Spike-timing dependent plasticity and the cognitive map

Directory of Open Access Journals (Sweden)

Daniel eBush

2010-10-01

Full Text Available Since the discovery of place cells – single pyramidal neurons that encode spatial location – it has been hypothesised that the hippocampus may act as a cognitive map of known environments. This putative function has been extensively modelled using auto-associative networks, which utilise rate-coded synaptic plasticity rules in order to generate strong bi-directional connections between concurrently active place cells that encode for neighbouring place fields. However, empirical studies using hippocampal cultures have demonstrated that the magnitude and direction of changes in synaptic strength can also be dictated by the relative timing of pre- and post- synaptic firing according to a spike-timing dependent plasticity (STDP rule. Furthermore, electrophysiology studies have identified persistent ‘theta-coded’ temporal correlations in place cell activity in vivo, characterised by phase precession of firing as the corresponding place field is traversed. It is not yet clear if STDP and theta-coded neural dynamics are compatible with cognitive map theory and previous rate-coded models of spatial learning in the hippocampus. Here, we demonstrate that an STDP rule based on empirical data obtained from the hippocampus can mediate rate-coded Hebbian learning when pre- and post- synaptic activity is stochastic and has no persistent sequence bias. We subsequently demonstrate that a spiking recurrent neural network that utilises this STDP rule, alongside theta-coded neural activity, allows the rapid development of a cognitive map during directed or random exploration of an environment of overlapping place fields. Hence, we establish that STDP and phase precession are compatible with rate-coded models of cognitive map development.

A theory of loop formation and elimination by spike timing-dependent plasticity

Directory of Open Access Journals (Sweden)

James Kozloski

2010-03-01

Full Text Available We show that the local Spike Timing-Dependent Plasticity (STDP rule has the effect of regulating the trans-synaptic weights of loops of any length within a simulated network of neurons. We show that depending on STDP's polarity, functional loops are formed or eliminated in networks driven to normal spiking conditions by random, partially correlated inputs, where functional loops comprise synaptic weights that exceed a non-zero threshold. We further prove that STDP is a form of loop-regulating plasticity for the case of a linear network driven by noise. Thus a notable local synaptic learning rule makes a specific prediction about synapses in the brain in which standard STDP is present: that under normal spiking conditions, they should participate in predominantly feed-forward connections at all scales. Our model implies that any deviations from this prediction would require a substantial modification to the hypothesized role for standard STDP. Given its widespread occurrence in the brain, we predict that STDP could also regulate long range functional loops among individual neurons across all brain scales, up to, and including, the scale of global brain network topology.

Physical implementation of pair-based spike timing dependent plasticity

International Nuclear Information System (INIS)

Azghadi, M.R.; Al-Sarawi, S.; Iannella, N.; Abbott, D.

2011-01-01

Full text: Objective Spike-timing-dependent plasticity (STOP) is one of several plasticity rules which leads to learning and memory in the brain. STOP induces synaptic weight changes based on the timing of the pre- and post-synaptic neurons. A neural network which can mimic the adaptive capability of biological brains in the temporal domain, requires the weight of single connections to be altered by spike timing. To physically realise this network into silicon, a large number of interconnected STOP circuits on the same substrate is required. This imposes two significant limitations in terms of power and area. To cover these limitations, very large scale integrated circuit (VLSI) technology provides attractive features in terms of low power and small area requirements. An example is demonstrated by (lndiveli et al. 2006). The objective of this paper is to present a new implementation of the STOP circuit which demonstrates better power and area in comparison to previous implementations. Methods The proposed circuit uses complementary metal oxide semiconductor (CMOS) technology as depicted in Fig. I. The synaptic weight can be stored on a capacitor and charging/discharging current can lead to potentiation and depression. HSpice simulation results demonstrate that the average power, peak power, and area of the proposed circuit have been reduced by 6, 8 and 15%, respectively, in comparison with Indiveri's implementation. These improvements naturally lead to packing more STOP circuits onto the same substrate, when compared to previous proposals. Hence, this new implementation is quite interesting for real-world large neural networks.

Perceptron learning rule derived from spike-frequency adaptation and spike-time-dependent plasticity.

Science.gov (United States)

D'Souza, Prashanth; Liu, Shih-Chii; Hahnloser, Richard H R

2010-03-09

It is widely believed that sensory and motor processing in the brain is based on simple computational primitives rooted in cellular and synaptic physiology. However, many gaps remain in our understanding of the connections between neural computations and biophysical properties of neurons. Here, we show that synaptic spike-time-dependent plasticity (STDP) combined with spike-frequency adaptation (SFA) in a single neuron together approximate the well-known perceptron learning rule. Our calculations and integrate-and-fire simulations reveal that delayed inputs to a neuron endowed with STDP and SFA precisely instruct neural responses to earlier arriving inputs. We demonstrate this mechanism on a developmental example of auditory map formation guided by visual inputs, as observed in the external nucleus of the inferior colliculus (ICX) of barn owls. The interplay of SFA and STDP in model ICX neurons precisely transfers the tuning curve from the visual modality onto the auditory modality, demonstrating a useful computation for multimodal and sensory-guided processing.

Spike Train Auto-Structure Impacts Post-Synaptic Firing and Timing-Based Plasticity

Science.gov (United States)

Scheller, Bertram; Castellano, Marta; Vicente, Raul; Pipa, Gordon

2011-01-01

Cortical neurons are typically driven by several thousand synapses. The precise spatiotemporal pattern formed by these inputs can modulate the response of a post-synaptic cell. In this work, we explore how the temporal structure of pre-synaptic inhibitory and excitatory inputs impact the post-synaptic firing of a conductance-based integrate and fire neuron. Both the excitatory and inhibitory input was modeled by renewal gamma processes with varying shape factors for modeling regular and temporally random Poisson activity. We demonstrate that the temporal structure of mutually independent inputs affects the post-synaptic firing, while the strength of the effect depends on the firing rates of both the excitatory and inhibitory inputs. In a second step, we explore the effect of temporal structure of mutually independent inputs on a simple version of Hebbian learning, i.e., hard bound spike-timing-dependent plasticity. We explore both the equilibrium weight distribution and the speed of the transient weight dynamics for different mutually independent gamma processes. We find that both the equilibrium distribution of the synaptic weights and the speed of synaptic changes are modulated by the temporal structure of the input. Finally, we highlight that the sensitivity of both the post-synaptic firing as well as the spike-timing-dependent plasticity on the auto-structure of the input of a neuron could be used to modulate the learning rate of synaptic modification. PMID:22203800

Unsupervised Learning of Digit Recognition Using Spike-Timing-Dependent Plasticity

Directory of Open Access Journals (Sweden)

Peter U. Diehl

2015-08-01

Full Text Available In order to understand how the mammalian neocortex is performing computations, two things are necessary; we need to have a good understanding of the available neuronal processing units and mechanisms, and we need to gain a better understanding of how those mechanisms are combined to build functioning systems. Therefore, in recent years there is an increasing interest in how spiking neural networks (SNN can be used to perform complex computations or solve pattern recognition tasks. However, it remains a challenging task to design SNNs which use biologically plausible mechanisms (especially for learning new patterns, since most of such SNN architectures rely on training in a rate-based network and subsequent conversion to a SNN. We present a SNN for digit recognition which is based on mechanisms with increased biological plausibility, i.e. conductance-based instead of current-based synapses, spike-timing-dependent plasticity with time-dependent weight change, lateral inhibition, and an adaptive spiking threshold. Unlike most other systems, we do not use a teaching signal and do not present any class labels to the network. Using this unsupervised learning scheme, our architecture achieves 95% accuracy on the MNIST benchmark, which is better than previous SNN implementations without supervision. The fact that we used no domain-specific knowledge points toward the general applicability of our network design. Also, the performance of our network scales well with the number of neurons used and shows similar performance for four different learning rules, indicating robustness of the full combination of mechanisms, which suggests applicability in heterogeneous biological neural networks.

Reward-timing-dependent bidirectional modulation of cortical microcircuits during optical single-neuron operant conditioning.

Science.gov (United States)

Hira, Riichiro; Ohkubo, Fuki; Masamizu, Yoshito; Ohkura, Masamichi; Nakai, Junichi; Okada, Takashi; Matsuzaki, Masanori

2014-11-24

Animals rapidly adapt to environmental change. To reveal how cortical microcircuits are rapidly reorganized when an animal recognizes novel reward contingency, we conduct two-photon calcium imaging of layer 2/3 motor cortex neurons in mice and simultaneously reinforce the activity of a single cortical neuron with water delivery. Here we show that when the target neuron is not relevant to a pre-trained forelimb movement, the mouse increases the target neuron activity and the number of rewards delivered during 15-min operant conditioning without changing forelimb movement behaviour. The reinforcement bidirectionally modulates the activity of subsets of non-target neurons, independent of distance from the target neuron. The bidirectional modulation depends on the relative timing between the reward delivery and the neuronal activity, and is recreated by pairing reward delivery and photoactivation of a subset of neurons. Reward-timing-dependent bidirectional modulation may be one of the fundamental processes in microcircuit reorganization for rapid adaptation.

Diverse spike-timing-dependent plasticity based on multilevel HfO x memristor for neuromorphic computing

Science.gov (United States)

Lu, Ke; Li, Yi; He, Wei-Fan; Chen, Jia; Zhou, Ya-Xiong; Duan, Nian; Jin, Miao-Miao; Gu, Wei; Xue, Kan-Hao; Sun, Hua-Jun; Miao, Xiang-Shui

2018-06-01

Memristors have emerged as promising candidates for artificial synaptic devices, serving as the building block of brain-inspired neuromorphic computing. In this letter, we developed a Pt/HfO x /Ti memristor with nonvolatile multilevel resistive switching behaviors due to the evolution of the conductive filaments and the variation in the Schottky barrier. Diverse state-dependent spike-timing-dependent-plasticity (STDP) functions were implemented with different initial resistance states. The measured STDP forms were adopted as the learning rule for a three-layer spiking neural network which achieves a 75.74% recognition accuracy for MNIST handwritten digit dataset. This work has shown the capability of memristive synapse in spiking neural networks for pattern recognition application.

Learning Probabilistic Inference through Spike-Timing-Dependent Plasticity.

Science.gov (United States)

Pecevski, Dejan; Maass, Wolfgang

2016-01-01

Numerous experimental data show that the brain is able to extract information from complex, uncertain, and often ambiguous experiences. Furthermore, it can use such learnt information for decision making through probabilistic inference. Several models have been proposed that aim at explaining how probabilistic inference could be performed by networks of neurons in the brain. We propose here a model that can also explain how such neural network could acquire the necessary information for that from examples. We show that spike-timing-dependent plasticity in combination with intrinsic plasticity generates in ensembles of pyramidal cells with lateral inhibition a fundamental building block for that: probabilistic associations between neurons that represent through their firing current values of random variables. Furthermore, by combining such adaptive network motifs in a recursive manner the resulting network is enabled to extract statistical information from complex input streams, and to build an internal model for the distribution p (*) that generates the examples it receives. This holds even if p (*) contains higher-order moments. The analysis of this learning process is supported by a rigorous theoretical foundation. Furthermore, we show that the network can use the learnt internal model immediately for prediction, decision making, and other types of probabilistic inference.

Artificial neuron operations and spike-timing-dependent plasticity using memristive devices for brain-inspired computing

Science.gov (United States)

Marukame, Takao; Nishi, Yoshifumi; Yasuda, Shin-ichi; Tanamoto, Tetsufumi

2018-04-01

The use of memristive devices for creating artificial neurons is promising for brain-inspired computing from the viewpoints of computation architecture and learning protocol. We present an energy-efficient multiplier accumulator based on a memristive array architecture incorporating both analog and digital circuitries. The analog circuitry is used to full advantage for neural networks, as demonstrated by the spike-timing-dependent plasticity (STDP) in fabricated AlO x /TiO x -based metal-oxide memristive devices. STDP protocols for controlling periodic analog resistance with long-range stability were experimentally verified using a variety of voltage amplitudes and spike timings.

Activity-Dependent Plasticity of Spike Pauses in Cerebellar Purkinje Cells

Directory of Open Access Journals (Sweden)

Giorgio Grasselli

2016-03-01

Full Text Available The plasticity of intrinsic excitability has been described in several types of neurons, but the significance of non-synaptic mechanisms in brain plasticity and learning remains elusive. Cerebellar Purkinje cells are inhibitory neurons that spontaneously fire action potentials at high frequencies and regulate activity in their target cells in the cerebellar nuclei by generating a characteristic spike burst-pause sequence upon synaptic activation. Using patch-clamp recordings from mouse Purkinje cells, we find that depolarization-triggered intrinsic plasticity enhances spike firing and shortens the duration of spike pauses. Pause plasticity is absent from mice lacking SK2-type potassium channels (SK2−/− mice and in occlusion experiments using the SK channel blocker apamin, while apamin wash-in mimics pause reduction. Our findings demonstrate that spike pauses can be regulated through an activity-dependent, exclusively non-synaptic, SK2 channel-dependent mechanism and suggest that pause plasticity—by altering the Purkinje cell output—may be crucial to cerebellar information storage and learning.

Spike-timing dependent plasticity in a transistor-selected resistive switching memory

International Nuclear Information System (INIS)

Ambrogio, S; Balatti, S; Nardi, F; Facchinetti, S; Ielmini, D

2013-01-01

In a neural network, neuron computation is achieved through the summation of input signals fed by synaptic connections. The synaptic activity (weight) is dictated by the synchronous firing of neurons, inducing potentiation/depression of the synaptic connection. This learning function can be supported by the resistive switching memory (RRAM), which changes its resistance depending on the amplitude, the pulse width and the bias polarity of the applied signal. This work shows a new synapse circuit comprising a MOS transistor as a selector and a RRAM as a variable resistance, displaying spike-timing dependent plasticity (STDP) similar to the one originally experienced in biological neural networks. We demonstrate long-term potentiation and long-term depression by simulations with an analytical model of resistive switching. Finally, the experimental demonstration of the new STDP scheme is presented. (paper)

Anti-hebbian spike-timing-dependent plasticity and adaptive sensory processing.

Science.gov (United States)

Roberts, Patrick D; Leen, Todd K

2010-01-01

Adaptive sensory processing influences the central nervous system's interpretation of incoming sensory information. One of the functions of this adaptive sensory processing is to allow the nervous system to ignore predictable sensory information so that it may focus on important novel information needed to improve performance of specific tasks. The mechanism of spike-timing-dependent plasticity (STDP) has proven to be intriguing in this context because of its dual role in long-term memory and ongoing adaptation to maintain optimal tuning of neural responses. Some of the clearest links between STDP and adaptive sensory processing have come from in vitro, in vivo, and modeling studies of the electrosensory systems of weakly electric fish. Plasticity in these systems is anti-Hebbian, so that presynaptic inputs that repeatedly precede, and possibly could contribute to, a postsynaptic neuron's firing are weakened. The learning dynamics of anti-Hebbian STDP learning rules are stable if the timing relations obey strict constraints. The stability of these learning rules leads to clear predictions of how functional consequences can arise from the detailed structure of the plasticity. Here we review the connection between theoretical predictions and functional consequences of anti-Hebbian STDP, focusing on adaptive processing in the electrosensory system of weakly electric fish. After introducing electrosensory adaptive processing and the dynamics of anti-Hebbian STDP learning rules, we address issues of predictive sensory cancelation and novelty detection, descending control of plasticity, synaptic scaling, and optimal sensory tuning. We conclude with examples in other systems where these principles may apply.

Anti-Hebbian Spike Timing Dependent Plasticity and Adaptive Sensory Processing

Directory of Open Access Journals (Sweden)

Patrick D Roberts

2010-12-01

Full Text Available Adaptive processing influences the central nervous system's interpretation of incoming sensory information. One of the functions of this adaptative sensory processing is to allow the nervous system to ignore predictable sensory information so that it may focus on important new information needed to improve performance of specific tasks. The mechanism of spike timing-dependent plasticity (STDP has proven to be intriguing in this context because of its dual role in long-term memory and ongoing adaptation to maintain optimal tuning of neural responses. Some of the clearest links between STDP and adaptive sensory processing have come from in vitro, in vivo, and modeling studies of the electrosensory systems of fish. Plasticity in such systems is anti-Hebbian, i.e. presynaptic inputs that repeatedly precede and hence could contribute to a postsynaptic neuron’s firing are weakened. The learning dynamics of anti-Hebbian STDP learning rules are stable if the timing relations obey strict constraints. The stability of these learning rules leads to clear predictions of how functional consequences can arise from the detailed structure of the plasticity. Here we review the connection between theoretical predictions and functional consequences of anti-Hebbian STDP, focusing on adaptive processing in the electrosensory system of weakly electric fish. After introducing electrosensory adaptive processing and the dynamics of anti-Hebbian STDP learning rules, we address issues of predictive sensory cancellation and novelty detection, descending control of plasticity, synaptic scaling, and optimal sensory tuning. We conclude with examples in other systems where these principles may apply.

Network evolution induced by asynchronous stimuli through spike-timing-dependent plasticity.

Directory of Open Access Journals (Sweden)

Wu-Jie Yuan

Full Text Available In sensory neural system, external asynchronous stimuli play an important role in perceptual learning, associative memory and map development. However, the organization of structure and dynamics of neural networks induced by external asynchronous stimuli are not well understood. Spike-timing-dependent plasticity (STDP is a typical synaptic plasticity that has been extensively found in the sensory systems and that has received much theoretical attention. This synaptic plasticity is highly sensitive to correlations between pre- and postsynaptic firings. Thus, STDP is expected to play an important role in response to external asynchronous stimuli, which can induce segregative pre- and postsynaptic firings. In this paper, we study the impact of external asynchronous stimuli on the organization of structure and dynamics of neural networks through STDP. We construct a two-dimensional spatial neural network model with local connectivity and sparseness, and use external currents to stimulate alternately on different spatial layers. The adopted external currents imposed alternately on spatial layers can be here regarded as external asynchronous stimuli. Through extensive numerical simulations, we focus on the effects of stimulus number and inter-stimulus timing on synaptic connecting weights and the property of propagation dynamics in the resulting network structure. Interestingly, the resulting feedforward structure induced by stimulus-dependent asynchronous firings and its propagation dynamics reflect both the underlying property of STDP. The results imply a possible important role of STDP in generating feedforward structure and collective propagation activity required for experience-dependent map plasticity in developing in vivo sensory pathways and cortices. The relevance of the results to cue-triggered recall of learned temporal sequences, an important cognitive function, is briefly discussed as well. Furthermore, this finding suggests a potential

Learning Probabilistic Inference through Spike-Timing-Dependent Plasticity123

Science.gov (United States)

Pecevski, Dejan

2016-01-01

Abstract Numerous experimental data show that the brain is able to extract information from complex, uncertain, and often ambiguous experiences. Furthermore, it can use such learnt information for decision making through probabilistic inference. Several models have been proposed that aim at explaining how probabilistic inference could be performed by networks of neurons in the brain. We propose here a model that can also explain how such neural network could acquire the necessary information for that from examples. We show that spike-timing-dependent plasticity in combination with intrinsic plasticity generates in ensembles of pyramidal cells with lateral inhibition a fundamental building block for that: probabilistic associations between neurons that represent through their firing current values of random variables. Furthermore, by combining such adaptive network motifs in a recursive manner the resulting network is enabled to extract statistical information from complex input streams, and to build an internal model for the distribution p* that generates the examples it receives. This holds even if p* contains higher-order moments. The analysis of this learning process is supported by a rigorous theoretical foundation. Furthermore, we show that the network can use the learnt internal model immediately for prediction, decision making, and other types of probabilistic inference. PMID:27419214

Spike-timing-dependent plasticity in the human dorso-lateral prefrontal cortex.

Science.gov (United States)

Casula, Elias Paolo; Pellicciari, Maria Concetta; Picazio, Silvia; Caltagirone, Carlo; Koch, Giacomo

2016-12-01

Changes in the synaptic strength of neural connections are induced by repeated coupling of activity of interconnected neurons with precise timing, a phenomenon known as spike-timing-dependent plasticity (STDP). It is debated if this mechanism exists in large-scale cortical networks in humans. We combined transcranial magnetic stimulation (TMS) with concurrent electroencephalography (EEG) to directly investigate the effects of two paired associative stimulation (PAS) protocols (fronto-parietal and parieto-frontal) of pre and post-synaptic inputs within the human fronto-parietal network. We found evidence that the dorsolateral prefrontal cortex (DLPFC) has the potential to form robust STDP. Long-term potentiation/depression of TMS-evoked cortical activity is prompted after that DLPFC stimulation is followed/preceded by posterior parietal stimulation. Such bidirectional changes are paralleled by sustained increase/decrease of high-frequency oscillatory activity, likely reflecting STDP responsivity. The current findings could be important to drive plasticity of damaged cortical circuits in patients with cognitive or psychiatric disorders. Copyright © 2016 Elsevier Inc. All rights reserved.

Does spike-timing-dependent synaptic plasticity couple or decouple neurons firing in synchrony?

Directory of Open Access Journals (Sweden)

Andreas eKnoblauch

2012-08-01

Full Text Available Spike synchronization is thought to have a constructive role for feature integration, attention, associativelearning, and the formation of bidirectionally connected Hebbian cell assemblies. By contrast, theoreticalstudies on spike-timing-dependent plasticity (STDP report an inherently decoupling influence of spikesynchronization on synaptic connections of coactivated neurons. For example, bidirectional synapticconnections as found in cortical areas could be reproduced only by assuming realistic models of STDP andrate coding. We resolve this conflict by theoretical analysis and simulation of various simple and realisticSTDP models that provide a more complete characterization of conditions when STDP leads to eithercoupling or decoupling of neurons firing in synchrony. In particular, we show that STDP consistentlycouples synchronized neurons if key model parameters are matched to physiological data: First, synapticpotentiation must be significantly stronger than synaptic depression for small (positive or negative timelags between presynaptic and postsynaptic spikes. Second, spike synchronization must be sufficientlyimprecise, for example, within a time window of 5-10msec instead of 1msec. Third, axonal propagationdelays should not be much larger than dendritic delays. Under these assumptions synchronized neuronswill be strongly coupled leading to a dominance of bidirectional synaptic connections even for simpleSTDP models and low mean firing rates at the level of spontaneous activity.

Supervised spike-timing-dependent plasticity: a spatiotemporal neuronal learning rule for function approximation and decisions.

Science.gov (United States)

Franosch, Jan-Moritz P; Urban, Sebastian; van Hemmen, J Leo

2013-12-01

How can an animal learn from experience? How can it train sensors, such as the auditory or tactile system, based on other sensory input such as the visual system? Supervised spike-timing-dependent plasticity (supervised STDP) is a possible answer. Supervised STDP trains one modality using input from another one as "supervisor." Quite complex time-dependent relationships between the senses can be learned. Here we prove that under very general conditions, supervised STDP converges to a stable configuration of synaptic weights leading to a reconstruction of primary sensory input.

A neuromorphic implementation of multiple spike-timing synaptic plasticity rules for large-scale neural networks

Directory of Open Access Journals (Sweden)

Runchun Mark Wang

2015-05-01

Full Text Available We present a neuromorphic implementation of multiple synaptic plasticity learning rules, which include both Spike Timing Dependent Plasticity (STDP and Spike Timing Dependent Delay Plasticity (STDDP. We present a fully digital implementation as well as a mixed-signal implementation, both of which use a novel dynamic-assignment time-multiplexing approach and support up to 2^26 (64M synaptic plasticity elements. Rather than implementing dedicated synapses for particular types of synaptic plasticity, we implemented a more generic synaptic plasticity adaptor array that is separate from the neurons in the neural network. Each adaptor performs synaptic plasticity according to the arrival times of the pre- and post-synaptic spikes assigned to it, and sends out a weighted and/or delayed pre-synaptic spike to the target synapse in the neural network. This strategy provides great flexibility for building complex large-scale neural networks, as a neural network can be configured for multiple synaptic plasticity rules without changing its structure. We validate the proposed neuromorphic implementations with measurement results and illustrate that the circuits are capable of performing both STDP and STDDP. We argue that it is practical to scale the work presented here up to 2^36 (64G synaptic adaptors on a current high-end FPGA platform.

Reward-dependent modulation of movement variability.

Science.gov (United States)

Pekny, Sarah E; Izawa, Jun; Shadmehr, Reza

2015-03-04

Movement variability is often considered an unwanted byproduct of a noisy nervous system. However, variability can signal a form of implicit exploration, indicating that the nervous system is intentionally varying the motor commands in search of actions that yield the greatest success. Here, we investigated the role of the human basal ganglia in controlling reward-dependent motor variability as measured by trial-to-trial changes in performance during a reaching task. We designed an experiment in which the only performance feedback was success or failure and quantified how reach variability was modulated as a function of the probability of reward. In healthy controls, reach variability increased as the probability of reward decreased. Control of variability depended on the history of past rewards, with the largest trial-to-trial changes occurring immediately after an unrewarded trial. In contrast, in participants with Parkinson's disease, a known example of basal ganglia dysfunction, reward was a poor modulator of variability; that is, the patients showed an impaired ability to increase variability in response to decreases in the probability of reward. This was despite the fact that, after rewarded trials, reach variability in the patients was comparable to healthy controls. In summary, we found that movement variability is partially a form of exploration driven by the recent history of rewards. When the function of the human basal ganglia is compromised, the reward-dependent control of movement variability is impaired, particularly affecting the ability to increase variability after unsuccessful outcomes. Copyright © 2015 the authors 0270-6474/15/354015-10$15.00/0.

A network of spiking neurons that can represent interval timing: mean field analysis.

Science.gov (United States)

Gavornik, Jeffrey P; Shouval, Harel Z

2011-04-01

Despite the vital importance of our ability to accurately process and encode temporal information, the underlying neural mechanisms are largely unknown. We have previously described a theoretical framework that explains how temporal representations, similar to those reported in the visual cortex, can form in locally recurrent cortical networks as a function of reward modulated synaptic plasticity. This framework allows networks of both linear and spiking neurons to learn the temporal interval between a stimulus and paired reward signal presented during training. Here we use a mean field approach to analyze the dynamics of non-linear stochastic spiking neurons in a network trained to encode specific time intervals. This analysis explains how recurrent excitatory feedback allows a network structure to encode temporal representations.

Spike timing analysis in neural networks with unsupervised synaptic plasticity

Science.gov (United States)

Mizusaki, B. E. P.; Agnes, E. J.; Brunnet, L. G.; Erichsen, R., Jr.

2013-01-01

The synaptic plasticity rules that sculpt a neural network architecture are key elements to understand cortical processing, as they may explain the emergence of stable, functional activity, while avoiding runaway excitation. For an associative memory framework, they should be built in a way as to enable the network to reproduce a robust spatio-temporal trajectory in response to an external stimulus. Still, how these rules may be implemented in recurrent networks and the way they relate to their capacity of pattern recognition remains unclear. We studied the effects of three phenomenological unsupervised rules in sparsely connected recurrent networks for associative memory: spike-timing-dependent-plasticity, short-term-plasticity and an homeostatic scaling. The system stability is monitored during the learning process of the network, as the mean firing rate converges to a value determined by the homeostatic scaling. Afterwards, it is possible to measure the recovery efficiency of the activity following each initial stimulus. This is evaluated by a measure of the correlation between spike fire timings, and we analysed the full memory separation capacity and limitations of this system.

2D co-ordinate transformation based on a spike timing-dependent plasticity learning mechanism.

Science.gov (United States)

Wu, QingXiang; McGinnity, Thomas Martin; Maguire, Liam; Belatreche, Ammar; Glackin, Brendan

2008-11-01

In order to plan accurate motor actions, the brain needs to build an integrated spatial representation associated with visual stimuli and haptic stimuli. Since visual stimuli are represented in retina-centered co-ordinates and haptic stimuli are represented in body-centered co-ordinates, co-ordinate transformations must occur between the retina-centered co-ordinates and body-centered co-ordinates. A spiking neural network (SNN) model, which is trained with spike-timing-dependent-plasticity (STDP), is proposed to perform a 2D co-ordinate transformation of the polar representation of an arm position to a Cartesian representation, to create a virtual image map of a haptic input. Through the visual pathway, a position signal corresponding to the haptic input is used to train the SNN with STDP synapses such that after learning the SNN can perform the co-ordinate transformation to generate a representation of the haptic input with the same co-ordinates as a visual image. The model can be applied to explain co-ordinate transformation in spiking neuron based systems. The principle can be used in artificial intelligent systems to process complex co-ordinate transformations represented by biological stimuli.

Frequency dependent changes in NMDAR-dependent synaptic plasticity

Directory of Open Access Journals (Sweden)

Arvind eKumar

2011-09-01

Full Text Available The NMDAR-dependent synaptic plasticity is thought to mediate several forms of learning, and can be induced by spike trains containing a small number of spikes occurring with varying rates and timing, as well as with oscillations. We computed the influence of these variables on the plasticity induced at a single NMDAR containing synapse using a reduced model that was analytically tractable, and these findings were confirmed using detailed, multi-compartment model. In addition to explaining diverse experimental results about the rate and timing dependence of synaptic plasticity, the model made several novel and testable predictions. We found that there was a preferred frequency for inducing long-term potentiation (LTP such that higher frequency stimuli induced lesser LTP, decreasing as 1/f when the number of spikes in the stimulus was kept fixed. Among other things, the preferred frequency for inducing LTP varied as a function of the distance of the synapse from the soma. In fact, same stimulation frequencies could induce LTP or LTD depending on the dendritic location of the synapse. Next, we found that rhythmic stimuli induced greater plasticity then irregular stimuli. Furthermore, brief bursts of spikes significantly expanded the timing dependence of plasticity. Finally, we found that in the ~5-15Hz frequency range both rate- and timing-dependent plasticity mechanisms work synergistically to render the synaptic plasticity most sensitive to spike-timing. These findings provide computational evidence that oscillations can have a profound influence on the plasticity of an NMDAR-dependent synapse, and show a novel role for the dendritic morphology in this process.

Unsupervised learning by spike timing dependent plasticity in phase change memory (PCM synapses

Directory of Open Access Journals (Sweden)

Stefano eAmbrogio

2016-03-01

Full Text Available We present a novel one-transistor/one-resistor (1T1R synapse for neuromorphic networks, based on phase change memory (PCM technology. The synapse is capable of spike-timing dependent plasticity (STDP, where gradual potentiation relies on set transition, namely crystallization, in the PCM, while depression is achieved via reset or amorphization of a chalcogenide active volume. STDP characteristics are demonstrated by experiments under variable initial conditions and number of pulses. Finally, we support the applicability of the 1T1R synapse for learning and recognition of visual patterns by simulations of fully connected neuromorphic networks with 2 or 3 layers with high recognition efficiency. The proposed scheme provides a feasible low-power solution for on-line unsupervised machine learning in smart reconfigurable sensors.

Spike timing-dependent plasticity as the origin of the formation of clustered synaptic efficacy engrams

Directory of Open Access Journals (Sweden)

Nicolangelo L Iannella

2010-07-01

Full Text Available Synapse location, dendritic active properties and synaptic plasticity are all known to play some role in shaping the different input streams impinging onto a neuron. It remains unclear however, how the magnitude and spatial distribution of synaptic efficacies emerge from this interplay. Here, we investigate this interplay using a biophysically detailed neuron model of a reconstructed layer 2/3 pyramidal cell and spike timing-dependent plasticity (STDP. Specifically, we focus on the issue of how the efficacy of synapses contributed by different input streams are spatially represented in dendrites after STDP learning. We construct a simple feed forward network where a detailed model neuron receives synaptic inputs independently from multiple yet equally sized groups of afferent fibres with correlated activity, mimicking the spike activity from different neuronal populations encoding, for example, different sensory modalities. Interestingly, ensuing STDP learning, we observe that for all afferent groups, STDP leads to synaptic efficacies arranged into spatially segregated clusters effectively partitioning the dendritic tree. These segregated clusters possess a characteristic global organisation in space, where they form a tessellation in which each group dominates mutually exclusive regions of the dendrite.Put simply, the dendritic imprint from different input streams left after STDP learning effectively forms what we term a dendritic efficacy mosaic. Furthermore, we show how variations of the inputs and STDP rule affect such an organization. Our model suggests that STDP may be an important mechanism for creating a clustered plasticity engram, which shapes how different input streams are spatially represented in dendrite.

Analog memory and spike-timing-dependent plasticity characteristics of a nanoscale titanium oxide bilayer resistive switching device

International Nuclear Information System (INIS)

Seo, Kyungah; Park, Sangsu; Lee, Kwanghee; Lee, Byounghun; Hwang, Hyunsang; Kim, Insung; Jung, Seungjae; Jo, Minseok; Park, Jubong; Shin, Jungho; Biju, Kuyyadi P; Kong, Jaemin

2011-01-01

We demonstrated analog memory, synaptic plasticity, and a spike-timing-dependent plasticity (STDP) function with a nanoscale titanium oxide bilayer resistive switching device with a simple fabrication process and good yield uniformity. We confirmed the multilevel conductance and analog memory characteristics as well as the uniformity and separated states for the accuracy of conductance change. Finally, STDP and a biological triple model were analyzed to demonstrate the potential of titanium oxide bilayer resistive switching device as synapses in neuromorphic devices. By developing a simple resistive switching device that can emulate a synaptic function, the unique characteristics of synapses in the brain, e.g. combined memory and computing in one synapse and adaptation to the outside environment, were successfully demonstrated in a solid state device.

Emergence of task-dependent representations in working memory circuits

Directory of Open Access Journals (Sweden)

Cristina eSavin

2014-05-01

Full Text Available A wealth of experimental evidence suggests that working memory circuits preferentially represent information that is behaviorally relevant. Still, we are missing a mechanistic account of how these representations come about. Here we provide a simple explanation for a range of experimental findings, in light of prefrontal circuits adapting to task constraints by reward-dependent learning. In particular, we model a neural network shaped by reward-modulated spike-timing dependent plasticity (r-STDP and homeostatic plasticity (intrinsic excitability and synaptic scaling. We show that the experimentally-observed neural representations naturally emerge in an initially unstructured circuit as it learns to solve several working memory tasks. These results point to a critical, and previously unappreciated, role for reward-dependent learning in shaping prefrontal cortex activity.

Supervised Learning Using Spike-Timing-Dependent Plasticity of Memristive Synapses.

Science.gov (United States)

Nishitani, Yu; Kaneko, Yukihiro; Ueda, Michihito

2015-12-01

We propose a supervised learning model that enables error backpropagation for spiking neural network hardware. The method is modeled by modifying an existing model to suit the hardware implementation. An example of a network circuit for the model is also presented. In this circuit, a three-terminal ferroelectric memristor (3T-FeMEM), which is a field-effect transistor with a gate insulator composed of ferroelectric materials, is used as an electric synapse device to store the analog synaptic weight. Our model can be implemented by reflecting the network error to the write voltage of the 3T-FeMEMs and introducing a spike-timing-dependent learning function to the device. An XOR problem was successfully demonstrated as a benchmark learning by numerical simulations using the circuit properties to estimate the learning performance. In principle, the learning time per step of this supervised learning model and the circuit is independent of the number of neurons in each layer, promising a high-speed and low-power calculation in large-scale neural networks.

Double Dissociation of Spike Timing-Dependent Potentiation and Depression by Subunit-Preferring NMDA Receptor Antagonists in Mouse Barrel Cortex

NARCIS (Netherlands)

Banerjee, A.; Meredith, R.M.; Rodriguez-Moreno, A.; Mierau, S.B.; Auberson, Y.P.; Paulsen, O.

2009-01-01

Spike timing-dependent plasticity (STDP) is a strong candidate for an N-methyl-D-aspartate (NMDA) receptor-dependent form of synaptic plasticity that could underlie the development of receptive field properties in sensory neocortices. Whilst induction of timing-dependent long-term potentiation

Reward-modulated motor information in identified striatum neurons.

Science.gov (United States)

Isomura, Yoshikazu; Takekawa, Takashi; Harukuni, Rie; Handa, Takashi; Aizawa, Hidenori; Takada, Masahiko; Fukai, Tomoki

2013-06-19

It is widely accepted that dorsal striatum neurons participate in either the direct pathway (expressing dopamine D1 receptors) or the indirect pathway (expressing D2 receptors), controlling voluntary movements in an antagonistically balancing manner. The D1- and D2-expressing neurons are activated and inactivated, respectively, by dopamine released from substantia nigra neurons encoding reward expectation. However, little is known about the functional representation of motor information and its reward modulation in individual striatal neurons constituting the two pathways. In this study, we juxtacellularly recorded the spike activity of single neurons in the dorsolateral striatum of rats performing voluntary forelimb movement in a reward-predictable condition. Some of these neurons were identified morphologically by a combination of juxtacellular visualization and in situ hybridization for D1 mRNA. We found that the striatal neurons exhibited distinct functional activations before and during the forelimb movement, regardless of the expression of D1 mRNA. They were often positively, but rarely negatively, modulated by expecting a reward for the correct motor response. The positive reward modulation was independent of behavioral differences in motor performance. In contrast, regular-spiking and fast-spiking neurons in any layers of the motor cortex displayed only minor and unbiased reward modulation of their functional activation in relation to the execution of forelimb movement. Our results suggest that the direct and indirect pathway neurons cooperatively rather than antagonistically contribute to spatiotemporal control of voluntary movements, and that motor information is subcortically integrated with reward information through dopaminergic and other signals in the skeletomotor loop of the basal ganglia.

Dynamic Hebbian Cross-Correlation Learning Resolves the Spike Timing Dependent Plasticity Conundrum

Directory of Open Access Journals (Sweden)

Tjeerd V. olde Scheper

2018-01-01

Full Text Available Spike Timing-Dependent Plasticity has been found to assume many different forms. The classic STDP curve, with one potentiating and one depressing window, is only one of many possible curves that describe synaptic learning using the STDP mechanism. It has been shown experimentally that STDP curves may contain multiple LTP and LTD windows of variable width, and even inverted windows. The underlying STDP mechanism that is capable of producing such an extensive, and apparently incompatible, range of learning curves is still under investigation. In this paper, it is shown that STDP originates from a combination of two dynamic Hebbian cross-correlations of local activity at the synapse. The correlation of the presynaptic activity with the local postsynaptic activity is a robust and reliable indicator of the discrepancy between the presynaptic neuron and the postsynaptic neuron's activity. The second correlation is between the local postsynaptic activity with dendritic activity which is a good indicator of matching local synaptic and dendritic activity. We show that this simple time-independent learning rule can give rise to many forms of the STDP learning curve. The rule regulates synaptic strength without the need for spike matching or other supervisory learning mechanisms. Local differences in dendritic activity at the synapse greatly affect the cross-correlation difference which determines the relative contributions of different neural activity sources. Dendritic activity due to nearby synapses, action potentials, both forward and back-propagating, as well as inhibitory synapses will dynamically modify the local activity at the synapse, and the resulting STDP learning rule. The dynamic Hebbian learning rule ensures furthermore, that the resulting synaptic strength is dynamically stable, and that interactions between synapses do not result in local instabilities. The rule clearly demonstrates that synapses function as independent localized

Delay selection by spike-timing-dependent plasticity in recurrent networks of spiking neurons receiving oscillatory inputs.

Directory of Open Access Journals (Sweden)

Robert R Kerr

Full Text Available Learning rules, such as spike-timing-dependent plasticity (STDP, change the structure of networks of neurons based on the firing activity. A network level understanding of these mechanisms can help infer how the brain learns patterns and processes information. Previous studies have shown that STDP selectively potentiates feed-forward connections that have specific axonal delays, and that this underlies behavioral functions such as sound localization in the auditory brainstem of the barn owl. In this study, we investigate how STDP leads to the selective potentiation of recurrent connections with different axonal and dendritic delays during oscillatory activity. We develop analytical models of learning with additive STDP in recurrent networks driven by oscillatory inputs, and support the results using simulations with leaky integrate-and-fire neurons. Our results show selective potentiation of connections with specific axonal delays, which depended on the input frequency. In addition, we demonstrate how this can lead to a network becoming selective in the amplitude of its oscillatory response to this frequency. We extend this model of axonal delay selection within a single recurrent network in two ways. First, we show the selective potentiation of connections with a range of both axonal and dendritic delays. Second, we show axonal delay selection between multiple groups receiving out-of-phase, oscillatory inputs. We discuss the application of these models to the formation and activation of neuronal ensembles or cell assemblies in the cortex, and also to missing fundamental pitch perception in the auditory brainstem.

Dopamine-signalled reward predictions generated by competitive excitation and inhibition in a spiking neural network model

Directory of Open Access Journals (Sweden)

Paul eChorley

2011-05-01

Full Text Available Dopaminergic neurons in the mammalian substantia nigra displaycharacteristic phasic responses to stimuli which reliably predict thereceipt of primary rewards. These responses have been suggested toencode reward prediction-errors similar to those used in reinforcementlearning. Here, we propose a model of dopaminergic activity in whichprediction error signals are generated by the joint action ofshort-latency excitation and long-latency inhibition, in a networkundergoing dopaminergic neuromodulation of both spike-timing dependentsynaptic plasticity and neuronal excitability. In contrast toprevious models, sensitivity to recent events is maintained by theselective modification of specific striatal synapses, efferent tocortical neurons exhibiting stimulus-specific, temporally extendedactivity patterns. Our model shows, in the presence of significantbackground activity, (i a shift in dopaminergic response from rewardto reward predicting stimuli, (ii preservation of a response tounexpected rewards, and (iii a precisely-timed below-baseline dip inactivity observed when expected rewards are omitted.

Learning of spiking networks with different forms of long-term synaptic plasticity

International Nuclear Information System (INIS)

Vlasov, D.S.; Sboev, A.G.; Serenko, A.V.; Rybka, R.B.; Moloshnikov, I.A.

2016-01-01

The possibility of modeling the learning process based on different forms of spike timing-dependent plasticity (STDP) has been studied. It has been shown that the learnability depends on the choice of the spike pairing scheme in the STDP rule and the type of the input signal used during learning [ru

Self-Organization of Microcircuits in Networks of Spiking Neurons with Plastic Synapses.

Directory of Open Access Journals (Sweden)

Gabriel Koch Ocker

2015-08-01

Full Text Available The synaptic connectivity of cortical networks features an overrepresentation of certain wiring motifs compared to simple random-network models. This structure is shaped, in part, by synaptic plasticity that promotes or suppresses connections between neurons depending on their joint spiking activity. Frequently, theoretical studies focus on how feedforward inputs drive plasticity to create this network structure. We study the complementary scenario of self-organized structure in a recurrent network, with spike timing-dependent plasticity driven by spontaneous dynamics. We develop a self-consistent theory for the evolution of network structure by combining fast spiking covariance with a slow evolution of synaptic weights. Through a finite-size expansion of network dynamics we obtain a low-dimensional set of nonlinear differential equations for the evolution of two-synapse connectivity motifs. With this theory in hand, we explore how the form of the plasticity rule drives the evolution of microcircuits in cortical networks. When potentiation and depression are in approximate balance, synaptic dynamics depend on weighted divergent, convergent, and chain motifs. For additive, Hebbian STDP these motif interactions create instabilities in synaptic dynamics that either promote or suppress the initial network structure. Our work provides a consistent theoretical framework for studying how spiking activity in recurrent networks interacts with synaptic plasticity to determine network structure.

Self-Organization of Microcircuits in Networks of Spiking Neurons with Plastic Synapses.

Science.gov (United States)

Ocker, Gabriel Koch; Litwin-Kumar, Ashok; Doiron, Brent

2015-08-01

The synaptic connectivity of cortical networks features an overrepresentation of certain wiring motifs compared to simple random-network models. This structure is shaped, in part, by synaptic plasticity that promotes or suppresses connections between neurons depending on their joint spiking activity. Frequently, theoretical studies focus on how feedforward inputs drive plasticity to create this network structure. We study the complementary scenario of self-organized structure in a recurrent network, with spike timing-dependent plasticity driven by spontaneous dynamics. We develop a self-consistent theory for the evolution of network structure by combining fast spiking covariance with a slow evolution of synaptic weights. Through a finite-size expansion of network dynamics we obtain a low-dimensional set of nonlinear differential equations for the evolution of two-synapse connectivity motifs. With this theory in hand, we explore how the form of the plasticity rule drives the evolution of microcircuits in cortical networks. When potentiation and depression are in approximate balance, synaptic dynamics depend on weighted divergent, convergent, and chain motifs. For additive, Hebbian STDP these motif interactions create instabilities in synaptic dynamics that either promote or suppress the initial network structure. Our work provides a consistent theoretical framework for studying how spiking activity in recurrent networks interacts with synaptic plasticity to determine network structure.

A model of human motor sequence learning explains facilitation and interference effects based on spike-timing dependent plasticity.

Directory of Open Access Journals (Sweden)

Quan Wang

2017-08-01

Full Text Available The ability to learn sequential behaviors is a fundamental property of our brains. Yet a long stream of studies including recent experiments investigating motor sequence learning in adult human subjects have produced a number of puzzling and seemingly contradictory results. In particular, when subjects have to learn multiple action sequences, learning is sometimes impaired by proactive and retroactive interference effects. In other situations, however, learning is accelerated as reflected in facilitation and transfer effects. At present it is unclear what the underlying neural mechanism are that give rise to these diverse findings. Here we show that a recently developed recurrent neural network model readily reproduces this diverse set of findings. The self-organizing recurrent neural network (SORN model is a network of recurrently connected threshold units that combines a simplified form of spike-timing dependent plasticity (STDP with homeostatic plasticity mechanisms ensuring network stability, namely intrinsic plasticity (IP and synaptic normalization (SN. When trained on sequence learning tasks modeled after recent experiments we find that it reproduces the full range of interference, facilitation, and transfer effects. We show how these effects are rooted in the network's changing internal representation of the different sequences across learning and how they depend on an interaction of training schedule and task similarity. Furthermore, since learning in the model is based on fundamental neuronal plasticity mechanisms, the model reveals how these plasticity mechanisms are ultimately responsible for the network's sequence learning abilities. In particular, we find that all three plasticity mechanisms are essential for the network to learn effective internal models of the different training sequences. This ability to form effective internal models is also the basis for the observed interference and facilitation effects. This suggests that

Stimulus-dependent modulation of spike burst length in cat striate cortical cells.

Science.gov (United States)

DeBusk, B C; DeBruyn, E J; Snider, R K; Kabara, J F; Bonds, A B

1997-07-01

Burst activity, defined by groups of two or more spikes with intervals of cats. Bursting varied broadly across a population of 507 simple and complex cells. Half of this population had > or = 42% of their spikes contained in bursts. The fraction of spikes in bursts did not vary as a function of average firing rate and was stationary over time. Peaks in the interspike interval histograms were found at both 3-5 ms and 10-30 ms. In many cells the locations of these peaks were independent of firing rate, indicating a quantized control of firing behavior at two different time scales. The activity at the shorter time scale most likely results from intrinsic properties of the cell membrane, and that at the longer scale from recurrent network excitation. Burst frequency (bursts per s) and burst length (spikes per burst) both depended on firing rate. Burst frequency was essentially linear with firing rate, whereas burst length was a nonlinear function of firing rate and was also governed by stimulus orientation. At a given firing rate, burst length was greater for optimal orientations than for nonoptimal orientations. No organized orientation dependence was seen in bursts from lateral geniculate nucleus cells. Activation of cortical contrast gain control at low response amplitudes resulted in no burst length modulation, but burst shortening at optimal orientations was found in responses characterized by supersaturation. At a given firing rate, cortical burst length was shortened by microinjection of gamma-aminobutyric acid (GABA), and bursts became longer in the presence of N-methyl-bicuculline, a GABA(A) receptor blocker. These results are consistent with a model in which responses are reduced at nonoptimal orientations, at least in part, by burst shortening that is mediated by GABA. A similar mechanism contributes to response supersaturation at high contrasts via recruitment of inhibitory responses that are tuned to adjacent orientations. Burst length modulation can serve

Minimizing the effect of process mismatch in a neuromorphic system using spike-timing-dependent adaptation.

Science.gov (United States)

Cameron, Katherine; Murray, Alan

2008-05-01

This paper investigates whether spike-timing-dependent plasticity (STDP) can minimize the effect of mismatch within the context of a depth-from-motion algorithm. To improve noise rejection, this algorithm contains a spike prediction element, whose performance is degraded by analog very large scale integration (VLSI) mismatch. The error between the actual spike arrival time and the prediction is used as the input to an STDP circuit, to improve future predictions. Before STDP adaptation, the error reflects the degree of mismatch within the prediction circuitry. After STDP adaptation, the error indicates to what extent the adaptive circuitry can minimize the effect of transistor mismatch. The circuitry is tested with static and varying prediction times and chip results are presented. The effect of noisy spikes is also investigated. Under all conditions the STDP adaptation is shown to improve performance.

Effects of spike-time-dependent plasticity on the stochastic resonance of small-world neuronal networks

Energy Technology Data Exchange (ETDEWEB)

Yu, Haitao; Guo, Xinmeng; Wang, Jiang, E-mail: jiangwang@tju.edu.cn; Deng, Bin; Wei, Xile [School of Electrical Engineering and Automation, Tianjin University, Tianjin 300072 (China)

2014-09-01

The phenomenon of stochastic resonance in Newman-Watts small-world neuronal networks is investigated when the strength of synaptic connections between neurons is adaptively adjusted by spike-time-dependent plasticity (STDP). It is shown that irrespective of the synaptic connectivity is fixed or adaptive, the phenomenon of stochastic resonance occurs. The efficiency of network stochastic resonance can be largely enhanced by STDP in the coupling process. Particularly, the resonance for adaptive coupling can reach a much larger value than that for fixed one when the noise intensity is small or intermediate. STDP with dominant depression and small temporal window ratio is more efficient for the transmission of weak external signal in small-world neuronal networks. In addition, we demonstrate that the effect of stochastic resonance can be further improved via fine-tuning of the average coupling strength of the adaptive network. Furthermore, the small-world topology can significantly affect stochastic resonance of excitable neuronal networks. It is found that there exists an optimal probability of adding links by which the noise-induced transmission of weak periodic signal peaks.

Effects of spike-time-dependent plasticity on the stochastic resonance of small-world neuronal networks

International Nuclear Information System (INIS)

Yu, Haitao; Guo, Xinmeng; Wang, Jiang; Deng, Bin; Wei, Xile

2014-01-01

The phenomenon of stochastic resonance in Newman-Watts small-world neuronal networks is investigated when the strength of synaptic connections between neurons is adaptively adjusted by spike-time-dependent plasticity (STDP). It is shown that irrespective of the synaptic connectivity is fixed or adaptive, the phenomenon of stochastic resonance occurs. The efficiency of network stochastic resonance can be largely enhanced by STDP in the coupling process. Particularly, the resonance for adaptive coupling can reach a much larger value than that for fixed one when the noise intensity is small or intermediate. STDP with dominant depression and small temporal window ratio is more efficient for the transmission of weak external signal in small-world neuronal networks. In addition, we demonstrate that the effect of stochastic resonance can be further improved via fine-tuning of the average coupling strength of the adaptive network. Furthermore, the small-world topology can significantly affect stochastic resonance of excitable neuronal networks. It is found that there exists an optimal probability of adding links by which the noise-induced transmission of weak periodic signal peaks

Is a 4-bit synaptic weight resolution enough? - constraints on enabling spike-timing dependent plasticity in neuromorphic hardware.

Science.gov (United States)

Pfeil, Thomas; Potjans, Tobias C; Schrader, Sven; Potjans, Wiebke; Schemmel, Johannes; Diesmann, Markus; Meier, Karlheinz

2012-01-01

Large-scale neuromorphic hardware systems typically bear the trade-off between detail level and required chip resources. Especially when implementing spike-timing dependent plasticity, reduction in resources leads to limitations as compared to floating point precision. By design, a natural modification that saves resources would be reducing synaptic weight resolution. In this study, we give an estimate for the impact of synaptic weight discretization on different levels, ranging from random walks of individual weights to computer simulations of spiking neural networks. The FACETS wafer-scale hardware system offers a 4-bit resolution of synaptic weights, which is shown to be sufficient within the scope of our network benchmark. Our findings indicate that increasing the resolution may not even be useful in light of further restrictions of customized mixed-signal synapses. In addition, variations due to production imperfections are investigated and shown to be uncritical in the context of the presented study. Our results represent a general framework for setting up and configuring hardware-constrained synapses. We suggest how weight discretization could be considered for other backends dedicated to large-scale simulations. Thus, our proposition of a good hardware verification practice may rise synergy effects between hardware developers and neuroscientists.

Spike-timing theory of working memory.

Directory of Open Access Journals (Sweden)

Botond Szatmáry

Full Text Available Working memory (WM is the part of the brain's memory system that provides temporary storage and manipulation of information necessary for cognition. Although WM has limited capacity at any given time, it has vast memory content in the sense that it acts on the brain's nearly infinite repertoire of lifetime long-term memories. Using simulations, we show that large memory content and WM functionality emerge spontaneously if we take the spike-timing nature of neuronal processing into account. Here, memories are represented by extensively overlapping groups of neurons that exhibit stereotypical time-locked spatiotemporal spike-timing patterns, called polychronous patterns; and synapses forming such polychronous neuronal groups (PNGs are subject to associative synaptic plasticity in the form of both long-term and short-term spike-timing dependent plasticity. While long-term potentiation is essential in PNG formation, we show how short-term plasticity can temporarily strengthen the synapses of selected PNGs and lead to an increase in the spontaneous reactivation rate of these PNGs. This increased reactivation rate, consistent with in vivo recordings during WM tasks, results in high interspike interval variability and irregular, yet systematically changing, elevated firing rate profiles within the neurons of the selected PNGs. Additionally, our theory explains the relationship between such slowly changing firing rates and precisely timed spikes, and it reveals a novel relationship between WM and the perception of time on the order of seconds.

GABAergic activities control spike timing- and frequency-dependent long-term depression at hippocampal excitatory synapses

Directory of Open Access Journals (Sweden)

Makoto Nishiyama

2010-06-01

Full Text Available GABAergic interneuronal network activities in the hippocampus control a variety of neural functions, including learning and memory, by regulating θ and γ oscillations. How these GABAergic activities at pre- and post-synaptic sites of hippocampal CA1 pyramidal cells differentially contribute to synaptic function and plasticity during their repetitive pre- and post-synaptic spiking at θ and γ oscillations is largely unknown. We show here that activities mediated by postsynaptic GABAARs and presynaptic GABABRs determine, respectively, the spike timing- and frequency-dependence of activity-induced synaptic modifications at Schaffer collateral-CA1 excitatory synapses. We demonstrate that both feedforward and feedback GABAAR-mediated inhibition in the postsynaptic cell controls the spike timing-dependent long-term depression of excitatory inputs (“e-LTD” at the θ frequency. We also show that feedback postsynaptic inhibition specifically causes e-LTD of inputs that induce small postsynaptic currents (<70 pA with LTP timing, thus enforcing the requirement of cooperativity for induction of long-term potentiation at excitatory inputs (“e-LTP”. Furthermore, under spike-timing protocols that induce e-LTP and e-LTD at excitatory synapses, we observed parallel induction of LTP and LTD at inhibitory inputs (“i-LTP” and “i-LTD” to the same postsynaptic cells. Finally, we show that presynaptic GABABR-mediated inhibition plays a major role in the induction of frequency-dependent e-LTD at α and β frequencies. These observations demonstrate the critical influence of GABAergic interneuronal network activities in regulating the spike timing and frequency dependences of long-term synaptic modifications in the hippocampus.

Effects of bursting dynamic features on the generation of multi-clustered structure of neural network with symmetric spike-timing-dependent plasticity learning rule

International Nuclear Information System (INIS)

Liu, Hui; Song, Yongduan; Xue, Fangzheng; Li, Xiumin

2015-01-01

In this paper, the generation of multi-clustered structure of self-organized neural network with different neuronal firing patterns, i.e., bursting or spiking, has been investigated. The initially all-to-all-connected spiking neural network or bursting neural network can be self-organized into clustered structure through the symmetric spike-timing-dependent plasticity learning for both bursting and spiking neurons. However, the time consumption of this clustering procedure of the burst-based self-organized neural network (BSON) is much shorter than the spike-based self-organized neural network (SSON). Our results show that the BSON network has more obvious small-world properties, i.e., higher clustering coefficient and smaller shortest path length than the SSON network. Also, the results of larger structure entropy and activity entropy of the BSON network demonstrate that this network has higher topological complexity and dynamical diversity, which benefits for enhancing information transmission of neural circuits. Hence, we conclude that the burst firing can significantly enhance the efficiency of clustering procedure and the emergent clustered structure renders the whole network more synchronous and therefore more sensitive to weak input. This result is further confirmed from its improved performance on stochastic resonance. Therefore, we believe that the multi-clustered neural network which self-organized from the bursting dynamics has high efficiency in information processing

Effects of bursting dynamic features on the generation of multi-clustered structure of neural network with symmetric spike-timing-dependent plasticity learning rule

Energy Technology Data Exchange (ETDEWEB)

Liu, Hui; Song, Yongduan; Xue, Fangzheng; Li, Xiumin, E-mail: xmli@cqu.edu.cn [Key Laboratory of Dependable Service Computing in Cyber Physical Society of Ministry of Education, Chongqing University, Chongqing 400044 (China); College of Automation, Chongqing University, Chongqing 400044 (China)

2015-11-15

In this paper, the generation of multi-clustered structure of self-organized neural network with different neuronal firing patterns, i.e., bursting or spiking, has been investigated. The initially all-to-all-connected spiking neural network or bursting neural network can be self-organized into clustered structure through the symmetric spike-timing-dependent plasticity learning for both bursting and spiking neurons. However, the time consumption of this clustering procedure of the burst-based self-organized neural network (BSON) is much shorter than the spike-based self-organized neural network (SSON). Our results show that the BSON network has more obvious small-world properties, i.e., higher clustering coefficient and smaller shortest path length than the SSON network. Also, the results of larger structure entropy and activity entropy of the BSON network demonstrate that this network has higher topological complexity and dynamical diversity, which benefits for enhancing information transmission of neural circuits. Hence, we conclude that the burst firing can significantly enhance the efficiency of clustering procedure and the emergent clustered structure renders the whole network more synchronous and therefore more sensitive to weak input. This result is further confirmed from its improved performance on stochastic resonance. Therefore, we believe that the multi-clustered neural network which self-organized from the bursting dynamics has high efficiency in information processing.

Multiple coherence resonances and synchronization transitions by time delay in adaptive scale-free neuronal networks with spike-timing-dependent plasticity

International Nuclear Information System (INIS)

Xie, Huijuan; Gong, Yubing

2017-01-01

In this paper, we numerically study the effect of spike-timing-dependent plasticity (STDP) on multiple coherence resonances (MCR) and synchronization transitions (ST) induced by time delay in adaptive scale-free Hodgkin–Huxley neuronal networks. It is found that STDP has a big influence on MCR and ST induced by time delay and on the effect of network average degree on the MCR and ST. MCR is enhanced or suppressed as the adjusting rate A p of STDP decreases or increases, and there is optimal A p by which ST becomes strongest. As network average degree 〈k〉 increases, ST is enhanced and there is optimal 〈k〉 at which MCR becomes strongest. Moreover, for a larger A p value, ST is enhanced more rapidly with increasing 〈k〉 and the optimal 〈k〉 for MCR increases. These results show that STDP can either enhance or suppress MCR, and there is optimal STDP that can most strongly enhance ST induced by time delay in the adaptive neuronal networks. These findings could find potential implication for the information processing and transmission in neural systems.

Learning-dependent plasticity in human auditory cortex during appetitive operant conditioning.

Science.gov (United States)

Puschmann, Sebastian; Brechmann, André; Thiel, Christiane M

2013-11-01

Animal experiments provide evidence that learning to associate an auditory stimulus with a reward causes representational changes in auditory cortex. However, most studies did not investigate the temporal formation of learning-dependent plasticity during the task but rather compared auditory cortex receptive fields before and after conditioning. We here present a functional magnetic resonance imaging study on learning-related plasticity in the human auditory cortex during operant appetitive conditioning. Participants had to learn to associate a specific category of frequency-modulated tones with a reward. Only participants who learned this association developed learning-dependent plasticity in left auditory cortex over the course of the experiment. No differential responses to reward predicting and nonreward predicting tones were found in auditory cortex in nonlearners. In addition, learners showed similar learning-induced differential responses to reward-predicting and nonreward-predicting tones in the ventral tegmental area and the nucleus accumbens, two core regions of the dopaminergic neurotransmitter system. This may indicate a dopaminergic influence on the formation of learning-dependent plasticity in auditory cortex, as it has been suggested by previous animal studies. Copyright © 2012 Wiley Periodicals, Inc.

Mirror Neurons Modeled Through Spike-Timing-Dependent Plasticity are Affected by Channelopathies Associated with Autism Spectrum Disorder.

Science.gov (United States)

Antunes, Gabriela; Faria da Silva, Samuel F; Simoes de Souza, Fabio M

2018-06-01

Mirror neurons fire action potentials both when the agent performs a certain behavior and watches someone performing a similar action. Here, we present an original mirror neuron model based on the spike-timing-dependent plasticity (STDP) between two morpho-electrical models of neocortical pyramidal neurons. Both neurons fired spontaneously with basal firing rate that follows a Poisson distribution, and the STDP between them was modeled by the triplet algorithm. Our simulation results demonstrated that STDP is sufficient for the rise of mirror neuron function between the pairs of neocortical neurons. This is a proof of concept that pairs of neocortical neurons associating sensory inputs to motor outputs could operate like mirror neurons. In addition, we used the mirror neuron model to investigate whether channelopathies associated with autism spectrum disorder could impair the modeled mirror function. Our simulation results showed that impaired hyperpolarization-activated cationic currents (Ih) affected the mirror function between the pairs of neocortical neurons coupled by STDP.

Spike timing regulation on the millisecond scale by distributed synaptic plasticity at the cerebellum input stage: a simulation study

Directory of Open Access Journals (Sweden)

Jesus A Garrido

2013-05-01

Full Text Available The way long-term synaptic plasticity regulates neuronal spike patterns is not completely understood. This issue is especially relevant for the cerebellum, which is endowed with several forms of long-term synaptic plasticity and has been predicted to operate as a timing and a learning machine. Here we have used a computational model to simulate the impact of multiple distributed synaptic weights in the cerebellar granular layer network. In response to mossy fiber bursts, synaptic weights at multiple connections played a crucial role to regulate spike number and positioning in granule cells. The weight at mossy fiber to granule cell synapses regulated the delay of the first spike and the weight at mossy fiber and parallel fiber to Golgi cell synapses regulated the duration of the time-window during which the first-spike could be emitted. Moreover, the weights of synapses controlling Golgi cell activation regulated the intensity of granule cell inhibition and therefore the number of spikes that could be emitted. First spike timing was regulated with millisecond precision and the number of spikes ranged from 0 to 3. Interestingly, different combinations of synaptic weights optimized either first-spike timing precision or spike number, efficiently controlling transmission and filtering properties. These results predict that distributed synaptic plasticity regulates the emission of quasi-digital spike patterns on the millisecond time scale and allows the cerebellar granular layer to flexibly control burst transmission along the mossy fiber pathway.

A re-examination of Hebbian-covariance rules and spike timing-dependent plasticity in cat visual cortex in vivo

Directory of Open Access Journals (Sweden)

Yves Frégnac

2010-12-01

Full Text Available Spike-Timing-Dependent Plasticity (STDP is considered as an ubiquitous rule for associative plasticity in cortical networks in vitro. However, limited supporting evidence for its functional role has been provided in vivo. In particular, there are very few studies demonstrating the co-occurence of synaptic efficiency changes and alteration of sensory responses in adult cortex during Hebbian or STDP protocols. We addressed this issue by reviewing and comparing the functional effects of two types of cellular conditioning in cat visual cortex. The first one, referred to as the covariance protocol, obeys a generalized Hebbian framework, by imposing, for different stimuli, supervised positive and negative changes in covariance between postsynaptic and presynaptic activity rates. The second protocol, based on intracellular recordings, replicated in vivo variants of the theta-burst paradigm (TBS, proven successful in inducing long-term potentiation (LTP in vitro. Since it was shown to impose a precise correlation delay between the electrically activated thalamic input and the TBS-induced postsynaptic spike, this protocol can be seen as a probe of causal (pre-before-post STDP. By choosing a thalamic region where the visual field representation was in retinotopic overlap with the intracellularly recorded cortical receptive field as the afferent site for supervised electrical stimulation, this protocol allowed to look for possible correlates between STDP and functional reorganization of the conditioned cortical receptive field. The rate-based covariance protocol induced significant and large amplitude changes in receptive field properties, in both kitten and adult V1 cortex. The TBS STDP-like protocol produced in the adult significant changes in the synaptic gain of the electrically activated thalamic pathway, but the statistical significance of the functional correlates was detectable mostly at the population level. Comparison of our observations with the

Burst-time-dependent plasticity robustly guides ON/OFF segregation in the lateral geniculate nucleus.

Directory of Open Access Journals (Sweden)

Julijana Gjorgjieva

2009-12-01

Full Text Available Spontaneous retinal activity (known as "waves" remodels synaptic connectivity to the lateral geniculate nucleus (LGN during development. Analysis of retinal waves recorded with multielectrode arrays in mouse suggested that a cue for the segregation of functionally distinct (ON and OFF retinal ganglion cells (RGCs in the LGN may be a desynchronization in their firing, where ON cells precede OFF cells by one second. Using the recorded retinal waves as input, with two different modeling approaches we explore timing-based plasticity rules for the evolution of synaptic weights to identify key features underlying ON/OFF segregation. First, we analytically derive a linear model for the evolution of ON and OFF weights, to understand how synaptic plasticity rules extract input firing properties to guide segregation. Second, we simulate postsynaptic activity with a nonlinear integrate-and-fire model to compare findings with the linear model. We find that spike-time-dependent plasticity, which modifies synaptic weights based on millisecond-long timing and order of pre- and postsynaptic spikes, fails to segregate ON and OFF retinal inputs in the absence of normalization. Implementing homeostatic mechanisms results in segregation, but only with carefully-tuned parameters. Furthermore, extending spike integration timescales to match the second-long input correlation timescales always leads to ON segregation because ON cells fire before OFF cells. We show that burst-time-dependent plasticity can robustly guide ON/OFF segregation in the LGN without normalization, by integrating pre- and postsynaptic bursts irrespective of their firing order and over second-long timescales. We predict that an LGN neuron will become ON- or OFF-responsive based on a local competition of the firing patterns of neighboring RGCs connecting to it. Finally, we demonstrate consistency with ON/OFF segregation in ferret, despite differences in the firing properties of retinal waves. Our

Motor Planning under Unpredictable Reward: Modulations of Movement Vigor and Primate Striatum Activity

Directory of Open Access Journals (Sweden)

Ioan eOpris

2011-05-01

Full Text Available Although reward probability is an important factor that shapes animal behavior, it is not well understood however, how the primate brain translates reward expectation into the vigor of movement (reaction time and speed. To address this question, we trained two monkeys in a reaction time task that required wrist movements in response to vibrotactile and visual stimuli, with a variable reward schedule. Correct performance was rewarded in 75 % of the trials. Monkeys were certain that they would be rewarded only in the trials immediately following withheld rewards. In these trials, the animals responded sooner and moved faster. Single-unit recordings from the dorsal striatum revealed that modulations in striatal neurons reflected such modulations of movement vigor. First, in the trials with certain rewards, striatal neurons modulated their firing rates earlier. Second, magnitudes of changes in neuronal firing rates depended on whether or not monkeys were certain about the reward. Third, these modulations depended on the sensory modality of the cue (visual vs. vibratory and/or movement direction (flexions vs. extensions. We conclude that dorsal striatum may be a part of the mechanism responsible for the modulation of movement vigor in response to changes of reward predictability.

Frequency-dependent glycinergic inhibition modulates plasticity in hippocampus.

Science.gov (United States)

Keck, Tara; Lillis, Kyle P; Zhou, Yu-Dong; White, John A

2008-07-16

Previous studies have demonstrated the presence of functional glycine receptors (GlyRs) in hippocampus. In this work, we examine the baseline activity and activity-dependent modulation of GlyRs in region CA1. We find that strychnine-sensitive GlyRs are open in the resting CA1 pyramidal cell, creating a state of tonic inhibition that "shunts" the magnitude of EPSPs evoked by electrical stimulation of the Schaffer collateral inputs. This GlyR-mediated shunting conductance is independent of the presynaptic stimulation rate; however, pairs of presynaptic and postsynaptic action potentials, repeated at frequencies above 5 Hz, reduce the GlyR-mediated conductance and increase peak EPSP magnitudes to levels at least 20% larger than those seen with presynaptic stimulation alone. We refer to this phenomenon as rate-dependent efficacy (RDE). Exogenous GlyR agonists (glycine, taurine) block RDE by preventing the closure of postsynaptic GlyRs. The GlyR antagonist strychnine blocks postsynaptic GlyRs under all conditions, occluding RDE. During RDE, GlyRs are less responsive to local glycine application, suggesting that a reduction in the number or sensitivity of membrane-inserted GlyRs underlies RDE. By extending the RDE induction protocol to include 500 paired presynaptic and postsynaptic spikes, we can induce long-term synaptic depression (LTD). Manipulations that lead to reduced functionality of GlyRs, either pharmacologically or through RDE, also lead to increased LTD. This result suggests that RDE contributes to long-term synaptic plasticity in the hippocampus.

Reward-dependent modulation of working memory in lateral prefrontal cortex.

Science.gov (United States)

Kennerley, Steven W; Wallis, Jonathan D

2009-03-11

Although research implicates lateral prefrontal cortex (PFC) in executive control and goal-directed behavior, it remains unclear how goals influence executive processes. One possibility is that goal-relevant information, such as expected rewards, could modulate the representation of information relating to executive control, thereby ensuring the efficient allocation of cognitive resources. To investigate this, we examined how reward modulated spatial working memory. Past studies investigating spatial working memory have focused on dorsolateral PFC, but this area only weakly connects with areas processing reward. Ventrolateral PFC has better connections in this regard. Thus, we contrasted the functional properties of single neurons in ventrolateral and dorsolateral PFC as two subjects performed a task that required them to hold spatial information in working memory under different expectancies of reward for correct performance. We balanced the order of presentation of spatial and reward information so we could assess the neuronal encoding of the two pieces of information independently and conjointly. Neurons in ventrolateral PFC encoded both spatial and reward information earlier, stronger and in a more sustained manner than neurons in dorsolateral PFC. Within ventrolateral PFC, spatial selectivity was more prevalent on the inferior convexity than within the principal sulcus. Finally, when reward increased spatial selectivity, behavioral performance improved, whereas when reward decreased spatial selectivity, behavioral performance deteriorated. These results suggest that ventrolateral PFC may be a locus whereby information about expected rewards can modulate information in working memory. The pattern of results is consistent with a role for ventrolateral PFC in attentional control.

Spiking Neural Networks Based on OxRAM Synapses for Real-Time Unsupervised Spike Sorting.

Science.gov (United States)

Werner, Thilo; Vianello, Elisa; Bichler, Olivier; Garbin, Daniele; Cattaert, Daniel; Yvert, Blaise; De Salvo, Barbara; Perniola, Luca

2016-01-01

In this paper, we present an alternative approach to perform spike sorting of complex brain signals based on spiking neural networks (SNN). The proposed architecture is suitable for hardware implementation by using resistive random access memory (RRAM) technology for the implementation of synapses whose low latency (spike sorting. This offers promising advantages to conventional spike sorting techniques for brain-computer interfaces (BCI) and neural prosthesis applications. Moreover, the ultra-low power consumption of the RRAM synapses of the spiking neural network (nW range) may enable the design of autonomous implantable devices for rehabilitation purposes. We demonstrate an original methodology to use Oxide based RRAM (OxRAM) as easy to program and low energy (Spike Timing Dependent Plasticity. Real spiking data have been recorded both intra- and extracellularly from an in-vitro preparation of the Crayfish sensory-motor system and used for validation of the proposed OxRAM based SNN. This artificial SNN is able to identify, learn, recognize and distinguish between different spike shapes in the input signal with a recognition rate about 90% without any supervision.

Time Resolution Dependence of Information Measures for Spiking Neurons: Scaling and Universality

Directory of Open Access Journals (Sweden)

James P Crutchfield

2015-08-01

Full Text Available The mutual information between stimulus and spike-train response is commonly used to monitor neural coding efficiency, but neuronal computation broadly conceived requires more refined and targeted information measures of input-output joint processes. A first step towards that larger goal is todevelop information measures for individual output processes, including information generation (entropy rate, stored information (statisticalcomplexity, predictable information (excess entropy, and active information accumulation (bound information rate. We calculate these for spike trains generated by a variety of noise-driven integrate-and-fire neurons as a function of time resolution and for alternating renewal processes. We show that their time-resolution dependence reveals coarse-grained structural properties of interspike interval statistics; e.g., $tau$-entropy rates that diverge less quickly than the firing rate indicate interspike interval correlations. We also find evidence that the excess entropy and regularized statistical complexity of different types of integrate-and-fire neurons are universal in the continuous-time limit in the sense that they do not depend on mechanism details. This suggests a surprising simplicity in the spike trains generated by these model neurons. Interestingly, neurons with gamma-distributed ISIs and neurons whose spike trains are alternating renewal processes do not fall into the same universality class. These results lead to two conclusions. First, the dependence of information measures on time resolution reveals mechanistic details about spike train generation. Second, information measures can be used as model selection tools for analyzing spike train processes.

Stochastic Variational Learning in Recurrent Spiking Networks

Directory of Open Access Journals (Sweden)

Danilo eJimenez Rezende

2014-04-01

Full Text Available The ability to learn and perform statistical inference with biologically plausible recurrent network of spiking neurons is an important step towards understanding perception and reasoning. Here we derive and investigate a new learning rule for recurrent spiking networks with hidden neurons, combining principles from variational learning and reinforcement learning. Our network defines a generative model over spike train histories and the derived learning rule has the form of a local Spike Timing Dependent Plasticity rule modulated by global factors (neuromodulators conveying information about ``novelty on a statistically rigorous ground.Simulations show that our model is able to learn bothstationary and non-stationary patterns of spike trains.We also propose one experiment that could potentially be performed with animals in order to test the dynamics of the predicted novelty signal.

Stochastic variational learning in recurrent spiking networks.

Science.gov (United States)

Jimenez Rezende, Danilo; Gerstner, Wulfram

2014-01-01

The ability to learn and perform statistical inference with biologically plausible recurrent networks of spiking neurons is an important step toward understanding perception and reasoning. Here we derive and investigate a new learning rule for recurrent spiking networks with hidden neurons, combining principles from variational learning and reinforcement learning. Our network defines a generative model over spike train histories and the derived learning rule has the form of a local Spike Timing Dependent Plasticity rule modulated by global factors (neuromodulators) conveying information about "novelty" on a statistically rigorous ground. Simulations show that our model is able to learn both stationary and non-stationary patterns of spike trains. We also propose one experiment that could potentially be performed with animals in order to test the dynamics of the predicted novelty signal.

Is a 4-bit synaptic weight resolution enough? - Constraints on enabling spike-timing dependent plasticity in neuromorphic hardware

Directory of Open Access Journals (Sweden)

Thomas ePfeil

2012-07-01

Full Text Available Large-scale neuromorphic hardware systems typically bear the trade-off be-tween detail level and required chip resources. Especially when implementingspike-timing-dependent plasticity, reduction in resources leads to limitations ascompared to floating point precision. By design, a natural modification that savesresources would be reducing synaptic weight resolution. In this study, we give anestimate for the impact of synaptic weight discretization on different levels, rangingfrom random walks of individual weights to computer simulations of spiking neuralnetworks. The FACETS wafer-scale hardware system offers a 4-bit resolution ofsynaptic weights, which is shown to be sufficient within the scope of our networkbenchmark. Our findings indicate that increasing the resolution may not even beuseful in light of further restrictions of customized mixed-signal synapses. In ad-dition, variations due to production imperfections are investigated and shown tobe uncritical in the context of the presented study. Our results represent a generalframework for setting up and configuring hardware-constrained synapses. We sug-gest how weight discretization could be considered for other backends dedicatedto large-scale simulations. Thus, our proposition of a good hardware verificationpractice may rise synergy effects between hardware developers and neuroscientists.

Timing is not everything: neuromodulation opens the STDP gate

Directory of Open Access Journals (Sweden)

Verena Pawlak

2010-10-01

Full Text Available Spike timing dependent plasticity (STDP is a temporally specific extension of Hebbian associative plasticity that has tied together the timing of presynaptic inputs relative to the postsynaptic single spike. However, it is difficult to translate this mechanism to in vivo conditions where there is an abundance of presynaptic activity constantly impinging upon the dendritic tree as well as ongoing postsynaptic spiking activity that backpropagates along the dendrite. Theoretical studies have proposed that, in addition to this pre- and postsynaptic activity, a ‘third factor’ would enable the association of specific inputs to specific outputs. Experimentally, the picture that is beginning to emerge, is that in addition to the precise timing of pre- and postsynaptic spikes, this third factor involves neuromodulators that have a distinctive influence on STDP rules. Specifically, neuromodulatory systems can influence STDP rules by acting via dopaminergic, noradrenergic, muscarinic and nicotinic receptors. Neuromodulator actions can enable STDP induction or - by increasing or decreasing the threshold - can change the conditions for plasticity induction. Because some of the neuromodulators are also involved in reward, a link between STDP and reward-mediated learning is emerging. However, many outstanding questions concerning the relationship between neuromodulatory systems and STDP rules remain, that once solved, will help make the crucial link from timing-based synaptic plasticity rules to behaviorally-based learning.

Synaptic plasticity in drug reward circuitry.

Science.gov (United States)

Winder, Danny G; Egli, Regula E; Schramm, Nicole L; Matthews, Robert T

2002-11-01

Drug addiction is a major public health issue worldwide. The persistence of drug craving coupled with the known recruitment of learning and memory centers in the brain has led investigators to hypothesize that the alterations in glutamatergic synaptic efficacy brought on by synaptic plasticity may play key roles in the addiction process. Here we review the present literature, examining the properties of synaptic plasticity within drug reward circuitry, and the effects that drugs of abuse have on these forms of plasticity. Interestingly, multiple forms of synaptic plasticity can be induced at glutamatergic synapses within the dorsal striatum, its ventral extension the nucleus accumbens, and the ventral tegmental area, and at least some of these forms of plasticity are regulated by behaviorally meaningful administration of cocaine and/or amphetamine. Thus, the present data suggest that regulation of synaptic plasticity in reward circuits is a tractable candidate mechanism underlying aspects of addiction.

Computational modeling of spiking neural network with learning rules from STDP and intrinsic plasticity

Science.gov (United States)

Li, Xiumin; Wang, Wei; Xue, Fangzheng; Song, Yongduan

2018-02-01

Recently there has been continuously increasing interest in building up computational models of spiking neural networks (SNN), such as the Liquid State Machine (LSM). The biologically inspired self-organized neural networks with neural plasticity can enhance the capability of computational performance, with the characteristic features of dynamical memory and recurrent connection cycles which distinguish them from the more widely used feedforward neural networks. Despite a variety of computational models for brain-like learning and information processing have been proposed, the modeling of self-organized neural networks with multi-neural plasticity is still an important open challenge. The main difficulties lie in the interplay among different forms of neural plasticity rules and understanding how structures and dynamics of neural networks shape the computational performance. In this paper, we propose a novel approach to develop the models of LSM with a biologically inspired self-organizing network based on two neural plasticity learning rules. The connectivity among excitatory neurons is adapted by spike-timing-dependent plasticity (STDP) learning; meanwhile, the degrees of neuronal excitability are regulated to maintain a moderate average activity level by another learning rule: intrinsic plasticity (IP). Our study shows that LSM with STDP+IP performs better than LSM with a random SNN or SNN obtained by STDP alone. The noticeable improvement with the proposed method is due to the better reflected competition among different neurons in the developed SNN model, as well as the more effectively encoded and processed relevant dynamic information with its learning and self-organizing mechanism. This result gives insights to the optimization of computational models of spiking neural networks with neural plasticity.

Spike-Timing Dependent Plasticity and the Cognitive Map

OpenAIRE

Bush, Daniel; Philippides, Andrew; Husbands, Phil; O'Shea, Michael

2010-01-01

Since the discovery of place cells – single pyramidal neurons that encode spatial location – it has been hypothesized that the hippocampus may act as a cognitive map of known environments. This putative function has been extensively modeled using auto-associative networks, which utilize rate-coded synaptic plasticity rules in order to generate strong bi-directional connections between concurrently active place cells that encode for neighboring place fields. However, empirical studies using hi...

Modeling spiking behavior of neurons with time-dependent Poisson processes.

Science.gov (United States)

Shinomoto, S; Tsubo, Y

2001-10-01

Three kinds of interval statistics, as represented by the coefficient of variation, the skewness coefficient, and the correlation coefficient of consecutive intervals, are evaluated for three kinds of time-dependent Poisson processes: pulse regulated, sinusoidally regulated, and doubly stochastic. Among these three processes, the sinusoidally regulated and doubly stochastic Poisson processes, in the case when the spike rate varies slowly compared with the mean interval between spikes, are found to be consistent with the three statistical coefficients exhibited by data recorded from neurons in the prefrontal cortex of monkeys.

Learning to Generate Sequences with Combination of Hebbian and Non-hebbian Plasticity in Recurrent Spiking Neural Networks.

Science.gov (United States)

Panda, Priyadarshini; Roy, Kaushik

2017-01-01

Synaptic Plasticity, the foundation for learning and memory formation in the human brain, manifests in various forms. Here, we combine the standard spike timing correlation based Hebbian plasticity with a non-Hebbian synaptic decay mechanism for training a recurrent spiking neural model to generate sequences. We show that inclusion of the adaptive decay of synaptic weights with standard STDP helps learn stable contextual dependencies between temporal sequences, while reducing the strong attractor states that emerge in recurrent models due to feedback loops. Furthermore, we show that the combined learning scheme suppresses the chaotic activity in the recurrent model substantially, thereby enhancing its' ability to generate sequences consistently even in the presence of perturbations.

Oscillation-Driven Spike-Timing Dependent Plasticity Allows Multiple Overlapping Pattern Recognition in Inhibitory Interneuron Networks

DEFF Research Database (Denmark)

Garrido, Jesús A.; Luque, Niceto R.; Tolu, Silvia

2016-01-01

The majority of operations carried out by the brain require learning complex signal patterns for future recognition, retrieval and reuse. Although learning is thought to depend on multiple forms of long-term synaptic plasticity, the way this latter contributes to pattern recognition is still poorly...... and at the inhibitory interneuron-interneuron synapses, the interneurons rapidly learned complex input patterns. Interestingly, induction of plasticity required that the network be entrained into theta-frequency band oscillations, setting the internal phase-reference required to drive STDP. Inhibitory plasticity...... effectively distributed multiple patterns among available interneurons, thus allowing the simultaneous detection of multiple overlapping patterns. The addition of plasticity in intrinsic excitability made the system more robust allowing self-adjustment and rescaling in response to a broad range of input...

Reinforcement learning of targeted movement in a spiking neuronal model of motor cortex.

Directory of Open Access Journals (Sweden)

George L Chadderdon

Full Text Available Sensorimotor control has traditionally been considered from a control theory perspective, without relation to neurobiology. In contrast, here we utilized a spiking-neuron model of motor cortex and trained it to perform a simple movement task, which consisted of rotating a single-joint "forearm" to a target. Learning was based on a reinforcement mechanism analogous to that of the dopamine system. This provided a global reward or punishment signal in response to decreasing or increasing distance from hand to target, respectively. Output was partially driven by Poisson motor babbling, creating stochastic movements that could then be shaped by learning. The virtual forearm consisted of a single segment rotated around an elbow joint, controlled by flexor and extensor muscles. The model consisted of 144 excitatory and 64 inhibitory event-based neurons, each with AMPA, NMDA, and GABA synapses. Proprioceptive cell input to this model encoded the 2 muscle lengths. Plasticity was only enabled in feedforward connections between input and output excitatory units, using spike-timing-dependent eligibility traces for synaptic credit or blame assignment. Learning resulted from a global 3-valued signal: reward (+1, no learning (0, or punishment (-1, corresponding to phasic increases, lack of change, or phasic decreases of dopaminergic cell firing, respectively. Successful learning only occurred when both reward and punishment were enabled. In this case, 5 target angles were learned successfully within 180 s of simulation time, with a median error of 8 degrees. Motor babbling allowed exploratory learning, but decreased the stability of the learned behavior, since the hand continued moving after reaching the target. Our model demonstrated that a global reinforcement signal, coupled with eligibility traces for synaptic plasticity, can train a spiking sensorimotor network to perform goal-directed motor behavior.

Reinforcement learning of targeted movement in a spiking neuronal model of motor cortex.

Science.gov (United States)

Chadderdon, George L; Neymotin, Samuel A; Kerr, Cliff C; Lytton, William W

2012-01-01

Sensorimotor control has traditionally been considered from a control theory perspective, without relation to neurobiology. In contrast, here we utilized a spiking-neuron model of motor cortex and trained it to perform a simple movement task, which consisted of rotating a single-joint "forearm" to a target. Learning was based on a reinforcement mechanism analogous to that of the dopamine system. This provided a global reward or punishment signal in response to decreasing or increasing distance from hand to target, respectively. Output was partially driven by Poisson motor babbling, creating stochastic movements that could then be shaped by learning. The virtual forearm consisted of a single segment rotated around an elbow joint, controlled by flexor and extensor muscles. The model consisted of 144 excitatory and 64 inhibitory event-based neurons, each with AMPA, NMDA, and GABA synapses. Proprioceptive cell input to this model encoded the 2 muscle lengths. Plasticity was only enabled in feedforward connections between input and output excitatory units, using spike-timing-dependent eligibility traces for synaptic credit or blame assignment. Learning resulted from a global 3-valued signal: reward (+1), no learning (0), or punishment (-1), corresponding to phasic increases, lack of change, or phasic decreases of dopaminergic cell firing, respectively. Successful learning only occurred when both reward and punishment were enabled. In this case, 5 target angles were learned successfully within 180 s of simulation time, with a median error of 8 degrees. Motor babbling allowed exploratory learning, but decreased the stability of the learned behavior, since the hand continued moving after reaching the target. Our model demonstrated that a global reinforcement signal, coupled with eligibility traces for synaptic plasticity, can train a spiking sensorimotor network to perform goal-directed motor behavior.

Reward-based learning under hardware constraints-using a RISC processor embedded in a neuromorphic substrate.

Science.gov (United States)

Friedmann, Simon; Frémaux, Nicolas; Schemmel, Johannes; Gerstner, Wulfram; Meier, Karlheinz

2013-01-01

In this study, we propose and analyze in simulations a new, highly flexible method of implementing synaptic plasticity in a wafer-scale, accelerated neuromorphic hardware system. The study focuses on globally modulated STDP, as a special use-case of this method. Flexibility is achieved by embedding a general-purpose processor dedicated to plasticity into the wafer. To evaluate the suitability of the proposed system, we use a reward modulated STDP rule in a spike train learning task. A single layer of neurons is trained to fire at specific points in time with only the reward as feedback. This model is simulated to measure its performance, i.e., the increase in received reward after learning. Using this performance as baseline, we then simulate the model with various constraints imposed by the proposed implementation and compare the performance. The simulated constraints include discretized synaptic weights, a restricted interface between analog synapses and embedded processor, and mismatch of analog circuits. We find that probabilistic updates can increase the performance of low-resolution weights, a simple interface between analog synapses and processor is sufficient for learning, and performance is insensitive to mismatch. Further, we consider communication latency between wafer and the conventional control computer system that is simulating the environment. This latency increases the delay, with which the reward is sent to the embedded processor. Because of the time continuous operation of the analog synapses, delay can cause a deviation of the updates as compared to the not delayed situation. We find that for highly accelerated systems latency has to be kept to a minimum. This study demonstrates the suitability of the proposed implementation to emulate the selected reward modulated STDP learning rule. It is therefore an ideal candidate for implementation in an upgraded version of the wafer-scale system developed within the BrainScaleS project.

Higher Order Spike Synchrony in Prefrontal Cortex during visual memory

Directory of Open Access Journals (Sweden)

Gordon ePipa

2011-06-01

Full Text Available Precise temporal synchrony of spike firing has been postulated as an important neuronal mechanism for signal integration and the induction of plasticity in neocortex. As prefrontal cortex plays an important role in organizing memory and executive functions, the convergence of multiple visual pathways onto PFC predicts that neurons should preferentially synchronize their spiking when stimulus information is processed. Furthermore, synchronous spike firing should intensify if memory processes require the induction of neuronal plasticity, even if this is only for short-term. Here we show with multiple simultaneously recorded units in ventral prefrontal cortex that neurons participate in 3 ms precise synchronous discharges distributed across multiple sites separated by at least 500 µm. The frequency of synchronous firing is modulated by behavioral performance and is specific for the memorized visual stimuli. In particular, during the memory period in which activity is not stimulus driven, larger groups of up to 7 sites exhibit performance dependent modulation of their spike synchronization.

Inferring oscillatory modulation in neural spike trains.

Science.gov (United States)

Arai, Kensuke; Kass, Robert E

2017-10-01

Oscillations are observed at various frequency bands in continuous-valued neural recordings like the electroencephalogram (EEG) and local field potential (LFP) in bulk brain matter, and analysis of spike-field coherence reveals that spiking of single neurons often occurs at certain phases of the global oscillation. Oscillatory modulation has been examined in relation to continuous-valued oscillatory signals, and independently from the spike train alone, but behavior or stimulus triggered firing-rate modulation, spiking sparseness, presence of slow modulation not locked to stimuli and irregular oscillations with large variability in oscillatory periods, present challenges to searching for temporal structures present in the spike train. In order to study oscillatory modulation in real data collected under a variety of experimental conditions, we describe a flexible point-process framework we call the Latent Oscillatory Spike Train (LOST) model to decompose the instantaneous firing rate in biologically and behaviorally relevant factors: spiking refractoriness, event-locked firing rate non-stationarity, and trial-to-trial variability accounted for by baseline offset and a stochastic oscillatory modulation. We also extend the LOST model to accommodate changes in the modulatory structure over the duration of the experiment, and thereby discover trial-to-trial variability in the spike-field coherence of a rat primary motor cortical neuron to the LFP theta rhythm. Because LOST incorporates a latent stochastic auto-regressive term, LOST is able to detect oscillations when the firing rate is low, the modulation is weak, and when the modulating oscillation has a broad spectral peak.

Spike-timing computation properties of a feed-forward neural network model

Directory of Open Access Journals (Sweden)

Drew Benjamin Sinha

2014-01-01

Full Text Available Brain function is characterized by dynamical interactions among networks of neurons. These interactions are mediated by network topology at many scales ranging from microcircuits to brain areas. Understanding how networks operate can be aided by understanding how the transformation of inputs depends upon network connectivity patterns, e.g. serial and parallel pathways. To tractably determine how single synapses or groups of synapses in such pathways shape transformations, we modeled feed-forward networks of 7-22 neurons in which synaptic strength changed according to a spike-timing dependent plasticity rule. We investigated how activity varied when dynamics were perturbed by an activity-dependent electrical stimulation protocol (spike-triggered stimulation; STS in networks of different topologies and background input correlations. STS can successfully reorganize functional brain networks in vivo, but with a variability in effectiveness that may derive partially from the underlying network topology. In a simulated network with a single disynaptic pathway driven by uncorrelated background activity, structured spike-timing relationships between polysynaptically connected neurons were not observed. When background activity was correlated or parallel disynaptic pathways were added, however, robust polysynaptic spike timing relationships were observed, and application of STS yielded predictable changes in synaptic strengths and spike-timing relationships. These observations suggest that precise input-related or topologically induced temporal relationships in network activity are necessary for polysynaptic signal propagation. Such constraints for polysynaptic computation suggest potential roles for higher-order topological structure in network organization, such as maintaining polysynaptic correlation in the face of relatively weak synapses.

Synaptic convergence regulates synchronization-dependent spike transfer in feedforward neural networks.

Science.gov (United States)

Sailamul, Pachaya; Jang, Jaeson; Paik, Se-Bum

2017-12-01

Correlated neural activities such as synchronizations can significantly alter the characteristics of spike transfer between neural layers. However, it is not clear how this synchronization-dependent spike transfer can be affected by the structure of convergent feedforward wiring. To address this question, we implemented computer simulations of model neural networks: a source and a target layer connected with different types of convergent wiring rules. In the Gaussian-Gaussian (GG) model, both the connection probability and the strength are given as Gaussian distribution as a function of spatial distance. In the Uniform-Constant (UC) and Uniform-Exponential (UE) models, the connection probability density is a uniform constant within a certain range, but the connection strength is set as a constant value or an exponentially decaying function, respectively. Then we examined how the spike transfer function is modulated under these conditions, while static or synchronized input patterns were introduced to simulate different levels of feedforward spike synchronization. We observed that the synchronization-dependent modulation of the transfer function appeared noticeably different for each convergence condition. The modulation of the spike transfer function was largest in the UC model, and smallest in the UE model. Our analysis showed that this difference was induced by the different spike weight distributions that was generated from convergent synapses in each model. Our results suggest that, the structure of the feedforward convergence is a crucial factor for correlation-dependent spike control, thus must be considered important to understand the mechanism of information transfer in the brain.

Modulation of neural activity by reward in medial intraparietal cortex is sensitive to temporal sequence of reward

Science.gov (United States)

Rajalingham, Rishi; Stacey, Richard Greg; Tsoulfas, Georgios

2014-01-01

To restore movements to paralyzed patients, neural prosthetic systems must accurately decode patients' intentions from neural signals. Despite significant advancements, current systems are unable to restore complex movements. Decoding reward-related signals from the medial intraparietal area (MIP) could enhance prosthetic performance. However, the dynamics of reward sensitivity in MIP is not known. Furthermore, reward-related modulation in premotor areas has been attributed to behavioral confounds. Here we investigated the stability of reward encoding in MIP by assessing the effect of reward history on reward sensitivity. We recorded from neurons in MIP while monkeys performed a delayed-reach task under two reward schedules. In the variable schedule, an equal number of small- and large-rewards trials were randomly interleaved. In the constant schedule, one reward size was delivered for a block of trials. The memory period firing rate of most neurons in response to identical rewards varied according to schedule. Using systems identification tools, we attributed the schedule sensitivity to the dependence of neural activity on the history of reward. We did not find schedule-dependent behavioral changes, suggesting that reward modulates neural activity in MIP. Neural discrimination between rewards was less in the variable than in the constant schedule, degrading our ability to decode reach target and reward simultaneously. The effect of schedule was mitigated by adding Haar wavelet coefficients to the decoding model. This raises the possibility of multiple encoding schemes at different timescales and reinforces the potential utility of reward information for prosthetic performance. PMID:25008408

Modulation of neural activity by reward in medial intraparietal cortex is sensitive to temporal sequence of reward.

Science.gov (United States)

Rajalingham, Rishi; Stacey, Richard Greg; Tsoulfas, Georgios; Musallam, Sam

2014-10-01

To restore movements to paralyzed patients, neural prosthetic systems must accurately decode patients' intentions from neural signals. Despite significant advancements, current systems are unable to restore complex movements. Decoding reward-related signals from the medial intraparietal area (MIP) could enhance prosthetic performance. However, the dynamics of reward sensitivity in MIP is not known. Furthermore, reward-related modulation in premotor areas has been attributed to behavioral confounds. Here we investigated the stability of reward encoding in MIP by assessing the effect of reward history on reward sensitivity. We recorded from neurons in MIP while monkeys performed a delayed-reach task under two reward schedules. In the variable schedule, an equal number of small- and large-rewards trials were randomly interleaved. In the constant schedule, one reward size was delivered for a block of trials. The memory period firing rate of most neurons in response to identical rewards varied according to schedule. Using systems identification tools, we attributed the schedule sensitivity to the dependence of neural activity on the history of reward. We did not find schedule-dependent behavioral changes, suggesting that reward modulates neural activity in MIP. Neural discrimination between rewards was less in the variable than in the constant schedule, degrading our ability to decode reach target and reward simultaneously. The effect of schedule was mitigated by adding Haar wavelet coefficients to the decoding model. This raises the possibility of multiple encoding schemes at different timescales and reinforces the potential utility of reward information for prosthetic performance. Copyright © 2014 the American Physiological Society.

Double dissociation of spike timing-dependent potentiation and depression by subunit-preferring NMDA receptor antagonists in mouse barrel cortex.

Science.gov (United States)

Banerjee, Abhishek; Meredith, Rhiannon M; Rodríguez-Moreno, Antonio; Mierau, Susanna B; Auberson, Yves P; Paulsen, Ole

2009-12-01

Spike timing-dependent plasticity (STDP) is a strong candidate for an N-methyl-D-aspartate (NMDA) receptor-dependent form of synaptic plasticity that could underlie the development of receptive field properties in sensory neocortices. Whilst induction of timing-dependent long-term potentiation (t-LTP) requires postsynaptic NMDA receptors, timing-dependent long-term depression (t-LTD) requires the activation of presynaptic NMDA receptors at layer 4-to-layer 2/3 synapses in barrel cortex. Here we investigated the developmental profile of t-LTD at layer 4-to-layer 2/3 synapses of mouse barrel cortex and studied their NMDA receptor subunit dependence. Timing-dependent LTD emerged in the first postnatal week, was present during the second week and disappeared in the adult, whereas t-LTP persisted in adulthood. An antagonist at GluN2C/D subunit-containing NMDA receptors blocked t-LTD but not t-LTP. Conversely, a GluN2A subunit-preferring antagonist blocked t-LTP but not t-LTD. The GluN2C/D subunit requirement for t-LTD appears to be synapse specific, as GluN2C/D antagonists did not block t-LTD at horizontal cross-columnar layer 2/3-to-layer 2/3 synapses, which was blocked by a GluN2B antagonist instead. These data demonstrate an NMDA receptor subunit-dependent double dissociation of t-LTD and t-LTP mechanisms at layer 4-to-layer 2/3 synapses, and suggest that t-LTD is mediated by distinct molecular mechanisms at different synapses on the same postsynaptic neuron.

Possible evidence for re-regulation of HPA axis and brain reward systems over time in treatment in prescription opioid-dependent patients.

Science.gov (United States)

Bunce, Scott C; Harris, Jonathan D; Bixler, Edward O; Taylor, Megan; Muelly, Emilie; Deneke, Erin; Thompson, Kenneth W; Meyer, Roger E

2015-01-01

There is growing evidence for a neuroadaptive model underlying vulnerability to relapse in opioid dependence. The purpose of this study was to evaluate clinical measures hypothesized to mirror elements of allostatic dysregulation in patients dependent on prescription opioids at 2 time points after withdrawal, compared with healthy control participants. Recently withdrawn (n = 7) prescription opioid-dependent patients were compared with the patients in supervised residential care for 2 to 3 months (extended care; n = 7) and healthy controls (n = 7) using drug cue reactivity, affect-modulated startle response tasks, salivary cortisol, and 8 days of sleep actigraphy. Prefrontal cortex was monitored with functional near-infrared spectroscopy during the cue reactivity task. Startle response results indicated reduced hedonic response to natural rewards among patients recently withdrawn from opioids relative to extended care patients. The recently withdrawn patients showed increased activation to pill stimuli in right dorsolateral prefrontal cortex relative to extended care patients. Cortisol levels were elevated among recently withdrawn patients and intermediate for extended care relative to healthy controls. Actigraphy indicated disturbed sleep between recently withdrawn patients and extended care patients; extended care patients were similar to controls. Dorsolateral prefrontal cortex activation to drug and natural reward cues, startle responses to natural reward cues, day-time cortisol levels, time in bed, and total time spent sleeping were all correlated with the number of days since last drug use (ie, time in supervised residential treatment). These results suggest possible re-regulation of dysregulated hypothalamic-pituitary-adrenal axis and brain reward systems in prescription opioid-dependent patients over the drug-free period in residential treatment.

Cellular Signal Mechanisms of Reward-Related Plasticity in the Hippocampus

Directory of Open Access Journals (Sweden)

Masako Isokawa

2012-01-01

Full Text Available The hippocampus has the extraordinary capacity to process and store information. Consequently, there is an intense interest in the mechanisms that underline learning and memory. Synaptic plasticity has been hypothesized to be the neuronal substrate for learning. Ca2+ and Ca2+-activated kinases control cellular processes of most forms of hippocampal synapse plasticity. In this paper, I aim to integrate our current understanding of Ca2+-mediated synaptic plasticity and metaplasticity in motivational and reward-related learning in the hippocampus. I will introduce two representative neuromodulators that are widely studied in reward-related learning (e.g., ghrelin and endocannabinoids and show how they might contribute to hippocampal neuron activities and Ca2+-mediated signaling processes in synaptic plasticity. Additionally, I will discuss functional significance of these two systems and their signaling pathways for its relevance to maladaptive reward learning leading to addiction.

Reward modulates perception in binocular rivalry.

Science.gov (United States)

Marx, Svenja; Einhäuser, Wolfgang

2015-01-14

Our perception does not provide us with an exact imprint of the outside world, but is continuously adapted to our internal expectations, task sets, and behavioral goals. Although effects of reward-or value in general-on perception therefore seem likely, how valuation modulates perception and how such modulation relates to attention is largely unknown. We probed effects of reward on perception by using a binocular-rivalry paradigm. Distinct gratings drifting in opposite directions were presented to each observer's eyes. To objectify their subjective perceptual experience, the optokinetic nystagmus was used as measure of current perceptual dominance. In a first experiment, one of the percepts was either rewarded or attended. We found that reward and attention similarly biased perception. In a second experiment, observers performed an attentionally demanding task either on the rewarded stimulus, the other stimulus, or both. We found that-on top of an attentional effect on perception-at each level of attentional load, reward still modulated perception by increasing the dominance of the rewarded percept. Similarly, penalizing one percept increased dominance of the other at each level of attentional load. In turn, rewarding-and similarly nonpunishing-a percept yielded performance benefits that are typically associated with selective attention. In conclusion, our data show that value modulates perception in a similar way as the volitional deployment of attention, even though the relative effect of value is largely unaffected by an attention task. © 2015 ARVO.

Hybrid Spintronic-CMOS Spiking Neural Network with On-Chip Learning: Devices, Circuits, and Systems

Science.gov (United States)

Sengupta, Abhronil; Banerjee, Aparajita; Roy, Kaushik

2016-12-01

Over the past decade, spiking neural networks (SNNs) have emerged as one of the popular architectures to emulate the brain. In SNNs, information is temporally encoded and communication between neurons is accomplished by means of spikes. In such networks, spike-timing-dependent plasticity mechanisms require the online programing of synapses based on the temporal information of spikes transmitted by spiking neurons. In this work, we propose a spintronic synapse with decoupled spike-transmission and programing-current paths. The spintronic synapse consists of a ferromagnet-heavy-metal heterostructure where the programing current through the heavy metal generates spin-orbit torque to modulate the device conductance. Low programing energy and fast programing times demonstrate the efficacy of the proposed device as a nanoelectronic synapse. We perform a simulation study based on an experimentally benchmarked device-simulation framework to demonstrate the interfacing of such spintronic synapses with CMOS neurons and learning circuits operating in the transistor subthreshold region to form a network of spiking neurons that can be utilized for pattern-recognition problems.

Goal-Directed Decision Making with Spiking Neurons.

Science.gov (United States)

Friedrich, Johannes; Lengyel, Máté

2016-02-03

Behavioral and neuroscientific data on reward-based decision making point to a fundamental distinction between habitual and goal-directed action selection. The formation of habits, which requires simple updating of cached values, has been studied in great detail, and the reward prediction error theory of dopamine function has enjoyed prominent success in accounting for its neural bases. In contrast, the neural circuit mechanisms of goal-directed decision making, requiring extended iterative computations to estimate values online, are still unknown. Here we present a spiking neural network that provably solves the difficult online value estimation problem underlying goal-directed decision making in a near-optimal way and reproduces behavioral as well as neurophysiological experimental data on tasks ranging from simple binary choice to sequential decision making. Our model uses local plasticity rules to learn the synaptic weights of a simple neural network to achieve optimal performance and solves one-step decision-making tasks, commonly considered in neuroeconomics, as well as more challenging sequential decision-making tasks within 1 s. These decision times, and their parametric dependence on task parameters, as well as the final choice probabilities match behavioral data, whereas the evolution of neural activities in the network closely mimics neural responses recorded in frontal cortices during the execution of such tasks. Our theory provides a principled framework to understand the neural underpinning of goal-directed decision making and makes novel predictions for sequential decision-making tasks with multiple rewards. Goal-directed actions requiring prospective planning pervade decision making, but their circuit-level mechanisms remain elusive. We show how a model circuit of biologically realistic spiking neurons can solve this computationally challenging problem in a novel way. The synaptic weights of our network can be learned using local plasticity rules

A Plastic Cortico-Striatal Circuit Model of Adaptation in Perceptual Decision

Directory of Open Access Journals (Sweden)

Pao-Yueh eHsiao

2013-12-01

Full Text Available The ability to optimize decisions and adapt them to changing environments is a crucial brain function that increase survivability. Although much has been learned about the neuronal activity in various brain regions that are associated with decision making, and about how the nervous systems may learn to achieve optimization, the underlying neuronal mechanisms of how the nervous systems optimize decision strategies with preference given to speed or accuracy, and how the systems adapt to changes in the environment, remain unclear. Based on extensive empirical observations, we addressed the question by extending a previously described cortico-basal ganglia circuit model of perceptual decisions with the inclusion of a dynamic dopamine (DA system that modulates spike-timing dependent plasticity. We found that, once an optimal model setting that maximized the reward rate was selected, the same setting automatically optimized decisions across different task environments through dynamic balancing between the facilitating and depressing components of the DA dynamics. Interestingly, other model parameters were also optimal if we considered the reward rate that was weighted by the subject’s preferences for speed or accuracy. Specifically, the circuit model favored speed if we increased the phasic DA response to the reward prediction error, whereas the model favored accuracy if we reduced the tonic DA activity or the phasic DA responses to the estimated reward probability. The proposed model provides insight into the roles of different components of DA responses in decision adaptation and optimization in a changing environment.

Serial Spike Time Correlations Affect Probability Distribution of Joint Spike Events.

Science.gov (United States)

Shahi, Mina; van Vreeswijk, Carl; Pipa, Gordon

2016-01-01

Detecting the existence of temporally coordinated spiking activity, and its role in information processing in the cortex, has remained a major challenge for neuroscience research. Different methods and approaches have been suggested to test whether the observed synchronized events are significantly different from those expected by chance. To analyze the simultaneous spike trains for precise spike correlation, these methods typically model the spike trains as a Poisson process implying that the generation of each spike is independent of all the other spikes. However, studies have shown that neural spike trains exhibit dependence among spike sequences, such as the absolute and relative refractory periods which govern the spike probability of the oncoming action potential based on the time of the last spike, or the bursting behavior, which is characterized by short epochs of rapid action potentials, followed by longer episodes of silence. Here we investigate non-renewal processes with the inter-spike interval distribution model that incorporates spike-history dependence of individual neurons. For that, we use the Monte Carlo method to estimate the full shape of the coincidence count distribution and to generate false positives for coincidence detection. The results show that compared to the distributions based on homogeneous Poisson processes, and also non-Poisson processes, the width of the distribution of joint spike events changes. Non-renewal processes can lead to both heavy tailed or narrow coincidence distribution. We conclude that small differences in the exact autostructure of the point process can cause large differences in the width of a coincidence distribution. Therefore, manipulations of the autostructure for the estimation of significance of joint spike events seem to be inadequate.

Precise-spike-driven synaptic plasticity: learning hetero-association of spatiotemporal spike patterns.

Directory of Open Access Journals (Sweden)

Qiang Yu

Full Text Available A new learning rule (Precise-Spike-Driven (PSD Synaptic Plasticity is proposed for processing and memorizing spatiotemporal patterns. PSD is a supervised learning rule that is analytically derived from the traditional Widrow-Hoff rule and can be used to train neurons to associate an input spatiotemporal spike pattern with a desired spike train. Synaptic adaptation is driven by the error between the desired and the actual output spikes, with positive errors causing long-term potentiation and negative errors causing long-term depression. The amount of modification is proportional to an eligibility trace that is triggered by afferent spikes. The PSD rule is both computationally efficient and biologically plausible. The properties of this learning rule are investigated extensively through experimental simulations, including its learning performance, its generality to different neuron models, its robustness against noisy conditions, its memory capacity, and the effects of its learning parameters. Experimental results show that the PSD rule is capable of spatiotemporal pattern classification, and can even outperform a well studied benchmark algorithm with the proposed relative confidence criterion. The PSD rule is further validated on a practical example of an optical character recognition problem. The results again show that it can achieve a good recognition performance with a proper encoding. Finally, a detailed discussion is provided about the PSD rule and several related algorithms including tempotron, SPAN, Chronotron and ReSuMe.

Precise-spike-driven synaptic plasticity: learning hetero-association of spatiotemporal spike patterns.

Science.gov (United States)

Yu, Qiang; Tang, Huajin; Tan, Kay Chen; Li, Haizhou

2013-01-01

A new learning rule (Precise-Spike-Driven (PSD) Synaptic Plasticity) is proposed for processing and memorizing spatiotemporal patterns. PSD is a supervised learning rule that is analytically derived from the traditional Widrow-Hoff rule and can be used to train neurons to associate an input spatiotemporal spike pattern with a desired spike train. Synaptic adaptation is driven by the error between the desired and the actual output spikes, with positive errors causing long-term potentiation and negative errors causing long-term depression. The amount of modification is proportional to an eligibility trace that is triggered by afferent spikes. The PSD rule is both computationally efficient and biologically plausible. The properties of this learning rule are investigated extensively through experimental simulations, including its learning performance, its generality to different neuron models, its robustness against noisy conditions, its memory capacity, and the effects of its learning parameters. Experimental results show that the PSD rule is capable of spatiotemporal pattern classification, and can even outperform a well studied benchmark algorithm with the proposed relative confidence criterion. The PSD rule is further validated on a practical example of an optical character recognition problem. The results again show that it can achieve a good recognition performance with a proper encoding. Finally, a detailed discussion is provided about the PSD rule and several related algorithms including tempotron, SPAN, Chronotron and ReSuMe.

Fast-Spiking Interneurons Supply Feedforward Control of Bursting, Calcium, and Plasticity for Efficient Learning.

Science.gov (United States)

Owen, Scott F; Berke, Joshua D; Kreitzer, Anatol C

2018-02-08

Fast-spiking interneurons (FSIs) are a prominent class of forebrain GABAergic cells implicated in two seemingly independent network functions: gain control and network plasticity. Little is known, however, about how these roles interact. Here, we use a combination of cell-type-specific ablation, optogenetics, electrophysiology, imaging, and behavior to describe a unified mechanism by which striatal FSIs control burst firing, calcium influx, and synaptic plasticity in neighboring medium spiny projection neurons (MSNs). In vivo silencing of FSIs increased bursting, calcium transients, and AMPA/NMDA ratios in MSNs. In a motor sequence task, FSI silencing increased the frequency of calcium transients but reduced the specificity with which transients aligned to individual task events. Consistent with this, ablation of FSIs disrupted the acquisition of striatum-dependent egocentric learning strategies. Together, our data support a model in which feedforward inhibition from FSIs temporally restricts MSN bursting and calcium-dependent synaptic plasticity to facilitate striatum-dependent sequence learning. Copyright © 2018 Elsevier Inc. All rights reserved.

Reward Expectancy Strengthens CA1 Theta and Beta Band Synchronization and Hippocampal-Ventral Striatal Coupling.

Science.gov (United States)

Lansink, Carien S; Meijer, Guido T; Lankelma, Jan V; Vinck, Martin A; Jackson, Jadin C; Pennartz, Cyriel M A

2016-10-12

The use of information from the hippocampal memory system in motivated behavior depends on its communication with the ventral striatum. When an animal encounters cues that signal subsequent reward, its reward expectancy is raised. It is unknown, however, how this process affects hippocampal dynamics and their influence on target structures, such as ventral striatum. We show that, in rats, reward-predictive cues result in enhanced hippocampal theta and beta band rhythmic activity during subsequent action, compared with uncued goal-directed navigation. The beta band component, also labeled theta's harmonic, involves selective hippocampal CA1 cell groups showing frequency doubling of firing periodicity relative to theta rhythmicity and it partitions the theta cycle into segments showing clear versus poor spike timing organization. We found that theta phase precession occurred over a wider range than previously reported. This was apparent from spikes emitted near the peak of the theta cycle exhibiting large "phase precessing jumps" relative to spikes in foregoing cycles. Neither this phenomenon nor the regular manifestation of theta phase precession was affected by reward expectancy. Ventral striatal neuronal firing phase-locked not only to hippocampal theta, but also to beta band activity. Both hippocampus and ventral striatum showed increased synchronization between neuronal firing and local field potential activity during cued compared with uncued goal approaches. These results suggest that cue-triggered reward expectancy intensifies hippocampal output to target structures, such as the ventral striatum, by which the hippocampus may gain prioritized access to systems modulating motivated behaviors. Here we show that temporally discrete cues raising reward expectancy enhance both theta and beta band activity in the hippocampus once goal-directed navigation has been initiated. These rhythmic activities are associated with increased synchronization of neuronal firing

The race to learn: spike timing and STDP can coordinate learning and recall in CA3.

Science.gov (United States)

Nolan, Christopher R; Wyeth, Gordon; Milford, Michael; Wiles, Janet

2011-06-01

The CA3 region of the hippocampus has long been proposed as an autoassociative network performing pattern completion on known inputs. The dentate gyrus (DG) region is often proposed as a network performing the complementary function of pattern separation. Neural models of pattern completion and separation generally designate explicit learning phases to encode new information and assume an ideal fixed threshold at which to stop learning new patterns and begin recalling known patterns. Memory systems are significantly more complex in practice, with the degree of memory recall depending on context-specific goals. Here, we present our spike-timing separation and completion (STSC) model of the entorhinal cortex (EC), DG, and CA3 network, ascribing to each region a role similar to that in existing models but adding a temporal dimension by using a spiking neural network. Simulation results demonstrate that (a) spike-timing dependent plasticity in the EC-CA3 synapses provides a pattern completion ability without recurrent CA3 connections, (b) the race between activation of CA3 cells via EC-CA3 synapses and activation of the same cells via DG-CA3 synapses distinguishes novel from known inputs, and (c) modulation of the EC-CA3 synapses adjusts the learned versus test input similarity required to evoke a direct CA3 response prior to any DG activity, thereby adjusting the pattern completion threshold. These mechanisms suggest that spike timing can arbitrate between learning and recall based on the novelty of each individual input, ensuring control of the learn-recall decision resides in the same subsystem as the learned memories themselves. The proposed modulatory signal does not override this decision but biases the system toward either learning or recall. The model provides an explanation for empirical observations that a reduction in novelty produces a corresponding reduction in the latency of responses in CA3 and CA1. Copyright © 2010 Wiley-Liss, Inc.

Human synapses show a wide temporal window for spike-timing-dependent plasticity

NARCIS (Netherlands)

Testa-Silva, G.; Verhoog, M.B.; Goriounova, N.A.; Loebel, A.; Hjorth, J.; Baayen, J.C.; de Kock, C.P.J.; Mansvelder, H.D.

2010-01-01

Throughout our lifetime, activity-dependent changes in neuronal connection strength enable the brain to refine neural circuits and learn based on experience. Synapses can bi-directionally alter strength and the magnitude and sign depend on the millisecond timing of presynaptic and postsynaptic

A Reward-Maximizing Spiking Neuron as a Bounded Rational Decision Maker.

Science.gov (United States)

Leibfried, Felix; Braun, Daniel A

2015-08-01

Rate distortion theory describes how to communicate relevant information most efficiently over a channel with limited capacity. One of the many applications of rate distortion theory is bounded rational decision making, where decision makers are modeled as information channels that transform sensory input into motor output under the constraint that their channel capacity is limited. Such a bounded rational decision maker can be thought to optimize an objective function that trades off the decision maker's utility or cumulative reward against the information processing cost measured by the mutual information between sensory input and motor output. In this study, we interpret a spiking neuron as a bounded rational decision maker that aims to maximize its expected reward under the computational constraint that the mutual information between the neuron's input and output is upper bounded. This abstract computational constraint translates into a penalization of the deviation between the neuron's instantaneous and average firing behavior. We derive a synaptic weight update rule for such a rate distortion optimizing neuron and show in simulations that the neuron efficiently extracts reward-relevant information from the input by trading off its synaptic strengths against the collected reward.

Examining the time dependence of DAMA's modulation amplitude

Science.gov (United States)

Kelso, Chris; Savage, Christopher; Sandick, Pearl; Freese, Katherine; Gondolo, Paolo

2018-03-01

If dark matter is composed of weakly interacting particles, Earth's orbital motion may induce a small annual variation in the rate at which these particles interact in a terrestrial detector. The DAMA collaboration has identified at a 9.3σ confidence level such an annual modulation in their event rate over two detector iterations, DAMA/NaI and DAMA/LIBRA, each with ˜ 7 years of observations. This data is well fit by a constant modulation amplitude for the two iterations of the experiment. We statistically examine the time dependence of the modulation amplitudes, which "by eye" appear to be decreasing with time in certain energy ranges. We perform a chi-squared goodness of fit test of the average modulation amplitudes measured by the two detector iterations which rejects the hypothesis of a consistent modulation amplitude at greater than 80, 96, and 99.6% for the 2-4, 2-5 and 2-6 keVee energy ranges, respectively. We also find that among the 14 annual cycles there are three ≳ 3σ departures from the average in our estimated data in the 5-6 keVee energy range. In addition, we examined several phenomenological models for the time dependence of the modulation amplitude. Using a maximum likelihood test, we find that descriptions of the modulation amplitude as decreasing with time are preferred over a constant modulation amplitude at anywhere between 1σ and 3σ , depending on the phenomenological model for the time dependence and the signal energy range considered. A time dependent modulation amplitude is not expected for a dark matter signal, at least for dark matter halo morphologies consistent with the DAMA signal. New data from DAMA/LIBRA-phase2 will certainly aid in determining whether any apparent time dependence is a real effect or a statistical fluctuation.

Learning with three factors: modulating Hebbian plasticity with errors.

Science.gov (United States)

Kuśmierz, Łukasz; Isomura, Takuya; Toyoizumi, Taro

2017-10-01

Synaptic plasticity is a central theme in neuroscience. A framework of three-factor learning rules provides a powerful abstraction, helping to navigate through the abundance of models of synaptic plasticity. It is well-known that the dopamine modulation of learning is related to reward, but theoretical models predict other functional roles of the modulatory third factor; it may encode errors for supervised learning, summary statistics of the population activity for unsupervised learning or attentional feedback. Specialized structures may be needed in order to generate and propagate third factors in the neural network. Copyright © 2017 The Authors. Published by Elsevier Ltd.. All rights reserved.

Spike Pattern Structure Influences Synaptic Efficacy Variability Under STDP and Synaptic Homeostasis. II: Spike Shuffling Methods on LIF Networks

Directory of Open Access Journals (Sweden)

Zedong Bi

2016-08-01

Full Text Available Synapses may undergo variable changes during plasticity because of the variability of spike patterns such as temporal stochasticity and spatial randomness. Here, we call the variability of synaptic weight changes during plasticity to be efficacy variability. In this paper, we investigate how four aspects of spike pattern statistics (i.e., synchronous firing, burstiness/regularity, heterogeneity of rates and heterogeneity of cross-correlations influence the efficacy variability under pair-wise additive spike-timing dependent plasticity (STDP and synaptic homeostasis (the mean strength of plastic synapses into a neuron is bounded, by implementing spike shuffling methods onto spike patterns self-organized by a network of excitatory and inhibitory leaky integrate-and-fire (LIF neurons. With the increase of the decay time scale of the inhibitory synaptic currents, the LIF network undergoes a transition from asynchronous state to weak synchronous state and then to synchronous bursting state. We first shuffle these spike patterns using a variety of methods, each designed to evidently change a specific pattern statistics; and then investigate the change of efficacy variability of the synapses under STDP and synaptic homeostasis, when the neurons in the network fire according to the spike patterns before and after being treated by a shuffling method. In this way, we can understand how the change of pattern statistics may cause the change of efficacy variability. Our results are consistent with those of our previous study which implements spike-generating models on converging motifs. We also find that burstiness/regularity is important to determine the efficacy variability under asynchronous states, while heterogeneity of cross-correlations is the main factor to cause efficacy variability when the network moves into synchronous bursting states (the states observed in epilepsy.

Optical implementation of neural learning algorithms based on cross-gain modulation in a semiconductor optical amplifier

Science.gov (United States)

Li, Qiang; Wang, Zhi; Le, Yansi; Sun, Chonghui; Song, Xiaojia; Wu, Chongqing

2016-10-01

Neuromorphic engineering has a wide range of applications in the fields of machine learning, pattern recognition, adaptive control, etc. Photonics, characterized by its high speed, wide bandwidth, low power consumption and massive parallelism, is an ideal way to realize ultrafast spiking neural networks (SNNs). Synaptic plasticity is believed to be critical for learning, memory and development in neural circuits. Experimental results have shown that changes of synapse are highly dependent on the relative timing of pre- and postsynaptic spikes. Synaptic plasticity in which presynaptic spikes preceding postsynaptic spikes results in strengthening, while the opposite timing results in weakening is called antisymmetric spike-timing-dependent plasticity (STDP) learning rule. And synaptic plasticity has the opposite effect under the same conditions is called antisymmetric anti-STDP learning rule. We proposed and experimentally demonstrated an optical implementation of neural learning algorithms, which can achieve both of antisymmetric STDP and anti-STDP learning rule, based on the cross-gain modulation (XGM) within a single semiconductor optical amplifier (SOA). The weight and height of the potentitation and depression window can be controlled by adjusting the injection current of the SOA, to mimic the biological antisymmetric STDP and anti-STDP learning rule more realistically. As the injection current increases, the width of depression and potentitation window decreases and height increases, due to the decreasing of recovery time and increasing of gain under a stronger injection current. Based on the demonstrated optical STDP circuit, ultrafast learning in optical SNNs can be realized.

Adenosine A2A Receptors Control Glutamatergic Synaptic Plasticity in Fast Spiking Interneurons of the Prefrontal Cortex

Directory of Open Access Journals (Sweden)

Amber Kerkhofs

2018-03-01

Full Text Available Adenosine A2A receptors (A2AR are activated upon increased synaptic activity to assist in the implementation of long-term plastic changes at synapses. While it is reported that A2AR are involved in the control of prefrontal cortex (PFC-dependent behavior such as working memory, reversal learning and effort-based decision making, it is not known whether A2AR control glutamatergic synapse plasticity within the medial PFC (mPFC. To elucidate that, we tested whether A2AR blockade affects long-term plasticity (LTP of excitatory post-synaptic potentials in pyramidal neurons and fast spiking (FS interneurons in layer 5 of the mPFC and of population spikes. Our results show that A2AR are enriched at mPFC synapses, where their blockade reversed the direction of plasticity at excitatory synapses onto layer 5 FS interneurons from LTP to long-term depression, while their blockade had no effect on the induction of LTP at excitatory synapses onto layer 5 pyramidal neurons. At the network level, extracellularly induced LTP of population spikes was reduced by A2AR blockade. The interneuron-specificity of A2AR in controlling glutamatergic synapse LTP may ensure that during periods of high synaptic activity, a proper excitation/inhibition balance is maintained within the mPFC.

Evolving spiking networks with variable resistive memories.

Science.gov (United States)

Howard, Gerard; Bull, Larry; de Lacy Costello, Ben; Gale, Ella; Adamatzky, Andrew

2014-01-01

Neuromorphic computing is a brainlike information processing paradigm that requires adaptive learning mechanisms. A spiking neuro-evolutionary system is used for this purpose; plastic resistive memories are implemented as synapses in spiking neural networks. The evolutionary design process exploits parameter self-adaptation and allows the topology and synaptic weights to be evolved for each network in an autonomous manner. Variable resistive memories are the focus of this research; each synapse has its own conductance profile which modifies the plastic behaviour of the device and may be altered during evolution. These variable resistive networks are evaluated on a noisy robotic dynamic-reward scenario against two static resistive memories and a system containing standard connections only. The results indicate that the extra behavioural degrees of freedom available to the networks incorporating variable resistive memories enable them to outperform the comparative synapse types.

A note on the optimal pricing strategy in the discrete-time Geo/Geo/1 queuing system with sojourn time-dependent reward

Directory of Open Access Journals (Sweden)

Doo Ho Lee

Full Text Available This work studies the optimal pricing strategy in a discrete-time Geo/Geo/1 queuing system under the sojourn time-dependent reward. We consider two types of pricing schemes. The first one is called the ex-post payment scheme where the server charges a price that is proportional to the time a customer spends in the system, and the second one is called ex-ante payment scheme where the server charges a flat price for all services. In each pricing scheme, a departing customer receives the reward that is inversely proportional to his/her sojourn time. The server should make the optimal pricing decisions in order to maximize its expected profits per time unit in each pricing scheme. This work also investigates customer's equilibrium joining or balking behavior under server's optimal pricing strategy. Numerical experiments are also conducted to validate our analysis. Keywords: Optimal pricing, Equilibrium behavior, Geo/Geo/1 queue, Sojourn time-dependent reward

Timed Synaptic Inhibition Shapes NMDA Spikes, Influencing Local Dendritic Processing and Global I/O Properties of Cortical Neurons

Directory of Open Access Journals (Sweden)

Michael Doron

2017-11-01

Full Text Available The NMDA spike is a long-lasting nonlinear phenomenon initiated locally in the dendritic branches of a variety of cortical neurons. It plays a key role in synaptic plasticity and in single-neuron computations. Combining dynamic system theory and computational approaches, we now explore how the timing of synaptic inhibition affects the NMDA spike and its associated membrane current. When impinging on its early phase, individual inhibitory synapses strongly, but transiently, dampen the NMDA spike; later inhibition prematurely terminates it. A single inhibitory synapse reduces the NMDA-mediated Ca2+ current, a key player in plasticity, by up to 45%. NMDA spikes in distal dendritic branches/spines are longer-lasting and more resilient to inhibition, enhancing synaptic plasticity at these branches. We conclude that NMDA spikes are highly sensitive to dendritic inhibition; sparse weak inhibition can finely tune synaptic plasticity both locally at the dendritic branch level and globally at the level of the neuron’s output.

Striatal fast-spiking interneurons selectively modulate circuit output and are required for habitual behavior.

Science.gov (United States)

O'Hare, Justin K; Li, Haofang; Kim, Namsoo; Gaidis, Erin; Ade, Kristen; Beck, Jeff; Yin, Henry; Calakos, Nicole

2017-09-05

Habit formation is a behavioral adaptation that automates routine actions. Habitual behavior correlates with broad reconfigurations of dorsolateral striatal (DLS) circuit properties that increase gain and shift pathway timing. The mechanism(s) for these circuit adaptations are unknown and could be responsible for habitual behavior. Here we find that a single class of interneuron, fast-spiking interneurons (FSIs), modulates all of these habit-predictive properties. Consistent with a role in habits, FSIs are more excitable in habitual mice compared to goal-directed and acute chemogenetic inhibition of FSIs in DLS prevents the expression of habitual lever pressing. In vivo recordings further reveal a previously unappreciated selective modulation of SPNs based on their firing patterns; FSIs inhibit most SPNs but paradoxically promote the activity of a subset displaying high fractions of gamma-frequency spiking. These results establish a microcircuit mechanism for habits and provide a new example of how interneurons mediate experience-dependent behavior.

Associative stimulation of the supraorbital nerve fails to induce timing-specific plasticity in the human blink reflex

DEFF Research Database (Denmark)

Zeuner, Kirsten E; Knutzen, Arne; Al-Ali, Asmaa

2010-01-01

Associative high-frequency electrical stimulation (HFS) of the supraorbital nerve in five healthy individuals induced long-term potentiation (LTP)-like or depression (LTD)-like changes in the human blink reflex circuit according to the rules of spike timing-dependent plasticity (Mao and Evinger...

GABAA receptor drugs and neuronal plasticity in reward and aversion: focus on the ventral tegmental area

Directory of Open Access Journals (Sweden)

Elena eVashchinkina

2014-11-01

Full Text Available GABAA receptors are the main fast inhibitory neurotransmitter receptors in the mammalian brain, and targets for many clinically important drugs widely used in the treatment of anxiety disorders, insomnia and in anesthesia. Nonetheless, there are significant risks associated with the long-term use of these drugs particularly related to development of tolerance and addiction. Addictive mechanisms of GABAA receptor drugs are poorly known, but recent findings suggest that those drugs may induce aberrant neuroadaptations in the brain reward circuitry. Recently, benzodiazepines, acting on synaptic GABAA receptors, and modulators of extrasynaptic GABAA receptors (THIP and neurosteroids have been found to induce plasticity in the ventral tegmental area (VTA dopamine neurons and their main target projections. Furthermore, depending whether synaptic or extrasynaptic GABAA receptor populations are activated, the behavioral outcome of repeated administration seems to correlate with rewarding or aversive behavioral responses, respectively. The VTA dopamine neurons project to forebrain centers such as the nucleus accumbens and medial prefrontal cortex, and receive afferent projections from these brain regions and especially from the extended amygdala and lateral habenula, forming the major part of the reward and aversion circuitry. Both synaptic and extrasynaptic GABAA drugs inhibit the VTA GABAergic interneurons, thus activating the VTA DA neurons by disinhibition and this way inducing glutamatergic synaptic plasticity. However, the GABAA drugs failed to alter synaptic spine numbers as studied from Golgi-Cox-stained VTA dendrites. Since the GABAergic drugs are known to depress the brain metabolism and gene expression, their likely way of inducing neuroplasticity in mature neurons is by disinhibiting the principal neurons, which remains to be rigorously tested for a number of clinically important anxiolytics, sedatives and anesthetics in different parts of

A supervised learning rule for classification of spatiotemporal spike patterns.

Science.gov (United States)

Lilin Guo; Zhenzhong Wang; Adjouadi, Malek

2016-08-01

This study introduces a novel supervised algorithm for spiking neurons that take into consideration synapse delays and axonal delays associated with weights. It can be utilized for both classification and association and uses several biologically influenced properties, such as axonal and synaptic delays. This algorithm also takes into consideration spike-timing-dependent plasticity as in Remote Supervised Method (ReSuMe). This paper focuses on the classification aspect alone. Spiked neurons trained according to this proposed learning rule are capable of classifying different categories by the associated sequences of precisely timed spikes. Simulation results have shown that the proposed learning method greatly improves classification accuracy when compared to the Spike Pattern Association Neuron (SPAN) and the Tempotron learning rule.

Modulation of the spike activity of neocortex neurons during a conditioned reflex.

Science.gov (United States)

Storozhuk, V M; Sanzharovskii, A V; Sachenko, V V; Busel, B I

2000-01-01

Experiments were conducted on cats to study the effects of iontophoretic application of glutamate and a number of modulators on the spike activity of neurons in the sensorimotor cortex during a conditioned reflex. These studies showed that glutamate, as well as exerting a direct influence on neuron spike activity, also had a delayed facilitatory action lasting 10-20 min after iontophoresis was finished. Adrenomimetics were found to have a double modulatory effect on intracortical glutamate connections: inhibitory and facilitatory effects were mediated by beta1 and beta2 adrenoceptors respectively. Although dopamine, like glutamate, facilitated neuron spike activity during the period of application, the simultaneous facilitatory actions of glutamate and L-DOPA were accompanied by occlusion of spike activity, and simultaneous application of glutamate and haloperidol suppressed spike activity associated with the conditioned reflex response. Facilitation thus appears to show a significant level of dependence on metabotropic glutamate receptors which, like dopamine receptors, are linked to the intracellular medium via Gi proteins.

Neural processing of calories in brain reward areas can be modulated by reward sensitivity

Directory of Open Access Journals (Sweden)

Inge eVan Rijn

2016-01-01

Full Text Available A food’s reward value is dependent on its caloric content. Furthermore, a food’s acute reward value also depends on hunger state. The drive to obtain rewards (reward sensitivity, however, differs between individuals. Here, we assessed the association between brain responses to calories in the mouth and trait reward sensitivity in different hunger states. Firstly, we assessed this in data from a functional neuroimaging study (van Rijn et al., 2015, in which participants (n=30 tasted simple solutions of a non-caloric sweetener with or without a non-sweet carbohydrate (maltodextrin during hunger and satiety. Secondly, we expanded these analyses to regular drinks by assessing the same relationship in data from a study in which soft drinks sweetened with either sucrose or a non-caloric sweetener were administered during hunger (n=18 (Griffioen-Roose et al., 2013. First, taste activation by the non-caloric solution/soft drink was subtracted from that by the caloric solution/soft drink to eliminate sweetness effects and retain activation induced by calories. Subsequently, this difference in taste activation was correlated with reward sensitivity as measured with the BAS drive subscale of the Behavioral Activation System (BAS questionnaire.When participants were hungry and tasted calories from the simple solution, brain activation in the right ventral striatum (caudate, right amygdala and anterior cingulate cortex (bilaterally correlated negatively with BAS drive scores. In contrast, when participants were satiated, taste responses correlated positively with BAS drive scores in the left caudate. These results were not replicated for soft drinks. Thus, neural responses to oral calories from maltodextrin were modulated by reward sensitivity in reward-related brain areas. This was not the case for sucrose. This may be due to the direct detection of maltodextrin, but not sucrose in the oral cavity. Also, in a familiar beverage, detection of calories per

Neural Processing of Calories in Brain Reward Areas Can be Modulated by Reward Sensitivity.

Science.gov (United States)

van Rijn, Inge; Griffioen-Roose, Sanne; de Graaf, Cees; Smeets, Paul A M

2015-01-01

A food's reward value is dependent on its caloric content. Furthermore, a food's acute reward value also depends on hunger state. The drive to obtain rewards (reward sensitivity), however, differs between individuals. Here, we assessed the association between brain responses to calories in the mouth and trait reward sensitivity in different hunger states. Firstly, we assessed this in data from a functional neuroimaging study (van Rijn et al., 2015), in which participants (n = 30) tasted simple solutions of a non-caloric sweetener with or without a non-sweet carbohydrate (maltodextrin) during hunger and satiety. Secondly, we expanded these analyses to regular drinks by assessing the same relationship in data from a study in which soft drinks sweetened with either sucrose or a non-caloric sweetener were administered during hunger (n = 18) (Griffioen-Roose et al., 2013). First, taste activation by the non-caloric solution/soft drink was subtracted from that by the caloric solution/soft drink to eliminate sweetness effects and retain activation induced by calories. Subsequently, this difference in taste activation was correlated with reward sensitivity as measured with the BAS drive subscale of the Behavioral Activation System (BAS) questionnaire. When participants were hungry and tasted calories from the simple solution, brain activation in the right ventral striatum (caudate), right amygdala and anterior cingulate cortex (bilaterally) correlated negatively with BAS drive scores. In contrast, when participants were satiated, taste responses correlated positively with BAS drive scores in the left caudate. These results were not replicated for soft drinks. Thus, neural responses to oral calories from maltodextrin were modulated by reward sensitivity in reward-related brain areas. This was not the case for sucrose. This may be due to the direct detection of maltodextrin, but not sucrose in the oral cavity. Also, in a familiar beverage, detection of calories per se may be

Monetary rewards modulate inhibitory control

Directory of Open Access Journals (Sweden)

Paula Marcela Herrera

2014-05-01

Full Text Available The ability to override a dominant response, often referred to as behavioural inhibiton, is considered a key element of executive cognition. Poor behavioural inhibition is a defining characteristic of several neurological and psychiatric populations. Recently, there has been increasing interest in the motivational dimension of behavioural inhibition, with some experiments incorporating emotional contingencies in classical inhibitory paradigms such as the Go/Nogo and Stop Signal Tasks. Several studies have reported a positive modulatory effect of reward on the performance of such tasks in pathological conditions such as substance abuse, pathological gambling, and ADHD. However, experiments that directly investigate the modulatory effects of reward magnitudes on the performance of inhibitory paradigms are rare and consequently, little is known about the finer grained relationship between motivation and self-control. Here, we probed the effect of reward and reward magnitude on behavioural inhibition using two modified version of the widely used Stop Signal Task. The first task compared no reward with reward, whilst the other compared two different reward magnitudes. The reward magnitude effect was confirmed by the second study, whereas it was less compelling in the first study, possibly due to the effect of having no reward in some conditions. In addition, our results showed a kick start effect over global performance measures. More specifically, there was a long lasting improvement in performance throughout the task, when participants received the highest reward magnitudes at the beginning of the protocol. These results demonstrate that individuals’ behavioural inhibition capacities are dynamic not static because they are modulated by the reward magnitude and initial reward history of the task at hand.

Different propagation speeds of recalled sequences in plastic spiking neural networks

Science.gov (United States)

Huang, Xuhui; Zheng, Zhigang; Hu, Gang; Wu, Si; Rasch, Malte J.

2015-03-01

Neural networks can generate spatiotemporal patterns of spike activity. Sequential activity learning and retrieval have been observed in many brain areas, and e.g. is crucial for coding of episodic memory in the hippocampus or generating temporal patterns during song production in birds. In a recent study, a sequential activity pattern was directly entrained onto the neural activity of the primary visual cortex (V1) of rats and subsequently successfully recalled by a local and transient trigger. It was observed that the speed of activity propagation in coordinates of the retinotopically organized neural tissue was constant during retrieval regardless how the speed of light stimulation sweeping across the visual field during training was varied. It is well known that spike-timing dependent plasticity (STDP) is a potential mechanism for embedding temporal sequences into neural network activity. How training and retrieval speeds relate to each other and how network and learning parameters influence retrieval speeds, however, is not well described. We here theoretically analyze sequential activity learning and retrieval in a recurrent neural network with realistic synaptic short-term dynamics and STDP. Testing multiple STDP rules, we confirm that sequence learning can be achieved by STDP. However, we found that a multiplicative nearest-neighbor (NN) weight update rule generated weight distributions and recall activities that best matched the experiments in V1. Using network simulations and mean-field analysis, we further investigated the learning mechanisms and the influence of network parameters on recall speeds. Our analysis suggests that a multiplicative STDP rule with dominant NN spike interaction might be implemented in V1 since recall speed was almost constant in an NMDA-dominant regime. Interestingly, in an AMPA-dominant regime, neural circuits might exhibit recall speeds that instead follow the change in stimulus speeds. This prediction could be tested in

Spike Pattern Structure Influences Synaptic Efficacy Variability Under STDP and Synaptic Homeostasis. I: Spike Generating Models on Converging Motifs

Directory of Open Access Journals (Sweden)

Zedong eBi

2016-02-01

Full Text Available In neural systems, synaptic plasticity is usually driven by spike trains. Due to the inherent noises of neurons and synapses as well as the randomness of connection details, spike trains typically exhibit variability such as spatial randomness and temporal stochasticity, resulting in variability of synaptic changes under plasticity, which we call efficacy variability. How the variability of spike trains influences the efficacy variability of synapses remains unclear. In this paper, we try to understand this influence under pair-wise additive spike-timing dependent plasticity (STDP when the mean strength of plastic synapses into a neuron is bounded (synaptic homeostasis. Specifically, we systematically study, analytically and numerically, how four aspects of statistical features, i.e. synchronous firing, burstiness/regularity, heterogeneity of rates and heterogeneity of cross-correlations, as well as their interactions influence the efficacy variability in converging motifs (simple networks in which one neuron receives from many other neurons. Neurons (including the post-synaptic neuron in a converging motif generate spikes according to statistical models with tunable parameters. In this way, we can explicitly control the statistics of the spike patterns, and investigate their influence onto the efficacy variability, without worrying about the feedback from synaptic changes onto the dynamics of the post-synaptic neuron. We separate efficacy variability into two parts: the drift part (DriftV induced by the heterogeneity of change rates of different synapses, and the diffusion part (DiffV induced by weight diffusion caused by stochasticity of spike trains. Our main findings are: (1 synchronous firing and burstiness tend to increase DiffV, (2 heterogeneity of rates induces DriftV when potentiation and depression in STDP are not balanced, and (3 heterogeneity of cross-correlations induces DriftV together with heterogeneity of rates. We anticipate our

Role of the origin of glutamatergic synaptic inputs in controlling synaptic plasticity and its modulation by alcohol in mice nucleus accumbens

Directory of Open Access Journals (Sweden)

Gilles Erwann Martin

2015-07-01

Full Text Available It is widely accepted that long-lasting changes of synaptic strength in the nucleus accumbens, a brain region involved in drug reward, mediate acute and chronic effects of alcohol. However, our understanding of the mechanisms underlying the effects of alcohol on synaptic plasticity is limited by the fact that the nucleus accumbens receives glutamatergic inputs from distinct brain regions (e.g. the prefrontal cortex, the amygdala and the hippocampus, each region providing different information (e.g. spatial, emotional and cognitive. Combining whole-cell patch-clamp recordings and the optogenetic technique, we examined synaptic plasticity, and its regulation by alcohol, at cortical, hippocampal and amygdala inputs in fresh slices of mouse tissue. We showed that the origin of synaptic inputs determines the basic properties of glutamatergic synaptic transmission, the expression of spike-timing dependent long-term depression (tLTD and long-term potentiation (tLTP and their regulation by alcohol. While we observed both tLTP and tLTD at amygadala and hippocampal synapses, we showed that cortical inputs only undergo tLTD. Functionally, we provide evidence that acute EtOH has little effects on higher order information coming from the prefrontal cortex (PFCx, while severely impacting the ability of emotional and contextual information to induce long-lasting changes of synaptic strength.

NMDA receptor blockade in the prelimbic cortex activates the mesolimbic system and dopamine-dependent opiate reward signaling.

Science.gov (United States)

Tan, Huibing; Rosen, Laura G; Ng, Garye A; Rushlow, Walter J; Laviolette, Steven R

2014-12-01

N-Methyl-D-aspartate (NMDA) receptors in the medial prefrontal cortex (mPFC) are involved in opiate reward processing and modulate sub-cortical dopamine (DA) activity. NMDA receptor blockade in the prelimbic (PLC) division of the mPFC strongly potentiates the rewarding behavioural properties of normally sub-reward threshold doses of opiates. However, the possible functional interactions between cortical NMDA and sub-cortical DAergic motivational neural pathways underlying these effects are not understood. This study examines how NMDA receptor modulation in the PLC influences opiate reward processing via interactions with sub-cortical DAergic transmission. We further examined whether direct intra-PLC NMDA receptor modulation may activate DA-dependent opiate reward signaling via interactions with the ventral tegmental area (VTA). Using an unbiased place conditioning procedure (CPP) in rats, we performed bilateral intra-PLC microinfusions of the competitive NMDA receptor antagonist, (2R)-amino-5-phosphonovaleric acid (AP-5), prior to behavioural morphine place conditioning and challenged the rewarding effects of morphine with DA receptor blockade. We next examined the effects of intra-PLC NMDA receptor blockade on the spontaneous activity patterns of presumptive VTA DA or GABAergic neurons, using single-unit, extracellular in vivo neuronal recordings. We show that intra-PLC NMDA receptor blockade strongly activates sub-cortical DA neurons within the VTA while inhibiting presumptive non-DA GABAergic neurons. Behaviourally, NMDA receptor blockade activates a DA-dependent opiate reward system, as pharmacological blockade of DA transmission blocked morphine reward only in the presence of intra-PLC NMDA receptor antagonism. These findings demonstrate a cortical NMDA-mediated mechanism controlling mesolimbic DAergic modulation of opiate reward processing.

Neural Processing of Calories in Brain Reward Areas Can be Modulated by Reward Sensitivity

NARCIS (Netherlands)

van Rijn, Inge; Griffioen-Roose, Sanne; de Graaf, Cees; Smeets, Paul A M

A food's reward value is dependent on its caloric content. Furthermore, a food's acute reward value also depends on hunger state. The drive to obtain rewards (reward sensitivity), however, differs between individuals. Here, we assessed the association between brain responses to calories in the mouth

Bimodal stimulus timing-dependent plasticity in primary auditory cortex is altered after noise exposure with and without tinnitus.

Science.gov (United States)

Basura, Gregory J; Koehler, Seth D; Shore, Susan E

2015-12-01

Central auditory circuits are influenced by the somatosensory system, a relationship that may underlie tinnitus generation. In the guinea pig dorsal cochlear nucleus (DCN), pairing spinal trigeminal nucleus (Sp5) stimulation with tones at specific intervals and orders facilitated or suppressed subsequent tone-evoked neural responses, reflecting spike timing-dependent plasticity (STDP). Furthermore, after noise-induced tinnitus, bimodal responses in DCN were shifted from Hebbian to anti-Hebbian timing rules with less discrete temporal windows, suggesting a role for bimodal plasticity in tinnitus. Here, we aimed to determine if multisensory STDP principles like those in DCN also exist in primary auditory cortex (A1), and whether they change following noise-induced tinnitus. Tone-evoked and spontaneous neural responses were recorded before and 15 min after bimodal stimulation in which the intervals and orders of auditory-somatosensory stimuli were randomized. Tone-evoked and spontaneous firing rates were influenced by the interval and order of the bimodal stimuli, and in sham-controls Hebbian-like timing rules predominated as was seen in DCN. In noise-exposed animals with and without tinnitus, timing rules shifted away from those found in sham-controls to more anti-Hebbian rules. Only those animals with evidence of tinnitus showed increased spontaneous firing rates, a purported neurophysiological correlate of tinnitus in A1. Together, these findings suggest that bimodal plasticity is also evident in A1 following noise damage and may have implications for tinnitus generation and therapeutic intervention across the central auditory circuit. Copyright © 2015 the American Physiological Society.

A model for rate-dependent but time-independent material behavior in cyclic plasticity

International Nuclear Information System (INIS)

Dafalias, Y.F.; Ramey, M.R.; Sheikh, I.

1977-01-01

It is the purpose of this paper to present a model for rate-dependent but time independent material behavior under cyclic loading in the plastic range. What is referred to as time independent behavior here, is the absence of creep and relaxation phenomena from the behavior of the model. The notion of plastic internal variables (piv) is introduced, as properly invariant scalars or second order tensors, whose constitutive relations are rate-type equations not necessarily homogeneous of oder one in the rates, as it would be required for independent plasticity. The concept of a yield surface in the strain space and a loading function in terms of the total strain rate is introduced, where the sign of the loading function defines zero or non-zero value of the rate of piv. Thus rate dependence is achieved without time dependent behavior (no creep or relaxation). In addition, discrete memory parameters associated with the most recent event of unloading-reloading in different directions enter the constitutive relations for the piv. A particular form of the constitutive relations is assumed, where the rate of piv is a linear combination of the strain rate components, with coefficients depending on the second invariant of the strain rate tensor, which can be viewed as a scalar measure of the rate of deformation in the multiaxial case and a direct generalization of the uniaxial strain rate. This leads to a particularly simple form of the constitutive relations resembling the ones for rate independent plasticity. The uniaxial counterpart would be a relation between the plastic strain rate (as one of the piv) and the total strain rate through a plastic modulus which depends on the strain rate, the piv, and the discrete memory parameters

Learning Touch Preferences with a Tactile Robot Using Dopamine Modulated STDP in a Model of Insular Cortex

Directory of Open Access Journals (Sweden)

Ting-Shuo eChou

2015-07-01

Full Text Available Neurorobots enable researchers to study how behaviors are produced by neural mechanisms in an uncertain, noisy, real-world environment. To investigate how the somatosensory system processes noisy, real-world touch inputs, we introduce a neurorobot called CARL-SJR, which has a full-body tactile sensory area. The design of CARL-SJR is such that it encourages people to communicate with it through gentle touch. CARL-SJR provides feedback to users by displaying bright colors on its surface. In the present study, we show that CARL-SJR is capable of learning associations between conditioned stimuli (CS; a color pattern on its surface and unconditioned stimuli (US; a preferred touch pattern by applying a spiking neural network (SNN with neurobiologically inspired plasticity. Specifically, we modeled the primary somatosensory cortex, prefrontal cortex, striatum, and the insular cortex, which is important for hedonic touch, to process noisy data generated directly from CARL-SJR’s tactile sensory area. To facilitate learning, we applied dopamine-modulated Spike Timing Dependent Plasticity (STDP to our simulated prefrontal cortex, striatum and insular cortex. To cope with noisy, varying inputs, the SNN was tuned to produce traveling waves of activity that carried spatiotemporal information. Despite the noisy tactile sensors, spike trains, and variations in subject hand swipes, the learning was quite robust. Further, the plasticity (i.e., STDP in primary somatosensory cortex and insular cortex in the incremental pathway of dopaminergic reward system allowed us to control CARL-SJR’s preference for touch direction without heavily pre-processed inputs. The emerged behaviors we found in this model match animal’s behaviors wherein they prefer touch in particular areas and directions. Thus, the results in this paper could serve as an explanation on the underlying neural mechanisms for developing tactile preferences and hedonic touch.

A model for rate-dependent but time-independent material behavior in cyclic plasticity

International Nuclear Information System (INIS)

Dafalias, Y.F.; Ramey, M.R.; Sheikh, I.

1977-01-01

This paper presents a model for rate-dependent but time independent material behavior under cyclic loading in the plastic range. What is referred to as time independent behavior here, is the absence of creep and relaxation phenomena from the behavior of the model. The notion of plastic internal variables (piv) is introduced, as properly invariant scalars or second order tensors, whose constitutive relations are rate-type equations not necessarily homogeneous of order one in the rates, as it would be required for independent plasticity. The concept of a yield surface in the strain space and a loading function in terms of the total strain rate is introduced, where the sign of the loading function defines zero or non-zero value of the rate of piv. Thus rate dependence is achieved without time dependent behaviour (no creep or relaxation). In addition, discrete memory parameters associated with the most recent event of unloading-reloading in different directions enter the constitutive relations for the piv. (Auth.)

Spike-timing-based computation in sound localization.

Directory of Open Access Journals (Sweden)

Dan F M Goodman

2010-11-01

Full Text Available Spike timing is precise in the auditory system and it has been argued that it conveys information about auditory stimuli, in particular about the location of a sound source. However, beyond simple time differences, the way in which neurons might extract this information is unclear and the potential computational advantages are unknown. The computational difficulty of this task for an animal is to locate the source of an unexpected sound from two monaural signals that are highly dependent on the unknown source signal. In neuron models consisting of spectro-temporal filtering and spiking nonlinearity, we found that the binaural structure induced by spatialized sounds is mapped to synchrony patterns that depend on source location rather than on source signal. Location-specific synchrony patterns would then result in the activation of location-specific assemblies of postsynaptic neurons. We designed a spiking neuron model which exploited this principle to locate a variety of sound sources in a virtual acoustic environment using measured human head-related transfer functions. The model was able to accurately estimate the location of previously unknown sounds in both azimuth and elevation (including front/back discrimination in a known acoustic environment. We found that multiple representations of different acoustic environments could coexist as sets of overlapping neural assemblies which could be associated with spatial locations by Hebbian learning. The model demonstrates the computational relevance of relative spike timing to extract spatial information about sources independently of the source signal.

Abstinence duration modulates striatal functioning during monetary reward processing in cocaine patients.

Science.gov (United States)

Bustamante, Juan-Carlos; Barrós-Loscertales, Alfonso; Costumero, Víctor; Fuentes-Claramonte, Paola; Rosell-Negre, Patricia; Ventura-Campos, Noelia; Llopis, Juan-José; Ávila, César

2014-09-01

Pre-clinical and clinical studies in cocaine addiction highlight alterations in the striatal dopaminergic reward system that subserve maintenance of cocaine use. Using an instrumental conditioning paradigm with monetary reinforcement, we studied striatal functional alterations in long-term abstinent cocaine-dependent patients and striatal functioning as a function of abstinence and treatment duration. Eighteen patients and 20 controls underwent functional magnetic resonance imaging during a Monetary Incentive Delay task. Region of interest analyses based on masks of the dorsal and ventral striatum were conducted to test between-group differences and the functional effects in the cocaine group of time (in months) with no more than two lapses from the first time patients visited the clinical service to seek treatment at the scanning time (duration of treatment), and the functional effects of the number of months with no lapses or relapses at the scanning session time (length of abstinence). We applied a voxel-wise and a cluster-wise FWE-corrected level (pFWE) at a threshold of P reward anticipation than the control group. The regression analyses in the patients group revealed a positive correlation between duration of treatment and brain activity in the left caudate during reward anticipation. Likewise, length of abstinence negatively correlated with brain activity in the bilateral nucleus accumbens during monetary outcome processing. In conclusion, caudate and nucleus accumbens show a different brain response pattern to non-drug rewards during cocaine addiction, which can be modulated by treatment success. © 2013 The Authors, Addiction Biology © 2013 Society for the Study of Addiction.

Motivational orientation modulates the neural response to reward.

Science.gov (United States)

Linke, Julia; Kirsch, Peter; King, Andrea V; Gass, Achim; Hennerici, Michael G; Bongers, André; Wessa, Michèle

2010-02-01

Motivational orientation defines the source of motivation for an individual to perform a particular action and can either originate from internal desires (e.g., interest) or external compensation (e.g., money). To this end, motivational orientation should influence the way positive or negative feedback is processed during learning situations and this might in turn have an impact on the learning process. In the present study, we thus investigated whether motivational orientation, i.e., extrinsic and intrinsic motivation modulates the neural response to reward and punishment as well as learning from reward and punishment in 33 healthy individuals. To assess neural responses to reward, punishment and learning of reward contingencies we employed a probabilistic reversal learning task during functional magnetic resonance imaging. Extrinsic and intrinsic motivation were assessed with a self-report questionnaire. Rewarding trials fostered activation in the medial orbitofrontal cortex and anterior cingulate gyrus (ACC) as well as the amygdala and nucleus accumbens, whereas for punishment an increased neural response was observed in the medial and inferior prefrontal cortex, the superior parietal cortex and the insula. High extrinsic motivation was positively correlated to increased neural responses to reward in the ACC, amygdala and putamen, whereas a negative relationship between intrinsic motivation and brain activation in these brain regions was observed. These findings show that motivational orientation indeed modulates the responsiveness to reward delivery in major components of the human reward system and therefore extends previous results showing a significant influence of individual differences in reward-related personality traits on the neural processing of reward. Copyright (c) 2009 Elsevier Inc. All rights reserved.

Spike-Timing of Orbitofrontal Neurons Is Synchronized With Breathing.

Science.gov (United States)

Kőszeghy, Áron; Lasztóczi, Bálint; Forro, Thomas; Klausberger, Thomas

2018-01-01

The orbitofrontal cortex (OFC) has been implicated in a multiplicity of complex brain functions, including representations of expected outcome properties, post-decision confidence, momentary food-reward values, complex flavors and odors. As breathing rhythm has an influence on odor processing at primary olfactory areas, we tested the hypothesis that it may also influence neuronal activity in the OFC, a prefrontal area involved also in higher order processing of odors. We recorded spike timing of orbitofrontal neurons as well as local field potentials (LFPs) in awake, head-fixed mice, together with the breathing rhythm. We observed that a large majority of orbitofrontal neurons showed robust phase-coupling to breathing during immobility and running. The phase coupling of action potentials to breathing was significantly stronger in orbitofrontal neurons compared to cells in the medial prefrontal cortex. The characteristic synchronization of orbitofrontal neurons with breathing might provide a temporal framework for multi-variable processing of olfactory, gustatory and reward-value relationships.

Spike-Timing of Orbitofrontal Neurons Is Synchronized With Breathing

Directory of Open Access Journals (Sweden)

Áron Kőszeghy

2018-04-01

Full Text Available The orbitofrontal cortex (OFC has been implicated in a multiplicity of complex brain functions, including representations of expected outcome properties, post-decision confidence, momentary food-reward values, complex flavors and odors. As breathing rhythm has an influence on odor processing at primary olfactory areas, we tested the hypothesis that it may also influence neuronal activity in the OFC, a prefrontal area involved also in higher order processing of odors. We recorded spike timing of orbitofrontal neurons as well as local field potentials (LFPs in awake, head-fixed mice, together with the breathing rhythm. We observed that a large majority of orbitofrontal neurons showed robust phase-coupling to breathing during immobility and running. The phase coupling of action potentials to breathing was significantly stronger in orbitofrontal neurons compared to cells in the medial prefrontal cortex. The characteristic synchronization of orbitofrontal neurons with breathing might provide a temporal framework for multi-variable processing of olfactory, gustatory and reward-value relationships.

Stochastic synaptic plasticity with memristor crossbar arrays

KAUST Repository

Naous, Rawan

2016-11-01

Memristive devices have been shown to exhibit slow and stochastic resistive switching behavior under low-voltage, low-current operating conditions. Here we explore such mechanisms to emulate stochastic plasticity in memristor crossbar synapse arrays. Interfaced with integrate-and-fire spiking neurons, the memristive synapse arrays are capable of implementing stochastic forms of spike-timing dependent plasticity which parallel mean-rate models of stochastic learning with binary synapses. We present theory and experiments with spike-based stochastic learning in memristor crossbar arrays, including simplified modeling as well as detailed physical simulation of memristor stochastic resistive switching characteristics due to voltage and current induced filament formation and collapse. © 2016 IEEE.

Stochastic synaptic plasticity with memristor crossbar arrays

KAUST Repository

Naous, Rawan; Al-Shedivat, Maruan; Neftci, Emre; Cauwenberghs, Gert; Salama, Khaled N.

2016-01-01

Memristive devices have been shown to exhibit slow and stochastic resistive switching behavior under low-voltage, low-current operating conditions. Here we explore such mechanisms to emulate stochastic plasticity in memristor crossbar synapse arrays. Interfaced with integrate-and-fire spiking neurons, the memristive synapse arrays are capable of implementing stochastic forms of spike-timing dependent plasticity which parallel mean-rate models of stochastic learning with binary synapses. We present theory and experiments with spike-based stochastic learning in memristor crossbar arrays, including simplified modeling as well as detailed physical simulation of memristor stochastic resistive switching characteristics due to voltage and current induced filament formation and collapse. © 2016 IEEE.

Opposite modulation of brain stimulation reward by NMDA and AMPA receptors in the ventral tegmental area.

Science.gov (United States)

Ducrot, Charles; Fortier, Emmanuel; Bouchard, Claude; Rompré, Pierre-Paul

2013-01-01

Previous studies have shown that blockade of ventral tegmental area (VTA) glutamate N-Methyl-D-Aspartate (NMDA) receptors induces reward, stimulates forward locomotion and enhances brain stimulation reward. Glutamate induces two types of excitatory response on VTA neurons, a fast and short lasting depolarization mediated by α-amino-3-hydroxy-5-methyl-4-isoxazole propionate (AMPA) receptors and a longer lasting depolarization mediated by NMDA receptors. A role for the two glutamate receptors in modulation of VTA neuronal activity is evidenced by the functional change in AMPA and NMDA synaptic responses that result from repeated exposure to reward. Since both receptors contribute to the action of glutamate on VTA neuronal activity, we studied the effects of VTA AMPA and NMDA receptor blockade on reward induced by electrical brain stimulation. Experiments were performed on rats trained to self-administer electrical pulses in the medial posterior mesencephalon. Reward thresholds were measured with the curve-shift paradigm before and for 2 h after bilateral VTA microinjections of the AMPA antagonist, NBQX (2,3,-Dioxo-6-nitro-1,2,3,4-tetrahydrobenzo(f)quinoxaline-7-sulfonamide, 0, 80, and 800 pmol/0.5 μl/side) and of a single dose (0.825 nmol/0.5 μl/side) of the NMDA antagonist, PPPA (2R,4S)-4-(3-Phosphonopropyl)-2-piperidinecarboxylic acid). NBQX produced a dose-dependent increase in reward threshold with no significant change in maximum rate of responding. Whereas PPPA injected at the same VTA sites produced a significant time dependent decrease in reward threshold and increase in maximum rate of responding. We found a negative correlation between the magnitude of the attenuation effect of NBQX and the enhancement effect of PPPA; moreover, NBQX and PPPA were most effective when injected, respectively, into the anterior and posterior VTA. These results suggest that glutamate acts on different receptor sub-types, most likely located on different VTA neurons, to

Opposite modulation of brain stimulation reward by NMDA and AMPA receptors in the ventral tegmental area.

Directory of Open Access Journals (Sweden)

Charles eDucrot

2013-10-01

Full Text Available Previous studies have shown that blockade of ventral midbrain (VM glutamate N-Methyl-D-Aspartate (NMDA receptors induces reward, stimulates forward locomotion and enhances brain stimulation reward. Glutamate induces two types of excitatory response on VM neurons, a fast and short lasting depolarisation mediated by a-amino-3-hydroxy-5-methyl-4-isoxazole propionate (AMPA receptors and a longer lasting depolarization mediated by NMDA receptors. A role for the two glutamate receptors in modulation of VM neuronal activity is evidenced by the functional change in AMPA and NMDA synaptic responses that result from repeated exposure to reward. Since both receptors contribute to the action of glutamate on VM neuronal activity, we studied the effects of VM AMPA and NMDA receptor blockade on reward induced by electrical brain stimulation. Experiments were performed on rats trained to self-administer electrical pulses in the medial posterior mesencephalon. Reward thresholds were measured with the curve-shift paradigm before and for two hours after bilateral VM microinjections of the AMPA antagonist, NBQX (2,3,-Dioxo-6-nitro-1,2,3,4-tetrahydrobenzo(fquinoxaline-7-sulfonamide, 0, 80, and 800 pmol/0.5ul/side and of a single dose (0.825 nmol/0.5ul/side of the NMDA antagonist, PPPA (2R,4S-4-(3-Phosphonopropyl-2-piperidinecarboxylic acid. NBQX produced a dose-dependent increase in reward threshold with no significant change in maximum rate of responding. Whereas PPPA injected at the same VM sites produced a significant time dependent decrease in reward threshold and increase in maximum rate of responding. We found a negative correlation between the magnitude of the attenuation effect of NBQX and the enhancement effect of PPPA; moreover, NBQX and PPPA were most effective when injected respectively into the anterior and posterior VM. These results suggest that glutamate acts on different receptor sub-types, most likely located on different VM neurons, to modulate

A Multiple-Plasticity Spiking Neural Network Embedded in a Closed-Loop Control System to Model Cerebellar Pathologies.

Science.gov (United States)

Geminiani, Alice; Casellato, Claudia; Antonietti, Alberto; D'Angelo, Egidio; Pedrocchi, Alessandra

2018-06-01

The cerebellum plays a crucial role in sensorimotor control and cerebellar disorders compromise adaptation and learning of motor responses. However, the link between alterations at network level and cerebellar dysfunction is still unclear. In principle, this understanding would benefit of the development of an artificial system embedding the salient neuronal and plastic properties of the cerebellum and operating in closed-loop. To this aim, we have exploited a realistic spiking computational model of the cerebellum to analyze the network correlates of cerebellar impairment. The model was modified to reproduce three different damages of the cerebellar cortex: (i) a loss of the main output neurons (Purkinje Cells), (ii) a lesion to the main cerebellar afferents (Mossy Fibers), and (iii) a damage to a major mechanism of synaptic plasticity (Long Term Depression). The modified network models were challenged with an Eye-Blink Classical Conditioning test, a standard learning paradigm used to evaluate cerebellar impairment, in which the outcome was compared to reference results obtained in human or animal experiments. In all cases, the model reproduced the partial and delayed conditioning typical of the pathologies, indicating that an intact cerebellar cortex functionality is required to accelerate learning by transferring acquired information to the cerebellar nuclei. Interestingly, depending on the type of lesion, the redistribution of synaptic plasticity and response timing varied greatly generating specific adaptation patterns. Thus, not only the present work extends the generalization capabilities of the cerebellar spiking model to pathological cases, but also predicts how changes at the neuronal level are distributed across the network, making it usable to infer cerebellar circuit alterations occurring in cerebellar pathologies.

The dependence of neuronal encoding efficiency on Hebbian plasticity and homeostatic regulation of neurotransmitter release

Directory of Open Access Journals (Sweden)

Faramarz eFaghihi

2015-04-01

Full Text Available Synapses act as information filters by different molecular mechanisms including retrograde messenger that affect neuronal spiking activity. One of the well-known effects of retrograde messenger in presynaptic neurons is a change of the probability of neurotransmitter release. Hebbian learning describe a strengthening of a synapse between a presynaptic input onto a postsynaptic neuron when both pre- and postsynaptic neurons are coactive. In this work, a theory of homeostatic regulation of neurotransmitter release by retrograde messenger and Hebbian plasticity in neuronal encoding is presented. Encoding efficiency was measured for different synaptic conditions. In order to gain high encoding efficiency, the spiking pattern of a neuron should be dependent on the intensity of the input and show low levels of noise. In this work, we represent spiking trains as zeros and ones (corresponding to non-spike or spike in a time bin, respectively as words with length equal to three. Then the frequency of each word (here eight words is measured using spiking trains. These frequencies are used to measure neuronal efficiency in different conditions and for different parameter values. Results show that neurons that have synapses acting as band-pass filters show the highest efficiency to encode their input when both Hebbian mechanism and homeostatic regulation of neurotransmitter release exist in synapses. Specifically, the integration of homeostatic regulation of feedback inhibition with Hebbian mechanism and homeostatic regulation of neurotransmitter release in the synapses leads to even higher efficiency when high stimulus intensity is presented to the neurons. However, neurons with synapses acting as high-pass filters show no remarkable increase in encoding efficiency for all simulated synaptic plasticity mechanisms.

Monetary reward modulates task-irrelevant perceptual learning for invisible stimuli.

Science.gov (United States)

Pascucci, David; Mastropasqua, Tommaso; Turatto, Massimo

2015-01-01

Task Irrelevant Perceptual Learning (TIPL) shows that the brain's discriminative capacity can improve also for invisible and unattended visual stimuli. It has been hypothesized that this form of "unconscious" neural plasticity is mediated by an endogenous reward mechanism triggered by the correct task performance. Although this result has challenged the mandatory role of attention in perceptual learning, no direct evidence exists of the hypothesized link between target recognition, reward and TIPL. Here, we manipulated the reward value associated with a target to demonstrate the involvement of reinforcement mechanisms in sensory plasticity for invisible inputs. Participants were trained in a central task associated with either high or low monetary incentives, provided only at the end of the experiment, while subliminal stimuli were presented peripherally. Our results showed that high incentive-value targets induced a greater degree of perceptual improvement for the subliminal stimuli, supporting the role of reinforcement mechanisms in TIPL.

A Reinforcement Learning Framework for Spiking Networks with Dynamic Synapses

Directory of Open Access Journals (Sweden)

Karim El-Laithy

2011-01-01

Full Text Available An integration of both the Hebbian-based and reinforcement learning (RL rules is presented for dynamic synapses. The proposed framework permits the Hebbian rule to update the hidden synaptic model parameters regulating the synaptic response rather than the synaptic weights. This is performed using both the value and the sign of the temporal difference in the reward signal after each trial. Applying this framework, a spiking network with spike-timing-dependent synapses is tested to learn the exclusive-OR computation on a temporally coded basis. Reward values are calculated with the distance between the output spike train of the network and a reference target one. Results show that the network is able to capture the required dynamics and that the proposed framework can reveal indeed an integrated version of Hebbian and RL. The proposed framework is tractable and less computationally expensive. The framework is applicable to a wide class of synaptic models and is not restricted to the used neural representation. This generality, along with the reported results, supports adopting the introduced approach to benefit from the biologically plausible synaptic models in a wide range of intuitive signal processing.

Effects of Spike Anticipation on the Spiking Dynamics of Neural Networks

Directory of Open Access Journals (Sweden)

Daniel ede Santos-Sierra

2015-11-01

Full Text Available Synchronization is one of the central phenomena involved in information processing in living systems. It is known that the nervous system requires the coordinated activity of both local and distant neural populations. Such an interplay allows to merge different information modalities in a whole processing supporting high-level mental skills as understanding, memory, abstraction, etc. Though the biological processes underlying synchronization in the brain are not fully understood there have been reported a variety of mechanisms supporting different types of synchronization both at theoretical and experimental level. One of the more intriguing of these phenomena is the anticipating synchronization, which has been recently reported in a pair of unidirectionally coupled artificial neurons under simple conditions cite{Pyragas}, where the slave neuron is able to anticipate in time the behaviour of the master one. In this paper we explore the effect of spike anticipation over the information processing performed by a neural network at functional and structural level. We show that the introduction of intermediary neurons in the network enhances spike anticipation and analyse how these variations in spike anticipation can significantly change the firing regime of the neural network according to its functional and structural properties. In addition we show that the interspike interval (ISI, one of the main features of the neural response associated to the information coding, can be closely related to spike anticipation by each spike, and how synaptic plasticity can be modulated through that relationship. This study has been performed through numerical simulation of a coupled system of Hindmarsh-Rose neurons.

Effects of Spike Anticipation on the Spiking Dynamics of Neural Networks.

Science.gov (United States)

de Santos-Sierra, Daniel; Sanchez-Jimenez, Abel; Garcia-Vellisca, Mariano A; Navas, Adrian; Villacorta-Atienza, Jose A

2015-01-01

Synchronization is one of the central phenomena involved in information processing in living systems. It is known that the nervous system requires the coordinated activity of both local and distant neural populations. Such an interplay allows to merge different information modalities in a whole processing supporting high-level mental skills as understanding, memory, abstraction, etc. Though, the biological processes underlying synchronization in the brain are not fully understood there have been reported a variety of mechanisms supporting different types of synchronization both at theoretical and experimental level. One of the more intriguing of these phenomena is the anticipating synchronization, which has been recently reported in a pair of unidirectionally coupled artificial neurons under simple conditions (Pyragiene and Pyragas, 2013), where the slave neuron is able to anticipate in time the behavior of the master one. In this paper, we explore the effect of spike anticipation over the information processing performed by a neural network at functional and structural level. We show that the introduction of intermediary neurons in the network enhances spike anticipation and analyse how these variations in spike anticipation can significantly change the firing regime of the neural network according to its functional and structural properties. In addition we show that the interspike interval (ISI), one of the main features of the neural response associated with the information coding, can be closely related to spike anticipation by each spike, and how synaptic plasticity can be modulated through that relationship. This study has been performed through numerical simulation of a coupled system of Hindmarsh-Rose neurons.

Endocannabinoid signaling in reward and addiction

Science.gov (United States)

Parsons, Loren H.; Hurd, Yasmin L.

2015-01-01

Brain endocannabinoid signaling influences the motivation for natural rewards (such as palatable food, sexual activity and social interaction) and modulates the rewarding effects of addictive drugs. Pathological forms of natural and drug-induced reward are associated with dysregulated endocannabinoid signaling that may derive from pre-existing genetic factors or from prolonged drug exposure. Impaired endocannabinoid signaling contributes to dysregulated synaptic plasticity, increased stress responsivity, negative emotional states, and craving that propel addiction. Understanding the contributions of endocannabinoid disruptions to behavioral and physiological traits provides insight into the endocannabinoid influence on addiction vulnerability. PMID:26373473

Cannabinoid transmission in the prelimbic cortex bidirectionally controls opiate reward and aversion signaling through dissociable kappa versus μ-opiate receptor dependent mechanisms.

Science.gov (United States)

Ahmad, Tasha; Lauzon, Nicole M; de Jaeger, Xavier; Laviolette, Steven R

2013-09-25

Cannabinoid, dopamine (DA), and opiate receptor pathways play integrative roles in emotional learning, associative memory, and sensory perception. Modulation of cannabinoid CB1 receptor transmission within the medial prefrontal cortex (mPFC) regulates the emotional valence of both rewarding and aversive experiences. Furthermore, CB1 receptor substrates functionally interact with opiate-related motivational processing circuits, particularly in the context of reward-related learning and memory. Considerable evidence demonstrates functional interactions between CB1 and DA signaling pathways during the processing of motivationally salient information. However, the role of mPFC CB1 receptor transmission in the modulation of behavioral opiate-reward processing is not currently known. Using an unbiased conditioned place preference paradigm with rats, we examined the role of intra-mPFC CB1 transmission during opiate reward learning. We report that activation or inhibition of CB1 transmission within the prelimbic cortical (PLC) division of the mPFC bidirectionally regulates the motivational valence of opiates; whereas CB1 activation switched morphine reward signaling into an aversive stimulus, blockade of CB1 transmission potentiated the rewarding properties of normally sub-reward threshold conditioning doses of morphine. Both of these effects were dependent upon DA transmission as systemic blockade of DAergic transmission prevented CB1-dependent modulation of morphine reward and aversion behaviors. We further report that CB1-mediated intra-PLC opiate motivational signaling is mediated through a μ-opiate receptor-dependent reward pathway, or a κ-opiate receptor-dependent aversion pathway, directly within the ventral tegmental area. Our results provide evidence for a novel CB1-mediated motivational valence switching mechanism within the PLC, controlling dissociable subcortical reward and aversion pathways.

Monetary reward modulates task-irrelevant perceptual learning for invisible stimuli.

Directory of Open Access Journals (Sweden)

David Pascucci

Full Text Available Task Irrelevant Perceptual Learning (TIPL shows that the brain's discriminative capacity can improve also for invisible and unattended visual stimuli. It has been hypothesized that this form of "unconscious" neural plasticity is mediated by an endogenous reward mechanism triggered by the correct task performance. Although this result has challenged the mandatory role of attention in perceptual learning, no direct evidence exists of the hypothesized link between target recognition, reward and TIPL. Here, we manipulated the reward value associated with a target to demonstrate the involvement of reinforcement mechanisms in sensory plasticity for invisible inputs. Participants were trained in a central task associated with either high or low monetary incentives, provided only at the end of the experiment, while subliminal stimuli were presented peripherally. Our results showed that high incentive-value targets induced a greater degree of perceptual improvement for the subliminal stimuli, supporting the role of reinforcement mechanisms in TIPL.

Cortical plasticity induced by spike-triggered microstimulation in primate somatosensory cortex.

Directory of Open Access Journals (Sweden)

Weiguo Song

Full Text Available Electrical stimulation of the nervous system for therapeutic purposes, such as deep brain stimulation in the treatment of Parkinson's disease, has been used for decades. Recently, increased attention has focused on using microstimulation to restore functions as diverse as somatosensation and memory. However, how microstimulation changes the neural substrate is still not fully understood. Microstimulation may cause cortical changes that could either compete with or complement natural neural processes, and could result in neuroplastic changes rendering the region dysfunctional or even epileptic. As part of our efforts to produce neuroprosthetic devices and to further study the effects of microstimulation on the cortex, we stimulated and recorded from microelectrode arrays in the hand area of the primary somatosensory cortex (area 1 in two awake macaque monkeys. We applied a simple neuroprosthetic microstimulation protocol to a pair of electrodes in the area 1 array, using either random pulses or pulses time-locked to the recorded spiking activity of a reference neuron. This setup was replicated using a computer model of the thalamocortical system, which consisted of 1980 spiking neurons distributed among six cortical layers and two thalamic nuclei. Experimentally, we found that spike-triggered microstimulation induced cortical plasticity, as shown by increased unit-pair mutual information, while random microstimulation did not. In addition, there was an increased response to touch following spike-triggered microstimulation, along with decreased neural variability. The computer model successfully reproduced both qualitative and quantitative aspects of the experimental findings. The physiological findings of this study suggest that even simple microstimulation protocols can be used to increase somatosensory information flow.

Differential Activation of Fast-Spiking and Regular-Firing Neuron Populations During Movement and Reward in the Dorsal Medial Frontal Cortex

Science.gov (United States)

Insel, Nathan; Barnes, Carol A.

2015-01-01

The medial prefrontal cortex is thought to be important for guiding behavior according to an animal's expectations. Efforts to decode the region have focused not only on the question of what information it computes, but also how distinct circuit components become engaged during behavior. We find that the activity of regular-firing, putative projection neurons contains rich information about behavioral context and firing fields cluster around reward sites, while activity among putative inhibitory and fast-spiking neurons is most associated with movement and accompanying sensory stimulation. These dissociations were observed even between adjacent neurons with apparently reciprocal, inhibitory–excitatory connections. A smaller population of projection neurons with burst-firing patterns did not show clustered firing fields around rewards; these neurons, although heterogeneous, were generally less selective for behavioral context than regular-firing cells. The data suggest a network that tracks an animal's behavioral situation while, at the same time, regulating excitation levels to emphasize high valued positions. In this scenario, the function of fast-spiking inhibitory neurons is to constrain network output relative to incoming sensory flow. This scheme could serve as a bridge between abstract sensorimotor information and single-dimensional codes for value, providing a neural framework to generate expectations from behavioral state. PMID:24700585

A framework for plasticity implementation on the SpiNNaker neural architecture.

Science.gov (United States)

Galluppi, Francesco; Lagorce, Xavier; Stromatias, Evangelos; Pfeiffer, Michael; Plana, Luis A; Furber, Steve B; Benosman, Ryad B

2014-01-01

Many of the precise biological mechanisms of synaptic plasticity remain elusive, but simulations of neural networks have greatly enhanced our understanding of how specific global functions arise from the massively parallel computation of neurons and local Hebbian or spike-timing dependent plasticity rules. For simulating large portions of neural tissue, this has created an increasingly strong need for large scale simulations of plastic neural networks on special purpose hardware platforms, because synaptic transmissions and updates are badly matched to computing style supported by current architectures. Because of the great diversity of biological plasticity phenomena and the corresponding diversity of models, there is a great need for testing various hypotheses about plasticity before committing to one hardware implementation. Here we present a novel framework for investigating different plasticity approaches on the SpiNNaker distributed digital neural simulation platform. The key innovation of the proposed architecture is to exploit the reconfigurability of the ARM processors inside SpiNNaker, dedicating a subset of them exclusively to process synaptic plasticity updates, while the rest perform the usual neural and synaptic simulations. We demonstrate the flexibility of the proposed approach by showing the implementation of a variety of spike- and rate-based learning rules, including standard Spike-Timing dependent plasticity (STDP), voltage-dependent STDP, and the rate-based BCM rule. We analyze their performance and validate them by running classical learning experiments in real time on a 4-chip SpiNNaker board. The result is an efficient, modular, flexible and scalable framework, which provides a valuable tool for the fast and easy exploration of learning models of very different kinds on the parallel and reconfigurable SpiNNaker system.

Coincidence Detection Using Spiking Neurons with Application to Face Recognition

Directory of Open Access Journals (Sweden)

Fadhlan Kamaruzaman

2015-01-01

Full Text Available We elucidate the practical implementation of Spiking Neural Network (SNN as local ensembles of classifiers. Synaptic time constant τs is used as learning parameter in representing the variations learned from a set of training data at classifier level. This classifier uses coincidence detection (CD strategy trained in supervised manner using a novel supervised learning method called τs Prediction which adjusts the precise timing of output spikes towards the desired spike timing through iterative adaptation of τs. This paper also discusses the approximation of spike timing in Spike Response Model (SRM for the purpose of coincidence detection. This process significantly speeds up the whole process of learning and classification. Performance evaluations with face datasets such as AR, FERET, JAFFE, and CK+ datasets show that the proposed method delivers better face classification performance than the network trained with Supervised Synaptic-Time Dependent Plasticity (STDP. We also found that the proposed method delivers better classification accuracy than k nearest neighbor, ensembles of kNN, and Support Vector Machines. Evaluation on several types of spike codings also reveals that latency coding delivers the best result for face classification as well as for classification of other multivariate datasets.

Delay discounting of oral morphine and sweetened juice rewards in dependent and non-dependent rats.

Science.gov (United States)

Harvey-Lewis, Colin; Perdrizet, Johnna; Franklin, Keith B J

2014-07-01

Opioid-dependent humans are reported to show accelerated delay discounting of opioid rewards when compared to monetary rewards. It has been suggested that this may reflect a difference in discounting of consumable and non-consumable goods not specific to dependent individuals. Here, we evaluate the discounting of similar morphine and non-morphine oral rewards in dependent and non-dependent rats We first tested the analgesic and rewarding effects of our morphine solution. In a second experiment, we assigned rats randomly to either dependent or non-dependent groups that, 30 min after daily testing, received 30 mg/kg subcutaneous dose of morphine, or saline, respectively. Delay discounting of drug-free reward was examined prior to initiation of the dosing regimen. We tested discounting of the morphine reward in half the rats and retested the discounting of the drug-free reward in the other half. All tests were run 22.5 h after the daily maintenance dose. Rats preferred the morphine cocktail to the drug-free solution and consumed enough to induce significant analgesia. The control quinine solution did not produce these effects. Dependent rats discounted morphine rewards more rapidly than before dependence and when compared to discounting drug-free rewards. In non-dependent rats both reward types were discounted similarly. These results show that morphine dependence increases impulsiveness specifically towards a drug reward while morphine experience without dependence does not.

MPTP-meditated hippocampal dopamine deprivation modulates synaptic transmission and activity-dependent synaptic plasticity

International Nuclear Information System (INIS)

Zhu Guoqi; Chen Ying; Huang Yuying; Li Qinglin; Behnisch, Thomas

2011-01-01

Parkinson's disease (PD)-like symptoms including learning deficits are inducible by 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP). Therefore, it is possible that MPTP may disturb hippocampal memory processing by modulation of dopamine (DA)- and activity-dependent synaptic plasticity. We demonstrate here that intraperitoneal (i.p.) MPTP injection reduces the number of tyrosine hydroxylase (TH)-positive neurons in the substantia nigra (SN) within 7 days. Subsequently, the TH expression level in SN and hippocampus and the amount of DA and its metabolite DOPAC in striatum and hippocampus decrease. DA depletion does not alter basal synaptic transmission and changes pair-pulse facilitation (PPF) of field excitatory postsynaptic potentials (fEPSPs) only at the 30 ms inter-pulse interval. In addition, the induction of long-term potentiation (LTP) is impaired whereas the duration of long-term depression (LTD) becomes prolonged. Since both LTP and LTD depend critically on activation of NMDA and DA receptors, we also tested the effect of DA depletion on NMDA receptor-mediated synaptic transmission. Seven days after MPTP injection, the NMDA receptor-mediated fEPSPs are decreased by about 23%. Blocking the NMDA receptor-mediated fEPSP does not mimic the MPTP-LTP. Only co-application of D1/D5 and NMDA receptor antagonists during tetanization resembled the time course of fEPSP potentiation as observed 7 days after i.p. MPTP injection. Together, our data demonstrate that MPTP-induced degeneration of DA neurons and the subsequent hippocampal DA depletion alter NMDA receptor-mediated synaptic transmission and activity-dependent synaptic plasticity. - Highlights: → I.p. MPTP-injection mediates death of dopaminergic neurons. → I.p. MPTP-injection depletes DA and DOPAC in striatum and hippocampus. → I.p. MPTP-injection does not alter basal synaptic transmission. → Reduction of LTP and enhancement of LTD after i.p. MPTP-injection. → Attenuation of NMDA-receptors mediated

Transformation-invariant visual representations in self-organizing spiking neural networks.

Science.gov (United States)

Evans, Benjamin D; Stringer, Simon M

2012-01-01

The ventral visual pathway achieves object and face recognition by building transformation-invariant representations from elementary visual features. In previous computer simulation studies with rate-coded neural networks, the development of transformation-invariant representations has been demonstrated using either of two biologically plausible learning mechanisms, Trace learning and Continuous Transformation (CT) learning. However, it has not previously been investigated how transformation-invariant representations may be learned in a more biologically accurate spiking neural network. A key issue is how the synaptic connection strengths in such a spiking network might self-organize through Spike-Time Dependent Plasticity (STDP) where the change in synaptic strength is dependent on the relative times of the spikes emitted by the presynaptic and postsynaptic neurons rather than simply correlated activity driving changes in synaptic efficacy. Here we present simulations with conductance-based integrate-and-fire (IF) neurons using a STDP learning rule to address these gaps in our understanding. It is demonstrated that with the appropriate selection of model parameters and training regime, the spiking network model can utilize either Trace-like or CT-like learning mechanisms to achieve transform-invariant representations.

Transform-invariant visual representations in self-organizing spiking neural networks

Directory of Open Access Journals (Sweden)

Benjamin eEvans

2012-07-01

Full Text Available The ventral visual pathway achieves object and face recognition by building transform-invariant representations from elementary visual features. In previous computer simulation studies with rate-coded neural networks, the development of transform invariant representations has been demonstrated using either of two biologically plausible learning mechanisms, Trace learning and Continuous Transformation (CT learning. However, it has not previously been investigated how transform invariant representations may be learned in a more biologically accurate spiking neural network. A key issue is how the synaptic connection strengths in such a spiking network might self-organize through Spike-Time Dependent Plasticity (STDP where the change in synaptic strength is dependent on the relative times of the spikes emitted by the pre- and postsynaptic neurons rather than simply correlated activity driving changes in synaptic efficacy. Here we present simulations with conductance-based integrate-and-fire (IF neurons using a STDP learning rule to address these gaps in our understanding. It is demonstrated that with the appropriate selection of model pa- rameters and training regime, the spiking network model can utilize either Trace-like or CT-like learning mechanisms to achieve transform-invariant representations.

Reward-Enhanced Memory in Younger and Older Adults

OpenAIRE

Julia Spaniol; Cécile Schain; Holly J. Bowen

2014-01-01

Objectives. We investigated how the anticipation of remote monetary reward modulates intentional episodic memory formation in younger and older adults. On the basis of prior findings of preserved reward–cognition interactions in aging, we predicted that reward anticipation would be associated with enhanced memory in both younger and older adults. On the basis of previous demonstrations of a time-dependent effect of reward anticipation on memory, we expected the memory enhancement to increase ...

Real-time cerebellar neuroprosthetic system based on a spiking neural network model of motor learning.

Science.gov (United States)

Xu, Tao; Xiao, Na; Zhai, Xiaolong; Kwan Chan, Pak; Tin, Chung

2018-02-01

Damage to the brain, as a result of various medical conditions, impacts the everyday life of patients and there is still no complete cure to neurological disorders. Neuroprostheses that can functionally replace the damaged neural circuit have recently emerged as a possible solution to these problems. Here we describe the development of a real-time cerebellar neuroprosthetic system to substitute neural function in cerebellar circuitry for learning delay eyeblink conditioning (DEC). The system was empowered by a biologically realistic spiking neural network (SNN) model of the cerebellar neural circuit, which considers the neuronal population and anatomical connectivity of the network. The model simulated synaptic plasticity critical for learning DEC. This SNN model was carefully implemented on a field programmable gate array (FPGA) platform for real-time simulation. This hardware system was interfaced in in vivo experiments with anesthetized rats and it used neural spikes recorded online from the animal to learn and trigger conditioned eyeblink in the animal during training. This rat-FPGA hybrid system was able to process neuronal spikes in real-time with an embedded cerebellum model of ~10 000 neurons and reproduce learning of DEC with different inter-stimulus intervals. Our results validated that the system performance is physiologically relevant at both the neural (firing pattern) and behavioral (eyeblink pattern) levels. This integrated system provides the sufficient computation power for mimicking the cerebellar circuit in real-time. The system interacts with the biological system naturally at the spike level and can be generalized for including other neural components (neuron types and plasticity) and neural functions for potential neuroprosthetic applications.

Effect of tensile properties on time-dependent C(t) and J(t) integrals in elastic-plastic-creep FE analysis

International Nuclear Information System (INIS)

Lee, So-Dam; Lee, Han-Sang; Kim, Yun-Jae; Ainsworth, Robert A.; Dean, David W.

2016-01-01

This technical note presents the effect of elastic-plastic properties on calculated time-dependent C(t) and J(t) values. This is investigated via systematic elastic-plastic-creep finite element (FE) analysis. Three different stress-strain curves are used, having essentially the same plastic properties at large strains but different tensile data near the 0.2% proof (yield) strength. It is found that the plastic property in stress-strain curve affects the FE C(t) values only at short times (within approximately 20% of the redistribution time). The plastic property affects the initial J values at time t = 0 but not the rate of change of J(t) with time. - Highlights: • The effect of elastic-plastic properties on calculated time-dependent C(t) and J(t) values is presented via FE analysis. • The plastic property affects the FE C(t) values only at short times up to ∼20% of the redistribution time. • The plastic property affects the initial J values at time t = 0 but not the rate of change of J(t) with time.

Motor control by precisely timed spike patterns

DEFF Research Database (Denmark)

Srivastava, Kyle H; Holmes, Caroline M; Vellema, Michiel

2017-01-01

whether the information in spike timing actually plays a role in brain function. By examining the activity of individual motor units (the muscle fibers innervated by a single motor neuron) and manipulating patterns of activation of these neurons, we provide both correlative and causal evidence......A fundamental problem in neuroscience is understanding how sequences of action potentials ("spikes") encode information about sensory signals and motor outputs. Although traditional theories assume that this information is conveyed by the total number of spikes fired within a specified time...... interval (spike rate), recent studies have shown that additional information is carried by the millisecond-scale timing patterns of action potentials (spike timing). However, it is unknown whether or how subtle differences in spike timing drive differences in perception or behavior, leaving it unclear...

Limits to high-speed simulations of spiking neural networks using general-purpose computers.

Science.gov (United States)

Zenke, Friedemann; Gerstner, Wulfram

2014-01-01

To understand how the central nervous system performs computations using recurrent neuronal circuitry, simulations have become an indispensable tool for theoretical neuroscience. To study neuronal circuits and their ability to self-organize, increasing attention has been directed toward synaptic plasticity. In particular spike-timing-dependent plasticity (STDP) creates specific demands for simulations of spiking neural networks. On the one hand a high temporal resolution is required to capture the millisecond timescale of typical STDP windows. On the other hand network simulations have to evolve over hours up to days, to capture the timescale of long-term plasticity. To do this efficiently, fast simulation speed is the crucial ingredient rather than large neuron numbers. Using different medium-sized network models consisting of several thousands of neurons and off-the-shelf hardware, we compare the simulation speed of the simulators: Brian, NEST and Neuron as well as our own simulator Auryn. Our results show that real-time simulations of different plastic network models are possible in parallel simulations in which numerical precision is not a primary concern. Even so, the speed-up margin of parallelism is limited and boosting simulation speeds beyond one tenth of real-time is difficult. By profiling simulation code we show that the run times of typical plastic network simulations encounter a hard boundary. This limit is partly due to latencies in the inter-process communications and thus cannot be overcome by increased parallelism. Overall, these results show that to study plasticity in medium-sized spiking neural networks, adequate simulation tools are readily available which run efficiently on small clusters. However, to run simulations substantially faster than real-time, special hardware is a prerequisite.

Theta-band phase locking of orbitofrontal neurons during reward expectancy

NARCIS (Netherlands)

van Wingerden, M.; Vinck, M.; Lankelma, J.; Pennartz, C.M.A.

2010-01-01

The expectancy of a rewarding outcome following actions and cues is coded by a network of brain structures including the orbitofrontal cortex. Thus far, predicted reward was considered to be coded by time-averaged spike rates of neurons. However, besides firing rate, the precise timing of action

Structural and Functional Plasticity within the Nucleus Accumbens and Prefrontal Cortex Associated with Time-Dependent Increases in Food Cue-Seeking Behavior.

Science.gov (United States)

Dingess, Paige M; Darling, Rebecca A; Derman, Rifka C; Wulff, Shaun S; Hunter, Melissa L; Ferrario, Carrie R; Brown, Travis E

2017-11-01

Urges to consume food can be driven by stimuli in the environment that are associated with previous food experience. Identifying adaptations within brain reward circuits that facilitate cue-induced food seeking is critical for understanding and preventing the overconsumption of food and subsequent weight gain. Utilizing electrophysiological, biochemical, and DiI labeling, we examined functional and structural changes in the nucleus accumbens (NAc) and prefrontal cortex (PFC) associated with time-dependent increases in food craving ('incubation of craving'). Rats self-administered 60% high fat or chow 45 mg pellets and were then tested for incubation of craving either 1 or 30 days after training. High fat was chosen for comparison to determine whether palatability differentially affected incubation and/or plasticity. Rats showed robust incubation of craving for both food rewards, although responding for cues previously associated with high fat was greater than chow at both 1 and 30 days. In addition, previous experience with high-fat consumption reduced dendritic spine density in the PFC at both time points. In contrast, incubation was associated with an increase in NAc spine density and α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptor (AMPAR)-mediated transmission at 30 days in both groups. Finally, incubation of craving for chow and high fat was accompanied by an increase in calcium-permeable and calcium-impermeable AMPARs, respectively. Our results suggest that incubation of food craving alters brain reward circuitry and macronutrient composition specifically induces cortical changes in a way that may facilitate maladaptive food-seeking behaviors.

Learning and retrieval behavior in recurrent neural networks with pre-synaptic dependent homeostatic plasticity

Science.gov (United States)

Mizusaki, Beatriz E. P.; Agnes, Everton J.; Erichsen, Rubem; Brunnet, Leonardo G.

2017-08-01

The plastic character of brain synapses is considered to be one of the foundations for the formation of memories. There are numerous kinds of such phenomenon currently described in the literature, but their role in the development of information pathways in neural networks with recurrent architectures is still not completely clear. In this paper we study the role of an activity-based process, called pre-synaptic dependent homeostatic scaling, in the organization of networks that yield precise-timed spiking patterns. It encodes spatio-temporal information in the synaptic weights as it associates a learned input with a specific response. We introduce a correlation measure to evaluate the precision of the spiking patterns and explore the effects of different inhibitory interactions and learning parameters. We find that large learning periods are important in order to improve the network learning capacity and discuss this ability in the presence of distinct inhibitory currents.

Toward an autonomous brain machine interface: integrating sensorimotor reward modulation and reinforcement learning.

Science.gov (United States)

Marsh, Brandi T; Tarigoppula, Venkata S Aditya; Chen, Chen; Francis, Joseph T

2015-05-13

For decades, neurophysiologists have worked on elucidating the function of the cortical sensorimotor control system from the standpoint of kinematics or dynamics. Recently, computational neuroscientists have developed models that can emulate changes seen in the primary motor cortex during learning. However, these simulations rely on the existence of a reward-like signal in the primary sensorimotor cortex. Reward modulation of the primary sensorimotor cortex has yet to be characterized at the level of neural units. Here we demonstrate that single units/multiunits and local field potentials in the primary motor (M1) cortex of nonhuman primates (Macaca radiata) are modulated by reward expectation during reaching movements and that this modulation is present even while subjects passively view cursor motions that are predictive of either reward or nonreward. After establishing this reward modulation, we set out to determine whether we could correctly classify rewarding versus nonrewarding trials, on a moment-to-moment basis. This reward information could then be used in collaboration with reinforcement learning principles toward an autonomous brain-machine interface. The autonomous brain-machine interface would use M1 for both decoding movement intention and extraction of reward expectation information as evaluative feedback, which would then update the decoding algorithm as necessary. In the work presented here, we show that this, in theory, is possible. Copyright © 2015 the authors 0270-6474/15/357374-14$15.00/0.

Dopaminergic modulation of reward-guided decision making in alcohol-preferring AA rats.

Science.gov (United States)

Oinio, Ville; Bäckström, Pia; Uhari-Väänänen, Johanna; Raasmaja, Atso; Piepponen, Petteri; Kiianmaa, Kalervo

2017-05-30

R**esults from animal gambling models have highlighted the importance of dopaminergic neurotransmission in modulating decision making when large sucrose rewards are combined with uncertainty. The majority of these models use food restriction as a tool to motivate animals to accomplish operant behavioral tasks, in which sucrose is used as a reward. As enhanced motivation to obtain sucrose due to hunger may impact its reward-seeking effect, we wanted to examine the decision-making behavior of rats in a situation where rats were fed ad libitum. For this purpose, we chose alcohol-preferring AA (alko alcohol) rats, as these rats have been shown to have high preference for sweet agents. In the present study, AA rats were trained to self-administer sucrose pellet rewards in a two-lever choice task (one pellet vs. three pellets). Once rational choice behavior had been established, the probability of gaining three pellets was decreased over time (50%, 33%, 25% then 20%). The effect of d-amphetamine on decision making was studied at every probability level, as well as the effect of the dopamine D 1 receptor agonist SKF-81297 and D 2 agonist quinpirole at probability levels of 100% and 25%. d-Amphetamine increased unprofitable choices in a dose-dependent manner at the two lowest probability levels. Quinpirole increased the frequency of unprofitable decisions at the 25% probability level, and SKF-82197 did not affect choice behavior. These results mirror the findings of probabilistic discounting studies using food-restricted rats. Based on this, the use of AA rats provides a new approach for studies on reward-guided decision making. Copyright © 2017 Elsevier B.V. All rights reserved.

Real-time cerebellar neuroprosthetic system based on a spiking neural network model of motor learning

Science.gov (United States)

Xu, Tao; Xiao, Na; Zhai, Xiaolong; Chan, Pak Kwan; Tin, Chung

2018-02-01

Objective. Damage to the brain, as a result of various medical conditions, impacts the everyday life of patients and there is still no complete cure to neurological disorders. Neuroprostheses that can functionally replace the damaged neural circuit have recently emerged as a possible solution to these problems. Here we describe the development of a real-time cerebellar neuroprosthetic system to substitute neural function in cerebellar circuitry for learning delay eyeblink conditioning (DEC). Approach. The system was empowered by a biologically realistic spiking neural network (SNN) model of the cerebellar neural circuit, which considers the neuronal population and anatomical connectivity of the network. The model simulated synaptic plasticity critical for learning DEC. This SNN model was carefully implemented on a field programmable gate array (FPGA) platform for real-time simulation. This hardware system was interfaced in in vivo experiments with anesthetized rats and it used neural spikes recorded online from the animal to learn and trigger conditioned eyeblink in the animal during training. Main results. This rat-FPGA hybrid system was able to process neuronal spikes in real-time with an embedded cerebellum model of ~10 000 neurons and reproduce learning of DEC with different inter-stimulus intervals. Our results validated that the system performance is physiologically relevant at both the neural (firing pattern) and behavioral (eyeblink pattern) levels. Significance. This integrated system provides the sufficient computation power for mimicking the cerebellar circuit in real-time. The system interacts with the biological system naturally at the spike level and can be generalized for including other neural components (neuron types and plasticity) and neural functions for potential neuroprosthetic applications.

Implementing Signature Neural Networks with Spiking Neurons.

Science.gov (United States)

Carrillo-Medina, José Luis; Latorre, Roberto

2016-01-01

of inhibitory connections. These parameters also modulate the memory capabilities of the network. The dynamical modes observed in the different informational dimensions in a given moment are independent and they only depend on the parameters shaping the information processing in this dimension. In view of these results, we argue that plasticity mechanisms inside individual cells and multicoding strategies can provide additional computational properties to spiking neural networks, which could enhance their capacity and performance in a wide variety of real-world tasks.

The dependence of neuronal encoding efficiency on Hebbian plasticity and homeostatic regulation of neurotransmitter release

Science.gov (United States)

Faghihi, Faramarz; Moustafa, Ahmed A.

2015-01-01

Synapses act as information filters by different molecular mechanisms including retrograde messenger that affect neuronal spiking activity. One of the well-known effects of retrograde messenger in presynaptic neurons is a change of the probability of neurotransmitter release. Hebbian learning describe a strengthening of a synapse between a presynaptic input onto a postsynaptic neuron when both pre- and postsynaptic neurons are coactive. In this work, a theory of homeostatic regulation of neurotransmitter release by retrograde messenger and Hebbian plasticity in neuronal encoding is presented. Encoding efficiency was measured for different synaptic conditions. In order to gain high encoding efficiency, the spiking pattern of a neuron should be dependent on the intensity of the input and show low levels of noise. In this work, we represent spiking trains as zeros and ones (corresponding to non-spike or spike in a time bin, respectively) as words with length equal to three. Then the frequency of each word (here eight words) is measured using spiking trains. These frequencies are used to measure neuronal efficiency in different conditions and for different parameter values. Results show that neurons that have synapses acting as band-pass filters show the highest efficiency to encode their input when both Hebbian mechanism and homeostatic regulation of neurotransmitter release exist in synapses. Specifically, the integration of homeostatic regulation of feedback inhibition with Hebbian mechanism and homeostatic regulation of neurotransmitter release in the synapses leads to even higher efficiency when high stimulus intensity is presented to the neurons. However, neurons with synapses acting as high-pass filters show no remarkable increase in encoding efficiency for all simulated synaptic plasticity mechanisms. This study demonstrates the importance of cooperation of Hebbian mechanism with regulation of neurotransmitter release induced by rapid diffused retrograde

Unsupervised discrimination of patterns in spiking neural networks with excitatory and inhibitory synaptic plasticity.

Science.gov (United States)

Srinivasa, Narayan; Cho, Youngkwan

2014-01-01

A spiking neural network model is described for learning to discriminate among spatial patterns in an unsupervised manner. The network anatomy consists of source neurons that are activated by external inputs, a reservoir that resembles a generic cortical layer with an excitatory-inhibitory (EI) network and a sink layer of neurons for readout. Synaptic plasticity in the form of STDP is imposed on all the excitatory and inhibitory synapses at all times. While long-term excitatory STDP enables sparse and efficient learning of the salient features in inputs, inhibitory STDP enables this learning to be stable by establishing a balance between excitatory and inhibitory currents at each neuron in the network. The synaptic weights between source and reservoir neurons form a basis set for the input patterns. The neural trajectories generated in the reservoir due to input stimulation and lateral connections between reservoir neurons can be readout by the sink layer neurons. This activity is used for adaptation of synapses between reservoir and sink layer neurons. A new measure called the discriminability index (DI) is introduced to compute if the network can discriminate between old patterns already presented in an initial training session. The DI is also used to compute if the network adapts to new patterns without losing its ability to discriminate among old patterns. The final outcome is that the network is able to correctly discriminate between all patterns-both old and new. This result holds as long as inhibitory synapses employ STDP to continuously enable current balance in the network. The results suggest a possible direction for future investigation into how spiking neural networks could address the stability-plasticity question despite having continuous synaptic plasticity.

How musical training affects cognitive development: rhythm, reward and other modulating variables.

Science.gov (United States)

Miendlarzewska, Ewa A; Trost, Wiebke J

2013-01-01

Musical training has recently gained additional interest in education as increasing neuroscientific research demonstrates its positive effects on brain development. Neuroimaging revealed plastic changes in the brains of adult musicians but it is still unclear to what extent they are the product of intensive music training rather than of other factors, such as preexisting biological markers of musicality. In this review, we synthesize a large body of studies demonstrating that benefits of musical training extend beyond the skills it directly aims to train and last well into adulthood. For example, children who undergo musical training have better verbal memory, second language pronunciation accuracy, reading ability and executive functions. Learning to play an instrument as a child may even predict academic performance and IQ in young adulthood. The degree of observed structural and functional adaptation in the brain correlates with intensity and duration of practice. Importantly, the effects on cognitive development depend on the timing of musical initiation due to sensitive periods during development, as well as on several other modulating variables. Notably, we point to motivation, reward and social context of musical education, which are important yet neglected factors affecting the long-term benefits of musical training. Further, we introduce the notion of rhythmic entrainment and suggest that it may represent a mechanism supporting learning and development of executive functions. It also hones temporal processing and orienting of attention in time that may underlie enhancements observed in reading and verbal memory. We conclude that musical training uniquely engenders near and far transfer effects, preparing a foundation for a range of skills, and thus fostering cognitive development.

How musical training affects cognitive development: rhythm, reward and other modulating variables.

Directory of Open Access Journals (Sweden)

Ewa Aurelia Miendlarzewska

2014-01-01

Full Text Available Musical training has recently gained additional interest in education as increasing neuroscientific research demonstrates its positive effects on brain development. Neuroimaging revealed plastic changes in the brains of adult musicians but it is still unclear to what extent they are the product of intensive music training rather than of other factors, such as preexisting biological markers of musicality. In this review, we synthesize a large body of studies demonstrating that benefits of musical training extend beyond the skills it directly aims to train and last well into adulthood. For example, children who undergo musical training have better verbal memory, second language pronunciation accuracy, reading ability and executive functions. Learning to play an instrument as a child may even predict academic performance and IQ in young adulthood. The degree of observed structural and functional adaptation in the brain correlates with intensity and duration of practice. Importantly, the effects on cognitive development depend on the timing of musical initiation due to sensitive periods during development, as well as on several other modulating variables. Notably, we point to motivation, reward and social context of musical education, which are important yet neglected factors affecting the long-term benefits of musical training. Further, we introduce the notion of rhythmic entrainment and suggest that it may represent a mechanism supporting learning and development of executive functions. It also hones temporal processing and orienting of attention in time that may underlie enhancements observed in reading and verbal memory. We conclude that musical training uniquely engenders near and far transfer effects, preparing a foundation for a range of skills, and thus fostering cognitive development.

Spike timing precision of neuronal circuits.

Science.gov (United States)

Kilinc, Deniz; Demir, Alper

2018-04-17

Spike timing is believed to be a key factor in sensory information encoding and computations performed by the neurons and neuronal circuits. However, the considerable noise and variability, arising from the inherently stochastic mechanisms that exist in the neurons and the synapses, degrade spike timing precision. Computational modeling can help decipher the mechanisms utilized by the neuronal circuits in order to regulate timing precision. In this paper, we utilize semi-analytical techniques, which were adapted from previously developed methods for electronic circuits, for the stochastic characterization of neuronal circuits. These techniques, which are orders of magnitude faster than traditional Monte Carlo type simulations, can be used to directly compute the spike timing jitter variance, power spectral densities, correlation functions, and other stochastic characterizations of neuronal circuit operation. We consider three distinct neuronal circuit motifs: Feedback inhibition, synaptic integration, and synaptic coupling. First, we show that both the spike timing precision and the energy efficiency of a spiking neuron are improved with feedback inhibition. We unveil the underlying mechanism through which this is achieved. Then, we demonstrate that a neuron can improve on the timing precision of its synaptic inputs, coming from multiple sources, via synaptic integration: The phase of the output spikes of the integrator neuron has the same variance as that of the sample average of the phases of its inputs. Finally, we reveal that weak synaptic coupling among neurons, in a fully connected network, enables them to behave like a single neuron with a larger membrane area, resulting in an improvement in the timing precision through cooperation.

Structural Plasticity Denoises Responses and Improves Learning Speed

Directory of Open Access Journals (Sweden)

Robin Spiess

2016-09-01

Full Text Available Despite an abundance of computational models for learning of synaptic weights, there has been relatively little research on structural plasticity, i.e. the creation and elimination of synapses. Especially, it is not clear how structural plasticity works in concert with spike-timing-dependent plasticity (STDP and what advantages their combination offers.Here we present a fairly large-scale functional model that uses leaky integrate-and-fire neurons, STDP, homeostasis, recurrent connections, and structural plasticity to learn the input encoding, the relation between inputs, and to infer missing inputs. Using this model, we compare the error and the amount of noise in the network's responses with and without structural plasticity and the influence of structural plasticity on the learning speed of the network.Using structural plasticity during learning shows good results for learning the representation of input values, i.e. structural plasticity strongly reduces the noise of the response by preventing spikes with a high error.For inferring missing inputs we see similar results, with responses having less noise if the network was trained using structural plasticity.Additionally, using structural plasticity with pruning significantly decreased the time to learn weights suitable for inference.Presumably, this is due to the clearer signal containing less spikes that misrepresent the desired value. Therefore, this work shows that structural plasticity is not only able to improve upon the performance using STDP without structural plasticity but also speeds up learning.Additionally, it addresses the practical problem of limited resources for connectivity that is not only apparent in the mammalian neocortex but also in computer hardware or neuromorphic (brain-inspired hardware by efficiently pruning synapses without losing performance.

Inferior Olive HCN1 Channels Coordinate Synaptic Integration and Complex Spike Timing

Directory of Open Access Journals (Sweden)

Derek L.F. Garden

2018-02-01

Full Text Available Cerebellar climbing-fiber-mediated complex spikes originate from neurons in the inferior olive (IO, are critical for motor coordination, and are central to theories of cerebellar learning. Hyperpolarization-activated cyclic-nucleotide-gated (HCN channels expressed by IO neurons have been considered as pacemaker currents important for oscillatory and resonant dynamics. Here, we demonstrate that in vitro, network actions of HCN1 channels enable bidirectional glutamatergic synaptic responses, while local actions of HCN1 channels determine the timing and waveform of synaptically driven action potentials. These roles are distinct from, and may complement, proposed pacemaker functions of HCN channels. We find that in behaving animals HCN1 channels reduce variability in the timing of cerebellar complex spikes, which serve as a readout of IO spiking. Our results suggest that spatially distributed actions of HCN1 channels enable the IO to implement network-wide rules for synaptic integration that modulate the timing of cerebellar climbing fiber signals.

Reward-based learning under hardware constraints - Using a RISC processor embedded in a neuromorphic substrate

Directory of Open Access Journals (Sweden)

Simon eFriedmann

2013-09-01

Full Text Available In this study, we propose and analyze in simulations a new, highly flexible method of imple-menting synaptic plasticity in a wafer-scale, accelerated neuromorphic hardware system. Thestudy focuses on globally modulated STDP, as a special use-case of this method. Flexibility isachieved by embedding a general-purpose processor dedicated to plasticity into the wafer. Toevaluate the suitability of the proposed system, we use a reward modulated STDP rule in a spiketrain learning task. A single layer of neurons is trained to fire at specific points in time withonly the reward as feedback. This model is simulated to measure its performance, i.e. the in-crease in received reward after learning. Using this performance as baseline, we then simulatethe model with various constraints imposed by the proposed implementation and compare theperformance. The simulated constraints include discretized synaptic weights, a restricted inter-face between analog synapses and embedded processor, and mismatch of analog circuits. Wefind that probabilistic updates can increase the performance of low-resolution weights, a simpleinterface between analog synapses and processor is sufficient for learning, and performance isinsensitive to mismatch. Further, we consider communication latency between wafer and theconventional control computer system that is simulating the environment. This latency increasesthe delay, with which the reward is sent to the embedded processor. Because of the time continu-ous operation of the analog synapses, delay can cause a deviation of the updates as compared tothe not delayed situation. We find that for highly accelerated systems latency has to be kept to aminimum. This study demonstrates the suitability of the proposed implementation to emulatethe selected reward modulated STDP learning rule. It is therefore an ideal candidate for imple-mentation in an upgraded version of the wafer-scale system developed within the BrainScaleSproject.

STDP and STDP Variations with Memristors for Spiking Neuromorphic Learning Systems

Directory of Open Access Journals (Sweden)

Teresa eSerrano-Gotarredona

2013-02-01

Full Text Available In this paper we review several ways of realizing asynchronous Spike-Timing Dependent Plasticity (STDP using memristors as synapses. Our focus is on how to use individual memristors to implement synaptic weight multiplications, in a way such that it is not necessary to (a introduce global synchronization and (b to separate memristor learning phases from memristor performing phases. In the approaches described, neurons fire spikes asynchronously when they wish and memristive synapses perform computation and learn at their own pace, as it happens in biological neural systems. We distinguish between two different memristor physics, depending on whether they respond to the original ``moving wall'' or to the ``filament creation and annihilation'' models. Independent of the memristor physics, we discuss two different types of STDP rules that can be implemented with memristors: either the pure timing-based rule that takes into account the arrival time of the spikes from the pre- and the post-synaptic neurons, or a hybrid rule that takes into account only the timing of pre-synaptic spikes and the membrane potential and other state variables of the post-synaptic neuron. We show how to implement these rules in cross-bar architectures that comprise massive arrays of memristors, and we discuss applications for artificial vision.

Risk of punishment influences discrete and coordinated encoding of reward-guided actions by prefrontal cortex and VTA neurons

Science.gov (United States)

Park, Junchol

2017-01-01

Actions motivated by rewards are often associated with risk of punishment. Little is known about the neural representation of punishment risk during reward-seeking behavior. We modeled this circumstance in rats by designing a task where actions were consistently rewarded but probabilistically punished. Spike activity and local field potentials were recorded during task performance simultaneously from VTA and mPFC, two reciprocally connected regions implicated in reward-seeking and aversive behaviors. At the single unit level, we found that ensembles of putative dopamine and non-dopamine VTA neurons and mPFC neurons encode the relationship between action and punishment. At the network level, we found that coherent theta oscillations synchronize VTA and mPFC in a bottom-up direction, effectively phase-modulating the neuronal spike activity in the two regions during punishment-free actions. This synchrony declined as a function of punishment probability, suggesting that during reward-seeking actions, risk of punishment diminishes VTA-driven neural synchrony between the two regions. PMID:29058673

Spike rate and spike timing contributions to coding taste quality information in rat periphery

Directory of Open Access Journals (Sweden)

Vernon eLawhern

2011-05-01

Full Text Available There is emerging evidence that individual sensory neurons in the rodent brain rely on temporal features of the discharge pattern to code differences in taste quality information. In contrast, in-vestigations of individual sensory neurons in the periphery have focused on analysis of spike rate and mostly disregarded spike timing as a taste quality coding mechanism. The purpose of this work was to determine the contribution of spike timing to taste quality coding by rat geniculate ganglion neurons using computational methods that have been applied successfully in other sys-tems. We recorded the discharge patterns of narrowly-tuned and broadly-tuned neurons in the rat geniculate ganglion to representatives of the five basic taste qualities. We used mutual in-formation to determine significant responses and the van Rossum metric to characterize their temporal features. While our findings show that spike timing contributes a significant part of the message, spike rate contributes the largest portion of the message relayed by afferent neurons from rat fungiform taste buds to the brain. Thus, spike rate and spike timing together are more effective than spike rate alone in coding stimulus quality information to a single basic taste in the periphery for both narrowly-tuned specialist and broadly-tuned generalist neurons.

Cannabinoid modulation of drug reward and the implications of marijuana legalization.

Science.gov (United States)

Covey, Dan P; Wenzel, Jennifer M; Cheer, Joseph F

2015-12-02

Marijuana is the most popular illegal drug worldwide. Recent trends indicate that this may soon change; not due to decreased marijuana use, but to an amendment in marijuana's illegal status. The cannabinoid type 1 (CB1) receptor mediates marijuana's psychoactive and reinforcing properties. CB1 receptors are also part of the brain endocannabinoid (eCB) system and support numerous forms of learning and memory, including the conditioned reinforcing properties of cues predicting reward or punishment. This is accomplished via eCB-dependent alterations in mesolimbic dopamine function, which plays an obligatory role in reward learning and motivation. Presynaptic CB1 receptors control midbrain dopamine neuron activity and thereby shape phasic dopamine release in target regions, particularly the nucleus accumbens (NAc). By also regulating synaptic input to the NAc, CB1 receptors modulate NAc output onto downstream neurons of the basal ganglia motor circuit, and thereby support goal-directed behaviors. Abused drugs promote short- and long-term adaptations in eCB-regulation of mesolimbic dopamine function, and thereby hijack neural systems related to the pursuit of rewards to promote drug abuse. By pharmacologically targeting the CB1 receptors, marijuana has preferential access to this neuronal system and can potently alter eCB-dependent processing of reward-related stimuli. As marijuana legalization progresses, greater access to this drug should increase the utility of marijuana as a research tool to better understand the eCB system, which has the potential to advance cannabinoid-based treatments for drug addiction. Copyright © 2014 Elsevier B.V. All rights reserved.

Neuromodulated Synaptic Plasticity on the SpiNNaker Neuromorphic System

Directory of Open Access Journals (Sweden)

Mantas Mikaitis

2018-02-01

Full Text Available SpiNNaker is a digital neuromorphic architecture, designed specifically for the low power simulation of large-scale spiking neural networks at speeds close to biological real-time. Unlike other neuromorphic systems, SpiNNaker allows users to develop their own neuron and synapse models as well as specify arbitrary connectivity. As a result SpiNNaker has proved to be a powerful tool for studying different neuron models as well as synaptic plasticity—believed to be one of the main mechanisms behind learning and memory in the brain. A number of Spike-Timing-Dependent-Plasticity(STDP rules have already been implemented on SpiNNaker and have been shown to be capable of solving various learning tasks in real-time. However, while STDP is an important biological theory of learning, it is a form of Hebbian or unsupervised learning and therefore does not explain behaviors that depend on feedback from the environment. Instead, learning rules based on neuromodulated STDP (three-factor learning rules have been shown to be capable of solving reinforcement learning tasks in a biologically plausible manner. In this paper we demonstrate for the first time how a model of three-factor STDP, with the third-factor representing spikes from dopaminergic neurons, can be implemented on the SpiNNaker neuromorphic system. Using this learning rule we first show how reward and punishment signals can be delivered to a single synapse before going on to demonstrate it in a larger network which solves the credit assignment problem in a Pavlovian conditioning experiment. Because of its extra complexity, we find that our three-factor learning rule requires approximately 2× as much processing time as the existing SpiNNaker STDP learning rules. However, we show that it is still possible to run our Pavlovian conditioning model with up to 1 × 104 neurons in real-time, opening up new research opportunities for modeling behavioral learning on SpiNNaker.

Analog memristive synapse in spiking networks implementing unsupervised learning

Directory of Open Access Journals (Sweden)

Erika Covi

2016-10-01

Full Text Available Emerging brain-inspired architectures call for devices that can emulate the functionality of biological synapses in order to implement new efficient computational schemes able to solve ill-posed problems. Various devices and solutions are still under investigation and, in this respect, a challenge is opened to the researchers in the field. Indeed, the optimal candidate is a device able to reproduce the complete functionality of a synapse, i.e. the typical synaptic process underlying learning in biological systems (activity-dependent synaptic plasticity. This implies a device able to change its resistance (synaptic strength, or weight upon proper electrical stimuli (synaptic activity and showing several stable resistive states throughout its dynamic range (analog behavior. Moreover, it should be able to perform spike timing dependent plasticity (STDP, an associative homosynaptic plasticity learning rule based on the delay time between the two firing neurons the synapse is connected to. This rule is a fundamental learning protocol in state-of-art networks, because it allows unsupervised learning. Notwithstanding this fact, STDP-based unsupervised learning has been proposed several times mainly for binary synapses rather than multilevel synapses composed of many binary memristors. This paper proposes an HfO2-based analog memristor as a synaptic element which performs STDP within a small spiking neuromorphic network operating unsupervised learning for character recognition. The trained network is able to recognize five characters even in case incomplete or noisy characters are displayed and it is robust to a device-to-device variability of up to +/-30%.

Analog Memristive Synapse in Spiking Networks Implementing Unsupervised Learning.

Science.gov (United States)

Covi, Erika; Brivio, Stefano; Serb, Alexander; Prodromakis, Themis; Fanciulli, Marco; Spiga, Sabina

2016-01-01

Emerging brain-inspired architectures call for devices that can emulate the functionality of biological synapses in order to implement new efficient computational schemes able to solve ill-posed problems. Various devices and solutions are still under investigation and, in this respect, a challenge is opened to the researchers in the field. Indeed, the optimal candidate is a device able to reproduce the complete functionality of a synapse, i.e., the typical synaptic process underlying learning in biological systems (activity-dependent synaptic plasticity). This implies a device able to change its resistance (synaptic strength, or weight) upon proper electrical stimuli (synaptic activity) and showing several stable resistive states throughout its dynamic range (analog behavior). Moreover, it should be able to perform spike timing dependent plasticity (STDP), an associative homosynaptic plasticity learning rule based on the delay time between the two firing neurons the synapse is connected to. This rule is a fundamental learning protocol in state-of-art networks, because it allows unsupervised learning. Notwithstanding this fact, STDP-based unsupervised learning has been proposed several times mainly for binary synapses rather than multilevel synapses composed of many binary memristors. This paper proposes an HfO 2 -based analog memristor as a synaptic element which performs STDP within a small spiking neuromorphic network operating unsupervised learning for character recognition. The trained network is able to recognize five characters even in case incomplete or noisy images are displayed and it is robust to a device-to-device variability of up to ±30%.

A Ca2+-based computational model for NDMA receptor-dependent synaptic plasticity at individual post-synaptic spines in the hippocampus

Directory of Open Access Journals (Sweden)

Owen Rackham

2010-07-01

Full Text Available Associative synaptic plasticity is synapse specific and requires coincident activity in presynaptic and postsynaptic neurons to activate NMDA receptors (NMDARs. The resultant Ca2+ influx is the critical trigger for the induction of synaptic plasticity. Given its centrality for the induction of synaptic plasticity, a model for NMDAR activation incorporating the timing of presynaptic glutamate release and postsynaptic depolarization by back-propagating action potentials could potentially predict the pre- and post-synaptic spike patterns required to induce synaptic plasticity. We have developed such a model by incorporating currently available data on the timecourse and amplitude of the postsynaptic membrane potential within individual spines. We couple this with data on the kinetics of synaptic NMDARs and then use the model to predict the continuous spine [Ca2+] in response to regular or irregular pre- and post-synaptic spike patterns. We then incorporate experimental data from synaptic plasticity induction protocols by regular activity patterns to couple the predicted local peak [Ca2+] to changes in synaptic strength. We find that our model accurately describes [Ca2+] in dendritic spines resulting from NMDAR activation during presynaptic and postsynaptic activity when compared to previous experimental observations. The model also replicates the experimentally determined plasticity outcome of regular and irregular spike patterns when applied to a single synapse. This model could therefore be used to predict the induction of synaptic plasticity under a variety of experimental conditions and spike patterns.

A Simple Network Architecture Accounts for Diverse Reward Time Responses in Primary Visual Cortex.

Science.gov (United States)

Huertas, Marco A; Hussain Shuler, Marshall G; Shouval, Harel Z

2015-09-16

Many actions performed by animals and humans depend on an ability to learn, estimate, and produce temporal intervals of behavioral relevance. Exemplifying such learning of cued expectancies is the observation of reward-timing activity in the primary visual cortex (V1) of rodents, wherein neural responses to visual cues come to predict the time of future reward as behaviorally experienced in the past. These reward-timing responses exhibit significant heterogeneity in at least three qualitatively distinct classes: sustained increase or sustained decrease in firing rate until the time of expected reward, and a class of cells that reach a peak in firing at the expected delay. We elaborate upon our existing model by including inhibitory and excitatory units while imposing simple connectivity rules to demonstrate what role these inhibitory elements and the simple architectures play in sculpting the response dynamics of the network. We find that simply adding inhibition is not sufficient for obtaining the different distinct response classes, and that a broad distribution of inhibitory projections is necessary for obtaining peak-type responses. Furthermore, although changes in connection strength that modulate the effects of inhibition onto excitatory units have a strong impact on the firing rate profile of these peaked responses, the network exhibits robustness in its overall ability to predict the expected time of reward. Finally, we demonstrate how the magnitude of expected reward can be encoded at the expected delay in the network and how peaked responses express this reward expectancy. Heterogeneity in single-neuron responses is a common feature of neuronal systems, although sometimes, in theoretical approaches, it is treated as a nuisance and seldom considered as conveying a different aspect of a signal. In this study, we focus on the heterogeneous responses in the primary visual cortex of rodents trained with a predictable delayed reward time. We describe under what

Galanin-Expressing GABA Neurons in the Lateral Hypothalamus Modulate Food Reward and Noncompulsive Locomotion

OpenAIRE

Qualls-Creekmore, Emily; Yu, Sangho; Francois, Marie; Hoang, John; Huesing, Clara; Bruce-Keller, Annadora; Burk, David; Berthoud, Hans-Rudolf; Morrison, Christopher D.; Münzberg, Heike

2017-01-01

The lateral hypothalamus (LHA) integrates reward and appetitive behavior and is composed of many overlapping neuronal populations. Recent studies associated LHA GABAergic neurons (LHAGABA), which densely innervate the ventral tegmental area (VTA), with modulation of food reward and consumption; yet, LHAGABA projections to the VTA exclusively modulated food consumption, not reward. We identified a subpopulation of LHAGABA neurons that coexpress the neuropeptide galanin (LHAGal). These LHAGal n...

A compound memristive synapse model for statistical learning through STDP in spiking neural networks

Directory of Open Access Journals (Sweden)

Johannes eBill

2014-12-01

Full Text Available Memristors have recently emerged as promising circuit elements to mimic the function of biological synapses in neuromorphic computing. The fabrication of reliable nanoscale memristive synapses, that feature continuous conductance changes based on the timing of pre- and postsynaptic spikes, has however turned out to be challenging. In this article, we propose an alternative approach, the compound memristive synapse, that circumvents this problem by the use of memristors with binary memristive states. A compound memristive synapse employs multiple bistable memristors in parallel to jointly form one synapse, thereby providing a spectrum of synaptic efficacies. We investigate the computational implications of synaptic plasticity in the compound synapse by integrating the recently observed phenomenon of stochastic filament formation into an abstract model of stochastic switching. Using this abstract model, we first show how standard pulsing schemes give rise to spike-timing dependent plasticity (STDP with a stabilizing weight dependence in compound synapses. In a next step, we study unsupervised learning with compound synapses in networks of spiking neurons organized in a winner-take-all architecture. Our theoretical analysis reveals that compound-synapse STDP implements generalized Expectation-Maximization in the spiking network. Specifically, the emergent synapse configuration represents the most salient features of the input distribution in a Mixture-of-Gaussians generative model. Furthermore, the network’s spike response to spiking input streams approximates a well-defined Bayesian posterior distribution. We show in computer simulations how such networks learn to represent high-dimensional distributions over images of handwritten digits with high fidelity even in presence of substantial device variations and under severe noise conditions. Therefore, the compound memristive synapse may provide a synaptic design principle for future neuromorphic

A compound memristive synapse model for statistical learning through STDP in spiking neural networks.

Science.gov (United States)

Bill, Johannes; Legenstein, Robert

2014-01-01

Memristors have recently emerged as promising circuit elements to mimic the function of biological synapses in neuromorphic computing. The fabrication of reliable nanoscale memristive synapses, that feature continuous conductance changes based on the timing of pre- and postsynaptic spikes, has however turned out to be challenging. In this article, we propose an alternative approach, the compound memristive synapse, that circumvents this problem by the use of memristors with binary memristive states. A compound memristive synapse employs multiple bistable memristors in parallel to jointly form one synapse, thereby providing a spectrum of synaptic efficacies. We investigate the computational implications of synaptic plasticity in the compound synapse by integrating the recently observed phenomenon of stochastic filament formation into an abstract model of stochastic switching. Using this abstract model, we first show how standard pulsing schemes give rise to spike-timing dependent plasticity (STDP) with a stabilizing weight dependence in compound synapses. In a next step, we study unsupervised learning with compound synapses in networks of spiking neurons organized in a winner-take-all architecture. Our theoretical analysis reveals that compound-synapse STDP implements generalized Expectation-Maximization in the spiking network. Specifically, the emergent synapse configuration represents the most salient features of the input distribution in a Mixture-of-Gaussians generative model. Furthermore, the network's spike response to spiking input streams approximates a well-defined Bayesian posterior distribution. We show in computer simulations how such networks learn to represent high-dimensional distributions over images of handwritten digits with high fidelity even in presence of substantial device variations and under severe noise conditions. Therefore, the compound memristive synapse may provide a synaptic design principle for future neuromorphic architectures.

Cigarette craving is associated with blunted reward processing in nicotine-dependent smokers.

Science.gov (United States)

Peechatka, Alyssa L; Whitton, Alexis E; Farmer, Stacey L; Pizzagalli, Diego A; Janes, Amy C

2015-10-01

Dysfunctional reward processing leading to the undervaluation of non-drug rewards is hypothesized to play a crucial role in nicotine dependence. However, it is unclear if blunted reward responsivity and the desire to use nicotine are directly linked after a brief period of abstinence. Such an association would suggest that individuals with reduced reward responsivity may be at increased risk to experience nicotine craving. Reward function was evaluated with a probabilistic reward task (PRT), which measures reward responsivity to monetary incentives. To identify whether smoking status influenced reward function, PRT performance was compared between non-depressed, nicotine-dependent smokers and non-smokers. Within smokers, correlations were conducted to determine if blunted reward responsivity on the PRT was associated with increased nicotine craving. Time since last nicotine exposure was standardized to 4h for all smokers. Smokers and non-smokers did not differ in reward responsivity on the PRT. However, within smokers, a significant negative correlation was found between reward responsivity and intensity of nicotine craving. The current findings show that, among smokers, the intensity of nicotine craving is linked to lower sensitivity to non-drug rewards. This finding is in line with prior theories that suggest reward dysfunction in some clinical populations (e.g., depressive disorders, schizophrenia) may facilitate nicotine use. The current study expands on such theories by indicating that sub-clinical variations in reward function are related to motivation for nicotine use. Identifying smokers who show blunted sensitivity to non-drug rewards may help guide treatments aimed at mitigating the motivation to smoke. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

Probability differently modulating the effects of reward and punishment on visuomotor adaptation.

Science.gov (United States)

Song, Yanlong; Smiley-Oyen, Ann L

2017-12-01

Recent human motor learning studies revealed that punishment seemingly accelerated motor learning but reward enhanced consolidation of motor memory. It is not evident how intrinsic properties of reward and punishment modulate the potentially dissociable effects of reward and punishment on motor learning and motor memory. It is also not clear what causes the dissociation of the effects of reward and punishment. By manipulating probability of distribution, a critical property of reward and punishment, the present study demonstrated that probability had distinct modulation on the effects of reward and punishment in adapting to a sudden visual rotation and consolidation of the adaptation memory. Specifically, two probabilities of monetary reward and punishment distribution, 50 and 100%, were applied during young adult participants adapting to a sudden visual rotation. Punishment and reward showed distinct effects on motor adaptation and motor memory. The group that received punishments in 100% of the adaptation trials adapted significantly faster than the other three groups, but the group that received rewards in 100% of the adaptation trials showed marked savings in re-adapting to the same rotation. In addition, the group that received punishments in 50% of the adaptation trials that were randomly selected also had savings in re-adapting to the same rotation. Sensitivity to sensory prediction error or difference in explicit process induced by reward and punishment may likely contribute to the distinct effects of reward and punishment.

Natural Rewards, Neuroplasticity, and Non-Drug Addictions

Science.gov (United States)

Olsen, Christopher M.

2011-01-01

There is a high degree of overlap between brain regions involved in processing natural rewards and drugs of abuse. “Non-drug” or “behavioral” addictions have become increasingly documented in the clinic, and pathologies include compulsive activities such as shopping, eating, exercising, sexual behavior, and gambling. Like drug addiction, non-drug addictions manifest in symptoms including craving, impaired control over the behavior, tolerance, withdrawal, and high rates of relapse. These alterations in behavior suggest that plasticity may be occurring in brain regions associated with drug addiction. In this review, I summarize data demonstrating that exposure to non-drug rewards can alter neural plasticity in regions of the brain that are affected by drugs of abuse. Research suggests that there are several similarities between neuroplasticity induced by natural and drug rewards and that, depending on the reward, repeated exposure to natural rewards might induce neuroplasticity that either promotes or counteracts addictive behavior. PMID:21459101

Dose Dependent Dopaminergic Modulation of Reward-Based Learning in Parkinson's Disease

Science.gov (United States)

van Wouwe, N. C.; Ridderinkhof, K. R.; Band, G. P. H.; van den Wildenberg, W. P. M.; Wylie, S. A.

2012-01-01

Learning to select optimal behavior in new and uncertain situations is a crucial aspect of living and requires the ability to quickly associate stimuli with actions that lead to rewarding outcomes. Mathematical models of reinforcement-based learning to select rewarding actions distinguish between (1) the formation of stimulus-action-reward…

Discrete-time rewards model-checked

NARCIS (Netherlands)

Larsen, K.G.; Andova, S.; Niebert, Peter; Hermanns, H.; Katoen, Joost P.

2003-01-01

This paper presents a model-checking approach for analyzing discrete-time Markov reward models. For this purpose, the temporal logic probabilistic CTL is extended with reward constraints. This allows to formulate complex measures – involving expected as well as accumulated rewards – in a precise and

Neuromodulation, development and synaptic plasticity.

Science.gov (United States)

Foehring, R C; Lorenzon, N M

1999-03-01

We discuss parallels in the mechanisms underlying use-dependent synaptic plasticity during development and long-term potentiation (LTP) and long-term depression (LTD) in neocortical synapses. Neuromodulators, such as norepinephrine, serotonin, and acetylcholine have also been implicated in regulating both developmental plasticity and LTP/LTD. There are many potential levels of interaction between neuromodulators and plasticity. Ion channels are substrates for modulation in many cell types. We discuss examples of modulation of voltage-gated Ca2+ channels and Ca(2+)-dependent K+ channels and the consequences for neocortical pyramidal cell firing behaviour. At the time when developmental plasticity is most evident in rat cortex, the substrate for modulation is changing as the densities and relative proportions of various ion channels types are altered during ontogeny. We discuss examples of changes in K+ and Ca2+ channels and the consequence for modulation of neuronal activity.

Announced reward counteracts the effects of chronic social stress on anticipatory behavior and hippocampal synaptic plasticity in rats.

Science.gov (United States)

Kamal, Amer; Van der Harst, Johanneke E; Kapteijn, Chantal M; Baars, Annemarie J M; Spruijt, Berry M; Ramakers, Geert M J

2010-04-01

Chronic stress causes insensitivity to rewards (anhedonia) in rats, reflected by the absence of anticipatory behavior for a sucrose-reward, which can be reversed by antidepressant treatment or repeated announced transfer to an enriched cage. It was, however, not clear whether the highly rewarding properties of the enriched cage alone caused this reversal or whether the anticipation of this reward as such had an additional effect. Therefore, the present study compared the consequences of the announcement of a reward to the mere effect of a reward alone with respect to their efficacy to counteract the consequences of chronic stress. Two forms of synaptic plasticity, long-term potentiation and long-term depression were investigated in area CA1 of the hippocampus. This was done in socially stressed rats (induced by defeat and subsequent long-term individual housing), socially stressed rats that received a reward (short-term enriched housing) and socially stressed rats to which this reward was announced by means of a stimulus that was repeatedly paired to the reward. The results were compared to corresponding control rats. We show that announcement of enriched housing appeared to have had an additional effect compared to the enriched housing per se as indicated by a significant higher amount of LTP. In conclusion, announced short-term enriched housing has a high and long-lasting counteracting efficacy on stress-induced alterations of hippocampal synaptic plasticity. This information is important for counteracting the consequences of chronic stress in both human and captive rats.

Dual roles for spike signaling in cortical neural populations

Directory of Open Access Journals (Sweden)

Dana eBallard

2011-06-01

Full Text Available A prominent feature of signaling in cortical neurons is that of randomness in the action potential. The output of a typical pyramidal cell can be well fit with a Poisson model, and variations in the Poisson rate repeatedly have been shown to be correlated with stimuli. However while the rate provides a very useful characterization of neural spike data, it may not be the most fundamental description of the signaling code. Recent data showing γ frequency range multi-cell action potential correlations, together with spike timing dependent plasticity, are spurring a re-examination of the classical model, since precise timing codes imply that the generation of spikes is essentially deterministic. Could the observed Poisson randomness and timing determinism reflect two separate modes of communication, or do they somehow derive from a single process? We investigate in a timing-based model whether the apparent incompatibility between these probabilistic and deterministic observations may be resolved by examining how spikes could be used in the underlying neural circuits. The crucial component of this model draws on dual roles for spike signaling. In learning receptive fields from ensembles of inputs, spikes need to behave probabilistically, whereas for fast signaling of individual stimuli, the spikes need to behave deterministically. Our simulations show that this combination is possible if deterministic signals using γ latency coding are probabilistically routed through different members of a cortical cell population at different times. This model exhibits standard features characteristic of Poisson models such as orientation tuning post-stimulus histograms and exponential interval histograms. In addition it makes testable predictions that follow from the γ latency coding.

Enhanced polychronisation in a spiking network with metaplasticity

Directory of Open Access Journals (Sweden)

Mira eGuise

2015-02-01

Full Text Available Computational models of metaplasticity have usually focused on the modeling of single synapses (Shouval et al., 2002. In this paper we study the effect of metaplasticity on network behavior. Our guiding assumption is that the primary purpose of metaplasticity is to regulate synaptic plasticity, by increasing it when input is low and decreasing it when input is high. For our experiments we adopt a model of metaplasticity that demonstrably has this effect for a single synapse; our primary interest is in how metaplasticity thus defined affects network-level phenomena. We focus on a network-level phenomenon called polychronicity, that has a potential role in representation and memory. A network with polychronicity has the ability to produce non-synchronous but precisely timed sequences of neural firing events that can arise from strongly connected groups of neurons called polychronous neural groups (Izhikevich et al., 2004; Izhikevich, 2006a. Polychronous groups (PNGs develop readily when spiking networks are exposed to repeated spatio-temporal stimuli under the influence of spike-timing-dependent plasticity (STDP, but are sensitive to changes in synaptic weight distribution. We use a technique we have recently developed called Response Fingerprinting to show that PNGs formed in the presence of metaplasticity are significantly larger than those with no metaplasticity. A potential mechanism for this enhancement is proposed that links an inherent property of integrator type neurons called spike latency to an increase in the tolerance of PNG neurons to jitter in their inputs.

Molecular and Neuronal Plasticity Mechanisms in the Amygdala-Prefrontal Cortical Circuit: Implications for Opiate Addiction Memory Formation

Directory of Open Access Journals (Sweden)

Laura G Rosen

2015-11-01

Full Text Available The persistence of associative memories linked to the rewarding properties of drugs of abuse is a core underlying feature of the addiction process. Opiate class drugs in particular, possess potent euphorigenic effects which, when linked to environmental cues, can produce drug-related ‘trigger’ memories that may persist for lengthy periods of time, even during abstinence, in both humans and other animals. Furthermore, the transitional switch from the drug-naïve, non-dependent state to states of dependence and withdrawal, represents a critical boundary between distinct neuronal and molecular substrates associated with opiate-reward memory formation. Identifying the functional molecular and neuronal mechanisms related to the acquisition, consolidation, recall and extinction phases of opiate-related reward memories is critical for understanding, and potentially reversing, addiction-related memory plasticity characteristic of compulsive drug-seeking behaviors. The mammalian prefrontal cortex (PFC and basolateral nucleus of the amygdala (BLA share important functional and anatomical connections that are involved importantly in the processing of associative memories linked to drug reward. In addition, both regions share interconnections with the mesolimbic pathway’s ventral tegmental area (VTA and nucleus accumbens (NAc and can modulate dopamine (DA transmission and neuronal activity associated with drug-related DAergic signaling dynamics. In this review, we will summarize research from both human and animal modelling studies highlighting the importance of neuronal and molecular plasticity mechanisms within this circuitry during critical phases of opiate addiction-related learning and memory processing. Specifically, we will focus on two molecular signaling pathways known to be involved in both drug-related neuroadaptations and in memory-related plasticity mechanisms; the extracellular-signal-regulated kinase system (ERK and the Ca2+/calmodulin-dependent

The impact of reward and punishment on skill learning depends on task demands.

Science.gov (United States)

Steel, Adam; Silson, Edward H; Stagg, Charlotte J; Baker, Chris I

2016-10-27

Reward and punishment motivate behavior, but it is unclear exactly how they impact skill performance and whether the effect varies across skills. The present study investigated the effect of reward and punishment in both a sequencing skill and a motor skill context. Participants trained on either a sequencing skill (serial reaction time task) or a motor skill (force-tracking task). Skill knowledge was tested immediately after training, and again 1 hour, 24-48 hours, and 30 days after training. We found a dissociation of the effects of reward and punishment on the tasks, primarily reflecting the impact of punishment. While punishment improved serial reaction time task performance, it impaired force-tracking task performance. In contrast to prior literature, neither reward nor punishment benefitted memory retention, arguing against the common assumption that reward ubiquitously benefits skill retention. Collectively, these results suggest that punishment impacts skilled behavior more than reward in a complex, task dependent fashion.

Pramipexole modulates the neural network of reward anticipation.

Science.gov (United States)

Ye, Zheng; Hammer, Anke; Camara, Estela; Münte, Thomas F

2011-05-01

Pramipexole is widely prescribed to treat Parkinson's disease. It has been found to cause impulse control disorders such as pathological gambling. To examine how pramipexole modulates the network of reward anticipation, we carried out a pharmacological functional magnetic resonance imaging study with a double-blind, within-subject design. During the anticipation of monetary rewards, pramipexole increased the activity of the nucleus accumbens (NAcc), enhanced the interaction between the NAcc and the anterior insula, but weakened the interaction between the NAcc and the prefrontal cortex. These results suggest that pramipexole may exaggerate incentive and affective responses to possible rewards, but reduce the top-down control of impulses, leading to an increase in impulsive behaviors. This imbalance between the prefrontal-striatum connectivity and the insula-striatum connectivity may represent the neural mechanism of pathological gambling caused by pramipexole. Copyright © 2010 Wiley-Liss, Inc.

Self-control with spiking and non-spiking neural networks playing games.

Science.gov (United States)

Christodoulou, Chris; Banfield, Gaye; Cleanthous, Aristodemos

2010-01-01

Self-control can be defined as choosing a large delayed reward over a small immediate reward, while precommitment is the making of a choice with the specific aim of denying oneself future choices. Humans recognise that they have self-control problems and attempt to overcome them by applying precommitment. Problems in exercising self-control, suggest a conflict between cognition and motivation, which has been linked to competition between higher and lower brain functions (representing the frontal lobes and the limbic system respectively). This premise of an internal process conflict, lead to a behavioural model being proposed, based on which, we implemented a computational model for studying and explaining self-control through precommitment behaviour. Our model consists of two neural networks, initially non-spiking and then spiking ones, representing the higher and lower brain systems viewed as cooperating for the benefit of the organism. The non-spiking neural networks are of simple feed forward multilayer type with reinforcement learning, one with selective bootstrap weight update rule, which is seen as myopic, representing the lower brain and the other with the temporal difference weight update rule, which is seen as far-sighted, representing the higher brain. The spiking neural networks are implemented with leaky integrate-and-fire neurons with learning based on stochastic synaptic transmission. The differentiating element between the two brain centres in this implementation is based on the memory of past actions determined by an eligibility trace time constant. As the structure of the self-control problem can be likened to the Iterated Prisoner's Dilemma (IPD) game in that cooperation is to defection what self-control is to impulsiveness or what compromising is to insisting, we implemented the neural networks as two players, learning simultaneously but independently, competing in the IPD game. With a technique resembling the precommitment effect, whereby the

Reward-enhanced memory in younger and older adults.

Science.gov (United States)

Spaniol, Julia; Schain, Cécile; Bowen, Holly J

2014-09-01

We investigated how the anticipation of remote monetary reward modulates intentional episodic memory formation in younger and older adults. On the basis of prior findings of preserved reward-cognition interactions in aging, we predicted that reward anticipation would be associated with enhanced memory in both younger and older adults. On the basis of previous demonstrations of a time-dependent effect of reward anticipation on memory, we expected the memory enhancement to increase with study-test delay. In Experiment 1, younger and older participants encoded a series of picture stimuli associated with high- or low-reward values. At test (24-hr postencoding), recognition hits resulted in either high or low monetary rewards, whereas false alarms were penalized to discourage guessing. Experiment 2 was similar to Experiment 1, but the study-test delay was manipulated within subjects (immediate vs 24hr). In Experiment 1, younger and older adults showed enhanced recognition for high-reward pictures compared with low-reward pictures. Experiment 2 replicated this finding and additionally showed that the effect did not extend to immediate recognition. The current findings provide support for a time-dependent mechanism of reward-based memory enhancement. They also suggest that aging leaves intact the positive influence of reward anticipation on intentional long-term memory formation. © The Author 2013. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Spine formation pattern of adult-born neurons is differentially modulated by the induction timing and location of hippocampal plasticity.

Directory of Open Access Journals (Sweden)

Noriaki Ohkawa

Full Text Available In the adult hippocampus dentate gyrus (DG, newly born neurons are functionally integrated into existing circuits and play important roles in hippocampus-dependent memory. However, it remains unclear how neural plasticity regulates the integration pattern of new neurons into preexisting circuits. Because dendritic spines are major postsynaptic sites for excitatory inputs, spines of new neurons were visualized by retrovirus-mediated labeling to evaluate integration. Long-term potentiation (LTP was induced at 12, 16, or 21 days postinfection (dpi, at which time new neurons have no, few, or many spines, respectively. The spine expression patterns were investigated at one or two weeks after LTP induction. Induction at 12 dpi increased later spinogenesis, although the new neurons at 12 dpi didn't respond to the stimulus for LTP induction. Induction at 21 dpi transiently mediated spine enlargement. Surprisingly, LTP induction at 16 dpi reduced the spine density of new neurons. All LTP-mediated changes specifically appeared within the LTP-induced layer. Therefore, neural plasticity differentially regulates the integration of new neurons into the activated circuit, dependent on their developmental stage. Consequently, new neurons at different developmental stages may play distinct roles in processing the acquired information by modulating the connectivity of activated circuits via their integration.

Acute D3 Antagonist GSK598809 Selectively Enhances Neural Response During Monetary Reward Anticipation in Drug and Alcohol Dependence

Science.gov (United States)

Murphy, Anna; Nestor, Liam J; McGonigle, John; Paterson, Louise; Boyapati, Venkataramana; Ersche, Karen D; Flechais, Remy; Kuchibatla, Shankar; Metastasio, Antonio; Orban, Csaba; Passetti, Filippo; Reed, Laurence; Smith, Dana; Suckling, John; Taylor, Eleanor; Robbins, Trevor W; Lingford-Hughes, Anne; Nutt, David J; Deakin, John FW; Elliott, Rebecca

2017-01-01

Evidence suggests that disturbances in neurobiological mechanisms of reward and inhibitory control maintain addiction and provoke relapse during abstinence. Abnormalities within the dopamine system may contribute to these disturbances and pharmacologically targeting the D3 dopamine receptor (DRD3) is therefore of significant clinical interest. We used functional magnetic resonance imaging to investigate the acute effects of the DRD3 antagonist GSK598809 on anticipatory reward processing, using the monetary incentive delay task (MIDT), and response inhibition using the Go/No-Go task (GNGT). A double-blind, placebo-controlled, crossover design approach was used in abstinent alcohol dependent, abstinent poly-drug dependent and healthy control volunteers. For the MIDT, there was evidence of blunted ventral striatal response to reward in the poly-drug-dependent group under placebo. GSK598809 normalized ventral striatal reward response and enhanced response in the DRD3-rich regions of the ventral pallidum and substantia nigra. Exploratory investigations suggested that the effects of GSK598809 were mainly driven by those with primary dependence on alcohol but not on opiates. Taken together, these findings suggest that GSK598809 may remediate reward deficits in substance dependence. For the GNGT, enhanced response in the inferior frontal cortex of the poly-drug group was found. However, there were no effects of GSK598809 on the neural network underlying response inhibition nor were there any behavioral drug effects on response inhibition. GSK598809 modulated the neural network underlying reward anticipation but not response inhibition, suggesting that DRD3 antagonists may restore reward deficits in addiction. PMID:28042871

Operant Conditioning: A Minimal Components Requirement in Artificial Spiking Neurons Designed for Bio-Inspired Robot’s Controller

Directory of Open Access Journals (Sweden)

André eCyr

2014-07-01

Full Text Available We demonstrate the operant conditioning (OC learning process within a basic bio-inspired robot controller paradigm, using an artificial spiking neural network (ASNN with minimal component count as artificial brain. In biological agents, OC results in behavioral changes that are learned from the consequences of previous actions, using progressive prediction adjustment triggered by reinforcers. In a robotics context, virtual and physical robots may benefit from a similar learning skill when facing unknown environments with no supervision. In this work, we demonstrate that a simple ASNN can efficiently realise many OC scenarios. The elementary learning kernel that we describe relies on a few critical neurons, synaptic links and the integration of habituation and spike-timing dependent plasticity (STDP as learning rules. Using four tasks of incremental complexity, our experimental results show that such minimal neural component set may be sufficient to implement many OC procedures. Hence, with the described bio-inspired module, OC can be implemented in a wide range of robot controllers, including those with limited computational resources.

Spiking irregularity and frequency modulate the behavioral report of single-neuron stimulation.

Science.gov (United States)

Doron, Guy; von Heimendahl, Moritz; Schlattmann, Peter; Houweling, Arthur R; Brecht, Michael

2014-02-05

The action potential activity of single cortical neurons can evoke measurable sensory effects, but it is not known how spiking parameters and neuronal subtypes affect the evoked sensations. Here, we examined the effects of spike train irregularity, spike frequency, and spike number on the detectability of single-neuron stimulation in rat somatosensory cortex. For regular-spiking, putative excitatory neurons, detectability increased with spike train irregularity and decreasing spike frequencies but was not affected by spike number. Stimulation of single, fast-spiking, putative inhibitory neurons led to a larger sensory effect compared to regular-spiking neurons, and the effect size depended only on spike irregularity. An ideal-observer analysis suggests that, under our experimental conditions, rats were using integration windows of a few hundred milliseconds or more. Our data imply that the behaving animal is sensitive to single neurons' spikes and even to their temporal patterning. Copyright © 2014 Elsevier Inc. All rights reserved.

Modulation of risk/reward decision making by dopaminergic transmission within the basolateral amygdala.

Science.gov (United States)

Larkin, Joshua D; Jenni, Nicole L; Floresco, Stan B

2016-01-01

Dopamine (DA) transmission within cortico-limbic-striatal circuitry is integral in modulating decisions involving reward uncertainty. The basolateral amygdala (BLA) also plays a role in these processes, yet how DA transmission within this nucleus regulates cost/benefit decision making is unknown. We investigated the contribution of DA transmission within the BLA to risk/reward decision making assessed with a probabilistic discounting task. Rats were well-trained to choose between a small/certain reward and a large/risky reward, with the probability of obtaining the larger reward decreasing (100-12.5 %) or increasing (12.5-100 %) over a session. We examined the effects of antagonizing BLA D1 (SCH 23390, 0.1-1 μg) or D2 (eticlopride, 0.1-1 μg) receptors, as well as intra-BLA infusions of agonists for D1 (SKF 81297, 0.1-1 μg) and D2 (quinpirole, 1-10 μg) receptors. We also assessed how DA receptor stimulation may induce differential effects related to baseline levels of risky choice. BLA D1 receptor antagonism reduced risky choice by decreasing reward sensitivity, whereas D2 antagonism did not affect overall choice patterns. Stimulation of BLA D1 receptors optimized decision making in a baseline-dependent manner: in risk-averse rats, infusions of a lower dose of SKF81297 increased risky choice when reward probabilities were high (50 %), whereas in risk-prone rats, this drug reduced risky choice when probabilities were low (12.5 %). Quinpirole reduced risky choice in risk-prone rats, enhancing lose-shift behavior. These data highlight previously uncharacterized roles for BLA DA D1 and D2 receptors in biasing choice during risk/reward decision making through mediation of reward/negative feedback sensitivity.

Linear stability analysis of retrieval state in associative memory neural networks of spiking neurons

International Nuclear Information System (INIS)

Yoshioka, Masahiko

2002-01-01

We study associative memory neural networks of the Hodgkin-Huxley type of spiking neurons in which multiple periodic spatiotemporal patterns of spike timing are memorized as limit-cycle-type attractors. In encoding the spatiotemporal patterns, we assume the spike-timing-dependent synaptic plasticity with the asymmetric time window. Analysis for periodic solution of retrieval state reveals that if the area of the negative part of the time window is equivalent to the positive part, then crosstalk among encoded patterns vanishes. Phase transition due to the loss of the stability of periodic solution is observed when we assume fast α function for direct interaction among neurons. In order to evaluate the critical point of this phase transition, we employ Floquet theory in which the stability problem of the infinite number of spiking neurons interacting with α function is reduced to the eigenvalue problem with the finite size of matrix. Numerical integration of the single-body dynamics yields the explicit value of the matrix, which enables us to determine the critical point of the phase transition with a high degree of precision

Deficient neural activity subserving decision-making during reward waiting time in intertemporal choice in adult attention-deficit hyperactivity disorder.

Science.gov (United States)

Todokoro, Ayako; Tanaka, Saori C; Kawakubo, Yuki; Yahata, Noriaki; Ishii-Takahashi, Ayaka; Nishimura, Yukika; Kano, Yukiko; Ohtake, Fumio; Kasai, Kiyoto

2018-04-24

Impulsivity, which significantly affects social adaptation, is an important target behavioral characteristic in interventions for attention-deficit hyperactivity disorder (ADHD). Typically, people are willing to wait longer to acquire greater rewards. Impulsivity in ADHD may be associated with brain dysfunction in decision-making involving waiting behavior under such situations. We tested the hypothesis that brain circuitry during a period of waiting (i.e., prior to the acquisition of reward) is altered in adults with ADHD. The participants included 14 medication-free adults with ADHD and 16 healthy controls matched for age, sex, IQ, and handedness. The behavioral task had participants choose between a delayed, larger monetary reward and an immediate, smaller monetary reward, where the reward waiting time actually occurred during functional magnetic resonance imaging measurement. We tested for group differences in the contrast values of blood-oxygen-level dependent signals associated with the length of waiting time, calculated using the parametric modulation method. While the two groups did not differ in the time discounting rate, the delay-sensitive contrast values were significantly lower in the caudate and visual cortex in individuals with ADHD. The higher impulsivity scores were significantly associated with lower delay-sensitive contrast values in the caudate and visual cortex. These results suggest that deficient neural activity affects decision-making involving reward waiting time during intertemporal choice tasks, and provide an explanation for the basis of impulsivity in adult ADHD. © 2018 The Author. Psychiatry and Clinical Neurosciences © 2018 Japanese Society of Psychiatry and Neurology.

Expected reward modulates encoding-related theta activity before an event.

Science.gov (United States)

Gruber, Matthias J; Watrous, Andrew J; Ekstrom, Arne D; Ranganath, Charan; Otten, Leun J

2013-01-01

Oscillatory brain activity in the theta frequency range (4-8 Hz) before the onset of an event has been shown to affect the likelihood of successfully encoding the event into memory. Recent work has also indicated that frontal theta activity might be modulated by reward, but it is not clear how reward expectancy, anticipatory theta activity, and memory formation might be related. Here, we used scalp electroencephalography (EEG) to assess the relationship between these factors. EEG was recorded from healthy adults while they memorized a series of words. Each word was preceded by a cue that indicated whether a high or low monetary reward would be earned if the word was successfully remembered in a later recognition test. Frontal theta power between the presentation of the reward cue and the onset of a word was predictive of later memory for the word, but only in the high reward condition. No theta differences were observed before word onset following low reward cues. The magnitude of prestimulus encoding-related theta activity in the high reward condition was correlated with the number of high reward words that were later confidently recognized. These findings provide strong evidence for a link between reward expectancy, theta activity, and memory encoding. Theta activity before event onset seems to be especially important for the encoding of motivationally significant stimuli. One possibility is that dopaminergic activity during reward anticipation mediates frontal theta activity related to memory. Copyright © 2012 Elsevier Inc. All rights reserved.

Modulating Hippocampal Plasticity with In Vivo Brain Stimulation

Science.gov (United States)

2016-11-17

wires were left unhooked from stimulation device. Following stimulation , the animals were returned to their homecage until time of euthanasia and...current stimulation (tDCS) to enhance cognitive training: effect of timing of stimulation . Exp Brain Res 232:3345-3351. 15 DISTRIBUTION...AFRL-RH-WP-TR-2016-0082 MODULATING HIPPOCAMPAL PLASTICITY WITH IN-VIVO BRAIN STIMULATION Joyce G. Rohan Oakridge Institute

Unified pre- and postsynaptic long-term plasticity enables reliable and flexible learning.

Science.gov (United States)

Costa, Rui Ponte; Froemke, Robert C; Sjöström, P Jesper; van Rossum, Mark Cw

2015-08-26

Although it is well known that long-term synaptic plasticity can be expressed both pre- and postsynaptically, the functional consequences of this arrangement have remained elusive. We show that spike-timing-dependent plasticity with both pre- and postsynaptic expression develops receptive fields with reduced variability and improved discriminability compared to postsynaptic plasticity alone. These long-term modifications in receptive field statistics match recent sensory perception experiments. Moreover, learning with this form of plasticity leaves a hidden postsynaptic memory trace that enables fast relearning of previously stored information, providing a cellular substrate for memory savings. Our results reveal essential roles for presynaptic plasticity that are missed when only postsynaptic expression of long-term plasticity is considered, and suggest an experience-dependent distribution of pre- and postsynaptic strength changes.

Time dependence of resonance γ-radiation modulated by acoustic excitations

International Nuclear Information System (INIS)

Mkrtchyan, A.R.; Arakelyan, A.R.; Gabrielyan, R.G.; Kocharyan, L.A.; Grigoryan, G.R.; Slavinskii, M.M.

1984-01-01

Experimental investigations of the time dependence of the γ-resonance absorption line intensity in case of modulation by acoustic waves are presented. 57 Co was used as source and a stainless steel foil was chosen as an absorber. The time dependences of the counting rate of the resonant γ-quanta corresponding to excitations with 3400 Hz and with 1.5 or 7 V at the vibrosystem transducer are plotted. The measurements show that the method has principal advantages over the conventional Moessbauer spectroscopy

Bayesian population decoding of spiking neurons.

Science.gov (United States)

Gerwinn, Sebastian; Macke, Jakob; Bethge, Matthias

2009-01-01

The timing of action potentials in spiking neurons depends on the temporal dynamics of their inputs and contains information about temporal fluctuations in the stimulus. Leaky integrate-and-fire neurons constitute a popular class of encoding models, in which spike times depend directly on the temporal structure of the inputs. However, optimal decoding rules for these models have only been studied explicitly in the noiseless case. Here, we study decoding rules for probabilistic inference of a continuous stimulus from the spike times of a population of leaky integrate-and-fire neurons with threshold noise. We derive three algorithms for approximating the posterior distribution over stimuli as a function of the observed spike trains. In addition to a reconstruction of the stimulus we thus obtain an estimate of the uncertainty as well. Furthermore, we derive a 'spike-by-spike' online decoding scheme that recursively updates the posterior with the arrival of each new spike. We use these decoding rules to reconstruct time-varying stimuli represented by a Gaussian process from spike trains of single neurons as well as neural populations.

Bayesian population decoding of spiking neurons

Directory of Open Access Journals (Sweden)

Sebastian Gerwinn

2009-10-01

Full Text Available The timing of action potentials in spiking neurons depends on the temporal dynamics of their inputs and contains information about temporal fluctuations in the stimulus. Leaky integrate-and-fire neurons constitute a popular class of encoding models, in which spike times depend directly on the temporal structure of the inputs. However, optimal decoding rules for these models have only been studied explicitly in the noiseless case. Here, we study decoding rules for probabilistic inference of a continuous stimulus from the spike times of a population of leaky integrate-and-fire neurons with threshold noise. We derive three algorithms for approximating the posterior distribution over stimuli as a function of the observed spike trains. In addition to a reconstruction of the stimulus we thus obtain an estimate of the uncertainty as well. Furthermore, we derive a `spike-by-spike' online decoding scheme that recursively updates the posterior with the arrival of each new spike. We use these decoding rules to reconstruct time-varying stimuli represented by a Gaussian process from spike trains of single neurons as well as neural populations.

An imperfect dopaminergic error signal can drive temporal-difference learning.

Directory of Open Access Journals (Sweden)

Wiebke Potjans

2011-05-01

Full Text Available An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards.

Neural plasticity and its initiating conditions in tinnitus.

Science.gov (United States)

Roberts, L E

2018-03-01

Deafferentation caused by cochlear pathology (which can be hidden from the audiogram) activates forms of neural plasticity in auditory pathways, generating tinnitus and its associated conditions including hyperacusis. This article discusses tinnitus mechanisms and suggests how these mechanisms may relate to those involved in normal auditory information processing. Research findings from animal models of tinnitus and from electromagnetic imaging of tinnitus patients are reviewed which pertain to the role of deafferentation and neural plasticity in tinnitus and hyperacusis. Auditory neurons compensate for deafferentation by increasing their input/output functions (gain) at multiple levels of the auditory system. Forms of homeostatic plasticity are believed to be responsible for this neural change, which increases the spontaneous and driven activity of neurons in central auditory structures in animals expressing behavioral evidence of tinnitus. Another tinnitus correlate, increased neural synchrony among the affected neurons, is forged by spike-timing-dependent neural plasticity in auditory pathways. Slow oscillations generated by bursting thalamic neurons verified in tinnitus animals appear to modulate neural plasticity in the cortex, integrating tinnitus neural activity with information in brain regions supporting memory, emotion, and consciousness which exhibit increased metabolic activity in tinnitus patients. The latter process may be induced by transient auditory events in normal processing but it persists in tinnitus, driven by phantom signals from the auditory pathway. Several tinnitus therapies attempt to suppress tinnitus through plasticity, but repeated sessions will likely be needed to prevent tinnitus activity from returning owing to deafferentation as its initiating condition.

The solar modulation of galactic comic rays as described by a time-dependent drift model

International Nuclear Information System (INIS)

Le Roux, J.A.

1990-09-01

The modulation process is understood to be an interaction between cosmic rays and the solar wind. The heliosphere and the observed modulation of cosmic rays in the heliosphere was reviewed and the time-dependence nature of the long-term modulation of cosmic rays highligted. A two-dimensional time-dependent drift model that describes the long-term modulation of cosmic-rays is presented. Application of the time-dependent drift model during times of increased solar activity showed that drift should be reduced during such periods. Isolated Forbush decreases were also studied in an effort to explain some observed trends in the properties of the Forbush decrease as a function of radial distance. The magnitude of the Forbush decrease and its recovery time were therefore studied as a function of radial distance in the equatorial plane. 154 refs., 95 figs., 1 tab

Reward Circuitry in Addiction.

Science.gov (United States)

Cooper, Sarah; Robison, A J; Mazei-Robison, Michelle S

2017-07-01

Understanding the brain circuitry that underlies reward is critical to improve treatment for many common health issues, including obesity, depression, and addiction. Here we focus on insights into the organization and function of reward circuitry and its synaptic and structural adaptations in response to cocaine exposure. While the importance of certain circuits, such as the mesocorticolimbic dopamine pathway, are well established in drug reward, recent studies using genetics-based tools have revealed functional changes throughout the reward circuitry that contribute to different facets of addiction, such as relapse and craving. The ability to observe and manipulate neuronal activity within specific cell types and circuits has led to new insight into not only the basic connections between brain regions, but also the molecular changes within these specific microcircuits, such as neurotrophic factor and GTPase signaling or α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) receptor function, that underlie synaptic and structural plasticity evoked by drugs of abuse. Excitingly, these insights from preclinical rodent work are now being translated into the clinic, where transcranial magnetic simulation and deep brain stimulation therapies are being piloted in human cocaine dependence. Thus, this review seeks to summarize current understanding of the major brain regions implicated in drug-related behaviors and the molecular mechanisms that contribute to altered connectivity between these regions, with the postulation that increased knowledge of the plasticity within the drug reward circuit will lead to new and improved treatments for addiction.

Reward-Modulated Hebbian Plasticity as Leverage for Partially Embodied Control in Compliant Robotics

Science.gov (United States)

Burms, Jeroen; Caluwaerts, Ken; Dambre, Joni

2015-01-01

In embodied computation (or morphological computation), part of the complexity of motor control is offloaded to the body dynamics. We demonstrate that a simple Hebbian-like learning rule can be used to train systems with (partial) embodiment, and can be extended outside of the scope of traditional neural networks. To this end, we apply the learning rule to optimize the connection weights of recurrent neural networks with different topologies and for various tasks. We then apply this learning rule to a simulated compliant tensegrity robot by optimizing static feedback controllers that directly exploit the dynamics of the robot body. This leads to partially embodied controllers, i.e., hybrid controllers that naturally integrate the computations that are performed by the robot body into a neural network architecture. Our results demonstrate the universal applicability of reward-modulated Hebbian learning. Furthermore, they demonstrate the robustness of systems trained with the learning rule. This study strengthens our belief that compliant robots should or can be seen as computational units, instead of dumb hardware that needs a complex controller. This link between compliant robotics and neural networks is also the main reason for our search for simple universal learning rules for both neural networks and robotics. PMID:26347645

Improving Spiking Dynamical Networks: Accurate Delays, Higher-Order Synapses, and Time Cells.

Science.gov (United States)

Voelker, Aaron R; Eliasmith, Chris

2018-03-01

Researchers building spiking neural networks face the challenge of improving the biological plausibility of their model networks while maintaining the ability to quantitatively characterize network behavior. In this work, we extend the theory behind the neural engineering framework (NEF), a method of building spiking dynamical networks, to permit the use of a broad class of synapse models while maintaining prescribed dynamics up to a given order. This theory improves our understanding of how low-level synaptic properties alter the accuracy of high-level computations in spiking dynamical networks. For completeness, we provide characterizations for both continuous-time (i.e., analog) and discrete-time (i.e., digital) simulations. We demonstrate the utility of these extensions by mapping an optimal delay line onto various spiking dynamical networks using higher-order models of the synapse. We show that these networks nonlinearly encode rolling windows of input history, using a scale invariant representation, with accuracy depending on the frequency content of the input signal. Finally, we reveal that these methods provide a novel explanation of time cell responses during a delay task, which have been observed throughout hippocampus, striatum, and cortex.

Reward modulation of hippocampal subfield activation during successful associative encoding and retrieval

Science.gov (United States)

Wolosin, Sasha M.; Zeithamova, Dagmar; Preston, Alison R.

2012-01-01

Emerging evidence suggests that motivation enhances episodic memory formation through interactions between medial temporal lobe (MTL) structures and dopaminergic midbrain. In addition, recent theories propose that motivation specifically facilitates hippocampal associative binding processes, resulting in more detailed memories that are readily reinstated from partial input. Here, we used high-resolution functional magnetic resonance imaging to determine how motivation influences associative encoding and retrieval processes within human MTL subregions and dopaminergic midbrain. Participants intentionally encoded object associations under varying conditions of reward and performed a retrieval task during which studied associations were cued from partial input. Behaviorally, cued recall performance was superior for high-value relative to low-value associations; however, participants differed in the degree to which rewards influenced memory. The magnitude of behavioral reward modulation was associated with reward-related activation changes in dentate gyrus/CA2,3 during encoding and enhanced functional connectivity between dentate gyrus/CA2,3 and dopaminergic midbrain during both the encoding and retrieval phases of the task. These findings suggests that within the hippocampus, reward-based motivation specifically enhances dentate gyrus/CA2,3 associative encoding mechanisms through interactions with dopaminergic midbrain. Furthermore, within parahippocampal cortex and dopaminergic midbrain regions, activation associated with successful memory formation was modulated by reward across the group. During the retrieval phase, we also observed enhanced activation in hippocampus and dopaminergic midbrain for high-value associations that occurred in the absence of any explicit cues to reward. Collectively, these findings shed light on fundamental mechanisms through which reward impacts associative memory formation and retrieval through facilitation of MTL and VTA/SN processing

Cigarette smoking modulates medication-associated deficits in a monetary reward task in patients with schizophrenia.

Science.gov (United States)

Lernbass, Birgit; Grön, Georg; Wolf, Nadine D; Abler, Birgit

2013-09-01

Imaging studies of reward processing have demonstrated a mesolimbic-mesocortical dopaminergic dysfunction in schizophrenia. Such studies on reward processing in patients and also in healthy controls showed that differential activations of dopaminergic brain areas are associated with adaptive changes in response speed related to different reward values. Given this relationship, we investigated reward processing on the behavioural level in a larger sample of 49 medicated patients with a diagnosis of schizophrenia (ICD-10 F20) and 49 healthy controls. Subjects were instructed to react by button press upon two different stimuli in order to retain a 60 % chance winning a previously announced high (1$) or low (20¢) amount of money paid to participants after the experiment. Concordant with previous reports on deficits in reward processing, acceleration of reaction times in patients upon low rewards differed significantly (p non-smoking subgroup of patients (n = 24). In this subgroup, we also observed a significant (p monetary reward task might constitute a feasible behavioural proxy for dopaminergic dysfunction and its different dimensions regarding psychopathology but also medication in patients with schizophrenia. In line with clinical observations, our findings support the notion that smoking modulates medication-associated side effects on reward processing in patients with schizophrenia.

Atypical valuation of monetary and cigarette rewards in substance dependent smokers.

Science.gov (United States)

Baker, Travis E; Wood, Jonathan M A; Holroyd, Clay B

2016-02-01

Substance dependent (SD) relative to non-dependent (ND) individuals exhibit an attenuated reward positivity, an electrophysiological signal believed to index sensitivity of anterior cingulate cortex (ACC) to rewards. Here we asked whether this altered neural response reflects a specific devaluation of monetary rewards relative to drug-related rewards by ACC. We recorded the reward positivity from SD and ND individuals who currently smoke, following an overnight period of abstinence, while they engaged in two feedback tasks. In a money condition the feedback indicated either a monetary reward or no reward, and in a cigarette condition the feedback indicated either a drug-related reward or no reward. Overall, cigarette relative to monetary rewards elicited a larger reward positivity. Further, for the subjects who engaged in the money condition first, the reward positivity was smaller for the SD compared to the ND participants, but for the subjects who engaged in the cigarette condition first, the reward positivity was larger for the SD compared to the ND participants. Our results suggest that the initial category of feedback "primed" the response of the ACC to the alternative feedback type on subsequent trials, and that SD and ND individuals responded differently to this priming effect. We propose that for people who misuse addictive substances, the prospect of obtaining drug-related rewards engages the ACC to exert control over extended behaviors. Copyright © 2015 International Federation of Clinical Neurophysiology. Published by Elsevier Ireland Ltd. All rights reserved.

Behavioral plasticity through the modulation of switch neurons.

Science.gov (United States)

Vassiliades, Vassilis; Christodoulou, Chris

2016-02-01

A central question in artificial intelligence is how to design agents capable of switching between different behaviors in response to environmental changes. Taking inspiration from neuroscience, we address this problem by utilizing artificial neural networks (NNs) as agent controllers, and mechanisms such as neuromodulation and synaptic gating. The novel aspect of this work is the introduction of a type of artificial neuron we call "switch neuron". A switch neuron regulates the flow of information in NNs by selectively gating all but one of its incoming synaptic connections, effectively allowing only one signal to propagate forward. The allowed connection is determined by the switch neuron's level of modulatory activation which is affected by modulatory signals, such as signals that encode some information about the reward received by the agent. An important aspect of the switch neuron is that it can be used in appropriate "switch modules" in order to modulate other switch neurons. As we show, the introduction of the switch modules enables the creation of sequences of gating events. This is achieved through the design of a modulatory pathway capable of exploring in a principled manner all permutations of the connections arriving on the switch neurons. We test the model by presenting appropriate architectures in nonstationary binary association problems and T-maze tasks. The results show that for all tasks, the switch neuron architectures generate optimal adaptive behaviors, providing evidence that the switch neuron model could be a valuable tool in simulations where behavioral plasticity is required. Copyright © 2015 Elsevier Ltd. All rights reserved.

Accelerated iTBS treatment in depressed patients differentially modulates reward system activity based on anhedonia.

Science.gov (United States)

Duprat, Romain; Wu, Guo-Rong; De Raedt, Rudi; Baeken, Chris

2017-08-09

Accelerated intermittent theta-burst stimulation (aiTBS) anti-depressive working mechanisms are still unclear. Because aiTBS may work through modulating the reward system and the level of anhedonia may influence this modulation, we investigated the effect of aiTBS on reward responsiveness in high and low anhedonic MDD patients. In this registered RCT (NCT01832805), 50 MDD patients were randomised to a sham-controlled cross-over aiTBS treatment protocol over the left dorsolateral prefrontal cortex (DLPFC). Patients performed a probabilistic learning task in fMRI before and after each week of stimulation. Task performance analyses did not show any significant effects of aiTBS on reward responsiveness, nor differences between both groups of MDD patients. However, at baseline, low anhedonic patients displayed higher neural activity in the caudate and putamen. After the first week of aiTBS treatment, in low anhedonic patients we found a decreased neural activity within the reward system, in contrast to an increased activity observed in high anhedonic patients. No changes were observed in reward related neural regions after the first week of sham stimulation. Although both MDD groups showed no differences in task performance, our brain imaging findings suggest that left DLPFC aiTBS treatment modulates the reward system differently according to anhedonia severity.

PLASTIC PIPE DEFECT DETECTION USING NONLINEAR ACOUSTIC MODULATION

Directory of Open Access Journals (Sweden)

Gigih Priyandoko

2015-02-01

Full Text Available This project discuss about the defect detection of plastic pipe by using nonlinear acoustic wave modulation method. Nonlinaer acoustic modulations are investigated for fatigue crack detection. It is a sensitive method for damage detection and it is based on the propagation of high frequency acoustic waves in plastic pipe with low frequency excitation. The plastic pipe is excited simultaneously with a slow amplitude modulated vibration pumping wave and a constant amplitude probing wave. The frequency of both the excitation signals coincides with the resonances of the plastic pipe. An actuator is used for frequencies generation while sensor is used for the frequencies detection. Besides that, a PVP pipe is used as the specimen as it is commonly used for the conveyance of liquid in many fields. The results obtained are being observed and the difference between uncrack specimen and cracked specimen can be distinguished.

Statistical characteristics of climbing fiber spikes necessary for efficient cerebellar learning.

Science.gov (United States)

Kuroda, S; Yamamoto, K; Miyamoto, H; Doya, K; Kawat, M

2001-03-01

Mean firing rates (MFRs), with analogue values, have thus far been used as information carriers of neurons in most brain theories of learning. However, the neurons transmit the signal by spikes, which are discrete events. The climbing fibers (CFs), which are known to be essential for cerebellar motor learning, fire at the ultra-low firing rates (around 1 Hz), and it is not yet understood theoretically how high-frequency information can be conveyed and how learning of smooth and fast movements can be achieved. Here we address whether cerebellar learning can be achieved by CF spikes instead of conventional MFR in an eye movement task, such as the ocular following response (OFR), and an arm movement task. There are two major afferents into cerebellar Purkinje cells: parallel fiber (PF) and CF, and the synaptic weights between PFs and Purkinje cells have been shown to be modulated by the stimulation of both types of fiber. The modulation of the synaptic weights is regulated by the cerebellar synaptic plasticity. In this study we simulated cerebellar learning using CF signals as spikes instead of conventional MFR. To generate the spikes we used the following four spike generation models: (1) a Poisson model in which the spike interval probability follows a Poisson distribution, (2) a gamma model in which the spike interval probability follows the gamma distribution, (3) a max model in which a spike is generated when a synaptic input reaches maximum, and (4) a threshold model in which a spike is generated when the input crosses a certain small threshold. We found that, in an OFR task with a constant visual velocity, learning was successful with stochastic models, such as Poisson and gamma models, but not in the deterministic models, such as max and threshold models. In an OFR with a stepwise velocity change and an arm movement task, learning could be achieved only in the Poisson model. In addition, for efficient cerebellar learning, the distribution of CF spike

Mixed signal learning by spike correlation propagation in feedback inhibitory circuits.

Directory of Open Access Journals (Sweden)

Naoki Hiratani

2015-04-01

Full Text Available The brain can learn and detect mixed input signals masked by various types of noise, and spike-timing-dependent plasticity (STDP is the candidate synaptic level mechanism. Because sensory inputs typically have spike correlation, and local circuits have dense feedback connections, input spikes cause the propagation of spike correlation in lateral circuits; however, it is largely unknown how this secondary correlation generated by lateral circuits influences learning processes through STDP, or whether it is beneficial to achieve efficient spike-based learning from uncertain stimuli. To explore the answers to these questions, we construct models of feedforward networks with lateral inhibitory circuits and study how propagated correlation influences STDP learning, and what kind of learning algorithm such circuits achieve. We derive analytical conditions at which neurons detect minor signals with STDP, and show that depending on the origin of the noise, different correlation timescales are useful for learning. In particular, we show that non-precise spike correlation is beneficial for learning in the presence of cross-talk noise. We also show that by considering excitatory and inhibitory STDP at lateral connections, the circuit can acquire a lateral structure optimal for signal detection. In addition, we demonstrate that the model performs blind source separation in a manner similar to the sequential sampling approximation of the Bayesian independent component analysis algorithm. Our results provide a basic understanding of STDP learning in feedback circuits by integrating analyses from both dynamical systems and information theory.

Differential encoding of factors influencing predicted reward value in monkey rostral anterior cingulate cortex.

Science.gov (United States)

Toda, Koji; Sugase-Miyamoto, Yasuko; Mizuhiki, Takashi; Inaba, Kiyonori; Richmond, Barry J; Shidara, Munetaka

2012-01-01

The value of a predicted reward can be estimated based on the conjunction of both the intrinsic reward value and the length of time to obtain it. The question we addressed is how the two aspects, reward size and proximity to reward, influence the responses of neurons in rostral anterior cingulate cortex (rACC), a brain region thought to play an important role in reward processing. We recorded from single neurons while two monkeys performed a multi-trial reward schedule task. The monkeys performed 1-4 sequential color discrimination trials to obtain a reward of 1-3 liquid drops. There were two task conditions, a valid cue condition, where the number of trials and reward amount were associated with visual cues, and a random cue condition, where the cue was picked from the cue set at random. In the valid cue condition, the neuronal firing is strongly modulated by the predicted reward proximity during the trials. Information about the predicted reward amount is almost absent at those times. In substantial subpopulations, the neuronal responses decreased or increased gradually through schedule progress to the predicted outcome. These two gradually modulating signals could be used to calculate the effect of time on the perception of reward value. In the random cue condition, little information about the reward proximity or reward amount is encoded during the course of the trial before reward delivery, but when the reward is actually delivered the responses reflect both the reward proximity and reward amount. Our results suggest that the rACC neurons encode information about reward proximity and amount in a manner that is dependent on utility of reward information. The manner in which the information is represented could be used in the moment-to-moment calculation of the effect of time and amount on predicted outcome value.

Homeostatic plasticity and STDP: keeping a neuron's cool in a fluctuating world

Directory of Open Access Journals (Sweden)

Alanna J Watt

2010-06-01

Full Text Available Spike-timing dependent plasticity (STDP offers a powerful means of forming and modifying neural circuits. Experimental and theoretical studies have demonstrated its potential usefulness for functions as varied as cortical map development, sharpening of sensory receptive fields, working memory, and associative learning. Even so, it is unlikely that STDP works alone. Unless changes in synaptic strength are coordinated across multiple synapses and with other neuronal properties, it is difficult to maintain the stability and functionality of neural circuits. Moreover, there are certain features of early postnatal development (e.g., rapid changes in sensory input that threaten neural circuit stability in ways that STDP may not be well placed to counter. These considerations have led researchers to investigate additional types of plasticity, complementary to STDP, that may serve to constrain synaptic weights and/or neuronal firing. These are collectively known as “homeostatic plasticity” and include schemes that control the total synaptic strength of a neuron, that modulate its intrinsic excitability as a function of average activity, or that make the ability of synapses to undergo Hebbian modification depend upon their history of use. In this article, we will review the experimental evidence for homeostatic forms of plasticity and consider how they might interact with STDP during development and learning & memory.

Correlation-based model of artificially induced plasticity in motor cortex by a bidirectional brain-computer interface.

Science.gov (United States)

Lajoie, Guillaume; Krouchev, Nedialko I; Kalaska, John F; Fairhall, Adrienne L; Fetz, Eberhard E

2017-02-01

Experiments show that spike-triggered stimulation performed with Bidirectional Brain-Computer-Interfaces (BBCI) can artificially strengthen connections between separate neural sites in motor cortex (MC). When spikes from a neuron recorded at one MC site trigger stimuli at a second target site after a fixed delay, the connections between sites eventually strengthen. It was also found that effective spike-stimulus delays are consistent with experimentally derived spike-timing-dependent plasticity (STDP) rules, suggesting that STDP is key to drive these changes. However, the impact of STDP at the level of circuits, and the mechanisms governing its modification with neural implants remain poorly understood. The present work describes a recurrent neural network model with probabilistic spiking mechanisms and plastic synapses capable of capturing both neural and synaptic activity statistics relevant to BBCI conditioning protocols. Our model successfully reproduces key experimental results, both established and new, and offers mechanistic insights into spike-triggered conditioning. Using analytical calculations and numerical simulations, we derive optimal operational regimes for BBCIs, and formulate predictions concerning the efficacy of spike-triggered conditioning in different regimes of cortical activity.

Dopamine modulates reward system activity during subconscious processing of sexual stimuli.

Science.gov (United States)

Oei, Nicole Y L; Rombouts, Serge Arb; Soeter, Roelof P; van Gerven, Joop M; Both, Stephanie

2012-06-01

Dopaminergic medication influences conscious processing of rewarding stimuli, and is associated with impulsive-compulsive behaviors, such as hypersexuality. Previous studies have shown that subconscious subliminal presentation of sexual stimuli activates brain areas known to be part of the 'reward system'. In this study, it was hypothesized that dopamine modulates activation in key areas of the reward system, such as the nucleus accumbens, during subconscious processing of sexual stimuli. Young healthy males (n=53) were randomly assigned to two experimental groups or a control group, and were administered a dopamine antagonist (haloperidol), a dopamine agonist (levodopa), or placebo. Brain activation was assessed during a backward-masking task with subliminally presented sexual stimuli. Results showed that levodopa significantly enhanced the activation in the nucleus accumbens and dorsal anterior cingulate when subliminal sexual stimuli were shown, whereas haloperidol decreased activations in those areas. Dopamine thus enhances activations in regions thought to regulate 'wanting' in response to potentially rewarding sexual stimuli that are not consciously perceived. This running start of the reward system might explain the pull of rewards in individuals with compulsive reward-seeking behaviors such as hypersexuality and patients who receive dopaminergic medication.

The Severe Acute Respiratory Syndrome (SARS-coronavirus 3a protein may function as a modulator of the trafficking properties of the spike protein

Directory of Open Access Journals (Sweden)

Tan Yee-Joo

2005-02-01

Full Text Available Abstract Background A recent publication reported that a tyrosine-dependent sorting signal, present in cytoplasmic tail of the spike protein of most coronaviruses, mediates the intracellular retention of the spike protein. This motif is missing from the spike protein of the severe acute respiratory syndrome-coronavirus (SARS-CoV, resulting in high level of surface expression of the spike protein when it is expressed on its own in vitro. Presentation of the hypothesis It has been shown that the severe acute respiratory syndrome-coronavirus genome contains open reading frames that encode for proteins with no homologue in other coronaviruses. One of them is the 3a protein, which is expressed during infection in vitro and in vivo. The 3a protein, which contains a tyrosine-dependent sorting signal in its cytoplasmic domain, is expressed on the cell surface and can undergo internalization. In addition, 3a can bind to the spike protein and through this interaction, it may be able to cause the spike protein to become internalized, resulting in a decrease in its surface expression. Testing the hypothesis The effects of 3a on the internalization of cell surface spike protein can be examined biochemically and the significance of the interplay between these two viral proteins during viral infection can be studied using reverse genetics methodology. Implication of the hypothesis If this hypothesis is proven, it will indicate that the severe acute respiratory syndrome-coronavirus modulates the surface expression of the spike protein via a different mechanism from other coronaviruses. The interaction between 3a and S, which are expressed from separate subgenomic RNA, would be important for controlling the trafficking properties of S. The cell surface expression of S in infected cells significantly impacts viral assembly, viral spread and viral pathogenesis. Modulation by this unique pathway could confer certain advantages during the replication of the severe

Differential roles of nonsynaptic and synaptic plasticity in operant reward learning-induced compulsive behavior.

Science.gov (United States)

Sieling, Fred; Bédécarrats, Alexis; Simmers, John; Prinz, Astrid A; Nargeot, Romuald

2014-05-05

Rewarding stimuli in associative learning can transform the irregularly and infrequently generated motor patterns underlying motivated behaviors into output for accelerated and stereotyped repetitive action. This transition to compulsive behavioral expression is associated with modified synaptic and membrane properties of central neurons, but establishing the causal relationships between cellular plasticity and motor adaptation has remained a challenge. We found previously that changes in the intrinsic excitability and electrical synapses of identified neurons in Aplysia's central pattern-generating network for feeding are correlated with a switch to compulsive-like motor output expression induced by in vivo operant conditioning. Here, we used specific computer-simulated ionic currents in vitro to selectively replicate or suppress the membrane and synaptic plasticity resulting from this learning. In naive in vitro preparations, such experimental manipulation of neuronal membrane properties alone increased the frequency but not the regularity of feeding motor output found in preparations from operantly trained animals. On the other hand, changes in synaptic strength alone switched the regularity but not the frequency of feeding output from naive to trained states. However, simultaneously imposed changes in both membrane and synaptic properties reproduced both major aspects of the motor plasticity. Conversely, in preparations from trained animals, experimental suppression of the membrane and synaptic plasticity abolished the increase in frequency and regularity of the learned motor output expression. These data establish direct causality for the contributions of distinct synaptic and nonsynaptic adaptive processes to complementary facets of a compulsive behavior resulting from operant reward learning. Copyright © 2014 Elsevier Ltd. All rights reserved.

Reward modulates oculomotor competition between differently valued stimuli.

Science.gov (United States)

Bucker, Berno; Silvis, Jeroen D; Donk, Mieke; Theeuwes, Jan

2015-03-01

The present work explored the effects of reward in the well-known global effect paradigm in which two objects appear simultaneously in close spatial proximity. The experiment consisted of three phases (i) a pre-training phase that served as a baseline, (ii) a reward-training phase to associate differently colored stimuli with high, low and no reward value, and (iii) a post-training phase in which rewards were no longer delivered, to examine whether objects previously associated with higher reward value attracted the eyes more strongly than those associated with low or no reward value. Unlike previous reward studies, the differently valued objects directly competed with each other on the same trial. The results showed that initially eye movements were not biased towards any particular stimulus, while in the reward-training phase, eye movements started to land progressively closer towards stimuli that were associated with a high reward value. Even though rewards were no longer delivered, this bias remained robustly present in the post-training phase. A time course analysis showed that the effect of reward was present for the fastest saccades (around 170 ms) and increased with increasing latency. Although strategic effects for slower saccades cannot be ruled out, we suggest that fast oculomotor responses became habituated and were no longer under strategic attentional control. Together the results imply that reward affects oculomotor competition in favor of stimuli previously associated high reward, when multiple reward associated objects compete for selection. Copyright © 2015 Elsevier Ltd. All rights reserved.

A correction term for the covariance of renewal-reward processes with multivariate rewards

NARCIS (Netherlands)

Patch, B.; Nazarathy, Y.; Taimre, T.

We consider a renewal-reward process with multivariate rewards. Such a process is constructed from an i.i.d. sequence of time periods, to each of which there is associated a multivariate reward vector. The rewards in each time period may depend on each other and on the period length, but not on the

Heterogeneity of Purkinje cell simple spike-complex spike interactions: zebrin- and non-zebrin-related variations.

Science.gov (United States)

Tang, Tianyu; Xiao, Jianqiang; Suh, Colleen Y; Burroughs, Amelia; Cerminara, Nadia L; Jia, Linjia; Marshall, Sarah P; Wise, Andrew K; Apps, Richard; Sugihara, Izumi; Lang, Eric J

2017-08-01

Cerebellar Purkinje cells (PCs) generate two types of action potentials, simple and complex spikes. Although they are generated by distinct mechanisms, interactions between the two spike types exist. Zebrin staining produces alternating positive and negative stripes of PCs across most of the cerebellar cortex. Thus, here we compared simple spike-complex spike interactions both within and across zebrin populations. Simple spike activity undergoes a complex modulation preceding and following a complex spike. The amplitudes of the pre- and post-complex spike modulation phases were correlated across PCs. On average, the modulation was larger for PCs in zebrin positive regions. Correlations between aspects of the complex spike waveform and simple spike activity were found, some of which varied between zebrin positive and negative PCs. The implications of the results are discussed with regard to hypotheses that complex spikes are triggered by rises in simple spike activity for either motor learning or homeostatic functions. Purkinje cells (PCs) generate two types of action potentials, called simple and complex spikes (SSs and CSs). We first investigated the CS-associated modulation of SS activity and its relationship to the zebrin status of the PC. The modulation pattern consisted of a pre-CS rise in SS activity, and then, following the CS, a pause, a rebound, and finally a late inhibition of SS activity for both zebrin positive (Z+) and negative (Z-) cells, though the amplitudes of the phases were larger in Z+ cells. Moreover, the amplitudes of the pre-CS rise with the late inhibitory phase of the modulation were correlated across PCs. In contrast, correlations between modulation phases across CSs of individual PCs were generally weak. Next, the relationship between CS spikelets and SS activity was investigated. The number of spikelets/CS correlated with the average SS firing rate only for Z+ cells. In contrast, correlations across CSs between spikelet numbers and the

BAS-drive trait modulates dorsomedial striatum activity during reward response-outcome associations.

Science.gov (United States)

Costumero, Víctor; Barrós-Loscertales, Alfonso; Fuentes, Paola; Rosell-Negre, Patricia; Bustamante, Juan Carlos; Ávila, César

2016-09-01

According to the Reinforcement Sensitivity Theory, behavioral studies have found that individuals with stronger reward sensitivity easily detect cues of reward and establish faster associations between instrumental responses and reward. Neuroimaging studies have shown that processing anticipatory cues of reward is accompanied by stronger ventral striatum activity in individuals with stronger reward sensitivity. Even though establishing response-outcome contingencies has been consistently associated with dorsal striatum, individual differences in this process are poorly understood. Here, we aimed to study the relation between reward sensitivity and brain activity while processing response-reward contingencies. Forty-five participants completed the BIS/BAS questionnaire and performed a gambling task paradigm in which they received monetary rewards or punishments. Overall, our task replicated previous results that have related processing high reward outcomes with activation of striatum and medial frontal areas, whereas processing high punishment outcomes was associated with stronger activity in insula and middle cingulate. As expected, the individual differences in the activity of dorsomedial striatum correlated positively with BAS-Drive. Our results agree with previous studies that have related the dorsomedial striatum with instrumental performance, and suggest that the individual differences in this area may form part of the neural substrate responsible for modulating instrumental conditioning by reward sensitivity.

Automatic spike sorting using tuning information.

Science.gov (United States)

Ventura, Valérie

2009-09-01

Current spike sorting methods focus on clustering neurons' characteristic spike waveforms. The resulting spike-sorted data are typically used to estimate how covariates of interest modulate the firing rates of neurons. However, when these covariates do modulate the firing rates, they provide information about spikes' identities, which thus far have been ignored for the purpose of spike sorting. This letter describes a novel approach to spike sorting, which incorporates both waveform information and tuning information obtained from the modulation of firing rates. Because it efficiently uses all the available information, this spike sorter yields lower spike misclassification rates than traditional automatic spike sorters. This theoretical result is verified empirically on several examples. The proposed method does not require additional assumptions; only its implementation is different. It essentially consists of performing spike sorting and tuning estimation simultaneously rather than sequentially, as is currently done. We used an expectation-maximization maximum likelihood algorithm to implement the new spike sorter. We present the general form of this algorithm and provide a detailed implementable version under the assumptions that neurons are independent and spike according to Poisson processes. Finally, we uncover a systematic flaw of spike sorting based on waveform information only.

Changes in reward contingency modulate the trial-to-trial variability of hippocampal place cells.

Science.gov (United States)

Wikenheiser, Andrew M; Redish, A David

2011-08-01

Pyramidal cells in the rodent hippocampus often exhibit clear spatial tuning. Theories of hippocampal function suggest that these "place cells" implement multiple, independent neural representations of position (maps), based on different reference frames or environmental features. Consistent with the "multiple maps" theory, previous studies have shown that manipulating spatial factors related to task performance modulates the within-session variability (overdispersion) of cells in the hippocampus. However, the influence of changes in reward contingency on overdispersion has not been examined. To test this, we first trained rats to collect food from three feeders positioned around a circular track (task(1)). When subjects were proficient, the reward contingency was altered such that every other feeder delivered food (task(2)). We recorded ensembles of hippocampal neurons as rats performed both tasks. Place cell overdispersion was high during task(1) but decreased significantly during task(2), and this increased reliability could not be accounted for by changes in running speed or familiarity with the task. Intuitively, decreased variability might be expected to improve neural representations of position. To test this, we used Bayesian decoding of hippocampal spike trains to estimate subjects' location. Neither the amount of probability decoded to subjects' position (local probability) nor the difference between estimated position and true location (decoding accuracy) differed between tasks. However, we found that hippocampal ensembles were significantly more self-consistent during task(2) performance. These results suggest that changes in task demands can affect the firing statistics of hippocampal neurons, leading to changes in the properties of decoded neural representations.

It's about time: Earlier rewards increase intrinsic motivation.

Science.gov (United States)

Woolley, Kaitlin; Fishbach, Ayelet

2018-06-01

Can immediate (vs. delayed) rewards increase intrinsic motivation? Prior research compared the presence versus absence of rewards. By contrast, this research compared immediate versus delayed rewards, predicting that more immediate rewards increase intrinsic motivation by creating a perceptual fusion between the activity and its goal (i.e., the reward). In support of the hypothesis, framing a reward from watching a news program as more immediate (vs. delayed) increased intrinsic motivation to watch the program (Study 1), and receiving more immediate bonus (vs. delayed, Study 2; and vs. delayed and no bonus, Study 3) increased intrinsic motivation in an experimental task. The effect of reward timing was mediated by the strength of the association between an activity and a reward, and was specific to intrinsic (vs. extrinsic) motivation-immediacy influenced the positive experience of an activity, but not perceived outcome importance (Study 4). In addition, the effect of the timing of rewards was independent of the effect of the magnitude of the rewards (Study 5). (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Spike-Based Bayesian-Hebbian Learning of Temporal Sequences.

Directory of Open Access Journals (Sweden)

Philip J Tully

2016-05-01

Full Text Available Many cognitive and motor functions are enabled by the temporal representation and processing of stimuli, but it remains an open issue how neocortical microcircuits can reliably encode and replay such sequences of information. To better understand this, a modular attractor memory network is proposed in which meta-stable sequential attractor transitions are learned through changes to synaptic weights and intrinsic excitabilities via the spike-based Bayesian Confidence Propagation Neural Network (BCPNN learning rule. We find that the formation of distributed memories, embodied by increased periods of firing in pools of excitatory neurons, together with asymmetrical associations between these distinct network states, can be acquired through plasticity. The model's feasibility is demonstrated using simulations of adaptive exponential integrate-and-fire model neurons (AdEx. We show that the learning and speed of sequence replay depends on a confluence of biophysically relevant parameters including stimulus duration, level of background noise, ratio of synaptic currents, and strengths of short-term depression and adaptation. Moreover, sequence elements are shown to flexibly participate multiple times in the sequence, suggesting that spiking attractor networks of this type can support an efficient combinatorial code. The model provides a principled approach towards understanding how multiple interacting plasticity mechanisms can coordinate hetero-associative learning in unison.

Silicon synaptic transistor for hardware-based spiking neural network and neuromorphic system

Science.gov (United States)

Kim, Hyungjin; Hwang, Sungmin; Park, Jungjin; Park, Byung-Gook

2017-10-01

Brain-inspired neuromorphic systems have attracted much attention as new computing paradigms for power-efficient computation. Here, we report a silicon synaptic transistor with two electrically independent gates to realize a hardware-based neural network system without any switching components. The spike-timing dependent plasticity characteristics of the synaptic devices are measured and analyzed. With the help of the device model based on the measured data, the pattern recognition capability of the hardware-based spiking neural network systems is demonstrated using the modified national institute of standards and technology handwritten dataset. By comparing systems with and without inhibitory synapse part, it is confirmed that the inhibitory synapse part is an essential element in obtaining effective and high pattern classification capability.

Event-Driven Contrastive Divergence for Spiking Neuromorphic Systems

Directory of Open Access Journals (Sweden)

Emre eNeftci

2014-01-01

Full Text Available Restricted Boltzmann Machines (RBMs and Deep Belief Networks have been demonstrated to perform efficiently in variety of applications, such as dimensionality reduction, feature learning, and classification. Their implementation on neuromorphic hardware platforms emulating large-scale networks of spiking neurons can have significant advantages from the perspectives of scalability, power dissipation and real-time interfacing with the environment. However the traditional RBM architecture and the commonly used training algorithm known as Contrastive Divergence (CD are based on discrete updates and exact arithmetics which do not directly map onto a dynamical neural substrate. Here, we present an event-driven variation of CD to train a RBM constructed with Integrate & Fire (I&F neurons, that is constrained by the limitations of existing and near future neuromorphic hardware platforms. Our strategy is based on neural sampling, which allows us to synthesize a spiking neural network that samples from a target Boltzmann distribution. The reverberating activity of the network replaces the discrete steps of the CD algorithm, while Spike Time Dependent Plasticity (STDP carries out the weight updates in an online, asynchronous fashion.We demonstrate our approach by training an RBM composed of leaky I&F neurons with STDP synapses to learn a generative model of the MNIST hand-written digit dataset, and by testing it in recognition, generation and cue integration tasks. Our results contribute to a machine learning-driven approach for synthesizing networks of spiking neurons capable of carrying out practical, high-level functionality.

Event-driven contrastive divergence for spiking neuromorphic systems.

Science.gov (United States)

Neftci, Emre; Das, Srinjoy; Pedroni, Bruno; Kreutz-Delgado, Kenneth; Cauwenberghs, Gert

2013-01-01

Restricted Boltzmann Machines (RBMs) and Deep Belief Networks have been demonstrated to perform efficiently in a variety of applications, such as dimensionality reduction, feature learning, and classification. Their implementation on neuromorphic hardware platforms emulating large-scale networks of spiking neurons can have significant advantages from the perspectives of scalability, power dissipation and real-time interfacing with the environment. However, the traditional RBM architecture and the commonly used training algorithm known as Contrastive Divergence (CD) are based on discrete updates and exact arithmetics which do not directly map onto a dynamical neural substrate. Here, we present an event-driven variation of CD to train a RBM constructed with Integrate & Fire (I&F) neurons, that is constrained by the limitations of existing and near future neuromorphic hardware platforms. Our strategy is based on neural sampling, which allows us to synthesize a spiking neural network that samples from a target Boltzmann distribution. The recurrent activity of the network replaces the discrete steps of the CD algorithm, while Spike Time Dependent Plasticity (STDP) carries out the weight updates in an online, asynchronous fashion. We demonstrate our approach by training an RBM composed of leaky I&F neurons with STDP synapses to learn a generative model of the MNIST hand-written digit dataset, and by testing it in recognition, generation and cue integration tasks. Our results contribute to a machine learning-driven approach for synthesizing networks of spiking neurons capable of carrying out practical, high-level functionality.

The Attraction Effect Modulates Reward Prediction Errors and Intertemporal Choices.

Science.gov (United States)

Gluth, Sebastian; Hotaling, Jared M; Rieskamp, Jörg

2017-01-11

Classical economic theory contends that the utility of a choice option should be independent of other options. This view is challenged by the attraction effect, in which the relative preference between two options is altered by the addition of a third, asymmetrically dominated option. Here, we leveraged the attraction effect in the context of intertemporal choices to test whether both decisions and reward prediction errors (RPE) in the absence of choice violate the independence of irrelevant alternatives principle. We first demonstrate that intertemporal decision making is prone to the attraction effect in humans. In an independent group of participants, we then investigated how this affects the neural and behavioral valuation of outcomes using a novel intertemporal lottery task and fMRI. Participants' behavioral responses (i.e., satisfaction ratings) were modulated systematically by the attraction effect and this modulation was correlated across participants with the respective change of the RPE signal in the nucleus accumbens. Furthermore, we show that, because exponential and hyperbolic discounting models are unable to account for the attraction effect, recently proposed sequential sampling models might be more appropriate to describe intertemporal choices. Our findings demonstrate for the first time that the attraction effect modulates subjective valuation even in the absence of choice. The findings also challenge the prospect of using neuroscientific methods to measure utility in a context-free manner and have important implications for theories of reinforcement learning and delay discounting. Many theories of value-based decision making assume that people first assess the attractiveness of each option independently of each other and then pick the option with the highest subjective value. The attraction effect, however, shows that adding a new option to a choice set can change the relative value of the existing options, which is a violation of the independence

Individual differences in sensitivity to reward and punishment and neural activity during reward and avoidance learning.

Science.gov (United States)

Kim, Sang Hee; Yoon, HeungSik; Kim, Hackjin; Hamann, Stephan

2015-09-01

In this functional neuroimaging study, we investigated neural activations during the process of learning to gain monetary rewards and to avoid monetary loss, and how these activations are modulated by individual differences in reward and punishment sensitivity. Healthy young volunteers performed a reinforcement learning task where they chose one of two fractal stimuli associated with monetary gain (reward trials) or avoidance of monetary loss (avoidance trials). Trait sensitivity to reward and punishment was assessed using the behavioral inhibition/activation scales (BIS/BAS). Functional neuroimaging results showed activation of the striatum during the anticipation and reception periods of reward trials. During avoidance trials, activation of the dorsal striatum and prefrontal regions was found. As expected, individual differences in reward sensitivity were positively associated with activation in the left and right ventral striatum during reward reception. Individual differences in sensitivity to punishment were negatively associated with activation in the left dorsal striatum during avoidance anticipation and also with activation in the right lateral orbitofrontal cortex during receiving monetary loss. These results suggest that learning to attain reward and learning to avoid loss are dependent on separable sets of neural regions whose activity is modulated by trait sensitivity to reward or punishment. © The Author (2015). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.

The sleep and circadian modulation of neural reward pathways: a protocol for a pair of systematic reviews.

Science.gov (United States)

Byrne, Jamie E M; Murray, Greg

2017-12-02

Animal research suggests that neural reward activation may be systematically modulated by sleep and circadian function. Whether humans also exhibit sleep and circadian modulation of neural reward pathways is unclear. This area is in need of further research, as it has implications for the involvement of sleep and circadian function in reward-related disorders. The aim of this paper is to describe the protocol for a pair of systematic literature reviews to synthesise existing literature related to (1) sleep and (2) circadian modulation of neural reward pathways in healthy human populations. A systematic review of relevant online databases (Scopus, PubMed, Web of Science, ProQuest, PsycINFO and EBSCOhost) will be conducted. Reference lists, relevant reviews and supplementary data will be searched for additional articles. Articles will be included if (a) they contain a sleep- or circadian-related predictor variable with a neural reward outcome variable, (b) use a functional magnetic resonance imaging protocol and (c) use human samples. Articles will be excluded if study participants had disorders known to affect the reward system. The articles will be screened by two independent authors. Two authors will complete the data extraction form, with two authors independently completing the quality assessment tool for the selected articles, with a consensus reached with a third author if needed. Narrative synthesis methods will be used to analyse the data. The findings from this pair of systematic literature reviews will assist in the identification of the pathways involved in the sleep and circadian function modulation of neural reward in healthy individuals, with implications for disorders characterised by dysregulation in sleep, circadian rhythms and reward function. PROSPERO CRD42017064994.

Interplay of multiple synaptic plasticity features in filamentary memristive devices for neuromorphic computing

Science.gov (United States)

La Barbera, Selina; Vincent, Adrien F.; Vuillaume, Dominique; Querlioz, Damien; Alibart, Fabien

2016-12-01

Bio-inspired computing represents today a major challenge at different levels ranging from material science for the design of innovative devices and circuits to computer science for the understanding of the key features required for processing of natural data. In this paper, we propose a detail analysis of resistive switching dynamics in electrochemical metallization cells for synaptic plasticity implementation. We show how filament stability associated to joule effect during switching can be used to emulate key synaptic features such as short term to long term plasticity transition and spike timing dependent plasticity. Furthermore, an interplay between these different synaptic features is demonstrated for object motion detection in a spike-based neuromorphic circuit. System level simulation presents robust learning and promising synaptic operation paving the way to complex bio-inspired computing systems composed of innovative memory devices.

Temporal sequence learning in winner-take-all networks of spiking neurons demonstrated in a brain-based device.

Science.gov (United States)

McKinstry, Jeffrey L; Edelman, Gerald M

2013-01-01

Animal behavior often involves a temporally ordered sequence of actions learned from experience. Here we describe simulations of interconnected networks of spiking neurons that learn to generate patterns of activity in correct temporal order. The simulation consists of large-scale networks of thousands of excitatory and inhibitory neurons that exhibit short-term synaptic plasticity and spike-timing dependent synaptic plasticity. The neural architecture within each area is arranged to evoke winner-take-all (WTA) patterns of neural activity that persist for tens of milliseconds. In order to generate and switch between consecutive firing patterns in correct temporal order, a reentrant exchange of signals between these areas was necessary. To demonstrate the capacity of this arrangement, we used the simulation to train a brain-based device responding to visual input by autonomously generating temporal sequences of motor actions.

Striatal activation and frontostriatal connectivity during non-drug reward anticipation in alcohol dependence.

Science.gov (United States)

Becker, Alena; Kirsch, Martina; Gerchen, Martin Fungisai; Kiefer, Falk; Kirsch, Peter

2017-05-01

According to prevailing neurobiological theories of addiction, altered function in neural reward circuitry is a central mechanism of alcohol dependence. Growing evidence postulates that the ventral striatum (VS), as well as areas of the prefrontal cortex, contribute to the increased incentive salience of alcohol-associated cues, diminished motivation to pursue non-drug rewards and weakened strength of inhibitory cognitive control, which are central to addiction. The present study aims to investigate the neural response and functional connectivity underlying monetary, non-drug reward processing in alcohol dependence. We utilized a reward paradigm to investigate the anticipation of monetary reward in 32 alcohol-dependent inpatients and 35 healthy controls. Functional magnetic resonance imaging was used to measure task-related brain activation and connectivity. Alcohol-dependent patients showed increased activation of the VS during anticipation of monetary gain compared with healthy controls. Generalized psychophysiological interaction analyses revealed decreased functional connectivity between the VS and the dorsolateral prefrontal cortex in alcohol dependent patients relative to controls. Increased activation of the VS and reduced frontostriatal connectivity were associated with increased craving. These findings provide evidence that alcohol dependence is rather associated with disrupted integration of striatal and prefrontal processes than with a global reward anticipation deficit. © 2016 Society for the Study of Addiction.

A kinetic model of dopamine- and calcium-dependent striatal synaptic plasticity.

Directory of Open Access Journals (Sweden)

Takashi Nakano

2010-02-01

Full Text Available Corticostriatal synapse plasticity of medium spiny neurons is regulated by glutamate input from the cortex and dopamine input from the substantia nigra. While cortical stimulation alone results in long-term depression (LTD, the combination with dopamine switches LTD to long-term potentiation (LTP, which is known as dopamine-dependent plasticity. LTP is also induced by cortical stimulation in magnesium-free solution, which leads to massive calcium influx through NMDA-type receptors and is regarded as calcium-dependent plasticity. Signaling cascades in the corticostriatal spines are currently under investigation. However, because of the existence of multiple excitatory and inhibitory pathways with loops, the mechanisms regulating the two types of plasticity remain poorly understood. A signaling pathway model of spines that express D1-type dopamine receptors was constructed to analyze the dynamic mechanisms of dopamine- and calcium-dependent plasticity. The model incorporated all major signaling molecules, including dopamine- and cyclic AMP-regulated phosphoprotein with a molecular weight of 32 kDa (DARPP32, as well as AMPA receptor trafficking in the post-synaptic membrane. Simulations with dopamine and calcium inputs reproduced dopamine- and calcium-dependent plasticity. Further in silico experiments revealed that the positive feedback loop consisted of protein kinase A (PKA, protein phosphatase 2A (PP2A, and the phosphorylation site at threonine 75 of DARPP-32 (Thr75 served as the major switch for inducing LTD and LTP. Calcium input modulated this loop through the PP2B (phosphatase 2B-CK1 (casein kinase 1-Cdk5 (cyclin-dependent kinase 5-Thr75 pathway and PP2A, whereas calcium and dopamine input activated the loop via PKA activation by cyclic AMP (cAMP. The positive feedback loop displayed robust bi-stable responses following changes in the reaction parameters. Increased basal dopamine levels disrupted this dopamine-dependent plasticity. The

Endogenous nociceptin modulates diet preference independent of motivation and reward.

Science.gov (United States)

Koizumi, Miwako; Cagniard, Barbara; Murphy, Niall P

2009-04-20

Previous studies show that the opioid peptide nociceptin stimulates food intake. Here, we studied nociceptin receptor knockout (NOP KO) mice in various behavioral paradigms designed to differentiate psychological and physiological loci at which endogenous nociceptin might control feeding. When presented a choice under food restriction, NOP KO mice displayed reduced preference for high sucrose diet, but lower intake of high fat diet under no-choice conditions. These responses were absent under ad libitum feeding conditions. Conditioned place preference to high fat diet under food-deprived conditions was unaltered in NOP KO mice, suggesting no difference in reward responses. Furthermore, operant food self-administration under a variety of conditions showed no genotype-dependent differences, suggesting no differences in the motivational properties of food. Taste reactivity to sucrose was unchanged in NOP KO mice, though NOP KO mice had altered aversive reactions to quinine solutions under ad libitum feeding, suggesting minor differences in the affective impact of palatable and unpalatable tastants. Although NOP KO mice re-fed following food-deprivation showed normal increases in plasma glucose and insulin, multidimensional scaling analysis showed that the relationship between these measures, body weight and plasma leptin was substantially disrupted in NOP KO, particularly in fasted mice. Additionally, the typical positive relationship between body weight and plasma leptin was considerably weaker in NOP KO mice. Together, these findings suggest that endogenous nociceptin differentially modulates diet preference depending on macronutrient content and homeostatic state, independently of the motivating, rewarding or orosensory properties of food, but may involve metabolic or postingestive processes.

Correlation-based model of artificially induced plasticity in motor cortex by a bidirectional brain-computer interface.

Directory of Open Access Journals (Sweden)

Guillaume Lajoie

2017-02-01

Full Text Available Experiments show that spike-triggered stimulation performed with Bidirectional Brain-Computer-Interfaces (BBCI can artificially strengthen connections between separate neural sites in motor cortex (MC. When spikes from a neuron recorded at one MC site trigger stimuli at a second target site after a fixed delay, the connections between sites eventually strengthen. It was also found that effective spike-stimulus delays are consistent with experimentally derived spike-timing-dependent plasticity (STDP rules, suggesting that STDP is key to drive these changes. However, the impact of STDP at the level of circuits, and the mechanisms governing its modification with neural implants remain poorly understood. The present work describes a recurrent neural network model with probabilistic spiking mechanisms and plastic synapses capable of capturing both neural and synaptic activity statistics relevant to BBCI conditioning protocols. Our model successfully reproduces key experimental results, both established and new, and offers mechanistic insights into spike-triggered conditioning. Using analytical calculations and numerical simulations, we derive optimal operational regimes for BBCIs, and formulate predictions concerning the efficacy of spike-triggered conditioning in different regimes of cortical activity.

Trial-by-Trial Modulation of Associative Memory Formation by Reward Prediction Error and Reward Anticipation as Revealed by a Biologically Plausible Computational Model.

Science.gov (United States)

Aberg, Kristoffer C; Müller, Julia; Schwartz, Sophie

2017-01-01

Anticipation and delivery of rewards improves memory formation, but little effort has been made to disentangle their respective contributions to memory enhancement. Moreover, it has been suggested that the effects of reward on memory are mediated by dopaminergic influences on hippocampal plasticity. Yet, evidence linking memory improvements to actual reward computations reflected in the activity of the dopaminergic system, i.e., prediction errors and expected values, is scarce and inconclusive. For example, different previous studies reported that the magnitude of prediction errors during a reinforcement learning task was a positive, negative, or non-significant predictor of successfully encoding simultaneously presented images. Individual sensitivities to reward and punishment have been found to influence the activation of the dopaminergic reward system and could therefore help explain these seemingly discrepant results. Here, we used a novel associative memory task combined with computational modeling and showed independent effects of reward-delivery and reward-anticipation on memory. Strikingly, the computational approach revealed positive influences from both reward delivery, as mediated by prediction error magnitude, and reward anticipation, as mediated by magnitude of expected value, even in the absence of behavioral effects when analyzed using standard methods, i.e., by collapsing memory performance across trials within conditions. We additionally measured trait estimates of reward and punishment sensitivity and found that individuals with increased reward (vs. punishment) sensitivity had better memory for associations encoded during positive (vs. negative) prediction errors when tested after 20 min, but a negative trend when tested after 24 h. In conclusion, modeling trial-by-trial fluctuations in the magnitude of reward, as we did here for prediction errors and expected value computations, provides a comprehensive and biologically plausible description of

Serotonergic modulation of reward and punishment: evidence from pharmacological fMRI studies.

Science.gov (United States)

Macoveanu, Julian

2014-03-27

Until recently, the bulk of research on the human reward system was focused on studying the dopaminergic and opioid neurotransmitter systems. However, extending the initial data from animal studies on reward, recent pharmacological brain imaging studies on human participants bring a new line of evidence on the key role serotonin plays in reward processing. The reviewed research has revealed how central serotonin availability and receptor specific transmission modulates the neural response to both appetitive (rewarding) and aversive (punishing) stimuli in putative reward-related brain regions. Thus, serotonin is suggested to be involved in behavioral control when there is a prospect of reward or punishment. The new findings may have implications in understanding psychiatric disorders such as major depression which is characterized by abnormal serotonergic function and reward-related processing and may also provide a neural correlated for the emotional blunting observed in the clinical treatment of psychiatric disorders with selective serotonin reuptake inhibitors. Given the unique profile of action of each serotonergic receptor subtype, future pharmacological studies may favor receptor specific investigations to complement present research mainly focused on global serotonergic manipulations. Copyright © 2014 Elsevier B.V. All rights reserved.

Nalmefene Reduces Reward Anticipation in Alcohol Dependence: An Experimental Functional Magnetic Resonance Imaging Study.

Science.gov (United States)

Quelch, Darren R; Mick, Inge; McGonigle, John; Ramos, Anna C; Flechais, Remy S A; Bolstridge, Mark; Rabiner, Eugenii; Wall, Matthew B; Newbould, Rexford D; Steiniger-Brach, Björn; van den Berg, Franz; Boyce, Malcolm; Østergaard Nilausen, Dorrit; Breuning Sluth, Lasse; Meulien, Didier; von der Goltz, Christoph; Nutt, David; Lingford-Hughes, Anne

2017-06-01

Nalmefene is a µ and δ opioid receptor antagonist, κ opioid receptor partial agonist that has recently been approved in Europe for treating alcohol dependence. It offers a treatment approach for alcohol-dependent individuals with "high-risk drinking levels" to reduce their alcohol consumption. However, the neurobiological mechanism underpinning its effects on alcohol consumption remains to be determined. Using a randomized, double-blind, placebo-controlled, within-subject crossover design we aimed to determine the effect of a single dose of nalmefene on striatal blood oxygen level-dependent (BOLD) signal change during anticipation of monetary reward using the monetary incentive delay task following alcohol challenge. Twenty-two currently heavy-drinking, non-treatment-seeking alcohol-dependent males were recruited. The effect of single dose nalmefene (18 mg) on changes in a priori defined striatal region of interest BOLD signal change during reward anticipation compared with placebo was investigated using functional magnetic resonance imaging. Both conditions were performed under intravenous alcohol administration (6% vol/vol infusion to achieve a target level of 80 mg/dL). Datasets from 18 participants were available and showed that in the presence of the alcohol infusion, nalmefene significantly reduced the BOLD response in the striatal region of interest compared with placebo. Nalmefene did not alter brain perfusion. Nalmefene blunts BOLD response in the mesolimbic system during anticipation of monetary reward and an alcohol infusion. This is consistent with nalmefene's actions on opioid receptors, which modulate the mesolimbic dopaminergic system, and provides a neurobiological basis for its efficacy. Copyright © 2017 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.

How Sequential Changes in Reward Magnitude Modulate Cognitive Flexibility: Evidence from Voluntary Task Switching

Science.gov (United States)

Fröber, Kerstin; Dreisbach, Gesine

2016-01-01

There is much evidence that the prospect of reward modulates cognitive control in terms of more stable behavior. Increases in expected reward magnitude, however, have been suggested to increase flexible behavior as evidenced by reduced switch costs. In a series of experiments, the authors provide evidence that this increased cognitive flexibility…

Possible contributions of a novel form of synaptic plasticity in Aplysia to reward, memory, and their dysfunctions in mammalian brain.

Science.gov (United States)

Hawkins, Robert D

2013-09-18

Recent studies in Aplysia have identified a new variation of synaptic plasticity in which modulatory transmitters enhance spontaneous release of glutamate, which then acts on postsynaptic receptors to recruit mechanisms of intermediate- and long-term plasticity. In this review I suggest the hypothesis that similar plasticity occurs in mammals, where it may contribute to reward, memory, and their dysfunctions in several psychiatric disorders. In Aplysia, spontaneous release is enhanced by activation of presynaptic serotonin receptors, but presynaptic D1 dopamine receptors or nicotinic acetylcholine receptors could play a similar role in mammals. Those receptors enhance spontaneous release of glutamate in hippocampus, entorhinal cortex, prefrontal cortex, ventral tegmental area, and nucleus accumbens. In all of those brain areas, glutamate can activate postsynaptic receptors to elevate Ca(2+) and engage mechanisms of early-phase long-term potentiation (LTP), including AMPA receptor insertion, and of late-phase LTP, including protein synthesis and growth. Thus, presynaptic receptors and spontaneous release may contribute to postsynaptic mechanisms of plasticity in brain regions involved in reward and memory, and could play roles in disorders that affect plasticity in those regions, including addiction, Alzheimer's disease, schizophrenia, and attention deficit hyperactivity disorder (ADHD).

Financial time series prediction using spiking neural networks.

Science.gov (United States)

Reid, David; Hussain, Abir Jaafar; Tawfik, Hissam

2014-01-01

In this paper a novel application of a particular type of spiking neural network, a Polychronous Spiking Network, was used for financial time series prediction. It is argued that the inherent temporal capabilities of this type of network are suited to non-stationary data such as this. The performance of the spiking neural network was benchmarked against three systems: two "traditional", rate-encoded, neural networks; a Multi-Layer Perceptron neural network and a Dynamic Ridge Polynomial neural network, and a standard Linear Predictor Coefficients model. For this comparison three non-stationary and noisy time series were used: IBM stock data; US/Euro exchange rate data, and the price of Brent crude oil. The experiments demonstrated favourable prediction results for the Spiking Neural Network in terms of Annualised Return and prediction error for 5-Step ahead predictions. These results were also supported by other relevant metrics such as Maximum Drawdown and Signal-To-Noise ratio. This work demonstrated the applicability of the Polychronous Spiking Network to financial data forecasting and this in turn indicates the potential of using such networks over traditional systems in difficult to manage non-stationary environments.

A novel analytical characterization for short-term plasticity parameters in spiking neural networks.

Science.gov (United States)

O'Brien, Michael J; Thibeault, Corey M; Srinivasa, Narayan

2014-01-01

Short-term plasticity (STP) is a phenomenon that widely occurs in the neocortex with implications for learning and memory. Based on a widely used STP model, we develop an analytical characterization of the STP parameter space to determine the nature of each synapse (facilitating, depressing, or both) in a spiking neural network based on presynaptic firing rate and the corresponding STP parameters. We demonstrate consistency with previous work by leveraging the power of our characterization to replicate the functional volumes that are integral for the previous network stabilization results. We then use our characterization to predict the precise transitional point from the facilitating regime to the depressing regime in a simulated synapse, suggesting in vitro experiments to verify the underlying STP model. We conclude the work by integrating our characterization into a framework for finding suitable STP parameters for self-sustaining random, asynchronous activity in a prescribed recurrent spiking neural network. The systematic process resulting from our analytical characterization improves the success rate of finding the requisite parameters for such networks by three orders of magnitude over a random search.

Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition.

Science.gov (United States)

Kasabov, Nikola; Dhoble, Kshitij; Nuntalid, Nuttapod; Indiveri, Giacomo

2013-05-01

On-line learning and recognition of spatio- and spectro-temporal data (SSTD) is a very challenging task and an important one for the future development of autonomous machine learning systems with broad applications. Models based on spiking neural networks (SNN) have already proved their potential in capturing spatial and temporal data. One class of them, the evolving SNN (eSNN), uses a one-pass rank-order learning mechanism and a strategy to evolve a new spiking neuron and new connections to learn new patterns from incoming data. So far these networks have been mainly used for fast image and speech frame-based recognition. Alternative spike-time learning methods, such as Spike-Timing Dependent Plasticity (STDP) and its variant Spike Driven Synaptic Plasticity (SDSP), can also be used to learn spatio-temporal representations, but they usually require many iterations in an unsupervised or semi-supervised mode of learning. This paper introduces a new class of eSNN, dynamic eSNN, that utilise both rank-order learning and dynamic synapses to learn SSTD in a fast, on-line mode. The paper also introduces a new model called deSNN, that utilises rank-order learning and SDSP spike-time learning in unsupervised, supervised, or semi-supervised modes. The SDSP learning is used to evolve dynamically the network changing connection weights that capture spatio-temporal spike data clusters both during training and during recall. The new deSNN model is first illustrated on simple examples and then applied on two case study applications: (1) moving object recognition using address-event representation (AER) with data collected using a silicon retina device; (2) EEG SSTD recognition for brain-computer interfaces. The deSNN models resulted in a superior performance in terms of accuracy and speed when compared with other SNN models that use either rank-order or STDP learning. The reason is that the deSNN makes use of both the information contained in the order of the first input spikes

Measures of spike train synchrony for data with multiple time scales

NARCIS (Netherlands)

Satuvuori, Eero; Mulansky, Mario; Bozanic, Nebojsa; Malvestio, Irene; Zeldenrust, Fleur; Lenk, Kerstin; Kreuz, Thomas

2017-01-01

Background Measures of spike train synchrony are widely used in both experimental and computational neuroscience. Time-scale independent and parameter-free measures, such as the ISI-distance, the SPIKE-distance and SPIKE-synchronization, are preferable to time scale parametric measures, since by

Introduction to spiking neural networks: Information processing, learning and applications.

Science.gov (United States)

Ponulak, Filip; Kasinski, Andrzej

2011-01-01

The concept that neural information is encoded in the firing rate of neurons has been the dominant paradigm in neurobiology for many years. This paradigm has also been adopted by the theory of artificial neural networks. Recent physiological experiments demonstrate, however, that in many parts of the nervous system, neural code is founded on the timing of individual action potentials. This finding has given rise to the emergence of a new class of neural models, called spiking neural networks. In this paper we summarize basic properties of spiking neurons and spiking networks. Our focus is, specifically, on models of spike-based information coding, synaptic plasticity and learning. We also survey real-life applications of spiking models. The paper is meant to be an introduction to spiking neural networks for scientists from various disciplines interested in spike-based neural processing.

Utilization of reward-prospect enhances preparatory attention and reduces stimulus conflict.

Science.gov (United States)

van den Berg, Berry; Krebs, Ruth M; Lorist, Monicque M; Woldorff, Marty G

2014-06-01

The prospect of gaining money is an incentive widely at play in the real world. Such monetary motivation might have particularly strong influence when the cognitive system is challenged, such as when needing to process conflicting stimulus inputs. Here, we employed manipulations of reward-prospect and attentional-preparation levels in a cued-Stroop stimulus conflict task, along with the high temporal resolution of electrical brain recordings, to provide insight into the mechanisms by which reward-prospect and attention interact and modulate cognitive task performance. In this task, the cue indicated whether or not the participant needed to prepare for an upcoming Stroop stimulus and, if so, whether there was the potential for monetary reward (dependent on performance on that trial). Both cued attention and cued reward-prospect enhanced preparatory neural activity, as reflected by increases in the hallmark attention-related negative-polarity ERP slow wave (contingent negative variation [CNV]) and reductions in oscillatory Alpha activity, which was followed by enhanced processing of the subsequent Stroop stimulus. In addition, similar modulations of preparatory neural activity (larger CNVs and reduced Alpha) predicted shorter versus longer response times (RTs) to the subsequent target stimulus, consistent with such modulations reflecting trial-to-trial variations in attention. Particularly striking were the individual differences in the utilization of reward-prospect information. In particular, the size of the reward effects on the preparatory neural activity correlated across participants with the degree to which reward-prospect both facilitated overall task performance (shorter RTs) and reduced conflict-related behavioral interference. Thus, the prospect of reward appears to recruit attentional preparation circuits to enhance processing of task-relevant target information.

Modulation of short-term plasticity in the corticothalamic circuit by group III metabotropic glutamate receptors.

Science.gov (United States)

Kyuyoung, Christine L; Huguenard, John R

2014-01-08

Recurrent connections in the corticothalamic circuit underlie oscillatory behavior in this network and range from normal sleep rhythms to the abnormal spike-wave discharges seen in absence epilepsy. The propensity of thalamic neurons to fire postinhibitory rebound bursts mediated by low-threshold calcium spikes renders the circuit vulnerable to both increased excitation and increased inhibition, such as excessive excitatory cortical drive to thalamic reticular (RT) neurons or heightened inhibition of thalamocortical relay (TC) neurons by RT. In this context, a protective role may be played by group III metabotropic receptors (mGluRs), which are uniquely located in the presynaptic active zone and typically act as autoreceptors or heteroceptors to depress synaptic release. Here, we report that these receptors regulate short-term plasticity at two loci in the corticothalamic circuit in rats: glutamatergic cortical synapses onto RT neurons and GABAergic synapses onto TC neurons in somatosensory ventrobasal thalamus. The net effect of group III mGluR activation at these synapses is to suppress thalamic oscillations as assayed in vitro. These findings suggest a functional role of these receptors to modulate corticothalamic transmission and protect against prolonged activity in the network.

Adaptive robotic control driven by a versatile spiking cerebellar network.

Directory of Open Access Journals (Sweden)

Claudia Casellato

Full Text Available The cerebellum is involved in a large number of different neural processes, especially in associative learning and in fine motor control. To develop a comprehensive theory of sensorimotor learning and control, it is crucial to determine the neural basis of coding and plasticity embedded into the cerebellar neural circuit and how they are translated into behavioral outcomes in learning paradigms. Learning has to be inferred from the interaction of an embodied system with its real environment, and the same cerebellar principles derived from cell physiology have to be able to drive a variety of tasks of different nature, calling for complex timing and movement patterns. We have coupled a realistic cerebellar spiking neural network (SNN with a real robot and challenged it in multiple diverse sensorimotor tasks. Encoding and decoding strategies based on neuronal firing rates were applied. Adaptive motor control protocols with acquisition and extinction phases have been designed and tested, including an associative Pavlovian task (Eye blinking classical conditioning, a vestibulo-ocular task and a perturbed arm reaching task operating in closed-loop. The SNN processed in real-time mossy fiber inputs as arbitrary contextual signals, irrespective of whether they conveyed a tone, a vestibular stimulus or the position of a limb. A bidirectional long-term plasticity rule implemented at parallel fibers-Purkinje cell synapses modulated the output activity in the deep cerebellar nuclei. In all tasks, the neurorobot learned to adjust timing and gain of the motor responses by tuning its output discharge. It succeeded in reproducing how human biological systems acquire, extinguish and express knowledge of a noisy and changing world. By varying stimuli and perturbations patterns, real-time control robustness and generalizability were validated. The implicit spiking dynamics of the cerebellar model fulfill timing, prediction and learning functions.

Adaptive robotic control driven by a versatile spiking cerebellar network.

Science.gov (United States)

Casellato, Claudia; Antonietti, Alberto; Garrido, Jesus A; Carrillo, Richard R; Luque, Niceto R; Ros, Eduardo; Pedrocchi, Alessandra; D'Angelo, Egidio

2014-01-01

The cerebellum is involved in a large number of different neural processes, especially in associative learning and in fine motor control. To develop a comprehensive theory of sensorimotor learning and control, it is crucial to determine the neural basis of coding and plasticity embedded into the cerebellar neural circuit and how they are translated into behavioral outcomes in learning paradigms. Learning has to be inferred from the interaction of an embodied system with its real environment, and the same cerebellar principles derived from cell physiology have to be able to drive a variety of tasks of different nature, calling for complex timing and movement patterns. We have coupled a realistic cerebellar spiking neural network (SNN) with a real robot and challenged it in multiple diverse sensorimotor tasks. Encoding and decoding strategies based on neuronal firing rates were applied. Adaptive motor control protocols with acquisition and extinction phases have been designed and tested, including an associative Pavlovian task (Eye blinking classical conditioning), a vestibulo-ocular task and a perturbed arm reaching task operating in closed-loop. The SNN processed in real-time mossy fiber inputs as arbitrary contextual signals, irrespective of whether they conveyed a tone, a vestibular stimulus or the position of a limb. A bidirectional long-term plasticity rule implemented at parallel fibers-Purkinje cell synapses modulated the output activity in the deep cerebellar nuclei. In all tasks, the neurorobot learned to adjust timing and gain of the motor responses by tuning its output discharge. It succeeded in reproducing how human biological systems acquire, extinguish and express knowledge of a noisy and changing world. By varying stimuli and perturbations patterns, real-time control robustness and generalizability were validated. The implicit spiking dynamics of the cerebellar model fulfill timing, prediction and learning functions.

Real-time computing platform for spiking neurons (RT-spike).

Science.gov (United States)

Ros, Eduardo; Ortigosa, Eva M; Agís, Rodrigo; Carrillo, Richard; Arnold, Michael

2006-07-01

A computing platform is described for simulating arbitrary networks of spiking neurons in real time. A hybrid computing scheme is adopted that uses both software and hardware components to manage the tradeoff between flexibility and computational power; the neuron model is implemented in hardware and the network model and the learning are implemented in software. The incremental transition of the software components into hardware is supported. We focus on a spike response model (SRM) for a neuron where the synapses are modeled as input-driven conductances. The temporal dynamics of the synaptic integration process are modeled with a synaptic time constant that results in a gradual injection of charge. This type of model is computationally expensive and is not easily amenable to existing software-based event-driven approaches. As an alternative we have designed an efficient time-based computing architecture in hardware, where the different stages of the neuron model are processed in parallel. Further improvements occur by computing multiple neurons in parallel using multiple processing units. This design is tested using reconfigurable hardware and its scalability and performance evaluated. Our overall goal is to investigate biologically realistic models for the real-time control of robots operating within closed action-perception loops, and so we evaluate the performance of the system on simulating a model of the cerebellum where the emulation of the temporal dynamics of the synaptic integration process is important.

Reward anticipation modulates the effect of stress-related increases in cortisol on episodic memory.

Science.gov (United States)

Quent, Jörn A; McCullough, Andrew M; Sazma, Matt; Wolf, Oliver T; Yonelinas, Andrew P

2018-01-01

When acute stress is experienced shortly after an event is encoded into memory, this can slow the forgetting of the study event, which is thought to reflect the effect of cortisol on consolidation. In addition, when events are encoded under conditions of high reward they tend to be remembered better than those encoded under non-rewarding conditions, and these effects are thought to reflect the operation of the dopaminergic reward system. Although both modulatory systems are believed to impact the medial temporal lobe regions critical for episodic memory, the manner, and even the extent, to which these two systems interact is currently unknown. To address this question in the current study, participants encoded words under reward or non-reward conditions, then one half of the participants were stressed using the social evaluation cold pressor task and the other half completed a non-stress control task. After a two-hour delay, all participants received a free recall and recognition memory test. There were no significant effects of stress or reward on overall memory performance. However, for the non-reward items, increases in stress-related cortisol in stressed participants were related to increases in recall and increases in recollection-based recognition responses. In contrast, for the reward items, increases in stress-related cortisol were not related to increases in memory performance. The results indicate that the stress and the reward systems interact in the way they impact episodic memory. The results are consistent with tag and capture models in the sense that cortisol reactivity can only affect non-reward items because plasticity-related products are already provided by reward anticipation. Copyright © 2017 Elsevier Inc. All rights reserved.

Reward-dependent modulation of working memory is associated with negative symptoms in schizophrenia.

Science.gov (United States)

Hager, Oliver M; Kirschner, Matthias; Bischof, Martin; Hartmann-Riemer, Matthias N; Kluge, Agne; Seifritz, Erich; Tobler, Philippe N; Kaiser, Stefan

2015-10-01

The negative symptoms of schizophrenia have been associated with altered neural activity during both reward processing and cognitive processing. Even though increasing evidence suggests a strong interaction between these two domains, it has not been studied in relation to negative symptoms. To elucidate neural mechanisms of the reward-cognition interaction, we applied a letter variant of the n-back working memory task and varied the financial incentives for performance. In the interaction contrast, we found a significantly activated cluster in the rostral anterior cingulate cortex (ACC), the middle frontal gyrus, and the bilateral superior frontal gyrus. The interaction did not differ significantly between the patient group and a healthy control group, suggesting that patients with schizophrenia are on average able to integrate reward information and utilize this information to maximize cognitive performance. However within the patient group, we found a significant inverse correlation of ACC activity with the factor diminished expression. This finding is consistent with the model that a lack of available cognitive resources leads to diminished expression. We therefore argue that patients with diminished expression have difficulties in recruiting additional cognitive resources (as implemented in the ACC) in response to an anticipated reward. Due to this lack of cognitive resources, less processing capacity is available for effective expression, resulting in diminished expressive behavior. Copyright © 2015 Elsevier B.V. All rights reserved.

Cocaine Promotes Coincidence Detection and Lowers Induction Threshold during Hebbian Associative Synaptic Potentiation in Prefrontal Cortex.

Science.gov (United States)

Ruan, Hongyu; Yao, Wei-Dong

2017-01-25

Addictive drugs usurp neural plasticity mechanisms that normally serve reward-related learning and memory, primarily by evoking changes in glutamatergic synaptic strength in the mesocorticolimbic dopamine circuitry. Here, we show that repeated cocaine exposure in vivo does not alter synaptic strength in the mouse prefrontal cortex during an early period of withdrawal, but instead modifies a Hebbian quantitative synaptic learning rule by broadening the temporal window and lowers the induction threshold for spike-timing-dependent LTP (t-LTP). After repeated, but not single, daily cocaine injections, t-LTP in layer V pyramidal neurons is induced at +30 ms, a normally ineffective timing interval for t-LTP induction in saline-exposed mice. This cocaine-induced, extended-timing t-LTP lasts for ∼1 week after terminating cocaine and is accompanied by an increased susceptibility to potentiation by fewer pre-post spike pairs, indicating a reduced t-LTP induction threshold. Basal synaptic strength and the maximal attainable t-LTP magnitude remain unchanged after cocaine exposure. We further show that the cocaine facilitation of t-LTP induction is caused by sensitized D1-cAMP/protein kinase A dopamine signaling in pyramidal neurons, which then pathologically recruits voltage-gated l-type Ca 2+ channels that synergize with GluN2A-containing NMDA receptors to drive t-LTP at extended timing. Our results illustrate a mechanism by which cocaine, acting on a key neuromodulation pathway, modifies the coincidence detection window during Hebbian plasticity to facilitate associative synaptic potentiation in prefrontal excitatory circuits. By modifying rules that govern activity-dependent synaptic plasticity, addictive drugs can derail the experience-driven neural circuit remodeling process important for executive control of reward and addiction. It is believed that addictive drugs often render an addict's brain reward system hypersensitive, leaving the individual more susceptible to

Contingency learning in alcohol dependence and pathological gambling: learning and unlearning reward contingencies

NARCIS (Netherlands)

Vanes, L.D.; Holst, R.J. van; Jansen, J.M.; Brink, W. van den; Oosterlaan, J.; Goudriaan, A.E.

2014-01-01

BACKGROUND: Patients with alcohol dependence (AD) and pathological gambling (PG) are characterized by dysfunctional reward processing and their ability to adapt to alterations of reward contingencies is impaired. However, most neurocognitive tasks investigating reward processing involve a complex

Contingency Learning in Alcohol Dependence and Pathological Gambling: Learning and Unlearning Reward Contingencies

NARCIS (Netherlands)

Vanes, L.D.; Holst, R.; Jansen, J.D.; van den Brink, W.A.; Oosterlaan, J.; Goudriaan, A.E.

2014-01-01

Background: Patients with alcohol dependence (AD) and pathological gambling (PG) are characterized by dysfunctional reward processing and their ability to adapt to alterations of reward contingencies is impaired. However, most neurocognitive tasks investigating reward processing involve a complex

Stress-Induced Impairment of a Working Memory Task: Role of Spiking Rate and Spiking History Predicted Discharge

Science.gov (United States)

Devilbiss, David M.; Jenison, Rick L.; Berridge, Craig W.

2012-01-01

Stress, pervasive in society, contributes to over half of all work place accidents a year and over time can contribute to a variety of psychiatric disorders including depression, schizophrenia, and post-traumatic stress disorder. Stress impairs higher cognitive processes, dependent on the prefrontal cortex (PFC) and that involve maintenance and integration of information over extended periods, including working memory and attention. Substantial evidence has demonstrated a relationship between patterns of PFC neuron spiking activity (action-potential discharge) and components of delayed-response tasks used to probe PFC-dependent cognitive function in rats and monkeys. During delay periods of these tasks, persistent spiking activity is posited to be essential for the maintenance of information for working memory and attention. However, the degree to which stress-induced impairment in PFC-dependent cognition involves changes in task-related spiking rates or the ability for PFC neurons to retain information over time remains unknown. In the current study, spiking activity was recorded from the medial PFC of rats performing a delayed-response task of working memory during acute noise stress (93 db). Spike history-predicted discharge (SHPD) for PFC neurons was quantified as a measure of the degree to which ongoing neuronal discharge can be predicted by past spiking activity and reflects the degree to which past information is retained by these neurons over time. We found that PFC neuron discharge is predicted by their past spiking patterns for nearly one second. Acute stress impaired SHPD, selectively during delay intervals of the task, and simultaneously impaired task performance. Despite the reduction in delay-related SHPD, stress increased delay-related spiking rates. These findings suggest that neural codes utilizing SHPD within PFC networks likely reflects an additional important neurophysiological mechanism for maintenance of past information over time. Stress

Rewards modulate saccade latency but not exogenous spatial attention.

Science.gov (United States)

Dunne, Stephen; Ellison, Amanda; Smith, Daniel T

2015-01-01

The eye movement system is sensitive to reward. However, whilst the eye movement system is extremely flexible, the extent to which changes to oculomotor behavior induced by reward paradigms persist beyond the training period or transfer to other oculomotor tasks is unclear. To address these issues we examined the effects of presenting feedback that represented small monetary rewards to spatial locations on the latency of saccadic eye movements, the time-course of learning and extinction of the effects of rewarding saccades on exogenous spatial attention and oculomotor inhibition of return. Reward feedback produced a relative facilitation of saccadic latency in a stimulus driven saccade task which persisted for three blocks of extinction trials. However, this hemifield-specific effect failed to transfer to peripheral cueing tasks. We conclude that rewarding specific spatial locations is unlikely to induce long-term, systemic changes to the human oculomotor or attention systems.

Rewards modulate saccade latency but not exogenous spatial attention.

Directory of Open Access Journals (Sweden)

Stephen eDunne

2015-07-01

Full Text Available The eye movement system is sensitive to reward. However, whilst the eye movement system is extremely flexible, the extent to which changes to oculomotor behaviour induced by reward paradigms persist beyond the training period or transfer to other oculomotor tasks is unclear. To address these issues we examined the effects of presenting feedback that represented small monetary rewards to spatial locations on the latency of saccadic eye movements, the time-course of learning and extinction of the effects of rewarding saccades on exogenous spatial attention and oculomotor IOR. Reward feedback produced a relative facilitation of saccadic latency in a stimulus driven saccade task which persisted for 3 blocks of extinction trials. However this hemifield-specific effect failed to transfer to peripheral cueing tasks. We conclude that rewarding specific spatial locations is unlikely to induce long-term, systemic changes to the human oculomotor or attention systems.

Contingency learning in alcohol dependence and pathological gambling: learning and unlearning reward contingencies

NARCIS (Netherlands)

Vanes, Lucy D.; van Holst, Ruth J.; Jansen, Jochem M.; van den Brink, Wim; Oosterlaan, Jaap; Goudriaan, Anna E.

2014-01-01

Patients with alcohol dependence (AD) and pathological gambling (PG) are characterized by dysfunctional reward processing and their ability to adapt to alterations of reward contingencies is impaired. However, most neurocognitive tasks investigating reward processing involve a complex mix of

The expression of plasticity-related genes in an acute model of stress is modulated by chronic desipramine in a time-dependent manner within medial prefrontal cortex.

Science.gov (United States)

Nava, Nicoletta; Treccani, Giulia; Müller, Heidi Kaastrup; Popoli, Maurizio; Wegener, Gregers; Elfving, Betina

2017-01-01

It is well established that stress plays a major role in the pathogenesis of neuropsychiatric diseases. Stress-induced alteration of synaptic plasticity has been hypothesized to underlie the morphological changes observed by neuroimaging in psychiatric patients in key regions such as hippocampus and prefrontal cortex (PFC). We have recently shown that a single acute stress exposure produces significant short-term alterations of structural plasticity within medial PFC. These alterations were partially prevented by previous treatment with chronic desipramine (DMI). In the present study we evaluated the effects of acute Foot-shock (FS)-stress and pre-treatment with the traditional antidepressant DMI on the gene expression of key regulators of synaptic plasticity and structure. Expression of Homer, Shank, Spinophilin, Densin-180, and the small RhoGTPase related gene Rac1 and downstream target genes, Limk1, Cofilin1 and Rock1 were investigated 1 day (1d), 7 d and 14d after FS-stress exposure. We found that DMI specifically increases the short-term expression of Spinophilin, as well as Homer and Shank family genes, and that both acute stress and DMI exert significant long-term effects on mRNA levels of genes involved in spine plasticity. These findings support the knowledge that acute FS stress and antidepressant treatment induce both rapid and sustained time-dependent alterations in structural components of synaptic plasticity in rodent medial PFC. Copyright © 2016 Elsevier B.V. and ECNP. All rights reserved.

Reward and Attentional Control in Visual Search

Science.gov (United States)

Anderson, Brian A.; Wampler, Emma K.; Laurent, Patryk A.

2015-01-01

It has long been known that the control of attention in visual search depends both on voluntary, top-down deployment according to context-specific goals, and on involuntary, stimulus-driven capture based on the physical conspicuity of perceptual objects. Recent evidence suggests that pairing target stimuli with reward can modulate the voluntary deployment of attention, but there is little evidence that reward modulates the involuntary deployment of attention to task-irrelevant distractors. We report several experiments that investigate the role of reward learning on attentional control. Each experiment involved a training phase and a test phase. In the training phase, different colors were associated with different amounts of monetary reward. In the test phase, color was not task-relevant and participants searched for a shape singleton; in most experiments no reward was delivered in the test phase. We first show that attentional capture by physically salient distractors is magnified by a previous association with reward. In subsequent experiments we demonstrate that physically inconspicuous stimuli previously associated with reward capture attention persistently during extinction—even several days after training. Furthermore, vulnerability to attentional capture by high-value stimuli is negatively correlated across individuals with working memory capacity and positively correlated with trait impulsivity. An analysis of intertrial effects reveals that value-driven attentional capture is spatially specific. Finally, when reward is delivered at test contingent on the task-relevant shape feature, recent reward history modulates value-driven attentional capture by the irrelevant color feature. The influence of learned value on attention may provide a useful model of clinical syndromes characterized by similar failures of cognitive control, including addiction, attention-deficit/hyperactivity disorder, and obesity. PMID:23437631

Improved SpikeProp for Using Particle Swarm Optimization

Directory of Open Access Journals (Sweden)

Falah Y. H. Ahmed

2013-01-01

Full Text Available A spiking neurons network encodes information in the timing of individual spike times. A novel supervised learning rule for SpikeProp is derived to overcome the discontinuities introduced by the spiking thresholding. This algorithm is based on an error-backpropagation learning rule suited for supervised learning of spiking neurons that use exact spike time coding. The SpikeProp is able to demonstrate the spiking neurons that can perform complex nonlinear classification in fast temporal coding. This study proposes enhancements of SpikeProp learning algorithm for supervised training of spiking networks which can deal with complex patterns. The proposed methods include the SpikeProp particle swarm optimization (PSO and angle driven dependency learning rate. These methods are presented to SpikeProp network for multilayer learning enhancement and weights optimization. Input and output patterns are encoded as spike trains of precisely timed spikes, and the network learns to transform the input trains into target output trains. With these enhancements, our proposed methods outperformed other conventional neural network architectures.

On the Spike Train Variability Characterized by Variance-to-Mean Power Relationship.

Science.gov (United States)

Koyama, Shinsuke

2015-07-01

We propose a statistical method for modeling the non-Poisson variability of spike trains observed in a wide range of brain regions. Central to our approach is the assumption that the variance and the mean of interspike intervals are related by a power function characterized by two parameters: the scale factor and exponent. It is shown that this single assumption allows the variability of spike trains to have an arbitrary scale and various dependencies on the firing rate in the spike count statistics, as well as in the interval statistics, depending on the two parameters of the power function. We also propose a statistical model for spike trains that exhibits the variance-to-mean power relationship. Based on this, a maximum likelihood method is developed for inferring the parameters from rate-modulated spike trains. The proposed method is illustrated on simulated and experimental spike trains.

Prediction-error in the context of real social relationships modulates reward system activity.

Science.gov (United States)

Poore, Joshua C; Pfeifer, Jennifer H; Berkman, Elliot T; Inagaki, Tristen K; Welborn, Benjamin L; Lieberman, Matthew D

2012-01-01

The human reward system is sensitive to both social (e.g., validation) and non-social rewards (e.g., money) and is likely integral for relationship development and reputation building. However, data is sparse on the question of whether implicit social reward processing meaningfully contributes to explicit social representations such as trust and attachment security in pre-existing relationships. This event-related fMRI experiment examined reward system prediction-error activity in response to a potent social reward-social validation-and this activity's relation to both attachment security and trust in the context of real romantic relationships. During the experiment, participants' expectations for their romantic partners' positive regard of them were confirmed (validated) or violated, in either positive or negative directions. Primary analyses were conducted using predefined regions of interest, the locations of which were taken from previously published research. Results indicate that activity for mid-brain and striatal reward system regions of interest was modulated by social reward expectation violation in ways consistent with prior research on reward prediction-error. Additionally, activity in the striatum during viewing of disconfirmatory information was associated with both increases in post-scan reports of attachment anxiety and decreases in post-scan trust, a finding that follows directly from representational models of attachment and trust.

Low and high gamma oscillations in rat ventral striatum have distinct relationships to behavior, reward, and spiking activity on a learned spatial decision task

Directory of Open Access Journals (Sweden)

Matthijs A A Van Der Meer

2009-06-01

Full Text Available Local field potential (LFP oscillations in the brain reflect organization thought to be important for perception, attention, movement, and memory. In the basal ganglia, including dorsal striatum, dysfunctional LFP states are associated with Parkinson’s disease, while in healthy subjects, dorsal striatal LFPs have been linked to decision-making processes. However, LFPs in ventral striatum have been less studied. We report that in rats running a spatial decision task, prominent gamma-50 (45-55 Hz and gamma-80 (70-85 Hz oscillations in ventral striatum had distinct relationships to behavior, task events, and spiking activity. Gamma-50 power increased sharply following reward delivery and before movement initiation, while in contrast, gamma-80 power ramped up gradually to reward locations. Gamma-50 power was low and contained little structure during early learning, but rapidly developed a stable pattern, while gamma-80 power was initially high before returning to a stable level within a similar timeframe. Putative fast-spiking interneurons (FSIs showed phase, firing rate, and coherence relationships with gamma-50 and gamma-80, indicating that the observed LFP patterns are locally relevant. Furthermore, in a number of FSIs such relationships were specific to gamma-50 or gamma-80, suggesting that partially distinct FSI populations mediate the effects of gamma-50 and gamma-80.

Association of contextual cues with morphine reward increases neural and synaptic plasticity in the ventral hippocampus of rats

NARCIS (Netherlands)

Alvandi, M.S.; Bourmpoula, M.; Homberg, J.R.; Fathollahi, Y.

2017-01-01

Drug addiction is associated with aberrant memory and permanent functional changes in neural circuits. It is known that exposure to drugs like morphine is associated with positive emotional states and reward-related memory. However, the underlying mechanisms in terms of neural plasticity in the

Reward salience and risk aversion underlie differential ACC activity in substance dependence.

Science.gov (United States)

Alexander, William H; Fukunaga, Rena; Finn, Peter; Brown, Joshua W

2015-01-01

The medial prefrontal cortex, especially the dorsal anterior cingulate cortex (ACC), has long been implicated in cognitive control and error processing. Although the association between ACC and behavior has been established, it is less clear how ACC contributes to dysfunctional behavior such as substance dependence. Evidence from neuroimaging studies investigating ACC function in substance users is mixed, with some studies showing disengagement of ACC in substance dependent individuals (SDs), while others show increased ACC activity related to substance use. In this study, we investigate ACC function in SDs and healthy individuals performing a change signal task for monetary rewards. Using a priori predictions derived from a recent computational model of ACC, we find that ACC activity differs between SDs and controls in factors related to reward salience and risk aversion between SDs and healthy individuals. Quantitative fits of a computational model to fMRI data reveal significant differences in best fit parameters for reward salience and risk preferences. Specifically, the ACC in SDs shows greater risk aversion, defined as concavity in the utility function, and greater attention to rewards relative to reward omission. Furthermore, across participants risk aversion and reward salience are positively correlated. The results clarify the role that ACC plays in both the reduced sensitivity to omitted rewards and greater reward valuation in SDs. Clinical implications of applying computational modeling in psychiatry are also discussed.

Towards building hybrid biological/in silico neural networks for motor neuroprosthetic control

Directory of Open Access Journals (Sweden)

Mehmet eKocaturk

2015-08-01

Full Text Available In this article, we introduce the Bioinspired Neuroprosthetic Design Environment (BNDE as a practical platform for the development of novel brain machine interface (BMI controllers which are based on spiking model neurons. We built the BNDE around a hard real-time system so that it is capable of creating simulated synapses from extracellularly recorded neurons to model neurons. In order to evaluate the practicality of the BNDE for neuroprosthetic control experiments, a novel, adaptive BMI controller was developed and tested using real-time closed-loop simulations. The present controller consists of two in silico medium spiny neurons which receive simulated synaptic inputs from recorded motor cortical neurons. In the closed-loop simulations, the recordings from the cortical neurons were imitated using an external, hardware-based neural signal synthesizer. By implementing a reward-modulated spike timing-dependent plasticity rule, the controller achieved perfect target reach accuracy for a two target reaching task in one dimensional space. The BNDE combines the flexibility of software-based spiking neural network (SNN simulations with powerful online data visualization tools and is a low-cost, PC-based and all-in-one solution for developing neurally-inspired BMI controllers. We believe the BNDE is the first implementation which is capable of creating hybrid biological/in silico neural networks for motor neuroprosthetic control and utilizes multiple CPU cores for computationally intensive real-time SNN simulations.

Motivation and timing: clues for modeling the reward system.

Science.gov (United States)

Galtress, Tiffany; Marshall, Andrew T; Kirkpatrick, Kimberly

2012-05-01

There is growing evidence that a change in reward magnitude or value alters interval timing, indicating that motivation and timing are not independent processes as was previously believed. The present paper reviews several recent studies, as well as presenting some new evidence with further manipulations of reward value during training vs. testing on a peak procedure. The combined results cannot be accounted for by any of the current psychological timing theories. However, in examining the neural circuitry of the reward system, it is not surprising that motivation has an impact on timing because the motivation/valuation system directly interfaces with the timing system. A new approach is proposed for the development of the next generation of timing models, which utilizes knowledge of the neuroanatomy and neurophysiology of the reward system to guide the development of a neurocomputational model of the reward system. The initial foundation along with heuristics for proceeding with developing such a model is unveiled in an attempt to stimulate new theoretical approaches in the field. Copyright © 2012 Elsevier B.V. All rights reserved.

Motivation and timing: Clues for modeling the reward system

Science.gov (United States)

Galtress, Tiffany; Marshall, Andrew T.; Kirkpatrick, Kimberly

2012-01-01

There is growing evidence that a change in reward magnitude or value alters interval timing, indicating that motivation and timing are not independent processes as was previously believed. The present paper reviews several recent studies, as well as presenting some new evidence with further manipulations of reward value during training vs. testing on a peak procedure. The combined results cannot be accounted for by any of the current psychological timing theories. However, in examining the neural circuitry of the reward system, it is not surprising that motivation has an impact on timing because the motivation/valuation system directly interfaces with the timing system. A new approach is proposed for the development of the next generation of timing models, which utilizes knowledge of the neuroanatomy and neurophysiology of the reward system to guide the development of a neurocomputational model of the reward system. The initial foundation along with heuristics for proceeding with developing such a model is unveiled in an attempt to stimulate new theoretical approaches in the field. PMID:22421220

[Experimental determination of the time-dependent extent of after-burning with reference to possibilities of the plastic surgery reconstruction of 3d degree burns].

Science.gov (United States)

Bäumer, F; Henrich, H A; Ussmüller, J

1986-02-01

The present experiments try to answer the question as to the time-dependent extent of the after-burning process after full-thickness burn (third degree). For an early plastic surgical treatment it was of interest to determine the most early time of escharotomy. The time-dependent spreading of the after-burning area reached its maximum five days after the burn injury. The after-burning area was marked by intravenous injections of Patentblau which caused distinct intravital colouring. Subsequently no further progress could be observed. In the present experiments we suggest this time as the earliest time for plastic covering in case it would be dependent upon the end of the after-burning process.

Stimulus Intensity-dependent Modulations of Hippocampal Long-term Potentiation by Basolateral Amygdala Priming

Directory of Open Access Journals (Sweden)

Zexuan eLi

2012-05-01

Full Text Available There is growing realization that the relationship between memory and stress/emotionality is complicated, and may include both memory enhancing and memory impairing aspects. It has been suggested that the underlying mechanisms involve amygdalar modulation of hippocampal synaptic plasticity, such as long-term potentiation (LTP. We recently reported that while in CA1 basolateral amygdala (BLA priming impaired theta stimulation induced LTP, it enhanced LTP in the dentate gyrus (DG. However, emotional and stressfull experiences were found to activate synaptic plasticity within the BLA, rasing the possibility that BLA modulation of other brain regions may be altered as well, as it may depend on the way the BLA is activated or is responding. In previous studies BLA priming stimulation was relatively weak (1V, 50 µs pulse duration. In the present study we assessed the effects of two stronger levels of BLA priming stimulation (1V or 2V, 100 µs pulse duration on LTP induction in hippocampal DG and CA1, in anesthetized rats. Results show that 1V-BLA priming stimulation enhanced but 2V-BLA priming stimulation impaired DG LTP; however, both levels of BLA priming stimulation impaired CA1 LTP, suggesting that modulation of hippocampal synaptic plasticity by amygdala is dependent on the degree of amygdala activation. These findings suggest that plasticity induced within the amygdala, by stressful experiences induces a form of metaplasticity that would alter the way the amygdala may modulate memory-related processes in other brain areas, such as the hippocampus.

Prediction-error in the context of real social relationships modulates reward system activity

Directory of Open Access Journals (Sweden)

Joshua ePoore

2012-08-01

Full Text Available The human reward system is sensitive to both social (e.g., validation and non-social rewards (e.g., money and is likely integral for relationship development and reputation building. However, data is sparse on the question of whether implicit social reward processing meaningfully contributes to explicit social representations such as trust and attachment security in pre-existing relationships. This event-related fMRI experiment examined reward system prediction-error activity in response to a potent social reward—social validation—and this activity’s relation to both attachment security and trust in the context of real romantic relationships. During the experiment, participants’ expectations for their romantic partners’ positive regard of them were confirmed (validated or violated, in either positive or negative directions. Primary analyses were conducted using predefined regions of interest, the locations of which were taken from previously published research. Results indicate that activity for mid-brain and striatal reward system regions of interest was modulated by social reward expectation violation in ways consistent with prior research on reward prediction-error. Additionally, activity in the striatum during viewing of disconfirmatory information was associated with both increases in post-scan reports of attachment anxiety and decreases in post-scan trust, a finding that follows directly from representational models of attachment and trust.

CCR5 is a suppressor for cortical plasticity and hippocampal learning and memory.

Science.gov (United States)

Zhou, Miou; Greenhill, Stuart; Huang, Shan; Silva, Tawnie K; Sano, Yoshitake; Wu, Shumin; Cai, Ying; Nagaoka, Yoshiko; Sehgal, Megha; Cai, Denise J; Lee, Yong-Seok; Fox, Kevin; Silva, Alcino J

2016-12-20

Although the role of CCR5 in immunity and in HIV infection has been studied widely, its role in neuronal plasticity, learning and memory is not understood. Here, we report that decreasing the function of CCR5 increases MAPK/CREB signaling, long-term potentiation (LTP), and hippocampus-dependent memory in mice, while neuronal CCR5 overexpression caused memory deficits. Decreasing CCR5 function in mouse barrel cortex also resulted in enhanced spike timing dependent plasticity and consequently, dramatically accelerated experience-dependent plasticity. These results suggest that CCR5 is a powerful suppressor for plasticity and memory, and CCR5 over-activation by viral proteins may contribute to HIV-associated cognitive deficits. Consistent with this hypothesis, the HIV V3 peptide caused LTP, signaling and memory deficits that were prevented by Ccr5 knockout or knockdown. Overall, our results demonstrate that CCR5 plays an important role in neuroplasticity, learning and memory, and indicate that CCR5 has a role in the cognitive deficits caused by HIV.

The phenotypic plasticity of developmental modules

Directory of Open Access Journals (Sweden)

Aabha I. Sharma

2016-08-01

Full Text Available Abstract Background Organisms develop and evolve in a modular fashion, but how individual modules interact with the environment remains poorly understood. Phenotypically plastic traits are often under selection, and studies are needed to address how traits respond to the environment in a modular fashion. In this study, tissue-specific plasticity of melanic spots was examined in the large milkweed bug, Oncopeltus fasciatus. Results Although the size of the abdominal melanic bands varied according to rearing temperatures, wing melanic bands were more robust. To explore the regulation of abdominal pigmentation plasticity, candidate genes involved in abdominal melanic spot patterning and biosynthesis of melanin were analyzed. While the knockdown of dopa decarboxylase (Ddc led to lighter pigmentation in both the wings and the abdomen, the shape of the melanic elements remained unaffected. Although the knockdown of Abdominal-B (Abd-B partially phenocopied the low-temperature phenotype, the abdominal bands were still sensitive to temperature shifts. These observations suggest that regulators downstream of Abd-B but upstream of DDC are responsible for the temperature response of the abdomen. Ablation of wings led to the regeneration of a smaller wing with reduced melanic bands that were shifted proximally. In addition, the knockdown of the Wnt signaling nuclear effector genes, armadillo 1 and armadillo 2, altered both the melanic bands and the wing shape. Thus, the pleiotropic effects of Wnt signaling may constrain the amount of plasticity in wing melanic bands. Conclusions We propose that when traits are regulated by distinct pre-patterning mechanisms, they can respond to the environment in a modular fashion, whereas when the environment impacts developmental regulators that are shared between different modules, phenotypic plasticity can manifest as a developmentally integrated system.

Monetary reward speeds up voluntary saccades.

Science.gov (United States)

Chen, Lewis L; Chen, Y Mark; Zhou, Wu; Mustain, William D

2014-01-01

Past studies have shown that reward contingency is critical for sensorimotor learning, and reward expectation speeds up saccades in animals. Whether monetary reward speeds up saccades in human remains unknown. Here we addressed this issue by employing a conditional saccade task, in which human subjects performed a series of non-reflexive, visually-guided horizontal saccades. The subjects were (or were not) financially compensated for making a saccade in response to a centrally-displayed visual congruent (or incongruent) stimulus. Reward modulation of saccadic velocities was quantified independently of the amplitude-velocity coupling. We found that reward expectation significantly sped up voluntary saccades up to 30°/s, and the reward modulation was consistent across tests. These findings suggest that monetary reward speeds up saccades in human in a fashion analogous to how juice reward sped up saccades in monkeys. We further noticed that the idiosyncratic nasal-temporal velocity asymmetry was highly consistent regardless of test order, and its magnitude was not correlated with the magnitude of reward modulation. This suggests that reward modulation and the intrinsic velocity asymmetry may be governed by separate mechanisms that regulate saccade generation.

A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses.

Science.gov (United States)

Qiao, Ning; Mostafa, Hesham; Corradi, Federico; Osswald, Marc; Stefanini, Fabio; Sumislawska, Dora; Indiveri, Giacomo

2015-01-01

Implementing compact, low-power artificial neural processing systems with real-time on-line learning abilities is still an open challenge. In this paper we present a full-custom mixed-signal VLSI device with neuromorphic learning circuits that emulate the biophysics of real spiking neurons and dynamic synapses for exploring the properties of computational neuroscience models and for building brain-inspired computing systems. The proposed architecture allows the on-chip configuration of a wide range of network connectivities, including recurrent and deep networks, with short-term and long-term plasticity. The device comprises 128 K analog synapse and 256 neuron circuits with biologically plausible dynamics and bi-stable spike-based plasticity mechanisms that endow it with on-line learning abilities. In addition to the analog circuits, the device comprises also asynchronous digital logic circuits for setting different synapse and neuron properties as well as different network configurations. This prototype device, fabricated using a 180 nm 1P6M CMOS process, occupies an area of 51.4 mm(2), and consumes approximately 4 mW for typical experiments, for example involving attractor networks. Here we describe the details of the overall architecture and of the individual circuits and present experimental results that showcase its potential. By supporting a wide range of cortical-like computational modules comprising plasticity mechanisms, this device will enable the realization of intelligent autonomous systems with on-line learning capabilities.

Multichannel interictal spike activity detection using time-frequency entropy measure.

Science.gov (United States)

Thanaraj, Palani; Parvathavarthini, B

2017-06-01

Localization of interictal spikes is an important clinical step in the pre-surgical assessment of pharmacoresistant epileptic patients. The manual selection of interictal spike periods is cumbersome and involves a considerable amount of analysis workload for the physician. The primary focus of this paper is to automate the detection of interictal spikes for clinical applications in epilepsy localization. The epilepsy localization procedure involves detection of spikes in a multichannel EEG epoch. Therefore, a multichannel Time-Frequency (T-F) entropy measure is proposed to extract features related to the interictal spike activity. Least squares support vector machine is used to train the proposed feature to classify the EEG epochs as either normal or interictal spike period. The proposed T-F entropy measure, when validated with epilepsy dataset of 15 patients, shows an interictal spike classification accuracy of 91.20%, sensitivity of 100% and specificity of 84.23%. Moreover, the area under the curve of Receiver Operating Characteristics plot of 0.9339 shows the superior classification performance of the proposed T-F entropy measure. The results of this paper show a good spike detection accuracy without any prior information about the spike morphology.

Association of contextual cues with morphine reward increases neural and synaptic plasticity in the ventral hippocampus of rats.

Science.gov (United States)

Alvandi, Mina Sadighi; Bourmpoula, Maria; Homberg, Judith R; Fathollahi, Yaghoub

2017-11-01

Drug addiction is associated with aberrant memory and permanent functional changes in neural circuits. It is known that exposure to drugs like morphine is associated with positive emotional states and reward-related memory. However, the underlying mechanisms in terms of neural plasticity in the ventral hippocampus, a region involved in associative memory and emotional behaviors, are not fully understood. Therefore, we measured adult neurogenesis, dendritic spine density and brain-derived neurotrophic factor (BDNF) and TrkB mRNA expression as parameters for synaptic plasticity in the ventral hippocampus. Male Sprague Dawley rats were subjected to the CPP (conditioned place preference) paradigm and received 10 mg/kg morphine. Half of the rats were used to evaluate neurogenesis by immunohistochemical markers Ki67 and doublecortin (DCX). The other half was used for Golgi staining to measure spine density and real-time quantitative reverse transcription-polymerase chain reaction to assess BDNF/TrkB expression levels. We found that morphine-treated rats exhibited more place conditioning as compared with saline-treated rats and animals that were exposed to the CPP without any injections. Locomotor activity did not change significantly. Morphine-induced CPP significantly increased the number of Ki67 and DCX-labeled cells in the ventral dentate gyrus. Additionally, we found increased dendritic spine density in both CA1 and dentate gyrus and an enhancement of BDNF/TrkB mRNA levels in the whole ventral hippocampus. Ki67, DCX and spine density were significantly correlated with CPP scores. In conclusion, we show that morphine-induced reward-related memory is associated with neural and synaptic plasticity changes in the ventral hippocampus. Such neural changes could underlie context-induced drug relapse. © 2017 Society for the Study of Addiction.

Effects of dopamine and glutamate on synaptic plasticity: a computational modeling approach for drug abuse as comorbidity in mood disorders.

Science.gov (United States)

Qi, Z; Kikuchi, S; Tretter, F; Voit, E O

2011-05-01

Major depressive disorder (MDD) affects about 16% of the general population and is a leading cause of death in the United States and around the world. Aggravating the situation is the fact that "drug use disorders" are highly comorbid in MDD patients, and VICE VERSA. Drug use and MDD share a common component, the dopamine system, which is critical in many motivation and reward processes, as well as in the regulation of stress responses in MDD. A potentiating mechanism in drug use disorders appears to be synaptic plasticity, which is regulated by dopamine transmission. In this article, we describe a computational model of the synaptic plasticity of GABAergic medium spiny neurons in the nucleus accumbens, which is critical in the reward system. The model accounts for effects of both dopamine and glutamate transmission. Model simulations show that GABAergic medium spiny neurons tend to respond to dopamine stimuli with synaptic potentiation and to glutamate signals with synaptic depression. Concurrent dopamine and glutamate signals cause various types of synaptic plasticity, depending on input scenarios. Interestingly, the model shows that a single 0.5 mg/kg dose of amphetamine can cause synaptic potentiation for over 2 h, a phenomenon that makes synaptic plasticity of medium spiny neurons behave quasi as a bistable system. The model also identifies mechanisms that could potentially be critical to correcting modifications of synaptic plasticity caused by drugs in MDD patients. An example is the feedback loop between protein kinase A, phosphodiesterase, and the second messenger cAMP in the postsynapse. Since reward mechanisms activated by psychostimulants could be crucial in establishing addiction comorbidity in patients with MDD, this model might become an aid for identifying and targeting specific modules within the reward system and lead to a better understanding and potential treatment of comorbid drug use disorders in MDD. © Georg Thieme Verlag KG Stuttgart · New

Reward-dependent learning in neuronal networks for planning and decision making.

Science.gov (United States)

Dehaene, S; Changeux, J P

2000-01-01

Neuronal network models have been proposed for the organization of evaluation and decision processes in prefrontal circuitry and their putative neuronal and molecular bases. The models all include an implementation and simulation of an elementary reward mechanism. Their central hypothesis is that tentative rules of behavior, which are coded by clusters of active neurons in prefrontal cortex, are selected or rejected based on an evaluation by this reward signal, which may be conveyed, for instance, by the mesencephalic dopaminergic neurons with which the prefrontal cortex is densely interconnected. At the molecular level, the reward signal is postulated to be a neurotransmitter such as dopamine, which exerts a global modulatory action on prefrontal synaptic efficacies, either via volume transmission or via targeted synaptic triads. Negative reinforcement has the effect of destabilizing the currently active rule-coding clusters; subsequently, spontaneous activity varies again from one cluster to another, giving the organism the chance to discover and learn a new rule. Thus, reward signals function as effective selection signals that either maintain or suppress currently active prefrontal representations as a function of their current adequacy. Simulations of this variation-selection have successfully accounted for the main features of several major tasks that depend on prefrontal cortex integrity, such as the delayed-response test, the Wisconsin card sorting test, the Tower of London test and the Stroop test. For the more complex tasks, we have found it necessary to supplement the external reward input with a second mechanism that supplies an internal reward; it consists of an auto-evaluation loop which short-circuits the reward input from the exterior. This allows for an internal evaluation of covert motor intentions without actualizing them as behaviors, by simply testing them covertly by comparison with memorized former experiences. This element of architecture

Enriched encoding: reward motivation organizes cortical networks for hippocampal detection of unexpected events.

Science.gov (United States)

Murty, Vishnu P; Adcock, R Alison

2014-08-01

Learning how to obtain rewards requires learning about their contexts and likely causes. How do long-term memory mechanisms balance the need to represent potential determinants of reward outcomes with the computational burden of an over-inclusive memory? One solution would be to enhance memory for salient events that occur during reward anticipation, because all such events are potential determinants of reward. We tested whether reward motivation enhances encoding of salient events like expectancy violations. During functional magnetic resonance imaging, participants performed a reaction-time task in which goal-irrelevant expectancy violations were encountered during states of high- or low-reward motivation. Motivation amplified hippocampal activation to and declarative memory for expectancy violations. Connectivity of the ventral tegmental area (VTA) with medial prefrontal, ventrolateral prefrontal, and visual cortices preceded and predicted this increase in hippocampal sensitivity. These findings elucidate a novel mechanism whereby reward motivation can enhance hippocampus-dependent memory: anticipatory VTA-cortical-hippocampal interactions. Further, the findings integrate literatures on dopaminergic neuromodulation of prefrontal function and hippocampus-dependent memory. We conclude that during reward motivation, VTA modulation induces distributed neural changes that amplify hippocampal signals and records of expectancy violations to improve predictions-a potentially unique contribution of the hippocampus to reward learning. © The Author 2013. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.

Modulation of synaptic plasticity by stress hormone associates with plastic alteration of synaptic NMDA receptor in the adult hippocampus.

Directory of Open Access Journals (Sweden)

Yiu Chung Tse

Full Text Available Stress exerts a profound impact on learning and memory, in part, through the actions of adrenal corticosterone (CORT on synaptic plasticity, a cellular model of learning and memory. Increasing findings suggest that CORT exerts its impact on synaptic plasticity by altering the functional properties of glutamate receptors, which include changes in the motility and function of α-amino-3-hydroxy-5-methylisoxazole-4-propionic acid subtype of glutamate receptor (AMPAR that are responsible for the expression of synaptic plasticity. Here we provide evidence that CORT could also regulate synaptic plasticity by modulating the function of synaptic N-methyl-D-aspartate receptors (NMDARs, which mediate the induction of synaptic plasticity. We found that stress level CORT applied to adult rat hippocampal slices potentiated evoked NMDAR-mediated synaptic responses within 30 min. Surprisingly, following this fast-onset change, we observed a slow-onset (>1 hour after termination of CORT exposure increase in synaptic expression of GluN2A-containing NMDARs. To investigate the consequences of the distinct fast- and slow-onset modulation of NMDARs for synaptic plasticity, we examined the formation of long-term potentiation (LTP and long-term depression (LTD within relevant time windows. Paralleling the increased NMDAR function, both LTP and LTD were facilitated during CORT treatment. However, 1-2 hours after CORT treatment when synaptic expression of GluN2A-containing NMDARs is increased, bidirectional plasticity was no longer facilitated. Our findings reveal the remarkable plasticity of NMDARs in the adult hippocampus in response to CORT. CORT-mediated slow-onset increase in GluN2A in hippocampal synapses could be a homeostatic mechanism to normalize synaptic plasticity following fast-onset stress-induced facilitation.

Modulation of Food Reward by Endocrine and Environmental Factors: Update and Perspective.

Science.gov (United States)

Figlewicz, Dianne P

2015-01-01

Palatable foods are frequently high in energy density. Chronic consumption of high-energy density foods can contribute to the development of cardiometabolic pathology including obesity, diabetes, and cardiovascular disease. This article reviews the contributions of extrinsic and intrinsic factors that influence the reward components of food intake. A narrative review was conducted to determine the behavioral and central nervous system (CNS) related processes involved in the reward components of high-energy density food intake. The rewarding aspects of food, particularly palatable and preferred foods, are regulated by CNS circuitry. Overlaying this regulation is modulation by intrinsic endocrine systems and metabolic hormones relating to energy homeostasis, developmental stage, or gender. It is now recognized that extrinsic or environmental factors, including ambient diet composition and the provocation of stress or anxiety, also contribute substantially to the expression of food reward behaviors such as motivation for, and seeking of, preferred foods. High-energy density food intake is influenced by both physiological and pathophysiological processes. Contextual, behavioral, and psychological factors and CNS-related processes represent potential targets for multiple types of therapeutic intervention.

Differential reward coding in the subdivisions of the primate caudate during an oculomotor task.

Science.gov (United States)

Nakamura, Kae; Santos, Gustavo S; Matsuzaki, Ryuichi; Nakahara, Hiroyuki

2012-11-07

The basal ganglia play a pivotal role in reward-oriented behavior. The striatum, an input channel of the basal ganglia, is composed of subdivisions that are topographically connected with different cortical and subcortical areas. To test whether reward information is differentially processed in the different parts of the striatum, we compared reward-related neuronal activity along the dorsolateral-ventromedial axis in the caudate nucleus of monkeys performing an asymmetrically rewarded oculomotor task. In a given block, a target in one position was associated with a large reward, whereas the other target was associated with a small reward. The target position-reward value contingency was switched between blocks. We found the following: (1) activity that reflected the block-wise reward contingency emerged before the appearance of a visual target, and it was more prevalent in the dorsal, rather than central and ventral, caudate; (2) activity that was positively related to the reward size of the current trial was evident, especially after reward delivery, and it was more prevalent in the ventral and central, rather than dorsal, caudate; and (3) activity that was modulated by the memory of the outcomes of the previous trials was evident in the dorsal and central caudate. This multiple reward information, together with the target-direction information, was represented primarily by individual caudate neurons, and the different reward information was represented in caudate subpopulations with distinct electrophysiological properties, e.g., baseline firing and spike width. These results suggest parallel processing of different reward information by the basal ganglia subdivisions defined by extrinsic connections and intrinsic properties.

The globus pallidus sends reward-related signals to the lateral habenula.

Science.gov (United States)

Hong, Simon; Hikosaka, Okihide

2008-11-26

As a major output station of the basal ganglia, the globus pallidus internal segment (GPi) projects to the thalamus and brainstem nuclei thereby controlling motor behavior. A less well known fact is that the GPi also projects to the lateral habenula (LHb) which is often associated with the limbic system. Using the monkey performing a saccade task with positionally biased reward outcomes, we found that antidromically identified LHb-projecting neurons were distributed mainly in the dorsal and ventral borders of the GPi and that their activity was strongly modulated by expected reward outcomes. A majority of them were excited by the no-reward-predicting target and inhibited by the reward-predicting target. These reward-dependent modulations were similar to those in LHb neurons but started earlier than those in LHb neurons. These results suggest that GPi may initiate reward-related signals through its effects on the LHb, which then influences the dopaminergic and serotonergic systems.

Modulation of Hippocampal Neural Plasticity by Glucose-Related Signaling

Directory of Open Access Journals (Sweden)

Marco Mainardi

2015-01-01

Full Text Available Hormones and peptides involved in glucose homeostasis are emerging as important modulators of neural plasticity. In this regard, increasing evidence shows that molecules such as insulin, insulin-like growth factor-I, glucagon-like peptide-1, and ghrelin impact on the function of the hippocampus, which is a key area for learning and memory. Indeed, all these factors affect fundamental hippocampal properties including synaptic plasticity (i.e., synapse potentiation and depression, structural plasticity (i.e., dynamics of dendritic spines, and adult neurogenesis, thus leading to modifications in cognitive performance. Here, we review the main mechanisms underlying the effects of glucose metabolism on hippocampal physiology. In particular, we discuss the role of these signals in the modulation of cognitive functions and their potential implications in dysmetabolism-related cognitive decline.

Reward speeds up and increases consistency of visual selective attention: a lifespan comparison.

Science.gov (United States)

Störmer, Viola; Eppinger, Ben; Li, Shu-Chen

2014-06-01

Children and older adults often show less favorable reward-based learning and decision making, relative to younger adults. It is unknown, however, whether reward-based processes that influence relatively early perceptual and attentional processes show similar lifespan differences. In this study, we investigated whether stimulus-reward associations affect selective visual attention differently across the human lifespan. Children, adolescents, younger adults, and older adults performed a visual search task in which the target colors were associated with either high or low monetary rewards. We discovered that high reward value speeded up response times across all four age groups, indicating that reward modulates attentional selection across the lifespan. This speed-up in response time was largest in younger adults, relative to the other three age groups. Furthermore, only younger adults benefited from high reward value in increasing response consistency (i.e., reduction of trial-by-trial reaction time variability). Our findings suggest that reward-based modulations of relatively early and implicit perceptual and attentional processes are operative across the lifespan, and the effects appear to be greater in adulthood. The age-specific effect of reward on reducing intraindividual response variability in younger adults likely reflects mechanisms underlying the development and aging of reward processing, such as lifespan age differences in the efficacy of dopaminergic modulation. Overall, the present results indicate that reward shapes visual perception across different age groups by biasing attention to motivationally salient events.

A Re-configurable On-line Learning Spiking Neuromorphic Processor comprising 256 neurons and 128K synapses

Directory of Open Access Journals (Sweden)

Ning eQiao

2015-04-01

Full Text Available Implementing compact, low-power artificial neural processing systems with real-time on-line learning abilities is still an open challenge. In this paper we present a full-custom mixed-signal VLSI device with neuromorphic learning circuits that emulate the biophysics of real spiking neurons and dynamic synapses for exploring the properties of computational neuroscience models and for building brain-inspired computing systems. The proposed architecture allows the on-chip configuration of a wide range of network connectivities, including recurrent and deep networks with short-term and long-term plasticity. The device comprises 128 K analog synapse and 256 neuron circuits with biologically plausible dynamics and bi-stable spike-based plasticity mechanisms that endow it with on-line learning abilities. In addition to the analog circuits, the device comprises also asynchronous digital logic circuits for setting different synapse and neuron properties as well as different network configurations. This prototype device, fabricated using a 180 nm 1P6M CMOS process, occupies an area of 51.4 mm 2 , and consumes approximately 4 mW for typical experiments, for example involving attractor networks. Here we describe the details of the overall architecture and of the individual circuits and present experimental results that showcase its potential. By supporting a wide range of cortical-like computational modules comprising plasticity mechanisms, this device will enable the realization of intelligent autonomous systems with on-line learning capabilities.

The endocannabinoid system in brain reward processes.

Science.gov (United States)

Solinas, M; Goldberg, S R; Piomelli, D

2008-05-01

Food, drugs and brain stimulation can serve as strong rewarding stimuli and are all believed to activate common brain circuits that evolved in mammals to favour fitness and survival. For decades, endogenous dopaminergic and opioid systems have been considered the most important systems in mediating brain reward processes. Recent evidence suggests that the endogenous cannabinoid (endocannabinoid) system also has an important role in signalling of rewarding events. First, CB(1) receptors are found in brain areas involved in reward processes, such as the dopaminergic mesolimbic system. Second, activation of CB(1) receptors by plant-derived, synthetic or endogenous CB(1) receptor agonists stimulates dopaminergic neurotransmission, produces rewarding effects and increases rewarding effects of abused drugs and food. Third, pharmacological or genetic blockade of CB(1) receptors prevents activation of dopaminergic neurotransmission by several addictive drugs and reduces rewarding effects of food and these drugs. Fourth, brain levels of the endocannabinoids anandamide and 2-arachidonoylglycerol are altered by activation of reward processes. However, the intrinsic activity of the endocannabinoid system does not appear to play a facilitatory role in brain stimulation reward and some evidence suggests it may even oppose it. The influence of the endocannabinoid system on brain reward processes may depend on the degree of activation of the different brain areas involved and might represent a mechanism for fine-tuning dopaminergic activity. Although involvement of the various components of the endocannabinoid system may differ depending on the type of rewarding event investigated, this system appears to play a major role in modulating reward processes.

Spike and burst coding in thalamocortical relay cells.

Directory of Open Access Journals (Sweden)

Fleur Zeldenrust

2018-02-01

Full Text Available Mammalian thalamocortical relay (TCR neurons switch their firing activity between a tonic spiking and a bursting regime. In a combined experimental and computational study, we investigated the features in the input signal that single spikes and bursts in the output spike train represent and how this code is influenced by the membrane voltage state of the neuron. Identical frozen Gaussian noise current traces were injected into TCR neurons in rat brain slices as well as in a validated three-compartment TCR model cell. The resulting membrane voltage traces and spike trains were analyzed by calculating the coherence and impedance. Reverse correlation techniques gave the Event-Triggered Average (ETA and the Event-Triggered Covariance (ETC. This demonstrated that the feature selectivity started relatively long before the events (up to 300 ms and showed a clear distinction between spikes (selective for fluctuations and bursts (selective for integration. The model cell was fine-tuned to mimic the frozen noise initiated spike and burst responses to within experimental accuracy, especially for the mixed mode regimes. The information content carried by the various types of events in the signal as well as by the whole signal was calculated. Bursts phase-lock to and transfer information at lower frequencies than single spikes. On depolarization the neuron transits smoothly from the predominantly bursting regime to a spiking regime, in which it is more sensitive to high-frequency fluctuations. The model was then used to elucidate properties that could not be assessed experimentally, in particular the role of two important subthreshold voltage-dependent currents: the low threshold activated calcium current (IT and the cyclic nucleotide modulated h current (Ih. The ETAs of those currents and their underlying activation/inactivation states not only explained the state dependence of the firing regime but also the long-lasting concerted dynamic action of the two

The Two Faces of Social Interaction Reward in Animal Models of Drug Dependence.

Science.gov (United States)

El Rawas, Rana; Saria, Alois

2016-03-01

Drug dependence is a serious health and social problem. Social factors can modify vulnerability to developing drug dependence, acting as risk factors or protective factors. Whereas stress and peer environment that encourage substance use may increase drug taking, strong attachments between family members and peer environment that do not experience drug use may protect against drug taking and, ultimately, drug dependence. The rewarding effects of drug abuse and social interaction can be evaluated using animal models. In this review we focus on evaluating social interaction reward in the conditioned place preference paradigm. We give an overview of how social interaction, if made available within the drug context, may facilitate, promote and interact with the drug's effects. However, social interaction, if offered alternatively outside the drug context, may have pronounced protective effects against drug abuse and relapse. We also address the importance of the weight difference parameter between the social partners in determining the positive or "agonistic" versus the hostile or "antagonistic" social interaction. We conclude that understanding social interaction reward and its subsequent effects on drug reward is sorely needed for therapeutic interventions against drug dependence.

P3-9: Roles of Subthreshold LFP Induced by Receptive Field Surround for Response Modulation in Monkey V1

Directory of Open Access Journals (Sweden)

Kayeon Kim

2012-10-01

Full Text Available A focal stimulus outside the receptive field robustly induces LFP change, while the same stimulus evokes no spike activity. We determined how this subthreshold LFP change interacted with spike response to the RF stimulus. Specifically, we sequentially presented two identical Gabor stimuli with a variable stimulus onset asynchrony (SOA; the first one (S1 was presented outside RF inducing a subthreshold LFP change, and the second one (S2 was subsequently presented within RF generating a spiking response. This enabled us to manipulate the temporal relation between subthreshold LFP and evoked spike activity and to determine whether subthreshold LFP contributed to modulation of spike activity in a SOA-dependent manner. We found that the subthreshold LFP propagated a considerably long distance, estimated to be more than 10 mm of cortical distance. The cross-correlation between the time course of subthreshold LFP and the pattern of SOA-dependency of spike activity was significant. These results indicate that signal integration is farther beyond the RF than previously estimated based on spike-triggered average, and suggest that subthreshold LFP modulate spike activity in a SOA-dependent manner.

The Timing Effects of Reward, Business Longevity, and Involvement on Consumers’ Responses to a Reward Program

Directory of Open Access Journals (Sweden)

Badri Munir Sukoco

2015-06-01

Full Text Available Managers could elicit customers’ repeat purchase behavior through a well-designed reward program. This study examines two extrinsic cues - business longevity and timing effects of reward – to determine the consumers’ perceived risk and intention to participate in this kind of program. Moreover, this study discusses how different levels of involvement might interact with these two cues. An experiment with a 2 (business longevity: long vs. short x 2 (timing of reward: delayed vs. immediate x 2 (involvement: high vs. low between-subject factorial design is conducted to validate the proposed research hypotheses. The results show that an immediate reward offered by an older, more established, firm for a highly-involved product, make loyalty programs less risky and consequently attract consumers to participate. Interestingly, immediate rewards that are offered by older firms for a product that customers are less involved in has the opposite effects. Managerial and academic implications are further presented in this study.

Plasticity in memristive devices for Spiking Neural Networks

Directory of Open Access Journals (Sweden)

Sylvain eSaïghi

2015-03-01

Full Text Available Memristive devices present a new device technology allowing for the realization of compact nonvolatile memories. Some of them are already in the process of industrialization. Additionally, they exhibit complex multilevel and plastic behaviors, which make them good candidates for the implementation of artificial synapses in neuromorphic engineering. However, memristive effects rely on diverse physical mechanisms, and their plastic behaviors differ strongly from one technology to another. Here, we present measurements performed on different memristive devices and the opportunities that they provide. We show that they can be used to implement different learning rules whose properties emerge directly from device physics: real time or accelerated operation, deterministic or stochastic behavior, long term or short term plasticity. We then discuss how such devices might be integrated into a complete architecture. These results highlight that there is no unique way to exploit memristive devices in neuromorphic systems. Understanding and embracing device physics is the key for their optimal use.

Enhancement of signal sensitivity in a heterogeneous neural network refined from synaptic plasticity

Energy Technology Data Exchange (ETDEWEB)

Li Xiumin; Small, Michael, E-mail: ensmall@polyu.edu.h, E-mail: 07901216r@eie.polyu.edu.h [Department of Electronic and Information Engineering, Hong Kong Polytechnic University, Hung Hom, Kowloon (Hong Kong)

2010-08-15

Long-term synaptic plasticity induced by neural activity is of great importance in informing the formation of neural connectivity and the development of the nervous system. It is reasonable to consider self-organized neural networks instead of prior imposition of a specific topology. In this paper, we propose a novel network evolved from two stages of the learning process, which are respectively guided by two experimentally observed synaptic plasticity rules, i.e. the spike-timing-dependent plasticity (STDP) mechanism and the burst-timing-dependent plasticity (BTDP) mechanism. Due to the existence of heterogeneity in neurons that exhibit different degrees of excitability, a two-level hierarchical structure is obtained after the synaptic refinement. This self-organized network shows higher sensitivity to afferent current injection compared with alternative archetypal networks with different neural connectivity. Statistical analysis also demonstrates that it has the small-world properties of small shortest path length and high clustering coefficients. Thus the selectively refined connectivity enhances the ability of neuronal communications and improves the efficiency of signal transmission in the network.

Enhancement of signal sensitivity in a heterogeneous neural network refined from synaptic plasticity

International Nuclear Information System (INIS)

Li Xiumin; Small, Michael

2010-01-01

Long-term synaptic plasticity induced by neural activity is of great importance in informing the formation of neural connectivity and the development of the nervous system. It is reasonable to consider self-organized neural networks instead of prior imposition of a specific topology. In this paper, we propose a novel network evolved from two stages of the learning process, which are respectively guided by two experimentally observed synaptic plasticity rules, i.e. the spike-timing-dependent plasticity (STDP) mechanism and the burst-timing-dependent plasticity (BTDP) mechanism. Due to the existence of heterogeneity in neurons that exhibit different degrees of excitability, a two-level hierarchical structure is obtained after the synaptic refinement. This self-organized network shows higher sensitivity to afferent current injection compared with alternative archetypal networks with different neural connectivity. Statistical analysis also demonstrates that it has the small-world properties of small shortest path length and high clustering coefficients. Thus the selectively refined connectivity enhances the ability of neuronal communications and improves the efficiency of signal transmission in the network.

Synaptic Plasticity and Spike Synchronisation in Neuronal Networks

Science.gov (United States)

Borges, Rafael R.; Borges, Fernando S.; Lameu, Ewandson L.; Protachevicz, Paulo R.; Iarosz, Kelly C.; Caldas, Iberê L.; Viana, Ricardo L.; Macau, Elbert E. N.; Baptista, Murilo S.; Grebogi, Celso; Batista, Antonio M.

2017-12-01

Brain plasticity, also known as neuroplasticity, is a fundamental mechanism of neuronal adaptation in response to changes in the environment or due to brain injury. In this review, we show our results about the effects of synaptic plasticity on neuronal networks composed by Hodgkin-Huxley neurons. We show that the final topology of the evolved network depends crucially on the ratio between the strengths of the inhibitory and excitatory synapses. Excitation of the same order of inhibition revels an evolved network that presents the rich-club phenomenon, well known to exist in the brain. For initial networks with considerably larger inhibitory strengths, we observe the emergence of a complex evolved topology, where neurons sparsely connected to other neurons, also a typical topology of the brain. The presence of noise enhances the strength of both types of synapses, but if the initial network has synapses of both natures with similar strengths. Finally, we show how the synchronous behaviour of the evolved network will reflect its evolved topology.

Persistence and storage of activity patterns in spiking recurrent cortical networks: modulation of sigmoid signals by after-hyperpolarization currents and acetylcholine.

Science.gov (United States)

Palma, Jesse; Grossberg, Stephen; Versace, Massimiliano

2012-01-01

Many cortical networks contain recurrent architectures that transform input patterns before storing them in short-term memory (STM). Theorems in the 1970's showed how feedback signal functions in rate-based recurrent on-center off-surround networks control this process. A sigmoid signal function induces a quenching threshold below which inputs are suppressed as noise and above which they are contrast-enhanced before pattern storage. This article describes how changes in feedback signaling, neuromodulation, and recurrent connectivity may alter pattern processing in recurrent on-center off-surround networks of spiking neurons. In spiking neurons, fast, medium, and slow after-hyperpolarization (AHP) currents control sigmoid signal threshold and slope. Modulation of AHP currents by acetylcholine (ACh) can change sigmoid shape and, with it, network dynamics. For example, decreasing signal function threshold and increasing slope can lengthen the persistence of a partially contrast-enhanced pattern, increase the number of active cells stored in STM, or, if connectivity is distance-dependent, cause cell activities to cluster. These results clarify how cholinergic modulation by the basal forebrain may alter the vigilance of category learning circuits, and thus their sensitivity to predictive mismatches, thereby controlling whether learned categories code concrete or abstract features, as predicted by Adaptive Resonance Theory. The analysis includes global, distance-dependent, and interneuron-mediated circuits. With an appropriate degree of recurrent excitation and inhibition, spiking networks maintain a partially contrast-enhanced pattern for 800 ms or longer after stimuli offset, then resolve to no stored pattern, or to winner-take-all (WTA) stored patterns with one or multiple winners. Strengthening inhibition prolongs a partially contrast-enhanced pattern by slowing the transition to stability, while strengthening excitation causes more winners when the network

Persistence and storage of activity patterns in spiking recurrent cortical networks:Modulation of sigmoid signals by after-hyperpolarization currents and acetylcholine

Directory of Open Access Journals (Sweden)

Jesse ePalma

2012-06-01

Full Text Available Many cortical networks contain recurrent architectures that transform input patterns before storing them in short-term memory (STM. Theorems in the 1970’s showed how feedback signal functions in rate-based recurrent on-center off-surround networks control this process. A sigmoid signal function induces a quenching threshold below which inputs are suppressed as noise and above which they are contrast-enhanced before pattern storage. This article describes how changes in feedback signaling, neuromodulation, and recurrent connectivity may alter pattern processing in recurrent on-center off-surround networks of spiking neurons. In spiking neurons, fast, medium, and slow after-hyperpolarization (AHP currents control sigmoid signal threshold and slope. Modulation of AHP currents by acetylcholine (ACh can change sigmoid shape and, with it, network dynamics. For example, decreasing signal function threshold and increasing slope can lengthen the persistence of a partially contrast-enhanced pattern, increase the number of active cells stored in STM, or, if connectivity is distance-dependent, cause cell activities to cluster. These results clarify how cholinergic modulation by the basal forebrain may alter the vigilance of category learning circuits, and thus their sensitivity to predictive mismatches, thereby controlling whether learned categories code concrete or abstract features, as predicted by Adaptive Resonance Theory. The analysis includes global, distance-dependent, and interneuron-mediated circuits. With an appropriate degree of recurrent excitation and inhibition, spiking networks maintain a partially contrast-enhanced pattern for 800 milliseconds or longer after stimuli offset, then resolve to no stored pattern, or to winner-take-all stored patterns with one or multiple winners. Strengthening inhibition prolongs a partially contrast-enhanced pattern by slowing the transition to stability, while strengthening excitation causes more winners

The role of the dorsal raphé nucleus in reward-seeking behavior

Directory of Open Access Journals (Sweden)

Kae eNakamura

2013-08-01

Full Text Available Pharmacological experiments have shown that the modulation of brain serotonin levels has a strong impact on value-based decision making. Anatomical and physiological evidence also revealed that the dorsal raphé nucleus (DRN, a major source of serotonin, and the dopamine system receive common inputs from brain regions associated with appetitive and aversive information processing. The serotonin and dopamine systems also have reciprocal functional influences on each other. However, the specific mechanism by which serotonin affects value-based decision making is not clear.To understand the information carried by the DRN for reward-seeking behavior, we measured single neuron activity in the primate DRN during the performance of saccade tasks to obtain different amounts of a reward. We found that DRN neuronal activity was characterized by tonic modulation that was altered by the expected and received reward value. Consistent reward-dependent modulation across different task periods suggested that DRN activity kept track of the reward value throughout a trial. The DRN was also characterized by modulation of its activity in the opposite direction by different neuronal subgroups, one firing strongly for the prediction and receipt of large rewards, with the other firing strongly for small rewards. Conversely, putative dopamine neurons showed positive phasic responses to reward-indicating cues and the receipt of an unexpected reward amount, which supports the reward prediction error signal hypothesis of dopamine.I suggest that the tonic reward monitoring signal of the DRN, possibly together with its interaction with the dopamine system, reports a continuous level of motivation throughout the performance of a task. Such a signal may provide reward context information to the targets of DRN projections, where it may be integrated further with incoming motivationally salient information.

Associative and non-associative plasticity in Kenyon cells of the honeybee mushroom body

Directory of Open Access Journals (Sweden)

2008-06-01

Full Text Available The insect mushroom bodies are higher-order brain centers and critical for odor learning. We investigated experience dependent plasticity of their intrinsic neurons, the Kenyon cells. Using calcium imaging, we recorded Kenyon cell responses and investigated non-associative plasticity by applying repeated odor stimuli. Associative plasticity was examined by performing appetitive odor learning experiments. Olfactory, gustatory and tactile antennal stimuli evoked phasic calcium transients in sparse ensembles of responding Kenyon cells. Repeated stimulation with an odor led to a decrease in Kenyon cells’ response strength. The pairing of an odor (CS with a sucrose reward (US induced a prolongation of Kenyon cell responses. After conditioning, Kenyon cell responses to a rewarded odor (CS+ recovered from repetition-induced decrease, while the responses to a non-rewarded odor (CS- decreased further. The spatio-temporal pattern of activated Kenyon cells changed for both odors when compared with the response before conditioning but the change was stronger for the CS-. These results demonstrate that Kenyon cell responses are subject to non-associative plasticity during odor repetition and undergo associative plasticity after appetitive odor learning.

Temperature dependence of plastic scintillators

Science.gov (United States)

Peralta, L.

2018-03-01

Plastic scintillator detectors have been studied as dosimeters, since they provide a cost-effective alternative to conventional ionization chambers. Several articles have reported undesired response dependencies on beam energy and temperature, which provides the motivation to determine appropriate correction factors. In this work, we studied the light yield temperature dependency of four plastic scintillators, BCF-10, BCF-60, BC-404, RP-200A and two clear fibers, BCF-98 and SK-80. Measurements were made using a 50 kVp X-ray beam to produce the scintillation and/or radioluminescence signal. The 0 to 40 °C temperature range was scanned for each scintillator, and temperature coefficients were obtained.

Dopamine reward prediction error coding.

Science.gov (United States)

Schultz, Wolfram

2016-03-01

Reward prediction errors consist of the differences between received and predicted rewards. They are crucial for basic forms of learning about rewards and make us strive for more rewards-an evolutionary beneficial trait. Most dopamine neurons in the midbrain of humans, monkeys, and rodents signal a reward prediction error; they are activated by more reward than predicted (positive prediction error), remain at baseline activity for fully predicted rewards, and show depressed activity with less reward than predicted (negative prediction error). The dopamine signal increases nonlinearly with reward value and codes formal economic utility. Drugs of addiction generate, hijack, and amplify the dopamine reward signal and induce exaggerated, uncontrolled dopamine effects on neuronal plasticity. The striatum, amygdala, and frontal cortex also show reward prediction error coding, but only in subpopulations of neurons. Thus, the important concept of reward prediction errors is implemented in neuronal hardware.

Event-related EEG responses to anticipation and delivery of monetary and social reward.

Science.gov (United States)

Flores, Amanda; Münte, Thomas F; Doñamayor, Nuria

2015-07-01

Monetary and a social incentive delay tasks were used to characterize reward anticipation and delivery with electroencephalography. During reward anticipation, N1, P2 and P3 components were modulated by both prospective reward value and incentive type (monetary or social), suggesting distinctive allocation of attentional and motivational resources depending not only on whether rewards or non-rewards were cued, but also on the monetary and social nature of the prospective outcomes. In the delivery phase, P2, FRN and P3 components were also modulated by levels of reward value and incentive type, illustrating how distinctive affective and cognitive processes were attached to the different outcomes. Our findings imply that neural processing of both reward anticipation and delivery can be specific to incentive type, which might have implications for basic as well as translational research. These results are discussed in the light of previous electrophysiological and neuroimaging work using similar tasks. Copyright © 2015 Elsevier B.V. All rights reserved.

Reconstructing stimuli from the spike-times of leaky integrate and fire neurons

Directory of Open Access Journals (Sweden)

Sebastian eGerwinn

2011-02-01

Full Text Available Reconstructing stimuli from the spike-trains of neurons is an important approach for understanding the neural code. One of the difficulties associated with this task is that signals which are varying continuously in time are encoded into sequences of discrete events or spikes. An important problem is to determine how much information about the continuously varying stimulus can be extracted from the time-points at which spikes were observed, especially if these time-points are subject to some sort of randomness. For the special case of spike trains generated by leaky integrate and fire neurons, noise can be introduced by allowing variations in the threshold every time a spike is released. A simple decoding algorithm previously derived for the noiseless case can be extended to the stochastic case, but turns out to be biased. Here, we review a solution to this problem, by presenting a simple yet efficient algorithm which greatly reduces the bias, and therefore leads to better decoding performance in the stochastic case.

Detection of bursts in neuronal spike trains by the mean inter-spike interval method

Institute of Scientific and Technical Information of China (English)

Lin Chen; Yong Deng; Weihua Luo; Zhen Wang; Shaoqun Zeng

2009-01-01

Bursts are electrical spikes firing with a high frequency, which are the most important property in synaptic plasticity and information processing in the central nervous system. However, bursts are difficult to identify because bursting activities or patterns vary with phys-iological conditions or external stimuli. In this paper, a simple method automatically to detect bursts in spike trains is described. This method auto-adaptively sets a parameter (mean inter-spike interval) according to intrinsic properties of the detected burst spike trains, without any arbitrary choices or any operator judgrnent. When the mean value of several successive inter-spike intervals is not larger than the parameter, a burst is identified. By this method, bursts can be automatically extracted from different bursting patterns of cultured neurons on multi-electrode arrays, as accurately as by visual inspection. Furthermore, significant changes of burst variables caused by electrical stimulus have been found in spontaneous activity of neuronal network. These suggest that the mean inter-spike interval method is robust for detecting changes in burst patterns and characteristics induced by environmental alterations.

Influence of time-dependent elastic-plastic material behaviour on the load-carrying capacity of shells of revolution

International Nuclear Information System (INIS)

Schnabel, F.

1987-01-01

The present report deals with the influence of time-dependent material behavior on the load-carrying capacity of thin-walled shells of revolution. In the first part various creep-hardening hypotheses as well as the spatial and temporal discretization procedures employed are described. The adaptation of a well-tested finite element method based on ring elements to the treatment of creep problems and several time-integration procedures, in particular the iterative treatment of the coupling between creep and elastic-plastic strains as well as the important aspect of time-step-control are discussed in detail. In the second part several typical shell configurations are analyzed and a comparison with available theoretical and experimental results is made. Finally, the time-dependent load-carrying behavior of torispherical pressure vessel ends subjected to internal and external pressure is investigated and design aids for the determination of creep collapse times are proposed. (orig.) [de

Magnesium sulfate differentially modulates fetal membrane inflammation in a time-dependent manner.

Science.gov (United States)

Cross, Sarah N; Nelson, Rachel A; Potter, Julie A; Norwitz, Errol R; Abrahams, Vikki M

2018-04-30

Chorioamnionitis and infection-associated inflammation are major causes of preterm birth. Magnesium sulfate (MgSO 4 ) is widely used in obstetrics as a tocolytic; however, its mechanism of action is unclear. This study sought to investigate how MgSO 4 modulates infection-associated inflammation in fetal membranes (FMs), and whether the response was time dependent. Human FM explants were treated with or without bacterial lipopolysaccharide (LPS); with or without MgSO 4 added either: 1 hour before LPS; at the same time as LPS; 1 hour post-LPS; or 2 hours post-LPS. Explants were also treated with or without viral dsRNA and LPS, alone or in combination; and MgSO 4 added 1 hour post-LPS After 24 hours, supernatants were measured for cytokines/chemokines; and tissue lysates measured for caspase-1 activity. Lipopolysaccharide-induced FM inflammation by upregulating the secretion of a number of inflammatory cytokines/chemokines. Magnesium sulfate administered 1-hour post-LPS inhibited FM secretion of IL-1β, IL-6, G-CSF, RANTES, and TNFα. Magnesium sulfate administered 2 hours post-LPS augmented FM secretion of these factors as well as IL-8, IFNγ, VEGF, GROα and IP-10. Magnesium sulfate delivered 1- hour post-LPS inhibited LPS-induced caspase-1 activity, and inhibited the augmented IL-1β response triggered by combination viral dsRNA and LPS. Magnesium sulfate differentially modulates LPS-induced FM inflammation in a time-dependent manner, in part through its modulation of caspase-1 activity. Thus, the timing of MgSO 4 administration may be critical in optimizing its anti-inflammatory effects in the clinical setting. MgSO 4 might also be useful at preventing FM inflammation triggered by a polymicrobial viral-bacterial infection. © 2018 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Reward retroactively enhances memory consolidation for related items

OpenAIRE

Patil, Anuya; Murty, Vishnu P.; Dunsmoor, Joseph E.; Phelps, Elizabeth A.; Davachi, Lila

2017-01-01

Reward motivation has been shown to modulate episodic memory processes in order to support future adaptive behavior. However, for a memory system to be truly adaptive, it should enhance memory for rewarded events as well as for neutral events that may seem inconsequential at the time of encoding but can gain importance later. Here, we investigated the influence of reward motivation on retroactive memory enhancement selectively for conceptually related information. We found behavioral evidence...

Kalirin-7 is necessary for normal NMDA receptor-dependent synaptic plasticity

Directory of Open Access Journals (Sweden)

Lemtiri-Chlieh Fouad

2011-12-01

Full Text Available Abstract Background Dendritic spines represent the postsynaptic component of the vast majority of excitatory synapses present in the mammalian forebrain. The ability of spines to rapidly alter their shape, size, number and receptor content in response to stimulation is considered to be of paramount importance during the development of synaptic plasticity. Indeed, long-term potentiation (LTP, widely believed to be a cellular correlate of learning and memory, has been repeatedly shown to induce both spine enlargement and the formation of new dendritic spines. In our studies, we focus on Kalirin-7 (Kal7, a Rho GDP/GTP exchange factor (Rho-GEF localized to the postsynaptic density that plays a crucial role in the development and maintenance of dendritic spines both in vitro and in vivo. Previous studies have shown that mice lacking Kal7 (Kal7KO have decreased dendritic spine density in the hippocampus as well as focal hippocampal-dependent learning impairments. Results We have performed a detailed electrophysiological characterization of the role of Kal7 in hippocampal synaptic plasticity. We show that loss of Kal7 results in impaired NMDA receptor-dependent LTP and long-term depression, whereas a NMDA receptor-independent form of LTP is shown to be normal in the absence of Kal7. Conclusions These results indicate that Kal7 is an essential and selective modulator of NMDA receptor-dependent synaptic plasticity in the hippocampus.

Kalirin-7 is necessary for normal NMDA receptor-dependent synaptic plasticity

KAUST Repository

Lemtiri-Chlieh, Fouad

2011-12-19

Background: Dendritic spines represent the postsynaptic component of the vast majority of excitatory synapses present in the mammalian forebrain. The ability of spines to rapidly alter their shape, size, number and receptor content in response to stimulation is considered to be of paramount importance during the development of synaptic plasticity. Indeed, long-term potentiation (LTP), widely believed to be a cellular correlate of learning and memory, has been repeatedly shown to induce both spine enlargement and the formation of new dendritic spines. In our studies, we focus on Kalirin-7 (Kal7), a Rho GDP/GTP exchange factor (Rho-GEF) localized to the postsynaptic density that plays a crucial role in the development and maintenance of dendritic spines both in vitro and in vivo. Previous studies have shown that mice lacking Kal7 (Kal7 KO) have decreased dendritic spine density in the hippocampus as well as focal hippocampal-dependent learning impairments.Results: We have performed a detailed electrophysiological characterization of the role of Kal7 in hippocampal synaptic plasticity. We show that loss of Kal7 results in impaired NMDA receptor-dependent LTP and long-term depression, whereas a NMDA receptor-independent form of LTP is shown to be normal in the absence of Kal7.Conclusions: These results indicate that Kal7 is an essential and selective modulator of NMDA receptor-dependent synaptic plasticity in the hippocampus. 2011 Lemtiri-Chlieh et al; licensee BioMed Central Ltd.

Threat/reward-sensitivity and hypomanic-personality modulate cognitive-control and attentional neural processes to emotional stimuli.

Science.gov (United States)

Pornpattananangkul, Narun; Hu, Xiaoqing; Nusslock, Robin

2015-11-01

Temperamental-traits (e.g. threat/reward-sensitivity) are found to modulate cognitive-control and attentional-processes. Yet, it is unclear exactly how these traits interact with emotional-stimuli in the modulation of cognitive-control, as reflected by the N2 event-related potential (ERP), and attentional-processes, as reflected by the P2 and P3 ERPs. Here in an ERP emotional-Go/NoGo task, 36 participants were instructed to inhibit their response to Fearful- and Happy-faces. Individual-differences in threat-sensitivity, reward-sensitivity and hypomanic-personality were assessed through self-report. Hypomanic-personality was assessed, given its relationship with reward-sensitivity and relevance to mood-disorder symptoms. Concerning cognitive-control, individuals with elevated threat-sensitivity displayed more-negative N2s to Happy-NoGo (relative to Fearful-NoGo) faces, whereas both individuals with elevated reward-sensitivity and hypomanic-personality displayed more-negative N2s to Fearful-NoGo (relative to Happy-NoGo) faces. Accordingly, when cognitive-control is required (during Go/NoGo), a mismatch between one's temperament and the valence of the NoGo-stimulus elevates detection of the need for cognitive-control. Conversely, the modulation of attentional-processing was specific to threat-sensitivity, as there was no relationship between either reward-sensitivity or hypomanic-personality and attentional-processing. Elevated threat-sensitivity was associated with enhanced early (P2s) and later (P3s) attentional-processing to Fearful-NoGo (relative to Happy-NoGo) faces. These latter findings support the negative attentional-bias model relating elevated threat-sensitivity with attentional-biases toward negative-stimuli and away from positive-stimuli. © The Author (2015). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.

NeuroFlow: A General Purpose Spiking Neural Network Simulation Platform using Customizable Processors.

Science.gov (United States)

Cheung, Kit; Schultz, Simon R; Luk, Wayne

2015-01-01

NeuroFlow is a scalable spiking neural network simulation platform for off-the-shelf high performance computing systems using customizable hardware processors such as Field-Programmable Gate Arrays (FPGAs). Unlike multi-core processors and application-specific integrated circuits, the processor architecture of NeuroFlow can be redesigned and reconfigured to suit a particular simulation to deliver optimized performance, such as the degree of parallelism to employ. The compilation process supports using PyNN, a simulator-independent neural network description language, to configure the processor. NeuroFlow supports a number of commonly used current or conductance based neuronal models such as integrate-and-fire and Izhikevich models, and the spike-timing-dependent plasticity (STDP) rule for learning. A 6-FPGA system can simulate a network of up to ~600,000 neurons and can achieve a real-time performance of 400,000 neurons. Using one FPGA, NeuroFlow delivers a speedup of up to 33.6 times the speed of an 8-core processor, or 2.83 times the speed of GPU-based platforms. With high flexibility and throughput, NeuroFlow provides a viable environment for large-scale neural network simulation.

Cholinergic modulation of mesolimbic dopamine function and reward.

Science.gov (United States)

Mark, Gregory P; Shabani, Shkelzen; Dobbs, Lauren K; Hansen, Stephen T

2011-07-25

The substantial health risk posed by obesity and compulsive drug use has compelled a serious research effort to identify the neurobiological substrates that underlie the development these pathological conditions. Despite substantial progress, an understanding of the neurochemical systems that mediate the motivational aspects of drug-seeking and craving remains incomplete. Important work from the laboratory of Bart Hoebel has provided key information on neurochemical systems that interact with dopamine (DA) as potentially important components in both the development of addiction and the expression of compulsive behaviors such as binge eating. One such modulatory system appears to be cholinergic pathways that interact with DA systems at all levels of the reward circuit. Cholinergic cells in the pons project to DA-rich cell body regions in the ventral tegmental area (VTA) and substantial nigra (SN) where they modulate the activity of dopaminergic neurons and reward processing. The DA terminal region of the nucleus accumbens (NAc) contains a small but particularly important group of cholinergic interneurons, which have extensive dendritic arbors that make synapses with a vast majority of NAc neurons and afferents. Together with acetylcholine (ACh) input onto DA cell bodies, cholinergic systems could serve a vital role in gating information flow concerning the motivational value of stimuli through the mesolimbic system. In this report we highlight evidence that CNS cholinergic systems play a pivotal role in behaviors that are motivated by both natural and drug rewards. We argue that the search for underlying neurochemical substrates of compulsive behaviors, as well as attempts to identify potential pharmacotherapeutic targets to combat them, must include a consideration of central cholinergic systems. Copyright © 2011 Elsevier Inc. All rights reserved.

A new framework for cortico-striatal plasticity: behavioural theory meets in vitro data at the reinforcement-action interface.

Directory of Open Access Journals (Sweden)

Kevin N Gurney

2015-01-01

Full Text Available Operant learning requires that reinforcement signals interact with action representations at a suitable neural interface. Much evidence suggests that this occurs when phasic dopamine, acting as a reinforcement prediction error, gates plasticity at cortico-striatal synapses, and thereby changes the future likelihood of selecting the action(s coded by striatal neurons. But this hypothesis faces serious challenges. First, cortico-striatal plasticity is inexplicably complex, depending on spike timing, dopamine level, and dopamine receptor type. Second, there is a credit assignment problem-action selection signals occur long before the consequent dopamine reinforcement signal. Third, the two types of striatal output neuron have apparently opposite effects on action selection. Whether these factors rule out the interface hypothesis and how they interact to produce reinforcement learning is unknown. We present a computational framework that addresses these challenges. We first predict the expected activity changes over an operant task for both types of action-coding striatal neuron, and show they co-operate to promote action selection in learning and compete to promote action suppression in extinction. Separately, we derive a complete model of dopamine and spike-timing dependent cortico-striatal plasticity from in vitro data. We then show this model produces the predicted activity changes necessary for learning and extinction in an operant task, a remarkable convergence of a bottom-up data-driven plasticity model with the top-down behavioural requirements of learning theory. Moreover, we show the complex dependencies of cortico-striatal plasticity are not only sufficient but necessary for learning and extinction. Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity. By bridging the levels between the single synapse and

Phasic spike patterning in rat supraoptic neurones in vivo and in vitro

Science.gov (United States)

Sabatier, Nancy; Brown, Colin H; Ludwig, Mike; Leng, Gareth

2004-01-01

In vivo, most vasopressin cells of the hypothalamic supraoptic nucleus fire action potentials in a ‘phasic’ pattern when the systemic osmotic pressure is elevated, while most oxytocin cells fire continuously. The phasic firing pattern is believed to arise as a consequence of intrinsic activity-dependent changes in membrane potential, and these have been extensively studied in vitro. Here we analysed the discharge patterning of supraoptic nucleus neurones in vivo, to infer the characteristics of the post-spike sequence of hyperpolarization and depolarization from the observed spike patterning. We then compared patterning in phasic cells in vivo and in vitro, and we found systematic differences in the interspike interval distributions, and in other statistical parameters that characterized activity patterns within bursts. Analysis of hazard functions (probability of spike initiation as a function of time since the preceding spike) revealed that phasic firing in vitro appears consistent with a regenerative process arising from a relatively slow, late depolarizing afterpotential that approaches or exceeds spike threshold. By contrast, in vivo activity appears to be dominated by stochastic rather than deterministic mechanisms, and appears consistent with a relatively early and fast depolarizing afterpotential that modulates the probability that random synaptic input exceeds spike threshold. Despite superficial similarities in the phasic firing patterns observed in vivo and in vitro, there are thus fundamental differences in the underlying mechanisms. PMID:15146047

5-HT4-receptors modulate induction of long-term depression but not potentiation at hippocampal output synapses in acute rat brain slices.

Directory of Open Access Journals (Sweden)

Matthias Wawra

Full Text Available The subiculum is the principal target of CA1 pyramidal cells and mediates hippocampal output to various cortical and subcortical regions of the brain. The majority of subicular pyramidal cells are burst-spiking neurons. Previous studies indicated that high frequency stimulation in subicular burst-spiking cells causes presynaptic NMDA-receptor dependent long-term potentiation (LTP whereas low frequency stimulation induces postsynaptic NMDA-receptor-dependent long-term depression (LTD. In the present study, we investigate the effect of 5-hydroxytryptamine type 4 (5-HT4 receptor activation and blockade on both forms of synaptic plasticity in burst-spiking cells. We demonstrate that neither activation nor block of 5-HT4 receptors modulate the induction or expression of LTP. In contrast, activation of 5-HT4 receptors facilitates expression of LTD, and block of the 5-HT4 receptor prevents induction of short-term depression and LTD. As 5-HT4 receptors are positively coupled to adenylate cyclase 1 (AC1, 5-HT4 receptors might modulate PKA activity through AC1. Since LTD is blocked in the presence of 5-HT4 receptor antagonists, our data are consistent with 5-HT4 receptor activation by ambient serotonin or intrinsically active 5-HT4 receptors. Our findings provide new insight into aminergic modulation of hippocampal output.

CX3CR1 deficiency alters hippocampal-dependent plasticity phenomena blunting the effects of enriched environment

Directory of Open Access Journals (Sweden)

Laura eMaggi

2011-10-01

Full Text Available In recent years several evidence demonstrated that some features of hippocampal biology, like neurogenesis, synaptic transmission, learning and memory performances are deeply modulated by social, motor and sensorial experiences. Fractalkine/CX3CL1 is a transmembrane chemokine abundantly expressed in the brain by neurons, where it modulates glutamatergic transmission and long-term plasticity processes regulating the intercellular communication between glia and neurons, being its specific receptor CX3CR1 expressed by microglia. In this paper we investigated the role of CX3CL1/CX3CR1 signaling on experience-dependent hippocampal plasticity processes. At this aim wt and CX3CR1GFP/GFP mice were exposed to long-lasting-enriched environment (EE and the effects on hippocampal functions were studied by electrophysiological recordings of long-term potentiation (LTP of synaptic activity, behavioral tests of learning and memory in the Morris water maze paradigm and analysis of neurogenesis in the subgranular zone of the dentate gyrus (DG.We found that CX3CR1 deficiency increases hippocampal plasticity and spatial memory blunting the potentiating effects of EE. In contrast, exposure to EE increased the number and migration of neural progenitors in the DG of both wt and CX3CR1GFP/GFP mice. These data indicate that CX3CL1/CX3CR1-mediated signaling is crucial for a normal experience-dependent modulation of hippocampal functions.

Self-rewards and personal motivation

DEFF Research Database (Denmark)

Koch, Alexander Karl; Nafziger, Julia; Suvorov, Anton

2014-01-01

Self-administered rewards are ubiquitous. They serve as incentives for personal accomplishments and are widely recommended to increase personal motivation. We show that in a model with time-inconsistent and reference-dependent preferences, self-rewards can be a credible and effective tool...

Self-Rewards and Personal Motivation

DEFF Research Database (Denmark)

Koch, Alexander Karl; Nafziger, Julia; Suvorov, Anton

Self-administered rewards are ubiquitous. They serve as incentives for personal accomplishments and are widely recommended to increase personal motivation. We show that in a model with time-inconsistent and reference-dependent preferences, self-rewards can be a credible and effective tool...

Self-rewards and personal motivation

NARCIS (Netherlands)

Koch, A.K.; Nafziger, J.; Suvorov, A.; van de Ven, J.

2014-01-01

Self-administered rewards are ubiquitous. They serve as incentives for personal accomplishments and are widely recommended to increase personal motivation. We show that in a model with time-inconsistent and reference-dependent preferences, self-rewards can be a credible and effective tool to

Reward prediction error signal enhanced by striatum-amygdala interaction explains the acceleration of probabilistic reward learning by emotion.

Science.gov (United States)

Watanabe, Noriya; Sakagami, Masamichi; Haruno, Masahiko

2013-03-06

Learning does not only depend on rationality, because real-life learning cannot be isolated from emotion or social factors. Therefore, it is intriguing to determine how emotion changes learning, and to identify which neural substrates underlie this interaction. Here, we show that the task-independent presentation of an emotional face before a reward-predicting cue increases the speed of cue-reward association learning in human subjects compared with trials in which a neutral face is presented. This phenomenon was attributable to an increase in the learning rate, which regulates reward prediction errors. Parallel to these behavioral findings, functional magnetic resonance imaging demonstrated that presentation of an emotional face enhanced reward prediction error (RPE) signal in the ventral striatum. In addition, we also found a functional link between this enhanced RPE signal and increased activity in the amygdala following presentation of an emotional face. Thus, this study revealed an acceleration of cue-reward association learning by emotion, and underscored a role of striatum-amygdala interactions in the modulation of the reward prediction errors by emotion.

Reward acts on the pFC to enhance distractor resistance of working memory representations.

Science.gov (United States)

Fallon, Sean James; Cools, Roshan

2014-12-01

Working memory and reward processing are often thought to be separate, unrelated processes. However, most daily activities involve integrating these two types of information, and the two processes rarely, if ever, occur in isolation. Here, we show that working memory and reward interact in a task-dependent manner and that this task-dependent interaction involves modulation of the pFC by the ventral striatum. Specifically, BOLD signal during gains relative to losses in the ventral striatum and pFC was associated not only with enhanced distractor resistance but also with impairment in the ability to update working memory representations. Furthermore, the effect of reward on working memory was accompanied by differential coupling between the ventral striatum and ignore-related regions in the pFC. Together, these data demonstrate that reward-related signals modulate the balance between cognitive stability and cognitive flexibility by altering functional coupling between the ventral striatum and the pFC.

How motivation and reward learning modulate selective attention.

Science.gov (United States)

Bourgeois, A; Chelazzi, L; Vuilleumier, P

2016-01-01

Motivational stimuli such as rewards elicit adaptive responses and influence various cognitive functions. Notably, increasing evidence suggests that stimuli with particular motivational values can strongly shape perception and attention. These effects resemble both selective top-down and stimulus-driven attentional orienting, as they depend on internal states but arise without conscious will, yet they seem to reflect attentional systems that are functionally and anatomically distinct from those classically associated with frontoparietal cortical networks in the brain. Recent research in human and nonhuman primates has begun to reveal how reward can bias attentional selection, and where within the cognitive system the signals providing attentional priority are generated. This review aims at describing the different mechanisms sustaining motivational attention, their impact on different behavioral tasks, and current knowledge concerning the neural networks governing the integration of motivational influences on attentional behavior. © 2016 Elsevier B.V. All rights reserved.

Time Series Analysis of Wheat Futures Reward in China

Institute of Scientific and Technical Information of China (English)

无

2005-01-01

Different from the fact that the main researches are focused on single futures contract and lack of the comparison of different periods, this paper described the statistical characteristics of wheat futures reward time series of Zhengzhou Commodity Exchange in recent three years. Besides the basic statistic analysis, the paper used the GARCH and EGARCH model to describe the time series which had the ARCH effect and analyzed the persistence of volatility shocks and the leverage effect. The results showed that compared with that of normal one,wheat futures reward series were abnormality, leptokurtic and thick tail distribution. The study also found that two-part of the reward series had no autocorrelation. Among the six correlative series, three ones presented the ARCH effect. By using of the Auto-regressive Distributed Lag Model, GARCH model and EGARCH model, the paper demonstrates the persistence of volatility shocks and the leverage effect on the wheat futures reward time series. The results reveal that on the one hand, the statistical characteristics of the wheat futures reward are similar to the aboard mature futures market as a whole. But on the other hand, the results reflect some shortages such as the immatureness and the over-control by the government in the Chinese future market.

Theta-frequency resonance at the cerebellum input stage improves spike-timing on the millisecond time-scale

Directory of Open Access Journals (Sweden)

Daniela eGandolfi

2013-04-01

Full Text Available The neuronal circuits of the brain are thought to use resonance and oscillations to improve communication over specific frequency bands (Llinas, 1988; Buzsaki, 2006. However, the properties and mechanism of these phenomena in brain circuits remain largely unknown. Here we show that, at the cerebellum input stage, the granular layer generates its maximum response at 5-7 Hz both in vivo following tactile sensory stimulation of the whisker pad and in acute slices following mossy fiber-bundle stimulation. The spatial analysis of granular layer activity performed using voltage-sensitive dye (VSD imaging revealed 5-7 Hz resonance covering large granular layer areas. In single granule cells, resonance appeared as a reorganization of output spike bursts on the millisecond time-scale, such that the first spike occurred earlier and with higher temporal precision and the probability of spike generation increased. Resonance was independent from circuit inhibition, as it persisted with little variation in the presence of the GABAA receptor blocker, gabazine. However, circuit inhibition reduced the resonance area more markedly at 7 Hz. Simulations with detailed computational models suggested that resonance depended on intrinsic granule cells ionic mechanisms: specifically, Kslow (M-like and KA currents acted as resonators and the persistent Na current and NMDA current acted as amplifiers. This form of resonance may play an important role for enhancing coherent spike emission from the granular layer when theta-frequency bursts are transmitted by the cerebral cortex and peripheral sensory structures during sensory-motor processing, cognition and learning.

Reward Retroactively Enhances Memory Consolidation for Related Items

Science.gov (United States)

Patil, Anuya; Murty, Vishnu P.; Dunsmoor, Joseph E.; Phelps, Elizabeth A.; Davachi, Lila

2017-01-01

Reward motivation has been shown to modulate episodic memory processes in order to support future adaptive behavior. However, for a memory system to be truly adaptive, it should enhance memory for rewarded events as well as for neutral events that may seem inconsequential at the time of encoding but can gain importance later. Here, we investigated…

Performance analysis of chi models using discrete-time probabilistic reward graphs

NARCIS (Netherlands)

Trcka, N.; Georgievska, S.; Markovski, J.; Andova, S.; Vink, de E.P.

2008-01-01

We propose the model of discrete-time probabilistic reward graphs (DTPRGs) for performance analysis of systems exhibiting discrete deterministic time delays and probabilistic behavior, via their interpretation as discrete-time Markov reward chains, full-fledged platform for qualitative and

The time-dependence of exchange-induced relaxation during modulated radio frequency pulses.

Science.gov (United States)

Sorce, Dennis J; Michaeli, Shalom; Garwood, Michael

2006-03-01

The problem of the relaxation of identical spins 1/2 induced by chemical exchange between spins with different chemical shifts in the presence of time-dependent RF irradiation (in the first rotating frame) is considered for the fast exchange regime. The solution for the time evolution under the chemical exchange Hamiltonian in the tilted doubly rotating frame (TDRF) is presented. Detailed derivation is specified to the case of a two-site chemical exchange system with complete randomization between jumps of the exchanging spins. The derived theory can be applied to describe the modulation of the chemical exchange relaxation rate constants when using a train of adiabatic pulses, such as the hyperbolic secant pulse. Theory presented is valid for quantification of the exchange-induced time-dependent rotating frame longitudinal T1rho,ex and transverse T2rho,ex relaxations in the fast chemical exchange regime.

Mindfulness meditation modulates reward prediction errors in the striatum in a passive conditioning task

Directory of Open Access Journals (Sweden)

Ulrich eKirk

2015-02-01

Full Text Available Reinforcement learning models have demonstrated that phasic activity of dopamine neurons during reward expectation encodes information about the predictability of rewards and cues that predict reward. Evidence indicates that mindfulness-based approaches reduce reward anticipation signal in the striatum to negative and positive incentives suggesting the hypothesis that such training influence basic reward processing. Using a passive conditioning task and fMRI in a group of experienced mindfulness meditators and age-matched controls, we tested the hypothesis that mindfulness meditation influence reward and reward prediction error signals. We found diminished positive and negative prediction error-related blood-oxygen level-dependent (BOLD responses in the putamen in meditators compared with controls. In the meditators, this decrease in striatal BOLD responses to reward prediction was paralleled by increased activity in posterior insula, a primary interoceptive region. Critically, responses in the putamen during early trials of the conditioning procedure (run 1 were elevated in both meditators and controls. These results provide evidence that experienced mindfulness meditators show attenuated reward prediction signals to valenced stimuli, which may be related to interoceptive processes encoded in the posterior insula.

Enabling an Integrated Rate-temporal Learning Scheme on Memristor

Science.gov (United States)

He, Wei; Huang, Kejie; Ning, Ning; Ramanathan, Kiruthika; Li, Guoqi; Jiang, Yu; Sze, Jiayin; Shi, Luping; Zhao, Rong; Pei, Jing

2014-04-01

Learning scheme is the key to the utilization of spike-based computation and the emulation of neural/synaptic behaviors toward realization of cognition. The biological observations reveal an integrated spike time- and spike rate-dependent plasticity as a function of presynaptic firing frequency. However, this integrated rate-temporal learning scheme has not been realized on any nano devices. In this paper, such scheme is successfully demonstrated on a memristor. Great robustness against the spiking rate fluctuation is achieved by waveform engineering with the aid of good analog properties exhibited by the iron oxide-based memristor. The spike-time-dependence plasticity (STDP) occurs at moderate presynaptic firing frequencies and spike-rate-dependence plasticity (SRDP) dominates other regions. This demonstration provides a novel approach in neural coding implementation, which facilitates the development of bio-inspired computing systems.

The evolution of age-dependent plasticity

NARCIS (Netherlands)

Fischer, Barbara; van Doorn, G. Sander; Dieckmann, Ulf; Taborsky, Barbara

2014-01-01

When organisms encounter environments that are heterogeneous in time, phenotypic plasticity is often favored by selection. The degree of such plasticity can vary during an organism''s lifetime, but the factors promoting differential plastic responses at different ages or life stages remain poorly

Mirrored STDP Implements Autoencoder Learning in a Network of Spiking Neurons.

Science.gov (United States)

Burbank, Kendra S

2015-12-01

The autoencoder algorithm is a simple but powerful unsupervised method for training neural networks. Autoencoder networks can learn sparse distributed codes similar to those seen in cortical sensory areas such as visual area V1, but they can also be stacked to learn increasingly abstract representations. Several computational neuroscience models of sensory areas, including Olshausen & Field's Sparse Coding algorithm, can be seen as autoencoder variants, and autoencoders have seen extensive use in the machine learning community. Despite their power and versatility, autoencoders have been difficult to implement in a biologically realistic fashion. The challenges include their need to calculate differences between two neuronal activities and their requirement for learning rules which lead to identical changes at feedforward and feedback connections. Here, we study a biologically realistic network of integrate-and-fire neurons with anatomical connectivity and synaptic plasticity that closely matches that observed in cortical sensory areas. Our choice of synaptic plasticity rules is inspired by recent experimental and theoretical results suggesting that learning at feedback connections may have a different form from learning at feedforward connections, and our results depend critically on this novel choice of plasticity rules. Specifically, we propose that plasticity rules at feedforward versus feedback connections are temporally opposed versions of spike-timing dependent plasticity (STDP), leading to a symmetric combined rule we call Mirrored STDP (mSTDP). We show that with mSTDP, our network follows a learning rule that approximately minimizes an autoencoder loss function. When trained with whitened natural image patches, the learned synaptic weights resemble the receptive fields seen in V1. Our results use realistic synaptic plasticity rules to show that the powerful autoencoder learning algorithm could be within the reach of real biological networks.

Spike persistence and normalization in benign epilepsy with centrotemporal spikes - Implications for management.

Science.gov (United States)

Kim, Hunmin; Kim, Soo Yeon; Lim, Byung Chan; Hwang, Hee; Chae, Jong-Hee; Choi, Jieun; Kim, Ki Joong; Dlugos, Dennis J

2018-05-10

This study was performed 1) to determine the timing of spike normalization in patients with benign epilepsy with centrotemporal spikes (BECTS); 2) to identify relationships between age of seizure onset, age of spike normalization, years of spike persistence and treatment; and 3) to assess final outcomes between groups of patients with or without spikes at the time of medication tapering. Retrospective analysis of BECTS patients confirmed by clinical data, including age of onset, seizure semiology and serial electroencephalography (EEG) from diagnosis to remission. Age at spike normalization, years of spike persistence, and time of treatment onset to spike normalization were assessed. Final seizure and EEG outcome were compared between the groups with or without spikes at the time of AED tapering. One hundred and thirty-four patients were included. Mean age at seizure onset was 7.52 ± 2.11 years. Mean age at spike normalization was 11.89 ± 2.11 (range: 6.3-16.8) years. Mean time of treatment onset to spike normalization was 4.11 ± 2.13 (range: 0.24-10.08) years. Younger age of seizure onset was correlated with longer duration of spike persistence (r = -0.41, p < 0.001). In treated patients, spikes persisted for 4.1 ± 1.95 years, compared with 2.9 ± 1.97 years in untreated patients. No patients had recurrent seizures after AED was discontinued, regardless of the presence/absence of spikes at time of AED tapering. Years of spike persistence was longer in early onset BECTS patients. Treatment with AEDs did not shorten years of spike persistence. Persistence of spikes at time of treatment withdrawal was not associated with seizure recurrence. Copyright © 2018 The Japanese Society of Child Neurology. Published by Elsevier B.V. All rights reserved.

Amphetamine sensitization alters reward processing in the human striatum and amygdala.

Directory of Open Access Journals (Sweden)

Owen G O'Daly

Full Text Available Dysregulation of mesolimbic dopamine transmission is implicated in a number of psychiatric illnesses characterised by disruption of reward processing and goal-directed behaviour, including schizophrenia, drug addiction and impulse control disorders associated with chronic use of dopamine agonists. Amphetamine sensitization (AS has been proposed to model the development of this aberrant dopamine signalling and the subsequent dysregulation of incentive motivational processes. However, in humans the effects of AS on the dopamine-sensitive neural circuitry associated with reward processing remains unclear. Here we describe the effects of acute amphetamine administration, following a sensitising dosage regime, on blood oxygen level dependent (BOLD signal in dopaminoceptive brain regions during a rewarded gambling task performed by healthy volunteers. Using a randomised, double-blind, parallel-groups design, we found clear evidence for sensitization to the subjective effects of the drug, while rewarded reaction times were unchanged. Repeated amphetamine exposure was associated with reduced dorsal striatal BOLD signal during decision making, but enhanced ventromedial caudate activity during reward anticipation. The amygdala BOLD response to reward outcomes was blunted following repeated amphetamine exposure. Positive correlations between subjective sensitization and changes in anticipation- and outcome-related BOLD signal were seen for the caudate nucleus and amygdala, respectively. These data show for the first time in humans that AS changes the functional impact of acute stimulant exposure on the processing of reward-related information within dopaminoceptive regions. Our findings accord with pathophysiological models which implicate aberrant dopaminergic modulation of striatal and amygdala activity in psychosis and drug-related compulsive disorders.

Quality and Timing of Stressors Differentially Impact on Brain Plasticity and Neuroendocrine-Immune Function in Mice

Directory of Open Access Journals (Sweden)

Sara Capoccia

2013-01-01

Full Text Available A growing body of evidence suggests that psychological stress is a major risk factor for psychiatric disorders. The basic mechanisms are still under investigation but involve changes in neuroendocrine-immune interactions, ultimately affecting brain plasticity. In this study we characterized central and peripheral effects of different stressors, applied for different time lengths, in adult male C57BL/6J mice. We compared the effects of repeated (7 versus 21 days restraint stress (RS and chronic disruption of social hierarchy (SS on neuroendocrine (corticosterone and immune function (cytokines and splenic apoptosis and on a marker of brain plasticity (brain-derived neurotrophic factor, BDNF . Neuroendocrine activation did not differ between SS and control subjects; by contrast, the RS group showed a strong neuroendocrine response characterized by a specific time-dependent profile. Immune function and hippocampal BDNF levels were inversely related to hypothalamic-pituitary-adrenal axis activation. These data show a fine modulation of the crosstalk between central and peripheral pathways of adaptation and plasticity and suggest that the length of stress exposure is crucial to determine its final outcome on health or disease.

Emergent Auditory Feature Tuning in a Real-Time Neuromorphic VLSI System.

Science.gov (United States)

Sheik, Sadique; Coath, Martin; Indiveri, Giacomo; Denham, Susan L; Wennekers, Thomas; Chicca, Elisabetta

2012-01-01

Many sounds of ecological importance, such as communication calls, are characterized by time-varying spectra. However, most neuromorphic auditory models to date have focused on distinguishing mainly static patterns, under the assumption that dynamic patterns can be learned as sequences of static ones. In contrast, the emergence of dynamic feature sensitivity through exposure to formative stimuli has been recently modeled in a network of spiking neurons based on the thalamo-cortical architecture. The proposed network models the effect of lateral and recurrent connections between cortical layers, distance-dependent axonal transmission delays, and learning in the form of Spike Timing Dependent Plasticity (STDP), which effects stimulus-driven changes in the pattern of network connectivity. In this paper we demonstrate how these principles can be efficiently implemented in neuromorphic hardware. In doing so we address two principle problems in the design of neuromorphic systems: real-time event-based asynchronous communication in multi-chip systems, and the realization in hybrid analog/digital VLSI technology of neural computational principles that we propose underlie plasticity in neural processing of dynamic stimuli. The result is a hardware neural network that learns in real-time and shows preferential responses, after exposure, to stimuli exhibiting particular spectro-temporal patterns. The availability of hardware on which the model can be implemented, makes this a significant step toward the development of adaptive, neurobiologically plausible, spike-based, artificial sensory systems.

Emergent auditory feature tuning in a real-time neuromorphic VLSI system

Directory of Open Access Journals (Sweden)

Sadique eSheik

2012-02-01

Full Text Available Many sounds of ecological importance, such as communication calls, are characterised by time-varying spectra. However, most neuromorphic auditory models to date have focused on distinguishing mainly static patterns, under the assumption that dynamic patterns can be learned as sequences of static ones. In contrast, the emergence of dynamic feature sensitivity through exposure to formative stimuli has been recently modeled in a network of spiking neurons based on the thalamocortical architecture. The proposed network models the effect of lateral and recurrent connections between cortical layers, distance-dependent axonal transmission delays, and learning in the form of Spike Timing Dependent Plasticity (STDP, which effects stimulus-driven changes in the pattern of network connectivity. In this paper we demonstrate how these principles can be efficiently implemented in neuromorphic hardware. In doing so we address two principle problems in the design of neuromorphic systems: real-time event-based asynchronous communication in multi-chip systems, and the realization in hybrid analog/digital VLSI technology of neural computational principles that we propose underlie plasticity in neural processing of dynamic stimuli. The result is a hardware neural network that learns in real-time and shows preferential responses, after exposure, to stimuli exhibiting particular spectrotemporal patterns. The availability of hardware on which the model can be implemented, makes this a significant step towards the development of adaptive, neurobiologically plausible, spike-based, artificial sensory systems.

Galanin-Expressing GABA Neurons in the Lateral Hypothalamus Modulate Food Reward and Noncompulsive Locomotion.