Estimation of the neural drive to the muscle from surface electromyograms

Science.gov (United States)

Hofmann, David

Muscle force is highly correlated with the standard deviation of the surface electromyogram (sEMG) produced by the active muscle. Correctly estimating this quantity of non-stationary sEMG and understanding its relation to neural drive and muscle force is of paramount importance. The single constituents of the sEMG are called motor unit action potentials whose biphasic amplitude can interfere (named amplitude cancellation), potentially affecting the standard deviation (Keenan etal. 2005). However, when certain conditions are met the Campbell-Hardy theorem suggests that amplitude cancellation does not affect the standard deviation. By simulation of the sEMG, we verify the applicability of this theorem to myoelectric signals and investigate deviations from its conditions to obtain a more realistic setting. We find no difference in estimated standard deviation with and without interference, standing in stark contrast to previous results (Keenan etal. 2008, Farina etal. 2010). Furthermore, since the theorem provides us with the functional relationship between standard deviation and neural drive we conclude that complex methods based on high density electrode arrays and blind source separation might not bear substantial advantages for neural drive estimation (Farina and Holobar 2016). Funded by NIH Grant Number 1 R01 EB022872 and NSF Grant Number 1208126.

Neural Control of a Tracking Task via Attention-Gated Reinforcement Learning for Brain-Machine Interfaces.

Science.gov (United States)

Wang, Yiwen; Wang, Fang; Xu, Kai; Zhang, Qiaosheng; Zhang, Shaomin; Zheng, Xiaoxiang

2015-05-01

Reinforcement learning (RL)-based brain machine interfaces (BMIs) enable the user to learn from the environment through interactions to complete the task without desired signals, which is promising for clinical applications. Previous studies exploited Q-learning techniques to discriminate neural states into simple directional actions providing the trial initial timing. However, the movements in BMI applications can be quite complicated, and the action timing explicitly shows the intention when to move. The rich actions and the corresponding neural states form a large state-action space, imposing generalization difficulty on Q-learning. In this paper, we propose to adopt attention-gated reinforcement learning (AGREL) as a new learning scheme for BMIs to adaptively decode high-dimensional neural activities into seven distinct movements (directional moves, holdings and resting) due to the efficient weight-updating. We apply AGREL on neural data recorded from M1 of a monkey to directly predict a seven-action set in a time sequence to reconstruct the trajectory of a center-out task. Compared to Q-learning techniques, AGREL could improve the target acquisition rate to 90.16% in average with faster convergence and more stability to follow neural activity over multiple days, indicating the potential to achieve better online decoding performance for more complicated BMI tasks.

Complexity and competition in appetitive and aversive neural circuits

Directory of Open Access Journals (Sweden)

Crista L. Barberini

2012-11-01

Full Text Available Decision-making often involves using sensory cues to predict possible rewarding or punishing reinforcement outcomes before selecting a course of action. Recent work has revealed complexity in how the brain learns to predict rewards and punishments. Analysis of neural signaling during and after learning in the amygdala and orbitofrontal cortex, two brain areas that process appetitive and aversive stimuli, reveals a dynamic relationship between appetitive and aversive circuits. Specifically, the relationship between signaling in appetitive and aversive circuits in these areas shifts as a function of learning. Furthermore, although appetitive and aversive circuits may often drive opposite behaviors – approaching or avoiding reinforcement depending upon its valence – these circuits can also drive similar behaviors, such as enhanced arousal or attention; these processes also may influence choice behavior. These data highlight the formidable challenges ahead in dissecting how appetitive and aversive neural circuits interact to produce a complex and nuanced range of behaviors.

FRAMEWORK OF TAILORMADE DRIVING SUPPORT SYSTEMS AND NEURAL NETWORK DRIVER MODEL

Directory of Open Access Journals (Sweden)

Toshiya HIROSE, M.S.

2004-01-01

Nowadays, tailormade medical treatment is receiving much attention in the field of medical care. It is also desirable for driving support systems to reflect the driving characteristics of individuals as much as possible, begin monitoring the driver when a driver starts driving and calculates the driver model, and supports them with a model that makes the driver feel quite normal. That is the construction of Tailormade Driving Support Systems (TDSS. This research proposes a concept and a framework of TDSS, and presents a driver model that uses a neural network to build the system. As for the feasibility of this system, the research selects braking as a typical constituent element, and illustrates and reviews the results of experiments and simulations.

Higher incentives can impair performance: neural evidence on reinforcement and rationality.

Science.gov (United States)

Achtziger, Anja; Alós-Ferrer, Carlos; Hügelschäfer, Sabine; Steinhauser, Marco

2015-11-01

Standard economic thinking postulates that increased monetary incentives should increase performance. Human decision makers, however, frequently focus on past performance, a form of reinforcement learning occasionally at odds with rational decision making. We used an incentivized belief-updating task from economics to investigate this conflict through measurements of neural correlates of reward processing. We found that higher incentives fail to improve performance when immediate feedback on decision outcomes is provided. Subsequent analysis of the feedback-related negativity, an early event-related potential following feedback, revealed the mechanism behind this paradoxical effect. As incentives increase, the win/lose feedback becomes more prominent, leading to an increased reliance on reinforcement and more errors. This mechanism is relevant for economic decision making and the debate on performance-based payment. © The Author (2015). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.

Adolescent-specific patterns of behavior and neural activity during social reinforcement learning.

Science.gov (United States)

Jones, Rebecca M; Somerville, Leah H; Li, Jian; Ruberry, Erika J; Powers, Alisa; Mehta, Natasha; Dyke, Jonathan; Casey, B J

2014-06-01

Humans are sophisticated social beings. Social cues from others are exceptionally salient, particularly during adolescence. Understanding how adolescents interpret and learn from variable social signals can provide insight into the observed shift in social sensitivity during this period. The present study tested 120 participants between the ages of 8 and 25 years on a social reinforcement learning task where the probability of receiving positive social feedback was parametrically manipulated. Seventy-eight of these participants completed the task during fMRI scanning. Modeling trial-by-trial learning, children and adults showed higher positive learning rates than did adolescents, suggesting that adolescents demonstrated less differentiation in their reaction times for peers who provided more positive feedback. Forming expectations about receiving positive social reinforcement correlated with neural activity within the medial prefrontal cortex and ventral striatum across age. Adolescents, unlike children and adults, showed greater insular activity during positive prediction error learning and increased activity in the supplementary motor cortex and the putamen when receiving positive social feedback regardless of the expected outcome, suggesting that peer approval may motivate adolescents toward action. While different amounts of positive social reinforcement enhanced learning in children and adults, all positive social reinforcement equally motivated adolescents. Together, these findings indicate that sensitivity to peer approval during adolescence goes beyond simple reinforcement theory accounts and suggest possible explanations for how peers may motivate adolescent behavior.

Adapting without reinforcement.

Science.gov (United States)

Kheifets, Aaron; Gallistel, C Randy

2012-11-01

Our data rule out a broad class of behavioral models in which behavioral change is guided by differential reinforcement. To demonstrate this, we showed that the number of reinforcers missed before the subject shifted its behavior was not sufficient to drive behavioral change. What's more, many subjects shifted their behavior to a more optimal strategy even when they had not yet missed a single reinforcer. Naturally, differential reinforcement cannot be said to drive a process that shifts to accommodate to new conditions so adeptly that it doesn't miss a single reinforcer: it would have no input on which to base this shift.

A Model to Explain the Emergence of Reward Expectancy neurons using Reinforcement Learning and Neural Network

OpenAIRE

Shinya, Ishii; Munetaka, Shidara; Katsunari, Shibata

2006-01-01

In an experiment of multi-trial task to obtain a reward, reward expectancy neurons,###which responded only in the non-reward trials that are necessary to advance###toward the reward, have been observed in the anterior cingulate cortex of monkeys.###In this paper, to explain the emergence of the reward expectancy neuron in###terms of reinforcement learning theory, a model that consists of a recurrent neural###network trained based on reinforcement learning is proposed. The analysis of the###hi...

Brain Dynamics in Predicting Driving Fatigue Using a Recurrent Self-Evolving Fuzzy Neural Network.

Science.gov (United States)

Liu, Yu-Ting; Lin, Yang-Yin; Wu, Shang-Lin; Chuang, Chun-Hsiang; Lin, Chin-Teng

2016-02-01

This paper proposes a generalized prediction system called a recurrent self-evolving fuzzy neural network (RSEFNN) that employs an on-line gradient descent learning rule to address the electroencephalography (EEG) regression problem in brain dynamics for driving fatigue. The cognitive states of drivers significantly affect driving safety; in particular, fatigue driving, or drowsy driving, endangers both the individual and the public. For this reason, the development of brain-computer interfaces (BCIs) that can identify drowsy driving states is a crucial and urgent topic of study. Many EEG-based BCIs have been developed as artificial auxiliary systems for use in various practical applications because of the benefits of measuring EEG signals. In the literature, the efficacy of EEG-based BCIs in recognition tasks has been limited by low resolutions. The system proposed in this paper represents the first attempt to use the recurrent fuzzy neural network (RFNN) architecture to increase adaptability in realistic EEG applications to overcome this bottleneck. This paper further analyzes brain dynamics in a simulated car driving task in a virtual-reality environment. The proposed RSEFNN model is evaluated using the generalized cross-subject approach, and the results indicate that the RSEFNN is superior to competing models regardless of the use of recurrent or nonrecurrent structures.

Buffering social influence: neural correlates of response inhibition predict driving safety in the presence of a peer.

Science.gov (United States)

Cascio, Christopher N; Carp, Joshua; O'Donnell, Matthew Brook; Tinney, Francis J; Bingham, C Raymond; Shope, Jean T; Ouimet, Marie Claude; Pradhan, Anuj K; Simons-Morton, Bruce G; Falk, Emily B

2015-01-01

Adolescence is a period characterized by increased sensitivity to social cues, as well as increased risk-taking in the presence of peers. For example, automobile crashes are the leading cause of death for adolescents, and driving with peers increases the risk of a fatal crash. Growing evidence points to an interaction between neural systems implicated in cognitive control and social and emotional context in predicting adolescent risk. We tested such a relationship in recently licensed teen drivers. Participants completed an fMRI session in which neural activity was measured during a response inhibition task, followed by a separate driving simulator session 1 week later. Participants drove alone and with a peer who was randomly assigned to express risk-promoting or risk-averse social norms. The experimentally manipulated social context during the simulated drive moderated the relationship between individual differences in neural activity in the hypothesized cognitive control network (right inferior frontal gyrus, BG) and risk-taking in the driving context a week later. Increased activity in the response inhibition network was not associated with risk-taking in the presence of a risky peer but was significantly predictive of safer driving in the presence of a cautious peer, above and beyond self-reported susceptibility to peer pressure. Individual differences in recruitment of the response inhibition network may allow those with stronger inhibitory control to override risky tendencies when in the presence of cautious peers. This relationship between social context and individual differences in brain function expands our understanding of neural systems involved in top-down cognitive control during adolescent development.

Analysis of tribological behaviour of zirconia reinforced Al-SiC hybrid composites using statistical and artificial neural network technique

Science.gov (United States)

Arif, Sajjad; Tanwir Alam, Md; Ansari, Akhter H.; Bilal Naim Shaikh, Mohd; Arif Siddiqui, M.

2018-05-01

The tribological performance of aluminium hybrid composites reinforced with micro SiC (5 wt%) and nano zirconia (0, 3, 6 and 9 wt%) fabricated through powder metallurgy technique were investigated using statistical and artificial neural network (ANN) approach. The influence of zirconia reinforcement, sliding distance and applied load were analyzed with test based on full factorial design of experiments. Analysis of variance (ANOVA) was used to evaluate the percentage contribution of each process parameters on wear loss. ANOVA approach suggested that wear loss be mainly influenced by sliding distance followed by zirconia reinforcement and applied load. Further, a feed forward back propagation neural network was applied on input/output date for predicting and analyzing the wear behaviour of fabricated composite. A very close correlation between experimental and ANN output were achieved by implementing the model. Finally, ANN model was effectively used to find the influence of various control factors on wear behaviour of hybrid composites.

Habituation of reinforcer effectiveness

OpenAIRE

David R Lloyd; David R Lloyd; Douglas J Medina; Larry W Hawk; Whitney D Fosco; Jerry B Richards

2014-01-01

In this paper we propose an integrative model of habituation of reinforcer effectiveness (HRE) that links behavioral and neural based explanations of reinforcement. We argue that habituation of reinforcer effectiveness (HRE) is a fundamental property of reinforcing stimuli. Most reinforcement models implicitly suggest that the effectiveness of a reinforcer is stable across repeated presentations. In contrast, an HRE approach predicts decreased effectiveness due to repeated presentation. We ar...

On-Line Tracking Controller for Brushless DC Motor Drives Using Artificial Neural Networks

Science.gov (United States)

Rubaai, Ahmed

1996-01-01

A real-time control architecture is developed for time-varying nonlinear brushless dc motors operating in a high performance drives environment. The developed control architecture possesses the capabilities of simultaneous on-line identification and control. The dynamics of the motor are modeled on-line and controlled using an artificial neural network, as the system runs. The control architecture combines the experience and dependability of adaptive tracking systems with potential and promise of the neural computing technology. The sensitivity of real-time controller to parametric changes that occur during training is investigated. Such changes are usually manifested by rapid changes in the load of the brushless motor drives. This sudden change in the external load is simulated for the sigmoidal and sinusoidal reference tracks. The ability of the neuro-controller to maintain reasonable tracking accuracy in the presence of external noise is also verified for a number of desired reference trajectories.

Dissociable neural representations of reinforcement and belief prediction errors underlie strategic learning.

Science.gov (United States)

Zhu, Lusha; Mathewson, Kyle E; Hsu, Ming

2012-01-31

Decision-making in the presence of other competitive intelligent agents is fundamental for social and economic behavior. Such decisions require agents to behave strategically, where in addition to learning about the rewards and punishments available in the environment, they also need to anticipate and respond to actions of others competing for the same rewards. However, whereas we know much about strategic learning at both theoretical and behavioral levels, we know relatively little about the underlying neural mechanisms. Here, we show using a multi-strategy competitive learning paradigm that strategic choices can be characterized by extending the reinforcement learning (RL) framework to incorporate agents' beliefs about the actions of their opponents. Furthermore, using this characterization to generate putative internal values, we used model-based functional magnetic resonance imaging to investigate neural computations underlying strategic learning. We found that the distinct notions of prediction errors derived from our computational model are processed in a partially overlapping but distinct set of brain regions. Specifically, we found that the RL prediction error was correlated with activity in the ventral striatum. In contrast, activity in the ventral striatum, as well as the rostral anterior cingulate (rACC), was correlated with a previously uncharacterized belief-based prediction error. Furthermore, activity in rACC reflected individual differences in degree of engagement in belief learning. These results suggest a model of strategic behavior where learning arises from interaction of dissociable reinforcement and belief-based inputs.

Drive Control Scheme of Electric Power Assisted Wheelchair Based on Neural Network Learning of Human Wheelchair Operation Characteristics

Science.gov (United States)

Tanohata, Naoki; Seki, Hirokazu

This paper describes a novel drive control scheme of electric power assisted wheelchairs based on neural network learning of human wheelchair operation characteristics. “Electric power assisted wheelchair” which enhances the drive force of the operator by employing electric motors is expected to be widely used as a mobility support system for elderly and disabled people. However, some handicapped people with paralysis of the muscles of one side of the body cannot maneuver the wheelchair as desired because of the difference in the right and left input force. Therefore, this study proposes a neural network learning system of such human wheelchair operation characteristics and a drive control scheme with variable distribution and assistance ratios. Some driving experiments will be performed to confirm the effectiveness of the proposed control system.

Correlated evolution of male and female reproductive traits drive a cascading effect of reinforcement in Drosophila yakuba

Science.gov (United States)

Comeault, Aaron A.; Venkat, Aarti; Matute, Daniel R.

2016-01-01

Selection against maladaptive hybridization can drive the evolution of reproductive isolation in a process called reinforcement. While the importance of reinforcement in evolution has been historically debated, many examples now exist. Despite these examples, we typically lack a detailed understanding of the mechanisms limiting the spread of reinforced phenotypes throughout a species' range. Here we address this issue in the fruit fly Drosophila yakuba, a species that hybridizes with its sister species D. santomea and is undergoing reinforcement in a well-defined hybrid zone on the island of São Tomé. Within this region, female D. yakuba show increased postmating-prezygotic (gametic) isolation towards D. santomea when compared with females from allopatric populations. We use a combination of natural collections, fertility assays, and experimental evolution to understand why reinforced gametic isolation in D. yakuba is confined to this hybrid zone. We show that, among other traits, D. yakuba males from sympatric populations sire fewer progeny than allopatric males when mated to allopatric D. yakuba females. Our results provide a novel example of reinforcement acting on a postmating-prezygotic trait in males, resulting in a cascade of reproductive isolation among conspecific populations. PMID:27440664

Synaptic energy drives the information processing mechanisms in spiking neural networks.

Science.gov (United States)

El Laithy, Karim; Bogdan, Martin

2014-04-01

Flow of energy and free energy minimization underpins almost every aspect of naturally occurring physical mechanisms. Inspired by this fact this work establishes an energy-based framework that spans the multi-scale range of biological neural systems and integrates synaptic dynamic, synchronous spiking activity and neural states into one consistent working paradigm. Following a bottom-up approach, a hypothetical energy function is proposed for dynamic synaptic models based on the theoretical thermodynamic principles and the Hopfield networks. We show that a synapse exposes stable operating points in terms of its excitatory postsynaptic potential as a function of its synaptic strength. We postulate that synapses in a network operating at these stable points can drive this network to an internal state of synchronous firing. The presented analysis is related to the widely investigated temporal coherent activities (cell assemblies) over a certain range of time scales (binding-by-synchrony). This introduces a novel explanation of the observed (poly)synchronous activities within networks regarding the synaptic (coupling) functionality. On a network level the transitions from one firing scheme to the other express discrete sets of neural states. The neural states exist as long as the network sustains the internal synaptic energy.

Habituation of reinforcer effectiveness

Directory of Open Access Journals (Sweden)

David R Lloyd

2014-01-01

Full Text Available In this paper we propose an integrative model of habituation of reinforcer effectiveness (HRE that links behavioral and neural based explanations of reinforcement. We argue that habituation of reinforcer effectiveness (HRE is a fundamental property of reinforcing stimuli. Most reinforcement models implicitly suggest that the effectiveness of a reinforcer is stable across repeated presentations. In contrast, an HRE approach predicts decreased effectiveness due to repeated presentation. We argue that repeated presentation of reinforcing stimuli decreases their effectiveness and that these decreases are described by the behavioral characteristics of habituation (McSweeney and Murphy, 2009;Rankin et al., 2009. We describe a neural model that postulates a positive association between dopamine neurotransmission and HRE. We present evidence that stimulant drugs, which artificially increase dopamine neurotransmission, disrupt (slow normally occurring HRE and also provide evidence that stimulant drugs have differential effects on operant responding maintained by reinforcers with rapid vs. slow HRE rates. We hypothesize that abnormal HRE due to genetic and/or environmental factors may underlie some behavioral disorders. For example, recent research indicates that slow-HRE is predictive of obesity. In contrast ADHD may reflect ‘accelerated-HRE’. Consideration of HRE is important for the development of effective reinforcement based treatments. Finally, we point out that most of the reinforcing stimuli that regulate daily behavior are non-consumable environmental/social reinforcers which have rapid-HRE. The almost exclusive use of consumable reinforcers with slow-HRE in pre-clinical studies with animals may have caused the importance of HRE to be overlooked. Further study of reinforcing stimuli with rapid-HRE is needed in order to understand how habituation and reinforcement interact and regulate behavior.

Reinforcement principles for addiction medicine; from recreational drug use to psychiatric disorder.

Science.gov (United States)

Edwards, Scott

2016-01-01

The transition from recreational drug use to addiction can be conceptualized as a pathological timeline whereby the psychological mechanisms responsible for disordered drug use evolve from positive reinforcement to favor elements of negative reinforcement. Abused substances (ranging from alcohol to psychostimulants) are initially ingested at regular occasions according to their positive reinforcing properties. Importantly, repeated exposure to rewarding substances sets off a chain of secondary reinforcing events, whereby cues and contexts associated with drug use may themselves become reinforcing and thereby contribute to the continued use and possible abuse of the substance(s) of choice. Indeed, the powerful reinforcing efficacy of certain drugs may eclipse that of competing social rewards (such as career and family) and lead to an aberrant narrowing of behavioral repertoire. In certain vulnerable individuals, escalation of drug use over time is thought to drive specific molecular neuroadaptations that foster the development of addiction. Research has identified neurobiological elements of altered reinforcement following excessive drug use that comprise within-circuit and between-circuit neuroadaptations, both of which contribute to addiction. Central to this process is the eventual potentiation of negative reinforcement mechanisms that may represent the final definitive criterion locking vulnerable individuals into a persistent state of addiction. Targeting the neural substrates of reinforcement likely represents our best chances for therapeutic intervention for this devastating disease. © 2016 Elsevier B.V. All rights reserved.

Prediction of strain values in reinforcements and concrete of a RC frame using neural networks

Science.gov (United States)

Vafaei, Mohammadreza; Alih, Sophia C.; Shad, Hossein; Falah, Ali; Halim, Nur Hajarul Falahi Abdul

2018-03-01

The level of strain in structural elements is an important indicator for the presence of damage and its intensity. Considering this fact, often structural health monitoring systems employ strain gauges to measure strains in critical elements. However, because of their sensitivity to the magnetic fields, inadequate long-term durability especially in harsh environments, difficulties in installation on existing structures, and maintenance cost, installation of strain gauges is not always possible for all structural components. Therefore, a reliable method that can accurately estimate strain values in critical structural elements is necessary for damage identification. In this study, a full-scale test was conducted on a planar RC frame to investigate the capability of neural networks for predicting the strain values. Two neural networks each of which having a single hidden layer was trained to relate the measured rotations and vertical displacements of the frame to the strain values measured at different locations of the frame. Results of trained neural networks indicated that they accurately estimated the strain values both in reinforcements and concrete. In addition, the trained neural networks were capable of predicting strains for the unseen input data set.

Habituation of reinforcer effectiveness.

Science.gov (United States)

Lloyd, David R; Medina, Douglas J; Hawk, Larry W; Fosco, Whitney D; Richards, Jerry B

2014-01-09

In this paper we propose an integrative model of habituation of reinforcer effectiveness (HRE) that links behavioral- and neural-based explanations of reinforcement. We argue that HRE is a fundamental property of reinforcing stimuli. Most reinforcement models implicitly suggest that the effectiveness of a reinforcer is stable across repeated presentations. In contrast, an HRE approach predicts decreased effectiveness due to repeated presentation. We argue that repeated presentation of reinforcing stimuli decreases their effectiveness and that these decreases are described by the behavioral characteristics of habituation (McSweeney and Murphy, 2009; Rankin etal., 2009). We describe a neural model that postulates a positive association between dopamine neurotransmission and HRE. We present evidence that stimulant drugs, which artificially increase dopamine neurotransmission, disrupt (slow) normally occurring HRE and also provide evidence that stimulant drugs have differential effects on operant responding maintained by reinforcers with rapid vs. slow HRE rates. We hypothesize that abnormal HRE due to genetic and/or environmental factors may underlie some behavioral disorders. For example, recent research indicates that slow-HRE is predictive of obesity. In contrast ADHD may reflect "accelerated-HRE." Consideration of HRE is important for the development of effective reinforcement-based treatments. Finally, we point out that most of the reinforcing stimuli that regulate daily behavior are non-consumable environmental/social reinforcers which have rapid-HRE. The almost exclusive use of consumable reinforcers with slow-HRE in pre-clinical studies with animals may have caused the importance of HRE to be overlooked. Further study of reinforcing stimuli with rapid-HRE is needed in order to understand how habituation and reinforcement interact and regulate behavior.

Vicarious Reinforcement In Rhesus Macaques (Macaca mulatta

Directory of Open Access Journals (Sweden)

Steve W. C. Chang

2011-03-01

Full Text Available What happens to others profoundly influences our own behavior. Such other-regarding outcomes can drive observational learning, as well as motivate cooperation, charity, empathy, and even spite. Vicarious reinforcement may serve as one of the critical mechanisms mediating the influence of other-regarding outcomes on behavior and decision-making in groups. Here we show that rhesus macaques spontaneously derive vicarious reinforcement from observing rewards given to another monkey, and that this reinforcement can motivate them to subsequently deliver or withhold rewards from the other animal. We exploited Pavlovian and instrumental conditioning to associate rewards to self (M1 and/or rewards to another monkey (M2 with visual cues. M1s made more errors in the instrumental trials when cues predicted reward to M2 compared to when cues predicted reward to M1, but made even more errors when cues predicted reward to no one. In subsequent preference tests between pairs of conditioned cues, M1s preferred cues paired with reward to M2 over cues paired with reward to no one. By contrast, M1s preferred cues paired with reward to self over cues paired with reward to both monkeys simultaneously. Rates of attention to M2 strongly predicted the strength and valence of vicarious reinforcement. These patterns of behavior, which were absent in nonsocial control trials, are consistent with vicarious reinforcement based upon sensitivity to observed, or counterfactual, outcomes with respect to another individual. Vicarious reward may play a critical role in shaping cooperation and competition, as well as motivating observational learning and group coordination in rhesus macaques, much as it does in humans. We propose that vicarious reinforcement signals mediate these behaviors via homologous neural circuits involved in reinforcement learning and decision-making.

Vicarious reinforcement in rhesus macaques (macaca mulatta).

Science.gov (United States)

Chang, Steve W C; Winecoff, Amy A; Platt, Michael L

2011-01-01

What happens to others profoundly influences our own behavior. Such other-regarding outcomes can drive observational learning, as well as motivate cooperation, charity, empathy, and even spite. Vicarious reinforcement may serve as one of the critical mechanisms mediating the influence of other-regarding outcomes on behavior and decision-making in groups. Here we show that rhesus macaques spontaneously derive vicarious reinforcement from observing rewards given to another monkey, and that this reinforcement can motivate them to subsequently deliver or withhold rewards from the other animal. We exploited Pavlovian and instrumental conditioning to associate rewards to self (M1) and/or rewards to another monkey (M2) with visual cues. M1s made more errors in the instrumental trials when cues predicted reward to M2 compared to when cues predicted reward to M1, but made even more errors when cues predicted reward to no one. In subsequent preference tests between pairs of conditioned cues, M1s preferred cues paired with reward to M2 over cues paired with reward to no one. By contrast, M1s preferred cues paired with reward to self over cues paired with reward to both monkeys simultaneously. Rates of attention to M2 strongly predicted the strength and valence of vicarious reinforcement. These patterns of behavior, which were absent in non-social control trials, are consistent with vicarious reinforcement based upon sensitivity to observed, or counterfactual, outcomes with respect to another individual. Vicarious reward may play a critical role in shaping cooperation and competition, as well as motivating observational learning and group coordination in rhesus macaques, much as it does in humans. We propose that vicarious reinforcement signals mediate these behaviors via homologous neural circuits involved in reinforcement learning and decision-making.

On-road magnetic emissions prediction of electric cars in terms of driving dynamics using neural networks

NARCIS (Netherlands)

Wefky, Ahmed M.; Espinosa, Felipe; Leferink, Frank Bernardus Johannes; Gardel, Alfredo; Vogt-Ardatjew, R.A.

2013-01-01

This paper presents a novel artificial neural network (ANN) model estimating vehicle-level radiated magnetic emissions of an electric car as a function of the corresponding driving pattern. Real world electromagnetic interference (EMI) experiments have been realized in a semi-anechoic chamber using

Dynamic neural networks based on-line identification and control of high performance motor drives

Science.gov (United States)

Rubaai, Ahmed; Kotaru, Raj

1995-01-01

In the automated and high-tech industries of the future, there wil be a need for high performance motor drives both in the low-power range and in the high-power range. To meet very straight demands of tracking and regulation in the two quadrants of operation, advanced control technologies are of a considerable interest and need to be developed. In response a dynamics learning control architecture is developed with simultaneous on-line identification and control. the feature of the proposed approach, to efficiently combine the dual task of system identification (learning) and adaptive control of nonlinear motor drives into a single operation is presented. This approach, therefore, not only adapts to uncertainties of the dynamic parameters of the motor drives but also learns about their inherent nonlinearities. In fact, most of the neural networks based adaptive control approaches in use have an identification phase entirely separate from the control phase. Because these approaches separate the identification and control modes, it is not possible to cope with dynamic changes in a controlled process. Extensive simulation studies have been conducted and good performance was observed. The robustness characteristics of neuro-controllers to perform efficiently in a noisy environment is also demonstrated. With this initial success, the principal investigator believes that the proposed approach with the suggested neural structure can be used successfully for the control of high performance motor drives. Two identification and control topologies based on the model reference adaptive control technique are used in this present analysis. No prior knowledge of load dynamics is assumed in either topology while the second topology also assumes no knowledge of the motor parameters.

Rational and Mechanistic Perspectives on Reinforcement Learning

Science.gov (United States)

Chater, Nick

2009-01-01

This special issue describes important recent developments in applying reinforcement learning models to capture neural and cognitive function. But reinforcement learning, as a theoretical framework, can apply at two very different levels of description: "mechanistic" and "rational." Reinforcement learning is often viewed in mechanistic terms--as…

Damage Level Prediction of Reinforced Concrete Building Based on Earthquake Time History Using Artificial Neural Network

Directory of Open Access Journals (Sweden)

Suryanita Reni

2017-01-01

Full Text Available The strong motion earthquake could cause the building damage in case of the building not considered in the earthquake design of the building. The study aims to predict the damage-level of building due to earthquake using Artificial Neural Networks method. The building model is a reinforced concrete building with ten floors and height between floors is 3.6 m. The model building received a load of the earthquake based on nine earthquake time history records. Each time history scaled to 0,5g, 0,75g, and 1,0g. The Artificial Neural Networks are designed in 4 architectural models using the MATLAB program. Model 1 used the displacement, velocity, and acceleration as input and Model 2 used the displacement only as the input. Model 3 used the velocity as input, and Model 4 used the acceleration just as input. The output of the Neural Networks is the damage level of the building with the category of Safe (1, Immediate Occupancy (2, Life Safety (3 or in a condition of Collapse Prevention (4. According to the results, Neural Network models have the prediction rate of the damage level between 85%-95%. Therefore, one of the solutions for analyzing the structural responses and the damage level promptly and efficiently when the earthquake occurred is by using Artificial Neural Network

Shear strength estimation of the concrete beams reinforced with FRP; comparison of artificial neural network and equations of regulations

Directory of Open Access Journals (Sweden)

Mahmood Akbari

2017-12-01

Full Text Available In recent years, numerous experimental tests were done on the concrete beams reinforced with the fiber-reinforced polymer (FRP. In this way, some equations were proposed to estimate the shear strength of the beams reinforced with FRP. The aim of this study is to explore the feasibility of using a feed-forward artificial neural network (ANN model to predict the ultimate shear strength of the beams strengthened with FRP composites. For this purpose, a database consists of 304 reinforced FRP concrete beams have been collected from the available articles on the analysis of shear behavior of these beams. The inputs to the ANN model consists of the 11 variables including the geometric dimensions of the section, steel reinforcement amount, FRP amount and the properties of the concrete, steel reinforcement and FRP materials while the output variable is the shear strength of the FRP beam. To assess the performance of the ANN model for estimating the shear strength of the reinforced beams, the outputs of the ANN are compared to those of equations of the Iranian code (Publication No. 345 and the American code (ACI 440. The comparisons between the outputs of Iran and American regulations with those of the proposed model indicates that the predictive power of this model is much better than the experimental codes. Specifically, for under study data, mean absolute relative error (MARE criteria is 13%, 34% and 39% for the ANN model, the American and the Iranian codes, respectively.

A plausible neural circuit for decision making and its formation based on reinforcement learning.

Science.gov (United States)

Wei, Hui; Dai, Dawei; Bu, Yijie

2017-06-01

A human's, or lower insects', behavior is dominated by its nervous system. Each stable behavior has its own inner steps and control rules, and is regulated by a neural circuit. Understanding how the brain influences perception, thought, and behavior is a central mandate of neuroscience. The phototactic flight of insects is a widely observed deterministic behavior. Since its movement is not stochastic, the behavior should be dominated by a neural circuit. Based on the basic firing characteristics of biological neurons and the neural circuit's constitution, we designed a plausible neural circuit for this phototactic behavior from logic perspective. The circuit's output layer, which generates a stable spike firing rate to encode flight commands, controls the insect's angular velocity when flying. The firing pattern and connection type of excitatory and inhibitory neurons are considered in this computational model. We simulated the circuit's information processing using a distributed PC array, and used the real-time average firing rate of output neuron clusters to drive a flying behavior simulation. In this paper, we also explored how a correct neural decision circuit is generated from network flow view through a bee's behavior experiment based on the reward and punishment feedback mechanism. The significance of this study: firstly, we designed a neural circuit to achieve the behavioral logic rules by strictly following the electrophysiological characteristics of biological neurons and anatomical facts. Secondly, our circuit's generality permits the design and implementation of behavioral logic rules based on the most general information processing and activity mode of biological neurons. Thirdly, through computer simulation, we achieved new understanding about the cooperative condition upon which multi-neurons achieve some behavioral control. Fourthly, this study aims in understanding the information encoding mechanism and how neural circuits achieve behavior control

Construction of multi-agent mobile robots control system in the problem of persecution with using a modified reinforcement learning method based on neural networks

Science.gov (United States)

Patkin, M. L.; Rogachev, G. N.

2018-02-01

A method for constructing a multi-agent control system for mobile robots based on training with reinforcement using deep neural networks is considered. Synthesis of the management system is proposed to be carried out with reinforcement training and the modified Actor-Critic method, in which the Actor module is divided into Action Actor and Communication Actor in order to simultaneously manage mobile robots and communicate with partners. Communication is carried out by sending partners at each step a vector of real numbers that are added to the observation vector and affect the behaviour. Functions of Actors and Critic are approximated by deep neural networks. The Critics value function is trained by using the TD-error method and the Actor’s function by using DDPG. The Communication Actor’s neural network is trained through gradients received from partner agents. An environment in which a cooperative multi-agent interaction is present was developed, computer simulation of the application of this method in the control problem of two robots pursuing two goals was carried out.

Neural Crest-Derived Mesenchymal Cells Require Wnt Signaling for Their Development and Drive Invagination of the Telencephalic Midline

Science.gov (United States)

Choe, Youngshik; Zarbalis, Konstantinos S.; Pleasure, Samuel J.

2014-01-01

Embryonic neural crest cells contribute to the development of the craniofacial mesenchyme, forebrain meninges and perivascular cells. In this study, we investigated the function of ß-catenin signaling in neural crest cells abutting the dorsal forebrain during development. In the absence of ß-catenin signaling, neural crest cells failed to expand in the interhemispheric region and produced ectopic smooth muscle cells instead of generating dermal and calvarial mesenchyme. In contrast, constitutive expression of stabilized ß-catenin in neural crest cells increased the number of mesenchymal lineage precursors suggesting that ß-catenin signaling is necessary for the expansion of neural crest-derived mesenchymal cells. Interestingly, the loss of neural crest-derived mesenchymal stem cells (MSCs) leads to failure of telencephalic midline invagination and causes ventricular system defects. This study shows that ß-catenin signaling is required for the switch of neural crest cells to MSCs and mediates the expansion of MSCs to drive the formation of mesenchymal structures of the head. Furthermore, loss of these structures causes striking defects in forebrain morphogenesis. PMID:24516524

Neural crest-derived mesenchymal cells require Wnt signaling for their development and drive invagination of the telencephalic midline.

Directory of Open Access Journals (Sweden)

Youngshik Choe

Full Text Available Embryonic neural crest cells contribute to the development of the craniofacial mesenchyme, forebrain meninges and perivascular cells. In this study, we investigated the function of ß-catenin signaling in neural crest cells abutting the dorsal forebrain during development. In the absence of ß-catenin signaling, neural crest cells failed to expand in the interhemispheric region and produced ectopic smooth muscle cells instead of generating dermal and calvarial mesenchyme. In contrast, constitutive expression of stabilized ß-catenin in neural crest cells increased the number of mesenchymal lineage precursors suggesting that ß-catenin signaling is necessary for the expansion of neural crest-derived mesenchymal cells. Interestingly, the loss of neural crest-derived mesenchymal stem cells (MSCs leads to failure of telencephalic midline invagination and causes ventricular system defects. This study shows that ß-catenin signaling is required for the switch of neural crest cells to MSCs and mediates the expansion of MSCs to drive the formation of mesenchymal structures of the head. Furthermore, loss of these structures causes striking defects in forebrain morphogenesis.

GA-based fuzzy reinforcement learning for control of a magnetic bearing system.

Science.gov (United States)

Lin, C T; Jou, C P

2000-01-01

This paper proposes a TD (temporal difference) and GA (genetic algorithm)-based reinforcement (TDGAR) learning method and applies it to the control of a real magnetic bearing system. The TDGAR learning scheme is a new hybrid GA, which integrates the TD prediction method and the GA to perform the reinforcement learning task. The TDGAR learning system is composed of two integrated feedforward networks. One neural network acts as a critic network to guide the learning of the other network (the action network) which determines the outputs (actions) of the TDGAR learning system. The action network can be a normal neural network or a neural fuzzy network. Using the TD prediction method, the critic network can predict the external reinforcement signal and provide a more informative internal reinforcement signal to the action network. The action network uses the GA to adapt itself according to the internal reinforcement signal. The key concept of the TDGAR learning scheme is to formulate the internal reinforcement signal as the fitness function for the GA such that the GA can evaluate the candidate solutions (chromosomes) regularly, even during periods without external feedback from the environment. This enables the GA to proceed to new generations regularly without waiting for the arrival of the external reinforcement signal. This can usually accelerate the GA learning since a reinforcement signal may only be available at a time long after a sequence of actions has occurred in the reinforcement learning problem. The proposed TDGAR learning system has been used to control an active magnetic bearing (AMB) system in practice. A systematic design procedure is developed to achieve successful integration of all the subsystems including magnetic suspension, mechanical structure, and controller training. The results show that the TDGAR learning scheme can successfully find a neural controller or a neural fuzzy controller for a self-designed magnetic bearing system.

Could LC-NE-Dependent Adjustment of Neural Gain Drive Functional Brain Network Reorganization?

Directory of Open Access Journals (Sweden)

Carole Guedj

2017-01-01

Full Text Available The locus coeruleus-norepinephrine (LC-NE system is thought to act at synaptic, cellular, microcircuit, and network levels to facilitate cognitive functions through at least two different processes, not mutually exclusive. Accordingly, as a reset signal, the LC-NE system could trigger brain network reorganizations in response to salient information in the environment and/or adjust the neural gain within its target regions to optimize behavioral responses. Here, we provide evidence of the co-occurrence of these two mechanisms at the whole-brain level, in resting-state conditions following a pharmacological stimulation of the LC-NE system. We propose that these two mechanisms are interdependent such that the LC-NE-dependent adjustment of the neural gain inferred from the clustering coefficient could drive functional brain network reorganizations through coherence in the gamma rhythm. Via the temporal dynamic of gamma-range band-limited power, the release of NE could adjust the neural gain, promoting interactions only within the neuronal populations whose amplitude envelopes are correlated, thus making it possible to reorganize neuronal ensembles, functional networks, and ultimately, behavioral responses. Thus, our proposal offers a unified framework integrating the putative influence of the LC-NE system on both local- and long-range adjustments of brain dynamics underlying behavioral flexibility.

CAPES: Unsupervised Storage Performance Tuning Using Neural Network-Based Deep Reinforcement Learning

CERN Multimedia

CERN. Geneva

2017-01-01

Parameter tuning is an important task of storage performance optimization. Current practice usually involves numerous tweak-benchmark cycles that are slow and costly. To address this issue, we developed CAPES, a model-less deep reinforcement learning-based unsupervised parameter tuning system driven by a deep neural network (DNN). It is designed to nd the optimal values of tunable parameters in computer systems, from a simple client-server system to a large data center, where human tuning can be costly and often cannot achieve optimal performance. CAPES takes periodic measurements of a target computer system’s state, and trains a DNN which uses Q-learning to suggest changes to the system’s current parameter values. CAPES is minimally intrusive, and can be deployed into a production system to collect training data and suggest tuning actions during the system’s daily operation. Evaluation of a prototype on a Lustre system demonstrates an increase in I/O throughput up to 45% at saturation point. About the...

Phasic dopamine as a prediction error of intrinsic and extrinsic reinforcements driving both action acquisition and reward maximization: a simulated robotic study.

Science.gov (United States)

Mirolli, Marco; Santucci, Vieri G; Baldassarre, Gianluca

2013-03-01

An important issue of recent neuroscientific research is to understand the functional role of the phasic release of dopamine in the striatum, and in particular its relation to reinforcement learning. The literature is split between two alternative hypotheses: one considers phasic dopamine as a reward prediction error similar to the computational TD-error, whose function is to guide an animal to maximize future rewards; the other holds that phasic dopamine is a sensory prediction error signal that lets the animal discover and acquire novel actions. In this paper we propose an original hypothesis that integrates these two contrasting positions: according to our view phasic dopamine represents a TD-like reinforcement prediction error learning signal determined by both unexpected changes in the environment (temporary, intrinsic reinforcements) and biological rewards (permanent, extrinsic reinforcements). Accordingly, dopamine plays the functional role of driving both the discovery and acquisition of novel actions and the maximization of future rewards. To validate our hypothesis we perform a series of experiments with a simulated robotic system that has to learn different skills in order to get rewards. We compare different versions of the system in which we vary the composition of the learning signal. The results show that only the system reinforced by both extrinsic and intrinsic reinforcements is able to reach high performance in sufficiently complex conditions. Copyright © 2013 Elsevier Ltd. All rights reserved.

Multi-Objective Reinforcement Learning-Based Deep Neural Networks for Cognitive Space Communications

Science.gov (United States)

Ferreria, Paulo Victor R.; Paffenroth, Randy; Wyglinski, Alexander M.; Hackett, Timothy M.; Bilen, Sven G.; Reinhart, Richard C.; Mortensen, Dale J.

2017-01-01

Future communication subsystems of space exploration missions can potentially benefit from software-defined radios (SDRs) controlled by machine learning algorithms. In this paper, we propose a novel hybrid radio resource allocation management control algorithm that integrates multi-objective reinforcement learning and deep artificial neural networks. The objective is to efficiently manage communications system resources by monitoring performance functions with common dependent variables that result in conflicting goals. The uncertainty in the performance of thousands of different possible combinations of radio parameters makes the trade-off between exploration and exploitation in reinforcement learning (RL) much more challenging for future critical space-based missions. Thus, the system should spend as little time as possible on exploring actions, and whenever it explores an action, it should perform at acceptable levels most of the time. The proposed approach enables on-line learning by interactions with the environment and restricts poor resource allocation performance through virtual environment exploration. Improvements in the multiobjective performance can be achieved via transmitter parameter adaptation on a packet-basis, with poorly predicted performance promptly resulting in rejected decisions. Simulations presented in this work considered the DVB-S2 standard adaptive transmitter parameters and additional ones expected to be present in future adaptive radio systems. Performance results are provided by analysis of the proposed hybrid algorithm when operating across a satellite communication channel from Earth to GEO orbit during clear sky conditions. The proposed approach constitutes part of the core cognitive engine proof-of-concept to be delivered to the NASA Glenn Research Center SCaN Testbed located onboard the International Space Station.

Instructional control of reinforcement learning: a behavioral and neurocomputational investigation.

Science.gov (United States)

Doll, Bradley B; Jacobs, W Jake; Sanfey, Alan G; Frank, Michael J

2009-11-24

Humans learn how to behave directly through environmental experience and indirectly through rules and instructions. Behavior analytic research has shown that instructions can control behavior, even when such behavior leads to sub-optimal outcomes (Hayes, S. (Ed.). 1989. Rule-governed behavior: cognition, contingencies, and instructional control. Plenum Press.). Here we examine the control of behavior through instructions in a reinforcement learning task known to depend on striatal dopaminergic function. Participants selected between probabilistically reinforced stimuli, and were (incorrectly) told that a specific stimulus had the highest (or lowest) reinforcement probability. Despite experience to the contrary, instructions drove choice behavior. We present neural network simulations that capture the interactions between instruction-driven and reinforcement-driven behavior via two potential neural circuits: one in which the striatum is inaccurately trained by instruction representations coming from prefrontal cortex/hippocampus (PFC/HC), and another in which the striatum learns the environmentally based reinforcement contingencies, but is "overridden" at decision output. Both models capture the core behavioral phenomena but, because they differ fundamentally on what is learned, make distinct predictions for subsequent behavioral and neuroimaging experiments. Finally, we attempt to distinguish between the proposed computational mechanisms governing instructed behavior by fitting a series of abstract "Q-learning" and Bayesian models to subject data. The best-fitting model supports one of the neural models, suggesting the existence of a "confirmation bias" in which the PFC/HC system trains the reinforcement system by amplifying outcomes that are consistent with instructions while diminishing inconsistent outcomes.

Reinforcing Saccadic Amplitude Variability

Science.gov (United States)

Paeye, Celine; Madelain, Laurent

2011-01-01

Saccadic endpoint variability is often viewed as the outcome of neural noise occurring during sensorimotor processing. However, part of this variability might result from operant learning. We tested this hypothesis by reinforcing dispersions of saccadic amplitude distributions, while maintaining constant their medians. In a first experiment we…

Goal-seeking neural net for recall and recognition

Science.gov (United States)

Omidvar, Omid M.

1990-07-01

Neural networks have been used to mimic cognitive processes which take place in animal brains. The learning capability inherent in neural networks makes them suitable candidates for adaptive tasks such as recall and recognition. The synaptic reinforcements create a proper condition for adaptation, which results in memorization, formation of perception, and higher order information processing activities. In this research a model of a goal seeking neural network is studied and the operation of the network with regard to recall and recognition is analyzed. In these analyses recall is defined as retrieval of stored information where little or no matching is involved. On the other hand recognition is recall with matching; therefore it involves memorizing a piece of information with complete presentation. This research takes the generalized view of reinforcement in which all the signals are potential reinforcers. The neuronal response is considered to be the source of the reinforcement. This local approach to adaptation leads to the goal seeking nature of the neurons as network components. In the proposed model all the synaptic strengths are reinforced in parallel while the reinforcement among the layers is done in a distributed fashion and pipeline mode from the last layer inward. A model of complex neuron with varying threshold is developed to account for inhibitory and excitatory behavior of real neuron. A goal seeking model of a neural network is presented. This network is utilized to perform recall and recognition tasks. The performance of the model with regard to the assigned tasks is presented.

Safety Assessment for Electrical Motor Drive System Based on SOM Neural Network

Directory of Open Access Journals (Sweden)

Linghui Meng

2016-01-01

Full Text Available With the development of the urban rail train, safety and reliability have become more and more important. In this paper, the fault degree and health degree of the system are put forward based on the analysis of electric motor drive system’s control principle. With the self-organizing neural network’s advantage of competitive learning and unsupervised clustering, the system’s health clustering and safety identification are worked out. With the switch devices’ faults data obtained from the dSPACE simulation platform, the health assessment algorithm is verified. And the results show that the algorithm can achieve the system’s fault diagnosis and health assessment, which has a point in the health assessment and maintenance for the train.

Effects of reinforcement-blocking doses of pimozide on neural systems driven by rewarding stimulation of the MFB: a /sup 14/C-2-deoxyglucose analysis

Energy Technology Data Exchange (ETDEWEB)

Gomita, Y.; Gallistel, C.R.

1982-10-01

An analysis by means of /sup 14/C-2-deoxyglucose autoradiography of the neural systems unilaterally activated by the reinforcing stimulation used in the two accompanying papers revealed strong and reliable effects in the nucleus of the diagonal band of Broca, in the medial forebrain bundle (MFB) and/or the fornix throughout the diencephalon, and in the part of the anterior ventral tegmentum where the dopaminergic projection to the lateral habenula originates. The terminal fields of the dopaminergic forebrain projections were not affected, but there was bilateral suppression of lateral habenular activity. A second experiment found that the same systems are still activated by (automatically administered) reinforcing stimulation in rats treated with reinforcement blocking doses of pimozide. The only clear effect of pimozide was to reverse the bilateral suppressive effect of the stimulation on lateral habenular activity. Animals treated with pimozide show greatly elevated activity in the lateral habenula, whether or not they receive reinforcing stimulation. The results suggest that pimozide's effect on reinforcement is mediated by the circuitry interconnecting the lateral habenula with the nucleus of the diagonal band of Broca and/or the anterior ventral tegmentum.

The Racer’s Brain – How Domain Expertise is Reflected in the Neural Substrates of Driving

Directory of Open Access Journals (Sweden)

Otto eLappi

2015-11-01

Full Text Available A fundamental question in human brain plasticity is how sensory, motor, and cognitive functions adapt in the process of skill acquisition extended over a period of many years. Recently, there has emerged a growing interest in cognitive neuroscience on studying the functional and structural differences in the brains of elite athletes. Elite performance in sports, music or the arts, allows us to observe sensorimotor and cognitive performance at the limits of human capability. In this mini-review we look at driving expertise. The emerging brain imaging literature on the neural substrates of real and simulated driving is reviewed (for the first time, and used as the context for interpreting recent findings on the differences between racing drivers and non-athlete controls. Also the cognitive psychology and cognitive neuroscience of expertise are discussed.

Closed loop interactions between spiking neural network and robotic simulators based on MUSIC and ROS

Directory of Open Access Journals (Sweden)

Philipp Weidel

2016-08-01

Full Text Available In order to properly assess the function and computational properties of simulated neural systems, it is necessary to account for the nature of the stimuli that drive the system. However, providing stimuli that are rich and yet both reproducible and amenable to experimental manipulations is technically challenging, and even more so if a closed-loop scenario is required. In this work, we present a novel approach to solve this problem, connecting robotics and neural network simulators. We implement a middleware solution that bridges the Robotic Operating System (ROS to the Multi-Simulator Coordinator (MUSIC. This enables any robotic and neural simulators that implement the corresponding interfaces to be efficiently coupled, allowing real-time performance for a wide range of configurations. This work extends the toolset available for researchers in both neurorobotics and computational neuroscience, and creates the opportunity to perform closed-loop experiments of arbitrary complexity to address questions in multiple areas, including embodiment, agency, and reinforcement learning.

Evidence for a neural law of effect.

Science.gov (United States)

Athalye, Vivek R; Santos, Fernando J; Carmena, Jose M; Costa, Rui M

2018-03-02

Thorndike's law of effect states that actions that lead to reinforcements tend to be repeated more often. Accordingly, neural activity patterns leading to reinforcement are also reentered more frequently. Reinforcement relies on dopaminergic activity in the ventral tegmental area (VTA), and animals shape their behavior to receive dopaminergic stimulation. Seeking evidence for a neural law of effect, we found that mice learn to reenter more frequently motor cortical activity patterns that trigger optogenetic VTA self-stimulation. Learning was accompanied by gradual shaping of these patterns, with participating neurons progressively increasing and aligning their covariance to that of the target pattern. Motor cortex patterns that lead to phasic dopaminergic VTA activity are progressively reinforced and shaped, suggesting a mechanism by which animals select and shape actions to reliably achieve reinforcement. Copyright © 2018 The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original U.S. Government Works.

Kernel Temporal Differences for Neural Decoding

Science.gov (United States)

Bae, Jihye; Sanchez Giraldo, Luis G.; Pohlmeyer, Eric A.; Francis, Joseph T.; Sanchez, Justin C.; Príncipe, José C.

2015-01-01

We study the feasibility and capability of the kernel temporal difference (KTD)(λ) algorithm for neural decoding. KTD(λ) is an online, kernel-based learning algorithm, which has been introduced to estimate value functions in reinforcement learning. This algorithm combines kernel-based representations with the temporal difference approach to learning. One of our key observations is that by using strictly positive definite kernels, algorithm's convergence can be guaranteed for policy evaluation. The algorithm's nonlinear functional approximation capabilities are shown in both simulations of policy evaluation and neural decoding problems (policy improvement). KTD can handle high-dimensional neural states containing spatial-temporal information at a reasonable computational complexity allowing real-time applications. When the algorithm seeks a proper mapping between a monkey's neural states and desired positions of a computer cursor or a robot arm, in both open-loop and closed-loop experiments, it can effectively learn the neural state to action mapping. Finally, a visualization of the coadaptation process between the decoder and the subject shows the algorithm's capabilities in reinforcement learning brain machine interfaces. PMID:25866504

Pragmatically Framed Cross-Situational Noun Learning Using Computational Reinforcement Models.

Science.gov (United States)

Najnin, Shamima; Banerjee, Bonny

2018-01-01

Cross-situational learning and social pragmatic theories are prominent mechanisms for learning word meanings (i.e., word-object pairs). In this paper, the role of reinforcement is investigated for early word-learning by an artificial agent. When exposed to a group of speakers, the agent comes to understand an initial set of vocabulary items belonging to the language used by the group. Both cross-situational learning and social pragmatic theory are taken into account. As social cues, joint attention and prosodic cues in caregiver's speech are considered. During agent-caregiver interaction, the agent selects a word from the caregiver's utterance and learns the relations between that word and the objects in its visual environment. The "novel words to novel objects" language-specific constraint is assumed for computing rewards. The models are learned by maximizing the expected reward using reinforcement learning algorithms [i.e., table-based algorithms: Q-learning, SARSA, SARSA-λ, and neural network-based algorithms: Q-learning for neural network (Q-NN), neural-fitted Q-network (NFQ), and deep Q-network (DQN)]. Neural network-based reinforcement learning models are chosen over table-based models for better generalization and quicker convergence. Simulations are carried out using mother-infant interaction CHILDES dataset for learning word-object pairings. Reinforcement is modeled in two cross-situational learning cases: (1) with joint attention (Attentional models), and (2) with joint attention and prosodic cues (Attentional-prosodic models). Attentional-prosodic models manifest superior performance to Attentional ones for the task of word-learning. The Attentional-prosodic DQN outperforms existing word-learning models for the same task.

Design, Fabrication and Testing of Carbon Fiber Reinforced Epoxy Drive Shaft for All Terrain Vehicle using Filament Winding

Directory of Open Access Journals (Sweden)

Yeshwant Nayak Suhas

2018-01-01

Full Text Available Filament winding is a composite material fabrication technique that is used to manufacture concentric hollow components. In this study Carbon/Epoxy composite drive shafts were fabricated using filament winding process with a fiber orientation of [852/±452/252]s. Carbon in the form of multifilament fibers of Tairyfil TC-33 having 3000 filaments/strand was used as reinforcement with low viscosity epoxy resin as the matrix material. The driveshaft is designed to be used in SAE Baja All Terrain Vehicle (ATV that makes use of a fully floating axle in its rear wheel drive system. The torsional strength of the shaft was tested and compared to that of an OEM steel shaft that was previously used in the ATV. Results show that the composite shaft had 8.5% higher torsional strength in comparison to the OEM steel shaft and was also lighter by 60%. Scanning electron microscopy (SEM micrographs were studied to investigate the probable failure mechanism. Delamination, matrix agglomeration, fiber pull-out and matrix cracking were the prominent failure mechanisms identified.

Invariant recognition drives neural representations of action sequences.

Directory of Open Access Journals (Sweden)

Andrea Tacchetti

2017-12-01

Full Text Available Recognizing the actions of others from visual stimuli is a crucial aspect of human perception that allows individuals to respond to social cues. Humans are able to discriminate between similar actions despite transformations, like changes in viewpoint or actor, that substantially alter the visual appearance of a scene. This ability to generalize across complex transformations is a hallmark of human visual intelligence. Advances in understanding action recognition at the neural level have not always translated into precise accounts of the computational principles underlying what representations of action sequences are constructed by human visual cortex. Here we test the hypothesis that invariant action discrimination might fill this gap. Recently, the study of artificial systems for static object perception has produced models, Convolutional Neural Networks (CNNs, that achieve human level performance in complex discriminative tasks. Within this class, architectures that better support invariant object recognition also produce image representations that better match those implied by human and primate neural data. However, whether these models produce representations of action sequences that support recognition across complex transformations and closely follow neural representations of actions remains unknown. Here we show that spatiotemporal CNNs accurately categorize video stimuli into action classes, and that deliberate model modifications that improve performance on an invariant action recognition task lead to data representations that better match human neural recordings. Our results support our hypothesis that performance on invariant discrimination dictates the neural representations of actions computed in the brain. These results broaden the scope of the invariant recognition framework for understanding visual intelligence from perception of inanimate objects and faces in static images to the study of human perception of action sequences.

Glaucoma and Driving: On-Road Driving Characteristics

Science.gov (United States)

Wood, Joanne M.; Black, Alex A.; Mallon, Kerry; Thomas, Ravi; Owsley, Cynthia

2016-01-01

Purpose To comprehensively investigate the types of driving errors and locations that are most problematic for older drivers with glaucoma compared to those without glaucoma using a standardized on-road assessment. Methods Participants included 75 drivers with glaucoma (mean = 73.2±6.0 years) with mild to moderate field loss (better-eye MD = -1.21 dB; worse-eye MD = -7.75 dB) and 70 age-matched controls without glaucoma (mean = 72.6 ± 5.0 years). On-road driving performance was assessed in a dual-brake vehicle by an occupational therapist using a standardized scoring system which assessed the types of driving errors and the locations where they were made and the number of critical errors that required an instructor intervention. Driving safety was rated on a 10-point scale. Self-reported driving ability and difficulties were recorded using the Driving Habits Questionnaire. Results Drivers with glaucoma were rated as significantly less safe, made more driving errors, and had almost double the rate of critical errors than those without glaucoma. Driving errors involved lane positioning and planning/approach, and were significantly more likely to occur at traffic lights and yield/give-way intersections. There were few between group differences in self-reported driving ability. Conclusions Older drivers with glaucoma with even mild to moderate field loss exhibit impairments in driving ability, particularly during complex driving situations that involve tactical problems with lane-position, planning ahead and observation. These results, together with the fact that these drivers self-report their driving to be relatively good, reinforce the need for evidence-based on-road assessments for evaluating driving fitness. PMID:27472221

Glaucoma and Driving: On-Road Driving Characteristics.

Directory of Open Access Journals (Sweden)

Joanne M Wood

Full Text Available To comprehensively investigate the types of driving errors and locations that are most problematic for older drivers with glaucoma compared to those without glaucoma using a standardized on-road assessment.Participants included 75 drivers with glaucoma (mean = 73.2±6.0 years with mild to moderate field loss (better-eye MD = -1.21 dB; worse-eye MD = -7.75 dB and 70 age-matched controls without glaucoma (mean = 72.6 ± 5.0 years. On-road driving performance was assessed in a dual-brake vehicle by an occupational therapist using a standardized scoring system which assessed the types of driving errors and the locations where they were made and the number of critical errors that required an instructor intervention. Driving safety was rated on a 10-point scale. Self-reported driving ability and difficulties were recorded using the Driving Habits Questionnaire.Drivers with glaucoma were rated as significantly less safe, made more driving errors, and had almost double the rate of critical errors than those without glaucoma. Driving errors involved lane positioning and planning/approach, and were significantly more likely to occur at traffic lights and yield/give-way intersections. There were few between group differences in self-reported driving ability.Older drivers with glaucoma with even mild to moderate field loss exhibit impairments in driving ability, particularly during complex driving situations that involve tactical problems with lane-position, planning ahead and observation. These results, together with the fact that these drivers self-report their driving to be relatively good, reinforce the need for evidence-based on-road assessments for evaluating driving fitness.

Glaucoma and Driving: On-Road Driving Characteristics.

Science.gov (United States)

Wood, Joanne M; Black, Alex A; Mallon, Kerry; Thomas, Ravi; Owsley, Cynthia

2016-01-01

To comprehensively investigate the types of driving errors and locations that are most problematic for older drivers with glaucoma compared to those without glaucoma using a standardized on-road assessment. Participants included 75 drivers with glaucoma (mean = 73.2±6.0 years) with mild to moderate field loss (better-eye MD = -1.21 dB; worse-eye MD = -7.75 dB) and 70 age-matched controls without glaucoma (mean = 72.6 ± 5.0 years). On-road driving performance was assessed in a dual-brake vehicle by an occupational therapist using a standardized scoring system which assessed the types of driving errors and the locations where they were made and the number of critical errors that required an instructor intervention. Driving safety was rated on a 10-point scale. Self-reported driving ability and difficulties were recorded using the Driving Habits Questionnaire. Drivers with glaucoma were rated as significantly less safe, made more driving errors, and had almost double the rate of critical errors than those without glaucoma. Driving errors involved lane positioning and planning/approach, and were significantly more likely to occur at traffic lights and yield/give-way intersections. There were few between group differences in self-reported driving ability. Older drivers with glaucoma with even mild to moderate field loss exhibit impairments in driving ability, particularly during complex driving situations that involve tactical problems with lane-position, planning ahead and observation. These results, together with the fact that these drivers self-report their driving to be relatively good, reinforce the need for evidence-based on-road assessments for evaluating driving fitness.

Evaluation of axial pile bearing capacity based on pile driving analyzer (PDA) test using Neural Network

Science.gov (United States)

Maizir, H.; Suryanita, R.

2018-01-01

A few decades, many methods have been developed to predict and evaluate the bearing capacity of driven piles. The problem of the predicting and assessing the bearing capacity of the pile is very complicated and not yet established, different soil testing and evaluation produce a widely different solution. However, the most important thing is to determine methods used to predict and evaluate the bearing capacity of the pile to the required degree of accuracy and consistency value. Accurate prediction and evaluation of axial bearing capacity depend on some variables, such as the type of soil, diameter, and length of pile, etc. The aims of the study of Artificial Neural Networks (ANNs) are utilized to obtain more accurate and consistent axial bearing capacity of a driven pile. ANNs can be described as mapping an input to the target output data. The method using the ANN model developed to predict and evaluate the axial bearing capacity of the pile based on the pile driving analyzer (PDA) test data for more than 200 selected data. The results of the predictions obtained by the ANN model and the PDA test were then compared. This research as the neural network models give a right prediction and evaluation of the axial bearing capacity of piles using neural networks.

Neural activity in ventral medial prefrontal cortex is modulated more before approach than avoidance during reinforced and extinction trial blocks.

Science.gov (United States)

Gentry, Ronny N; Roesch, Matthew R

2018-04-16

Ventromedial prefrontal cortex (vmPFC) is thought to provide regulatory control over Pavlovian fear responses and has recently been implicated in appetitive approach behavior, but much less is known about its role in contexts where appetitive and aversive outcomes can be obtained and avoided, respectively. To address this issue, we recorded from single neurons in vmPFC while male rats performed our combined approach and avoidance task under reinforced and non-reinforced (extinction) conditions. Surprisingly, we found that cues predicting reward modulated cell firing in vmPFC more often and more robustly than cues preceding avoidable shock; additionally, firing of vmPFC neurons was both response (press or no-press) and outcome (reinforced or extinction) selective. These results suggest a complex role for vmPFC in regulating behavior and supports its role in appetitive contexts during both reinforced and non-reinforced conditions. SIGNIFICANCE STATEMENT Selecting context-appropriate behaviors to gain reward or avoid punishment is critical for survival. While the role of ventromedial prefrontal cortex (vmPFC) in mediating fear responses is well-established, vmPFC has also been implicated in the regulation of reward-guided approach and extinction. Many studies have used indirect methods and simple behavioral procedures to study vmPFC, which leaves the literature incomplete. We recorded vmFPC neural activity during a complex cue-driven combined approach and avoidance task and during extinction. Surprisingly, we found very little vmPFC modulation to cues predicting avoidable shock, while cues predicting reward approach robustly modulated vmPFC firing in a response- and outcome-selective manner. This suggests a more complex role for vmPFC than current theories suggest, specifically regarding context-specific behavioral optimization. Copyright © 2018 the authors.

Multi-mode energy management strategy for fuel cell electric vehicles based on driving pattern identification using learning vector quantization neural network algorithm

Science.gov (United States)

Song, Ke; Li, Feiqiang; Hu, Xiao; He, Lin; Niu, Wenxu; Lu, Sihao; Zhang, Tong

2018-06-01

The development of fuel cell electric vehicles can to a certain extent alleviate worldwide energy and environmental issues. While a single energy management strategy cannot meet the complex road conditions of an actual vehicle, this article proposes a multi-mode energy management strategy for electric vehicles with a fuel cell range extender based on driving condition recognition technology, which contains a patterns recognizer and a multi-mode energy management controller. This paper introduces a learning vector quantization (LVQ) neural network to design the driving patterns recognizer according to a vehicle's driving information. This multi-mode strategy can automatically switch to the genetic algorithm optimized thermostat strategy under specific driving conditions in the light of the differences in condition recognition results. Simulation experiments were carried out based on the model's validity verification using a dynamometer test bench. Simulation results show that the proposed strategy can obtain better economic performance than the single-mode thermostat strategy under dynamic driving conditions.

New Smith Internal Model Control of Two-Motor Drive System Based on Neural Network Generalized Inverse

Directory of Open Access Journals (Sweden)

Guohai Liu

2016-01-01

Full Text Available Multimotor drive system is widely applied in industrial control system. Considering the characteristics of multi-input multioutput, nonlinear, strong-coupling, and time-varying delay in two-motor drive systems, this paper proposes a new Smith internal model (SIM control method, which is based on neural network generalized inverse (NNGI. This control strategy adopts the NNGI system to settle the decoupling issue and utilizes the SIM control structure to solve the delay problem. The NNGI method can decouple the original system into several composite pseudolinear subsystems and also complete the pole-zero allocation of subsystems. Furthermore, based on the precise model of pseudolinear system, the proposed SIM control structure is used to compensate the network delay and enhance the interference resisting the ability of the whole system. Both simulation and experimental results are given, verifying that the proposed control strategy can effectively solve the decoupling problem and exhibits the strong robustness to load impact disturbance at various operations.

Enrichment of skin-derived neural precursor cells from dermal cell populations by altering culture conditions.

Science.gov (United States)

Bayati, Vahid; Gazor, Rohoullah; Nejatbakhsh, Reza; Negad Dehbashi, Fereshteh

2016-01-01

As stem cells play a critical role in tissue repair, their manipulation for being applied in regenerative medicine is of great importance. Skin-derived precursors (SKPs) may be good candidates for use in cell-based therapy as the only neural stem cells which can be isolated from an accessible tissue, skin. Herein, we presented a simple protocol to enrich neural SKPs by monolayer adherent cultivation to prove the efficacy of this method. To enrich neural SKPs from dermal cell populations, we have found that a monolayer adherent cultivation helps to increase the numbers of neural precursor cells. Indeed, we have cultured dermal cells as monolayer under serum-supplemented (control) and serum-supplemented culture, followed by serum free cultivation (test) and compared. Finally, protein markers of SKPs were assessed and compared in both experimental groups and differentiation potential was evaluated in enriched culture. The cells of enriched culture concurrently expressed fibronectin, vimentin and nestin, an intermediate filament protein expressed in neural and skeletal muscle precursors as compared to control culture. In addition, they possessed a multipotential capacity to differentiate into neurogenic, glial, adipogenic, osteogenic and skeletal myogenic cell lineages. It was concluded that serum-free adherent culture reinforced by growth factors have been shown to be effective on proliferation of skin-derived neural precursor cells (skin-NPCs) and drive their selective and rapid expansion.

Convolutional Neural Network-Based Classification of Driver's Emotion during Aggressive and Smooth Driving Using Multi-Modal Camera Sensors.

Science.gov (United States)

Lee, Kwan Woo; Yoon, Hyo Sik; Song, Jong Min; Park, Kang Ryoung

2018-03-23

Because aggressive driving often causes large-scale loss of life and property, techniques for advance detection of adverse driver emotional states have become important for the prevention of aggressive driving behaviors. Previous studies have primarily focused on systems for detecting aggressive driver emotion via smart-phone accelerometers and gyro-sensors, or they focused on methods of detecting physiological signals using electroencephalography (EEG) or electrocardiogram (ECG) sensors. Because EEG and ECG sensors cause discomfort to drivers and can be detached from the driver's body, it becomes difficult to focus on bio-signals to determine their emotional state. Gyro-sensors and accelerometers depend on the performance of GPS receivers and cannot be used in areas where GPS signals are blocked. Moreover, if driving on a mountain road with many quick turns, a driver's emotional state can easily be misrecognized as that of an aggressive driver. To resolve these problems, we propose a convolutional neural network (CNN)-based method of detecting emotion to identify aggressive driving using input images of the driver's face, obtained using near-infrared (NIR) light and thermal camera sensors. In this research, we conducted an experiment using our own database, which provides a high classification accuracy for detecting driver emotion leading to either aggressive or smooth (i.e., relaxed) driving. Our proposed method demonstrates better performance than existing methods.

Human-level control through deep reinforcement learning

Science.gov (United States)

Mnih, Volodymyr; Kavukcuoglu, Koray; Silver, David; Rusu, Andrei A.; Veness, Joel; Bellemare, Marc G.; Graves, Alex; Riedmiller, Martin; Fidjeland, Andreas K.; Ostrovski, Georg; Petersen, Stig; Beattie, Charles; Sadik, Amir; Antonoglou, Ioannis; King, Helen; Kumaran, Dharshan; Wierstra, Daan; Legg, Shane; Hassabis, Demis

2015-02-01

The theory of reinforcement learning provides a normative account, deeply rooted in psychological and neuroscientific perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems, the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms. While reinforcement learning agents have achieved some successes in a variety of domains, their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.

Human-level control through deep reinforcement learning.

Science.gov (United States)

Mnih, Volodymyr; Kavukcuoglu, Koray; Silver, David; Rusu, Andrei A; Veness, Joel; Bellemare, Marc G; Graves, Alex; Riedmiller, Martin; Fidjeland, Andreas K; Ostrovski, Georg; Petersen, Stig; Beattie, Charles; Sadik, Amir; Antonoglou, Ioannis; King, Helen; Kumaran, Dharshan; Wierstra, Daan; Legg, Shane; Hassabis, Demis

2015-02-26

The theory of reinforcement learning provides a normative account, deeply rooted in psychological and neuroscientific perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems, the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms. While reinforcement learning agents have achieved some successes in a variety of domains, their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.

Reinforcement Learning in Autism Spectrum Disorder

Directory of Open Access Journals (Sweden)

Manuela Schuetze

2017-11-01

Full Text Available Early behavioral interventions are recognized as integral to standard care in autism spectrum disorder (ASD, and often focus on reinforcing desired behaviors (e.g., eye contact and reducing the presence of atypical behaviors (e.g., echoing others' phrases. However, efficacy of these programs is mixed. Reinforcement learning relies on neurocircuitry that has been reported to be atypical in ASD: prefrontal-sub-cortical circuits, amygdala, brainstem, and cerebellum. Thus, early behavioral interventions rely on neurocircuitry that may function atypically in at least a subset of individuals with ASD. Recent work has investigated physiological, behavioral, and neural responses to reinforcers to uncover differences in motivation and learning in ASD. We will synthesize this work to identify promising avenues for future research that ultimately can be used to enhance the efficacy of early intervention.

Tackling Error Propagation through Reinforcement Learning: A Case of Greedy Dependency Parsing

OpenAIRE

Le, Minh; Fokkens, Antske

2017-01-01

Error propagation is a common problem in NLP. Reinforcement learning explores erroneous states during training and can therefore be more robust when mistakes are made early in a process. In this paper, we apply reinforcement learning to greedy dependency parsing which is known to suffer from error propagation. Reinforcement learning improves accuracy of both labeled and unlabeled dependencies of the Stanford Neural Dependency Parser, a high performance greedy parser, while maintaining its eff...

Estimating the behavior of RC beams strengthened with NSM system using artificial neural networks

Directory of Open Access Journals (Sweden)

Seyed Rohollah Hosseini Vaez

2017-12-01

Full Text Available In the last decade, conventional materials such as steel and concrete are being replaced by fiber reinforced polymer (FRP materials for the strengthening of concrete structures. Among the strengthening techniques based on Fiber Reinforced Polymer composites, the use of near-surface mounted (NSM FRP rods is emerging as a promising technology for increasing flexural and shear strength of deficient concrete, masonry and timber members. An artificial neural network is an information processing tool that is inspired by the way biological nervous systems (such as the brain process the information. The key element of this tool is the novel structure of the information processing system. In engineering applications, a neural network can be a vector mapper which maps an input vector to an output one. In the present study, a new approach is developed to predict the behavior of strengthened concrete beam using a large number of experimental data by applying artificial neural networks. Having parameters used as input nodes in ANN modeling such as elastic modulus of the FRP reinforcement, the ratio of the steel longitudinal reinforcement, dimensions of the beam section, the ratio of the NSM-FRP reinforcement and characteristics of concrete, the output node was the flexural strength of beams. The idealized neural network was employed to generate empirical charts and equations to be used in design. The aim of this study is to investigate the behavior of strengthened RC beam using artificial neural networks.

Convolutional Neural Network-Based Classification of Driver’s Emotion during Aggressive and Smooth Driving Using Multi-Modal Camera Sensors

Directory of Open Access Journals (Sweden)

Kwan Woo Lee

2018-03-01

Full Text Available Because aggressive driving often causes large-scale loss of life and property, techniques for advance detection of adverse driver emotional states have become important for the prevention of aggressive driving behaviors. Previous studies have primarily focused on systems for detecting aggressive driver emotion via smart-phone accelerometers and gyro-sensors, or they focused on methods of detecting physiological signals using electroencephalography (EEG or electrocardiogram (ECG sensors. Because EEG and ECG sensors cause discomfort to drivers and can be detached from the driver’s body, it becomes difficult to focus on bio-signals to determine their emotional state. Gyro-sensors and accelerometers depend on the performance of GPS receivers and cannot be used in areas where GPS signals are blocked. Moreover, if driving on a mountain road with many quick turns, a driver’s emotional state can easily be misrecognized as that of an aggressive driver. To resolve these problems, we propose a convolutional neural network (CNN-based method of detecting emotion to identify aggressive driving using input images of the driver’s face, obtained using near-infrared (NIR light and thermal camera sensors. In this research, we conducted an experiment using our own database, which provides a high classification accuracy for detecting driver emotion leading to either aggressive or smooth (i.e., relaxed driving. Our proposed method demonstrates better performance than existing methods.

Artificial neural network and regression modelling to study the effect of reinforcement and deformation on volumetric wear of red mud nano particle reinforced aluminium matrix composites synthesized by stir casting

Directory of Open Access Journals (Sweden)

Gampala Satyanarayana

2018-05-01

Full Text Available Artificial neural network (ANN approach was used for the prediction of effect of reinforcement and deformation on volumetric wear of red mud nano particle reinforced aluminium matrix composites synthesized by stir casting. Red mud obtained from alumina processing industry was milled in a high energy ball mill and the particle size was reduced to 40 nm in 30 h. Sliding wear characteristics of the composites were evaluated on pin on disc wear tester at different loads of 10 N, 20 N and 30 N and sliding speeds of 200, 400, and 600 RPM. The wear rate of the composite was decreased with increase in weight fraction of red mud up to 10% and beyond that the wear rate was increased. The interfacial area between the matrix and the reinforcement increases with increase in red mud volume fraction, leading to increase in strength and wear resistance. Mathematical regression model and ANN model have been developed to predict theoretical wear rate of the composite and observed that ANN predictions have excellent agreement with measured values than other models. Thus, the prediction of wear rate of the nano composites using artificial neural network before actual manufacture will considerably saves the project time, effort and cost. Resumen: Se utilizó el método de red neuronal artificial (RNA para predecir el efecto del refuerzo y la deformación sobre el desgaste volumétrico de los materiales compuestos de matriz de aluminio reforzada con nanopartículas de barro rojo sintetizados por agitación. El barro rojo obtenido de la industria de procesamiento de alúmina se molió en un molino de bolas de alta energía y el tamaño de la partícula se redujo a 40 nm en 30 h. Las características de desgaste de los materiales compuestos se evaluaron en los probadores pin-on-disk de desgaste en diferentes cargas de 10N, 20N y 30N, y velocidades de deslizamiento de 200, 400 y 600 rpm. El índice de desgaste del material compuesto se redujo con el aumento en

Capacitance estimation algorithm based on DC-link voltage harmonics using artificial neural network in three-phase motor drive systems

DEFF Research Database (Denmark)

Soliman, Hammam Abdelaal Hammam; Davari, Pooya; Wang, Huai

2017-01-01

to industry. In this digest, a condition monitoring methodology that estimates the capacitance value of the dc-link capacitor in a three phase Front-End diode bridge motor drive is proposed. The proposed software methodology is based on Artificial Neural Network (ANN) algorithm. The harmonics of the dc......-link voltage are used as training data to the Artificial Neural Network. Fast Fourier Transform (FFT) of the dc-link voltage is analysed in order to study the impact of capacitance variation on the harmonics order. Laboratory experiments are conducted to validate the proposed methodology and the error analysis......In modern design of power electronic converters, reliability of dc-link capacitors is one of the critical considered aspects. The industrial field have been attracted to the monitoring of their health condition and the estimation of their ageing process status. However, the existing condition...

Model analysis of adaptive car driving behavior

NARCIS (Netherlands)

Wewerinke, P.H.

1996-01-01

This paper deals with two modeling approaches to car driving. The first one is a system theoretic approach to describe adaptive human driving behavior. The second approach utilizes neural networks. As an illustrative example the overtaking task is considered and modeled in system theoretic terms.

Program Helps Simulate Neural Networks

Science.gov (United States)

Villarreal, James; Mcintire, Gary

1993-01-01

Neural Network Environment on Transputer System (NNETS) computer program provides users high degree of flexibility in creating and manipulating wide variety of neural-network topologies at processing speeds not found in conventional computing environments. Supports back-propagation and back-propagation-related algorithms. Back-propagation algorithm used is implementation of Rumelhart's generalized delta rule. NNETS developed on INMOS Transputer(R). Predefines back-propagation network, Jordan network, and reinforcement network to assist users in learning and defining own networks. Also enables users to configure other neural-network paradigms from NNETS basic architecture. Small portion of software written in OCCAM(R) language.

A Mechanistic Neural Field Theory of How Anesthesia Suppresses Consciousness: Synaptic Drive Dynamics, Bifurcations, Attractors, and Partial State Equipartitioning.

Science.gov (United States)

Hou, Saing Paul; Haddad, Wassim M; Meskin, Nader; Bailey, James M

2015-12-01

With the advances in biochemistry, molecular biology, and neurochemistry there has been impressive progress in understanding the molecular properties of anesthetic agents. However, there has been little focus on how the molecular properties of anesthetic agents lead to the observed macroscopic property that defines the anesthetic state, that is, lack of responsiveness to noxious stimuli. In this paper, we use dynamical system theory to develop a mechanistic mean field model for neural activity to study the abrupt transition from consciousness to unconsciousness as the concentration of the anesthetic agent increases. The proposed synaptic drive firing-rate model predicts the conscious-unconscious transition as the applied anesthetic concentration increases, where excitatory neural activity is characterized by a Poincaré-Andronov-Hopf bifurcation with the awake state transitioning to a stable limit cycle and then subsequently to an asymptotically stable unconscious equilibrium state. Furthermore, we address the more general question of synchronization and partial state equipartitioning of neural activity without mean field assumptions. This is done by focusing on a postulated subset of inhibitory neurons that are not themselves connected to other inhibitory neurons. Finally, several numerical experiments are presented to illustrate the different aspects of the proposed theory.

Dopamine-Dependent Reinforcement of Motor Skill Learning: Evidence from Gilles de la Tourette Syndrome

Science.gov (United States)

Palminteri, Stefano; Lebreton, Mael; Worbe, Yulia; Hartmann, Andreas; Lehericy, Stephane; Vidailhet, Marie; Grabli, David; Pessiglione, Mathias

2011-01-01

Reinforcement learning theory has been extensively used to understand the neural underpinnings of instrumental behaviour. A central assumption surrounds dopamine signalling reward prediction errors, so as to update action values and ensure better choices in the future. However, educators may share the intuitive idea that reinforcements not only…

In vitro reinforcement of hippocampal bursting: a search for Skinner's atoms of behavior.

OpenAIRE

Stein, L; Xue, B G; Belluzzi, J D

1994-01-01

A novel "in vitro reinforcement" paradigm was used to investigate Skinner's (1953) hypotheses (a) that operant behavior is made up of infinitesimal "response elements" or "behavioral atoms" and (b) that these very small units, and not whole responses, are the functional units of reinforcement. Our tests are based on the assumption that behavioral atoms may plausibly be represented at the neural level by individual cellular responses. As a first approach, we attempted to reinforce the bursting...

Neural responses to exclusion predict susceptibility to social influence.

Science.gov (United States)

Falk, Emily B; Cascio, Christopher N; O'Donnell, Matthew Brook; Carp, Joshua; Tinney, Francis J; Bingham, C Raymond; Shope, Jean T; Ouimet, Marie Claude; Pradhan, Anuj K; Simons-Morton, Bruce G

2014-05-01

Social influence is prominent across the lifespan, but sensitivity to influence is especially high during adolescence and is often associated with increased risk taking. Such risk taking can have dire consequences. For example, in American adolescents, traffic-related crashes are leading causes of nonfatal injury and death. Neural measures may be especially useful in understanding the basic mechanisms of adolescents' vulnerability to peer influence. We examined neural responses to social exclusion as potential predictors of risk taking in the presence of peers in recently licensed adolescent drivers. Risk taking was assessed in a driving simulator session occurring approximately 1 week after the neuroimaging session. Increased activity in neural systems associated with the distress of social exclusion and mentalizing during an exclusion episode predicted increased risk taking in the presence of a peer (controlling for solo risk behavior) during a driving simulator session outside the neuroimaging laboratory 1 week later. These neural measures predicted risky driving behavior above and beyond self-reports of susceptibility to peer pressure and distress during exclusion. These results address the neural bases of social influence and risk taking; contribute to our understanding of social and emotional function in the adolescent brain; and link neural activity in specific, hypothesized, regions to risk-relevant outcomes beyond the neuroimaging laboratory. Results of this investigation are discussed in terms of the mechanisms underlying risk taking in adolescents and the public health implications for adolescent driving. Copyright © 2014 Society for Adolescent Health and Medicine. All rights reserved.

Towards autonomous neuroprosthetic control using Hebbian reinforcement learning.

Science.gov (United States)

Mahmoudi, Babak; Pohlmeyer, Eric A; Prins, Noeline W; Geng, Shijia; Sanchez, Justin C

2013-12-01

Our goal was to design an adaptive neuroprosthetic controller that could learn the mapping from neural states to prosthetic actions and automatically adjust adaptation using only a binary evaluative feedback as a measure of desirability/undesirability of performance. Hebbian reinforcement learning (HRL) in a connectionist network was used for the design of the adaptive controller. The method combines the efficiency of supervised learning with the generality of reinforcement learning. The convergence properties of this approach were studied using both closed-loop control simulations and open-loop simulations that used primate neural data from robot-assisted reaching tasks. The HRL controller was able to perform classification and regression tasks using its episodic and sequential learning modes, respectively. In our experiments, the HRL controller quickly achieved convergence to an effective control policy, followed by robust performance. The controller also automatically stopped adapting the parameters after converging to a satisfactory control policy. Additionally, when the input neural vector was reorganized, the controller resumed adaptation to maintain performance. By estimating an evaluative feedback directly from the user, the HRL control algorithm may provide an efficient method for autonomous adaptation of neuroprosthetic systems. This method may enable the user to teach the controller the desired behavior using only a simple feedback signal.

Neural network regulation driven by autonomous neural firings

Science.gov (United States)

Cho, Myoung Won

2016-07-01

Biological neurons naturally fire spontaneously due to the existence of a noisy current. Such autonomous firings may provide a driving force for network formation because synaptic connections can be modified due to neural firings. Here, we study the effect of autonomous firings on network formation. For the temporally asymmetric Hebbian learning, bidirectional connections lose their balance easily and become unidirectional ones. Defining the difference between reciprocal connections as new variables, we could express the learning dynamics as if Ising model spins interact with each other in magnetism. We present a theoretical method to estimate the interaction between the new variables in a neural system. We apply the method to some network systems and find some tendencies of autonomous neural network regulation.

Neural correlates of reinforcement learning and social preferences in competitive bidding.

Science.gov (United States)

van den Bos, Wouter; Talwar, Arjun; McClure, Samuel M

2013-01-30

In competitive social environments, people often deviate from what rational choice theory prescribes, resulting in losses or suboptimal monetary gains. We investigate how competition affects learning and decision-making in a common value auction task. During the experiment, groups of five human participants were simultaneously scanned using MRI while playing the auction task. We first demonstrate that bidding is well characterized by reinforcement learning with biased reward representations dependent on social preferences. Indicative of reinforcement learning, we found that estimated trial-by-trial prediction errors correlated with activity in the striatum and ventromedial prefrontal cortex. Additionally, we found that individual differences in social preferences were related to activity in the temporal-parietal junction and anterior insula. Connectivity analyses suggest that monetary and social value signals are integrated in the ventromedial prefrontal cortex and striatum. Based on these results, we argue for a novel mechanistic account for the integration of reinforcement history and social preferences in competitive decision-making.

Music mood induction and maintenance while driving : A simulator study

NARCIS (Netherlands)

Dijksterhuis, Chris; van der Zwaag, Marjolein; de Waard, Dick; Westerink, Joyce; Brookhuis, Karel; Mulder, Ben L. J. M.

2010-01-01

It is common knowledge that mood can influence our everyday behaviour and people often seek to reinforce, or to alter their mood, for example by turning on music. Music listening while driving is a common activity. However, the actual impact of music listening while driving on physical state and

Reinforced dynamics for enhanced sampling in large atomic and molecular systems

Science.gov (United States)

Zhang, Linfeng; Wang, Han; E, Weinan

2018-03-01

A new approach for efficiently exploring the configuration space and computing the free energy of large atomic and molecular systems is proposed, motivated by an analogy with reinforcement learning. There are two major components in this new approach. Like metadynamics, it allows for an efficient exploration of the configuration space by adding an adaptively computed biasing potential to the original dynamics. Like deep reinforcement learning, this biasing potential is trained on the fly using deep neural networks, with data collected judiciously from the exploration and an uncertainty indicator from the neural network model playing the role of the reward function. Parameterization using neural networks makes it feasible to handle cases with a large set of collective variables. This has the potential advantage that selecting precisely the right set of collective variables has now become less critical for capturing the structural transformations of the system. The method is illustrated by studying the full-atom explicit solvent models of alanine dipeptide and tripeptide, as well as the system of a polyalanine-10 molecule with 20 collective variables.

Neural systems for control

National Research Council Canada - National Science Library

Omidvar, Omid; Elliott, David L

1997-01-01

... is reprinted with permission from A. Barto, "Reinforcement Learning," Handbook of Brain Theory and Neural Networks, M.A. Arbib, ed.. The MIT Press, Cambridge, MA, pp. 804-809, 1995. Chapter 4, Figures 4-5 and 7-9 and Tables 2-5, are reprinted with permission, from S. Cho, "Map Formation in Proprioceptive Cortex," International Jour...

Smart sensorless prediction diagnosis of electric drives

Science.gov (United States)

Kruglova, TN; Glebov, NA; Shoshiashvili, ME

2017-10-01

In this paper, the discuss diagnostic method and prediction of the technical condition of an electrical motor using artificial intelligent method, based on the combination of fuzzy logic and neural networks, are discussed. The fuzzy sub-model determines the degree of development of each fault. The neural network determines the state of the object as a whole and the number of serviceable work periods for motors actuator. The combination of advanced techniques reduces the learning time and increases the forecasting accuracy. The experimental implementation of the method for electric drive diagnosis and associated equipment is carried out at different speeds. As a result, it was found that this method allows troubleshooting the drive at any given speed.

Classification of amyotrophic lateral sclerosis disease based on convolutional neural network and reinforcement sample learning algorithm.

Science.gov (United States)

Sengur, Abdulkadir; Akbulut, Yaman; Guo, Yanhui; Bajaj, Varun

2017-12-01

Electromyogram (EMG) signals contain useful information of the neuromuscular diseases like amyotrophic lateral sclerosis (ALS). ALS is a well-known brain disease, which can progressively degenerate the motor neurons. In this paper, we propose a deep learning based method for efficient classification of ALS and normal EMG signals. Spectrogram, continuous wavelet transform (CWT), and smoothed pseudo Wigner-Ville distribution (SPWVD) have been employed for time-frequency (T-F) representation of EMG signals. A convolutional neural network is employed to classify these features. In it, Two convolution layers, two pooling layer, a fully connected layer and a lost function layer is considered in CNN architecture. The CNN architecture is trained with the reinforcement sample learning strategy. The efficiency of the proposed implementation is tested on publicly available EMG dataset. The dataset contains 89 ALS and 133 normal EMG signals with 24 kHz sampling frequency. Experimental results show 96.80% accuracy. The obtained results are also compared with other methods, which show the superiority of the proposed method.

Autonomous reinforcement learning with experience replay.

Science.gov (United States)

Wawrzyński, Paweł; Tanwani, Ajay Kumar

2013-05-01

This paper considers the issues of efficiency and autonomy that are required to make reinforcement learning suitable for real-life control tasks. A real-time reinforcement learning algorithm is presented that repeatedly adjusts the control policy with the use of previously collected samples, and autonomously estimates the appropriate step-sizes for the learning updates. The algorithm is based on the actor-critic with experience replay whose step-sizes are determined on-line by an enhanced fixed point algorithm for on-line neural network training. An experimental study with simulated octopus arm and half-cheetah demonstrates the feasibility of the proposed algorithm to solve difficult learning control problems in an autonomous way within reasonably short time. Copyright © 2012 Elsevier Ltd. All rights reserved.

Systems and processes within Halliburton Canada to reinforce safe driving behaviors

Energy Technology Data Exchange (ETDEWEB)

Karowich, P.; Mallett, C. [Halliburton Energy Processing Canada, Calgary, AB (Canada)

2005-07-01

An overview of Halliburton Canada's driver training program was presented. Vehicle incident statistics for the year 2000 were provided. Various causes of poor driving were examined, including over driving road conditions, fatigue and complacency. A speeding model was presented, with details of activators, driver behavior and potential consequences. It was noted that direction alone is not sufficient to change behavior. Different factors contributing to fatigue included overworking, pressure, diet and exercise issues. It was suggested that initial safety awareness and carefulness is often short-lived because of a natural learning process called drift. Elements of the Halliburton training program were reviewed. The skid car system is used by the organization, as well as collision avoidance techniques. All employees are trained and classroom discussions and commentaries are provided. Pre-hire driving evaluations with a third-party assessor are conducted, with emphasis on past driving experience. Fatigue management skills are also taught, with a focus on the daily cycles that bodies go through that cause fatigue and triggers that can stimulate alertness. Interactive assessments must be passed to complete the course. Journey management techniques are used as well as traffic safety awareness. Transportation Tuesday is a communication tool that teaches employees about reckless driving and how to avoid incidents. It was concluded that the Halliburton program is designed to improve safety performance at all levels of the organization by changing the way individuals think about safety, as well as understanding why a person drives at risk before exploring possible solutions. Vehicle incident statistics from before and after the program was implemented were presented, along with near miss reports. tabs, figs.

Within- and across-trial dynamics of human EEG reveal cooperative interplay between reinforcement learning and working memory.

Science.gov (United States)

Collins, Anne G E; Frank, Michael J

2018-03-06

Learning from rewards and punishments is essential to survival and facilitates flexible human behavior. It is widely appreciated that multiple cognitive and reinforcement learning systems contribute to decision-making, but the nature of their interactions is elusive. Here, we leverage methods for extracting trial-by-trial indices of reinforcement learning (RL) and working memory (WM) in human electro-encephalography to reveal single-trial computations beyond that afforded by behavior alone. Neural dynamics confirmed that increases in neural expectation were predictive of reduced neural surprise in the following feedback period, supporting central tenets of RL models. Within- and cross-trial dynamics revealed a cooperative interplay between systems for learning, in which WM contributes expectations to guide RL, despite competition between systems during choice. Together, these results provide a deeper understanding of how multiple neural systems interact for learning and decision-making and facilitate analysis of their disruption in clinical populations.

Human reinforcement learning subdivides structured action spaces by learning effector-specific values

OpenAIRE

Gershman, Samuel J.; Pesaran, Bijan; Daw, Nathaniel D.

2009-01-01

Humans and animals are endowed with a large number of effectors. Although this enables great behavioral flexibility, it presents an equally formidable reinforcement learning problem of discovering which actions are most valuable, due to the high dimensionality of the action space. An unresolved question is how neural systems for reinforcement learning – such as prediction error signals for action valuation associated with dopamine and the striatum – can cope with this “curse of dimensionality...

What Can Reinforcement Learning Teach Us About Non-Equilibrium Quantum Dynamics

Science.gov (United States)

Bukov, Marin; Day, Alexandre; Sels, Dries; Weinberg, Phillip; Polkovnikov, Anatoli; Mehta, Pankaj

Equilibrium thermodynamics and statistical physics are the building blocks of modern science and technology. Yet, our understanding of thermodynamic processes away from equilibrium is largely missing. In this talk, I will reveal the potential of what artificial intelligence can teach us about the complex behaviour of non-equilibrium systems. Specifically, I will discuss the problem of finding optimal drive protocols to prepare a desired target state in quantum mechanical systems by applying ideas from Reinforcement Learning [one can think of Reinforcement Learning as the study of how an agent (e.g. a robot) can learn and perfect a given policy through interactions with an environment.]. The driving protocols learnt by our agent suggest that the non-equilibrium world features possibilities easily defying intuition based on equilibrium physics.

hmmr mediates anterior neural tube closure and morphogenesis in the frog Xenopus.

Science.gov (United States)

Prager, Angela; Hagenlocher, Cathrin; Ott, Tim; Schambony, Alexandra; Feistel, Kerstin

2017-10-01

Development of the central nervous system requires orchestration of morphogenetic processes which drive elevation and apposition of the neural folds and their fusion into a neural tube. The newly formed tube gives rise to the brain in anterior regions and continues to develop into the spinal cord posteriorly. Conspicuous differences between the anterior and posterior neural tube become visible already during neural tube closure (NTC). Planar cell polarity (PCP)-mediated convergent extension (CE) movements are restricted to the posterior neural plate, i.e. hindbrain and spinal cord, where they propagate neural fold apposition. The lack of CE in the anterior neural plate correlates with a much slower mode of neural fold apposition anteriorly. The morphogenetic processes driving anterior NTC have not been addressed in detail. Here, we report a novel role for the breast cancer susceptibility gene and microtubule (MT) binding protein Hmmr (Hyaluronan-mediated motility receptor, RHAMM) in anterior neurulation and forebrain development in Xenopus laevis. Loss of hmmr function resulted in a lack of telencephalic hemisphere separation, arising from defective roof plate formation, which in turn was caused by impaired neural tissue narrowing. hmmr regulated polarization of neural cells, a function which was dependent on the MT binding domains. hmmr cooperated with the core PCP component vangl2 in regulating cell polarity and neural morphogenesis. Disrupted cell polarization and elongation in hmmr and vangl2 morphants prevented radial intercalation (RI), a cell behavior essential for neural morphogenesis. Our results pinpoint a novel role of hmmr in anterior neural development and support the notion that RI is a major driving force for anterior neurulation and forebrain morphogenesis. Copyright © 2017 Elsevier Inc. All rights reserved.

The neural basis of the imitation drive.

Science.gov (United States)

Hanawa, Sugiko; Sugiura, Motoaki; Nozawa, Takayuki; Kotozaki, Yuka; Yomogida, Yukihito; Ihara, Mizuki; Akimoto, Yoritaka; Thyreau, Benjamin; Izumi, Shinichi; Kawashima, Ryuta

2016-01-01

Spontaneous imitation is assumed to underlie the acquisition of important skills by infants, including language and social interaction. In this study, functional magnetic resonance imaging (fMRI) was used to examine the neural basis of 'spontaneously' driven imitation, which has not yet been fully investigated. Healthy participants were presented with movie clips of meaningless bimanual actions and instructed to observe and imitate them during an fMRI scan. The participants were subsequently shown the movie clips again and asked to evaluate the strength of their 'urge to imitate' (Urge) for each action. We searched for cortical areas where the degree of activation positively correlated with Urge scores; significant positive correlations were observed in the right supplementary motor area (SMA) and bilateral midcingulate cortex (MCC) under the imitation condition. These areas were not explained by explicit reasons for imitation or the kinematic characteristics of the actions. Previous studies performed in monkeys and humans have implicated the SMA and MCC/caudal cingulate zone in voluntary actions. This study also confirmed the functional connectivity between Urge and imitation performance using a psychophysiological interaction analysis. Thus, our findings reveal the critical neural components that underlie spontaneous imitation and provide possible reasons why infants imitate spontaneously. © The Author (2015). Published by Oxford University Press.

Global reinforcement training of CrossNets

Science.gov (United States)

Ma, Xiaolong

2007-10-01

Hybrid "CMOL" integrated circuits, incorporating advanced CMOS devices for neural cell bodies, nanowires as axons and dendrites, and latching switches as synapses, may be used for the hardware implementation of extremely dense (107 cells and 1012 synapses per cm2) neuromorphic networks, operating up to 10 6 times faster than their biological prototypes. We are exploring several "Cross- Net" architectures that accommodate the limitations imposed by CMOL hardware and should allow effective training of the networks without a direct external access to individual synapses. Our studies have show that CrossNets based on simple (two-terminal) crosspoint devices can work well in at least two modes: as Hop-field networks for associative memory and multilayer perceptrons for classification tasks. For more intelligent tasks (such as robot motion control or complex games), which do not have "examples" for supervised learning, more advanced training methods such as the global reinforcement learning are necessary. For application of global reinforcement training algorithms to CrossNets, we have extended Williams's REINFORCE learning principle to a more general framework and derived several learning rules that are more suitable for CrossNet hardware implementation. The results of numerical experiments have shown that these new learning rules can work well for both classification tasks and reinforcement tasks such as the cartpole balancing control problem. Some limitations imposed by the CMOL hardware need to be carefully addressed for the the successful application of in situ reinforcement training to CrossNets.

Brain Inspired Cognitive Model with Attention for Self-Driving Cars

OpenAIRE

Chen, Shitao; Zhang, Songyi; Shang, Jinghao; Chen, Badong; Zheng, Nanning

2017-01-01

Perception-driven approach and end-to-end system are two major vision-based frameworks for self-driving cars. However, it is difficult to introduce attention and historical information of autonomous driving process, which are the essential factors for achieving human-like driving into these two methods. In this paper, we propose a novel model for self-driving cars named brain-inspired cognitive model with attention (CMA). This model consists of three parts: a convolutional neural network for ...

Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving

OpenAIRE

Shalev-Shwartz, Shai; Shammah, Shaked; Shashua, Amnon

2016-01-01

Autonomous driving is a multi-agent setting where the host vehicle must apply sophisticated negotiation skills with other road users when overtaking, giving way, merging, taking left and right turns and while pushing ahead in unstructured urban roadways. Since there are many possible scenarios, manually tackling all possible cases will likely yield a too simplistic policy. Moreover, one must balance between unexpected behavior of other drivers/pedestrians and at the same time not to be too de...

Batch Policy Gradient Methods for Improving Neural Conversation Models

OpenAIRE

Kandasamy, Kirthevasan; Bachrach, Yoram; Tomioka, Ryota; Tarlow, Daniel; Carter, David

2017-01-01

We study reinforcement learning of chatbots with recurrent neural network architectures when the rewards are noisy and expensive to obtain. For instance, a chatbot used in automated customer service support can be scored by quality assurance agents, but this process can be expensive, time consuming and noisy. Previous reinforcement learning work for natural language processing uses on-policy updates and/or is designed for on-line learning settings. We demonstrate empirically that such strateg...

International Conference on Artificial Neural Networks (ICANN)

CERN Document Server

Mladenov, Valeri; Kasabov, Nikola; Artificial Neural Networks : Methods and Applications in Bio-/Neuroinformatics

2015-01-01

The book reports on the latest theories on artificial neural networks, with a special emphasis on bio-neuroinformatics methods. It includes twenty-three papers selected from among the best contributions on bio-neuroinformatics-related issues, which were presented at the International Conference on Artificial Neural Networks, held in Sofia, Bulgaria, on September 10-13, 2013 (ICANN 2013). The book covers a broad range of topics concerning the theory and applications of artificial neural networks, including recurrent neural networks, super-Turing computation and reservoir computing, double-layer vector perceptrons, nonnegative matrix factorization, bio-inspired models of cell communities, Gestalt laws, embodied theory of language understanding, saccadic gaze shifts and memory formation, and new training algorithms for Deep Boltzmann Machines, as well as dynamic neural networks and kernel machines. It also reports on new approaches to reinforcement learning, optimal control of discrete time-delay systems, new al...

Reinforcement Learning Based Novel Adaptive Learning Framework for Smart Grid Prediction

Directory of Open Access Journals (Sweden)

Tian Li

2017-01-01

Full Text Available Smart grid is a potential infrastructure to supply electricity demand for end users in a safe and reliable manner. With the rapid increase of the share of renewable energy and controllable loads in smart grid, the operation uncertainty of smart grid has increased briskly during recent years. The forecast is responsible for the safety and economic operation of the smart grid. However, most existing forecast methods cannot account for the smart grid due to the disabilities to adapt to the varying operational conditions. In this paper, reinforcement learning is firstly exploited to develop an online learning framework for the smart grid. With the capability of multitime scale resolution, wavelet neural network has been adopted in the online learning framework to yield reinforcement learning and wavelet neural network (RLWNN based adaptive learning scheme. The simulations on two typical prediction problems in smart grid, including wind power prediction and load forecast, validate the effectiveness and the scalability of the proposed RLWNN based learning framework and algorithm.

Energy Management Strategy for a Hybrid Electric Vehicle Based on Deep Reinforcement Learning

OpenAIRE

Yue Hu; Weimin Li; Kun Xu; Taimoor Zahid; Feiyan Qin; Chenming Li

2018-01-01

An energy management strategy (EMS) is important for hybrid electric vehicles (HEVs) since it plays a decisive role on the performance of the vehicle. However, the variation of future driving conditions deeply influences the effectiveness of the EMS. Most existing EMS methods simply follow predefined rules that are not adaptive to different driving conditions online. Therefore, it is useful that the EMS can learn from the environment or driving cycle. In this paper, a deep reinforcement learn...

Reinforcement-driven spread of innovations and fads

International Nuclear Information System (INIS)

Krapivsky, P L; Redner, S; Volovik, D

2011-01-01

We investigate how social reinforcement drives the spread of permanent innovations and transient fads. We account for social reinforcement by endowing each individual with M + 1 possible awareness states 0, 1, 2,..., M, with state M corresponding to adopting an innovation. An individual with awareness k 1−1/M for M > 1. When individuals can abandon the innovation at rate λ, the population fraction that remains clueless about the fad undergoes a phase transition at a critical rate λ c ; this transition is second order for M = 1 and first order for M > 1, with macroscopic fluctuations accompanying the latter. The time for the fad to disappear has an intriguing non-monotonic dependence on λ

Neural networks and statistical learning

CERN Document Server

Du, Ke-Lin

2014-01-01

Providing a broad but in-depth introduction to neural network and machine learning in a statistical framework, this book provides a single, comprehensive resource for study and further research. All the major popular neural network models and statistical learning approaches are covered with examples and exercises in every chapter to develop a practical working understanding of the content. Each of the twenty-five chapters includes state-of-the-art descriptions and important research results on the respective topics. The broad coverage includes the multilayer perceptron, the Hopfield network, associative memory models, clustering models and algorithms, the radial basis function network, recurrent neural networks, principal component analysis, nonnegative matrix factorization, independent component analysis, discriminant analysis, support vector machines, kernel methods, reinforcement learning, probabilistic and Bayesian networks, data fusion and ensemble learning, fuzzy sets and logic, neurofuzzy models, hardw...

Reading the Freudian theory of sexual drives from a functional neuroimaging perspective

Directory of Open Access Journals (Sweden)

Serge eStoléru

2014-03-01

Full Text Available One of the essential tasks of neuropsychoanalysis is to investigate the neural correlates of sexual drives. Here, we consider the four defining characteristics of sexual drives as delineated by Freud: their pressure, aim, object, and source. We systematically examine the relations between these characteristics and the four-component neurophenomenological model that we have proposed based on functional neuroimaging studies, which comprises a cognitive, a motivational, an emotional and an autonomic/neuroendocrine component. Functional neuroimaging studies of sexual arousal have thrown a new light on the four fundamental characteristics of sexual drives by identifying their potential neural correlates. While these studies are essentally consistent with the Freudian model of drives, the main difference emerging between the functional neuroimaging perspective on sexual drives and the Freudian theory relates to the source of drives. From a functional neuroimaging perspective sources of sexual drives, conceived by psychoanalysis as processes of excitation occurring in a peripheral organ, do not seem, at least in adult subjects, to be an essential part of the determinants of sexual arousal. It is rather the central processing of visual or genital stimuli that gives to these stimuli their sexually arousing and sexually pleasurable character.

Hybrid computing using a neural network with dynamic external memory.

Science.gov (United States)

Graves, Alex; Wayne, Greg; Reynolds, Malcolm; Harley, Tim; Danihelka, Ivo; Grabska-Barwińska, Agnieszka; Colmenarejo, Sergio Gómez; Grefenstette, Edward; Ramalho, Tiago; Agapiou, John; Badia, Adrià Puigdomènech; Hermann, Karl Moritz; Zwols, Yori; Ostrovski, Georg; Cain, Adam; King, Helen; Summerfield, Christopher; Blunsom, Phil; Kavukcuoglu, Koray; Hassabis, Demis

2016-10-27

Artificial neural networks are remarkably adept at sensory processing, sequence learning and reinforcement learning, but are limited in their ability to represent variables and data structures and to store data over long timescales, owing to the lack of an external memory. Here we introduce a machine learning model called a differentiable neural computer (DNC), which consists of a neural network that can read from and write to an external memory matrix, analogous to the random-access memory in a conventional computer. Like a conventional computer, it can use its memory to represent and manipulate complex data structures, but, like a neural network, it can learn to do so from data. When trained with supervised learning, we demonstrate that a DNC can successfully answer synthetic questions designed to emulate reasoning and inference problems in natural language. We show that it can learn tasks such as finding the shortest path between specified points and inferring the missing links in randomly generated graphs, and then generalize these tasks to specific graphs such as transport networks and family trees. When trained with reinforcement learning, a DNC can complete a moving blocks puzzle in which changing goals are specified by sequences of symbols. Taken together, our results demonstrate that DNCs have the capacity to solve complex, structured tasks that are inaccessible to neural networks without external read-write memory.

The role of the lateral amygdala in the retrieval and maintenance of fear-memories formed by probabilistic reinforcement

Directory of Open Access Journals (Sweden)

Jeffrey C. Erlich

2012-04-01

Full Text Available The lateral nucleus of the amygdala (LA is a key element in the neural circuit subserving Pavlovian fear conditioning, an animal model of fear and anxiety. Most studies have focused on the role of the LA in fear acquisition and extinction, i.e. how neural plasticity results from changing contingencies between a neutral conditioned stimulus (e.g. a tone and an aversive unconditioned stimulus (e.g. a shock. However, outside of the lab, fear memories are often the result of repeated and unpredictable experiences. Examples include domestic violence, child abuse or combat. To better understand the role of the LA in the expression of fear resulting from repeated and uncertain reinforcement, rats experienced a 30% partial reinforcement fear-conditioning schedule four days a week for four weeks. Rats reached asymptotic levels of conditioned fear expression after the first week. We then manipulated LA activity with drug (or vehicle infusions once a week, for the next three weeks, before the training session. LA infusions of muscimol, a GABA-A agonist that inhibits neural activity, reduced conditioned stimulus (CS evoked fear behavior to pre-conditioning levels. LA infusions of pentagastrin, a cholecystokinin-2 (CCK agonist that increases neural excitability, resulted in CS-evoked fear behavior that continued past the offset of the CS. This suggests that neural activity in the LA is required for the retrieval of fear memories that stem from repeated and uncertain reinforcement, and that CCK signaling in the LA plays a role in the recovery from fear after the removal of the fear-evoking stimulus.

A Difficult Journey: Reflections on Driving and Driving Cessation From a Team of Clinical Researchers.

Science.gov (United States)

Liddle, Jacki; Gustafsson, Louise; Mitchell, Geoffrey; Pachana, Nancy A

2017-02-01

Recognizing the clinical importance and safety and well-being implications for the population, a multidisciplinary team has been researching older drivers and driving cessation issues for more than 15 years. Using empirical approaches, the team has explored quality of life and participation outcomes related to driving and nondriving for older people and has developed interventions to improve outcomes after driving cessation. The team members represent occupational therapists, medical practitioners, and clinical and neuropsychologists. While building the evidence base for driving- and driving cessation-related clinical practice, the researchers have also had first-hand experiences of interruptions to their own or parents' driving; involvement of older family members in road crashes; and provision of support during family members' driving assessment and cessation. This has led to reflection on their understandings and re-evaluation and refocusing of their perspectives in driving cessation research. This work will share the narratives of the authors and note their developing perspectives and foci within research as well as their clinical practice. Personal reflections have indicated the far-reaching implications for older drivers and family members of involvement in road crashes: the potential for interruptions to driving as a time for support and future planning and the conflicting and difficult roles of family members within the driving cessation process. Overall the lived, personal experience of the authors has reinforced the complex nature of driving and changes to driving status for the driver and their support team and the need for further research and support. © The Author 2016. Published by Oxford University Press on behalf of The Gerontological Society of America. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

The combination of appetitive and aversive reinforcers and the nature of their interaction during auditory learning.

Science.gov (United States)

Ilango, A; Wetzel, W; Scheich, H; Ohl, F W

2010-03-31

Learned changes in behavior can be elicited by either appetitive or aversive reinforcers. It is, however, not clear whether the two types of motivation, (approaching appetitive stimuli and avoiding aversive stimuli) drive learning in the same or different ways, nor is their interaction understood in situations where the two types are combined in a single experiment. To investigate this question we have developed a novel learning paradigm for Mongolian gerbils, which not only allows rewards and punishments to be presented in isolation or in combination with each other, but also can use these opposite reinforcers to drive the same learned behavior. Specifically, we studied learning of tone-conditioned hurdle crossing in a shuttle box driven by either an appetitive reinforcer (brain stimulation reward) or an aversive reinforcer (electrical footshock), or by a combination of both. Combination of the two reinforcers potentiated speed of acquisition, led to maximum possible performance, and delayed extinction as compared to either reinforcer alone. Additional experiments, using partial reinforcement protocols and experiments in which one of the reinforcers was omitted after the animals had been previously trained with the combination of both reinforcers, indicated that appetitive and aversive reinforcers operated together but acted in different ways: in this particular experimental context, punishment appeared to be more effective for initial acquisition and reward more effective to maintain a high level of conditioned responses (CRs). The results imply that learning mechanisms in problem solving were maximally effective when the initial punishment of mistakes was combined with the subsequent rewarding of correct performance. Copyright 2010 IBRO. Published by Elsevier Ltd. All rights reserved.

artificial neural network model for low strength rc beam shear capacity

African Journals Online (AJOL)

User

RESEARCH PAPER. Keywords: Shear strength, reinforced concrete, Artificial Neural Network, design equations ... searchers using artificial intelligence to im- prove on theoretical ...... benefit to humanity or a waste of time?” The. Structural ...

Adversarial Reinforcement Learning in a Cyber Security Simulation}

OpenAIRE

Elderman, Richard; Pater, Leon; Thie, Albert; Drugan, Madalina; Wiering, Marco

2017-01-01

This paper focuses on cyber-security simulations in networks modeled as a Markov game with incomplete information and stochastic elements. The resulting game is an adversarial sequential decision making problem played with two agents, the attacker and defender. The two agents pit one reinforcement learning technique, like neural networks, Monte Carlo learning and Q-learning, against each other and examine their effectiveness against learning opponents. The results showed that Monte Carlo lear...

NNETS - NEURAL NETWORK ENVIRONMENT ON A TRANSPUTER SYSTEM

Science.gov (United States)

Villarreal, J.

1994-01-01

The primary purpose of NNETS (Neural Network Environment on a Transputer System) is to provide users a high degree of flexibility in creating and manipulating a wide variety of neural network topologies at processing speeds not found in conventional computing environments. To accomplish this purpose, NNETS supports back propagation and back propagation related algorithms. The back propagation algorithm used is an implementation of Rumelhart's Generalized Delta Rule. NNETS was developed on the INMOS Transputer. NNETS predefines a Back Propagation Network, a Jordan Network, and a Reinforcement Network to assist users in learning and defining their own networks. The program also allows users to configure other neural network paradigms from the NNETS basic architecture. The Jordan network is basically a feed forward network that has the outputs connected to a pseudo input layer. The state of the network is dependent on the inputs from the environment plus the state of the network. The Reinforcement network learns via a scalar feedback signal called reinforcement. The network propagates forward randomly. The environment looks at the outputs of the network to produce a reinforcement signal that is fed back to the network. NNETS was written for the INMOS C compiler D711B version 1.3 or later (MS-DOS version). A small portion of the software was written in the OCCAM language to perform the communications routing between processors. NNETS is configured to operate on a 4 X 10 array of Transputers in sequence with a Transputer based graphics processor controlled by a master IBM PC 286 (or better) Transputer. A RGB monitor is required which must be capable of 512 X 512 resolution. It must be able to receive red, green, and blue signals via BNC connectors. NNETS is meant for experienced Transputer users only. The program is distributed on 5.25 inch 1.2Mb MS-DOS format diskettes. NNETS was developed in 1991. Transputer and OCCAM are registered trademarks of Inmos Corporation. MS

Evolving Neural Turing Machines for Reward-based Learning

DEFF Research Database (Denmark)

Greve, Rasmus Boll; Jacobsen, Emil Juul; Risi, Sebastian

2016-01-01

An unsolved problem in neuroevolution (NE) is to evolve artificial neural networks (ANN) that can store and use information to change their behavior online. While plastic neural networks have shown promise in this context, they have difficulties retaining information over longer periods of time...... version of the double T-Maze, a complex reinforcement-like learning problem. In the T-Maze learning task the agent uses the memory bank to display adaptive behavior that normally requires a plastic ANN, thereby suggesting a complementary and effective mechanism for adaptive behavior in NE....

How we learn to make decisions: rapid propagation of reinforcement learning prediction errors in humans.

Science.gov (United States)

Krigolson, Olav E; Hassall, Cameron D; Handy, Todd C

2014-03-01

Our ability to make decisions is predicated upon our knowledge of the outcomes of the actions available to us. Reinforcement learning theory posits that actions followed by a reward or punishment acquire value through the computation of prediction errors-discrepancies between the predicted and the actual reward. A multitude of neuroimaging studies have demonstrated that rewards and punishments evoke neural responses that appear to reflect reinforcement learning prediction errors [e.g., Krigolson, O. E., Pierce, L. J., Holroyd, C. B., & Tanaka, J. W. Learning to become an expert: Reinforcement learning and the acquisition of perceptual expertise. Journal of Cognitive Neuroscience, 21, 1833-1840, 2009; Bayer, H. M., & Glimcher, P. W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47, 129-141, 2005; O'Doherty, J. P. Reward representations and reward-related learning in the human brain: Insights from neuroimaging. Current Opinion in Neurobiology, 14, 769-776, 2004; Holroyd, C. B., & Coles, M. G. H. The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679-709, 2002]. Here, we used the brain ERP technique to demonstrate that not only do rewards elicit a neural response akin to a prediction error but also that this signal rapidly diminished and propagated to the time of choice presentation with learning. Specifically, in a simple, learnable gambling task, we show that novel rewards elicited a feedback error-related negativity that rapidly decreased in amplitude with learning. Furthermore, we demonstrate the existence of a reward positivity at choice presentation, a previously unreported ERP component that has a similar timing and topography as the feedback error-related negativity that increased in amplitude with learning. The pattern of results we observed mirrored the output of a computational model that we implemented to compute reward

Integration of silicon-based neural probes and micro-drive arrays for chronic recording of large populations of neurons in behaving animals.

Science.gov (United States)

Michon, Frédéric; Aarts, Arno; Holzhammer, Tobias; Ruther, Patrick; Borghs, Gustaaf; McNaughton, Bruce; Kloosterman, Fabian

2016-08-01

Understanding how neuronal assemblies underlie cognitive function is a fundamental question in system neuroscience. It poses the technical challenge to monitor the activity of populations of neurons, potentially widely separated, in relation to behaviour. In this paper, we present a new system which aims at simultaneously recording from a large population of neurons from multiple separated brain regions in freely behaving animals. The concept of the new device is to combine the benefits of two existing electrophysiological techniques, i.e. the flexibility and modularity of micro-drive arrays and the high sampling ability of electrode-dense silicon probes. Newly engineered long bendable silicon probes were integrated into a micro-drive array. The resulting device can carry up to 16 independently movable silicon probes, each carrying 16 recording sites. Populations of neurons were recorded simultaneously in multiple cortical and/or hippocampal sites in two freely behaving implanted rats. Current approaches to monitor neuronal activity either allow to flexibly record from multiple widely separated brain regions (micro-drive arrays) but with a limited sampling density or to provide denser sampling at the expense of a flexible placement in multiple brain regions (neural probes). By combining these two approaches and their benefits, we present an alternative solution for flexible and simultaneous recordings from widely distributed populations of neurons in freely behaving rats.

Driving range estimation for electric vehicles based on driving condition identification and forecast

Science.gov (United States)

Pan, Chaofeng; Dai, Wei; Chen, Liao; Chen, Long; Wang, Limei

2017-10-01

With the impact of serious environmental pollution in our cities combined with the ongoing depletion of oil resources, electric vehicles are becoming highly favored as means of transport. Not only for the advantage of low noise, but for their high energy efficiency and zero pollution. The Power battery is used as the energy source of electric vehicles. However, it does currently still have a few shortcomings, noticeably the low energy density, with high costs and short cycle life results in limited mileage compared with conventional passenger vehicles. There is great difference in vehicle energy consumption rate under different environment and driving conditions. Estimation error of current driving range is relatively large due to without considering the effects of environmental temperature and driving conditions. The development of a driving range estimation method will have a great impact on the electric vehicles. A new driving range estimation model based on the combination of driving cycle identification and prediction is proposed and investigated. This model can effectively eliminate mileage errors and has good convergence with added robustness. Initially the identification of the driving cycle is based on Kernel Principal Component feature parameters and fuzzy C referring to clustering algorithm. Secondly, a fuzzy rule between the characteristic parameters and energy consumption is established under MATLAB/Simulink environment. Furthermore the Markov algorithm and BP(Back Propagation) neural network method is utilized to predict the future driving conditions to improve the accuracy of the remaining range estimation. Finally, driving range estimation method is carried out under the ECE 15 condition by using the rotary drum test bench, and the experimental results are compared with the estimation results. Results now show that the proposed driving range estimation method can not only estimate the remaining mileage, but also eliminate the fluctuation of the

Intelligent Method for Identifying Driving Risk Based on V2V Multisource Big Data

Directory of Open Access Journals (Sweden)

Jinshuan Peng

2018-01-01

Full Text Available Risky driving behavior is a major cause of traffic conflicts, which can develop into road traffic accidents, making the timely and accurate identification of such behavior essential to road safety. A platform was therefore established for analyzing the driving behavior of 20 professional drivers in field tests, in which overclose car following and lane departure were used as typical risky driving behaviors. Characterization parameters for identification were screened and used to determine threshold values and an appropriate time window for identification. A neural network-Bayesian filter identification model was established and data samples were selected to identify risky driving behavior and evaluate the identification efficiency of the model. The results obtained indicated a successful identification rate of 83.6% when the neural network model was solely used to identify risky driving behavior, but this could be increased to 92.46% once corrected by the Bayesian filter. This has important theoretical and practical significance in relation to evaluating the efficiency of existing driver assist systems, as well as the development of future intelligent driving systems.

Neural basis of decision making guided by emotional outcomes.

Science.gov (United States)

Katahira, Kentaro; Matsuda, Yoshi-Taka; Fujimura, Tomomi; Ueno, Kenichi; Asamizuya, Takeshi; Suzuki, Chisato; Cheng, Kang; Okanoya, Kazuo; Okada, Masato

2015-05-01

Emotional events resulting from a choice influence an individual's subsequent decision making. Although the relationship between emotion and decision making has been widely discussed, previous studies have mainly investigated decision outcomes that can easily be mapped to reward and punishment, including monetary gain/loss, gustatory stimuli, and pain. These studies regard emotion as a modulator of decision making that can be made rationally in the absence of emotions. In our daily lives, however, we often encounter various emotional events that affect decisions by themselves, and mapping the events to a reward or punishment is often not straightforward. In this study, we investigated the neural substrates of how such emotional decision outcomes affect subsequent decision making. By using functional magnetic resonance imaging (fMRI), we measured brain activities of humans during a stochastic decision-making task in which various emotional pictures were presented as decision outcomes. We found that pleasant pictures differentially activated the midbrain, fusiform gyrus, and parahippocampal gyrus, whereas unpleasant pictures differentially activated the ventral striatum, compared with neutral pictures. We assumed that the emotional decision outcomes affect the subsequent decision by updating the value of the options, a process modeled by reinforcement learning models, and that the brain regions representing the prediction error that drives the reinforcement learning are involved in guiding subsequent decisions. We found that some regions of the striatum and the insula were separately correlated with the prediction error for either pleasant pictures or unpleasant pictures, whereas the precuneus was correlated with prediction errors for both pleasant and unpleasant pictures. Copyright © 2015 the American Physiological Society.

Neural basis of decision making guided by emotional outcomes

Science.gov (United States)

Matsuda, Yoshi-Taka; Fujimura, Tomomi; Ueno, Kenichi; Asamizuya, Takeshi; Suzuki, Chisato; Cheng, Kang; Okanoya, Kazuo; Okada, Masato

2015-01-01

Emotional events resulting from a choice influence an individual's subsequent decision making. Although the relationship between emotion and decision making has been widely discussed, previous studies have mainly investigated decision outcomes that can easily be mapped to reward and punishment, including monetary gain/loss, gustatory stimuli, and pain. These studies regard emotion as a modulator of decision making that can be made rationally in the absence of emotions. In our daily lives, however, we often encounter various emotional events that affect decisions by themselves, and mapping the events to a reward or punishment is often not straightforward. In this study, we investigated the neural substrates of how such emotional decision outcomes affect subsequent decision making. By using functional magnetic resonance imaging (fMRI), we measured brain activities of humans during a stochastic decision-making task in which various emotional pictures were presented as decision outcomes. We found that pleasant pictures differentially activated the midbrain, fusiform gyrus, and parahippocampal gyrus, whereas unpleasant pictures differentially activated the ventral striatum, compared with neutral pictures. We assumed that the emotional decision outcomes affect the subsequent decision by updating the value of the options, a process modeled by reinforcement learning models, and that the brain regions representing the prediction error that drives the reinforcement learning are involved in guiding subsequent decisions. We found that some regions of the striatum and the insula were separately correlated with the prediction error for either pleasant pictures or unpleasant pictures, whereas the precuneus was correlated with prediction errors for both pleasant and unpleasant pictures. PMID:25695644

Prediction of punching shear capacities of two-way concrete slabs reinforced with FRP bars

Directory of Open Access Journals (Sweden)

Ibrahim M. Metwally

2013-08-01

Full Text Available Where corrosion of steel reinforcement is a concern, fiber-reinforced polymer (FRP reinforcing bar or grid reinforcement provides an alternative reinforcement for concrete flat slabs. The existing provisions for punching of slabs in most international design standards for reinforced concrete are based on tests of steel reinforced slabs. The elastic stiffness and bonding characteristics of FRP reinforcement are sufficiently different from those of steel to affect punching strength [1]. This paper evaluates the punching shear strength of concrete flat slabs reinforced with different types of fiber-reinforced polymer (FRP. A total of 59 full-size slabs were constructed and tested collected from the literature of FRP bars reinforced concrete slabs. The test parameters were the amount of FRP reinforcing bars, Young’s modulus of FRP bars, slab thickness, loaded areas and concrete compressive strength. The experimental punching shear strengths were compared with the available theoretical predictions, including the ACI 318 Code, BS 8110 Code, ACI 440 design guidelines, and a number of models proposed by some researchers in the literature. Two approaches for predicting the punching strength of FRP-reinforced slabs are examined. The first is an empirical new model which is considered as a modification of El-Gamal et al. [2] model. The second is a Neural Networks Technique; which has been developed to predict the punching shear capacity of FRP reinforced concrete slabs. The accuracies of both methods were evaluated against the experimental test data. They attained excellent agreement with available test results compared to the existing design formulas.

Simple artificial neural networks that match probability and exploit and explore when confronting a multiarmed bandit.

Science.gov (United States)

Dawson, Michael R W; Dupuis, Brian; Spetch, Marcia L; Kelly, Debbie M

2009-08-01

The matching law (Herrnstein 1961) states that response rates become proportional to reinforcement rates; this is related to the empirical phenomenon called probability matching (Vulkan 2000). Here, we show that a simple artificial neural network generates responses consistent with probability matching. This behavior was then used to create an operant procedure for network learning. We use the multiarmed bandit (Gittins 1989), a classic problem of choice behavior, to illustrate that operant training balances exploiting the bandit arm expected to pay off most frequently with exploring other arms. Perceptrons provide a medium for relating results from neural networks, genetic algorithms, animal learning, contingency theory, reinforcement learning, and theories of choice.

Enhancing neural activity to drive respiratory plasticity following cervical spinal cord injury

Science.gov (United States)

Hormigo, Kristiina M.; Zholudeva, Lyandysha V.; Spruance, Victoria M.; Marchenko, Vitaliy; Cote, Marie-Pascale; Vinit, Stephane; Giszter, Simon; Bezdudnaya, Tatiana; Lane, Michael A.

2016-01-01

Cervical spinal cord injury (SCI) results in permanent life-altering sensorimotor deficits, among which impaired breathing is one of the most devastating and life-threatening. While clinical and experimental research has revealed that some spontaneous respiratory improvement (functional plasticity) can occur post-SCI, the extent of the recovery is limited and significant deficits persist. Thus, increasing effort is being made to develop therapies that harness and enhance this neuroplastic potential to optimize long-term recovery of breathing in injured individuals. One strategy with demonstrated therapeutic potential is the use of treatments that increase neural and muscular activity (e.g. locomotor training, neural and muscular stimulation) and promote plasticity. With a focus on respiratory function post-SCI, this review will discuss advances in the use of neural interfacing strategies and activity-based treatments, and highlights some recent results from our own research. PMID:27582085

Emergence of gamma motor activity in an artificial neural network model of the corticospinal system.

Science.gov (United States)

Grandjean, Bernard; Maier, Marc A

2017-02-01

Muscle spindle discharge during active movement is a function of mechanical and neural parameters. Muscle length changes (and their derivatives) represent its primary mechanical, fusimotor drive its neural component. However, neither the action nor the function of fusimotor and in particular of γ-drive, have been clearly established, since γ-motor activity during voluntary, non-locomotor movements remains largely unknown. Here, using a computational approach, we explored whether γ-drive emerges in an artificial neural network model of the corticospinal system linked to a biomechanical antagonist wrist simulator. The wrist simulator included length-sensitive and γ-drive-dependent type Ia and type II muscle spindle activity. Network activity and connectivity were derived by a gradient descent algorithm to generate reciprocal, known target α-motor unit activity during wrist flexion-extension (F/E) movements. Two tasks were simulated: an alternating F/E task and a slow F/E tracking task. Emergence of γ-motor activity in the alternating F/E network was a function of α-motor unit drive: if muscle afferent (together with supraspinal) input was required for driving α-motor units, then γ-drive emerged in the form of α-γ coactivation, as predicted by empirical studies. In the slow F/E tracking network, γ-drive emerged in the form of α-γ dissociation and provided critical, bidirectional muscle afferent activity to the cortical network, containing known bidirectional target units. The model thus demonstrates the complementary aspects of spindle output and hence γ-drive: i) muscle spindle activity as a driving force of α-motor unit activity, and ii) afferent activity providing continuous sensory information, both of which crucially depend on γ-drive.

Reinforcement function design and bias for efficient learning in mobile robots

International Nuclear Information System (INIS)

Touzet, C.; Santos, J.M.

1998-01-01

The main paradigm in sub-symbolic learning robot domain is the reinforcement learning method. Various techniques have been developed to deal with the memorization/generalization problem, demonstrating the superior ability of artificial neural network implementations. In this paper, the authors address the issue of designing the reinforcement so as to optimize the exploration part of the learning. They also present and summarize works relative to the use of bias intended to achieve the effective synthesis of the desired behavior. Demonstrative experiments involving a self-organizing map implementation of the Q-learning and real mobile robots (Nomad 200 and Khepera) in a task of obstacle avoidance behavior synthesis are described. 3 figs., 5 tabs

TensorFlow Agents: Efficient Batched Reinforcement Learning in TensorFlow

OpenAIRE

Hafner, Danijar; Davidson, James; Vanhoucke, Vincent

2017-01-01

We introduce TensorFlow Agents, an efficient infrastructure paradigm for building parallel reinforcement learning algorithms in TensorFlow. We simulate multiple environments in parallel, and group them to perform the neural network computation on a batch rather than individual observations. This allows the TensorFlow execution engine to parallelize computation, without the need for manual synchronization. Environments are stepped in separate Python processes to progress them in parallel witho...

OptoZIF Drive: a 3D printed implant and assembly tool package for neural recording and optical stimulation in freely moving mice

Science.gov (United States)

Freedman, David S.; Schroeder, Joseph B.; Telian, Gregory I.; Zhang, Zhengyang; Sunil, Smrithi; Ritt, Jason T.

2016-12-01

Objective. Behavioral neuroscience studies in freely moving rodents require small, light-weight implants to facilitate neural recording and stimulation. Our goal was to develop an integrated package of 3D printed parts and assembly aids for labs to rapidly fabricate, with minimal training, an implant that combines individually positionable microelectrodes, an optical fiber, zero insertion force (ZIF-clip) headstage connection, and secondary recording electrodes, e.g. for electromyography (EMG). Approach. Starting from previous implant designs that position recording electrodes using a control screw, we developed an implant where the main drive body, protective shell, and non-metal components of the microdrives are 3D printed in parallel. We compared alternative shapes and orientations of circuit boards for electrode connection to the headstage, in terms of their size, weight, and ease of wire insertion. We iteratively refined assembly methods, and integrated additional assembly aids into the 3D printed casing. Main results. We demonstrate the effectiveness of the OptoZIF Drive by performing real time optogenetic feedback in behaving mice. A novel feature of the OptoZIF Drive is its vertical circuit board, which facilities direct ZIF-clip connection. This feature requires angled insertion of an optical fiber that still can exit the drive from the center of a ring of recording electrodes. We designed an innovative 2-part protective shell that can be installed during the implant surgery to facilitate making additional connections to the circuit board. We use this feature to show that facial EMG in mice can be used as a control signal to lock stimulation to the animal’s motion, with stable EMG signal over several months. To decrease assembly time, reduce assembly errors, and improve repeatability, we fabricate assembly aids including a drive holder, a drill guide, an implant fixture for microelectode ‘pinning’, and a gold plating fixture. Significance. The

Optimisation of hybrid high-modulus/high-strength carbon fiber reinforced plastic composite drive

OpenAIRE

Montagnier, Olivier; Hochard, Christian

2011-01-01

International audience; This study deals with the optimisation of hybrid composite drive shafts operating at subcritical or supercritical speeds, using a genetic algorithm. A formulation for the flexural vibrations of a composite drive shaft mounted on viscoelastic supports including shear effects is developed. In particular, an analytic stability criterion is developed to ensure the integrity of the system in the supercritical regime. Then it is shown that the torsional strength can be compu...

Nano-topography Enhances Communication in Neural Cells Networks

KAUST Repository

Onesto, V.

2017-08-23

Neural cells are the smallest building blocks of the central and peripheral nervous systems. Information in neural networks and cell-substrate interactions have been heretofore studied separately. Understanding whether surface nano-topography can direct nerve cells assembly into computational efficient networks may provide new tools and criteria for tissue engineering and regenerative medicine. In this work, we used information theory approaches and functional multi calcium imaging (fMCI) techniques to examine how information flows in neural networks cultured on surfaces with controlled topography. We found that substrate roughness Sa affects networks topology. In the low nano-meter range, S-a = 0-30 nm, information increases with Sa. Moreover, we found that energy density of a network of cells correlates to the topology of that network. This reinforces the view that information, energy and surface nano-topography are tightly inter-connected and should not be neglected when studying cell-cell interaction in neural tissue repair and regeneration.

Moods as ups and downs of the motivation pendulum: Revisiting Reinforcement Sensitivity Theory (RST in Bipolar Disorder

Directory of Open Access Journals (Sweden)

Tal eGonen

2014-11-01

Full Text Available Motivation is a key neurobehavioral concept underlying adaptive responses to environmental incentives and threats. As such, dysregulation of motivational processes may be critical in the formation of abnormal behavioral patterns/tendencies. According to the long standing model of the Reinforcement Sensitivity Theory (RST, motivation behaviors are driven by three neurobehavioral systems mediating the sensitivity to punishment, reward or goal-conflict. Corresponding to current neurobehavioral theories in psychiatry, this theory links abnormal motivational drives to abnormal behavior; viewing depression and mania as two abnormal extremes of reward driven processes leading to either under or over approach tendencies, respectively. We revisit the RST framework in the context of bipolar disorder (BD and challenge this concept by suggesting that dysregulated interactions of both punishment and reward related processes better account for the psychological and neural abnormalities observed in BD. We further present an integrative model positing that the three parallel motivation systems currently proposed by the RST model, can be viewed as subsystems in a large-scale neurobehavioral network of motivational decision making.

Neural Networks Integrated Circuit for Biomimetics MEMS Microrobot

Directory of Open Access Journals (Sweden)

Ken Saito

2014-06-01

Full Text Available In this paper, we will propose the neural networks integrated circuit (NNIC which is the driving waveform generator of the 4.0, 2.7, 2.5 mm, width, length, height in size biomimetics microelectromechanical systems (MEMS microrobot. The microrobot was made from silicon wafer fabricated by micro fabrication technology. The mechanical system of the robot was equipped with small size rotary type actuators, link mechanisms and six legs to realize the ant-like switching behavior. The NNIC generates the driving waveform using synchronization phenomena such as biological neural networks. The driving waveform can operate the actuators of the MEMS microrobot directly. Therefore, the NNIC bare chip realizes the robot control without using any software programs or A/D converters. The microrobot performed forward and backward locomotion, and also changes direction by inputting an external single trigger pulse. The locomotion speed of the microrobot was 26.4 mm/min when the step width was 0.88 mm. The power consumption of the system was 250 mWh when the room temperature was 298 K.

Simulation-based optimization parametric optimization techniques and reinforcement learning

CERN Document Server

Gosavi, Abhijit

2003-01-01

Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning introduces the evolving area of simulation-based optimization. The book's objective is two-fold: (1) It examines the mathematical governing principles of simulation-based optimization, thereby providing the reader with the ability to model relevant real-life problems using these techniques. (2) It outlines the computational technology underlying these methods. Taken together these two aspects demonstrate that the mathematical and computational methods discussed in this book do work. Broadly speaking, the book has two parts: (1) parametric (static) optimization and (2) control (dynamic) optimization. Some of the book's special features are: *An accessible introduction to reinforcement learning and parametric-optimization techniques. *A step-by-step description of several algorithms of simulation-based optimization. *A clear and simple introduction to the methodology of neural networks. *A gentle introduction to converg...

The Neural Representation of Goal-Directed Actions and Outcomes in the Ventral Striatum's Olfactory Tubercle

Science.gov (United States)

Gadziola, Marie A.

2016-01-01

The ventral striatum is critical for evaluating reward information and the initiation of goal-directed behaviors. The many cellular, afferent, and efferent similarities between the ventral striatum's nucleus accumbens and olfactory tubercle (OT) suggests the distributed involvement of neurons within the ventral striatopallidal complex in motivated behaviors. Although the nucleus accumbens has an established role in representing goal-directed actions and their outcomes, it is not known whether this function is localized within the nucleus accumbens or distributed also within the OT. Answering such a fundamental question will expand our understanding of the neural mechanisms underlying motivated behaviors. Here we address whether the OT encodes natural reinforcers and serves as a substrate for motivational information processing. In recordings from mice engaged in a novel water-motivated instrumental task, we report that OT neurons modulate their firing rate during initiation and progression of the instrumental licking behavior, with some activity being internally generated and preceding the first lick. We further found that as motivational drive decreases throughout a session, the activity of OT neurons is enhanced earlier relative to the behavioral action. Additionally, OT neurons discriminate the types and magnitudes of fluid reinforcers. Together, these data suggest that the processing of reward information and the orchestration of goal-directed behaviors is a global principle of the ventral striatum and have important implications for understanding the neural systems subserving addiction and mood disorders. SIGNIFICANCE STATEMENT Goal-directed behaviors are widespread among animals and underlie complex behaviors ranging from food intake, social behavior, and even pathological conditions, such as gambling and drug addiction. The ventral striatum is a neural system critical for evaluating reward information and the initiation of goal-directed behaviors. Here we

Biologically Inspired Modular Neural Control for a Leg-Wheel Hybrid Robot

DEFF Research Database (Denmark)

Manoonpong, Poramate; Wörgötter, Florentin; Laksanacharoen, Pudit

2014-01-01

In this article we present modular neural control for a leg-wheel hybrid robot consisting of three legs with omnidirectional wheels. This neural control has four main modules having their functional origin in biological neural systems. A minimal recurrent control (MRC) module is for sensory signal...... processing and state memorization. Its outputs drive two front wheels while the rear wheel is controlled through a velocity regulating network (VRN) module. In parallel, a neural oscillator network module serves as a central pattern generator (CPG) controls leg movements for sidestepping. Stepping directions...... or they can serve as useful modules for other module-based neural control applications....

Behavior of reinforced concrete beams reinforced with GFRP bars

Directory of Open Access Journals (Sweden)

D. H. Tavares

Full Text Available The use of fiber reinforced polymer (FRP bars is one of the alternatives presented in recent studies to prevent the drawbacks related to the steel reinforcement in specific reinforced concrete members. In this work, six reinforced concrete beams were submitted to four point bending tests. One beam was reinforced with CA-50 steel bars and five with glass fiber reinforced polymer (GFRP bars. The tests were carried out in the Department of Structural Engineering in São Carlos Engineering School, São Paulo University. The objective of the test program was to compare strength, reinforcement deformation, displacement, and some anchorage aspects between the GFRP-reinforced concrete beams and the steel-reinforced concrete beam. The results show that, even though four GFRP-reinforced concrete beams were designed with the same internal tension force as that with steel reinforcement, their capacity was lower than that of the steel-reinforced beam. The results also show that similar flexural capacity can be achieved for the steel- and for the GFRP-reinforced concrete beams by controlling the stiffness (reinforcement modulus of elasticity multiplied by the bar cross-sectional area - EA and the tension force of the GFRP bars.

Delay-range-dependent exponential H∞ synchronization of a class of delayed neural networks

International Nuclear Information System (INIS)

Karimi, Hamid Reza; Maass, Peter

2009-01-01

This article aims to present a multiple delayed state-feedback control design for exponential H ∞ synchronization problem of a class of delayed neural networks with multiple time-varying discrete delays. On the basis of the drive-response concept and by introducing a descriptor technique and using Lyapunov-Krasovskii functional, new delay-range-dependent sufficient conditions for exponential H ∞ synchronization of the drive-response structure of neural networks are driven in terms of linear matrix inequalities (LMIs). The explicit expression of the controller gain matrices are parameterized based on the solvability conditions such that the drive system and the response system can be exponentially synchronized. A numerical example is included to illustrate the applicability of the proposed design method.

One central oscillatory drive is compatible with experimental motor unit behaviour in essential and Parkinsonian tremor

Science.gov (United States)

Dideriksen, Jakob L.; Gallego, Juan A.; Holobar, Ales; Rocon, Eduardo; Pons, Jose L.; Farina, Dario

2015-08-01

Objective. Pathological tremors are symptomatic to several neurological disorders that are difficult to differentiate and the way by which central oscillatory networks entrain tremorogenic contractions is unknown. We considered the alternative hypotheses that tremor arises from one oscillator (at the tremor frequency) or, as suggested by recent findings from the superimposition of two separate inputs (at the tremor frequency and twice that frequency). Approach. Assuming one central oscillatory network we estimated analytically the relative amplitude of the harmonics of the tremor frequency in the motor neuron output for different temporal behaviors of the oscillator. Next, we analyzed the bias in the relative harmonics amplitude introduced by superimposing oscillations at twice the tremor frequency. These findings were validated using experimental measurements of wrist angular velocity and surface electromyography (EMG) from 22 patients (11 essential tremor, 11 Parkinson’s disease). The ensemble motor unit action potential trains identified from the EMG represented the neural drive to the muscles. Main results. The analytical results showed that the relative power of the tremor harmonics in the analytical models of the neural drive was determined by the variability and duration of the tremor bursts and the presence of the second oscillator biased this power towards higher values. The experimental findings accurately matched the analytical model assuming one oscillator, indicating a negligible functional role of secondary oscillatory inputs. Furthermore, a significant difference in the relative power of harmonics in the neural drive was found across the patient groups, suggesting a diagnostic value of this measure (classification accuracy: 86%). This diagnostic power decreased substantially when estimated from limb acceleration or the EMG. Signficance. The results indicate that the neural drive in pathological tremor is compatible with one central network

Modeling Heavy/Medium-Duty Fuel Consumption Based on Drive Cycle Properties

Energy Technology Data Exchange (ETDEWEB)

Wang, Lijuan; Duran, Adam; Gonder, Jeffrey; Kelly, Kenneth

2015-10-13

This paper presents multiple methods for predicting heavy/medium-duty vehicle fuel consumption based on driving cycle information. A polynomial model, a black box artificial neural net model, a polynomial neural network model, and a multivariate adaptive regression splines (MARS) model were developed and verified using data collected from chassis testing performed on a parcel delivery diesel truck operating over the Heavy Heavy-Duty Diesel Truck (HHDDT), City Suburban Heavy Vehicle Cycle (CSHVC), New York Composite Cycle (NYCC), and hydraulic hybrid vehicle (HHV) drive cycles. Each model was trained using one of four drive cycles as a training cycle and the other three as testing cycles. By comparing the training and testing results, a representative training cycle was chosen and used to further tune each method. HHDDT as the training cycle gave the best predictive results, because HHDDT contains a variety of drive characteristics, such as high speed, acceleration, idling, and deceleration. Among the four model approaches, MARS gave the best predictive performance, with an average absolute percent error of -1.84% over the four chassis dynamometer drive cycles. To further evaluate the accuracy of the predictive models, the approaches were first applied to real-world data. MARS outperformed the other three approaches, providing an average absolute percent error of -2.2% of four real-world road segments. The MARS model performance was then compared to HHDDT, CSHVC, NYCC, and HHV drive cycles with the performance from Future Automotive System Technology Simulator (FASTSim). The results indicated that the MARS method achieved a comparative predictive performance with FASTSim.

Implementation of real-time energy management strategy based on reinforcement learning for hybrid electric vehicles and simulation validation.

Science.gov (United States)

Kong, Zehui; Zou, Yuan; Liu, Teng

2017-01-01

To further improve the fuel economy of series hybrid electric tracked vehicles, a reinforcement learning (RL)-based real-time energy management strategy is developed in this paper. In order to utilize the statistical characteristics of online driving schedule effectively, a recursive algorithm for the transition probability matrix (TPM) of power-request is derived. The reinforcement learning (RL) is applied to calculate and update the control policy at regular time, adapting to the varying driving conditions. A facing-forward powertrain model is built in detail, including the engine-generator model, battery model and vehicle dynamical model. The robustness and adaptability of real-time energy management strategy are validated through the comparison with the stationary control strategy based on initial transition probability matrix (TPM) generated from a long naturalistic driving cycle in the simulation. Results indicate that proposed method has better fuel economy than stationary one and is more effective in real-time control.

Implementation of real-time energy management strategy based on reinforcement learning for hybrid electric vehicles and simulation validation.

Directory of Open Access Journals (Sweden)

Zehui Kong

Full Text Available To further improve the fuel economy of series hybrid electric tracked vehicles, a reinforcement learning (RL-based real-time energy management strategy is developed in this paper. In order to utilize the statistical characteristics of online driving schedule effectively, a recursive algorithm for the transition probability matrix (TPM of power-request is derived. The reinforcement learning (RL is applied to calculate and update the control policy at regular time, adapting to the varying driving conditions. A facing-forward powertrain model is built in detail, including the engine-generator model, battery model and vehicle dynamical model. The robustness and adaptability of real-time energy management strategy are validated through the comparison with the stationary control strategy based on initial transition probability matrix (TPM generated from a long naturalistic driving cycle in the simulation. Results indicate that proposed method has better fuel economy than stationary one and is more effective in real-time control.

Neural modularity helps organisms evolve to learn new skills without forgetting old skills.

Science.gov (United States)

Ellefsen, Kai Olav; Mouret, Jean-Baptiste; Clune, Jeff

2015-04-01

A long-standing goal in artificial intelligence is creating agents that can learn a variety of different skills for different problems. In the artificial intelligence subfield of neural networks, a barrier to that goal is that when agents learn a new skill they typically do so by losing previously acquired skills, a problem called catastrophic forgetting. That occurs because, to learn the new task, neural learning algorithms change connections that encode previously acquired skills. How networks are organized critically affects their learning dynamics. In this paper, we test whether catastrophic forgetting can be reduced by evolving modular neural networks. Modularity intuitively should reduce learning interference between tasks by separating functionality into physically distinct modules in which learning can be selectively turned on or off. Modularity can further improve learning by having a reinforcement learning module separate from sensory processing modules, allowing learning to happen only in response to a positive or negative reward. In this paper, learning takes place via neuromodulation, which allows agents to selectively change the rate of learning for each neural connection based on environmental stimuli (e.g. to alter learning in specific locations based on the task at hand). To produce modularity, we evolve neural networks with a cost for neural connections. We show that this connection cost technique causes modularity, confirming a previous result, and that such sparsely connected, modular networks have higher overall performance because they learn new skills faster while retaining old skills more and because they have a separate reinforcement learning module. Our results suggest (1) that encouraging modularity in neural networks may help us overcome the long-standing barrier of networks that cannot learn new skills without forgetting old ones, and (2) that one benefit of the modularity ubiquitous in the brains of natural animals might be to

Neural modularity helps organisms evolve to learn new skills without forgetting old skills.

Directory of Open Access Journals (Sweden)

Kai Olav Ellefsen

2015-04-01

Full Text Available A long-standing goal in artificial intelligence is creating agents that can learn a variety of different skills for different problems. In the artificial intelligence subfield of neural networks, a barrier to that goal is that when agents learn a new skill they typically do so by losing previously acquired skills, a problem called catastrophic forgetting. That occurs because, to learn the new task, neural learning algorithms change connections that encode previously acquired skills. How networks are organized critically affects their learning dynamics. In this paper, we test whether catastrophic forgetting can be reduced by evolving modular neural networks. Modularity intuitively should reduce learning interference between tasks by separating functionality into physically distinct modules in which learning can be selectively turned on or off. Modularity can further improve learning by having a reinforcement learning module separate from sensory processing modules, allowing learning to happen only in response to a positive or negative reward. In this paper, learning takes place via neuromodulation, which allows agents to selectively change the rate of learning for each neural connection based on environmental stimuli (e.g. to alter learning in specific locations based on the task at hand. To produce modularity, we evolve neural networks with a cost for neural connections. We show that this connection cost technique causes modularity, confirming a previous result, and that such sparsely connected, modular networks have higher overall performance because they learn new skills faster while retaining old skills more and because they have a separate reinforcement learning module. Our results suggest (1 that encouraging modularity in neural networks may help us overcome the long-standing barrier of networks that cannot learn new skills without forgetting old ones, and (2 that one benefit of the modularity ubiquitous in the brains of natural animals

Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills

Science.gov (United States)

Ellefsen, Kai Olav; Mouret, Jean-Baptiste; Clune, Jeff

2015-01-01

A long-standing goal in artificial intelligence is creating agents that can learn a variety of different skills for different problems. In the artificial intelligence subfield of neural networks, a barrier to that goal is that when agents learn a new skill they typically do so by losing previously acquired skills, a problem called catastrophic forgetting. That occurs because, to learn the new task, neural learning algorithms change connections that encode previously acquired skills. How networks are organized critically affects their learning dynamics. In this paper, we test whether catastrophic forgetting can be reduced by evolving modular neural networks. Modularity intuitively should reduce learning interference between tasks by separating functionality into physically distinct modules in which learning can be selectively turned on or off. Modularity can further improve learning by having a reinforcement learning module separate from sensory processing modules, allowing learning to happen only in response to a positive or negative reward. In this paper, learning takes place via neuromodulation, which allows agents to selectively change the rate of learning for each neural connection based on environmental stimuli (e.g. to alter learning in specific locations based on the task at hand). To produce modularity, we evolve neural networks with a cost for neural connections. We show that this connection cost technique causes modularity, confirming a previous result, and that such sparsely connected, modular networks have higher overall performance because they learn new skills faster while retaining old skills more and because they have a separate reinforcement learning module. Our results suggest (1) that encouraging modularity in neural networks may help us overcome the long-standing barrier of networks that cannot learn new skills without forgetting old ones, and (2) that one benefit of the modularity ubiquitous in the brains of natural animals might be to

Traffic sign recognition with deep convolutional neural networks

OpenAIRE

Karamatić, Boris

2016-01-01

The problem of detection and recognition of traffic signs is becoming an important problem when it comes to the development of self driving cars and advanced driver assistance systems. In this thesis we will develop a system for detection and recognition of traffic signs. For the problem of detection we will use aggregate channel features and for the problem of recognition we will use a deep convolutional neural network. We will describe how convolutional neural networks work, how they are co...

Optimizing microstimulation using a reinforcement learning framework.

Science.gov (United States)

Brockmeier, Austin J; Choi, John S; Distasio, Marcello M; Francis, Joseph T; Príncipe, José C

2011-01-01

The ability to provide sensory feedback is desired to enhance the functionality of neuroprosthetics. Somatosensory feedback provides closed-loop control to the motor system, which is lacking in feedforward neuroprosthetics. In the case of existing somatosensory function, a template of the natural response can be used as a template of desired response elicited by electrical microstimulation. In the case of no initial training data, microstimulation parameters that produce responses close to the template must be selected in an online manner. We propose using reinforcement learning as a framework to balance the exploration of the parameter space and the continued selection of promising parameters for further stimulation. This approach avoids an explicit model of the neural response from stimulation. We explore a preliminary architecture--treating the task as a k-armed bandit--using offline data recorded for natural touch and thalamic microstimulation, and we examine the methods efficiency in exploring the parameter space while concentrating on promising parameter forms. The best matching stimulation parameters, from k = 68 different forms, are selected by the reinforcement learning algorithm consistently after 334 realizations.

Inactivity-induced respiratory plasticity: Protecting the drive to breathe in disorders that reduce respiratory neural activity☆

Science.gov (United States)

Strey, K.A.; Baertsch, N.A.; Baker-Herman, T.L.

2013-01-01

Multiple forms of plasticity are activated following reduced respiratory neural activity. For example, in ventilated rats, a central neural apnea elicits a rebound increase in phrenic and hypoglossal burst amplitude upon resumption of respiratory neural activity, forms of plasticity called inactivity-induced phrenic and hypoglossal motor facilitation (iPMF and iHMF), respectively. Here, we provide a conceptual framework for plasticity following reduced respiratory neural activity to guide future investigations. We review mechanisms giving rise to iPMF and iHMF, present new data suggesting that inactivity-induced plasticity is observed in inspiratory intercostals (iIMF) and point out gaps in our knowledge. We then survey conditions relevant to human health characterized by reduced respiratory neural activity and discuss evidence that inactivity-induced plasticity is elicited during these conditions. Understanding the physiological impact and circumstances in which inactivity-induced respiratory plasticity is elicited may yield novel insights into the treatment of disorders characterized by reductions in respiratory neural activity. PMID:23816599

Wind Turbine Driving a PM Synchronous Generator Using Novel Recurrent Chebyshev Neural Network Control with the Ideal Learning Rate

Directory of Open Access Journals (Sweden)

Chih-Hong Lin

2016-06-01

Full Text Available A permanent magnet (PM synchronous generator system driven by wind turbine (WT, connected with smart grid via AC-DC converter and DC-AC converter, are controlled by the novel recurrent Chebyshev neural network (NN and amended particle swarm optimization (PSO to regulate output power and output voltage in two power converters in this study. Because a PM synchronous generator system driven by WT is an unknown non-linear and time-varying dynamic system, the on-line training novel recurrent Chebyshev NN control system is developed to regulate DC voltage of the AC-DC converter and AC voltage of the DC-AC converter connected with smart grid. Furthermore, the variable learning rate of the novel recurrent Chebyshev NN is regulated according to discrete-type Lyapunov function for improving the control performance and enhancing convergent speed. Finally, some experimental results are shown to verify the effectiveness of the proposed control method for a WT driving a PM synchronous generator system in smart grid.

Braided reinforced composite rods for the internal reinforcement of concrete

Science.gov (United States)

Gonilho Pereira, C.; Fangueiro, R.; Jalali, S.; Araujo, M.; Marques, P.

2008-05-01

This paper reports on the development of braided reinforced composite rods as a substitute for the steel reinforcement in concrete. The research work aims at understanding the mechanical behaviour of core-reinforced braided fabrics and braided reinforced composite rods, namely concerning the influence of the braiding angle, the type of core reinforcement fibre, and preloading and postloading conditions. The core-reinforced braided fabrics were made from polyester fibres for producing braided structures, and E-glass, carbon, HT polyethylene, and sisal fibres were used for the core reinforcement. The braided reinforced composite rods were obtained by impregnating the core-reinforced braided fabric with a vinyl ester resin. The preloading of the core-reinforced braided fabrics and the postloading of the braided reinforced composite rods were performed in three and two stages, respectively. The results of tensile tests carried out on different samples of core-reinforced braided fabrics are presented and discussed. The tensile and bending properties of the braided reinforced composite rods have been evaluated, and the results obtained are presented, discussed, and compared with those of conventional materials, such as steel.

On the Application of TLS Techniques to AC Electrical Drives

Directory of Open Access Journals (Sweden)

M. Cirrincione

2005-03-01

Full Text Available This paper deals with the application of a new neuron, the TLS EXIN neuron, to AC induction motor drives. In particular, it addresses two important subjects of AC induction motor drives: the on-line estimation of the electrical parameters of the machine and the speed estimation in sensorless drives. On this basis, this work summarizes the parameter estimation and sensorless techniques already developed by the authors over these last few years, all based on the TLS EXIN. With regard to sensorless, two techniques are proposed: one based on the MRAS and the other based on the full-order Luenberger observer. The work show some of the most significant results obtained by the authors in these fields and stresses the important potentiality of this new neural technique in AC induction machine drives.

Flexural reinforced concrete member with FRP reinforcement

OpenAIRE

Putzolu, Mariana

2017-01-01

One of the most problematic point in construction is the durability of the concrete especially related to corrosion of the steel reinforcement. Due to this problem the construction sector, introduced the use of Fiber Reinforced Polymer, the main fibers used in construction are Glass, Carbon and Aramid. In this study, the author aim to analyse the flexural behaviour of concrete beams reinforced with FRP. This aim is achieved by the analysis of specimens reinforced with GFRP bars, with theoreti...

The drift diffusion model as the choice rule in reinforcement learning.

Science.gov (United States)

Pedersen, Mads Lund; Frank, Michael J; Biele, Guido

2017-08-01

Current reinforcement-learning models often assume simplified decision processes that do not fully reflect the dynamic complexities of choice processes. Conversely, sequential-sampling models of decision making account for both choice accuracy and response time, but assume that decisions are based on static decision values. To combine these two computational models of decision making and learning, we implemented reinforcement-learning models in which the drift diffusion model describes the choice process, thereby capturing both within- and across-trial dynamics. To exemplify the utility of this approach, we quantitatively fit data from a common reinforcement-learning paradigm using hierarchical Bayesian parameter estimation, and compared model variants to determine whether they could capture the effects of stimulant medication in adult patients with attention-deficit hyperactivity disorder (ADHD). The model with the best relative fit provided a good description of the learning process, choices, and response times. A parameter recovery experiment showed that the hierarchical Bayesian modeling approach enabled accurate estimation of the model parameters. The model approach described here, using simultaneous estimation of reinforcement-learning and drift diffusion model parameters, shows promise for revealing new insights into the cognitive and neural mechanisms of learning and decision making, as well as the alteration of such processes in clinical groups.

BAS-drive trait modulates dorsomedial striatum activity during reward response-outcome associations.

Science.gov (United States)

Costumero, Víctor; Barrós-Loscertales, Alfonso; Fuentes, Paola; Rosell-Negre, Patricia; Bustamante, Juan Carlos; Ávila, César

2016-09-01

According to the Reinforcement Sensitivity Theory, behavioral studies have found that individuals with stronger reward sensitivity easily detect cues of reward and establish faster associations between instrumental responses and reward. Neuroimaging studies have shown that processing anticipatory cues of reward is accompanied by stronger ventral striatum activity in individuals with stronger reward sensitivity. Even though establishing response-outcome contingencies has been consistently associated with dorsal striatum, individual differences in this process are poorly understood. Here, we aimed to study the relation between reward sensitivity and brain activity while processing response-reward contingencies. Forty-five participants completed the BIS/BAS questionnaire and performed a gambling task paradigm in which they received monetary rewards or punishments. Overall, our task replicated previous results that have related processing high reward outcomes with activation of striatum and medial frontal areas, whereas processing high punishment outcomes was associated with stronger activity in insula and middle cingulate. As expected, the individual differences in the activity of dorsomedial striatum correlated positively with BAS-Drive. Our results agree with previous studies that have related the dorsomedial striatum with instrumental performance, and suggest that the individual differences in this area may form part of the neural substrate responsible for modulating instrumental conditioning by reward sensitivity.

Adaptive exponential synchronization of delayed neural networks with reaction-diffusion terms

International Nuclear Information System (INIS)

Sheng Li; Yang Huizhong; Lou Xuyang

2009-01-01

This paper presents an exponential synchronization scheme for a class of neural networks with time-varying and distributed delays and reaction-diffusion terms. An adaptive synchronization controller is derived to achieve the exponential synchronization of the drive-response structure of neural networks by using the Lyapunov stability theory. At the same time, the update laws of parameters are proposed to guarantee the synchronization of delayed neural networks with all parameters unknown. It is shown that the approaches developed here extend and improve the ideas presented in recent literatures.

Variability in prescription opioid intake and reinforcement amongst 129 substrains.

Science.gov (United States)

Jimenez, S M; Healy, A F; Coelho, M A; Brown, C N; Kippin, T E; Szumlinski, K K

2017-09-01

Opioid abuse in the United States has reached epidemic proportions, with treatment admissions and deaths associated with prescription opioid abuse quadrupling over the past 10 years. Although genetics are theorized to contribute substantially to inter-individual variability in the development, severity and treatment outcomes of opioid abuse/addiction, little direct preclinical study has focused on the behavioral genetics of prescription opioid reinforcement and drug-taking. Herein, we employed different 129 substrains of mice currently available from The Jackson Laboratory (129S1/SvlmJ, 129X1/SvJ, 129S4/SvJaeJ and 129P3/J) as a model system of genetic variation and assayed mice for oral opioid intake and reinforcement, as well as behavioral and somatic signs of dependence. All substrains exhibited a dose-dependent increase in oral oxycodone and heroin preference and intake under limited-access procedures and all, but 129S1/SvlmJ mice, exhibited oxycodone reinforcement. Relative to the other substrains, 129P3/J mice exhibited higher heroin and oxycodone intake. While 129X1/SvJ exhibited the highest anxiety-like behavior during natural opioid withdrawal, somatic and behavior signs of precipitated withdrawal were most robust in 129P3/J mice. These results demonstrate the feasibility and relative sensitivity of our oral opioid self-administration procedures for detecting substrain differences in drug reinforcement/intake among 129 mice, of relevance to the identification of genetic variants contributing to high vs. low oxycodone reinforcement and intake. © 2017 John Wiley & Sons Ltd and International Behavioural and Neural Genetics Society.

Human Embryonic Stem Cells: A Model for the Study of Neural Development and Neurological Diseases

Directory of Open Access Journals (Sweden)

Piya Prajumwongs

2016-01-01

Full Text Available Although the mechanism of neurogenesis has been well documented in other organisms, there might be fundamental differences between human and those species referring to species-specific context. Based on principles learned from other systems, it is found that the signaling pathways required for neural induction and specification of human embryonic stem cells (hESCs recapitulated those in the early embryo development in vivo at certain degree. This underscores the usefulness of hESCs in understanding early human neural development and reinforces the need to integrate the principles of developmental biology and hESC biology for an efficient neural differentiation.

Fluctuation-Driven Neural Dynamics Reproduce Drosophila Locomotor Patterns.

Directory of Open Access Journals (Sweden)

Andrea Maesani

2015-11-01

Full Text Available The neural mechanisms determining the timing of even simple actions, such as when to walk or rest, are largely mysterious. One intriguing, but untested, hypothesis posits a role for ongoing activity fluctuations in neurons of central action selection circuits that drive animal behavior from moment to moment. To examine how fluctuating activity can contribute to action timing, we paired high-resolution measurements of freely walking Drosophila melanogaster with data-driven neural network modeling and dynamical systems analysis. We generated fluctuation-driven network models whose outputs-locomotor bouts-matched those measured from sensory-deprived Drosophila. From these models, we identified those that could also reproduce a second, unrelated dataset: the complex time-course of odor-evoked walking for genetically diverse Drosophila strains. Dynamical models that best reproduced both Drosophila basal and odor-evoked locomotor patterns exhibited specific characteristics. First, ongoing fluctuations were required. In a stochastic resonance-like manner, these fluctuations allowed neural activity to escape stable equilibria and to exceed a threshold for locomotion. Second, odor-induced shifts of equilibria in these models caused a depression in locomotor frequency following olfactory stimulation. Our models predict that activity fluctuations in action selection circuits cause behavioral output to more closely match sensory drive and may therefore enhance navigation in complex sensory environments. Together these data reveal how simple neural dynamics, when coupled with activity fluctuations, can give rise to complex patterns of animal behavior.

Adolescent-specific patterns of behavior and neural activity during social reinforcement learning

OpenAIRE

Jones, Rebecca M.; Somerville, Leah H.; Li, Jian; Ruberry, Erika J.; Powers, Alisa; Mehta, Natasha; Dyke, Jonathan; Casey, BJ

2014-01-01

Humans are sophisticated social beings. Social cues from others are exceptionally salient, particularly during adolescence. Understanding how adolescents interpret and learn from variable social signals can provide insight into the observed shift in social sensitivity during this period. The current study tested 120 participants between the ages of 8 and 25 years on a social reinforcement learning task where the probability of receiving positive social feedback was parametrically manipulated....

The probability of reinforcement per trial affects posttrial responding and subsequent extinction but not within-trial responding.

Science.gov (United States)

Harris, Justin A; Kwok, Dorothy W S

2018-01-01

During magazine approach conditioning, rats do not discriminate between a conditional stimulus (CS) that is consistently reinforced with food and a CS that is occasionally (partially) reinforced, as long as the CSs have the same overall reinforcement rate per second. This implies that rats are indifferent to the probability of reinforcement per trial. However, in the same rats, the per-trial reinforcement rate will affect subsequent extinction-responding extinguishes more rapidly for a CS that was consistently reinforced than for a partially reinforced CS. Here, we trained rats with consistently and partially reinforced CSs that were matched for overall reinforcement rate per second. We measured conditioned responding both during and immediately after the CSs. Differences in the per-trial probability of reinforcement did not affect the acquisition of responding during the CS but did affect subsequent extinction of that responding, and also affected the post-CS response rates during conditioning. Indeed, CSs with the same probability of reinforcement per trial evoked the same amount of post-CS responding even when they differed in overall reinforcement rate and thus evoked different amounts of responding during the CS. We conclude that reinforcement rate per second controls rats' acquisition of responding during the CS, but at the same time, rats also learn specifically about the probability of reinforcement per trial. The latter learning affects the rats' expectation of reinforcement as an outcome of the trial, which influences their ability to detect retrospectively that an opportunity for reinforcement was missed, and, in turn, drives extinction. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Disrupted expected value signaling in youth with disruptive behavior disorders to environmental reinforcers.

Science.gov (United States)

White, Stuart F; Fowler, Katherine A; Sinclair, Stephen; Schechter, Julia C; Majestic, Catherine M; Pine, Daniel S; Blair, R James

2014-05-01

Youth with disruptive behavior disorders (DBD), including conduct disorder (CD) and oppositional defiant disorder (ODD), have difficulties in reinforcement-based decision making, the neural basis of which is poorly understood. Studies examining decision making in youth with DBD have revealed reduced reward responses within the ventromedial prefrontal cortex/orbitofrontal cortex (vmPFC/OFC), increased responses to unexpected punishment within the vmPFC and striatum, and reduced use of expected value information in the anterior insula cortex and dorsal anterior cingulate cortex during the avoidance of suboptimal choices. Previous work has used only monetary reinforcement. The current study examined whether dysfunction in youth with DBD during decision making extended to environmental reinforcers. A total of 30 youth (15 healthy youth and 15 youth with DBD) completed a novel reinforcement-learning paradigm using environmental reinforcers (physical threat images, e.g., striking snake image; contamination threat images, e.g., rotting food; appetitive images, e.g., puppies) while undergoing functional magnetic resonance imaging (fMRI). Behaviorally, healthy youth were significantly more likely to avoid physical threat, but not contamination threat, stimuli than youth with DBD. Imaging results revealed that youth with DBD showed significantly reduced use of expected value information in the bilateral caudate, thalamus, and posterior cingulate cortex during the avoidance of suboptimal responses. The current data suggest that youth with DBD show deficits to environmental reinforcers similar to the deficits seen to monetary reinforcers. Importantly, this deficit was unrelated to callous-unemotional (CU) traits, suggesting that caudate impairment may be a common deficit across youth with DBD. Published by Elsevier Inc.

Hedgehog regulates Norrie disease protein to drive neural progenitor self-renewal.

Science.gov (United States)

McNeill, Brian; Mazerolle, Chantal; Bassett, Erin A; Mears, Alan J; Ringuette, Randy; Lagali, Pamela; Picketts, David J; Paes, Kim; Rice, Dennis; Wallace, Valerie A

2013-03-01

Norrie disease (ND) is a congenital disorder characterized by retinal hypovascularization and cognitive delay. ND has been linked to mutations in 'Norrie Disease Protein' (Ndp), which encodes the secreted protein Norrin. Norrin functions as a secreted angiogenic factor, although its role in neural development has not been assessed. Here, we show that Ndp expression is initiated in retinal progenitors in response to Hedgehog (Hh) signaling, which induces Gli2 binding to the Ndp promoter. Using a combination of genetic epistasis and acute RNAi-knockdown approaches, we show that Ndp is required downstream of Hh activation to induce retinal progenitor proliferation in the retina. Strikingly, Ndp regulates the rate of cell-cycle re-entry and not cell-cycle kinetics, thereby uncoupling the self-renewal and cell-cycle progression functions of Hh. Taken together, we have uncovered a cell autonomous function for Ndp in retinal progenitor proliferation that is independent of its function in the retinal vasculature, which could explain the neural defects associated with ND.

EEG Artifact Removal Using a Wavelet Neural Network

Science.gov (United States)

Nguyen, Hoang-Anh T.; Musson, John; Li, Jiang; McKenzie, Frederick; Zhang, Guangfan; Xu, Roger; Richey, Carl; Schnell, Tom

2011-01-01

!n this paper we developed a wavelet neural network. (WNN) algorithm for Electroencephalogram (EEG) artifact removal without electrooculographic (EOG) recordings. The algorithm combines the universal approximation characteristics of neural network and the time/frequency property of wavelet. We. compared the WNN algorithm with .the ICA technique ,and a wavelet thresholding method, which was realized by using the Stein's unbiased risk estimate (SURE) with an adaptive gradient-based optimal threshold. Experimental results on a driving test data set show that WNN can remove EEG artifacts effectively without diminishing useful EEG information even for very noisy data.

Gaze-contingent reinforcement learning reveals incentive value of social signals in young children and adults.

Science.gov (United States)

Vernetti, Angélina; Smith, Tim J; Senju, Atsushi

2017-03-15

While numerous studies have demonstrated that infants and adults preferentially orient to social stimuli, it remains unclear as to what drives such preferential orienting. It has been suggested that the learned association between social cues and subsequent reward delivery might shape such social orienting. Using a novel, spontaneous indication of reinforcement learning (with the use of a gaze contingent reward-learning task), we investigated whether children and adults' orienting towards social and non-social visual cues can be elicited by the association between participants' visual attention and a rewarding outcome. Critically, we assessed whether the engaging nature of the social cues influences the process of reinforcement learning. Both children and adults learned to orient more often to the visual cues associated with reward delivery, demonstrating that cue-reward association reinforced visual orienting. More importantly, when the reward-predictive cue was social and engaging, both children and adults learned the cue-reward association faster and more efficiently than when the reward-predictive cue was social but non-engaging. These new findings indicate that social engaging cues have a positive incentive value. This could possibly be because they usually coincide with positive outcomes in real life, which could partly drive the development of social orienting. © 2017 The Authors.

Study on reinforced concrete beams with helical transverse reinforcement

Science.gov (United States)

Kaarthik Krishna, N.; Sandeep, S.; Mini, K. M.

2018-02-01

In a Reinforced Concrete (R.C) structure, major reinforcement is used for taking up tensile stresses acting on the structure due to applied loading. The present paper reports the behavior of reinforced concrete beams with helical reinforcement (transverse reinforcement) subjected to monotonous loading by 3-point flexure test. The results were compared with identically similar reinforced concrete beams with rectangular stirrups. During the test crack evolution, load carrying capacity and deflection of the beams were monitored, analyzed and compared. Test results indicate that the use of helical reinforcement provides enhanced load carrying capacity and a lower deflection proving to be more ductile, clearly indicating the advantage in carrying horizontal loads. An analysis was also carried out using ANSYS software in order to compare the test results of both the beams.

Homeostatic scaling of excitability in recurrent neural networks.

NARCIS (Netherlands)

Remme, M.W.H.; Wadman, W.J.

2012-01-01

Neurons adjust their intrinsic excitability when experiencing a persistent change in synaptic drive. This process can prevent neural activity from moving into either a quiescent state or a saturated state in the face of ongoing plasticity, and is thought to promote stability of the network in which

The Neural Foundations of Reaction and Action in Aversive Motivation.

Science.gov (United States)

Campese, Vincent D; Sears, Robert M; Moscarello, Justin M; Diaz-Mataix, Lorenzo; Cain, Christopher K; LeDoux, Joseph E

2016-01-01

Much of the early research in aversive learning concerned motivation and reinforcement in avoidance conditioning and related paradigms. When the field transitioned toward the focus on Pavlovian threat conditioning in isolation, this paved the way for the clear understanding of the psychological principles and neural and molecular mechanisms responsible for this type of learning and memory that has unfolded over recent decades. Currently, avoidance conditioning is being revisited, and with what has been learned about associative aversive learning, rapid progress is being made. We review, below, the literature on the neural substrates critical for learning in instrumental active avoidance tasks and conditioned aversive motivation.

A Neural Network Approach to Fluid Level Measurement in Dynamic Environments Using a Single Capacitive Sensor

Directory of Open Access Journals (Sweden)

Edin TERZIC

2010-03-01

Full Text Available A measurement system has been developed using a single tube capacitive sensor to accurately determine the fluid level in vehicular fuel tanks. A novel approach based on artificial neural networks based signal pre-processing and classification has been described in this article. A broad investigation on the Backpropagation neural network and some selected signal pre-processing filters, namely, Moving Mean, Moving Median, and Wavelet Filter has also been presented. An on field drive trial was conducted under normal driving conditions at various fuel volumes ranging from 5 L to 50 L to acquire training samples from the capacitive sensor. A second field trial was conducted to obtain test samples to verify the performance of the neural network. The neural network was trained and verified with 50 % of the training and test samples. The results obtained using the neural network approach having different filtration methods are compared with the results obtained using simple Moving Mean and Moving Median functions. It is demonstrated that the Backpropagation neural network with Moving Median filter produced the most accurate outcome compared with the other signal filtration methods.

Driving profile modeling and recognition based on soft computing approach.

Science.gov (United States)

Wahab, Abdul; Quek, Chai; Tan, Chin Keong; Takeda, Kazuya

2009-04-01

Advancements in biometrics-based authentication have led to its increasing prominence and are being incorporated into everyday tasks. Existing vehicle security systems rely only on alarms or smart card as forms of protection. A biometric driver recognition system utilizing driving behaviors is a highly novel and personalized approach and could be incorporated into existing vehicle security system to form a multimodal identification system and offer a greater degree of multilevel protection. In this paper, detailed studies have been conducted to model individual driving behavior in order to identify features that may be efficiently and effectively used to profile each driver. Feature extraction techniques based on Gaussian mixture models (GMMs) are proposed and implemented. Features extracted from the accelerator and brake pedal pressure were then used as inputs to a fuzzy neural network (FNN) system to ascertain the identity of the driver. Two fuzzy neural networks, namely, the evolving fuzzy neural network (EFuNN) and the adaptive network-based fuzzy inference system (ANFIS), are used to demonstrate the viability of the two proposed feature extraction techniques. The performances were compared against an artificial neural network (NN) implementation using the multilayer perceptron (MLP) network and a statistical method based on the GMM. Extensive testing was conducted and the results show great potential in the use of the FNN for real-time driver identification and verification. In addition, the profiling of driver behaviors has numerous other potential applications for use by law enforcement and companies dealing with buses and truck drivers.

REINFORCED COMPOSITE PANEL

DEFF Research Database (Denmark)

2003-01-01

A composite panel having front and back faces, the panel comprising facing reinforcement, backing reinforcement and matrix material binding to the facing and backing reinforcements, the facing and backing reinforcements each independently comprising one or more reinforcing sheets, the facing rein...... by matrix material, the facing and backing reinforcements being interconnected to resist out-of-plane relative movement. The reinforced composite panel is useful as a barrier element for shielding structures, equipment and personnel from blast and/or ballistic impact damage....

Model-Based Fault Diagnosis in Electric Drive Inverters Using Artificial Neural Network

National Research Council Canada - National Science Library

Masrur, Abul; Chen, ZhiHang; Zhang, Baifang; Jia, Hongbin; Murphey, Yi-Lu

2006-01-01

.... A normal model and various faulted models of the inverter-motor combination were developed, and voltages and current signals were generated from those models to train an artificial neural network for fault diagnosis...

Cracking the Neural Code for Sensory Perception by Combining Statistics, Intervention, and Behavior.

Science.gov (United States)

Panzeri, Stefano; Harvey, Christopher D; Piasini, Eugenio; Latham, Peter E; Fellin, Tommaso

2017-02-08

The two basic processes underlying perceptual decisions-how neural responses encode stimuli, and how they inform behavioral choices-have mainly been studied separately. Thus, although many spatiotemporal features of neural population activity, or "neural codes," have been shown to carry sensory information, it is often unknown whether the brain uses these features for perception. To address this issue, we propose a new framework centered on redefining the neural code as the neural features that carry sensory information used by the animal to drive appropriate behavior; that is, the features that have an intersection between sensory and choice information. We show how this framework leads to a new statistical analysis of neural activity recorded during behavior that can identify such neural codes, and we discuss how to combine intersection-based analysis of neural recordings with intervention on neural activity to determine definitively whether specific neural activity features are involved in a task. Copyright © 2017 Elsevier Inc. All rights reserved.

Brain Electrodynamic and Hemodynamic Signatures Against Fatigue During Driving

Directory of Open Access Journals (Sweden)

Chun-Hsiang Chuang

2018-03-01

Full Text Available Fatigue is likely to be gradually cumulated in a prolonged and attention-demanding task that may adversely affect task performance. To address the brain dynamics during a driving task, this study recruited 16 subjects to participate in an event-related lane-departure driving experiment. Each subject was instructed to maintain attention and task performance throughout an hour-long driving experiment. The subjects' brain electrodynamics and hemodynamics were simultaneously recorded via 32-channel electroencephalography (EEG and 8-source/16-detector functional near-infrared spectroscopy (fNIRS. The behavior performance demonstrated that all subjects were able to promptly respond to lane-deviation events, even if the sign of fatigue arose in the brain, which suggests that the subjects were fighting fatigue during the driving experiment. The EEG event-related analysis showed strengthening alpha suppression in the occipital cortex, a common brain region of fatigue. Furthermore, we noted increasing oxygenated hemoglobin (HbO of the brain to fight driving fatigue in the frontal cortex, primary motor cortex, parieto-occipital cortex and supplementary motor area. In conclusion, the increasing neural activity and cortical activations were aimed at maintaining driving performance when fatigue emerged. The electrodynamic and hemodynamic signatures of fatigue fighting contribute to our understanding of the brain dynamics of driving fatigue and address driving safety issues through the maintenance of attention and behavioral performance.

Adolescent development of context-dependent stimulus-reward association memory and its neural correlates.

Science.gov (United States)

Voss, Joel L; O'Neil, Jonathan T; Kharitonova, Maria; Briggs-Gowan, Margaret J; Wakschlag, Lauren S

2015-01-01

Expression of learned stimulus-reward associations based on context is essential for regulation of behavior to meet situational demands. Contextual regulation improves during development, although the developmental progression of relevant neural and cognitive processes is not fully specified. We therefore measured neural correlates of flexible, contextual expression of stimulus-reward associations in pre/early-adolescent children (ages 9-13 years) and young adults (ages 19-22 years). After reinforcement learning using standard parameters, a contextual reversal manipulation was used whereby contextual cues indicated that stimulus-reward associations were the same as previously reinforced for some trials (consistent trials) or were reversed on other trials (inconsistent trials). Subjects were thus required to respond according to original stimulus-reward associations vs. reversed associations based on trial-specific contextual cues. Children and young adults did not differ in reinforcement learning or in relevant functional magnetic resonance imaging (fMRI) correlates. In contrast, adults outperformed children during contextual reversal, with better performance specifically for inconsistent trials. fMRI signals corresponding to this selective advantage included greater activity in lateral prefrontal cortex (LPFC), hippocampus, and dorsal striatum for young adults relative to children. Flexible expression of stimulus-reward associations based on context thus improves via adolescent development, as does recruitment of brain regions involved in reward learning and contextual expression of memory. HighlightsEarly-adolescent children and young adults were equivalent in reinforcement learning.Adults outperformed children in contextual expression of stimulus-reward associations.Adult advantages correlated with increased activity of relevant brain regions.Specific neurocognitive developmental changes support better contextual regulation.

Reinforcement Magnitude: An Evaluation of Preference and Reinforcer Efficacy

OpenAIRE

Trosclair-Lasserre, Nicole M; Lerman, Dorothea C; Call, Nathan A; Addison, Laura R; Kodak, Tiffany

2008-01-01

Consideration of reinforcer magnitude may be important for maximizing the efficacy of treatment for problem behavior. Nonetheless, relatively little is known about children's preferences for different magnitudes of social reinforcement or the extent to which preference is related to differences in reinforcer efficacy. The purpose of the current study was to evaluate the relations among reinforcer magnitude, preference, and efficacy by drawing on the procedures and results of basic experimenta...

Modeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning.

Science.gov (United States)

Mkrtchian, Anahit; Aylward, Jessica; Dayan, Peter; Roiser, Jonathan P; Robinson, Oliver J

2017-10-01

Serious and debilitating symptoms of anxiety are the most common mental health problem worldwide, accounting for around 5% of all adult years lived with disability in the developed world. Avoidance behavior-avoiding social situations for fear of embarrassment, for instance-is a core feature of such anxiety. However, as for many other psychiatric symptoms the biological mechanisms underlying avoidance remain unclear. Reinforcement learning models provide formal and testable characterizations of the mechanisms of decision making; here, we examine avoidance in these terms. A total of 101 healthy participants and individuals with mood and anxiety disorders completed an approach-avoidance go/no-go task under stress induced by threat of unpredictable shock. We show an increased reliance in the mood and anxiety group on a parameter of our reinforcement learning model that characterizes a prepotent (pavlovian) bias to withhold responding in the face of negative outcomes. This was particularly the case when the mood and anxiety group was under stress. This formal description of avoidance within the reinforcement learning framework provides a new means of linking clinical symptoms with biophysically plausible models of neural circuitry and, as such, takes us closer to a mechanistic understanding of mood and anxiety disorders. Copyright © 2017 Society of Biological Psychiatry. Published by Elsevier Inc. All rights reserved.

On the Drive Specificity of Freudian Drives for the Generation of SEEKING Activities: The Importance of the Underestimated Imperative Motor Factor

Directory of Open Access Journals (Sweden)

Michael Kirsch

2018-05-01

Full Text Available Doubters of Freud’s theory of drives frequently mentioned that his approach is outdated and therefore cannot be useful for solving current problems in patients with mental disorders. At present, many scientists believe that affects rather than drives are of utmost importance for the emotional life and the theoretical framework of affective neuroscience, developed by Panksepp, strongly underpinned this view. Panksepp evaluated seven so-called command systems and the SEEKING system is therein of central importance. Panksepp used Pankseppian drives as inputs for the SEEKING system but noted the missing explanation of drive-specific generation of SEEKING activities in his description. Drive specificity requires dual action of the drive: the activation of a drive-specific brain area and the release of the neurotransmitter dopamine. Noticeably, as Freud claimed drive specificity too, it was here analyzed whether a Freudian drive can evoke the generation of drive-specific SEEKING activities. Special importance was addressed to the imperative motor factor in Freud’s drive theory because Panksepp’s formulations focused on neural pathways without specifying underlying neurotransmitter/endocrine factors impelling motor activity. As Panksepp claimed sleep as a Pankseppian drive, we firstly had to classified sleep as a Freudian drive by using three evaluated criteria for a Freudian drive. After that it was possible to identify the imperative motor factors of hunger, thirst, sex, and sleep. Most importantly, all of these imperative motor factors can both activate a drive-specific brain area and release dopamine from dopaminergic neurons, i.e., they can achieve the so-called drive specificity. Surprisingly, an impaired Freudian drive can alter via endocrinological pathways the concentration of the imperative motor factor of a second Freudian drive, obviously in some independence to the level of the metabolic deficit, thereby offering the possibility to

Central neural pathways for thermoregulation

Science.gov (United States)

Morrison, Shaun F.; Nakamura, Kazuhiro

2010-01-01

Central neural circuits orchestrate a homeostatic repertoire to maintain body temperature during environmental temperature challenges and to alter body temperature during the inflammatory response. This review summarizes the functional organization of the neural pathways through which cutaneous thermal receptors alter thermoregulatory effectors: the cutaneous circulation for heat loss, the brown adipose tissue, skeletal muscle and heart for thermogenesis and species-dependent mechanisms (sweating, panting and saliva spreading) for evaporative heat loss. These effectors are regulated by parallel but distinct, effector-specific neural pathways that share a common peripheral thermal sensory input. The thermal afferent circuits include cutaneous thermal receptors, spinal dorsal horn neurons and lateral parabrachial nucleus neurons projecting to the preoptic area to influence warm-sensitive, inhibitory output neurons which control thermogenesis-promoting neurons in the dorsomedial hypothalamus that project to premotor neurons in the rostral ventromedial medulla, including the raphe pallidus, that descend to provide the excitation necessary to drive thermogenic thermal effectors. A distinct population of warm-sensitive preoptic neurons controls heat loss through an inhibitory input to raphe pallidus neurons controlling cutaneous vasoconstriction. PMID:21196160

RM-SORN: a reward-modulated self-organizing recurrent neural network.

Science.gov (United States)

Aswolinskiy, Witali; Pipa, Gordon

2015-01-01

Neural plasticity plays an important role in learning and memory. Reward-modulation of plasticity offers an explanation for the ability of the brain to adapt its neural activity to achieve a rewarded goal. Here, we define a neural network model that learns through the interaction of Intrinsic Plasticity (IP) and reward-modulated Spike-Timing-Dependent Plasticity (STDP). IP enables the network to explore possible output sequences and STDP, modulated by reward, reinforces the creation of the rewarded output sequences. The model is tested on tasks for prediction, recall, non-linear computation, pattern recognition, and sequence generation. It achieves performance comparable to networks trained with supervised learning, while using simple, biologically motivated plasticity rules, and rewarding strategies. The results confirm the importance of investigating the interaction of several plasticity rules in the context of reward-modulated learning and whether reward-modulated self-organization can explain the amazing capabilities of the brain.

Integrating neural network technology and noise analysis

International Nuclear Information System (INIS)

Uhrig, R.E.; Oak Ridge National Lab., TN

1995-01-01

The integrated use of neural network and noise analysis technologies offers advantages not available by the use of either technology alone. The application of neural network technology to noise analysis offers an opportunity to expand the scope of problems where noise analysis is useful and unique ways in which the integration of these technologies can be used productively. The two-sensor technique, in which the responses of two sensors to an unknown driving source are related, is used to demonstration such integration. The relationship between power spectral densities (PSDs) of accelerometer signals is derived theoretically using noise analysis to demonstrate its uniqueness. This relationship is modeled from experimental data using a neural network when the system is working properly, and the actual PSD of one sensor is compared with the PSD of that sensor predicted by the neural network using the PSD of the other sensor as an input. A significant deviation between the actual and predicted PSDs indicate that system is changing (i.e., failing). Experiments carried out on check values and bearings illustrate the usefulness of the methodology developed. (Author)

Nonequilibrium landscape theory of neural networks

Science.gov (United States)

Yan, Han; Zhao, Lei; Hu, Liang; Wang, Xidi; Wang, Erkang; Wang, Jin

2013-01-01

The brain map project aims to map out the neuron connections of the human brain. Even with all of the wirings mapped out, the global and physical understandings of the function and behavior are still challenging. Hopfield quantified the learning and memory process of symmetrically connected neural networks globally through equilibrium energy. The energy basins of attractions represent memories, and the memory retrieval dynamics is determined by the energy gradient. However, the realistic neural networks are asymmetrically connected, and oscillations cannot emerge from symmetric neural networks. Here, we developed a nonequilibrium landscape–flux theory for realistic asymmetrically connected neural networks. We uncovered the underlying potential landscape and the associated Lyapunov function for quantifying the global stability and function. We found the dynamics and oscillations in human brains responsible for cognitive processes and physiological rhythm regulations are determined not only by the landscape gradient but also by the flux. We found that the flux is closely related to the degrees of the asymmetric connections in neural networks and is the origin of the neural oscillations. The neural oscillation landscape shows a closed-ring attractor topology. The landscape gradient attracts the network down to the ring. The flux is responsible for coherent oscillations on the ring. We suggest the flux may provide the driving force for associations among memories. We applied our theory to rapid-eye movement sleep cycle. We identified the key regulation factors for function through global sensitivity analysis of landscape topography against wirings, which are in good agreements with experiments. PMID:24145451

Nonequilibrium landscape theory of neural networks.

Science.gov (United States)

Yan, Han; Zhao, Lei; Hu, Liang; Wang, Xidi; Wang, Erkang; Wang, Jin

2013-11-05

The brain map project aims to map out the neuron connections of the human brain. Even with all of the wirings mapped out, the global and physical understandings of the function and behavior are still challenging. Hopfield quantified the learning and memory process of symmetrically connected neural networks globally through equilibrium energy. The energy basins of attractions represent memories, and the memory retrieval dynamics is determined by the energy gradient. However, the realistic neural networks are asymmetrically connected, and oscillations cannot emerge from symmetric neural networks. Here, we developed a nonequilibrium landscape-flux theory for realistic asymmetrically connected neural networks. We uncovered the underlying potential landscape and the associated Lyapunov function for quantifying the global stability and function. We found the dynamics and oscillations in human brains responsible for cognitive processes and physiological rhythm regulations are determined not only by the landscape gradient but also by the flux. We found that the flux is closely related to the degrees of the asymmetric connections in neural networks and is the origin of the neural oscillations. The neural oscillation landscape shows a closed-ring attractor topology. The landscape gradient attracts the network down to the ring. The flux is responsible for coherent oscillations on the ring. We suggest the flux may provide the driving force for associations among memories. We applied our theory to rapid-eye movement sleep cycle. We identified the key regulation factors for function through global sensitivity analysis of landscape topography against wirings, which are in good agreements with experiments.

Reinforced sulphur concrete

NARCIS (Netherlands)

2014-01-01

Reinforced sulphur concrete wherein one or more metal reinforcing members are in contact with sulphur concrete is disclosed. The reinforced sulphur concrete comprises an adhesion promoter that enhances the interaction between the sulphur and the one or more metal reinforcing members.

Pinning synchronization of memristor-based neural networks with time-varying delays.

Science.gov (United States)

Yang, Zhanyu; Luo, Biao; Liu, Derong; Li, Yueheng

2017-09-01

In this paper, the synchronization of memristor-based neural networks with time-varying delays via pinning control is investigated. A novel pinning method is introduced to synchronize two memristor-based neural networks which denote drive system and response system, respectively. The dynamics are studied by theories of differential inclusions and nonsmooth analysis. In addition, some sufficient conditions are derived to guarantee asymptotic synchronization and exponential synchronization of memristor-based neural networks via the presented pinning control. Furthermore, some improvements about the proposed control method are also discussed in this paper. Finally, the effectiveness of the obtained results is demonstrated by numerical simulations. Copyright © 2017 Elsevier Ltd. All rights reserved.

CMOS On-Chip Optoelectronic Neural Interface Device with Integrated Light Source for Optogenetics

International Nuclear Information System (INIS)

Sawadsaringkarn, Y; Kimura, H; Maezawa, Y; Nakajima, A; Kobayashi, T; Sasagawa, K; Noda, T; Tokuda, T; Ohta, J

2012-01-01

A novel optoelectronic neural interface device is proposed for target applications in optogenetics for neural science. The device consists of a light emitting diode (LED) array implemented on a CMOS image sensor for on-chip local light stimulation. In this study, we designed a suitable CMOS image sensor equipped with on-chip electrodes to drive the LEDs, and developed a device structure and packaging process for LED integration. The prototype device produced an illumination intensity of approximately 1 mW with a driving current of 2.0 mA, which is expected to be sufficient to activate channelrhodopsin (ChR2). We also demonstrated the functions of light stimulation and on-chip imaging using a brain slice from a mouse as a target sample.

Risk-taking on the road and in the mind: behavioural and neural patterns of decision making between risky and safe drivers.

Science.gov (United States)

Ba, Yutao; Zhang, Wei; Peng, QiJia; Salvendy, Gavriel; Crundall, David

2016-01-01

Drivers' risk-taking is a key issue of road safety. This study explored individual differences in drivers' decision-making, linking external behaviours to internal neural activity, to reveal the cognitive mechanisms of risky driving. Twenty-four male drivers were split into two groups (risky vs. safe drivers) via the Drivier Behaviour Questionnaire-violation. The risky drivers demonstrated higher preference for the risky choices in the paradigms of Iowa Gambling Task and Balloon Analogue Risk Task. More importantly, the risky drivers showed lower amplitudes of feedback-related negativity (FRN) and loss-minus-gain FRN in both paradigms, which indicated their neural processing of error-detection. A significant difference of P300 amplitudes was also reported between groups, which indicated their neural processing of reward-evaluation and were modified by specific paradigm and feedback. These results suggested that the neural basis of risky driving was the decision patterns less revised by losses and more motivated by rewards. Risk-taking on the road is largely determined by inherent cognitive mechanisms, which can be indicated by the behavioural and neural patterns of decision-making. In this regard, it is feasible to quantize drivers’ riskiness in the cognitive stage before actual risky driving or accidents, and intervene accordingly.

Sensorless FOC Performance Improved with On-Line Speed and Rotor Resistance Estimator Based on an Artificial Neural Network for an Induction Motor Drive

Directory of Open Access Journals (Sweden)

Jose. M. Gutierrez-Villalobos

2015-06-01

Full Text Available Three-phase induction motor drive requires high accuracy in high performance processes in industrial applications. Field oriented control, which is one of the most employed control schemes for induction motors, bases its function on the electrical parameter estimation coming from the motor. These parameters make an electrical machine driver work improperly, since these electrical parameter values change at low speeds, temperature changes, and especially with load and duty changes. The focus of this paper is the real-time and on-line electrical parameters with a CMAC-ADALINE block added in the standard FOC scheme to improve the IM driver performance and endure the driver and the induction motor lifetime. Two kinds of neural network structures are used; one to estimate rotor speed and the other one to estimate rotor resistance of an induction motor.

Reinforcement Magnitude: An Evaluation of Preference and Reinforcer Efficacy

Science.gov (United States)

Trosclair-Lasserre, Nicole M.; Lerman, Dorothea C.; Call, Nathan A.; Addison, Laura R.; Kodak, Tiffany

2008-01-01

Consideration of reinforcer magnitude may be important for maximizing the efficacy of treatment for problem behavior. Nonetheless, relatively little is known about children's preferences for different magnitudes of social reinforcement or the extent to which preference is related to differences in reinforcer efficacy. The purpose of the current…

Autoshaping Chicks with Heat Reinforcement: The Role of Stimulus-Reinforcer and Response-Reinforcer Relations

Science.gov (United States)

Wasserman, Edward A.; And Others

1975-01-01

The present series of experiments attempted to analyze more fully the contributions of stimulus-reinforcer and response-reinforcer relations to autoshaping within a single conditioning situation. (Author)

Deep learning in neural networks: an overview.

Science.gov (United States)

Schmidhuber, Jürgen

2015-01-01

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarizes relevant work, much of it from the previous millennium. Shallow and Deep Learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.

Sensation and perception of sucrose and fat stimuli predict the reinforcing value of food.

Science.gov (United States)

Panek-Scarborough, Leah M; Dewey, Amber M; Temple, Jennifer L

2012-03-20

Chronic overeating can lead to weight gain and obesity. Sensory system function may play a role in the types of foods people select and the amount of food people eat. Several studies have shown that the orosensory components of eating play a strong role in driving food intake and food selection. In addition, previous work has shown that motivation to get food, or the reinforcing value of food, is a predictor of energy intake. The purpose of this study was to test the hypothesis that higher detection thresholds and lower suprathreshold intensity ratings of sweet and fat stimuli are associated with greater reinforcing value of food. In addition, we sought to determine if the sensory ratings of the stimuli would differ depending on whether they were expectorated or swallowed. The reinforcing value of food was measured by having participants perform operant responses for food on progressive ratio schedules of reinforcement. Taste detection thresholds and suprathresholds for solutions containing varied concentrations of sucrose and fat were also measured in two different Experiments. In Experiment 1, we found that sucrose, but not fat, detection predicted the reinforcing value of food with the reinforcing value of food increasing as sucrose detection threshold increased (indicating poorer detection). In Experiment 2, we found that lower suprathreshold ratings of expectorated fat and sucrose predicted greater reinforcing value of food. In addition, higher detection thresholds for fat stimuli (indicating poorer detection) were associated with greater reinforcing value of food. When taken together, these studies suggest that there is a relationship between taste detection and perception and reinforcing value of food and that these relationships vary based on whether the stimulus is swallowed or expectorated. Copyright © 2012 Elsevier Inc. All rights reserved.

Investigation of Drive-Reinforcement Learning and Application of Learning to Flight Control

Science.gov (United States)

1993-08-01

WL-TR-93-1153 INVESTIGATION OF DRIVE-REINFORCEMEN% LEARNING AND APPLICATION OF LEARNING TO FLIGHT CONTROL AD-A277 442 WALTER L. BAKER (ED), STEPHEN ...OF LEARNING TO FUIGHT CONTROL PE 62204 ___ ___ ___ ___ __ ___ ___ ___ ___ ___ ___ __ PR 2003 6. AUTHOR(S) TA 05 WALTER L. BAKER (ED), STEPHEN C. ATKINS...34 Computers and Thought, E. A. Freigenbaum and J. Feldman (eds.), Mc- Graw Hill, New York, (1959). [19] Holland, J. H., "Escaping Brittleness: The Possibility

Genomic Signatures of Reinforcement

Directory of Open Access Journals (Sweden)

Austin G. Garner

2018-04-01

Full Text Available Reinforcement is the process by which selection against hybridization increases reproductive isolation between taxa. Much research has focused on demonstrating the existence of reinforcement, yet relatively little is known about the genetic basis of reinforcement or the evolutionary conditions under which reinforcement can occur. Inspired by reinforcement’s characteristic phenotypic pattern of reproductive trait divergence in sympatry but not in allopatry, we discuss whether reinforcement also leaves a distinct genomic pattern. First, we describe three patterns of genetic variation we expect as a consequence of reinforcement. Then, we discuss a set of alternative processes and complicating factors that may make the identification of reinforcement at the genomic level difficult. Finally, we consider how genomic analyses can be leveraged to inform if and to what extent reinforcement evolved in the face of gene flow between sympatric lineages and between allopatric and sympatric populations of the same lineage. Our major goals are to understand if genome scans for particular patterns of genetic variation could identify reinforcement, isolate the genetic basis of reinforcement, or infer the conditions under which reinforcement evolved.

Genomic Signatures of Reinforcement

Science.gov (United States)

Goulet, Benjamin E.

2018-01-01

Reinforcement is the process by which selection against hybridization increases reproductive isolation between taxa. Much research has focused on demonstrating the existence of reinforcement, yet relatively little is known about the genetic basis of reinforcement or the evolutionary conditions under which reinforcement can occur. Inspired by reinforcement’s characteristic phenotypic pattern of reproductive trait divergence in sympatry but not in allopatry, we discuss whether reinforcement also leaves a distinct genomic pattern. First, we describe three patterns of genetic variation we expect as a consequence of reinforcement. Then, we discuss a set of alternative processes and complicating factors that may make the identification of reinforcement at the genomic level difficult. Finally, we consider how genomic analyses can be leveraged to inform if and to what extent reinforcement evolved in the face of gene flow between sympatric lineages and between allopatric and sympatric populations of the same lineage. Our major goals are to understand if genome scans for particular patterns of genetic variation could identify reinforcement, isolate the genetic basis of reinforcement, or infer the conditions under which reinforcement evolved. PMID:29614048

Synchronization of chaotic neural networks via output or state coupling

International Nuclear Information System (INIS)

Lu Hongtao; Leeuwen, C. van

2006-01-01

We consider the problem of global exponential synchronization between two identical chaotic neural networks that are linearly and unidirectionally coupled. We formulate a general framework for the synchronization problem in which one chaotic neural network, working as the driving system (or master), sends its output or state values to the other, which serves as the response system (or slave). We use Lyapunov functions to establish general theoretical conditions for designing the coupling matrix. Neither symmetry nor negative (positive) definiteness of the coupling matrix are required; under less restrictive conditions, the two coupled chaotic neural networks can achieve global exponential synchronization regardless of their initial states. Detailed comparisons with existing results are made and numerical simulations are carried out to demonstrate the effectiveness of the established synchronization laws

Modeling the behavioral substrates of associate learning and memory - Adaptive neural models

Science.gov (United States)

Lee, Chuen-Chien

1991-01-01

Three adaptive single-neuron models based on neural analogies of behavior modification episodes are proposed, which attempt to bridge the gap between psychology and neurophysiology. The proposed models capture the predictive nature of Pavlovian conditioning, which is essential to the theory of adaptive/learning systems. The models learn to anticipate the occurrence of a conditioned response before the presence of a reinforcing stimulus when training is complete. Furthermore, each model can find the most nonredundant and earliest predictor of reinforcement. The behavior of the models accounts for several aspects of basic animal learning phenomena in Pavlovian conditioning beyond previous related models. Computer simulations show how well the models fit empirical data from various animal learning paradigms.

Accessing and constructing driving data to develop fuel consumption forecast model

Science.gov (United States)

Yamashita, Rei-Jo; Yao, Hsiu-Hsen; Hung, Shih-Wei; Hackman, Acquah

2018-02-01

In this study, we develop a forecasting models, to estimate fuel consumption based on the driving behavior, in which vehicles and routes are known. First, the driving data are collected via telematics and OBDII. Then, the driving fuel consumption formula is used to calculate the estimate fuel consumption, and driving behavior indicators are generated for analysis. Based on statistical analysis method, the driving fuel consumption forecasting model is constructed. Some field experiment results were done in this study to generate hundreds of driving behavior indicators. Based on data mining approach, the Pearson coefficient correlation analysis is used to filter highly fuel consumption related DBIs. Only highly correlated DBI will be used in the model. These DBIs are divided into four classes: speed class, acceleration class, Left/Right/U-turn class and the other category. We then use K-means cluster analysis to group to the driver class and the route class. Finally, more than 12 aggregate models are generated by those highly correlated DBIs, using the neural network model and regression analysis. Based on Mean Absolute Percentage Error (MAPE) to evaluate from the developed AMs. The best MAPE values among these AM is below 5%.

Structural Behavior of Concrete Beams Reinforced with Basalt Fiber Reinforced Polymer (BFRP) Bars

Science.gov (United States)

Ovitigala, Thilan

The main challenge for civil engineers is to provide sustainable, environmentally friendly and financially feasible structures to the society. Finding new materials such as fiber reinforced polymer (FRP) material that can fulfill the above requirements is a must. FRP material was expensive and it was limited to niche markets such as space shuttles and air industry in the 1960s. Over the time, it became cheaper and spread to other industries such as sporting goods in the 1980-1990, and then towards the infrastructure industry. Design and construction guidelines are available for carbon fiber reinforced polymer (CFRP), aramid fiber reinforced polymer (AFRP) and glass fiber reinforced polymer (GFRP) and they are currently used in structural applications. Since FRP is linear elastic brittle material, design guidelines for the steel reinforcement are not valid for FRP materials. Corrosion of steel reinforcement affects the durability of the concrete structures. FRP reinforcement is identified as an alternative to steel reinforcement in corrosive environments. Although basalt fiber reinforced polymer (BFRP) has many advantages over other FRP materials, but limited studies have been done. These studies didn't include larger BFRP bar diameters that are mostly used in practice. Therefore, larger beam sizes with larger BFRP reinforcement bar diameters are needed to investigate the flexural and shear behavior of BFRP reinforced concrete beams. Also, shear behavior of BFRP reinforced concrete beams was not yet studied. Experimental testing of mechanical properties and bond strength of BFRP bars and flexural and shear behavior of BFRP reinforced concrete beams are needed to include BFRP reinforcement bars in the design codes. This study mainly focuses on the use of BFRP bars as internal reinforcement. The test results of the mechanical properties of BFRP reinforcement bars, the bond strength of BFRP reinforcement bars, and the flexural and shear behavior of concrete beams

Upset Prediction in Friction Welding Using Radial Basis Function Neural Network

Directory of Open Access Journals (Sweden)

Wei Liu

2013-01-01

Full Text Available This paper addresses the upset prediction problem of friction welded joints. Based on finite element simulations of inertia friction welding (IFW, a radial basis function (RBF neural network was developed initially to predict the final upset for a number of welding parameters. The predicted joint upset by the RBF neural network was compared to validated finite element simulations, producing an error of less than 8.16% which is reasonable. Furthermore, the effects of initial rotational speed and axial pressure on the upset were investigated in relation to energy conversion with the RBF neural network. The developed RBF neural network was also applied to linear friction welding (LFW and continuous drive friction welding (CDFW. The correlation coefficients of RBF prediction for LFW and CDFW were 0.963 and 0.998, respectively, which further suggest that an RBF neural network is an effective method for upset prediction of friction welded joints.

Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making.

Science.gov (United States)

Schönberg, Tom; Daw, Nathaniel D; Joel, Daphna; O'Doherty, John P

2007-11-21

The computational framework of reinforcement learning has been used to forward our understanding of the neural mechanisms underlying reward learning and decision-making behavior. It is known that humans vary widely in their performance in decision-making tasks. Here, we used a simple four-armed bandit task in which subjects are almost evenly split into two groups on the basis of their performance: those who do learn to favor choice of the optimal action and those who do not. Using models of reinforcement learning we sought to determine the neural basis of these intrinsic differences in performance by scanning both groups with functional magnetic resonance imaging. We scanned 29 subjects while they performed the reward-based decision-making task. Our results suggest that these two groups differ markedly in the degree to which reinforcement learning signals in the striatum are engaged during task performance. While the learners showed robust prediction error signals in both the ventral and dorsal striatum during learning, the nonlearner group showed a marked absence of such signals. Moreover, the magnitude of prediction error signals in a region of dorsal striatum correlated significantly with a measure of behavioral performance across all subjects. These findings support a crucial role of prediction error signals, likely originating from dopaminergic midbrain neurons, in enabling learning of action selection preferences on the basis of obtained rewards. Thus, spontaneously observed individual differences in decision making performance demonstrate the suggested dependence of this type of learning on the functional integrity of the dopaminergic striatal system in humans.

Distracted driving in elderly and middle-aged drivers.

Science.gov (United States)

Thompson, Kelsey R; Johnson, Amy M; Emerson, Jamie L; Dawson, Jeffrey D; Boer, Erwin R; Rizzo, Matthew

2012-03-01

Automobile driving is a safety-critical real-world example of multitasking. A variety of roadway and in-vehicle distracter tasks create information processing loads that compete for the neural resources needed to drive safely. Drivers with mind and brain aging may be particularly susceptible to distraction due to waning cognitive resources and control over attention. This study examined distracted driving performance in an instrumented vehicle (IV) in 86 elderly (mean=72.5 years, SD=5.0 years) and 51 middle-aged drivers (mean=53.7 years, SD=9.3 year) under a concurrent auditory-verbal processing load created by the Paced Auditory Serial Addition Task (PASAT). Compared to baseline (no-task) driving performance, distraction was associated with reduced steering control in both groups, with middle-aged drivers showing a greater increase in steering variability. The elderly drove slower and showed decreased speed variability during distraction compared to middle-aged drivers. They also tended to "freeze up", spending significantly more time holding the gas pedal steady, another tactic that may mitigate time pressured integration and control of information, thereby freeing mental resources to maintain situation awareness. While 39% of elderly and 43% of middle-aged drivers committed significantly more driving safety errors during distraction, 28% and 18%, respectively, actually improved, compatible with allocation of attention resources to safety critical tasks under a cognitive load. Copyright © 2011 Elsevier Ltd. All rights reserved.

Reinforcement shapes clines in female mate discrimination in Drosophila subquinaria

Science.gov (United States)

Bewick, Emily R.; Dyer, Kelly A.

2014-01-01

Reinforcement of species boundaries may alter mate recognition in a way that also affects patterns of mate preference among conspecific populations. In the fly Drosophila subquinaria, females sympatric with the closely related species D. recens reject mating with heterospecific males as well as with conspecific males from allopatric populations. Here, we assess geographic variation in behavioral isolation within and among populations of D. subquinaria and use cline theory to understand patterns of selection on reinforced discrimination and its consequences for sexual isolation within species. We find that selection has fixed rejection of D. recens males in sympatry, while significant genetic variation in this behavior occurs within allopatric populations. In conspecific matings sexual isolation is also asymmetric and stronger in populations that are sympatric with D. recens. The clines in behavioral discrimination within and between species are similar in shape and are maintained by strong selection in the face of gene flow, and we show that some of their genetic basis may be either shared or linked. Thus, while reinforcement can drive extremely strong phenotypic divergence, the long-term consequences for incipient speciation depend on gene flow, genetic linkage of discrimination traits, and the cost of these behaviors in allopatry. PMID:25163510

Presentations of the PTAC driving safety technology forum

Energy Technology Data Exchange (ETDEWEB)

NONE

2005-07-01

Issues concerning road safety and driver training in relation to the petroleum industry were the focus of this conference. The results of a driving safety survey from the Petroleum Technology Alliance Canada were presented. High risk driver identification methods were discussed along with coaching case studies, including details of continuous and advanced driver training by various industry members. Road hazard analysis was discussed as an effective means of reducing vehicular incidents. The dangers posed by cell phones were reviewed, as well as issues concerning motor vehicle event data recorders and personal privacy. The safety aspects of global positioning system (GPS) enabled fleet management systems were evaluated. Tactical plans for Alberta's traffic collision fatality and injury reduction strategy were examined. An overview of the Schlumberger Canada Driving Program was presented. The systems and processes used within Halliburton Canada to reinforce safe driving behaviors were reviewed. Real time personnel monitoring was discussed, as well as various asset management technologies. It was suggested that new communications technologies for mobile resource management can improve oilfield fleet safety and productivity. The forum featured 12 presentations, of which 8 have been catalogued separately for inclusion in this database.

Chitosan derived co-spheroids of neural stem cells and mesenchymal stem cells for neural regeneration.

Science.gov (United States)

Han, Hao-Wei; Hsu, Shan-Hui

2017-10-01

Chitosan has been considered as candidate biomaterials for neural applications. The effective treatment of neurodegeneration or injury to the central nervous system (CNS) is still in lack nowadays. Adult neural stem cells (NSCs) represents a promising cell source to treat the CNS diseases but they are limited in number. Here, we developed the core-shell spheroids of NSCs (shell) and mesenchymal stem cells (MSCs, core) by co-culturing cells on the chitosan surface. The NSCs in chitosan derived co-spheroids displayed a higher survival rate than those in NSC homo-spheroids. The direct interaction of NSCs with MSCs in the co-spheroids increased the Notch activity and differentiation tendency of NSCs. Meanwhile, the differentiation potential of MSCs in chitosan derived co-spheroids was significantly enhanced toward neural lineages. Furthermore, NSC homo-spheroids and NSC/MSC co-spheroids derived on chitosan were evaluated for their in vivo efficacy by the embryonic and adult zebrafish brain injury models. The locomotion activity of zebrafish receiving chitosan derived NSC homo-spheroids or NSC/MSC co-spheroids was partially rescued in both models. Meanwhile, the higher survival rate was observed in the group of adult zebrafish implanted with chitosan derived NSC/MSC co-spheroids as compared to NSC homo-spheroids. These evidences indicate that chitosan may provide an extracellular matrix-like environment to drive the interaction and the morphological assembly between NSCs and MSCs and promote their neural differentiation capacities, which can be used for neural regeneration. Copyright © 2017 Elsevier B.V. All rights reserved.

Anti-synchronization control of BAM memristive neural networks with multiple proportional delays and stochastic perturbations

Science.gov (United States)

Wang, Weiping; Yuan, Manman; Luo, Xiong; Liu, Linlin; Zhang, Yao

2018-01-01

Proportional delay is a class of unbounded time-varying delay. A class of bidirectional associative memory (BAM) memristive neural networks with multiple proportional delays is concerned in this paper. First, we propose the model of BAM memristive neural networks with multiple proportional delays and stochastic perturbations. Furthermore, by choosing suitable nonlinear variable transformations, the BAM memristive neural networks with multiple proportional delays can be transformed into the BAM memristive neural networks with constant delays. Based on the drive-response system concept, differential inclusions theory and Lyapunov stability theory, some anti-synchronization criteria are obtained. Finally, the effectiveness of proposed criteria are demonstrated through numerical examples.

neural control system

International Nuclear Information System (INIS)

Elshazly, A.A.E.

2002-01-01

Automatic power stabilization control is the desired objective for any reactor operation , especially, nuclear power plants. A major problem in this area is inevitable gap between a real plant ant the theory of conventional analysis and the synthesis of linear time invariant systems. in particular, the trajectory tracking control of a nonlinear plant is a class of problems in which the classical linear transfer function methods break down because no transfer function can represent the system over the entire operating region . there is a considerable amount of research on the model-inverse approach using feedback linearization technique. however, this method requires a prices plant model to implement the exact linearizing feedback, for nuclear reactor systems, this approach is not an easy task because of the uncertainty in the plant parameters and un-measurable state variables . therefore, artificial neural network (ANN) is used either in self-tuning control or in improving the conventional rule-based exper system.the main objective of this thesis is to suggest an ANN, based self-learning controller structure . this method is capable of on-line reinforcement learning and control for a nuclear reactor with a totally unknown dynamics model. previously, researches are based on back- propagation algorithm . back -propagation (BP), fast back -propagation (FBP), and levenberg-marquardt (LM), algorithms are discussed and compared for reinforcement learning. it is found that, LM algorithm is quite superior

Additive manufacturing of short and mixed fibre-reinforced polymer

Science.gov (United States)

Lewicki, James; Duoss, Eric B.; Rodriguez, Jennifer Nicole; Worsley, Marcus A.; King, Michael J.

2018-01-09

Additive manufacturing of a fiber-reinforced polymer (FRP) product using an additive manufacturing print head; a reservoir in the additive manufacturing print head; short carbon fibers in the reservoir, wherein the short carbon fibers are randomly aligned in the reservoir; an acrylate, methacrylate, epoxy, cyanate ester or isocyanate resin in the reservoir, wherein the short carbon fibers are dispersed in the acrylate, methacrylate, epoxy, cyanate ester or isocyanate resin; a tapered nozzle in the additive manufacturing print head operatively connected to the reservoir, the tapered nozzle produces an extruded material that forms the fiber-reinforced polymer product; baffles in the tapered nozzle that receive the acrylate, methacrylate, epoxy, cyanate ester or isocyanate resin with the short carbon fibers dispersed in the acrylate, methacrylate, epoxy, cyanate ester or isocyanate resin; and a system for driving the acrylate, methacrylate, epoxy, cyanate ester or isocyanate resin with the short carbon fibers dispersed in the acrylate, methacrylate, epoxy, cyanate ester or isocyanate resin from the reservoir through the tapered nozzle wherein the randomly aligned short carbon fibers in the acrylate, methacrylate, epoxy, cyanate ester or isocyanate resin are aligned by the baffles and wherein the extruded material has the short carbon fibers aligned in the acrylate, methacrylate, epoxy, cyanate ester or isocyanate resin that forms the fiber-reinforced polymer product.

Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement.

Science.gov (United States)

Belke, Terry W; Christie-Fougere, Melissa M

2006-11-01

Across two experiments, a peak procedure was used to assess the timing of the onset and offset of an opportunity to run as a reinforcer. The first experiment investigated the effect of reinforcer duration on temporal discrimination of the onset of the reinforcement interval. Three male Wistar rats were exposed to fixed-interval (FI) 30-s schedules of wheel-running reinforcement and the duration of the opportunity to run was varied across values of 15, 30, and 60s. Each session consisted of 50 reinforcers and 10 probe trials. Results showed that as reinforcer duration increased, the percentage of postreinforcement pauses longer than the 30-s schedule interval increased. On probe trials, peak response rates occurred near the time of reinforcer delivery and peak times varied with reinforcer duration. In a second experiment, seven female Long-Evans rats were exposed to FI 30-s schedules leading to 30-s opportunities to run. Timing of the onset and offset of the reinforcement period was assessed by probe trials during the schedule interval and during the reinforcement interval in separate conditions. The results provided evidence of timing of the onset, but not the offset of the wheel-running reinforcement period. Further research is required to assess if timing occurs during a wheel-running reinforcement period.

Human reinforcement learning subdivides structured action spaces by learning effector-specific values.

Science.gov (United States)

Gershman, Samuel J; Pesaran, Bijan; Daw, Nathaniel D

2009-10-28

Humans and animals are endowed with a large number of effectors. Although this enables great behavioral flexibility, it presents an equally formidable reinforcement learning problem of discovering which actions are most valuable because of the high dimensionality of the action space. An unresolved question is how neural systems for reinforcement learning-such as prediction error signals for action valuation associated with dopamine and the striatum-can cope with this "curse of dimensionality." We propose a reinforcement learning framework that allows for learned action valuations to be decomposed into effector-specific components when appropriate to a task, and test it by studying to what extent human behavior and blood oxygen level-dependent (BOLD) activity can exploit such a decomposition in a multieffector choice task. Subjects made simultaneous decisions with their left and right hands and received separate reward feedback for each hand movement. We found that choice behavior was better described by a learning model that decomposed the values of bimanual movements into separate values for each effector, rather than a traditional model that treated the bimanual actions as unitary with a single value. A decomposition of value into effector-specific components was also observed in value-related BOLD signaling, in the form of lateralized biases in striatal correlates of prediction error and anticipatory value correlates in the intraparietal sulcus. These results suggest that the human brain can use decomposed value representations to "divide and conquer" reinforcement learning over high-dimensional action spaces.

Enhancing corrosion resistance of reinforced concrete structures with hybrid fiber reinforced concrete

International Nuclear Information System (INIS)

Blunt, J.; Jen, G.; Ostertag, C.P.

2015-01-01

Highlights: • Reinforced concrete beams were subjected to cyclic flexural loading. • Hybrid fiber reinforced composites were effective in reducing corrosion rates. • Crack resistance due to fibers increased corrosion resistance of steel rebar. • Galvanic corrosion measurements underestimated corrosion rates. • Polarization resistance measurements predicted mass loss more accurately. - Abstract: Service loads well below the yield strength of steel reinforcing bars lead to cracking of reinforced concrete. This paper investigates whether the crack resistance of Hybrid Fiber Reinforced Concrete (HyFRC) reduces the corrosion rate of steel reinforcing bars in concrete after cyclic flexural loading. The reinforcing bars were extracted to examine their surface for corrosion and compare microcell and macrocell corrosion mass loss estimates against direct gravimetric measurements. A delay in corrosion initiation and lower active corrosion rates were observed in the HyFRC beam specimens when compared to reinforced specimens containing plain concrete matrices cycled at the same flexural load

Improved Object Proposals with Geometrical Features for Autonomous Driving

Directory of Open Access Journals (Sweden)

Yiliu Feng

2017-01-01

Full Text Available This paper aims at generating high-quality object proposals for object detection in autonomous driving. Most existing proposal generation methods are designed for the general object detection, which may not perform well in a particular scene. We propose several geometrical features suited for autonomous driving and integrate them into state-of-the-art general proposal generation methods. In particular, we formulate the integration as a feature fusion problem by fusing the geometrical features with existing proposal generation methods in a Bayesian framework. Experiments on the challenging KITTI benchmark demonstrate that our approach improves the existing methods significantly. Combined with a convolutional neural net detector, our approach achieves state-of-the-art performance on all three KITTI object classes.

Prebiotics Supplementation Impact on the Reinforcing and Motivational Aspect of Feeding

Directory of Open Access Journals (Sweden)

Anne-Sophie Delbès

2018-05-01

(regular chow or HFHS as well as the timing at which prebiotic supplementation is introduced (preventive or curative greatly influence the efficacy of the gut–microbiota–brain axis. This crosstalk selectively alters the hedonic or motivational drive to eat and triggers molecular changes in neural substrates involved in the homeostatic and non-homeostatic control of body weight.

Stimulus-dependent suppression of chaos in recurrent neural networks

International Nuclear Information System (INIS)

Rajan, Kanaka; Abbott, L. F.; Sompolinsky, Haim

2010-01-01

Neuronal activity arises from an interaction between ongoing firing generated spontaneously by neural circuits and responses driven by external stimuli. Using mean-field analysis, we ask how a neural network that intrinsically generates chaotic patterns of activity can remain sensitive to extrinsic input. We find that inputs not only drive network responses, but they also actively suppress ongoing activity, ultimately leading to a phase transition in which chaos is completely eliminated. The critical input intensity at the phase transition is a nonmonotonic function of stimulus frequency, revealing a 'resonant' frequency at which the input is most effective at suppressing chaos even though the power spectrum of the spontaneous activity peaks at zero and falls exponentially. A prediction of our analysis is that the variance of neural responses should be most strongly suppressed at frequencies matching the range over which many sensory systems operate.

Theoretical and numerical analysis of reinforced concrete beams with confinement reinforcement

Directory of Open Access Journals (Sweden)

R. G. Delalibera

Full Text Available This paper discusses the use of confinement in over-reinforced concrete beams. This reinforcement consists of square stirrups, placed in the compression zone of the beam cross-section, in order to improve its ductility. A parametric numerical study is initially performed, using a finite element computational program that considers the material nonlinearities and the confinement effect. To investigate the influence of the transverse reinforcing ratio on the beam ductility, an experimental program was also conducted. Four over-reinforced beams were tested; three beam specimens with additional transverse reinforcement to confine the beams, and one without it. All specimens were fabricated with a concrete designed for a compressive strength of 25 MPa. The experimental results show that the post-peak ductility factor is proportional to the confining reinforcement ratio, however the same is not observed for the pre-peak ductility factor, which varied randomly with changes in the confining reinforcement ratio. It was also observed from the experiments that the confinement effect tends to be smaller close to the beam neutral axis.

Proposal for an All-Spin Artificial Neural Network: Emulating Neural and Synaptic Functionalities Through Domain Wall Motion in Ferromagnets.

Science.gov (United States)

Sengupta, Abhronil; Shim, Yong; Roy, Kaushik

2016-12-01

Non-Boolean computing based on emerging post-CMOS technologies can potentially pave the way for low-power neural computing platforms. However, existing work on such emerging neuromorphic architectures have either focused on solely mimicking the neuron, or the synapse functionality. While memristive devices have been proposed to emulate biological synapses, spintronic devices have proved to be efficient at performing the thresholding operation of the neuron at ultra-low currents. In this work, we propose an All-Spin Artificial Neural Network where a single spintronic device acts as the basic building block of the system. The device offers a direct mapping to synapse and neuron functionalities in the brain while inter-layer network communication is accomplished via CMOS transistors. To the best of our knowledge, this is the first demonstration of a neural architecture where a single nanoelectronic device is able to mimic both neurons and synapses. The ultra-low voltage operation of low resistance magneto-metallic neurons enables the low-voltage operation of the array of spintronic synapses, thereby leading to ultra-low power neural architectures. Device-level simulations, calibrated to experimental results, was used to drive the circuit and system level simulations of the neural network for a standard pattern recognition problem. Simulation studies indicate energy savings by  ∼  100× in comparison to a corresponding digital/analog CMOS neuron implementation.

[Interaction between neurons of the frontal cortex and hippocampus during the realization of choice of food reinforcement quality in cats].

Science.gov (United States)

Merzhanova, G Kh; Dolbakian, E E; Khokhlova, V N

2003-01-01

Six cats were subjected to the procedure of appetitive instrumental conditioning (with light as a conditioned stimuls) by the method of the "active choice" of reinforcement quality. Short-delay conditioned bar-press responses were rewarded with bread-meat mixture, and the delayed responses were reinforced by meat. The animals differed in behavior strategy: four animals preferred the bar-pressing with a long delay (the so-called "self-control" group), and two cats preferred the bar-pressing with a short delay (the so-called "impulsive" group). Multiunit activity in the frontal cortex and hippocampus (CA3) was recorded via chronically implanted nichrome wire semimicroelectrodes. An interaction between the neighboring neurons in the frontal cortex and hippocampus (within local neural networks) and between the neurons of the frontal cortex and hippocampus (distributed neural networks in frontal-hippocampal and hippocampal-frontal directions) was evaluated by means of statistical crosscorrelation analysis of spike trains. Crosscorrelations between neuronal spike trains in the delay range of 0-100 ms were explored. It was shown that the number of crosscorrelations between the neuronal discharges both in the local and distributed networks was significantly higher in the "self-control" cats. It was suggested that the local and distributed neural networks of the frontal cortex and hippocampus are involved in the system of brain structures which determine the behavioral strategy of animals in the "self-control" group.

Novel Adaptive Forward Neural MIMO NARX Model for the Identification of Industrial 3-DOF Robot Arm Kinematics

OpenAIRE

Ho Pham Huy Anh; Nguyen Thanh Nam

2012-01-01

In this paper, a novel forward adaptive neural MIMO NARX model is used for modelling and identifying the forward kinematics of an industrial 3‐DOF robot arm system. The nonlinear features of the forward kinematics of the industrial robot arm drive are thoroughly modelled based on the forward adaptive neural NARX model‐based identification process using experimental input‐output training data. This paper proposes a novel use of a back propagation (BP) algorithm to generate the forward neural M...

Synchronization of chaotic recurrent neural networks with time-varying delays using nonlinear feedback control

International Nuclear Information System (INIS)

Cui Baotong; Lou Xuyang

2009-01-01

In this paper, a new method to synchronize two identical chaotic recurrent neural networks is proposed. Using the drive-response concept, a nonlinear feedback control law is derived to achieve the state synchronization of the two identical chaotic neural networks. Furthermore, based on the Lyapunov method, a delay independent sufficient synchronization condition in terms of linear matrix inequality (LMI) is obtained. A numerical example with graphical illustrations is given to illuminate the presented synchronization scheme

DSP implementation of a PV system with GA-MLP-NN based MPPT controller supplying BLDC motor drive

International Nuclear Information System (INIS)

Akkaya, R.; Kulaksiz, A.A.; Aydogdu, O.

2007-01-01

This paper presents a brushless dc motor drive for heating, ventilating and air conditioning fans, which is utilized as the load of a photovoltaic system with a maximum power point tracking (MPPT) controller. The MPPT controller is based on a genetic assisted, multi-layer perceptron neural network (GA-MLP-NN) structure and includes a DC-DC boost converter. Genetic assistance in the neural network is used to optimize the size of the hidden layer. Also, for training the network, a genetic assisted, Levenberg-Marquardt (GA-LM) algorithm is utilized. The off line GA-MLP-NN, trained by this hybrid algorithm, is utilized for online estimation of the voltage and current values in the maximum power point. A brushless dc (BLDC) motor drive system that incorporates a motor controller with proportional integral (PI) speed control loop is successfully implemented to operate the fans. The digital signal processor (DSP) based unit provides rapid achievement of the MPPT and current control of the BLDC motor drive. The performance results of the system are given, and experimental results are presented for a laboratory prototype of 120 W

Physiological mechanisms of dyspnea during exercise with external thoracic restriction: Role of increased neural respiratory drive

Science.gov (United States)

Mendonca, Cassandra T.; Schaeffer, Michele R.; Riley, Patrick

2013-01-01

We tested the hypothesis that neuromechanical uncoupling of the respiratory system forms the mechanistic basis of dyspnea during exercise in the setting of “abnormal” restrictive constraints on ventilation (VE). To this end, we examined the effect of chest wall strapping (CWS) sufficient to mimic a “mild” restrictive lung deficit on the interrelationships between VE, breathing pattern, dynamic operating lung volumes, esophageal electrode-balloon catheter-derived measures of the diaphragm electromyogram (EMGdi) and the transdiaphragmatic pressure time product (PTPdi), and sensory intensity and unpleasantness ratings of dyspnea during exercise. Twenty healthy men aged 25.7 ± 1.1 years (means ± SE) completed symptom-limited incremental cycle exercise tests under two randomized conditions: unrestricted control and CWS to reduce vital capacity (VC) by 21.6 ± 0.5%. Compared with control, exercise with CWS was associated with 1) an exaggerated EMGdi and PTPdi response; 2) no change in the relationship between EMGdi and each of tidal volume (expressed as a percentage of VC), inspiratory reserve volume, and PTPdi, thus indicating relative preservation of neuromechanical coupling; 3) increased sensory intensity and unpleasantness ratings of dyspnea; and 4) no change in the relationship between increasing EMGdi and each of the intensity and unpleasantness of dyspnea. In conclusion, the increased intensity and unpleasantness of dyspnea during exercise with CWS could not be readily explained by increased neuromechanical uncoupling but likely reflected the awareness of increased neural respiratory drive (EMGdi) needed to achieve any given VE during exercise in the setting of “abnormal” restrictive constraints on tidal volume expansion. PMID:24356524

Axial Compression Tests on Corroded Reinforced Concrete Columns Consolidated with Fibre Reinforced Polymers

Directory of Open Access Journals (Sweden)

Bin Ding

2017-06-01

Full Text Available Reinforced concrete structure featured by strong bearing capacity, high rigidity, good integrity, good fire resistance, and extensive applicability occupies a mainstream position in contemporary architecture. However, with the development of social economy, people need higher requirements on architectural structure; durability, especially, has been extensively researched. Because of the higher requirement on building material, ordinary reinforced concrete structure has not been able to satisfy the demand. As a result, some new materials and structures have emerged, for example, fibre reinforced polymers. Compared to steel reinforcement, fibre reinforced polymers have many advantages, such as high tensile strength, good durability, good shock absorption, low weight, and simple construction. The application of fibre reinforced polymers in architectural structure can effectively improve the durability of the concrete structure and lower the maintenance, reinforcement, and construction costs in severe environments. Based on the concepts of steel tube concrete, fibre reinforced composite material confined concrete, and fibre reinforced composite material tubed concrete, this study proposes a novel composite structure, i.e., fibre reinforced composite material and steel tube concrete composite structure. The structure was developed by pasting fibre around steel tube concrete and restraining core concrete using fibre reinforced composite material and steel tubes. The bearing capacity and ultimate deformation capacity of the structure was tested using column axial compression test.

Neural Behavior Chain Learning of Mobile Robot Actions

Directory of Open Access Journals (Sweden)

Lejla Banjanovic-Mehmedovic

2012-01-01

Full Text Available This paper presents a visual/motor behavior learning approach, based on neural networks. We propose Behavior Chain Model (BCM in order to create a way of behavior learning. Our behavior-based system evolution task is a mobile robot detecting a target and driving/acting towards it. First, the mapping relations between the image feature domain of the object and the robot action domain are derived. Second, a multilayer neural network for offline learning of the mapping relations is used. This learning structure through neural network training process represents a connection between the visual perceptions and motor sequence of actions in order to grip a target. Last, using behavior learning through a noticed action chain, we can predict mobile robot behavior for a variety of similar tasks in similar environment. Prediction results suggest that the methodology is adequate and could be recognized as an idea for designing different mobile robot behaviour assistance.

Reinforcement Learning of Linking and Tracing Contours in Recurrent Neural Networks

Science.gov (United States)

Brosch, Tobias; Neumann, Heiko; Roelfsema, Pieter R.

2015-01-01

The processing of a visual stimulus can be subdivided into a number of stages. Upon stimulus presentation there is an early phase of feedforward processing where the visual information is propagated from lower to higher visual areas for the extraction of basic and complex stimulus features. This is followed by a later phase where horizontal connections within areas and feedback connections from higher areas back to lower areas come into play. In this later phase, image elements that are behaviorally relevant are grouped by Gestalt grouping rules and are labeled in the cortex with enhanced neuronal activity (object-based attention in psychology). Recent neurophysiological studies revealed that reward-based learning influences these recurrent grouping processes, but it is not well understood how rewards train recurrent circuits for perceptual organization. This paper examines the mechanisms for reward-based learning of new grouping rules. We derive a learning rule that can explain how rewards influence the information flow through feedforward, horizontal and feedback connections. We illustrate the efficiency with two tasks that have been used to study the neuronal correlates of perceptual organization in early visual cortex. The first task is called contour-integration and demands the integration of collinear contour elements into an elongated curve. We show how reward-based learning causes an enhancement of the representation of the to-be-grouped elements at early levels of a recurrent neural network, just as is observed in the visual cortex of monkeys. The second task is curve-tracing where the aim is to determine the endpoint of an elongated curve composed of connected image elements. If trained with the new learning rule, neural networks learn to propagate enhanced activity over the curve, in accordance with neurophysiological data. We close the paper with a number of model predictions that can be tested in future neurophysiological and computational studies

Function of FEZF1 during early neural differentiation of human embryonic stem cells.

Science.gov (United States)

Liu, Xin; Su, Pei; Lu, Lisha; Feng, Zicen; Wang, Hongtao; Zhou, Jiaxi

2018-01-01

The understanding of the mechanism underlying human neural development has been hampered due to lack of a cellular system and complicated ethical issues. Human embryonic stem cells (hESCs) provide an invaluable model for dissecting human development because of unlimited self-renewal and the capacity to differentiate into nearly all cell types in the human body. In this study, using a chemical defined neural induction protocol and molecular profiling, we identified Fez family zinc finger 1 (FEZF1) as a potential regulator of early human neural development. FEZF1 is rapidly up-regulated during neural differentiation in hESCs and expressed before PAX6, a well-established marker of early human neural induction. We generated FEZF1-knockout H1 hESC lines using CRISPR-CAS9 technology and found that depletion of FEZF1 abrogates neural differentiation of hESCs. Moreover, loss of FEZF1 impairs the pluripotency exit of hESCs during neural specification, which partially explains the neural induction defect caused by FEZF1 deletion. However, enforced expression of FEZF1 itself fails to drive neural differentiation in hESCs, suggesting that FEZF1 is necessary but not sufficient for neural differentiation from hESCs. Taken together, our findings identify one of the earliest regulators expressed upon neural induction and provide insight into early neural development in human.

Interfacial fracture of dentin adhesively bonded to quartz-fiber reinforced composite

International Nuclear Information System (INIS)

Melo, Renata M.; Rahbar, Nima; Soboyejo, Wole

2011-01-01

The paper presents the results of an experimental study of interfacial failure in a multilayered structure consisting of a dentin/resin cement/quartz-fiber reinforced composite (FRC). Slices of dentin close to the pulp chamber were sandwiched by two half-circle discs made of a quartz-fiber reinforced composite, bonded with bonding agent (All-bond 2, BISCO, Schaumburg) and resin cement (Duo-link, BISCO, Schaumburg) to make Brazil-nut sandwich specimens for interfacial toughness testing. Interfacial fracture toughness (strain energy release rate, G) was measured as a function of mode mixity by changing loading angles from 0 deg. to 15 deg. The interfacial fracture surfaces were then examined using Scanning Electron Microscopy (SEM) and Energy Dispersive X-ray Spectroscopy (EDX) to determine the failure modes when loading angles changed. A computational model was also developed to calculate the driving forces, stress intensity factors and mode mixities. Interfacial toughness increased from ∼ 1.5 to 3.2 J/m 2 when the loading angle increases from ∼ 0 to 15 deg. The hybridized dentin/cement interface appeared to be tougher than the resin cement/quartz-fiber reinforced epoxy. The Brazil-nut sandwich specimen was a suitable method to investigate the mechanical integrity of dentin/cement/FRC interfaces.

Artificial Neural Networks for the Prediction of Wear Properties of Al6061-TiO2 Composites

Science.gov (United States)

Veeresh Kumar, G. B.; Pramod, R.; Shivakumar Gouda, P. S.; Rao, C. S. P.

2017-08-01

The exceptional performance of composite materials in comparison with the monolithic materials have been extensively studied by researchers. Among the metal matrix composites Aluminium matrix based composites have displayed superior mechanical properties. The aluminium 6061 alloy has been used in aeronautical and automotive components, but their resistance against the wear is poor. To enhance the wear properties, Titanium dioxide (TiO2) particulates have been used as reinforcements. In the present investigation Back propagation (BP) technique has been adopted for Artificial Neural Network [ANN] modelling. The wear experimentations were carried out on a pin-on-disc wear monitoring apparatus. For conduction of wear tests ASTM G99 was adopted. Experimental design was carried out using Taguchi L27 orthogonal array. The sliding distance, weight percentage of the reinforcement material and applied load have a substantial influence on the height damage due to wear of the Al6061 and Al6061-TiO2 filled composites. The Al6061 with 3 wt% TiO2 composite displayed an excellent wear resistance in comparison with other composites investigated. A non-linear relationship between density, applied load, weight percentage of reinforcement, sliding distance and height decrease due to wear has been established using an artificial neural network. A good agreement has been observed between experimental and ANN model predicted results.

A canonical neural mechanism for behavioral variability

Science.gov (United States)

Darshan, Ran; Wood, William E.; Peters, Susan; Leblois, Arthur; Hansel, David

2017-05-01

The ability to generate variable movements is essential for learning and adjusting complex behaviours. This variability has been linked to the temporal irregularity of neuronal activity in the central nervous system. However, how neuronal irregularity actually translates into behavioural variability is unclear. Here we combine modelling, electrophysiological and behavioural studies to address this issue. We demonstrate that a model circuit comprising topographically organized and strongly recurrent neural networks can autonomously generate irregular motor behaviours. Simultaneous recordings of neurons in singing finches reveal that neural correlations increase across the circuit driving song variability, in agreement with the model predictions. Analysing behavioural data, we find remarkable similarities in the babbling statistics of 5-6-month-old human infants and juveniles from three songbird species and show that our model naturally accounts for these `universal' statistics.

Brain activation during fast driving in a driving simulator: the role of the lateral prefrontal cortex.

Science.gov (United States)

Jäncke, Lutz; Brunner, Béatrice; Esslen, Michaela

2008-07-16

Little is currently known about the neural underpinnings of the cognitive control of driving behavior in realistic situations and of the driver's speeding behavior in particular. In this study, participants drove in realistic scenarios presented in a high-end driving simulator. Scalp-recorded EEG oscillations in the alpha-band (8-13 Hz) with a 30-electrode montage were recorded while the participants drove under different conditions: (i) excessively fast (Fast), (ii) in a controlled manner at a safe speed (Correct), and (iii) impatiently in the context of testing traffic conditions (Impatient). Intracerebral sources of alpha-band activation were estimated using low resolution electrical tomography. Given that previous studies have shown a strong negative correlation between the Bold response in the frontal cortex and the alpha-band power, we used alpha-band-related activity as an estimation of frontal activation. Statistical analysis revealed more alpha-band-related activity (i.e. less neuronal activation) in the right lateral prefrontal cortex, including the dorsolateral prefrontal cortex, during fast driving. Those participants who speeded most and exhibited greater risk-taking behavior demonstrated stronger alpha-related activity (i.e. less neuronal activation) in the left anterior lateral prefrontal cortex. These findings are discussed in the context of current theories about the role of the lateral prefrontal cortex in controlling risk-taking behavior, task switching, and multitasking.

Flexural strength using Steel Plate, Carbon Fiber Reinforced Polymer (CFRP) and Glass Fiber Reinforced Polymer (GFRP) on reinforced concrete beam in building technology

Science.gov (United States)

Tarigan, Johannes; Patra, Fadel Muhammad; Sitorus, Torang

2018-03-01

Reinforced concrete structures are very commonly used in buildings because they are cheaper than the steel structures. But in reality, many concrete structures are damaged, so there are several ways to overcome this problem, by providing reinforcement with Fiber Reinforced Polymer (FRP) and reinforcement with steel plates. Each type of reinforcements has its advantages and disadvantages. In this study, researchers discuss the comparison between flexural strength of reinforced concrete beam using steel plates and Fiber Reinforced Polymer (FRP). In this case, the researchers use Carbon Fiber Reinforced Polymer (CFRP) and Glass Fiber Reinforced Polymer (GFRP) as external reinforcements. The dimension of the beams is 15 x 25 cm with the length of 320 cm. Based on the analytical results, the strength of the beam with CFRP is 1.991 times its initial, GFRP is 1.877 times while with the steel plate is 1.646 times. Based on test results, the strength of the beam with CFRP is 1.444 times its initial, GFRP is 1.333 times while the steel plate is 1.167 times. Based on these test results, the authors conclude that beam with CFRP is the best choice for external reinforcement in building technology than the others.

Energy Management Strategy for a Hybrid Electric Vehicle Based on Deep Reinforcement Learning

Directory of Open Access Journals (Sweden)

Yue Hu

2018-01-01

Full Text Available An energy management strategy (EMS is important for hybrid electric vehicles (HEVs since it plays a decisive role on the performance of the vehicle. However, the variation of future driving conditions deeply influences the effectiveness of the EMS. Most existing EMS methods simply follow predefined rules that are not adaptive to different driving conditions online. Therefore, it is useful that the EMS can learn from the environment or driving cycle. In this paper, a deep reinforcement learning (DRL-based EMS is designed such that it can learn to select actions directly from the states without any prediction or predefined rules. Furthermore, a DRL-based online learning architecture is presented. It is significant for applying the DRL algorithm in HEV energy management under different driving conditions. Simulation experiments have been conducted using MATLAB and Advanced Vehicle Simulator (ADVISOR co-simulation. Experimental results validate the effectiveness of the DRL-based EMS compared with the rule-based EMS in terms of fuel economy. The online learning architecture is also proved to be effective. The proposed method ensures the optimality, as well as real-time applicability, in HEVs.

Neural crest stem cell population in craniomaxillofacial development and tissue repair

Directory of Open Access Journals (Sweden)

M La Noce

2014-10-01

Full Text Available Neural crest cells, delaminating from the neural tube during migration, undergo an epithelial-mesenchymal transition and differentiate into several cell types strongly reinforcing the mesoderm of the craniofacial body area – giving rise to bone, cartilage and other tissues and cells of this human body area. Recent studies on craniomaxillofacial neural crest-derived cells have provided evidence for the tremendous plasticity of these cells. Actually, neural crest cells can respond and adapt to the environment in which they migrate and the cranial mesoderm plays an important role toward patterning the identity of the migrating neural crest cells. In our experience, neural crest-derived stem cells, such as dental pulp stem cells, can actively proliferate, repair bone and give rise to other tissues and cytotypes, including blood vessels, smooth muscle, adipocytes and melanocytes, highlighting that their use in tissue engineering is successful. In this review, we provide an overview of the main pathways involved in neural crest formation, delamination, migration and differentiation; and, in particular, we concentrate our attention on the translatability of the latest scientific progress. Here we try to suggest new ideas and strategies that are needed to fully develop the clinical use of these cells. This effort should involve both researchers/clinicians and improvements in good manufacturing practice procedures. It is important to address studies towards clinical application or take into consideration that studies must have an effective therapeutic prospect for humans. New approaches and ideas must be concentrated also toward stem cell recruitment and activation within the human body, overcoming the classical grafting.

Measures of Coupling between Neural Populations Based on Granger Causality Principle.

Science.gov (United States)

Kaminski, Maciej; Brzezicka, Aneta; Kaminski, Jan; Blinowska, Katarzyna J

2016-01-01

This paper shortly reviews the measures used to estimate neural synchronization in experimental settings. Our focus is on multivariate measures of dependence based on the Granger causality (G-causality) principle, their applications and performance in respect of robustness to noise, volume conduction, common driving, and presence of a "weak node." Application of G-causality measures to EEG, intracranial signals and fMRI time series is addressed. G-causality based measures defined in the frequency domain allow the synchronization between neural populations and the directed propagation of their electrical activity to be determined. The time-varying G-causality based measure Short-time Directed Transfer Function (SDTF) supplies information on the dynamics of synchronization and the organization of neural networks. Inspection of effective connectivity patterns indicates a modular structure of neural networks, with a stronger coupling within modules than between them. The hypothetical plausible mechanism of information processing, suggested by the identified synchronization patterns, is communication between tightly coupled modules intermitted by sparser interactions providing synchronization of distant structures.

Review On Applications Of Neural Network To Computer Vision

Science.gov (United States)

Li, Wei; Nasrabadi, Nasser M.

1989-03-01

Neural network models have many potential applications to computer vision due to their parallel structures, learnability, implicit representation of domain knowledge, fault tolerance, and ability of handling statistical data. This paper demonstrates the basic principles, typical models and their applications in this field. Variety of neural models, such as associative memory, multilayer back-propagation perceptron, self-stabilized adaptive resonance network, hierarchical structured neocognitron, high order correlator, network with gating control and other models, can be applied to visual signal recognition, reinforcement, recall, stereo vision, motion, object tracking and other vision processes. Most of the algorithms have been simulated on com-puters. Some have been implemented with special hardware. Some systems use features, such as edges and profiles, of images as the data form for input. Other systems use raw data as input signals to the networks. We will present some novel ideas contained in these approaches and provide a comparison of these methods. Some unsolved problems are mentioned, such as extracting the intrinsic properties of the input information, integrating those low level functions to a high-level cognitive system, achieving invariances and other problems. Perspectives of applications of some human vision models and neural network models are analyzed.

Cognitive deficits caused by prefrontal cortical and hippocampal neural disinhibition.

Science.gov (United States)

Bast, Tobias; Pezze, Marie; McGarrity, Stephanie

2017-10-01

We review recent evidence concerning the significance of inhibitory GABA transmission and of neural disinhibition, that is, deficient GABA transmission, within the prefrontal cortex and the hippocampus, for clinically relevant cognitive functions. Both regions support important cognitive functions, including attention and memory, and their dysfunction has been implicated in cognitive deficits characterizing neuropsychiatric disorders. GABAergic inhibition shapes cortico-hippocampal neural activity, and, recently, prefrontal and hippocampal neural disinhibition has emerged as a pathophysiological feature of major neuropsychiatric disorders, especially schizophrenia and age-related cognitive decline. Regional neural disinhibition, disrupting spatio-temporal control of neural activity and causing aberrant drive of projections, may disrupt processing within the disinhibited region and efferent regions. Recent studies in rats showed that prefrontal and hippocampal neural disinhibition (by local GABA antagonist microinfusion) dysregulates burst firing, which has been associated with important aspects of neural information processing. Using translational tests of clinically relevant cognitive functions, these studies showed that prefrontal and hippocampal neural disinhibition disrupts regional cognitive functions (including prefrontal attention and hippocampal memory function). Moreover, hippocampal neural disinhibition disrupted attentional performance, which does not require the hippocampus but requires prefrontal-striatal circuits modulated by the hippocampus. However, some prefrontal and hippocampal functions (including inhibitory response control) are spared by regional disinhibition. We consider conceptual implications of these findings, regarding the distinct relationships of distinct cognitive functions to prefrontal and hippocampal GABA tone and neural activity. Moreover, the findings support the proposition that prefrontal and hippocampal neural disinhibition

Neural network-based adaptive dynamic surface control for permanent magnet synchronous motors.

Science.gov (United States)

Yu, Jinpeng; Shi, Peng; Dong, Wenjie; Chen, Bing; Lin, Chong

2015-03-01

This brief considers the problem of neural networks (NNs)-based adaptive dynamic surface control (DSC) for permanent magnet synchronous motors (PMSMs) with parameter uncertainties and load torque disturbance. First, NNs are used to approximate the unknown and nonlinear functions of PMSM drive system and a novel adaptive DSC is constructed to avoid the explosion of complexity in the backstepping design. Next, under the proposed adaptive neural DSC, the number of adaptive parameters required is reduced to only one, and the designed neural controllers structure is much simpler than some existing results in literature, which can guarantee that the tracking error converges to a small neighborhood of the origin. Then, simulations are given to illustrate the effectiveness and potential of the new design technique.

Experimental analysis of reinforced concrete beams strengthened in bending with carbon fiber reinforced polymer

Directory of Open Access Journals (Sweden)

M. M. VIEIRA

Full Text Available The use of carbon fiber reinforced polymer (CFRP has been widely used for the reinforcement of concrete structures due to its practicality and versatility in application, low weight, high tensile strength and corrosion resistance. Some construction companies use CFRP in flexural strengthening of reinforced concrete beams, but without anchor systems. Therefore, the aim of this study is analyze, through an experimental program, the structural behavior of reinforced concrete beams flexural strengthened by CFRP without anchor fibers, varying steel reinforcement and the amount of carbon fibers reinforcement layers. Thus, two groups of reinforced concrete beams were produced with the same geometric feature but with different steel reinforcement. Each group had five beams: one that is not reinforced with CFRP (reference and other reinforced with two, three, four and five layers of carbon fibers. Beams were designed using a computational routine developed in MAPLE software and subsequently tested in 4-point points flexural test up to collapse. Experimental tests have confirmed the effectiveness of the reinforcement, ratifying that beams collapse at higher loads and lower deformation as the amount of fibers in the reinforcing layers increased. However, the increase in the number of layers did not provide a significant increase in the performance of strengthened beams, indicating that it was not possible to take full advantage of strengthening applied due to the occurrence of premature failure mode in the strengthened beams for pullout of the cover that could have been avoided through the use of a suitable anchoring system for CFRP.

Reactive Neural Control for Phototaxis and Obstacle Avoidance Behavior of Walking Machines

DEFF Research Database (Denmark)

Manoonpong, Poramate; Pasemann, Frank; Wörgötter, Florentin

2007-01-01

as a sensory fusion unit. It filters sensory noise and shapes sensory data to drive the corresponding reactive behavior. On the other hand, modular neural control based on a central pattern generator is applied for locomotion of walking machines. It coordinates leg movements and can generate omnidirectional...

Reinforcement Schedules in a Verbal Reinforcement Combination and Renection-Impulsivity

OpenAIRE

TAMASE, Koji; UEDA, Masako

1986-01-01

It was predicted that higher proportion of the negative reinforcement "Wrong" than that of the positive reinforcement "Right" in a reinforcement combination will produce higher proportion of the correct response and this trend will be greater in reflective children than in impulsive children. From 140 kindergarten children 30 reflective and 30 impulsive children were selected and they were given a two-hole marble-dropping task. The best performance in the ratio of correct responses was obtain...

Application of neural networks and its prospect. 4. Prediction of major disruptions in tokamak plasmas, analyses of time series data

International Nuclear Information System (INIS)

Yoshino, Ryuji

2006-01-01

Disruption prediction of tokamak plasma has been studied by neural network. The disruption prediction performances by neural network are estimated by the prediction success rate, false alarm rate, and time prior to disruption. The current driving type disruption is predicted by time series data, and plasma lifetime, risk of disruption and plasma stability. Some disruptions generated by density limit, impurity mixture, error magnetic field can be predicted 100 % of prediction success rate by the premonitory symptoms. The pressure driving type disruption phenomena generate some hundred micro seconds before, so that the operation limits such as β N limit of DIII-D and density limit of ADITYA were investigated. The false alarm rate was decreased by β N limit training under stable discharge. The pressure driving disruption generated with increasing plasma pressure can be predicted about 90 % by evaluating plasma stability. (S.Y.)

Heavy-duty mentor : Inthinc's trucker-coaching software reinforces best-practices driving

Energy Technology Data Exchange (ETDEWEB)

Menzies, J.

2010-12-15

This article described Inthinc's tiwi and waySmart driver mentoring systems and how they are being put to use in the Canadian oil patch. The automated monitoring system for truck drivers generates audible alerts for unsafe practices, including verbal warnings. The use of the technology in the United States resulted in substantial improvements in seat-belt use, speeding violations, aggressive driving behaviours, and crash rates. The resulting safety improvements result in cost savings that justify the cost of the system. The system can also monitor and reduce vehicle idle-time and electronically log driver hours of service, including changes from vehicle to vehicle, which provides particularly useful data for the oil industry. Accelerometers are used to detect aggressive driving maneuvers, and the system can be customized with different degrees of leeway. The system is designed to be corrective rather than punitive. Drivers are given time to adjust their behaviour. Only repeated failure results in notification to the designated supervisor. In the future, the system will be able to prevent calling and texting from cell phones while the vehicle is in motion and provide instruction for walk-around, pre-trip truck inspections. The mentoring capability is the key distinction system, as the focus is on changing the behaviour of the driver in the vehicle. 1 fig.

Vocal production complexity correlates with neural instructions in the oyster toadfish (Opsanus tau)

DEFF Research Database (Denmark)

Elemans, C. P. H.; Mensinger, A. F.; Rome, L. C.

2014-01-01

frequencies are determined directly by the firing rate of a vocal-acoustic neural network that drives the contraction frequency of superfast swimbladder muscles. The oyster toadfish boatwhistle call starts with an irregular sound waveform that could be an emergent property of the peripheral nonlinear sound...

The effect of the inner-hair-cell mediated transduction on the shape of neural tuning curves

Science.gov (United States)

Altoè, Alessandro; Pulkki, Ville; Verhulst, Sarah

2018-05-01

The inner hair cells of the mammalian cochlea transform the vibrations of their stereocilia into releases of neurotransmitter at the ribbon synapses, thereby controlling the activity of the afferent auditory fibers. The mechanical-to-neural transduction is a highly nonlinear process and it introduces differences between the frequency-tuning of the stereocilia and that of the afferent fibers. Using a computational model of the inner hair cell that is based on in vitro data, we estimated that smaller vibrations of the stereocilia are necessary to drive the afferent fibers above threshold at low (≤0.5 kHz) than at high (≥4 kHz) driving frequencies. In the base of the cochlea, the transduction process affects the low-frequency tails of neural tuning curves. In particular, it introduces differences between the frequency-tuning of the stereocilia and that of the auditory fibers resembling those between basilar membrane velocity and auditory fibers tuning curves in the chinchilla base. For units with a characteristic frequency between 1 and 4 kHz, the transduction process yields shallower neural than stereocilia tuning curves as the characteristic frequency decreases. This study proposes that transduction contributes to the progressive broadening of neural tuning curves from the base to the apex.

Detection of braking intention in diverse situations during simulated driving based on EEG feature combination.

Science.gov (United States)

Kim, Il-Hwa; Kim, Jeong-Woo; Haufe, Stefan; Lee, Seong-Whan

2015-02-01

We developed a simulated driving environment for studying neural correlates of emergency braking in diversified driving situations. We further investigated to what extent these neural correlates can be used to detect a participant's braking intention prior to the behavioral response. We measured electroencephalographic (EEG) and electromyographic signals during simulated driving. Fifteen participants drove a virtual vehicle and were exposed to several kinds of traffic situations in a simulator system, while EEG signals were measured. After that, we extracted characteristic features to categorize whether the driver intended to brake or not. Our system shows excellent detection performance in a broad range of possible emergency situations. In particular, we were able to distinguish three different kinds of emergency situations (sudden stop of a preceding vehicle, sudden cutting-in of a vehicle from the side and unexpected appearance of a pedestrian) from non-emergency (soft) braking situations, as well as from situations in which no braking was required, but the sensory stimulation was similar to stimulations inducing an emergency situation (e.g., the sudden stop of a vehicle on a neighboring lane). We proposed a novel feature combination comprising movement-related potentials such as the readiness potential, event-related desynchronization features besides the event-related potentials (ERP) features used in a previous study. The performance of predicting braking intention based on our proposed feature combination was superior compared to using only ERP features. Our study suggests that emergency situations are characterized by specific neural patterns of sensory perception and processing, as well as motor preparation and execution, which can be utilized by neurotechnology based braking assistance systems.

Detection of braking intention in diverse situations during simulated driving based on EEG feature combination

Science.gov (United States)

Kim, Il-Hwa; Kim, Jeong-Woo; Haufe, Stefan; Lee, Seong-Whan

2015-02-01

Objective. We developed a simulated driving environment for studying neural correlates of emergency braking in diversified driving situations. We further investigated to what extent these neural correlates can be used to detect a participant's braking intention prior to the behavioral response. Approach. We measured electroencephalographic (EEG) and electromyographic signals during simulated driving. Fifteen participants drove a virtual vehicle and were exposed to several kinds of traffic situations in a simulator system, while EEG signals were measured. After that, we extracted characteristic features to categorize whether the driver intended to brake or not. Main results. Our system shows excellent detection performance in a broad range of possible emergency situations. In particular, we were able to distinguish three different kinds of emergency situations (sudden stop of a preceding vehicle, sudden cutting-in of a vehicle from the side and unexpected appearance of a pedestrian) from non-emergency (soft) braking situations, as well as from situations in which no braking was required, but the sensory stimulation was similar to stimulations inducing an emergency situation (e.g., the sudden stop of a vehicle on a neighboring lane). Significance. We proposed a novel feature combination comprising movement-related potentials such as the readiness potential, event-related desynchronization features besides the event-related potentials (ERP) features used in a previous study. The performance of predicting braking intention based on our proposed feature combination was superior compared to using only ERP features. Our study suggests that emergency situations are characterized by specific neural patterns of sensory perception and processing, as well as motor preparation and execution, which can be utilized by neurotechnology based braking assistance systems.

Functional Contour-following via Haptic Perception and Reinforcement Learning.

Science.gov (United States)

Hellman, Randall B; Tekin, Cem; van der Schaar, Mihaela; Santos, Veronica J

2018-01-01

Many tasks involve the fine manipulation of objects despite limited visual feedback. In such scenarios, tactile and proprioceptive feedback can be leveraged for task completion. We present an approach for real-time haptic perception and decision-making for a haptics-driven, functional contour-following task: the closure of a ziplock bag. This task is challenging for robots because the bag is deformable, transparent, and visually occluded by artificial fingertip sensors that are also compliant. A deep neural net classifier was trained to estimate the state of a zipper within a robot's pinch grasp. A Contextual Multi-Armed Bandit (C-MAB) reinforcement learning algorithm was implemented to maximize cumulative rewards by balancing exploration versus exploitation of the state-action space. The C-MAB learner outperformed a benchmark Q-learner by more efficiently exploring the state-action space while learning a hard-to-code task. The learned C-MAB policy was tested with novel ziplock bag scenarios and contours (wire, rope). Importantly, this work contributes to the development of reinforcement learning approaches that account for limited resources such as hardware life and researcher time. As robots are used to perform complex, physically interactive tasks in unstructured or unmodeled environments, it becomes important to develop methods that enable efficient and effective learning with physical testbeds.

Reward-based training of recurrent neural networks for cognitive and value-based tasks.

Science.gov (United States)

Song, H Francis; Yang, Guangyu R; Wang, Xiao-Jing

2017-01-13

Trained neural network models, which exhibit features of neural activity recorded from behaving animals, may provide insights into the circuit mechanisms of cognitive functions through systematic analysis of network activity and connectivity. However, in contrast to the graded error signals commonly used to train networks through supervised learning, animals learn from reward feedback on definite actions through reinforcement learning. Reward maximization is particularly relevant when optimal behavior depends on an animal's internal judgment of confidence or subjective preferences. Here, we implement reward-based training of recurrent neural networks in which a value network guides learning by using the activity of the decision network to predict future reward. We show that such models capture behavioral and electrophysiological findings from well-known experimental paradigms. Our work provides a unified framework for investigating diverse cognitive and value-based computations, and predicts a role for value representation that is essential for learning, but not executing, a task.

Cost efficient carbon fibre reinforced thermoplastics with in-situ polymerization of polyamide

Science.gov (United States)

Köhler, T.; Akdere, M.; Röding, T.; Gries, T.; Seide, G.

2017-10-01

Lightweight design has gained more and more relevance over the last decades. Especially in automotive industry it is of paramount importance to reduce weight and save fuel. At the same time the demand for safety and performance increases the components’ weight. To reach a trade-off between driving comfort and efficiency new lightweight materials have to be developed. One possible solution is the usage of carbon fibre reinforced thermoplastics (CFRTP) as a lightweight substitute material. In contrast to conventional carbon fibre reinforced plastics (CFRP), CFRTPs are cheaper and have a higher impact resistance. Furthermore they are characterized by hot forming ability, weldability and recyclability. However, the impregnation of the textile requires high pressure, because of the melted polymer’s high viscosity. A new innovative approach for CFRTP is the usage of in-situ polymerization with ɛ-caprolactam as matrix, which has a much lower viscosity and thus requires much lower pressure for impregnation and consolidation.

Management of Reinforcement Corrosion

DEFF Research Database (Denmark)

Küter, André; Geiker, Mette Rica; Møller, Per

Reinforcement corrosion is the most important cause for deterioration of reinforced concrete structures, both with regard to costs and consequences. Thermodynamically consistent descriptions of corrosion mechanisms are expected to allow the development of innovative concepts for the management...... of reinforcement corrosion....

Applications of neural networks to mechanics

International Nuclear Information System (INIS)

1997-01-01

Neural networks have become powerful tools in engineer's techniques. The aim of this conference was to present their application to concrete cases in the domain of mechanics, including the preparation and use of materials. Artificial neurons are non-linear organs which provide an output signal that depends on several differently weighted input signals. Their connection into networks allows to solve problems for which the driving laws are not well known. The applications discussed during this conference deal with: the driving of machines or processes, the control of machines, materials or products, the simulation and forecasting, and the optimization. Three papers dealing with the control of spark ignition engines, the regulation of heating floors and the optimization of energy consumptions in industrial buildings were selected for ETDE and one paper dealing with the optimization of the management of a reprocessed plutonium stock was selected for INIS. (J.S.)

Novel Adaptive Forward Neural MIMO NARX Model for the Identification of Industrial 3-DOF Robot Arm Kinematics

Directory of Open Access Journals (Sweden)

Ho Pham Huy Anh

2012-10-01

Full Text Available In this paper, a novel forward adaptive neural MIMO NARX model is used for modelling and identifying the forward kinematics of an industrial 3-DOF robot arm system. The nonlinear features of the forward kinematics of the industrial robot arm drive are thoroughly modelled based on the forward adaptive neural NARX model-based identification process using experimental input-output training data. This paper proposes a novel use of a back propagation (BP algorithm to generate the forward neural MIMO NARX (FNMN model for the forward kinematics of the industrial 3-DOF robot arm. The results show that the proposed adaptive neural NARX model trained by a Back Propagation learning algorithm yields outstanding performance and perfect accuracy.

Dynamic pricing and automated resource allocation for complex information services reinforcement learning and combinatorial auctions

CERN Document Server

Schwind, Michael; Fandel, G

2007-01-01

Many firms provide their customers with online information products which require limited resources such as server capacity. This book develops allocation mechanisms that aim to ensure an efficient resource allocation in modern IT-services. Recent methods of artificial intelligence, such as neural networks and reinforcement learning, and nature-oriented optimization methods, such as genetic algorithms and simulated annealing, are advanced and applied to allocation processes in distributed IT-infrastructures, e.g. grid systems. The author presents two methods, both of which using the users??? w

Neural networks for feedback feedforward nonlinear control systems.

Science.gov (United States)

Parisini, T; Zoppoli, R

1994-01-01

This paper deals with the problem of designing feedback feedforward control strategies to drive the state of a dynamic system (in general, nonlinear) so as to track any desired trajectory joining the points of given compact sets, while minimizing a certain cost function (in general, nonquadratic). Due to the generality of the problem, conventional methods are difficult to apply. Thus, an approximate solution is sought by constraining control strategies to take on the structure of multilayer feedforward neural networks. After discussing the approximation properties of neural control strategies, a particular neural architecture is presented, which is based on what has been called the "linear-structure preserving principle". The original functional problem is then reduced to a nonlinear programming one, and backpropagation is applied to derive the optimal values of the synaptic weights. Recursive equations to compute the gradient components are presented, which generalize the classical adjoint system equations of N-stage optimal control theory. Simulation results related to nonlinear nonquadratic problems show the effectiveness of the proposed method.

Adaptive Neural Network Sliding Mode Control for Quad Tilt Rotor Aircraft

Directory of Open Access Journals (Sweden)

Yanchao Yin

2017-01-01

Full Text Available A novel neural network sliding mode control based on multicommunity bidirectional drive collaborative search algorithm (M-CBDCS is proposed to design a flight controller for performing the attitude tracking control of a quad tilt rotors aircraft (QTRA. Firstly, the attitude dynamic model of the QTRA concerning propeller tension, channel arm, and moment of inertia is formulated, and the equivalent sliding mode control law is stated. Secondly, an adaptive control algorithm is presented to eliminate the approximation error, where a radial basis function (RBF neural network is used to online regulate the equivalent sliding mode control law, and the novel M-CBDCS algorithm is developed to uniformly update the unknown neural network weights and essential model parameters adaptively. The nonlinear approximation error is obtained and serves as a novel leakage term in the adaptations to guarantee the sliding surface convergence and eliminate the chattering phenomenon, which benefit the overall attitude control performance for QTRA. Finally, the appropriate comparisons among the novel adaptive neural network sliding mode control, the classical neural network sliding mode control, and the dynamic inverse PID control are examined, and comparative simulations are included to verify the efficacy of the proposed control method.

Dynamic Sensor Tasking for Space Situational Awareness via Reinforcement Learning

Science.gov (United States)

Linares, R.; Furfaro, R.

2016-09-01

This paper studies the Sensor Management (SM) problem for optical Space Object (SO) tracking. The tasking problem is formulated as a Markov Decision Process (MDP) and solved using Reinforcement Learning (RL). The RL problem is solved using the actor-critic policy gradient approach. The actor provides a policy which is random over actions and given by a parametric probability density function (pdf). The critic evaluates the policy by calculating the estimated total reward or the value function for the problem. The parameters of the policy action pdf are optimized using gradients with respect to the reward function. Both the critic and the actor are modeled using deep neural networks (multi-layer neural networks). The policy neural network takes the current state as input and outputs probabilities for each possible action. This policy is random, and can be evaluated by sampling random actions using the probabilities determined by the policy neural network's outputs. The critic approximates the total reward using a neural network. The estimated total reward is used to approximate the gradient of the policy network with respect to the network parameters. This approach is used to find the non-myopic optimal policy for tasking optical sensors to estimate SO orbits. The reward function is based on reducing the uncertainty for the overall catalog to below a user specified uncertainty threshold. This work uses a 30 km total position error for the uncertainty threshold. This work provides the RL method with a negative reward as long as any SO has a total position error above the uncertainty threshold. This penalizes policies that take longer to achieve the desired accuracy. A positive reward is provided when all SOs are below the catalog uncertainty threshold. An optimal policy is sought that takes actions to achieve the desired catalog uncertainty in minimum time. This work trains the policy in simulation by letting it task a single sensor to "learn" from its performance

DYNAMIC AND INCREMENTAL EXPLORATION STRATEGY IN FUSION ADAPTIVE RESONANCE THEORY FOR ONLINE REINFORCEMENT LEARNING

Directory of Open Access Journals (Sweden)

Budhitama Subagdja

2016-06-01

Full Text Available One of the fundamental challenges in reinforcement learning is to setup a proper balance between exploration and exploitation to obtain the maximum cummulative reward in the long run. Most protocols for exploration bound the overall values to a convergent level of performance. If new knowledge is inserted or the environment is suddenly changed, the issue becomes more intricate as the exploration must compromise the pre-existing knowledge. This paper presents a type of multi-channel adaptive resonance theory (ART neural network model called fusion ART which serves as a fuzzy approximator for reinforcement learning with inherent features that can regulate the exploration strategy. This intrinsic regulation is driven by the condition of the knowledge learnt so far by the agent. The model offers a stable but incremental reinforcement learning that can involve prior rules as bootstrap knowledge for guiding the agent to select the right action. Experiments in obstacle avoidance and navigation tasks demonstrate that in the configuration of learning wherein the agent learns from scratch, the inherent exploration model in fusion ART model is comparable to the basic E-greedy policy. On the other hand, the model is demonstrated to deal with prior knowledge and strike a balance between exploration and exploitation.

Effects of partial reinforcement and time between reinforced trials on terminal response rate in pigeon autoshaping.

Science.gov (United States)

Gottlieb, Daniel A

2006-03-01

Partial reinforcement often leads to asymptotically higher rates of responding and number of trials with a response than does continuous reinforcement in pigeon autoshaping. However, comparisons typically involve a partial reinforcement schedule that differs from the continuous reinforcement schedule in both time between reinforced trials and probability of reinforcement. Two experiments examined the relative contributions of these two manipulations to asymptotic response rate. Results suggest that the greater responding previously seen with partial reinforcement is primarily due to differential probability of reinforcement and not differential time between reinforced trials. Further, once established, differences in responding are resistant to a change in stimulus and contingency. Secondary response theories of autoshaped responding (theories that posit additional response-augmenting or response-attenuating mechanisms specific to partial or continuous reinforcement) cannot fully accommodate the current body of data. It is suggested that researchers who study pigeon autoshaping train animals on a common task prior to training them under different conditions.

Influence of transverse reinforcement on perforation resistance of reinforced concrete slabs under hard missile impact

International Nuclear Information System (INIS)

Orbovic, Nebojsa; Sagals, Genadijs; Blahoianu, Andrei

2015-01-01

This paper describes the work conducted by the Canadian Nuclear Safety Commission (CNSC) related to the influence of transverse reinforcement on perforation capacity of reinforced concrete (RC) slabs under “hard” missile impact (impact with negligible missile deformations). The paper presents the results of three tests on reinforced concrete slabs conducted at VTT Technical Research Centre (Finland), along with the numerical simulations as well as a discussion of the current code provisions related to impactive loading. Transverse reinforcement is widely used for improving the shear and punching strength of concrete structures. However, the effect of this reinforcement on the perforation resistance under localized missile impact is still unclear. The goal of this paper is to fill the gap in the current literature related to this topic. Based on similar tests designed by the authors with missile velocity below perforation velocity, it was expected that transverse reinforcement would improve the perforation resistance. Three slabs were tested under almost identical conditions with the only difference being the transverse reinforcement. One slab was designed without transverse reinforcement, the second one with the transverse reinforcement in form of conventional stirrups with hooks and the third one with the transverse reinforcement in form of T-headed bars. Although the transverse reinforcement reduced the overall damage of the slabs (the rear face scabbing), the conclusion from the tests is that the transverse reinforcement does not have important influence on perforation capacity of concrete slabs under rigid missile impact. The slab with T-headed bars presented a slight improvement compared to the baseline specimen without transverse reinforcement. The slab with conventional stirrups presented slightly lower perforation capacity (higher residual missile velocity) than the slab without transverse reinforcement. In conclusion, the performed tests show slightly

Investigation and modeling on protective textiles using artificial neural networks for defense applications

International Nuclear Information System (INIS)

Ramaiah, Gurumurthy B.; Chennaiah, Radhalakshmi Y.; Satyanarayanarao, Gurumurthy K.

2010-01-01

Kevlar 29 is a class of Kevlar fiber used for protective applications primarily by the military and law enforcement agencies for bullet resistant vests, hence for these reasons military has found that armors reinforced with Kevlar 29 multilayer fabrics which offer 25-40% better fragmentation resistance and provide better fit with greater comfort. The objective of this study is to investigate and develop an artificial neural network model for analyzing the performance of ballistic fabrics made from Kevlar 29 single layer fabrics using their material properties as inputs. Data from fragment simulation projectile (FSP) ballistic penetration measurements at 244 m/s has been used to demonstrate the modeling aspects of artificial neural networks. The neural network models demonstrated in this paper is based on back propagation (BP) algorithm which is inbuilt in MATLAB 7.1 software and is used for studies in science, technology and engineering. In the present research, comparisons are also made between the measured values of samples selected for building the neural network model and network predicted results. The analysis of the results for network predicted and experimental samples used in this study showed similarity.

Finite element modelling of concrete beams reinforced with hybrid fiber reinforced bars

Science.gov (United States)

Smring, Santa binti; Salleh, Norhafizah; Hamid, NoorAzlina Abdul; Majid, Masni A.

2017-11-01

Concrete is a heterogeneous composite material made up of cement, sand, coarse aggregate and water mixed in a desired proportion to obtain the required strength. Plain concrete does not with stand tension as compared to compression. In order to compensate this drawback steel reinforcement are provided in concrete. Now a day, for improving the properties of concrete and also to take up tension combination of steel and glass fibre-reinforced polymer (GFRP) bars promises favourable strength, serviceability, and durability. To verify its promise and support design concrete structures with hybrid type of reinforcement, this study have investigated the load-deflection behaviour of concrete beams reinforced with hybrid GFRP and steel bars by using ATENA software. Fourteen beams, including six control beams reinforced with only steel or only GFRP bars, were analysed. The ratio and the ordinate of GFRP to steel were the main parameters investigated. The behaviour of these beams was investigated via the load-deflection characteristics, cracking behaviour and mode of failure. Hybrid GFRP-Steel reinforced concrete beam showed the improvement in both ultimate capacity and deflection concomitant to the steel reinforced concrete beam. On the other hand, finite element (FE) modelling which is ATENA were validated with previous experiment and promising the good result to be used for further analyses and development in the field of present study.

Iterative free-energy optimization for recurrent neural networks (INFERNO)

Science.gov (United States)

2017-01-01

The intra-parietal lobe coupled with the Basal Ganglia forms a working memory that demonstrates strong planning capabilities for generating robust yet flexible neuronal sequences. Neurocomputational models however, often fails to control long range neural synchrony in recurrent spiking networks due to spontaneous activity. As a novel framework based on the free-energy principle, we propose to see the problem of spikes’ synchrony as an optimization problem of the neurons sub-threshold activity for the generation of long neuronal chains. Using a stochastic gradient descent, a reinforcement signal (presumably dopaminergic) evaluates the quality of one input vector to move the recurrent neural network to a desired activity; depending on the error made, this input vector is strengthened to hill-climb the gradient or elicited to search for another solution. This vector can be learned then by one associative memory as a model of the basal-ganglia to control the recurrent neural network. Experiments on habit learning and on sequence retrieving demonstrate the capabilities of the dual system to generate very long and precise spatio-temporal sequences, above two hundred iterations. Its features are applied then to the sequential planning of arm movements. In line with neurobiological theories, we discuss its relevance for modeling the cortico-basal working memory to initiate flexible goal-directed neuronal chains of causation and its relation to novel architectures such as Deep Networks, Neural Turing Machines and the Free-Energy Principle. PMID:28282439

Racial bias in neural empathic responses to pain.

Directory of Open Access Journals (Sweden)

Luis Sebastian Contreras-Huerta

Full Text Available Recent studies have shown that perceiving the pain of others activates brain regions in the observer associated with both somatosensory and affective-motivational aspects of pain, principally involving regions of the anterior cingulate and anterior insula cortex. The degree of these empathic neural responses is modulated by racial bias, such that stronger neural activation is elicited by observing pain in people of the same racial group compared with people of another racial group. The aim of the present study was to examine whether a more general social group category, other than race, could similarly modulate neural empathic responses and perhaps account for the apparent racial bias reported in previous studies. Using a minimal group paradigm, we assigned participants to one of two mixed-race teams. We use the term race to refer to the Chinese or Caucasian appearance of faces and whether the ethnic group represented was the same or different from the appearance of the participant' own face. Using fMRI, we measured neural empathic responses as participants observed members of their own group or other group, and members of their own race or other race, receiving either painful or non-painful touch. Participants showed clear group biases, with no significant effect of race, on behavioral measures of implicit (affective priming and explicit group identification. Neural responses to observed pain in the anterior cingulate cortex, insula cortex, and somatosensory areas showed significantly greater activation when observing pain in own-race compared with other-race individuals, with no significant effect of minimal groups. These results suggest that racial bias in neural empathic responses is not influenced by minimal forms of group categorization, despite the clear association participants showed with in-group more than out-group members. We suggest that race may be an automatic and unconscious mechanism that drives the initial neural responses to

Racial Bias in Neural Empathic Responses to Pain

Science.gov (United States)

Contreras-Huerta, Luis Sebastian; Baker, Katharine S.; Reynolds, Katherine J.; Batalha, Luisa; Cunnington, Ross

2013-01-01

Recent studies have shown that perceiving the pain of others activates brain regions in the observer associated with both somatosensory and affective-motivational aspects of pain, principally involving regions of the anterior cingulate and anterior insula cortex. The degree of these empathic neural responses is modulated by racial bias, such that stronger neural activation is elicited by observing pain in people of the same racial group compared with people of another racial group. The aim of the present study was to examine whether a more general social group category, other than race, could similarly modulate neural empathic responses and perhaps account for the apparent racial bias reported in previous studies. Using a minimal group paradigm, we assigned participants to one of two mixed-race teams. We use the term race to refer to the Chinese or Caucasian appearance of faces and whether the ethnic group represented was the same or different from the appearance of the participant' own face. Using fMRI, we measured neural empathic responses as participants observed members of their own group or other group, and members of their own race or other race, receiving either painful or non-painful touch. Participants showed clear group biases, with no significant effect of race, on behavioral measures of implicit (affective priming) and explicit group identification. Neural responses to observed pain in the anterior cingulate cortex, insula cortex, and somatosensory areas showed significantly greater activation when observing pain in own-race compared with other-race individuals, with no significant effect of minimal groups. These results suggest that racial bias in neural empathic responses is not influenced by minimal forms of group categorization, despite the clear association participants showed with in-group more than out-group members. We suggest that race may be an automatic and unconscious mechanism that drives the initial neural responses to observed pain in

Extended driving impairs nocturnal driving performances.

Directory of Open Access Journals (Sweden)

Patricia Sagaspe

Full Text Available Though fatigue and sleepiness at the wheel are well-known risk factors for traffic accidents, many drivers combine extended driving and sleep deprivation. Fatigue-related accidents occur mainly at night but there is no experimental data available to determine if the duration of prior driving affects driving performance at night. Participants drove in 3 nocturnal driving sessions (3-5 am, 1-5 am and 9 pm-5 am on open highway. Fourteen young healthy men (mean age [+/-SD] = 23.4 [+/-1.7] years participated Inappropriate line crossings (ILC in the last hour of driving of each session, sleep variables, self-perceived fatigue and sleepiness were measured. Compared to the short (3-5 am driving session, the incidence rate ratio of inappropriate line crossings increased by 2.6 (95% CI, 1.1 to 6.0; P<.05 for the intermediate (1-5 am driving session and by 4.0 (CI, 1.7 to 9.4; P<.001 for the long (9 pm-5 am driving session. Compared to the reference session (9-10 pm, the incidence rate ratio of inappropriate line crossings were 6.0 (95% CI, 2.3 to 15.5; P<.001, 15.4 (CI, 4.6 to 51.5; P<.001 and 24.3 (CI, 7.4 to 79.5; P<.001, respectively, for the three different durations of driving. Self-rated fatigue and sleepiness scores were both positively correlated to driving impairment in the intermediate and long duration sessions (P<.05 and increased significantly during the nocturnal driving sessions compared to the reference session (P<.01. At night, extended driving impairs driving performances and therefore should be limited.

Measures of coupling between neural populations based on Granger causality principle

Directory of Open Access Journals (Sweden)

Maciej Kaminski

2016-10-01

Full Text Available This paper shortly reviews the measures used to estimate neural synchronization in experimental settings. Our focus is on multivariate measures of dependence based on the Granger causality (G-causality principle, their applications and performance in respect of robustness to noise, volume conduction, common driving, and presence of a weak node. Application of G-causality measures to EEG, intracranial signals and fMRI time series is addressed. G-causality based measures defined in the frequency domain allow the synchronization between neural populations and the directed propagation of their electrical activity to be determined. The time-varying G-causality based measure Short-time Directed Transfer Function (SDTF supplies information on the dynamics of synchronization and the organization of neural networks. Inspection of effective connectivity patterns indicates a modular structure of neural networks, with a stronger coupling within modules than between them. The hypothetical plausible mechanism of information processing, suggested by the identified synchronization patterns, is communication between tightly coupled modules intermitted by sparser interactions providing synchronization of distant structures.

Intelligent Energy Management Control for Extended Range Electric Vehicles Based on Dynamic Programming and Neural Network

Directory of Open Access Journals (Sweden)

Lihe Xi

2017-11-01

Full Text Available The extended range electric vehicle (EREV can store much clean energy from the electric grid when it arrives at the charging station with lower battery energy. Consuming minimum gasoline during the trip is a common goal for most energy management controllers. To achieve these objectives, an intelligent energy management controller for EREV based on dynamic programming and neural networks (IEMC_NN is proposed. The power demand split ratio between the extender and battery are optimized by DP, and the control objectives are presented as a cost function. The online controller is trained by neural networks. Three trained controllers, constructing the controller library in IEMC_NN, are obtained from training three typical lengths of the driving cycle. To determine an appropriate NN controller for different driving distance purposes, the selection module in IEMC_NN is developed based on the remaining battery energy and the driving distance to the charging station. Three simulation conditions are adopted to validate the performance of IEMC_NN. They are target driving distance information, known and unknown, changing the destination during the trip. Simulation results using these simulation conditions show that the IEMC_NN had better fuel economy than the charging deplete/charging sustain (CD/CS algorithm. More significantly, with known driving distance information, the battery SOC controlled by IEMC_NN can just reach the lower bound as the EREV arrives at the charging station, which was also feasible when the driver changed the destination during the trip.

Progress of research on cytoskeleton and neural cell migration obstacle induced by ionizing radiation

International Nuclear Information System (INIS)

Qiu Jun; Wu Cuiping; Wang Mingming

2012-01-01

The dynamic changes of the microtubules and microfilaments provide the main force that drives the normal migration. Biological effects in tissues and cells induced by ionizing radiation are closely correlated with the changes happening to the cytoskeleton. It is that the ionizing radiation can induce the depolymeration of microfilaments and the assembly obstacles of microtubules, and make neural cell incapable of entering the model of migration or abnormally migrate. The effects of relevant changes of the cytoskeleton induced by irradiation on neural cell migration were discussed in this paper. (authors)

DRIVEN POLYSTRONG REINFORCED CONCRETE PILES AND NEW DESIGN OF PILE CAPS

Directory of Open Access Journals (Sweden)

I. I. Bekbasarov

2015-01-01

Full Text Available The paper presents constructional and technological features for manufacturing driven piles with variable strength of pile shaft. Economical efficiency of their production has been shown in the paper. The paper provides a pile cap design that ensures perception of hammer impacts with the help of lateral edges of the pile cap. Driven reinforced concrete piles which are manufactured from three shaft sections having various strength have been proposed in the paper. Material strength (concrete grade and diameter of bars and length of shaft sections are given on a case by case basis in accordance with nature and rate of stresses in piles during their driving process. Manufacturing of polystrong piles provides an opportunity to select them for a particular construction site with due account of their preservation during driving process.A pile cap has been developed that as opposed to existing analogous designs makes it possible to transmit impact efforts from a hammer to the pile through lateral surface of its head part. The pile cap provides the possibility to increase an area for perception of hammer impact efforts by the pile and in doing so it is possible significantly to reduce a damage risk and destruction of pile concrete during its driving. Application of polystrong piles and their driving with the help of new pile cap are considered as a basis for defect-free and resource-saving technology for pile foundations in the construction.

Fuzzylot: a novel self-organising fuzzy-neural rule-based pilot system for automated vehicles.

Science.gov (United States)

Pasquier, M; Quek, C; Toh, M

2001-10-01

This paper presents part of our research work concerned with the realisation of an Intelligent Vehicle and the technologies required for its routing, navigation, and control. An automated driver prototype has been developed using a self-organising fuzzy rule-based system (POPFNN-CRI(S)) to model and subsequently emulate human driving expertise. The ability of fuzzy logic to represent vague information using linguistic variables makes it a powerful tool to develop rule-based control systems when an exact working model is not available, as is the case of any vehicle-driving task. Designing a fuzzy system, however, is a complex endeavour, due to the need to define the variables and their associated fuzzy sets, and determine a suitable rule base. Many efforts have thus been devoted to automating this process, yielding the development of learning and optimisation techniques. One of them is the family of POP-FNNs, or Pseudo-Outer Product Fuzzy Neural Networks (TVR, AARS(S), AARS(NS), CRI, Yager). These generic self-organising neural networks developed at the Intelligent Systems Laboratory (ISL/NTU) are based on formal fuzzy mathematical theory and are able to objectively extract a fuzzy rule base from training data. In this application, a driving simulator has been developed, that integrates a detailed model of the car dynamics, complete with engine characteristics and environmental parameters, and an OpenGL-based 3D-simulation interface coupled with driving wheel and accelerator/ brake pedals. The simulator has been used on various road scenarios to record from a human pilot driving data consisting of steering and speed control actions associated to road features. Specifically, the POPFNN-CRI(S) system is used to cluster the data and extract a fuzzy rule base modelling the human driving behaviour. Finally, the effectiveness of the generated rule base has been validated using the simulator in autopilot mode.

Methods for producing reinforced carbon nanotubes

Science.gov (United States)

Ren, Zhifen [Newton, MA; Wen, Jian Guo [Newton, MA; Lao, Jing Y [Chestnut Hill, MA; Li, Wenzhi [Brookline, MA

2008-10-28

Methods for producing reinforced carbon nanotubes having a plurality of microparticulate carbide or oxide materials formed substantially on the surface of such reinforced carbon nanotubes composite materials are disclosed. In particular, the present invention provides reinforced carbon nanotubes (CNTs) having a plurality of boron carbide nanolumps formed substantially on a surface of the reinforced CNTs that provide a reinforcing effect on CNTs, enabling their use as effective reinforcing fillers for matrix materials to give high-strength composites. The present invention also provides methods for producing such carbide reinforced CNTs.

Synchronization of Switched Interval Networks and Applications to Chaotic Neural Networks

Directory of Open Access Journals (Sweden)

Jinde Cao

2013-01-01

Full Text Available This paper investigates synchronization problem of switched delay networks with interval parameters uncertainty, based on the theories of the switched systems and drive-response technique, a mathematical model of the switched interval drive-response error system is established. Without constructing Lyapunov-Krasovskii functions, introducing matrix measure method for the first time to switched time-varying delay networks, combining Halanay inequality technique, synchronization criteria are derived for switched interval networks under the arbitrary switching rule, which are easy to verify in practice. Moreover, as an application, the proposed scheme is then applied to chaotic neural networks. Finally, numerical simulations are provided to illustrate the effectiveness of the theoretical results.

Early Model of Traffic Sign Reminder Based on Neural Network

Directory of Open Access Journals (Sweden)

Budi Rahmani

2012-12-01

Full Text Available Recognizing the traffic signs installed on the streets is one of the requirements of driving on the road. Laxity in driving may result in traffic accident. This paper describes a real-time reminder model, by utilizing a camera that can be installed in a car to capture image of traffic signs, and is processed and later to inform the driver. The extracting feature harnessing the morphological elements (strel is used in this paper. Artificial Neural Networks is used to train the system and to produce a final decision. The result shows that the accuracy in detecting and recognizing the ten types of traffic signs in real-time is 80%.

Visibility Enhancement of Scene Images Degraded by Foggy Weather Conditions with Deep Neural Networks

Directory of Open Access Journals (Sweden)

Farhan Hussain

2016-01-01

Full Text Available Nowadays many camera-based advanced driver assistance systems (ADAS have been introduced to assist the drivers and ensure their safety under various driving conditions. One of the problems faced by drivers is the faded scene visibility and lower contrast while driving in foggy conditions. In this paper, we present a novel approach to provide a solution to this problem by employing deep neural networks. We assume that the fog in an image can be mathematically modeled by an unknown complex function and we utilize the deep neural network to approximate the corresponding mathematical model for the fog. The advantages of our technique are as follows: (i its real-time operation and (ii being based on minimal input, that is, a single image, and exhibiting robustness/generalization for various unseen image data. Experiments carried out on various synthetic images indicate that our proposed technique has the abilities to approximate the corresponding fog function reasonably and remove it for better visibility and safety.

Peer Passenger Norms and Pressure: Experimental Effects on Simulated Driving Among Teenage Males.

Science.gov (United States)

Bingham, C Raymond; Simons-Morton, Bruce G; Pradhan, Anuj K; Li, Kaigang; Almani, Farideh; Falk, Emily B; Shope, Jean T; Buckley, Lisa; Ouimet, Marie Claude; Albert, Paul S

2016-08-01

Serious crashes are more likely when teenage drivers have teenage passengers. One likely source of this increased risk is social influences on driving performance. This driving simulator study experimentally tested the effects of peer influence (i.e., risk-accepting compared to risk-averse peer norms reinforced by pressure) on the driving risk behavior (i.e., risky driving behavior and inattention to hazards) of male teenagers. It was hypothesized that peer presence would result in greater driving risk behavior (i.e., increased driving risk and reduced latent hazard anticipation), and that the effect would be greater when the peer was risk-accepting. Fifty-three 16- and 17-year-old male participants holding a provisional U.S., State of Michigan driver license were randomized to either a risk-accepting or risk-averse condition. Each participant operated a driving simulator while alone and separately with a confederate peer passenger. The simulator world included scenarios designed to elicit variation in driving risk behavior with a teen passenger present in the vehicle. Significant interactions of passenger presence (passenger present vs. alone) by risk condition (risk-accepting vs. risk-averse) were observed for variables measuring: failure to stop at yellow light intersections (Incident Rate Ratio (IRR)=2.16; 95% Confidence Interval [95CI]=1.06, 4.43); higher probability of overtaking (IRR=10.17; 95CI=1.43, 73.35); shorter left turn latency (IRR=0.43; 95CI=0.31,0.60); and, failure to stop at an intersection with an occluded stop sign (IRR=7.90; 95CI=2.06,30.35). In all cases, greater risky driving by participants was more likely with a risk-accepting passenger versus a risk-averse passenger present and a risk-accepting passenger present versus driving alone. Exposure of male teenagers to a risk-accepting confederate peer passenger who applied peer influence increased simulated risky driving behavior compared with exposure to a risk-averse confederate peer

Carbon Fiber Reinforced Polymer Grids for Shear and End Zone Reinforcement in Bridge Beams

Science.gov (United States)

2018-01-01

Corrosion of reinforcing steel reduces life spans of bridges throughout the United States; therefore, using non-corroding carbon fiber reinforced polymer (CFRP) reinforcement is seen as a way to increase service life. The use of CFRP as the flexural ...

On the synchronization of neural networks containing time-varying delays and sector nonlinearity

International Nuclear Information System (INIS)

Yan, J.-J.; Lin, J.-S.; Hung, M.-L.; Liao, T.-L.

2007-01-01

We present a systematic design procedure for synchronization of neural networks subject to time-varying delays and sector nonlinearity in the control input. Based on the drive-response concept and the Lyapunov stability theorem, a memoryless decentralized control law is proposed which guarantees exponential synchronization even when input nonlinearity is present. The supplementary requirement that the time-derivative of time-varying delays must be smaller than one is released for the proposed control scheme. A four-dimensional Hopfield neural network with time-varying delays is presented as the illustrative example to demonstrate the effectiveness of the proposed synchronization scheme

Techniques for extracting single-trial activity patterns from large-scale neural recordings

Science.gov (United States)

Churchland, Mark M; Yu, Byron M; Sahani, Maneesh; Shenoy, Krishna V

2008-01-01

Summary Large, chronically-implanted arrays of microelectrodes are an increasingly common tool for recording from primate cortex, and can provide extracellular recordings from many (order of 100) neurons. While the desire for cortically-based motor prostheses has helped drive their development, such arrays also offer great potential to advance basic neuroscience research. Here we discuss the utility of array recording for the study of neural dynamics. Neural activity often has dynamics beyond that driven directly by the stimulus. While governed by those dynamics, neural responses may nevertheless unfold differently for nominally identical trials, rendering many traditional analysis methods ineffective. We review recent studies – some employing simultaneous recording, some not – indicating that such variability is indeed present both during movement generation, and during the preceding premotor computations. In such cases, large-scale simultaneous recordings have the potential to provide an unprecedented view of neural dynamics at the level of single trials. However, this enterprise will depend not only on techniques for simultaneous recording, but also on the use and further development of analysis techniques that can appropriately reduce the dimensionality of the data, and allow visualization of single-trial neural behavior. PMID:18093826

Cortical drive to breathe in amyotrophic lateral sclerosis: a dyspnoea-worsening defence?

Science.gov (United States)

Georges, Marjolaine; Morawiec, Elise; Raux, Mathieu; Gonzalez-Bermejo, Jésus; Pradat, Pierre-François; Similowski, Thomas; Morélot-Panzini, Capucine

2016-06-01

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease causing diaphragm weakness that can be partially compensated by inspiratory neck muscle recruitment. This disappears during sleep, which is compatible with a cortical contribution to the drive to breathe. We hypothesised that ALS patients with respiratory failure exhibit respiratory-related cortical activity, relieved by noninvasive ventilation (NIV) and related to dyspnoea.We studied 14 ALS patients with respiratory failure. Electroencephalographic recordings (EEGs) and electromyographic recordings of inspiratory neck muscles were performed during spontaneous breathing and NIV. Dyspnoea was evaluated using the Multidimensional Dyspnea Profile.Eight patients exhibited slow EEG negativities preceding inspiration (pre-inspiratory potentials) during spontaneous breathing. Pre-inspiratory potentials were attenuated during NIV (p=0.04). Patients without pre-inspiratory potentials presented more advanced forms of ALS and more severe respiratory impairment, but less severe dyspnoea. Patients with pre-inspiratory potentials had stronger inspiratory neck muscle activation and more severe dyspnoea during spontaneous breathing.ALS-related diaphragm weakness can engage cortical resources to augment the neural drive to breathe. This might reflect a compensatory mechanism, with the intensity of dyspnoea a negative consequence. Disease progression and the corresponding neural loss could abolish this phenomenon. A putative cognitive cost should be investigated. Copyright ©ERS 2016.

Reinforcement Learning–Based Energy Management Strategy for a Hybrid Electric Tracked Vehicle

Directory of Open Access Journals (Sweden)

Teng Liu

2015-07-01

Full Text Available This paper presents a reinforcement learning (RL–based energy management strategy for a hybrid electric tracked vehicle. A control-oriented model of the powertrain and vehicle dynamics is first established. According to the sample information of the experimental driving schedule, statistical characteristics at various velocities are determined by extracting the transition probability matrix of the power request. Two RL-based algorithms, namely Q-learning and Dyna algorithms, are applied to generate optimal control solutions. The two algorithms are simulated on the same driving schedule, and the simulation results are compared to clarify the merits and demerits of these algorithms. Although the Q-learning algorithm is faster (3 h than the Dyna algorithm (7 h, its fuel consumption is 1.7% higher than that of the Dyna algorithm. Furthermore, the Dyna algorithm registers approximately the same fuel consumption as the dynamic programming–based global optimal solution. The computational cost of the Dyna algorithm is substantially lower than that of the stochastic dynamic programming.

Reinforcement learning for optimal control of low exergy buildings

International Nuclear Information System (INIS)

Yang, Lei; Nagy, Zoltan; Goffin, Philippe; Schlueter, Arno

2015-01-01

Highlights: • Implementation of reinforcement learning control for LowEx Building systems. • Learning allows adaptation to local environment without prior knowledge. • Presentation of reinforcement learning control for real-life applications. • Discussion of the applicability for real-life situations. - Abstract: Over a third of the anthropogenic greenhouse gas (GHG) emissions stem from cooling and heating buildings, due to their fossil fuel based operation. Low exergy building systems are a promising approach to reduce energy consumption as well as GHG emissions. They consists of renewable energy technologies, such as PV, PV/T and heat pumps. Since careful tuning of parameters is required, a manual setup may result in sub-optimal operation. A model predictive control approach is unnecessarily complex due to the required model identification. Therefore, in this work we present a reinforcement learning control (RLC) approach. The studied building consists of a PV/T array for solar heat and electricity generation, as well as geothermal heat pumps. We present RLC for the PV/T array, and the full building model. Two methods, Tabular Q-learning and Batch Q-learning with Memory Replay, are implemented with real building settings and actual weather conditions in a Matlab/Simulink framework. The performance is evaluated against standard rule-based control (RBC). We investigated different neural network structures and find that some outperformed RBC already during the learning phase. Overall, every RLC strategy for PV/T outperformed RBC by over 10% after the third year. Likewise, for the full building, RLC outperforms RBC in terms of meeting the heating demand, maintaining the optimal operation temperature and compensating more effectively for ground heat. This allows to reduce engineering costs associated with the setup of these systems, as well as decrease the return-of-invest period, both of which are necessary to create a sustainable, zero-emission building

Sensorless Speed/Torque Control of DC Machine Using Artificial Neural Network Technique

Directory of Open Access Journals (Sweden)

Rakan Kh. Antar

2017-12-01

Full Text Available In this paper, Artificial Neural Network (ANN technique is implemented to improve speed and torque control of a separately excited DC machine drive. The speed and torque sensorless scheme based on ANN is estimated adaptively. The proposed controller is designed to estimate rotor speed and mechanical load torque as a Model Reference Adaptive System (MRAS method for DC machine. The DC drive system consists of four quadrant DC/DC chopper with MOSFET transistors, ANN, logic gates and routing circuits. The DC drive circuit is designed, evaluated and modeled by Matlab/Simulink in the forward and reverse operation modes as a motor and generator, respectively. The DC drive system is simulated at different speed values (±1200 rpm and mechanical torque (±7 N.m in steady state and dynamic conditions. The simulation results illustratethe effectiveness of the proposed controller without speed or torque sensors.

Continual and One-Shot Learning Through Neural Networks with Dynamic External Memory

DEFF Research Database (Denmark)

Lüders, Benno; Schläger, Mikkel; Korach, Aleksandra

2017-01-01

it easier to find unused memory location and therefor facilitates the evolution of continual learning networks. Our results suggest that augmenting evolving networks with an external memory component is not only a viable mechanism for adaptive behaviors in neuroevolution but also allows these networks...... a new task is learned. This paper takes a step in overcoming this limitation by building on the recently proposed Evolving Neural Turing Machine (ENTM) approach. In the ENTM, neural networks are augmented with an external memory component that they can write to and read from, which allows them to store...... associations quickly and over long periods of time. The results in this paper demonstrate that the ENTM is able to perform one-shot learning in reinforcement learning tasks without catastrophic forgetting of previously stored associations. Additionally, we introduce a new ENTM default jump mechanism that makes...

The neural basis of responsibility attribution in decision-making.

Science.gov (United States)

Li, Peng; Shen, Yue; Sui, Xue; Chen, Changming; Feng, Tingyong; Li, Hong; Holroyd, Clay

2013-01-01

Social responsibility links personal behavior with societal expectations and plays a key role in affecting an agent's emotional state following a decision. However, the neural basis of responsibility attribution remains unclear. In two previous event-related brain potential (ERP) studies we found that personal responsibility modulated outcome evaluation in gambling tasks. Here we conducted a functional magnetic resonance imaging (fMRI) study to identify particular brain regions that mediate responsibility attribution. In a context involving team cooperation, participants completed a task with their teammates and on each trial received feedback about team success and individual success sequentially. We found that brain activity differed between conditions involving team success vs. team failure. Further, different brain regions were associated with reinforcement of behavior by social praise vs. monetary reward. Specifically, right temporoparietal junction (RTPJ) was associated with social pride whereas dorsal striatum and dorsal anterior cingulate cortex (ACC) were related to reinforcement of behaviors leading to personal gain. The present study provides evidence that the RTPJ is an important region for determining whether self-generated behaviors are deserving of praise in a social context.

Combining noncontingent reinforcement and differential reinforcement schedules as treatment for aberrant behavior.

OpenAIRE

Marcus, B A; Vollmer, T R

1996-01-01

Research has shown that noncontingent reinforcement (NCR) can be an effective behavior-reduction procedure when based on a functional analysis. The effects of NCR may be a result of elimination of the contingency between aberrant behavior and reinforcing consequences (extinction) or frequent and free access to reinforcers that may reduce the participant's motivation to engage in aberrant behaviors or mands. If motivation is momentarily reduced, behavior such as mands may not be sensitive to p...

Spinal interneurons differentiate sequentially from those driving the fastest swimming movements in larval zebrafish to those driving the slowest ones.

Science.gov (United States)

McLean, David L; Fetcho, Joseph R

2009-10-28

Studies of neuronal networks have revealed few general principles that link patterns of development with later functional roles. While investigating the neural control of movements, we recently discovered a topographic map in the spinal cord of larval zebrafish that relates the position of motoneurons and interneurons to their order of recruitment during swimming. Here, we show that the map reflects an orderly pattern of differentiation of neurons driving different movements. First, we use high-speed filming to show that large-amplitude swimming movements with bending along much of the body appear first, with smaller, regional swimming movements emerging later. Next, using whole-cell patch recordings, we demonstrate that the excitatory circuits that drive large-amplitude, fast swimming movements at larval stages are present and functional early on in embryos. Finally, we systematically assess the orderly emergence of spinal circuits according to swimming speed using transgenic fish expressing the photoconvertible protein Kaede to track neuronal differentiation in vivo. We conclude that a simple principle governs the development of spinal networks in which the neurons driving the fastest, most powerful swimming in larvae develop first with ones that drive increasingly weaker and slower larval movements layered on over time. Because the neurons are arranged by time of differentiation in the spinal cord, the result is a topographic map that represents the speed/strength of movements at which neurons are recruited and the temporal emergence of networks. This pattern may represent a general feature of neuronal network development throughout the brain and spinal cord.

Opponent appetitive-aversive neural processes underlie predictive learning of pain relief.

Science.gov (United States)

Seymour, Ben; O'Doherty, John P; Koltzenburg, Martin; Wiech, Katja; Frackowiak, Richard; Friston, Karl; Dolan, Raymond

2005-09-01

Termination of a painful or unpleasant event can be rewarding. However, whether the brain treats relief in a similar way as it treats natural reward is unclear, and the neural processes that underlie its representation as a motivational goal remain poorly understood. We used fMRI (functional magnetic resonance imaging) to investigate how humans learn to generate expectations of pain relief. Using a pavlovian conditioning procedure, we show that subjects experiencing prolonged experimentally induced pain can be conditioned to predict pain relief. This proceeds in a manner consistent with contemporary reward-learning theory (average reward/loss reinforcement learning), reflected by neural activity in the amygdala and midbrain. Furthermore, these reward-like learning signals are mirrored by opposite aversion-like signals in lateral orbitofrontal cortex and anterior cingulate cortex. This dual coding has parallels to 'opponent process' theories in psychology and promotes a formal account of prediction and expectation during pain.

Degradation of Waterfront Reinforced Concrete Structures

African Journals Online (AJOL)

Key words: Degradation, reinforced concrete, Dar es Salaam port. Abstract—One of the ... especially corrosion of the reinforcement. ... Corrosion of steel reinforcement contributes .... cracks along the line of reinforcement bars and most of the ...

Fiber breakage phenomena in long fiber reinforced plastic preparation

International Nuclear Information System (INIS)

Huang, Chao-Tsai; Tseng, Huan-Chang; Chang, Rong-Yeu; Vlcek, Jiri

2015-01-01

Due to the high demand of smart green, the lightweight technologies have become the driving force for the development of automotives and other industries in recent years. Among those technologies, using short and long fiber-reinforced plastics (FRP) to replace some metal components can reduce the weight of an automotive significantly. However, the microstructures of fibers inside plastic matrix are too complicated to manage and control during the injection molding through the screw, the runner, the gate, and then into the cavity. This study focuses on the fiber breakage phenomena during the screw plastification. Results show that fiber breakage is strongly dependent on screw design and operation. When the screw geometry changes, the fiber breakage could be larger even with lower compression ratio. (paper)

Confinement of Reinforced-Concrete Columns with Non-Code Compliant Confining Reinforcement plus Supplemental Pen-Binder

Directory of Open Access Journals (Sweden)

Anang Kristianto

2012-11-01

Full Text Available One of the important requirements for earthquake resistant building related to confinement is the use of seismic hooks in the hoop or confining reinforcement of reinforced-concrete column elements. However, installation of a confining reinforcement with a 135-degree hook is not easy. Therefore, in practice, many construction workers apply a confining reinforcement with a 90-degreehook (non-code compliant. Based on research and records of recent earthquakes in Indonesia, the use of a non-code compliant confining reinforcement for concrete columns produces structures with poor seismic performance. This paper presents a study that introduces an additional element that is expected to improve the effectiveness of concrete columns confined with a non-code compliant confining reinforcement. The additional element, named a pen-binder, is used to keep the non-code compliant confining reinforcement in place. The effectiveness of this element under pure axial concentric loading was investigatedcomprehensively.The specimens tested in this study were 18 concrete columns,with a cross-section of 170 mm x 170 mm and a height of 480 mm. The main test variables were the material type of the pen-binder, the angle of the hook, and the confining reinforcement configuration.The test results indicate that adding pen-binders can effectively improve the strength and ductility of the column specimens confined with a non-code compliant confining reinforcement

Influence of reinforcement's corrosion into hyperstatic reinforced concrete beams: a probabilistic failure scenarios analysis

Directory of Open Access Journals (Sweden)

G. P. PELLIZZER

Full Text Available AbstractThis work aims to study the mechanical effects of reinforcement's corrosion in hyperstatic reinforced concrete beams. The focus is the probabilistic determination of individual failure scenarios change as well as global failure change along time. The limit state functions assumed describe analytically bending and shear resistance of reinforced concrete rectangular cross sections as a function of steel and concrete resistance and section dimensions. It was incorporated empirical laws that penalize the steel yield stress and the reinforcement's area along time in addition to Fick's law, which models the chloride penetration into concrete pores. The reliability theory was applied based on Monte Carlo simulation method, which assesses each individual probability of failure. The probability of global structural failure was determined based in the concept of failure tree. The results of a hyperstatic reinforced concrete beam showed that reinforcements corrosion make change into the failure scenarios modes. Therefore, unimportant failure modes in design phase become important after corrosion start.

Neural Network Control-Based Drive Design of Servomotor and Its Application to Automatic Guided Vehicle

Directory of Open Access Journals (Sweden)

Ming-Shyan Wang

2015-01-01

Full Text Available An automatic guided vehicle (AGV is extensively used for productions in a flexible manufacture system with high efficiency and high flexibility. A servomotor-based AGV is designed and implemented in this paper. In order to steer the AGV to go along a predefined path with corner or arc, the conventional proportional-integral-derivative (PID control is used in the system. However, it is difficult to tune PID gains at various conditions. As a result, the neural network (NN control is considered to assist the PID control for gain tuning. The experimental results are first provided to verify the correctness of the neural network plus PID control for 400 W-motor control system. Secondly, the AGV includes two sets of the designed motor systems and CAN BUS transmission so that it can move along the straight line and curve paths shown in the taped videos.

Learning Similar Actions by Reinforcement or Sensory-Prediction Errors Rely on Distinct Physiological Mechanisms.

Science.gov (United States)

Uehara, Shintaro; Mawase, Firas; Celnik, Pablo

2017-09-14

Humans can acquire knowledge of new motor behavior via different forms of learning. The two forms most commonly studied have been the development of internal models based on sensory-prediction errors (error-based learning) and success-based feedback (reinforcement learning). Human behavioral studies suggest these are distinct learning processes, though the neurophysiological mechanisms that are involved have not been characterized. Here, we evaluated physiological markers from the cerebellum and the primary motor cortex (M1) using noninvasive brain stimulations while healthy participants trained finger-reaching tasks. We manipulated the extent to which subjects rely on error-based or reinforcement by providing either vector or binary feedback about task performance. Our results demonstrated a double dissociation where learning the task mainly via error-based mechanisms leads to cerebellar plasticity modifications but not long-term potentiation (LTP)-like plasticity changes in M1; while learning a similar action via reinforcement mechanisms elicited M1 LTP-like plasticity but not cerebellar plasticity changes. Our findings indicate that learning complex motor behavior is mediated by the interplay of different forms of learning, weighing distinct neural mechanisms in M1 and the cerebellum. Our study provides insights for designing effective interventions to enhance human motor learning. © The Author 2017. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.

Exponential H(infinity) synchronization of general discrete-time chaotic neural networks with or without time delays.

Science.gov (United States)

Qi, Donglian; Liu, Meiqin; Qiu, Meikang; Zhang, Senlin

2010-08-01

This brief studies exponential H(infinity) synchronization of a class of general discrete-time chaotic neural networks with external disturbance. On the basis of the drive-response concept and H(infinity) control theory, and using Lyapunov-Krasovskii (or Lyapunov) functional, state feedback controllers are established to not only guarantee exponential stable synchronization between two general chaotic neural networks with or without time delays, but also reduce the effect of external disturbance on the synchronization error to a minimal H(infinity) norm constraint. The proposed controllers can be obtained by solving the convex optimization problems represented by linear matrix inequalities. Most discrete-time chaotic systems with or without time delays, such as Hopfield neural networks, cellular neural networks, bidirectional associative memory networks, recurrent multilayer perceptrons, Cohen-Grossberg neural networks, Chua's circuits, etc., can be transformed into this general chaotic neural network to be H(infinity) synchronization controller designed in a unified way. Finally, some illustrated examples with their simulations have been utilized to demonstrate the effectiveness of the proposed methods.

Finite element modeling of reinforced concrete beams with a hybrid combination of steel and aramid reinforcement

International Nuclear Information System (INIS)

Hawileh, R.A.

2015-01-01

Highlights: • Modeling of concrete beams reinforced steel and FRP bars. • Developed finite element models achieved good results. • The models are validated via comparison with experimental results. • Parametric studies are performed. - Abstract: Corrosion of steel bars has an adverse effect on the life-span of reinforced concrete (RC) members and is usually associated with crack development in RC beams. Fiber reinforced polymer (FRP) bars have been recently used to reinforce concrete members in flexure due to their high tensile strength and superior corrosion resistance properties. However, FRP materials are brittle in nature, thus RC beams reinforced with such materials would exhibit a less ductile behavior when compared to similar members reinforced with conventional steel reinforcement. Recently, researchers investigated the performance of concrete beams reinforced with a hybrid combination of steel and Aramid Fiber Reinforced Polymer (AFRP) reinforcement to maintain a reasonable level of ductility in such members. The function of the AFRP bars is to increase the load-carrying capacity, while the function of the steel bars is to ensure ductility of the flexural member upon yielding in tension. This paper presents a three-dimensional (3D) finite element (FE) model that predicted the load versus mid-span deflection response of tested RC beams conducted by other researchers with a hybrid combination of steel and AFRP bars. The developed FE models account for the constituent material nonlinearities and bond–slip behavior between the reinforcing bars and adjacent concrete surfaces. It was concluded that the developed models can accurately capture the behavior and predicts the load-carrying capacity of such RC members. In addition, a parametric study is conducted using the validated models to investigate the effect of AFRP bar size, FRP material type, bond–slip action, and concrete compressive strength on the performance of concrete beams when reinforced

Health monitoring of precast bridge deck panels reinforced with glass fiber reinforced polymer (GFRP) bars.

Science.gov (United States)

2012-03-01

The present research project investigates monitoring concrete precast panels for bridge decks that are reinforced with Glass Fiber Reinforced Polymer (GFRP) bars. Due to the lack of long term research on concrete members reinforced with GFRP bars, lo...

Caregiver preference for reinforcement-based interventions for problem behavior maintained by positive reinforcement.

Science.gov (United States)

Gabor, Anne M; Fritz, Jennifer N; Roath, Christopher T; Rothe, Brittany R; Gourley, Denise A

2016-06-01

Social validity of behavioral interventions typically is assessed with indirect methods or by determining preferences of the individuals who receive treatment, and direct observation of caregiver preference rarely is described. In this study, preferences of 5 caregivers were determined via a concurrent-chains procedure. Caregivers were neurotypical, and children had been diagnosed with developmental disabilities and engaged in problem behavior maintained by positive reinforcement. Caregivers were taught to implement noncontingent reinforcement (NCR), differential reinforcement of alternative behavior (DRA), and differential reinforcement of other behavior (DRO), and the caregivers selected interventions to implement during sessions with the child after they had demonstrated proficiency in implementing the interventions. Three caregivers preferred DRA, 1 caregiver preferred differential reinforcement procedures, and 1 caregiver did not exhibit a preference. Direct observation of implementation in concurrent-chains procedures may allow the identification of interventions that are implemented with sufficient integrity and preferred by caregivers. © 2016 Society for the Experimental Analysis of Behavior.

New directions at UNISOR and the importance of reinforcing spherical and deformed shell gaps

International Nuclear Information System (INIS)

Hamilton, J.H.

1985-01-01

An on-line nuclear orientation facility under construction for UNISOR is described. The strong competition between shell gaps at spherical, prolate and oblate deformation is shown to give rise to various structures from spherical double closed shell, to coexisting near-spherical and deformed shapes to deformed double closed shell nuclei in the region of A = 70-104. The importance of the reinforcing of the shape driving forces when the nucleus has shell gaps for the protons and neutrons at the same deformation on nuclear shapes and the switching of magic numbers is described

Effects of reinforcer magnitude on responding under differential-reinforcement-of-low-rate schedules of rats and pigeons.

Science.gov (United States)

Doughty, Adam H; Richards, Jerry B

2002-07-01

Experiment I investigated the effects of reinforcer magnitude on differential-reinforcement-of-low-rate (DRL) schedule performance in three phases. In Phase 1, two groups of rats (n = 6 and 5) responded under a DRI. 72-s schedule with reinforcer magnitudes of either 30 or 300 microl of water. After acquisition, the water amounts were reversed for each rat. In Phase 2, the effects of the same reinforcer magnitudes on DRL 18-s schedule performance were examined across conditions. In Phase 3, each rat responded unider a DR1. 18-s schedule in which the water amotnts alternated between 30 and 300 microl daily. Throughout each phase of Experiment 1, the larger reinforcer magnitude resulted in higher response rates and lower reinforcement rates. The peak of the interresponse-time distributions was at a lower value tinder the larger reinforcer magnitude. In Experiment 2, 3 pigeons responded under a DRL 20-s schedule in which reinforcer magnitude (1-s or 6-s access to grain) varied iron session to session. Higher response rates and lower reinforcement rates occurred tinder the longer hopper duration. These results demonstrate that larger reinforcer magnitudes engender less efficient DRL schedule performance in both rats and pigeons, and when reinforcer magnitude was held constant between sessions or was varied daily. The present results are consistent with previous research demonstrating a decrease in efficiency as a function of increased reinforcer magnituide tinder procedures that require a period of time without a specified response. These findings also support the claim that DRI. schedule performance is not governed solely by a timing process.

Australia's long-term electricity demand forecasting using deep neural networks

OpenAIRE

Hamedmoghadam, Homayoun; Joorabloo, Nima; Jalili, Mahdi

2018-01-01

Accurate prediction of long-term electricity demand has a significant role in demand side management and electricity network planning and operation. Demand over-estimation results in over-investment in network assets, driving up the electricity prices, while demand under-estimation may lead to under-investment resulting in unreliable and insecure electricity. In this manuscript, we apply deep neural networks to predict Australia's long-term electricity demand. A stacked autoencoder is used in...

Aging differentially affects male and female neural stem cell neurogenic properties

Directory of Open Access Journals (Sweden)

Jay Waldron

2010-09-01

Full Text Available Jay Waldron1, Althea McCourty1, Laurent Lecanu1,21The Research Institute of the McGill University Health Centre, Montreal, Canada; 2Department of Medicine, McGill University, Montreal, Quebec, CanadaPurpose: Neural stem cell transplantation as a brain repair strategy is a very promising technology. However, despite many attempts, the clinical success remains very deceiving. Despite clear evidence that sexual dimorphism rules many aspects of human biology, the occurrence of a sex difference in neural stem cell biology is largely understudied. Herein, we propose to determine whether gender is a dimension that drives the fate of neural stem cells through aging. Should it occur, we believe that neural stem cell sexual dimorphism and its variation during aging should be taken into account to refine clinical approaches of brain repair strategies.Methods: Neural stem cells were isolated from the subventricular zone of three- and 20-month-old male and female Long-Evans rats. Expression of the estrogen receptors, ERα and ERβ, progesterone receptor, androgen receptor, and glucocorticoid receptor was analyzed and quantified by Western blotting on undifferentiated neural stem cells. A second set of neural stem cells was treated with retinoic acid to trigger differentiation, and the expression of neuronal, astroglial, and oligodendroglial markers was determined using Western blotting.Conclusion: We provided in vitro evidence that the fate of neural stem cells is affected by sex and aging. Indeed, young male neural stem cells mainly expressed markers of neuronal and oligodendroglial fate, whereas young female neural stem cells underwent differentiation towards an astroglial phenotype. Aging resulted in a lessened capacity to express neuron and astrocyte markers. Undifferentiated neural stem cells displayed sexual dimorphism in the expression of steroid receptors, in particular ERα and ERβ, and the expression level of several steroid receptors increased

The usage of carbon fiber reinforcement polymer and glass fiber reinforcement polymer for retrofit technology building

Science.gov (United States)

Tarigan, Johannes; Meka, Randi; Nursyamsi

2018-03-01

Fiber Reinforcement Polymer has been used as a material technology since the 1970s in Europe. Fiber Reinforcement Polymer can reinforce the structure externally, and used in many types of buildings like beams, columns, and slabs. It has high tensile strength. Fiber Reinforcement Polymer also has high rigidity and strength. The profile of Fiber Reinforcement Polymer is thin and light, installation is simple to conduct. One of Fiber Reinforcement Polymer material is Carbon Fiber Reinforcement Polymer and Glass Fiber Reinforcement Polymer. These materials is tested when it is installed on concrete cylinders, to obtain the comparison of compressive strength CFRP and GFRP. The dimension of concrete is diameter of 15 cm and height of 30 cm. It is amounted to 15 and divided into three groups. The test is performed until it collapsed to obtain maximum load. The results of research using CFRP and GFRP have shown the significant enhancement in compressive strength. CFRP can increase the compressive strength of 26.89%, and GFRP of 14.89%. For the comparison of two materials, CFRP is more strengthening than GFRP regarding increasing compressive strength. The usage of CFRP and GFRP can increase the loading capacity.

Research on FBG-Based CFRP Structural Damage Identification Using BP Neural Network

Science.gov (United States)

Geng, Xiangyi; Lu, Shizeng; Jiang, Mingshun; Sui, Qingmei; Lv, Shanshan; Xiao, Hang; Jia, Yuxi; Jia, Lei

2018-06-01

A damage identification system of carbon fiber reinforced plastics (CFRP) structures is investigated using fiber Bragg grating (FBG) sensors and back propagation (BP) neural network. FBG sensors are applied to construct the sensing network to detect the structural dynamic response signals generated by active actuation. The damage identification model is built based on the BP neural network. The dynamic signal characteristics extracted by the Fourier transform are the inputs, and the damage states are the outputs of the model. Besides, damages are simulated by placing lumped masses with different weights instead of inducing real damages, which is confirmed to be feasible by finite element analysis (FEA). At last, the damage identification system is verified on a CFRP plate with 300 mm × 300 mm experimental area, with the accurate identification of varied damage states. The system provides a practical way for CFRP structural damage identification.

Synchronization stability of memristor-based complex-valued neural networks with time delays.

Science.gov (United States)

Liu, Dan; Zhu, Song; Ye, Er

2017-12-01

This paper focuses on the dynamical property of a class of memristor-based complex-valued neural networks (MCVNNs) with time delays. By constructing the appropriate Lyapunov functional and utilizing the inequality technique, sufficient conditions are proposed to guarantee exponential synchronization of the coupled systems based on drive-response concept. The proposed results are very easy to verify, and they also extend some previous related works on memristor-based real-valued neural networks. Meanwhile, the obtained sufficient conditions of this paper may be conducive to qualitative analysis of some complex-valued nonlinear delayed systems. A numerical example is given to demonstrate the effectiveness of our theoretical results. Copyright © 2017 Elsevier Ltd. All rights reserved.

A Neural Marker for Social Bias Toward In-group Accents.

Science.gov (United States)

Bestelmeyer, Patricia E G; Belin, Pascal; Ladd, D Robert

2015-10-01

Accents provide information about the speaker's geographical, socio-economic, and ethnic background. Research in applied psychology and sociolinguistics suggests that we generally prefer our own accent to other varieties of our native language and attribute more positive traits to it. Despite the widespread influence of accents on social interactions, educational and work settings the neural underpinnings of this social bias toward our own accent and, what may drive this bias, are unexplored. We measured brain activity while participants from two different geographical backgrounds listened passively to 3 English accent types embedded in an adaptation design. Cerebral activity in several regions, including bilateral amygdalae, revealed a significant interaction between the participants' own accent and the accent they listened to: while repetition of own accents elicited an enhanced neural response, repetition of the other group's accent resulted in reduced responses classically associated with adaptation. Our findings suggest that increased social relevance of, or greater emotional sensitivity to in-group accents, may underlie the own-accent bias. Our results provide a neural marker for the bias associated with accents, and show, for the first time, that the neural response to speech is partly shaped by the geographical background of the listener. © The Author 2014. Published by Oxford University Press.

Reinforced concrete tomography

International Nuclear Information System (INIS)

Mariscotti, M.A.J.; Morixe, M.; Tarela, P.A.; Thieberger, P.

1997-01-01

In this paper we describe the technique of reinforced concrete tomography, its historical background, recent technological developments and main applications. Gamma radiation sensitive plates are imprinted with radiation going through the concrete sample under study, and then processed to reveal the presence of reinforcement and defects in the material density. The three dimensional reconstruction, or tomography, of the reinforcement out of a single gammagraphy is an original development alternative to conventional methods. Re-bar diameters and positions may be determined with an accuracy of ± 1 mm 0.5-1 cm, respectively. The non-destructive character of this technique makes it particularly attractive in cases of inhabited buildings and diagnoses of balconies. (author) [es

TRIGA control rod position and reactivity transient Monitoring by Neural Networks

International Nuclear Information System (INIS)

Rosa, R.; Palomba, M.; Sepielli, M.

2008-01-01

Plant sensors drift or malfunction and operator actions in nuclear reactor control can be supported by sensor on-line monitoring, and data validation through soft-computing process. On-line recalibration can often avoid manual calibration or drifting component replacement. DSP requires prompt response to the modified conditions. Artificial Neural Network (ANN) and Fuzzy logic ensure: prompt response, link with field measurement and physical system behaviour, data incoming interpretation, and detection of discrepancy for mis-calibration or sensor faults. ANN (Artificial Neural Network) is a system based on the operation of biological neural networks. Although computing is day by day advancing, there are certain tasks that a program made for a common microprocessor is unable to perform. A software implementation of an ANN can be made with Pros and Cons. Pros: A neural network can perform tasks that a linear program can not; When an element of the neural network fails, it can continue without any problem by their parallel nature; A neural network learns and does not need to be reprogrammed; It can be implemented in any application; It can be implemented without any problem. Cons: The architecture of a neural network is different from the architecture of microprocessors therefore needs to be emulated; it requires high processing time for large neural networks; and the neural network needs training to operate. Three possibilities of training exist: Supervised learning: the network is trained providing input and matching output patterns; Unsupervised learning: input patterns are not a priori classified and the system must develop its own representation of the input stimuli; Reinforcement Learning: intermediate form of the above two types of learning, the learning machine does some action on the environment and gets a feedback response from the environment. Two TRIGAN ANN applications are considered: control rod position and fuel temperature. The outcome obtained in this

Dynamics of neuronal circuits in addiction: reward, antireward, and emotional memory.

Science.gov (United States)

Koob, G F

2009-05-01

Drug addiction is conceptualized as chronic, relapsing compulsive use of drugs with significant dysregulation of brain hedonic systems. Compulsive drug use is accompanied by decreased function of brain substrates for drug positive reinforcement and recruitment of brain substrates mediating the negative reinforcement of motivational withdrawal. The neural substrates for motivational withdrawal ("dark side" of addiction) involve recruitment of elements of the extended amygdala and the brain stress systems, including corticotropin-releasing factor and norepinephrine. These changes, combined with decreased reward function, are hypothesized to persist in the form of an allostatic state that forms a powerful motivational background for relapse. Relapse also involves a key role for the basolateral amygdala in mediating the motivational effects of stimuli previously paired with drug seeking and drug motivational withdrawal. The basolateral amygdala has a key role in mediating emotional memories in general. The hypothesis argued here is that brain stress systems activated by the motivational consequences of drug withdrawal can not only form the basis for negative reinforcement that drives drug seeking, but also potentiate associative mechanisms that perpetuate the emotional state and help drive the allostatic state of addiction.

Detection of an inhibitory cortical gradient underlying peak shift in learning: a neural basis for a false memory.

Science.gov (United States)

Miasnikov, Alexandre A; Weinberger, Norman M

2012-11-01

Experience often does not produce veridical memory. Understanding false attribution of events constitutes an important problem in memory research. "Peak shift" is a well-characterized, controllable phenomenon in which human and animal subjects that receive reinforcement associated with one sensory stimulus later respond maximally to another stimulus in post-training stimulus generalization tests. Peak shift ordinarily develops in discrimination learning (reinforced CS+, unreinforced CS-) and has long been attributed to the interaction of an excitatory gradient centered on the CS+ and an inhibitory gradient centered on the CS-; the shift is away from the CS-. In contrast, we have obtained peak shifts during single tone frequency training, using stimulation of the cholinergic nucleus basalis (NB) to implant behavioral memory into the rat. As we also recorded cortical activity, we took the opportunity to investigate the possible existence of a neural frequency gradient that could account for behavioral peak shift. Behavioral frequency generalization gradients (FGGs, interruption of ongoing respiration) were determined twice before training while evoked potentials were recorded from the primary auditory cortex (A1), to obtain a baseline gradient of "habituatory" neural decrement. A post-training behavioral FGG obtained 24h after three daily sessions of a single tone paired with NB stimulation (200 trials/day) revealed a peak shift. The peak of the FGG was at a frequency lower than the CS while the cortical inhibitory gradient was at a frequency higher than the CS frequency. Further analysis indicated that the frequency location and magnitude of the gradient could account for the behavioral peak shift. These results provide a neural basis for a systematic case of memory misattribution and may provide an animal model for the study of the neural bases of a type of "false memory". Published by Elsevier Inc.

Characterization of Pax3 and Sox10 transgenic Xenopus laevis embryos as tools to study neural crest development.

Science.gov (United States)

Alkobtawi, Mansour; Ray, Heather; Barriga, Elias H; Moreno, Mauricio; Kerney, Ryan; Monsoro-Burq, Anne-Helene; Saint-Jeannet, Jean-Pierre; Mayor, Roberto

2018-03-06

The neural crest is a multipotent population of cells that originates a variety of cell types. Many animal models are used to study neural crest induction, migration and differentiation, with amphibians and birds being the most widely used systems. A major technological advance to study neural crest development in mouse, chick and zebrafish has been the generation of transgenic animals in which neural crest specific enhancers/promoters drive the expression of either fluorescent proteins for use as lineage tracers, or modified genes for use in functional studies. Unfortunately, no such transgenic animals currently exist for the amphibians Xenopus laevis and tropicalis, key model systems for studying neural crest development. Here we describe the generation and characterization of two transgenic Xenopus laevis lines, Pax3-GFP and Sox10-GFP, in which GFP is expressed in the pre-migratory and migratory neural crest, respectively. We show that Pax3-GFP could be a powerful tool to study neural crest induction, whereas Sox10-GFP could be used in the study of neural crest migration in living embryos. Copyright © 2018 The Authors. Published by Elsevier Inc. All rights reserved.

Identifying Method of Drunk Driving Based on Driving Behavior

Directory of Open Access Journals (Sweden)

Xiaohua Zhao

2011-05-01

Full Text Available Drunk driving is one of the leading causes contributing to traffic crashes. There are numerous issues that need to be resolved with the current method of identifying drunk driving. Driving behavior, with the characteristic of real-time, was extensively researched to identify impaired driving behaviors. In this paper, the drives with BACs above 0.05% were defined as drunk driving state. A detailed comparison was made between normal driving and drunk driving. The experiment in driving simulator was designed to collect the driving performance data of the groups. According to the characteristics analysis for the effect of alcohol on driving performance, seven significant indicators were extracted and the drunk driving was identified by the Fisher Discriminant Method. The discriminant function demonstrated a high accuracy of classification. The optimal critical score to differentiate normal from drinking state was found to be 0. The evaluation result verifies the accuracy of classification method.

Prediction of reinforcement corrosion using corrosion induced cracks width in corroded reinforced concrete beams

International Nuclear Information System (INIS)

Khan, Inamullah; François, Raoul; Castel, Arnaud

2014-01-01

This paper studies the evolution of reinforcement corrosion in comparison to corrosion crack width in a highly corroded reinforced concrete beam. Cracking and corrosion maps of the beam were drawn and steel reinforcement was recovered from the beam to observe the corrosion pattern and to measure the loss of mass of steel reinforcement. Maximum steel cross-section loss of the main reinforcement and average steel cross-section loss between stirrups were plotted against the crack width. The experimental results were compared with existing models proposed by Rodriguez et al., Vidal et al. and Zhang et al. Time prediction models for a given opening threshold are also compared to experimental results. Steel cross-section loss for stirrups was also measured and was plotted against the crack width. It was observed that steel cross-section loss in the stirrups had no relationship with the crack width of longitudinal corrosion cracks. -- Highlights: •Relationship between crack and corrosion of reinforcement was investigated. •Corrosion results of natural process and then corresponds to in-situ conditions. •Comparison with time predicting model is provided. •Prediction of load-bearing capacity from crack pattern was studied

Study of the internal confinement of concrete reinforced (in civil engineering) with woven reinforcement

Science.gov (United States)

Dalal, M.; Goumairi, O.; El Malik, A.

2017-10-01

Concrete is generally the most used material in the field of construction. Despite its extensive use in structures, it represents some drawbacks related to its properties including its low tensile strength and low ductility. To solve this problem, the use of steel reinforcement in concrete structures is possible. Another possibility is the introduction of different types of continuous fibre / staple in the concrete, such as steel fibres or synthetic fibres, to obtain ″Concretes bundles″. Many types of fibre concrete, which have been developed and for many of them, the gain provided by the fibre was rather low and no significant improvement in tensile strength was really reaching. By cons, the ductility was higher than that of ordinary concrete. The objective of this study is to examine concrete reinforcement by inserting reinforcements woven polyester. These are either woven bidirectional (2D) or three-dimensional woven (3D). So we will report the properties of each type of reinforcement and the influence of the method of weaving on the strength reinforcements and on the strength of concrete in which they are incorporated. Such influence should contribute to improving the sustainability and enhancement of reinforcement

Blunted perception of neural respiratory drive and breathlessness in patients with cystic fibrosis

Directory of Open Access Journals (Sweden)

Charles C. Reilly

2016-03-01

Full Text Available The electromyogram recorded from the diaphragm (EMGdi and parasternal intercostal muscle using surface electrodes (sEMGpara provides a measure of neural respiratory drive (NRD, the magnitude of which reflects lung disease severity in stable cystic fibrosis. The aim of this study was to explore perception of NRD and breathlessness in both healthy individuals and patients with cystic fibrosis. Given chronic respiratory loading and increased NRD in cystic fibrosis, often in the absence of breathlessness at rest, we hypothesised that patients with cystic fibrosis would be able to tolerate higher levels of NRD for a given level of breathlessness compared to healthy individuals during exercise. 15 cystic fibrosis patients (mean forced expiratory volume in 1 s (FEV1 53.5% predicted and 15 age-matched, healthy controls were studied. Spirometry was measured in all subjects and lung volumes measured in the cystic fibrosis patients. EMGdi and sEMGpara were recorded at rest and during incremental cycle exercise to exhaustion and expressed as a percentage of maximum (% max obtained from maximum respiratory manoeuvres. Borg breathlessness scores were recorded at rest and during each minute of exercise. EMGdi % max and sEMGpara % max and associated Borg breathlessness scores differed significantly between healthy subjects and cystic fibrosis patients at rest and during exercise. The relationship between EMGdi % max and sEMGpara % max and Borg score was shifted to the right in the cystic fibrosis patients, such that at comparable levels of EMGdi % max and sEMGpara % max the cystic fibrosis patients reported significantly lower Borg breathlessness scores compared to the healthy individuals. At Borg score 1 (clinically significant increase in breathlessness from baseline corresponding levels of EMGdi % max (20.2±12% versus 32.15±15%, p=0.02 and sEMGpara % max (18.9±8% versus 29.2±15%, p=0.04 were lower in the healthy individuals compared to the cystic

Driving the brain towards creativity and intelligence: A network control theory analysis.

Science.gov (United States)

Kenett, Yoed N; Medaglia, John D; Beaty, Roger E; Chen, Qunlin; Betzel, Richard F; Thompson-Schill, Sharon L; Qiu, Jiang

2018-01-04

High-level cognitive constructs, such as creativity and intelligence, entail complex and multiple processes, including cognitive control processes. Recent neurocognitive research on these constructs highlight the importance of dynamic interaction across neural network systems and the role of cognitive control processes in guiding such a dynamic interaction. How can we quantitatively examine the extent and ways in which cognitive control contributes to creativity and intelligence? To address this question, we apply a computational network control theory (NCT) approach to structural brain imaging data acquired via diffusion tensor imaging in a large sample of participants, to examine how NCT relates to individual differences in distinct measures of creative ability and intelligence. Recent application of this theory at the neural level is built on a model of brain dynamics, which mathematically models patterns of inter-region activity propagated along the structure of an underlying network. The strength of this approach is its ability to characterize the potential role of each brain region in regulating whole-brain network function based on its anatomical fingerprint and a simplified model of node dynamics. We find that intelligence is related to the ability to "drive" the brain system into easy to reach neural states by the right inferior parietal lobe and lower integration abilities in the left retrosplenial cortex. We also find that creativity is related to the ability to "drive" the brain system into difficult to reach states by the right dorsolateral prefrontal cortex (inferior frontal junction) and higher integration abilities in sensorimotor areas. Furthermore, we found that different facets of creativity-fluency, flexibility, and originality-relate to generally similar but not identical network controllability processes. We relate our findings to general theories on intelligence and creativity. Copyright © 2018 Elsevier Ltd. All rights reserved.

Synchronization of cellular neural networks of neutral type via dynamic feedback controller

International Nuclear Information System (INIS)

Park, Ju H.

2009-01-01

In this paper, we aim to study global synchronization for neural networks with neutral delay. A dynamic feedback control scheme is proposed to achieve the synchronization between drive network and response network. By utilizing the Lyapunov function and linear matrix inequalities (LMIs), we derive simple and efficient criterion in terms of LMIs for synchronization. The feedback controllers can be easily obtained by solving the derived LMIs.

From Creatures of Habit to Goal-Directed Learners: Tracking the Developmental Emergence of Model-Based Reinforcement Learning.

Science.gov (United States)

Decker, Johannes H; Otto, A Ross; Daw, Nathaniel D; Hartley, Catherine A

2016-06-01

Theoretical models distinguish two decision-making strategies that have been formalized in reinforcement-learning theory. A model-based strategy leverages a cognitive model of potential actions and their consequences to make goal-directed choices, whereas a model-free strategy evaluates actions based solely on their reward history. Research in adults has begun to elucidate the psychological mechanisms and neural substrates underlying these learning processes and factors that influence their relative recruitment. However, the developmental trajectory of these evaluative strategies has not been well characterized. In this study, children, adolescents, and adults performed a sequential reinforcement-learning task that enabled estimation of model-based and model-free contributions to choice. Whereas a model-free strategy was apparent in choice behavior across all age groups, a model-based strategy was absent in children, became evident in adolescents, and strengthened in adults. These results suggest that recruitment of model-based valuation systems represents a critical cognitive component underlying the gradual maturation of goal-directed behavior. © The Author(s) 2016.

The Reinforcing Event (RE) Menu

Science.gov (United States)

Addison, Roger M.; Homme, Lloyd E.

1973-01-01

A motivational system, the Contingency Management System, uses contracts in which some amount of defined task behavior is demanded for some interval of reinforcing event. The Reinforcing Event Menu, a list of high probability reinforcing behaviors, is used in the system as a prompting device for the learner and as an aid for the administrator in…

Self-rated driving and driving safety in older adults.

Science.gov (United States)

Ross, Lesley A; Dodson, Joan E; Edwards, Jerri D; Ackerman, Michelle L; Ball, Karlene

2012-09-01

Many U.S. states rely on older adults to self-regulate their driving and determine when driving is no longer a safe option. However, the relationship of older adults' self-rated driving in terms of actual driving competency outcomes is unclear. The current study investigates self-rated driving in terms of (1) systematic differences between older adults with high (good/excellent) versus low (poor/fair/average) self-ratings, and (2) the predictive nature of self-rated driving to adverse driving outcomes in older adults (n=350; mean age 73.9, SD=5.25, range 65-91). Adverse driving outcomes included self-reported incidences of (1) being pulled over by the police, (2) receiving a citation, (3) receiving a recommendation to cease or limit driving, (4) crashes, and (5) state-reported crashes. Results found that older drivers with low self-ratings reported more medical conditions, less driving frequency, and had been given more suggestions to stop/limit their driving; there were no other significant differences between low and high self-raters. Logistic regression revealed older drivers were more likely to have a state-reported crash and receive a suggestion to stop or limit driving. Men were more likely to report all adverse driving outcomes except for receiving a suggestion to stop or limit driving. Regarding self-rated driving, older adults with high ratings were 66% less likely (OR=0.34, 95% CI=0.14-0.85) to have received suggestions to limit or stop driving after accounting for demographics, health and driving frequency. Self-ratings were not predictive of other driving outcomes (being pulled over by the police, receiving a citation, self-reported crashes, or state-reported crashes, ps>0.05). Most older drivers (85.14%) rated themselves as either good or excellent drivers regardless of their actual previous citation or crash rates. Self-rated driving is likely not related to actual driving proficiency as indicated by previous crash involvement in older adults

Steel fiber reinforced concrete

International Nuclear Information System (INIS)

Baloch, S.U.

2005-01-01

Steel-Fiber Reinforced Concrete is constructed by adding short fibers of small cross-sectional size .to the fresh concrete. These fibers reinforce the concrete in all directions, as they are randomly oriented. The improved mechanical properties of concrete include ductility, impact-resistance, compressive, tensile and flexural strength and abrasion-resistance. These uniqlte properties of the fiber- reinforcement can be exploited to great advantage in concrete structural members containing both conventional bar-reinforcement and steel fibers. The improvements in mechanical properties of cementitious materials resulting from steel-fiber reinforcement depend on the type, geometry, volume fraction and material-properties of fibers, the matrix mix proportions and the fiber-matrix interfacial bond characteristics. Effects of steel fibers on the mechanical properties of concrete have been investigated in this paper through a comprehensive testing-programme, by varying the fiber volume fraction and the aspect-ratio (Lid) of fibers. Significant improvements are observed in compressive, tensile, flexural strength and impact-resistance of concrete, accompanied by marked improvement in ductility. optimum fiber-volume fraction and aspect-ratio of steel fibers is identified. Test results are analyzed in details and relevant conclusions drawn. The research is finally concluded with future research needs. (author)

Corrosion of reinforcement bars in steel ibre reinforced concrete structures

DEFF Research Database (Denmark)

Solgaard, Anders Ole Stubbe

and the influence of steel fibres on initiation and propagation of cracks in concrete. Moreover, the impact of fibres on corrosion-induced cover cracking was covered. The impact of steel fibres on propagation of reinforcement corrosion was investigated through studies of their impact on the electrical resistivity...... of concrete, which is known to affect the corrosion process of embedded reinforcement. The work concerning the impact of steel fibres on initiation and propagation of cracks was linked to corrosion initiation and propagation of embedded reinforcement bars via additional studies. Cracks in the concrete cover...... are known to alter the ingress rate of depassivating substances and thereby influence the corrosion process. The Ph.D. study covered numerical as well as experimental studies. Electrochemically passive steel fibres are electrically isolating thus not changing the electrical resistivity of concrete, whereas...

The Drive-Wise Project: Driving Simulator Training increases real driving performance in healthy older drivers

Directory of Open Access Journals (Sweden)

Gianclaudio eCasutt

2014-05-01

Full Text Available Background: Age-related cognitive decline is often associated with unsafe driving behavior. We hypothesized that 10 active training sessions in a driving simulator increase cognitive and on-road driving performance. In addition, driving simulator training should outperform cognitive training.Methods: Ninety-one healthy active drivers (62 – 87 years were randomly assigned to either (1 a driving simulator training group, (2 an attention training group (vigilance and selective attention, or (3 a control group. The main outcome variables were on-road driving and cognitive performance. Seventy-seven participants (85% completed the training and were included in the analyses. Training gains were analyzed using a multiple regression analysis with planned comparisons.Results: The driving simulator training group showed an improvement in on-road driving performance compared to the attention training group. In addition, both training groups increased cognitive performance compared to the control group. Conclusion: Driving simulator training offers the potential to enhance driving skills in older drivers. Compared to the attention training, the simulator training seems to be a more powerful program for increasing older drivers’ safety on the road.

Algorithms for Reinforcement Learning

CERN Document Server

Szepesvari, Csaba

2010-01-01

Reinforcement learning is a learning paradigm concerned with learning to control a system so as to maximize a numerical performance measure that expresses a long-term objective. What distinguishes reinforcement learning from supervised learning is that only partial feedback is given to the learner about the learner's predictions. Further, the predictions may have long term effects through influencing the future state of the controlled system. Thus, time plays a special role. The goal in reinforcement learning is to develop efficient learning algorithms, as well as to understand the algorithms'

Electrical drives for direct drive renewable energy systems

CERN Document Server

Mueller, Markus

2013-01-01

Wind turbine gearboxes present major reliability issues, leading to great interest in the current development of gearless direct-drive wind energy systems. Offering high reliability, high efficiency and low maintenance, developments in these direct-drive systems point the way to the next generation of wind power, and Electrical drives for direct drive renewable energy systems is an authoritative guide to their design, development and operation. Part one outlines electrical drive technology, beginning with an overview of electrical generators for direct drive systems. Principles of electrical design for permanent magnet generators are discussed, followed by electrical, thermal and structural generator design and systems integration. A review of power electronic converter technology and power electronic converter systems for direct drive renewable energy applications is then conducted. Part two then focuses on wind and marine applications, beginning with a commercial overview of wind turbine drive systems and a...

Self-rated Driving and Driving Safety in Older Adults

OpenAIRE

Ross, Lesley A.; Dodson, Joan; Edwards, Jerri D.; Ackerman, Michelle L.; Ball, Karlene

2012-01-01

Many U.S. states rely on older adults to self-regulate their driving and determine when driving is no longer a safe option. However, the relationship of older adults’ self-rated driving in terms of actual driving competency outcomes is unclear. The current study investigates self-rated driving in terms of (1) systematic differences between older adults with high (good/excellent) versus low (poor/fair/average) self-ratings, and (2) the predictive nature of self-rated driving to adverse driving...

The neural basis of responsibility attribution in decision-making.

Directory of Open Access Journals (Sweden)

Peng Li

Full Text Available Social responsibility links personal behavior with societal expectations and plays a key role in affecting an agent's emotional state following a decision. However, the neural basis of responsibility attribution remains unclear. In two previous event-related brain potential (ERP studies we found that personal responsibility modulated outcome evaluation in gambling tasks. Here we conducted a functional magnetic resonance imaging (fMRI study to identify particular brain regions that mediate responsibility attribution. In a context involving team cooperation, participants completed a task with their teammates and on each trial received feedback about team success and individual success sequentially. We found that brain activity differed between conditions involving team success vs. team failure. Further, different brain regions were associated with reinforcement of behavior by social praise vs. monetary reward. Specifically, right temporoparietal junction (RTPJ was associated with social pride whereas dorsal striatum and dorsal anterior cingulate cortex (ACC were related to reinforcement of behaviors leading to personal gain. The present study provides evidence that the RTPJ is an important region for determining whether self-generated behaviors are deserving of praise in a social context.

The role of multisensor data fusion in neuromuscular control of a sagittal arm with a pair of muscles using actor-critic reinforcement learning method.

Science.gov (United States)

Golkhou, V; Parnianpour, M; Lucas, C

2004-01-01

In this study, we consider the role of multisensor data fusion in neuromuscular control using an actor-critic reinforcement learning method. The model we use is a single link system actuated by a pair of muscles that are excited with alpha and gamma signals. Various physiological sensor information such as proprioception, spindle sensors, and Golgi tendon organs have been integrated to achieve an oscillatory movement with variable amplitude and frequency, while achieving a stable movement with minimum metabolic cost and coactivation. The system is highly nonlinear in all its physical and physiological attributes. Transmission delays are included in the afferent and efferent neural paths to account for a more accurate representation of the reflex loops. This paper proposes a reinforcement learning method with an Actor-Critic architecture instead of middle and low level of central nervous system (CNS). The Actor in this structure is a two layer feedforward neural network and the Critic is a model of the cerebellum. The Critic is trained by the State-Action-Reward-State-Action (SARSA) method. The Critic will train the Actor by supervisory learning based on previous experiences. The reinforcement signal in SARSA is evaluated based on available alternatives concerning the concept of multisensor data fusion. The effectiveness and the biological plausibility of the present model are demonstrated by several simulations. The system showed excellent tracking capability when we integrated the available sensor information. Addition of a penalty for activation of muscles resulted in much lower muscle coactivation while keeping the movement stable.

Feedback amplification loop drives malignant growth in epithelial tissues.

Science.gov (United States)

Muzzopappa, Mariana; Murcia, Lada; Milán, Marco

2017-08-29

Interactions between cells bearing oncogenic mutations and the surrounding microenvironment, and cooperation between clonally distinct cell populations, can contribute to the growth and malignancy of epithelial tumors. The genetic techniques available in Drosophila have contributed to identify important roles of the TNF-α ligand Eiger and mitogenic molecules in mediating these interactions during the early steps of tumor formation. Here we unravel the existence of a tumor-intrinsic-and microenvironment-independent-self-reinforcement mechanism that drives tumor initiation and growth in an Eiger-independent manner. This mechanism relies on cell interactions between two functionally distinct cell populations, and we present evidence that these cell populations are not necessarily genetically different. Tumor-specific and cell-autonomous activation of the tumorigenic JNK stress-activated pathway drives the expression of secreted signaling molecules and growth factors to delaminating cells, which nonautonomously promote proliferative growth of the partially transformed epithelial tissue. We present evidence that cross-feeding interactions between delaminating and nondelaminating cells increase each other's sizes and that these interactions can explain the unlimited growth potential of these tumors. Our results will open avenues toward our molecular understanding of those social cell interactions with a relevant function in tumor initiation in humans.

Deep reinforcement learning for automated radiation adaptation in lung cancer.

Science.gov (United States)

Tseng, Huan-Hsin; Luo, Yi; Cui, Sunan; Chien, Jen-Tzung; Ten Haken, Randall K; Naqa, Issam El

2017-12-01

To investigate deep reinforcement learning (DRL) based on historical treatment plans for developing automated radiation adaptation protocols for nonsmall cell lung cancer (NSCLC) patients that aim to maximize tumor local control at reduced rates of radiation pneumonitis grade 2 (RP2). In a retrospective population of 114 NSCLC patients who received radiotherapy, a three-component neural networks framework was developed for deep reinforcement learning (DRL) of dose fractionation adaptation. Large-scale patient characteristics included clinical, genetic, and imaging radiomics features in addition to tumor and lung dosimetric variables. First, a generative adversarial network (GAN) was employed to learn patient population characteristics necessary for DRL training from a relatively limited sample size. Second, a radiotherapy artificial environment (RAE) was reconstructed by a deep neural network (DNN) utilizing both original and synthetic data (by GAN) to estimate the transition probabilities for adaptation of personalized radiotherapy patients' treatment courses. Third, a deep Q-network (DQN) was applied to the RAE for choosing the optimal dose in a response-adapted treatment setting. This multicomponent reinforcement learning approach was benchmarked against real clinical decisions that were applied in an adaptive dose escalation clinical protocol. In which, 34 patients were treated based on avid PET signal in the tumor and constrained by a 17.2% normal tissue complication probability (NTCP) limit for RP2. The uncomplicated cure probability (P+) was used as a baseline reward function in the DRL. Taking our adaptive dose escalation protocol as a blueprint for the proposed DRL (GAN + RAE + DQN) architecture, we obtained an automated dose adaptation estimate for use at ∼2/3 of the way into the radiotherapy treatment course. By letting the DQN component freely control the estimated adaptive dose per fraction (ranging from 1-5 Gy), the DRL automatically favored dose

Continuous Reinforced Concrete Beams

DEFF Research Database (Denmark)

Hoang, Cao Linh; Nielsen, Mogens Peter

1996-01-01

This report deals with stress and stiffness estimates of continuous reinforced concrete beams with different stiffnesses for negative and positive moments e.g. corresponding to different reinforcement areas in top and bottom. Such conditions are often met in practice.The moment distribution...

Neural changes underlying early stages of L2 vocabulary acquisition.

Science.gov (United States)

Pu, He; Holcomb, Phillip J; Midgley, Katherine J

2016-11-01

Research has shown neural changes following second language (L2) acquisition after weeks or months of instruction. But are such changes detectable even earlier than previously shown? The present study examines the electrophysiological changes underlying the earliest stages of second language vocabulary acquisition by recording event-related potentials (ERPs) within the first week of learning. Adult native English speakers with no previous Spanish experience completed less than four hours of Spanish vocabulary training, with pre- and post-training ERPs recorded to a backward translation task. Results indicate that beginning L2 learners show rapid neural changes following learning, manifested in changes to the N400 - an ERP component sensitive to lexicosemantic processing and degree of L2 proficiency. Specifically, learners in early stages of L2 acquisition show growth in N400 amplitude to L2 words following learning as well as a backward translation N400 priming effect that was absent pre-training. These results were shown within days of minimal L2 training, suggesting that the neural changes captured during adult second language acquisition are more rapid than previously shown. Such findings are consistent with models of early stages of bilingualism in adult learners of L2 ( e.g. Kroll and Stewart's RHM) and reinforce the use of ERP measures to assess L2 learning.

Neural Regulation of Pancreatic Cancer: A Novel Target for Intervention

International Nuclear Information System (INIS)

Chang, Aeson; Kim-Fuchs, Corina; Le, Caroline P.; Hollande, Frédéric; Sloan, Erica K.

2015-01-01

The tumor microenvironment is known to play a pivotal role in driving cancer progression and governing response to therapy. This is of significance in pancreatic cancer where the unique pancreatic tumor microenvironment, characterized by its pronounced desmoplasia and fibrosis, drives early stages of tumor progression and dissemination, and contributes to its associated low survival rates. Several molecular factors that regulate interactions between pancreatic tumors and their surrounding stroma are beginning to be identified. Yet broader physiological factors that influence these interactions remain unclear. Here, we discuss a series of preclinical and mechanistic studies that highlight the important role chronic stress plays as a physiological regulator of neural-tumor interactions in driving the progression of pancreatic cancer. These studies propose several approaches to target stress signaling via the β-adrenergic signaling pathway in order to slow pancreatic tumor growth and metastasis. They also provide evidence to support the use of β-blockers as a novel therapeutic intervention to complement current clinical strategies to improve cancer outcome in patients with pancreatic cancer

Neural Regulation of Pancreatic Cancer: A Novel Target for Intervention

Energy Technology Data Exchange (ETDEWEB)

Chang, Aeson [Monash Institute of Pharmaceutical Sciences, Monash University, Parkville, Victoria 3052 (Australia); Kim-Fuchs, Corina [Monash Institute of Pharmaceutical Sciences, Monash University, Parkville, Victoria 3052 (Australia); Department of Visceral Surgery and Medicine, University Hospital Bern, Bern 3010 (Switzerland); Le, Caroline P. [Monash Institute of Pharmaceutical Sciences, Monash University, Parkville, Victoria 3052 (Australia); Hollande, Frédéric [Monash Institute of Pharmaceutical Sciences, Monash University, Parkville, Victoria 3052 (Australia); Department of Pathology, University of Melbourne, Parkville 3010 (Australia); Sloan, Erica K., E-mail: erica.sloan@monash.edu [Monash Institute of Pharmaceutical Sciences, Monash University, Parkville, Victoria 3052 (Australia); Cousins Center for PNI, UCLA Semel Institute, Jonsson Comprehensive Cancer Center, and UCLA AIDS Institute, University of California Los Angeles, Los Angeles, CA 90095 (United States); Peter MacCallum Cancer Centre, Division of Cancer Surgery, East Melbourne, Victoria 3002 (Australia)

2015-07-17

The tumor microenvironment is known to play a pivotal role in driving cancer progression and governing response to therapy. This is of significance in pancreatic cancer where the unique pancreatic tumor microenvironment, characterized by its pronounced desmoplasia and fibrosis, drives early stages of tumor progression and dissemination, and contributes to its associated low survival rates. Several molecular factors that regulate interactions between pancreatic tumors and their surrounding stroma are beginning to be identified. Yet broader physiological factors that influence these interactions remain unclear. Here, we discuss a series of preclinical and mechanistic studies that highlight the important role chronic stress plays as a physiological regulator of neural-tumor interactions in driving the progression of pancreatic cancer. These studies propose several approaches to target stress signaling via the β-adrenergic signaling pathway in order to slow pancreatic tumor growth and metastasis. They also provide evidence to support the use of β-blockers as a novel therapeutic intervention to complement current clinical strategies to improve cancer outcome in patients with pancreatic cancer.

The Reinforcement Sensitivity Theory of Personality Questionnaire (RST-PQ): Development and validation.

Science.gov (United States)

Corr, Philip J; Cooper, Andrew J

2016-11-01

We report the development and validation of a questionnaire measure of the revised reinforcement sensitivity theory (rRST) of personality. Starting with qualitative responses to defensive and approach scenarios modeled on typical rodent ethoexperimental situations, exploratory and confirmatory factor analyses (CFAs) revealed a robust 6-factor structure: 2 unitary defensive factors, fight-flight-freeze system (FFFS; related to fear) and the behavioral inhibition system (BIS; related to anxiety); and 4 behavioral approach system (BAS) factors (Reward Interest, Goal-Drive Persistence, Reward Reactivity, and Impulsivity). Theoretically motivated thematic facets were employed to sample the breadth of defensive space, comprising FFFS (Flight, Freeze, and Active Avoidance) and BIS (Motor Planning Interruption, Worry, Obsessive Thoughts, and Behavioral Disengagement). Based on theoretical considerations, and statistically confirmed, a separate scale for Defensive Fight was developed. Validation evidence for the 6-factor structure came from convergent and discriminant validity shown by correlations with existing personality scales. We offer the Reinforcement Sensitivity Theory of Personality Questionnaire to facilitate future research specifically on rRST and, more broadly, on approach-avoidance theories of personality. (PsycINFO Database Record (c) 2016 APA, all rights reserved).

Modelling reinforcement corrosion in concrete

DEFF Research Database (Denmark)

Michel, Alexander; Geiker, Mette Rica; Stang, Henrik

2012-01-01

A physio-chemical model for the simulation of reinforcement corrosion in concrete struc-tures was developed. The model allows for simulation of initiation and subsequent propaga-tion of reinforcement corrosion. Corrosion is assumed to be initiated once a defined critical chloride threshold......, a numerical example is pre-sented, that illustrates the formation of corrosion cells as well as propagation of corrosion in a reinforced concrete structure....

The Reinforcement Learning Competition 2014

OpenAIRE

Dimitrakakis, Christos; Li, Guangliang; Tziortziotis, Nikoalos

2014-01-01

Reinforcement learning is one of the most general problems in artificial intelligence. It has been used to model problems in automated experiment design, control, economics, game playing, scheduling and telecommunications. The aim of the reinforcement learning competition is to encourage the development of very general learning agents for arbitrary reinforcement learning problems and to provide a test-bed for the unbiased evaluation of algorithms.

Sensorless control for permanent magnet synchronous motor using a neural network based adaptive estimator

Science.gov (United States)

Kwon, Chung-Jin; Kim, Sung-Joong; Han, Woo-Young; Min, Won-Kyoung

2005-12-01

The rotor position and speed estimation of permanent-magnet synchronous motor(PMSM) was dealt with. By measuring the phase voltages and currents of the PMSM drive, two diagonally recurrent neural network(DRNN) based observers, a neural current observer and a neural velocity observer were developed. DRNN which has self-feedback of the hidden neurons ensures that the outputs of DRNN contain the whole past information of the system even if the inputs of DRNN are only the present states and inputs of the system. Thus the structure of DRNN may be simpler than that of feedforward and fully recurrent neural networks. If the backpropagation method was used for the training of the DRNN the problem of slow convergence arise. In order to reduce this problem, recursive prediction error(RPE) based learning method for the DRNN was presented. The simulation results show that the proposed approach gives a good estimation of rotor speed and position, and RPE based training has requires a shorter computation time compared to backpropagation based training.

Impulsivity in spontaneously hypertensive rats: Within-subjects comparison of sensitivity to delay and to amount of reinforcement.

Science.gov (United States)

Orduña, Vladimir; Mercado, Eduardo

2017-06-15

Previous research has shown that spontaneously hypertensive rats (SHR) display higher levels of impulsive choice behavior, which is accompanied by a higher sensitivity to the delay of reinforcement, and by a normal sensitivity to the amount of reinforcement. Because those results were based on three different samples of subjects, in the present report we evaluated these three processes in the same individuals. SHR and WIS rats were exposed to concurrent-chains schedules in which the terminal links were manipulated to assess impulsivity, sensitivity to delay, and sensitivity to amount. For exploring impulsivity, a terminal link was associated with a small reinforcer (1 pellet) delivered after a short delay (2s) while the other terminal link was associated with a larger reinforcer (4 pellets) delivered after a longer delay (28s). For assessing sensitivity to delay, both alternatives delivered the same amount of reinforcement (1 pellet) and the only difference between them was in the delay before reinforcement delivery (2s vs 28s). For assessing sensitivity to amount, both alternatives were associated with the same delay (15s), but the alternatives differed in the amount of reinforcement (1 vs 4 pellets). In addition to replicating previously observed effects within-subjects, we were interested in analyzing different aspects of the regularity of rats' actions in the choice task. The results confirmed that previous findings were not a consequence of between-group differences: SHR were more impulsive and more sensitive to delay, while their sensitivity to amount was normal. Analyses of response regularity indicated that SHR subjects were more periodic in their responses to levers and in their feeder entries, had a higher number of short-duration bouts of responding, and made a substantially higher number of switches between the alternatives. We discuss the potential implications of these findings for the possible behavioral mechanisms driving the increased sensitivity

Origins of food reinforcement in infants12345

Science.gov (United States)

Kong, Kai Ling; Feda, Denise M; Eiden, Rina D; Epstein, Leonard H

2015-01-01

Background: Rapid weight gain in infancy is associated with a higher risk of obesity in children and adults. A high relative reinforcing value of food is cross-sectionally related to obesity; lean children find nonfood alternatives more reinforcing than do overweight/obese children. However, to our knowledge, there is no research on how and when food reinforcement develops. Objective: This study was designed to assess whether the reinforcing value of food and nonfood alternatives could be tested in 9- to 18-mo-old infants and whether the reinforcing value of food and nonfood alternatives is differentially related to infant weight status. Design: Reinforcing values were assessed by using absolute progressive ratio schedules of reinforcement, with presentation of food and nonfood alternatives counterbalanced in 2 separate studies. Two nonfood reinforcers [Baby Einstein–Baby MacDonald shows (study 1, n = 27) or bubbles (study 2, n = 30)] were tested against the baby’s favorite food. Food reinforcing ratio (FRR) was quantified by measuring the reinforcing value of food (Food Pmax) in proportion to the total reinforcing value of food and a nonfood alternative (DVD Pmax or BUB Pmax). Results: Greater weight-for-length z score was associated with a greater FRR of a favorite food in study 1 (FRR-DVD) (r = 0.60, P positively associated with FRR-DVD (r = 0.57, P = 0.009) and FRR-BUB (r = 0.37, P = 0.047). Conclusions: Our newly developed paradigm, which tested 2 different nonfood alternatives, demonstrated that lean infants find nonfood alternatives more reinforcing than do overweight/obese infants. This observation suggests that strengthening the alternative reinforcers may have a protective effect against childhood obesity. This research was registered at clinicaltrials.gov as NCT02229552. PMID:25733636

Early life stress accelerates behavioral and neural maturation of the hippocampus in male mice.

Science.gov (United States)

Bath, K; Manzano-Nieves, G; Goodwill, H

2016-06-01

Early life stress (ELS) increases the risk for later cognitive and emotional dysfunction. ELS is known to truncate neural development through effects on suppressing cell birth, increasing cell death, and altering neuronal morphology, effects that have been associated with behavioral profiles indicative of precocious maturation. However, how earlier silencing of growth drives accelerated behavioral maturation has remained puzzling. Here, we test the novel hypothesis that, ELS drives a switch from growth to maturation to accelerate neural and behavioral development. To test this, we used a mouse model of ELS, fragmented maternal care, and a cross-sectional dense sampling approach focusing on hippocampus and measured effects of ELS on the ontogeny of behavioral development and biomarkers of neural maturation. Consistent with previous work, ELS was associated with an earlier developmental decline in expression of markers of cell proliferation (Ki-67) and differentiation (doublecortin). However, ELS also led to a precocious arrival of Parvalbumin-positive cells, led to an earlier switch in NMDA receptor subunit expression (marker of synaptic maturity), and was associated with an earlier rise in myelin basic protein expression (key component of the myelin sheath). In addition, in a contextual fear-conditioning task, ELS accelerated the timed developmental suppression of contextual fear. Together, these data provide support for the hypothesis that ELS serves to switch neurodevelopment from processes of growth to maturation and promotes accelerated development of some forms of emotional learning. Copyright © 2016 Elsevier Inc. All rights reserved.

Effect of hybrid fiber reinforcement on the cracking process in fiber reinforced cementitious composites

DEFF Research Database (Denmark)

Pereira, Eduardo B.; Fischer, Gregor; Barros, Joaquim A.O.

2012-01-01

The simultaneous use of different types of fibers as reinforcement in cementitious matrix composites is typically motivated by the underlying principle of a multi-scale nature of the cracking processes in fiber reinforced cementitious composites. It has been hypothesized that while undergoing...... tensile deformations in the composite, the fibers with different geometrical and mechanical properties restrain the propagation and further development of cracking at different scales from the micro- to the macro-scale. The optimized design of the fiber reinforcing systems requires the objective...... materials is carried out by assessing directly their tensile stress-crack opening behavior. The efficiency of hybrid fiber reinforcements and the multi-scale nature of cracking processes are discussed based on the experimental results obtained, as well as the micro-mechanisms underlying the contribution...

Dementia & Driving

Science.gov (United States)

... have to give up driving. Many people associate driving with self-reliance and freedom; the loss of driving privileges ... familiar roads and avoid long distances. Avoid heavy traffic and heavily traveled roads. Avoid driving at night and in bad weather. Reduce the ...

Adaptive Sliding Mode Control of Chaos in Permanent Magnet Synchronous Motor via Fuzzy Neural Networks

Directory of Open Access Journals (Sweden)

Tat-Bao-Thien Nguyen

2014-01-01

Full Text Available In this paper, based on fuzzy neural networks, we develop an adaptive sliding mode controller for chaos suppression and tracking control in a chaotic permanent magnet synchronous motor (PMSM drive system. The proposed controller consists of two parts. The first is an adaptive sliding mode controller which employs a fuzzy neural network to estimate the unknown nonlinear models for constructing the sliding mode controller. The second is a compensational controller which adaptively compensates estimation errors. For stability analysis, the Lyapunov synthesis approach is used to ensure the stability of controlled systems. Finally, simulation results are provided to verify the validity and superiority of the proposed method.

A Novel Robot System Integrating Biological and Mechanical Intelligence Based on Dissociated Neural Network-Controlled Closed-Loop Environment.

Science.gov (United States)

Li, Yongcheng; Sun, Rong; Wang, Yuechao; Li, Hongyi; Zheng, Xiongfei

2016-01-01

We propose the architecture of a novel robot system merging biological and artificial intelligence based on a neural controller connected to an external agent. We initially built a framework that connected the dissociated neural network to a mobile robot system to implement a realistic vehicle. The mobile robot system characterized by a camera and two-wheeled robot was designed to execute the target-searching task. We modified a software architecture and developed a home-made stimulation generator to build a bi-directional connection between the biological and the artificial components via simple binomial coding/decoding schemes. In this paper, we utilized a specific hierarchical dissociated neural network for the first time as the neural controller. Based on our work, neural cultures were successfully employed to control an artificial agent resulting in high performance. Surprisingly, under the tetanus stimulus training, the robot performed better and better with the increasement of training cycle because of the short-term plasticity of neural network (a kind of reinforced learning). Comparing to the work previously reported, we adopted an effective experimental proposal (i.e. increasing the training cycle) to make sure of the occurrence of the short-term plasticity, and preliminarily demonstrated that the improvement of the robot's performance could be caused independently by the plasticity development of dissociated neural network. This new framework may provide some possible solutions for the learning abilities of intelligent robots by the engineering application of the plasticity processing of neural networks, also for the development of theoretical inspiration for the next generation neuro-prostheses on the basis of the bi-directional exchange of information within the hierarchical neural networks.

A Novel Robot System Integrating Biological and Mechanical Intelligence Based on Dissociated Neural Network-Controlled Closed-Loop Environment.

Directory of Open Access Journals (Sweden)

Yongcheng Li

Full Text Available We propose the architecture of a novel robot system merging biological and artificial intelligence based on a neural controller connected to an external agent. We initially built a framework that connected the dissociated neural network to a mobile robot system to implement a realistic vehicle. The mobile robot system characterized by a camera and two-wheeled robot was designed to execute the target-searching task. We modified a software architecture and developed a home-made stimulation generator to build a bi-directional connection between the biological and the artificial components via simple binomial coding/decoding schemes. In this paper, we utilized a specific hierarchical dissociated neural network for the first time as the neural controller. Based on our work, neural cultures were successfully employed to control an artificial agent resulting in high performance. Surprisingly, under the tetanus stimulus training, the robot performed better and better with the increasement of training cycle because of the short-term plasticity of neural network (a kind of reinforced learning. Comparing to the work previously reported, we adopted an effective experimental proposal (i.e. increasing the training cycle to make sure of the occurrence of the short-term plasticity, and preliminarily demonstrated that the improvement of the robot's performance could be caused independently by the plasticity development of dissociated neural network. This new framework may provide some possible solutions for the learning abilities of intelligent robots by the engineering application of the plasticity processing of neural networks, also for the development of theoretical inspiration for the next generation neuro-prostheses on the basis of the bi-directional exchange of information within the hierarchical neural networks.

Reinforcement Learning Approach to Generate Goal-directed Locomotion of a Snake-Like Robot with Screw-Drive Units

DEFF Research Database (Denmark)

Chatterjee, Sromona; Nachstedt, Timo; Tamosiunaite, Minija

2014-01-01

Abstract—In this paper we apply a policy improvement algorithm called Policy Improvement using Path Integrals (PI2) to generate goal-directed locomotion of a complex snake-like robot with screw-drive units. PI2 is numerically simple and has an ability to deal with high dimensional systems. Here...

Neural signals of vicarious extinction learning.

Science.gov (United States)

Golkar, Armita; Haaker, Jan; Selbing, Ida; Olsson, Andreas

2016-10-01

Social transmission of both threat and safety is ubiquitous, but little is known about the neural circuitry underlying vicarious safety learning. This is surprising given that these processes are critical to flexibly adapt to a changeable environment. To address how the expression of previously learned fears can be modified by the transmission of social information, two conditioned stimuli (CS + s) were paired with shock and the third was not. During extinction, we held constant the amount of direct, non-reinforced, exposure to the CSs (i.e. direct extinction), and critically varied whether another individual-acting as a demonstrator-experienced safety (CS + vic safety) or aversive reinforcement (CS + vic reinf). During extinction, ventromedial prefrontal cortex (vmPFC) responses to the CS + vic reinf increased but decreased to the CS + vic safety This pattern of vmPFC activity was reversed during a subsequent fear reinstatement test, suggesting a temporal shift in the involvement of the vmPFC. Moreover, only the CS + vic reinf association recovered. Our data suggest that vicarious extinction prevents the return of conditioned fear responses, and that this efficacy is reflected by diminished vmPFC involvement during extinction learning. The present findings may have important implications for understanding how social information influences the persistence of fear memories in individuals suffering from emotional disorders. © The Author (2016). Published by Oxford University Press. For Permissions, please email: journals.permissions@oup.com.

DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars

OpenAIRE

Tian, Yuchi; Pei, Kexin; Jana, Suman; Ray, Baishakhi

2017-01-01

Recent advances in Deep Neural Networks (DNNs) have led to the development of DNN-driven autonomous cars that, using sensors like camera, LiDAR, etc., can drive without any human intervention. Most major manufacturers including Tesla, GM, Ford, BMW, and Waymo/Google are working on building and testing different types of autonomous vehicles. The lawmakers of several US states including California, Texas, and New York have passed new legislation to fast-track the process of testing and deployme...

A qualitative exploration of driving stress and driving discourtesy.

Science.gov (United States)

Scott-Parker, B; Jones, C M; Rune, K; Tucker, J

2018-05-31

Driving courtesy, and conversely driving discourtesy, recently has been of great interest in the public domain. In addition, there has been increasing recognition of the negative impact of stress upon the individual's health and wellbeing, with a plethora of interventions aimed at minimising stress more generally. The research literature regarding driving dis/courtesy, in comparison, is scant, with a handful of studies examining the dis/courteous driving behaviour of road users, and the relationship between driving discourtesy and driving stress. To examine courteous and discourteous driving experiences, and to explore the impact of stress associated with such driving experiences. Thirty-eight drivers (20 females) from the Sunshine Coast region volunteered to participate in one of four 1-1.5 h focus groups. Content analysis used the verbatim utterances captured via an Mp3 device. Three themes pertaining to stressful and discourteous interactions were identified. Theme one pertained to the driving context: road infrastructure (eg, roundabouts, roadwork), vehicles (eg, features), location (eg, country vs city, unfamiliar areas), and temporal aspects (eg, holidays). Theme two pertained to other road users: their behaviour (eg, tailgating, merging), and unknown factors (eg, illicit and licit drug use). Theme three pertained to the self as road user: their own behaviours (eg, deliberate intimidation), and their emotions (eg, angry reaction to other drivers, being in control). Driving dis/courtesy and driving stress is a complex phenomenon, suggesting complex intervention efforts are required. Driving discourtesy was reported as being highly stressful, therefore intervention efforts which encourage driving courtesy and which foster emotional capacity to cope with stressful circumstances appear warranted. Copyright © 2018. Published by Elsevier Ltd.

The drive to eat: comparisons and distinctions between mechanisms of food reward and drug addiction.

Science.gov (United States)

DiLeone, Ralph J; Taylor, Jane R; Picciotto, Marina R

2012-10-01

The growing rates of obesity have prompted comparisons between the uncontrolled intake of food and drugs; however, an evaluation of the equivalence of food- and drug-related behaviors requires a thorough understanding of the underlying neural circuits driving each behavior. Although it has been attractive to borrow neurobiological concepts from addiction to explore compulsive food seeking, a more integrated model is needed to understand how food and drugs differ in their ability to drive behavior. In this Review, we will examine the commonalities and differences in the systems-level and behavioral responses to food and to drugs of abuse, with the goal of identifying areas of research that would address gaps in our understanding and ultimately identify new treatments for obesity or drug addiction.

Seismic Retrofitting: Reinforced Concrete (RC shear wall versus Reinforcement of RC element by Carbon Fiber Reinforced Polymer (CFRP using PUSHOVER analysis

Directory of Open Access Journals (Sweden)

Yahya RIYAD

2016-12-01

Full Text Available Seismic retrofitting of constructions vulnerable to earthquakes is a current problem of great political and social relevance. During the last sixty years, moderate to severe earthquakes have occurred in Morocco (specifically in Agadir 1960 and Hoceima 2004. Such events have clearly shown the vulnerability of the building stock in particular and of the built environment in general. Hence, it is very much essential to retrofit the vulnerable building to cope up for the next damaging earthquake. In this paper, the focus will be on a comparative study between two techniques of seismic retrofitting, the first one is a reinforcement using carbon fiber reinforced polymer (CFRP applied to RC elements by bonding , and the second one is a reinforcement with a shear wall. For this study, we will use a non-linear static analysis -also known as Pushover analysis - on a reinforced concrete structure consisting of beams and columns, and composed from eight storey with a gross area of 240 m², designed conforming to the Moroccan Seismic code[1].

A neural tracking and motor control approach to improve rehabilitation of upper limb movements

Directory of Open Access Journals (Sweden)

Schmid Maurizio

2008-02-01

Full Text Available Abstract Background Restoration of upper limb movements in subjects recovering from stroke is an essential keystone in rehabilitative practices. Rehabilitation of arm movements, in fact, is usually a far more difficult one as compared to that of lower extremities. For these reasons, researchers are developing new methods and technologies so that the rehabilitative process could be more accurate, rapid and easily accepted by the patient. This paper introduces the proof of concept for a new non-invasive FES-assisted rehabilitation system for the upper limb, called smartFES (sFES, where the electrical stimulation is controlled by a biologically inspired neural inverse dynamics model, fed by the kinematic information associated with the execution of a planar goal-oriented movement. More specifically, this work details two steps of the proposed system: an ad hoc markerless motion analysis algorithm for the estimation of kinematics, and a neural controller that drives a synthetic arm. The vision of the entire system is to acquire kinematics from the analysis of video sequences during planar arm movements and to use it together with a neural inverse dynamics model able to provide the patient with the electrical stimulation patterns needed to perform the movement with the assisted limb. Methods The markerless motion tracking system aims at localizing and monitoring the arm movement by tracking its silhouette. It uses a specifically designed motion estimation method, that we named Neural Snakes, which predicts the arm contour deformation as a first step for a silhouette extraction algorithm. The starting and ending points of the arm movement feed an Artificial Neural Controller, enclosing the muscular Hill's model, which solves the inverse dynamics to obtain the FES patterns needed to move a simulated arm from the starting point to the desired point. Both position error with respect to the requested arm trajectory and comparison between curvature factors

Robust Adaptive Exponential Synchronization of Stochastic Perturbed Chaotic Delayed Neural Networks with Parametric Uncertainties

Directory of Open Access Journals (Sweden)

Yang Fang

2014-01-01

Full Text Available This paper investigates the robust adaptive exponential synchronization in mean square of stochastic perturbed chaotic delayed neural networks with nonidentical parametric uncertainties. A robust adaptive feedback controller is proposed based on Gronwally’s inequality, drive-response concept, and adaptive feedback control technique with the update laws of nonidentical parametric uncertainties as well as linear matrix inequality (LMI approach. The sufficient conditions for robust adaptive exponential synchronization in mean square of uncoupled uncertain stochastic chaotic delayed neural networks are derived in terms of linear matrix inequalities (LMIs. The effect of nonidentical uncertain parameter uncertainties is suppressed by the designed robust adaptive feedback controller rapidly. A numerical example is provided to validate the effectiveness of the proposed method.

A new framework for cortico-striatal plasticity: behavioural theory meets in vitro data at the reinforcement-action interface.

Directory of Open Access Journals (Sweden)

Kevin N Gurney

2015-01-01

Full Text Available Operant learning requires that reinforcement signals interact with action representations at a suitable neural interface. Much evidence suggests that this occurs when phasic dopamine, acting as a reinforcement prediction error, gates plasticity at cortico-striatal synapses, and thereby changes the future likelihood of selecting the action(s coded by striatal neurons. But this hypothesis faces serious challenges. First, cortico-striatal plasticity is inexplicably complex, depending on spike timing, dopamine level, and dopamine receptor type. Second, there is a credit assignment problem-action selection signals occur long before the consequent dopamine reinforcement signal. Third, the two types of striatal output neuron have apparently opposite effects on action selection. Whether these factors rule out the interface hypothesis and how they interact to produce reinforcement learning is unknown. We present a computational framework that addresses these challenges. We first predict the expected activity changes over an operant task for both types of action-coding striatal neuron, and show they co-operate to promote action selection in learning and compete to promote action suppression in extinction. Separately, we derive a complete model of dopamine and spike-timing dependent cortico-striatal plasticity from in vitro data. We then show this model produces the predicted activity changes necessary for learning and extinction in an operant task, a remarkable convergence of a bottom-up data-driven plasticity model with the top-down behavioural requirements of learning theory. Moreover, we show the complex dependencies of cortico-striatal plasticity are not only sufficient but necessary for learning and extinction. Validating the model, we show it can account for behavioural data describing extinction, renewal, and reacquisition, and replicate in vitro experimental data on cortico-striatal plasticity. By bridging the levels between the single synapse and

Electric drives

CERN Document Server

Boldea, Ion

2005-01-01

ENERGY CONVERSION IN ELECTRIC DRIVESElectric Drives: A DefinitionApplication Range of Electric DrivesEnergy Savings Pay Off RapidlyGlobal Energy Savings Through PEC DrivesMotor/Mechanical Load MatchMotion/Time Profile MatchLoad Dynamics and StabilityMultiquadrant OperationPerformance IndexesProblemsELECTRIC MOTORS FOR DRIVESElectric Drives: A Typical ConfigurationElectric Motors for DrivesDC Brush MotorsConventional AC MotorsPower Electronic Converter Dependent MotorsEnergy Conversion in Electric Motors/GeneratorsPOWER ELECTRONIC CONVERTERS (PECs) FOR DRIVESPower Electronic Switches (PESs)The

FOAM CONCRETE REINFORCEMENT BY BASALT FIBRES

Directory of Open Access Journals (Sweden)

Zhukov Aleksey Dmitrievich

2012-10-01

Full Text Available The authors demonstrate that the foam concrete performance can be improved by dispersed reinforcement, including methods that involve basalt fibres. They address the results of the foam concrete modeling technology and assess the importance of technology-related parameters. Reinforcement efficiency criteria are also provided in the article. Dispersed reinforcement improves the plasticity of the concrete mix and reduces the settlement crack formation rate. Conventional reinforcement that involves metal laths and rods demonstrates its limited application in the production of concrete used for thermal insulation and structural purposes. Dispersed reinforcement is preferable. This technology contemplates the infusion of fibres into porous mixes. Metal, polymeric, basalt and glass fibres are used as reinforcing components. It has been identified that products reinforced by polypropylene fibres demonstrate substantial abradability and deformability rates even under the influence of minor tensile stresses due to the low adhesion strength of polypropylene in the cement matrix. The objective of the research was to develop the type of polypropylene of D500 grade that would demonstrate the operating properties similar to those of Hebel and Ytong polypropylenes. Dispersed reinforcement was performed by the basalt fibre. This project contemplates an autoclave-free technology to optimize the consumption of electricity. Dispersed reinforcement is aimed at the reduction of the block settlement in the course of hardening at early stages of their operation, the improvement of their strength and other operating properties. Reduction in the humidity rate of the mix is based on the plasticizing properties of fibres, as well as the application of the dry mineralization method. Selection of optimal parameters of the process-related technology was performed with the help of G-BAT-2011 Software, developed at Moscow State University of Civil Engineering. The authors also

Influence of neural adaptation on dynamics and equilibrium state of neural activities in a ring neural network

Science.gov (United States)

Takiyama, Ken

2017-12-01

How neural adaptation affects neural information processing (i.e. the dynamics and equilibrium state of neural activities) is a central question in computational neuroscience. In my previous works, I analytically clarified the dynamics and equilibrium state of neural activities in a ring-type neural network model that is widely used to model the visual cortex, motor cortex, and several other brain regions. The neural dynamics and the equilibrium state in the neural network model corresponded to a Bayesian computation and statistically optimal multiple information integration, respectively, under a biologically inspired condition. These results were revealed in an analytically tractable manner; however, adaptation effects were not considered. Here, I analytically reveal how the dynamics and equilibrium state of neural activities in a ring neural network are influenced by spike-frequency adaptation (SFA). SFA is an adaptation that causes gradual inhibition of neural activity when a sustained stimulus is applied, and the strength of this inhibition depends on neural activities. I reveal that SFA plays three roles: (1) SFA amplifies the influence of external input in neural dynamics; (2) SFA allows the history of the external input to affect neural dynamics; and (3) the equilibrium state corresponds to the statistically optimal multiple information integration independent of the existence of SFA. In addition, the equilibrium state in a ring neural network model corresponds to the statistically optimal integration of multiple information sources under biologically inspired conditions, independent of the existence of SFA.

The possibility of using high strength reinforced concrete

International Nuclear Information System (INIS)

Miura, Nobuaki

1991-01-01

There is recently much research about and developments in reinforced concrete using high strength concrete and reinforcement. As a result, some high-rise buildings and nuclear buildings have been constructed with such concrete. Reinforced concrete will be stronger in the future, but there is a limit to its strength defined by the character of the materials and also by the character of the reinforced concrete members made of the concrete and reinforcement. This report describes the merits and demerits of using high strength reinforced concrete. (author)

Older drivers with cognitive impairment: Perceived changes in driving skills, driving-related discomfort and self-regulation of driving

DEFF Research Database (Denmark)

Meng, A.; Siren, A.; Teasdale, Thomas William

2013-01-01

The results of a previous study indicate that in general, older drivers who recognise cognitive problems show realistic self-assessment of changes in their driving skills and that driving-related discomfort may function as an indirect monitoring of driving ability, contributing to their safe...... drivers may recognise cognitive problems, they tend not to recognise changes to their driving, which may reflect reluctance to acknowledge the impact of cognitive impairment on their driving. Furthermore, the results suggest that driving-related discomfort plays an important role in the self......-regulation of driving among cognitively impaired older drivers. However, it is less clear what triggers driving-related discomfort among cognitively impaired older drivers indicating that it may be a less reliable aspect of their self-monitoring of driving ability....

Reinforced concrete wall under hydrogen detonation

International Nuclear Information System (INIS)

Saarenheimo, A.

2000-11-01

The structural integrity of a reinforced concrete wall in the BWR reactor building under hydrogen detonation conditions has been analysed. Of particular interest is whether the containment integrity can be jeopardised by an external hydrogen detonation. The load carrying capacity of a reinforced concrete wall was studied. The detonation pressure loads were estimated with computerised hand calculations assuming a direct initiation of detonation and applying the strong explosion theory. The results can be considered as rough and conservative estimates for the first shock pressure impact induced by a reflecting detonation wave. Structural integrity may be endangered due to slow pressurisation or dynamic impulse loads associated with local detonations. The static pressure following the passage of a shock front may be relatively high, thus this static or slowly decreasing pressure after a detonation may damage the structure severely. The mitigating effects of the opening of a door on pressure history and structural response were also studied. The non-linear behaviour of the wall was studied under detonations corresponding a detonable hydrogen mass of 0.5 kg and 1.428 kg. Non-linear finite element analyses of the reinforced concrete structure were carried out by the ABAQUS/Explicit program. The reinforcement and its non-linear material behaviour and the tensile cracking of concrete were modelled. Reinforcement was defined as layers of uniformly spaced reinforcing bars in shell elements. In these studies the surrounding structures of the non-linearly modelled reinforced concrete wall were modelled using idealised boundary conditions. Especially concrete cracking and yielding of the reinforcement was monitored during the numerical simulation. (au)

Culture in the mind's mirror: how anthropology and neuroscience can inform a model of the neural substrate for cultural imitative learning.

Science.gov (United States)

Losin, Elizabeth A Reynolds; Dapretto, Mirella; Iacoboni, Marco

2009-01-01

Cultural neuroscience, the study of how cultural experience shapes the brain, is an emerging subdiscipline in the neurosciences. Yet, a foundational question to the study of culture and the brain remains neglected by neuroscientific inquiry: "How does cultural information get into the brain in the first place?" Fortunately, the tools needed to explore the neural architecture of cultural learning - anthropological theories and cognitive neuroscience methodologies - already exist; they are merely separated by disciplinary boundaries. Here we review anthropological theories of cultural learning derived from fieldwork and modeling; since cultural learning theory suggests that sophisticated imitation abilities are at the core of human cultural learning, we focus our review on cultural imitative learning. Accordingly we proceed to discuss the neural underpinnings of imitation and other mechanisms important for cultural learning: learning biases, mental state attribution, and reinforcement learning. Using cultural neuroscience theory and cognitive neuroscience research as our guides, we then propose a preliminary model of the neural architecture of cultural learning. Finally, we discuss future studies needed to test this model and fully explore and explain the neural underpinnings of cultural imitative learning.

Hedonic Hunger Is Related to Increased Neural and Perceptual Responses to Cues of Palatable Food and Motivation to Consume: Evidence from 3 Independent Investigations.

Science.gov (United States)

Burger, Kyle S; Sanders, Abigail J; Gilbert, Jennifer R

2016-09-01

The Power of Food Scale (PFS) seeks to identify individuals who experience high appetitive drive in response to food cues, which is a construct termed "hedonic hunger." The purpose of this study was to assess cross-sectional correlates and predictive power of PFS scores to probe the construct of hedonic hunger. Separate data from 3 studies (study 1, n = 44; study 2, n = 398; study 3, n = 100) were used to evaluate the construct of hedonic hunger. We examined the correlations between the PFS and neural responsivity during intake and anticipated intake of palatable foods, behavioral food reinforcement, perceptual hedonic ratings of food images, and change in body mass index (BMI) and binge eating over time. Hedonic hunger was strongly related to bilateral brain response in regions implicated in oral somatosensory processing during cue-elicited anticipation of food intake (study 1; right postcentral gyrus: r = 0.67, P hunger was not associated with baseline BMI (studies 1-3: P = 0.14, 0.21, and 0.37, respectively) or change in BMI over the 2-y follow-up (studies 1 and 2: P = 0.14 and 0.37, respectively) but was significantly correlated with baseline binge eating in 2 samples (study 1: r = 0.58, P = 0.001; study 2: r = 0.31, P = 0.02; and study 3: P = 0.02). Hedonic hunger was not predictive of weight regulation. However, individuals who report high hedonic hunger are likely to show increased neural and perceptual responses to cues of palatable foods, increased motivation to consume such foods, and a greater likelihood of current binge eating. © 2016 American Society for Nutrition.

Drive Stands

Data.gov (United States)

Federal Laboratory Consortium — The Electrical Systems Laboratory (ESL)houses numerous electrically driven drive stands. A drive stand consists of an electric motor driving a gearbox and a mounting...

Comparing Expert and Novice Driving Behavior in a Driving Simulator

Directory of Open Access Journals (Sweden)

Hiran B. Ekanayake

2014-02-01

Full Text Available This paper presents a study focused on comparing driving behavior of expert and novice drivers in a mid-range driving simulator with the intention of evaluating the validity of driving simulators for driver training. For the investigation, measurements of performance, psychophysiological measurements, and self-reported user experience under different conditions of driving tracks and driving sessions were analyzed. We calculated correlations between quantitative and qualitative measures to enhance the reliability of the findings. The experiment was conducted involving 14 experienced drivers and 17 novice drivers. The results indicate that driving behaviors of expert and novice drivers differ from each other in several ways but it heavily depends on the characteristics of the task. Moreover, our belief is that the analytical framework proposed in this paper can be used as a tool for selecting appropriate driving tasks as well as for evaluating driving performance in driving simulators.

On the weld strength of in situ tape placed reinforcements on weave reinforced structures

NARCIS (Netherlands)

Grouve, Wouter Johannes Bernardus; Warnet, Laurent; Rietman, Bert; Akkerman, Remko

2012-01-01

Unidirectionally reinforced thermoplastic tapes were welded onto woven fabric reinforced laminates using a laser assisted tape placement process. A mandrel peel setup was used to quantify the interfacial fracture toughness between the tape and the laminate as a measure for weld strength. The tape

Reinforcement Learning State-of-the-Art

CERN Document Server

Wiering, Marco

2012-01-01

Reinforcement learning encompasses both a science of adaptive behavior of rational beings in uncertain environments and a computational methodology for finding optimal behaviors for challenging problems in control, optimization and adaptive behavior of intelligent agents. As a field, reinforcement learning has progressed tremendously in the past decade. The main goal of this book is to present an up-to-date series of survey articles on the main contemporary sub-fields of reinforcement learning. This includes surveys on partially observable environments, hierarchical task decompositions, relational knowledge representation and predictive state representations. Furthermore, topics such as transfer, evolutionary methods and continuous spaces in reinforcement learning are surveyed. In addition, several chapters review reinforcement learning methods in robotics, in games, and in computational neuroscience. In total seventeen different subfields are presented by mostly young experts in those areas, and together the...

Autoimmunity as a Driving Force of Cognitive Evolution

Directory of Open Access Journals (Sweden)

Serge Nataf

2017-10-01

Full Text Available In the last decades, increasingly robust experimental approaches have formally demonstrated that autoimmunity is a physiological process involved in a large range of functions including cognition. On this basis, the recently enunciated “brain superautoantigens” theory proposes that autoimmunity has been a driving force of cognitive evolution. It is notably suggested that the immune and nervous systems have somehow co-evolved and exerted a mutual selection pressure benefiting to both systems. In this two-way process, the evolutionary-determined emergence of neurons expressing specific immunogenic antigens (brain superautoantigens has exerted a selection pressure on immune genes shaping the T-cell repertoire. Such a selection pressure on immune genes has translated into the emergence of a finely tuned autoimmune T-cell repertoire that promotes cognition. In another hand, the evolutionary-determined emergence of brain-autoreactive T-cells has exerted a selection pressure on neural genes coding for brain superautoantigens. Such a selection pressure has translated into the emergence of a neural repertoire (defined here as the whole of neurons, synapses and non-neuronal cells involved in cognitive functions expressing brain superautoantigens. Overall, the brain superautoantigens theory suggests that cognitive evolution might have been primarily driven by internal cues rather than external environmental conditions. Importantly, while providing a unique molecular connection between neural and T-cell repertoires under physiological conditions, brain superautoantigens may also constitute an Achilles heel responsible for the particular susceptibility of Homo sapiens to “neuroimmune co-pathologies” i.e., disorders affecting both neural and T-cell repertoires. These may notably include paraneoplastic syndromes, multiple sclerosis as well as autism, schizophrenia and neurodegenerative diseases. In the context of this theoretical frame, a specific

Heparan Sulfate Proteoglycans as Drivers of Neural Progenitors Derived From Human Mesenchymal Stem Cells.

Science.gov (United States)

Okolicsanyi, Rachel K; Oikari, Lotta E; Yu, Chieh; Griffiths, Lyn R; Haupt, Larisa M

2018-01-01

Background: Due to their relative ease of isolation and their high ex vivo and in vitro expansive potential, human mesenchymal stem cells (hMSCs) are an attractive candidate for therapeutic applications in the treatment of brain injury and neurological diseases. Heparan sulfate proteoglycans (HSPGs) are a family of ubiquitous proteins involved in a number of vital cellular processes including proliferation and stem cell lineage differentiation. Methods: Following the determination that hMSCs maintain neural potential throughout extended in vitro expansion, we examined the role of HSPGs in mediating the neural potential of hMSCs. hMSCs cultured in basal conditions (undifferentiated monolayer cultures) were found to co-express neural markers and HSPGs throughout expansion with modulation of the in vitro niche through the addition of exogenous HS influencing cellular HSPG and neural marker expression. Results: Conversion of hMSCs into hMSC Induced Neurospheres (hMSC IN) identified distinctly localized HSPG staining within the spheres along with altered gene expression of HSPG core protein and biosynthetic enzymes when compared to undifferentiated hMSCs. Conclusion: Comparison of markers of pluripotency, neural self-renewal and neural lineage specification between hMSC IN, hMSC and human neural stem cell (hNSC H9) cultures suggest that in vitro generated hMSC IN may represent an intermediary neurogenic cell type, similar to a common neural progenitor cell. In addition, this data demonstrates HSPGs and their biosynthesis machinery, are associated with hMSC IN formation. The identification of specific HSPGs driving hMSC lineage-specification will likely provide new markers to allow better use of hMSCs in therapeutic applications and improve our understanding of human neurogenesis.

Cognitive problems, self-rated changes in driving skills, driving-related discomfort and self-regulation of driving in old drivers

DEFF Research Database (Denmark)

Meng, Annette; Siren, Anu Kristiina

2012-01-01

Ageing in general is associated with functional decline that may have an adverse effect on driving. Nevertheless, older drivers have been found to show good judgement and to self-regulate their driving, which may enable them to continue driving safely despite functional decline. The process...... of the self-monitoring of driving ability and the awareness of functional decline, and its association with the self-regulation of driving is, however, not fully understood. The aim of the present study was to examine the perceived changes in driving skills, the discomfort experienced in driving, and the self......-related discomfort is an important factor affecting the self-regulation of driving. Finally, the findings indicate that driving-related discomfort functions as an indirect self-monitoring of driving ability and may contribute to the safe driving performance of Danish older drivers....

Facilitating tolerance of delayed reinforcement during functional communication training.

Science.gov (United States)

Fisher, W W; Thompson, R H; Hagopian, L P; Bowman, L G; Krug, A

2000-01-01

Few clinical investigations have addressed the problem of delayed reinforcement. In this investigation, three individuals whose destructive behavior was maintained by positive reinforcement were treated using functional communication training (FCT) with extinction (EXT). Next, procedures used in the basic literature on delayed reinforcement and self-control (reinforcer delay fading, punishment of impulsive responding, and provision of an alternative activity during reinforcer delay) were used to teach participants to tolerate delayed reinforcement. With the first case, reinforcer delay fading alone was effective at maintaining low rates of destructive behavior while introducing delayed reinforcement. In the second case, the addition of a punishment component reduced destructive behavior to near-zero levels and facilitated reinforcer delay fading. With the third case, reinforcer delay fading was associated with increases in masturbation and head rolling, but prompting and praising the individual for completing work during the delay interval reduced all problem behaviors and facilitated reinforcer delay fading.

Reinforced Conductive Polyaniline-Paper Composites

Directory of Open Access Journals (Sweden)

Jinhua Yan

2015-05-01

Full Text Available A method for direct aniline interfacial polymerization on polyamideamine-epichlorohydrin (PAE-reinforced paper substrate is introduced in this paper. Cellulose-based papers with and without reinforcement were considered. The polyaniline (PANI-paper composites had surface resistivity lower than 100 Ω/sq after more than 3 polymerizations. Their mechanical strength and thermal stability were analyzed by tensile tests and thermogravimetric analysis (TGA. Fourier transform infrared (FTIR results revealed that there was strong interaction between NH groups in aniline monomers and OH groups in fibers, which did not disappear until after 3 polymerizations. Scanning electron microscopy (SEM and field emission (FE SEM images showed morphological differences between composites using reinforced and untreated base papers. Conductive composites made with PAE-reinforced base paper had both good thermal stability and good mechanical strength, with high conductivity and a smaller PANI amount.

Superelastic SMA–FRP composite reinforcement for concrete structures

International Nuclear Information System (INIS)

Wierschem, Nicholas; Andrawes, Bassem

2010-01-01

For many years there has been interest in using fiber-reinforced polymers (FRPs) as reinforcement in concrete structures. Unfortunately, due to their linear elastic behavior, FRP reinforcing bars are never considered for structural damping or dynamic applications. With the aim of improving the ductility and damping capability of concrete structures reinforced with FRP reinforcement, this paper studies the application of SMA–FRP, a relatively novel type of composite reinforced with superelastic shape memory alloy (SMA) wires. The cyclic tensile behavior of SMA–FRP composites are studied experimentally and analytically. Tests of SMA–FRP composite coupons are conducted to determine their constitutive behavior. The experimental results are used to develop and calibrate a uniaxial SMA–FRP analytical model. Parametric and case studies are performed to determine the efficacy of the SMA–FRP reinforcement in concrete structures and the key factors governing its behavior. The results show significant potential for SMA–FRP reinforcement to improve the ductility and damping of concrete structures while still maintaining its elastic characteristic, typical of FRP reinforcement

Positive behavioral contrast across food and alcohol reinforcers.

OpenAIRE

McSweeney, F K; Melville, C L; Higa, J

1988-01-01

The present study examined behavioral contrast during concurrent and multiple schedules that provided food and alcohol reinforcers. Concurrent-schedule contrast occurred in the responding reinforced by food when alcohol reinforcers were removed. It also occurred in the responding reinforced by alcohol when food was removed. Multiple-schedule contrast appeared for food when alcohol reinforcers were removed, but not for alcohol when food was removed. These results show that behavioral contrast ...

Brain imaging studies of the cocaine addict: Implications for reinforcement and addiction

International Nuclear Information System (INIS)

Volkow, N.D.; Fowler, J.S.; SUNY, Stony Brook, Stony Brook, NY

1995-01-01

These studies document dopaminergic abnormalities in cocaine abusers. They also suggest a regulatory role of Dopamine (DA) in frontal metabolism. The correlation of striatal D 2 receptor availability with metabolism was strongest for orbital frontal cortex (OFC) cingulate and prefrontal cortices. In cocaine abusers tested during early withdrawal (<1 week) the OFC was found to be hypermetabolic and metabolism in OFC and prefrontal cortices were found to be significantly associated with cocaine craving . Thus, we postulate that repeated and intermittent DA stimulation, as seen during a cocaine binge, activates the prefrontal and OFC cortices increasing the drive to compulsively self-administer cocaine. During cocaine discontinuation and protracted withdrawal and with decreased DA stimulation, these frontal cortical regions become hyponietabolic. Dopaminergic stimulation by a DA-enhancing drug and/or environmental conditioning will reactivate these frontal regions resetting the compulsion to self-administer cocaine and the inability to terminate this behavior. The pharmacokionetic studies with [11C]cocaine are consistent with behavioral and pharmacological studies in animals as well as in vitro studies which have revealed that while the mechanisms for cocaine's reinforcing properties are complex, they partly involve the brain's dopamine system and also highlight the importance of cocaine's pharmacokinetic on its unique reinforcing properties

Brain imaging studies of the cocaine addict: Implications for reinforcement and addiction

Energy Technology Data Exchange (ETDEWEB)

Volkow, N.D.; Fowler, J.S. [Brookhaven National Lab., Upton, NY (United States)]|[SUNY, Stony Brook, Stony Brook, NY (United States). Dept. of Psychiatry

1995-07-01

These studies document dopaminergic abnormalities in cocaine abusers. They also suggest a regulatory role of Dopamine (DA) in frontal metabolism. The correlation of striatal D{sub 2} receptor availability with metabolism was strongest for orbital frontal cortex (OFC) cingulate and prefrontal cortices. In cocaine abusers tested during early withdrawal (<1 week) the OFC was found to be hypermetabolic and metabolism in OFC and prefrontal cortices were found to be significantly associated with cocaine craving . Thus, we postulate that repeated and intermittent DA stimulation, as seen during a cocaine binge, activates the prefrontal and OFC cortices increasing the drive to compulsively self-administer cocaine. During cocaine discontinuation and protracted withdrawal and with decreased DA stimulation, these frontal cortical regions become hyponietabolic. Dopaminergic stimulation by a DA-enhancing drug and/or environmental conditioning will reactivate these frontal regions resetting the compulsion to self-administer cocaine and the inability to terminate this behavior. The pharmacokionetic studies with [11C]cocaine are consistent with behavioral and pharmacological studies in animals as well as in vitro studies which have revealed that while the mechanisms for cocaine`s reinforcing properties are complex, they partly involve the brain`s dopamine system and also highlight the importance of cocaine`s pharmacokinetic on its unique reinforcing properties.

Learning to trade via direct reinforcement.

Science.gov (United States)

Moody, J; Saffell, M

2001-01-01

We present methods for optimizing portfolios, asset allocations, and trading systems based on direct reinforcement (DR). In this approach, investment decision-making is viewed as a stochastic control problem, and strategies are discovered directly. We present an adaptive algorithm called recurrent reinforcement learning (RRL) for discovering investment policies. The need to build forecasting models is eliminated, and better trading performance is obtained. The direct reinforcement approach differs from dynamic programming and reinforcement algorithms such as TD-learning and Q-learning, which attempt to estimate a value function for the control problem. We find that the RRL direct reinforcement framework enables a simpler problem representation, avoids Bellman's curse of dimensionality and offers compelling advantages in efficiency. We demonstrate how direct reinforcement can be used to optimize risk-adjusted investment returns (including the differential Sharpe ratio), while accounting for the effects of transaction costs. In extensive simulation work using real financial data, we find that our approach based on RRL produces better trading strategies than systems utilizing Q-learning (a value function method). Real-world applications include an intra-daily currency trader and a monthly asset allocation system for the S&P 500 Stock Index and T-Bills.

Enhancement of shear strength and ductility for reinforced concrete wide beams due to web reinforcement

Directory of Open Access Journals (Sweden)

M. Said

2013-12-01

Full Text Available The shear behavior of reinforced concrete wide beams was investigated. The experimental program consisted of nine beams of 29 MPa concrete strength tested with a shear span-depth ratio equal to 3.0. One of the tested beams had no web reinforcement as a control specimen. The flexure mode of failure was secured for all of the specimens to allow for shear mode of failure. The key parameters covered in this investigation are the effect of the existence, spacing, amount and yield stress of the vertical stirrups on the shear capacity and ductility of the tested wide beams. The study shows that the contribution of web reinforcement to the shear capacity is significant and directly proportional to the amount and spacing of the shear reinforcement. The increase in the shear capacity ranged from 32% to 132% for the range of the tested beams compared with the control beam. High grade steel was more effective in the contribution of the shear strength of wide beams. Also, test results demonstrate that the shear reinforcement significantly enhances the ductility of the wide beams. In addition, shear resistances at failure recorded in this study are compared to the analytical strengths calculated according to the current Egyptian Code and the available international codes. The current study highlights the need to include the contribution of shear reinforcement in the Egyptian Code requirements for shear capacity of wide beams.

Fibre Reinforced Polymer Composites as Internal and External Reinforcements for Building Elements

Directory of Open Access Journals (Sweden)

Cătălin Banu

2008-01-01

Full Text Available During the latest decades fibre reinforced polymer (FRP composite materials have proven valuable properties and suitable to be used in construction of new buildings and in upgrading the existing ones. These materials have covered the road from research laboratory and demonstration projects to implementation in actual structures. Nowadays the civil and structural engineering communities are about to commence the stage in which the use of FRP composites is becoming a routine similar to that of traditional material such as concrete, masonry and wood. Two main issues are presented in this paper, the use of FRP composite materials for new structural members (internal reinforcements and strengthening of existing members (externally bonded reinforcements. The advantages and disadvantages as well as the problems and constraints associated with both issues are discussed in detail mainly related to concrete members.

Hedonic Hunger Is Related to Increased Neural and Perceptual Responses to Cues of Palatable Food and Motivation to Consume: Evidence from 3 Independent Investigations12

Science.gov (United States)

Burger, Kyle S; Sanders, Abigail J; Gilbert, Jennifer R

2016-01-01

Background: The Power of Food Scale (PFS) seeks to identify individuals who experience high appetitive drive in response to food cues, which is a construct termed “hedonic hunger.” Objective: The purpose of this study was to assess cross-sectional correlates and predictive power of PFS scores to probe the construct of hedonic hunger. Methods: Separate data from 3 studies (study 1, n = 44; study 2, n = 398; study 3, n = 100) were used to evaluate the construct of hedonic hunger. We examined the correlations between the PFS and neural responsivity during intake and anticipated intake of palatable foods, behavioral food reinforcement, perceptual hedonic ratings of food images, and change in body mass index (BMI) and binge eating over time. Results: Hedonic hunger was strongly related to bilateral brain response in regions implicated in oral somatosensory processing during cue-elicited anticipation of food intake (study 1; right postcentral gyrus: r = 0.67, P hunger was not associated with baseline BMI (studies 1–3: P = 0.14, 0.21, and 0.37, respectively) or change in BMI over the 2-y follow-up (studies 1 and 2: P = 0.14 and 0.37, respectively) but was significantly correlated with baseline binge eating in 2 samples (study 1: r = 0.58, P = 0.001; study 2: r = 0.31, P = 0.02; and study 3: P = 0.02). Conclusions: Hedonic hunger was not predictive of weight regulation. However, individuals who report high hedonic hunger are likely to show increased neural and perceptual responses to cues of palatable foods, increased motivation to consume such foods, and a greater likelihood of current binge eating. PMID:27489006

Strength Characteristics of Reinforced Sandy Soil

OpenAIRE

S. N. Bannikov; Mahamed Al Fayez

2005-01-01

Laboratory tests on determination of reinforced sandy soil strength characteristics (angle of internal friction, specific cohesive force) have been carried out with the help of a specially designed instrument and proposed methodology. Analysis of the obtained results has revealed that cohesive forces are brought about in reinforced sandy soil and an angle of internal soil friction becomes larger in comparison with non-reinforced soil.

Analytical, Numerical and Experimental Examination of Reinforced Composites Beams Covered with Carbon Fiber Reinforced Plastic

Science.gov (United States)

Kasimzade, A. A.; Tuhta, S.

2012-03-01

In the article, analytical, numerical (Finite Element Method) and experimental investigation results of beam that was strengthened with fiber reinforced plastic-FRP composite has been given as comparative, the effect of FRP wrapping number to the maximum load and moment capacity has been evaluated depending on this results. Carbon FRP qualitative dependences have been occurred between wrapping number and beam load and moment capacity for repair-strengthen the reinforced concrete beams with carbon fiber. Shown possibilities of application traditional known analysis programs, for the analysis of Carbon Fiber Reinforced Plastic (CFRP) strengthened structures.

Retrospective revaluation and its neural circuit in rats.

Science.gov (United States)

San-Galli, Aurore; Marchand, Alain R; Decorte, Laurence; Di Scala, Georges

2011-10-01

Contingency learning is essential for establishing predictive or causal judgements. Retrospective revaluation captures essential aspects of the updating of this knowledge, according to new experience. In the present study, retrospective revaluation and its neural substrate was investigated in a rat conditioned magazine approach. One element of a previously food-reinforced Tone-Light compound stimulus was either further reinforced (inflation) or extinguished (extinction). These treatments affected the predictive value of the alternate stimulus (target), but only when the target was a weakly salient stimulus such as a Light, and the inflation/extinction procedure concerned the more salient element, that is the Tone. As the predictive value of the Light was decreased in comparison with a relevant control group, this revaluation was interpreted as backward blocking, and not unovershadowing. This observation challenges retrospective revaluation models focused on acquisition and prediction error detection, and is better accounted for by retrieval-based associative theories such as the comparator model (Miller and Matzel) [5]. Immunohistochemical detection of the Fos protein after the test phase revealed activation of the orbitofrontal and infralimbic cortices as well as nucleus accumbens core and shell, in rats that exhibited retrospective revaluation. Our results suggest that rats integrate successive experiences at the retrieval stage of retrospective revaluation, and that prefronto-accumbal interactions are involved in this function. Copyright © 2011 Elsevier B.V. All rights reserved.

Producing Durable Continuously Reinforced Concrete Pavement using Glass-ceramic Coated Reinforcing Steel

Science.gov (United States)

2010-02-01

reinforcement if the enamel is broken  Embedded cement grains hydrate if enamel is cracked to self-heal with the formation of calcium silicate hydrate Goal...Reinforced Concrete Pavement The 600% volume change in the iron to iron oxide formation put the concrete in tension and it cracks an spalls BUILDING...corrodes prematurely and delaminates the pavement  Moisture and chlorides can move through the natural porosity of concrete and the cracks in the

Constitutive model for reinforced concrete

NARCIS (Netherlands)

Feenstra, P.H.; Borst, de R.

1995-01-01

A numerical model is proposed for reinforced-concrete behavior that combines the commonly accepted ideas from modeling plain concrete, reinforcement, and interaction behavior in a consistent manner. The behavior of plain concrete is govern by fracture-energy-level-based formulation both in tension

Tangible Reinforcers: Bonuses or Bribes?

Science.gov (United States)

O'Leary, K. Daniel; And Others

1972-01-01

Objections to the use of tangible reinforcers, such as prizes, candy, cigarettes, and money, are discussed. Treatment programs using tangible reinforcers are recommended as powerful modifers of behavior to be implemented only after less powerful means of modification have been tried. (Author)

Pharmacological stimuli decreasing nucleus accumbens dopamine can act as positive reinforcers but have a low addictive potential.

Science.gov (United States)

Marinelli, M; Barrot, M; Simon, H; Oberlander, C; Dekeyne, A; Le Moal, M; Piazza, P V

1998-10-01

Opioid peptides, through mu and delta receptors, play an important part in reward. In contrast, the role of kappa receptors is more controversial. We examined the possible positive reinforcing effects of a selective kappa agonist, RU 51599, by studying intravenous self-administration in the rat. The effect of RU 51599 on dopamine release in the nucleus accumbens was also studied, as opioids and dopamine seem to interact in the mediation of reward. The behavioural and dopaminergic effects of RU 51599 were compared with those of the mu agonist heroin. Rats self-administered both RU 51599 (6.5, 20 and 60 microg/inj) and heroin (30 microg/inj) at low ratio requirement. When the ratio requirement, i.e. the number of responses necessary to receive one drug infusion, was increased, self-administration of RU 51599 rapidly extinguished, whereas self-administration of heroin was maintained. Intravenous infusion of RU 51599 (100, 200 and 400 microg) dose-dependently decreased (25, 30 and 40%, respectively) extracellular concentrations of dopamine, as measured by means of microdialysis in freely moving rats. In contrast, heroin increased accumbens dopamine (130% over baseline). These results indicate that kappa receptors, similarly to mu ones, can mediate positive reinforcing effects of opioid peptides. However, the strength of the reinforcement is very low for kappa receptors. This suggests that changes in accumbens dopamine do not correlate with the capacity of a stimulus to induce reward or aversion. In contrast, a parallel seems to exist between an increase in accumbens dopamine and the drive to reach or obtain a positive reinforcer.

Dissecting neural pathways for forgetting in Drosophila olfactory aversive memory.

Science.gov (United States)

Shuai, Yichun; Hirokawa, Areekul; Ai, Yulian; Zhang, Min; Li, Wanhe; Zhong, Yi

2015-12-01

Recent studies have identified molecular pathways driving forgetting and supported the notion that forgetting is a biologically active process. The circuit mechanisms of forgetting, however, remain largely unknown. Here we report two sets of Drosophila neurons that account for the rapid forgetting of early olfactory aversive memory. We show that inactivating these neurons inhibits memory decay without altering learning, whereas activating them promotes forgetting. These neurons, including a cluster of dopaminergic neurons (PAM-β'1) and a pair of glutamatergic neurons (MBON-γ4>γ1γ2), terminate in distinct subdomains in the mushroom body and represent parallel neural pathways for regulating forgetting. Interestingly, although activity of these neurons is required for memory decay over time, they are not required for acute forgetting during reversal learning. Our results thus not only establish the presence of multiple neural pathways for forgetting in Drosophila but also suggest the existence of diverse circuit mechanisms of forgetting in different contexts.

Neural basis of the undermining effect of monetary reward on intrinsic motivation.

Science.gov (United States)

Murayama, Kou; Matsumoto, Madoka; Izuma, Keise; Matsumoto, Kenji

2010-12-07

Contrary to the widespread belief that people are positively motivated by reward incentives, some studies have shown that performance-based extrinsic reward can actually undermine a person's intrinsic motivation to engage in a task. This "undermining effect" has timely practical implications, given the burgeoning of performance-based incentive systems in contemporary society. It also presents a theoretical challenge for economic and reinforcement learning theories, which tend to assume that monetary incentives monotonically increase motivation. Despite the practical and theoretical importance of this provocative phenomenon, however, little is known about its neural basis. Herein we induced the behavioral undermining effect using a newly developed task, and we tracked its neural correlates using functional MRI. Our results show that performance-based monetary reward indeed undermines intrinsic motivation, as assessed by the number of voluntary engagements in the task. We found that activity in the anterior striatum and the prefrontal areas decreased along with this behavioral undermining effect. These findings suggest that the corticobasal ganglia valuation system underlies the undermining effect through the integration of extrinsic reward value and intrinsic task value.

TEACHING SELF-CONTROL WITH QUALITATIVELY DIFFERENT REINFORCERS

OpenAIRE

Passage, Michael; Tincani, Matt; Hantula, Donald A.

2012-01-01

This study examined the effectiveness of using qualitatively different reinforcers to teach self-control to an adolescent boy who had been diagnosed with an intellectual disability. First, he was instructed to engage in an activity without programmed reinforcement. Next, he was instructed to engage in the activity under a two-choice fixed-duration schedule of reinforcement. Finally, he was exposed to self-control training, during which the delay to a more preferred reinforcer was initially sh...

Reinforcement learning for a biped robot based on a CPG-actor-critic method.

Science.gov (United States)

Nakamura, Yutaka; Mori, Takeshi; Sato, Masa-aki; Ishii, Shin

2007-08-01

Animals' rhythmic movements, such as locomotion, are considered to be controlled by neural circuits called central pattern generators (CPGs), which generate oscillatory signals. Motivated by this biological mechanism, studies have been conducted on the rhythmic movements controlled by CPG. As an autonomous learning framework for a CPG controller, we propose in this article a reinforcement learning method we call the "CPG-actor-critic" method. This method introduces a new architecture to the actor, and its training is roughly based on a stochastic policy gradient algorithm presented recently. We apply this method to an automatic acquisition problem of control for a biped robot. Computer simulations show that training of the CPG can be successfully performed by our method, thus allowing the biped robot to not only walk stably but also adapt to environmental changes.

DrivingSense: Dangerous Driving Behavior Identification Based on Smartphone Autocalibration

Directory of Open Access Journals (Sweden)

Chunmei Ma

2017-01-01

Full Text Available Since pervasive smartphones own advanced computing capability and are equipped with various sensors, they have been used for dangerous driving behaviors detection, such as drunk driving. However, sensory data gathered by smartphones are noisy, which results in inaccurate driving behaviors estimations. Some existing works try to filter noise from sensor readings, but usually only the outlier data are filtered. The noises caused by hardware of the smartphone cannot be removed from the sensor reading. In this paper, we propose DrivingSense, a reliable dangerous driving behavior identification scheme based on smartphone autocalibration. We first theoretically analyze the impact of the sensor error on the vehicle driving behavior estimation. Then, we propose a smartphone autocalibration algorithm based on sensor noise distribution determination when a vehicle is being driven. DrivingSense leverages the corrected sensor parameters to identify three kinds of dangerous behaviors: speeding, irregular driving direction change, and abnormal speed control. We evaluate the effectiveness of our scheme under realistic environments. The results show that DrivingSense, on average, is able to detect the driving direction change event and abnormal speed control event with 93.95% precision and 90.54% recall, respectively. In addition, the speed estimation error is less than 2.1 m/s, which is an acceptable range.

The role of attention in the tinnitus decompensation: reinforcement of a large-scale neural decompensation measure.

Science.gov (United States)

Low, Yin Fen; Trenado, Carlos; Delb, Wolfgang; Corona-Strauss, Farah I; Strauss, Daniel J

2007-01-01

Large-scale neural correlates of the tinnitus decompensation have been identified by using wavelet phase stability criteria of single sweep sequences of auditory late responses (ALRs). The suggested measure provided an objective quantification of the tinnitus decompensation and allowed for a reliable discrimination between a group of compensated and decompensated tinnitus patients. By interpreting our results with an oscillatory tinnitus model, our synchronization stability measure of ALRs can be linked to the focus of attention on the tinnitus signal. In the following study, we examined in detail the correlates of this attentional mechanism in healthy subjects. The results support our previous findings of the phase synchronization stability measure that reflected neural correlates of the fixation of attention to the tinnitus signal. In this case, enabling the differentiation between the attended and unattended conditions. It is concluded that the wavelet phase synchronization stability of ALRs single sweeps can be used as objective tinnitus decompensation measure and can be interpreted in the framework of the Jastreboff tinnitus model and adaptive resonance theory. Our studies confirm that the synchronization stability in ALR sequences is linked to attention. This measure is not only able to serve as objective quantification of the tinnitus decompensation, but also can be applied in all online and real time neurofeedback therapeutic approach where a direct stimulus locked attention monitoring is compulsory as if it based on a single sweeps processing.

PARTIAL REINFORCEMENT (ACQUISITION) EFFECTS WITHIN SUBJECTS.

Science.gov (United States)

AMSEL, A; MACKINNON, J R; RASHOTTE, M E; SURRIDGE, C T

1964-03-01

Acquisition performance of 22 rats in a straight alley runway was examined. The animals were subjected to partial reinforcement when the alley was black (B+/-) and continuous reinforcement when it was white (W+). The results indicated (a) higher terminal performance, for partial as against continuous reinforcement conditions, for starting-time and running-time measures, and (b) lower terminal performance under partial conditions for a goal-entry-time measure. These results confirm within subjects an effect previously demonstrated, in the runway, only in between-groups tests, where one group is run under partial reinforcement and a separate group is run under continuous reinforcement in the presence of the same external stimuli. Differences between the runway situation, employing a discrete-trial procedure and performance measures at three points in the response chain, and the Skinner box situation, used in its free-operant mode with a single performance measure, are discussed in relation to the present findings.

Neural Population Dynamics during Reaching Are Better Explained by a Dynamical System than Representational Tuning.

Science.gov (United States)

Michaels, Jonathan A; Dann, Benjamin; Scherberger, Hansjörg

2016-11-01

Recent models of movement generation in motor cortex have sought to explain neural activity not as a function of movement parameters, known as representational models, but as a dynamical system acting at the level of the population. Despite evidence supporting this framework, the evaluation of representational models and their integration with dynamical systems is incomplete in the literature. Using a representational velocity-tuning based simulation of center-out reaching, we show that incorporating variable latency offsets between neural activity and kinematics is sufficient to generate rotational dynamics at the level of neural populations, a phenomenon observed in motor cortex. However, we developed a covariance-matched permutation test (CMPT) that reassigns neural data between task conditions independently for each neuron while maintaining overall neuron-to-neuron relationships, revealing that rotations based on the representational model did not uniquely depend on the underlying condition structure. In contrast, rotations based on either a dynamical model or motor cortex data depend on this relationship, providing evidence that the dynamical model more readily explains motor cortex activity. Importantly, implementing a recurrent neural network we demonstrate that both representational tuning properties and rotational dynamics emerge, providing evidence that a dynamical system can reproduce previous findings of representational tuning. Finally, using motor cortex data in combination with the CMPT, we show that results based on small numbers of neurons or conditions should be interpreted cautiously, potentially informing future experimental design. Together, our findings reinforce the view that representational models lack the explanatory power to describe complex aspects of single neuron and population level activity.

Reinforcement Learning in Repeated Portfolio Decisions

OpenAIRE

Diao, Linan; Rieskamp, Jörg

2011-01-01

How do people make investment decisions when they receive outcome feedback? We examined how well the standard mean-variance model and two reinforcement models predict people's portfolio decisions. The basic reinforcement model predicts a learning process that relies solely on the portfolio's overall return, whereas the proposed extended reinforcement model also takes the risk and covariance of the investments into account. The experimental results illustrate that people reacted sensitively to...

Learning by stimulation avoidance: A principle to control spiking neural networks dynamics.

Science.gov (United States)

Sinapayen, Lana; Masumori, Atsushi; Ikegami, Takashi

2017-01-01

Learning based on networks of real neurons, and learning based on biologically inspired models of neural networks, have yet to find general learning rules leading to widespread applications. In this paper, we argue for the existence of a principle allowing to steer the dynamics of a biologically inspired neural network. Using carefully timed external stimulation, the network can be driven towards a desired dynamical state. We term this principle "Learning by Stimulation Avoidance" (LSA). We demonstrate through simulation that the minimal sufficient conditions leading to LSA in artificial networks are also sufficient to reproduce learning results similar to those obtained in biological neurons by Shahaf and Marom, and in addition explains synaptic pruning. We examined the underlying mechanism by simulating a small network of 3 neurons, then scaled it up to a hundred neurons. We show that LSA has a higher explanatory power than existing hypotheses about the response of biological neural networks to external simulation, and can be used as a learning rule for an embodied application: learning of wall avoidance by a simulated robot. In other works, reinforcement learning with spiking networks can be obtained through global reward signals akin simulating the dopamine system; we believe that this is the first project demonstrating sensory-motor learning with random spiking networks through Hebbian learning relying on environmental conditions without a separate reward system.

Memory Transformation Enhances Reinforcement Learning in Dynamic Environments.

Science.gov (United States)

Santoro, Adam; Frankland, Paul W; Richards, Blake A

2016-11-30

Over the course of systems consolidation, there is a switch from a reliance on detailed episodic memories to generalized schematic memories. This switch is sometimes referred to as "memory transformation." Here we demonstrate a previously unappreciated benefit of memory transformation, namely, its ability to enhance reinforcement learning in a dynamic environment. We developed a neural network that is trained to find rewards in a foraging task where reward locations are continuously changing. The network can use memories for specific locations (episodic memories) and statistical patterns of locations (schematic memories) to guide its search. We find that switching from an episodic to a schematic strategy over time leads to enhanced performance due to the tendency for the reward location to be highly correlated with itself in the short-term, but regress to a stable distribution in the long-term. We also show that the statistics of the environment determine the optimal utilization of both types of memory. Our work recasts the theoretical question of why memory transformation occurs, shifting the focus from the avoidance of memory interference toward the enhancement of reinforcement learning across multiple timescales. As time passes, memories transform from a highly detailed state to a more gist-like state, in a process called "memory transformation." Theories of memory transformation speak to its advantages in terms of reducing memory interference, increasing memory robustness, and building models of the environment. However, the role of memory transformation from the perspective of an agent that continuously acts and receives reward in its environment is not well explored. In this work, we demonstrate a view of memory transformation that defines it as a way of optimizing behavior across multiple timescales. Copyright © 2016 the authors 0270-6474/16/3612228-15$15.00/0.

Driving Style Analysis Using Primitive Driving Patterns With Bayesian Nonparametric Approaches

OpenAIRE

Wang, Wenshuo; Xi, Junqiang; Zhao, Ding

2017-01-01

Analysis and recognition of driving styles are profoundly important to intelligent transportation and vehicle calibration. This paper presents a novel driving style analysis framework using the primitive driving patterns learned from naturalistic driving data. In order to achieve this, first, a Bayesian nonparametric learning method based on a hidden semi-Markov model (HSMM) is introduced to extract primitive driving patterns from time series driving data without prior knowledge of the number...

Responding for sucrose and wheel-running reinforcement: effects of sucrose concentration and wheel-running reinforcer duration.

Science.gov (United States)

Belke, Terry W; Hancock, Stephanie D

2003-03-01

Six male albino rats were placed in running wheels and exposed to a fixed-interval 30-s schedule of lever pressing that produced either a drop of sucrose solution or the opportunity to run for a fixed duration as reinforcers. Each reinforcer type was signaled by a different stimulus. In Experiment 1, the duration of running was held constant at 15 s while the concentration of sucrose solution was varied across values of 0, 2.5. 5, 10, and 15%. As concentration decreased, postreinforcement pause duration increased and local rates decreased in the presence of the stimulus signaling sucrose. Consequently, the difference between responding in the presence of stimuli signaling wheel-running and sucrose reinforcers diminished, and at 2.5%, response functions for the two reinforcers were similar. In Experiment 2, the concentration of sucrose solution was held constant at 15% while the duration of the opportunity to run was first varied across values of 15, 45, and 90 s then subsequently across values of 5, 10, and 15 s. As run duration increased, postreinforcement pause duration in the presence of the wheel-running stimulus increased and local rates increased then decreased. In summary, inhibitory aftereffects of previous reinforcers occurred when both sucrose concentration and run duration varied; changes in responding were attributable to changes in the excitatory value of the stimuli signaling the two reinforcers.

Neural stem cell-encoded temporal patterning delineates an early window of malignant susceptibility in Drosophila.

Science.gov (United States)

Narbonne-Reveau, Karine; Lanet, Elodie; Dillard, Caroline; Foppolo, Sophie; Chen, Ching-Huan; Parrinello, Hugues; Rialle, Stéphanie; Sokol, Nicholas S; Maurange, Cédric

2016-06-14

Pediatric neural tumors are often initiated during early development and can undergo very rapid transformation. However, the molecular basis of this early malignant susceptibility remains unknown. During Drosophila development, neural stem cells (NSCs) divide asymmetrically and generate intermediate progenitors that rapidly differentiate in neurons. Upon gene inactivation, these progeny can dedifferentiate and generate malignant tumors. Here, we find that intermediate progenitors are prone to malignancy only when born during an early window of development while expressing the transcription factor Chinmo, and the mRNA-binding proteins Imp/IGF2BP and Lin-28. These genes compose an oncogenic module that is coopted upon dedifferentiation of early-born intermediate progenitors to drive unlimited tumor growth. In late larvae, temporal transcription factor progression in NSCs silences the module, thereby limiting mitotic potential and terminating the window of malignant susceptibility. Thus, this study identifies the gene regulatory network that confers malignant potential to neural tumors with early developmental origins.

Control rod drive

International Nuclear Information System (INIS)

Okutani, Tetsuro.

1988-01-01

Purpose: To provide a simple and economical control rod drive using a control circuit requiring no pulse circuit. Constitution: Control rods in a BWR type reactor are driven by hydraulic pressure and inserted or withdrawn in the direction of applying the hydraulic pressure. The direction of the hydraulic pressure is controlled by a direction control valve. Since the driving for the control rod is extremely important in view of the operation, a self diagnosis function is disposed for rapid inspection of possible abnormality. In the present invention, two driving contacts are disposed each by one between the both ends of a solenoid valve of the direction control valve for driving the control rod and the driving power source, and diagnosis is conducted by alternately operating them. Therefore, since it is only necessary that the control circuit issues a driving instruction only to one of the two driving contacts, the pulse circuit is no more required. Further, since the control rod driving is conducted upon alignment of the two driving instructions, the reliability of the control rod drive can be improved. (Horiuchi, T.)

Reinforcement learning on slow features of high-dimensional input streams.

Directory of Open Access Journals (Sweden)

Robert Legenstein

Full Text Available Humans and animals are able to learn complex behaviors based on a massive stream of sensory information from different modalities. Early animal studies have identified learning mechanisms that are based on reward and punishment such that animals tend to avoid actions that lead to punishment whereas rewarded actions are reinforced. However, most algorithms for reward-based learning are only applicable if the dimensionality of the state-space is sufficiently small or its structure is sufficiently simple. Therefore, the question arises how the problem of learning on high-dimensional data is solved in the brain. In this article, we propose a biologically plausible generic two-stage learning system that can directly be applied to raw high-dimensional input streams. The system is composed of a hierarchical slow feature analysis (SFA network for preprocessing and a simple neural network on top that is trained based on rewards. We demonstrate by computer simulations that this generic architecture is able to learn quite demanding reinforcement learning tasks on high-dimensional visual input streams in a time that is comparable to the time needed when an explicit highly informative low-dimensional state-space representation is given instead of the high-dimensional visual input. The learning speed of the proposed architecture in a task similar to the Morris water maze task is comparable to that found in experimental studies with rats. This study thus supports the hypothesis that slowness learning is one important unsupervised learning principle utilized in the brain to form efficient state representations for behavioral learning.

Conditioned Reinforcement Value and Resistance to Change

Science.gov (United States)

Shahan, Timothy A.; Podlesnik, Christopher A.

2008-01-01

Three experiments examined the effects of conditioned reinforcement value and primary reinforcement rate on resistance to change using a multiple schedule of observing-response procedures with pigeons. In the absence of observing responses in both components, unsignaled periods of variable-interval (VI) schedule food reinforcement alternated with…

Dimensions of driving anger and their relationships with aberrant driving.

Science.gov (United States)

Zhang, Tingru; Chan, Alan H S; Zhang, Wei

2015-08-01

The purpose of this study was to investigate the relationship between driving anger and aberrant driving behaviours. An internet-based questionnaire survey was administered to a sample of Chinese drivers, with driving anger measured by a 14-item short Driving Anger Scale (DAS) and the aberrant driving behaviours measured by a 23-item Driver Behaviour Questionnaire (DBQ). The results of Confirmatory Factor Analysis demonstrated that the three-factor model (hostile gesture, arrival-blocking and safety-blocking) of the DAS fitted the driving anger data well. The Exploratory Factor Analysis on DBQ data differentiated four types of aberrant driving, viz. emotional violation, error, deliberate violation and maintaining progress violation. For the anger-aberration relation, it was found that only "arrival-blocking" anger was a significant positive predictor for all four types of aberrant driving behaviours. The "safety-blocking" anger revealed a negative impact on deliberate violations, a finding different from previously established positive anger-aberration relation. These results suggest that drivers with different patterns of driving anger would show different behavioural tendencies and as a result intervention strategies may be differentially effective for drivers of different profiles. Copyright © 2015 Elsevier Ltd. All rights reserved.

Glass FRP reinforcement in rehabilitation of concrete marine infrastructure

International Nuclear Information System (INIS)

Newhook, John P.

2006-01-01

Fiber reinforced polymer (FRP) reinforcements for concrete structures are gaining wide acceptance as a suitable alternative to steel reinforcements. The primary advantage is that they do not suffer corrosion and hence they promise to be more durable in environments where steel reinforced concrete has a limited life span. Concrete wharves and jetties are examples of structures subjected to such harsh environments and represent the general class of marine infrastructure in which glass FRP (GFRP) reinforcement should be used for improved durability and service life. General design considerations which make glass FRP suitable for use in marine concrete rehabilitation projects are discussed. A case study of recent wharf rehabilitation project in Canada is used to reinforce these considerations. The structure consisted of a GFRP reinforced concrete deck panel and steel - GFRP hybrid reinforced concrete pile cap. A design methodology is developed for the hybrid reinforcement design and verified through testing. The results of a field monitoring program are used to establish the satisfactory field performance of the GFRP reinforcement. The design concepts presented in the paper are applicable to many concrete marine components and other structures where steel reinforcement corrosion is a problem. (author)

Multi-physics corrosion modeling for sustainability assessment of steel reinforced high performance fiber reinforced cementitious composites

DEFF Research Database (Denmark)

Lepech, M.; Michel, Alexander; Geiker, Mette

2016-01-01

and widespread depassivation, are the mechanism behind experimental results of HPFRCC steel corrosion studies found in the literature. Such results provide an indication of the fundamental mechanisms by which steel reinforced HPFRCC materials may be more durable than traditional reinforced concrete and other......Using a newly developed multi-physics transport, corrosion, and cracking model, which models these phenomena as a coupled physiochemical processes, the role of HPFRCC crack control and formation in regulating steel reinforcement corrosion is investigated. This model describes transport of water...... and chemical species, the electric potential distribution in the HPFRCC, the electrochemical propagation of steel corrosion, and the role of microcracks in the HPFRCC material. Numerical results show that the reduction in anode and cathode size on the reinforcing steel surface, due to multiple crack formation...

Why we stay with our social partners: Neural mechanisms of stay/leave decision-making.

Science.gov (United States)

Heijne, Amber; Rossi, Filippo; Sanfey, Alan G

2017-09-03

How do we decide to keep interacting (e.g., stay) with a social partner or to switch (e.g., leave) to another? This paper investigated the neural mechanisms of stay/leave decision-making. We hypothesized that these decisions fit within a framework of value-based decision-making, and explored four potential mechanisms underlying a hypothesized bias to stay. Twenty-six participants underwent functional Magnetic Resonance Imaging (fMRI) while completing social and nonsocial versions of a stay/leave decision-making task. On each trial, participants chose between four alternative options, after which they received a monetary reward. Crucially, in the social condition, reward magnitude was ostensibly determined by the generosity of social partners, whereas in the nonsocial condition, reward amounts were ostensibly determined in a pre-programmed manner. Results demonstrated that participants were more likely to stay with options of relatively high expected value, with these values updated through Reinforcement Learning mechanisms and represented neurally within ventromedial prefrontal cortex. Moreover, we demonstrated that greater brain activity in ventromedial prefrontal cortex, caudate nucleus, and septo-hypothalamic regions for social versus nonsocial decisions to stay may underlie a bias towards staying with social partners in particular. These findings complement existing social psychological theories by investigating the neural mechanisms of actual stay/leave decisions.

The critical dimensions of the response-reinforcer contingency.

Science.gov (United States)

Williams, B A.

2001-05-03

Two major dimensions of any contingency of reinforcement are the temporal relation between a response and its reinforcer, and the relative frequency of the reinforcer given the response versus when the response has not occurred. Previous data demonstrate that time, per se, is not sufficient to explain the effects of delay-of-reinforcement procedures; needed in addition is some account of the events occurring in the delay interval. Moreover, the effects of the same absolute time values vary greatly across situations, such that any notion of a standard delay-of-reinforcement gradient is simplistic. The effects of reinforcers occurring in the absence of a response depend critically upon the stimulus conditions paired with those reinforcers, in much the same manner as has been shown with Pavlovian contingency effects. However, it is unclear whether the underlying basis of such effects is response competition or changes in the calculus of causation.

Autonomous dynamics in neural networks: the dHAN concept and associative thought processes

Science.gov (United States)

Gros, Claudius

2007-02-01

The neural activity of the human brain is dominated by self-sustained activities. External sensory stimuli influence this autonomous activity but they do not drive the brain directly. Most standard artificial neural network models are however input driven and do not show spontaneous activities. It constitutes a challenge to develop organizational principles for controlled, self-sustained activity in artificial neural networks. Here we propose and examine the dHAN concept for autonomous associative thought processes in dense and homogeneous associative networks. An associative thought-process is characterized, within this approach, by a time-series of transient attractors. Each transient state corresponds to a stored information, a memory. The subsequent transient states are characterized by large associative overlaps, which are identical to acquired patterns. Memory states, the acquired patterns, have such a dual functionality. In this approach the self-sustained neural activity has a central functional role. The network acquires a discrimination capability, as external stimuli need to compete with the autonomous activity. Noise in the input is readily filtered-out. Hebbian learning of external patterns occurs coinstantaneous with the ongoing associative thought process. The autonomous dynamics needs a long-term working-point optimization which acquires within the dHAN concept a dual functionality: It stabilizes the time development of the associative thought process and limits runaway synaptic growth, which generically occurs otherwise in neural networks with self-induced activities and Hebbian-type learning rules.

Modelling and simulating the forming of new dry automated lay-up reinforcements for primary structures

Science.gov (United States)

Bouquerel, Laure; Moulin, Nicolas; Drapier, Sylvain; Boisse, Philippe; Beraud, Jean-Marc

2017-10-01

While weight has been so far the main driver for the development of prepreg based-composites solutions for aeronautics, a new weight-cost trade-off tends to drive choices for next-generation aircrafts. As a response, Hexcel has designed a new dry reinforcement type for aircraft primary structures, which combines the benefits of automation, out-of-autoclave process cost-effectiveness, and mechanical performances competitive to prepreg solutions: HiTape® is a unidirectional (UD) dry carbon reinforcement with thermoplastic veil on each side designed for aircraft primary structures [1-3]. One privileged process route for HiTape® in high volume automated processes consists in forming initially flat dry reinforcement stacks, before resin infusion [4] or injection. Simulation of the forming step aims at predicting the geometry and mechanical properties of the formed stack (so-called preform) for process optimisation. Extensive work has been carried out on prepreg and dry woven fabrics forming behaviour and simulation, but the interest for dry non-woven reinforcements has emerged more recently. Some work has been achieved on non crimp fabrics but studies on the forming behaviour of UDs are seldom and deal with UD prepregs only. Tension and bending in the fibre direction, along with inter-ply friction have been identified as the main mechanisms controlling the HiTape® response during forming. Bending has been characterised using a modified Peirce's flexometer [5] and inter-ply friction study is under development. Anisotropic hyperelastic constitutive models have been selected to represent the assumed decoupled deformation mechanisms. Model parameters are then identified from associated experimental results. For forming simulation, a continuous approach at the macroscopic scale has been selected first, and simulation is carried out in the Zset framework [6] using proper shell finite elements.

Empathic concern drives costly altruism

Science.gov (United States)

FeldmanHall, Oriel; Dalgleish, Tim; Evans, Davy; Mobbs, Dean

2015-01-01

Why do we self-sacrifice to help others in distress? Two competing theories have emerged, one suggesting that prosocial behavior is primarily motivated by feelings of empathic other-oriented concern, the other that we help mainly because we are egoistically focused on reducing our own discomfort. Here we explore the relationship between costly altruism and these two sub-processes of empathy, specifically drawing on the caregiving model to test the theory that trait empathic concern (e.g. general tendency to have sympathy for another) and trait personal distress (e.g. predisposition to experiencing aversive arousal states) may differentially drive altruistic behavior. We find that trait empathic concern – and not trait personal distress – motivates costly altruism, and this relationship is supported by activity in the ventral tegmental area, caudate and subgenual anterior cingulate, key regions for promoting social attachment and caregiving. Together, this data helps identify the behavioral and neural mechanisms motivating costly altruism, while demonstrating that individual differences in empathic concern-related brain responses can predict real prosocial choice. PMID:25462694

Altered neural processing of reward and punishment in adolescents with Major Depressive Disorder.

Science.gov (United States)

Landes, I; Bakos, S; Kohls, G; Bartling, J; Schulte-Körne, G; Greimel, E

2018-05-01

Altered reward and punishment function has been suggested as an important vulnerability factor for the development of Major Depressive Disorder (MDD). Prior ERP studies found evidence for neurophysiological dysfunctions in reinforcement processes in adults with MDD. To date, only few ERP studies have examined the neural underpinnings of reinforcement processing in adolescents diagnosed with MDD. The present event-related potential (ERP) study aimed to investigate neurophysiological mechanisms of anticipation and consumption of reward and punishment in adolescents with MDD in one comprehensive paradigm. During ERP recording, 25 adolescents with MDD and 29 healthy controls (12-17 years) completed a Monetary Incentive Delay Task comprising both a monetary reward and a monetary punishment condition. During anticipation, the cue-P3 signaling attentional allocation was recorded. During consumption, the feedback-P3 and Reward Positivity (RewP) were recorded to capture attentional allocation and outcome evaluation, respectively. Compared to controls, adolescents with MDD showed prolonged cue-P3 latencies to reward cues. Furthermore, unlike controls, adolescents with MDD displayed shorter feedback-P3 latencies in the reward versus punishment condition. RewPs did not differ between groups. It remains unanswered whether the observed alterations in adolescent MDD represent a state or trait. Delayed neural processing of reward cues corresponds to the clinical presentation of adolescent MDD with reduced motivational tendencies to obtain rewards. Relatively shorter feedback-P3 latencies in the reward versus punishment condition could indicate a high salience of performance-contingent reward. Frequent exposure of negatively biased adolescents with MDD to performance-contingent rewards might constitute a promising intervention approach. Copyright © 2018 Elsevier B.V. All rights reserved.

Constitutively active Notch1 converts cranial neural crest-derived frontonasal mesenchyme to perivascular cells in vivo

Directory of Open Access Journals (Sweden)

Sophie R. Miller

2017-03-01

Full Text Available Perivascular/mural cells originate from either the mesoderm or the cranial neural crest. Regardless of their origin, Notch signalling is necessary for their formation. Furthermore, in both chicken and mouse, constitutive Notch1 activation (via expression of the Notch1 intracellular domain is sufficient in vivo to convert trunk mesoderm-derived somite cells to perivascular cells, at the expense of skeletal muscle. In experiments originally designed to investigate the effect of premature Notch1 activation on the development of neural crest-derived olfactory ensheathing glial cells (OECs, we used in ovo electroporation to insert a tetracycline-inducible NotchΔE construct (encoding a constitutively active mutant of mouse Notch1 into the genome of chicken cranial neural crest cell precursors, and activated NotchΔE expression by doxycycline injection at embryonic day 4. NotchΔE-targeted cells formed perivascular cells within the frontonasal mesenchyme, and expressed a perivascular marker on the olfactory nerve. Hence, constitutively activating Notch1 is sufficient in vivo to drive not only somite cells, but also neural crest-derived frontonasal mesenchyme and perhaps developing OECs, to a perivascular cell fate. These results also highlight the plasticity of neural crest-derived mesenchyme and glia.

Evolvable synthetic neural system

Science.gov (United States)

Curtis, Steven A. (Inventor)

2009-01-01

An evolvable synthetic neural system includes an evolvable neural interface operably coupled to at least one neural basis function. Each neural basis function includes an evolvable neural interface operably coupled to a heuristic neural system to perform high-level functions and an autonomic neural system to perform low-level functions. In some embodiments, the evolvable synthetic neural system is operably coupled to one or more evolvable synthetic neural systems in a hierarchy.

Neural electrical activity and neural network growth.

Science.gov (United States)

Gafarov, F M

2018-05-01

The development of central and peripheral neural system depends in part on the emergence of the correct functional connectivity in its input and output pathways. Now it is generally accepted that molecular factors guide neurons to establish a primary scaffold that undergoes activity-dependent refinement for building a fully functional circuit. However, a number of experimental results obtained recently shows that the neuronal electrical activity plays an important role in the establishing of initial interneuronal connections. Nevertheless, these processes are rather difficult to study experimentally, due to the absence of theoretical description and quantitative parameters for estimation of the neuronal activity influence on growth in neural networks. In this work we propose a general framework for a theoretical description of the activity-dependent neural network growth. The theoretical description incorporates a closed-loop growth model in which the neural activity can affect neurite outgrowth, which in turn can affect neural activity. We carried out the detailed quantitative analysis of spatiotemporal activity patterns and studied the relationship between individual cells and the network as a whole to explore the relationship between developing connectivity and activity patterns. The model, developed in this work will allow us to develop new experimental techniques for studying and quantifying the influence of the neuronal activity on growth processes in neural networks and may lead to a novel techniques for constructing large-scale neural networks by self-organization. Copyright © 2018 Elsevier Ltd. All rights reserved.

Modelling root reinforcement in shallow forest soils

Science.gov (United States)

Skaugset, Arne E.

1997-01-01

A hypothesis used to explain the relationship between timber harvesting and landslides is that tree roots add mechanical support to soil, thus increasing soil strength. Upon harvest, the tree roots decay which reduces soil strength and increases the risk of management -induced landslides. The technical literature does not adequately support this hypothesis. Soil strength values attributed to root reinforcement that are in the technical literature are such that forested sites can't fail and all high risk, harvested sites must fail. Both unstable forested sites and stable harvested sites exist, in abundance, in the real world thus, the literature does not adequately describe the real world. An analytical model was developed to calculate soil strength increase due to root reinforcement. Conceptually, the model is composed of a reinforcing element with high tensile strength, i.e. a conifer root, embedded in a material with little tensile strength, i.e. a soil. As the soil fails and deforms, the reinforcing element also deforms and stretches. The lateral deformation of the reinforcing element is treated analytically as a laterally loaded pile in a flexible foundation and the axial deformation is treated as an axially loaded pile. The governing differential equations are solved using finite-difference approximation techniques. The root reinforcement model was tested by comparing the final shape of steel and aluminum rods, parachute cord, wooden dowels, and pine roots in direct shear with predicted shapes from the output of the root reinforcement model. The comparisons were generally satisfactory, were best for parachute cord and wooden dowels, and were poorest for steel and aluminum rods. A parameter study was performed on the root reinforcement model which showed reinforced soil strength increased with increasing root diameter and soil depth. Output from the root reinforcement model showed a strain incompatibility between large and small diameter roots. The peak

Reinforcement learning for dpm of embedded visual sensor nodes

International Nuclear Information System (INIS)

Khani, U.; Sadhayo, I. H.

2014-01-01

This paper proposes a RL (Reinforcement Learning) based DPM (Dynamic Power Management) technique to learn time out policies during a visual sensor node's operation which has multiple power/performance states. As opposed to the widely used static time out policies, our proposed DPM policy which is also referred to as OLTP (Online Learning of Time out Policies), learns to dynamically change the time out decisions in the different node states including the non-operational states. The selection of time out values in different power/performance states of a visual sensing platform is based on the workload estimates derived from a ML-ANN (Multi-Layer Artificial Neural Network) and an objective function given by weighted performance and power parameters. The DPM approach is also able to dynamically adjust the power-performance weights online to satisfy a given constraint of either power consumption or performance. Results show that the proposed learning algorithm explores the power-performance tradeoff with non-stationary workload and outperforms other DPM policies. It also performs the online adjustment of the tradeoff parameters in order to meet a user-specified constraint. (author)

A Case–Control Study Investigating Simulated Driving Errors in Ischemic Stroke and Subarachnoid Hemorrhage

Directory of Open Access Journals (Sweden)

Megan A. Hird

2018-02-01

performance. Current results support the importance of differentiating between stroke subtypes and considering other important clinical characteristics (e.g., side of lesion, vascular territory when assessing driving performance and reinforce the importance of physicians discussing driving safety with patients after stroke.

Modeling of geosynthetic reinforced capping systems

International Nuclear Information System (INIS)

Viswanadham, B.V.S.; Koenig, D.; Jessberger, H.L.

1997-01-01

The investigation deals with the influence of a geosynthetic reinforcement on the deformation behavior and sealing efficiency of the reinforced mineral sealing layer at the onset of non-uniform settlements. The research program is mainly concentrated in studying the influence of reinforcement inclusion in restraining cracks and crack propagation due to soil-geosynthetic bond efficiency. Centrifuge model tests are conducted in the 500 gt capacity balanced beam Bochum geotechnical Centrifuge (Z1) simulating a differential deformation of a mineral sealing layer of a landfill with the help of trap-door arrangement. By comparing the performance of the deformed mineral sealing layer with and without geogrid, the reinforcement ability of the geogrid in controlling the crack propagation and permeability of the mineral swing layer is evaluated

Quantum Entanglement in Neural Network States

Directory of Open Access Journals (Sweden)

Dong-Ling Deng

2017-05-01

Full Text Available Machine learning, one of today’s most rapidly growing interdisciplinary fields, promises an unprecedented perspective for solving intricate quantum many-body problems. Understanding the physical aspects of the representative artificial neural-network states has recently become highly desirable in the applications of machine-learning techniques to quantum many-body physics. In this paper, we explore the data structures that encode the physical features in the network states by studying the quantum entanglement properties, with a focus on the restricted-Boltzmann-machine (RBM architecture. We prove that the entanglement entropy of all short-range RBM states satisfies an area law for arbitrary dimensions and bipartition geometry. For long-range RBM states, we show by using an exact construction that such states could exhibit volume-law entanglement, implying a notable capability of RBM in representing quantum states with massive entanglement. Strikingly, the neural-network representation for these states is remarkably efficient, in the sense that the number of nonzero parameters scales only linearly with the system size. We further examine the entanglement properties of generic RBM states by randomly sampling the weight parameters of the RBM. We find that their averaged entanglement entropy obeys volume-law scaling, and the meantime strongly deviates from the Page entropy of the completely random pure states. We show that their entanglement spectrum has no universal part associated with random matrix theory and bears a Poisson-type level statistics. Using reinforcement learning, we demonstrate that RBM is capable of finding the ground state (with power-law entanglement of a model Hamiltonian with a long-range interaction. In addition, we show, through a concrete example of the one-dimensional symmetry-protected topological cluster states, that the RBM representation may also be used as a tool to analytically compute the entanglement spectrum. Our

Geo synthetic-reinforced Pavement systems

International Nuclear Information System (INIS)

Zornberg, J. G.

2014-01-01

Geo synthetics have been used as reinforcement inclusions to improve pavement performance. while there are clear field evidence of the benefit of using geo synthetic reinforcements, the specific conditions or mechanisms that govern the reinforcement of pavements are, at best, unclear and have remained largely unmeasured. Significant research has been recently conducted with the objectives of: (i) determining the relevant properties of geo synthetics that contribute to the enhanced performance of pavement systems, (ii) developing appropriate analytical, laboratory and field methods capable of quantifying the pavement performance, and (iii) enabling the prediction of pavement performance as a function of the properties of the various types of geo synthetics. (Author)

Reinforcement of Conducting Silver-based Materials

Directory of Open Access Journals (Sweden)

Heike JUNG

2014-09-01

Full Text Available Silver is a well-known material in the field of contact materials because of its high electrical and thermal conductivity. However, due to its bad mechanical and switching properties, silver alloys or reinforcements of the ductile silver matrix are required. Different reinforcements, e. g. tungsten, tungsten carbide, nickel, cadmium oxide or tin oxide, are used in different sectors of switches. To reach an optimal distribution of these reinforcements, various manufacturing techniques (e. g. powder blending, preform infiltration, wet-chemical methods, internal oxidation are being used for the production of these contact materials. Each of these manufacturing routes offers different advantages and disadvantages. The mechanical alloying process displays a successful and efficient method to produce particle-reinforced metal-matrix composite powders. This contribution presents the obtained fine disperse microstructure of tungsten-particle-reinforced silver composite powders produced by the mechanical alloying process and displays this technique as possible route to provide feedstock powders for subsequent consolidation processes. DOI: http://dx.doi.org/10.5755/j01.ms.20.3.4889

Decoupling control of a five-phase fault-tolerant permanent magnet motor by radial basis function neural network inverse

Science.gov (United States)

Chen, Qian; Liu, Guohai; Xu, Dezhi; Xu, Liang; Xu, Gaohong; Aamir, Nazir

2018-05-01

This paper proposes a new decoupled control for a five-phase in-wheel fault-tolerant permanent magnet (IW-FTPM) motor drive, in which radial basis function neural network inverse (RBF-NNI) and internal model control (IMC) are combined. The RBF-NNI system is introduced into original system to construct a pseudo-linear system, and IMC is used as a robust controller. Hence, the newly proposed control system incorporates the merits of the IMC and RBF-NNI methods. In order to verify the proposed strategy, an IW-FTPM motor drive is designed based on dSPACE real-time control platform. Then, the experimental results are offered to verify that the d-axis current and the rotor speed are successfully decoupled. Besides, the proposed motor drive exhibits strong robustness even under load torque disturbance.

Sequence conservation and combinatorial complexity of Drosophila neural precursor cell enhancers

Directory of Open Access Journals (Sweden)

Kuzin Alexander

2008-08-01

Full Text Available Abstract Background The presence of highly conserved sequences within cis-regulatory regions can serve as a valuable starting point for elucidating the basis of enhancer function. This study focuses on regulation of gene expression during the early events of Drosophila neural development. We describe the use of EvoPrinter and cis-Decoder, a suite of interrelated phylogenetic footprinting and alignment programs, to characterize highly conserved sequences that are shared among co-regulating enhancers. Results Analysis of in vivo characterized enhancers that drive neural precursor gene expression has revealed that they contain clusters of highly conserved sequence blocks (CSBs made up of shorter shared sequence elements which are present in different combinations and orientations within the different co-regulating enhancers; these elements contain either known consensus transcription factor binding sites or consist of novel sequences that have not been functionally characterized. The CSBs of co-regulated enhancers share a large number of sequence elements, suggesting that a diverse repertoire of transcription factors may interact in a highly combinatorial fashion to coordinately regulate gene expression. We have used information gained from our comparative analysis to discover an enhancer that directs expression of the nervy gene in neural precursor cells of the CNS and PNS. Conclusion The combined use EvoPrinter and cis-Decoder has yielded important insights into the combinatorial appearance of fundamental sequence elements required for neural enhancer function. Each of the 30 enhancers examined conformed to a pattern of highly conserved blocks of sequences containing shared constituent elements. These data establish a basis for further analysis and understanding of neural enhancer function.

Topology optimization of reinforced concrete beams by a spread-over reinforcement model with fixed grid mesh

Directory of Open Access Journals (Sweden)

Benjapon Wethyavivorn

2011-02-01

Full Text Available For this investigation, topology optimization was used as a tool to determine the optimal reinforcement for reinforcedconcrete beam. The topology optimization process was based on a unit finite element cell with layers of concrete and steel.The thickness of the reinforced steel layer of this unit cell was then adjusted when the concrete layer could not carry thetensile or compressive stress. At the same time, unit cells which carried very low stress were eliminated. The process wasperformed iteratively to create a topology of reinforced concrete beam which satisfied design conditions.

Acquisition with partial and continuous reinforcement in pigeon autoshaping.

Science.gov (United States)

Gottlieb, Daniel A

2004-08-01

Contemporary time accumulation models make the unique prediction that acquisition of a conditioned response will be equally rapid with partial and continuous reinforcement, if the time between conditioned stimuli is held constant. To investigate this, acquisition of conditioned responding was examined in pigeon autoshaping under conditions of 100% and 25% reinforcement, holding intertrial interval constant. Contrary to what was predicted, evidence for slowed acquisition in partially reinforced animals was observed with several response measures. However, asymptotic performance was superior with 25% reinforcement. A switching of reinforcement contingencies after initial acquisition did not immediately affect responding. After further sessions, partial reinforcement augmented responding, whereas continuous reinforcement did not, irrespective of an animal's reinforcement history. Subsequent training with a novel stimulus maintained the response patterns. These acquisition results generally support associative, rather than time accumulation, accounts of conditioning.

Brain network response underlying decisions about abstract reinforcers.

Science.gov (United States)

Mills-Finnerty, Colleen; Hanson, Catherine; Hanson, Stephen Jose

2014-12-01

Decision making studies typically use tasks that involve concrete action-outcome contingencies, in which subjects do something and get something. No studies have addressed decision making involving abstract reinforcers, where there are no action-outcome contingencies and choices are entirely hypothetical. The present study examines these kinds of choices, as well as whether the same biases that exist for concrete reinforcer decisions, specifically framing effects, also apply during abstract reinforcer decisions. We use both General Linear Model as well as Bayes network connectivity analysis using the Independent Multi-sample Greedy Equivalence Search (IMaGES) algorithm to examine network response underlying choices for abstract reinforcers under positive and negative framing. We find for the first time that abstract reinforcer decisions activate the same network of brain regions as concrete reinforcer decisions, including the striatum, insula, anterior cingulate, and VMPFC, results that are further supported via comparison to a meta-analysis of decision making studies. Positive and negative framing activated different parts of this network, with stronger activation in VMPFC during negative framing and in DLPFC during positive, suggesting different decision making pathways depending on frame. These results were further clarified using connectivity analysis, which revealed stronger connections between anterior cingulate, insula, and accumbens during negative framing compared to positive. Taken together, these results suggest that not only do abstract reinforcer decisions rely on the same brain substrates as concrete reinforcers, but that the response underlying framing effects on abstract reinforcers also resemble those for concrete reinforcers, specifically increased limbic system connectivity during negative frames. Copyright © 2014 Elsevier Inc. All rights reserved.

Long-term performance of GFRP reinforcement : technical report.

Science.gov (United States)

2009-12-01

Significant research has been performed on glass fiber-reinforced polymer (GFRP) concrete reinforcement. : This research has shown that GFRP reinforcement exhibits high strengths, is lightweight, can decrease time of : construction, and is corrosion ...

Analysis of Neural-BOLD Coupling through Four Models of the Neural Metabolic Demand

Directory of Open Access Journals (Sweden)

Christopher W Tyler

2015-12-01

Full Text Available The coupling of the neuronal energetics to the blood-oxygen-level-dependent (BOLD response is still incompletely understood. To address this issue, we compared the fits of four plausible models of neurometabolic coupling dynamics to available data for simultaneous recordings of the local field potential (LFP and the local BOLD response recorded from monkey primary visual cortex over a wide range of stimulus durations. The four models of the metabolic demand driving the BOLD response were: direct coupling with the overall LFP; rectified coupling to the LFP; coupling with a slow adaptive component of the implied neural population response; and coupling with the non-adaptive intracellular input signal defined by the stimulus time course. Taking all stimulus durations into account, the results imply that the BOLD response is most closely coupled with metabolic demand derived from the intracellular input waveform, without significant influence from the adaptive transients and nonlinearities exhibited by the LFP waveform.

Older driver fitness-to-drive evaluation using naturalistic driving data.

Science.gov (United States)

Guo, Feng; Fang, Youjia; Antin, Jonathan F

2015-09-01

As our driving population continues to age, it is becoming increasingly important to find a small set of easily administered fitness metrics that can meaningfully and reliably identify at-risk seniors requiring more in-depth evaluation of their driving skills and weaknesses. Sixty driver assessment metrics related to fitness-to-drive were examined for 20 seniors who were followed for a year using the naturalistic driving paradigm. Principal component analysis and negative binomial regression modeling approaches were used to develop parsimonious models relating the most highly predictive of the driver assessment metrics to the safety-related outcomes observed in the naturalistic driving data. This study provides important confirmation using naturalistic driving methods of the relationship between contrast sensitivity and crash-related events. The results of this study provide crucial information on the continuing journey to identify metrics and protocols that could be applied to determine seniors' fitness to drive. Published by Elsevier Ltd.

The effect of concrete strength and reinforcement on toughness of reinforced concrete beams

OpenAIRE

Carneiro, Joaquim A. O.; Jalali, Said; Teixeira, Vasco M. P.; Tomás, M.

2005-01-01

The objective pursued with this work includes the evaluating of the strength and the total energy absorption capacity (toughness) of reinforced concrete beams using different amounts of steel-bar reinforcement. The experimental campaign deals with the evaluation of the threshold load prior collapse, ultimate load and deformation, as well as the beam total energy absorption capacity, using a three point bending test. The beam half span displacement was measured using a displacement transducer,...

Automated driving safer and more efficient future driving

CERN Document Server

Horn, Martin

2017-01-01

The main topics of this book include advanced control, cognitive data processing, high performance computing, functional safety, and comprehensive validation. These topics are seen as technological bricks to drive forward automated driving. The current state of the art of automated vehicle research, development and innovation is given. The book also addresses industry-driven roadmaps for major new technology advances as well as collaborative European initiatives supporting the evolvement of automated driving. Various examples highlight the state of development of automated driving as well as the way forward. The book will be of interest to academics and researchers within engineering, graduate students, automotive engineers at OEMs and suppliers, ICT and software engineers, managers, and other decision-makers.

Concrete cover cracking due to uniform reinforcement corrosion

DEFF Research Database (Denmark)

Solgaard, Anders Ole Stubbe; Michel, Alexander; Geiker, Mette Rica

2013-01-01

and reinforcement de-passivation is a frequently used limit state. The present paper investigates an alternative limit state: corrosion-induced cover cracking. Results from numerical simulations of concrete cover cracking due to reinforcement corrosion are presented. The potential additional service life...... is calculated using literature data on corrosion rate and Faraday’s law. The parameters varied comprise reinforcement diameter, concrete cover thickness and concrete material properties, viz. concrete tensile strength and ductility (plain concrete and fibre reinforced concrete). Results obtained from......Service life design (SLD) is an important tool for civil engineers to ensure that the structural integrity and functionality of the structure is not compromised within a given time frame, i.e. the service life. In SLD of reinforced concrete structures, reinforcement corrosion is of major concern...

Quenched Reinforcement Exposed to Fire

DEFF Research Database (Denmark)

Hertz, Kristian Dahl

2006-01-01

.0% is seldom found in “slack” (not prestressed) reinforcement, but 2.0% stresses might be relevant for reinforcement in T shaped cross sections and for prestressed structures, where large strains can be applied. All data are provided in a “HOT” condition during a fire and in a “COLD” condition after a fire...

Design of reinforced concrete plates and shells

International Nuclear Information System (INIS)

Schulz, M.

1984-01-01

Nowadays, the internal forces of reinforced concrete laminar structures can be easily evaluated by the finite element procedures. The longitudinal design in each direction is not adequate, since the whole set of internal forces in each point must be concomitantly considered. The classic formulation for the design and new design charts which bring reduction of the amount of necessary reinforcement are presented. A rational reinforced concrete mathematical theory which makes possible the limit state design of plates and shells is discussed. This model can also be applied to define the constitutive relationships of laminar finite elements of reinforced concrete. (Author) [pt

Effect of steel reinforcement with different degree of corrosion on degeneration of mechanical performance of reinforced concrete frame joints

Directory of Open Access Journals (Sweden)

Wu Xu

2016-02-01

Full Text Available Beam-column joints which shoulders high-level and vertical shearing effect that maintains balance of beam and column end is the major component influencing the performance of the whole framework. Post earthquake investigation suggests that collapse of frame structure is induced by failure of joints in most cases. Thus, beam-column joints must have strong bearing capacity and good ductility, and reinforced concrete structure just meets the above requirement. But corrosion caused by long time use of reinforced concrete framework will lead to degeneration of mechanical performance of joints. To find out the rule of effect of steel reinforcement with different corrosion rate on degeneration of bearing capacity of reinforced concrete framework joints, this study made a nonlinear numerical analysis on fifteen models without stirrup in the core area of reinforced concrete frame joints using displacement method considering axial load ratio of column end and constraint condition. This work aims to find out the key factor that influences mechanical performance of joints, thus to provide a basis for repair and reinforcement of degenerated framework joints.