Recurrent Neural Network for Single Machine Power System Stabilizer
Directory of Open Access Journals (Sweden)
Widi Aribowo
2010-04-01
Full Text Available In this paper, recurrent neural network (RNN is used to design power system stabilizer (PSS due to its advantage on the dependence not only on present input but also on past condition. A RNN-PSS is able to capture the dynamic response of a system without any delays caused by external feedback, primarily by the internal feedback loop in recurrent neuron. In this paper, RNNPSS consists of a RNN-identifier and a RNN-controller. The RNN-Identifier functions as the tracker of dynamics characteristics of the plant, while the RNN-controller is used to damp the system’s low frequency oscillations. Simulation results using MATLAB demonstrate that the RNNPSS can successfully damp out oscillation and improve the performance of the system.
Chaotic diagonal recurrent neural network
International Nuclear Information System (INIS)
Wang Xing-Yuan; Zhang Yi
2012-01-01
We propose a novel neural network based on a diagonal recurrent neural network and chaos, and its structure and learning algorithm are designed. The multilayer feedforward neural network, diagonal recurrent neural network, and chaotic diagonal recurrent neural network are used to approach the cubic symmetry map. The simulation results show that the approximation capability of the chaotic diagonal recurrent neural network is better than the other two neural networks. (interdisciplinary physics and related areas of science and technology)
Natural Language Video Description using Deep Recurrent Neural Networks
2015-11-23
language with a single deep neural network. We use deep recurrent nets (RNNs), which have recently demonstrated strong results for machine translation (MT...Donahue, Marcus Rohrbach, Raymond Mooney, and Kate Saenko. Translating videos to natural language using deep recurrent neural net - works. In NAACL, 2015...Natural Language Video Description using Deep Recurrent Neural Networks Subhashini Venugopalan University of Texas at Austin vsub@cs.utexas.edu
Czech Academy of Sciences Publication Activity Database
Doubravová, Jana; Wiszniowski, J.; Horálek, Josef
2016-01-01
Roč. 93, August (2016), s. 138-149 ISSN 0098-3004 R&D Projects: GA ČR GAP210/12/2336; GA MŠk LM2010008 Institutional support: RVO:67985530 Keywords : event detection * artificial neural network * West Bohemia/Vogtland Subject RIV: DC - Siesmology, Volcanology, Earth Structure Impact factor: 2.533, year: 2016
Directory of Open Access Journals (Sweden)
Faa-Jeng Lin
2014-01-01
Full Text Available This study presents a new active and reactive power control scheme for a single-stage three-phase grid-connected photovoltaic (PV system during grid faults. The presented PV system utilizes a single-stage three-phase current-controlled voltage-source inverter to achieve the maximum power point tracking (MPPT control of the PV panel with the function of low voltage ride through (LVRT. Moreover, a formula based on positive sequence voltage for evaluating the percentage of voltage sag is derived to determine the ratio of the injected reactive current to satisfy the LVRT regulations. To reduce the risk of overcurrent during LVRT operation, a current limit is predefined for the injection of reactive current. Furthermore, the control of active and reactive power is designed using a two-dimensional recurrent fuzzy cerebellar model articulation neural network (2D-RFCMANN. In addition, the online learning laws of 2D-RFCMANN are derived according to gradient descent method with varied learning-rate coefficients for network parameters to assure the convergence of the tracking error. Finally, some experimental tests are realized to validate the effectiveness of the proposed control scheme.
Interpretation of Recurrent Neural Networks
DEFF Research Database (Denmark)
Pedersen, Morten With; Larsen, Jan
1997-01-01
This paper addresses techniques for interpretation and characterization of trained recurrent nets for time series problems. In particular, we focus on assessment of effective memory and suggest an operational definition of memory. Further we discuss the evaluation of learning curves. Various...
Optimization of recurrent neural networks for time series modeling
DEFF Research Database (Denmark)
Pedersen, Morten With
1997-01-01
The present thesis is about optimization of recurrent neural networks applied to time series modeling. In particular is considered fully recurrent networks working from only a single external input, one layer of nonlinear hidden units and a li near output unit applied to prediction of discrete time...... series. The overall objective s are to improve training by application of second-order methods and to improve generalization ability by architecture optimization accomplished by pruning. The major topics covered in the thesis are: 1. The problem of training recurrent networks is analyzed from a numerical...... of solution obtained as well as computation time required. 3. A theoretical definition of the generalization error for recurrent networks is provided. This definition justifies a commonly adopted approach for estimating generalization ability. 4. The viability of pruning recurrent networks by the Optimal...
Recurrent Neural Network for Computing Outer Inverse.
Živković, Ivan S; Stanimirović, Predrag S; Wei, Yimin
2016-05-01
Two linear recurrent neural networks for generating outer inverses with prescribed range and null space are defined. Each of the proposed recurrent neural networks is based on the matrix-valued differential equation, a generalization of dynamic equations proposed earlier for the nonsingular matrix inversion, the Moore-Penrose inversion, as well as the Drazin inversion, under the condition of zero initial state. The application of the first approach is conditioned by the properties of the spectrum of a certain matrix; the second approach eliminates this drawback, though at the cost of increasing the number of matrix operations. The cases corresponding to the most common generalized inverses are defined. The conditions that ensure stability of the proposed neural network are presented. Illustrative examples present the results of numerical simulations.
Time series prediction with simple recurrent neural networks ...
African Journals Online (AJOL)
Simple recurrent neural networks are widely used in time series prediction. Most researchers and application developers often choose arbitrarily between Elman or Jordan simple recurrent neural networks for their applications. A hybrid of the two called Elman-Jordan (or Multi-recurrent) neural network is also being used.
Local Dynamics in Trained Recurrent Neural Networks
Rivkind, Alexander; Barak, Omri
2017-06-01
Learning a task induces connectivity changes in neural circuits, thereby changing their dynamics. To elucidate task-related neural dynamics, we study trained recurrent neural networks. We develop a mean field theory for reservoir computing networks trained to have multiple fixed point attractors. Our main result is that the dynamics of the network's output in the vicinity of attractors is governed by a low-order linear ordinary differential equation. The stability of the resulting equation can be assessed, predicting training success or failure. As a consequence, networks of rectified linear units and of sigmoidal nonlinearities are shown to have diametrically different properties when it comes to learning attractors. Furthermore, a characteristic time constant, which remains finite at the edge of chaos, offers an explanation of the network's output robustness in the presence of variability of the internal neural dynamics. Finally, the proposed theory predicts state-dependent frequency selectivity in the network response.
Local Dynamics in Trained Recurrent Neural Networks.
Rivkind, Alexander; Barak, Omri
2017-06-23
Learning a task induces connectivity changes in neural circuits, thereby changing their dynamics. To elucidate task-related neural dynamics, we study trained recurrent neural networks. We develop a mean field theory for reservoir computing networks trained to have multiple fixed point attractors. Our main result is that the dynamics of the network's output in the vicinity of attractors is governed by a low-order linear ordinary differential equation. The stability of the resulting equation can be assessed, predicting training success or failure. As a consequence, networks of rectified linear units and of sigmoidal nonlinearities are shown to have diametrically different properties when it comes to learning attractors. Furthermore, a characteristic time constant, which remains finite at the edge of chaos, offers an explanation of the network's output robustness in the presence of variability of the internal neural dynamics. Finally, the proposed theory predicts state-dependent frequency selectivity in the network response.
Supervised Sequence Labelling with Recurrent Neural Networks
Graves, Alex
2012-01-01
Supervised sequence labelling is a vital area of machine learning, encompassing tasks such as speech, handwriting and gesture recognition, protein secondary structure prediction and part-of-speech tagging. Recurrent neural networks are powerful sequence learning tools—robust to input noise and distortion, able to exploit long-range contextual information—that would seem ideally suited to such problems. However their role in large-scale sequence labelling systems has so far been auxiliary. The goal of this book is a complete framework for classifying and transcribing sequential data with recurrent neural networks only. Three main innovations are introduced in order to realise this goal. Firstly, the connectionist temporal classification output layer allows the framework to be trained with unsegmented target sequences, such as phoneme-level speech transcriptions; this is in contrast to previous connectionist approaches, which were dependent on error-prone prior segmentation. Secondly, multidimensional...
Turing Computation with Recurrent Artificial Neural Networks
Carmantini, Giovanni S; Graben, Peter beim; Desroches, Mathieu; Rodrigues, Serafim
2015-01-01
We improve the results by Siegelmann & Sontag (1995) by providing a novel and parsimonious constructive mapping between Turing Machines and Recurrent Artificial Neural Networks, based on recent developments of Nonlinear Dynamical Automata. The architecture of the resulting R-ANNs is simple and elegant, stemming from its transparent relation with the underlying NDAs. These characteristics yield promise for developments in machine learning methods and symbolic computation with continuous time d...
Adaptive Filtering Using Recurrent Neural Networks
Parlos, Alexander G.; Menon, Sunil K.; Atiya, Amir F.
2005-01-01
A method for adaptive (or, optionally, nonadaptive) filtering has been developed for estimating the states of complex process systems (e.g., chemical plants, factories, or manufacturing processes at some level of abstraction) from time series of measurements of system inputs and outputs. The method is based partly on the fundamental principles of the Kalman filter and partly on the use of recurrent neural networks. The standard Kalman filter involves an assumption of linearity of the mathematical model used to describe a process system. The extended Kalman filter accommodates a nonlinear process model but still requires linearization about the state estimate. Both the standard and extended Kalman filters involve the often unrealistic assumption that process and measurement noise are zero-mean, Gaussian, and white. In contrast, the present method does not involve any assumptions of linearity of process models or of the nature of process noise; on the contrary, few (if any) assumptions are made about process models, noise models, or the parameters of such models. In this regard, the method can be characterized as one of nonlinear, nonparametric filtering. The method exploits the unique ability of neural networks to approximate nonlinear functions. In a given case, the process model is limited mainly by limitations of the approximation ability of the neural networks chosen for that case. Moreover, despite the lack of assumptions regarding process noise, the method yields minimum- variance filters. In that they do not require statistical models of noise, the neural- network-based state filters of this method are comparable to conventional nonlinear least-squares estimators.
Neural tube defects in Gauteng, South Africa: Recurrence risks and ...
African Journals Online (AJOL)
Neural tube defects in Gauteng, South Africa: Recurrence risks and associated factors. G Teckie, A Krause, JGR Kromberg. Abstract. Background. After congenital heart disease, neural tube defects (NTDs) are the most common serious structural birth defects in human infants. Objectives. To (i) determine the recurrence risks ...
Identification of Non-Linear Structures using Recurrent Neural Networks
DEFF Research Database (Denmark)
Kirkegaard, Poul Henning; Nielsen, Søren R. K.; Hansen, H. I.
Two different partially recurrent neural networks structured as Multi Layer Perceptrons (MLP) are investigated for time domain identification of a non-linear structure.......Two different partially recurrent neural networks structured as Multi Layer Perceptrons (MLP) are investigated for time domain identification of a non-linear structure....
Identification of Non-Linear Structures using Recurrent Neural Networks
DEFF Research Database (Denmark)
Kirkegaard, Poul Henning; Nielsen, Søren R. K.; Hansen, H. I.
1995-01-01
Two different partially recurrent neural networks structured as Multi Layer Perceptrons (MLP) are investigated for time domain identification of a non-linear structure.......Two different partially recurrent neural networks structured as Multi Layer Perceptrons (MLP) are investigated for time domain identification of a non-linear structure....
Precipitation Nowcast using Deep Recurrent Neural Network
Akbari Asanjan, A.; Yang, T.; Gao, X.; Hsu, K. L.; Sorooshian, S.
2016-12-01
An accurate precipitation nowcast (0-6 hours) with a fine temporal and spatial resolution has always been an important prerequisite for flood warning, streamflow prediction and risk management. Most of the popular approaches used for forecasting precipitation can be categorized into two groups. One type of precipitation forecast relies on numerical modeling of the physical dynamics of atmosphere and another is based on empirical and statistical regression models derived by local hydrologists or meteorologists. Given the recent advances in artificial intelligence, in this study a powerful Deep Recurrent Neural Network, termed as Long Short-Term Memory (LSTM) model, is creatively used to extract the patterns and forecast the spatial and temporal variability of Cloud Top Brightness Temperature (CTBT) observed from GOES satellite. Then, a 0-6 hours precipitation nowcast is produced using a Precipitation Estimation from Remote Sensing Information using Artificial Neural Network (PERSIANN) algorithm, in which the CTBT nowcast is used as the PERSIANN algorithm's raw inputs. Two case studies over the continental U.S. have been conducted that demonstrate the improvement of proposed approach as compared to a classical Feed Forward Neural Network and a couple simple regression models. The advantages and disadvantages of the proposed method are summarized with regard to its capability of pattern recognition through time, handling of vanishing gradient during model learning, and working with sparse data. The studies show that the LSTM model performs better than other methods, and it is able to learn the temporal evolution of the precipitation events through over 1000 time lags. The uniqueness of PERSIANN's algorithm enables an alternative precipitation nowcast approach as demonstrated in this study, in which the CTBT prediction is produced and used as the inputs for generating precipitation nowcast.
Baruch, Ieroham; Mariaca-Gaspar, Carlos; Barrera-Cortes, Josefina
2008-01-01
The chapter proposes a new Kalman filter closed loop topology of recurrent neural network for identification and modeling of an unknown hydrocarbon degradation process carried out in a biopile system and a rotating drum. The proposed KF RNN contained a recurrent neural plant model, a recurrent neural output plant filter and posses global and local feedbacks. The learning algorithm is a modified version of the dynamic Backpropagation one derived using the adjoint KF RNN topology by means of th...
Deep Recurrent Neural Networks for Supernovae Classification
Charnock, Tom; Moss, Adam
2017-03-01
We apply deep recurrent neural networks, which are capable of learning complex sequential information, to classify supernovae (code available at https://github.com/adammoss/supernovae). The observational time and filter fluxes are used as inputs to the network, but since the inputs are agnostic, additional data such as host galaxy information can also be included. Using the Supernovae Photometric Classification Challenge (SPCC) data, we find that deep networks are capable of learning about light curves, however the performance of the network is highly sensitive to the amount of training data. For a training size of 50% of the representational SPCC data set (around 104 supernovae) we obtain a type-Ia versus non-type-Ia classification accuracy of 94.7%, an area under the Receiver Operating Characteristic curve AUC of 0.986 and an SPCC figure-of-merit F 1 = 0.64. When using only the data for the early-epoch challenge defined by the SPCC, we achieve a classification accuracy of 93.1%, AUC of 0.977, and F 1 = 0.58, results almost as good as with the whole light curve. By employing bidirectional neural networks, we can acquire impressive classification results between supernovae types I, II and III at an accuracy of 90.4% and AUC of 0.974. We also apply a pre-trained model to obtain classification probabilities as a function of time and show that it can give early indications of supernovae type. Our method is competitive with existing algorithms and has applications for future large-scale photometric surveys.
Bayesian Recurrent Neural Network for Language Modeling.
Chien, Jen-Tzung; Ku, Yuan-Chu
2016-02-01
A language model (LM) is calculated as the probability of a word sequence that provides the solution to word prediction for a variety of information systems. A recurrent neural network (RNN) is powerful to learn the large-span dynamics of a word sequence in the continuous space. However, the training of the RNN-LM is an ill-posed problem because of too many parameters from a large dictionary size and a high-dimensional hidden layer. This paper presents a Bayesian approach to regularize the RNN-LM and apply it for continuous speech recognition. We aim to penalize the too complicated RNN-LM by compensating for the uncertainty of the estimated model parameters, which is represented by a Gaussian prior. The objective function in a Bayesian classification network is formed as the regularized cross-entropy error function. The regularized model is constructed not only by calculating the regularized parameters according to the maximum a posteriori criterion but also by estimating the Gaussian hyperparameter by maximizing the marginal likelihood. A rapid approximation to a Hessian matrix is developed to implement the Bayesian RNN-LM (BRNN-LM) by selecting a small set of salient outer-products. The proposed BRNN-LM achieves a sparser model than the RNN-LM. Experiments on different corpora show the robustness of system performance by applying the rapid BRNN-LM under different conditions.
Representation of linguistic form and function in recurrent neural networks
Kadar, Akos; Chrupala, Grzegorz; Alishahi, Afra
2017-01-01
We present novel methods for analyzing the activation patterns of recurrent neural networks from a linguistic point of view and explore the types of linguistic structure they learn. As a case study, we use a standard standalone language model, and a multi-task gated recurrent network architecture
A hierarchical classification of first-order recurrent neural networks.
Cabessa, Jérémie; Vill, Alessandro E P
2010-12-31
We provide a decidable hierarchical classification of first-order recurrent neural networks made up of McCulloch and Pitts cells. This classification is achieved by proving an equivalence result between such neural networks and deterministic Büuchi automata, and then translating the Wadge classification theory from the abstract machine to the neural network context. The obtained hierarchy of neural networks is proved to have width 2 and height omega + 1, and a decidability procedure of this hierarchy is provided. Notably, this classification is shown to be intimately related to the attractive properties of the considered networks.
Noise-enhanced categorization in a recurrently reconnected neural network
Monterola, Christopher; Zapotocky, Martin
2005-03-01
We investigate the interplay of recurrence and noise in neural networks trained to categorize spatial patterns of neural activity. We develop the following procedure to demonstrate how, in the presence of noise, the introduction of recurrence permits to significantly extend and homogenize the operating range of a feed-forward neural network. We first train a two-level perceptron in the absence of noise. Following training, we identify the input and output units of the feed-forward network, and thus convert it into a two-layer recurrent network. We show that the performance of the reconnected network has features reminiscent of nondynamic stochastic resonance: the addition of noise enables the network to correctly categorize stimuli of subthreshold strength, with optimal noise magnitude significantly exceeding the stimulus strength. We characterize the dynamics leading to this effect and contrast it to the behavior of a more simple associative memory network in which noise-mediated categorization fails.
Noise-enhanced categorization in a recurrently reconnected neural network
International Nuclear Information System (INIS)
Monterola, Christopher; Zapotocky, Martin
2005-01-01
We investigate the interplay of recurrence and noise in neural networks trained to categorize spatial patterns of neural activity. We develop the following procedure to demonstrate how, in the presence of noise, the introduction of recurrence permits to significantly extend and homogenize the operating range of a feed-forward neural network. We first train a two-level perceptron in the absence of noise. Following training, we identify the input and output units of the feed-forward network, and thus convert it into a two-layer recurrent network. We show that the performance of the reconnected network has features reminiscent of nondynamic stochastic resonance: the addition of noise enables the network to correctly categorize stimuli of subthreshold strength, with optimal noise magnitude significantly exceeding the stimulus strength. We characterize the dynamics leading to this effect and contrast it to the behavior of a more simple associative memory network in which noise-mediated categorization fails
The computational power of interactive recurrent neural networks.
Cabessa, Jérémie; Siegelmann, Hava T
2012-04-01
In classical computation, rational- and real-weighted recurrent neural networks were shown to be respectively equivalent to and strictly more powerful than the standard Turing machine model. Here, we study the computational power of recurrent neural networks in a more biologically oriented computational framework, capturing the aspects of sequential interactivity and persistence of memory. In this context, we prove that so-called interactive rational- and real-weighted neural networks show the same computational powers as interactive Turing machines and interactive Turing machines with advice, respectively. A mathematical characterization of each of these computational powers is also provided. It follows from these results that interactive real-weighted neural networks can perform uncountably many more translations of information than interactive Turing machines, making them capable of super-Turing capabilities.
Energy Complexity of Recurrent Neural Networks
Czech Academy of Sciences Publication Activity Database
Šíma, Jiří
2014-01-01
Roč. 26, č. 5 (2014), s. 953-973 ISSN 0899-7667 R&D Projects: GA ČR GAP202/10/1333 Institutional support: RVO:67985807 Keywords : neural network * finite automaton * energy complexity * optimal size Subject RIV: IN - Informatics, Computer Science Impact factor: 2.207, year: 2014
Bach in 2014: Music Composition with Recurrent Neural Network
Liu, I-Ting; Ramakrishnan, Bhiksha
2014-01-01
We propose a framework for computer music composition that uses resilient propagation (RProp) and long short term memory (LSTM) recurrent neural network. In this paper, we show that LSTM network learns the structure and characteristics of music pieces properly by demonstrating its ability to recreate music. We also show that predicting existing music using RProp outperforms Back propagation through time (BPTT).
Probing the basins of attraction of a recurrent neural network
Heerema, M.; van Leeuwen, W.A.
2000-01-01
Analytical expressions for the weights $w_{ij}(b)$ of the connections of a recurrent neural network are found by taking explicitly into account basins of attraction, the size of which is characterized by a basin parameter $b$. It is shown that a network with $b \
Bayesian model ensembling using meta-trained recurrent neural networks
Ambrogioni, L.; Berezutskaya, Y.; Gü ç lü , U.; Borne, E.W.P. van den; Gü ç lü tü rk, Y.; Gerven, M.A.J. van; Maris, E.G.G.
2017-01-01
In this paper we demonstrate that a recurrent neural network meta-trained on an ensemble of arbitrary classification tasks can be used as an approximation of the Bayes optimal classifier. This result is obtained by relying on the framework of e-free approximate Bayesian inference, where the Bayesian
Railway track circuit fault diagnosis using recurrent neural networks
de Bruin, T.D.; Verbert, K.A.J.; Babuska, R.
2017-01-01
Timely detection and identification of faults in railway track circuits are crucial for the safety and availability of railway networks. In this paper, the use of the long-short-term memory (LSTM) recurrent neural network is proposed to accomplish these tasks based on the commonly available
A recurrent neural network with ever changing synapses
Heerema, M.; van Leeuwen, W.A.
2000-01-01
A recurrent neural network with noisy input is studied analytically, on the basis of a Discrete Time Master Equation. The latter is derived from a biologically realizable learning rule for the weights of the connections. In a numerical study it is found that the fixed points of the dynamics of the
Active Control of Sound based on Diagonal Recurrent Neural Network
Jayawardhana, Bayu; Xie, Lihua; Yuan, Shuqing
2002-01-01
Recurrent neural network has been known for its dynamic mapping and better suited for nonlinear dynamical system. Nonlinear controller may be needed in cases where the actuators exhibit the nonlinear characteristics, or in cases when the structure to be controlled exhibits nonlinear behavior. The
Convolutional over Recurrent Encoder for Neural Machine Translation
Directory of Open Access Journals (Sweden)
Dakwale Praveen
2017-06-01
Full Text Available Neural machine translation is a recently proposed approach which has shown competitive results to traditional MT approaches. Standard neural MT is an end-to-end neural network where the source sentence is encoded by a recurrent neural network (RNN called encoder and the target words are predicted using another RNN known as decoder. Recently, various models have been proposed which replace the RNN encoder with a convolutional neural network (CNN. In this paper, we propose to augment the standard RNN encoder in NMT with additional convolutional layers in order to capture wider context in the encoder output. Experiments on English to German translation demonstrate that our approach can achieve significant improvements over a standard RNN-based baseline.
Synthesis of recurrent neural networks for dynamical system simulation.
Trischler, Adam P; D'Eleuterio, Gabriele M T
2016-08-01
We review several of the most widely used techniques for training recurrent neural networks to approximate dynamical systems, then describe a novel algorithm for this task. The algorithm is based on an earlier theoretical result that guarantees the quality of the network approximation. We show that a feedforward neural network can be trained on the vector-field representation of a given dynamical system using backpropagation, then recast it as a recurrent network that replicates the original system's dynamics. After detailing this algorithm and its relation to earlier approaches, we present numerical examples that demonstrate its capabilities. One of the distinguishing features of our approach is that both the original dynamical systems and the recurrent networks that simulate them operate in continuous time. Copyright © 2016 Elsevier Ltd. All rights reserved.
Fractional Hopfield Neural Networks: Fractional Dynamic Associative Recurrent Neural Networks.
Pu, Yi-Fei; Yi, Zhang; Zhou, Ji-Liu
2017-10-01
This paper mainly discusses a novel conceptual framework: fractional Hopfield neural networks (FHNN). As is commonly known, fractional calculus has been incorporated into artificial neural networks, mainly because of its long-term memory and nonlocality. Some researchers have made interesting attempts at fractional neural networks and gained competitive advantages over integer-order neural networks. Therefore, it is naturally makes one ponder how to generalize the first-order Hopfield neural networks to the fractional-order ones, and how to implement FHNN by means of fractional calculus. We propose to introduce a novel mathematical method: fractional calculus to implement FHNN. First, we implement fractor in the form of an analog circuit. Second, we implement FHNN by utilizing fractor and the fractional steepest descent approach, construct its Lyapunov function, and further analyze its attractors. Third, we perform experiments to analyze the stability and convergence of FHNN, and further discuss its applications to the defense against chip cloning attacks for anticounterfeiting. The main contribution of our work is to propose FHNN in the form of an analog circuit by utilizing a fractor and the fractional steepest descent approach, construct its Lyapunov function, prove its Lyapunov stability, analyze its attractors, and apply FHNN to the defense against chip cloning attacks for anticounterfeiting. A significant advantage of FHNN is that its attractors essentially relate to the neuron's fractional order. FHNN possesses the fractional-order-stability and fractional-order-sensitivity characteristics.
Analysis of surface ozone using a recurrent neural network.
Biancofiore, Fabio; Verdecchia, Marco; Di Carlo, Piero; Tomassetti, Barbara; Aruffo, Eleonora; Busilacchio, Marcella; Bianco, Sebastiano; Di Tommaso, Sinibaldo; Colangeli, Carlo
2015-05-01
Hourly concentrations of ozone (O₃) and nitrogen dioxide (NO₂) have been measured for 16 years, from 1998 to 2013, in a seaside town in central Italy. The seasonal trends of O₃ and NO₂ recorded in this period have been studied. Furthermore, we used the data collected during one year (2005), to define the characteristics of a multiple linear regression model and a neural network model. Both models are used to model the hourly O₃ concentration, using, two scenarios: 1) in the first as inputs, only meteorological parameters and 2) in the second adding photochemical parameters at those of the first scenario. In order to evaluate the performance of the model four statistical criteria are used: correlation coefficient, fractional bias, normalized mean squared error and a factor of two. All the criteria show that the neural network gives better results, compared to the regression model, in all the model scenarios. Predictions of O₃ have been carried out by many authors using a feed forward neural architecture. In this paper we show that a recurrent architecture significantly improves the performances of neural predictors. Using only the meteorological parameters as input, the recurrent architecture shows performance better than the multiple linear regression model that uses meteorological and photochemical data as input, making the neural network model with recurrent architecture a more useful tool in areas where only weather measurements are available. Finally, we used the neural network model to forecast the O₃ hourly concentrations 1, 3, 6, 12, 24 and 48 h ahead. The performances of the model in predicting O₃ levels are discussed. Emphasis is given to the possibility of using the neural network model in operational ways in areas where only meteorological data are available, in order to predict O₃ also in sites where it has not been measured yet. Copyright © 2015 Elsevier B.V. All rights reserved.
A one-layer recurrent neural network for constrained nonsmooth optimization.
Liu, Qingshan; Wang, Jun
2011-10-01
This paper presents a novel one-layer recurrent neural network modeled by means of a differential inclusion for solving nonsmooth optimization problems, in which the number of neurons in the proposed neural network is the same as the number of decision variables of optimization problems. Compared with existing neural networks for nonsmooth optimization problems, the global convexity condition on the objective functions and constraints is relaxed, which allows the objective functions and constraints to be nonconvex. It is proven that the state variables of the proposed neural network are convergent to optimal solutions if a single design parameter in the model is larger than a derived lower bound. Numerical examples with simulation results substantiate the effectiveness and illustrate the characteristics of the proposed neural network.
Iterative free-energy optimization for recurrent neural networks (INFERNO.
Directory of Open Access Journals (Sweden)
Alexandre Pitti
Full Text Available The intra-parietal lobe coupled with the Basal Ganglia forms a working memory that demonstrates strong planning capabilities for generating robust yet flexible neuronal sequences. Neurocomputational models however, often fails to control long range neural synchrony in recurrent spiking networks due to spontaneous activity. As a novel framework based on the free-energy principle, we propose to see the problem of spikes' synchrony as an optimization problem of the neurons sub-threshold activity for the generation of long neuronal chains. Using a stochastic gradient descent, a reinforcement signal (presumably dopaminergic evaluates the quality of one input vector to move the recurrent neural network to a desired activity; depending on the error made, this input vector is strengthened to hill-climb the gradient or elicited to search for another solution. This vector can be learned then by one associative memory as a model of the basal-ganglia to control the recurrent neural network. Experiments on habit learning and on sequence retrieving demonstrate the capabilities of the dual system to generate very long and precise spatio-temporal sequences, above two hundred iterations. Its features are applied then to the sequential planning of arm movements. In line with neurobiological theories, we discuss its relevance for modeling the cortico-basal working memory to initiate flexible goal-directed neuronal chains of causation and its relation to novel architectures such as Deep Networks, Neural Turing Machines and the Free-Energy Principle.
Iterative free-energy optimization for recurrent neural networks (INFERNO)
2017-01-01
The intra-parietal lobe coupled with the Basal Ganglia forms a working memory that demonstrates strong planning capabilities for generating robust yet flexible neuronal sequences. Neurocomputational models however, often fails to control long range neural synchrony in recurrent spiking networks due to spontaneous activity. As a novel framework based on the free-energy principle, we propose to see the problem of spikes’ synchrony as an optimization problem of the neurons sub-threshold activity for the generation of long neuronal chains. Using a stochastic gradient descent, a reinforcement signal (presumably dopaminergic) evaluates the quality of one input vector to move the recurrent neural network to a desired activity; depending on the error made, this input vector is strengthened to hill-climb the gradient or elicited to search for another solution. This vector can be learned then by one associative memory as a model of the basal-ganglia to control the recurrent neural network. Experiments on habit learning and on sequence retrieving demonstrate the capabilities of the dual system to generate very long and precise spatio-temporal sequences, above two hundred iterations. Its features are applied then to the sequential planning of arm movements. In line with neurobiological theories, we discuss its relevance for modeling the cortico-basal working memory to initiate flexible goal-directed neuronal chains of causation and its relation to novel architectures such as Deep Networks, Neural Turing Machines and the Free-Energy Principle. PMID:28282439
Spatial Clockwork Recurrent Neural Network for Muscle Perimysium Segmentation.
Xie, Yuanpu; Zhang, Zizhao; Sapkota, Manish; Yang, Lin
2016-10-01
Accurate segmentation of perimysium plays an important role in early diagnosis of many muscle diseases because many diseases contain different perimysium inflammation. However, it remains as a challenging task due to the complex appearance of the perymisum morphology and its ambiguity to the background area. The muscle perimysium also exhibits strong structure spanned in the entire tissue, which makes it difficult for current local patch-based methods to capture this long-range context information. In this paper, we propose a novel spatial clockwork recurrent neural network (spatial CW-RNN) to address those issues. Specifically, we split the entire image into a set of non-overlapping image patches, and the semantic dependencies among them are modeled by the proposed spatial CW-RNN. Our method directly takes the 2D structure of the image into consideration and is capable of encoding the context information of the entire image into the local representation of each patch. Meanwhile, we leverage on the structured regression to assign one prediction mask rather than a single class label to each local patch, which enables both efficient training and testing. We extensively test our method for perimysium segmentation using digitized muscle microscopy images. Experimental results demonstrate the superiority of the novel spatial CW-RNN over other existing state of the arts.
A recurrent neural network for solving bilevel linear programming problem.
He, Xing; Li, Chuandong; Huang, Tingwen; Li, Chaojie; Huang, Junjian
2014-04-01
In this brief, based on the method of penalty functions, a recurrent neural network (NN) modeled by means of a differential inclusion is proposed for solving the bilevel linear programming problem (BLPP). Compared with the existing NNs for BLPP, the model has the least number of state variables and simple structure. Using nonsmooth analysis, the theory of differential inclusions, and Lyapunov-like method, the equilibrium point sequence of the proposed NNs can approximately converge to an optimal solution of BLPP under certain conditions. Finally, the numerical simulations of a supply chain distribution model have shown excellent performance of the proposed recurrent NNs.
Global robust stability of delayed recurrent neural networks
International Nuclear Information System (INIS)
Cao Jinde; Huang Deshuang; Qu Yuzhong
2005-01-01
This paper is concerned with the global robust stability of a class of delayed interval recurrent neural networks which contain time-invariant uncertain parameters whose values are unknown but bounded in given compact sets. A new sufficient condition is presented for the existence, uniqueness, and global robust stability of equilibria for interval neural networks with time delays by constructing Lyapunov functional and using matrix-norm inequality. An error is corrected in an earlier publication, and an example is given to show the effectiveness of the obtained results
Web server's reliability improvements using recurrent neural networks
DEFF Research Database (Denmark)
Madsen, Henrik; Albu, Rǎzvan-Daniel; Felea, Ioan
2012-01-01
In this paper we describe an interesting approach to error prediction illustrated by experimental results. The application consists of monitoring the activity for the web servers in order to collect the specific data. Predicting an error with severe consequences for the performance of a server (t...... usage, network usage and memory usage. We collect different data sets from monitoring the web server's activity and for each one we predict the server's reliability with the proposed recurrent neural network. © 2012 Taylor & Francis Group...
On the induction of temporal structure by recurrent neural networks
Shertil, MS
2014-01-01
Language acquisition is one of the core problems in artificial intelligence (AI) and it is generally accepted that any successful AI account of the mind will stand or fall depending on its ability to model human language. Simple Recurrent Networks (SRNs) are a class of so-called artificial neural networks that have a long history in language modelling via learning to predict the next word in a sentence. However, SRNs have also been shown to suffer from catastrophic forgetting, lack of syntact...
Homeostatic scaling of excitability in recurrent neural networks.
Directory of Open Access Journals (Sweden)
Michiel W H Remme
Full Text Available Neurons adjust their intrinsic excitability when experiencing a persistent change in synaptic drive. This process can prevent neural activity from moving into either a quiescent state or a saturated state in the face of ongoing plasticity, and is thought to promote stability of the network in which neurons reside. However, most neurons are embedded in recurrent networks, which require a delicate balance between excitation and inhibition to maintain network stability. This balance could be disrupted when neurons independently adjust their intrinsic excitability. Here, we study the functioning of activity-dependent homeostatic scaling of intrinsic excitability (HSE in a recurrent neural network. Using both simulations of a recurrent network consisting of excitatory and inhibitory neurons that implement HSE, and a mean-field description of adapting excitatory and inhibitory populations, we show that the stability of such adapting networks critically depends on the relationship between the adaptation time scales of both neuron populations. In a stable adapting network, HSE can keep all neurons functioning within their dynamic range, while the network is undergoing several (pathophysiologically relevant types of plasticity, such as persistent changes in external drive, changes in connection strengths, or the loss of inhibitory cells from the network. However, HSE cannot prevent the unstable network dynamics that result when, due to such plasticity, recurrent excitation in the network becomes too strong compared to feedback inhibition. This suggests that keeping a neural network in a stable and functional state requires the coordination of distinct homeostatic mechanisms that operate not only by adjusting neural excitability, but also by controlling network connectivity.
Parameter estimation in space systems using recurrent neural networks
Parlos, Alexander G.; Atiya, Amir F.; Sunkel, John W.
1991-01-01
The identification of time-varying parameters encountered in space systems is addressed, using artificial neural systems. A hybrid feedforward/feedback neural network, namely a recurrent multilayer perception, is used as the model structure in the nonlinear system identification. The feedforward portion of the network architecture provides its well-known interpolation property, while through recurrency and cross-talk, the local information feedback enables representation of temporal variations in the system nonlinearities. The standard back-propagation-learning algorithm is modified and it is used for both the off-line and on-line supervised training of the proposed hybrid network. The performance of recurrent multilayer perceptron networks in identifying parameters of nonlinear dynamic systems is investigated by estimating the mass properties of a representative large spacecraft. The changes in the spacecraft inertia are predicted using a trained neural network, during two configurations corresponding to the early and late stages of the spacecraft on-orbit assembly sequence. The proposed on-line mass properties estimation capability offers encouraging results, though, further research is warranted for training and testing the predictive capabilities of these networks beyond nominal spacecraft operations.
Prediction of Bladder Cancer Recurrences Using Artificial Neural Networks
Zulueta Guerrero, Ekaitz; Garay, Naiara Telleria; Lopez-Guede, Jose Manuel; Vilches, Borja Ayerdi; Iragorri, Eider Egilegor; Castaños, David Lecumberri; de La Hoz Rastrollo, Ana Belén; Peña, Carlos Pertusa
Even if considerable advances have been made in the field of early diagnosis, there is no simple, cheap and non-invasive method that can be applied to the clinical monitorisation of bladder cancer patients. Moreover, bladder cancer recurrences or the reappearance of the tumour after its surgical resection cannot be predicted in the current clinical setting. In this study, Artificial Neural Networks (ANN) were used to assess how different combinations of classical clinical parameters (stage-grade and age) and two urinary markers (growth factor and pro-inflammatory mediator) could predict post surgical recurrences in bladder cancer patients. Different ANN methods, input parameter combinations and recurrence related output variables were used and the resulting positive and negative prediction rates compared. MultiLayer Perceptron (MLP) was selected as the most predictive model and urinary markers showed the highest sensitivity, predicting correctly 50% of the patients that would recur in a 2 year follow-up period.
Effective learning in recurrent max-min neural networks.
Loe, Kia Fock; Teow, Loo Nin
1998-04-01
Max and min operations have interesting properties that facilitate the exchange of information between the symbolic and real-valued domains. As such, neural networks that employ max-min activation functions have been a subject of interest in recent years. Since max-min functions are not strictly differentiable, we propose a mathematically sound learning method based on using Fourier convergence analysis of side-derivatives to derive a gradient descent technique for max-min error functions. We then propose a novel recurrent max-min neural network model that is trained to perform grammatical inference as an application example. Comparisons made between this model and recurrent sigmoidal neural networks show that our model not only performs better in terms of learning speed and generalization, but that its final weight configuration allows a deterministic finite automation (DFA) to be extracted in a straightforward manner. In essence, we are able to demonstrate that our proposed gradient descent technique does allow max-min neural networks to learn effectively.
Recurrent Neural Network for Computing the Drazin Inverse.
Stanimirović, Predrag S; Zivković, Ivan S; Wei, Yimin
2015-11-01
This paper presents a recurrent neural network (RNN) for computing the Drazin inverse of a real matrix in real time. This recurrent neural network (RNN) is composed of n independent parts (subnetworks), where n is the order of the input matrix. These subnetworks can operate concurrently, so parallel and distributed processing can be achieved. In this way, the computational advantages over the existing sequential algorithms can be attained in real-time applications. The RNN defined in this paper is convenient for an implementation in an electronic circuit. The number of neurons in the neural network is the same as the number of elements in the output matrix, which represents the Drazin inverse. The difference between the proposed RNN and the existing ones for the Drazin inverse computation lies in their network architecture and dynamics. The conditions that ensure the stability of the defined RNN as well as its convergence toward the Drazin inverse are considered. In addition, illustrative examples and examples of application to the practical engineering problems are discussed to show the efficacy of the proposed neural network.
Ideomotor feedback control in a recurrent neural network.
Galtier, Mathieu
2015-06-01
The architecture of a neural network controlling an unknown environment is presented. It is based on a randomly connected recurrent neural network from which both perception and action are simultaneously read and fed back. There are two concurrent learning rules implementing a sort of ideomotor control: (i) perception is learned along the principle that the network should predict reliably its incoming stimuli; (ii) action is learned along the principle that the prediction of the network should match a target time series. The coherent behavior of the neural network in its environment is a consequence of the interaction between the two principles. Numerical simulations show a promising performance of the approach, which can be turned into a local and better "biologically plausible" algorithm.
A Recurrent Neural Network for Nonlinear Fractional Programming
Directory of Open Access Journals (Sweden)
Quan-Ju Zhang
2012-01-01
Full Text Available This paper presents a novel recurrent time continuous neural network model which performs nonlinear fractional optimization subject to interval constraints on each of the optimization variables. The network is proved to be complete in the sense that the set of optima of the objective function to be minimized with interval constraints coincides with the set of equilibria of the neural network. It is also shown that the network is primal and globally convergent in the sense that its trajectory cannot escape from the feasible region and will converge to an exact optimal solution for any initial point being chosen in the feasible interval region. Simulation results are given to demonstrate further the global convergence and good performance of the proposing neural network for nonlinear fractional programming problems with interval constraints.
Convolutional neural networks for prostate cancer recurrence prediction
Kumar, Neeraj; Verma, Ruchika; Arora, Ashish; Kumar, Abhay; Gupta, Sanchit; Sethi, Amit; Gann, Peter H.
2017-03-01
Accurate prediction of the treatment outcome is important for cancer treatment planning. We present an approach to predict prostate cancer (PCa) recurrence after radical prostatectomy using tissue images. We used a cohort whose case vs. control (recurrent vs. non-recurrent) status had been determined using post-treatment follow up. Further, to aid the development of novel biomarkers of PCa recurrence, cases and controls were paired based on matching of other predictive clinical variables such as Gleason grade, stage, age, and race. For this cohort, tissue resection microarray with up to four cores per patient was available. The proposed approach is based on deep learning, and its novelty lies in the use of two separate convolutional neural networks (CNNs) - one to detect individual nuclei even in the crowded areas, and the other to classify them. To detect nuclear centers in an image, the first CNN predicts distance transform of the underlying (but unknown) multi-nuclear map from the input HE image. The second CNN classifies the patches centered at nuclear centers into those belonging to cases or controls. Voting across patches extracted from image(s) of a patient yields the probability of recurrence for the patient. The proposed approach gave 0.81 AUC for a sample of 30 recurrent cases and 30 non-recurrent controls, after being trained on an independent set of 80 case-controls pairs. If validated further, such an approach might help in choosing between a combination of treatment options such as active surveillance, radical prostatectomy, radiation, and hormone therapy. It can also generalize to the prediction of treatment outcomes in other cancers.
Land Cover Classification via Multitemporal Spatial Data by Deep Recurrent Neural Networks
Ienco, Dino; Gaetano, Raffaele; Dupaquier, Claire; Maurel, Pierre
2017-10-01
Nowadays, modern earth observation programs produce huge volumes of satellite images time series (SITS) that can be useful to monitor geographical areas through time. How to efficiently analyze such kind of information is still an open question in the remote sensing field. Recently, deep learning methods proved suitable to deal with remote sensing data mainly for scene classification (i.e. Convolutional Neural Networks - CNNs - on single images) while only very few studies exist involving temporal deep learning approaches (i.e Recurrent Neural Networks - RNNs) to deal with remote sensing time series. In this letter we evaluate the ability of Recurrent Neural Networks, in particular the Long-Short Term Memory (LSTM) model, to perform land cover classification considering multi-temporal spatial data derived from a time series of satellite images. We carried out experiments on two different datasets considering both pixel-based and object-based classification. The obtained results show that Recurrent Neural Networks are competitive compared to state-of-the-art classifiers, and may outperform classical approaches in presence of low represented and/or highly mixed classes. We also show that using the alternative feature representation generated by LSTM can improve the performances of standard classifiers.
Sensitivity analysis of linear programming problem through a recurrent neural network
Das, Raja
2017-11-01
In this paper we study the recurrent neural network for solving linear programming problems. To achieve optimality in accuracy and also in computational effort, an algorithm is presented. We investigate the sensitivity analysis of linear programming problem through the neural network. A detailed example is also presented to demonstrate the performance of the recurrent neural network.
Fine-tuning and the stability of recurrent neural networks.
Directory of Open Access Journals (Sweden)
David MacNeil
Full Text Available A central criticism of standard theoretical approaches to constructing stable, recurrent model networks is that the synaptic connection weights need to be finely-tuned. This criticism is severe because proposed rules for learning these weights have been shown to have various limitations to their biological plausibility. Hence it is unlikely that such rules are used to continuously fine-tune the network in vivo. We describe a learning rule that is able to tune synaptic weights in a biologically plausible manner. We demonstrate and test this rule in the context of the oculomotor integrator, showing that only known neural signals are needed to tune the weights. We demonstrate that the rule appropriately accounts for a wide variety of experimental results, and is robust under several kinds of perturbation. Furthermore, we show that the rule is able to achieve stability as good as or better than that provided by the linearly optimal weights often used in recurrent models of the integrator. Finally, we discuss how this rule can be generalized to tune a wide variety of recurrent attractor networks, such as those found in head direction and path integration systems, suggesting that it may be used to tune a wide variety of stable neural systems.
Software reliability prediction using recurrent neural network with Bayesian regularization.
Tian, Liang; Noore, Afzel
2004-06-01
A recurrent neural network modeling approach for software reliability prediction with respect to cumulative failure time is proposed. Our proposed network structure has the capability of learning and recognizing the inherent internal temporal property of cumulative failure time sequence. Further, by adding a penalty term of sum of network connection weights, Bayesian regularization is applied to our network training scheme to improve the generalization capability and lower the susceptibility of overfitting. The performance of our proposed approach has been tested using four real-time control and flight dynamic application data sets. Numerical results show that our proposed approach is robust across different software projects, and has a better performance with respect to both goodness-of-fit and next-step-predictability compared to existing neural network models for failure time prediction.
Inferring network interactions using recurrent neural networks and swarm intelligence.
Ressom, Habtom W; Zhang, Yuji; Xuan, Jianhua; Wang, Yue; Clarke, Robert
2006-01-01
We present a novel algorithm combining artificial neural networks and swarm intelligence (SI) methods to infer network interactions. The algorithm uses ant colony optimization (ACO) to identify the optimal architecture of a recurrent neural network (RNN), while the weights of the RNN are optimized using particle swarm optimization (PSO). Our goal is to construct an RNN that mimics the true structure of an unknown network and the time-series data that the network generated. We applied the proposed hybrid SI-RNN algorithm to infer a simulated genetic network. The results indicate that the algorithm has a promising potential to infer complex interactions such as gene regulatory networks from time-series gene expression data.
Estimating Ads’ Click through Rate with Recurrent Neural Network
Directory of Open Access Journals (Sweden)
Chen Qiao-Hong
2016-01-01
Full Text Available With the development of the Internet, online advertising spreads across every corner of the world, the ads' click through rate (CTR estimation is an important method to improve the online advertising revenue. Compared with the linear model, the nonlinear models can study much more complex relationships between a large number of nonlinear characteristics, so as to improve the accuracy of the estimation of the ads’ CTR. The recurrent neural network (RNN based on Long-Short Term Memory (LSTM is an improved model of the feedback neural network with ring structure. The model overcomes the problem of the gradient of the general RNN. Experiments show that the RNN based on LSTM exceeds the linear models, and it can effectively improve the estimation effect of the ads’ click through rate.
Very deep recurrent convolutional neural network for object recognition
Brahimi, Sourour; Ben Aoun, Najib; Ben Amar, Chokri
2017-03-01
In recent years, Computer vision has become a very active field. This field includes methods for processing, analyzing, and understanding images. The most challenging problems in computer vision are image classification and object recognition. This paper presents a new approach for object recognition task. This approach exploits the success of the Very Deep Convolutional Neural Network for object recognition. In fact, it improves the convolutional layers by adding recurrent connections. This proposed approach was evaluated on two object recognition benchmarks: Pascal VOC 2007 and CIFAR-10. The experimental results prove the efficiency of our method in comparison with the state of the art methods.
Optimizing Markovian modeling of chaotic systems with recurrent neural networks
International Nuclear Information System (INIS)
Cechin, Adelmo L.; Pechmann, Denise R.; Oliveira, Luiz P.L. de
2008-01-01
In this paper, we propose a methodology for optimizing the modeling of an one-dimensional chaotic time series with a Markov Chain. The model is extracted from a recurrent neural network trained for the attractor reconstructed from the data set. Each state of the obtained Markov Chain is a region of the reconstructed state space where the dynamics is approximated by a specific piecewise linear map, obtained from the network. The Markov Chain represents the dynamics of the time series in its statistical essence. An application to a time series resulted from Lorenz system is included
Emergent latent symbol systems in recurrent neural networks
Monner, Derek; Reggia, James A.
2012-12-01
Fodor and Pylyshyn [(1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1-2), 3-71] famously argued that neural networks cannot behave systematically short of implementing a combinatorial symbol system. A recent response from Frank et al. [(2009). Connectionist semantic systematicity. Cognition, 110(3), 358-379] claimed to have trained a neural network to behave systematically without implementing a symbol system and without any in-built predisposition towards combinatorial representations. We believe systems like theirs may in fact implement a symbol system on a deeper and more interesting level: one where the symbols are latent - not visible at the level of network structure. In order to illustrate this possibility, we demonstrate our own recurrent neural network that learns to understand sentence-level language in terms of a scene. We demonstrate our model's learned understanding by testing it on novel sentences and scenes. By paring down our model into an architecturally minimal version, we demonstrate how it supports combinatorial computation over distributed representations by using the associative memory operations of Vector Symbolic Architectures. Knowledge of the model's memory scheme gives us tools to explain its errors and construct superior future models. We show how the model designs and manipulates a latent symbol system in which the combinatorial symbols are patterns of activation distributed across the layers of a neural network, instantiating a hybrid of classical symbolic and connectionist representations that combines advantages of both.
Encoding Time in Feedforward Trajectories of a Recurrent Neural Network Model.
Hardy, N F; Buonomano, Dean V
2018-02-01
Brain activity evolves through time, creating trajectories of activity that underlie sensorimotor processing, behavior, and learning and memory. Therefore, understanding the temporal nature of neural dynamics is essential to understanding brain function and behavior. In vivo studies have demonstrated that sequential transient activation of neurons can encode time. However, it remains unclear whether these patterns emerge from feedforward network architectures or from recurrent networks and, furthermore, what role network structure plays in timing. We address these issues using a recurrent neural network (RNN) model with distinct populations of excitatory and inhibitory units. Consistent with experimental data, a single RNN could autonomously produce multiple functionally feedforward trajectories, thus potentially encoding multiple timed motor patterns lasting up to several seconds. Importantly, the model accounted for Weber's law, a hallmark of timing behavior. Analysis of network connectivity revealed that efficiency-a measure of network interconnectedness-decreased as the number of stored trajectories increased. Additionally, the balance of excitation (E) and inhibition (I) shifted toward excitation during each unit's activation time, generating the prediction that observed sequential activity relies on dynamic control of the E/I balance. Our results establish for the first time that the same RNN can generate multiple functionally feedforward patterns of activity as a result of dynamic shifts in the E/I balance imposed by the connectome of the RNN. We conclude that recurrent network architectures account for sequential neural activity, as well as for a fundamental signature of timing behavior: Weber's law.
Classification of conductance traces with recurrent neural networks.
Lauritzen, Kasper P; Magyarkuti, András; Balogh, Zoltán; Halbritter, András; Solomon, Gemma C
2018-02-28
We present a new automated method for structural classification of the traces obtained in break junction experiments. Using recurrent neural networks trained on the traces of minimal cross-sectional area in molecular dynamics simulations, we successfully separate the traces into two classes: point contact or nanowire. This is done without any assumptions about the expected features of each class. The trained neural network is applied to experimental break junction conductance traces, and it separates the classes as well as the previously used experimental methods. The effect of using partial conductance traces is explored, and we show that the method performs equally well using full or partial traces (as long as the trace just prior to breaking is included). When only the initial part of the trace is included, the results are still better than random chance. Finally, we show that the neural network classification method can be used to classify experimental conductance traces without using simulated results for training, but instead training the network on a few representative experimental traces. This offers a tool to recognize some characteristic motifs of the traces, which can be hard to find by simple data selection algorithms.
A modular architecture for transparent computation in recurrent neural networks.
Carmantini, Giovanni S; Beim Graben, Peter; Desroches, Mathieu; Rodrigues, Serafim
2017-01-01
Computation is classically studied in terms of automata, formal languages and algorithms; yet, the relation between neural dynamics and symbolic representations and operations is still unclear in traditional eliminative connectionism. Therefore, we suggest a unique perspective on this central issue, to which we would like to refer as transparent connectionism, by proposing accounts of how symbolic computation can be implemented in neural substrates. In this study we first introduce a new model of dynamics on a symbolic space, the versatile shift, showing that it supports the real-time simulation of a range of automata. We then show that the Gödelization of versatile shifts defines nonlinear dynamical automata, dynamical systems evolving on a vectorial space. Finally, we present a mapping between nonlinear dynamical automata and recurrent artificial neural networks. The mapping defines an architecture characterized by its granular modularity, where data, symbolic operations and their control are not only distinguishable in activation space, but also spatially localizable in the network itself, while maintaining a distributed encoding of symbolic representations. The resulting networks simulate automata in real-time and are programmed directly, in the absence of network training. To discuss the unique characteristics of the architecture and their consequences, we present two examples: (i) the design of a Central Pattern Generator from a finite-state locomotive controller, and (ii) the creation of a network simulating a system of interactive automata that supports the parsing of garden-path sentences as investigated in psycholinguistics experiments. Copyright © 2016 Elsevier Ltd. All rights reserved.
Classification of conductance traces with recurrent neural networks
Lauritzen, Kasper P.; Magyarkuti, András; Balogh, Zoltán; Halbritter, András; Solomon, Gemma C.
2018-02-01
We present a new automated method for structural classification of the traces obtained in break junction experiments. Using recurrent neural networks trained on the traces of minimal cross-sectional area in molecular dynamics simulations, we successfully separate the traces into two classes: point contact or nanowire. This is done without any assumptions about the expected features of each class. The trained neural network is applied to experimental break junction conductance traces, and it separates the classes as well as the previously used experimental methods. The effect of using partial conductance traces is explored, and we show that the method performs equally well using full or partial traces (as long as the trace just prior to breaking is included). When only the initial part of the trace is included, the results are still better than random chance. Finally, we show that the neural network classification method can be used to classify experimental conductance traces without using simulated results for training, but instead training the network on a few representative experimental traces. This offers a tool to recognize some characteristic motifs of the traces, which can be hard to find by simple data selection algorithms.
Tuning Recurrent Neural Networks for Recognizing Handwritten Arabic Words
Qaralleh, Esam
2013-10-01
Artificial neural networks have the abilities to learn by example and are capable of solving problems that are hard to solve using ordinary rule-based programming. They have many design parameters that affect their performance such as the number and sizes of the hidden layers. Large sizes are slow and small sizes are generally not accurate. Tuning the neural network size is a hard task because the design space is often large and training is often a long process. We use design of experiments techniques to tune the recurrent neural network used in an Arabic handwriting recognition system. We show that best results are achieved with three hidden layers and two subsampling layers. To tune the sizes of these five layers, we use fractional factorial experiment design to limit the number of experiments to a feasible number. Moreover, we replicate the experiment configuration multiple times to overcome the randomness in the training process. The accuracy and time measurements are analyzed and modeled. The two models are then used to locate network sizes that are on the Pareto optimal frontier. The approach described in this paper reduces the label error from 26.2% to 19.8%.
Region stability analysis and tracking control of memristive recurrent neural network.
Bao, Gang; Zeng, Zhigang; Shen, Yanjun
2018-02-01
Memristor is firstly postulated by Leon Chua and realized by Hewlett-Packard (HP) laboratory. Research results show that memristor can be used to simulate the synapses of neurons. This paper presents a class of recurrent neural network with HP memristors. Firstly, it shows that memristive recurrent neural network has more compound dynamics than the traditional recurrent neural network by simulations. Then it derives that n dimensional memristive recurrent neural network is composed of [Formula: see text] sub neural networks which do not have a common equilibrium point. By designing the tracking controller, it can make memristive neural network being convergent to the desired sub neural network. At last, two numerical examples are given to verify the validity of our result. Copyright © 2017 Elsevier Ltd. All rights reserved.
A novel recurrent neural network with finite-time convergence for linear programming.
Liu, Qingshan; Cao, Jinde; Chen, Guanrong
2010-11-01
In this letter, a novel recurrent neural network based on the gradient method is proposed for solving linear programming problems. Finite-time convergence of the proposed neural network is proved by using the Lyapunov method. Compared with the existing neural networks for linear programming, the proposed neural network is globally convergent to exact optimal solutions in finite time, which is remarkable and rare in the literature of neural networks for optimization. Some numerical examples are given to show the effectiveness and excellent performance of the new recurrent neural network.
Recurrent Neural Network Approach Based on the Integral Representation of the Drazin Inverse.
Stanimirović, Predrag S; Živković, Ivan S; Wei, Yimin
2015-10-01
In this letter, we present the dynamical equation and corresponding artificial recurrent neural network for computing the Drazin inverse for arbitrary square real matrix, without any restriction on its eigenvalues. Conditions that ensure the stability of the defined recurrent neural network as well as its convergence toward the Drazin inverse are considered. Several illustrative examples present the results of computer simulations.
A recurrent neural network for adaptive beamforming and array correction.
Che, Hangjun; Li, Chuandong; He, Xing; Huang, Tingwen
2016-08-01
In this paper, a recurrent neural network (RNN) is proposed for solving adaptive beamforming problem. In order to minimize sidelobe interference, the problem is described as a convex optimization problem based on linear array model. RNN is designed to optimize system's weight values in the feasible region which is derived from arrays' state and plane wave's information. The new algorithm is proven to be stable and converge to optimal solution in the sense of Lyapunov. So as to verify new algorithm's performance, we apply it to beamforming under array mismatch situation. Comparing with other optimization algorithms, simulations suggest that RNN has strong ability to search for exact solutions under the condition of large scale constraints. Copyright © 2016 Elsevier Ltd. All rights reserved.
Global robust exponential stability analysis for interval recurrent neural networks
International Nuclear Information System (INIS)
Xu Shengyuan; Lam, James; Ho, Daniel W.C.; Zou Yun
2004-01-01
This Letter investigates the problem of robust global exponential stability analysis for interval recurrent neural networks (RNNs) via the linear matrix inequality (LMI) approach. The values of the time-invariant uncertain parameters are assumed to be bounded within given compact sets. An improved condition for the existence of a unique equilibrium point and its global exponential stability of RNNs with known parameters is proposed. Based on this, a sufficient condition for the global robust exponential stability for interval RNNs is obtained. Both of the conditions are expressed in terms of LMIs, which can be checked easily by various recently developed convex optimization algorithms. Examples are provided to demonstrate the reduced conservatism of the proposed exponential stability condition
Web server's reliability improvements using recurrent neural networks
DEFF Research Database (Denmark)
Madsen, Henrik; Albu, Rǎzvan-Daniel; Felea, Ioan
2012-01-01
result of which is that its functionality becomes totally inaccessible or hard to access for clients) requires measuring the capacity of a server at any given time. This measurement is highly complex, if not impossible. There are several variables which we can measure on a running system, such as: CPU......In this paper we describe an interesting approach to error prediction illustrated by experimental results. The application consists of monitoring the activity for the web servers in order to collect the specific data. Predicting an error with severe consequences for the performance of a server (the...... usage, network usage and memory usage. We collect different data sets from monitoring the web server's activity and for each one we predict the server's reliability with the proposed recurrent neural network. © 2012 Taylor & Francis Group...
Drawing and Recognizing Chinese Characters with Recurrent Neural Network.
Zhang, Xu-Yao; Yin, Fei; Zhang, Yan-Ming; Liu, Cheng-Lin; Bengio, Yoshua
2018-04-01
Recent deep learning based approaches have achieved great success on handwriting recognition. Chinese characters are among the most widely adopted writing systems in the world. Previous research has mainly focused on recognizing handwritten Chinese characters. However, recognition is only one aspect for understanding a language, another challenging and interesting task is to teach a machine to automatically write (pictographic) Chinese characters. In this paper, we propose a framework by using the recurrent neural network (RNN) as both a discriminative model for recognizing Chinese characters and a generative model for drawing (generating) Chinese characters. To recognize Chinese characters, previous methods usually adopt the convolutional neural network (CNN) models which require transforming the online handwriting trajectory into image-like representations. Instead, our RNN based approach is an end-to-end system which directly deals with the sequential structure and does not require any domain-specific knowledge. With the RNN system (combining an LSTM and GRU), state-of-the-art performance can be achieved on the ICDAR-2013 competition database. Furthermore, under the RNN framework, a conditional generative model with character embedding is proposed for automatically drawing recognizable Chinese characters. The generated characters (in vector format) are human-readable and also can be recognized by the discriminative RNN model with high accuracy. Experimental results verify the effectiveness of using RNNs as both generative and discriminative models for the tasks of drawing and recognizing Chinese characters.
Recurrent Neural Network Applications for Astronomical Time Series
Protopapas, Pavlos
2017-06-01
The benefits of good predictive models in astronomy lie in early event prediction systems and effective resource allocation. Current time series methods applicable to regular time series have not evolved to generalize for irregular time series. In this talk, I will describe two Recurrent Neural Network methods, Long Short-Term Memory (LSTM) and Echo State Networks (ESNs) for predicting irregular time series. Feature engineering along with a non-linear modeling proved to be an effective predictor. For noisy time series, the prediction is improved by training the network on error realizations using the error estimates from astronomical light curves. In addition to this, we propose a new neural network architecture to remove correlation from the residuals in order to improve prediction and compensate for the noisy data. Finally, I show how to set hyperparameters for a stable and performant solution correctly. In this work, we circumvent this obstacle by optimizing ESN hyperparameters using Bayesian optimization with Gaussian Process priors. This automates the tuning procedure, enabling users to employ the power of RNN without needing an in-depth understanding of the tuning procedure.
Recurrent Neural Networks to Correct Satellite Image Classification Maps
Maggiori, Emmanuel; Charpiat, Guillaume; Tarabalka, Yuliya; Alliez, Pierre
2017-09-01
While initially devised for image categorization, convolutional neural networks (CNNs) are being increasingly used for the pixelwise semantic labeling of images. However, the proper nature of the most common CNN architectures makes them good at recognizing but poor at localizing objects precisely. This problem is magnified in the context of aerial and satellite image labeling, where a spatially fine object outlining is of paramount importance. Different iterative enhancement algorithms have been presented in the literature to progressively improve the coarse CNN outputs, seeking to sharpen object boundaries around real image edges. However, one must carefully design, choose and tune such algorithms. Instead, our goal is to directly learn the iterative process itself. For this, we formulate a generic iterative enhancement process inspired from partial differential equations, and observe that it can be expressed as a recurrent neural network (RNN). Consequently, we train such a network from manually labeled data for our enhancement task. In a series of experiments we show that our RNN effectively learns an iterative process that significantly improves the quality of satellite image classification maps.
Global dissipativity of continuous-time recurrent neural networks with time delay
International Nuclear Information System (INIS)
Liao Xiaoxin; Wang Jun
2003-01-01
This paper addresses the global dissipativity of a general class of continuous-time recurrent neural networks. First, the concepts of global dissipation and global exponential dissipation are defined and elaborated. Next, the sets of global dissipativity and global exponentially dissipativity are characterized using the parameters of recurrent neural network models. In particular, it is shown that the Hopfield network and cellular neural networks with or without time delays are dissipative systems
A generalized LSTM-like training algorithm for second-order recurrent neural networks.
Monner, Derek; Reggia, James A
2012-01-01
The long short term memory (LSTM) is a second-order recurrent neural network architecture that excels at storing sequential short-term memories and retrieving them many time-steps later. LSTM's original training algorithm provides the important properties of spatial and temporal locality, which are missing from other training approaches, at the cost of limiting its applicability to a small set of network architectures. Here we introduce the generalized long short-term memory(LSTM-g) training algorithm, which provides LSTM-like locality while being applicable without modification to a much wider range of second-order network architectures. With LSTM-g, all units have an identical set of operating instructions for both activation and learning, subject only to the configuration of their local environment in the network; this is in contrast to the original LSTM training algorithm, where each type of unit has its own activation and training instructions. When applied to LSTM architectures with peephole connections, LSTM-g takes advantage of an additional source of back-propagated error which can enable better performance than the original algorithm. Enabled by the broad architectural applicability of LSTM-g, we demonstrate that training recurrent networks engineered for specific tasks can produce better results than single-layer networks. We conclude that LSTM-g has the potential to both improve the performance and broaden the applicability of spatially and temporally local gradient-based training algorithms for recurrent neural networks. Copyright © 2011 Elsevier Ltd. All rights reserved.
Directory of Open Access Journals (Sweden)
Eduard eGrinke
2015-10-01
Full Text Available Walking animals, like insects, with little neural computing can effectively perform complex behaviors. They can walk around their environment, escape from corners/deadlocks, and avoid or climb over obstacles. While performing all these behaviors, they can also adapt their movements to deal with an unknown situation. As a consequence, they successfully navigate through their complex environment. The versatile and adaptive abilities are the result of an integration of several ingredients embedded in their sensorimotor loop. Biological studies reveal that the ingredients include neural dynamics, plasticity, sensory feedback, and biomechanics. Generating such versatile and adaptive behaviors for a walking robot is a challenging task. In this study, we present a bio-inspired approach to solve this task. Specifically, the approach combines neural mechanisms with plasticity, sensory feedback, and biomechanics. The neural mechanisms consist of adaptive neural sensory processing and modular neural locomotion control. The sensory processing is based on a small recurrent network consisting of two fully connected neurons. Online correlation-based learning with synaptic scaling is applied to adequately change the connections of the network. By doing so, we can effectively exploit neural dynamics (i.e., hysteresis effects and single attractors in the network to generate different turning angles with short-term memory for a biomechanical walking robot. The turning information is transmitted as descending steering signals to the locomotion control which translates the signals into motor actions. As a result, the robot can walk around and adapt its turning angle for avoiding obstacles in different situations as well as escaping from sharp corners or deadlocks. Using backbone joint control embedded in the locomotion control allows the robot to climb over small obstacles. Consequently, it can successfully explore and navigate in complex environments.
Recurrent Neural Network Based Boolean Factor Analysis and its Application to Word Clustering
Czech Academy of Sciences Publication Activity Database
Frolov, A. A.; Húsek, Dušan; Polyakov, P.Y.
2009-01-01
Roč. 20, č. 7 (2009), s. 1073-1086 ISSN 1045-9227 R&D Projects: GA MŠk(CZ) 1M0567 Institutional research plan: CEZ:AV0Z10300504 Keywords : recurrent neural network * Hopfield-like neural network * associative memory * unsupervised learning * neural network architecture * neural network application * statistics * Boolean factor analysis * concepts search * information retrieval Subject RIV: BB - Applied Statistics, Operational Research Impact factor: 2.889, year: 2009
Application of recurrent neural networks for drought projections in California
Le, J. A.; El-Askary, H. M.; Allali, M.; Struppa, D. C.
2017-05-01
We use recurrent neural networks (RNNs) to investigate the complex interactions between the long-term trend in dryness and a projected, short but intense, period of wetness due to the 2015-2016 El Niño. Although it was forecasted that this El Niño season would bring significant rainfall to the region, our long-term projections of the Palmer Z Index (PZI) showed a continuing drought trend, contrasting with the 1998-1999 El Niño event. RNN training considered PZI data during 1896-2006 that was validated against the 2006-2015 period to evaluate the potential of extreme precipitation forecast. We achieved a statistically significant correlation of 0.610 between forecasted and observed PZI on the validation set for a lead time of 1 month. This gives strong confidence to the forecasted precipitation indicator. The 2015-2016 El Niño season proved to be relatively weak as compared with the 1997-1998, with a peak PZI anomaly of 0.242 standard deviations below historical averages, continuing drought conditions.
Recurrent Neural Network Model for Constructive Peptide Design.
Müller, Alex T; Hiss, Jan A; Schneider, Gisbert
2018-02-26
We present a generative long short-term memory (LSTM) recurrent neural network (RNN) for combinatorial de novo peptide design. RNN models capture patterns in sequential data and generate new data instances from the learned context. Amino acid sequences represent a suitable input for these machine-learning models. Generative models trained on peptide sequences could therefore facilitate the design of bespoke peptide libraries. We trained RNNs with LSTM units on pattern recognition of helical antimicrobial peptides and used the resulting model for de novo sequence generation. Of these sequences, 82% were predicted to be active antimicrobial peptides compared to 65% of randomly sampled sequences with the same amino acid distribution as the training set. The generated sequences also lie closer to the training data than manually designed amphipathic helices. The results of this study showcase the ability of LSTM RNNs to construct new amino acid sequences within the applicability domain of the model and motivate their prospective application to peptide and protein design without the need for the exhaustive enumeration of sequence libraries.
Multiplex visibility graphs to investigate recurrent neural network dynamics
Bianchi, Filippo Maria; Livi, Lorenzo; Alippi, Cesare; Jenssen, Robert
2017-03-01
A recurrent neural network (RNN) is a universal approximator of dynamical systems, whose performance often depends on sensitive hyperparameters. Tuning them properly may be difficult and, typically, based on a trial-and-error approach. In this work, we adopt a graph-based framework to interpret and characterize internal dynamics of a class of RNNs called echo state networks (ESNs). We design principled unsupervised methods to derive hyperparameters configurations yielding maximal ESN performance, expressed in terms of prediction error and memory capacity. In particular, we propose to model time series generated by each neuron activations with a horizontal visibility graph, whose topological properties have been shown to be related to the underlying system dynamics. Successively, horizontal visibility graphs associated with all neurons become layers of a larger structure called a multiplex. We show that topological properties of such a multiplex reflect important features of ESN dynamics that can be used to guide the tuning of its hyperparamers. Results obtained on several benchmarks and a real-world dataset of telephone call data records show the effectiveness of the proposed methods.
Fast computation with spikes in a recurrent neural network
International Nuclear Information System (INIS)
Jin, Dezhe Z.; Seung, H. Sebastian
2002-01-01
Neural networks with recurrent connections are sometimes regarded as too slow at computation to serve as models of the brain. Here we analytically study a counterexample, a network consisting of N integrate-and-fire neurons with self excitation, all-to-all inhibition, instantaneous synaptic coupling, and constant external driving inputs. When the inhibition and/or excitation are large enough, the network performs a winner-take-all computation for all possible external inputs and initial states of the network. The computation is done very quickly: As soon as the winner spikes once, the computation is completed since no other neurons will spike. For some initial states, the winner is the first neuron to spike, and the computation is done at the first spike of the network. In general, there are M potential winners, corresponding to the top M external inputs. When the external inputs are close in magnitude, M tends to be larger. If M>1, the selection of the actual winner is strongly influenced by the initial states. If a special relation between the excitation and inhibition is satisfied, the network always selects the neuron with the maximum external input as the winner
Railway Track Circuit Fault Diagnosis Using Recurrent Neural Networks.
de Bruin, Tim; Verbert, Kim; Babuska, Robert
2017-03-01
Timely detection and identification of faults in railway track circuits are crucial for the safety and availability of railway networks. In this paper, the use of the long-short-term memory (LSTM) recurrent neural network is proposed to accomplish these tasks based on the commonly available measurement signals. By considering the signals from multiple track circuits in a geographic area, faults are diagnosed from their spatial and temporal dependences. A generative model is used to show that the LSTM network can learn these dependences directly from the data. The network correctly classifies 99.7% of the test input sequences, with no false positive fault detections. In addition, the t-Distributed Stochastic Neighbor Embedding (t-SNE) method is used to examine the resulting network, further showing that it has learned the relevant dependences in the data. Finally, we compare our LSTM network with a convolutional network trained on the same task. From this comparison, we conclude that the LSTM network architecture is better suited for the railway track circuit fault detection and identification tasks than the convolutional network.
Scene Segmentation with DAG-Recurrent Neural Networks.
Shuai, Bing; Zup, Zhen; Wang, Bing; Wang, Gang
2017-06-06
In this paper, we address the challenging task of scene segmentation. In order to capture the rich contextual dependencies over image regions, we propose Directed Acyclic Graph - Recurrent Neural Networks (DAG-RNN) to perform context aggregation over locally connected feature maps. More specifically, DAG-RNN is placed on top of pre-trained CNN (feature extractor) to embed context into local features so that their representative capability can be enhanced. In comparison with plain CNN (as in Fully Convolutional Networks - FCN), DAG-RNN is empirically found to be significantly more effective at aggregating context. Therefore, DAG-RNN demonstrates noticeably performance superiority over FCNs on scene segmentation. Besides, DAG-RNN entails dramatically less parameters as well as demands fewer computation operations, which makes DAG-RNN more favorable to be potentially applied on resource-constrained embedded devices. Meanwhile, the class occurrence frequencies are extremely imbalanced in scene segmentation, so we propose a novel class-weighted loss to train the segmentation network. The loss distributes reasonably higher attention weights to infrequent classes during network training, which is essential to boost their parsing performance. We evaluate our segmentation network on three challenging public scene segmentation benchmarks: Sift Flow, Pascal Context and COCO Stuff. On top of them, we achieve very impressive segmentation performance.
Liu, Qingshan; Cao, Jinde
2010-06-01
Based on the projection operator, a recurrent neural network is proposed for solving extended general variational inequalities (EGVIs). Sufficient conditions are provided to ensure the global convergence of the proposed neural network based on Lyapunov methods. Compared with the existing neural networks for variational inequalities, the proposed neural network is a modified version of the general projection neural network existing in the literature and capable of solving the EGVI problems. In addition, simulation results on numerical examples show the effectiveness and performance of the proposed neural network.
Low-dimensional recurrent neural network-based Kalman filter for speech enhancement.
Xia, Youshen; Wang, Jun
2015-07-01
This paper proposes a new recurrent neural network-based Kalman filter for speech enhancement, based on a noise-constrained least squares estimate. The parameters of speech signal modeled as autoregressive process are first estimated by using the proposed recurrent neural network and the speech signal is then recovered from Kalman filtering. The proposed recurrent neural network is globally asymptomatically stable to the noise-constrained estimate. Because the noise-constrained estimate has a robust performance against non-Gaussian noise, the proposed recurrent neural network-based speech enhancement algorithm can minimize the estimation error of Kalman filter parameters in non-Gaussian noise. Furthermore, having a low-dimensional model feature, the proposed neural network-based speech enhancement algorithm has a much faster speed than two existing recurrent neural networks-based speech enhancement algorithms. Simulation results show that the proposed recurrent neural network-based speech enhancement algorithm can produce a good performance with fast computation and noise reduction. Copyright © 2015 Elsevier Ltd. All rights reserved.
Solving differential equations with unknown constitutive relations as recurrent neural networks
Energy Technology Data Exchange (ETDEWEB)
Hagge, Tobias J.; Stinis, Panagiotis; Yeung, Enoch H.; Tartakovsky, Alexandre M.
2017-12-08
We solve a system of ordinary differential equations with an unknown functional form of a sink (reaction rate) term. We assume that the measurements (time series) of state variables are partially available, and use a recurrent neural network to “learn” the reaction rate from this data. This is achieved by including discretized ordinary differential equations as part of a recurrent neural network training problem. We extend TensorFlow’s recurrent neural network architecture to create a simple but scalable and effective solver for the unknown functions, and apply it to a fedbatch bioreactor simulation problem. Use of techniques from recent deep learning literature enables training of functions with behavior manifesting over thousands of time steps. Our networks are structurally similar to recurrent neural networks, but differ in purpose, and require modified training strategies.
DEFF Research Database (Denmark)
Chon, K H; Hoyer, D; Armoundas, A A
1999-01-01
In this study, we introduce a new approach for estimating linear and nonlinear stochastic autoregressive moving average (ARMA) model parameters, given a corrupt signal, using artificial recurrent neural networks. This new approach is a two-step approach in which the parameters of the deterministic...... error is obtained by subtracting the corrupt signal of the estimated ARMA model obtained via the deterministic estimation step from the system output response. We present computer simulation examples to show the efficacy of the proposed stochastic recurrent neural network approach in obtaining accurate...... model predictions. Furthermore, we compare the performance of the new approach to that of the deterministic recurrent neural network approach. Using this simple two-step procedure, we obtain more robust model predictions than with the deterministic recurrent neural network approach despite the presence...
Zhao, Haiquan; Zeng, Xiangping; Zhang, Jiashu; Liu, Yangguang; Wang, Xiaomin; Li, Tianrui
2011-01-01
To eliminate nonlinear channel distortion in chaotic communication systems, a novel joint-processing adaptive nonlinear equalizer based on a pipelined recurrent neural network (JPRNN) is proposed, using a modified real-time recurrent learning (RTRL) algorithm. Furthermore, an adaptive amplitude RTRL algorithm is adopted to overcome the deteriorating effect introduced by the nesting process. Computer simulations illustrate that the proposed equalizer outperforms the pipelined recurrent neural network (PRNN) and recurrent neural network (RNN) equalizers. Copyright © 2010 Elsevier Ltd. All rights reserved.
Grinke, Eduard; Tetzlaff, Christian; Wörgötter, Florentin; Manoonpong, Poramate
2015-01-01
Walking animals, like insects, with little neural computing can effectively perform complex behaviors. For example, they can walk around their environment, escape from corners/deadlocks, and avoid or climb over obstacles. While performing all these behaviors, they can also adapt their movements to deal with an unknown situation. As a consequence, they successfully navigate through their complex environment. The versatile and adaptive abilities are the result of an integration of several ingredients embedded in their sensorimotor loop. Biological studies reveal that the ingredients include neural dynamics, plasticity, sensory feedback, and biomechanics. Generating such versatile and adaptive behaviors for a many degrees-of-freedom (DOFs) walking robot is a challenging task. Thus, in this study, we present a bio-inspired approach to solve this task. Specifically, the approach combines neural mechanisms with plasticity, exteroceptive sensory feedback, and biomechanics. The neural mechanisms consist of adaptive neural sensory processing and modular neural locomotion control. The sensory processing is based on a small recurrent neural network consisting of two fully connected neurons. Online correlation-based learning with synaptic scaling is applied to adequately change the connections of the network. By doing so, we can effectively exploit neural dynamics (i.e., hysteresis effects and single attractors) in the network to generate different turning angles with short-term memory for a walking robot. The turning information is transmitted as descending steering signals to the neural locomotion control which translates the signals into motor actions. As a result, the robot can walk around and adapt its turning angle for avoiding obstacles in different situations. The adaptation also enables the robot to effectively escape from sharp corners or deadlocks. Using backbone joint control embedded in the the locomotion control allows the robot to climb over small obstacles
International Nuclear Information System (INIS)
Liang Jinling; Cao Jinde
2003-01-01
Employing general Halanay inequality, we analyze the global exponential stability of a class of reaction-diffusion recurrent neural networks with time-varying delays. Several new sufficient conditions are obtained to ensure existence, uniqueness and global exponential stability of the equilibrium point of delayed reaction-diffusion recurrent neural networks. The results extend and improve the earlier publications. In addition, an example is given to show the effectiveness of the obtained result
Encoding sensory and motor patterns as time-invariant trajectories in recurrent neural networks.
Goudar, Vishwa; Buonomano, Dean V
2018-03-14
Much of the information the brain processes and stores is temporal in nature-a spoken word or a handwritten signature, for example, is defined by how it unfolds in time. However, it remains unclear how neural circuits encode complex time-varying patterns. We show that by tuning the weights of a recurrent neural network (RNN), it can recognize and then transcribe spoken digits. The model elucidates how neural dynamics in cortical networks may resolve three fundamental challenges: first, encode multiple time-varying sensory and motor patterns as stable neural trajectories; second, generalize across relevant spatial features; third, identify the same stimuli played at different speeds-we show that this temporal invariance emerges because the recurrent dynamics generate neural trajectories with appropriately modulated angular velocities. Together our results generate testable predictions as to how recurrent networks may use different mechanisms to generalize across the relevant spatial and temporal features of complex time-varying stimuli. © 2018, Goudar et al.
Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition.
Spoerer, Courtney J; McClure, Patrick; Kriegeskorte, Nikolaus
2017-01-01
Feedforward neural networks provide the dominant model of how the brain performs visual object recognition. However, these networks lack the lateral and feedback connections, and the resulting recurrent neuronal dynamics, of the ventral visual pathway in the human and non-human primate brain. Here we investigate recurrent convolutional neural networks with bottom-up (B), lateral (L), and top-down (T) connections. Combining these types of connections yields four architectures (B, BT, BL, and BLT), which we systematically test and compare. We hypothesized that recurrent dynamics might improve recognition performance in the challenging scenario of partial occlusion. We introduce two novel occluded object recognition tasks to test the efficacy of the models, digit clutter (where multiple target digits occlude one another) and digit debris (where target digits are occluded by digit fragments). We find that recurrent neural networks outperform feedforward control models (approximately matched in parametric complexity) at recognizing objects, both in the absence of occlusion and in all occlusion conditions. Recurrent networks were also found to be more robust to the inclusion of additive Gaussian noise. Recurrent neural networks are better in two respects: (1) they are more neurobiologically realistic than their feedforward counterparts; (2) they are better in terms of their ability to recognize objects, especially under challenging conditions. This work shows that computer vision can benefit from using recurrent convolutional architectures and suggests that the ubiquitous recurrent connections in biological brains are essential for task performance.
A One-Layer Recurrent Neural Network for Constrained Complex-Variable Convex Optimization.
Qin, Sitian; Feng, Jiqiang; Song, Jiahui; Wen, Xingnan; Xu, Chen
2018-03-01
In this paper, based on calculus and penalty method, a one-layer recurrent neural network is proposed for solving constrained complex-variable convex optimization. It is proved that for any initial point from a given domain, the state of the proposed neural network reaches the feasible region in finite time and converges to an optimal solution of the constrained complex-variable convex optimization finally. In contrast to existing neural networks for complex-variable convex optimization, the proposed neural network has a lower model complexity and better convergence. Some numerical examples and application are presented to substantiate the effectiveness of the proposed neural network.
Multi-step-prediction of chaotic time series based on co-evolutionary recurrent neural network
International Nuclear Information System (INIS)
Ma Qianli; Zheng Qilun; Peng Hong; Qin Jiangwei; Zhong Tanwei
2008-01-01
This paper proposes a co-evolutionary recurrent neural network (CERNN) for the multi-step-prediction of chaotic time series, it estimates the proper parameters of phase space reconstruction and optimizes the structure of recurrent neural networks by co-evolutionary strategy. The searching space was separated into two subspaces and the individuals are trained in a parallel computational procedure. It can dynamically combine the embedding method with the capability of recurrent neural network to incorporate past experience due to internal recurrence. The effectiveness of CERNN is evaluated by using three benchmark chaotic time series data sets: the Lorenz series, Mackey-Glass series and real-world sun spot series. The simulation results show that CERNN improves the performances of multi-step-prediction of chaotic time series
A statistical framework for evaluating neural networks to predict recurrent events in breast cancer
Gorunescu, Florin; Gorunescu, Marina; El-Darzi, Elia; Gorunescu, Smaranda
2010-07-01
Breast cancer is the second leading cause of cancer deaths in women today. Sometimes, breast cancer can return after primary treatment. A medical diagnosis of recurrent cancer is often a more challenging task than the initial one. In this paper, we investigate the potential contribution of neural networks (NNs) to support health professionals in diagnosing such events. The NN algorithms are tested and applied to two different datasets. An extensive statistical analysis has been performed to verify our experiments. The results show that a simple network structure for both the multi-layer perceptron and radial basis function can produce equally good results, not all attributes are needed to train these algorithms and, finally, the classification performances of all algorithms are statistically robust. Moreover, we have shown that the best performing algorithm will strongly depend on the features of the datasets, and hence, there is not necessarily a single best classifier.
An Attractor-Based Complexity Measurement for Boolean Recurrent Neural Networks
Cabessa, Jérémie; Villa, Alessandro E. P.
2014-01-01
We provide a novel refined attractor-based complexity measurement for Boolean recurrent neural networks that represents an assessment of their computational power in terms of the significance of their attractor dynamics. This complexity measurement is achieved by first proving a computational equivalence between Boolean recurrent neural networks and some specific class of -automata, and then translating the most refined classification of -automata to the Boolean neural network context. As a result, a hierarchical classification of Boolean neural networks based on their attractive dynamics is obtained, thus providing a novel refined attractor-based complexity measurement for Boolean recurrent neural networks. These results provide new theoretical insights to the computational and dynamical capabilities of neural networks according to their attractive potentialities. An application of our findings is illustrated by the analysis of the dynamics of a simplified model of the basal ganglia-thalamocortical network simulated by a Boolean recurrent neural network. This example shows the significance of measuring network complexity, and how our results bear new founding elements for the understanding of the complexity of real brain circuits. PMID:24727866
Entity recognition from clinical texts via recurrent neural network.
Liu, Zengjian; Yang, Ming; Wang, Xiaolong; Chen, Qingcai; Tang, Buzhou; Wang, Zhe; Xu, Hua
2017-07-05
Entity recognition is one of the most primary steps for text analysis and has long attracted considerable attention from researchers. In the clinical domain, various types of entities, such as clinical entities and protected health information (PHI), widely exist in clinical texts. Recognizing these entities has become a hot topic in clinical natural language processing (NLP), and a large number of traditional machine learning methods, such as support vector machine and conditional random field, have been deployed to recognize entities from clinical texts in the past few years. In recent years, recurrent neural network (RNN), one of deep learning methods that has shown great potential on many problems including named entity recognition, also has been gradually used for entity recognition from clinical texts. In this paper, we comprehensively investigate the performance of LSTM (long-short term memory), a representative variant of RNN, on clinical entity recognition and protected health information recognition. The LSTM model consists of three layers: input layer - generates representation of each word of a sentence; LSTM layer - outputs another word representation sequence that captures the context information of each word in this sentence; Inference layer - makes tagging decisions according to the output of LSTM layer, that is, outputting a label sequence. Experiments conducted on corpora of the 2010, 2012 and 2014 i2b2 NLP challenges show that LSTM achieves highest micro-average F1-scores of 85.81% on the 2010 i2b2 medical concept extraction, 92.29% on the 2012 i2b2 clinical event detection, and 94.37% on the 2014 i2b2 de-identification, which is considerably competitive with other state-of-the-art systems. LSTM that requires no hand-crafted feature has great potential on entity recognition from clinical texts. It outperforms traditional machine learning methods that suffer from fussy feature engineering. A possible future direction is how to integrate knowledge
The neural legacy of a single concussion.
Kraus, Nina; Lindley, Tory; Colegrove, Danielle; Krizman, Jennifer; Otto-Meyer, Sebastian; Thompson, Elaine C; White-Schwoch, Travis
2017-04-12
It has been hypothesized that concussions impart lasting brain damage, even after a patient has ostensibly recovered. This hypothesis is based largely upon neuropathological studies in deceased athletes, however, leaving open the question of whether it can be detected in vivo. We measured neural responses to speech in collegiate student-athletes with a history of a single concussion from which they had recovered. These student-athletes had weaker responses to speech than age- and position-matched peers. This group difference suggests that concussions engender small, but detectable, changes in brain function prior to the emergence of frank behavioral indications. Copyright © 2017 Elsevier B.V. All rights reserved.
Liu, Qingshan; Dang, Chuangyin; Huang, Tingwen
2013-02-01
This paper presents a decision-making model described by a recurrent neural network for dynamic portfolio optimization. The portfolio-optimization problem is first converted into a constrained fractional programming problem. Since the objective function in the programming problem is not convex, the traditional optimization techniques are no longer applicable for solving this problem. Fortunately, the objective function in the fractional programming is pseudoconvex on the feasible region. It leads to a one-layer recurrent neural network modeled by means of a discontinuous dynamic system. To ensure the optimal solutions for portfolio optimization, the convergence of the proposed neural network is analyzed and proved. In fact, the neural network guarantees to get the optimal solutions for portfolio-investment advice if some mild conditions are satisfied. A numerical example with simulation results substantiates the effectiveness and illustrates the characteristics of the proposed neural network.
Directory of Open Access Journals (Sweden)
Daniel Durstewitz
2017-06-01
Full Text Available The computational and cognitive properties of neural systems are often thought to be implemented in terms of their (stochastic network dynamics. Hence, recovering the system dynamics from experimentally observed neuronal time series, like multiple single-unit recordings or neuroimaging data, is an important step toward understanding its computations. Ideally, one would not only seek a (lower-dimensional state space representation of the dynamics, but would wish to have access to its statistical properties and their generative equations for in-depth analysis. Recurrent neural networks (RNNs are a computationally powerful and dynamically universal formal framework which has been extensively studied from both the computational and the dynamical systems perspective. Here we develop a semi-analytical maximum-likelihood estimation scheme for piecewise-linear RNNs (PLRNNs within the statistical framework of state space models, which accounts for noise in both the underlying latent dynamics and the observation process. The Expectation-Maximization algorithm is used to infer the latent state distribution, through a global Laplace approximation, and the PLRNN parameters iteratively. After validating the procedure on toy examples, and using inference through particle filters for comparison, the approach is applied to multiple single-unit recordings from the rodent anterior cingulate cortex (ACC obtained during performance of a classical working memory task, delayed alternation. Models estimated from kernel-smoothed spike time data were able to capture the essential computational dynamics underlying task performance, including stimulus-selective delay activity. The estimated models were rarely multi-stable, however, but rather were tuned to exhibit slow dynamics in the vicinity of a bifurcation point. In summary, the present work advances a semi-analytical (thus reasonably fast maximum-likelihood estimation framework for PLRNNs that may enable to recover
Durstewitz, Daniel
2017-06-01
The computational and cognitive properties of neural systems are often thought to be implemented in terms of their (stochastic) network dynamics. Hence, recovering the system dynamics from experimentally observed neuronal time series, like multiple single-unit recordings or neuroimaging data, is an important step toward understanding its computations. Ideally, one would not only seek a (lower-dimensional) state space representation of the dynamics, but would wish to have access to its statistical properties and their generative equations for in-depth analysis. Recurrent neural networks (RNNs) are a computationally powerful and dynamically universal formal framework which has been extensively studied from both the computational and the dynamical systems perspective. Here we develop a semi-analytical maximum-likelihood estimation scheme for piecewise-linear RNNs (PLRNNs) within the statistical framework of state space models, which accounts for noise in both the underlying latent dynamics and the observation process. The Expectation-Maximization algorithm is used to infer the latent state distribution, through a global Laplace approximation, and the PLRNN parameters iteratively. After validating the procedure on toy examples, and using inference through particle filters for comparison, the approach is applied to multiple single-unit recordings from the rodent anterior cingulate cortex (ACC) obtained during performance of a classical working memory task, delayed alternation. Models estimated from kernel-smoothed spike time data were able to capture the essential computational dynamics underlying task performance, including stimulus-selective delay activity. The estimated models were rarely multi-stable, however, but rather were tuned to exhibit slow dynamics in the vicinity of a bifurcation point. In summary, the present work advances a semi-analytical (thus reasonably fast) maximum-likelihood estimation framework for PLRNNs that may enable to recover relevant aspects
Predicting recurrent aphthous ulceration using genetic algorithms-optimized neural networks
Directory of Open Access Journals (Sweden)
Najla S Dar-Odeh
2010-05-01
Full Text Available Najla S Dar-Odeh1, Othman M Alsmadi2, Faris Bakri3, Zaer Abu-Hammour2, Asem A Shehabi3, Mahmoud K Al-Omiri1, Shatha M K Abu-Hammad4, Hamzeh Al-Mashni4, Mohammad B Saeed4, Wael Muqbil4, Osama A Abu-Hammad1 1Faculty of Dentistry, 2Faculty of Engineering and Technology, 3Faculty of Medicine, University of Jordan, Amman, Jordan; 4Dental Department, University of Jordan Hospital, Amman, JordanObjective: To construct and optimize a neural network that is capable of predicting the occurrence of recurrent aphthous ulceration (RAU based on a set of appropriate input data.Participants and methods: Artificial neural networks (ANN software employing genetic algorithms to optimize the architecture neural networks was used. Input and output data of 86 participants (predisposing factors and status of the participants with regards to recurrent aphthous ulceration were used to construct and train the neural networks. The optimized neural networks were then tested using untrained data of a further 10 participants.Results: The optimized neural network, which produced the most accurate predictions for the presence or absence of recurrent aphthous ulceration was found to employ: gender, hematological (with or without ferritin and mycological data of the participants, frequency of tooth brushing, and consumption of vegetables and fruits.Conclusions: Factors appearing to be related to recurrent aphthous ulceration and appropriate for use as input data to construct ANNs that predict recurrent aphthous ulceration were found to include the following: gender, hemoglobin, serum vitamin B12, serum ferritin, red cell folate, salivary candidal colony count, frequency of tooth brushing, and the number of fruits or vegetables consumed daily.Keywords: artifical neural networks, recurrent, aphthous ulceration, ulcer
Multistability and instability analysis of recurrent neural networks with time-varying delays.
Zhang, Fanghai; Zeng, Zhigang
2018-01-01
This paper provides new theoretical results on the multistability and instability analysis of recurrent neural networks with time-varying delays. It is shown that such n-neuronal recurrent neural networks have exactly [Formula: see text] equilibria, [Formula: see text] of which are locally exponentially stable and the others are unstable, where k 0 is a nonnegative integer such that k 0 ≤n. By using the combination method of two different divisions, recurrent neural networks can possess more dynamic properties. This method improves and extends the existing results in the literature. Finally, one numerical example is provided to show the superiority and effectiveness of the presented results. Copyright © 2017 Elsevier Ltd. All rights reserved.
Financial Time Series Prediction Using Elman Recurrent Random Neural Networks
Directory of Open Access Journals (Sweden)
Jie Wang
2016-01-01
(ERNN, the empirical results show that the proposed neural network displays the best performance among these neural networks in financial time series forecasting. Further, the empirical research is performed in testing the predictive effects of SSE, TWSE, KOSPI, and Nikkei225 with the established model, and the corresponding statistical comparisons of the above market indices are also exhibited. The experimental results show that this approach gives good performance in predicting the values from the stock market indices.
A novel nonlinear adaptive filter using a pipelined second-order Volterra recurrent neural network.
Zhao, Haiquan; Zhang, Jiashu
2009-12-01
To enhance the performance and overcome the heavy computational complexity of recurrent neural networks (RNN), a novel nonlinear adaptive filter based on a pipelined second-order Volterra recurrent neural network (PSOVRNN) is proposed in this paper. A modified real-time recurrent learning (RTRL) algorithm of the proposed filter is derived in much more detail. The PSOVRNN comprises of a number of simple small-scale second-order Volterra recurrent neural network (SOVRNN) modules. In contrast to the standard RNN, these modules of a PSOVRNN can be performed simultaneously in a pipelined parallelism fashion, which can lead to a significant improvement in its total computational efficiency. Moreover, since each module of the PSOVRNN is a SOVRNN in which nonlinearity is introduced by the recursive second-order Volterra (RSOV) expansion, its performance can be further improved. Computer simulations have demonstrated that the PSOVRNN performs better than the pipelined recurrent neural network (PRNN) and RNN for nonlinear colored signals prediction and nonlinear channel equalization. However, the superiority of the PSOVRNN over the PRNN is at the cost of increasing computational complexity due to the introduced nonlinear expansion of each module.
Lin, Yang-Yin; Chang, Jyh-Yeong; Lin, Chin-Teng
2013-02-01
This paper presents a novel recurrent fuzzy neural network, called an interactively recurrent self-evolving fuzzy neural network (IRSFNN), for prediction and identification of dynamic systems. The recurrent structure in an IRSFNN is formed as an external loops and internal feedback by feeding the rule firing strength of each rule to others rules and itself. The consequent part in the IRSFNN is composed of a Takagi-Sugeno-Kang (TSK) or functional-link-based type. The proposed IRSFNN employs a functional link neural network (FLNN) to the consequent part of fuzzy rules for promoting the mapping ability. Unlike a TSK-type fuzzy neural network, the FLNN in the consequent part is a nonlinear function of input variables. An IRSFNNs learning starts with an empty rule base and all of the rules are generated and learned online through a simultaneous structure and parameter learning. An on-line clustering algorithm is effective in generating fuzzy rules. The consequent update parameters are derived by a variable-dimensional Kalman filter algorithm. The premise and recurrent parameters are learned through a gradient descent algorithm. We test the IRSFNN for the prediction and identification of dynamic plants and compare it to other well-known recurrent FNNs. The proposed model obtains enhanced performance results.
DEFF Research Database (Denmark)
Chon, K H; Hoyer, D; Armoundas, A A
1999-01-01
In this study, we introduce a new approach for estimating linear and nonlinear stochastic autoregressive moving average (ARMA) model parameters, given a corrupt signal, using artificial recurrent neural networks. This new approach is a two-step approach in which the parameters of the deterministic...... part of the stochastic ARMA model are first estimated via a three-layer artificial neural network (deterministic estimation step) and then reestimated using the prediction error as one of the inputs to the artificial neural networks in an iterative algorithm (stochastic estimation step). The prediction...... of significant amounts of either dynamic or measurement noise in the output signal. The comparison between the deterministic and stochastic recurrent neural network approaches is furthered by applying both approaches to experimentally obtained renal blood pressure and flow signals....
International Nuclear Information System (INIS)
Liang Jinling; Cao Jinde
2003-01-01
In this Letter, the problems of boundedness and stability for a general class of non-autonomous recurrent neural networks with variable coefficients and time-varying delays are analyzed via employing Young inequality technique and Lyapunov method. Some simple sufficient conditions are given for boundedness and stability of the solutions for the recurrent neural networks. These results generalize and improve the previous works, and they are easy to check and apply in practice. Two illustrative examples and their numerical simulations are also given to demonstrate the effectiveness of the proposed results
ReSeg: A Recurrent Neural Network-based Model for Semantic Segmentation
Visin, Francesco; Ciccone, Marco; Romero, Adriana; Kastner, Kyle; Cho, Kyunghyun; Bengio, Yoshua; Matteucci, Matteo; Courville, Aaron
2015-01-01
We propose a structured prediction architecture, which exploits the local generic features extracted by Convolutional Neural Networks and the capacity of Recurrent Neural Networks (RNN) to retrieve distant dependencies. The proposed architecture, called ReSeg, is based on the recently introduced ReNet model for image classification. We modify and extend it to perform the more challenging task of semantic segmentation. Each ReNet layer is composed of four RNN that sweep the image horizontally ...
Training the Recurrent neural network by the Fuzzy Min-Max algorithm for fault prediction
International Nuclear Information System (INIS)
Zemouri, Ryad; Racoceanu, Daniel; Zerhouni, Noureddine; Minca, Eugenia; Filip, Florin
2009-01-01
In this paper, we present a training technique of a Recurrent Radial Basis Function neural network for fault prediction. We use the Fuzzy Min-Max technique to initialize the k-center of the RRBF neural network. The k-means algorithm is then applied to calculate the centers that minimize the mean square error of the prediction task. The performances of the k-means algorithm are then boosted by the Fuzzy Min-Max technique.
International Nuclear Information System (INIS)
Cui Baotong; Lou Xuyang
2009-01-01
In this paper, a new method to synchronize two identical chaotic recurrent neural networks is proposed. Using the drive-response concept, a nonlinear feedback control law is derived to achieve the state synchronization of the two identical chaotic neural networks. Furthermore, based on the Lyapunov method, a delay independent sufficient synchronization condition in terms of linear matrix inequality (LMI) is obtained. A numerical example with graphical illustrations is given to illuminate the presented synchronization scheme
International Nuclear Information System (INIS)
Lou, X.; Cui, B.
2008-01-01
In this paper we consider the problem of exponential stability for recurrent neural networks with multiple time varying delays and reaction-diffusion terms. The activation functions are supposed to be bounded and globally Lipschitz continuous. By means of Lyapunov functional, sufficient conditions are derived, which guarantee global exponential stability of the delayed neural network. Finally, a numerical example is given to show the correctness of our analysis. (author)
Reduced-Order Modeling for Flutter/LCO Using Recurrent Artificial Neural Network
Yao, Weigang; Liou, Meng-Sing
2012-01-01
The present study demonstrates the efficacy of a recurrent artificial neural network to provide a high fidelity time-dependent nonlinear reduced-order model (ROM) for flutter/limit-cycle oscillation (LCO) modeling. An artificial neural network is a relatively straightforward nonlinear method for modeling an input-output relationship from a set of known data, for which we use the radial basis function (RBF) with its parameters determined through a training process. The resulting RBF neural network, however, is only static and is not yet adequate for an application to problems of dynamic nature. The recurrent neural network method [1] is applied to construct a reduced order model resulting from a series of high-fidelity time-dependent data of aero-elastic simulations. Once the RBF neural network ROM is constructed properly, an accurate approximate solution can be obtained at a fraction of the cost of a full-order computation. The method derived during the study has been validated for predicting nonlinear aerodynamic forces in transonic flow and is capable of accurate flutter/LCO simulations. The obtained results indicate that the present recurrent RBF neural network is accurate and efficient for nonlinear aero-elastic system analysis
Liu, Qingshan; Wang, Jun
2011-04-01
This paper presents a one-layer recurrent neural network for solving a class of constrained nonsmooth optimization problems with piecewise-linear objective functions. The proposed neural network is guaranteed to be globally convergent in finite time to the optimal solutions under a mild condition on a derived lower bound of a single gain parameter in the model. The number of neurons in the neural network is the same as the number of decision variables of the optimization problem. Compared with existing neural networks for optimization, the proposed neural network has a couple of salient features such as finite-time convergence and a low model complexity. Specific models for two important special cases, namely, linear programming and nonsmooth optimization, are also presented. In addition, applications to the shortest path problem and constrained least absolute deviation problem are discussed with simulation results to demonstrate the effectiveness and characteristics of the proposed neural network.
Stimulus-dependent suppression of chaos in recurrent neural networks
International Nuclear Information System (INIS)
Rajan, Kanaka; Abbott, L. F.; Sompolinsky, Haim
2010-01-01
Neuronal activity arises from an interaction between ongoing firing generated spontaneously by neural circuits and responses driven by external stimuli. Using mean-field analysis, we ask how a neural network that intrinsically generates chaotic patterns of activity can remain sensitive to extrinsic input. We find that inputs not only drive network responses, but they also actively suppress ongoing activity, ultimately leading to a phase transition in which chaos is completely eliminated. The critical input intensity at the phase transition is a nonmonotonic function of stimulus frequency, revealing a 'resonant' frequency at which the input is most effective at suppressing chaos even though the power spectrum of the spontaneous activity peaks at zero and falls exponentially. A prediction of our analysis is that the variance of neural responses should be most strongly suppressed at frequencies matching the range over which many sensory systems operate.
Homeostatic scaling of excitability in recurrent neural networks.
Remme, M.W.H.; Wadman, W.J.
2012-01-01
Neurons adjust their intrinsic excitability when experiencing a persistent change in synaptic drive. This process can prevent neural activity from moving into either a quiescent state or a saturated state in the face of ongoing plasticity, and is thought to promote stability of the network in which
Recurrent Artificial Neural Networks and Finite State Natural Language Processing.
Moisl, Hermann
It is argued that pessimistic assessments of the adequacy of artificial neural networks (ANNs) for natural language processing (NLP) on the grounds that they have a finite state architecture are unjustified, and that their adequacy in this regard is an empirical issue. First, arguments that counter standard objections to finite state NLP on the…
Individual Identification Using Functional Brain Fingerprint Detected by Recurrent Neural Network.
Chen, Shiyang; Hu, Xiaoping P
2018-03-20
Individual identification based on brain function has gained traction in literature. Investigating individual differences in brain function can provide additional insights into the brain. In this work, we introduce a recurrent neural network based model for identifying individuals based on only a short segment of resting state functional MRI data. In addition, we demonstrate how the global signal and differences in atlases affect the individual identifiability. Furthermore, we investigate neural network features that exhibit the uniqueness of each individual. The results indicate that our model is able to identify individuals based on neural features and provides additional information regarding brain dynamics.
Xia, Youshen; Feng, Gang; Wang, Jun
2004-09-01
This paper presents a recurrent neural network for solving strict convex quadratic programming problems and related linear piecewise equations. Compared with the existing neural networks for quadratic program, the proposed neural network has a one-layer structure with a low model complexity. Moreover, the proposed neural network is shown to have a finite-time convergence and exponential convergence. Illustrative examples further show the good performance of the proposed neural network in real-time applications.
Qin, Sitian; Yang, Xiudong; Xue, Xiaoping; Song, Jiahui
2017-10-01
Pseudoconvex optimization problem, as an important nonconvex optimization problem, plays an important role in scientific and engineering applications. In this paper, a recurrent one-layer neural network is proposed for solving the pseudoconvex optimization problem with equality and inequality constraints. It is proved that from any initial state, the state of the proposed neural network reaches the feasible region in finite time and stays there thereafter. It is also proved that the state of the proposed neural network is convergent to an optimal solution of the related problem. Compared with the related existing recurrent neural networks for the pseudoconvex optimization problems, the proposed neural network in this paper does not need the penalty parameters and has a better convergence. Meanwhile, the proposed neural network is used to solve three nonsmooth optimization problems, and we make some detailed comparisons with the known related conclusions. In the end, some numerical examples are provided to illustrate the effectiveness of the performance of the proposed neural network.
International Nuclear Information System (INIS)
Hannen, Jennifer C; Buckner, Gregory D; Crews, John H
2012-01-01
This paper introduces an indirect intelligent sliding mode controller (IISMC) for shape memory alloy (SMA) actuators, specifically a flexible beam deflected by a single offset SMA tendon. The controller manipulates applied voltage, which alters SMA tendon temperature to track reference bending angles. A hysteretic recurrent neural network (HRNN) captures the nonlinear, hysteretic relationship between SMA temperature and bending angle. The variable structure control strategy provides robustness to model uncertainties and parameter variations, while effectively compensating for system nonlinearities, achieving superior tracking compared to an optimized PI controller. (paper)
Global stability of discrete-time recurrent neural networks with impulse effects
International Nuclear Information System (INIS)
Zhou, L; Li, C; Wan, J
2008-01-01
This paper formulates and studies a class of discrete-time recurrent neural networks with impulse effects. A stability criterion, which characterizes the effects of impulse and stability property of the corresponding impulse-free networks on the stability of the impulsive networks in an aggregate form, is established. Two simplified and numerically tractable criteria are also provided
Recurrent Neural Network For Forecasting Time Series With Long Memory Pattern
Walid; Alamsyah
2017-04-01
Recurrent Neural Network as one of the hybrid models are often used to predict and estimate the issues related to electricity, can be used to describe the cause of the swelling of electrical load which experienced by PLN. In this research will be developed RNN forecasting procedures at the time series with long memory patterns. Considering the application is the national electrical load which of course has a different trend with the condition of the electrical load in any country. This research produces the algorithm of time series forecasting which has long memory pattern using E-RNN after this referred to the algorithm of integrated fractional recurrent neural networks (FIRNN).The prediction results of long memory time series using models Fractional Integrated Recurrent Neural Network (FIRNN) showed that the model with the selection of data difference in the range of [-1,1] and the model of Fractional Integrated Recurrent Neural Network (FIRNN) (24,6,1) provides the smallest MSE value, which is 0.00149684.
Encoding of phonology in a recurrent neural model of grounded speech
Alishahi, Afra; Barking, Marie; Chrupala, Grzegorz; Levy, Roger; Specia, Lucia
2017-01-01
We study the representation and encoding of phonemes in a recurrent neural network model of grounded speech. We use a model which processes images and their spoken descriptions, and projects the visual and auditory representations into the same semantic space. We perform a number of analyses on how
Direction-of-change forecasting using a volatility-based recurrent neural network
Bekiros, S.D.; Georgoutsos, D.A.
2008-01-01
This paper investigates the profitability of a trading strategy, based on recurrent neural networks, that attempts to predict the direction-of-change of the market in the case of the NASDAQ composite index. The sample extends over the period 8 February 1971 to 7 April 1998, while the sub-period 8
Anti-periodic solutions for recurrent neural networks without assuming global Lipschitz conditions
Directory of Open Access Journals (Sweden)
Hong Zhang
2010-04-01
Full Text Available In this paper we study recurrent neural networks with time-varying delays and continuously distributed delays. Without assuming global Lipschitz conditions on the activation functions, we establish the existence and local exponential stability of anti-periodic solutions.
A one-layer recurrent neural network for constrained nonconvex optimization.
Li, Guocheng; Yan, Zheng; Wang, Jun
2015-01-01
In this paper, a one-layer recurrent neural network is proposed for solving nonconvex optimization problems subject to general inequality constraints, designed based on an exact penalty function method. It is proved herein that any neuron state of the proposed neural network is convergent to the feasible region in finite time and stays there thereafter, provided that the penalty parameter is sufficiently large. The lower bounds of the penalty parameter and convergence time are also estimated. In addition, any neural state of the proposed neural network is convergent to its equilibrium point set which satisfies the Karush-Kuhn-Tucker conditions of the optimization problem. Moreover, the equilibrium point set is equivalent to the optimal solution to the nonconvex optimization problem if the objective function and constraints satisfy given conditions. Four numerical examples are provided to illustrate the performances of the proposed neural network.
A one-layer recurrent neural network for constrained nonsmooth invex optimization.
Li, Guocheng; Yan, Zheng; Wang, Jun
2014-02-01
Invexity is an important notion in nonconvex optimization. In this paper, a one-layer recurrent neural network is proposed for solving constrained nonsmooth invex optimization problems, designed based on an exact penalty function method. It is proved herein that any state of the proposed neural network is globally convergent to the optimal solution set of constrained invex optimization problems, with a sufficiently large penalty parameter. In addition, any neural state is globally convergent to the unique optimal solution, provided that the objective function and constraint functions are pseudoconvex. Moreover, any neural state is globally convergent to the feasible region in finite time and stays there thereafter. The lower bounds of the penalty parameter and convergence time are also estimated. Two numerical examples are provided to illustrate the performances of the proposed neural network. Copyright © 2013 Elsevier Ltd. All rights reserved.
Statistical downscaling of precipitation using long short-term memory recurrent neural networks
Misra, Saptarshi; Sarkar, Sudeshna; Mitra, Pabitra
2017-11-01
Hydrological impacts of global climate change on regional scale are generally assessed by downscaling large-scale climatic variables, simulated by General Circulation Models (GCMs), to regional, small-scale hydrometeorological variables like precipitation, temperature, etc. In this study, we propose a new statistical downscaling model based on Recurrent Neural Network with Long Short-Term Memory which captures the spatio-temporal dependencies in local rainfall. The previous studies have used several other methods such as linear regression, quantile regression, kernel regression, beta regression, and artificial neural networks. Deep neural networks and recurrent neural networks have been shown to be highly promising in modeling complex and highly non-linear relationships between input and output variables in different domains and hence we investigated their performance in the task of statistical downscaling. We have tested this model on two datasets—one on precipitation in Mahanadi basin in India and the second on precipitation in Campbell River basin in Canada. Our autoencoder coupled long short-term memory recurrent neural network model performs the best compared to other existing methods on both the datasets with respect to temporal cross-correlation, mean squared error, and capturing the extremes.
Spatiotemporal Dynamics and Reliable Computations in Recurrent Spiking Neural Networks
Pyle, Ryan; Rosenbaum, Robert
2017-01-01
Randomly connected networks of excitatory and inhibitory spiking neurons provide a parsimonious model of neural variability, but are notoriously unreliable for performing computations. We show that this difficulty is overcome by incorporating the well-documented dependence of connection probability on distance. Spatially extended spiking networks exhibit symmetry-breaking bifurcations and generate spatiotemporal patterns that can be trained to perform dynamical computations under a reservoir computing framework.
Hysteretic recurrent neural networks: a tool for modeling hysteretic materials and systems
International Nuclear Information System (INIS)
Veeramani, Arun S; Crews, John H; Buckner, Gregory D
2009-01-01
This paper introduces a novel recurrent neural network, the hysteretic recurrent neural network (HRNN), that is ideally suited to modeling hysteretic materials and systems. This network incorporates a hysteretic neuron consisting of conjoined sigmoid activation functions. Although similar hysteretic neurons have been explored previously, the HRNN is unique in its utilization of simple recurrence to 'self-select' relevant activation functions. Furthermore, training is facilitated by placing the network weights on the output side, allowing standard backpropagation of error training algorithms to be used. We present two- and three-phase versions of the HRNN for modeling hysteretic materials with distinct phases. These models are experimentally validated using data collected from shape memory alloys and ferromagnetic materials. The results demonstrate the HRNN's ability to accurately generalize hysteretic behavior with a relatively small number of neurons. Additional benefits lie in the network's ability to identify statistical information concerning the macroscopic material by analyzing the weights of the individual neurons
Directory of Open Access Journals (Sweden)
Xing Yin
2011-01-01
uncertain periodic switched recurrent neural networks with time-varying delays. When uncertain discrete-time recurrent neural network is a periodic system, it is expressed as switched neural network for the finite switching state. Based on the switched quadratic Lyapunov functional approach (SQLF and free-weighting matrix approach (FWM, some linear matrix inequality criteria are found to guarantee the delay-dependent asymptotical stability of these systems. Two examples illustrate the exactness of the proposed criteria.
Using a multi-state recurrent neural network to optimize loading patterns in BWRs
International Nuclear Information System (INIS)
Ortiz, Juan Jose; Requena, Ignacio
2004-01-01
A Multi-State Recurrent Neural Network is used to optimize Loading Patterns (LP) in BWRs. We have proposed an energy function that depends on fuel assembly positions and their nuclear cross sections to carry out optimisation. Multi-State Recurrent Neural Networks creates LPs that satisfy the Radial Power Peaking Factor and maximize the effective multiplication factor at the Beginning of the Cycle, and also satisfy the Minimum Critical Power Ratio and Maximum Linear Heat Generation Rate at the End of the Cycle, thereby maximizing the effective multiplication factor. In order to evaluate the LPs, we have used a trained back-propagation neural network to predict the parameter values, instead of using a reactor core simulator, which saved considerable computation time in the search process. We applied this method to find optimal LPs for five cycles of Laguna Verde Nuclear Power Plant (LVNPP) in Mexico
Directory of Open Access Journals (Sweden)
Mattia Rigotti
2010-10-01
Full Text Available Neural activity of behaving animals, especially in the prefrontal cortex, is highly heterogeneous, with selective responses to diverse aspects of the executed task. We propose a general model of recurrent neural networks that perform complex rule-based tasks, and we show that the diversity of neuronal responses plays a fundamental role when the behavioral responses are context dependent. Specifically, we found that when the inner mental states encoding the task rules are represented by stable patterns of neural activity (attractors of the neural dynamics, the neurons must be selective for combinations of sensory stimuli and inner mental states. Such mixed selectivity is easily obtained by neurons that connect with random synaptic strengths both to the recurrent network and to neurons encoding sensory inputs. The number of randomly connected neurons needed to solve a task is on average only three times as large as the number of neurons needed in a network designed ad hoc. Moreover, the number of needed neurons grows only linearly with the number of task-relevant events and mental states, provided that each neuron responds to a large proportion of events (dense/distributed coding. A biologically realistic implementation of the model captures several aspects of the activity recorded from monkeys performing context dependent tasks. Our findings explain the importance of the diversity of neural responses and provide us with simple and general principles for designing attractor neural networks that perform complex computation.
Firing rate dynamics in recurrent spiking neural networks with intrinsic and network heterogeneity.
Ly, Cheng
2015-12-01
Heterogeneity of neural attributes has recently gained a lot of attention and is increasing recognized as a crucial feature in neural processing. Despite its importance, this physiological feature has traditionally been neglected in theoretical studies of cortical neural networks. Thus, there is still a lot unknown about the consequences of cellular and circuit heterogeneity in spiking neural networks. In particular, combining network or synaptic heterogeneity and intrinsic heterogeneity has yet to be considered systematically despite the fact that both are known to exist and likely have significant roles in neural network dynamics. In a canonical recurrent spiking neural network model, we study how these two forms of heterogeneity lead to different distributions of excitatory firing rates. To analytically characterize how these types of heterogeneities affect the network, we employ a dimension reduction method that relies on a combination of Monte Carlo simulations and probability density function equations. We find that the relationship between intrinsic and network heterogeneity has a strong effect on the overall level of heterogeneity of the firing rates. Specifically, this relationship can lead to amplification or attenuation of firing rate heterogeneity, and these effects depend on whether the recurrent network is firing asynchronously or rhythmically firing. These observations are captured with the aforementioned reduction method, and furthermore simpler analytic descriptions based on this dimension reduction method are developed. The final analytic descriptions provide compact and descriptive formulas for how the relationship between intrinsic and network heterogeneity determines the firing rate heterogeneity dynamics in various settings.
A two-layer recurrent neural network for nonsmooth convex optimization problems.
Qin, Sitian; Xue, Xiaoping
2015-06-01
In this paper, a two-layer recurrent neural network is proposed to solve the nonsmooth convex optimization problem subject to convex inequality and linear equality constraints. Compared with existing neural network models, the proposed neural network has a low model complexity and avoids penalty parameters. It is proved that from any initial point, the state of the proposed neural network reaches the equality feasible region in finite time and stays there thereafter. Moreover, the state is unique if the initial point lies in the equality feasible region. The equilibrium point set of the proposed neural network is proved to be equivalent to the Karush-Kuhn-Tucker optimality set of the original optimization problem. It is further proved that the equilibrium point of the proposed neural network is stable in the sense of Lyapunov. Moreover, from any initial point, the state is proved to be convergent to an equilibrium point of the proposed neural network. Finally, as applications, the proposed neural network is used to solve nonlinear convex programming with linear constraints and L1 -norm minimization problems.
Hierarchical singleton-type recurrent neural fuzzy networks for noisy speech recognition.
Juang, Chia-Feng; Chiou, Chyi-Tian; Lai, Chun-Lung
2007-05-01
This paper proposes noisy speech recognition using hierarchical singleton-type recurrent neural fuzzy networks (HSRNFNs). The proposed HSRNFN is a hierarchical connection of two singleton-type recurrent neural fuzzy networks (SRNFNs), where one is used for noise filtering and the other for recognition. The SRNFN is constructed by recurrent fuzzy if-then rules with fuzzy singletons in the consequences, and their recurrent properties make them suitable for processing speech patterns with temporal characteristics. In n words recognition, n SRNFNs are created for modeling n words, where each SRNFN receives the current frame feature and predicts the next one of its modeling word. The prediction error of each SRNFN is used as recognition criterion. In filtering, one SRNFN is created, and each SRNFN recognizer is connected to the same SRNFN filter, which filters noisy speech patterns in the feature domain before feeding them to the SRNFN recognizer. Experiments with Mandarin word recognition under different types of noise are performed. Other recognizers, including multilayer perceptron (MLP), time-delay neural networks (TDNNs), and hidden Markov models (HMMs), are also tested and compared. These experiments and comparisons demonstrate good results with HSRNFN for noisy speech recognition tasks.
Simultaneous multichannel signal transfers via chaos in a recurrent neural network.
Soma, Ken-ichiro; Mori, Ryota; Sato, Ryuichi; Furumai, Noriyuki; Nara, Shigetoshi
2015-05-01
We propose neural network model that demonstrates the phenomenon of signal transfer between separated neuron groups via other chaotic neurons that show no apparent correlations with the input signal. The model is a recurrent neural network in which it is supposed that synchronous behavior between small groups of input and output neurons has been learned as fragments of high-dimensional memory patterns, and depletion of neural connections results in chaotic wandering dynamics. Computer experiments show that when a strong oscillatory signal is applied to an input group in the chaotic regime, the signal is successfully transferred to the corresponding output group, although no correlation is observed between the input signal and the intermediary neurons. Signal transfer is also observed when multiple signals are applied simultaneously to separate input groups belonging to different memory attractors. In this sense simultaneous multichannel communications are realized, and the chaotic neural dynamics acts as a signal transfer medium in which the signal appears to be hidden.
A non-penalty recurrent neural network for solving a class of constrained optimization problems.
Hosseini, Alireza
2016-01-01
In this paper, we explain a methodology to analyze convergence of some differential inclusion-based neural networks for solving nonsmooth optimization problems. For a general differential inclusion, we show that if its right hand-side set valued map satisfies some conditions, then solution trajectory of the differential inclusion converges to optimal solution set of its corresponding in optimization problem. Based on the obtained methodology, we introduce a new recurrent neural network for solving nonsmooth optimization problems. Objective function does not need to be convex on R(n) nor does the new neural network model require any penalty parameter. We compare our new method with some penalty-based and non-penalty based models. Moreover for differentiable cases, we implement circuit diagram of the new neural network. Copyright © 2015 Elsevier Ltd. All rights reserved.
A Novel Recurrent Neural Network for Manipulator Control With Improved Noise Tolerance.
Li, Shuai; Wang, Huanqing; Rafique, Muhammad Usman
2017-04-12
In this paper, we propose a novel recurrent neural network to resolve the redundancy of manipulators for efficient kinematic control in the presence of noises in a polynomial type. Leveraging the high-order derivative properties of polynomial noises, a deliberately devised neural network is proposed to eliminate the impact of noises and recover the accurate tracking of desired trajectories in workspace. Rigorous analysis shows that the proposed neural law stabilizes the system dynamics and the position tracking error converges to zero in the presence of noises. Extensive simulations verify the theoretical results. Numerical comparisons show that existing dual neural solutions lose stability when exposed to large constant noises or time-varying noises. In contrast, the proposed approach works well and has a low tracking error comparable to noise-free situations.
Multi-stability and almost periodic solutions of a class of recurrent neural networks
International Nuclear Information System (INIS)
Liu Yiguang; You Zhisheng
2007-01-01
This paper studies multi-stability, existence of almost periodic solutions of a class of recurrent neural networks with bounded activation functions. After introducing a sufficient condition insuring multi-stability, many criteria guaranteeing existence of almost periodic solutions are derived using Mawhin's coincidence degree theory. All the criteria are constructed without assuming the activation functions are smooth, monotonic or Lipschitz continuous, and that the networks contains periodic variables (such as periodic coefficients, periodic inputs or periodic activation functions), so all criteria can be easily extended to fit many concrete forms of neural networks such as Hopfield neural networks, or cellular neural networks, etc. Finally, all kinds of simulations are employed to illustrate the criteria
Intelligent Noise Removal from EMG Signal Using Focused Time-Lagged Recurrent Neural Network
Directory of Open Access Journals (Sweden)
S. N. Kale
2009-01-01
Full Text Available Electromyography (EMG signals can be used for clinical/biomedical application and modern human computer interaction. EMG signals acquire noise while traveling through tissue, inherent noise in electronics equipment, ambient noise, and so forth. ANN approach is studied for reduction of noise in EMG signal. In this paper, it is shown that Focused Time-Lagged Recurrent Neural Network (FTLRNN can elegantly solve to reduce the noise from EMG signal. After rigorous computer simulations, authors developed an optimal FTLRNN model, which removes the noise from the EMG signal. Results show that the proposed optimal FTLRNN model has an MSE (Mean Square Error as low as 0.000067 and 0.000048, correlation coefficient as high as 0.99950 and 0.99939 for noise signal and EMG signal, respectively, when validated on the test dataset. It is also noticed that the output of the estimated FTLRNN model closely follows the real one. This network is indeed robust as EMG signal tolerates the noise variance from 0.1 to 0.4 for uniform noise and 0.30 for Gaussian noise. It is clear that the training of the network is independent of specific partitioning of dataset. It is seen that the performance of the proposed FTLRNN model clearly outperforms the best Multilayer perceptron (MLP and Radial Basis Function NN (RBF models. The simple NN model such as the FTLRNN with single-hidden layer can be employed to remove noise from EMG signal.
Fei, Juntao; Lu, Cheng
2018-04-01
In this paper, an adaptive sliding mode control system using a double loop recurrent neural network (DLRNN) structure is proposed for a class of nonlinear dynamic systems. A new three-layer RNN is proposed to approximate unknown dynamics with two different kinds of feedback loops where the firing weights and output signal calculated in the last step are stored and used as the feedback signals in each feedback loop. Since the new structure has combined the advantages of internal feedback NN and external feedback NN, it can acquire the internal state information while the output signal is also captured, thus the new designed DLRNN can achieve better approximation performance compared with the regular NNs without feedback loops or the regular RNNs with a single feedback loop. The new proposed DLRNN structure is employed in an equivalent controller to approximate the unknown nonlinear system dynamics, and the parameters of the DLRNN are updated online by adaptive laws to get favorable approximation performance. To investigate the effectiveness of the proposed controller, the designed adaptive sliding mode controller with the DLRNN is applied to a -axis microelectromechanical system gyroscope to control the vibrating dynamics of the proof mass. Simulation results demonstrate that the proposed methodology can achieve good tracking property, and the comparisons of the approximation performance between radial basis function NN, RNN, and DLRNN show that the DLRNN can accurately estimate the unknown dynamics with a fast speed while the internal states of DLRNN are more stable.
Luitel, Bipul; Venayagamoorthy, Ganesh Kumar
2010-06-01
Training a single simultaneous recurrent neural network (SRN) to learn all outputs of a multiple-input-multiple-output (MIMO) system is a difficult problem. A new training algorithm developed from combined concepts of swarm intelligence and quantum principles is presented. The training algorithm is called particle swarm optimization with quantum infusion (PSO-QI). To improve the effectiveness of learning, a two-step learning approach is introduced in the training. The objective of the learning in the first step is to find the optimal set of weights in the SRN considering all output errors. In the second step, the objective is to maximize the learning of each output dynamics by fine tuning the respective SRN output weights. To demonstrate the effectiveness of the PSO-QI training algorithm and the two-step learning approach, two examples of an SRN learning MIMO systems are presented. The first example is learning a benchmark MIMO system and the second one is the design of a wide area monitoring system for a multimachine power system. From the results, it is observed that SRNs can effectively learn MIMO systems when trained using the PSO-QI algorithm and the two-step learning approach. Copyright 2009 Elsevier Ltd. All rights reserved.
Juang, Chia-Feng; Yeh, Yen-Ting
2017-06-30
This paper proposes the optimization of a fully connected recurrent neural network (FCRNN) using advanced multiobjective continuous ant colony optimization (AMO-CACO) for the multiobjective gait generation of a biped robot (the NAO). The FCRNN functions as a central pattern generator and is optimized to generate angles of the hip roll and pitch, the knee pitch, and the ankle pitch and roll. The performance of the FCRNN-generated gait is evaluated according to the walking speed, trajectory straightness, oscillations of the body in the pitch and yaw directions, and walking posture, subject to the basic constraints that the robot cannot fall down and must walk forward. This paper formulates this gait generation task as a constrained multiobjective optimization problem and solves this problem through an AMO-CACO-based evolutionary learning approach. The AMO-CACO finds Pareto optimal solutions through ant-path selection and sampling operations by introducing an accumulated rank for the solutions in each single-objective function into solution sorting to improve learning performance. Simulations are conducted to verify the AMO-CACO-based FCRNN gait generation performance through comparisons with different multiobjective optimization algorithms. Selected software-designed Pareto optimal FCRNNs are then applied to control the gait of a real NAO robot.
Bengoetxea, Ana; Leurs, Françoise; Hoellinger, Thomas; Cebolla, Ana M; Dan, Bernard; McIntyre, Joseph; Cheron, Guy
2014-01-01
In this study we employed a dynamic recurrent neural network (DRNN) in a novel fashion to reveal characteristics of control modules underlying the generation of muscle activations when drawing figures with the outstretched arm. We asked healthy human subjects to perform four different figure-eight movements in each of two workspaces (frontal plane and sagittal plane). We then trained a DRNN to predict the movement of the wrist from information in the EMG signals from seven different muscles. We trained different instances of the same network on a single movement direction, on all four movement directions in a single movement plane, or on all eight possible movement patterns and looked at the ability of the DRNN to generalize and predict movements for trials that were not included in the training set. Within a single movement plane, a DRNN trained on one movement direction was not able to predict movements of the hand for trials in the other three directions, but a DRNN trained simultaneously on all four movement directions could generalize across movement directions within the same plane. Similarly, the DRNN was able to reproduce the kinematics of the hand for both movement planes, but only if it was trained on examples performed in each one. As we will discuss, these results indicate that there are important dynamical constraints on the mapping of EMG to hand movement that depend on both the time sequence of the movement and on the anatomical constraints of the musculoskeletal system. In a second step, we injected EMG signals constructed from different synergies derived by the PCA in order to identify the mechanical significance of each of these components. From these results, one can surmise that discrete-rhythmic movements may be constructed from three different fundamental modules, one regulating the co-activation of all muscles over the time span of the movement and two others elliciting patterns of reciprocal activation operating in orthogonal directions.
Directory of Open Access Journals (Sweden)
Ana eBengoetxea
2014-09-01
Full Text Available In this study we employed a dynamic recurrent neural network (DRNN in a novel fashion to reveal characteristics of control modules underlying the generation of muscle activations when drawing figures with the outstretched arm. We asked healthy human subjects to perform four different figure-eight movements in each of two workspaces (frontal plane and sagittal plane. We then trained a DRNN to predict the movement of the wrist from information in the EMG signals from seven different muscles. We trained different instances of the same network on a single movement direction, on all four movement directions in a single movement plane, or on all eight possible movement patterns and looked at the ability of the DRNN to generalize and predict movements for trials that were not included in the training set. Within a single movement plane, a DRNN trained on one movement direction was not able to predict movements of the hand for trials in the other three directions, but a DRNN trained simultaneously on all four movement directions could generalize across movement directions within the same plane. Similarly, the DRNN was able to reproduce the kinematics of the hand for both movement planes, but only if it was trained on examples performed in each one. As we will discuss, these results indicate that there are important dynamical constraints on the mapping of EMG to hand movement that depend on both the time sequence of the movement and on the anatomical constraints of the musculoskeletal system. In a second step, we injected EMG signals constructed from different synergies derived by the PCA in order to identify the mechanical significance of each of these components. From these results, one can surmise that discrete-rhythmic movements may be constructed from three different fundamental modules, one regulating the co-activation of all muscles over the time span of the movement and two others patterns of reciprocal activation operating in orthogonal
Dynamic recurrent neural networks for stable adaptive control of wing rock motion
Kooi, Steven Boon-Lam
Wing rock is a self-sustaining limit cycle oscillation (LCO) which occurs as the result of nonlinear coupling between the dynamic response of the aircraft and the unsteady aerodynamic forces. In this thesis, dynamic recurrent RBF (Radial Basis Function) network control methodology is proposed to control the wing rock motion. The concept based on the properties of the Presiach hysteresis model is used in the design of dynamic neural networks. The structure and memory mechanism in the Preisach model is analogous to the parallel connectivity and memory formation in the RBF neural networks. The proposed dynamic recurrent neural network has a feature for adding or pruning the neurons in the hidden layer according to the growth criteria based on the properties of ensemble average memory formation of the Preisach model. The recurrent feature of the RBF network deals with the dynamic nonlinearities and endowed temporal memories of the hysteresis model. The control of wing rock is a tracking problem, the trajectory starts from non-zero initial conditions and it tends to zero as time goes to infinity. In the proposed neural control structure, the recurrent dynamic RBF network performs identification process in order to approximate the unknown non-linearities of the physical system based on the input-output data obtained from the wing rock phenomenon. The design of the RBF networks together with the network controllers are carried out in discrete time domain. The recurrent RBF networks employ two separate adaptation schemes where the RBF's centre and width are adjusted by the Extended Kalman Filter in order to give a minimum networks size, while the outer networks layer weights are updated using the algorithm derived from Lyapunov stability analysis for the stable closed loop control. The issue of the robustness of the recurrent RBF networks is also addressed. The effectiveness of the proposed dynamic recurrent neural control methodology is demonstrated through simulations to
International Nuclear Information System (INIS)
Huang Yu-Jiao; Hu Hai-Gen
2015-01-01
In this paper, the multistability issue is discussed for delayed complex-valued recurrent neural networks with discontinuous real-imaginary-type activation functions. Based on a fixed theorem and stability definition, sufficient criteria are established for the existence and stability of multiple equilibria of complex-valued recurrent neural networks. The number of stable equilibria is larger than that of real-valued recurrent neural networks, which can be used to achieve high-capacity associative memories. One numerical example is provided to show the effectiveness and superiority of the presented results. (paper)
Chen, Guiling; Li, Dingshi; Shi, Lin; van Gaans, Onno; Verduyn Lunel, Sjoerd
2018-03-01
We present new conditions for asymptotic stability and exponential stability of a class of stochastic recurrent neural networks with discrete and distributed time varying delays. Our approach is based on the method using fixed point theory, which do not resort to any Liapunov function or Liapunov functional. Our results neither require the boundedness, monotonicity and differentiability of the activation functions nor differentiability of the time varying delays. In particular, a class of neural networks without stochastic perturbations is also considered. Examples are given to illustrate our main results.
International Nuclear Information System (INIS)
Wang Linshan; Zhang Zhe; Wang Yangfan
2008-01-01
Some criteria for the global stochastic exponential stability of the delayed reaction-diffusion recurrent neural networks with Markovian jumping parameters are presented. The jumping parameters considered here are generated from a continuous-time discrete-state homogeneous Markov process, which are governed by a Markov process with discrete and finite state space. By employing a new Lyapunov-Krasovskii functional, a linear matrix inequality (LMI) approach is developed to establish some easy-to-test criteria of global exponential stability in the mean square for the stochastic neural networks. The criteria are computationally efficient, since they are in the forms of some linear matrix inequalities
Recurrent neural networks for NO{sub x} prediction in fossil plants
Energy Technology Data Exchange (ETDEWEB)
Reifman, J.; Vitela, J.E.; Feldman, E.E.; Wei, T.Y.C.
1996-04-01
The authors discuss the application of recurrent (dynamic) neural networks for time-dependent modeling of NO{sub x} emissions in coal-fired fossil plants. They use plant data from one of ComEd`s plants to train and test the network model. Additional tests, parametric studies, and sensitivity analyses are performed to determine if the dynamic behavior of the model matches the expected behavior of the physical system. The results are also compared with feedforward (static) neural network models trained to represent temporal information.
Design and analysis of a novel chaotic diagonal recurrent neural network
Wang, Libiao; Meng, Zhuo; Sun, Yize; Guo, Lei; Zhou, Mingxing
2015-09-01
A chaotic neural network model with logistic mapping is proposed to improve the performance of the conventional diagonal recurrent neural network. The network shows rich dynamic behaviors that contribute to escaping from a local minimum to reach the global minimum easily. Then, a simple parameter modulated chaos controller is adopted to enhance convergence speed of the network. Furthermore, an adaptive learning algorithm with the robust adaptive dead zone vector is designed to improve the generalization performance of the network, and weights convergence for the network with the adaptive dead zone vectors is proved in the sense of Lyapunov functions. Finally, the numerical simulation is carried out to demonstrate the correctness of the theory.
Observer design for switched recurrent neural networks: an average dwell time approach.
Lian, Jie; Feng, Zhi; Shi, Peng
2011-10-01
This paper is concerned with the problem of observer design for switched recurrent neural networks with time-varying delay. The attention is focused on designing the full-order observers that guarantee the global exponential stability of the error dynamic system. Based on the average dwell time approach and the free-weighting matrix technique, delay-dependent sufficient conditions are developed for the solvability of such problem and formulated as linear matrix inequalities. The error-state decay estimate is also given. Then, the stability analysis problem for the switched recurrent neural networks can be covered as a special case of our results. Finally, four illustrative examples are provided to demonstrate the effectiveness and the superiority of the proposed methods. © 2011 IEEE
DeepNano: Deep recurrent neural networks for base calling in MinION nanopore reads.
Directory of Open Access Journals (Sweden)
Vladimír Boža
Full Text Available The MinION device by Oxford Nanopore produces very long reads (reads over 100 kBp were reported; however it suffers from high sequencing error rate. We present an open-source DNA base caller based on deep recurrent neural networks and show that the accuracy of base calling is much dependent on the underlying software and can be improved by considering modern machine learning methods. By employing carefully crafted recurrent neural networks, our tool significantly improves base calling accuracy on data from R7.3 version of the platform compared to the default base caller supplied by the manufacturer. On R9 version, we achieve results comparable to Nanonet base caller provided by Oxford Nanopore. Availability of an open source tool with high base calling accuracy will be useful for development of new applications of the MinION device, including infectious disease detection and custom target enrichment during sequencing.
International Nuclear Information System (INIS)
Han, Seong Ik; Jeong, Chan Se; Yang, Soon Yong
2012-01-01
A robust positioning control scheme has been developed using friction parameter observer and recurrent fuzzy neural networks based on the sliding mode control. As a dynamic friction model, the LuGre model is adopted for handling friction compensation because it has been known to capture sufficiently the properties of a nonlinear dynamic friction. A developed friction parameter observer has a simple structure and also well estimates friction parameters of the LuGre friction model. In addition, an approximation method for the system uncertainty is developed using recurrent fuzzy neural networks technology to improve the precision positioning degree. Some simulation and experiment provide the verification on the performance of a proposed robust control scheme
Stability Analysis of Recurrent Neural Networks with Random Delay and Markovian Switching
Directory of Open Access Journals (Sweden)
Enwen Zhu
2010-01-01
Full Text Available In this paper, the exponential stability analysis problem is considered for a class of recurrent neural networks (RNNs with random delay and Markovian switching. The evolution of the delay is modeled by a continuous-time homogeneous Markov process with a finite number of states. The main purpose of this paper is to establish easily verifiable conditions under which the random delayed recurrent neural network with Markovian switching is exponentially stable. The analysis is based on the Lyapunov-Krasovskii functional and stochastic analysis approach, and the conditions are expressed in terms of linear matrix inequalities, which can be readily checked by using some standard numerical packages such as the Matlab LMI Toolbox. A numerical example is exploited to show the usefulness of the derived LMI-based stability conditions.
End-to-End Tracking and Semantic Segmentation Using Recurrent Neural Networks
Ondruska, Peter; Dequaire, Julie; Wang, Dominic Zeng; Posner, Ingmar
2016-01-01
In this work we present a novel end-to-end framework for tracking and classifying a robot's surroundings in complex, dynamic and only partially observable real-world environments. The approach deploys a recurrent neural network to filter an input stream of raw laser measurements in order to directly infer object locations, along with their identity in both visible and occluded areas. To achieve this we first train the network using unsupervised Deep Tracking, a recently proposed theoretical f...
International Nuclear Information System (INIS)
Cadini, F.; Zio, E.; Pedroni, N.
2007-01-01
In this paper, a locally recurrent neural network (LRNN) is employed for approximating the temporal evolution of a nonlinear dynamic system model of a simplified nuclear reactor. To this aim, an infinite impulse response multi-layer perceptron (IIR-MLP) is trained according to a recursive back-propagation (RBP) algorithm. The network nodes contain internal feedback paths and their connections are realized by means of IIR synaptic filters, which provide the LRNN with the necessary system state memory
Some new results for recurrent neural networks with varying-time coefficients and delays
International Nuclear Information System (INIS)
Jiang Haijun; Teng Zhidong
2005-01-01
In this Letter, we consider the recurrent neural networks with varying-time coefficients and delays. By constructing new Lyapunov functional, introducing ingeniously many real parameters and applying the technique of Young inequality, we establish a series of criteria on the boundedness, global exponential stability and the existence of periodic solutions. In these criteria, we do not require that the response functions are differentiable, bounded and monotone nondecreasing. Some previous works are improved and extended
Complex Dynamical Network Control for Trajectory Tracking Using Delayed Recurrent Neural Networks
Directory of Open Access Journals (Sweden)
Jose P. Perez
2014-01-01
Full Text Available In this paper, the problem of trajectory tracking is studied. Based on the V-stability and Lyapunov theory, a control law that achieves the global asymptotic stability of the tracking error between a delayed recurrent neural network and a complex dynamical network is obtained. To illustrate the analytic results, we present a tracking simulation of a dynamical network with each node being just one Lorenz’s dynamical system and three identical Chen’s dynamical systems.
Online Sequence Training of Recurrent Neural Networks with Connectionist Temporal Classification
Hwang, Kyuyeon; Sung, Wonyong
2015-01-01
Connectionist temporal classification (CTC) based supervised sequence training of recurrent neural networks (RNNs) has shown great success in many machine learning areas including end-to-end speech and handwritten character recognition. For the CTC training, however, it is required to unroll (or unfold) the RNN by the length of an input sequence. This unrolling requires a lot of memory and hinders a small footprint implementation of online learning or adaptation. Furthermore, the length of tr...
Model for a flexible motor memory based on a self-active recurrent neural network.
Boström, Kim Joris; Wagner, Heiko; Prieske, Markus; de Lussanet, Marc
2013-10-01
Using recent recurrent network architecture based on the reservoir computing approach, we propose and numerically simulate a model that is focused on the aspects of a flexible motor memory for the storage of elementary movement patterns into the synaptic weights of a neural network, so that the patterns can be retrieved at any time by simple static commands. The resulting motor memory is flexible in that it is capable to continuously modulate the stored patterns. The modulation consists in an approximately linear inter- and extrapolation, generating a large space of possible movements that have not been learned before. A recurrent network of thousand neurons is trained in a manner that corresponds to a realistic exercising scenario, with experimentally measured muscular activations and with kinetic data representing proprioceptive feedback. The network is "self-active" in that it maintains recurrent flow of activation even in the absence of input, a feature that resembles the "resting-state activity" found in the human and animal brain. The model involves the concept of "neural outsourcing" which amounts to the permanent shifting of computational load from higher to lower-level neural structures, which might help to explain why humans are able to execute learned skills in a fluent and flexible manner without the need for attention to the details of the movement. Copyright © 2013 Elsevier B.V. All rights reserved.
Kumar, Rajesh; Srivastava, Smriti; Gupta, J R P
2017-03-01
In this paper adaptive control of nonlinear dynamical systems using diagonal recurrent neural network (DRNN) is proposed. The structure of DRNN is a modification of fully connected recurrent neural network (FCRNN). Presence of self-recurrent neurons in the hidden layer of DRNN gives it an ability to capture the dynamic behaviour of the nonlinear plant under consideration (to be controlled). To ensure stability, update rules are developed using lyapunov stability criterion. These rules are then used for adjusting the various parameters of DRNN. The responses of plants obtained with DRNN are compared with those obtained when multi-layer feed forward neural network (MLFFNN) is used as a controller. Also, in example 4, FCRNN is also investigated and compared with DRNN and MLFFNN. Robustness of the proposed control scheme is also tested against parameter variations and disturbance signals. Four simulation examples including one-link robotic manipulator and inverted pendulum are considered on which the proposed controller is applied. The results so obtained show the superiority of DRNN over MLFFNN as a controller. Copyright © 2017 ISA. Published by Elsevier Ltd. All rights reserved.
International Nuclear Information System (INIS)
Hajihosseini, Amirhossein; Maleki, Farzaneh; Rokni Lamooki, Gholam Reza
2011-01-01
Highlights: → We construct a recurrent neural network by generalizing a specific n-neuron network. → Several codimension 1 and 2 bifurcations take place in the newly constructed network. → The newly constructed network has higher capabilities to learn periodic signals. → The normal form theorem is applied to investigate dynamics of the network. → A series of bifurcation diagrams is given to support theoretical results. - Abstract: A class of recurrent neural networks is constructed by generalizing a specific class of n-neuron networks. It is shown that the newly constructed network experiences generic pitchfork and Hopf codimension one bifurcations. It is also proved that the emergence of generic Bogdanov-Takens, pitchfork-Hopf and Hopf-Hopf codimension two, and the degenerate Bogdanov-Takens bifurcation points in the parameter space is possible due to the intersections of codimension one bifurcation curves. The occurrence of bifurcations of higher codimensions significantly increases the capability of the newly constructed recurrent neural network to learn broader families of periodic signals.
Budzinski, R. C.; Boaretto, B. R. R.; Prado, T. L.; Lopes, S. R.
2017-07-01
We study the stability of asymptotic states displayed by a complex neural network. We focus on the loss of stability of a stationary state of networks using recurrence quantifiers as tools to diagnose local and global stabilities as well as the multistability of a coupled neural network. Numerical simulations of a neural network composed of 1024 neurons in a small-world connection scheme are performed using the model of Braun et al. [Int. J. Bifurcation Chaos 08, 881 (1998), 10.1142/S0218127498000681], which is a modified model from the Hodgkin-Huxley model [J. Phys. 117, 500 (1952)]. To validate the analyses, the results are compared with those produced by Kuramoto's order parameter [Chemical Oscillations, Waves, and Turbulence (Springer-Verlag, Berlin Heidelberg, 1984)]. We show that recurrence tools making use of just integrated signals provided by the networks, such as local field potential (LFP) (LFP signals) or mean field values bring new results on the understanding of neural behavior occurring before the synchronization states. In particular we show the occurrence of different stationary and nonstationarity asymptotic states.
Interpretation of correlated neural variability from models of feed-forward and recurrent circuits
2018-01-01
Neural populations respond to the repeated presentations of a sensory stimulus with correlated variability. These correlations have been studied in detail, with respect to their mechanistic origin, as well as their influence on stimulus discrimination and on the performance of population codes. A number of theoretical studies have endeavored to link network architecture to the nature of the correlations in neural activity. Here, we contribute to this effort: in models of circuits of stochastic neurons, we elucidate the implications of various network architectures—recurrent connections, shared feed-forward projections, and shared gain fluctuations—on the stimulus dependence in correlations. Specifically, we derive mathematical relations that specify the dependence of population-averaged covariances on firing rates, for different network architectures. In turn, these relations can be used to analyze data on population activity. We examine recordings from neural populations in mouse auditory cortex. We find that a recurrent network model with random effective connections captures the observed statistics. Furthermore, using our circuit model, we investigate the relation between network parameters, correlations, and how well different stimuli can be discriminated from one another based on the population activity. As such, our approach allows us to relate properties of the neural circuit to information processing. PMID:29408930
International Nuclear Information System (INIS)
Cao Jiacong; Lin Xingchun
2008-01-01
An accurate forecast of solar irradiation is required for various solar energy applications and environmental impact analyses in recent years. Comparatively, various irradiation forecast models based on artificial neural networks (ANN) perform much better in accuracy than many conventional prediction models. However, the forecast precision of most existing ANN based forecast models has not been satisfactory to researchers and engineers so far, and the generalization capability of these networks needs further improving. Combining the prominent dynamic properties of a recurrent neural network (RNN) with the enhanced ability of a wavelet neural network (WNN) in mapping nonlinear functions, a diagonal recurrent wavelet neural network (DRWNN) is newly established in this paper to perform fine forecasting of hourly and daily global solar irradiance. Some additional steps, e.g. applying historical information of cloud cover to sample data sets and the cloud cover from the weather forecast to network input, are adopted to help enhance the forecast precision. Besides, a specially scheduled two phase training algorithm is adopted. As examples, both hourly and daily irradiance forecasts are completed using sample data sets in Shanghai and Macau, and comparisons between irradiation models show that the DRWNN models are definitely more accurate
Reward-based training of recurrent neural networks for cognitive and value-based tasks.
Song, H Francis; Yang, Guangyu R; Wang, Xiao-Jing
2017-01-13
Trained neural network models, which exhibit features of neural activity recorded from behaving animals, may provide insights into the circuit mechanisms of cognitive functions through systematic analysis of network activity and connectivity. However, in contrast to the graded error signals commonly used to train networks through supervised learning, animals learn from reward feedback on definite actions through reinforcement learning. Reward maximization is particularly relevant when optimal behavior depends on an animal's internal judgment of confidence or subjective preferences. Here, we implement reward-based training of recurrent neural networks in which a value network guides learning by using the activity of the decision network to predict future reward. We show that such models capture behavioral and electrophysiological findings from well-known experimental paradigms. Our work provides a unified framework for investigating diverse cognitive and value-based computations, and predicts a role for value representation that is essential for learning, but not executing, a task.
Robust recurrent neural network modeling for software fault detection and correction prediction
International Nuclear Information System (INIS)
Hu, Q.P.; Xie, M.; Ng, S.H.; Levitin, G.
2007-01-01
Software fault detection and correction processes are related although different, and they should be studied together. A practical approach is to apply software reliability growth models to model fault detection, and fault correction process is assumed to be a delayed process. On the other hand, the artificial neural networks model, as a data-driven approach, tries to model these two processes together with no assumptions. Specifically, feedforward backpropagation networks have shown their advantages over analytical models in fault number predictions. In this paper, the following approach is explored. First, recurrent neural networks are applied to model these two processes together. Within this framework, a systematic networks configuration approach is developed with genetic algorithm according to the prediction performance. In order to provide robust predictions, an extra factor characterizing the dispersion of prediction repetitions is incorporated into the performance function. Comparisons with feedforward neural networks and analytical models are developed with respect to a real data set
Predicting five-year recurrence rates of kidney stones: an artificial neural network model.
Caudarella, Renata; Tonello, Lucio; Rizzoli, Elisabetta; Vescini, Fabio
2011-03-01
Due to high recurrence rates of urolithiasis, many attempts have been performed to identify tools for predicting the risk of stone formation. The application of Artificial Neural Networks (ANNs) seems to be a valid candidate for reaching this endpoint. The aim of this study was to find a set of parameters able to predict recurrence episodes immediately after clinical and metabolic evaluation performed at the first visit in a 5-year window. Data were collected from 80 outpatients who presented idiopathic calcium stone disease both at baseline and after 5 years; patients underwent treatment including both general measures and medical therapy. After 5 years, patients were classified into two subsets, namely SSFs (without recurrence episodes), consisting of 45 subjects (56.25%) and RSFs, with at least one episode of recurrence after the baseline, consisting of 35 subjects (43.75%). Helped by conventional statistics (One-way ANOVA and three Discriminant Analyses: standard, backward stepwise and forward stepwise), an Artificial Neural Network (ANN) approach was used to predict recurrence episodes. An optimal set of 6 parameters was identified from amongst the different combinations in order to efficiently predict the outcome of stone recurrence in approximately 90% of cases. This set consist of serum Na and K as well as Na, P, Oxalate and AP (CaP) index from urine. The results obtained with ANN seem to suggest that some kind of relationship is present between the identified parameters and future stone recurrence. This relationship is probably very complex (in the mathematical sense) and non-linear In fact, a Logistic Regression was built as a comparative method and performed less good results at least in terms of accuracy and sensitivity. The application of ANN to the database led to a promising predicting algorithm and suggests that a strongly non-linear relationship seems to exist between the parameters and the recurrence episodes. In particular, the ANN approach
Single-site neural tube closure in human embryos revisited.
de Bakker, Bernadette S; Driessen, Stan; Boukens, Bastiaan J D; van den Hoff, Maurice J B; Oostra, Roelof-Jan
2017-10-01
Since the multi-site closure theory was first proposed in 1991 as explanation for the preferential localizations of neural tube defects, the closure of the neural tube has been debated. Although the multi-site closure theory is much cited in clinical literature, single-site closure is most apparent in literature concerning embryology. Inspired by Victor Hamburgers (1900-2001) statement that "our real teacher has been and still is the embryo, who is, incidentally, the only teacher who is always right", we decided to critically review both theories of neural tube closure. To verify the theories of closure, we studied serial histological sections of 10 mouse embryos between 8.5 and 9.5 days of gestation and 18 human embryos of the Carnegie collection between Carnegie stage 9 (19-21 days) and 13 (28-32 days). Neural tube closure was histologically defined by the neuroepithelial remodeling of the two adjoining neural fold tips in the midline. We did not observe multiple fusion sites in neither mouse nor human embryos. A meta-analysis of case reports on neural tube defects showed that defects can occur at any level of the neural axis. Our data indicate that the human neural tube fuses at a single site and, therefore, we propose to reinstate the single-site closure theory for neural tube closure. We showed that neural tube defects are not restricted to a specific location, thereby refuting the reasoning underlying the multi-site closure theory. Clin. Anat. 30:988-999, 2017. © 2017 Wiley Periodicals, Inc. © 2017 Wiley Periodicals, Inc.
International Nuclear Information System (INIS)
Liu, Xiaolan; Zhou, Mi
2016-01-01
In this paper, a one-layer recurrent network is proposed for solving a non-smooth convex optimization subject to linear inequality constraints. Compared with the existing neural networks for optimization, the proposed neural network is capable of solving more general convex optimization with linear inequality constraints. The convergence of the state variables of the proposed neural network to achieve solution optimality is guaranteed as long as the designed parameters in the model are larger than the derived lower bounds.
Deep Recurrent Neural Network-Based Autoencoders for Acoustic Novelty Detection
Directory of Open Access Journals (Sweden)
Erik Marchi
2017-01-01
Full Text Available In the emerging field of acoustic novelty detection, most research efforts are devoted to probabilistic approaches such as mixture models or state-space models. Only recent studies introduced (pseudo-generative models for acoustic novelty detection with recurrent neural networks in the form of an autoencoder. In these approaches, auditory spectral features of the next short term frame are predicted from the previous frames by means of Long-Short Term Memory recurrent denoising autoencoders. The reconstruction error between the input and the output of the autoencoder is used as activation signal to detect novel events. There is no evidence of studies focused on comparing previous efforts to automatically recognize novel events from audio signals and giving a broad and in depth evaluation of recurrent neural network-based autoencoders. The present contribution aims to consistently evaluate our recent novel approaches to fill this white spot in the literature and provide insight by extensive evaluations carried out on three databases: A3Novelty, PASCAL CHiME, and PROMETHEUS. Besides providing an extensive analysis of novel and state-of-the-art methods, the article shows how RNN-based autoencoders outperform statistical approaches up to an absolute improvement of 16.4% average F-measure over the three databases.
Deep Recurrent Neural Network-Based Autoencoders for Acoustic Novelty Detection.
Marchi, Erik; Vesperini, Fabio; Squartini, Stefano; Schuller, Björn
2017-01-01
In the emerging field of acoustic novelty detection, most research efforts are devoted to probabilistic approaches such as mixture models or state-space models. Only recent studies introduced (pseudo-)generative models for acoustic novelty detection with recurrent neural networks in the form of an autoencoder. In these approaches, auditory spectral features of the next short term frame are predicted from the previous frames by means of Long-Short Term Memory recurrent denoising autoencoders. The reconstruction error between the input and the output of the autoencoder is used as activation signal to detect novel events. There is no evidence of studies focused on comparing previous efforts to automatically recognize novel events from audio signals and giving a broad and in depth evaluation of recurrent neural network-based autoencoders. The present contribution aims to consistently evaluate our recent novel approaches to fill this white spot in the literature and provide insight by extensive evaluations carried out on three databases: A3Novelty, PASCAL CHiME, and PROMETHEUS. Besides providing an extensive analysis of novel and state-of-the-art methods, the article shows how RNN-based autoencoders outperform statistical approaches up to an absolute improvement of 16.4% average F -measure over the three databases.
Brain Dynamics in Predicting Driving Fatigue Using a Recurrent Self-Evolving Fuzzy Neural Network.
Liu, Yu-Ting; Lin, Yang-Yin; Wu, Shang-Lin; Chuang, Chun-Hsiang; Lin, Chin-Teng
2016-02-01
This paper proposes a generalized prediction system called a recurrent self-evolving fuzzy neural network (RSEFNN) that employs an on-line gradient descent learning rule to address the electroencephalography (EEG) regression problem in brain dynamics for driving fatigue. The cognitive states of drivers significantly affect driving safety; in particular, fatigue driving, or drowsy driving, endangers both the individual and the public. For this reason, the development of brain-computer interfaces (BCIs) that can identify drowsy driving states is a crucial and urgent topic of study. Many EEG-based BCIs have been developed as artificial auxiliary systems for use in various practical applications because of the benefits of measuring EEG signals. In the literature, the efficacy of EEG-based BCIs in recognition tasks has been limited by low resolutions. The system proposed in this paper represents the first attempt to use the recurrent fuzzy neural network (RFNN) architecture to increase adaptability in realistic EEG applications to overcome this bottleneck. This paper further analyzes brain dynamics in a simulated car driving task in a virtual-reality environment. The proposed RSEFNN model is evaluated using the generalized cross-subject approach, and the results indicate that the RSEFNN is superior to competing models regardless of the use of recurrent or nonrecurrent structures.
Directory of Open Access Journals (Sweden)
Renzhi Cao
2017-10-01
Full Text Available With the development of next generation sequencing techniques, it is fast and cheap to determine protein sequences but relatively slow and expensive to extract useful information from protein sequences because of limitations of traditional biological experimental techniques. Protein function prediction has been a long standing challenge to fill the gap between the huge amount of protein sequences and the known function. In this paper, we propose a novel method to convert the protein function problem into a language translation problem by the new proposed protein sequence language “ProLan” to the protein function language “GOLan”, and build a neural machine translation model based on recurrent neural networks to translate “ProLan” language to “GOLan” language. We blindly tested our method by attending the latest third Critical Assessment of Function Annotation (CAFA 3 in 2016, and also evaluate the performance of our methods on selected proteins whose function was released after CAFA competition. The good performance on the training and testing datasets demonstrates that our new proposed method is a promising direction for protein function prediction. In summary, we first time propose a method which converts the protein function prediction problem to a language translation problem and applies a neural machine translation model for protein function prediction.
Cao, Renzhi; Freitas, Colton; Chan, Leong; Sun, Miao; Jiang, Haiqing; Chen, Zhangxin
2017-10-17
With the development of next generation sequencing techniques, it is fast and cheap to determine protein sequences but relatively slow and expensive to extract useful information from protein sequences because of limitations of traditional biological experimental techniques. Protein function prediction has been a long standing challenge to fill the gap between the huge amount of protein sequences and the known function. In this paper, we propose a novel method to convert the protein function problem into a language translation problem by the new proposed protein sequence language "ProLan" to the protein function language "GOLan", and build a neural machine translation model based on recurrent neural networks to translate "ProLan" language to "GOLan" language. We blindly tested our method by attending the latest third Critical Assessment of Function Annotation (CAFA 3) in 2016, and also evaluate the performance of our methods on selected proteins whose function was released after CAFA competition. The good performance on the training and testing datasets demonstrates that our new proposed method is a promising direction for protein function prediction. In summary, we first time propose a method which converts the protein function prediction problem to a language translation problem and applies a neural machine translation model for protein function prediction.
Computing single step operators of logic programming in radial basis function neural networks
Energy Technology Data Exchange (ETDEWEB)
Hamadneh, Nawaf; Sathasivam, Saratha; Choon, Ong Hong [School of Mathematical Sciences, Universiti Sains Malaysia, 11800 USM, Penang (Malaysia)
2014-07-10
Logic programming is the process that leads from an original formulation of a computing problem to executable programs. A normal logic program consists of a finite set of clauses. A valuation I of logic programming is a mapping from ground atoms to false or true. The single step operator of any logic programming is defined as a function (T{sub p}:I→I). Logic programming is well-suited to building the artificial intelligence systems. In this study, we established a new technique to compute the single step operators of logic programming in the radial basis function neural networks. To do that, we proposed a new technique to generate the training data sets of single step operators. The training data sets are used to build the neural networks. We used the recurrent radial basis function neural networks to get to the steady state (the fixed point of the operators). To improve the performance of the neural networks, we used the particle swarm optimization algorithm to train the networks.
Computing single step operators of logic programming in radial basis function neural networks
Hamadneh, Nawaf; Sathasivam, Saratha; Choon, Ong Hong
2014-07-01
Logic programming is the process that leads from an original formulation of a computing problem to executable programs. A normal logic program consists of a finite set of clauses. A valuation I of logic programming is a mapping from ground atoms to false or true. The single step operator of any logic programming is defined as a function (Tp:I→I). Logic programming is well-suited to building the artificial intelligence systems. In this study, we established a new technique to compute the single step operators of logic programming in the radial basis function neural networks. To do that, we proposed a new technique to generate the training data sets of single step operators. The training data sets are used to build the neural networks. We used the recurrent radial basis function neural networks to get to the steady state (the fixed point of the operators). To improve the performance of the neural networks, we used the particle swarm optimization algorithm to train the networks.
Computing single step operators of logic programming in radial basis function neural networks
International Nuclear Information System (INIS)
Hamadneh, Nawaf; Sathasivam, Saratha; Choon, Ong Hong
2014-01-01
Logic programming is the process that leads from an original formulation of a computing problem to executable programs. A normal logic program consists of a finite set of clauses. A valuation I of logic programming is a mapping from ground atoms to false or true. The single step operator of any logic programming is defined as a function (T p :I→I). Logic programming is well-suited to building the artificial intelligence systems. In this study, we established a new technique to compute the single step operators of logic programming in the radial basis function neural networks. To do that, we proposed a new technique to generate the training data sets of single step operators. The training data sets are used to build the neural networks. We used the recurrent radial basis function neural networks to get to the steady state (the fixed point of the operators). To improve the performance of the neural networks, we used the particle swarm optimization algorithm to train the networks
Iterative prediction of chaotic time series using a recurrent neural network
Energy Technology Data Exchange (ETDEWEB)
Essawy, M.A.; Bodruzzaman, M. [Tennessee State Univ., Nashville, TN (United States). Dept. of Electrical and Computer Engineering; Shamsi, A.; Noel, S. [USDOE Morgantown Energy Technology Center, WV (United States)
1996-12-31
Chaotic systems are known for their unpredictability due to their sensitive dependence on initial conditions. When only time series measurements from such systems are available, neural network based models are preferred due to their simplicity, availability, and robustness. However, the type of neutral network used should be capable of modeling the highly non-linear behavior and the multi-attractor nature of such systems. In this paper the authors use a special type of recurrent neural network called the ``Dynamic System Imitator (DSI)``, that has been proven to be capable of modeling very complex dynamic behaviors. The DSI is a fully recurrent neural network that is specially designed to model a wide variety of dynamic systems. The prediction method presented in this paper is based upon predicting one step ahead in the time series, and using that predicted value to iteratively predict the following steps. This method was applied to chaotic time series generated from the logistic, Henon, and the cubic equations, in addition to experimental pressure drop time series measured from a Fluidized Bed Reactor (FBR), which is known to exhibit chaotic behavior. The time behavior and state space attractor of the actual and network synthetic chaotic time series were analyzed and compared. The correlation dimension and the Kolmogorov entropy for both the original and network synthetic data were computed. They were found to resemble each other, confirming the success of the DSI based chaotic system modeling.
Hanson, Jack; Yang, Yuedong; Paliwal, Kuldip; Zhou, Yaoqi
2017-03-01
Capturing long-range interactions between structural but not sequence neighbors of proteins is a long-standing challenging problem in bioinformatics. Recently, long short-term memory (LSTM) networks have significantly improved the accuracy of speech and image classification problems by remembering useful past information in long sequential events. Here, we have implemented deep bidirectional LSTM recurrent neural networks in the problem of protein intrinsic disorder prediction. The new method, named SPOT-Disorder, has steadily improved over a similar method using a traditional, window-based neural network (SPINE-D) in all datasets tested without separate training on short and long disordered regions. Independent tests on four other datasets including the datasets from critical assessment of structure prediction (CASP) techniques and >10 000 annotated proteins from MobiDB, confirmed SPOT-Disorder as one of the best methods in disorder prediction. Moreover, initial studies indicate that the method is more accurate in predicting functional sites in disordered regions. These results highlight the usefulness combining LSTM with deep bidirectional recurrent neural networks in capturing non-local, long-range interactions for bioinformatics applications. SPOT-disorder is available as a web server and as a standalone program at: http://sparks-lab.org/server/SPOT-disorder/index.php . j.hanson@griffith.edu.au or yuedong.yang@griffith.edu.au or yaoqi.zhou@griffith.edu.au. Supplementary data is available at Bioinformatics online.
Hou, Runmin; Wang, Li; Gao, Qiang; Hou, Yuanglong; Wang, Chao
2017-09-01
This paper proposes a novel indirect adaptive fuzzy wavelet neural network (IAFWNN) to control the nonlinearity, wide variations in loads, time-variation and uncertain disturbance of the ac servo system. In the proposed approach, the self-recurrent wavelet neural network (SRWNN) is employed to construct an adaptive self-recurrent consequent part for each fuzzy rule of TSK fuzzy model. For the IAFWNN controller, the online learning algorithm is based on back propagation (BP) algorithm. Moreover, an improved particle swarm optimization (IPSO) is used to adapt the learning rate. The aid of an adaptive SRWNN identifier offers the real-time gradient information to the adaptive fuzzy wavelet neural controller to overcome the impact of parameter variations, load disturbances and other uncertainties effectively, and has a good dynamic. The asymptotical stability of the system is guaranteed by using the Lyapunov method. The result of the simulation and the prototype test prove that the proposed are effective and suitable. Copyright © 2017. Published by Elsevier Ltd.
International Nuclear Information System (INIS)
Wang, Jie; Wang, Jun
2016-01-01
In an attempt to improve the forecasting accuracy of crude oil price fluctuations, a new neural network architecture is established in this work which combines Multilayer perception and ERNN (Elman recurrent neural networks) with stochastic time effective function. ERNN is a time-varying predictive control system and is developed with the ability to keep memory of recent events in order to predict future output. The stochastic time effective function represents that the recent information has a stronger effect for the investors than the old information. With the established model the empirical research has a good performance in testing the predictive effects on four different time series indices. Compared to other models, the present model is possible to evaluate data from 1990s to today with extreme accuracy and speedy. The applied CID (complexity invariant distance) analysis and multiscale CID analysis, are provided as the new useful measures to evaluate a better predicting ability of the proposed model than other traditional models. - Highlights: • A new forecasting model is developed by a random Elman recurrent neural network. • The forecasting accuracy of crude oil price fluctuations is improved by the model. • The forecasting results of the proposed model are more accurate than compared models. • Two new distance analysis methods are applied to confirm the predicting results.
Artun, E.; Ertekin, T.; Watson, R.; Miller, B.
2012-01-01
In this paper, an inverse looking approach is presented to efficiently design cyclic pressure pulsing (huff 'n' puff) with N 2 and CO 2, which is an effective improved oil recovery method in naturally fractured reservoirs. A numerical flow simulation model with compositional, dual-porosity formulation is constructed. The model characteristics are from the Big Andy Field, which is a depleted, naturally fractured oil reservoir in Kentucky. A set of cyclic pulsing design scenarios is created and run using this model. These scenarios and corresponding performance indicators are fed into the recurrent neural network for training. In order to capture the cyclic, time-dependent behavior of the process, recurrent neural networks are used to develop proxy models that can mimic the reservoir simulation model in an inverse looking manner. Two separate inverse looking proxy models for N 2 and CO 2 injections are constructed to predict the corresponding design scenarios, given a set of desired performance characteristics. Predictive capabilities of developed proxy models are evaluated by comparing simulation outputs with neural-network outputs. It is observed that networks are able to accurately predict the design parameters, such as the injection rate and the duration of injection, soaking and production periods.
DEFF Research Database (Denmark)
Dasgupta, Sakyasingha; Goldschmidt, Dennis; Wörgötter, Florentin
2015-01-01
Walking animals, like stick insects, cockroaches or ants, demonstrate a fascinating range of locomotive abilities and complex behaviors. The locomotive behaviors can consist of a variety of walking patterns along with adaptation that allow the animals to deal with changes in environmental...... conditions, like uneven terrains, gaps, obstacles etc. Biological study has revealed that such complex behaviors are a result of a combination of biomechanics and neural mechanism thus representing the true nature of embodied interactions. While the biomechanics helps maintain flexibility and sustain...... a variety of movements, the neural mechanisms generate movements while making appropriate predictions crucial for achieving adaptation. Such predictions or planning ahead can be achieved by way of internal models that are grounded in the overall behavior of the animal. Inspired by these findings, we present...
Directory of Open Access Journals (Sweden)
YuKang Jia
2017-01-01
Full Text Available Long Short-Term Memory (LSTM is a kind of Recurrent Neural Networks (RNN relating to time series, which has achieved good performance in speech recogniton and image recognition. Long Short-Term Memory Projection (LSTMP is a variant of LSTM to further optimize speed and performance of LSTM by adding a projection layer. As LSTM and LSTMP have performed well in pattern recognition, in this paper, we combine them with Connectionist Temporal Classification (CTC to study piano’s continuous note recognition for robotics. Based on the Beijing Forestry University music library, we conduct experiments to show recognition rates and numbers of iterations of LSTM with a single layer, LSTMP with a single layer, and Deep LSTM (DLSTM, LSTM with multilayers. As a result, the single layer LSTMP proves performing much better than the single layer LSTM in both time and the recognition rate; that is, LSTMP has fewer parameters and therefore reduces the training time, and, moreover, benefiting from the projection layer, LSTMP has better performance, too. The best recognition rate of LSTMP is 99.8%. As for DLSTM, the recognition rate can reach 100% because of the effectiveness of the deep structure, but compared with the single layer LSTMP, DLSTM needs more training time.
Czech Academy of Sciences Publication Activity Database
Frolov, A. A.; Sirota, A.M.; Húsek, Dušan; Muraviev, I. P.
2004-01-01
Roč. 14, č. 2 (2004), s. 139-152 ISSN 1210-0552 R&D Projects: GA ČR GA201/01/1192 Grant - others:BARRANDE(EU) 99010-2/99053; Intellectual computer Systems(EU) Grant 2.45 Institutional research plan: CEZ:AV0Z1030915 Keywords : nonlinear binary factor analysis * feature extraction * recurrent neural network * Single-Step approximation * neurodynamics simulation * attraction basins * Hebbian learning * unsupervised learning * neuroscience * brain function modeling Subject RIV: BA - General Mathematics
Neural Circuitry Based on Single Electron Transistors and Single Electron Memories
Directory of Open Access Journals (Sweden)
Aïmen BOUBAKER
2014-05-01
Full Text Available In this paper, we propose and explain a neural circuitry based on single electron transistors ‘SET’ which can be used in classification and recognition. We implement, after that, a Winner-Take-All ‘WTA’ neural network with lateral inhibition architecture. The original idea of this work is reflected, first, in the proposed new single electron memory ‘SEM’ design by hybridising two promising Single Electron Memory ‘SEM’ and the MTJ/Ring memory and second, in modeling and simulation results of neural memory based on SET. We prove the charge storage in quantum dot in two types of memories.
Identification of a nuclear reactor core (VVER) using recurrent neural networks
Energy Technology Data Exchange (ETDEWEB)
Boroushaki, M. E-mail: boroushaki@mehr.sharif.ac.ir; Ghofrani, M.B.; Lucas, C
2002-07-01
Recurrent neural networks (RNNs) in identification of complex nonlinear plants like nuclear reactor core, have difficulty in learning long-term dynamics. Therefore, in most papers in this area, the reactor core is used to identify just the short-term dynamics. In this paper we used a multi-NARX (nonlinear autoregressive with exogenous inputs) structure, including neural networks with different time steps and a heuristic compound learning method, consisting of off-line and on-line batch learnings. This multi-NARX was trained by an accurate 3-dimensional core calculation code. Network responses show that this procedure solves the difficulty in identification of complex nonlinear dynamic MIMO (multi-input multi-output) plants like nuclear reactor core, and can be used in fast prediction of nuclear reactor core dynamics behavior.
Identification of Jets Containing b-Hadrons with Recurrent Neural Networks at the ATLAS Experiment
CERN. Geneva
2017-01-01
A novel b-jet identification algorithm is constructed with a Recurrent Neural Network (RNN) at the ATLAS Experiment. This talk presents the expected performance of the RNN based b-tagging in simulated $t \\bar t$ events. The RNN based b-tagging processes properties of tracks associated to jets which are represented in sequences. In contrast to traditional impact-parameter-based b-tagging algorithms which assume the tracks of jets are independent from each other, RNN based b-tagging can exploit the spatial and kinematic correlations of tracks which are initiated from the same b-hadrons. The neural network nature of the tagging algorithm also allows the flexibility of extending input features to include more track properties than can be effectively used in traditional algorithms.
New results on global exponential stability of recurrent neural networks with time-varying delays
International Nuclear Information System (INIS)
Xu Shengyuan; Chu Yuming; Lu Junwei
2006-01-01
This Letter provides new sufficient conditions for the existence, uniqueness and global exponential stability of the equilibrium point of recurrent neural networks with time-varying delays by employing Lyapunov functions and using the Halanay inequality. The time-varying delays are not necessarily differentiable. Both Lipschitz continuous activation functions and monotone nondecreasing activation functions are considered. The derived stability criteria are expressed in terms of linear matrix inequalities (LMIs), which can be checked easily by resorting to recently developed algorithms solving LMIs. Furthermore, the proposed stability results are less conservative than some previous ones in the literature, which is demonstrated via some numerical examples
Hierarchical Recurrent Neural Hashing for Image Retrieval With Hierarchical Convolutional Features.
Lu, Xiaoqiang; Chen, Yaxiong; Li, Xuelong
Hashing has been an important and effective technology in image retrieval due to its computational efficiency and fast search speed. The traditional hashing methods usually learn hash functions to obtain binary codes by exploiting hand-crafted features, which cannot optimally represent the information of the sample. Recently, deep learning methods can achieve better performance, since deep learning architectures can learn more effective image representation features. However, these methods only use semantic features to generate hash codes by shallow projection but ignore texture details. In this paper, we proposed a novel hashing method, namely hierarchical recurrent neural hashing (HRNH), to exploit hierarchical recurrent neural network to generate effective hash codes. There are three contributions of this paper. First, a deep hashing method is proposed to extensively exploit both spatial details and semantic information, in which, we leverage hierarchical convolutional features to construct image pyramid representation. Second, our proposed deep network can exploit directly convolutional feature maps as input to preserve the spatial structure of convolutional feature maps. Finally, we propose a new loss function that considers the quantization error of binarizing the continuous embeddings into the discrete binary codes, and simultaneously maintains the semantic similarity and balanceable property of hash codes. Experimental results on four widely used data sets demonstrate that the proposed HRNH can achieve superior performance over other state-of-the-art hashing methods.Hashing has been an important and effective technology in image retrieval due to its computational efficiency and fast search speed. The traditional hashing methods usually learn hash functions to obtain binary codes by exploiting hand-crafted features, which cannot optimally represent the information of the sample. Recently, deep learning methods can achieve better performance, since deep
Automatic construction of a recurrent neural network based classifier for vehicle passage detection
Burnaev, Evgeny; Koptelov, Ivan; Novikov, German; Khanipov, Timur
2017-03-01
Recurrent Neural Networks (RNNs) are extensively used for time-series modeling and prediction. We propose an approach for automatic construction of a binary classifier based on Long Short-Term Memory RNNs (LSTM-RNNs) for detection of a vehicle passage through a checkpoint. As an input to the classifier we use multidimensional signals of various sensors that are installed on the checkpoint. Obtained results demonstrate that the previous approach to handcrafting a classifier, consisting of a set of deterministic rules, can be successfully replaced by an automatic RNN training on an appropriately labelled data.
Prenatal Diagnosis, Fetal Surgery, Recurrence Risk and Differential Diagnosis of Neural Tube Defects
Directory of Open Access Journals (Sweden)
Chih-Ping Chen
2008-09-01
Full Text Available Prenatal screening with α-fetoprotein (AFP and ultrasonography have allowed the prenatal diagnosis of neural tube defects (NTDs in current obstetric care, and open spina bifida has been considered a potential candidate for in utero treatment in modern pediatric surgery. This article provides an overview of maternal serum AFP screening, amniotic fluid AFP assays, amniotic fluid acetylcholinesterase immunoassays and level II ultrasound for NTDs, prenatal repair of fetal myelomeningocele, recurrence risk of NTDs, and differential diagnosis of NTDs on prenatal ultrasound.
The super-Turing computational power of plastic recurrent neural networks.
Cabessa, Jérémie; Siegelmann, Hava T
2014-12-01
We study the computational capabilities of a biologically inspired neural model where the synaptic weights, the connectivity pattern, and the number of neurons can evolve over time rather than stay static. Our study focuses on the mere concept of plasticity of the model so that the nature of the updates is assumed to be not constrained. In this context, we show that the so-called plastic recurrent neural networks (RNNs) are capable of the precise super-Turing computational power--as the static analog neural networks--irrespective of whether their synaptic weights are modeled by rational or real numbers, and moreover, irrespective of whether their patterns of plasticity are restricted to bi-valued updates or expressed by any other more general form of updating. Consequently, the incorporation of only bi-valued plastic capabilities in a basic model of RNNs suffices to break the Turing barrier and achieve the super-Turing level of computation. The consideration of more general mechanisms of architectural plasticity or of real synaptic weights does not further increase the capabilities of the networks. These results support the claim that the general mechanism of plasticity is crucially involved in the computational and dynamical capabilities of biological neural networks. They further show that the super-Turing level of computation reflects in a suitable way the capabilities of brain-like models of computation.
Nonlinear Model Predictive Control Based on a Self-Organizing Recurrent Neural Network.
Han, Hong-Gui; Zhang, Lu; Hou, Ying; Qiao, Jun-Fei
2016-02-01
A nonlinear model predictive control (NMPC) scheme is developed in this paper based on a self-organizing recurrent radial basis function (SR-RBF) neural network, whose structure and parameters are adjusted concurrently in the training process. The proposed SR-RBF neural network is represented in a general nonlinear form for predicting the future dynamic behaviors of nonlinear systems. To improve the modeling accuracy, a spiking-based growing and pruning algorithm and an adaptive learning algorithm are developed to tune the structure and parameters of the SR-RBF neural network, respectively. Meanwhile, for the control problem, an improved gradient method is utilized for the solution of the optimization problem in NMPC. The stability of the resulting control system is proved based on the Lyapunov stability theory. Finally, the proposed SR-RBF neural network-based NMPC (SR-RBF-NMPC) is used to control the dissolved oxygen (DO) concentration in a wastewater treatment process (WWTP). Comparisons with other existing methods demonstrate that the SR-RBF-NMPC can achieve a considerably better model fitting for WWTP and a better control performance for DO concentration.
Recurrence in Benign Paroxysmal Positional Vertigo: A Large, Single-Institution Study.
Luryi, Alexander L; Lawrence, Juliana; Bojrab, Dennis I; LaRouere, Michael; Babu, Seilesh; Zappia, John; Sargent, Eric W; Chan, Eleanor; Naumann, Ilka; Hong, Robert S; Schutt, Christopher A
2018-04-11
To report rates of recurrence in benign paroxysmal positional vertigo (BPPV) and associated patient and disease factors. Retrospective chart review. Single high-volume otology practice. Patients diagnosed with BPPV from 2007 to 2016 with documented resolution of symptoms. Diagnostic and particle repositioning maneuvers for BPPV. BPPV recurrence, time to recurrence, and ear(s) affected at recurrence. A total of 1,105 patients meeting criteria were identified. Of this population, 37% had recurrence of BPPV in either ear or both ears. Overall same-ear recurrence rate was 28%; 76% of recurrences involved the same ear(s) as initial presentation. Recurrences that occurred after longer disease-free intervals were more likely to involve the opposite ear than early recurrences (p = 0.02). Female sex (40.4% versus 32.7%, p = 0.01) and history of previous BPPV (57.5% versus 32.4%, p diabetes mellitus, and traumatic etiology were not. Approximately, half (56%) of recurrences occurred within 1 year of resolution. A large single-institution study of recurrence in BPPV is presented along with Kaplan-Meier disease-free survival curves. Female sex and history of previous BPPV were associated with increased recurrence, while previously suspected risk factors for recurrence including history of Menière's disease, diabetes, and trauma were not. Remote recurrence is more likely to involve the contralateral ear than early recurrence. These data solidify the expected course of treated BPPV allowing for improved clinical care and patient counseling.
Xiao, Lin; Liao, Bolin; Li, Shuai; Chen, Ke
2018-02-01
In order to solve general time-varying linear matrix equations (LMEs) more efficiently, this paper proposes two nonlinear recurrent neural networks based on two nonlinear activation functions. According to Lyapunov theory, such two nonlinear recurrent neural networks are proved to be convergent within finite-time. Besides, by solving differential equation, the upper bounds of the finite convergence time are determined analytically. Compared with existing recurrent neural networks, the proposed two nonlinear recurrent neural networks have a better convergence property (i.e., the upper bound is lower), and thus the accurate solutions of general time-varying LMEs can be obtained with less time. At last, various different situations have been considered by setting different coefficient matrices of general time-varying LMEs and a great variety of computer simulations (including the application to robot manipulators) have been conducted to validate the better finite-time convergence of the proposed two nonlinear recurrent neural networks. Copyright © 2017 Elsevier Ltd. All rights reserved.
International Nuclear Information System (INIS)
Lu Junguo
2008-01-01
In this paper, the global exponential stability and periodicity for a class of reaction-diffusion delayed recurrent neural networks with Dirichlet boundary conditions are addressed by constructing suitable Lyapunov functionals and utilizing some inequality techniques. We first prove global exponential converge to 0 of the difference between any two solutions of the original reaction-diffusion delayed recurrent neural networks with Dirichlet boundary conditions, the existence and uniqueness of equilibrium is the direct results of this procedure. This approach is different from the usually used one where the existence, uniqueness of equilibrium and stability are proved in two separate steps. Furthermore, we prove periodicity of the reaction-diffusion delayed recurrent neural networks with Dirichlet boundary conditions. Sufficient conditions ensuring the global exponential stability and the existence of periodic oscillatory solutions for the reaction-diffusion delayed recurrent neural networks with Dirichlet boundary conditions are given. These conditions are easy to check and have important leading significance in the design and application of reaction-diffusion recurrent neural networks with delays. Finally, two numerical examples are given to show the effectiveness of the obtained results
Bennett, C.; Dunne, J. F.; Trimby, S.; Richardson, D.
2017-02-01
A recurrent non-linear autoregressive with exogenous input (NARX) neural network is proposed, and a suitable fully-recurrent training methodology is adapted and tuned, for reconstructing cylinder pressure in multi-cylinder IC engines using measured crank kinematics. This type of indirect sensing is important for cost effective closed-loop combustion control and for On-Board Diagnostics. The challenge addressed is to accurately predict cylinder pressure traces within the cycle under generalisation conditions: i.e. using data not previously seen by the network during training. This involves direct construction and calibration of a suitable inverse crank dynamic model, which owing to singular behaviour at top-dead-centre (TDC), has proved difficult via physical model construction, calibration, and inversion. The NARX architecture is specialised and adapted to cylinder pressure reconstruction, using a fully-recurrent training methodology which is needed because the alternatives are too slow and unreliable for practical network training on production engines. The fully-recurrent Robust Adaptive Gradient Descent (RAGD) algorithm, is tuned initially using synthesised crank kinematics, and then tested on real engine data to assess the reconstruction capability. Real data is obtained from a 1.125 l, 3-cylinder, in-line, direct injection spark ignition (DISI) engine involving synchronised measurements of crank kinematics and cylinder pressure across a range of steady-state speed and load conditions. The paper shows that a RAGD-trained NARX network using both crank velocity and crank acceleration as input information, provides fast and robust training. By using the optimum epoch identified during RAGD training, acceptably accurate cylinder pressures, and especially accurate location-of-peak-pressure, can be reconstructed robustly under generalisation conditions, making it the most practical NARX configuration and recurrent training methodology for use on production engines.
International Nuclear Information System (INIS)
Ciammella, Patrizia; Podgornii, Ala; Galeandro, Maria; D’Abbiero, Nunziata; Pisanello, Anna; Botti, Andrea; Cagni, Elisabetta; Iori, Mauro; Iotti, Cinzia
2013-01-01
Glioblastoma (GBM) is the most common malignant primary brain tumor in adults. Tumor control and survival have improved with the use of radiotherapy (RT) plus concomitant and adjuvant chemotherapy, but the prognosis remain poor. In most cases the recurrence occurs within 7–9 months after primary treatment. Currently, many approaches are available for the salvage treatment of patients with recurrent GBM, including resection, re-irradiation or systemic agents, but no standard of care exists. We analysed a cohort of patients with recurrent GBM treated with frame-less hypofractionated stereotactic radiation therapy with a total dose of 25 Gy in 5 fractions. Of 91 consecutive patients with newly diagnosed GBM treated between 2007 and 2012 with conventional adjuvant chemo-radiation therapy, 15 underwent salvage RT at recurrence. The median time interval between primary RT and salvage RT was 10.8 months (range, 6–54 months). Overall, patients undergoing salvage RT showed a longer survival, with a median survival of 33 vs. 9.9 months (p= 0.00149). Median overall survival (OS) from salvage RT was 9.5 months. No patients demonstrated clinically significant acute morbidity, and all patients were able to complete the prescribed radiation therapy without interruption. Our results suggest that hypofractionated stereotactic radiation therapy is effective and safe in recurrent GBM. However, until prospective randomized trials will confirm these results, the decision for salvage treatment should remain individual and based on a multidisciplinary evaluation of each patient
Zheng, Jing; Lu, Jiren; Peng, Suping; Jiang, Tianqi
2018-02-01
The conventional arrival pick-up algorithms cannot avoid the manual modification of the parameters for the simultaneous identification of multiple events under different signal-to-noise ratios (SNRs). Therefore, in order to automatically obtain the arrivals of multiple events with high precision under different SNRs, in this study an algorithm was proposed which had the ability to pick up the arrival of microseismic or acoustic emission events based on deep recurrent neural networks. The arrival identification was performed using two important steps, which included a training phase and a testing phase. The training process was mathematically modelled by deep recurrent neural networks using Long Short-Term Memory architecture. During the testing phase, the learned weights were utilized to identify the arrivals through the microseismic/acoustic emission data sets. The data sets were obtained by rock physics experiments of the acoustic emission. In order to obtain the data sets under different SNRs, this study added random noise to the raw experiments' data sets. The results showed that the outcome of the proposed method was able to attain an above 80 per cent hit-rate at SNR 0 dB, and an approximately 70 per cent hit-rate at SNR -5 dB, with an absolute error in 10 sampling points. These results indicated that the proposed method had high selection precision and robustness.
Deep recurrent neural network reveals a hierarchy of process memory during dynamic natural vision.
Shi, Junxing; Wen, Haiguang; Zhang, Yizhen; Han, Kuan; Liu, Zhongming
2018-05-01
The human visual cortex extracts both spatial and temporal visual features to support perception and guide behavior. Deep convolutional neural networks (CNNs) provide a computational framework to model cortical representation and organization for spatial visual processing, but unable to explain how the brain processes temporal information. To overcome this limitation, we extended a CNN by adding recurrent connections to different layers of the CNN to allow spatial representations to be remembered and accumulated over time. The extended model, or the recurrent neural network (RNN), embodied a hierarchical and distributed model of process memory as an integral part of visual processing. Unlike the CNN, the RNN learned spatiotemporal features from videos to enable action recognition. The RNN better predicted cortical responses to natural movie stimuli than the CNN, at all visual areas, especially those along the dorsal stream. As a fully observable model of visual processing, the RNN also revealed a cortical hierarchy of temporal receptive window, dynamics of process memory, and spatiotemporal representations. These results support the hypothesis of process memory, and demonstrate the potential of using the RNN for in-depth computational understanding of dynamic natural vision. © 2018 Wiley Periodicals, Inc.
Optimal Formation of Multirobot Systems Based on a Recurrent Neural Network.
Wang, Yunpeng; Cheng, Long; Hou, Zeng-Guang; Yu, Junzhi; Tan, Min
2016-02-01
The optimal formation problem of multirobot systems is solved by a recurrent neural network in this paper. The desired formation is described by the shape theory. This theory can generate a set of feasible formations that share the same relative relation among robots. An optimal formation means that finding one formation from the feasible formation set, which has the minimum distance to the initial formation of the multirobot system. Then, the formation problem is transformed into an optimization problem. In addition, the orientation, scale, and admissible range of the formation can also be considered as the constraints in the optimization problem. Furthermore, if all robots are identical, their positions in the system are exchangeable. Then, each robot does not necessarily move to one specific position in the formation. In this case, the optimal formation problem becomes a combinational optimization problem, whose optimal solution is very hard to obtain. Inspired by the penalty method, this combinational optimization problem can be approximately transformed into a convex optimization problem. Due to the involvement of the Euclidean norm in the distance, the objective function of these optimization problems are nonsmooth. To solve these nonsmooth optimization problems efficiently, a recurrent neural network approach is employed, owing to its parallel computation ability. Finally, some simulations and experiments are given to validate the effectiveness and efficiency of the proposed optimal formation approach.
Analysis of recurrent neural networks for short-term energy load forecasting
Di Persio, Luca; Honchar, Oleksandr
2017-11-01
Short-term forecasts have recently gained an increasing attention because of the rise of competitive electricity markets. In fact, short-terms forecast of possible future loads turn out to be fundamental to build efficient energy management strategies as well as to avoid energy wastage. Such type of challenges are difficult to tackle both from a theoretical and applied point of view. Latter tasks require sophisticated methods to manage multidimensional time series related to stochastic phenomena which are often highly interconnected. In the present work we first review novel approaches to energy load forecasting based on recurrent neural network, focusing our attention on long/short term memory architectures (LSTMs). Such type of artificial neural networks have been widely applied to problems dealing with sequential data such it happens, e.g., in socio-economics settings, for text recognition purposes, concerning video signals, etc., always showing their effectiveness to model complex temporal data. Moreover, we consider different novel variations of basic LSTMs, such as sequence-to-sequence approach and bidirectional LSTMs, aiming at providing effective models for energy load data. Last but not least, we test all the described algorithms on real energy load data showing not only that deep recurrent networks can be successfully applied to energy load forecasting, but also that this approach can be extended to other problems based on time series prediction.
Askari, Marjan; Eslami, Saied; Scheffer, Alice C; Medlock, Stephanie; de Rooij, Sophia E; van der Velde, Nathalie; Abu-Hanna, Ameen
2013-10-01
Polypharmacy, and specifically the use of multiple fall-risk-increasing drugs (FRID), have been associated with increased risk of falling in older age. However, it is not yet clear whether the known set of FRIDs can be extrapolated to recurrent fallers, since they form a distinct group of more vulnerable older persons with different characteristics. We aim to investigate which classes of medications are associated with recurrent falls in elderly patients visiting the Emergency Department (ED) after a fall. This study had a cross-sectional design and was conducted in the ED of an academic medical center. Patients who sustained a fall, 65 years or older, and who visited the ED between 2004 and 2010 were invited to fill in a validated fall questionnaire designed to assess patient and fall characteristics (CAREFALL Triage Instrument [CTI]). We translated self-reported medications to anatomical therapeutic chemical (ATC) codes (at the second level). Univariate logistic regression analysis was performed to explore the association between medication classes and the outcome parameter (recurrent fall). Multivariate logistic regression was used to assess the associations after adjustment to potential confounders. In total 2,258 patients participated in our study, of whom 39 % (873) had sustained two or more falls within the previous year. After adjustment for the potential confounders, drugs for acid-related disorders (adjusted odds ratio [aOR] 1.29; 95 % CI 1.03–1.60), analgesics (aOR 1.22; 95 % CI 1.06–1.41), anti-Parkinson drugs (aOR 1.59; 95 % CI 1.02–2.46), nasal preparations (aOR 1.49; 95 % CI 1.07–2.08), ophthalmologicals (aOR 1.51; 95 % CI 1.10–2.09); antipsychotics (aOR 2.21; 95 % CI 1.08–4.52), and antidepressants (aOR 1.64; 95 % CI 1.13–2.37) remained statistically significantly associated with an ED visit due to a recurrent fall. Known FRIDs, such as psychotropic drugs, also increase the risk of recurrent falls. However, we found four relatively new
Kernel Function Tuning for Single-Layer Neural Networks
Czech Academy of Sciences Publication Activity Database
Vidnerová, Petra; Neruda, Roman
-, accepted 28.11. 2017 (2018) ISSN 2278-0149 R&D Projects: GA ČR GA15-18108S Institutional support: RVO:67985807 Keywords : single-layer neural networks * kernel methods * kernel function * optimisation Subject RIV: IN - Informatics, Computer Science http://www.ijmerr.com/
Using deep recurrent neural network for direct beam solar irradiance cloud screening
Chen, Maosi; Davis, John M.; Liu, Chaoshun; Sun, Zhibin; Zempila, Melina Maria; Gao, Wei
2017-09-01
Cloud screening is an essential procedure for in-situ calibration and atmospheric properties retrieval on (UV-)MultiFilter Rotating Shadowband Radiometer [(UV-)MFRSR]. Previous study has explored a cloud screening algorithm for direct-beam (UV-)MFRSR voltage measurements based on the stability assumption on a long time period (typically a half day or a whole day). To design such an algorithm requires in-depth understanding of radiative transfer and delicate data manipulation. Recent rapid developments on deep neural network and computation hardware have opened a window for modeling complicated End-to-End systems with a standardized strategy. In this study, a multi-layer dynamic bidirectional recurrent neural network is built for determining the cloudiness on each time point with a 17-year training dataset and tested with another 1-year dataset. The dataset is the daily 3-minute cosine corrected voltages, airmasses, and the corresponding cloud/clear-sky labels at two stations of the USDA UV-B Monitoring and Research Program. The results show that the optimized neural network model (3-layer, 250 hidden units, and 80 epochs of training) has an overall test accuracy of 97.87% (97.56% for the Oklahoma site and 98.16% for the Hawaii site). Generally, the neural network model grasps the key concept of the original model to use data in the entire day rather than short nearby measurements to perform cloud screening. A scrutiny of the logits layer suggests that the neural network model automatically learns a way to calculate a quantity similar to total optical depth and finds an appropriate threshold for cloud screening.
Reinforcement Learning of Linking and Tracing Contours in Recurrent Neural Networks
Brosch, Tobias; Neumann, Heiko; Roelfsema, Pieter R.
2015-01-01
The processing of a visual stimulus can be subdivided into a number of stages. Upon stimulus presentation there is an early phase of feedforward processing where the visual information is propagated from lower to higher visual areas for the extraction of basic and complex stimulus features. This is followed by a later phase where horizontal connections within areas and feedback connections from higher areas back to lower areas come into play. In this later phase, image elements that are behaviorally relevant are grouped by Gestalt grouping rules and are labeled in the cortex with enhanced neuronal activity (object-based attention in psychology). Recent neurophysiological studies revealed that reward-based learning influences these recurrent grouping processes, but it is not well understood how rewards train recurrent circuits for perceptual organization. This paper examines the mechanisms for reward-based learning of new grouping rules. We derive a learning rule that can explain how rewards influence the information flow through feedforward, horizontal and feedback connections. We illustrate the efficiency with two tasks that have been used to study the neuronal correlates of perceptual organization in early visual cortex. The first task is called contour-integration and demands the integration of collinear contour elements into an elongated curve. We show how reward-based learning causes an enhancement of the representation of the to-be-grouped elements at early levels of a recurrent neural network, just as is observed in the visual cortex of monkeys. The second task is curve-tracing where the aim is to determine the endpoint of an elongated curve composed of connected image elements. If trained with the new learning rule, neural networks learn to propagate enhanced activity over the curve, in accordance with neurophysiological data. We close the paper with a number of model predictions that can be tested in future neurophysiological and computational studies
Reinforcement Learning of Linking and Tracing Contours in Recurrent Neural Networks.
Directory of Open Access Journals (Sweden)
Tobias Brosch
2015-10-01
Full Text Available The processing of a visual stimulus can be subdivided into a number of stages. Upon stimulus presentation there is an early phase of feedforward processing where the visual information is propagated from lower to higher visual areas for the extraction of basic and complex stimulus features. This is followed by a later phase where horizontal connections within areas and feedback connections from higher areas back to lower areas come into play. In this later phase, image elements that are behaviorally relevant are grouped by Gestalt grouping rules and are labeled in the cortex with enhanced neuronal activity (object-based attention in psychology. Recent neurophysiological studies revealed that reward-based learning influences these recurrent grouping processes, but it is not well understood how rewards train recurrent circuits for perceptual organization. This paper examines the mechanisms for reward-based learning of new grouping rules. We derive a learning rule that can explain how rewards influence the information flow through feedforward, horizontal and feedback connections. We illustrate the efficiency with two tasks that have been used to study the neuronal correlates of perceptual organization in early visual cortex. The first task is called contour-integration and demands the integration of collinear contour elements into an elongated curve. We show how reward-based learning causes an enhancement of the representation of the to-be-grouped elements at early levels of a recurrent neural network, just as is observed in the visual cortex of monkeys. The second task is curve-tracing where the aim is to determine the endpoint of an elongated curve composed of connected image elements. If trained with the new learning rule, neural networks learn to propagate enhanced activity over the curve, in accordance with neurophysiological data. We close the paper with a number of model predictions that can be tested in future neurophysiological and
Askari, Marjan; Eslami, Saied; Scheffer, Alice C.; Medlock, Stephanie; de Rooij, Sophia E.; van der Velde, Nathalie; Abu-Hanna, Ameen
2013-01-01
Polypharmacy, and specifically the use of multiple fall-risk-increasing drugs (FRID), have been associated with increased risk of falling in older age. However, it is not yet clear whether the known set of FRIDs can be extrapolated to recurrent fallers, since they form a distinct group of more
A Three-Threshold Learning Rule Approaches the Maximal Capacity of Recurrent Neural Networks.
Directory of Open Access Journals (Sweden)
Alireza Alemi
2015-08-01
Full Text Available Understanding the theoretical foundations of how memories are encoded and retrieved in neural populations is a central challenge in neuroscience. A popular theoretical scenario for modeling memory function is the attractor neural network scenario, whose prototype is the Hopfield model. The model simplicity and the locality of the synaptic update rules come at the cost of a poor storage capacity, compared with the capacity achieved with perceptron learning algorithms. Here, by transforming the perceptron learning rule, we present an online learning rule for a recurrent neural network that achieves near-maximal storage capacity without an explicit supervisory error signal, relying only upon locally accessible information. The fully-connected network consists of excitatory binary neurons with plastic recurrent connections and non-plastic inhibitory feedback stabilizing the network dynamics; the memory patterns to be memorized are presented online as strong afferent currents, producing a bimodal distribution for the neuron synaptic inputs. Synapses corresponding to active inputs are modified as a function of the value of the local fields with respect to three thresholds. Above the highest threshold, and below the lowest threshold, no plasticity occurs. In between these two thresholds, potentiation/depression occurs when the local field is above/below an intermediate threshold. We simulated and analyzed a network of binary neurons implementing this rule and measured its storage capacity for different sizes of the basins of attraction. The storage capacity obtained through numerical simulations is shown to be close to the value predicted by analytical calculations. We also measured the dependence of capacity on the strength of external inputs. Finally, we quantified the statistics of the resulting synaptic connectivity matrix, and found that both the fraction of zero weight synapses and the degree of symmetry of the weight matrix increase with the
Pérez, Oswaldo; Merchant, Hugo
2018-04-03
Extensive research has described two key features of interval timing. The bias property is associated with accuracy and implies that time is overestimated for short intervals and underestimated for long intervals. The scalar property is linked to precision and states that the variability of interval estimates increases as a function of interval duration. The neural mechanisms behind these properties are not well understood. Here we implemented a recurrent neural network that mimics a cortical ensemble and includes cells that show paired-pulse facilitation and slow inhibitory synaptic currents. The network produces interval selective responses and reproduces both bias and scalar properties when a Bayesian decoder reads its activity. Notably, the interval-selectivity, timing accuracy, and precision of the network showed complex changes as a function of the decay time constants of the modeled synaptic properties and the level of background activity of the cells. These findings suggest that physiological values of the time constants for paired-pulse facilitation and GABAb, as well as the internal state of the network, determine the bias and scalar properties of interval timing. Significant Statement Timing is a fundamental element of complex behavior, including music and language. Temporal processing in a wide variety of contexts shows two primary features: time estimates exhibit a shift towards the mean (the bias property) and are more variable for longer intervals (the scalar property). We implemented a recurrent neural network that includes long-lasting synaptic currents, which can not only produce interval selective responses but also follow the bias and scalar properties. Interestingly, only physiological values of the time constants for paired-pulse facilitation and GABAb, as well as intermediate background activity within the network can reproduce the two key features of interval timing. Copyright © 2018 the authors.
Xia, Peng; Hu, Jie; Peng, Yinghong
2017-10-25
A novel model based on deep learning is proposed to estimate kinematic information for myoelectric control from multi-channel electromyogram (EMG) signals. The neural information of limb movement is embedded in EMG signals that are influenced by all kinds of factors. In order to overcome the negative effects of variability in signals, the proposed model employs the deep architecture combining convolutional neural networks (CNNs) and recurrent neural networks (RNNs). The EMG signals are transformed to time-frequency frames as the input to the model. The limb movement is estimated by the model that is trained with the gradient descent and backpropagation procedure. We tested the model for simultaneous and proportional estimation of limb movement in eight healthy subjects and compared it with support vector regression (SVR) and CNNs on the same data set. The experimental studies show that the proposed model has higher estimation accuracy and better robustness with respect to time. The combination of CNNs and RNNs can improve the model performance compared with using CNNs alone. The model of deep architecture is promising in EMG decoding and optimization of network structures can increase the accuracy and robustness. © 2017 International Center for Artificial Organs and Transplantation and Wiley Periodicals, Inc.
Liu, Qingshan; Guo, Zhishan; Wang, Jun
2012-02-01
In this paper, a one-layer recurrent neural network is proposed for solving pseudoconvex optimization problems subject to linear equality and bound constraints. Compared with the existing neural networks for optimization (e.g., the projection neural networks), the proposed neural network is capable of solving more general pseudoconvex optimization problems with equality and bound constraints. Moreover, it is capable of solving constrained fractional programming problems as a special case. The convergence of the state variables of the proposed neural network to achieve solution optimality is guaranteed as long as the designed parameters in the model are larger than the derived lower bounds. Numerical examples with simulation results illustrate the effectiveness and characteristics of the proposed neural network. In addition, an application for dynamic portfolio optimization is discussed. Copyright © 2011 Elsevier Ltd. All rights reserved.
Liu, Qingshan; Dang, Chuangyin; Cao, Jinde
2010-07-01
In this paper, based on a one-neuron recurrent neural network, a novel k-winners-take-all ( k -WTA) network is proposed. Finite time convergence of the proposed neural network is proved using the Lyapunov method. The k-WTA operation is first converted equivalently into a linear programming problem. Then, a one-neuron recurrent neural network is proposed to get the kth or (k+1)th largest inputs of the k-WTA problem. Furthermore, a k-WTA network is designed based on the proposed neural network to perform the k-WTA operation. Compared with the existing k-WTA networks, the proposed network has simple structure and finite time convergence. In addition, simulation results on numerical examples show the effectiveness and performance of the proposed k-WTA network.
Jin, Long; Liao, Bolin; Liu, Mei; Xiao, Lin; Guo, Dongsheng; Yan, Xiaogang
2017-01-01
By incorporating the physical constraints in joint space, a different-level simultaneous minimization scheme, which takes both the robot kinematics and robot dynamics into account, is presented and investigated for fault-tolerant motion planning of redundant manipulator in this paper. The scheme is reformulated as a quadratic program (QP) with equality and bound constraints, which is then solved by a discrete-time recurrent neural network. Simulative verifications based on a six-link planar redundant robot manipulator substantiate the efficacy and accuracy of the presented acceleration fault-tolerant scheme, the resultant QP and the corresponding discrete-time recurrent neural network.
Directory of Open Access Journals (Sweden)
Suhartono Suhartono
2009-07-01
Full Text Available Neural network (NN is one of many method used to predict the electricity consumption per hour in many countries. NN method which is used in many previous studies is Feed-Forward Neural Network (FFNN or Autoregressive Neural Network(AR-NN. AR-NN model is not able to capture and explain the effect of moving average (MA order on a time series of data. This research was conducted with the purpose of reviewing the application of other types of NN, that is Elman-Recurrent Neural Network (Elman-RNN which could explain MA order effect and compare the result of prediction accuracy with multiple seasonal ARIMA (Autoregressive Integrated Moving Average models. As a case study, we used data electricity consumption per hour in Mengare Gresik. Result of analysis showed that the best of double seasonal Arima models suited to short-term forecasting in the case study data is ARIMA([1,2,3,4,6,7,9,10,14,21,33],1,8(0,1,124 (1,1,0168. This model produces a white noise residuals, but it does not have a normal distribution due to suspected outlier. Outlier detection in iterative produce 14 innovation outliers. There are 4 inputs of Elman-RNN network that were examined and tested for forecasting the data, the input according to lag Arima, input such as lag Arima plus 14 dummy outlier, inputs are the lag-multiples of 24 up to lag 480, and the inputs are lag 1 and lag multiples of 24+1. All of four network uses one hidden layer with tangent sigmoid activation function and one output with a linear function. The result of comparative forecast accuracy through value of MAPE out-sample showed that the fourth networks, namely Elman-RNN (22, 3, 1, is the best model for forecasting electricity consumption per hour in short term in Mengare Gresik.
Wang, Xiao-Jing
2016-01-01
The ability to simultaneously record from large numbers of neurons in behaving animals has ushered in a new era for the study of the neural circuit mechanisms underlying cognitive functions. One promising approach to uncovering the dynamical and computational principles governing population responses is to analyze model recurrent neural networks (RNNs) that have been optimized to perform the same tasks as behaving animals. Because the optimization of network parameters specifies the desired output but not the manner in which to achieve this output, “trained” networks serve as a source of mechanistic hypotheses and a testing ground for data analyses that link neural computation to behavior. Complete access to the activity and connectivity of the circuit, and the ability to manipulate them arbitrarily, make trained networks a convenient proxy for biological circuits and a valuable platform for theoretical investigation. However, existing RNNs lack basic biological features such as the distinction between excitatory and inhibitory units (Dale’s principle), which are essential if RNNs are to provide insights into the operation of biological circuits. Moreover, trained networks can achieve the same behavioral performance but differ substantially in their structure and dynamics, highlighting the need for a simple and flexible framework for the exploratory training of RNNs. Here, we describe a framework for gradient descent-based training of excitatory-inhibitory RNNs that can incorporate a variety of biological knowledge. We provide an implementation based on the machine learning library Theano, whose automatic differentiation capabilities facilitate modifications and extensions. We validate this framework by applying it to well-known experimental paradigms such as perceptual decision-making, context-dependent integration, multisensory integration, parametric working memory, and motor sequence generation. Our results demonstrate the wide range of neural activity
Hoellinger, Thomas; Petieau, Mathieu; Duvinage, Matthieu; Castermans, Thierry; Seetharaman, Karthik; Cebolla, Ana-Maria; Bengoetxea, Ana; Ivanenko, Yuri; Dan, Bernard; Cheron, Guy
2013-01-01
The existence of dedicated neuronal modules such as those organized in the cerebral cortex, thalamus, basal ganglia, cerebellum, or spinal cord raises the question of how these functional modules are coordinated for appropriate motor behavior. Study of human locomotion offers an interesting field for addressing this central question. The coordination of the elevation of the 3 leg segments under a planar covariation rule (Borghese et al., 1996) was recently modeled (Barliya et al., 2009) by phase-adjusted simple oscillators shedding new light on the understanding of the central pattern generator (CPG) processing relevant oscillation signals. We describe the use of a dynamic recurrent neural network (DRNN) mimicking the natural oscillatory behavior of human locomotion for reproducing the planar covariation rule in both legs at different walking speeds. Neural network learning was based on sinusoid signals integrating frequency and amplitude features of the first three harmonics of the sagittal elevation angles of the thigh, shank, and foot of each lower limb. We verified the biological plausibility of the neural networks. Best results were obtained with oscillations extracted from the first three harmonics in comparison to oscillations outside the harmonic frequency peaks. Physiological replication steadily increased with the number of neuronal units from 1 to 80, where similarity index reached 0.99. Analysis of synaptic weighting showed that the proportion of inhibitory connections consistently increased with the number of neuronal units in the DRNN. This emerging property in the artificial neural networks resonates with recent advances in neurophysiology of inhibitory neurons that are involved in central nervous system oscillatory activities. The main message of this study is that this type of DRNN may offer a useful model of physiological central pattern generator for gaining insights in basic research and developing clinical applications.
Criticality meets learning: Criticality signatures in a self-organizing recurrent neural network.
Del Papa, Bruno; Priesemann, Viola; Triesch, Jochen
2017-01-01
Many experiments have suggested that the brain operates close to a critical state, based on signatures of criticality such as power-law distributed neuronal avalanches. In neural network models, criticality is a dynamical state that maximizes information processing capacities, e.g. sensitivity to input, dynamical range and storage capacity, which makes it a favorable candidate state for brain function. Although models that self-organize towards a critical state have been proposed, the relation between criticality signatures and learning is still unclear. Here, we investigate signatures of criticality in a self-organizing recurrent neural network (SORN). Investigating criticality in the SORN is of particular interest because it has not been developed to show criticality. Instead, the SORN has been shown to exhibit spatio-temporal pattern learning through a combination of neural plasticity mechanisms and it reproduces a number of biological findings on neural variability and the statistics and fluctuations of synaptic efficacies. We show that, after a transient, the SORN spontaneously self-organizes into a dynamical state that shows criticality signatures comparable to those found in experiments. The plasticity mechanisms are necessary to attain that dynamical state, but not to maintain it. Furthermore, onset of external input transiently changes the slope of the avalanche distributions - matching recent experimental findings. Interestingly, the membrane noise level necessary for the occurrence of the criticality signatures reduces the model's performance in simple learning tasks. Overall, our work shows that the biologically inspired plasticity and homeostasis mechanisms responsible for the SORN's spatio-temporal learning abilities can give rise to criticality signatures in its activity when driven by random input, but these break down under the structured input of short repeating sequences.
Impacts of single and recurrent wildfires on topsoil moisture regime
González-Pelayo, Oscar; Malvar, Maruxa; van den Elsen, Erik; Hosseini, Mohammadreza; Coelho, Celeste; Ritsema, Coen; Bautista, Susana; Keizer, Jacob
2017-04-01
The increasing fire recurrence on forest in the Mediterranean basin is well-established by future climate scenarios due to land use changes and climate predictions. By this, shifts on mature pine woodlands to shrub rangelands are of major importance on forest ecosystems buffer functions, since historical patterns of established vegetation help to recover from fire disturbances. This fact, together with the predicted expansion of the drought periods, will affect feedback processes of vegetation patterns since water availability on these seasons are driven by post-fire local soil properties. Although fire impacts of soil properties and water availability has been widely studied using the fire severity as the main factor, little research is developed on post-fire soil moisture patterns, including the fire recurrence as a key explanatory variable. The following research investigated, in pine woodlands of north central Portugal, the short-term consequences (one year after a fire) of wildfire recurrence on the surface soil moisture content (SMC) and on effective soil water (SWEFF, parameter that includes actual daily soil moisture, soil field capacity-FC and permanent wilting point-PWP). The study set-up includes analyses at two fire recurrence scenarios (1x- and 4x-burnt since 1975), at a patch level (shrub patch/interpatch) and at two soil depths (2.5 and 7.5 cm) in a nested approach. Understanding how fire recurrence affects water in soil over space and time is the main goal of this research. The use of soil moisture sensors in a nested approach, the rainfall features and analyses on basic soil properties as soil organic matter, texture, bulk density, pF curves, soil water repellency and soil surface components will establish which factors has the largest role in controlling soil moisture behavior. Main results displayed, in a seasonal and yearly basis, no differences on SMC as increasing fire recurrence (1x- vs 4x-burnt) neither between patch/interpatch microsites at
Using LSTM recurrent neural networks for monitoring the LHC superconducting magnets
Wielgosz, Maciej; Mertik, Matej
2017-09-21
The superconducting LHC magnets are coupled with an electronic monitoring system which records and analyzes voltage time series reflecting their performance. A currently used system is based on a range of preprogrammed triggers which launches protection procedures when a misbehavior of the magnets is detected. All the procedures used in the protection equipment were designed and implemented according to known working scenarios of the system and are updated and monitored by human operators. This paper proposes a novel approach to monitoring and fault protection of the Large Hadron Collider (LHC) superconducting magnets which employs state-of-the-art Deep Learning algorithms. Consequently, the authors of the paper decided to examine the performance of LSTM recurrent neural networks for modeling of voltage time series of the magnets. In order to address this challenging task different network architectures and hyper-parameters were used to achieve the best possible performance of the solution. The regression re...
Shahnazian, Danesh; Holroyd, Clay B
2018-02-01
Anterior cingulate cortex (ACC) has been the subject of intense debate over the past 2 decades, but its specific computational function remains controversial. Here we present a simple computational model of ACC that incorporates distributed representations across a network of interconnected processing units. Based on the proposal that ACC is concerned with the execution of extended, goal-directed action sequences, we trained a recurrent neural network to predict each successive step of several sequences associated with multiple tasks. In keeping with neurophysiological observations from nonhuman animals, the network yields distributed patterns of activity across ACC neurons that track the progression of each sequence, and in keeping with human neuroimaging data, the network produces discrepancy signals when any step of the sequence deviates from the predicted step. These simulations illustrate a novel approach for investigating ACC function.
Identification of Jets Containing $b$-Hadrons with Recurrent Neural Networks at the ATLAS Experiment
The ATLAS collaboration
2017-01-01
A novel $b$-jet identification algorithm is constructed with a Recurrent Neural Network (RNN) at the ATLAS experiment at the CERN Large Hadron Collider. The RNN based $b$-tagging algorithm processes charged particle tracks associated to jets without reliance on secondary vertex finding, and can augment existing secondary-vertex based taggers. In contrast to traditional impact-parameter-based $b$-tagging algorithms which assume that tracks associated to jets are independent from each other, the RNN based $b$-tagging algorithm can exploit the spatial and kinematic correlations between tracks which are initiated from the same $b$-hadrons. This new approach also accommodates an extended set of input variables. This note presents the expected performance of the RNN based $b$-tagging algorithm in simulated $t \\bar t$ events at $\\sqrt{s}=13$ TeV.
Automatic temporal segment detection via bilateral long short-term memory recurrent neural networks
Sun, Bo; Cao, Siming; He, Jun; Yu, Lejun; Li, Liandong
2017-03-01
Constrained by the physiology, the temporal factors associated with human behavior, irrespective of facial movement or body gesture, are described by four phases: neutral, onset, apex, and offset. Although they may benefit related recognition tasks, it is not easy to accurately detect such temporal segments. An automatic temporal segment detection framework using bilateral long short-term memory recurrent neural networks (BLSTM-RNN) to learn high-level temporal-spatial features, which synthesizes the local and global temporal-spatial information more efficiently, is presented. The framework is evaluated in detail over the face and body database (FABO). The comparison shows that the proposed framework outperforms state-of-the-art methods for solving the problem of temporal segment detection.
Han, Seong-Ik; Lee, Jang-Myung
2014-01-01
This paper proposes a backstepping control system that uses a tracking error constraint and recurrent fuzzy neural networks (RFNNs) to achieve a prescribed tracking performance for a strict-feedback nonlinear dynamic system. A new constraint variable was defined to generate the virtual control that forces the tracking error to fall within prescribed boundaries. An adaptive RFNN was also used to obtain the required improvement on the approximation performances in order to avoid calculating the explosive number of terms generated by the recursive steps of traditional backstepping control. The boundedness and convergence of the closed-loop system was confirmed based on the Lyapunov stability theory. The prescribed performance of the proposed control scheme was validated by using it to control the prescribed error of a nonlinear system and a robot manipulator. © 2013 ISA. Published by Elsevier Ltd. All rights reserved.
International Nuclear Information System (INIS)
Kim, Han Me; Kim, Jong Shik; Han, Seong Ik
2009-01-01
To improve position tracking performance of servo systems, a position tracking control using adaptive back-stepping control(ABSC) scheme and recurrent fuzzy neural networks(RFNN) is proposed. An adaptive rule of the ABSC based on system dynamics and dynamic friction model is also suggested to compensate nonlinear dynamic friction characteristics. However, it is difficult to reduce the position tracking error of servo systems by using only the ABSC scheme because of the system uncertainties which cannot be exactly identified during the modeling of servo systems. Therefore, in order to overcome system uncertainties and then to improve position tracking performance of servo systems, the RFNN technique is additionally applied to the servo system. The feasibility of the proposed control scheme for a servo system is validated through experiments. Experimental results show that the servo system with ABS controller based on the dual friction observer and RFNN including the reconstruction error estimator can achieve desired tracking performance and robustness
Mizusaki, Beatriz E. P.; Agnes, Everton J.; Erichsen, Rubem; Brunnet, Leonardo G.
2017-08-01
The plastic character of brain synapses is considered to be one of the foundations for the formation of memories. There are numerous kinds of such phenomenon currently described in the literature, but their role in the development of information pathways in neural networks with recurrent architectures is still not completely clear. In this paper we study the role of an activity-based process, called pre-synaptic dependent homeostatic scaling, in the organization of networks that yield precise-timed spiking patterns. It encodes spatio-temporal information in the synaptic weights as it associates a learned input with a specific response. We introduce a correlation measure to evaluate the precision of the spiking patterns and explore the effects of different inhibitory interactions and learning parameters. We find that large learning periods are important in order to improve the network learning capacity and discuss this ability in the presence of distinct inhibitory currents.
Robust passivity analysis for discrete-time recurrent neural networks with mixed delays
Huang, Chuan-Kuei; Shu, Yu-Jeng; Chang, Koan-Yuh; Shou, Ho-Nien; Lu, Chien-Yu
2015-02-01
This article considers the robust passivity analysis for a class of discrete-time recurrent neural networks (DRNNs) with mixed time-delays and uncertain parameters. The mixed time-delays that consist of both the discrete time-varying and distributed time-delays in a given range are presented, and the uncertain parameters are norm-bounded. The activation functions are assumed to be globally Lipschitz continuous. Based on new bounding technique and appropriate type of Lyapunov functional, a sufficient condition is investigated to guarantee the existence of the desired robust passivity condition for the DRNNs, which can be derived in terms of a family of linear matrix inequality (LMI). Some free-weighting matrices are introduced to reduce the conservatism of the criterion by using the bounding technique. A numerical example is given to illustrate the effectiveness and applicability.
Passivity and passification of memristor-based recurrent neural networks with time-varying delays.
Guo, Zhenyuan; Wang, Jun; Yan, Zheng
2014-11-01
This paper presents new theoretical results on the passivity and passification of a class of memristor-based recurrent neural networks (MRNNs) with time-varying delays. The casual assumptions on the boundedness and Lipschitz continuity of neuronal activation functions are relaxed. By constructing appropriate Lyapunov-Krasovskii functionals and using the characteristic function technique, passivity conditions are cast in the form of linear matrix inequalities (LMIs), which can be checked numerically using an LMI toolbox. Based on these conditions, two procedures for designing passification controllers are proposed, which guarantee that MRNNs with time-varying delays are passive. Finally, two illustrative examples are presented to show the characteristics of the main results in detail.
Generating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks.
Segler, Marwin H S; Kogej, Thierry; Tyrchan, Christian; Waller, Mark P
2018-01-24
In de novo drug design, computational strategies are used to generate novel molecules with good affinity to the desired biological target. In this work, we show that recurrent neural networks can be trained as generative models for molecular structures, similar to statistical language models in natural language processing. We demonstrate that the properties of the generated molecules correlate very well with the properties of the molecules used to train the model. In order to enrich libraries with molecules active toward a given biological target, we propose to fine-tune the model with small sets of molecules, which are known to be active against that target. Against Staphylococcus aureus , the model reproduced 14% of 6051 hold-out test molecules that medicinal chemists designed, whereas against Plasmodium falciparum (Malaria), it reproduced 28% of 1240 test molecules. When coupled with a scoring function, our model can perform the complete de novo drug design cycle to generate large sets of novel molecules for drug discovery.
Discrete-time recurrent neural networks with time-varying delays: Exponential stability analysis
Liu, Yurong; Wang, Zidong; Serrano, Alan; Liu, Xiaohui
2007-03-01
This Letter is concerned with the analysis problem of exponential stability for a class of discrete-time recurrent neural networks (DRNNs) with time delays. The delay is of the time-varying nature, and the activation functions are assumed to be neither differentiable nor strict monotonic. Furthermore, the description of the activation functions is more general than the recently commonly used Lipschitz conditions. Under such mild conditions, we first prove the existence of the equilibrium point. Then, by employing a Lyapunov Krasovskii functional, a unified linear matrix inequality (LMI) approach is developed to establish sufficient conditions for the DRNNs to be globally exponentially stable. It is shown that the delayed DRNNs are globally exponentially stable if a certain LMI is solvable, where the feasibility of such an LMI can be easily checked by using the numerically efficient Matlab LMI Toolbox. A simulation example is presented to show the usefulness of the derived LMI-based stability condition.
Discrete-time recurrent neural networks with time-varying delays: Exponential stability analysis
International Nuclear Information System (INIS)
Liu, Yurong; Wang, Zidong; Serrano, Alan; Liu, Xiaohui
2007-01-01
This Letter is concerned with the analysis problem of exponential stability for a class of discrete-time recurrent neural networks (DRNNs) with time delays. The delay is of the time-varying nature, and the activation functions are assumed to be neither differentiable nor strict monotonic. Furthermore, the description of the activation functions is more general than the recently commonly used Lipschitz conditions. Under such mild conditions, we first prove the existence of the equilibrium point. Then, by employing a Lyapunov-Krasovskii functional, a unified linear matrix inequality (LMI) approach is developed to establish sufficient conditions for the DRNNs to be globally exponentially stable. It is shown that the delayed DRNNs are globally exponentially stable if a certain LMI is solvable, where the feasibility of such an LMI can be easily checked by using the numerically efficient Matlab LMI Toolbox. A simulation example is presented to show the usefulness of the derived LMI-based stability condition
Neural processing of short-term recurrence in songbird vocal communication.
Directory of Open Access Journals (Sweden)
Gabriël J L Beckers
Full Text Available BACKGROUND: Many situations involving animal communication are dominated by recurring, stereotyped signals. How do receivers optimally distinguish between frequently recurring signals and novel ones? Cortical auditory systems are known to be pre-attentively sensitive to short-term delivery statistics of artificial stimuli, but it is unknown if this phenomenon extends to the level of behaviorally relevant delivery patterns, such as those used during communication. METHODOLOGY/PRINCIPAL FINDINGS: We recorded and analyzed complete auditory scenes of spontaneously communicating zebra finch (Taeniopygia guttata pairs over a week-long period, and show that they can produce tens of thousands of short-range contact calls per day. Individual calls recur at time scales (median interval 1.5 s matching those at which mammalian sensory systems are sensitive to recent stimulus history. Next, we presented to anesthetized birds sequences of frequently recurring calls interspersed with rare ones, and recorded, in parallel, action and local field potential responses in the medio-caudal auditory forebrain at 32 unique sites. Variation in call recurrence rate over natural ranges leads to widespread and significant modulation in strength of neural responses. Such modulation is highly call-specific in secondary auditory areas, but not in the main thalamo-recipient, primary auditory area. CONCLUSIONS/SIGNIFICANCE: Our results support the hypothesis that pre-attentive neural sensitivity to short-term stimulus recurrence is involved in the analysis of auditory scenes at the level of delivery patterns of meaningful sounds. This may enable birds to efficiently and automatically distinguish frequently recurring vocalizations from other events in their auditory scene.
Using recurrent neural network models for early detection of heart failure onset.
Choi, Edward; Schuetz, Andy; Stewart, Walter F; Sun, Jimeng
2017-03-01
We explored whether use of deep learning to model temporal relations among events in electronic health records (EHRs) would improve model performance in predicting initial diagnosis of heart failure (HF) compared to conventional methods that ignore temporality. Data were from a health system's EHR on 3884 incident HF cases and 28 903 controls, identified as primary care patients, between May 16, 2000, and May 23, 2013. Recurrent neural network (RNN) models using gated recurrent units (GRUs) were adapted to detect relations among time-stamped events (eg, disease diagnosis, medication orders, procedure orders, etc.) with a 12- to 18-month observation window of cases and controls. Model performance metrics were compared to regularized logistic regression, neural network, support vector machine, and K-nearest neighbor classifier approaches. Using a 12-month observation window, the area under the curve (AUC) for the RNN model was 0.777, compared to AUCs for logistic regression (0.747), multilayer perceptron (MLP) with 1 hidden layer (0.765), support vector machine (SVM) (0.743), and K-nearest neighbor (KNN) (0.730). When using an 18-month observation window, the AUC for the RNN model increased to 0.883 and was significantly higher than the 0.834 AUC for the best of the baseline methods (MLP). Deep learning models adapted to leverage temporal relations appear to improve performance of models for detection of incident heart failure with a short observation window of 12-18 months. © The Author 2016. Published by Oxford University Press on behalf of the American Medical Informatics Association.
Design of a heart rate controller for treadmill exercise using a recurrent fuzzy neural network.
Lu, Chun-Hao; Wang, Wei-Cheng; Tai, Cheng-Chi; Chen, Tien-Chi
2016-05-01
In this study, we developed a computer controlled treadmill system using a recurrent fuzzy neural network heart rate controller (RFNNHRC). Treadmill speeds and inclines were controlled by corresponding control servo motors. The RFNNHRC was used to generate the control signals to automatically control treadmill speed and incline to minimize the user heart rate deviations from a preset profile. The RFNNHRC combines a fuzzy reasoning capability to accommodate uncertain information and an artificial recurrent neural network learning process that corrects for treadmill system nonlinearities and uncertainties. Treadmill speeds and inclines are controlled by the RFNNHRC to achieve minimal heart rate deviation from a pre-set profile using adjustable parameters and an on-line learning algorithm that provides robust performance against parameter variations. The on-line learning algorithm of RFNNHRC was developed and implemented using a dsPIC 30F4011 DSP. Application of the proposed control scheme to heart rate responses of runners resulted in smaller fluctuations than those produced by using proportional integra control, and treadmill speeds and inclines were smoother. The present experiments demonstrate improved heart rate tracking performance with the proposed control scheme. The RFNNHRC scheme with adjustable parameters and an on-line learning algorithm was applied to a computer controlled treadmill system with heart rate control during treadmill exercise. Novel RFNNHRC structure and controller stability analyses were introduced. The RFNNHRC were tuned using a Lyapunov function to ensure system stability. The superior heart rate control with the proposed RFNNHRC scheme was demonstrated with various pre-set heart rates. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.
Fault diagnosis of rolling bearings with recurrent neural network-based autoencoders.
Liu, Han; Zhou, Jianzhong; Zheng, Yang; Jiang, Wei; Zhang, Yuncheng
2018-04-19
As the rolling bearings being the key part of rotary machine, its healthy condition is quite important for safety production. Fault diagnosis of rolling bearing has been research focus for the sake of improving the economic efficiency and guaranteeing the operation security. However, the collected signals are mixed with ambient noise during the operation of rotary machine, which brings great challenge to the exact diagnosis results. Using signals collected from multiple sensors can avoid the loss of local information and extract more helpful characteristics. Recurrent Neural Networks (RNN) is a type of artificial neural network which can deal with multiple time sequence data. The capacity of RNN has been proved outstanding for catching time relevance about time sequence data. This paper proposed a novel method for bearing fault diagnosis with RNN in the form of an autoencoder. In this approach, multiple vibration value of the rolling bearings of the next period are predicted from the previous period by means of Gated Recurrent Unit (GRU)-based denoising autoencoder. These GRU-based non-linear predictive denoising autoencoders (GRU-NP-DAEs) are trained with strong generalization ability for each different fault pattern. Then for the given input data, the reconstruction errors between the next period data and the output data generated by different GRU-NP-DAEs are used to detect anomalous conditions and classify fault type. Classic rotating machinery datasets have been employed to testify the effectiveness of the proposed diagnosis method and its preponderance over some state-of-the-art methods. The experiment results indicate that the proposed method achieves satisfactory performance with strong robustness and high classification accuracy. Copyright © 2018 ISA. Published by Elsevier Ltd. All rights reserved.
Recurrence of Intravenous Talc Granulomatosis following Single Lung Transplantation
Directory of Open Access Journals (Sweden)
Richard C Cook
1998-01-01
Full Text Available Advanced pulmonary disease is an unusual consequence of the intravenous injection of oral medications, usually developing over a period of several years. A number of patients with this condition have undergone lung transplantation for respiratory failure. However, a history of drug abuse is often considered to be a contraindication to transplantation in the context of limited donor resources. A patient with pulmonary talc granulomatosis secondary to intravenous methylphenidate injection who underwent successful lung transplantation and subsequently presented with recurrence of the underlying disease in the transplanted lung 18 months after transplantation is reported.
DEFF Research Database (Denmark)
Capaday, Charles; Ethier, C; Brizzi, L
2009-01-01
Capaday C, Ethier C, Brizzi L, Sik A, van Vreeswijk C, Gingras D. On the nature of the intrinsic connectivity of the cat motor cortex: evidence for a recurrent neural network topology. J Neurophysiol 102: 2131-2141, 2009. First published July 22, 2009; doi: 10.1152/jn.91319.2008. The details...
Interpretable deep neural networks for single-trial EEG classification.
Sturm, Irene; Lapuschkin, Sebastian; Samek, Wojciech; Müller, Klaus-Robert
2016-12-01
In cognitive neuroscience the potential of deep neural networks (DNNs) for solving complex classification tasks is yet to be fully exploited. The most limiting factor is that DNNs as notorious 'black boxes' do not provide insight into neurophysiological phenomena underlying a decision. Layer-wise relevance propagation (LRP) has been introduced as a novel method to explain individual network decisions. We propose the application of DNNs with LRP for the first time for EEG data analysis. Through LRP the single-trial DNN decisions are transformed into heatmaps indicating each data point's relevance for the outcome of the decision. DNN achieves classification accuracies comparable to those of CSP-LDA. In subjects with low performance subject-to-subject transfer of trained DNNs can improve the results. The single-trial LRP heatmaps reveal neurophysiologically plausible patterns, resembling CSP-derived scalp maps. Critically, while CSP patterns represent class-wise aggregated information, LRP heatmaps pinpoint neural patterns to single time points in single trials. We compare the classification performance of DNNs to that of linear CSP-LDA on two data sets related to motor-imaginary BCI. We have demonstrated that DNN is a powerful non-linear tool for EEG analysis. With LRP a new quality of high-resolution assessment of neural activity can be reached. LRP is a potential remedy for the lack of interpretability of DNNs that has limited their utility in neuroscientific applications. The extreme specificity of the LRP-derived heatmaps opens up new avenues for investigating neural activity underlying complex perception or decision-related processes. Copyright © 2016 Elsevier B.V. All rights reserved.
International Nuclear Information System (INIS)
Boroushaki, M.; Ghofrani, M.B.; Lucas, C.; Yazdanpanah, M.J.
2003-01-01
In the last decade, the intelligent control community has paid great attention to the topic of intelligent control systems for nuclear plants (core, steam generator...). Papers mostly used approximate and simple mathematical SISO (single-input-single-output) model of nuclear plants for testing and/or tuning of the control systems. They also tried to generalize theses models to a real MIMO (multi-input-multi-output) plant, while nuclear plants are typically of complex nonlinear and multivariable nature with high interactions between their state variables and therefore, many of these proposed intelligent control systems are not appropriate for real cases. In this paper, we designed an on-line intelligent core controller for load following operations, based on a heuristic control algorithm, using a valid and updatable recurrent neural network (RNN). We have used an accurate 3-dimensional core calculation code to represent the real plant and to train the RNN. The results of simulation show that this intelligent controller can control the reactor core during load following operations, using optimum control rod groups manoeuvre and variable overlapping strategy. This methodology represents a simple and reliable procedure for controlling other complex nonlinear MIMO plants, and may improve the responses, comparing to other control systems
Deep Recurrent Neural Networks for seizure detection and early seizure detection systems
Energy Technology Data Exchange (ETDEWEB)
Talathi, S. S. [Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
2017-06-05
Epilepsy is common neurological diseases, affecting about 0.6-0.8 % of world population. Epileptic patients suffer from chronic unprovoked seizures, which can result in broad spectrum of debilitating medical and social consequences. Since seizures, in general, occur infrequently and are unpredictable, automated seizure detection systems are recommended to screen for seizures during long-term electroencephalogram (EEG) recordings. In addition, systems for early seizure detection can lead to the development of new types of intervention systems that are designed to control or shorten the duration of seizure events. In this article, we investigate the utility of recurrent neural networks (RNNs) in designing seizure detection and early seizure detection systems. We propose a deep learning framework via the use of Gated Recurrent Unit (GRU) RNNs for seizure detection. We use publicly available data in order to evaluate our method and demonstrate very promising evaluation results with overall accuracy close to 100 %. We also systematically investigate the application of our method for early seizure warning systems. Our method can detect about 98% of seizure events within the first 5 seconds of the overall epileptic seizure duration.
Marginally Stable Triangular Recurrent Neural Network Architecture for Time Series Prediction.
Sivakumar, Seshadri; Sivakumar, Shyamala
2017-09-25
This paper introduces a discrete-time recurrent neural network architecture using triangular feedback weight matrices that allows a simplified approach to ensuring network and training stability. The triangular structure of the weight matrices is exploited to readily ensure that the eigenvalues of the feedback weight matrix represented by the block diagonal elements lie on the unit circle in the complex z-plane by updating these weights based on the differential of the angular error variable. Such placement of the eigenvalues together with the extended close interaction between state variables facilitated by the nondiagonal triangular elements, enhances the learning ability of the proposed architecture. Simulation results show that the proposed architecture is highly effective in time-series prediction tasks associated with nonlinear and chaotic dynamic systems with underlying oscillatory modes. This modular architecture with dual upper and lower triangular feedback weight matrices mimics fully recurrent network architectures, while maintaining learning stability with a simplified training process. While training, the block-diagonal weights (hence the eigenvalues) of the dual triangular matrices are constrained to the same values during weight updates aimed at minimizing the possibility of overfitting. The dual triangular architecture also exploits the benefit of parsing the input and selectively applying the parsed inputs to the two subnetworks to facilitate enhanced learning performance.
El-Nagar, Ahmad M
2018-01-01
In this study, a novel structure of a recurrent interval type-2 Takagi-Sugeno-Kang (TSK) fuzzy neural network (FNN) is introduced for nonlinear dynamic and time-varying systems identification. It combines the type-2 fuzzy sets (T2FSs) and a recurrent FNN to avoid the data uncertainties. The fuzzy firing strengths in the proposed structure are returned to the network input as internal variables. The interval type-2 fuzzy sets (IT2FSs) is used to describe the antecedent part for each rule while the consequent part is a TSK-type, which is a linear function of the internal variables and the external inputs with interval weights. All the type-2 fuzzy rules for the proposed RIT2TSKFNN are learned on-line based on structure and parameter learning, which are performed using the type-2 fuzzy clustering. The antecedent and consequent parameters of the proposed RIT2TSKFNN are updated based on the Lyapunov function to achieve network stability. The obtained results indicate that our proposed network has a small root mean square error (RMSE) and a small integral of square error (ISE) with a small number of rules and a small computation time compared with other type-2 FNNs. Copyright © 2017 ISA. Published by Elsevier Ltd. All rights reserved.
Using Long-Short-Term-Memory Recurrent Neural Networks to Predict Aviation Engine Vibrations
ElSaid, AbdElRahman Ahmed
This thesis examines building viable Recurrent Neural Networks (RNN) using Long Short Term Memory (LSTM) neurons to predict aircraft engine vibrations. The different networks are trained on a large database of flight data records obtained from an airline containing flights that suffered from excessive vibration. RNNs can provide a more generalizable and robust method for prediction over analytical calculations of engine vibration, as analytical calculations must be solved iteratively based on specific empirical engine parameters, and this database contains multiple types of engines. Further, LSTM RNNs provide a "memory" of the contribution of previous time series data which can further improve predictions of future vibration values. LSTM RNNs were used over traditional RNNs, as those suffer from vanishing/exploding gradients when trained with back propagation. The study managed to predict vibration values for 1, 5, 10, and 20 seconds in the future, with 2.84% 3.3%, 5.51% and 10.19% mean absolute error, respectively. These neural networks provide a promising means for the future development of warning systems so that suitable actions can be taken before the occurrence of excess vibration to avoid unfavorable situations during flight.
Nonlinear dynamics analysis of a self-organizing recurrent neural network: chaos waning.
Eser, Jürgen; Zheng, Pengsheng; Triesch, Jochen
2014-01-01
Self-organization is thought to play an important role in structuring nervous systems. It frequently arises as a consequence of plasticity mechanisms in neural networks: connectivity determines network dynamics which in turn feed back on network structure through various forms of plasticity. Recently, self-organizing recurrent neural network models (SORNs) have been shown to learn non-trivial structure in their inputs and to reproduce the experimentally observed statistics and fluctuations of synaptic connection strengths in cortex and hippocampus. However, the dynamics in these networks and how they change with network evolution are still poorly understood. Here we investigate the degree of chaos in SORNs by studying how the networks' self-organization changes their response to small perturbations. We study the effect of perturbations to the excitatory-to-excitatory weight matrix on connection strengths and on unit activities. We find that the network dynamics, characterized by an estimate of the maximum Lyapunov exponent, becomes less chaotic during its self-organization, developing into a regime where only few perturbations become amplified. We also find that due to the mixing of discrete and (quasi-)continuous variables in SORNs, small perturbations to the synaptic weights may become amplified only after a substantial delay, a phenomenon we propose to call deferred chaos.
Exponential stability of delayed recurrent neural networks with Markovian jumping parameters
International Nuclear Information System (INIS)
Wang Zidong; Liu Yurong; Yu Li; Liu Xiaohui
2006-01-01
In this Letter, the global exponential stability analysis problem is considered for a class of recurrent neural networks (RNNs) with time delays and Markovian jumping parameters. The jumping parameters considered here are generated from a continuous-time discrete-state homogeneous Markov process, which are governed by a Markov process with discrete and finite state space. The purpose of the problem addressed is to derive some easy-to-test conditions such that the dynamics of the neural network is stochastically exponentially stable in the mean square, independent of the time delay. By employing a new Lyapunov-Krasovskii functional, a linear matrix inequality (LMI) approach is developed to establish the desired sufficient conditions, and therefore the global exponential stability in the mean square for the delayed RNNs can be easily checked by utilizing the numerically efficient Matlab LMI toolbox, and no tuning of parameters is required. A numerical example is exploited to show the usefulness of the derived LMI-based stability conditions
Rakkiyappan, Rajan; Chandrasekar, Arunachalam; Cao, Jinde
2015-09-01
This paper presents a new design scheme for the passivity and passification of a class of memristor-based recurrent neural networks (MRNNs) with additive time-varying delays. The predictable assumptions on the boundedness and Lipschitz continuity of activation functions are formulated. The systems considered here are based on a different time-delay model suggested recently, which includes additive time-varying delay components in the state. The connection between the time-varying delay and its upper bound is considered when estimating the upper bound of the derivative of Lyapunov functional. It is recognized that the passivity condition can be expressed in a linear matrix inequality (LMI) format and by using characteristic function method. For state feedback passification, it is verified that it is apathetic to use immediate or delayed state feedback. By constructing a Lyapunov-Krasovskii functional and employing Jensen's inequality and reciprocal convex combination technique together with a tighter estimation of the upper bound of the cross-product terms derived from the derivatives of the Lyapunov functional, less conventional delay-dependent passivity criteria are established in terms of LMIs. Moreover, second-order reciprocally convex approach is employed for deriving the upper bound for terms with inverses of squared convex parameters. The model based on the memristor with additive time-varying delays widens the application scope for the design of neural networks. Finally, pertinent examples are given to show the advantages of the derived passivity criteria and the significant improvement of the theoretical approaches.
Exponential stability of delayed recurrent neural networks with Markovian jumping parameters
Wang, Zidong; Liu, Yurong; Yu, Li; Liu, Xiaohui
2006-08-01
In this Letter, the global exponential stability analysis problem is considered for a class of recurrent neural networks (RNNs) with time delays and Markovian jumping parameters. The jumping parameters considered here are generated from a continuous-time discrete-state homogeneous Markov process, which are governed by a Markov process with discrete and finite state space. The purpose of the problem addressed is to derive some easy-to-test conditions such that the dynamics of the neural network is stochastically exponentially stable in the mean square, independent of the time delay. By employing a new Lyapunov Krasovskii functional, a linear matrix inequality (LMI) approach is developed to establish the desired sufficient conditions, and therefore the global exponential stability in the mean square for the delayed RNNs can be easily checked by utilizing the numerically efficient Matlab LMI toolbox, and no tuning of parameters is required. A numerical example is exploited to show the usefulness of the derived LMI-based stability conditions.
International Nuclear Information System (INIS)
Sabahi, Kamel; Teshnehlab, Mohammad; Shoorhedeli, Mahdi Aliyari
2009-01-01
In this study, a new adaptive controller based on modified feedback error learning (FEL) approaches is proposed for load frequency control (LFC) problem. The FEL strategy consists of intelligent and conventional controllers in feedforward and feedback paths, respectively. In this strategy, a conventional feedback controller (CFC), i.e. proportional, integral and derivative (PID) controller, is essential to guarantee global asymptotic stability of the overall system; and an intelligent feedforward controller (INFC) is adopted to learn the inverse of the controlled system. Therefore, when the INFC learns the inverse of controlled system, the tracking of reference signal is done properly. Generally, the CFC is designed at nominal operating conditions of the system and, therefore, fails to provide the best control performance as well as global stability over a wide range of changes in the operating conditions of the system. So, in this study a supervised controller (SC), a lookup table based controller, is addressed for tuning of the CFC. During abrupt changes of the power system parameters, the SC adjusts the PID parameters according to these operating conditions. Moreover, for improving the performance of overall system, a recurrent fuzzy neural network (RFNN) is adopted in INFC instead of the conventional neural network, which was used in past studies. The proposed FEL controller has been compared with the conventional feedback error learning controller (CFEL) and the PID controller through some performance indices
Directory of Open Access Journals (Sweden)
Xingang Fu
2016-04-01
Full Text Available This paper investigates a novel recurrent neural network (NN-based vector control approach for single-phase grid-connected converters (GCCs with L (inductor, LC (inductor-capacitor and LCL (inductor-capacitor-inductor filters and provides their comparison study with the conventional standard vector control method. A single neural network controller replaces two current-loop PI controllers, and the NN training approximates the optimal control for the single-phase GCC system. The Levenberg–Marquardt (LM algorithm was used to train the NN controller based on the complete system equations without any decoupling policies. The proposed NN approach can solve the decoupling problem associated with the conventional vector control methods for L, LC and LCL-filter-based single-phase GCCs. Both simulation study and hardware experiments demonstrate that the neural network vector controller shows much more improved performance than that of conventional vector controllers, including faster response speed and lower overshoot. Especially, NN vector control could achieve very good performance using low switch frequency. More importantly, the neural network vector controller is a damping free controller, which is generally required by a conventional vector controller for an LCL-filter-based single-phase grid-connected converter and, therefore, can overcome the inefficiency problem caused by damping policies.
Cheng, Long; Hou, Zeng-Guang; Lin, Yingzi; Tan, Min; Zhang, Wenjun Chris; Wu, Fang-Xiang
2011-05-01
A recurrent neural network is proposed for solving the non-smooth convex optimization problem with the convex inequality and linear equality constraints. Since the objective function and inequality constraints may not be smooth, the Clarke's generalized gradients of the objective function and inequality constraints are employed to describe the dynamics of the proposed neural network. It is proved that the equilibrium point set of the proposed neural network is equivalent to the optimal solution of the original optimization problem by using the Lagrangian saddle-point theorem. Under weak conditions, the proposed neural network is proved to be stable, and the state of the neural network is convergent to one of its equilibrium points. Compared with the existing neural network models for non-smooth optimization problems, the proposed neural network can deal with a larger class of constraints and is not based on the penalty method. Finally, the proposed neural network is used to solve the identification problem of genetic regulatory networks, which can be transformed into a non-smooth convex optimization problem. The simulation results show the satisfactory identification accuracy, which demonstrates the effectiveness and efficiency of the proposed approach.
Recurrent neural network based hybrid model for reconstructing gene regulatory network.
Raza, Khalid; Alam, Mansaf
2016-10-01
One of the exciting problems in systems biology research is to decipher how genome controls the development of complex biological system. The gene regulatory networks (GRNs) help in the identification of regulatory interactions between genes and offer fruitful information related to functional role of individual gene in a cellular system. Discovering GRNs lead to a wide range of applications, including identification of disease related pathways providing novel tentative drug targets, helps to predict disease response, and also assists in diagnosing various diseases including cancer. Reconstruction of GRNs from available biological data is still an open problem. This paper proposes a recurrent neural network (RNN) based model of GRN, hybridized with generalized extended Kalman filter for weight update in backpropagation through time training algorithm. The RNN is a complex neural network that gives a better settlement between biological closeness and mathematical flexibility to model GRN; and is also able to capture complex, non-linear and dynamic relationships among variables. Gene expression data are inherently noisy and Kalman filter performs well for estimation problem even in noisy data. Hence, we applied non-linear version of Kalman filter, known as generalized extended Kalman filter, for weight update during RNN training. The developed model has been tested on four benchmark networks such as DNA SOS repair network, IRMA network, and two synthetic networks from DREAM Challenge. We performed a comparison of our results with other state-of-the-art techniques which shows superiority of our proposed model. Further, 5% Gaussian noise has been induced in the dataset and result of the proposed model shows negligible effect of noise on results, demonstrating the noise tolerance capability of the model. Copyright © 2016 Elsevier Ltd. All rights reserved.
Lu, Wenlian; Zheng, Ren; Chen, Tianping
2016-03-01
In this paper, we discuss outer-synchronization of the asymmetrically connected recurrent time-varying neural networks. By using both centralized and decentralized discretization data sampling principles, we derive several sufficient conditions based on three vector norms to guarantee that the difference of any two trajectories starting from different initial values of the neural network converges to zero. The lower bounds of the common time intervals between data samples in centralized and decentralized principles are proved to be positive, which guarantees exclusion of Zeno behavior. A numerical example is provided to illustrate the efficiency of the theoretical results. Copyright © 2015 Elsevier Ltd. All rights reserved.
Construction of Gene Regulatory Networks Using Recurrent Neural Networks and Swarm Intelligence.
Khan, Abhinandan; Mandal, Sudip; Pal, Rajat Kumar; Saha, Goutam
2016-01-01
We have proposed a methodology for the reverse engineering of biologically plausible gene regulatory networks from temporal genetic expression data. We have used established information and the fundamental mathematical theory for this purpose. We have employed the Recurrent Neural Network formalism to extract the underlying dynamics present in the time series expression data accurately. We have introduced a new hybrid swarm intelligence framework for the accurate training of the model parameters. The proposed methodology has been first applied to a small artificial network, and the results obtained suggest that it can produce the best results available in the contemporary literature, to the best of our knowledge. Subsequently, we have implemented our proposed framework on experimental (in vivo) datasets. Finally, we have investigated two medium sized genetic networks (in silico) extracted from GeneNetWeaver, to understand how the proposed algorithm scales up with network size. Additionally, we have implemented our proposed algorithm with half the number of time points. The results indicate that a reduction of 50% in the number of time points does not have an effect on the accuracy of the proposed methodology significantly, with a maximum of just over 15% deterioration in the worst case.
Emergence of unstable itinerant orbits in a recurrent neural network model
International Nuclear Information System (INIS)
Suemitsu, Yoshikazu; Nara, Shigetoshi
2005-01-01
A recurrent neural network model with time delay is investigated by numerical methods. The model functions as both conventional associative memory and also enables us to embed a new kind of memory attractor that cannot be realized in models without time delay, for example chain-ring attractors. This is attributed to the fact that the time delay extends the available state space dimension. The difference between the basin structures of chain-ring attractors and of isolated cycle attractors is investigated with respect to the two attractor pattern sets, random memory patterns and designed memory patterns with intended structures. Compared to isolated attractors with random memory patterns, the basins of chain-ring attractors are reduced considerably. Computer experiments confirm that the basin volume of each embedded chain-ring attractor shrinks and the emergence of unstable itinerant orbits in the outer state space of the memory attractor basins is discovered. The instability of such itinerant orbits is investigated. Results show that a 1-bit difference in initial conditions does not exceed 10% of a total dimension within 100 updating steps
A recurrent neural network for classification of unevenly sampled variable stars
Naul, Brett; Bloom, Joshua S.; Pérez, Fernando; van der Walt, Stéfan
2018-02-01
Astronomical surveys of celestial sources produce streams of noisy time series measuring flux versus time (`light curves'). Unlike in many other physical domains, however, large (and source-specific) temporal gaps in data arise naturally due to intranight cadence choices as well as diurnal and seasonal constraints1-5. With nightly observations of millions of variable stars and transients from upcoming surveys4,6, efficient and accurate discovery and classification techniques on noisy, irregularly sampled data must be employed with minimal human-in-the-loop involvement. Machine learning for inference tasks on such data traditionally requires the laborious hand-coding of domain-specific numerical summaries of raw data (`features')7. Here, we present a novel unsupervised autoencoding recurrent neural network8 that makes explicit use of sampling times and known heteroskedastic noise properties. When trained on optical variable star catalogues, this network produces supervised classification models that rival other best-in-class approaches. We find that autoencoded features learned in one time-domain survey perform nearly as well when applied to another survey. These networks can continue to learn from new unlabelled observations and may be used in other unsupervised tasks, such as forecasting and anomaly detection.
Using LSTM recurrent neural networks for monitoring the LHC superconducting magnets
Wielgosz, Maciej; Skoczeń, Andrzej; Mertik, Matej
2017-09-01
The superconducting LHC magnets are coupled with an electronic monitoring system which records and analyzes voltage time series reflecting their performance. A currently used system is based on a range of preprogrammed triggers which launches protection procedures when a misbehavior of the magnets is detected. All the procedures used in the protection equipment were designed and implemented according to known working scenarios of the system and are updated and monitored by human operators. This paper proposes a novel approach to monitoring and fault protection of the Large Hadron Collider (LHC) superconducting magnets which employs state-of-the-art Deep Learning algorithms. Consequently, the authors of the paper decided to examine the performance of LSTM recurrent neural networks for modeling of voltage time series of the magnets. In order to address this challenging task different network architectures and hyper-parameters were used to achieve the best possible performance of the solution. The regression results were measured in terms of RMSE for different number of future steps and history length taken into account for the prediction. The best result of RMSE = 0 . 00104 was obtained for a network of 128 LSTM cells within the internal layer and 16 steps history buffer.
Application of Recurrent Neural Networks on El Nino Impact on California Climate
Le, J.; El-Askary, H. M.; Allai, M.
2017-12-01
Following our successful paper on the application for the El Nino season of 2015-2016 over Southern California, we use recurrent neural networks (RNNs) to investigate the complex interactions between the long-term trend in dryness and a projected, short but intense, period of wetness due to the 2015-2016 El Niño. Although it was forecasted that this El Niño season would bring significant rainfall to the region, our long-term projections of the Palmer Z Index (PZI) showed a continuing drought trend. We achieved a statistically significant correlation of 0.610 between forecasted and observed PZI on the validation set for a lead time of 1 month. This gives strong confidence to the forecasted precipitation indicator. These predictions were bourne out in the resulting data. This paper details the expansion of our system to the climate of the entire California climate as a whole, dealing with inter-relationships and spatial variations within the state.
Local community detection as pattern restoration by attractor dynamics of recurrent neural networks.
Okamoto, Hiroshi
2016-08-01
Densely connected parts in networks are referred to as "communities". Community structure is a hallmark of a variety of real-world networks. Individual communities in networks form functional modules of complex systems described by networks. Therefore, finding communities in networks is essential to approaching and understanding complex systems described by networks. In fact, network science has made a great deal of effort to develop effective and efficient methods for detecting communities in networks. Here we put forward a type of community detection, which has been little examined so far but will be practically useful. Suppose that we are given a set of source nodes that includes some (but not all) of "true" members of a particular community; suppose also that the set includes some nodes that are not the members of this community (i.e., "false" members of the community). We propose to detect the community from this "imperfect" and "inaccurate" set of source nodes using attractor dynamics of recurrent neural networks. Community detection by the proposed method can be viewed as restoration of the original pattern from a deteriorated pattern, which is analogous to cue-triggered recall of short-term memory in the brain. We demonstrate the effectiveness of the proposed method using synthetic networks and real social networks for which correct communities are known. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.
Chherawala, Youssouf; Roy, Partha Pratim; Cheriet, Mohamed
2016-12-01
The performance of handwriting recognition systems is dependent on the features extracted from the word image. A large body of features exists in the literature, but no method has yet been proposed to identify the most promising of these, other than a straightforward comparison based on the recognition rate. In this paper, we propose a framework for feature set evaluation based on a collaborative setting. We use a weighted vote combination of recurrent neural network (RNN) classifiers, each trained with a particular feature set. This combination is modeled in a probabilistic framework as a mixture model and two methods for weight estimation are described. The main contribution of this paper is to quantify the importance of feature sets through the combination weights, which reflect their strength and complementarity. We chose the RNN classifier because of its state-of-the-art performance. Also, we provide the first feature set benchmark for this classifier. We evaluated several feature sets on the IFN/ENIT and RIMES databases of Arabic and Latin script, respectively. The resulting combination model is competitive with state-of-the-art systems.
Applying long short-term memory recurrent neural networks to intrusion detection
Directory of Open Access Journals (Sweden)
Ralf C. Staudemeyer
2015-07-01
Full Text Available We claim that modelling network traffic as a time series with a supervised learning approach, using known genuine and malicious behaviour, improves intrusion detection. To substantiate this, we trained long short-term memory (LSTM recurrent neural networks with the training data provided by the DARPA / KDD Cup ’99 challenge. To identify suitable LSTM-RNN network parameters and structure we experimented with various network topologies. We found networks with four memory blocks containing two cells each offer a good compromise between computational cost and detection performance. We applied forget gates and shortcut connections respectively. A learning rate of 0.1 and up to 1,000 epochs showed good results. We tested the performance on all features and on extracted minimal feature sets respectively. We evaluated different feature sets for the detection of all attacks within one network and also to train networks specialised on individual attack classes. Our results show that the LSTM classifier provides superior performance in comparison to results previously published results of strong static classifiers. With 93.82% accuracy and 22.13 cost, LSTM outperforms the winning entries of the KDD Cup ’99 challenge by far. This is due to the fact that LSTM learns to look back in time and correlate consecutive connection records. For the first time ever, we have demonstrated the usefulness of LSTM networks to intrusion detection.
Wei, Qikang; Chen, Tao; Xu, Ruifeng; He, Yulan; Gui, Lin
2016-01-01
The recognition of disease and chemical named entities in scientific articles is a very important subtask in information extraction in the biomedical domain. Due to the diversity and complexity of disease names, the recognition of named entities of diseases is rather tougher than those of chemical names. Although there are some remarkable chemical named entity recognition systems available online such as ChemSpot and tmChem, the publicly available recognition systems of disease named entities are rare. This article presents a system for disease named entity recognition (DNER) and normalization. First, two separate DNER models are developed. One is based on conditional random fields model with a rule-based post-processing module. The other one is based on the bidirectional recurrent neural networks. Then the named entities recognized by each of the DNER model are fed into a support vector machine classifier for combining results. Finally, each recognized disease named entity is normalized to a medical subject heading disease name by using a vector space model based method. Experimental results show that using 1000 PubMed abstracts for training, our proposed system achieves an F1-measure of 0.8428 at the mention level and 0.7804 at the concept level, respectively, on the testing data of the chemical-disease relation task in BioCreative V.Database URL: http://219.223.252.210:8080/SS/cdr.html. © The Author(s) 2016. Published by Oxford University Press.
Mandal, Sudip; Saha, Goutam; Pal, Rajat Kumar
2017-08-01
Correct inference of genetic regulations inside a cell from the biological database like time series microarray data is one of the greatest challenges in post genomic era for biologists and researchers. Recurrent Neural Network (RNN) is one of the most popular and simple approach to model the dynamics as well as to infer correct dependencies among genes. Inspired by the behavior of social elephants, we propose a new metaheuristic namely Elephant Swarm Water Search Algorithm (ESWSA) to infer Gene Regulatory Network (GRN). This algorithm is mainly based on the water search strategy of intelligent and social elephants during drought, utilizing the different types of communication techniques. Initially, the algorithm is tested against benchmark small and medium scale artificial genetic networks without and with presence of different noise levels and the efficiency was observed in term of parametric error, minimum fitness value, execution time, accuracy of prediction of true regulation, etc. Next, the proposed algorithm is tested against the real time gene expression data of Escherichia Coli SOS Network and results were also compared with others state of the art optimization methods. The experimental results suggest that ESWSA is very efficient for GRN inference problem and performs better than other methods in many ways.
An analysis of noise in recurrent neural networks: convergence and generalization.
Jim, K C; Giles, C L; Horne, B G
1996-01-01
Concerns the effect of noise on the performance of feedforward neural nets. We introduce and analyze various methods of injecting synaptic noise into dynamically driven recurrent nets during training. Theoretical results show that applying a controlled amount of noise during training may improve convergence and generalization performance. We analyze the effects of various noise parameters and predict that best overall performance can be achieved by injecting additive noise at each time step. Noise contributes a second-order gradient term to the error function which can be viewed as an anticipatory agent to aid convergence. This term appears to find promising regions of weight space in the beginning stages of training when the training error is large and should improve convergence on error surfaces with local minima. The first-order term is a regularization term that can improve generalization. Specifically, it can encourage internal representations where the state nodes operate in the saturated regions of the sigmoid discriminant function. While this effect can improve performance on automata inference problems with binary inputs and target outputs, it is unclear what effect it will have on other types of problems. To substantiate these predictions, we present simulations on learning the dual parity grammar from temporal strings for all noise models, and present simulations on learning a randomly generated six-state grammar using the predicted best noise model.
Generating Focused Molecule Libraries for Drug Discovery with Recurrent Neural Networks
Directory of Open Access Journals (Sweden)
Marwin H. S. Segler
2017-12-01
Full Text Available In de novo drug design, computational strategies are used to generate novel molecules with good affinity to the desired biological target. In this work, we show that recurrent neural networks can be trained as generative models for molecular structures, similar to statistical language models in natural language processing. We demonstrate that the properties of the generated molecules correlate very well with the properties of the molecules used to train the model. In order to enrich libraries with molecules active toward a given biological target, we propose to fine-tune the model with small sets of molecules, which are known to be active against that target. Against Staphylococcus aureus, the model reproduced 14% of 6051 hold-out test molecules that medicinal chemists designed, whereas against Plasmodium falciparum (Malaria, it reproduced 28% of 1240 test molecules. When coupled with a scoring function, our model can perform the complete de novo drug design cycle to generate large sets of novel molecules for drug discovery.
International Nuclear Information System (INIS)
Song Qiankun
2008-01-01
In this paper, the global exponential periodicity and stability of recurrent neural networks with time-varying delays are investigated by applying the idea of vector Lyapunov function, M-matrix theory and inequality technique. We assume neither the global Lipschitz conditions on these activation functions nor the differentiability on these time-varying delays, which were needed in other papers. Several novel criteria are found to ascertain the existence, uniqueness and global exponential stability of periodic solution for recurrent neural network with time-varying delays. Moreover, the exponential convergence rate index is estimated, which depends on the system parameters. Some previous results are improved and generalized, and an example is given to show the effectiveness of our method
Dequaire, Julie; Rao, Dushyant; Ondruska, Peter; Wang, Dominic; Posner, Ingmar
2016-01-01
This paper presents an end-to-end approach for tracking static and dynamic objects for an autonomous vehicle driving through crowded urban environments. Unlike traditional approaches to tracking, this method is learned end-to-end, and is able to directly predict a full unoccluded occupancy grid map from raw laser input data. Inspired by the recently presented DeepTracking approach [Ondruska, 2016], we employ a recurrent neural network (RNN) to capture the temporal evolution of the state of th...
International Nuclear Information System (INIS)
Yan, Ji; Bao-Tong, Cui
2010-01-01
In this paper, we have improved delay-dependent stability criteria for recurrent neural networks with a delay varying over a range and Markovian jumping parameters. The criteria improve over some previous ones in that they have fewer matrix variables yet less conservatism. In addition, a numerical example is provided to illustrate the applicability of the result using the linear matrix inequality toolbox in MATLAB. (general)
Ji, Yan; Cui, Bao-Tong
2010-06-01
In this paper, we have improved delay-dependent stability criteria for recurrent neural networks with a delay varying over a range and Markovian jumping parameters. The criteria improve over some previous ones in that they have fewer matrix variables yet less conservatism. In addition, a numerical example is provided to illustrate the applicability of the result using the linear matrix inequality toolbox in MATLAB.
Directory of Open Access Journals (Sweden)
Tamara eTosic
2016-01-01
Full Text Available For decades, research in neuroscience has supported the hypothesis that brain dynamics exhibits recurrent metastable states connected by transients, which together encode fundamental neural information processing. To understand the system’s dynamics it is important to detect such recurrence domains, but it is challenging to extract them from experimental neuroscience datasets due to the large trial-to-trial variability. The proposed methodology extracts recurrent metastable states in univariate time series by transforming datasets into their time-frequency representations and computing recurrence plots based on instantaneous spectral power values in various frequency bands. Additionally, a new statistical inference analysis compares different trial recurrence plots with corresponding surrogates to obtain statistically significant recurrent structures. This combination of methods is validated by applying it to two artificial datasets. In a final study of visually-evoked Local Field Potentials in partially anesthetized ferrets, the methodology is able to reveal recurrence structures of neural responses with trial-to-trial variability. Focusing on different frequency bands, the delta-band activity is much less recurrent than alpha-band activity. Moreover, alpha-activity is susceptible to pre-stimuli, while delta-activity is much less sensitive to pre-stimuli. This difference in recurrence structures in different frequency bands indicates diverse underlying information processing steps in the brain.
Mandal, Sudip; Khan, Abhinandan; Saha, Goutam; Pal, Rajat K
2016-01-01
The accurate prediction of genetic networks using computational tools is one of the greatest challenges in the postgenomic era. Recurrent Neural Network is one of the most popular but simple approaches to model the network dynamics from time-series microarray data. To date, it has been successfully applied to computationally derive small-scale artificial and real-world genetic networks with high accuracy. However, they underperformed for large-scale genetic networks. Here, a new methodology has been proposed where a hybrid Cuckoo Search-Flower Pollination Algorithm has been implemented with Recurrent Neural Network. Cuckoo Search is used to search the best combination of regulators. Moreover, Flower Pollination Algorithm is applied to optimize the model parameters of the Recurrent Neural Network formalism. Initially, the proposed method is tested on a benchmark large-scale artificial network for both noiseless and noisy data. The results obtained show that the proposed methodology is capable of increasing the inference of correct regulations and decreasing false regulations to a high degree. Secondly, the proposed methodology has been validated against the real-world dataset of the DNA SOS repair network of Escherichia coli. However, the proposed method sacrifices computational time complexity in both cases due to the hybrid optimization process.
Panda, Priyadarshini; Roy, Kaushik
2017-01-01
Synaptic Plasticity, the foundation for learning and memory formation in the human brain, manifests in various forms. Here, we combine the standard spike timing correlation based Hebbian plasticity with a non-Hebbian synaptic decay mechanism for training a recurrent spiking neural model to generate sequences. We show that inclusion of the adaptive decay of synaptic weights with standard STDP helps learn stable contextual dependencies between temporal sequences, while reducing the strong attractor states that emerge in recurrent models due to feedback loops. Furthermore, we show that the combined learning scheme suppresses the chaotic activity in the recurrent model substantially, thereby enhancing its' ability to generate sequences consistently even in the presence of perturbations.
Hirabayashi, Yu; Okumura, Akihisa; Kondo, Taiki; Magota, Miyuki; Kawabe, Shinji; Kando, Naoyuki; Yamaguchi, Hideaki; Natsume, Jun; Negoro, Tamiko; Watanabe, Kazuyoshi
2009-06-01
To assess the efficacy of diazepam suppositories at preventing febrile seizure recurrence during a single febrile illness to determine how to treat children with a febrile seizure on presentation at the hospital. We studied 203 children with febrile seizures from December 2004 through March 2006. On admission between December 2004 and May 2005, a diazepam suppository was administered to the patients. Patients seen between June 2005 and March 2006 were not treated with antiepileptic drugs on admission. We saw a significant difference in the rate of recurrence of febrile seizures between children treated with diazepam and those who were not. Recurrences were observed in 2 (2.1%) of 95 children treated with diazepam and in 16 (14.8%) of 108 untreated children. For the 108 untreated patients, the median age was 22.8 months in those with recurrences and 30.6 months in those without, confirming that a younger age was related to a recurrence. A diazepam suppository after a febrile seizure will reduce the incidence of recurrent febrile seizures during the same febrile illness. However, a diazepam suppository after a febrile seizure should be used after carefully considering the benefits and potential adverse effects.
Zhou, Caigen; Zeng, Xiaoqin; Luo, Chaomin; Zhang, Huaguang
In this paper, local bipolar auto-associative memories are presented based on discrete recurrent neural networks with a class of gain type activation function. The weight parameters of neural networks are acquired by a set of inequalities without the learning procedure. The global exponential stability criteria are established to ensure the accuracy of the restored patterns by considering time delays and external inputs. The proposed methodology is capable of effectively overcoming spurious memory patterns and achieving memory capacity. The effectiveness, robustness, and fault-tolerant capability are validated by simulated experiments.In this paper, local bipolar auto-associative memories are presented based on discrete recurrent neural networks with a class of gain type activation function. The weight parameters of neural networks are acquired by a set of inequalities without the learning procedure. The global exponential stability criteria are established to ensure the accuracy of the restored patterns by considering time delays and external inputs. The proposed methodology is capable of effectively overcoming spurious memory patterns and achieving memory capacity. The effectiveness, robustness, and fault-tolerant capability are validated by simulated experiments.
Energy Technology Data Exchange (ETDEWEB)
Bodruzzaman, M.; Essawy, M.A.
1996-03-31
Chaotic systems are known for their unpredictability due to their sensitive dependence on initial conditions. When only time series measurements from such systems are available, neural network based models are preferred due to their simplicity, availability, and robustness. However, the type of neural network used should be capable of modeling the highly non-linear behavior and the multi- attractor nature of such systems. In this paper we use a special type of recurrent neural network called the ``Dynamic System Imitator (DSI)``, that has been proven to be capable of modeling very complex dynamic behaviors. The DSI is a fully recurrent neural network that is specially designed to model a wide variety of dynamic systems. The prediction method presented in this paper is based upon predicting one step ahead in the time series, and using that predicted value to iteratively predict the following steps. This method was applied to chaotic time series generated from the logistic, Henon, and the cubic equations, in addition to experimental pressure drop time series measured from a Fluidized Bed Reactor (FBR), which is known to exhibit chaotic behavior. The time behavior and state space attractor of the actual and network synthetic chaotic time series were analyzed and compared. The correlation dimension and the Kolmogorov entropy for both the original and network synthetic data were computed. They were found to resemble each other, confirming the success of the DSI based chaotic system modeling.
Kordmahalleh, Mina Moradi; Sefidmazgi, Mohammad Gorji; Harrison, Scott H; Homaifar, Abdollah
2017-01-01
The modeling of genetic interactions within a cell is crucial for a basic understanding of physiology and for applied areas such as drug design. Interactions in gene regulatory networks (GRNs) include effects of transcription factors, repressors, small metabolites, and microRNA species. In addition, the effects of regulatory interactions are not always simultaneous, but can occur after a finite time delay, or as a combined outcome of simultaneous and time delayed interactions. Powerful biotechnologies have been rapidly and successfully measuring levels of genetic expression to illuminate different states of biological systems. This has led to an ensuing challenge to improve the identification of specific regulatory mechanisms through regulatory network reconstructions. Solutions to this challenge will ultimately help to spur forward efforts based on the usage of regulatory network reconstructions in systems biology applications. We have developed a hierarchical recurrent neural network (HRNN) that identifies time-delayed gene interactions using time-course data. A customized genetic algorithm (GA) was used to optimize hierarchical connectivity of regulatory genes and a target gene. The proposed design provides a non-fully connected network with the flexibility of using recurrent connections inside the network. These features and the non-linearity of the HRNN facilitate the process of identifying temporal patterns of a GRN. Our HRNN method was implemented with the Python language. It was first evaluated on simulated data representing linear and nonlinear time-delayed gene-gene interaction models across a range of network sizes and variances of noise. We then further demonstrated the capability of our method in reconstructing GRNs of the Saccharomyces cerevisiae synthetic network for in vivo benchmarking of reverse-engineering and modeling approaches (IRMA). We compared the performance of our method to TD-ARACNE, HCC-CLINDE, TSNI and ebdbNet across different network
Yamada, Tatsuro; Murata, Shingo; Arie, Hiroaki; Ogata, Tetsuya
2016-01-01
To work cooperatively with humans by using language, robots must not only acquire a mapping between language and their behavior but also autonomously utilize the mapping in appropriate contexts of interactive tasks online. To this end, we propose a novel learning method linking language to robot behavior by means of a recurrent neural network. In this method, the network learns from correct examples of the imposed task that are given not as explicitly separated sets of language and behavior but as sequential data constructed from the actual temporal flow of the task. By doing this, the internal dynamics of the network models both language-behavior relationships and the temporal patterns of interaction. Here, "internal dynamics" refers to the time development of the system defined on the fixed-dimensional space of the internal states of the context layer. Thus, in the execution phase, by constantly representing where in the interaction context it is as its current state, the network autonomously switches between recognition and generation phases without any explicit signs and utilizes the acquired mapping in appropriate contexts. To evaluate our method, we conducted an experiment in which a robot generates appropriate behavior responding to a human's linguistic instruction. After learning, the network actually formed the attractor structure representing both language-behavior relationships and the task's temporal pattern in its internal dynamics. In the dynamics, language-behavior mapping was achieved by the branching structure. Repetition of human's instruction and robot's behavioral response was represented as the cyclic structure, and besides, waiting to a subsequent instruction was represented as the fixed-point attractor. Thanks to this structure, the robot was able to interact online with a human concerning the given task by autonomously switching phases.
De-identification of clinical notes via recurrent neural network and conditional random field.
Liu, Zengjian; Tang, Buzhou; Wang, Xiaolong; Chen, Qingcai
2017-11-01
De-identification, identifying information from data, such as protected health information (PHI) present in clinical data, is a critical step to enable data to be shared or published. The 2016 Centers of Excellence in Genomic Science (CEGS) Neuropsychiatric Genome-scale and RDOC Individualized Domains (N-GRID) clinical natural language processing (NLP) challenge contains a de-identification track in de-identifying electronic medical records (EMRs) (i.e., track 1). The challenge organizers provide 1000 annotated mental health records for this track, 600 out of which are used as a training set and 400 as a test set. We develop a hybrid system for the de-identification task on the training set. Firstly, four individual subsystems, that is, a subsystem based on bidirectional LSTM (long-short term memory, a variant of recurrent neural network), a subsystem-based on bidirectional LSTM with features, a subsystem based on conditional random field (CRF) and a rule-based subsystem, are used to identify PHI instances. Then, an ensemble learning-based classifiers is deployed to combine all PHI instances predicted by above three machine learning-based subsystems. Finally, the results of the ensemble learning-based classifier and the rule-based subsystem are merged together. Experiments conducted on the official test set show that our system achieves the highest micro F1-scores of 93.07%, 91.43% and 95.23% under the "token", "strict" and "binary token" criteria respectively, ranking first in the 2016 CEGS N-GRID NLP challenge. In addition, on the dataset of 2014 i2b2 NLP challenge, our system achieves the highest micro F1-scores of 96.98%, 95.11% and 98.28% under the "token", "strict" and "binary token" criteria respectively, outperforming other state-of-the-art systems. All these experiments prove the effectiveness of our proposed method. Copyright © 2017. Published by Elsevier Inc.
Cocos, Anne; Fiks, Alexander G; Masino, Aaron J
2017-07-01
Social media is an important pharmacovigilance data source for adverse drug reaction (ADR) identification. Human review of social media data is infeasible due to data quantity, thus natural language processing techniques are necessary. Social media includes informal vocabulary and irregular grammar, which challenge natural language processing methods. Our objective is to develop a scalable, deep-learning approach that exceeds state-of-the-art ADR detection performance in social media. We developed a recurrent neural network (RNN) model that labels words in an input sequence with ADR membership tags. The only input features are word-embedding vectors, which can be formed through task-independent pretraining or during ADR detection training. Our best-performing RNN model used pretrained word embeddings created from a large, non-domain-specific Twitter dataset. It achieved an approximate match F-measure of 0.755 for ADR identification on the dataset, compared to 0.631 for a baseline lexicon system and 0.65 for the state-of-the-art conditional random field model. Feature analysis indicated that semantic information in pretrained word embeddings boosted sensitivity and, combined with contextual awareness captured in the RNN, precision. Our model required no task-specific feature engineering, suggesting generalizability to additional sequence-labeling tasks. Learning curve analysis showed that our model reached optimal performance with fewer training examples than the other models. ADR detection performance in social media is significantly improved by using a contextually aware model and word embeddings formed from large, unlabeled datasets. The approach reduces manual data-labeling requirements and is scalable to large social media datasets. © The Author 2017. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com
Jauregi Unanue, Iñigo; Zare Borzeshi, Ehsan; Piccardi, Massimo
2017-12-01
Previous state-of-the-art systems on Drug Name Recognition (DNR) and Clinical Concept Extraction (CCE) have focused on a combination of text "feature engineering" and conventional machine learning algorithms such as conditional random fields and support vector machines. However, developing good features is inherently heavily time-consuming. Conversely, more modern machine learning approaches such as recurrent neural networks (RNNs) have proved capable of automatically learning effective features from either random assignments or automated word "embeddings". (i) To create a highly accurate DNR and CCE system that avoids conventional, time-consuming feature engineering. (ii) To create richer, more specialized word embeddings by using health domain datasets such as MIMIC-III. (iii) To evaluate our systems over three contemporary datasets. Two deep learning methods, namely the Bidirectional LSTM and the Bidirectional LSTM-CRF, are evaluated. A CRF model is set as the baseline to compare the deep learning systems to a traditional machine learning approach. The same features are used for all the models. We have obtained the best results with the Bidirectional LSTM-CRF model, which has outperformed all previously proposed systems. The specialized embeddings have helped to cover unusual words in DrugBank and MedLine, but not in the i2b2/VA dataset. We present a state-of-the-art system for DNR and CCE. Automated word embeddings has allowed us to avoid costly feature engineering and achieve higher accuracy. Nevertheless, the embeddings need to be retrained over datasets that are adequate for the domain, in order to adequately cover the domain-specific vocabulary. Copyright © 2017 Elsevier Inc. All rights reserved.
Directory of Open Access Journals (Sweden)
Tatsuro Yamada
2016-07-01
Full Text Available To work cooperatively with humans by using language, robots must not only acquire a mapping between language and their behavior but also autonomously utilize the mapping in appropriate contexts of interactive tasks online. To this end, we propose a novel learning method linking language to robot behavior by means of a recurrent neural network. In this method, the network learns from correct examples of the imposed task that are given not as explicitly separated sets of language and behavior but as sequential data constructed from the actual temporal flow of the task. By doing this, the internal dynamics of the network models both language--behavior relationships and the temporal patterns of interaction. Here, ``internal dynamics'' refers to the time development of the system defined on the fixed-dimensional space of the internal states of the context layer. Thus, in the execution phase, by constantly representing where in the interaction context it is as its current state, the network autonomously switches between recognition and generation phases without any explicit signs and utilizes the acquired mapping in appropriate contexts. To evaluate our method, we conducted an experiment in which a robot generates appropriate behavior responding to a human's linguistic instruction. After learning, the network actually formed the attractor structure representing both language--behavior relationships and the task's temporal pattern in its internal dynamics. In the dynamics, language--behavior mapping was achieved by the branching structure. Repetition of human's instruction and robot's behavioral response was represented as the cyclic structure, and besides, waiting to a subsequent instruction was represented as the fixed-point attractor. Thanks to this structure, the robot was able to interact online with a human concerning the given task by autonomously switching phases.
Directory of Open Access Journals (Sweden)
Yu Mei O
2015-08-01
Full Text Available The purpose of this study was to identify gender differences in risk factors of fall accidents among older people, and whether these factors differ between single and recurrent fallers. A total of 4,426 individuals aged ≥65 years from two large-scale health surveys provided data. Logistic regression analyses were used to identify risk factors and to determine the risk model for falling and recurrent falling in men and women separately. Three major risk factors for falling regardless of gender or fall history are fear of falling, limitations in activities of daily living (ADL, and age ≥75 years. Fear of falling remains one of the common modifiable risk factors. Among those without a fall history, the use of sedatives or tranquilizers increases the risk of falling. Regarding gender differences, ADL limitations and fear of falling appear to be stronger fall risk factors for men than for women. Among women, alcohol use and educational level are significant risk factors for falling, while loneliness is associated with recurrent falling. Men with fear of falling or ADL limitations are at higher risk to have a recurrent fall accident than women with these conditions. Having a visual impairment or living with someone is associated with recurrent falling among men. Our findings emphasize the importance of multifactorial fall interventions, taking into account a variety of subgroup characteristics such as gender and fall history.
Directory of Open Access Journals (Sweden)
Sakyasingha eDasgupta
2015-09-01
Full Text Available Walking animals, like stick insects, cockroaches or ants, demonstrate a fascinating range of locomotive abilities and complex behaviors. The locomotive behaviors can consist of a variety of walking patterns along with adaptation that allow the animals to deal with changes in environmental conditions, like uneven terrains, gaps, obstacles etc. Biological study has revealed that such complex behaviors are a result of a combination of biomechanics and neural mechanism thus representing the true nature of embodied interactions. While the biomechanics helps maintain flexibility and sustain a variety of movements, the neural mechanisms generate movements while making appropriate predictions crucial for achieving adaptation. Such predictions or planning ahead can be achieved by way of internal models that are grounded in the overall behavior of the animal. Inspired by these findings, we present here, an artificial bio-inspired walking system which effectively combines biomechanics (in terms of the body and leg structures with the underlying neural mechanisms. The neural mechanisms consist of 1 central pattern generator based control for generating basic rhythmic patterns and coordinated movements, 2 distributed (at each leg recurrent neural network based adaptive forward models with efference copies as internal models for sensory predictions and instantaneous state estimations, and 3 searching and elevation control for adapting the movement of an individual leg to deal with different environmental conditions. Using simulations we show that this bio-inspired approach with adaptive internal models allows the walking robot to perform complex locomotive behaviors as observed in insects, including walking on undulated terrains, crossing large gaps as well as climbing over high obstacles. Furthermore we demonstrate that the newly developed recurrent network based approach to sensorimotor prediction outperforms the previous state of the art adaptive neuron
Directory of Open Access Journals (Sweden)
S. N. Naikwad
2009-01-01
Full Text Available A focused time lagged recurrent neural network (FTLR NN with gamma memory filter is designed to learn the subtle complex dynamics of a typical CSTR process. Continuous stirred tank reactor exhibits complex nonlinear operations where reaction is exothermic. It is noticed from literature review that process control of CSTR using neuro-fuzzy systems was attempted by many, but optimal neural network model for identification of CSTR process is not yet available. As CSTR process includes temporal relationship in the input-output mappings, time lagged recurrent neural network is particularly used for identification purpose. The standard back propagation algorithm with momentum term has been proposed in this model. The various parameters like number of processing elements, number of hidden layers, training and testing percentage, learning rule and transfer function in hidden and output layer are investigated on the basis of performance measures like MSE, NMSE, and correlation coefficient on testing data set. Finally effects of different norms are tested along with variation in gamma memory filter. It is demonstrated that dynamic NN model has a remarkable system identification capability for the problems considered in this paper. Thus FTLR NN with gamma memory filter can be used to learn underlying highly nonlinear dynamics of the system, which is a major contribution of this paper.
Samarasinghe, S; Ling, H
In this paper, we show how to extend our previously proposed novel continuous time Recurrent Neural Networks (RNN) approach that retains the advantage of continuous dynamics offered by Ordinary Differential Equations (ODE) while enabling parameter estimation through adaptation, to larger signalling networks using a modular approach. Specifically, the signalling network is decomposed into several sub-models based on important temporal events in the network. Each sub-model is represented by the proposed RNN and trained using data generated from the corresponding ODE model. Trained sub-models are assembled into a whole system RNN which is then subjected to systems dynamics and sensitivity analyses. The concept is illustrated by application to G1/S transition in cell cycle using Iwamoto et al. (2008) ODE model. We decomposed the G1/S network into 3 sub-models: (i) E2F transcription factor release; (ii) E2F and CycE positive feedback loop for elevating cyclin levels; and (iii) E2F and CycA negative feedback to degrade E2F. The trained sub-models accurately represented system dynamics and parameters were in good agreement with the ODE model. The whole system RNN however revealed couple of parameters contributing to compounding errors due to feedback and required refinement to sub-model 2. These related to the reversible reaction between CycE/CDK2 and p27, its inhibitor. The revised whole system RNN model very accurately matched dynamics of the ODE system. Local sensitivity analysis of the whole system model further revealed the most dominant influence of the above two parameters in perturbing G1/S transition, giving support to a recent hypothesis that the release of inhibitor p27 from Cyc/CDK complex triggers cell cycle stage transition. To make the model useful in a practical setting, we modified each RNN sub-model with a time relay switch to facilitate larger interval input data (≈20min) (original model used data for 30s or less) and retrained them that produced
Ling, Hong; Samarasinghe, Sandhya; Kulasiri, Don
2013-12-01
Understanding the control of cellular networks consisting of gene and protein interactions and their emergent properties is a central activity of Systems Biology research. For this, continuous, discrete, hybrid, and stochastic methods have been proposed. Currently, the most common approach to modelling accurate temporal dynamics of networks is ordinary differential equations (ODE). However, critical limitations of ODE models are difficulty in kinetic parameter estimation and numerical solution of a large number of equations, making them more suited to smaller systems. In this article, we introduce a novel recurrent artificial neural network (RNN) that addresses above limitations and produces a continuous model that easily estimates parameters from data, can handle a large number of molecular interactions and quantifies temporal dynamics and emergent systems properties. This RNN is based on a system of ODEs representing molecular interactions in a signalling network. Each neuron represents concentration change of one molecule represented by an ODE. Weights of the RNN correspond to kinetic parameters in the system and can be adjusted incrementally during network training. The method is applied to the p53-Mdm2 oscillation system - a crucial component of the DNA damage response pathways activated by a damage signal. Simulation results indicate that the proposed RNN can successfully represent the behaviour of the p53-Mdm2 oscillation system and solve the parameter estimation problem with high accuracy. Furthermore, we presented a modified form of the RNN that estimates parameters and captures systems dynamics from sparse data collected over relatively large time steps. We also investigate the robustness of the p53-Mdm2 system using the trained RNN under various levels of parameter perturbation to gain a greater understanding of the control of the p53-Mdm2 system. Its outcomes on robustness are consistent with the current biological knowledge of this system. As more
Cao, Jinde; Wang, Jun
2004-04-01
This paper investigates the absolute exponential stability of a general class of delayed neural networks, which require the activation functions to be partially Lipschitz continuous and monotone nondecreasing only, but not necessarily differentiable or bounded. Three new sufficient conditions are derived to ascertain whether or not the equilibrium points of the delayed neural networks with additively diagonally stable interconnection matrices are absolutely exponentially stable by using delay Halanay-type inequality and Lyapunov function. The stability criteria are also suitable for delayed optimization neural networks and delayed cellular neural networks whose activation functions are often nondifferentiable or unbounded. The results herein answer a question: if a neural network without any delay is absolutely exponentially stable, then under what additional conditions, the neural networks with delay is also absolutely exponentially stable.
Lu, Weizhao; Huang, Chunhui; Hou, Kun; Shi, Liting; Zhao, Huihui; Li, Zhengmei; Qiu, Jianfeng
2018-05-01
In continuous-variable quantum key distribution (CV-QKD), weak signal carrying information transmits from Alice to Bob; during this process it is easily influenced by unknown noise which reduces signal-to-noise ratio, and strongly impacts reliability and stability of the communication. Recurrent quantum neural network (RQNN) is an artificial neural network model which can perform stochastic filtering without any prior knowledge of the signal and noise. In this paper, a modified RQNN algorithm with expectation maximization algorithm is proposed to process the signal in CV-QKD, which follows the basic rule of quantum mechanics. After RQNN, noise power decreases about 15 dBm, coherent signal recognition rate of RQNN is 96%, quantum bit error rate (QBER) drops to 4%, which is 6.9% lower than original QBER, and channel capacity is notably enlarged.
Matos, Sérgio; Antunes, Rui
2017-12-13
Curation of protein interactions from scientific articles is an important task, since interaction networks are essential for the understanding of biological processes associated with disease or pharmacological action for example. However, the increase in the number of publications that potentially contain relevant information turns this into a very challenging and expensive task. In this work we used a convolutional recurrent neural network for identifying relevant articles for extracting information regarding protein interactions. Using the BioCreative III Article Classification Task dataset, we achieved an area under the precision-recall curve of 0.715 and a Matthew's correlation coefficient of 0.600, which represents an improvement over previous works.
International Nuclear Information System (INIS)
Ali, M. Syed
2011-01-01
In this paper, the global stability of Takagi—Sugeno (TS) uncertain stochastic fuzzy recurrent neural networks with discrete and distributed time-varying delays (TSUSFRNNs) is considered. A novel LMI-based stability criterion is obtained by using Lyapunov functional theory to guarantee the asymptotic stability of TSUSFRNNs. The proposed stability conditions are demonstrated through numerical examples. Furthermore, the supplementary requirement that the time derivative of time-varying delays must be smaller than one is removed. Comparison results are demonstrated to show that the proposed method is more able to guarantee the widest stability region than the other methods available in the existing literature. (general)
International Nuclear Information System (INIS)
Wang Shen-Quan; Feng Jian; Zhao Qing
2012-01-01
In this paper, the problem of delay-distribution-dependent stability is investigated for continuous-time recurrent neural networks (CRNNs) with stochastic delay. Different from the common assumptions on time delays, it is assumed that the probability distribution of the delay taking values in some intervals is known a priori. By making full use of the information concerning the probability distribution of the delay and by using a tighter bounding technique (the reciprocally convex combination method), less conservative asymptotic mean-square stable sufficient conditions are derived in terms of linear matrix inequalities (LMIs). Two numerical examples show that our results are better than the existing ones. (general)
DEFF Research Database (Denmark)
Pogrebnyakov, Nicolai; Maldonado, Edgar
2017-01-01
Classification of social media posts in emergency response is an important practical problem: accurate classification can help automate processing of such messages and help other responders and the public react to emergencies in a timely fashion. This research focused on classifying Facebook mess......, and models were constructed using support vector machines (SVMs) and convolutional (CNNs) and recurrent neural networks (RNNs). The best performing classifier was an RNN with a custom-trained word2vec model to represent features, which achieved the F1 measure of 0.839....
Identification of the non-linear systems using internal recurrent neural networks
Directory of Open Access Journals (Sweden)
Bogdan CODRES
2006-12-01
Full Text Available In the past years utilization of neural networks took a distinct ampleness because of the following properties: distributed representation of information, capacity of generalization in case of uncontained situation in training data set, tolerance to noise, resistance to partial destruction, parallel processing. Another major advantage of neural networks is that they allow us to obtain the model of the investigated system, systems that is not necessarily to be linear. In fact, the true value of neural networks is seen in the case of identification and control of nonlinear systems. In this paper there are presented some identification techniques using neural networks.
Single-Cell Phenotype Classification Using Deep Convolutional Neural Networks.
Dürr, Oliver; Sick, Beate
2016-10-01
Deep learning methods are currently outperforming traditional state-of-the-art computer vision algorithms in diverse applications and recently even surpassed human performance in object recognition. Here we demonstrate the potential of deep learning methods to high-content screening-based phenotype classification. We trained a deep learning classifier in the form of convolutional neural networks with approximately 40,000 publicly available single-cell images from samples treated with compounds from four classes known to lead to different phenotypes. The input data consisted of multichannel images. The construction of appropriate feature definitions was part of the training and carried out by the convolutional network, without the need for expert knowledge or handcrafted features. We compare our results against the recent state-of-the-art pipeline in which predefined features are extracted from each cell using specialized software and then fed into various machine learning algorithms (support vector machine, Fisher linear discriminant, random forest) for classification. The performance of all classification approaches is evaluated on an untouched test image set with known phenotype classes. Compared to the best reference machine learning algorithm, the misclassification rate is reduced from 8.9% to 6.6%. © 2016 Society for Laboratory Automation and Screening.
N.D.A. Boyé (Nicole); F.U.S. Mattace Raso (Francesco); E.M.M. van Lieshout (Esther); K.A. Hartholt (Klaas); E.F. van Beeck (Ed); T.J.M. van der Cammen (Tischa)
2014-01-01
textabstractAim: Although guidelines regarding falls prevention make a clear distinction between single and recurrent fallers, differences in functional status, physical performance, and quality of life in single and recurrent fallers have not been thoroughly investigated. Therefore, we investigated
Kabeshova, Anastasiia; Launay, Cyrille P; Gromov, Vasilii A; Annweiler, Cédric; Fantino, Bruno; Beauchet, Olivier
2015-04-01
Identification of the risk of recurrent falls is complex in older adults. The aim of this study was to examine the efficiency of 3 artificial neural networks (ANNs: multilayer perceptron [MLP], modified MLP, and neuroevolution of augmenting topologies [NEAT]) for the classification of recurrent fallers and nonrecurrent fallers using a set of clinical characteristics corresponding to risk factors of falls measured among community-dwelling older adults. Based on a cross-sectional design, 3289 community-dwelling volunteers aged 65 and older were recruited. Age, gender, body mass index (BMI), number of drugs daily taken, use of psychoactive drugs, diphosphonate, calcium, vitamin D supplements and walking aid, fear of falling, distance vision score, Timed Up and Go (TUG) score, lower-limb proprioception, handgrip strength, depressive symptoms, cognitive disorders, and history of falls were recorded. Participants were separated into 2 groups based on the number of falls that occurred over the past year: 0 or 1 fall and 2 or more falls. In addition, total population was separated into training and testing subgroups for ANN analysis. Among 3289 participants, 18.9% (n = 622) were recurrent fallers. NEAT, using 15 clinical characteristics (ie, use of walking aid, fear of falling, use of calcium, depression, use of vitamin D supplements, female, cognitive disorders, BMI 4, vision score 9 seconds, handgrip strength score ≤29 (N), and age ≥75 years), showed the best efficiency for identification of recurrent fallers, sensitivity (80.42%), specificity (92.54%), positive predictive value (84.38), negative predictive value (90.34), accuracy (88.39), and Cohen κ (0.74), compared with MLP and modified MLP. NEAT, using a set of 15 clinical characteristics, was an efficient ANN for the identification of recurrent fallers in older community-dwellers. Copyright © 2015 AMDA – The Society for Post-Acute and Long-Term Care Medicine. Published by Elsevier Inc. All rights reserved.
Association of the DIO2 gene single nucleotide polymorphisms with recurrent depressive disorder.
Gałecka, Elżbieta; Talarowska, Monika; Orzechowska, Agata; Górski, Paweł; Bieńkiewicz, Małgorzata; Szemraj, Janusz
2015-01-01
Genetic factors may play a role in the etiology of depressive disorder. The type 2 iodothyronine deiodinase gene (DIO2) encoding the enzyme catalyzing the conversion of T4 to T3 is suggested to play a role in the recurrent depressive disorder (rDD). The current study investigates whether a specific single nucleotide polymorphism (SNP) of the DIO2 gene, Thr92Ala (T/C); rs 225014 or ORFa-Gly3Asp (C/T); rs 12885300, correlate with the risk for recurrent depression. Genotypes for these two single nucleotide polymorphisms (SNPs) were determined in 179 patients meeting the ICD-10 criteria for rDD group and in 152 healthy individuals (control group) using a polymerase chain reaction (PCR) based method. The specific variant of the DIO2 gene, namely the CC genotype of the Thr92Ala polymorphism, was more frequently found in healthy subjects than in patients with depression, what suggests that it could potentially serve as a marker of a lower risk for recurrent depressive disorder. The distribution of four haplotypes was also significantly different between the two study groups with the TC (Thr-Gly) haplotype more frequently detected in patients with depression. In conclusion, data generated from this study suggest for the first time that DIO2 gene may play a role in the etiology of the disease, and thus should be further investigated.
A phase II trial of single-agent bevacizumab in patients with recurrent anaplastic glioma.
Kreisl, Teri N; Zhang, Weiting; Odia, Yazmin; Shih, Joanna H; Butman, John A; Hammoud, Dima; Iwamoto, Fabio M; Sul, Joohee; Fine, Howard A
2011-10-01
The purpose of this study was to evaluate the activity of single-agent bevacizumab in patients with recurrent anaplastic glioma and assess correlative advanced imaging parameters. Patients with recurrent anaplastic glioma were treated with bevacizumab 10 mg/kg every 2 weeks. Complete patient evaluations were repeated every 4 weeks. Correlative dynamic contrast-enhanced MR and (18)fluorodeoxyglucose PET imaging studies were obtained to evaluate physiologic changes in tumor and tumor vasculature at time points including baseline, 96 h after the first dose, and after the first 4 weeks of therapy. Median overall survival was 12 months (95% confidence interval [CI]: 6.08-22.8). Median progression-free survival was 2.93 months (95% CI: 2.01-4.93), and 6-month progression-free survival was 20.9% (95% CI: 10.3%-42.5%). Thirteen (43%) patients achieved a partial response. The most common grade ≥ 3 treatment-related toxicities were hypertension, hypophosphatemia, and thromboembolism. Single-agent bevacizumab produces significant radiographic response in patients with recurrent anaplastic glioma but did not meet the 6-month progression-free survival endpoint. Early change in enhancing tumor volume at 4 days after start of therapy was the most significant prognostic factor for overall and progression-free survival.
Sharma, Richa; Kumar, Vikas; Gaur, Prerna; Mittal, A P
2016-05-01
Being complex, non-linear and coupled system, the robotic manipulator cannot be effectively controlled using classical proportional-integral-derivative (PID) controller. To enhance the effectiveness of the conventional PID controller for the nonlinear and uncertain systems, gains of the PID controller should be conservatively tuned and should adapt to the process parameter variations. In this work, a mix locally recurrent neural network (MLRNN) architecture is investigated to mimic a conventional PID controller which consists of at most three hidden nodes which act as proportional, integral and derivative node. The gains of the mix locally recurrent neural network based PID (MLRNNPID) controller scheme are initialized with a newly developed cuckoo search algorithm (CSA) based optimization method rather than assuming randomly. A sequential learning based least square algorithm is then investigated for the on-line adaptation of the gains of MLRNNPID controller. The performance of the proposed controller scheme is tested against the plant parameters uncertainties and external disturbances for both links of the two link robotic manipulator with variable payload (TL-RMWVP). The stability of the proposed controller is analyzed using Lyapunov stability criteria. A performance comparison is carried out among MLRNNPID controller, CSA optimized NNPID (OPTNNPID) controller and CSA optimized conventional PID (OPTPID) controller in order to establish the effectiveness of the MLRNNPID controller. Copyright © 2016 ISA. Published by Elsevier Ltd. All rights reserved.
National Research Council Canada - National Science Library
Laine, Trevor
2003-01-01
.... Since input features extracted from sensor data for ATR algorithms are likely to contain significant correlation, models such as artificial neural networks that do not assume independent input data...
Directory of Open Access Journals (Sweden)
Chih-Hong Lin
2016-06-01
Full Text Available A permanent magnet (PM synchronous generator system driven by wind turbine (WT, connected with smart grid via AC-DC converter and DC-AC converter, are controlled by the novel recurrent Chebyshev neural network (NN and amended particle swarm optimization (PSO to regulate output power and output voltage in two power converters in this study. Because a PM synchronous generator system driven by WT is an unknown non-linear and time-varying dynamic system, the on-line training novel recurrent Chebyshev NN control system is developed to regulate DC voltage of the AC-DC converter and AC voltage of the DC-AC converter connected with smart grid. Furthermore, the variable learning rate of the novel recurrent Chebyshev NN is regulated according to discrete-type Lyapunov function for improving the control performance and enhancing convergent speed. Finally, some experimental results are shown to verify the effectiveness of the proposed control method for a WT driving a PM synchronous generator system in smart grid.
International Nuclear Information System (INIS)
Zio, Enrico; Pedroni, Nicola; Broggi, Matteo; Golea, Lucia Roxana
2009-01-01
In this paper, an infinite impulse response locally recurrent neural network (IIR-LRNN) is employed for modelling the dynamics of the Lead Bismuth Eutectic eXperimental Accelerator Driven System (LBE-XADS). The network is trained by recursive back-propagation (RBP) and its ability in estimating transients is tested under various conditions. The results demonstrate the robustness of the locally recurrent scheme in the reconstruction of complex nonlinear dynamic relationships
Knox, Dayan; Stanfield, Briana R.; Staib, Jennifer M.; David, Nina P.; Keller, Samantha M.; DePietro, Thomas
2016-01-01
Single prolonged stress (SPS) has been used to examine mechanisms via which stress exposure leads to post-traumatic stress disorder symptoms. SPS induces fear extinction retention deficits, but neural circuits critical for mediating these deficits are unknown. To address this gap, we examined the effect of SPS on neural activity in brain regions…
RM-SORN: a reward-modulated self-organizing recurrent neural network.
Aswolinskiy, Witali; Pipa, Gordon
2015-01-01
Neural plasticity plays an important role in learning and memory. Reward-modulation of plasticity offers an explanation for the ability of the brain to adapt its neural activity to achieve a rewarded goal. Here, we define a neural network model that learns through the interaction of Intrinsic Plasticity (IP) and reward-modulated Spike-Timing-Dependent Plasticity (STDP). IP enables the network to explore possible output sequences and STDP, modulated by reward, reinforces the creation of the rewarded output sequences. The model is tested on tasks for prediction, recall, non-linear computation, pattern recognition, and sequence generation. It achieves performance comparable to networks trained with supervised learning, while using simple, biologically motivated plasticity rules, and rewarding strategies. The results confirm the importance of investigating the interaction of several plasticity rules in the context of reward-modulated learning and whether reward-modulated self-organization can explain the amazing capabilities of the brain.
DEFF Research Database (Denmark)
Grinke, Eduard; Tetzlaff, Christian; Wörgötter, Florentin
2015-01-01
Walking animals, like insects, with little neural computing can effectively perform complex behaviors. For example, they can walk around their environment, escape from corners/deadlocks, and avoid or climb over obstacles. While performing all these behaviors, they can also adapt their movements...... to deal with an unknown situation. As a consequence, they successfully navigate through their complex environment. The versatile and adaptive abilities are the result of an integration of several ingredients embedded in their sensorimotor loop. Biological studies reveal that the ingredients include neural...... dynamics, plasticity, sensory feedback, and biomechanics. Generating such versatile and adaptive behaviors for a many degrees-of-freedom (DOFs) walking robot is a challenging task. Thus, in this study, we present a bio-inspired approach to solve this task. Specifically, the approach combines neural...
Xie, Jiaheng; Liu, Xiao; Dajun Zeng, Daniel
2018-01-01
Recent years have seen increased worldwide popularity of e-cigarette use. However, the risks of e-cigarettes are underexamined. Most e-cigarette adverse event studies have achieved low detection rates due to limited subject sample sizes in the experiments and surveys. Social media provides a large data repository of consumers' e-cigarette feedback and experiences, which are useful for e-cigarette safety surveillance. However, it is difficult to automatically interpret the informal and nontechnical consumer vocabulary about e-cigarettes in social media. This issue hinders the use of social media content for e-cigarette safety surveillance. Recent developments in deep neural network methods have shown promise for named entity extraction from noisy text. Motivated by these observations, we aimed to design a deep neural network approach to extract e-cigarette safety information in social media. Our deep neural language model utilizes word embedding as the representation of text input and recognizes named entity types with the state-of-the-art Bidirectional Long Short-Term Memory (Bi-LSTM) Recurrent Neural Network. Our Bi-LSTM model achieved the best performance compared to 3 baseline models, with a precision of 94.10%, a recall of 91.80%, and an F-measure of 92.94%. We identified 1591 unique adverse events and 9930 unique e-cigarette components (ie, chemicals, flavors, and devices) from our research testbed. Although the conditional random field baseline model had slightly better precision than our approach, our Bi-LSTM model achieved much higher recall, resulting in the best F-measure. Our method can be generalized to extract medical concepts from social media for other medical applications. © The Author 2017. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com
Neural correlates of single-vessel haemodynamic responses in vivo.
O'Herron, Philip; Chhatbar, Pratik Y; Levy, Manuel; Shen, Zhiming; Schramm, Adrien E; Lu, Zhongyang; Kara, Prakash
2016-06-16
Neural activation increases blood flow locally. This vascular signal is used by functional imaging techniques to infer the location and strength of neural activity. However, the precise spatial scale over which neural and vascular signals are correlated is unknown. Furthermore, the relative role of synaptic and spiking activity in driving haemodynamic signals is controversial. Previous studies recorded local field potentials as a measure of synaptic activity together with spiking activity and low-resolution haemodynamic imaging. Here we used two-photon microscopy to measure sensory-evoked responses of individual blood vessels (dilation, blood velocity) while imaging synaptic and spiking activity in the surrounding tissue using fluorescent glutamate and calcium sensors. In cat primary visual cortex, where neurons are clustered by their preference for stimulus orientation, we discovered new maps for excitatory synaptic activity, which were organized similarly to those for spiking activity but were less selective for stimulus orientation and direction. We generated tuning curves for individual vessel responses for the first time and found that parenchymal vessels in cortical layer 2/3 were orientation selective. Neighbouring penetrating arterioles had different orientation preferences. Pial surface arteries in cats, as well as surface arteries and penetrating arterioles in rat visual cortex (where orientation maps do not exist), responded to visual stimuli but had no orientation selectivity. We integrated synaptic or spiking responses around individual parenchymal vessels in cats and established that the vascular and neural responses had the same orientation preference. However, synaptic and spiking responses were more selective than vascular responses--vessels frequently responded robustly to stimuli that evoked little to no neural activity in the surrounding tissue. Thus, local neural and haemodynamic signals were partly decoupled. Together, these results indicate
Rossi, A.; Montefoschi, F.; Rizzo, A.; Diligenti, M.; Festucci, C.
2017-10-01
Machine Learning applied to Automatic Audio Surveillance has been attracting increasing attention in recent years. In spite of several investigations based on a large number of different approaches, little attention had been paid to the environmental temporal evolution of the input signal. In this work, we propose an exploration in this direction comparing the temporal correlations extracted at the feature level with the one learned by a representational structure. To this aim we analysed the prediction performances of a Recurrent Neural Network architecture varying the length of the processed input sequence and the size of the time window used in the feature extraction. Results corroborated the hypothesis that sequential models work better when dealing with data characterized by temporal order. However, so far the optimization of the temporal dimension remains an open issue.
Tutubalina, Elena; Nikolenko, Sergey
2017-01-01
Adverse drug reactions (ADRs) are an essential part of the analysis of drug use, measuring drug use benefits, and making policy decisions. Traditional channels for identifying ADRs are reliable but very slow and only produce a small amount of data. Text reviews, either on specialized web sites or in general-purpose social networks, may lead to a data source of unprecedented size, but identifying ADRs in free-form text is a challenging natural language processing problem. In this work, we propose a novel model for this problem, uniting recurrent neural architectures and conditional random fields. We evaluate our model with a comprehensive experimental study, showing improvements over state-of-the-art methods of ADR extraction.
Directory of Open Access Journals (Sweden)
Elena Tutubalina
2017-01-01
Full Text Available Adverse drug reactions (ADRs are an essential part of the analysis of drug use, measuring drug use benefits, and making policy decisions. Traditional channels for identifying ADRs are reliable but very slow and only produce a small amount of data. Text reviews, either on specialized web sites or in general-purpose social networks, may lead to a data source of unprecedented size, but identifying ADRs in free-form text is a challenging natural language processing problem. In this work, we propose a novel model for this problem, uniting recurrent neural architectures and conditional random fields. We evaluate our model with a comprehensive experimental study, showing improvements over state-of-the-art methods of ADR extraction.
Quang, Daniel; Xie, Xiaohui
2016-06-20
Modeling the properties and functions of DNA sequences is an important, but challenging task in the broad field of genomics. This task is particularly difficult for non-coding DNA, the vast majority of which is still poorly understood in terms of function. A powerful predictive model for the function of non-coding DNA can have enormous benefit for both basic science and translational research because over 98% of the human genome is non-coding and 93% of disease-associated variants lie in these regions. To address this need, we propose DanQ, a novel hybrid convolutional and bi-directional long short-term memory recurrent neural network framework for predicting non-coding function de novo from sequence. In the DanQ model, the convolution layer captures regulatory motifs, while the recurrent layer captures long-term dependencies between the motifs in order to learn a regulatory 'grammar' to improve predictions. DanQ improves considerably upon other models across several metrics. For some regulatory markers, DanQ can achieve over a 50% relative improvement in the area under the precision-recall curve metric compared to related models. We have made the source code available at the github repository http://github.com/uci-cbcl/DanQ. © The Author(s) 2016. Published by Oxford University Press on behalf of Nucleic Acids Research.
Khodabakhshi, Mohammad Bagher; Moradi, Mohammad Hassan
2017-05-01
The respiratory system dynamic is of high significance when it comes to the detection of lung abnormalities, which highlights the importance of presenting a reliable model for it. In this paper, we introduce a novel dynamic modelling method for the characterization of the lung sounds (LS), based on the attractor recurrent neural network (ARNN). The ARNN structure allows the development of an effective LS model. Additionally, it has the capability to reproduce the distinctive features of the lung sounds using its formed attractors. Furthermore, a novel ARNN topology based on fuzzy functions (FFs-ARNN) is developed. Given the utility of the recurrent quantification analysis (RQA) as a tool to assess the nature of complex systems, it was used to evaluate the performance of both the ARNN and the FFs-ARNN models. The experimental results demonstrate the effectiveness of the proposed approaches for multichannel LS analysis. In particular, a classification accuracy of 91% was achieved using FFs-ARNN with sequences of RQA features. Copyright © 2017 Elsevier Ltd. All rights reserved.
Hayashi, Hideaki; Shima, Keisuke; Shibanoki, Taro; Kurita, Yuichi; Tsuji, Toshio
2013-01-01
This paper outlines a probabilistic neural network developed on the basis of time-series discriminant component analysis (TSDCA) that can be used to classify high-dimensional time-series patterns. TSDCA involves the compression of high-dimensional time series into a lower-dimensional space using a set of orthogonal transformations and the calculation of posterior probabilities based on a continuous-density hidden Markov model that incorporates a Gaussian mixture model expressed in the reduced-dimensional space. The analysis can be incorporated into a neural network so that parameters can be obtained appropriately as network coefficients according to backpropagation-through-time-based training algorithm. The network is considered to enable high-accuracy classification of high-dimensional time-series patterns and to reduce the computation time taken for network training. In the experiments conducted during the study, the validity of the proposed network was demonstrated for EEG signals.
Directory of Open Access Journals (Sweden)
Run Min HOU
2014-05-01
Full Text Available To control the nonlinearity, widespread variations in loads and time varying characteristic of the high power ac servo system, the modeling and control techniques are studied here. A self-recurrent wavelet neural network (SRWNN modeling scheme is proposed, which successfully addresses the issue of the traditional wavelet neural network easily falling into local optimum, and significantly improves the network approximation capability and convergence rate. The control scheme of a SRWNN based on fuzzy compensation is expected. Gradient information is provided in real time for the controller by using a SRWNN identifier, so as to ensure that the learning and adjusting function of the controller of the SRWNN operate well, and fuzzy compensation control is applied to improve rapidity and accuracy of the entire system. Then the Lyapunov function is utilized to judge the stability of the system. The experimental analysis and comparisons with other modeling and control methods, it is clearly shown that the validities of the proposed modeling scheme and control scheme are effective.
Song, Yongli; Makarov, Valeri A; Velarde, Manuel G
2009-08-01
A model of time-delay recurrently coupled spatially segregated neural assemblies is here proposed. We show that it operates like some of the hierarchical architectures of the brain. Each assembly is a neural network with no delay in the local couplings between the units. The delay appears in the long range feedforward and feedback inter-assemblies communications. Bifurcation analysis of a simple four-units system in the autonomous case shows the richness of the dynamical behaviors in a biophysically plausible parameter region. We find oscillatory multistability, hysteresis, and stability switches of the rest state provoked by the time delay. Then we investigate the spatio-temporal patterns of bifurcating periodic solutions by using the symmetric local Hopf bifurcation theory of delay differential equations and derive the equation describing the flow on the center manifold that enables us determining the direction of Hopf bifurcations and stability of the bifurcating periodic orbits. We also discuss computational properties of the system due to the delay when an external drive of the network mimicks external sensory input.
Learning to Recognize Actions From Limited Training Examples Using a Recurrent Spiking Neural Model
Panda, Priyadarshini; Srinivasa, Narayan
2018-01-01
A fundamental challenge in machine learning today is to build a model that can learn from few examples. Here, we describe a reservoir based spiking neural model for learning to recognize actions with a limited number of labeled videos. First, we propose a novel encoding, inspired by how microsaccades influence visual perception, to extract spike information from raw video data while preserving the temporal correlation across different frames. Using this encoding, we show that the reservoir generalizes its rich dynamical activity toward signature action/movements enabling it to learn from few training examples. We evaluate our approach on the UCF-101 dataset. Our experiments demonstrate that our proposed reservoir achieves 81.3/87% Top-1/Top-5 accuracy, respectively, on the 101-class data while requiring just 8 video examples per class for training. Our results establish a new benchmark for action recognition from limited video examples for spiking neural models while yielding competitive accuracy with respect to state-of-the-art non-spiking neural models. PMID:29551962
Directory of Open Access Journals (Sweden)
Lars Buesing
2011-11-01
Full Text Available The organization of computations in networks of spiking neurons in the brain is still largely unknown, in particular in view of the inherently stochastic features of their firing activity and the experimentally observed trial-to-trial variability of neural systems in the brain. In principle there exists a powerful computational framework for stochastic computations, probabilistic inference by sampling, which can explain a large number of macroscopic experimental data in neuroscience and cognitive science. But it has turned out to be surprisingly difficult to create a link between these abstract models for stochastic computations and more detailed models of the dynamics of networks of spiking neurons. Here we create such a link and show that under some conditions the stochastic firing activity of networks of spiking neurons can be interpreted as probabilistic inference via Markov chain Monte Carlo (MCMC sampling. Since common methods for MCMC sampling in distributed systems, such as Gibbs sampling, are inconsistent with the dynamics of spiking neurons, we introduce a different approach based on non-reversible Markov chains that is able to reflect inherent temporal processes of spiking neuronal activity through a suitable choice of random variables. We propose a neural network model and show by a rigorous theoretical analysis that its neural activity implements MCMC sampling of a given distribution, both for the case of discrete and continuous time. This provides a step towards closing the gap between abstract functional models of cortical computation and more detailed models of networks of spiking neurons.
Recurrent-neural-network-based Boolean factor analysis and its application to word clustering.
Frolov, Alexander A; Husek, Dusan; Polyakov, Pavel Yu
2009-07-01
The objective of this paper is to introduce a neural-network-based algorithm for word clustering as an extension of the neural-network-based Boolean factor analysis algorithm (Frolov , 2007). It is shown that this extended algorithm supports even the more complex model of signals that are supposed to be related to textual documents. It is hypothesized that every topic in textual data is characterized by a set of words which coherently appear in documents dedicated to a given topic. The appearance of each word in a document is coded by the activity of a particular neuron. In accordance with the Hebbian learning rule implemented in the network, sets of coherently appearing words (treated as factors) create tightly connected groups of neurons, hence, revealing them as attractors of the network dynamics. The found factors are eliminated from the network memory by the Hebbian unlearning rule facilitating the search of other factors. Topics related to the found sets of words can be identified based on the words' semantics. To make the method complete, a special technique based on a Bayesian procedure has been developed for the following purposes: first, to provide a complete description of factors in terms of component probability, and second, to enhance the accuracy of classification of signals to determine whether it contains the factor. Since it is assumed that every word may possibly contribute to several topics, the proposed method might be related to the method of fuzzy clustering. In this paper, we show that the results of Boolean factor analysis and fuzzy clustering are not contradictory, but complementary. To demonstrate the capabilities of this attempt, the method is applied to two types of textual data on neural networks in two different languages. The obtained topics and corresponding words are at a good level of agreement despite the fact that identical topics in Russian and English conferences contain different sets of keywords.
Predictions of SEP events by means of a linear filter and layer-recurrent neural network
Czech Academy of Sciences Publication Activity Database
Valach, F.; Revallo, M.; Hejda, Pavel; Bochníček, Josef
2011-01-01
Roč. 69, č. 9-10 (2011), s. 758-766 ISSN 0094-5765 R&D Projects: GA AV ČR(CZ) IAA300120608; GA MŠk OC09070 Grant - others:VEGA(SK) 2/0015/11; VEGA(SK) 2/0022/11 Institutional research plan: CEZ:AV0Z30120515 Keywords : coronal mass ejection * X-ray flare * solar energetic particles * artificial neural network Subject RIV: DE - Earth Magnetism, Geodesy, Geography Impact factor: 0.614, year: 2011
Directory of Open Access Journals (Sweden)
Chien-Lin Huang
2015-11-01
Full Text Available This study applies Real-Time Recurrent Learning Neural Network (RTRLNN and Adaptive Network-based Fuzzy Inference System (ANFIS with novel heuristic techniques to develop an advanced prediction model of accumulated total inflow of a reservoir in order to solve the difficulties of future long lead-time highly varied uncertainty during typhoon attacks while using a real-time forecast. For promoting the temporal-spatial forecasted precision, the following original specialized heuristic inputs were coupled: observed-predicted inflow increase/decrease (OPIID rate, total precipitation, and duration from current time to the time of maximum precipitation and direct runoff ending (DRE. This study also investigated the temporal-spatial forecasted error feature to assess the feasibility of the developed models, and analyzed the output sensitivity of both single and combined heuristic inputs to determine whether the heuristic model is susceptible to the impact of future forecasted uncertainty/errors. Validation results showed that the long lead-time–predicted accuracy and stability of the RTRLNN-based accumulated total inflow model are better than that of the ANFIS-based model because of the real-time recurrent deterministic routing mechanism of RTRLNN. Simulations show that the RTRLNN-based model with coupled heuristic inputs (RTRLNN-CHI, average error percentage (AEP/average forecast lead-time (AFLT: 6.3%/49 h can achieve better prediction than the model with non-heuristic inputs (AEP of RTRLNN-NHI and ANFIS-NHI: 15.2%/31.8% because of the full consideration of real-time hydrological initial/boundary conditions. Besides, the RTRLNN-CHI model can promote the forecasted lead-time above 49 h with less than 10% of AEP which can overcome the previous forecasted limits of 6-h AFLT with above 20%–40% of AEP.
The general critical analysis for continuous-time UPPAM recurrent neural networks.
Qiao, Chen; Jing, Wen-Feng; Fang, Jian; Wang, Yu-Ping
2016-01-29
The uniformly pseudo-projection-anti-monotone (UPPAM) neural network model, which can be considered as the unified continuous-time neural networks (CNNs), includes almost all of the known CNNs individuals. Recently, studies on the critical dynamics behaviors of CNNs have drawn special attentions due to its importance in both theory and applications. In this paper, we will present the analysis of the UPPAM network under the general critical conditions. It is shown that the UPPAM network possesses the global convergence and asymptotical stability under the general critical conditions if the network satisfies one quasi-symmetric requirement on the connective matrices, which is easy to be verified and applied. The general critical dynamics have rarely been studied before, and this work is an attempt to gain an meaningful assurance of general critical convergence and stability of CNNs. Since UPPAM network is the unified model for CNNs, the results obtained here can generalize and extend the existing critical conclusions for CNNs individuals, let alone those non-critical cases. Moreover, the easily verified conditions for general critical convergence and stability can further promote the applications of CNNs.
On the approximation by single hidden layer feedforward neural networks with fixed weights
Guliyev, Namig J.; Ismailov, Vugar E.
2017-01-01
International audience; Feedforward neural networks have wide applicability in various disciplines of science due to their universal approximation property. Some authors have shown that single hidden layer feedforward neural networks (SLFNs) with fixed weights still possess the universal approximation property provided that approximated functions are univariate. But this phenomenon does not lay any restrictions on the number of neurons in the hidden layer. The more this number, the more the p...
Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network.
Gilra, Aditya; Gerstner, Wulfram
2017-11-27
The brain needs to predict how the body reacts to motor commands, but how a network of spiking neurons can learn non-linear body dynamics using local, online and stable learning rules is unclear. Here, we present a supervised learning scheme for the feedforward and recurrent connections in a network of heterogeneous spiking neurons. The error in the output is fed back through fixed random connections with a negative gain, causing the network to follow the desired dynamics. The rule for Feedback-based Online Local Learning Of Weights (FOLLOW) is local in the sense that weight changes depend on the presynaptic activity and the error signal projected onto the postsynaptic neuron. We provide examples of learning linear, non-linear and chaotic dynamics, as well as the dynamics of a two-link arm. Under reasonable approximations, we show, using the Lyapunov method, that FOLLOW learning is uniformly stable, with the error going to zero asymptotically.
Slowly evolving connectivity in recurrent neural networks: I. The extreme dilution regime
Wemmenhove, B.; Skantzos, N. S.; Coolen, A. C. C.
2004-08-01
We study extremely diluted spin models of neural networks in which the connectivity evolves in time, although adiabatically slowly compared to the neurons, according to stochastic equations which on average aim to reduce frustration. The (fast) neurons and (slow) connectivity variables equilibrate separately, but at different temperatures. Our model is exactly solvable in equilibrium. We obtain phase diagrams upon making the condensed ansatz (i.e. recall of one pattern). These show that, as the connectivity temperature is lowered, the volume of the retrieval phase diverges and the fraction of mis-aligned spins is reduced. Still one always retains a region in the retrieval phase where recall states other than the one corresponding to the 'condensed' pattern are locally stable, so the associative memory character of our model is preserved.
Slowly evolving connectivity in recurrent neural networks: I. The extreme dilution regime
International Nuclear Information System (INIS)
Wemmenhove, B; Skantzos, N S; Coolen, A C C
2004-01-01
We study extremely diluted spin models of neural networks in which the connectivity evolves in time, although adiabatically slowly compared to the neurons, according to stochastic equations which on average aim to reduce frustration. The (fast) neurons and (slow) connectivity variables equilibrate separately, but at different temperatures. Our model is exactly solvable in equilibrium. We obtain phase diagrams upon making the condensed ansatz (i.e. recall of one pattern). These show that, as the connectivity temperature is lowered, the volume of the retrieval phase diverges and the fraction of mis-aligned spins is reduced. Still one always retains a region in the retrieval phase where recall states other than the one corresponding to the 'condensed' pattern are locally stable, so the associative memory character of our model is preserved
Sengupta, Rakesh; Surampudi, Bapi Raju; Melcher, David
2014-09-25
It has been proposed that the ability of humans to quickly perceive numerosity involves a visual sense of number. Different paradigms of enumeration and numerosity comparison have produced a gamut of behavioral and neuroimaging data, but there has been no unified conceptual framework that can explain results across the entire range of numerosity. The current work tries to address the ongoing debate concerning whether the same mechanism operates for enumeration of small and large numbers, through a computational approach. We describe the workings of a single-layered, fully connected network characterized by self-excitation and recurrent inhibition that operates at both subitizing and estimation ranges. We show that such a network can account for classic numerical cognition effects (the distance effect, Fechner׳s law, Weber fraction for numerosity comparison) through the network steady state activation response across different recurrent inhibition values. The model also accounts for fMRI data previously reported for different enumeration related tasks. The model also allows us to generate an estimate of the pattern of reaction times in enumeration tasks. Overall, these findings suggest that a single network architecture can account for both small and large number processing. Copyright © 2014. Published by Elsevier B.V.
Single Positive Lymph Node Prostate Cancer Can Be Treated Surgically without Recurrence.
Directory of Open Access Journals (Sweden)
Dae Keun Kim
Full Text Available To investigate pN1 prostate cancer (PCa patients treated surgically without immediate adjuvant treatment.We analyzed the database of 2316 patients at our institution who underwent robot-assisted radical prostatectomy (RARP/radical prostatectomy (RP between July 2005 and November 2012. 87 patients with pN1 PCa and received no neoadjuvant and immediate adjuvant therapy were included in the study. Included pN1 PCa patients were followed up for median of 60 months. Biochemical recurrence (BCR-free survival, metastasis-free survival (MFS, cancer specific survival (CSS, and overall survival (OS rates were determined by using Kaplan-Meier analysis. Cox regression analysis was performed to investigate the impact of prostate-specific antigen (PSA level, Gleason score, extraprostatic extension, seminal vesicle invasion, perineural invasion, lymphovascular invasion, positive surgical margin, tumor volume, early post-operative PSA(6 weeks, PSA nadir, lymph node yield, and number of pathologically positive lymph nodes on survival.The 5-year OS rate of patients was 86.1%, while the CSS rate was 89.6%. The metastasis-free and BCR-free survival rates were 71% and 19.1%, respectively, and each was significantly correlated with the number of positive lymph nodes on log rank tests (p = 0.004 and p = 0.039, respectively. The presence of 2 or more pathologically positive LNs (HR:2.20; 95% CI 1.30-3.72; p = 0.003 and a Gleason score ≥8 (HR: 2.40;95% CI: 1.32-4.38; p = 0.04 were significant negative predictors of BCR free survival on multivariable regression analysis. Furthermore, the presence of 2 or more positive lymph nodes (HR: 1.06; 95% CI 1.01-1.11; p = 0.029 were significant negative predictors of metastasis-free survival on multivariable regression analysis. Additionally, in the patients who had no BCR without adjuvant treatment 9 patients out of 10 (90% had single positive LN and 5 patients out of 10 (50% had Gleason score 7. Therefore, single positive LN
Raghu, S; Sriraam, N; Kumar, G Pradeep
2017-02-01
Electroencephalogram shortly termed as EEG is considered as the fundamental segment for the assessment of the neural activities in the brain. In cognitive neuroscience domain, EEG-based assessment method is found to be superior due to its non-invasive ability to detect deep brain structure while exhibiting superior spatial resolutions. Especially for studying the neurodynamic behavior of epileptic seizures, EEG recordings reflect the neuronal activity of the brain and thus provide required clinical diagnostic information for the neurologist. This specific proposed study makes use of wavelet packet based log and norm entropies with a recurrent Elman neural network (REN) for the automated detection of epileptic seizures. Three conditions, normal, pre-ictal and epileptic EEG recordings were considered for the proposed study. An adaptive Weiner filter was initially applied to remove the power line noise of 50 Hz from raw EEG recordings. Raw EEGs were segmented into 1 s patterns to ensure stationarity of the signal. Then wavelet packet using Haar wavelet with a five level decomposition was introduced and two entropies, log and norm were estimated and were applied to REN classifier to perform binary classification. The non-linear Wilcoxon statistical test was applied to observe the variation in the features under these conditions. The effect of log energy entropy (without wavelets) was also studied. It was found from the simulation results that the wavelet packet log entropy with REN classifier yielded a classification accuracy of 99.70 % for normal-pre-ictal, 99.70 % for normal-epileptic and 99.85 % for pre-ictal-epileptic.
Abdelbar, Ashraf M; El-Hemaly, Mostafa A; Andrews, Emad A M; Wunsch, Donald C
2005-01-01
Abduction is the process of proceeding from data describing a set of observations or events, to a set of hypotheses which best explains or accounts for the data. Cost-based abduction (CBA) is an AI formalism in which evidence to be explained is treated as a goal to be proven, proofs have costs based on how much needs to be assumed to complete the proof, and the set of assumptions needed to complete the least-cost proof are taken as the best explanation for the given evidence. In this paper, we present two techniques for improving the performance of high order recurrent networks (HORN) applied to cost-based abduction. In the backtrack-points technique, we use heuristics to recognize early that the network trajectory is moving in the wrong direction; we then restore the network state to a previously stored point, and apply heuristic perturbations to nudge the network trajectory in a different direction. In the negative reinforcement technique, we add hyperedges to the network to reduce the attractiveness of local minima. We apply these techniques to a suite of six large CBA instances, systematically generated to be difficult.
Hayashi, Hideaki; Shibanoki, Taro; Shima, Keisuke; Kurita, Yuichi; Tsuji, Toshio
2015-12-01
This paper proposes a probabilistic neural network (NN) developed on the basis of time-series discriminant component analysis (TSDCA) that can be used to classify high-dimensional time-series patterns. TSDCA involves the compression of high-dimensional time series into a lower dimensional space using a set of orthogonal transformations and the calculation of posterior probabilities based on a continuous-density hidden Markov model with a Gaussian mixture model expressed in the reduced-dimensional space. The analysis can be incorporated into an NN, which is named a time-series discriminant component network (TSDCN), so that parameters of dimensionality reduction and classification can be obtained simultaneously as network coefficients according to a backpropagation through time-based learning algorithm with the Lagrange multiplier method. The TSDCN is considered to enable high-accuracy classification of high-dimensional time-series patterns and to reduce the computation time taken for network training. The validity of the TSDCN is demonstrated for high-dimensional artificial data and electroencephalogram signals in the experiments conducted during the study.
CloudScan - A Configuration-Free Invoice Analysis System Using Recurrent Neural Networks
DEFF Research Database (Denmark)
Palm, Rasmus Berg; Winther, Ole; Laws, Florian
2017-01-01
We present CloudScan; an invoice analysis system that requires zero configuration or upfront annotation. In contrast to previous work, CloudScan does not rely on templates of invoice layout, instead it learns a single global model of invoices that naturally generalizes to unseen invoice layouts. ...
Nigatu, Yeshambel T.; Bultmann, Ute; Reijneveld, Sijmen A.
2015-01-01
Background: Obesity and major depressive disorder (MDD) are important public health problems. MDD is a heterogeneous disorder and the direction of its association with obesity remains unclear. Evidence grows that recurrent MDD (MDD-R) differs in etiology and prognosis from single episode MDD
Jimeno Yepes, Antonio
2017-09-01
Word sense disambiguation helps identifying the proper sense of ambiguous words in text. With large terminologies such as the UMLS Metathesaurus ambiguities appear and highly effective disambiguation methods are required. Supervised learning algorithm methods are used as one of the approaches to perform disambiguation. Features extracted from the context of an ambiguous word are used to identify the proper sense of such a word. The type of features have an impact on machine learning methods, thus affect disambiguation performance. In this work, we have evaluated several types of features derived from the context of the ambiguous word and we have explored as well more global features derived from MEDLINE using word embeddings. Results show that word embeddings improve the performance of more traditional features and allow as well using recurrent neural network classifiers based on Long-Short Term Memory (LSTM) nodes. The combination of unigrams and word embeddings with an SVM sets a new state of the art performance with a macro accuracy of 95.97 in the MSH WSD data set. Copyright © 2017 Elsevier Inc. All rights reserved.
Pollastri, G; Baldi, P
2002-01-01
Accurate prediction of protein contact maps is an important step in computational structural proteomics. Because contact maps provide a translation and rotation invariant topological representation of a protein, they can be used as a fundamental intermediary step in protein structure prediction. We develop a new set of flexible machine learning architectures for the prediction of contact maps, as well as other information processing and pattern recognition tasks. The architectures can be viewed as recurrent neural network implemantations of a class of Bayesian networks we call generalized input-output HMMs (GIOHMMs). For the specific case of contact maps, contextual information is propagated laterally through four hidden planes, one for each cardinal corner. We show that these architectures can be trained from examples and yield contact map predictors that outperform previously reported methods. While several extensions and improvements are in progress, the current version can accurately predict 60.5% of contacts at a distance cutoff of 8 A and 45% of distant contacts at 10 A, for proteins of length up to 300.
International Nuclear Information System (INIS)
You, Seung Han; Hahn, Jin Oh
2012-01-01
By virtue of its ease of operation compared with its conventional manual counterpart, automatic transmissions are commonly used as automotive power transmission control system in today's passenger cars. In accordance with this trend, research efforts on closed-loop automatic transmission controls have been extensively carried out to improve ride quality and fuel economy. State-of-the-art power transmission control algorithms may have limitations in performance because they rely on the steady-state characteristics of the hydraulic actuator rather than fully exploit its dynamic characteristics. Since the ultimate viability of closed-loop power transmission control is dominated by precise pressure control at the level of hydraulic actuator, closed-loop control can potentially attain superior efficacy in case the hydraulic actuator can be easily incorporated into model-based observer/controller design. In this paper, we propose to use a recurrent neural network (RNN) to establish a nonlinear empirical model of a cascade hydraulic actuator in a passenger car automatic transmission, which has potential to be easily incorporated in designing observers and controllers. Experimental analysis is performed to grasp key system characteristics, based on which a nonlinear system identification procedure is carried out. Extensive experimental validation of the established model suggests that it has superb one-step-ahead prediction capability over appropriate frequency range, making it an attractive approach for model-based observer/controller design applications in automotive systems
Hwang, Chih-Lyang; Jan, Chau
2016-02-01
At the beginning, an approximate nonlinear autoregressive moving average (NARMA) model is employed to represent a class of multivariable nonlinear dynamic systems with time-varying delay. It is known that the disadvantages of robust control for the NARMA model are as follows: 1) suitable control parameters for larger time delay are more sensitive to achieving desirable performance; 2) it only deals with bounded uncertainty; and 3) the nominal NARMA model must be learned in advance. Due to the dynamic feature of the NARMA model, a recurrent neural network (RNN) is online applied to learn it. However, the system performance becomes deteriorated due to the poor learning of the larger variation of system vector functions. In this situation, a simple network is employed to compensate the upper bound of the residue caused by the linear parameterization of the approximation error of RNN. An e -modification learning law with a projection for weight matrix is applied to guarantee its boundedness without persistent excitation. Under suitable conditions, the semiglobally ultimately bounded tracking with the boundedness of estimated weight matrix is obtained by the proposed RNN-based multivariable adaptive control. Finally, simulations are presented to verify the effectiveness and robustness of the proposed control.
Xiao, Lin; Zhang, Yongsheng; Liao, Bolin; Zhang, Zhijun; Ding, Lei; Jin, Long
2017-01-01
A dual-robot system is a robotic device composed of two robot arms. To eliminate the joint-angle drift and prevent the occurrence of high joint velocity, a velocity-level bi-criteria optimization scheme, which includes two criteria (i.e., the minimum velocity norm and the repetitive motion), is proposed and investigated for coordinated path tracking of dual robot manipulators. Specifically, to realize the coordinated path tracking of dual robot manipulators, two subschemes are first presented for the left and right robot manipulators. After that, such two subschemes are reformulated as two general quadratic programs (QPs), which can be formulated as one unified QP. A recurrent neural network (RNN) is thus presented to solve effectively the unified QP problem. At last, computer simulation results based on a dual three-link planar manipulator further validate the feasibility and the efficacy of the velocity-level optimization scheme for coordinated path tracking using the recurrent neural network.
Directory of Open Access Journals (Sweden)
Chien-Yu Lu
2009-01-01
Full Text Available This paper examines a passivity analysis for a class of discrete-time recurrent neural networks (DRNNs with norm-bounded time-varying parameter uncertainties and interval time-varying delay. The activation functions are assumed to be globally Lipschitz continuous. Based on an appropriate type of Lyapunov functional, sufficient passivity conditions for the DRNNs are derived in terms of a family of linear matrix inequalities (LMIs. Two numerical examples are given to illustrate the effectiveness and applicability.
2016-02-11
INVESTIGATION OF BACK-OFF BASED INTERPOLATION BETWEEN RECURRENT NEURAL NETWORK AND N-GRAM LANGUAGE MODELS X. Chen, X. Liu, M. J. F. Gales, and P. C...weighting based linear interpolation in state-of-the-art ASR systems. However, previous work doesn’t fully exploit the difference of mod- elling power of the...back-off based compact representation of n-gram dependent interpolation weights is pro- posed in this paper. This approach allows weight parameters to
Recurrent Pneumonia in Children: A Reasoned Diagnostic Approach and a Single Centre Experience.
Montella, Silvia; Corcione, Adele; Santamaria, Francesca
2017-01-29
Recurrent pneumonia (RP), i.e., at least two episodes of pneumonia in one year or three episodes ever with intercritical radiographic clearing of densities, occurs in 7.7%-9% of children with community-acquired pneumonia. In RP, the challenge is to discriminate between children with self-limiting or minor problems, that do not require a diagnostic work-up, and those with an underlying disease. The aim of the current review is to discuss a reasoned diagnostic approach to RP in childhood. Particular emphasis has been placed on which children should undergo a diagnostic work-up and which tests should be performed. A pediatric case series is also presented, in order to document a single centre experience of RP. A management algorithm for the approach to children with RP, based on the evidence from a literature review, is proposed. Like all algorithms, it is not meant to replace clinical judgment, but it should drive physicians to adopt a systematic approach to pediatric RP and provide a useful guide to the clinician.
Management of recurrent ameloblastoma of the jaws; a 40-year single institution experience
Hertog, D.; Schulten, E.A.J.M.; Leemans, C.R.; Winters, H.A.H.; van der Waal, I.
2011-01-01
Ameloblastoma is a histologically almost always benign odontogenic tumor with a high rate of recurrence if not removed completely. Therefore, radical surgery is the treatment of choice of a primary ameloblastoma. Of 18 patients with a recurrent ameloblastoma, previously treated by enucleation,
Flow dynamic study of a single-phase square NCL using recurrence ...
Indian Academy of Sciences (India)
For this reason, in heat transfer, the natural circulation loop (NCL) is used extensively. NCL works as a cooling ... Natural circulation loop; nonlinear dynamics; prediction of chaos; recurrence plot; recurrence quantification. PACS Nos 05.60.-k. 1. ..... [25] M Llop, N Gascons and F X Llauró, Int. J. Multiphas. Flow 73, 43 (2015).
Single-trial analysis of the neural correlates of speech quality perception.
Porbadnigk, Anne K; Treder, Matthias S; Blankertz, Benjamin; Antons, Jan-Niklas; Schleicher, Robert; Möller, Sebastian; Curio, Gabriel; Müller, Klaus-Robert
2013-10-01
Assessing speech quality perception is a challenge typically addressed in behavioral and opinion-seeking experiments. Only recently, neuroimaging methods were introduced, which were used to study the neural processing of quality at group level. However, our electroencephalography (EEG) studies show that the neural correlates of quality perception are highly individual. Therefore, it became necessary to establish dedicated machine learning methods for decoding subject-specific effects. The effectiveness of our methods is shown by the data of an EEG study that investigates how the quality of spoken vowels is processed neurally. Participants were asked to indicate whether they had perceived a degradation of quality (signal-correlated noise) in vowels, presented in an oddball paradigm. We find that the P3 amplitude is attenuated with increasing noise. Single-trial analysis allows one to show that this is partly due to an increasing jitter of the P3 component. A novel classification approach helps to detect trials with presumably non-conscious processing at the threshold of perception. We show that this approach uncovers a non-trivial confounder between neural hits and neural misses. The combined use of EEG signals and machine learning methods results in a significant 'neural' gain in sensitivity (in processing quality loss) when compared to standard behavioral evaluation; averaged over 11 subjects, this amounts to a relative improvement in sensitivity of 35%.
Techniques for extracting single-trial activity patterns from large-scale neural recordings
Churchland, Mark M; Yu, Byron M; Sahani, Maneesh; Shenoy, Krishna V
2008-01-01
Summary Large, chronically-implanted arrays of microelectrodes are an increasingly common tool for recording from primate cortex, and can provide extracellular recordings from many (order of 100) neurons. While the desire for cortically-based motor prostheses has helped drive their development, such arrays also offer great potential to advance basic neuroscience research. Here we discuss the utility of array recording for the study of neural dynamics. Neural activity often has dynamics beyond that driven directly by the stimulus. While governed by those dynamics, neural responses may nevertheless unfold differently for nominally identical trials, rendering many traditional analysis methods ineffective. We review recent studies – some employing simultaneous recording, some not – indicating that such variability is indeed present both during movement generation, and during the preceding premotor computations. In such cases, large-scale simultaneous recordings have the potential to provide an unprecedented view of neural dynamics at the level of single trials. However, this enterprise will depend not only on techniques for simultaneous recording, but also on the use and further development of analysis techniques that can appropriately reduce the dimensionality of the data, and allow visualization of single-trial neural behavior. PMID:18093826
Siri, Benoît; Berry, Hugues; Cessac, Bruno; Delord, Bruno; Quoy, Mathias
2008-12-01
We present a mathematical analysis of the effects of Hebbian learning in random recurrent neural networks, with a generic Hebbian learning rule, including passive forgetting and different timescales, for neuronal activity and learning dynamics. Previous numerical work has reported that Hebbian learning drives the system from chaos to a steady state through a sequence of bifurcations. Here, we interpret these results mathematically and show that these effects, involving a complex coupling between neuronal dynamics and synaptic graph structure, can be analyzed using Jacobian matrices, which introduce both a structural and a dynamical point of view on neural network evolution. Furthermore, we show that sensitivity to a learned pattern is maximal when the largest Lyapunov exponent is close to 0. We discuss how neural networks may take advantage of this regime of high functional interest.
International Nuclear Information System (INIS)
Song, Qiankun; Wang, Zidong
2007-01-01
In this Letter, the analysis problem for the existence and stability of periodic solutions is investigated for a class of general discrete-time recurrent neural networks with time-varying delays. For the neural networks under study, a generalized activation function is considered, and the traditional assumptions on the boundedness, monotony and differentiability of the activation functions are removed. By employing the latest free-weighting matrix method, an appropriate Lyapunov-Krasovskii functional is constructed and several sufficient conditions are established to ensure the existence, uniqueness, and globally exponential stability of the periodic solution for the addressed neural network. The conditions are dependent on both the lower bound and upper bound of the time-varying time delays. Furthermore, the conditions are expressed in terms of the linear matrix inequalities (LMIs), which can be checked numerically using the effective LMI toolbox in MATLAB. Two simulation examples are given to show the effectiveness and less conservatism of the proposed criteria
Song, Qiankun; Wang, Zidong
2007-08-01
In this Letter, the analysis problem for the existence and stability of periodic solutions is investigated for a class of general discrete-time recurrent neural networks with time-varying delays. For the neural networks under study, a generalized activation function is considered, and the traditional assumptions on the boundedness, monotony and differentiability of the activation functions are removed. By employing the latest free-weighting matrix method, an appropriate Lyapunov Krasovskii functional is constructed and several sufficient conditions are established to ensure the existence, uniqueness, and globally exponential stability of the periodic solution for the addressed neural network. The conditions are dependent on both the lower bound and upper bound of the time-varying time delays. Furthermore, the conditions are expressed in terms of the linear matrix inequalities (LMIs), which can be checked numerically using the effective LMI toolbox in MATLAB. Two simulation examples are given to show the effectiveness and less conservatism of the proposed criteria.
Pavlov, Maja J; Ceranic, Miljan S; Latincic, Stojan M; Sabljak, Predrag V; Kecmanovic, Dragutin M; Sugarbaker, Paul H
2017-09-07
With standard treatment of epithelial ovarian cancer (EOC), prognosis is very poor. The aim of this study is to show early and late results in patients who underwent cytoreductive surgery and intraperitoneal chemotherapy. This was a retrospective single centre study. All patients with advanced and recurrent ovarian cancer treated with cytoreductive surgery and hyperthermic intraperitoneal chemotherapy (HIPEC) or modified early postoperative intraperitoneal chemotherapy (EPIC) were included in the study. In the period 1995-2014, 116 patients were treated, 55 with primary EOC and 61 with recurrent EOC. The mean age was 59 years (26-74). Statistically, median survival time was significantly longer in the group with primary advanced cancer of the ovary (41.3 months) compared to relapsed ovarian cancer (27.3 months). Survival for the primary EOC was 65 and 24% at 3 and 5 years, respectively. Survival for recurrent EOC was 33 and 16% at 3 and 5 years, respectively. Mortality was 1/116 (0.8%). Morbidity was 11/116 (9.5%). Peritoneal cancer index (PCI) was ≤20 in 59 (51%) patients and statistically, their average survival was significantly longer than in the group of 57 (49%) patients with PCI >20 (p = 0.014). In advanced or recurrent EOC, a curative therapeutic approach was pursued that combined optimal cytoreductive surgery and intraperitoneal chemotherapy. PCI and timing of the intervention (primary or recurrent) were the strongest independent prognostic factors.
A novel flow diverter(Tubridge) for the treatment of recurrent aneurysms: A single-center experience
Energy Technology Data Exchange (ETDEWEB)
Zhang, Yong Xin; Huang, Qing Hai; Fang, Yibin; Yang, Peng Fei; Xu, Yi; Hong Bo; Liu, Jian Min [Dept. of Neurosurgery, Changhai Hospital, Second Military Medical University, Shanghai (China)
2017-09-15
The Tubridge flow diverter (FD) is a novel device aimed at reconstructing the parent artery and occluding complex aneurysms. Retreatment of recurrent aneurysms using the FD is challenging. We report our initial experience in the repair of aneurysm recurrence with the FD. A database was reviewed prospectively, and 8 patients with 8 recurrent aneurysms (mean size, 16.7 mm) were identified. Four aneurysms had previously ruptured. The previous aneurysm treatment consisted of coiling in 1 aneurysm and single-stent-assisted coiling in 7 aneurysms. The procedural complications and clinical and angiographic outcomes were analyzed. Six aneurysms were treated by using a single Tubridge FD alone, while the remaining 2 were treated with FD + coiling. The immediate results of the 8 aneurysms were that they all showed incomplete occlusion. Neither major ischemic nor hemorrhagic complications occurred; however, 1 patient experienced a vasospasm. Follow-up angiographies were available for 7 aneurysms; the mean follow-up was 16.9 months (7–36 months). Five aneurysms were completely occluded, whereas 2 had a residual neck. Severe asymptomatic stenosis of 1 parent artery of a vertebral artery dissecting aneurysm was found. All visible branches covered by the FD were patent. All patients were clinically assessed as having attained a favorable outcome (modified Rankin Scale score ≤ 2) at discharge and follow-up. In selected patients, the Tubridge FD can provide a safe and efficient option for the retreatment of recurrent aneurysms. Nevertheless, attention should be paid to several technical points.
Sanches, Rafael Faria; de Lima Osório, Flávia; Dos Santos, Rafael G; Macedo, Ligia R H; Maia-de-Oliveira, João Paulo; Wichert-Ana, Lauro; de Araujo, Draulio Barros; Riba, Jordi; Crippa, José Alexandre S; Hallak, Jaime E C
2016-02-01
Ayahuasca is an Amazonian botanical hallucinogenic brew which contains dimethyltryptamine, a 5-HT2A receptor agonist, and harmine, a monoamine-oxidase A inhibitor. Our group recently reported that ayahuasca administration was associated with fast-acting antidepressive effects in 6 depressive patients. The objective of the present work was to assess the antidepressive potentials of ayahuasca in a bigger sample and to investigate its effects on regional cerebral blood flow. In an open-label trial conducted in an inpatient psychiatric unit, 17 patients with recurrent depression received an oral dose of ayahuasca (2.2 mL/kg) and were evaluated with the Hamilton Rating Scale for Depression, the Montgomery-Åsberg Depression Rating Scale, the Brief Psychiatric Rating Scale, the Young Mania Rating Scale, and the Clinician Administered Dissociative States Scale during acute ayahuasca effects and 1, 7, 14, and 21 days after drug intake. Blood perfusion was assessed eight hours after drug administration by means of single photon emission tomography. Ayahuasca administration was associated with increased psychoactivity (Clinician Administered Dissociative States Scale) and significant score decreases in depression-related scales (Hamilton Rating Scale for Depression, Montgomery-Åsberg Depression Rating Scale, Brief Psychiatric Rating Scale) from 80 minutes to day 21. Increased blood perfusion in the left nucleus accumbens, right insula and left subgenual area, brain regions implicated in the regulation of mood and emotions, were observed after ayahuasca intake. Ayahuasca was well tolerated. Vomiting was the only adverse effect recorded, being reported by 47% of the volunteers. Our results suggest that ayahuasca may have fast-acting and sustained antidepressive properties. These results should be replicated in randomized, double-blind, placebo-controlled trials.
Directory of Open Access Journals (Sweden)
Michel Pascal
2011-01-01
Full Text Available Abstract Background Although Campylobacter is the leading cause of reported bacterial gastro-enteritis in industrialized countries, little is known on its recurrence. The objective of this study is to describe the risk and the patient characteristics of recurrent episodes of human campylobacteriosis reported in Quebec. Methods Laboratory-confirmed cases of campylobacteriosis reported in the province of Quebec, Canada, through ongoing surveillance between 1996 and 2006 were analyzed. The risk of having a recurrent episode of campylobacteriosis was described using life table estimates. Logistic regression was used to assess if gender, age and patient residential location were associated with an increased risk of recurrence. Results Compared to the baseline risk, the risk for a recurrent disease event was higher for a period of four years and followed a decreasing trend. This increased risk of a recurrent event was similar across gender, but higher for people from rural areas and lower for children under four years of age. Conclusions These results may suggest the absence of durable immunity or clinical resilience following a first episode of campylobacteriosis and periodical re-exposure, at least among cases reported through the surveillance system.
2011-01-01
Background Although Campylobacter is the leading cause of reported bacterial gastro-enteritis in industrialized countries, little is known on its recurrence. The objective of this study is to describe the risk and the patient characteristics of recurrent episodes of human campylobacteriosis reported in Quebec. Methods Laboratory-confirmed cases of campylobacteriosis reported in the province of Quebec, Canada, through ongoing surveillance between 1996 and 2006 were analyzed. The risk of having a recurrent episode of campylobacteriosis was described using life table estimates. Logistic regression was used to assess if gender, age and patient residential location were associated with an increased risk of recurrence. Results Compared to the baseline risk, the risk for a recurrent disease event was higher for a period of four years and followed a decreasing trend. This increased risk of a recurrent event was similar across gender, but higher for people from rural areas and lower for children under four years of age. Conclusions These results may suggest the absence of durable immunity or clinical resilience following a first episode of campylobacteriosis and periodical re-exposure, at least among cases reported through the surveillance system. PMID:21226938
Maslow, Bat-Sheva L; Budinetz, Tara; Sueldo, Carolina; Anspach, Erica; Engmann, Lawrence; Benadiva, Claudio; Nulsen, John C
2015-07-01
To compare the analysis of chromosome number from paraffin-embedded products of conception using single-nucleotide polymorphism (SNP) microarray with the recommended screening for the evaluation of couples presenting with recurrent pregnancy loss who do not have previous fetal cytogenetic data. We performed a retrospective cohort study including all women who presented for a new evaluation of recurrent pregnancy loss over a 2-year period (January 1, 2012, to December 31, 2013). All participants had at least two documented first-trimester losses and both the recommended screening tests and SNP microarray performed on at least one paraffin-embedded products of conception sample. Single-nucleotide polymorphism microarray identifies all 24 chromosomes (22 autosomes, X, and Y). Forty-two women with a total of 178 losses were included in the study. Paraffin-embedded products of conception from 62 losses were sent for SNP microarray. Single-nucleotide polymorphism microarray successfully diagnosed fetal chromosome number in 71% (44/62) of samples, of which 43% (19/44) were euploid and 57% (25/44) were noneuploid. Seven of 42 (17%) participants had abnormalities on recurrent pregnancy loss screening. The per-person detection rate for a cause of pregnancy loss was significantly higher in the SNP microarray (0.50; 95% confidence interval [CI] 0.36-0.64) compared with recurrent pregnancy loss evaluation (0.17; 95% CI 0.08-0.31) (P=.002). Participants with one or more euploid loss identified on paraffin-embedded products of conception were significantly more likely to have an abnormality on recurrent pregnancy loss screening than those with only noneuploid results (P=.028). The significance remained when controlling for age, number of losses, number of samples, and total pregnancies. These results suggest that SNP microarray testing of paraffin-embedded products of conception is a valuable tool for the evaluation of recurrent pregnancy loss in patients without prior fetal
Single-trial analysis of the neural correlates of speech quality perception
Porbadnigk, Anne K.; Treder, Matthias S.; Blankertz, Benjamin; Antons, Jan-Niklas; Schleicher, Robert; Möller, Sebastian; Curio, Gabriel; Müller, Klaus-Robert
2013-10-01
Objective. Assessing speech quality perception is a challenge typically addressed in behavioral and opinion-seeking experiments. Only recently, neuroimaging methods were introduced, which were used to study the neural processing of quality at group level. However, our electroencephalography (EEG) studies show that the neural correlates of quality perception are highly individual. Therefore, it became necessary to establish dedicated machine learning methods for decoding subject-specific effects. Approach. The effectiveness of our methods is shown by the data of an EEG study that investigates how the quality of spoken vowels is processed neurally. Participants were asked to indicate whether they had perceived a degradation of quality (signal-correlated noise) in vowels, presented in an oddball paradigm. Main results. We find that the P3 amplitude is attenuated with increasing noise. Single-trial analysis allows one to show that this is partly due to an increasing jitter of the P3 component. A novel classification approach helps to detect trials with presumably non-conscious processing at the threshold of perception. We show that this approach uncovers a non-trivial confounder between neural hits and neural misses. Significance. The combined use of EEG signals and machine learning methods results in a significant ‘neural’ gain in sensitivity (in processing quality loss) when compared to standard behavioral evaluation; averaged over 11 subjects, this amounts to a relative improvement in sensitivity of 35%.
Yan, Liang; Zhang, Lu; Zhu, Bo; Zhang, Jingying; Jiao, Zongxia
2017-10-01
Permanent magnet spherical actuator (PMSA) is a multi-variable featured and inter-axis coupled nonlinear system, which unavoidably compromises its motion control implementation. Uncertainties such as external load and friction torque of ball bearing and manufacturing errors also influence motion performance significantly. Therefore, the objective of this paper is to propose a controller based on a single neural adaptive (SNA) algorithm and a neural network (NN) identifier optimized with a particle swarm optimization (PSO) algorithm to improve the motion stability of PMSA with three-dimensional magnet arrays. The dynamic model and computed torque model are formulated for the spherical actuator, and a dynamic decoupling control algorithm is developed. By utilizing the global-optimization property of the PSO algorithm, the NN identifier is trained to avoid locally optimal solution and achieve high-precision compensations to uncertainties. The employment of the SNA controller helps to reduce the effect of compensation errors and convert the system to a stable one, even if there is difference between the compensations and uncertainties due to external disturbances. A simulation model is established, and experiments are conducted on the research prototype to validate the proposed control algorithm. The amplitude of the parameter perturbation is set to 5%, 10%, and 15%, respectively. The strong robustness of the proposed hybrid algorithm is validated by the abundant simulation data. It shows that the proposed algorithm can effectively compensate the influence of uncertainties and eliminate the effect of inter-axis couplings of the spherical actuator.
International Nuclear Information System (INIS)
Tan, Jennifer; Hui, Andrew C; Heriot, Alexander G.; Mackay, Jack; Lynch, A. Craig; Van Dyk, Sylvia; Bressel, Mathias; Fox, Chris D.; Leong, Trevor; Ngan, Samuel Y.
2013-01-01
This study aims to evaluate the feasibility and outcomes of intraoperative radiotherapy (IORT) using high-dose-rate (HDR) brachytherapy for locally advanced or recurrent rectal cancers. Despite preoperative chemoradiation, patients with locally advanced or recurrent rectal cancers undergoing surgery remain at high risk of local recurrence. Intensification of radiation with IORT may improve local control. This is a prospective non-randomised study. Eligible patients were those with T4 rectal cancer or pelvic recurrence, deemed suitable for radical surgery but at high risk of positive resection margins, without evidence of metastasis. Chemoradiation was followed by radical surgery. Ten gray (Gy) was delivered to tumour bed via an IORT applicator at time of surgery. There were 15% primary and 85% recurrent cancers. The 71% received preoperative chemoradiation. R0, R1 and R2 resections were 70%, 22% and 7%, respectively. IORT was successfully delivered in 27 of 30 registered patients (90% (95% confidence interval (CI)=73–98)) at a median reported time of 12 weeks (interquartile range (IQR)=10–16) after chemoradiation. Mean IORT procedure and delivery times were 63 minutes (range 22–105 minutes). Ten patients (37% (95% CI=19–58)) experienced grade 3 or 4 toxicities (three wound, four abscesses, three soft tissue, three bowel obstructions, three ureteric obstructions and two sensory neuropathies). Local recurrence-free, failure-free and overall survival rates at 2.5 years were 68% (95% CI=52–89), 37% (95% CI=23–61) and 82% (95% CI=68–98), respectively. The addition of IORT to radical surgery for T4 or recurrent rectal cancer is feasible. It can be delivered safely with low morbidity and good tumour outcomes.
Aoki, Masahiko; Hatayama, Yoshiomi; Kawaguchi, Hideo; Hirose, Katsumi; Sato, Mariko; Akimoto, Hiroyoshi; Miura, Hiroyuki; Ono, Shuichi; Takai, Yoshihiro
2016-01-01
The purpose of this study was to investigate clinical outcomes following stereotactic body radiotherapy (SBRT) for lung metastases as oligo-recurrence. From May 2003 to June 2014, records for 66 patients with 76 oligo-recurrences in the lungs treated with SBRT were retrospectively reviewed. Oligo-recurrence primary sites and patient numbers were as follows: lungs, 31; colorectal, 13; head and neck, 10; esophagus, 3; uterus, 3; and others, 6. The median SBRT dose was 50 Gy (range, 45-60 Gy) administered in a median of 5 (range, 5-9) fractions. All patients received SBRT, with no acute toxicity. Surviving patients had a median follow-up time of 36.5 months. The 3-year rates of local control, overall survival and disease-free survival were 90.6%, 76.0% and 53.7%, respectively. Longer disease-free interval from initial treatment to SBRT, and non-colorectal cancer were both associated with favorable outcomes. Disease progression after SBRT occurred in 31 patients, most with distant metastases (n = 24) [among whom, 87.5% (n = 21) had new lung metastases]. Among these 21 patients, 12 were judged as having a second oligo-recurrence. Additional SBRT was performed for these 12 patients, and all 12 tumors were controlled without disease progression. Three patients (4.5%) developed Grade 2 radiation pneumonitis. No other late adverse events of Grade ≥2 were identified. Thus, SBRT for oligo-recurrence achieved acceptable tumor control, with additional SBRT also effective for selected patients with a second oligo-recurrence after primary SBRT. © The Author 2015. Published by Oxford University Press on behalf of The Japan Radiation Research Society and Japanese Society for Radiation Oncology.
NONLINEAR SYSTEM MODELING USING SINGLE NEURON CASCADED NEURAL NETWORK FOR REAL-TIME APPLICATIONS
Directory of Open Access Journals (Sweden)
S. Himavathi
2012-04-01
Full Text Available Neural Networks (NN have proved its efficacy for nonlinear system modeling. NN based controllers and estimators for nonlinear systems provide promising alternatives to the conventional counterpart. However, NN models have to meet the stringent requirements on execution time for its effective use in real time applications. This requires the NN model to be structurally compact and computationally less complex. In this paper a parametric method of analysis is adopted to determine the compact and faster NN model among various neural network architectures. This work proves through analysis and examples that the Single Neuron Cascaded (SNC architecture is distinct in providing compact and simpler models requiring lower execution time. The unique structural growth of SNC architecture enables automation in design. The SNC Network is shown to combine the advantages of both single and multilayer neural network architectures. Extensive analysis on selected architectures and their models for four benchmark nonlinear theoretical plants and a practical application are tested. A performance comparison of the NN models is presented to demonstrate the superiority of the single neuron cascaded architecture for online real time applications.
Ding, Lei; Xiao, Lin; Liao, Bolin; Lu, Rongbo; Peng, Hua
2017-01-01
To obtain the online solution of complex-valued systems of linear equation in complex domain with higher precision and higher convergence rate, a new neural network based on Zhang neural network (ZNN) is investigated in this paper. First, this new neural network for complex-valued systems of linear equation in complex domain is proposed and theoretically proved to be convergent within finite time. Then, the illustrative results show that the new neural network model has the higher precision and the higher convergence rate, as compared with the gradient neural network (GNN) model and the ZNN model. Finally, the application for controlling the robot using the proposed method for the complex-valued systems of linear equation is realized, and the simulation results verify the effectiveness and superiorness of the new neural network for the complex-valued systems of linear equation.
Neural-network-based single-sided non-enwrapping power loss tester
Passadis, K; Beckley, P
2003-01-01
It is preferable to be able to assess the power loss of electrical steels during production. When a single-sided tester is used, flux sensing is undertaken from one side only and hence some leakage flux above the strip may not captured by the sensing coils. Therefore, the disadvantage of a single-sided non-enwrapping tester lies in the measurement of the flux density in the material. A neural network was successfully used to 'predict' the correct level of flux density for accurate assessment of power loss.
Directory of Open Access Journals (Sweden)
Matheus Costa dos Reis
2014-01-01
Full Text Available This study was carried out to obtain the estimates of genetic variance and covariance components related to intra- and interpopulation in the original populations (C0 and in the third cycle (C3 of reciprocal recurrent selection (RRS which allows breeders to define the best breeding strategy. For that purpose, the half-sib progenies of intrapopulation (P11 and P22 and interpopulation (P12 and P21 from populations 1 and 2 derived from single-cross hybrids in the 0 and 3 cycles of the reciprocal recurrent selection program were used. The intra- and interpopulation progenies were evaluated in a 10×10 triple lattice design in two separate locations. The data for unhusked ear weight (ear weight without husk and plant height were collected. All genetic variance and covariance components were estimated from the expected mean squares. The breakdown of additive variance into intrapopulation and interpopulation additive deviations (στ2 and the covariance between these and their intrapopulation additive effects (CovAτ found predominance of the dominance effect for unhusked ear weight. Plant height for these components shows that the intrapopulation additive effect explains most of the variation. Estimates for intrapopulation and interpopulation additive genetic variances confirm that populations derived from single-cross hybrids have potential for recurrent selection programs.
Liu, Tao; Huang, Jie
2017-04-17
This paper presents a discrete-time recurrent neural network approach to solving systems of linear equations with two features. First, the system of linear equations may not have a unique solution. Second, the system matrix is not known precisely, but a sequence of matrices that converges to the unknown system matrix exponentially is known. The problem is motivated from solving the output regulation problem for linear systems. Thus, an application of our main result leads to an online solution to the output regulation problem for linear systems.
Single-Cell Memory Regulates a Neural Circuit for Sensory Behavior.
Kobayashi, Kyogo; Nakano, Shunji; Amano, Mutsuki; Tsuboi, Daisuke; Nishioka, Tomoki; Ikeda, Shingo; Yokoyama, Genta; Kaibuchi, Kozo; Mori, Ikue
2016-01-05
Unveiling the molecular and cellular mechanisms underlying memory has been a challenge for the past few decades. Although synaptic plasticity is proven to be essential for memory formation, the significance of "single-cell memory" still remains elusive. Here, we exploited a primary culture system for the analysis of C. elegans neurons and show that a single thermosensory neuron has an ability to form, retain, and reset a temperature memory. Genetic and proteomic analyses found that the expression of the single-cell memory exhibits inter-individual variability, which is controlled by the evolutionarily conserved CaMKI/IV and Raf pathway. The variable responses of a sensory neuron influenced the neural activity of downstream interneurons, suggesting that modulation of the sensory neurons ultimately determines the behavioral output in C. elegans. Our results provide proof of single-cell memory and suggest that the individual differences in neural responses at the single-cell level can confer individuality. Copyright © 2016 The Authors. Published by Elsevier Inc. All rights reserved.
Teraishi, Toshiya; Hori, Hiroaki; Sasayama, Daimei; Matsuo, Junko; Ogawa, Shintaro; Ishida, Ikki; Nagashima, Anna; Kinoshita, Yukiko; Ota, Miho; Hattori, Kotaro; Higuchi, Teruhiko; Kunugi, Hiroshi
2015-01-01
Previous studies consistently reported increased harm avoidance (HA) assessed with the Temperament and Character Inventory (TCI) in patients with major depressive disorder (MDD). However, such findings may have been related with depression severity and number of depressive episodes. The aims of the present study were twofold: to examine TCI personality profile in remitted MDD (DSM-IV) patients and to compare TCI personality between MDD patients with single episode (SGL-MDD) and those with recurrent episodes (REC-MDD) in order to elucidate personality profile associated with recurrence. TCI was administered to 86 outpatients with remitted SGL-MDD (12 male and 17 female patients; mean age 43.2 ± 12.1 years) and REC-MDD (26 male and 31 female patients; 40.3 ± 11.6 years), and 529 healthy controls (225 men and 304 women; 43.4 ± 15.5 years), matched for age, sex and education years. Logistic regression analyses were performed in which single/recurrent episodes of depression were the dependent variable and age, sex, age of onset, family history of psychiatric disease and TCI scores were entered as possible predictors. The remitted MDD patients had significantly higher scores on HA (P personality profile between remitted MDD patients and controls, and between remitted REC-MDD and SGL-MDD patients, suggesting that they are trait markers. HA and fatigability might be useful to assess risk for recurrence of depression. © 2014 The Authors. Psychiatry and Clinical Neurosciences © 2014 Japanese Society of Psychiatry and Neurology.
Directory of Open Access Journals (Sweden)
Xiaoqing Zhu
2016-02-01
Full Text Available Developing a model of primate neural tube (NT development is important to promote many NT disorder studies in model organisms. Here, we report a robust and stable system to allow for clonal expansion of single monkey neuroepithelial stem cells (NESCs to develop into miniature NT-like structures. Single NESCs can produce functional neurons in vitro, survive, and extensively regenerate neuron axons in monkey brain. NT formation and NESC maintenance depend on high metabolism activity and Wnt signaling. NESCs are regionally restricted to a telencephalic fate. Moreover, single NESCs can turn into radial glial progenitors (RGPCs. The transition is accurately regulated by Wnt signaling through regulation of Notch signaling and adhesion molecules. Finally, using the “NESC-TO-NTs” system, we model the functions of folic acid (FA on NT closure and demonstrate that FA can regulate multiple mechanisms to prevent NT defects. Our system is ideal for studying NT development and diseases.
Single- versus Multiobjective Optimization for Evolution of Neural Controllers in Ms. Pac-Man
Directory of Open Access Journals (Sweden)
Tse Guan Tan
2013-01-01
Full Text Available The objective of this study is to focus on the automatic generation of game artificial intelligence (AI controllers for Ms. Pac-Man agent by using artificial neural network (ANN and multiobjective artificial evolution. The Pareto Archived Evolution Strategy (PAES is used to generate a Pareto optimal set of ANNs that optimize the conflicting objectives of maximizing Ms. Pac-Man scores (screen-capture mode and minimizing neural network complexity. This proposed algorithm is called Pareto Archived Evolution Strategy Neural Network or PAESNet. Three different architectures of PAESNet were investigated, namely, PAESNet with fixed number of hidden neurons (PAESNet_F, PAESNet with varied number of hidden neurons (PAESNet_V, and the PAESNet with multiobjective techniques (PAESNet_M. A comparison between the single- versus multiobjective optimization is conducted in both training and testing processes. In general, therefore, it seems that PAESNet_F yielded better results in training phase. But the PAESNet_M successfully reduces the runtime operation and complexity of ANN by minimizing the number of hidden neurons needed in hidden layer and also it provides better generalization capability for controlling the game agent in a nondeterministic and dynamic environment.
Directory of Open Access Journals (Sweden)
Ahnsei Shon
2017-12-01
Full Text Available Recently, implantable devices have become widely used in neural prostheses because they eliminate endemic drawbacks of conventional percutaneous neural interface systems. However, there are still several issues to be considered: low-efficiency wireless power transmission; wireless data communication over restricted operating distance with high power consumption; and limited functionality, working either as a neural signal recorder or as a stimulator. To overcome these issues, we suggest a novel implantable wireless neural interface system for simultaneous neural signal recording and stimulation using a single cuff electrode. By using widely available commercial off-the-shelf (COTS components, an easily reconfigurable implantable wireless neural interface system was implemented into one compact module. The implantable device includes a wireless power consortium (WPC-compliant power transmission circuit, a medical implant communication service (MICS-band-based radio link and a cuff-electrode path controller for simultaneous neural signal recording and stimulation. During in vivo experiments with rabbit models, the implantable device successfully recorded and stimulated the tibial and peroneal nerves while communicating with the external device. The proposed system can be modified for various implantable medical devices, especially such as closed-loop control based implantable neural prostheses requiring neural signal recording and stimulation at the same time.
Sikora, Grzegorz; Wyłomańska, Agnieszka; Gajda, Janusz; Solé, Laura; Akin, Elizabeth J.; Tamkun, Michael M.; Krapf, Diego
2017-12-01
Protein and lipid nanodomains are prevalent on the surface of mammalian cells. In particular, it has been recently recognized that ion channels assemble into surface nanoclusters in the soma of cultured neurons. However, the interactions of these molecules with surface nanodomains display a considerable degree of heterogeneity. Here, we investigate this heterogeneity and develop statistical tools based on the recurrence of individual trajectories to identify subpopulations within ion channels in the neuronal surface. We specifically study the dynamics of the K+ channel Kv1.4 and the Na+ channel Nav1.6 on the surface of cultured hippocampal neurons at the single-molecule level. We find that both these molecules are expressed in two different forms with distinct kinetics with regards to surface interactions, emphasizing the complex proteomic landscape of the neuronal surface. Further, the tools presented in this work provide new methods for the analysis of membrane nanodomains, transient confinement, and identification of populations within single-particle trajectories.
International Nuclear Information System (INIS)
Beal, Kathryn; Abrey, Lauren E; Gutin, Philip H
2011-01-01
Surgical resection followed by radiotherapy and temozolomide in newly diagnosed glioblastoma can prolong survival, but it is not curative. For patients with disease progression after frontline therapy, there is no standard of care, although further surgery, chemotherapy, and radiotherapy may be used. Antiangiogenic therapies may be appropriate for treating glioblastomas because angiogenesis is critical to tumor growth. In a large, noncomparative phase II trial, bevacizumab was evaluated alone and with irinotecan in patients with recurrent glioblastoma; combination treatment was associated with an estimated 6-month progression-free survival (PFS) rate of 50.3%, a median overall survival of 8.9 months, and a response rate of 37.8%. Single-agent bevacizumab also exceeded the predetermined threshold of activity for salvage chemotherapy (6-month PFS rate, 15%), achieving a 6-month PFS rate of 42.6% (p < 0.0001). On the basis of these results and those from another phase II trial, the US Food and Drug Administration granted accelerated approval of single-agent bevacizumab for the treatment of glioblastoma that has progressed following prior therapy. Potential antiangiogenic agents-such as cilengitide and XL184-also show evidence of single-agent activity in recurrent glioblastoma. Moreover, the use of antiangiogenic agents with radiation at disease progression may improve the therapeutic ratio of single-modality approaches. Overall, these agents appear to be well tolerated, with adverse event profiles similar to those reported in studies of other solid tumors. Further research is needed to determine the role of antiangiogenic therapy in frontline treatment and to identify the optimal schedule and partnering agents for use in combination therapy
Investigating single-word syntactic primes in naming tasks: a recurrent network approach.
Farrar, W T
1998-04-01
Three experiments compared the qualitative pattern of participants' word-naming performance in a syntactic priming task with the qualitative pattern of performance generated by a recurrent network model. Experiments 1 and 2 demonstrated that when participants had a 600-ms response deadline, the appropriateness of a syntactic prime affected their naming times for high-frequency words but not low-frequency words. However, Experiment 2 also demonstrated that participants made the most pronunciation errors when naming inconsistent low-frequency words (e.g., pint) that were preceded by an inappropriate prime. The results of Experiment 3 suggest that participants' naming times to both high- and low-frequency words are affected by syntactic primes when there is no response deadline. The implication of these findings for the study of syntactic priming in English and other languages is discussed.
Baheti, Akshay D; Sewatkar, Rani; Hornick, Jason L; Saboo, Sachin S; Jagannathan, Jyothi P; Ramaiya, Nikhil H; Tirumani, Sree Harsha
2015-01-01
The objective was to study the imaging features of primary and metastatic intrathoracic synovial sarcoma (ISS). We reviewed the imaging of 42 patients with ISS (31 pleural, 7 intrapulmonary, 4 mediastinal), with baseline imaging of 19 patients and follow-up imaging in all. Primary tumors (n=19) were well circumscribed (15/19), homogeneous or heterogeneously enhancing, with mean size 9.1 cm (range: 1.8-25 cm). Recurrent/metastatic disease developed in 28/42 patients (67%). Lung was the most common site of metastases (21/28), with most of them predominantly pleural (11/21). On pathology, 30 patients had monophasic tumors, and 6 each had biphasic tumors and poorly differentiated tumors. Intrathoracic SS most commonly presents as a large heterogeneous pleural mass without associated adenopathy. Lung metastases are often pleural based and ipsilateral. Copyright © 2015 Elsevier Inc. All rights reserved.
2017-11-07
Adult Anaplastic Astrocytoma; Recurrent Grade III Glioma; Recurrent Grade IV Glioma; Adult Anaplastic Oligodendroglioma; Adult Brain Tumor; Adult Giant Cell Glioblastoma; Adult Glioblastoma; Adult Gliosarcoma; Adult Mixed Glioma; Recurrent Adult Brain Tumor; Adult Anaplastic Oligoastrocytoma; Recurrent High Grade Glioma
Liao, Chin-Wen; Lu, Chien-Yu
2011-06-01
The state estimation problem for discrete-time recurrent neural networks with both interval discrete and infinite-distributed time-varying delays is studied in this paper, where interval discrete time-varying delay is in a given range. The activation functions are assumed to be globally Lipschitz continuous. A delay-dependent condition for the existence of state estimators is proposed based on new bounding techniques. Via solutions to certain linear matrix inequalities, general full-order state estimators are designed that ensure globally asymptotic stability. The significant feature is that no inequality is needed for seeking upper bounds for the inner product between two vectors, which can reduce the conservatism of the criterion by employing the new bounding techniques. Two illustrative examples are given to demonstrate the effectiveness and applicability of the proposed approach.
Gu, Chengyuan; Qu, Yuanyuan; Zhang, Guiming; Sun, LiJiang; Zhu, Yao; Ye, Dingwei
2015-10-13
Adiponectin has been implicated in prostate cancer (PCa) aggressiveness. However, the role of genetic variations in the adiponectin (ADIPOQ) gene in PCa progression remains unknown. To determine whether genetic variants in ADIPOQ are associated with the risk of biochemical recurrence (BCR) after radical prostatectomy (RP). We evaluated three common ADIPOQ polymorphisms in 728 men with clinically localized PCa who underwent RP. Multivariable Cox proportional hazards models and Kaplan-Meier analysis were used to assess their prognostic significance on BCR. The plasma adiponectin concentrations were measured by enzyme-linked immunosorbent assay. ADIPOQ rs182052 variant allele was associated with both increased risk of BCR [HR: 2.44; 95% confidence interval (CI): 1.57-;3.79, P = 6×10-5] and decreased adiponectin level (β = -0.048, P = 0.004). Stratified analyses demonstrated that the association was more pronounced in men with higher visceral adipose tissue. Our data support that the ADIPOQ rs182052 SNP may be a predictive biomarker for BCR after RP by a possible mechanism of altering the adiponectin level. If validated, genetic predictors of outcome may help individualizing treatment for PCa.
Hutt, Axel; Buhry, Laure
2014-12-01
Anaesthetic agents are known to affect extra-synaptic GABAergic receptors, which induce tonic inhibitory currents. Since these receptors are very sensitive to small concentrations of agents, they are supposed to play an important role in the underlying neural mechanism of general anaesthesia. Moreover anaesthetic agents modulate the encephalographic activity (EEG) of subjects and hence show an effect on neural populations. To understand better the tonic inhibition effect in single neurons on neural populations and hence how it affects the EEG, the work considers single neurons and neural populations in a steady-state and studies numerically and analytically the modulation of their firing rate and nonlinear gain with respect to different levels of tonic inhibition. We consider populations of both type-I (Leaky Integrate-and-Fire model) and type-II (Morris-Lecar model) neurons. To bridge the single neuron description to the population description analytically, a recently proposed statistical approach is employed which allows to derive new analytical expressions for the population firing rate for type-I neurons. In addition, the work shows the derivation of a novel transfer function for type-I neurons as considered in neural mass models and studies briefly the interaction of synaptic and extra-synaptic inhibition. We reveal a strong subtractive and divisive effect of tonic inhibition in type-I neurons, i.e. a shift of the firing rate to higher excitation levels accompanied by a change of the nonlinear gain. Tonic inhibition shortens the excitation window of type-II neurons and their populations while maintaining the nonlinear gain. The gained results are interpreted in the context of recent experimental findings under propofol-induced anaesthesia.
Cui, Yiqian; Shi, Junyou; Wang, Zili
2017-11-01
Built-in tests (BITs) are widely used in mechanical systems to perform state identification, whereas the BIT false and missed alarms cause trouble to the operators or beneficiaries to make correct judgments. Artificial neural networks (ANN) are previously used for false and missed alarms identification, which has the features such as self-organizing and self-study. However, these ANN models generally do not incorporate the temporal effect of the bottom-level threshold comparison outputs and the historical temporal features are not fully considered. To improve the situation, this paper proposes a new integrated BIT design methodology by incorporating a novel type of dynamic neural networks (DNN) model. The new DNN model is termed as Forward IIR & Recurrent FIR DNN (FIRF-DNN), where its component neurons, network structures, and input/output relationships are discussed. The condition monitoring false and missed alarms reduction implementation scheme based on FIRF-DNN model is also illustrated, which is composed of three stages including model training, false and missed alarms detection, and false and missed alarms suppression. Finally, the proposed methodology is demonstrated in the application study and the experimental results are analyzed.
Single Image Super-Resolution Based on Multi-Scale Competitive Convolutional Neural Network.
Du, Xiaofeng; Qu, Xiaobo; He, Yifan; Guo, Di
2018-03-06
Deep convolutional neural networks (CNNs) are successful in single-image super-resolution. Traditional CNNs are limited to exploit multi-scale contextual information for image reconstruction due to the fixed convolutional kernel in their building modules. To restore various scales of image details, we enhance the multi-scale inference capability of CNNs by introducing competition among multi-scale convolutional filters, and build up a shallow network under limited computational resources. The proposed network has the following two advantages: (1) the multi-scale convolutional kernel provides the multi-context for image super-resolution, and (2) the maximum competitive strategy adaptively chooses the optimal scale of information for image reconstruction. Our experimental results on image super-resolution show that the performance of the proposed network outperforms the state-of-the-art methods.
Evaluation of deep neural networks for single image super-resolution in a maritime context
Nieuwenhuizen, Robert P. J.; Kruithof, Maarten; Schutte, Klamer
2017-10-01
High resolution imagery is of crucial importance for the performance on visual recognition tasks. Super-resolution (SR) reconstruction algorithms aim to enhance the image resolution beyond the capability of the image sensor being used. Traditional SR algorithms approach this inverse problem using physical models for the image formation combined with a regularization function to prevent instabilities in the solution. Recently deep neural networks have been put forward as an alternative approach to the SR reconstruction problem. They learn a mapping from low resolution images to their high resolution counterparts from pairs of training images, which allows them to capture more specific information about the space of possible solutions than traditional regularization functions. These networks have achieved state-of-the-art performance on single image SR for sets of generic test images. Here we investigate whether the same performance can be realized when these neural networks for single image SR are applied specifically in the maritime domain. In particular we investigate their ability to reconstruct undersampled images of ships at sea, and demonstrate that the performance is similar to what is achieved on generic test images. In addition we quantify the gain in performance that is achieved when the networks are trained specifically on images of ships, which allows the networks to capture more prior knowledge about the space of possible solutions. Finally we show that the performance deteriorates when the resolution of test images is limited by image blur, for example due to diffraction, rather than undersampling. This highlights the importance of using representative training data that account for the part of the image formation process that limits the resolution in the sensor data.
Single- and Multiple-Objective Optimization with Differential Evolution and Neural Networks
Rai, Man Mohan
2006-01-01
Genetic and evolutionary algorithms have been applied to solve numerous problems in engineering design where they have been used primarily as optimization procedures. These methods have an advantage over conventional gradient-based search procedures became they are capable of finding global optima of multi-modal functions and searching design spaces with disjoint feasible regions. They are also robust in the presence of noisy data. Another desirable feature of these methods is that they can efficiently use distributed and parallel computing resources since multiple function evaluations (flow simulations in aerodynamics design) can be performed simultaneously and independently on ultiple processors. For these reasons genetic and evolutionary algorithms are being used more frequently in design optimization. Examples include airfoil and wing design and compressor and turbine airfoil design. They are also finding increasing use in multiple-objective and multidisciplinary optimization. This lecture will focus on an evolutionary method that is a relatively new member to the general class of evolutionary methods called differential evolution (DE). This method is easy to use and program and it requires relatively few user-specified constants. These constants are easily determined for a wide class of problems. Fine-tuning the constants will off course yield the solution to the optimization problem at hand more rapidly. DE can be efficiently implemented on parallel computers and can be used for continuous, discrete and mixed discrete/continuous optimization problems. It does not require the objective function to be continuous and is noise tolerant. DE and applications to single and multiple-objective optimization will be included in the presentation and lecture notes. A method for aerodynamic design optimization that is based on neural networks will also be included as a part of this lecture. The method offers advantages over traditional optimization methods. It is more
Kember, Guy; Ardell, Jeffrey L; Shivkumar, Kalyanam; Armour, J Andrew
2017-01-01
The cardiac nervous system continuously controls cardiac function whether or not pathology is present. While myocardial infarction typically has a major and catastrophic impact, population studies have shown that longer-term risk for recurrent myocardial infarction and the related potential for sudden cardiac death depends mainly upon standard atherosclerotic variables and autonomic nervous system maladaptations. Investigative neurocardiology has demonstrated that autonomic control of cardiac function includes local circuit neurons for networked control within the peripheral nervous system. The structural and adaptive characteristics of such networked interactions define the dynamics and a new normal for cardiac control that results in the aftermath of recurrent myocardial infarction and/or unstable angina that may or may not precipitate autonomic derangement. These features are explored here via a mathematical model of cardiac regulation. A main observation is that the control environment during pathology is an extrapolation to a setting outside prior experience. Although global bounds guarantee stability, the resulting closed-loop dynamics exhibited while the network adapts during pathology are aptly described as 'free-floating' in order to emphasize their dependence upon details of the network structure. The totality of the results provide a mechanistic reasoning that validates the clinical practice of reducing sympathetic efferent neuronal tone while aggressively targeting autonomic derangement in the treatment of ischemic heart disease.
Directory of Open Access Journals (Sweden)
Guy Kember
Full Text Available The cardiac nervous system continuously controls cardiac function whether or not pathology is present. While myocardial infarction typically has a major and catastrophic impact, population studies have shown that longer-term risk for recurrent myocardial infarction and the related potential for sudden cardiac death depends mainly upon standard atherosclerotic variables and autonomic nervous system maladaptations. Investigative neurocardiology has demonstrated that autonomic control of cardiac function includes local circuit neurons for networked control within the peripheral nervous system. The structural and adaptive characteristics of such networked interactions define the dynamics and a new normal for cardiac control that results in the aftermath of recurrent myocardial infarction and/or unstable angina that may or may not precipitate autonomic derangement. These features are explored here via a mathematical model of cardiac regulation. A main observation is that the control environment during pathology is an extrapolation to a setting outside prior experience. Although global bounds guarantee stability, the resulting closed-loop dynamics exhibited while the network adapts during pathology are aptly described as 'free-floating' in order to emphasize their dependence upon details of the network structure. The totality of the results provide a mechanistic reasoning that validates the clinical practice of reducing sympathetic efferent neuronal tone while aggressively targeting autonomic derangement in the treatment of ischemic heart disease.
Directory of Open Access Journals (Sweden)
Oren J Becher
Full Text Available The PI3K/Akt/mTOR signaling pathway is aberrantly activated in various pediatric tumors. We conducted a phase I study of the Akt inhibitor perifosine in patients with recurrent/refractory pediatric CNS and solid tumors. This was a standard 3+3 open-label dose-escalation study to assess pharmacokinetics, describe toxicities, and identify the MTD for single-agent perifosine. Five dose levels were investigated, ranging from 25 to 125 mg/m2/day for 28 days per cycle. Twenty-three patients (median age 10 years, range 4-18 years with CNS tumors (DIPG [n = 3], high-grade glioma [n = 5], medulloblastoma [n = 2], ependymoma [n = 3], neuroblastoma (n = 8, Wilms tumor (n = 1, and Ewing sarcoma (n = 1 were treated. Only one DLT occurred (grade 4 hyperuricemia at dose level 4. The most common grade 3 or 4 toxicity at least possibly related to perifosine was neutropenia (8.7%, with the remaining grade 3 or 4 toxicities (fatigue, hyperglycemia, fever, hyperuricemia, and catheter-related infection occurring in one patient each. Pharmacokinetics was dose-saturable at doses above 50 mg/m2/day with significant inter-patient variability, consistent with findings reported in adult studies. One patient with DIPG (dose level 5 and 4 of 5 patients with high-grade glioma (dose levels 2 and 3 experienced stable disease for two months. Five subjects with neuroblastoma (dose levels 1 through 4 achieved stable disease which was prolonged (≥11 months in three. No objective responses were noted. In conclusion, the use of perifosine was safe and feasible in patients with recurrent/refractory pediatric CNS and solid tumors. An MTD was not defined by the 5 dose levels investigated. Our RP2D is 50 mg/m2/day.
Single-ended prediction of listening effort using deep neural networks.
Huber, Rainer; Krüger, Melanie; Meyer, Bernd T
2018-03-01
The effort required to listen to and understand noisy speech is an important factor in the evaluation of noise reduction schemes. This paper introduces a model for Listening Effort prediction from Acoustic Parameters (LEAP). The model is based on methods from automatic speech recognition, specifically on performance measures that quantify the degradation of phoneme posteriorgrams produced by a deep neural net: Noise or artifacts introduced by speech enhancement often result in a temporal smearing of phoneme representations, which is measured by comparison of phoneme vectors. This procedure does not require a priori knowledge about the processed speech, and is therefore single-ended. The proposed model was evaluated using three datasets of noisy speech signals with listening effort ratings obtained from normal hearing and hearing impaired subjects. The prediction quality was compared to several baseline models such as the ITU-T standard P.563 for single-ended speech quality assessment, the American National Standard ANIQUE+ for single-ended speech quality assessment, and a single-ended SNR estimator. In all three datasets, the proposed new model achieved clearly better prediction accuracies than the baseline models; correlations with subjective ratings were above 0.9. So far, the model is trained on the specific noise types used in the evaluation. Future work will be concerned with overcoming this limitation by training the model on a variety of different noise types in a multi-condition way in order to make it generalize to unknown noise types. Copyright © 2018 Elsevier B.V. All rights reserved.
Molinaro, Alyssa M; Pearson, Bret J
2016-04-27
The planarian Schmidtea mediterranea is a master regenerator with a large adult stem cell compartment. The lack of transgenic labeling techniques in this animal has hindered the study of lineage progression and has made understanding the mechanisms of tissue regeneration a challenge. However, recent advances in single-cell transcriptomics and analysis methods allow for the discovery of novel cell lineages as differentiation progresses from stem cell to terminally differentiated cell. Here we apply pseudotime analysis and single-cell transcriptomics to identify adult stem cells belonging to specific cellular lineages and identify novel candidate genes for future in vivo lineage studies. We purify 168 single stem and progeny cells from the planarian head, which were subjected to single-cell RNA sequencing (scRNAseq). Pseudotime analysis with Waterfall and gene set enrichment analysis predicts a molecularly distinct neoblast sub-population with neural character (νNeoblasts) as well as a novel alternative lineage. Using the predicted νNeoblast markers, we demonstrate that a novel proliferative stem cell population exists adjacent to the brain. scRNAseq coupled with in silico lineage analysis offers a new approach for studying lineage progression in planarians. The lineages identified here are extracted from a highly heterogeneous dataset with minimal prior knowledge of planarian lineages, demonstrating that lineage purification by transgenic labeling is not a prerequisite for this approach. The identification of the νNeoblast lineage demonstrates the usefulness of the planarian system for computationally predicting cellular lineages in an adult context coupled with in vivo verification.
Zhu, Yanan; Ouyang, Qi; Mao, Youdong
2017-07-21
Single-particle cryo-electron microscopy (cryo-EM) has become a mainstream tool for the structural determination of biological macromolecular complexes. However, high-resolution cryo-EM reconstruction often requires hundreds of thousands of single-particle images. Particle extraction from experimental micrographs thus can be laborious and presents a major practical bottleneck in cryo-EM structural determination. Existing computational methods for particle picking often use low-resolution templates for particle matching, making them susceptible to reference-dependent bias. It is critical to develop a highly efficient template-free method for the automatic recognition of particle images from cryo-EM micrographs. We developed a deep learning-based algorithmic framework, DeepEM, for single-particle recognition from noisy cryo-EM micrographs, enabling automated particle picking, selection and verification in an integrated fashion. The kernel of DeepEM is built upon a convolutional neural network (CNN) composed of eight layers, which can be recursively trained to be highly "knowledgeable". Our approach exhibits an improved performance and accuracy when tested on the standard KLH dataset. Application of DeepEM to several challenging experimental cryo-EM datasets demonstrated its ability to avoid the selection of un-wanted particles and non-particles even when true particles contain fewer features. The DeepEM methodology, derived from a deep CNN, allows automated particle extraction from raw cryo-EM micrographs in the absence of a template. It demonstrates an improved performance, objectivity and accuracy. Application of this novel method is expected to free the labor involved in single-particle verification, significantly improving the efficiency of cryo-EM data processing.
Recurrent Spatial Transformer Networks
DEFF Research Database (Denmark)
Sønderby, Søren Kaae; Sønderby, Casper Kaae; Maaløe, Lars
2015-01-01
We integrate the recently proposed spatial transformer network (SPN) [Jaderberg et. al 2015] into a recurrent neural network (RNN) to form an RNN-SPN model. We use the RNN-SPN to classify digits in cluttered MNIST sequences. The proposed model achieves a single digit error of 1.5% compared to 2.......9% for a convolutional networks and 2.0% for convolutional networks with SPN layers. The SPN outputs a zoomed, rotated and skewed version of the input image. We investigate different down-sampling factors (ratio of pixel in input and output) for the SPN and show that the RNN-SPN model is able to down-sample the input...
Sequential neural models with stochastic layers
DEFF Research Database (Denmark)
Fraccaro, Marco; Sønderby, Søren Kaae; Paquet, Ulrich
2016-01-01
How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural...... generative model. The clear separation of deterministic and stochastic layers allows a structured variational inference network to track the factorization of the model's posterior distribution. By retaining both the nonlinear recursive structure of a recurrent neural network and averaging over...
Blechert, J; Testa, G; Georgii, C; Klimesch, W; Wilhelm, F H
2016-05-01
Present-day environments are replete with tempting foods and the current obesity pandemic speaks to humans' inability to adjust to this. Pavlovian processes may be fundamental to such hedonic overeating. However, a lack of naturalistic Pavlovian paradigms in humans makes translational research difficult and important parameters such as implicitness and acquisition speed are unknown. Here we present a novel naturalistic conditioning task: an image of a neutral object was conditioned to marzipan taste in a single trial procedure by asking the participant to eat the 'object' (made from marzipan). Relative to control objects, results demonstrate robust pre- to post-conditioning changes of both subjective ratings and early as well as late event related brain potentials, suggesting contributions of implicit (attentional) and explicit (motivational) processes. Naturalistic single-trial taste-appetitive conditioning is potent in humans and shapes attentional and motivational neural processes that might challenge self-regulation during exposure to tempting foods. Thus, appetitive conditioning processes might contribute to overweight and obesity. Copyright © 2016 Elsevier Inc. All rights reserved.
Liebel, L.; Körner, M.
2016-06-01
In optical remote sensing, spatial resolution of images is crucial for numerous applications. Space-borne systems are most likely to be affected by a lack of spatial resolution, due to their natural disadvantage of a large distance between the sensor and the sensed object. Thus, methods for single-image super resolution are desirable to exceed the limits of the sensor. Apart from assisting visual inspection of datasets, post-processing operations—e.g., segmentation or feature extraction—can benefit from detailed and distinguishable structures. In this paper, we show that recently introduced state-of-the-art approaches for single-image super resolution of conventional photographs, making use of deep learning techniques, such as convolutional neural networks (CNN), can successfully be applied to remote sensing data. With a huge amount of training data available, end-to-end learning is reasonably easy to apply and can achieve results unattainable using conventional handcrafted algorithms. We trained our CNN on a specifically designed, domain-specific dataset, in order to take into account the special characteristics of multispectral remote sensing data. This dataset consists of publicly available SENTINEL-2 images featuring 13 spectral bands, a ground resolution of up to 10m, and a high radiometric resolution and thus satisfying our requirements in terms of quality and quantity. In experiments, we obtained results superior compared to competing approaches trained on generic image sets, which failed to reasonably scale satellite images with a high radiometric resolution, as well as conventional interpolation methods.
Directory of Open Access Journals (Sweden)
Flávia de L. Osório
2015-03-01
Full Text Available Objectives: Ayahuasca (AYA, a natural psychedelic brew prepared from Amazonian plants and rich in dimethyltryptamine (DMT and harmine, causes effects of subjective well-being and may therefore have antidepressant actions. This study sought to evaluate the effects of a single dose of AYA in six volunteers with a current depressive episode. Methods: Open-label trial conducted in an inpatient psychiatric unit. Results: Statistically significant reductions of up to 82% in depressive scores were observed between baseline and 1, 7, and 21 days after AYA administration, as measured on the Hamilton Rating Scale for Depression (HAM-D, the Montgomery-Åsberg Depression Rating Scale (MADRS, and the Anxious-Depression subscale of the Brief Psychiatric Rating Scale (BPRS. AYA administration resulted in nonsignificant changes in Young Mania Rating Scale (YMRS scores and in the thinking disorder subscale of the BPRS, suggesting that AYA does not induce episodes of mania and/or hypomania in patients with mood disorders and that modifications in thought content, which could indicate psychedelic effects, are not essential for mood improvement. Conclusions: These results suggest that AYA has fast-acting anxiolytic and antidepressant effects in patients with a depressive disorder.
Osório, Flávia de L; Sanches, Rafael F; Macedo, Ligia R; Santos, Rafael G dos; Maia-de-Oliveira, João P; Wichert-Ana, Lauro; Araujo, Draulio B de; Riba, Jordi; Crippa, José A; Hallak, Jaime E
2015-01-01
Ayahuasca (AYA), a natural psychedelic brew prepared from Amazonian plants and rich in dimethyltryptamine (DMT) and harmine, causes effects of subjective well-being and may therefore have antidepressant actions. This study sought to evaluate the effects of a single dose of AYA in six volunteers with a current depressive episode. Open-label trial conducted in an inpatient psychiatric unit. Statistically significant reductions of up to 82% in depressive scores were observed between baseline and 1, 7, and 21 days after AYA administration, as measured on the Hamilton Rating Scale for Depression (HAM-D), the Montgomery-Åsberg Depression Rating Scale (MADRS), and the Anxious-Depression subscale of the Brief Psychiatric Rating Scale (BPRS). AYA administration resulted in nonsignificant changes in Young Mania Rating Scale (YMRS) scores and in the thinking disorder subscale of the BPRS, suggesting that AYA does not induce episodes of mania and/or hypomania in patients with mood disorders and that modifications in thought content, which could indicate psychedelic effects, are not essential for mood improvement. These results suggest that AYA has fast-acting anxiolytic and antidepressant effects in patients with a depressive disorder.
Directory of Open Access Journals (Sweden)
J. B. Habarulema
2009-05-01
Full Text Available This paper attempts to describe the search for the parameter(s to represent solar wind effects in Global Positioning System total electron content (GPS TEC modelling using the technique of neural networks (NNs. A study is carried out by including solar wind velocity (Vsw, proton number density (Np and the Bz component of the interplanetary magnetic field (IMF Bz obtained from the Advanced Composition Explorer (ACE satellite as separate inputs to the NN each along with day number of the year (DN, hour (HR, a 4-month running mean of the daily sunspot number (R4 and the running mean of the previous eight 3-hourly magnetic A index values (A8. Hourly GPS TEC values derived from a dual frequency receiver located at Sutherland (32.38° S, 20.81° E, South Africa for 8 years (2000–2007 have been used to train the Elman neural network (ENN and the result has been used to predict TEC variations for a GPS station located at Cape Town (33.95° S, 18.47° E. Quantitative results indicate that each of the parameters considered may have some degree of influence on GPS TEC at certain periods although a decrease in prediction accuracy is also observed for some parameters for different days and seasons. It is also evident that there is still a difficulty in predicting TEC values during disturbed conditions. The improvements and degradation in prediction accuracies are both close to the benchmark values which lends weight to the belief that diurnal, seasonal, solar and magnetic variabilities may be the major determinants of TEC variability.
Directory of Open Access Journals (Sweden)
J. B. Habarulema
2009-05-01
Full Text Available This paper attempts to describe the search for the parameter(s to represent solar wind effects in Global Positioning System total electron content (GPS TEC modelling using the technique of neural networks (NNs. A study is carried out by including solar wind velocity (V_{sw}, proton number density (N_{p} and the B_{z} component of the interplanetary magnetic field (IMF B_{z} obtained from the Advanced Composition Explorer (ACE satellite as separate inputs to the NN each along with day number of the year (DN, hour (HR, a 4-month running mean of the daily sunspot number (R4 and the running mean of the previous eight 3-hourly magnetic A index values (A8. Hourly GPS TEC values derived from a dual frequency receiver located at Sutherland (32.38° S, 20.81° E, South Africa for 8 years (2000–2007 have been used to train the Elman neural network (ENN and the result has been used to predict TEC variations for a GPS station located at Cape Town (33.95° S, 18.47° E. Quantitative results indicate that each of the parameters considered may have some degree of influence on GPS TEC at certain periods although a decrease in prediction accuracy is also observed for some parameters for different days and seasons. It is also evident that there is still a difficulty in predicting TEC values during disturbed conditions. The improvements and degradation in prediction accuracies are both close to the benchmark values which lends weight to the belief that diurnal, seasonal, solar and magnetic variabilities may be the major determinants of TEC variability.
Directory of Open Access Journals (Sweden)
Yoshihide Ueda, MD, PhD
2017-12-01
Conclusions. PSC frequently recurred and progressed to graft failure after liver transplantation for PSC. Maintaining an inactive status of inflammatory bowel disease might offer protection against PSC recurrence.
Matsubara, Takashi; Torikai, Hiroyuki
2016-04-01
Modeling and implementation approaches for the reproduction of input-output relationships in biological nervous tissues contribute to the development of engineering and clinical applications. However, because of high nonlinearity, the traditional modeling and implementation approaches encounter difficulties in terms of generalization ability (i.e., performance when reproducing an unknown data set) and computational resources (i.e., computation time and circuit elements). To overcome these difficulties, asynchronous cellular automaton-based neuron (ACAN) models, which are described as special kinds of cellular automata that can be implemented as small asynchronous sequential logic circuits have been proposed. This paper presents a novel type of such ACAN and a theoretical analysis of its excitability. This paper also presents a novel network of such neurons, which can mimic input-output relationships of biological and nonlinear ordinary differential equation model neural networks. Numerical analyses confirm that the presented network has a higher generalization ability than other major modeling and implementation approaches. In addition, Field-Programmable Gate Array-implementations confirm that the presented network requires lower computational resources.
Currie, Dustin W; Comstock, R Dawn; Fields, Sarah K; Cantu, Robert C
To compare initial and recurrent concussions regarding average number of days between concussions, acute concussion symptoms and symptom resolution time, and return to play time. High school athletes sustaining multiple concussions linked within sport seasons drawn from a large sports injury surveillance study. Retrospective analysis of longitudinal surveillance data. Number of days between concussions, number of symptoms endorsed, specific symptoms endorsed, symptom resolution time, return to play time. Median time between initial and recurrent concussions was 21 days (interquartile range = 10-43 days). Loss of consciousness, the only significant symptom difference, occurred more frequently in recurrent (6.8%) than initial (1.7%) concussions (P = .04). No significant difference was found in the number of symptoms (P = .84) or symptom resolution time (P = .74). Recurrent concussions kept athletes from play longer than initial concussions (P concussions were season ending. We found that athletes' initial and recurrent concussions had similar symptom presentations and resolution time. Despite these similarities, athletes were restricted from returning to play for longer periods following a recurrent concussion, indicating clinicians are managing recurrent concussions more conservatively. It is probable that concussion recognition and management are superior now compared with when previous studies were published, possibly improving recurrent concussion outcomes.
Detection of single and multilayer clouds in an artificial neural network approach
Sun-Mack, Sunny; Minnis, Patrick; Smith, William L.; Hong, Gang; Chen, Yan
2017-10-01
Determining whether a scene observed with a satellite imager is composed of a thin cirrus over a water cloud or thick cirrus contiguous with underlying layers of ice and water clouds is often difficult because of similarities in the observed radiance values. In this paper an artificial neural network (ANN) algorithm, employing several Aqua MODIS infrared channels and the retrieved total cloud visible optical depth, is trained to detect multilayer ice-over-water cloud systems as identified by matched April 2009 CloudSat and CALIPSO (CC) data. The CC lidar and radar profiles provide the vertical structure that serves as output truth for a multilayer ANN, or MLANN, algorithm. Applying the trained MLANN to independent July 2008 MODIS data resulted in a combined ML and single layer hit rate of 75% (72%) for nonpolar regions during the day (night). The results are comparable to or more accurate than currently available methods. Areas of improvement are identified and will be addressed in future versions of the MLANN.
Energy Technology Data Exchange (ETDEWEB)
Wei, Xinyu, E-mail: xyuwei@mail.xjtu.edu.cn; Wang, Pengfei, E-mail: pengfeixiaoli@yahoo.cn; Zhao, Fuyu, E-mail: fuyuzhao_xj@163.com
2016-08-01
Highlights: • We establish a disperse dynamic model for AP1000 reactor core. • A digital PID control based on QDRNN is used to design a decoupling control system. • The decoupling performance is verified and discussed. • The decoupling control system is simulated under the load following operation. - Abstract: The control system of the AP1000 reactor core uses the mechanical shim (MSHIM) strategy, which includes a power control subsystem and an axial power distribution control subsystem. To address the strong coupling between the two subsystems, an interlock between the two subsystems is used, which can only alleviate but not eliminate the coupling. Therefore, sometimes the axial offset (AO) cannot be controlled tightly, and the flexibility of load-following operation is limited. Thus, the decoupling of the original AP1000 reactor core control system is the focus of this paper. First, a two-node disperse dynamic model is established for the AP1000 reactor core to use PID control. Then, a digital PID control system based on a quasi-diagonal recurrent neural network (QDRNN) is designed to decouple the original system. Finally, the decoupling of the control system is verified by the step signal and load-following condition. The results show that the designed control system can decouple the original system as expected and the AO can be controlled much more tightly. Moreover, the flexibility of the load following is increased.
Fairbank, Michael; Li, Shuhui; Fu, Xingang; Alonso, Eduardo; Wunsch, Donald
2014-01-01
We present a recurrent neural-network (RNN) controller designed to solve the tracking problem for control systems. We demonstrate that a major difficulty in training any RNN is the problem of exploding gradients, and we propose a solution to this in the case of tracking problems, by introducing a stabilization matrix and by using carefully constrained context units. This solution allows us to achieve consistently lower training errors, and hence allows us to more easily introduce adaptive capabilities. The resulting RNN is one that has been trained off-line to be rapidly adaptive to changing plant conditions and changing tracking targets. The case study we use is a renewable-energy generator application; that of producing an efficient controller for a three-phase grid-connected converter. The controller we produce can cope with the random variation of system parameters and fluctuating grid voltages. It produces tracking control with almost instantaneous response to changing reference states, and virtually zero oscillation. This compares very favorably to the classical proportional integrator (PI) controllers, which we show produce a much slower response and settling time. In addition, the RNN we propose exhibits better learning stability and convergence properties, and can exhibit faster adaptation, than has been achieved with adaptive critic designs. Copyright © 2013 Elsevier Ltd. All rights reserved.
Wielgosz, Maciej; Skoczeń, Andrzej
This paper focuses on an examination of an applicability of Recurrent Neural Network models for detecting anomalous behavior of the CERN superconducting magnets. In order to conduct the experiments, the authors designed and implemented an adaptive signal quantization algorithm and a custom GRU-based detector and developed a method for the detector parameters selection. Three different datasets were used for testing the detector. Two artificially generated datasets were used to assess the raw performance of the system whereas the 231 MB dataset composed of the signals acquired from HiLumi magnets was intended for real-life experiments and model training. Several different setups of the developed anomaly detection system were evaluated and compared with state-of-the-art OC-SVM reference model operating on the same data. The OC-SVM model was equipped with a rich set of feature extractors accounting for a range of the input signal properties. It was determined in the course of the experiments that the detector, a...
Directory of Open Access Journals (Sweden)
L. Liebel
2016-06-01
Full Text Available In optical remote sensing, spatial resolution of images is crucial for numerous applications. Space-borne systems are most likely to be affected by a lack of spatial resolution, due to their natural disadvantage of a large distance between the sensor and the sensed object. Thus, methods for single-image super resolution are desirable to exceed the limits of the sensor. Apart from assisting visual inspection of datasets, post-processing operations—e.g., segmentation or feature extraction—can benefit from detailed and distinguishable structures. In this paper, we show that recently introduced state-of-the-art approaches for single-image super resolution of conventional photographs, making use of deep learning techniques, such as convolutional neural networks (CNN, can successfully be applied to remote sensing data. With a huge amount of training data available, end-to-end learning is reasonably easy to apply and can achieve results unattainable using conventional handcrafted algorithms. We trained our CNN on a specifically designed, domain-specific dataset, in order to take into account the special characteristics of multispectral remote sensing data. This dataset consists of publicly available SENTINEL-2 images featuring 13 spectral bands, a ground resolution of up to 10m, and a high radiometric resolution and thus satisfying our requirements in terms of quality and quantity. In experiments, we obtained results superior compared to competing approaches trained on generic image sets, which failed to reasonably scale satellite images with a high radiometric resolution, as well as conventional interpolation methods.
Lu, I-Cheng; Wu, Che-Wei; Chang, Pi-Ying; Chen, Hsiu-Ya; Tseng, Kuang-Yi; Randolph, Gregory W; Cheng, Kuang-I; Chiang, Feng-Yu
2016-04-01
The use of neuromuscular blocking agent may effect intraoperative neuromonitoring (IONM) during thyroid surgery. An enhanced neuromuscular-blockade (NMB) recovery protocol was investigated in a porcine model and subsequently clinically applied during human thyroid neural monitoring surgery. Prospective animal and retrospective clinical study. In the animal experiment, 12 piglets were injected with rocuronium 0.6 mg/kg and randomly allocated to receive normal saline, sugammadex 2 mg/kg, or sugammadex 4 mg/kg to compare the recovery of laryngeal electromyography (EMG). In a subsequent clinical application study, 50 patients who underwent thyroidectomy with IONM followed an enhanced NMB recovery protocol-rocuronium 0.6 mg/kg at anesthesia induction and sugammadex 2 mg/kg at the operation start. The train-of-four (TOF) ratio was used for continuous quantitative monitoring of neuromuscular transmission. In our porcine model, it took 49 ± 15, 13.2 ± 5.6, and 4.2 ± 1.5 minutes for the 80% recovery of laryngeal EMG after injection of saline, sugammadex 2 mg/kg, and sugammadex 4 mg/kg, respectively. In subsequent clinical human application, the TOF ratio recovered from 0 to >0.9 within 5 minutes after administration of sugammadex 2 mg/kg at the operation start. All patients had positive and high EMG amplitude at the early stage of the operation, and intubation was without difficulty in 96% of patients. Both porcine modeling and clinical human application demonstrated that sugammadex 2 mg/kg allows effective and rapid restoration of neuromuscular function suppressed by rocuronium. Implementation of this enhanced NMB recovery protocol assures optimal conditions for tracheal intubation as well as IONM in thyroid surgery. NA. © 2016 The American Laryngological, Rhinological and Otological Society, Inc.
Dynamic neural controllers for induction motor.
Brdyś, M A; Kulawski, G J
1999-01-01
The paper reports application of recently developed adaptive control techniques based on neural networks to the induction motor control. This case study represents one of the more difficult control problems due to the complex, nonlinear, and time-varying dynamics of the motor and unavailability of full-state measurements. A partial solution is first presented based on a single input-single output (SISO) algorithm employing static multilayer perceptron (MLP) networks. A novel technique is subsequently described which is based on a recurrent neural network employed as a dynamical model of the plant. Recent stability results for this algorithm are reported. The technique is applied to multiinput-multioutput (MIMO) control of the motor. A simulation study of both methods is presented. It is argued that appropriately structured recurrent neural networks can provide conveniently parameterized dynamic models for many nonlinear systems for use in adaptive control.
Kalebic, Nereo; Taverna, Elena; Tavano, Stefania; Wong, Fong Kuan; Suchold, Dana; Winkler, Sylke; Huttner, Wieland B; Sarov, Mihail
2016-03-01
We have applied the CRISPR/Cas9 system in vivo to disrupt gene expression in neural stem cells in the developing mammalian brain. Two days after in utero electroporation of a single plasmid encoding Cas9 and an appropriate guide RNA (gRNA) into the embryonic neocortex of Tis21::GFP knock-in mice, expression of GFP, which occurs specifically in neural stem cells committed to neurogenesis, was found to be nearly completely (≈ 90%) abolished in the progeny of the targeted cells. Importantly, upon in utero electroporation directly of recombinant Cas9/gRNA complex, near-maximal efficiency of disruption of GFP expression was achieved already after 24 h. Furthermore, by using microinjection of the Cas9 protein/gRNA complex into neural stem cells in organotypic slice culture, we obtained disruption of GFP expression within a single cell cycle. Finally, we used either Cas9 plasmid in utero electroporation or Cas9 protein complex microinjection to disrupt the expression of Eomes/Tbr2, a gene fundamental for neocortical neurogenesis. This resulted in a reduction in basal progenitors and an increase in neuronal differentiation. Thus, the present in vivo application of the CRISPR/Cas9 system in neural stem cells provides a rapid, efficient and enduring disruption of expression of specific genes to dissect their role in mammalian brain development. © 2016 The Authors. Published under the terms of the CC BY NC ND 4.0 license.
Gałecka, Elżbieta; Szemraj, Janusz; Bieńkiewicz, Małgorzata; Majsterek, Ireneusz; Przybyłowska-Sygut, Karolina; Gałecki, Piotr; Lewiński, Andrzej
2013-02-01
Depressive disorder is a disease characterized by disturbances in the hypothalamo-pituitary-adrenal axis. Abnormalities include the increased level of glucocorticoids (GC) and changes in sensitivity to these hormones. The changes are related to glucocorticoid receptors gene (NR3C1) variants. The NR3C1 gene is suggested to be a candidate gene affecting depressive disorder risk and management. The aim of this study was to investigate polymorphisms within the NR3C1 gene and their role in the susceptibility to recurrent depressive disorder (rDD). 181 depressive patients and 149 healthy ethnically matched controls were included in the study. Single nucleotide polymorphisms were assessed using polymerase chain reaction/restriction fragment length polymorphism method. Statistical significance between rDD patients and controls was observed for the allele and genotype frequencies at three loci: BclI, N363S, and ER22/23EK. The presence of C allele, CC, and GC genotype of BclI polymorphism, G allele and GA genotype for N363S and ER22/23EK variants respectively were associated with increased rDD risk. Two haplotypes indicated higher susceptibility for rDD, while haplotype GAG played a protective role with OR(dis) 0.29 [95 % confidence interval (CI) = 0.13-0.64]. Data generated from this study support the earlier results that genetic variants of the NR3C1 gene are associated with rDD and suggest further consideration on the possible involvement of these variants in etiology of the disease.
Czarny, Piotr; Kwiatkowski, Dominik; Toma, Monika; Gałecki, Piotr; Orzechowska, Agata; Bobińska, Kinga; Bielecka-Kowalska, Anna; Szemraj, Janusz; Berk, Michael; Anderson, George; Śliwiński, Tomasz
2016-01-01
Background Depressive disorder, including recurrent type (rDD), is accompanied by increased oxidative stress and activation of inflammatory pathways, which may induce DNA damage. This thesis is supported by the presence of increased levels of DNA damage in depressed patients. Such DNA damage is repaired by the base excision repair (BER) pathway. BER efficiency may be influenced by polymorphisms in BER-related genes. Therefore, we genotyped nine single-nucleotide polymorphisms (SNPs) in six genes encoding BER proteins. Material/Methods Using TaqMan, we selected and genotyped the following SNPs: c.-441G>A (rs174538) of FEN1, c.2285T>C (rs1136410) of PARP1, c.580C>T (rs1799782) and c.1196A>G (rs25487) of XRCC1, c.*83A>C (rs4796030) and c.*50C>T (rs1052536) of LIG3, c.-7C>T (rs20579) of LIG1, and c.-468T>G (rs1760944) and c.444T>G (rs1130409) of APEX1 in 599 samples (288 rDD patients and 311 controls). Results We found a strong correlation between rDD and both SNPs of LIG3, their haplotypes, as well as a weaker association with the c.-468T>G of APEXI which diminished after Nyholt correction. Polymorphisms of LIG3 were also associated with early onset versus late onset depression, whereas the c.-468T>G polymorphism showed the opposite association. Conclusions The SNPs of genes involved in the repair of oxidative DNA damage may modulate rDD risk. Since this is an exploratory study, the results should to be treated with caution and further work needs to be done to elucidate the exact involvement of DNA damage and repair mechanisms in the development of this disease. PMID:27866211
Yoshihide Ueda, MD, PhD; Toshimi Kaido, MD, PhD; Hideaki Okajima, MD, PhD; Koichiro Hata, MD, PhD; Takayuki Anazawa, MD, PhD; Atsushi Yoshizawa, MD, PhD; Shintaro Yagi, MD, PhD; Kojiro Taura, MD, PhD; Toshihiko Masui, MD, PhD; Noriyo Yamashiki, MD, PhD; Hironori Haga, MD, PhD; Miki Nagao, MD, PhD; Hiroyuki Marusawa, MD, PhD; Hiroshi Seno, MD, PhD; Shinji Uemoto, MD, PhD
2017-01-01
Background. Primary sclerosing cholangitis (PSC) is a progressive cholestatic liver disease, with liver transplantation being the sole life-saving treatment for end-stage PSC-related liver disease. However, recurrence of PSC after liver transplantation is a common complication, with the risk factors for recurrence being controversial. Methods. We conducted a retrospective chart review of 45 patients who had undergone liver transplantation for PSC at our institute. The risk factors for PSC ...
Directory of Open Access Journals (Sweden)
Ying Song
Full Text Available The retreatment of recurrent intracranial vertebral artery dissecting aneurysms (VADAs after stent assisted coiling (SAC has not yet been studied. The purpose of this study was to evaluate the strategies and outcomes for retreatment of recurrent VADAs after SAC.Between September 2009 and November 2013, six consecutive patients presenting with recurrent intracranial VADAs after SAC were enrolled in this study. They were all male with age ranging from 29 to 54 years (mean age, 46.2 years. The procedures of treatments and angiographic and clinical follow-up were reviewed retrospectively. Retreatment modalities were selected individually according to the characteristics of recurrence. The outcomes of retreatment were evaluated by angiographic and clinical follow-up.Six patients with recurrent intracranial VADAs after SAC were retreated, with second SAC in three patients, coil embolization, double overlapping stents placement and endovascular occlusion with aneurysm trapping in one patient, respectively. Immediate angiographic outcomes of retreatment were: complete occlusion in three patients, nearly complete occlusion in two patients, and contrast medium retention in dissecting aneurysm in one patient. All cases were technically successful. No complications related to endovascular procedures occurred. Angiographic follow-up was available in all five patients treated with second SAC or double overlapping stents, which was complete occlusion in four patients, obliteration of parent artery in one patient, showing no recurrence at 4-11 months (mean: 8.6 months. Clinical follow-up was performed in all six patients at 11-51 months after initial endovascular treatment and at 9-43 months after retreatment. The mRS of last clinical follow-up was excellent in five patients and mild disability in only one patient.Endovascular retreatment is feasible and effective for recurrent intracranial VADAs after SAC. Individualized strategies of retreatment should be
Liu, Xueyou; Xu, Jianguo; Mao, Ke; Wang, Mengmeng; Ren, Peng; Lei, Ding; Fang, Yuan; Chen, Wenjing; Mao, Boyong; Zhou, Dong; Li, Jinmei; Hong, Zhen; Yan, Bo; An, Dongmei; Liu, Ling; Chen, Jiani; Luo, Rong; Zhou, Hui; Yu, Tao; Zhang, Heng
2018-04-01
To evaluate the potential risk factors associated with seizure recurrence in different periods after epilepsy surgery. A total of 303 patients with refractory epilepsy after epilepsy surgery were included. The Kaplan-Meier method with log-rank test and univariate and multivariate Cox proportional hazards model were performed to calculate the comparison of survival curves between groups and identify the risk factors associated with seizure recurrence in different periods after surgery. The significant predictors of seizure recurrence were determined, including duration of epilepsy (P = 0.018), seizure types (P = 0.009), magnetic resonance imaging findings (P = 0.007), intracranial electroencephalographic recordings (P = 0.002), sides of epileptogenic zone (P = 0.025), and types of surgery (P = 0.002). Moreover, the significant predictors of seizure recurrence within 12 months after surgery were also included, such as gender (P = 0.007), duration of epilepsy (P = 0.013), intracranial electroencephalographic recordings (P = 0.003), and types of surgery (P 36 months after surgery. We reconfirmed the well-known risk factors associated with seizure recurrence and also identified the controversial variables. In addition, we found that the risk factors associated with seizure recurrence were different in different periods after epilepsy surgery. Copyright © 2018 Elsevier Inc. All rights reserved.
Feedforward Approximations to Dynamic Recurrent Network Architectures.
Muir, Dylan R
2018-02-01
Recurrent neural network architectures can have useful computational properties, with complex temporal dynamics and input-sensitive attractor states. However, evaluation of recurrent dynamic architectures requires solving systems of differential equations, and the number of evaluations required to determine their response to a given input can vary with the input or can be indeterminate altogether in the case of oscillations or instability. In feedforward networks, by contrast, only a single pass through the network is needed to determine the response to a given input. Modern machine learning systems are designed to operate efficiently on feedforward architectures. We hypothesized that two-layer feedforward architectures with simple, deterministic dynamics could approximate the responses of single-layer recurrent network architectures. By identifying the fixed-point responses of a given recurrent network, we trained two-layer networks to directly approximate the fixed-point response to a given input. These feedforward networks then embodied useful computations, including competitive interactions, information transformations, and noise rejection. Our approach was able to find useful approximations to recurrent networks, which can then be evaluated in linear and deterministic time complexity.
Energy Technology Data Exchange (ETDEWEB)
Carnevale, Francisco Cesar, E-mail: francisco.carnevale@criep.com.br; Moreira, Airton Mota [University of Sao Paulo, Department of Radiology (Brazil); Harward, Sardis Honoria [The Dartmouth Institute for Health Policy and Clinical Practice (United States); Bhatia, Shivank [University of Miami Medical Center, Department of Interventional Radiology (United States); Assis, Andre Moreira de [University of Sao Paulo, Department of Radiology (Brazil); Srougi, Miguel [University of Sao Paulo, Department of Urology (Brazil); Cerri, Giovanni Guido [University of Sao Paulo, Department of Radiology (Brazil); Antunes, Alberto Azoubel [University of Sao Paulo, Department of Urology (Brazil)
2017-03-15
PurposeTo compare recurrence of lower urinary tract symptoms (LUTS) recurrence at 12 months following original prostate artery embolization (oPAE) or “proximal embolization first, then embolize distal” (PErFecTED) PAE for benign prostatic hyperplasia (BPH).Materials and Methods105 consecutive patients older than 45 years, with prostate size greater than 30 cm{sup 3}, International Prostate Symptom Score (IPSS) ≥ 8, quality of life (QoL) index ≥ 3, and refractory status or intolerance of medical management were prospectively enrolled between June 2008 and August 2013. The study was IRB-approved, and all patients provided informed consent. Patients underwent oPAE or PErFecTED PAE and were followed for at least 12 months. Technical success was defined as bilateral embolization and clinical success (non-recurrence) was defined as removal of the Foley catheter in patients with urinary retention, IPSS < 8 and QoL index < 3 at 12 months of follow-up. Nonparametric statistics were used to compare the study groups due to the size of the study population and distributions of clinical data.Results97 patients had 12-month data and were categorized as oPAE without recurrence (n = 46), oPAE with recurrence (n = 13), PErFecTED without recurrence (n = 36), or PErFecTED with recurrence (n = 2). Recurrence was significantly more common in oPAE patients (χ{sup 2}, p = 0.026). Unilateral embolization was significantly associated with recurrence among patients who underwent oPAE (χ{sup 2}, p = 0.032).ConclusionsBoth oPAE and PErFecTED PAE are safe and effective methods for treatment of LUTS, but PErFecTED PAE is associated with a significantly lower rate of symptom recurrence.
Sacks, Stephen L.; Shafran, Stephen D.; Diaz-Mitoma, Francisco; Trottier, Sylvie; Sibbald, R. Gary; Hughes, April; Safrin, Sharon; Rudy, Jeff; McGuire, Brian; Jaffe, Howard S.
1998-01-01
A randomized, double-blind, clinic-initiated, sequential dose-escalation pilot study was performed to compare the safety and efficacy of single applications of 1, 3, and 5% cidofovir gel with placebo in the treatment of early, lesional, recurrent genital herpes at five Canadian outpatient sites. Ninety-six patients began treatment within 12 h of lesion appearance and were evaluated twice daily until healing of the lesion occurred. Cidofovir gel at all strengths significantly decreased the med...
Cloud Height Estimation with a Single Digital Camera and Artificial Neural Networks
Carretas, Filipe; Janeiro, Fernando M.
2014-05-01
Clouds influence the local weather, the global climate and are an important parameter in the weather prediction models. Clouds are also an essential component of airplane safety when visual flight rules (VFR) are enforced, such as in most small aerodromes where it is not economically viable to install instruments for assisted flying. Therefore it is important to develop low cost and robust systems that can be easily deployed in the field, enabling large scale acquisition of cloud parameters. Recently, the authors developed a low-cost system for the measurement of cloud base height using stereo-vision and digital photography. However, due to the stereo nature of the system, some challenges were presented. In particular, the relative camera orientation requires calibration and the two cameras need to be synchronized so that the photos from both cameras are acquired simultaneously. In this work we present a new system that estimates the cloud height between 1000 and 5000 meters. This prototype is composed by one digital camera controlled by a Raspberry Pi and is installed at Centro de Geofísica de Évora (CGE) in Évora, Portugal. The camera is periodically triggered to acquire images of the overhead sky and the photos are downloaded to the Raspberry Pi which forwards them to a central computer that processes the images and estimates the cloud height in real time. To estimate the cloud height using just one image requires a computer model that is able to learn from previous experiences and execute pattern recognition. The model proposed in this work is an Artificial Neural Network (ANN) that was previously trained with cloud features at different heights. The chosen Artificial Neural Network is a three-layer network, with six parameters in the input layer, 12 neurons in the hidden intermediate layer, and an output layer with only one output. The six input parameters are the average intensity values and the intensity standard deviation of each RGB channel. The output
Ali-El-Dein, Bedeir; Sooriakumaran, Prasanna; Trinh, Quoc-Dien; Barakat, Tamer S; Nabeeh, Adel; Ibrahiem, El-Housseiny I
2013-06-01
To construct predictive models based on the objectively calculated risks of progression and recurrence of non-muscle-invasive bladder cancer (NMIBC) in a large cohort of patients from a single centre. Between October 1984 and March 2009 a cohort of 1019 patients (877 males; 142 females; median age 44 years) with histologically confirmed NMIBC was included in this study. Among these patients, 74% received bacillus Calmette-Guérin (BCG)-based therapy. Complete transurethral resection of bladder tumour of all visible tumours was carried out in all patients, and the stage and grade were determined. Univariate analysis and multivariate Cox regression were used to identify predictors of recurrence and progression. The studied predictors included age, sex, stage, grade, associated carcinoma in situ, tumour size, multiplicity, macroscopic appearance of the tumour, history of recurrence and type of adjuvant intravesical therapy. Multivariate logistic regression models were used to develop the 12- and 60-month recurrence and progression predictive models. The predictive accuracy of the models was assessed for discrimination as well as calibration. The median (range) follow-up was 44 (6-254) months. On multivariate analysis, stage, multiplicity, history of recurrence and adjuvant intravesical therapy were significantly associated with recurrence, whereas for progression only tumour grade and size were significant independent predictors. The constructed nomograms had a 64.9% and 69.4% chance of correctly distinguishing between two patients, one destined to have a recurrence and one not at 12 and 60 months, respectively. The constructed nomograms had a 70.2% and 73.5% chance of correctly distinguishing between two patients, one destined to progress and one not at 12 and 60 months, respectively. All predictive models were well calibrated. Based on multivariate analysis of the studied prognostic factors nomograms for predicting recurrence and progression in NMIBC were
Directory of Open Access Journals (Sweden)
Tomas Kazda
2016-01-01
Full Text Available The accurate identification of glioblastoma progression remains an unmet clinical need. The aim of this prospective single-institutional study is to determine and validate thresholds for the main metabolite concentrations obtained by MR spectroscopy (MRS and the values of the apparent diffusion coefficient (ADC to enable distinguishing tumor recurrence from pseudoprogression. Thirty-nine patients after the standard treatment of a glioblastoma underwent advanced imaging by MRS and ADC at the time of suspected recurrence — median time to progression was 6.7 months. The highest significant sensitivity and specificity to call the glioblastoma recurrence was observed for the total choline (tCho to total N-acetylaspartate (tNAA concentration ratio with the threshold ≥1.3 (sensitivity 100.0% and specificity 94.7%. The ADCmean value higher than 1313 × 10−6 mm2/s was associated with the pseudoprogression (sensitivity 98.3%, specificity 100.0%. The combination of MRS focused on the tCho/tNAA concentration ratio and the ADCmean value represents imaging methods applicable to early non-invasive differentiation between a glioblastoma recurrence and a pseudoprogression. However, the institutional definition and validation of thresholds for differential diagnostics is needed for the elimination of setup errors before implementation of these multimodal imaging techniques into clinical practice, as well as into clinical trials.
Neural Correlates of Single- and Dual-Task Walking in the Real World.
Pizzamiglio, Sara; Naeem, Usman; Abdalla, Hassan; Turner, Duncan L
2017-01-01
Recent developments in mobile brain-body imaging (MoBI) technologies have enabled studies of human locomotion where subjects are able to move freely in more ecologically valid scenarios. In this study, MoBI was employed to describe the behavioral and neurophysiological aspects of three different commonly occurring walking conditions in healthy adults. The experimental conditions were self-paced walking, walking while conversing with a friend and lastly walking while texting with a smartphone. We hypothesized that gait performance would decrease with increased cognitive demands and that condition-specific neural activation would involve condition-specific brain areas. Gait kinematics and high density electroencephalography (EEG) were recorded whilst walking around a university campus. Conditions with dual tasks were accompanied by decreased gait performance. Walking while conversing was associated with an increase of theta (θ) and beta (β) neural power in electrodes located over left-frontal and right parietal regions, whereas walking while texting was associated with a decrease of β neural power in a cluster of electrodes over the frontal-premotor and sensorimotor cortices when compared to walking whilst conversing. In conclusion, the behavioral "signatures" of common real-life activities performed outside the laboratory environment were accompanied by differing frequency-specific neural "biomarkers". The current findings encourage the study of the neural biomarkers of disrupted gait control in neurologically impaired patients.
Neural Correlates of Single- and Dual-Task Walking in the Real World
Directory of Open Access Journals (Sweden)
Sara Pizzamiglio
2017-09-01
Full Text Available Recent developments in mobile brain-body imaging (MoBI technologies have enabled studies of human locomotion where subjects are able to move freely in more ecologically valid scenarios. In this study, MoBI was employed to describe the behavioral and neurophysiological aspects of three different commonly occurring walking conditions in healthy adults. The experimental conditions were self-paced walking, walking while conversing with a friend and lastly walking while texting with a smartphone. We hypothesized that gait performance would decrease with increased cognitive demands and that condition-specific neural activation would involve condition-specific brain areas. Gait kinematics and high density electroencephalography (EEG were recorded whilst walking around a university campus. Conditions with dual tasks were accompanied by decreased gait performance. Walking while conversing was associated with an increase of theta (θ and beta (β neural power in electrodes located over left-frontal and right parietal regions, whereas walking while texting was associated with a decrease of β neural power in a cluster of electrodes over the frontal-premotor and sensorimotor cortices when compared to walking whilst conversing. In conclusion, the behavioral “signatures” of common real-life activities performed outside the laboratory environment were accompanied by differing frequency-specific neural “biomarkers”. The current findings encourage the study of the neural biomarkers of disrupted gait control in neurologically impaired patients.
Loss-efficiency model of single and variable-speed compressors using neural networks
Energy Technology Data Exchange (ETDEWEB)
Yang, Liang [Institute of Refrigeration and Cryogenics, Shanghai Jiaotong University, Shanghai 200240 (China); China R and D Center, Carrier Corporation, No.3239 Shen Jiang Road, Shanghai 201206 (China); Zhao, Ling-Xiao; Gu, Bo [Institute of Refrigeration and Cryogenics, Shanghai Jiaotong University, Shanghai 200240 (China); Zhang, Chun-Lu [China R and D Center, Carrier Corporation, No.3239 Shen Jiang Road, Shanghai 201206 (China)
2009-09-15
Compressor is the critical component to the performance of a vapor-compression refrigeration system. The loss-efficiency model including the volumetric efficiency and the isentropic efficiency is widely used for representing the compressor performance. A neural network loss-efficiency model is developed to simulate the performance of positive displacement compressors like the reciprocating, screw and scroll compressors. With one more input, frequency, it can be easily extended to the variable speed compressors. The three-layer polynomial perceptron network is developed because the polynomial transfer function is found very effective in training and free of over-learning. The selection of input parameters of neural networks is also found critical to the network prediction accuracy. The proposed neural networks give less than 0.4% standard deviations and {+-}1.3% maximum deviations against the manufacturer data. (author)
Deep Gate Recurrent Neural Network
2016-11-22
10.1177/0278364913495721. URL http://ijr.sagepub.com/ cgi /doi/10.1177/ 0278364913495721. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton...1008819414322. URL http://www.springerlink.com/index/K662611455651Q42. pdf . Hermann Mayer, Faustino Gomez, Daan Wierstra, Istvan Nagy, Alois Knoll, and Jürgen
Precision requirements for single-layer feed-forward neural networks
Annema, Anne J.; Hoen, K.; Hoen, Klaas; Wallinga, Hans
1994-01-01
This paper presents a mathematical analysis of the effect of limited precision analog hardware for weight adaptation to be used in on-chip learning feedforward neural networks. Easy-to-read equations and simple worst-case estimations for the maximum tolerable imprecision are presented. As an
Chatterjee, Avishek
A search for supersymmetry in proton-proton collisions at $\\sqrt{s} = 7$ TeV is presented, focusing on events with a single isolated lepton, energetic jets, and large missing transverse momentum. The analyzed data corresponds to a total integrated luminosity of 4.98 fb$^{−1}$ recorded by the CMS experiment. The search uses an artificial neural network to suppress Standard Model backgrounds, and estimates residual backgrounds using a fully data-driven method. The analysis is performed in both the muon and electron channels, and the combined result is interpreted in terms of limits on the CMSSM parameter space, as well as a simplified model.
Le, Hieu The
This thesis develops a new method to detect delaminations in composite laminates using a combination of finite element method, artificial neural networks, and genetic algorithms. Next, this newly developed method is applied to successfully solve delamination detection problems. Delaminations in a composite laminate with various sizes and locations are considered in the present studies. The improved layerwise shear deformation theory is implemented into the finite element method and used to calculate responses of laminates with single and multiple delaminations. Mappings between the natural frequencies and delamination characteristics are first determined from the developed models. These data are then used to train artificial neural networks of multiplayer perceptron using back-propagation. These trained artificial neural networks are in turn used as an approximate tool to calculate the responses of the delaminated laminates and to feed the data to the delamination detection process. Two different approaches for handling the neural network models are applied in the work and are presented for comparison. The delamination detection problem is formulated as an optimization problem with mixed type design variables. A genetic algorithm, which is a guided probabilistic search technique based on the simulation of Darwin's principle of evolution and natural selection, is developed to solve this optimization problem. Single through-the-width delamination, single internal delamination, and multiple through-the-width delaminations are separately considered for detection study. At last, the application is extended to the most challenging problem, which is the detection of general delamination. Various factors affecting the detection process such as the finite element convergence factor and the laminate geometry factor are also examined. Case studies are made and the findings are summarized in detail in each chapter of the dissertation. It is found that the newly developed
Neural networks and statistical learning
Du, Ke-Lin
2014-01-01
Providing a broad but in-depth introduction to neural network and machine learning in a statistical framework, this book provides a single, comprehensive resource for study and further research. All the major popular neural network models and statistical learning approaches are covered with examples and exercises in every chapter to develop a practical working understanding of the content. Each of the twenty-five chapters includes state-of-the-art descriptions and important research results on the respective topics. The broad coverage includes the multilayer perceptron, the Hopfield network, associative memory models, clustering models and algorithms, the radial basis function network, recurrent neural networks, principal component analysis, nonnegative matrix factorization, independent component analysis, discriminant analysis, support vector machines, kernel methods, reinforcement learning, probabilistic and Bayesian networks, data fusion and ensemble learning, fuzzy sets and logic, neurofuzzy models, hardw...
Directory of Open Access Journals (Sweden)
Ben W. Dulken
2017-01-01
Full Text Available Neural stem cells (NSCs in the adult mammalian brain serve as a reservoir for the generation of new neurons, oligodendrocytes, and astrocytes. Here, we use single-cell RNA sequencing to characterize adult NSC populations and examine the molecular identities and heterogeneity of in vivo NSC populations. We find that cells in the NSC lineage exist on a continuum through the processes of activation and differentiation. Interestingly, rare intermediate states with distinct molecular profiles can be identified and experimentally validated, and our analysis identifies putative surface markers and key intracellular regulators for these subpopulations of NSCs. Finally, using the power of single-cell profiling, we conduct a meta-analysis to compare in vivo NSCs and in vitro cultures, distinct fluorescence-activated cell sorting strategies, and different neurogenic niches. These data provide a resource for the field and contribute to an integrative understanding of the adult NSC lineage.
Directory of Open Access Journals (Sweden)
Ebrahim Shahbazi
2016-04-01
Full Text Available Direct conversion of somatic cells into neural stem cells (NSCs by defined factors holds great promise for mechanistic studies, drug screening, and potential cell therapies for different neurodegenerative diseases. Here, we report that a single zinc-finger transcription factor, Zfp521, is sufficient for direct conversion of human fibroblasts into long-term self-renewable and multipotent NSCs. In vitro, Zfp521-induced NSCs maintained their characteristics in the absence of exogenous factor expression and exhibited morphological, molecular, developmental, and functional properties that were similar to control NSCs. In addition, the single-seeded induced NSCs were able to form NSC colonies with efficiency comparable with control NSCs and expressed NSC markers. The converted cells were capable of surviving, migrating, and attaining neural phenotypes after transplantation into neonatal mouse and adult rat brains, without forming tumors. Moreover, the Zfp521-induced NSCs predominantly expressed rostral genes. Our results suggest a facilitated approach for establishing human NSCs through Zfp521-driven conversion of fibroblasts.
Hoyos, Sergio; Escobar, Jorge; Cardona, Doris; Guzmán, Carlos; Mena, Álvaro; Osorio, Germán; Pérez, Camilo; Restrepo, Juan C; Correa, Gonzalo
2015-01-01
Hepatocellular carcinoma is the most common primary tumor of the liver and is diagnosed in more than a half million people worldwide each year. This study aims to assess factors associated with the recurrence and survival of patients with hepatocellular carcinoma and liver transplantation in a cohort of patients from Medellín, Colombia. This was a descriptive retrospective study of a consecutive series of liver transplant patients from the Pablo Tobon Uribe Hospital of Medellín from January 2004 to May 2013. Demographic, clinical, imaging, and pathology variables were analyzed. Three hundred thirty liver transplants were performed during the study period, 54 cases (16.4%) had one or more hepatocellular carcinomas in the explant, and 79.6% of these patients were men. Cirrhotic patients had different etiologies, but most of them were due to alcohol abuse (22.2%), followed by hepatitis B virus infection (20.4 %), and hepatitis C virus infection (18.5%). In the pathology specimen, 51.9% had only one focus of hepatocellular carcinoma, 22.2% had two foci and 12.9% had three tumors. Recurrence of hepatocellular carcinoma occurred in 7.4% patients with an average time of 81 months. During follow-up, 25.9% of the patients died in an average time of 67.9 months (CI95 59.1-80.1 months). Recurrence and survival of patients with liver transplantation for hepatocellular carcinoma in this study had a similar behavior as that reported in the world literature. The factors associated with these outcomes were vascular invasion, poor tumor differentiation and satellitosis.
Survivin Improves Reprogramming Efficiency of Human Neural Progenitors by Single Molecule OCT4
Directory of Open Access Journals (Sweden)
Shixin Zhou
2016-01-01
Full Text Available Induced pluripotent stem (iPS cells have been generated from human somatic cells by ectopic expression of four Yamanaka factors. Here, we report that Survivin, an apoptosis inhibitor, can enhance iPS cells generation from human neural progenitor cells (NPCs together with one factor OCT4 (1F-OCT4-Survivin. Compared with 1F-OCT4, Survivin accelerates the process of reprogramming from human NPCs. The neurocyte-originated induced pluripotent stem (NiPS cells generated from 1F-OCT4-Survivin resemble human embryonic stem (hES cells in morphology, surface markers, global gene expression profiling, and epigenetic status. Survivin keeps high expression in both iPS and ES cells. During the process of NiPS cell to neural cell differentiation, the expression of Survivin is rapidly decreased in protein level. The mechanism of Survivin promotion of reprogramming efficiency from NPCs may be associated with stabilization of β-catenin in WNT signaling pathway. This hypothesis is supported by experiments of RT-PCR, chromatin immune-precipitation, and Western blot in human ES cells. Our results showed overexpression of Survivin could improve the efficiency of reprogramming from NPCs to iPS cells by one factor OCT4 through stabilization of the key molecule, β-catenin.
Directory of Open Access Journals (Sweden)
Francesco Chiancone
Full Text Available ABSTRACT Purpose To describe and analyze our experience with Anderson-Hynes transperitoneal laparoscopic pyeloplasty (LP in the treatment of recurrent ureteropelvic junction obstruction (UPJO. Materials and methods 38 consecutive patients who underwent transperitoneal laparoscopic redo-pyeloplasty between January 2007 and January 2015 at our department were included in the analysis. 36 patients were previously treated with dismembered pyeloplasty and 2 patients underwent a retrograde endopyelotomy. All patients were symptomatic and all patients had a T1/2>20 minutes at pre-operative DTPA (diethylene-triamine-pentaacetate renal scan. All data were collected in a prospectively maintained database and retrospectively analyzed. Intraoperative and postoperative complications have been reported according to the Satava and the Clavien-Dindo system. Treatment success was evaluated by a 12 month-postoperative renal scan. Total success was defined as T1/2≤10 minutes while relative success was defined as T1/2between 10 to 20 minutes. Post-operative hydronephrosis and flank pain were also evaluated. Results Mean operating time was 103.16±30 minutes. The mean blood loss was 122.37±73.25mL. The mean postoperative hospital stay was 4.47±0.86 days. No intraoperative complications occurred. 6 out of 38 patients (15.8% experienced postoperative complications. The success rate was 97.4% for flank pain and 97.4% for hydronephrosis. Post-operative renal scan showed radiological failure in one out of 38 (2.6% patients, relative success in 2 out of 38 (5.3% patients and total success in 35 out of 38 (92.1% of patients. Conclusion Laparoscopic redo-pyeloplasty is a feasible procedure for the treatment of recurrent ureteropelvic junction obstruction (UPJO, with a low rate of post-operative complications and a high success rate in high laparoscopic volume centers.
Chiancone, Francesco; Fedelini, Maurizio; Pucci, Luigi; Meccariello, Clemente; Fedelini, Paolo
2017-01-01
To describe and analyze our experience with Anderson-Hynes transperitoneal laparoscopic pyeloplasty (LP) in the treatment of recurrent ureteropelvic junction obstruction (UPJO). 38 consecutive patients who underwent transperitoneal laparoscopic redo-pyeloplasty between January 2007 and January 2015 at our department were included in the analysis. 36 patients were previously treated with dismembered pyeloplasty and 2 patients underwent a retrograde endopyelotomy. All patients were symptomatic and all patients had a T1/2>20 minutes at pre-operative DTPA (diethylene-triamine-pentaacetate) renal scan. All data were collected in a prospectively maintained database and retrospectively analyzed. Intraoperative and postoperative complications have been reported according to the Satava and the Clavien-Dindo system. Treatment success was evaluated by a 12 month-postoperative renal scan. Total success was defined as T1/2≤10 minutes while relative success was defined as T1/2between 10 to 20 minutes. Post-operative hydronephrosis and flank pain were also evaluated. Mean operating time was 103.16±30 minutes. The mean blood loss was 122.37±73.25mL. The mean postoperative hospital stay was 4.47±0.86 days. No intraoperative complications occurred. 6 out of 38 patients (15.8%) experienced postoperative complications. The success rate was 97.4% for flank pain and 97.4% for hydronephrosis. Post-operative renal scan showed radiological failure in one out of 38 (2.6%) patients, relative success in 2 out of 38 (5.3%) patients and total success in 35 out of 38 (92.1%) of patients. Laparoscopic redo-pyeloplasty is a feasible procedure for the treatment of recurrent ureteropelvic junction obstruction (UPJO), with a low rate of post-operative complications and a high success rate in high laparoscopic volume centers. Copyright® by the International Brazilian Journal of Urology.
DEFF Research Database (Denmark)
Miskowiak, Kamilla; Papadatou-Pastou, Marietta; Cowen, Philip J
2007-01-01
categorisation and recognition of self-referent personality trait words were assessed using event-related functional Magnetic Resonance Imaging (fMRI). Reboxetine had no effect on neuronal response during self-referent categorisation of positive or negative personality trait words. However, in a subsequent...... memory test, reboxetine reduced neuronal activation in a fronto-parietal network during correct recognition of positive target words vs. matched distractors. This was combined with increased speed to recognize positive vs. negative words compared to control subjects and suggests facilitated memory...... for positive self-referent material. These results support the hypothesis that antidepressants have early effects on the neural processing of emotional material which may be important in their therapeutic actions....
Directory of Open Access Journals (Sweden)
Chih-Ping Chen
2007-06-01
Full Text Available Omphalocele can be associated with single gene disorders, neural tube defects, diaphragmatic defects, fetal valproate syndrome, and syndromes of unknown etiology. This article provides a comprehensive review of omphalocele-related disorders: otopalatodigital syndrome type II; Melnick–Needles syndrome; Rieger syndrome; neural tube defects; Meckel syndrome; Shprintzen–Goldberg omphalocele syndrome; lethal omphalocele-cleft palate syndrome; cerebro-costo-mandibular syndrome; fetal valproate syndrome; Marshall–Smith syndrome; fibrochondrogenesis; hydrolethalus syndrome; Fryns syndrome; omphalocele, diaphragmatic defects, radial anomalies and various internal malformations; diaphragmatic defects, limb deficiencies and ossification defects of skull; Donnai–Barrow syndrome; CHARGE syndrome; Goltz syndrome; Carpenter syndrome; Toriello–Carey syndrome; familial omphalocele; Cornelia de Lange syndrome; C syndrome; Elejalde syndrome; Malpuech syndrome; cervical ribs, Sprengel anomaly, anal atresia and urethral obstruction; hydrocephalus with associated malformations; Kennerknecht syndrome; lymphedema, atrial septal defect and facial changes; and craniosynostosis- mental retardation syndrome of Lin and Gettig. Perinatal identification of omphalocele should alert one to the possibility of omphalocele-related disorders and familial inheritance and prompt a thorough genetic counseling for these disorders.
Directory of Open Access Journals (Sweden)
Matteo Cescon
2010-01-01
Full Text Available Background. Factors affecting outcomes after orthotopic liver transplantation (OLT for hepatocellular carcinoma (HCC have been extensively studied, but some of them have only recently been discovered or reassessed. Methods. We analyzed classical and more recently emerging variables with a hypothetical impact on recurrence-free survival (RFS in a single-center series of 283 patients transplanted for HCC between 1997 and 2009. Results. Five-year patient survival and RFS were 75% and 86%, respectively. Thirty-four (12% patients had HCC recurrence. Elevated preoperative alpha-fetoprotein (AFP levels, preoperative treatments of HCC, unfulfilled Milan and up-to-seven criteria at final histology, poor tumor differentiation, and tumor microvascular invasion negatively affected RFS by univariate analysis. Milan and up-to-seven criteria applied preoperatively, and the use of m-TOR inhibitors did not reach statistical significance. Cox's proportional hazard model showed that only elevated AFP levels (Odds Ratio=2.88; 95% C.I.=1.43–5.80; =.003, preoperative tumor treatments (Odds Ratio=4.84; 95% C.I.=1.42–16.42; =.01, and microvascular invasion (Odds Ratio=4.82; 95% C.I.=1.87–12.41; =.001 were predictors of lower RFS. Conclusions. Biological aggressiveness and preoperative tumor treatment, rather than traditional and expanded dimensional criteria, conditioned the outcomes in patients transplanted for HCC.
Directory of Open Access Journals (Sweden)
Di Micco Pierpaolo
2005-12-01
Full Text Available Abstract Background Antiphospholipid syndrome (APS has been often associated to RPL since 1980 and some reports in the Literature rarely described antibodies to factor XII in patients with APS. Case history We report the case history of 34-year-old caucasian women with recurrent fetal loss and persistent prolonged activated partial thromboplastin time. Haemostatic tests revealed persistent light decrease of clotting factor XII with normal values of IgG and IgM anticardiolipin antibodies and transient positivity for lupus anticoagulant (LA. Few reports in the Literature described antibodies to factor XII in patient with antiphospholipid syndrome (APS and transient LA. So, once other causes of RPL were excluded, the patient was diagnosed an unusual form of APS associated to antibodies to factor XII, reduced factor XII plasma levels, transient LA and prolonged activated partial thromboplastin time. Discussion We suggest to consider also antibodies directed to clotting factors (e.g. factor XII in our case as second step of thrombophilia screening in RPL, in particular if a persistent prolonged aPTT is present without an apparent cause.
Bocchi, L; Coppini, G; Nori, J; Valli, G
2004-05-01
Microcalcifications (microCas) are often early signs of breast cancer. However, detecting them is a difficult visual task and recognizing malignant lesions is a complex diagnostic problem. In recent years, several research groups have been working to develop computer-aided diagnosis (CAD) systems for X-ray mammography. In this paper, we propose a method to detect and classify microcalcifications. In order to discover the presence of microCas clusters, particular attention is paid to the analysis of the spatial arrangement of detected lesions. A fractal model has been used to describe the mammographic image, thus, allowing the use of a matched filtering stage to enhance microcalcifications against the background. A region growing algorithm, coupled with a neural classifier, detects existing lesions. Subsequently, a second fractal model is used to analyze their spatial arrangement so that the presence of microcalcification clusters can be detected and classified. Reported results indicate that fractal models provide an adequate framework for medical image processing; consequently high correct classification rates are achieved.
Volmer, M; de Vries, JCM; Goldschmidt, HMJ
Background: Preparation of KBr tablets, used for Fourier transform infrared (FT-IR) analysis of urinary calculus composition, is time-consuming and often hampered by pellet breakage. We developed a new F:T-IR method for urinary calculus analysis. This method makes use of a Golden Gate Single
International Nuclear Information System (INIS)
Karthik Raja, U; Leelamani, A; Raja, R; Samidurai, R
2013-01-01
In this paper, the exponential stability for a class of stochastic neural networks with time-varying delays and impulsive effects is considered. By constructing suitable Lyapunov functionals and by using the linear matrix inequality optimization approach, we obtain sufficient delay-dependent criteria to ensure the exponential stability of stochastic neural networks with time-varying delays and impulses. Two numerical examples with simulation results are provided to illustrate the effectiveness of the obtained results over those already existing in the literature. (paper)
Mintz, Michael; Khair, Shanawaj; Grewal, Suman; LaComb, Joseph F; Park, Jiyhe; Channer, Breana; Rajapakse, Ramona; Bucobo, Juan Carlos; Buscaglia, Jonathan M; Monzur, Farah; Chawla, Anupama; Yang, Jie; Robertson, Charlie E; Frank, Daniel N; Li, Ellen
2018-01-01
Studies of colonoscopic fecal microbiota transplant (FMT) in patients with recurrent CDI, indicate that this is a very effective treatment for preventing further relapses. In order to provide this service at Stony Brook University Hospital, we initiated an open-label prospective study of single colonoscopic FMT among patients with ≥ 2 recurrences of CDI, with the intention of monitoring microbial composition in the recipient before and after FMT, as compared with their respective donor. We also initiated a concurrent open label prospective trial of single colonoscopic FMT of patients with ulcerative colitis (UC) not responsive to therapy, after obtaining an IND permit (IND 15642). To characterize how FMT alters the fecal microbiota in patients with recurrent Clostridia difficile infections (CDI) and/or UC, we report the results of a pilot microbiome analysis of 11 recipients with a history of 2 or more recurrences of C. difficile infections without inflammatory bowel disease (CDI-only), 3 UC recipients with recurrent C. difficile infections (CDI + UC), and 5 UC recipients without a history of C. difficile infections (UC-only). V3V4 Illumina 16S ribosomal RNA (rRNA) gene sequencing was performed on the pre-FMT, 1-week post-FMT, and 3-months post-FMT recipient fecal samples along with those collected from the healthy donors. Fitted linear mixed models were used to examine the effects of Group (CDI-only, CDI + UC, UC-only), timing of FMT (Donor, pre-FMT, 1-week post-FMT, 3-months post-FMT) and first order Group*FMT interactions on the diversity and composition of fecal microbiota. Pairwise comparisons were then carried out on the recipient vs. donor and between the pre-FMT, 1-week post-FMT, and 3-months post-FMT recipient samples within each group. Significant effects of FMT on overall microbiota composition (e.g., beta diversity) were observed for the CDI-only and CDI + UC groups. Marked decreases in the relative abundances of the strictly anaerobic Bacteroidetes
Persistent and recurrent hyperparathyroidism.
Guerin, Carole; Paladino, Nunzia Cinzia; Lowery, Aoife; Castinetti, Fréderic; Taieb, David; Sebag, Fréderic
2017-06-01
Despite remarkable progress in imaging modalities and surgical management, persistence or recurrence of primary hyperparathyroidism (PHPT) still occurs in 2.5-5% of cases of PHPT. The aim of this review is to expose the management of persistent and recurrent hyperparathyroidism. A literature search was performed on MEDLINE using the search terms "recurrent" or "persistent" and "hyperparathyroidism" within the past 10 years. We also searched the reference lists of articles identified by this search strategy and selected those we judged relevant. Before considering reoperation, the surgeon must confirm the diagnosis of PHPT. Then, the patient must be evaluated with new imaging modalities. A single adenoma is found in 68% of cases, multiglandular disease in 28%, and parathyroid carcinoma in 3%. Others causes (<1%) include parathyromatosis and graft recurrence. The surgeon must balance the benefits against the risks of a reoperation (permanent hypocalcemia and recurrent laryngeal nerve palsy). If surgery is necessary, a focused approach can be considered in cases of significant imaging foci, but in the case of multiglandular disease, a bilateral neck exploration could be necessary. Patients with multiple endocrine neoplasia syndromes are at high risk of recurrence and should be managed regarding their hereditary pathology. The cure rate of persistent-PHPT or recurrent-PHPT in expert centers is estimated from 93 to 97%. After confirming the diagnosis of PHPT, patients with persistent-PHPT and recurrent-PHPT should be managed in an expert center with all dedicated competencies.
Directory of Open Access Journals (Sweden)
Ruijie Li
2018-04-01
Full Text Available In vivo two-photon Ca2+ imaging is a powerful tool for recording neuronal activities during perceptual tasks and has been increasingly applied to behaving animals for acute or chronic experiments. However, the auditory cortex is not easily accessible to imaging because of the abundant temporal muscles, arteries around the ears and their lateral locations. Here, we report a protocol for two-photon Ca2+ imaging in the auditory cortex of head-fixed behaving mice. By using a custom-made head fixation apparatus and a head-rotated fixation procedure, we achieved two-photon imaging and in combination with targeted cell-attached recordings of auditory cortical neurons in behaving mice. Using synthetic Ca2+ indicators, we recorded the Ca2+ transients at multiple scales, including neuronal populations, single neurons, dendrites and single spines, in auditory cortex during behavior. Furthermore, using genetically encoded Ca2+ indicators (GECIs, we monitored the neuronal dynamics over days throughout the process of associative learning. Therefore, we achieved two-photon functional imaging at multiple scales in auditory cortex of behaving mice, which extends the tool box for investigating the neural basis of audition-related behaviors.
Fu, Yunting; Wang, Lin-lin; Yi, Deqing; Jin, Lei; Liu, Jufen; Zhang, Yali; Ren, Aiguo
2015-06-01
Maternal pregestational hyperglycemia, diabetes, and obesity are well-established risk factors for neural tube defects (NTDs). As a common underlying mechanism, the imbalance of glucose homeostasis is directly related to the development of NTDs. Polymorphisms in genes regulating glucose metabolism in women may impact their chance of having an NTD-affected pregnancy. We conducted a two-stage case-control study to investigate the association between maternal genetic variants in genes regulating glucose metabolism and risk for NTDs. The cases were 547 women who gave birth to a child with an NTD (anencephaly, spina bifida, or encephalocele); the controls were 543 women who gave birth to a full-term healthy infant. In the first stage, 12 single nucleotide polymorphisms were genotyped in 160 cases and 162 controls. In the second stage, five single nucleotide polymorphisms found in the first stage and potentially associated with NTD risk were genotyped for validation, in an additional 387 cases and 381 controls. Combined analysis of data from the two stages showed an association between maternal AA genotype of GCKR rs780094 and increased risk for total NTDs [odds ratio, 1.73; 95% confidence interval, 1.16-2.59) and spina bifida subtype [odds ratio, 1.83; 95% confidence interval, 1.16-2.88). No association was found between the other four single nucleotide polymorphisms (LEPR rs1137100, HK1 rs748235, HHEX rs5015480, KCNQ1 rs2237892) and NTD risk. The AA genotype in maternal GCKR rs780094 is associated with an increased risk for NTDs and spina bifida in the Chinese population. © 2014 Wiley Periodicals, Inc.
Vasavada, Bhavin B; Chan, Chao Long
2015-02-01
Hepatitis C virus recurrence after transplant is universal. Histologic recurrence is observed in > 50% hepatitis C virus-infected grafts within the first year. The primary aim of our study was to evaluate factors responsible for hepatocellular carcinoma recurrence and mortality including histologic markers. All patients who had undergone transplant for hepatocellular carcinoma associated with hepatitis C virus from 2002 to 2012 were evaluated retrospectively. There were 109 patients with hepatocellular carcinoma associated with hepatitis C virus that underwent living-donor liver transplant from July 2002 to June 2012. On univariate analysis, preoperative Model for End-Stage Liver Disease Score (P = .026), α-fetoprotein level (P = .020), rapid fibrosis (P = .008), and Hepatitis Activity Index ≥ 6 (P = .008) were associated with recurrence. On multivariate Cox proportional hazards regression model, Model for End-Stage Liver Disease score (P Hepatitis C virus recurrence on biopsy is a poor prognostic indicator and is associated with a higher risk of hepatocellular carcinoma recurrence after liver transplant. Rapid fibrosis after liver transplant independently predicts hepatocellular carcinoma recurrence.
DEFF Research Database (Denmark)
Jensen, Ulrich Stab; Knudsen, Inge Jenny Dahl; Ostergaard, Christian
2010-01-01
A population-based nested case-control study was conducted in order to characterize patient factors and microbial species associated with recurrent bacteraemia.......A population-based nested case-control study was conducted in order to characterize patient factors and microbial species associated with recurrent bacteraemia....
Munro, Paul Wesley
A special form for modification of neuronal response properties is described in which the change in the synaptic state vector is parallel to the vector of afferent activity. This process is termed "parallel modification" and its theoretical and experimental implications are examined. A theoretical framework has been devised to describe the complementary functions of generalization and discrimination by single neurons. This constitutes a basis for three models each describing processes for the development of maximum selectivity (discrimination) and minimum selectivity (generalization) by neurons. Strengthening and weakening of synapses is expressed as a product of the presynaptic activity and a nonlinear modulatory function of two postsynaptic variables--namely a measure of the spatially integrated activity of the cell and a temporal integration (time-average) of that activity. Some theorems are given for low-dimensional systems and computer simulation results from more complex systems are discussed. Model neurons that achieve high selectivity mimic the development of cat visual cortex neurons in a wide variety of rearing conditions. A role for low-selectivity neurons is proposed in which they provide inhibitory input to neurons of the opposite type, thereby suppressing the common component of a pattern class and enhancing their selective properties. Such contrast-enhancing circuits are analyzed and supported by computer simulation. To enable maximum selectivity, the net inhibition to a cell must become strong enough to offset whatever excitation is produced by the non-preferred patterns. Ramifications of parallel models for certain experimental paradigms are analyzed. A methodology is outlined for testing synaptic modification hypotheses in the laboratory. A plastic projection from one neuronal population to another will attain stable equilibrium under periodic electrical stimulation of constant intensity. The perturbative effect of shifting this intensity level
Directory of Open Access Journals (Sweden)
Trong-Ngoc Le
2016-01-01
Full Text Available Objective. Our objective is to develop a computerized scheme for liver tumor segmentation in MR images. Materials and Methods. Our proposed scheme consists of four main stages. Firstly, the region of interest (ROI image which contains the liver tumor region in the T1-weighted MR image series was extracted by using seed points. The noise in this ROI image was reduced and the boundaries were enhanced. A 3D fast marching algorithm was applied to generate the initial labeled regions which are considered as teacher regions. A single hidden layer feedforward neural network (SLFN, which was trained by a noniterative algorithm, was employed to classify the unlabeled voxels. Finally, the postprocessing stage was applied to extract and refine the liver tumor boundaries. The liver tumors determined by our scheme were compared with those manually traced by a radiologist, used as the “ground truth.” Results. The study was evaluated on two datasets of 25 tumors from 16 patients. The proposed scheme obtained the mean volumetric overlap error of 27.43% and the mean percentage volume error of 15.73%. The mean of the average surface distance, the root mean square surface distance, and the maximal surface distance were 0.58 mm, 1.20 mm, and 6.29 mm, respectively.
Directory of Open Access Journals (Sweden)
Wust Peter
2010-05-01
Full Text Available Abstract Background To assess radiobiological restrictions and tolerance doses as well as other toxic effects derived from repeated applications of single-fraction high dose rate irradiation of small liver volumes in clinical practice. Methods Twenty patients with liver metastases were treated repeatedly (2 - 4 times at identical or intersecting locations by CT-guided interstitial brachytherapy with varying time intervals. Magnetic resonance imaging using the hepatocyte selective contrast media Gd-BOPTA was performed before and after treatment to determine the volume of hepatocyte function loss (called pseudolesion, and the last acquired MRI data set was merged with the dose distributions of all administered brachytherapies. We calculated the BED (biologically equivalent dose for a single dose d = 2 Gy for different α/β values (2, 3, 10, 20, 100 based on the linear-quadratic model and estimated the tolerance dose for liver parenchyma D90 as the BED exposing 90% of the pseudolesion in MRI. Results The tolerance doses D90 after repeated brachytherapy sessions were found between 22 - 24 Gy and proved only slightly dependent on α/β in the clinically relevant range of α/β = 2 - 10 Gy. Variance analysis showed a significant dependency of D90 with respect to the intervals between the first irradiation and the MRI control (p 90 and the pseudolesion's volume. No symptoms of liver dysfunction or other toxic effects such as abscess formation occurred during the follow-up time, neither acute nor on the long-term. Conclusions Inactivation of liver parenchyma occurs at a BED of approx. 22 - 24 Gy corresponding to a single dose of ~10 Gy (α/β ~ 5 Gy. This tolerance dose is consistent with the large potential to treat oligotopic and/or recurrent liver metastases by CT-guided HDR brachytherapy without radiation-induced liver disease (RILD. Repeated small volume irradiation may be applied safely within the limits of this study.
Rosenberg, Jon; Galen, Benjamin T
2017-07-01
Recurrent meningitis is a rare clinical scenario that can be self-limiting or life threatening depending on the underlying etiology. This review describes the causes, risk factors, treatment, and prognosis for recurrent meningitis. As a general overview of a broad topic, the aim of this review is to provide clinicians with a comprehensive differential diagnosis to aide in the evaluation and management of a patient with recurrent meningitis. New developments related to understanding the pathophysiology of recurrent meningitis are as scarce as studies evaluating the treatment and prevention of this rare disorder. A trial evaluating oral valacyclovir suppression after HSV-2 meningitis did not demonstrate a benefit in preventing recurrences. The data on prophylactic antibiotics after basilar skull fractures do not support their use. Intrathecal trastuzumab has shown promise in treating leptomeningeal carcinomatosis from HER-2 positive breast cancer. Monoclonal antibodies used to treat cancer and autoimmune diseases are new potential causes of drug-induced aseptic meningitis. Despite their potential for causing recurrent meningitis, the clinical entities reviewed herein are not frequently discussed together given that they are a heterogeneous collection of unrelated, rare diseases. Epidemiologic data on recurrent meningitis are lacking. The syndrome of recurrent benign lymphocytic meningitis described by Mollaret in 1944 was later found to be closely related to HSV-2 reactivation, but HSV-2 is by no means the only etiology of recurrent aseptic meningitis. While the mainstay of treatment for recurrent meningitis is supportive care, it is paramount to ensure that reversible and treatable causes have been addressed for further prevention.
Wang, Chih-Hung; Wu, Cheng-Yi; Yang, Justin Cheng-Ta; Lien, Wan-Ching; Wang, Hsiu-Po; Liu, Kao-Lang; Wu, Yao-Ming; Chen, Shyr-Chyr
2016-01-01
Background Percutaneous cholecystostomy tube (PCT) has been effectively used for the treatment of acute cholecystitis (AC) for patients unsuitable for early cholecystectomy. This retrospective study investigated the recurrence rate after successful PCT treatment and factors associated with recurrence. Methods We reviewed patients treated with PCT for AC from October 2004 through December 2013. Patients with successful PCT treatment were those who were free from persistent PCT drainage. We used multivariable logistic regression analysis sequentially to identify factors associated with each outcome. Results The study included 184 patients (mean age: 70.1 years). The average duration for parenteral antibiotics was 14.4 days and 20.0 days for PCT drainage. The one-year recurrence rate was 9.2% (17/184) with most recurrences occurring within two months (6.5%, 12/184) of the procedure. Complicated cholecystitis (odds ratio [OR]: 4.67; 95% confidence interval [CI]: 1.44–15.70; P = 0.01) and PCT drainage duration >32 days (OR: 4.92; 95% CI: 1.03–23.53; P = 0.05) positively correlated with one-year recurrence; parenteral antibiotics duration >10 days (OR: 0.21; 95% CI: 0.05–0.68; P = 0.01) was inversely associated with one-year recurrence. Conclusions The recurrence rate was low for patients after successful PCT treatment. Predictors for recurrence included the severity of initial AC and subsequently provided treatments. PMID:26821150
Directory of Open Access Journals (Sweden)
Mohamed Mohamed Elawdy
2017-09-01
Conclusions: In our present series, bladder cancer recurrence of urothelial malignancy occurred in nearly half of the patients after surgical management of UTUC. Ureteric tumour was the only identifiable risk factor, thus patients with ureteric tumours may benefit from prophylactic intravesical chemoimmunotherapy. Bladder recurrence does not appear to affect the cancer-specific survival after surgical management of UTUC.
Taal, Walter; Oosterkamp, Hendrika M.; Walenkamp, Annemiek M. E.; Dubbink, Hendrikus J.; Beerepoot, Laurens V.; Hanse, Monique C. J.; Buter, Jan; Honkoop, Aafke H.; Boerman, Dolf; de Vos, Filip Y. F.; Dinjens, Winand N. M.; Enting, Roeline; Taphoorn, Martin J. B.; van den Berkmortel, Franchette W. P. J.; Jansen, Rob L. H.; Brandsma, Dieta; Bromberg, Jacoline E. C.; van Heuvel, Irene; Vernhout, Rene M.; van der Holt, Bronno; van den Bent, Martin J.
BACKGROUND: Treatment options for recurrent glioblastoma are scarce, with second-line chemotherapy showing only modest activity against the tumour. Despite the absence of well controlled trials, bevacizumab is widely used in the treatment of recurrent glioblastoma. Nonetheless, whether the high
Invernizzi, Rosangela; Quaglia, Federica; Klersy, Catherine; Pagella, Fabio; Ornati, Federica; Chu, Francesco; Matti, Elina; Spinozzi, Giuseppe; Plumitallo, Sara; Grignani, Pierangela; Olivieri, Carla; Bastia, Raffaella; Bellistri, Francesca; Danesino, Cesare; Benazzo, Marco; Balduini, Carlo L
2015-11-01
Hereditary haemorrhagic telangiectasia is a genetic disease that leads to multiregional angiodysplasia. Severe recurrent epistaxis is the most common presentation, frequently leading to severe anaemia. Several therapeutic approaches have been investigated, but they are mostly palliative and have had variable results. We aimed to assess the efficacy of thalidomide for the reduction of epistaxis in patients with hereditary haemorrhagic telangiectasia that is refractory to standard therapy. We recruited patients aged 17 years or older with hereditary haemorrhagic telangiectasia who had severe recurrent epistaxis refractory to minimally invasive surgical procedures into an open-label, phase 2, non-randomised, single-centre study at IRCCS Policlinico San Matteo Foundation (Pavia, Italy). We gave patients thalidomide at a starting dose of 50 mg/day orally. If they had no response, we increased the thalidomide dose by 50 mg/day increments every 4 weeks, until a response was seen, up to a maximum dose of 200 mg/day. After patients had achieved a response, they continued treatment for 8-16 additional weeks. The primary endpoint was the efficacy of thalidomide measured as the percentage of patients who had reductions of at least one grade in the frequency, intensity, or duration of epistaxis. We followed up patients each month to assess epistaxis severity score and transfusion need, and any adverse events were reported. We included all patients who received any study drug and who participated in at least one post-baseline assessment in the primary efficacy population. The safety population consisted of all patients who received any dose of study treatment. This trial is registered with ClinicalTrials.gov, number NCT01485224. Between Dec 1, 2011, and May 12, 2014, we enrolled 31 patients. Median follow-up was 15·9 months (IQR 10·1-22·3). Three (10%, 95% CI 2-26) patients had a complete response, with bleeding stopped, 28 (90%, 95% CI 74-98) patients had partial responses
Mirbozorgi, S Abdollah; Bahrami, Hadi; Sawan, Mohamad; Rusch, Leslie A; Gosselin, Benoit
2016-06-01
We present a novel, fully-integrated, low-power full-duplex transceiver (FDT) to support high-density and bidirectional neural interfacing applications (high-channel count stimulating and recording) with asymmetric data rates: higher rates are required for recording (uplink signals) than stimulation (downlink signals). The transmitter (TX) and receiver (RX) share a single antenna to reduce implant size and complexity. The TX uses impulse radio ultra-wide band (IR-UWB) based on an edge combining approach, and the RX uses a novel 2.4-GHz on-off keying (OOK) receiver. Proper isolation (>20 dB) between the TX and RX path is implemented 1) by shaping the transmitted pulses to fall within the unregulated UWB spectrum (3.1-7 GHz), and 2) by space-efficient filtering (avoiding a circulator or diplexer) of the downlink OOK spectrum in the RX low-noise amplifier. The UWB 3.1-7 GHz transmitter can use either OOK or binary phase shift keying (BPSK) modulation schemes. The proposed FDT provides dual band 500-Mbps TX uplink data rate and 100 Mbps RX downlink data rate, and it is fully integrated into standard TSMC 0.18- μm CMOS within a total size of 0.8 mm(2). The total measured power consumption is 10.4 mW in full duplex mode (5 mW at 100 Mbps for RX, and 5.4 mW at 500 Mbps or 10.8 pJ/bit for TX). Additionally, a 3-coil inductive link along with on-chip power management circuits allows to powering up the implantable transceiver wirelessly by delivering 25 mW extracted from a 13.56-MHz carrier signal, at a total efficiency of 41.6%.
Knox, Dayan; Stanfield, Briana R; Staib, Jennifer M; David, Nina P; DePietro, Thomas; Chamness, Marisa; Schneider, Elizabeth K; Keller, Samantha M; Lawless, Caroline
2018-04-02
Neural circuits via which stress leads to disruptions in fear extinction is often explored in animal stress models. Using the single prolonged stress (SPS) model of post traumatic stress disorder and the immediate early gene (IEG) c-Fos as a measure of neural activity, we previously identified patterns of neural activity through which SPS disrupts extinction retention. However, none of these stress effects were specific to fear or extinction learning and memory. C-Jun is another IEG that is sometimes regulated in a different manner to c-Fos and could be used to identify emotional learning/memory specific patterns of neural activity that are sensitive to SPS. Animals were either fear conditioned (CS-fear) or presented with CSs only (CS-only) then subjected to extinction training and testing. C-Jun was then assayed within neural substrates critical for extinction memory. Inhibited c-Jun levels in the hippocampus (Hipp) and enhanced functional connectivity between the ventromedial prefrontal cortex (vmPFC) and basolateral amygdala (BLA) during extinction training was disrupted by SPS in the CS-fear group only. As a result, these effects were specific to emotional learning/memory. SPS also disrupted inhibited Hipp c-Jun levels, enhanced BLA c-Jun levels, and altered functional connectivity among the vmPFC, BLA, and Hipp during extinction testing in SPS rats in the CS-fear and CS-only groups. As a result, these effects were not specific to emotional learning/memory. Our findings suggest that SPS disrupts neural activity specific to extinction memory, but may also disrupt the retention of fear extinction by mechanisms that do not involve emotional learning/memory. Copyright © 2017 Elsevier B.V. All rights reserved.
Study of Single Top Quark Production Using Bayesian Neural Networks With D0 Detector at the Tevatron
Energy Technology Data Exchange (ETDEWEB)
Joshi, Jyoti [Panjab Univ., Chandigarh (India)
2012-01-01
Top quark, the heaviest and most intriguing among the six known quarks, can be created via two independent production mechanisms in {\\pp} collisions. The primary mode, strong {\\ttbar} pair production from a $gtt$ vertex, was used by the {\\d0} and CDF collaborations to establish the existence of the top quark in March 1995. The second mode is the electroweak production of a single top quark or antiquark, which has been observed recently in March 2009. Since single top quarks are produced at hadron colliders through a $Wtb$ vertex, thereby provide a direct probe of the nature of $Wtb$ coupling and of the Cabibbo-Kobayashi-Maskawa matrix element, $V_{tb}$. So this mechanism provides a sensitive probe for several, standard model and beyond standard model, parameters such as anomalous $Wtb$ couplings. In this thesis, we measure the cross section of the electroweak produced top quark in three different production modes, $s+t$, $s$ and $t$-channels using a technique based on the Bayesian neural networks. This technique is applied for analysis of the 5.4 $fb^{-1}$ of data collected by the {\\d0} detector. From a comparison of the Bayesian neural networks discriminants between data and the signal-background model using Bayesian statistics, the cross sections of the top quark produced through the electroweak mechanism have been measured as: \\[\\sigma(p\\bar{p}→tb+X,tqb+X) = 3.11^{+0.77}_{-0.71}\\;\\rm pb\\] \\[\\sigma(p\\bar{p}→tb+X) = 0.72^{+0.44}_{-0.43}\\;\\rm pb\\] \\[\\sigma(p\\bar{p}→tqb+X) = 2.92^{+0.87}_{-0.73}\\;\\rm pb\\] % The $s+t$-channel has a gaussian significance of $4.7\\sigma$, the $s$-channel $0.9\\sigma$ and the $t$-channel~$4.7\\sigma$. The results are consistent with the standard model predictions within one standard deviation. By combining these results with the results for two other analyses (using different MVA techniques) improved results \\[\\sigma(p\\bar{p}→tb+X,tqb+X) = 3.43^{+0.73}_{-0.74}\\;\\rm pb\\] \\[\\sigma
Directory of Open Access Journals (Sweden)
Prativa Sahoo
2018-01-01
Full Text Available Background. The aim of this study was to correlate T1-weighted dynamic contrast-enhanced MRI- (DCE-MRI- derived perfusion parameters with overall survival of recurrent high-grade glioma patients who received neural stem cell- (NSC- mediated enzyme/prodrug gene therapy. Methods. A total of 12 patients were included in this retrospective study. All patients were enrolled in a first-in-human study (NCT01172964 of NSC-mediated therapy for recurrent high-grade glioma. DCE-MRI data from all patients were collected and analyzed at three time points: MRI#1—day 1 postsurgery/treatment, MRI#2— day 7 ± 3 posttreatment, and MRI#3—one-month follow-up. Plasma volume (Vp, permeability (Ktr, and leakage (λtr perfusion parameters were calculated by fitting a pharmacokinetic model to the DCE-MRI data. The contrast-enhancing (CE volume was measured from the last dynamic phase acquired in the DCE sequence. Perfusion parameters and CE at each MRI time point were recorded along with their relative change between MRI#2 and MRI#3 (Δ32. Cox regression was used to analyze patient survival. Results. At MRI#1 and at MRI#3, none of the parameters showed a significant correlation with overall survival (OS. However, at MRI#2, CE and λtr were significantly associated with OS (p<0.05. The relative λtr and Vp from timepoint 2 to timepoint 3 (Δ32λtr and Δ32Vp were each associated with a higher hazard ratio (p<0.05. All parameters were highly correlated, resulting in a multivariate model for OS including only CE at MRI#2 and Δ32Vp, with an R2 of 0.89. Conclusion. The change in perfusion parameter values from 1 week to 1 month following NSC-mediated therapy combined with contrast-enhancing volume may be a useful biomarker to predict overall survival in patients with recurrent high-grade glioma.
Ichinohe, Y.; Yamada, S.; Miyazaki, N.; Saito, S.
2018-04-01
We present data preprocessing based on an artificial neural network to estimate the parameters of the X-ray emission spectra of a single-temperature thermal plasma. The method finds appropriate parameters close to the global optimum. The neural network is designed to learn the parameters of the thermal plasma (temperature, abundance, normalization and redshift) of the input spectra. After training using 9000 simulated X-ray spectra, the network has grown to predict all the unknown parameters with uncertainties of about a few per cent. The performance dependence on the network structure has been studied. We applied the neural network to an actual high-resolution spectrum obtained with Hitomi. The predicted plasma parameters agree with the known best-fitting parameters of the Perseus cluster within uncertainties of ≲10 per cent. The result shows that neural networks trained by simulated data might possibly be used to extract a feature built in the data. This would reduce human-intensive preprocessing costs before detailed spectral analysis, and would help us make the best use of the large quantities of spectral data that will be available in the coming decades.
Comparato, Giuseppe; Di Mario, Francesco
2008-01-01
The term "diverticulitis" indicates the inflammation of a diverticulum or diverticula, which is accompanied by detectable or microscopical perforation. Diverticulitis is a common condition with an estimated incidence of 25%. At present, elective sigmoid resection is recommended after 2 episodes of uncomplicated diverticulitis to prevent the serious complications of recurrent colonic diverticulitis. This guideline has been based on the assumption that recurrent episodes (2 or more) of diverticulitis will lead to complicated diverticulitis and higher mortality. The data to support this assumption are based on only a few small studies. Advances in diagnostic modalities, medical therapy, and surgical techniques over the past 2 decades have changed both the management and outcomes of diverticulitis. Many authors have shown that patients treated nonoperatively have a low risk of recurrent disease and would be expected to do well without elective colectomy.
Imazio, M; Battaglia, A; Gaido, L; Gaita, F
2017-05-01
Recurrent pericarditis is the most troublesome complication of pericarditis occurring in 15 to 30% of cases. The pathogenesis is often presumed to be immune-mediated although a specific rheumatologic diagnosis is commonly difficult to find. The clinical diagnosis is based on recurrent pericarditis chest pain and additional objective evidence of disease activity (e.g. pericardial rub, ECG changes, pericardial effusion, elevation of markers of inflammation, and/or imaging evidence of pericardial inflammation by CT or cardiac MR). The mainstay of medical therapy for recurrent pericarditis is aspirin or a non-steroidal anti-inflammatory drug (NSAID) plus colchicine. Second-line therapy is considered after failure of such treatments and it is generally based on low to moderate doses of corticosteroids (e.g. prednisone 0.2 to 0.5 mg/kg/day or equivalent) plus colchicine. More difficult cases are treated with combination of aspirin or NSAID, colchicine and corticosteroids. Refractory cases are managed by alternative medical options, including azathioprine, or intravenous human immunoglobulins or biological agents (e.g. anakinra). When all medical therapies fail, the last option may be surgical by pericardiectomy to be recommended in well-experienced centres. Despite a significant impairment of the quality of life, the most common forms of recurrent pericarditis (usually named as "idiopathic recurrent pericarditis" since without a well-defined etiological diagnosis) have good long-term outcomes with a negligible risk of developing constriction and rarely cardiac tamponade during follow-up. The present article reviews current knowledge on the definition, diagnosis, aetiology, therapy and prognosis of recurrent pericarditis with a focus on the more recent available literature. Copyright © 2016 Société Nationale Française de Médecine Interne (SNFMI). Published by Elsevier SAS. All rights reserved.
Directory of Open Access Journals (Sweden)
Zheng Lu
2017-06-01
Full Text Available A method using a nonlinear auto-regressive neural network with exogenous input (NARXnn to retrieve time series soil moisture (SM that is spatially and temporally continuous and high quality over the Heihe River Basin (HRB in China was investigated in this study. The input training data consisted of the X-band dual polarization brightness temperature (TB and the Ka-band V polarization TB from the Advanced Microwave Scanning Radiometer II (AMSR2, Global Land Satellite product (GLASS Leaf Area Index (LAI, precipitation from the Tropical Rainfall Measuring Mission (TRMM and the Global Precipitation Measurement (GPM, and a global 30 arc-second elevation (GTOPO-30. The output training data were generated from fused SM products of the Japan Aerospace Exploration Agency (JAXA and the Land Surface Parameter Model (LPRM. The reprocessed fused SM from two years (2013 and 2014 was inputted into the NARXnn for training; subsequently, SM during a third year (2015 was estimated. Direct and indirect validations were then performed during the period 2015 by comparing with in situ measurements, SM from JAXA, LPRM and the Global Land Data Assimilation System (GLDAS, as well as precipitation data from TRMM and GPM. The results showed that the SM predictions from NARXnn performed best, as indicated by their higher correlation coefficients (R ≥ 0.85 for the whole year of 2015, lower Bias values (absolute value of Bias ≤ 0.02 and root mean square error values (RMSE ≤ 0.06, and their improved response to precipitation. This method is being used to produce the NARXnn SM product over the HRB in China.
Directory of Open Access Journals (Sweden)
T. D. Xenos
2002-01-01
Full Text Available In this work, Neural-Network-based single-station hourly daily foF2 and M(3000F2 modelling of 15 European ionospheric stations is investigated. The data used are neural networks and hourly daily values from the period 1964- 1988 for training the neural networks and from the period 1989-1994 for checking the prediction accuracy. Two types of models are presented for the F2-layer critical frequency prediction and two for the propagation factor M(3000F2. The first foF2 model employs the E-layer local noon calculated daily critical frequency (foE12 and the local noon F2- layer critical frequency of the previous day. The second foF2 model, which introduces a new regional mapping technique, employs the Juliusruh neural network model and uses the E-layer local noon calculated daily critical frequency (foE12, and the previous day F2-layer critical frequency measured at Juliusruh at noon. The first M(3000F2 model employs the E-layer local noon calculated daily critical frequency (foE12, its ± 3 h deviations and the local noon cosine of the solar zenith angle (cos c12. The second model, which introduces a new M(3000F2 mapping technique, employs Juliusruh neural network model and uses the E-layer local noon calculated daily critical frequency (foE12, and the previous day F2-layer critical frequency measured at Juliusruh at noon.
Dynamic training algorithm for dynamic neural networks
International Nuclear Information System (INIS)
Tan, Y.; Van Cauwenberghe, A.; Liu, Z.
1996-01-01
The widely used backpropagation algorithm for training neural networks based on the gradient descent has a significant drawback of slow convergence. A Gauss-Newton method based recursive least squares (RLS) type algorithm with dynamic error backpropagation is presented to speed-up the learning procedure of neural networks with local recurrent terms. Finally, simulation examples concerning the applications of the RLS type algorithm to identification of nonlinear processes using a local recurrent neural network are also included in this paper
Directory of Open Access Journals (Sweden)
V. Rezan USLU
2010-01-01
Full Text Available Obtaining the inflation prediction is an important problem. Having this prediction accurately will lead to more accurate decisions. Various time series techniques have been used in the literature for inflation prediction. Recently, Artificial Neural Network (ANN is being preferred in the time series prediction problem due to its flexible modeling capacity. Artificial neural network can be applied easily to any time series since it does not require prior conditions such as a linear or curved specific model pattern, stationary and normal distribution. In this study, the predictions have been obtained using the feed forward and recurrent artificial neural network for the Consumer Price Index (CPI. A new combined forecast has been proposed based on ANN in which the ANN model predictions employed in analysis were used as data.
Metastable neural dynamics mediates expectation
Mazzucato, Luca; La Camera, Giancarlo; Fontanini, Alfredo
Sensory stimuli are processed faster when their presentation is expected compared to when they come as a surprise. We previously showed that, in multiple single-unit recordings from alert rat gustatory cortex, taste stimuli can be decoded faster from neural activity if preceded by a stimulus-predicting cue. However, the specific computational process mediating this anticipatory neural activity is unknown. Here, we propose a biologically plausible model based on a recurrent network of spiking neurons with clustered architecture. In the absence of stimulation, the model neural activity unfolds through sequences of metastable states, each state being a population vector of firing rates. We modeled taste stimuli and cue (the same for all stimuli) as two inputs targeting subsets of excitatory neurons. As observed in experiment, stimuli evoked specific state sequences, characterized in terms of `coding states', i.e., states occurring significantly more often for a particular stimulus. When stimulus presentation is preceded by a cue, coding states show a faster and more reliable onset, and expected stimuli can be decoded more quickly than unexpected ones. This anticipatory effect is unrelated to changes of firing rates in stimulus-selective neurons and is absent in homogeneous balanced networks, suggesting that a clustered organization is necessary to mediate the expectation of relevant events. Our results demonstrate a novel mechanism for speeding up sensory coding in cortical circuits. NIDCD K25-DC013557 (LM); NIDCD R01-DC010389 (AF); NSF IIS-1161852 (GL).
International Nuclear Information System (INIS)
Hu, Yu-Wen; Huang, Pin-I; Wong, Tai-Tong; Ho, Donald Ming-Tak; Chang, Kai-Ping; Guo, Wan-Yuo; Chang, Feng-Chi; Shiau, Cheng-Yin; Liang, Muh-Lii; Lee, Yi-Yen
2012-01-01
Purpose: Intracranial germinomas (IGs) are highly curable with radiotherapy (RT). However, recurrence still occurs, especially when limited-field RT is applied, and the optimal salvage therapy remains controversial. Methods and Materials: Between January 1989 and December 2010, 14 patients with clinically or pathologically diagnosed recurrent IGs after RT were reviewed at our institution. Of these, 11 received focal-field RT, and the other 3 received whole-brain irradiation, whole-ventricle irradiation, and Gamma Knife radiosurgery as the respective first course of RT. In addition, we identified from the literature 88 patients with recurrent IGs after reduced-volume RT, in whom the details of salvage therapy were recorded. Results: The median time to recurrence was 30.3 months (range, 3.8–134.9 months). One patient did not receive further treatment and was lost during follow-up. Of the patients, 7 underwent salvage with craniospinal irradiation (CSI) plus chemotherapy (CT), 4 with CSI alone, 1 with whole-brain irradiation plus CT, and 1 with Gamma Knife radiosurgery. The median follow-up time was 105.1 months (range, 24.2–180.9 months). Three patients died without evidence of disease progression: two from second malignancies and one from unknown cause. The others remained disease free. The 3-year survival rate after recurrence was 83.3%. A total of 102 patients from our study and the literature review were analyzed to determine the factors affecting prognosis and outcomes. After recurrence, the 5-year survival rates were 71% and 92.9% for all patients and for those receiving salvage CSI, respectively. Univariate analysis showed that initial RT volume, initial RT dose, initial CT, and salvage RT type were significant prognostic predictors of survival. On multivariable analysis, salvage CSI was the most significant factor (p = 0.03). Conclusions: Protracted follow-up is recommended because late recurrence is not uncommon. CSI with or without CT is an effective
End-to-End Neural Segmental Models for Speech Recognition
Tang, Hao; Lu, Liang; Kong, Lingpeng; Gimpel, Kevin; Livescu, Karen; Dyer, Chris; Smith, Noah A.; Renals, Steve
2017-12-01
Segmental models are an alternative to frame-based models for sequence prediction, where hypothesized path weights are based on entire segment scores rather than a single frame at a time. Neural segmental models are segmental models that use neural network-based weight functions. Neural segmental models have achieved competitive results for speech recognition, and their end-to-end training has been explored in several studies. In this work, we review neural segmental models, which can be viewed as consisting of a neural network-based acoustic encoder and a finite-state transducer decoder. We study end-to-end segmental models with different weight functions, including ones based on frame-level neural classifiers and on segmental recurrent neural networks. We study how reducing the search space size impacts performance under different weight functions. We also compare several loss functions for end-to-end training. Finally, we explore training approaches, including multi-stage vs. end-to-end training and multitask training that combines segmental and frame-level losses.
Directory of Open Access Journals (Sweden)
Elaheh Mesdaghinia
2017-12-01
Full Text Available Background: Recurrent spontaneous abortion has high incidence rate. The etiology is unknown in 30-40%. However high uterine artery resistance is accounted as one of the recurrent abortion reasons. Objective: The objective of the current study was to determine the impacts of vitamin E and aspirin on the uterine artery blood flow in women having recurrent abortions due to impaired uterine blood flow. Materials and Methods: This randomized clinical trial was conducted on 99 women having uterine pulsatility index (PI more than 2.5 and the history of more than two times abortions. The candidates were categorized into three groups; receiving aspirin, only vitamin E, and aspirin+vitamin E. After 2 months, uterine PIs were compared with each other. Results: All drug regimens caused an enhancement in uterine perfusion with a significant decline in uterine artery PI value. The women receiving vitamin E in accompanied with aspirin had the least mean PI of the uterine artery (p<0.001. The total average PI score of the right and left uterine arteries in groups receiving vitamin E in accompanied with aspirin was lower than the two counterparts significantly (p<0.001. Conclusion: Vitamin E, aspirin and especially their combination are effective in improving uterine artery blood flow in women with recurrent abortion due to impaired uterine blood flow. More well-designed studies are needed to find out whether the enhancement of uterine perfusion may lead to a better pregnancy outcome.
Directory of Open Access Journals (Sweden)
Xue Han
Full Text Available The quest to determine how precise neural activity patterns mediate computation, behavior, and pathology would be greatly aided by a set of tools for reliably activating and inactivating genetically targeted neurons, in a temporally precise and rapidly reversible fashion. Having earlier adapted a light-activated cation channel, channelrhodopsin-2 (ChR2, for allowing neurons to be stimulated by blue light, we searched for a complementary tool that would enable optical neuronal inhibition, driven by light of a second color. Here we report that targeting the codon-optimized form of the light-driven chloride pump halorhodopsin from the archaebacterium Natronomas pharaonis (hereafter abbreviated Halo to genetically-specified neurons enables them to be silenced reliably, and reversibly, by millisecond-timescale pulses of yellow light. We show that trains of yellow and blue light pulses can drive high-fidelity sequences of hyperpolarizations and depolarizations in neurons simultaneously expressing yellow light-driven Halo and blue light-driven ChR2, allowing for the first time manipulations of neural synchrony without perturbation of other parameters such as spiking rates. The Halo/ChR2 system thus constitutes a powerful toolbox for multichannel photoinhibition and photostimulation of virally or transgenically targeted neural circuits without need for exogenous chemicals, enabling systematic analysis and engineering of the brain, and quantitative bioengineering of excitable cells.
International Nuclear Information System (INIS)
Sato, Shunsuke; Tanaka, Toshiaki; Takahashi, Atsushi; Sasai, Masamichi; Kitamura, Hiroshi; Masumori, Naoya; Tsukamoto, Taiji
2010-01-01
We retrospectively evaluated long-term oncological outcomes in patients with germ cell tumors (GCTs) primarily treated at our institution and assessed late recurrence and second primary malignancies. This study included a total of 139 males with newly diagnosed GCTs of the testis or extragonadal origin who received treatment, including surgery, chemotherapy and radiation therapy, at our hospital between 1980 and 2005. We reviewed late recurrence that occurred at least 2 years after the initial disease-free status and secondary malignancies as well as oncological outcomes. In patients with seminoma, 5-year progression-free survival and cause-specific survival rates were 87.2% and 100% for Stage I, 88.9% and 100% for Stage II, and 50.0% and 50.0% for Stage III, respectively, whereas in those with non-seminomatous GCTs, they were 79.1% and 96.3% for Stage I, 89.5% and 89.4% for Stage II, and 85.7% and 78.4% for Stage III, respectively. Late recurrence was found in five (3.6%) patients and all of them responded to salvage treatment and achieved disease-free status. Second primary hematological neoplasms occurred in three (2.2%), although they had a long-term free of the primary disease. All died of the second primary disease. Late recurrence was successfully managed with appropriate treatments, although its incidence was not negligible. Periodic follow-up may be necessary for >5 years in patients with GCTs for early detection of late recurrence. In addition, care should be taken to watch for the development of life-threatening second primary malignant disease during long-term follow-up. (author)
The Criticality Hypothesis in Neural Systems
Karimipanah, Yahya
in studying neuronal avalanches. Finally, we show in a computational model that two prevalent features of cortical single-neuron activity, irregular spiking and the decline of response variability at stimulus onset, both are emergent properties of a recurrent network operating near criticality. Our findings establish criticality as a unifying principle for the statistics of single-neuron spiking and the collective behavior of recurrent circuits in cerebral cortex. Moreover, as the observed decline in response variability is regarded as an essential mechanism to enhance response fidelity to stimuli, our discovery of its relation to network criticality offers a starting point toward unraveling the possible roles of critical dynamics in neural coding.
Wei, Jing; Tan, Yu-Ting; Cai, Yu-Cen; Yuan, Zhong-Yu; Yang, Dong; Wang, Shu-Sen; Peng, Rou-Jun; Teng, Xiao-Yu; Liu, Dong-Geng; Shi, Yan-Xia
2014-01-01
The local recurrence rate of phyllodes tumors of the breast varies widely among different subtypes, and distant metastasis is associated with poor survival. This study aimed to identify factors that are predictive of local recurrence-free survival (LRFS), distant metastasis-free survival (DMFS), and overall survival (OS) in patients with phyllodes tumors of the breast. Clinical data of all patients with a phyllodes tumor of the breast (n = 192) treated at Sun Yat-sen University Cancer Center between March 1997 and December 2012 were reviewed. The Pearson χ2 test was used to investigate the relationship between clinical features of patients and histotypes of tumors. Univariate and multivariate Cox regression analyses were performed to identify factors that are predictive of LRFS, DMFS, and OS. In total, 31 (16.1%) patients developed local recurrence, and 12 (6.3%) developed distant metastasis. For the patients who developed local recurrence, the median age at the diagnosis of primary tumor was 33 years (range, 17-56 years), and the median size of primary tumor was 6.0 cm (range, 0.8-18 cm). For patients who developed distant metastasis, the median age at the diagnosis of primary tumor was 46 years (range, 24-68 years), and the median size of primary tumor was 5.0 cm (range, 0.8-18 cm). In univariate analysis, age, size, hemorrhage, and margin status were found to be predictive factors for LRFS (P = 0.009, 0.024, 0.004, and 0.001, respectively), whereas histotype, epithelial hyperplasia, margin status, and local recurrence were predictors of DMFS (P = 0.001, 0.007, 0.007, and tumor size (HR = 2.668, P = 0.013), histotype (HR = 1.715, P = 0.017), and margin status (HR = 4.530, Ptumor size, a higher tumor grade, and positive margins were associated with lower rates of LRFS. Histotype and margin status were found to be independent predictors of DMFS and OS. PMID:25104281
Directory of Open Access Journals (Sweden)
Behniafar Ali
2013-01-01
Full Text Available The electric marine instruments are newly inserted in the trade and industry, for which the existence of an equipped and reliable power system is necessitated. One of the features of such a power system is that it cannot have an earth system causing the protection relays not to be able to detect the single line to ground short circuit fault. While on the other hand, the occurrence of another similar fault at the same time can lead to the double line fault and thereby the tripping of relays and shortening of vital loads. This in turn endangers the personals' security and causes the loss of military plans. From the above considerations, it is inferred that detecting the single line to ground fault in the marine instruments is of a special importance. In this way, this paper intends to detect the single line to ground fault in the power systems of the marine instruments using the wavelet transform and Multi-Layer Perceptron (MLP neural network. In the numerical analysis, several different types of short circuit faults are simulated on several marine power systems and the proposed approach is applied to detect the single line to ground fault. The results are of a high quality and preciseness and perfectly demonstrate the effectiveness of the proposed approach.
Predicting Human Behaviour with Recurrent Neural Networks
Directory of Open Access Journals (Sweden)
Aitor Almeida
2018-02-01
Full Text Available As the average age of the urban population increases, cities must adapt to improve the quality of life of their citizens. The City4Age H2020 project is working on the early detection of the risks related to mild cognitive impairment and frailty and on providing meaningful interventions that prevent these risks. As part of the risk detection process, we have developed a multilevel conceptual model that describes the user behaviour using actions, activities, and intra- and inter-activity behaviour. Using this conceptual model, we have created a deep learning architecture based on long short-term memory networks (LSTMs that models the inter-activity behaviour. The presented architecture offers a probabilistic model that allows us to predict the user’s next actions and to identify anomalous user behaviours.
Subgrouped Real Time Recurrent Learning Neural Networks
1994-05-01
long ixlWix,ix3; static float 4[98]; float temp; static mnt iff=O; intj ; void nremnor; if (*idum iff ==0) 1 ix1=0IC1-(*idum)) % MI; ixlr(IAI*ixl+IC1...long ixlW,iix3; static float r[98]; float tenip; static int iff=O; intj ; void DrerrorO; if (idum <Oj11iff=O ){ ixl=QICI-(*idum)) % Ml; ixlF(IA1*ixl
Ocean wave forecasting using recurrent neural networks
Digital Repository Service at National Institute of Oceanography (India)
Mandal, S.; Prabaharan, N.
not accurately represent the measured values. The parametric or differential equation based on wind wave relationship and a differential equation of wave energy are solved numerically in wave forecasting. This is generally employed to give an estimate over... to the biological neurons, works on the input and output passing through a hidden layer. The ANN used here is a data- oriented modeling technique to find relations between input and output patterns by self learning and without any fixed mathematical form assumed...
Koley, Ebha; Verma, Khushaboo; Ghosh, Subhojit
2015-01-01
Restrictions on right of way and increasing power demand has boosted development of six phase transmission. It offers a viable alternative for transmitting more power, without major modification in existing structure of three phase double circuit transmission system. Inspite of the advantages, low acceptance of six phase system is attributed to the unavailability of a proper protection scheme. The complexity arising from large number of possible faults in six phase lines makes the protection quite challenging. The proposed work presents a hybrid wavelet transform and modular artificial neural network based fault detector, classifier and locator for six phase lines using single end data only. The standard deviation of the approximate coefficients of voltage and current signals obtained using discrete wavelet transform are applied as input to the modular artificial neural network for fault classification and location. The proposed scheme has been tested for all 120 types of shunt faults with variation in location, fault resistance, fault inception angles. The variation in power system parameters viz. short circuit capacity of the source and its X/R ratio, voltage, frequency and CT saturation has also been investigated. The result confirms the effectiveness and reliability of the proposed protection scheme which makes it ideal for real time implementation.
Beije, N.; Kraan, J.; Taal, W.; van der Holt, B.; Oosterkamp, H. M.; Walenkamp, A. M.; Beerepoot, L.; Hanse, M.; van Linde, M. E.; Otten, A.; Vernhout, R. M.; de Vos, F. Y. F.; Gratama, J. W.; Sleijfer, S.; van den Bent, M. J.
2015-01-01
Background: Angiogenesis is crucial for glioblastoma growth, and anti-vascular endothelial growth factor agents are widely used in recurrent glioblastoma patients. The number of circulating endothelial cells (CECs) is a surrogate marker for endothelial damage. We assessed their kinetics and explored
Schuster, David M; Nieh, Peter T; Jani, Ashesh B; Amzat, Rianot; Bowman, F Dubois; Halkar, Raghuveer K; Master, Viraj A; Nye, Jonathon A; Odewole, Oluwaseun A; Osunkoya, Adeboye O; Savir-Baruch, Bital; Alaei-Taleghani, Pooneh; Goodman, Mark M
2014-05-01
We prospectively evaluated the amino acid analogue positron emission tomography radiotracer anti-3-[(18)F]FACBC compared to ProstaScint® ((111)In-capromab pendetide) single photon emission computerized tomography-computerized tomography to detect recurrent prostate carcinoma. A total of 93 patients met study inclusion criteria who underwent anti-3-[(18)F]FACBC positron emission tomography-computerized tomography plus (111)In-capromab pendetide single photon emission computerized tomography-computerized tomography for suspected recurrent prostate carcinoma within 90 days. Reference standards were applied by a multidisciplinary board. We calculated diagnostic performance for detecting disease. In the 91 of 93 patients with sufficient data for a consensus on the presence or absence of prostate/bed disease anti-3-[(18)F]FACBC had 90.2% sensitivity, 40.0% specificity, 73.6% accuracy, 75.3% positive predictive value and 66.7% negative predictive value compared to (111)In-capromab pendetide with 67.2%, 56.7%, 63.7%, 75.9% and 45.9%, respectively. In the 70 of 93 patients with a consensus on the presence or absence of extraprostatic disease anti-3-[(18)F]FACBC had 55.0% sensitivity, 96.7% specificity, 72.9% accuracy, 95.7% positive predictive value and 61.7% negative predictive value compared to (111)In-capromab pendetide with 10.0%, 86.7%, 42.9%, 50.0% and 41.9%, respectively. Of 77 index lesions used to prove positivity histological proof was obtained in 74 (96.1%). Anti-3-[(18)F]FACBC identified 14 more positive prostate bed recurrences (55 vs 41) and 18 more patients with extraprostatic involvement (22 vs 4). Anti-3-[(18)F]FACBC positron emission tomography-computerized tomography correctly up-staged 18 of 70 cases (25.7%) in which there was a consensus on the presence or absence of extraprostatic involvement. Better diagnostic performance was noted for anti-3-[(18)F]FACBC positron emission tomography-computerized tomography than for (111)In-capromab pendetide single
A loop-based neural architecture for structured behavior encoding and decoding.
Gisiger, Thomas; Boukadoum, Mounir
2018-02-01
We present a new type of artificial neural network that generalizes on anatomical and dynamical aspects of the mammal brain. Its main novelty lies in its topological structure which is built as an array of interacting elementary motifs shaped like loops. These loops come in various types and can implement functions such as gating, inhibitory or executive control, or encoding of task elements to name a few. Each loop features two sets of neurons and a control region, linked together by non-recurrent projections. The two neural sets do the bulk of the loop's computations while the control unit specifies the timing and the conditions under which the computations implemented by the loop are to be performed. By functionally linking many such loops together, a neural network is obtained that may perform complex cognitive computations. To demonstrate the potential offered by such a system, we present two neural network simulations. The first illustrates the structure and dynamics of a single loop implementing a simple gating mechanism. The second simulation shows how connecting four loops in series can produce neural activity patterns that are sufficient to pass a simplified delayed-response task. We also show that this network reproduces electrophysiological measurements gathered in various regions of the brain of monkeys performing similar tasks. We also demonstrate connections between this type of neural network and recurrent or long short-term memory network models, and suggest ways to generalize them for future artificial intelligence research. Copyright © 2017 Elsevier Ltd. All rights reserved.
Keller, James M; Fogel, David B
2016-01-01
This book covers the three fundamental topics that form the basis of computational intelligence: neural networks, fuzzy systems, and evolutionary computation. The text focuses on inspiration, design, theory, and practical aspects of implementing procedures to solve real-world problems. While other books in the three fields that comprise computational intelligence are written by specialists in one discipline, this book is co-written by current former Editor-in-Chief of IEEE Transactions on Neural Networks and Learning Systems, a former Editor-in-Chief of IEEE Transactions on Fuzzy Systems, and the founding Editor-in-Chief of IEEE Transactions on Evolutionary Computation. The coverage across the three topics is both uniform and consistent in style and notation. Discusses single-layer and multilayer neural networks, radial-basi function networks, and recurrent neural networks Covers fuzzy set theory, fuzzy relations, fuzzy logic interference, fuzzy clustering and classification, fuzzy measures and fuzz...
Miskowiak, Kamilla W; Kessing, Lars V; Ott, Caroline V; Macoveanu, Julian; Harmer, Catherine J; Jørgensen, Anders; Revsbech, Rasmus; Jensen, Hans M; Paulson, Olaf B; Siebner, Hartwig R; Jørgensen, Martin B
2017-09-01
Negative neurocognitive bias is a core feature of major depressive disorder that is reversed by pharmacological and psychological treatments. This double-blind functional magnetic resonance imaging study investigated for the first time whether electroconvulsive therapy modulates negative neurocognitive bias in major depressive disorder. Patients with major depressive disorder were randomised to one active ( n=15) or sham electroconvulsive therapy ( n=12). The following day they underwent whole-brain functional magnetic resonance imaging at 3T while viewing emotional faces and performed facial expression recognition and dot-probe tasks. A single electroconvulsive therapy session had no effect on amygdala response to emotional faces. Whole-brain analysis revealed no effects of electroconvulsive therapy versus sham therapy after family-wise error correction at the cluster level, using a cluster-forming threshold of Z>3.1 ( p2.3; pelectroconvulsive therapy-induced changes in parahippocampal and superior frontal responses to fearful versus happy faces as well as in fear-specific functional connectivity between amygdala and occipito-temporal regions. Across all patients, greater fear-specific amygdala - occipital coupling correlated with lower fear vigilance. Despite no statistically significant shift in neural response to faces after a single electroconvulsive therapy session, the observed trend changes after a single electroconvulsive therapy session point to an early shift in emotional processing that may contribute to antidepressant effects of electroconvulsive therapy.
Contemporary deep recurrent learning for recognition
Iftekharuddin, K. M.; Alam, M.; Vidyaratne, L.
2017-05-01
Large-scale feed-forward neural networks have seen intense application in many computer vision problems. However, these networks can get hefty and computationally intensive with increasing complexity of the task. Our work, for the first time in literature, introduces a Cellular Simultaneous Recurrent Network (CSRN) based hierarchical neural network for object detection. CSRN has shown to be more effective to solving complex tasks such as maze traversal and image processing when compared to generic feed forward networks. While deep neural networks (DNN) have exhibited excellent performance in object detection and recognition, such hierarchical structure has largely been absent in neural networks with recurrency. Further, our work introduces deep hierarchy in SRN for object recognition. The simultaneous recurrency results in an unfolding effect of the SRN through time, potentially enabling the design of an arbitrarily deep network. This paper shows experiments using face, facial expression and character recognition tasks using novel deep recurrent model and compares recognition performance with that of generic deep feed forward model. Finally, we demonstrate the flexibility of incorporating our proposed deep SRN based recognition framework in a humanoid robotic platform called NAO.
Kuo, Yung; Park, Kyoungwon; Li, Jack; Ingargiola, Antonino; Park, Joonhyuck; Shvadchak, Volodymyr; Weiss, Shimon
2017-08-01
Monitoring membrane potential in neurons requires sensors with minimal invasiveness, high spatial and temporal (sub-ms) resolution, and large sensitivity for enabling detection of sub-threshold activities. While organic dyes and fluorescent proteins have been developed to possess voltage-sensing properties, photobleaching, cytotoxicity, low sensitivity, and low spatial resolution have obstructed further studies. Semiconductor nanoparticles (NPs), as prospective voltage sensors, have shown excellent sensitivity based on Quantum confined Stark effect (QCSE) at room temperature and at single particle level. Both theory and experiment have shown their voltage sensitivity can be increased significantly via material, bandgap, and structural engineering. Based on theoretical calculations, we synthesized one of the optimal candidates for voltage sensors: 12 nm type-II ZnSe/CdS nanorods (NRs), with an asymmetrically located seed. The voltage sensitivity and spectral shift were characterized in vitro using spectrally-resolved microscopy using electrodes grown by thin film deposition, which "sandwich" the NRs. We characterized multiple batches of such NRs and iteratively modified the synthesis to achieve higher voltage sensitivity (ΔF/F> 10%), larger spectral shift (>5 nm), better homogeneity, and better colloidal stability. Using a high throughput screening method, we were able to compare the voltage sensitivity of our NRs with commercial spherical quantum dots (QDs) with single particle statistics. Our method of high throughput screening with spectrally-resolved microscope also provides a versatile tool for studying single particles spectroscopy under field modulation.
Tendoscopic treatment of recurrent peroneal tendon dislocation
Scholten, Peter E.; Breugem, Stefan J. M.; van Dijk, C. Niek
2013-01-01
To study the possibility of tendoscopic treatment of recurrent peroneal tendon dislocation. The case of one patient is described including the tendoscopic technique to deepen the fibular groove. In this single case, there were no complications, recovery time was short, and there was no recurrence of
Directory of Open Access Journals (Sweden)
José Huygens Parente GARCIA
2015-09-01
Full Text Available BackgroundTreatment of hepatitis C virus infection in post-transplantation patients is a challenge due to poor tolerance and low success rates.ObjectiveTo determine the response rate to pegylated interferon and ribavirin in post-liver transplant patients with hepatitis C recurrence.MethodsBetween 18 May 2002 and 18 December 2011, 601 patients underwent liver transplantation at our service (Hospital Universitário Walter Cantídio, University of Ceará, 176 (29.2% of whom were hepatitis C virus positive. Forty received antiviral therapy and were included in this cohort study. Twenty-eight (70% completed the treatment protocol, which consisted of pegylated interferon and ribavirin for 48 weeks.ResultsThe sustained virological response rate was 55% according to intention-to-treat analysis. Recipient age and exposure to antiviral drugs prior to liver transplantation were associated with sustained virological response in the multivariate analysis. Patients were followed for 57 months on the average. Survival at 1 and 5 years was 100% in responders, versus 100% and 78%, respectively, in non-responders.ConclusionSustained virological response rates were satisfactory in our series of liver transplantation patients, and decreased with increasing recipient age. Non-exposure to antiviral drugs prior to liver transplantation was positively associated with sustained virological response. The overall survival of responders and non-responders was similar.
Oka, Mizuki; Nakaaki, Shutaro; Negi, Atsushi; Miyata, Jun; Nakagawa, Atsuo; Hirono, Nobutsugu; Mimura, Masaru
2016-03-01
A number of neuroimaging studies have addressed the specific effect of treatment with cholinesterase inhibitors on the frontal lobe in patients with Alzheimer's disease (AD). However, the neural effects of cholinesterase inhibitors on both apathy and executive dysfunction remain unclear. We examined whether baseline regional cerebral blood flow, as determined by using single-photon emission computed tomography, is capable of predicting changes in apathy and executive dysfunction in response to AD patients switching from donepezil to galantamine therapy. We conducted a 24-week, prospective, open-label study of AD patients treated with galantamine who did not respond to previous treatment with donepezil. Single-photon emission computed tomography was performed at baseline, and behaviour and cognitive assessments including the Mini-Mental State Examination, the Japanese version of the Alzheimer's Disease Assessment Scale-cognitive subscale, the Frontal Assessment Battery, the Neuropsychiatry Inventory Brief Questionnaire Form, and the Dysexecutive Questionnaire were conducted at three time points (baseline and after 12 and 24 weeks of galantamine therapy). After galantamine therapy, the Neuropsychiatry Inventory Brief Questionnaire Form scores (apathy, irritability, and aberrant motor symptoms) and the Dysexecutive Questionnaire score improved significantly. The single-photon emission computed tomography findings showed that lower baseline regional cerebral blood flow values in several frontal areas, including the dorsolateral and ventrolateral prefrontal cortex, the anterior cingulate, and the orbitofrontal cortex, predicted greater reductions in the score for apathy (distress) on the Neuropsychiatry Inventory Brief Questionnaire Form and the Dysexecutive Questionnaire score after patients switched from donepezil to galantamine therapy. Our study suggests that galantamine therapy, unlike donepezil, is characterized by a dual mechanism of action that may increase
Energy Technology Data Exchange (ETDEWEB)
Costa, Ederson D' Martin; Lemes, Nelson Henrique Teixeira, E-mail: nelson.lemes@unifal-mg.edu.br [Instituto de Ciencias Exatas, Universidade Federal de Alfenas, Alfenas, MG (Brazil); Santos, Marcelo Henrique dos [Instituto de Ciencias Farmaceuticas, Universidade Federal de Alfenas, Alfenas, MG (Brazil); Braga, Joao Pedro [Departamento de Quimica, Universidade Federal de Minas Gerais (UFMG), Belo Horizonte, MG (Brazil)
2012-07-01
This work propose a recursive neural network to solve inverse equilibrium problem. The acidity constants of 7-epiclusianone in ethanol-water binary mixtures were determined from multiwavelength spectrophotometric data. A linear relationship between acidity constants and the % w/v of ethanol in the solvent mixture was observed. The proposed method efficiency is compared with the Simplex method, commonly used in nonlinear optimization techniques. The neural network method is simple, numerically stable and has a broad range of applicability. (author)
International Nuclear Information System (INIS)
Ramackers, J.M.; Kotzki, P.O.; Couret, I.; Messner-Pellenc, P.; Davy, J.M.; Rossi, M.
1995-01-01
In this case report we present a patient with a recurrence of subacute bacterial infectious endocarditis (IE) complicating a transvenous endocardial pacemaker. Technetium-99m hexamethylpropylene amine oxime ( 99m Tc-HMPAO) labelled granulocytes were used for diagnosis and follow-up under medical treatment only, since surgical removal of the pacemaker lead was ruled out because of the general condition of the patient. Single-photon emission tomography (SPET) imaging displayed the active lesion previously suspected on echography. At the end of antibiotic therapy, SPET indicated a favourable disease outcome whereas echocardiographic abnormalities remained nearly unchanged. The medical treatment had eradicated the IE, and the patient did well for more than 1 year thereafter. (orig.)
Energy Technology Data Exchange (ETDEWEB)
Ramackers, J.M. [Department of Nuclear Medicine, CHU E. Herriot, Lyon (France); Kotzki, P.O. [Department of Nuclear Medicine, CHU Lapeyronie et A. de Villeneuve, Montpellier (France); Couret, I. [Department of Nuclear Medicine, CHU Lapeyronie et A. de Villeneuve, Montpellier (France); Messner-Pellenc, P. [Department of Cardiology, CHU Lapeyronie et A. Villeneuve, Montpellier (France); Davy, J.M. [Department of Cardiology, CHU Lapeyronie et A. Villeneuve, Montpellier (France); Rossi, M. [Department of Nuclear Medicine, CHU Lapeyronie et A. de Villeneuve, Montpellier (France)
1995-11-01
In this case report we present a patient with a recurrence of subacute bacterial infectious endocarditis (IE) complicating a transvenous endocardial pacemaker. Technetium-99m hexamethylpropylene amine oxime ({sup 99m}Tc-HMPAO) labelled granulocytes were used for diagnosis and follow-up under medical treatment only, since surgical removal of the pacemaker lead was ruled out because of the general condition of the patient. Single-photon emission tomography (SPET) imaging displayed the active lesion previously suspected on echography. At the end of antibiotic therapy, SPET indicated a favourable disease outcome whereas echocardiographic abnormalities remained nearly unchanged. The medical treatment had eradicated the IE, and the patient did well for more than 1 year thereafter. (orig.)
International Nuclear Information System (INIS)
Hegemann, Nina-Sophie; Morcinek, Sebastian; Corradini, Stefanie; Li, Minglun; Belka, Claus; Ganswindt, Ute; Buchner, Alexander; Karl, Alexander; Stief, Christian; Knuechel, Ruth
2016-01-01
Despite improved biochemical recurrence-free survival rates by the use of immediate adjuvant radiotherapy (RT) in patients with locally advanced prostate cancer, disagreement about the need and timing of RT remains. From 2005-2009, 94 patients presenting with a stage pT3a N0 and microscopic positive resection margin were retrospectively analyzed after radical prostatectomy. Special attention was given to patients' outcome, the frequency of additive RT, and its efficacy. Median follow-up was 80 months. A total of 71 patients had a negative postoperative prostate-specific antigen (PSA) level (<0.07 ng/ml). Thirty-six of them did not experience any PSA relapse (subgroup 1). Fourteen of them received additive RT and during follow-up all 36 patients remained PSA negative. Of 71 initially PSA-negative patients, 35 had a biochemical relapse (subgroup 2); 28 patients underwent salvage RT. The median PSA value before salvage RT was 0.24 ng/ml and was subsequently negative (<0.07 ng/ml) in 23 patients after RT. Of the entire cohort, 23 patients had persisting PSA after surgery (subgroup 3). Of these, 18 patients received salvage RT at a median PSA level of 0.4 ng/ml. One patient in subgroup 1, 5 patients in subgroup 2, and 9 patients in subgroup 3 had ongoing androgen deprivation therapy. The present study of 94 pT3a N0 R1 prostate cancer patients provides insight into medical care of this patient cohort and underlines the need for additive RT for the majority of patients to achieve long-term biochemical control. Although immediate adjuvant RT was applied with restraint (20 %), during the observation period 60 of 94 patients (63.8 %) received RT - highlighting the need of postoperative treatment. (orig.) [de
Mitchell, Deanna; Bergendahl, Genevieve; Ferguson, William; Roberts, William; Higgins, Timothy; Ashikaga, Takamaru; DeSarno, Mike; Kaplan, Joel; Kraveka, Jacqueline; Eslin, Don; Werff, Alyssa Vander; Hanna, Gina K; Sholler, Giselle L Saulnier
2016-01-01
The primary aim of this Phase I study was to determine the maximum tolerated dose (MTD) of TPI 287 and the safety and tolerability of TPI 287 alone and in combination with temozolomide (TMZ) in pediatric patients with refractory or recurrent neuroblastoma or medulloblastoma. The secondary aims were to evaluate the pharmacokinetics of TPI 287 and the treatment responses. Eighteen patients were enrolled to a phase I dose escalation trial of weekly intravenous infusion of TPI 287 for two 28-day cycles with toxicity monitoring to determine the MTD, followed by two cycles of TPI 287 in combination with TMZ. Samples were collected to determine the pharmacokinetic parameters C(max), AUC(0-24), t(1/2), CL, and Vd on day 1 of cycles 1 (TPI 287 alone) and 3 (TPI 287 + TMZ) following TPI 287 infusion. Treatment response was evaluated by radiographic (CT or MRI) and radionuclide (MIBG) imaging for neuroblastoma. We determined the MTD of TPI 287 alone and in combination with temozolomide to be 125 mg/m(2). The non-dose-limiting toxicities at this dose were mainly anorexia and pain. The dose-limiting toxicities (DLTs) of two patients at 135 mg/m(2) were grade 3 hemorrhagic cystitis and grade 3 sensory neuropathy. Overall, TPI 287 was well tolerated by pediatric patients with refractory and relapsed neuroblastoma and medulloblastoma at a dose of 125 mg/m(2) IV on days 1, 8, and 15 of a 28 day cycle. © 2015 Wiley Periodicals, Inc.
International Nuclear Information System (INIS)
Dawson, M.R.W.; Dobbs, A.; Hooper, H.R.; McEwan, A.J.B.; Triscott, J.; Cooney, J.
1994-01-01
Single-photon emission tomographic (SPET) images using technetium-99m labelled hexamethylpropylene amine oxime were obtained from 97 patients diagnosed as having Alzheimer's disease, as well as from a comparison group of 64 normal subjects. Multiple linear regression was used to predict subject type (Alzheimer's vs comparison) using scintillation counts from 14 different brain regions as predictors. These results were disappointing: the regression equation accounted for only 33.5% of the variance between subjects. However, the same data were also used to train parallel distributed processing (PDP) networks of different sizes to classify subjects. In general, the PDP networks accounted for substantially more (up to 95%) of the variance in the data, and in many instances were able to distinguish perfectly between the two subjects. These results suggest two conclusions. First, SPET images do provide sufficient information to distinguish patients with Alzheimer's disease from a normal comparison group. Second, to access this diagnostic information, it appears that one must take advantage of the ability of PDP networks to detect higher-order nonlinear relationships among the predictor variables. (orig.)
Factors that influence recurrent lumbar disc herniation.
Yaman, M E; Kazancı, A; Yaman, N D; Baş, F; Ayberk, G
2017-06-01
The most common cause of poor outcome following lumbar disc surgery is recurrent herniation. Recurrence has been noted in 5% to 15% of patients with surgically treated primary lumbar disc herniation. There have been many studies designed to determine the risk factors for recurrent lumbar disc herniation. In this study, we retrospectively analysed the influence of disc degeneration, endplate changes, surgical technique, and patient's clinical characteristics on recurrent lumbar disc herniation. Patients who underwent primary single-level L4-L5 lumbar discectomy and who were reoperated on for recurrent L4-L5 disc herniation were retrospectively reviewed. All these operations were performed between August 2004 and September 2009 at the Neurosurgery Department of Ataturk Education and Research Hospital in Ankara, Turkey. During the study period, 126 patients were reviewed, with 101 patients underwent primary single-level L4-L5 lumbar discectomy and 25 patients were reoperated on for recurrent L4-L5 disc herniation. Preoperative higher intervertebral disc height (Pdisc herniation had preoperative higher disc height and higher body mass index. Modic endplate changes had a higher tendency for recurrence of lumbar disc herniation. Well-planned and well-conducted large-scale prospective cohort studies are needed to confirm this and enable convenient treatment modalities to prevent recurrent disc pathology.
Bodzin, Adam S; Lunsford, Keri E; Markovic, Daniela; Harlander-Locke, Michael P; Busuttil, Ronald W; Agopian, Vatche G
2017-07-01
To evaluate predictors of mortality and impact of treatment in patients developing recurrent hepatocellular carcinoma (HCC) following liver transplantation (LT). Despite well-described clinicopathologic predictors of posttransplant HCC recurrence, data on prognosis following recurrence are scarce. Multivariate predictors of mortality following HCC recurrence were identified to develop a risk score model to stratify prognostic subgroups among 106 patients developing posttransplant recurrence from 1984 to 2014, including analysis of recurrence treatment modality on survival. Of 857 patients undergoing LT, 106 (12.4%) developed posttransplant HCC recurrence (median 15.8 months following LT) with a median post-recurrence survival of 10.6 months. Patients receiving surgical therapy (n = 25) had a median survival of 27.8 months, significantly superior to patients receiving nonsurgical therapy (10.6 months) and best supportive care (3.7 months, P 23, time to recurrence, >3 recurrent nodules, maximum recurrence size, bone recurrence, alphafetoprotein at recurrence, donor serum sodium, and pretransplant recipient neutrophil-lymphocyte ratio. A risk score model based on multivariate predictors accurately stratified recurrent HCC patients into prognostic subgroups, with low-risk patients (16 points) groups (C-statistic 0.75, P < 0.001). In the largest single-center report of recurrent HCC following LT, surgical treatment in well-selected patients is associated with significantly improved survival and should be pursued. A risk score model accurately stratifies prognostic subgroups, and may help guide treatment strategies.
Recurrent Intracerebral Hemorrhage
DEFF Research Database (Denmark)
Schmidt, Linnea Boegeskov; Goertz, Sanne; Wohlfahrt, Jan
2016-01-01
BACKGROUND: Intracerebral hemorrhage (ICH) is a disease with high mortality and a substantial risk of recurrence. However, the recurrence risk is poorly documented and the knowledge of potential predictors for recurrence among co-morbidities and medicine with antithrombotic effect is limited....... OBJECTIVES: 1) To estimate the short- and long-term cumulative risks of recurrent intracerebral hemorrhage (ICH). 2) To investigate associations between typical comorbid diseases, surgical treatment, use of medicine with antithrombotic effects, including antithrombotic treatment (ATT), selective serotonin...