WorldWideScience

Sample records for lyapunov-based recurrent neural

  1. Diagonal recurrent neural network based adaptive control of nonlinear dynamical systems using lyapunov stability criterion.

    Science.gov (United States)

    Kumar, Rajesh; Srivastava, Smriti; Gupta, J R P

    2017-03-01

    In this paper adaptive control of nonlinear dynamical systems using diagonal recurrent neural network (DRNN) is proposed. The structure of DRNN is a modification of fully connected recurrent neural network (FCRNN). Presence of self-recurrent neurons in the hidden layer of DRNN gives it an ability to capture the dynamic behaviour of the nonlinear plant under consideration (to be controlled). To ensure stability, update rules are developed using lyapunov stability criterion. These rules are then used for adjusting the various parameters of DRNN. The responses of plants obtained with DRNN are compared with those obtained when multi-layer feed forward neural network (MLFFNN) is used as a controller. Also, in example 4, FCRNN is also investigated and compared with DRNN and MLFFNN. Robustness of the proposed control scheme is also tested against parameter variations and disturbance signals. Four simulation examples including one-link robotic manipulator and inverted pendulum are considered on which the proposed controller is applied. The results so obtained show the superiority of DRNN over MLFFNN as a controller. Copyright © 2017 ISA. Published by Elsevier Ltd. All rights reserved.

  2. A recurrent neural network based on projection operator for extended general variational inequalities.

    Science.gov (United States)

    Liu, Qingshan; Cao, Jinde

    2010-06-01

    Based on the projection operator, a recurrent neural network is proposed for solving extended general variational inequalities (EGVIs). Sufficient conditions are provided to ensure the global convergence of the proposed neural network based on Lyapunov methods. Compared with the existing neural networks for variational inequalities, the proposed neural network is a modified version of the general projection neural network existing in the literature and capable of solving the EGVI problems. In addition, simulation results on numerical examples show the effectiveness and performance of the proposed neural network.

  3. A novel recurrent neural network with finite-time convergence for linear programming.

    Science.gov (United States)

    Liu, Qingshan; Cao, Jinde; Chen, Guanrong

    2010-11-01

    In this letter, a novel recurrent neural network based on the gradient method is proposed for solving linear programming problems. Finite-time convergence of the proposed neural network is proved by using the Lyapunov method. Compared with the existing neural networks for linear programming, the proposed neural network is globally convergent to exact optimal solutions in finite time, which is remarkable and rare in the literature of neural networks for optimization. Some numerical examples are given to show the effectiveness and excellent performance of the new recurrent neural network.

  4. Delay-Dependent Stability Criteria of Uncertain Periodic Switched Recurrent Neural Networks with Time-Varying Delays

    Directory of Open Access Journals (Sweden)

    Xing Yin

    2011-01-01

    uncertain periodic switched recurrent neural networks with time-varying delays. When uncertain discrete-time recurrent neural network is a periodic system, it is expressed as switched neural network for the finite switching state. Based on the switched quadratic Lyapunov functional approach (SQLF and free-weighting matrix approach (FWM, some linear matrix inequality criteria are found to guarantee the delay-dependent asymptotical stability of these systems. Two examples illustrate the exactness of the proposed criteria.

  5. Nonlinear Model Predictive Control Based on a Self-Organizing Recurrent Neural Network.

    Science.gov (United States)

    Han, Hong-Gui; Zhang, Lu; Hou, Ying; Qiao, Jun-Fei

    2016-02-01

    A nonlinear model predictive control (NMPC) scheme is developed in this paper based on a self-organizing recurrent radial basis function (SR-RBF) neural network, whose structure and parameters are adjusted concurrently in the training process. The proposed SR-RBF neural network is represented in a general nonlinear form for predicting the future dynamic behaviors of nonlinear systems. To improve the modeling accuracy, a spiking-based growing and pruning algorithm and an adaptive learning algorithm are developed to tune the structure and parameters of the SR-RBF neural network, respectively. Meanwhile, for the control problem, an improved gradient method is utilized for the solution of the optimization problem in NMPC. The stability of the resulting control system is proved based on the Lyapunov stability theory. Finally, the proposed SR-RBF neural network-based NMPC (SR-RBF-NMPC) is used to control the dissolved oxygen (DO) concentration in a wastewater treatment process (WWTP). Comparisons with other existing methods demonstrate that the SR-RBF-NMPC can achieve a considerably better model fitting for WWTP and a better control performance for DO concentration.

  6. Nonlinear recurrent neural networks for finite-time solution of general time-varying linear matrix equations.

    Science.gov (United States)

    Xiao, Lin; Liao, Bolin; Li, Shuai; Chen, Ke

    2018-02-01

    In order to solve general time-varying linear matrix equations (LMEs) more efficiently, this paper proposes two nonlinear recurrent neural networks based on two nonlinear activation functions. According to Lyapunov theory, such two nonlinear recurrent neural networks are proved to be convergent within finite-time. Besides, by solving differential equation, the upper bounds of the finite convergence time are determined analytically. Compared with existing recurrent neural networks, the proposed two nonlinear recurrent neural networks have a better convergence property (i.e., the upper bound is lower), and thus the accurate solutions of general time-varying LMEs can be obtained with less time. At last, various different situations have been considered by setting different coefficient matrices of general time-varying LMEs and a great variety of computer simulations (including the application to robot manipulators) have been conducted to validate the better finite-time convergence of the proposed two nonlinear recurrent neural networks. Copyright © 2017 Elsevier Ltd. All rights reserved.

  7. Evaluating Lyapunov exponent spectra with neural networks

    International Nuclear Information System (INIS)

    Maus, A.; Sprott, J.C.

    2013-01-01

    Highlights: • Cross-correlation is employed to remove spurious Lyapunov exponents from a spectrum. • Neural networks are shown to accurately model Lyapunov exponent spectra. • Neural networks compare favorably to local linear fits in modeling Lyapunov exponents. • Numerical experiments are performed with time series of varying length and noise. • Methods perform reasonably well on discrete time series. -- Abstract: A method using discrete cross-correlation for identifying and removing spurious Lyapunov exponents when embedding experimental data in a dimension greater than the original system is introduced. The method uses a distribution of calculated exponent values produced by modeling a single time series many times or multiple instances of a time series. For this task, global models are shown to compare favorably to local models traditionally used for time series taken from the Hénon map and delayed Hénon map, especially when the time series are short or contaminated by noise. An additional merit of global modeling is its ability to estimate the dynamical and geometrical properties of the original system such as the attractor dimension, entropy, and lag space, although consideration must be taken for the time it takes to train the global models

  8. Lyapunov Functions to Caputo Fractional Neural Networks with Time-Varying Delays

    Directory of Open Access Journals (Sweden)

    Ravi Agarwal

    2018-05-01

    Full Text Available One of the main properties of solutions of nonlinear Caputo fractional neural networks is stability and often the direct Lyapunov method is used to study stability properties (usually these Lyapunov functions do not depend on the time variable. In connection with the Lyapunov fractional method we present a brief overview of the most popular fractional order derivatives of Lyapunov functions among Caputo fractional delay differential equations. These derivatives are applied to various types of neural networks with variable coefficients and time-varying delays. We show that quadratic Lyapunov functions and their Caputo fractional derivatives are not applicable in some cases when one studies stability properties. Some sufficient conditions for stability of equilibrium of nonlinear Caputo fractional neural networks with time dependent transmission delays, time varying self-regulating parameters of all units and time varying functions of the connection between two neurons in the network are obtained. The cases of time varying Lipschitz coefficients as well as nonLipschitz activation functions are studied. We illustrate our theory on particular nonlinear Caputo fractional neural networks.

  9. A recurrent neural network for solving bilevel linear programming problem.

    Science.gov (United States)

    He, Xing; Li, Chuandong; Huang, Tingwen; Li, Chaojie; Huang, Junjian

    2014-04-01

    In this brief, based on the method of penalty functions, a recurrent neural network (NN) modeled by means of a differential inclusion is proposed for solving the bilevel linear programming problem (BLPP). Compared with the existing NNs for BLPP, the model has the least number of state variables and simple structure. Using nonsmooth analysis, the theory of differential inclusions, and Lyapunov-like method, the equilibrium point sequence of the proposed NNs can approximately converge to an optimal solution of BLPP under certain conditions. Finally, the numerical simulations of a supply chain distribution model have shown excellent performance of the proposed recurrent NNs.

  10. Synchronization of chaotic recurrent neural networks with time-varying delays using nonlinear feedback control

    International Nuclear Information System (INIS)

    Cui Baotong; Lou Xuyang

    2009-01-01

    In this paper, a new method to synchronize two identical chaotic recurrent neural networks is proposed. Using the drive-response concept, a nonlinear feedback control law is derived to achieve the state synchronization of the two identical chaotic neural networks. Furthermore, based on the Lyapunov method, a delay independent sufficient synchronization condition in terms of linear matrix inequality (LMI) is obtained. A numerical example with graphical illustrations is given to illuminate the presented synchronization scheme

  11. Global robust stability of delayed recurrent neural networks

    International Nuclear Information System (INIS)

    Cao Jinde; Huang Deshuang; Qu Yuzhong

    2005-01-01

    This paper is concerned with the global robust stability of a class of delayed interval recurrent neural networks which contain time-invariant uncertain parameters whose values are unknown but bounded in given compact sets. A new sufficient condition is presented for the existence, uniqueness, and global robust stability of equilibria for interval neural networks with time delays by constructing Lyapunov functional and using matrix-norm inequality. An error is corrected in an earlier publication, and an example is given to show the effectiveness of the obtained results

  12. Low-dimensional recurrent neural network-based Kalman filter for speech enhancement.

    Science.gov (United States)

    Xia, Youshen; Wang, Jun

    2015-07-01

    This paper proposes a new recurrent neural network-based Kalman filter for speech enhancement, based on a noise-constrained least squares estimate. The parameters of speech signal modeled as autoregressive process are first estimated by using the proposed recurrent neural network and the speech signal is then recovered from Kalman filtering. The proposed recurrent neural network is globally asymptomatically stable to the noise-constrained estimate. Because the noise-constrained estimate has a robust performance against non-Gaussian noise, the proposed recurrent neural network-based speech enhancement algorithm can minimize the estimation error of Kalman filter parameters in non-Gaussian noise. Furthermore, having a low-dimensional model feature, the proposed neural network-based speech enhancement algorithm has a much faster speed than two existing recurrent neural networks-based speech enhancement algorithms. Simulation results show that the proposed recurrent neural network-based speech enhancement algorithm can produce a good performance with fast computation and noise reduction. Copyright © 2015 Elsevier Ltd. All rights reserved.

  13. A novel recurrent neural network with one neuron and finite-time convergence for k-winners-take-all operation.

    Science.gov (United States)

    Liu, Qingshan; Dang, Chuangyin; Cao, Jinde

    2010-07-01

    In this paper, based on a one-neuron recurrent neural network, a novel k-winners-take-all ( k -WTA) network is proposed. Finite time convergence of the proposed neural network is proved using the Lyapunov method. The k-WTA operation is first converted equivalently into a linear programming problem. Then, a one-neuron recurrent neural network is proposed to get the kth or (k+1)th largest inputs of the k-WTA problem. Furthermore, a k-WTA network is designed based on the proposed neural network to perform the k-WTA operation. Compared with the existing k-WTA networks, the proposed network has simple structure and finite time convergence. In addition, simulation results on numerical examples show the effectiveness and performance of the proposed k-WTA network.

  14. Complex Dynamical Network Control for Trajectory Tracking Using Delayed Recurrent Neural Networks

    Directory of Open Access Journals (Sweden)

    Jose P. Perez

    2014-01-01

    Full Text Available In this paper, the problem of trajectory tracking is studied. Based on the V-stability and Lyapunov theory, a control law that achieves the global asymptotic stability of the tracking error between a delayed recurrent neural network and a complex dynamical network is obtained. To illustrate the analytic results, we present a tracking simulation of a dynamical network with each node being just one Lorenz’s dynamical system and three identical Chen’s dynamical systems.

  15. An Attractor-Based Complexity Measurement for Boolean Recurrent Neural Networks

    Science.gov (United States)

    Cabessa, Jérémie; Villa, Alessandro E. P.

    2014-01-01

    We provide a novel refined attractor-based complexity measurement for Boolean recurrent neural networks that represents an assessment of their computational power in terms of the significance of their attractor dynamics. This complexity measurement is achieved by first proving a computational equivalence between Boolean recurrent neural networks and some specific class of -automata, and then translating the most refined classification of -automata to the Boolean neural network context. As a result, a hierarchical classification of Boolean neural networks based on their attractive dynamics is obtained, thus providing a novel refined attractor-based complexity measurement for Boolean recurrent neural networks. These results provide new theoretical insights to the computational and dynamical capabilities of neural networks according to their attractive potentialities. An application of our findings is illustrated by the analysis of the dynamics of a simplified model of the basal ganglia-thalamocortical network simulated by a Boolean recurrent neural network. This example shows the significance of measuring network complexity, and how our results bear new founding elements for the understanding of the complexity of real brain circuits. PMID:24727866

  16. Chaotic diagonal recurrent neural network

    International Nuclear Information System (INIS)

    Wang Xing-Yuan; Zhang Yi

    2012-01-01

    We propose a novel neural network based on a diagonal recurrent neural network and chaos, and its structure and learning algorithm are designed. The multilayer feedforward neural network, diagonal recurrent neural network, and chaotic diagonal recurrent neural network are used to approach the cubic symmetry map. The simulation results show that the approximation capability of the chaotic diagonal recurrent neural network is better than the other two neural networks. (interdisciplinary physics and related areas of science and technology)

  17. Boundedness and stability for recurrent neural networks with variable coefficients and time-varying delays

    International Nuclear Information System (INIS)

    Liang Jinling; Cao Jinde

    2003-01-01

    In this Letter, the problems of boundedness and stability for a general class of non-autonomous recurrent neural networks with variable coefficients and time-varying delays are analyzed via employing Young inequality technique and Lyapunov method. Some simple sufficient conditions are given for boundedness and stability of the solutions for the recurrent neural networks. These results generalize and improve the previous works, and they are easy to check and apply in practice. Two illustrative examples and their numerical simulations are also given to demonstrate the effectiveness of the proposed results

  18. Global exponential stability for reaction-diffusion recurrent neural networks with multiple time varying delays

    International Nuclear Information System (INIS)

    Lou, X.; Cui, B.

    2008-01-01

    In this paper we consider the problem of exponential stability for recurrent neural networks with multiple time varying delays and reaction-diffusion terms. The activation functions are supposed to be bounded and globally Lipschitz continuous. By means of Lyapunov functional, sufficient conditions are derived, which guarantee global exponential stability of the delayed neural network. Finally, a numerical example is given to show the correctness of our analysis. (author)

  19. Global exponential stability and periodicity of reaction-diffusion delayed recurrent neural networks with Dirichlet boundary conditions

    International Nuclear Information System (INIS)

    Lu Junguo

    2008-01-01

    In this paper, the global exponential stability and periodicity for a class of reaction-diffusion delayed recurrent neural networks with Dirichlet boundary conditions are addressed by constructing suitable Lyapunov functionals and utilizing some inequality techniques. We first prove global exponential converge to 0 of the difference between any two solutions of the original reaction-diffusion delayed recurrent neural networks with Dirichlet boundary conditions, the existence and uniqueness of equilibrium is the direct results of this procedure. This approach is different from the usually used one where the existence, uniqueness of equilibrium and stability are proved in two separate steps. Furthermore, we prove periodicity of the reaction-diffusion delayed recurrent neural networks with Dirichlet boundary conditions. Sufficient conditions ensuring the global exponential stability and the existence of periodic oscillatory solutions for the reaction-diffusion delayed recurrent neural networks with Dirichlet boundary conditions are given. These conditions are easy to check and have important leading significance in the design and application of reaction-diffusion recurrent neural networks with delays. Finally, two numerical examples are given to show the effectiveness of the obtained results

  20. Indirect adaptive fuzzy wavelet neural network with self- recurrent consequent part for AC servo system.

    Science.gov (United States)

    Hou, Runmin; Wang, Li; Gao, Qiang; Hou, Yuanglong; Wang, Chao

    2017-09-01

    This paper proposes a novel indirect adaptive fuzzy wavelet neural network (IAFWNN) to control the nonlinearity, wide variations in loads, time-variation and uncertain disturbance of the ac servo system. In the proposed approach, the self-recurrent wavelet neural network (SRWNN) is employed to construct an adaptive self-recurrent consequent part for each fuzzy rule of TSK fuzzy model. For the IAFWNN controller, the online learning algorithm is based on back propagation (BP) algorithm. Moreover, an improved particle swarm optimization (IPSO) is used to adapt the learning rate. The aid of an adaptive SRWNN identifier offers the real-time gradient information to the adaptive fuzzy wavelet neural controller to overcome the impact of parameter variations, load disturbances and other uncertainties effectively, and has a good dynamic. The asymptotical stability of the system is guaranteed by using the Lyapunov method. The result of the simulation and the prototype test prove that the proposed are effective and suitable. Copyright © 2017. Published by Elsevier Ltd.

  1. Delay-slope-dependent stability results of recurrent neural networks.

    Science.gov (United States)

    Li, Tao; Zheng, Wei Xing; Lin, Chong

    2011-12-01

    By using the fact that the neuron activation functions are sector bounded and nondecreasing, this brief presents a new method, named the delay-slope-dependent method, for stability analysis of a class of recurrent neural networks with time-varying delays. This method includes more information on the slope of neuron activation functions and fewer matrix variables in the constructed Lyapunov-Krasovskii functional. Then some improved delay-dependent stability criteria with less computational burden and conservatism are obtained. Numerical examples are given to illustrate the effectiveness and the benefits of the proposed method.

  2. A recurrent neural network for adaptive beamforming and array correction.

    Science.gov (United States)

    Che, Hangjun; Li, Chuandong; He, Xing; Huang, Tingwen

    2016-08-01

    In this paper, a recurrent neural network (RNN) is proposed for solving adaptive beamforming problem. In order to minimize sidelobe interference, the problem is described as a convex optimization problem based on linear array model. RNN is designed to optimize system's weight values in the feasible region which is derived from arrays' state and plane wave's information. The new algorithm is proven to be stable and converge to optimal solution in the sense of Lyapunov. So as to verify new algorithm's performance, we apply it to beamforming under array mismatch situation. Comparing with other optimization algorithms, simulations suggest that RNN has strong ability to search for exact solutions under the condition of large scale constraints. Copyright © 2016 Elsevier Ltd. All rights reserved.

  3. Some new results for recurrent neural networks with varying-time coefficients and delays

    International Nuclear Information System (INIS)

    Jiang Haijun; Teng Zhidong

    2005-01-01

    In this Letter, we consider the recurrent neural networks with varying-time coefficients and delays. By constructing new Lyapunov functional, introducing ingeniously many real parameters and applying the technique of Young inequality, we establish a series of criteria on the boundedness, global exponential stability and the existence of periodic solutions. In these criteria, we do not require that the response functions are differentiable, bounded and monotone nondecreasing. Some previous works are improved and extended

  4. Novel criteria for global exponential periodicity and stability of recurrent neural networks with time-varying delays

    International Nuclear Information System (INIS)

    Song Qiankun

    2008-01-01

    In this paper, the global exponential periodicity and stability of recurrent neural networks with time-varying delays are investigated by applying the idea of vector Lyapunov function, M-matrix theory and inequality technique. We assume neither the global Lipschitz conditions on these activation functions nor the differentiability on these time-varying delays, which were needed in other papers. Several novel criteria are found to ascertain the existence, uniqueness and global exponential stability of periodic solution for recurrent neural network with time-varying delays. Moreover, the exponential convergence rate index is estimated, which depends on the system parameters. Some previous results are improved and generalized, and an example is given to show the effectiveness of our method

  5. Fractional Hopfield Neural Networks: Fractional Dynamic Associative Recurrent Neural Networks.

    Science.gov (United States)

    Pu, Yi-Fei; Yi, Zhang; Zhou, Ji-Liu

    2017-10-01

    This paper mainly discusses a novel conceptual framework: fractional Hopfield neural networks (FHNN). As is commonly known, fractional calculus has been incorporated into artificial neural networks, mainly because of its long-term memory and nonlocality. Some researchers have made interesting attempts at fractional neural networks and gained competitive advantages over integer-order neural networks. Therefore, it is naturally makes one ponder how to generalize the first-order Hopfield neural networks to the fractional-order ones, and how to implement FHNN by means of fractional calculus. We propose to introduce a novel mathematical method: fractional calculus to implement FHNN. First, we implement fractor in the form of an analog circuit. Second, we implement FHNN by utilizing fractor and the fractional steepest descent approach, construct its Lyapunov function, and further analyze its attractors. Third, we perform experiments to analyze the stability and convergence of FHNN, and further discuss its applications to the defense against chip cloning attacks for anticounterfeiting. The main contribution of our work is to propose FHNN in the form of an analog circuit by utilizing a fractor and the fractional steepest descent approach, construct its Lyapunov function, prove its Lyapunov stability, analyze its attractors, and apply FHNN to the defense against chip cloning attacks for anticounterfeiting. A significant advantage of FHNN is that its attractors essentially relate to the neuron's fractional order. FHNN possesses the fractional-order-stability and fractional-order-sensitivity characteristics.

  6. A two-layer recurrent neural network for nonsmooth convex optimization problems.

    Science.gov (United States)

    Qin, Sitian; Xue, Xiaoping

    2015-06-01

    In this paper, a two-layer recurrent neural network is proposed to solve the nonsmooth convex optimization problem subject to convex inequality and linear equality constraints. Compared with existing neural network models, the proposed neural network has a low model complexity and avoids penalty parameters. It is proved that from any initial point, the state of the proposed neural network reaches the equality feasible region in finite time and stays there thereafter. Moreover, the state is unique if the initial point lies in the equality feasible region. The equilibrium point set of the proposed neural network is proved to be equivalent to the Karush-Kuhn-Tucker optimality set of the original optimization problem. It is further proved that the equilibrium point of the proposed neural network is stable in the sense of Lyapunov. Moreover, from any initial point, the state is proved to be convergent to an equilibrium point of the proposed neural network. Finally, as applications, the proposed neural network is used to solve nonlinear convex programming with linear constraints and L1 -norm minimization problems.

  7. Exponential stability of delayed recurrent neural networks with Markovian jumping parameters

    International Nuclear Information System (INIS)

    Wang Zidong; Liu Yurong; Yu Li; Liu Xiaohui

    2006-01-01

    In this Letter, the global exponential stability analysis problem is considered for a class of recurrent neural networks (RNNs) with time delays and Markovian jumping parameters. The jumping parameters considered here are generated from a continuous-time discrete-state homogeneous Markov process, which are governed by a Markov process with discrete and finite state space. The purpose of the problem addressed is to derive some easy-to-test conditions such that the dynamics of the neural network is stochastically exponentially stable in the mean square, independent of the time delay. By employing a new Lyapunov-Krasovskii functional, a linear matrix inequality (LMI) approach is developed to establish the desired sufficient conditions, and therefore the global exponential stability in the mean square for the delayed RNNs can be easily checked by utilizing the numerically efficient Matlab LMI toolbox, and no tuning of parameters is required. A numerical example is exploited to show the usefulness of the derived LMI-based stability conditions

  8. Stochastic exponential stability of the delayed reaction-diffusion recurrent neural networks with Markovian jumping parameters

    International Nuclear Information System (INIS)

    Wang Linshan; Zhang Zhe; Wang Yangfan

    2008-01-01

    Some criteria for the global stochastic exponential stability of the delayed reaction-diffusion recurrent neural networks with Markovian jumping parameters are presented. The jumping parameters considered here are generated from a continuous-time discrete-state homogeneous Markov process, which are governed by a Markov process with discrete and finite state space. By employing a new Lyapunov-Krasovskii functional, a linear matrix inequality (LMI) approach is developed to establish some easy-to-test criteria of global exponential stability in the mean square for the stochastic neural networks. The criteria are computationally efficient, since they are in the forms of some linear matrix inequalities

  9. Robust stability analysis of Takagi—Sugeno uncertain stochastic fuzzy recurrent neural networks with mixed time-varying delays

    International Nuclear Information System (INIS)

    Ali, M. Syed

    2011-01-01

    In this paper, the global stability of Takagi—Sugeno (TS) uncertain stochastic fuzzy recurrent neural networks with discrete and distributed time-varying delays (TSUSFRNNs) is considered. A novel LMI-based stability criterion is obtained by using Lyapunov functional theory to guarantee the asymptotic stability of TSUSFRNNs. The proposed stability conditions are demonstrated through numerical examples. Furthermore, the supplementary requirement that the time derivative of time-varying delays must be smaller than one is removed. Comparison results are demonstrated to show that the proposed method is more able to guarantee the widest stability region than the other methods available in the existing literature. (general)

  10. Ads' click-through rates predicting based on gated recurrent unit neural networks

    Science.gov (United States)

    Chen, Qiaohong; Guo, Zixuan; Dong, Wen; Jin, Lingzi

    2018-05-01

    In order to improve the effect of online advertising and to increase the revenue of advertising, the gated recurrent unit neural networks(GRU) model is used as the ads' click through rates(CTR) predicting. Combined with the characteristics of gated unit structure and the unique of time sequence in data, using BPTT algorithm to train the model. Furthermore, by optimizing the step length algorithm of the gated unit recurrent neural networks, making the model reach optimal point better and faster in less iterative rounds. The experiment results show that the model based on the gated recurrent unit neural networks and its optimization of step length algorithm has the better effect on the ads' CTR predicting, which helps advertisers, media and audience achieve a win-win and mutually beneficial situation in Three-Side Game.

  11. Novel stability criteria for uncertain delayed Cohen-Grossberg neural networks using discretized Lyapunov functional

    International Nuclear Information System (INIS)

    Souza, Fernando O.; Palhares, Reinaldo M.; Ekel, Petr Ya.

    2009-01-01

    This paper deals with the stability analysis of delayed uncertain Cohen-Grossberg neural networks (CGNN). The proposed methodology consists in obtaining new robust stability criteria formulated as linear matrix inequalities (LMIs) via the Lyapunov-Krasovskii theory. Particularly one stability criterion is derived from the selection of a parameter-dependent Lyapunov-Krasovskii functional, which allied with the Gu's discretization technique and a simple strategy that decouples the system matrices from the functional matrices, assures a less conservative stability condition. Two computer simulations are presented to support the improved theoretical results.

  12. Collaborative Recurrent Neural Networks forDynamic Recommender Systems

    Science.gov (United States)

    2016-11-22

    JMLR: Workshop and Conference Proceedings 63:366–381, 2016 ACML 2016 Collaborative Recurrent Neural Networks for Dynamic Recommender Systems Young...an unprece- dented scale. Although such activity logs are abundantly available, most approaches to recommender systems are based on the rating...Recurrent Neural Network, Recommender System , Neural Language Model, Collaborative Filtering 1. Introduction As ever larger parts of the population

  13. Tracking Control Based on Recurrent Neural Networks for Nonlinear Systems with Multiple Inputs and Unknown Deadzone

    Directory of Open Access Journals (Sweden)

    J. Humberto Pérez-Cruz

    2012-01-01

    Full Text Available This paper deals with the problem of trajectory tracking for a broad class of uncertain nonlinear systems with multiple inputs each one subject to an unknown symmetric deadzone. On the basis of a model of the deadzone as a combination of a linear term and a disturbance-like term, a continuous-time recurrent neural network is directly employed in order to identify the uncertain dynamics. By using a Lyapunov analysis, the exponential convergence of the identification error to a bounded zone is demonstrated. Subsequently, by a proper control law, the state of the neural network is compelled to follow a bounded reference trajectory. This control law is designed in such a way that the singularity problem is conveniently avoided and the exponential convergence to a bounded zone of the difference between the state of the neural identifier and the reference trajectory can be proven. Thus, the exponential convergence of the tracking error to a bounded zone and the boundedness of all closed-loop signals can be guaranteed. One of the main advantages of the proposed strategy is that the controller can work satisfactorily without any specific knowledge of an upper bound for the unmodeled dynamics and/or the disturbance term.

  14. H∞ state estimation for discrete-time memristive recurrent neural networks with stochastic time-delays

    Science.gov (United States)

    Liu, Hongjian; Wang, Zidong; Shen, Bo; Alsaadi, Fuad E.

    2016-07-01

    This paper deals with the robust H∞ state estimation problem for a class of memristive recurrent neural networks with stochastic time-delays. The stochastic time-delays under consideration are governed by a Bernoulli-distributed stochastic sequence. The purpose of the addressed problem is to design the robust state estimator such that the dynamics of the estimation error is exponentially stable in the mean square, and the prescribed ? performance constraint is met. By utilizing the difference inclusion theory and choosing a proper Lyapunov-Krasovskii functional, the existence condition of the desired estimator is derived. Based on it, the explicit expression of the estimator gain is given in terms of the solution to a linear matrix inequality. Finally, a numerical example is employed to demonstrate the effectiveness and applicability of the proposed estimation approach.

  15. Deep Recurrent Neural Network-Based Autoencoders for Acoustic Novelty Detection

    Directory of Open Access Journals (Sweden)

    Erik Marchi

    2017-01-01

    Full Text Available In the emerging field of acoustic novelty detection, most research efforts are devoted to probabilistic approaches such as mixture models or state-space models. Only recent studies introduced (pseudo-generative models for acoustic novelty detection with recurrent neural networks in the form of an autoencoder. In these approaches, auditory spectral features of the next short term frame are predicted from the previous frames by means of Long-Short Term Memory recurrent denoising autoencoders. The reconstruction error between the input and the output of the autoencoder is used as activation signal to detect novel events. There is no evidence of studies focused on comparing previous efforts to automatically recognize novel events from audio signals and giving a broad and in depth evaluation of recurrent neural network-based autoencoders. The present contribution aims to consistently evaluate our recent novel approaches to fill this white spot in the literature and provide insight by extensive evaluations carried out on three databases: A3Novelty, PASCAL CHiME, and PROMETHEUS. Besides providing an extensive analysis of novel and state-of-the-art methods, the article shows how RNN-based autoencoders outperform statistical approaches up to an absolute improvement of 16.4% average F-measure over the three databases.

  16. Deep Recurrent Neural Network-Based Autoencoders for Acoustic Novelty Detection.

    Science.gov (United States)

    Marchi, Erik; Vesperini, Fabio; Squartini, Stefano; Schuller, Björn

    2017-01-01

    In the emerging field of acoustic novelty detection, most research efforts are devoted to probabilistic approaches such as mixture models or state-space models. Only recent studies introduced (pseudo-)generative models for acoustic novelty detection with recurrent neural networks in the form of an autoencoder. In these approaches, auditory spectral features of the next short term frame are predicted from the previous frames by means of Long-Short Term Memory recurrent denoising autoencoders. The reconstruction error between the input and the output of the autoencoder is used as activation signal to detect novel events. There is no evidence of studies focused on comparing previous efforts to automatically recognize novel events from audio signals and giving a broad and in depth evaluation of recurrent neural network-based autoencoders. The present contribution aims to consistently evaluate our recent novel approaches to fill this white spot in the literature and provide insight by extensive evaluations carried out on three databases: A3Novelty, PASCAL CHiME, and PROMETHEUS. Besides providing an extensive analysis of novel and state-of-the-art methods, the article shows how RNN-based autoencoders outperform statistical approaches up to an absolute improvement of 16.4% average F -measure over the three databases.

  17. New results on global exponential stability of recurrent neural networks with time-varying delays

    International Nuclear Information System (INIS)

    Xu Shengyuan; Chu Yuming; Lu Junwei

    2006-01-01

    This Letter provides new sufficient conditions for the existence, uniqueness and global exponential stability of the equilibrium point of recurrent neural networks with time-varying delays by employing Lyapunov functions and using the Halanay inequality. The time-varying delays are not necessarily differentiable. Both Lipschitz continuous activation functions and monotone nondecreasing activation functions are considered. The derived stability criteria are expressed in terms of linear matrix inequalities (LMIs), which can be checked easily by resorting to recently developed algorithms solving LMIs. Furthermore, the proposed stability results are less conservative than some previous ones in the literature, which is demonstrated via some numerical examples

  18. New results on global exponential stability of recurrent neural networks with time-varying delays

    Energy Technology Data Exchange (ETDEWEB)

    Xu Shengyuan [Department of Automation, Nanjing University of Science and Technology, Nanjing 210094 (China)]. E-mail: syxu02@yahoo.com.cn; Chu Yuming [Department of Mathematics, Huzhou Teacher' s College, Huzhou, Zhejiang 313000 (China); Lu Junwei [School of Electrical and Automation Engineering, Nanjing Normal University, 78 Bancang Street, Nanjing, 210042 (China)

    2006-04-03

    This Letter provides new sufficient conditions for the existence, uniqueness and global exponential stability of the equilibrium point of recurrent neural networks with time-varying delays by employing Lyapunov functions and using the Halanay inequality. The time-varying delays are not necessarily differentiable. Both Lipschitz continuous activation functions and monotone nondecreasing activation functions are considered. The derived stability criteria are expressed in terms of linear matrix inequalities (LMIs), which can be checked easily by resorting to recently developed algorithms solving LMIs. Furthermore, the proposed stability results are less conservative than some previous ones in the literature, which is demonstrated via some numerical examples.

  19. ReSeg: A Recurrent Neural Network-Based Model for Semantic Segmentation

    OpenAIRE

    Visin, Francesco; Ciccone, Marco; Romero, Adriana; Kastner, Kyle; Cho, Kyunghyun; Bengio, Yoshua; Matteucci, Matteo; Courville, Aaron

    2015-01-01

    We propose a structured prediction architecture, which exploits the local generic features extracted by Convolutional Neural Networks and the capacity of Recurrent Neural Networks (RNN) to retrieve distant dependencies. The proposed architecture, called ReSeg, is based on the recently introduced ReNet model for image classification. We modify and extend it to perform the more challenging task of semantic segmentation. Each ReNet layer is composed of four RNN that sweep the image horizontally ...

  20. An adaptive PID like controller using mix locally recurrent neural network for robotic manipulator with variable payload.

    Science.gov (United States)

    Sharma, Richa; Kumar, Vikas; Gaur, Prerna; Mittal, A P

    2016-05-01

    Being complex, non-linear and coupled system, the robotic manipulator cannot be effectively controlled using classical proportional-integral-derivative (PID) controller. To enhance the effectiveness of the conventional PID controller for the nonlinear and uncertain systems, gains of the PID controller should be conservatively tuned and should adapt to the process parameter variations. In this work, a mix locally recurrent neural network (MLRNN) architecture is investigated to mimic a conventional PID controller which consists of at most three hidden nodes which act as proportional, integral and derivative node. The gains of the mix locally recurrent neural network based PID (MLRNNPID) controller scheme are initialized with a newly developed cuckoo search algorithm (CSA) based optimization method rather than assuming randomly. A sequential learning based least square algorithm is then investigated for the on-line adaptation of the gains of MLRNNPID controller. The performance of the proposed controller scheme is tested against the plant parameters uncertainties and external disturbances for both links of the two link robotic manipulator with variable payload (TL-RMWVP). The stability of the proposed controller is analyzed using Lyapunov stability criteria. A performance comparison is carried out among MLRNNPID controller, CSA optimized NNPID (OPTNNPID) controller and CSA optimized conventional PID (OPTPID) controller in order to establish the effectiveness of the MLRNNPID controller. Copyright © 2016 ISA. Published by Elsevier Ltd. All rights reserved.

  1. Discrete-time recurrent neural networks with time-varying delays: Exponential stability analysis

    International Nuclear Information System (INIS)

    Liu, Yurong; Wang, Zidong; Serrano, Alan; Liu, Xiaohui

    2007-01-01

    This Letter is concerned with the analysis problem of exponential stability for a class of discrete-time recurrent neural networks (DRNNs) with time delays. The delay is of the time-varying nature, and the activation functions are assumed to be neither differentiable nor strict monotonic. Furthermore, the description of the activation functions is more general than the recently commonly used Lipschitz conditions. Under such mild conditions, we first prove the existence of the equilibrium point. Then, by employing a Lyapunov-Krasovskii functional, a unified linear matrix inequality (LMI) approach is developed to establish sufficient conditions for the DRNNs to be globally exponentially stable. It is shown that the delayed DRNNs are globally exponentially stable if a certain LMI is solvable, where the feasibility of such an LMI can be easily checked by using the numerically efficient Matlab LMI Toolbox. A simulation example is presented to show the usefulness of the derived LMI-based stability condition

  2. A novel word spotting method based on recurrent neural networks.

    Science.gov (United States)

    Frinken, Volkmar; Fischer, Andreas; Manmatha, R; Bunke, Horst

    2012-02-01

    Keyword spotting refers to the process of retrieving all instances of a given keyword from a document. In the present paper, a novel keyword spotting method for handwritten documents is described. It is derived from a neural network-based system for unconstrained handwriting recognition. As such it performs template-free spotting, i.e., it is not necessary for a keyword to appear in the training set. The keyword spotting is done using a modification of the CTC Token Passing algorithm in conjunction with a recurrent neural network. We demonstrate that the proposed systems outperform not only a classical dynamic time warping-based approach but also a modern keyword spotting system, based on hidden Markov models. Furthermore, we analyze the performance of the underlying neural networks when using them in a recognition task followed by keyword spotting on the produced transcription. We point out the advantages of keyword spotting when compared to classic text line recognition.

  3. Global exponential stability and periodicity of reaction-diffusion recurrent neural networks with distributed delays and Dirichlet boundary conditions

    International Nuclear Information System (INIS)

    Lu Junguo; Lu Linji

    2009-01-01

    In this paper, global exponential stability and periodicity of a class of reaction-diffusion recurrent neural networks with distributed delays and Dirichlet boundary conditions are studied by constructing suitable Lyapunov functionals and utilizing some inequality techniques. We first prove global exponential convergence to 0 of the difference between any two solutions of the original neural networks, the existence and uniqueness of equilibrium is the direct results of this procedure. This approach is different from the usually used one where the existence, uniqueness of equilibrium and stability are proved in two separate steps. Secondly, we prove periodicity. Sufficient conditions ensuring the existence, uniqueness, and global exponential stability of the equilibrium and periodic solution are given. These conditions are easy to verify and our results play an important role in the design and application of globally exponentially stable neural circuits and periodic oscillatory neural circuits.

  4. Direction-of-change forecasting using a volatility-based recurrent neural network

    NARCIS (Netherlands)

    Bekiros, S.D.; Georgoutsos, D.A.

    2008-01-01

    This paper investigates the profitability of a trading strategy, based on recurrent neural networks, that attempts to predict the direction-of-change of the market in the case of the NASDAQ composite index. The sample extends over the period 8 February 1971 to 7 April 1998, while the sub-period 8

  5. Evaluation of the cranial base in amnion rupture sequence involving the anterior neural tube: implications regarding recurrence risk.

    Science.gov (United States)

    Jones, Kenneth Lyons; Robinson, Luther K; Benirschke, Kurt

    2006-09-01

    Amniotic bands can cause disruption of the cranial end of the developing fetus, leading in some cases to a neural tube closure defect. Although recurrence for unaffected parents of an affected child with a defect in which the neural tube closed normally but was subsequently disrupted by amniotic bands is negligible; for a primary defect in closure of the neural tube to which amnion has subsequently adhered, recurrence risk is 1.7%. In that primary defects of neural tube closure are characterized by typical abnormalities of the base of the skull, evaluation of the cranial base in such fetuses provides an approach for making a distinction between these 2 mechanisms. This distinction has implications regarding recurrence risk. The skull base of 2 fetuses with amnion rupture sequence involving the cranial end of the neural tube were compared to that of 1 fetus with anencephaly as well as that of a structurally normal fetus. The skulls were cleaned, fixed in 10% formalin, recleaned, and then exposed to 10% KOH solution. After washing and recleaning, the skulls were exposed to hydrogen peroxide for bleaching and photography. Despite involvement of the anterior neural tube in both fetuses with amnion rupture sequence, in Case 3 the cranial base was normal while in Case 4 the cranial base was similar to that seen in anencephaly. This technique provides a method for determining the developmental pathogenesis of anterior neural tube defects in cases of amnion rupture sequence. As such, it provides information that can be used to counsel parents of affected children with respect to recurrence risk.

  6. Adaptive complementary fuzzy self-recurrent wavelet neural network controller for the electric load simulator system

    Directory of Open Access Journals (Sweden)

    Wang Chao

    2016-03-01

    Full Text Available Due to the complexities existing in the electric load simulator, this article develops a high-performance nonlinear adaptive controller to improve the torque tracking performance of the electric load simulator, which mainly consists of an adaptive fuzzy self-recurrent wavelet neural network controller with variable structure (VSFSWC and a complementary controller. The VSFSWC is clearly and easily used for real-time systems and greatly improves the convergence rate and control precision. The complementary controller is designed to eliminate the effect of the approximation error between the proposed neural network controller and the ideal feedback controller without chattering phenomena. Moreover, adaptive learning laws are derived to guarantee the system stability in the sense of the Lyapunov theory. Finally, the hardware-in-the-loop simulations are carried out to verify the feasibility and effectiveness of the proposed algorithms in different working styles.

  7. Recurrent Neural Network for Computing Outer Inverse.

    Science.gov (United States)

    Živković, Ivan S; Stanimirović, Predrag S; Wei, Yimin

    2016-05-01

    Two linear recurrent neural networks for generating outer inverses with prescribed range and null space are defined. Each of the proposed recurrent neural networks is based on the matrix-valued differential equation, a generalization of dynamic equations proposed earlier for the nonsingular matrix inversion, the Moore-Penrose inversion, as well as the Drazin inversion, under the condition of zero initial state. The application of the first approach is conditioned by the properties of the spectrum of a certain matrix; the second approach eliminates this drawback, though at the cost of increasing the number of matrix operations. The cases corresponding to the most common generalized inverses are defined. The conditions that ensure stability of the proposed neural network are presented. Illustrative examples present the results of numerical simulations.

  8. Recurrent fuzzy neural network backstepping control for the prescribed output tracking performance of nonlinear dynamic systems.

    Science.gov (United States)

    Han, Seong-Ik; Lee, Jang-Myung

    2014-01-01

    This paper proposes a backstepping control system that uses a tracking error constraint and recurrent fuzzy neural networks (RFNNs) to achieve a prescribed tracking performance for a strict-feedback nonlinear dynamic system. A new constraint variable was defined to generate the virtual control that forces the tracking error to fall within prescribed boundaries. An adaptive RFNN was also used to obtain the required improvement on the approximation performances in order to avoid calculating the explosive number of terms generated by the recursive steps of traditional backstepping control. The boundedness and convergence of the closed-loop system was confirmed based on the Lyapunov stability theory. The prescribed performance of the proposed control scheme was validated by using it to control the prescribed error of a nonlinear system and a robot manipulator. © 2013 ISA. Published by Elsevier Ltd. All rights reserved.

  9. Reward-based training of recurrent neural networks for cognitive and value-based tasks.

    Science.gov (United States)

    Song, H Francis; Yang, Guangyu R; Wang, Xiao-Jing

    2017-01-13

    Trained neural network models, which exhibit features of neural activity recorded from behaving animals, may provide insights into the circuit mechanisms of cognitive functions through systematic analysis of network activity and connectivity. However, in contrast to the graded error signals commonly used to train networks through supervised learning, animals learn from reward feedback on definite actions through reinforcement learning. Reward maximization is particularly relevant when optimal behavior depends on an animal's internal judgment of confidence or subjective preferences. Here, we implement reward-based training of recurrent neural networks in which a value network guides learning by using the activity of the decision network to predict future reward. We show that such models capture behavioral and electrophysiological findings from well-known experimental paradigms. Our work provides a unified framework for investigating diverse cognitive and value-based computations, and predicts a role for value representation that is essential for learning, but not executing, a task.

  10. Multi-step-prediction of chaotic time series based on co-evolutionary recurrent neural network

    International Nuclear Information System (INIS)

    Ma Qianli; Zheng Qilun; Peng Hong; Qin Jiangwei; Zhong Tanwei

    2008-01-01

    This paper proposes a co-evolutionary recurrent neural network (CERNN) for the multi-step-prediction of chaotic time series, it estimates the proper parameters of phase space reconstruction and optimizes the structure of recurrent neural networks by co-evolutionary strategy. The searching space was separated into two subspaces and the individuals are trained in a parallel computational procedure. It can dynamically combine the embedding method with the capability of recurrent neural network to incorporate past experience due to internal recurrence. The effectiveness of CERNN is evaluated by using three benchmark chaotic time series data sets: the Lorenz series, Mackey-Glass series and real-world sun spot series. The simulation results show that CERNN improves the performances of multi-step-prediction of chaotic time series

  11. Recurrent Neural Network Approach Based on the Integral Representation of the Drazin Inverse.

    Science.gov (United States)

    Stanimirović, Predrag S; Živković, Ivan S; Wei, Yimin

    2015-10-01

    In this letter, we present the dynamical equation and corresponding artificial recurrent neural network for computing the Drazin inverse for arbitrary square real matrix, without any restriction on its eigenvalues. Conditions that ensure the stability of the defined recurrent neural network as well as its convergence toward the Drazin inverse are considered. Several illustrative examples present the results of computer simulations.

  12. Application of a Self-recurrent Wavelet Neural Network in the Modeling and Control of an AC Servo System

    Directory of Open Access Journals (Sweden)

    Run Min HOU

    2014-05-01

    Full Text Available To control the nonlinearity, widespread variations in loads and time varying characteristic of the high power ac servo system, the modeling and control techniques are studied here. A self-recurrent wavelet neural network (SRWNN modeling scheme is proposed, which successfully addresses the issue of the traditional wavelet neural network easily falling into local optimum, and significantly improves the network approximation capability and convergence rate. The control scheme of a SRWNN based on fuzzy compensation is expected. Gradient information is provided in real time for the controller by using a SRWNN identifier, so as to ensure that the learning and adjusting function of the controller of the SRWNN operate well, and fuzzy compensation control is applied to improve rapidity and accuracy of the entire system. Then the Lyapunov function is utilized to judge the stability of the system. The experimental analysis and comparisons with other modeling and control methods, it is clearly shown that the validities of the proposed modeling scheme and control scheme are effective.

  13. Nonlinear dynamic systems identification using recurrent interval type-2 TSK fuzzy neural network - A novel structure.

    Science.gov (United States)

    El-Nagar, Ahmad M

    2018-01-01

    In this study, a novel structure of a recurrent interval type-2 Takagi-Sugeno-Kang (TSK) fuzzy neural network (FNN) is introduced for nonlinear dynamic and time-varying systems identification. It combines the type-2 fuzzy sets (T2FSs) and a recurrent FNN to avoid the data uncertainties. The fuzzy firing strengths in the proposed structure are returned to the network input as internal variables. The interval type-2 fuzzy sets (IT2FSs) is used to describe the antecedent part for each rule while the consequent part is a TSK-type, which is a linear function of the internal variables and the external inputs with interval weights. All the type-2 fuzzy rules for the proposed RIT2TSKFNN are learned on-line based on structure and parameter learning, which are performed using the type-2 fuzzy clustering. The antecedent and consequent parameters of the proposed RIT2TSKFNN are updated based on the Lyapunov function to achieve network stability. The obtained results indicate that our proposed network has a small root mean square error (RMSE) and a small integral of square error (ISE) with a small number of rules and a small computation time compared with other type-2 FNNs. Copyright © 2017 ISA. Published by Elsevier Ltd. All rights reserved.

  14. A mathematical analysis of the effects of Hebbian learning rules on the dynamics and structure of discrete-time random recurrent neural networks.

    Science.gov (United States)

    Siri, Benoît; Berry, Hugues; Cessac, Bruno; Delord, Bruno; Quoy, Mathias

    2008-12-01

    We present a mathematical analysis of the effects of Hebbian learning in random recurrent neural networks, with a generic Hebbian learning rule, including passive forgetting and different timescales, for neuronal activity and learning dynamics. Previous numerical work has reported that Hebbian learning drives the system from chaos to a steady state through a sequence of bifurcations. Here, we interpret these results mathematically and show that these effects, involving a complex coupling between neuronal dynamics and synaptic graph structure, can be analyzed using Jacobian matrices, which introduce both a structural and a dynamical point of view on neural network evolution. Furthermore, we show that sensitivity to a learned pattern is maximal when the largest Lyapunov exponent is close to 0. We discuss how neural networks may take advantage of this regime of high functional interest.

  15. A delay-dependent LMI approach to dynamics analysis of discrete-time recurrent neural networks with time-varying delays

    International Nuclear Information System (INIS)

    Song, Qiankun; Wang, Zidong

    2007-01-01

    In this Letter, the analysis problem for the existence and stability of periodic solutions is investigated for a class of general discrete-time recurrent neural networks with time-varying delays. For the neural networks under study, a generalized activation function is considered, and the traditional assumptions on the boundedness, monotony and differentiability of the activation functions are removed. By employing the latest free-weighting matrix method, an appropriate Lyapunov-Krasovskii functional is constructed and several sufficient conditions are established to ensure the existence, uniqueness, and globally exponential stability of the periodic solution for the addressed neural network. The conditions are dependent on both the lower bound and upper bound of the time-varying time delays. Furthermore, the conditions are expressed in terms of the linear matrix inequalities (LMIs), which can be checked numerically using the effective LMI toolbox in MATLAB. Two simulation examples are given to show the effectiveness and less conservatism of the proposed criteria

  16. Stability analysis of Markovian jumping stochastic Cohen—Grossberg neural networks with discrete and distributed time varying delays

    International Nuclear Information System (INIS)

    Ali, M. Syed

    2014-01-01

    In this paper, the global asymptotic stability problem of Markovian jumping stochastic Cohen—Grossberg neural networks with discrete and distributed time-varying delays (MJSCGNNs) is considered. A novel LMI-based stability criterion is obtained by constructing a new Lyapunov functional to guarantee the asymptotic stability of MJSCGNNs. Our results can be easily verified and they are also less restrictive than previously known criteria and can be applied to Cohen—Grossberg neural networks, recurrent neural networks, and cellular neural networks. Finally, the proposed stability conditions are demonstrated with numerical examples

  17. Predicting recurrent aphthous ulceration using genetic algorithms-optimized neural networks

    Directory of Open Access Journals (Sweden)

    Najla S Dar-Odeh

    2010-05-01

    Full Text Available Najla S Dar-Odeh1, Othman M Alsmadi2, Faris Bakri3, Zaer Abu-Hammour2, Asem A Shehabi3, Mahmoud K Al-Omiri1, Shatha M K Abu-Hammad4, Hamzeh Al-Mashni4, Mohammad B Saeed4, Wael Muqbil4, Osama A Abu-Hammad1 1Faculty of Dentistry, 2Faculty of Engineering and Technology, 3Faculty of Medicine, University of Jordan, Amman, Jordan; 4Dental Department, University of Jordan Hospital, Amman, JordanObjective: To construct and optimize a neural network that is capable of predicting the occurrence of recurrent aphthous ulceration (RAU based on a set of appropriate input data.Participants and methods: Artificial neural networks (ANN software employing genetic algorithms to optimize the architecture neural networks was used. Input and output data of 86 participants (predisposing factors and status of the participants with regards to recurrent aphthous ulceration were used to construct and train the neural networks. The optimized neural networks were then tested using untrained data of a further 10 participants.Results: The optimized neural network, which produced the most accurate predictions for the presence or absence of recurrent aphthous ulceration was found to employ: gender, hematological (with or without ferritin and mycological data of the participants, frequency of tooth brushing, and consumption of vegetables and fruits.Conclusions: Factors appearing to be related to recurrent aphthous ulceration and appropriate for use as input data to construct ANNs that predict recurrent aphthous ulceration were found to include the following: gender, hemoglobin, serum vitamin B12, serum ferritin, red cell folate, salivary candidal colony count, frequency of tooth brushing, and the number of fruits or vegetables consumed daily.Keywords: artifical neural networks, recurrent, aphthous ulceration, ulcer

  18. Wind Turbine Driving a PM Synchronous Generator Using Novel Recurrent Chebyshev Neural Network Control with the Ideal Learning Rate

    Directory of Open Access Journals (Sweden)

    Chih-Hong Lin

    2016-06-01

    Full Text Available A permanent magnet (PM synchronous generator system driven by wind turbine (WT, connected with smart grid via AC-DC converter and DC-AC converter, are controlled by the novel recurrent Chebyshev neural network (NN and amended particle swarm optimization (PSO to regulate output power and output voltage in two power converters in this study. Because a PM synchronous generator system driven by WT is an unknown non-linear and time-varying dynamic system, the on-line training novel recurrent Chebyshev NN control system is developed to regulate DC voltage of the AC-DC converter and AC voltage of the DC-AC converter connected with smart grid. Furthermore, the variable learning rate of the novel recurrent Chebyshev NN is regulated according to discrete-type Lyapunov function for improving the control performance and enhancing convergent speed. Finally, some experimental results are shown to verify the effectiveness of the proposed control method for a WT driving a PM synchronous generator system in smart grid.

  19. Model for a flexible motor memory based on a self-active recurrent neural network.

    Science.gov (United States)

    Boström, Kim Joris; Wagner, Heiko; Prieske, Markus; de Lussanet, Marc

    2013-10-01

    Using recent recurrent network architecture based on the reservoir computing approach, we propose and numerically simulate a model that is focused on the aspects of a flexible motor memory for the storage of elementary movement patterns into the synaptic weights of a neural network, so that the patterns can be retrieved at any time by simple static commands. The resulting motor memory is flexible in that it is capable to continuously modulate the stored patterns. The modulation consists in an approximately linear inter- and extrapolation, generating a large space of possible movements that have not been learned before. A recurrent network of thousand neurons is trained in a manner that corresponds to a realistic exercising scenario, with experimentally measured muscular activations and with kinetic data representing proprioceptive feedback. The network is "self-active" in that it maintains recurrent flow of activation even in the absence of input, a feature that resembles the "resting-state activity" found in the human and animal brain. The model involves the concept of "neural outsourcing" which amounts to the permanent shifting of computational load from higher to lower-level neural structures, which might help to explain why humans are able to execute learned skills in a fluent and flexible manner without the need for attention to the details of the movement. Copyright © 2013 Elsevier B.V. All rights reserved.

  20. Active Control of Sound based on Diagonal Recurrent Neural Network

    NARCIS (Netherlands)

    Jayawardhana, Bayu; Xie, Lihua; Yuan, Shuqing

    2002-01-01

    Recurrent neural network has been known for its dynamic mapping and better suited for nonlinear dynamical system. Nonlinear controller may be needed in cases where the actuators exhibit the nonlinear characteristics, or in cases when the structure to be controlled exhibits nonlinear behavior. The

  1. Recurrent Neural Network Based Boolean Factor Analysis and its Application to Word Clustering

    Czech Academy of Sciences Publication Activity Database

    Frolov, A. A.; Húsek, Dušan; Polyakov, P.Y.

    2009-01-01

    Roč. 20, č. 7 (2009), s. 1073-1086 ISSN 1045-9227 R&D Projects: GA MŠk(CZ) 1M0567 Institutional research plan: CEZ:AV0Z10300504 Keywords : recurrent neural network * Hopfield-like neural network * associative memory * unsupervised learning * neural network architecture * neural network application * statistics * Boolean factor analysis * concepts search * information retrieval Subject RIV: BB - Applied Statistics, Operational Research Impact factor: 2.889, year: 2009

  2. Synthesis of recurrent neural networks for dynamical system simulation.

    Science.gov (United States)

    Trischler, Adam P; D'Eleuterio, Gabriele M T

    2016-08-01

    We review several of the most widely used techniques for training recurrent neural networks to approximate dynamical systems, then describe a novel algorithm for this task. The algorithm is based on an earlier theoretical result that guarantees the quality of the network approximation. We show that a feedforward neural network can be trained on the vector-field representation of a given dynamical system using backpropagation, then recast it as a recurrent network that replicates the original system's dynamics. After detailing this algorithm and its relation to earlier approaches, we present numerical examples that demonstrate its capabilities. One of the distinguishing features of our approach is that both the original dynamical systems and the recurrent networks that simulate them operate in continuous time. Copyright © 2016 Elsevier Ltd. All rights reserved.

  3. A novel prosodic-information synthesizer based on recurrent fuzzy neural network for the Chinese TTS system.

    Science.gov (United States)

    Lin, Chin-Teng; Wu, Rui-Cheng; Chang, Jyh-Yeong; Liang, Sheng-Fu

    2004-02-01

    In this paper, a new technique for the Chinese text-to-speech (TTS) system is proposed. Our major effort focuses on the prosodic information generation. New methodologies for constructing fuzzy rules in a prosodic model simulating human's pronouncing rules are developed. The proposed Recurrent Fuzzy Neural Network (RFNN) is a multilayer recurrent neural network (RNN) which integrates a Self-cOnstructing Neural Fuzzy Inference Network (SONFIN) into a recurrent connectionist structure. The RFNN can be functionally divided into two parts. The first part adopts the SONFIN as a prosodic model to explore the relationship between high-level linguistic features and prosodic information based on fuzzy inference rules. As compared to conventional neural networks, the SONFIN can always construct itself with an economic network size in high learning speed. The second part employs a five-layer network to generate all prosodic parameters by directly using the prosodic fuzzy rules inferred from the first part as well as other important features of syllables. The TTS system combined with the proposed method can behave not only sandhi rules but also the other prosodic phenomena existing in the traditional TTS systems. Moreover, the proposed scheme can even find out some new rules about prosodic phrase structure. The performance of the proposed RFNN-based prosodic model is verified by imbedding it into a Chinese TTS system with a Chinese monosyllable database based on the time-domain pitch synchronous overlap add (TD-PSOLA) method. Our experimental results show that the proposed RFNN can generate proper prosodic parameters including pitch means, pitch shapes, maximum energy levels, syllable duration, and pause duration. Some synthetic sounds are online available for demonstration.

  4. Character recognition from trajectory by recurrent spiking neural networks.

    Science.gov (United States)

    Jiangrong Shen; Kang Lin; Yueming Wang; Gang Pan

    2017-07-01

    Spiking neural networks are biologically plausible and power-efficient on neuromorphic hardware, while recurrent neural networks have been proven to be efficient on time series data. However, how to use the recurrent property to improve the performance of spiking neural networks is still a problem. This paper proposes a recurrent spiking neural network for character recognition using trajectories. In the network, a new encoding method is designed, in which varying time ranges of input streams are used in different recurrent layers. This is able to improve the generalization ability of our model compared with general encoding methods. The experiments are conducted on four groups of the character data set from University of Edinburgh. The results show that our method can achieve a higher average recognition accuracy than existing methods.

  5. Nonlinear dynamics analysis of a self-organizing recurrent neural network: chaos waning.

    Science.gov (United States)

    Eser, Jürgen; Zheng, Pengsheng; Triesch, Jochen

    2014-01-01

    Self-organization is thought to play an important role in structuring nervous systems. It frequently arises as a consequence of plasticity mechanisms in neural networks: connectivity determines network dynamics which in turn feed back on network structure through various forms of plasticity. Recently, self-organizing recurrent neural network models (SORNs) have been shown to learn non-trivial structure in their inputs and to reproduce the experimentally observed statistics and fluctuations of synaptic connection strengths in cortex and hippocampus. However, the dynamics in these networks and how they change with network evolution are still poorly understood. Here we investigate the degree of chaos in SORNs by studying how the networks' self-organization changes their response to small perturbations. We study the effect of perturbations to the excitatory-to-excitatory weight matrix on connection strengths and on unit activities. We find that the network dynamics, characterized by an estimate of the maximum Lyapunov exponent, becomes less chaotic during its self-organization, developing into a regime where only few perturbations become amplified. We also find that due to the mixing of discrete and (quasi-)continuous variables in SORNs, small perturbations to the synaptic weights may become amplified only after a substantial delay, a phenomenon we propose to call deferred chaos.

  6. Region stability analysis and tracking control of memristive recurrent neural network.

    Science.gov (United States)

    Bao, Gang; Zeng, Zhigang; Shen, Yanjun

    2018-02-01

    Memristor is firstly postulated by Leon Chua and realized by Hewlett-Packard (HP) laboratory. Research results show that memristor can be used to simulate the synapses of neurons. This paper presents a class of recurrent neural network with HP memristors. Firstly, it shows that memristive recurrent neural network has more compound dynamics than the traditional recurrent neural network by simulations. Then it derives that n dimensional memristive recurrent neural network is composed of [Formula: see text] sub neural networks which do not have a common equilibrium point. By designing the tracking controller, it can make memristive neural network being convergent to the desired sub neural network. At last, two numerical examples are given to verify the validity of our result. Copyright © 2017 Elsevier Ltd. All rights reserved.

  7. Convolutional over Recurrent Encoder for Neural Machine Translation

    Directory of Open Access Journals (Sweden)

    Dakwale Praveen

    2017-06-01

    Full Text Available Neural machine translation is a recently proposed approach which has shown competitive results to traditional MT approaches. Standard neural MT is an end-to-end neural network where the source sentence is encoded by a recurrent neural network (RNN called encoder and the target words are predicted using another RNN known as decoder. Recently, various models have been proposed which replace the RNN encoder with a convolutional neural network (CNN. In this paper, we propose to augment the standard RNN encoder in NMT with additional convolutional layers in order to capture wider context in the encoder output. Experiments on English to German translation demonstrate that our approach can achieve significant improvements over a standard RNN-based baseline.

  8. A New Local Bipolar Autoassociative Memory Based on External Inputs of Discrete Recurrent Neural Networks With Time Delay.

    Science.gov (United States)

    Zhou, Caigen; Zeng, Xiaoqin; Luo, Chaomin; Zhang, Huaguang

    In this paper, local bipolar auto-associative memories are presented based on discrete recurrent neural networks with a class of gain type activation function. The weight parameters of neural networks are acquired by a set of inequalities without the learning procedure. The global exponential stability criteria are established to ensure the accuracy of the restored patterns by considering time delays and external inputs. The proposed methodology is capable of effectively overcoming spurious memory patterns and achieving memory capacity. The effectiveness, robustness, and fault-tolerant capability are validated by simulated experiments.In this paper, local bipolar auto-associative memories are presented based on discrete recurrent neural networks with a class of gain type activation function. The weight parameters of neural networks are acquired by a set of inequalities without the learning procedure. The global exponential stability criteria are established to ensure the accuracy of the restored patterns by considering time delays and external inputs. The proposed methodology is capable of effectively overcoming spurious memory patterns and achieving memory capacity. The effectiveness, robustness, and fault-tolerant capability are validated by simulated experiments.

  9. From Imitation to Prediction, Data Compression vs Recurrent Neural Networks for Natural Language Processing

    Directory of Open Access Journals (Sweden)

    Juan Andres Laura

    2018-03-01

    Full Text Available In recent studies Recurrent Neural Networks were used for generative processes and their surprising performance can be explained by their ability to create good predictions. In addition, Data Compression is also based on prediction. What the problem comes down to is whether a data compressor could be used to perform as well as recurrent neural networks in the natural language processing tasks of sentiment analysis and automatic text generation. If this is possible, then the problem comes down to determining if a compression algorithm is even more intelligent than a neural network in such tasks. In our journey, a fundamental difference between a Data Compression Algorithm and Recurrent Neural Networks has been discovered.

  10. Ocean wave forecasting using recurrent neural networks

    Digital Repository Service at National Institute of Oceanography (India)

    Mandal, S.; Prabaharan, N.

    , merchant vessel routing, nearshore construction, etc. more efficiently and safely. This paper describes an artificial neural network, namely recurrent neural network with rprop update algorithm and is applied for wave forecasting. Measured ocean waves off...

  11. Identification of Non-Linear Structures using Recurrent Neural Networks

    DEFF Research Database (Denmark)

    Kirkegaard, Poul Henning; Nielsen, Søren R. K.; Hansen, H. I.

    Two different partially recurrent neural networks structured as Multi Layer Perceptrons (MLP) are investigated for time domain identification of a non-linear structure.......Two different partially recurrent neural networks structured as Multi Layer Perceptrons (MLP) are investigated for time domain identification of a non-linear structure....

  12. Identification of Non-Linear Structures using Recurrent Neural Networks

    DEFF Research Database (Denmark)

    Kirkegaard, Poul Henning; Nielsen, Søren R. K.; Hansen, H. I.

    1995-01-01

    Two different partially recurrent neural networks structured as Multi Layer Perceptrons (MLP) are investigated for time domain identification of a non-linear structure.......Two different partially recurrent neural networks structured as Multi Layer Perceptrons (MLP) are investigated for time domain identification of a non-linear structure....

  13. Acoustic Event Detection in Multichannel Audio Using Gated Recurrent Neural Networks with High‐Resolution Spectral Features

    Directory of Open Access Journals (Sweden)

    Hyoung‐Gook Kim

    2017-12-01

    Full Text Available Recently, deep recurrent neural networks have achieved great success in various machine learning tasks, and have also been applied for sound event detection. The detection of temporally overlapping sound events in realistic environments is much more challenging than in monophonic detection problems. In this paper, we present an approach to improve the accuracy of polyphonic sound event detection in multichannel audio based on gated recurrent neural networks in combination with auditory spectral features. In the proposed method, human hearing perception‐based spatial and spectral‐domain noise‐reduced harmonic features are extracted from multichannel audio and used as high‐resolution spectral inputs to train gated recurrent neural networks. This provides a fast and stable convergence rate compared to long short‐term memory recurrent neural networks. Our evaluation reveals that the proposed method outperforms the conventional approaches.

  14. A novel joint-processing adaptive nonlinear equalizer using a modular recurrent neural network for chaotic communication systems.

    Science.gov (United States)

    Zhao, Haiquan; Zeng, Xiangping; Zhang, Jiashu; Liu, Yangguang; Wang, Xiaomin; Li, Tianrui

    2011-01-01

    To eliminate nonlinear channel distortion in chaotic communication systems, a novel joint-processing adaptive nonlinear equalizer based on a pipelined recurrent neural network (JPRNN) is proposed, using a modified real-time recurrent learning (RTRL) algorithm. Furthermore, an adaptive amplitude RTRL algorithm is adopted to overcome the deteriorating effect introduced by the nesting process. Computer simulations illustrate that the proposed equalizer outperforms the pipelined recurrent neural network (PRNN) and recurrent neural network (RNN) equalizers. Copyright © 2010 Elsevier Ltd. All rights reserved.

  15. Multistability of delayed complex-valued recurrent neural networks with discontinuous real-imaginary-type activation functions

    International Nuclear Information System (INIS)

    Huang Yu-Jiao; Hu Hai-Gen

    2015-01-01

    In this paper, the multistability issue is discussed for delayed complex-valued recurrent neural networks with discontinuous real-imaginary-type activation functions. Based on a fixed theorem and stability definition, sufficient criteria are established for the existence and stability of multiple equilibria of complex-valued recurrent neural networks. The number of stable equilibria is larger than that of real-valued recurrent neural networks, which can be used to achieve high-capacity associative memories. One numerical example is provided to show the effectiveness and superiority of the presented results. (paper)

  16. Land Cover Classification via Multitemporal Spatial Data by Deep Recurrent Neural Networks

    Science.gov (United States)

    Ienco, Dino; Gaetano, Raffaele; Dupaquier, Claire; Maurel, Pierre

    2017-10-01

    Nowadays, modern earth observation programs produce huge volumes of satellite images time series (SITS) that can be useful to monitor geographical areas through time. How to efficiently analyze such kind of information is still an open question in the remote sensing field. Recently, deep learning methods proved suitable to deal with remote sensing data mainly for scene classification (i.e. Convolutional Neural Networks - CNNs - on single images) while only very few studies exist involving temporal deep learning approaches (i.e Recurrent Neural Networks - RNNs) to deal with remote sensing time series. In this letter we evaluate the ability of Recurrent Neural Networks, in particular the Long-Short Term Memory (LSTM) model, to perform land cover classification considering multi-temporal spatial data derived from a time series of satellite images. We carried out experiments on two different datasets considering both pixel-based and object-based classification. The obtained results show that Recurrent Neural Networks are competitive compared to state-of-the-art classifiers, and may outperform classical approaches in presence of low represented and/or highly mixed classes. We also show that using the alternative feature representation generated by LSTM can improve the performances of standard classifiers.

  17. Exponential stabilization and synchronization for fuzzy model of memristive neural networks by periodically intermittent control.

    Science.gov (United States)

    Yang, Shiju; Li, Chuandong; Huang, Tingwen

    2016-03-01

    The problem of exponential stabilization and synchronization for fuzzy model of memristive neural networks (MNNs) is investigated by using periodically intermittent control in this paper. Based on the knowledge of memristor and recurrent neural network, the model of MNNs is formulated. Some novel and useful stabilization criteria and synchronization conditions are then derived by using the Lyapunov functional and differential inequality techniques. It is worth noting that the methods used in this paper are also applied to fuzzy model for complex networks and general neural networks. Numerical simulations are also provided to verify the effectiveness of theoretical results. Copyright © 2015 Elsevier Ltd. All rights reserved.

  18. Protein secondary structure prediction using modular reciprocal bidirectional recurrent neural networks.

    Science.gov (United States)

    Babaei, Sepideh; Geranmayeh, Amir; Seyyedsalehi, Seyyed Ali

    2010-12-01

    The supervised learning of recurrent neural networks well-suited for prediction of protein secondary structures from the underlying amino acids sequence is studied. Modular reciprocal recurrent neural networks (MRR-NN) are proposed to model the strong correlations between adjacent secondary structure elements. Besides, a multilayer bidirectional recurrent neural network (MBR-NN) is introduced to capture the long-range intramolecular interactions between amino acids in formation of the secondary structure. The final modular prediction system is devised based on the interactive integration of the MRR-NN and the MBR-NN structures to arbitrarily engage the neighboring effects of the secondary structure types concurrent with memorizing the sequential dependencies of amino acids along the protein chain. The advanced combined network augments the percentage accuracy (Q₃) to 79.36% and boosts the segment overlap (SOV) up to 70.09% when tested on the PSIPRED dataset in three-fold cross-validation. Copyright © 2010 Elsevier Ireland Ltd. All rights reserved.

  19. Fault diagnosis of rolling bearings with recurrent neural network-based autoencoders.

    Science.gov (United States)

    Liu, Han; Zhou, Jianzhong; Zheng, Yang; Jiang, Wei; Zhang, Yuncheng

    2018-04-19

    As the rolling bearings being the key part of rotary machine, its healthy condition is quite important for safety production. Fault diagnosis of rolling bearing has been research focus for the sake of improving the economic efficiency and guaranteeing the operation security. However, the collected signals are mixed with ambient noise during the operation of rotary machine, which brings great challenge to the exact diagnosis results. Using signals collected from multiple sensors can avoid the loss of local information and extract more helpful characteristics. Recurrent Neural Networks (RNN) is a type of artificial neural network which can deal with multiple time sequence data. The capacity of RNN has been proved outstanding for catching time relevance about time sequence data. This paper proposed a novel method for bearing fault diagnosis with RNN in the form of an autoencoder. In this approach, multiple vibration value of the rolling bearings of the next period are predicted from the previous period by means of Gated Recurrent Unit (GRU)-based denoising autoencoder. These GRU-based non-linear predictive denoising autoencoders (GRU-NP-DAEs) are trained with strong generalization ability for each different fault pattern. Then for the given input data, the reconstruction errors between the next period data and the output data generated by different GRU-NP-DAEs are used to detect anomalous conditions and classify fault type. Classic rotating machinery datasets have been employed to testify the effectiveness of the proposed diagnosis method and its preponderance over some state-of-the-art methods. The experiment results indicate that the proposed method achieves satisfactory performance with strong robustness and high classification accuracy. Copyright © 2018 ISA. Published by Elsevier Ltd. All rights reserved.

  20. Individual Identification Using Functional Brain Fingerprint Detected by Recurrent Neural Network.

    Science.gov (United States)

    Chen, Shiyang; Hu, Xiaoping P

    2018-03-20

    Individual identification based on brain function has gained traction in literature. Investigating individual differences in brain function can provide additional insights into the brain. In this work, we introduce a recurrent neural network based model for identifying individuals based on only a short segment of resting state functional MRI data. In addition, we demonstrate how the global signal and differences in atlases affect the individual identifiability. Furthermore, we investigate neural network features that exhibit the uniqueness of each individual. The results indicate that our model is able to identify individuals based on neural features and provides additional information regarding brain dynamics.

  1. Evaluation of the Performance of Feedforward and Recurrent Neural Networks in Active Cancellation of Sound Noise

    Directory of Open Access Journals (Sweden)

    Mehrshad Salmasi

    2012-07-01

    Full Text Available Active noise control is based on the destructive interference between the primary noise and generated noise from the secondary source. An antinoise of equal amplitude and opposite phase is generated and combined with the primary noise. In this paper, performance of the neural networks is evaluated in active cancellation of sound noise. For this reason, feedforward and recurrent neural networks are designed and trained. After training, performance of the feedforwrad and recurrent networks in noise attenuation are compared. We use Elman network as a recurrent neural network. For simulations, noise signals from a SPIB database are used. In order to compare the networks appropriately, equal number of layers and neurons are considered for the networks. Moreover, training and test samples are similar. Simulation results show that feedforward and recurrent neural networks present good performance in noise cancellation. As it is seen, the ability of recurrent neural network in noise attenuation is better than feedforward network.

  2. Dynamic stability analysis of fractional order leaky integrator echo state neural networks

    Science.gov (United States)

    Pahnehkolaei, Seyed Mehdi Abedi; Alfi, Alireza; Tenreiro Machado, J. A.

    2017-06-01

    The Leaky integrator echo state neural network (Leaky-ESN) is an improved model of the recurrent neural network (RNN) and adopts an interconnected recurrent grid of processing neurons. This paper presents a new proof for the convergence of a Lyapunov candidate function to zero when time tends to infinity by means of the Caputo fractional derivative with order lying in the range (0, 1). The stability of Fractional-Order Leaky-ESN (FO Leaky-ESN) is then analyzed, and the existence, uniqueness and stability of the equilibrium point are provided. A numerical example demonstrates the feasibility of the proposed method.

  3. Iterative free-energy optimization for recurrent neural networks (INFERNO)

    Science.gov (United States)

    2017-01-01

    The intra-parietal lobe coupled with the Basal Ganglia forms a working memory that demonstrates strong planning capabilities for generating robust yet flexible neuronal sequences. Neurocomputational models however, often fails to control long range neural synchrony in recurrent spiking networks due to spontaneous activity. As a novel framework based on the free-energy principle, we propose to see the problem of spikes’ synchrony as an optimization problem of the neurons sub-threshold activity for the generation of long neuronal chains. Using a stochastic gradient descent, a reinforcement signal (presumably dopaminergic) evaluates the quality of one input vector to move the recurrent neural network to a desired activity; depending on the error made, this input vector is strengthened to hill-climb the gradient or elicited to search for another solution. This vector can be learned then by one associative memory as a model of the basal-ganglia to control the recurrent neural network. Experiments on habit learning and on sequence retrieving demonstrate the capabilities of the dual system to generate very long and precise spatio-temporal sequences, above two hundred iterations. Its features are applied then to the sequential planning of arm movements. In line with neurobiological theories, we discuss its relevance for modeling the cortico-basal working memory to initiate flexible goal-directed neuronal chains of causation and its relation to novel architectures such as Deep Networks, Neural Turing Machines and the Free-Energy Principle. PMID:28282439

  4. SORN: a self-organizing recurrent neural network

    Directory of Open Access Journals (Sweden)

    Andreea Lazar

    2009-10-01

    Full Text Available Understanding the dynamics of recurrent neural networks is crucial for explaining how the brain processes information. In the neocortex, a range of different plasticity mechanisms are shaping recurrent networks into effective information processing circuits that learn appropriate representations for time-varying sensory stimuli. However, it has been difficult to mimic these abilities in artificial neural network models. Here we introduce SORN, a self-organizing recurrent network. It combines three distinct forms of local plasticity to learn spatio-temporal patterns in its input while maintaining its dynamics in a healthy regime suitable for learning. The SORN learns to encode information in the form of trajectories through its high-dimensional state space reminiscent of recent biological findings on cortical coding. All three forms of plasticity are shown to be essential for the network's success.

  5. Statistical downscaling of precipitation using long short-term memory recurrent neural networks

    Science.gov (United States)

    Misra, Saptarshi; Sarkar, Sudeshna; Mitra, Pabitra

    2017-11-01

    Hydrological impacts of global climate change on regional scale are generally assessed by downscaling large-scale climatic variables, simulated by General Circulation Models (GCMs), to regional, small-scale hydrometeorological variables like precipitation, temperature, etc. In this study, we propose a new statistical downscaling model based on Recurrent Neural Network with Long Short-Term Memory which captures the spatio-temporal dependencies in local rainfall. The previous studies have used several other methods such as linear regression, quantile regression, kernel regression, beta regression, and artificial neural networks. Deep neural networks and recurrent neural networks have been shown to be highly promising in modeling complex and highly non-linear relationships between input and output variables in different domains and hence we investigated their performance in the task of statistical downscaling. We have tested this model on two datasets—one on precipitation in Mahanadi basin in India and the second on precipitation in Campbell River basin in Canada. Our autoencoder coupled long short-term memory recurrent neural network model performs the best compared to other existing methods on both the datasets with respect to temporal cross-correlation, mean squared error, and capturing the extremes.

  6. Time series prediction with simple recurrent neural networks ...

    African Journals Online (AJOL)

    A hybrid of the two called Elman-Jordan (or Multi-recurrent) neural network is also being used. In this study, we evaluated the performance of these neural networks on three established bench mark time series prediction problems. Results from the experiments showed that Jordan neural network performed significantly ...

  7. Railway track circuit fault diagnosis using recurrent neural networks

    NARCIS (Netherlands)

    de Bruin, T.D.; Verbert, K.A.J.; Babuska, R.

    2017-01-01

    Timely detection and identification of faults in railway track circuits are crucial for the safety and availability of railway networks. In this paper, the use of the long-short-term memory (LSTM) recurrent neural network is proposed to accomplish these tasks based on the commonly available

  8. Solving differential equations with unknown constitutive relations as recurrent neural networks

    Energy Technology Data Exchange (ETDEWEB)

    Hagge, Tobias J.; Stinis, Panagiotis; Yeung, Enoch H.; Tartakovsky, Alexandre M.

    2017-12-08

    We solve a system of ordinary differential equations with an unknown functional form of a sink (reaction rate) term. We assume that the measurements (time series) of state variables are partially available, and use a recurrent neural network to “learn” the reaction rate from this data. This is achieved by including discretized ordinary differential equations as part of a recurrent neural network training problem. We extend TensorFlow’s recurrent neural network architecture to create a simple but scalable and effective solver for the unknown functions, and apply it to a fedbatch bioreactor simulation problem. Use of techniques from recent deep learning literature enables training of functions with behavior manifesting over thousands of time steps. Our networks are structurally similar to recurrent neural networks, but differ in purpose, and require modified training strategies.

  9. A novel nonlinear adaptive filter using a pipelined second-order Volterra recurrent neural network.

    Science.gov (United States)

    Zhao, Haiquan; Zhang, Jiashu

    2009-12-01

    To enhance the performance and overcome the heavy computational complexity of recurrent neural networks (RNN), a novel nonlinear adaptive filter based on a pipelined second-order Volterra recurrent neural network (PSOVRNN) is proposed in this paper. A modified real-time recurrent learning (RTRL) algorithm of the proposed filter is derived in much more detail. The PSOVRNN comprises of a number of simple small-scale second-order Volterra recurrent neural network (SOVRNN) modules. In contrast to the standard RNN, these modules of a PSOVRNN can be performed simultaneously in a pipelined parallelism fashion, which can lead to a significant improvement in its total computational efficiency. Moreover, since each module of the PSOVRNN is a SOVRNN in which nonlinearity is introduced by the recursive second-order Volterra (RSOV) expansion, its performance can be further improved. Computer simulations have demonstrated that the PSOVRNN performs better than the pipelined recurrent neural network (PRNN) and RNN for nonlinear colored signals prediction and nonlinear channel equalization. However, the superiority of the PSOVRNN over the PRNN is at the cost of increasing computational complexity due to the introduced nonlinear expansion of each module.

  10. Identification and prediction of dynamic systems using an interactively recurrent self-evolving fuzzy neural network.

    Science.gov (United States)

    Lin, Yang-Yin; Chang, Jyh-Yeong; Lin, Chin-Teng

    2013-02-01

    This paper presents a novel recurrent fuzzy neural network, called an interactively recurrent self-evolving fuzzy neural network (IRSFNN), for prediction and identification of dynamic systems. The recurrent structure in an IRSFNN is formed as an external loops and internal feedback by feeding the rule firing strength of each rule to others rules and itself. The consequent part in the IRSFNN is composed of a Takagi-Sugeno-Kang (TSK) or functional-link-based type. The proposed IRSFNN employs a functional link neural network (FLNN) to the consequent part of fuzzy rules for promoting the mapping ability. Unlike a TSK-type fuzzy neural network, the FLNN in the consequent part is a nonlinear function of input variables. An IRSFNNs learning starts with an empty rule base and all of the rules are generated and learned online through a simultaneous structure and parameter learning. An on-line clustering algorithm is effective in generating fuzzy rules. The consequent update parameters are derived by a variable-dimensional Kalman filter algorithm. The premise and recurrent parameters are learned through a gradient descent algorithm. We test the IRSFNN for the prediction and identification of dynamic plants and compare it to other well-known recurrent FNNs. The proposed model obtains enhanced performance results.

  11. Design of a heart rate controller for treadmill exercise using a recurrent fuzzy neural network.

    Science.gov (United States)

    Lu, Chun-Hao; Wang, Wei-Cheng; Tai, Cheng-Chi; Chen, Tien-Chi

    2016-05-01

    In this study, we developed a computer controlled treadmill system using a recurrent fuzzy neural network heart rate controller (RFNNHRC). Treadmill speeds and inclines were controlled by corresponding control servo motors. The RFNNHRC was used to generate the control signals to automatically control treadmill speed and incline to minimize the user heart rate deviations from a preset profile. The RFNNHRC combines a fuzzy reasoning capability to accommodate uncertain information and an artificial recurrent neural network learning process that corrects for treadmill system nonlinearities and uncertainties. Treadmill speeds and inclines are controlled by the RFNNHRC to achieve minimal heart rate deviation from a pre-set profile using adjustable parameters and an on-line learning algorithm that provides robust performance against parameter variations. The on-line learning algorithm of RFNNHRC was developed and implemented using a dsPIC 30F4011 DSP. Application of the proposed control scheme to heart rate responses of runners resulted in smaller fluctuations than those produced by using proportional integra control, and treadmill speeds and inclines were smoother. The present experiments demonstrate improved heart rate tracking performance with the proposed control scheme. The RFNNHRC scheme with adjustable parameters and an on-line learning algorithm was applied to a computer controlled treadmill system with heart rate control during treadmill exercise. Novel RFNNHRC structure and controller stability analyses were introduced. The RFNNHRC were tuned using a Lyapunov function to ensure system stability. The superior heart rate control with the proposed RFNNHRC scheme was demonstrated with various pre-set heart rates. Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.

  12. Deep Recurrent Convolutional Neural Network: Improving Performance For Speech Recognition

    OpenAIRE

    Zhang, Zewang; Sun, Zheng; Liu, Jiaqi; Chen, Jingwen; Huo, Zhao; Zhang, Xiao

    2016-01-01

    A deep learning approach has been widely applied in sequence modeling problems. In terms of automatic speech recognition (ASR), its performance has significantly been improved by increasing large speech corpus and deeper neural network. Especially, recurrent neural network and deep convolutional neural network have been applied in ASR successfully. Given the arising problem of training speed, we build a novel deep recurrent convolutional network for acoustic modeling and then apply deep resid...

  13. Sensitivity analysis of linear programming problem through a recurrent neural network

    Science.gov (United States)

    Das, Raja

    2017-11-01

    In this paper we study the recurrent neural network for solving linear programming problems. To achieve optimality in accuracy and also in computational effort, an algorithm is presented. We investigate the sensitivity analysis of linear programming problem through the neural network. A detailed example is also presented to demonstrate the performance of the recurrent neural network.

  14. Ideomotor feedback control in a recurrent neural network.

    Science.gov (United States)

    Galtier, Mathieu

    2015-06-01

    The architecture of a neural network controlling an unknown environment is presented. It is based on a randomly connected recurrent neural network from which both perception and action are simultaneously read and fed back. There are two concurrent learning rules implementing a sort of ideomotor control: (i) perception is learned along the principle that the network should predict reliably its incoming stimuli; (ii) action is learned along the principle that the prediction of the network should match a target time series. The coherent behavior of the neural network in its environment is a consequence of the interaction between the two principles. Numerical simulations show a promising performance of the approach, which can be turned into a local and better "biologically plausible" algorithm.

  15. ProLanGO: Protein Function Prediction Using Neural Machine Translation Based on a Recurrent Neural Network.

    Science.gov (United States)

    Cao, Renzhi; Freitas, Colton; Chan, Leong; Sun, Miao; Jiang, Haiqing; Chen, Zhangxin

    2017-10-17

    With the development of next generation sequencing techniques, it is fast and cheap to determine protein sequences but relatively slow and expensive to extract useful information from protein sequences because of limitations of traditional biological experimental techniques. Protein function prediction has been a long standing challenge to fill the gap between the huge amount of protein sequences and the known function. In this paper, we propose a novel method to convert the protein function problem into a language translation problem by the new proposed protein sequence language "ProLan" to the protein function language "GOLan", and build a neural machine translation model based on recurrent neural networks to translate "ProLan" language to "GOLan" language. We blindly tested our method by attending the latest third Critical Assessment of Function Annotation (CAFA 3) in 2016, and also evaluate the performance of our methods on selected proteins whose function was released after CAFA competition. The good performance on the training and testing datasets demonstrates that our new proposed method is a promising direction for protein function prediction. In summary, we first time propose a method which converts the protein function prediction problem to a language translation problem and applies a neural machine translation model for protein function prediction.

  16. Medical Concept Normalization in Social Media Posts with Recurrent Neural Networks.

    Science.gov (United States)

    Tutubalina, Elena; Miftahutdinov, Zulfat; Nikolenko, Sergey; Malykh, Valentin

    2018-06-12

    Text mining of scientific libraries and social media has already proven itself as a reliable tool for drug repurposing and hypothesis generation. The task of mapping a disease mention to a concept in a controlled vocabulary, typically to the standard thesaurus in the Unified Medical Language System (UMLS), is known as medical concept normalization. This task is challenging due to the differences in the use of medical terminology between health care professionals and social media texts coming from the lay public. To bridge this gap, we use sequence learning with recurrent neural networks and semantic representation of one- or multi-word expressions: we develop end-to-end architectures directly tailored to the task, including bidirectional Long Short-Term Memory, Gated Recurrent Units with an attention mechanism, and additional semantic similarity features based on UMLS. Our evaluation against a standard benchmark shows that recurrent neural networks improve results over an effective baseline for classification based on convolutional neural networks. A qualitative examination of mentions discovered in a dataset of user reviews collected from popular online health information platforms as well as a quantitative evaluation both show improvements in the semantic representation of health-related expressions in social media. Copyright © 2018. Published by Elsevier Inc.

  17. Neural network-based model reference adaptive control system.

    Science.gov (United States)

    Patino, H D; Liu, D

    2000-01-01

    In this paper, an approach to model reference adaptive control based on neural networks is proposed and analyzed for a class of first-order continuous-time nonlinear dynamical systems. The controller structure can employ either a radial basis function network or a feedforward neural network to compensate adaptively the nonlinearities in the plant. A stable controller-parameter adjustment mechanism, which is determined using the Lyapunov theory, is constructed using a sigma-modification-type updating law. The evaluation of control error in terms of the neural network learning error is performed. That is, the control error converges asymptotically to a neighborhood of zero, whose size is evaluated and depends on the approximation error of the neural network. In the design and analysis of neural network-based control systems, it is important to take into account the neural network learning error and its influence on the control error of the plant. Simulation results showing the feasibility and performance of the proposed approach are given.

  18. Automatic construction of a recurrent neural network based classifier for vehicle passage detection

    Science.gov (United States)

    Burnaev, Evgeny; Koptelov, Ivan; Novikov, German; Khanipov, Timur

    2017-03-01

    Recurrent Neural Networks (RNNs) are extensively used for time-series modeling and prediction. We propose an approach for automatic construction of a binary classifier based on Long Short-Term Memory RNNs (LSTM-RNNs) for detection of a vehicle passage through a checkpoint. As an input to the classifier we use multidimensional signals of various sensors that are installed on the checkpoint. Obtained results demonstrate that the previous approach to handcrafting a classifier, consisting of a set of deterministic rules, can be successfully replaced by an automatic RNN training on an appropriately labelled data.

  19. Learning text representation using recurrent convolutional neural network with highway layers

    OpenAIRE

    Wen, Ying; Zhang, Weinan; Luo, Rui; Wang, Jun

    2016-01-01

    Recently, the rapid development of word embedding and neural networks has brought new inspiration to various NLP and IR tasks. In this paper, we describe a staged hybrid model combining Recurrent Convolutional Neural Networks (RCNN) with highway layers. The highway network module is incorporated in the middle takes the output of the bi-directional Recurrent Neural Network (Bi-RNN) module in the first stage and provides the Convolutional Neural Network (CNN) module in the last stage with the i...

  20. Global dissipativity of continuous-time recurrent neural networks with time delay

    International Nuclear Information System (INIS)

    Liao Xiaoxin; Wang Jun

    2003-01-01

    This paper addresses the global dissipativity of a general class of continuous-time recurrent neural networks. First, the concepts of global dissipation and global exponential dissipation are defined and elaborated. Next, the sets of global dissipativity and global exponentially dissipativity are characterized using the parameters of recurrent neural network models. In particular, it is shown that the Hopfield network and cellular neural networks with or without time delays are dissipative systems

  1. Recurrent Neural Network For Forecasting Time Series With Long Memory Pattern

    Science.gov (United States)

    Walid; Alamsyah

    2017-04-01

    Recurrent Neural Network as one of the hybrid models are often used to predict and estimate the issues related to electricity, can be used to describe the cause of the swelling of electrical load which experienced by PLN. In this research will be developed RNN forecasting procedures at the time series with long memory patterns. Considering the application is the national electrical load which of course has a different trend with the condition of the electrical load in any country. This research produces the algorithm of time series forecasting which has long memory pattern using E-RNN after this referred to the algorithm of integrated fractional recurrent neural networks (FIRNN).The prediction results of long memory time series using models Fractional Integrated Recurrent Neural Network (FIRNN) showed that the model with the selection of data difference in the range of [-1,1] and the model of Fractional Integrated Recurrent Neural Network (FIRNN) (24,6,1) provides the smallest MSE value, which is 0.00149684.

  2. Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition.

    Science.gov (United States)

    Spoerer, Courtney J; McClure, Patrick; Kriegeskorte, Nikolaus

    2017-01-01

    Feedforward neural networks provide the dominant model of how the brain performs visual object recognition. However, these networks lack the lateral and feedback connections, and the resulting recurrent neuronal dynamics, of the ventral visual pathway in the human and non-human primate brain. Here we investigate recurrent convolutional neural networks with bottom-up (B), lateral (L), and top-down (T) connections. Combining these types of connections yields four architectures (B, BT, BL, and BLT), which we systematically test and compare. We hypothesized that recurrent dynamics might improve recognition performance in the challenging scenario of partial occlusion. We introduce two novel occluded object recognition tasks to test the efficacy of the models, digit clutter (where multiple target digits occlude one another) and digit debris (where target digits are occluded by digit fragments). We find that recurrent neural networks outperform feedforward control models (approximately matched in parametric complexity) at recognizing objects, both in the absence of occlusion and in all occlusion conditions. Recurrent networks were also found to be more robust to the inclusion of additive Gaussian noise. Recurrent neural networks are better in two respects: (1) they are more neurobiologically realistic than their feedforward counterparts; (2) they are better in terms of their ability to recognize objects, especially under challenging conditions. This work shows that computer vision can benefit from using recurrent convolutional architectures and suggests that the ubiquitous recurrent connections in biological brains are essential for task performance.

  3. Constructing Long Short-Term Memory based Deep Recurrent Neural Networks for Large Vocabulary Speech Recognition

    OpenAIRE

    Li, Xiangang; Wu, Xihong

    2014-01-01

    Long short-term memory (LSTM) based acoustic modeling methods have recently been shown to give state-of-the-art performance on some speech recognition tasks. To achieve a further performance improvement, in this research, deep extensions on LSTM are investigated considering that deep hierarchical model has turned out to be more efficient than a shallow one. Motivated by previous research on constructing deep recurrent neural networks (RNNs), alternative deep LSTM architectures are proposed an...

  4. A One-Layer Recurrent Neural Network for Constrained Complex-Variable Convex Optimization.

    Science.gov (United States)

    Qin, Sitian; Feng, Jiqiang; Song, Jiahui; Wen, Xingnan; Xu, Chen

    2018-03-01

    In this paper, based on calculus and penalty method, a one-layer recurrent neural network is proposed for solving constrained complex-variable convex optimization. It is proved that for any initial point from a given domain, the state of the proposed neural network reaches the feasible region in finite time and converges to an optimal solution of the constrained complex-variable convex optimization finally. In contrast to existing neural networks for complex-variable convex optimization, the proposed neural network has a lower model complexity and better convergence. Some numerical examples and application are presented to substantiate the effectiveness of the proposed neural network.

  5. Noise-enhanced categorization in a recurrently reconnected neural network

    International Nuclear Information System (INIS)

    Monterola, Christopher; Zapotocky, Martin

    2005-01-01

    We investigate the interplay of recurrence and noise in neural networks trained to categorize spatial patterns of neural activity. We develop the following procedure to demonstrate how, in the presence of noise, the introduction of recurrence permits to significantly extend and homogenize the operating range of a feed-forward neural network. We first train a two-level perceptron in the absence of noise. Following training, we identify the input and output units of the feed-forward network, and thus convert it into a two-layer recurrent network. We show that the performance of the reconnected network has features reminiscent of nondynamic stochastic resonance: the addition of noise enables the network to correctly categorize stimuli of subthreshold strength, with optimal noise magnitude significantly exceeding the stimulus strength. We characterize the dynamics leading to this effect and contrast it to the behavior of a more simple associative memory network in which noise-mediated categorization fails

  6. Noise-enhanced categorization in a recurrently reconnected neural network

    Science.gov (United States)

    Monterola, Christopher; Zapotocky, Martin

    2005-03-01

    We investigate the interplay of recurrence and noise in neural networks trained to categorize spatial patterns of neural activity. We develop the following procedure to demonstrate how, in the presence of noise, the introduction of recurrence permits to significantly extend and homogenize the operating range of a feed-forward neural network. We first train a two-level perceptron in the absence of noise. Following training, we identify the input and output units of the feed-forward network, and thus convert it into a two-layer recurrent network. We show that the performance of the reconnected network has features reminiscent of nondynamic stochastic resonance: the addition of noise enables the network to correctly categorize stimuli of subthreshold strength, with optimal noise magnitude significantly exceeding the stimulus strength. We characterize the dynamics leading to this effect and contrast it to the behavior of a more simple associative memory network in which noise-mediated categorization fails.

  7. Recurrent neural networks for breast lesion classification based on DCE-MRIs

    Science.gov (United States)

    Antropova, Natasha; Huynh, Benjamin; Giger, Maryellen

    2018-02-01

    Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) plays a significant role in breast cancer screening, cancer staging, and monitoring response to therapy. Recently, deep learning methods are being rapidly incorporated in image-based breast cancer diagnosis and prognosis. However, most of the current deep learning methods make clinical decisions based on 2-dimentional (2D) or 3D images and are not well suited for temporal image data. In this study, we develop a deep learning methodology that enables integration of clinically valuable temporal components of DCE-MRIs into deep learning-based lesion classification. Our work is performed on a database of 703 DCE-MRI cases for the task of distinguishing benign and malignant lesions, and uses the area under the ROC curve (AUC) as the performance metric in conducting that task. We train a recurrent neural network, specifically a long short-term memory network (LSTM), on sequences of image features extracted from the dynamic MRI sequences. These features are extracted with VGGNet, a convolutional neural network pre-trained on a large dataset of natural images ImageNet. The features are obtained from various levels of the network, to capture low-, mid-, and high-level information about the lesion. Compared to a classification method that takes as input only images at a single time-point (yielding an AUC = 0.81 (se = 0.04)), our LSTM method improves lesion classification with an AUC of 0.85 (se = 0.03).

  8. Embedding recurrent neural networks into predator-prey models.

    Science.gov (United States)

    Moreau, Yves; Louiès, Stephane; Vandewalle, Joos; Brenig, Leon

    1999-03-01

    We study changes of coordinates that allow the embedding of ordinary differential equations describing continuous-time recurrent neural networks into differential equations describing predator-prey models-also called Lotka-Volterra systems. We transform the equations for the neural network first into quasi-monomial form (Brenig, L. (1988). Complete factorization and analytic solutions of generalized Lotka-Volterra equations. Physics Letters A, 133(7-8), 378-382), where we express the vector field of the dynamical system as a linear combination of products of powers of the variables. In practice, this transformation is possible only if the activation function is the hyperbolic tangent or the logistic sigmoid. From this quasi-monomial form, we can directly transform the system further into Lotka-Volterra equations. The resulting Lotka-Volterra system is of higher dimension than the original system, but the behavior of its first variables is equivalent to the behavior of the original neural network. We expect that this transformation will permit the application of existing techniques for the analysis of Lotka-Volterra systems to recurrent neural networks. Furthermore, our results show that Lotka-Volterra systems are universal approximators of dynamical systems, just as are continuous-time neural networks.

  9. Distributed Recurrent Neural Forward Models with Synaptic Adaptation and CPG-based control for Complex Behaviors of Walking Robots

    Directory of Open Access Journals (Sweden)

    Sakyasingha eDasgupta

    2015-09-01

    Full Text Available Walking animals, like stick insects, cockroaches or ants, demonstrate a fascinating range of locomotive abilities and complex behaviors. The locomotive behaviors can consist of a variety of walking patterns along with adaptation that allow the animals to deal with changes in environmental conditions, like uneven terrains, gaps, obstacles etc. Biological study has revealed that such complex behaviors are a result of a combination of biomechanics and neural mechanism thus representing the true nature of embodied interactions. While the biomechanics helps maintain flexibility and sustain a variety of movements, the neural mechanisms generate movements while making appropriate predictions crucial for achieving adaptation. Such predictions or planning ahead can be achieved by way of internal models that are grounded in the overall behavior of the animal. Inspired by these findings, we present here, an artificial bio-inspired walking system which effectively combines biomechanics (in terms of the body and leg structures with the underlying neural mechanisms. The neural mechanisms consist of 1 central pattern generator based control for generating basic rhythmic patterns and coordinated movements, 2 distributed (at each leg recurrent neural network based adaptive forward models with efference copies as internal models for sensory predictions and instantaneous state estimations, and 3 searching and elevation control for adapting the movement of an individual leg to deal with different environmental conditions. Using simulations we show that this bio-inspired approach with adaptive internal models allows the walking robot to perform complex locomotive behaviors as observed in insects, including walking on undulated terrains, crossing large gaps as well as climbing over high obstacles. Furthermore we demonstrate that the newly developed recurrent network based approach to sensorimotor prediction outperforms the previous state of the art adaptive neuron

  10. Exponential H(infinity) synchronization of general discrete-time chaotic neural networks with or without time delays.

    Science.gov (United States)

    Qi, Donglian; Liu, Meiqin; Qiu, Meikang; Zhang, Senlin

    2010-08-01

    This brief studies exponential H(infinity) synchronization of a class of general discrete-time chaotic neural networks with external disturbance. On the basis of the drive-response concept and H(infinity) control theory, and using Lyapunov-Krasovskii (or Lyapunov) functional, state feedback controllers are established to not only guarantee exponential stable synchronization between two general chaotic neural networks with or without time delays, but also reduce the effect of external disturbance on the synchronization error to a minimal H(infinity) norm constraint. The proposed controllers can be obtained by solving the convex optimization problems represented by linear matrix inequalities. Most discrete-time chaotic systems with or without time delays, such as Hopfield neural networks, cellular neural networks, bidirectional associative memory networks, recurrent multilayer perceptrons, Cohen-Grossberg neural networks, Chua's circuits, etc., can be transformed into this general chaotic neural network to be H(infinity) synchronization controller designed in a unified way. Finally, some illustrated examples with their simulations have been utilized to demonstrate the effectiveness of the proposed methods.

  11. A one-layer recurrent neural network for constrained nonconvex optimization.

    Science.gov (United States)

    Li, Guocheng; Yan, Zheng; Wang, Jun

    2015-01-01

    In this paper, a one-layer recurrent neural network is proposed for solving nonconvex optimization problems subject to general inequality constraints, designed based on an exact penalty function method. It is proved herein that any neuron state of the proposed neural network is convergent to the feasible region in finite time and stays there thereafter, provided that the penalty parameter is sufficiently large. The lower bounds of the penalty parameter and convergence time are also estimated. In addition, any neural state of the proposed neural network is convergent to its equilibrium point set which satisfies the Karush-Kuhn-Tucker conditions of the optimization problem. Moreover, the equilibrium point set is equivalent to the optimal solution to the nonconvex optimization problem if the objective function and constraints satisfy given conditions. Four numerical examples are provided to illustrate the performances of the proposed neural network.

  12. Adaptive Filtering Using Recurrent Neural Networks

    Science.gov (United States)

    Parlos, Alexander G.; Menon, Sunil K.; Atiya, Amir F.

    2005-01-01

    A method for adaptive (or, optionally, nonadaptive) filtering has been developed for estimating the states of complex process systems (e.g., chemical plants, factories, or manufacturing processes at some level of abstraction) from time series of measurements of system inputs and outputs. The method is based partly on the fundamental principles of the Kalman filter and partly on the use of recurrent neural networks. The standard Kalman filter involves an assumption of linearity of the mathematical model used to describe a process system. The extended Kalman filter accommodates a nonlinear process model but still requires linearization about the state estimate. Both the standard and extended Kalman filters involve the often unrealistic assumption that process and measurement noise are zero-mean, Gaussian, and white. In contrast, the present method does not involve any assumptions of linearity of process models or of the nature of process noise; on the contrary, few (if any) assumptions are made about process models, noise models, or the parameters of such models. In this regard, the method can be characterized as one of nonlinear, nonparametric filtering. The method exploits the unique ability of neural networks to approximate nonlinear functions. In a given case, the process model is limited mainly by limitations of the approximation ability of the neural networks chosen for that case. Moreover, despite the lack of assumptions regarding process noise, the method yields minimum- variance filters. In that they do not require statistical models of noise, the neural- network-based state filters of this method are comparable to conventional nonlinear least-squares estimators.

  13. Recurrent Neural Network for Computing the Drazin Inverse.

    Science.gov (United States)

    Stanimirović, Predrag S; Zivković, Ivan S; Wei, Yimin

    2015-11-01

    This paper presents a recurrent neural network (RNN) for computing the Drazin inverse of a real matrix in real time. This recurrent neural network (RNN) is composed of n independent parts (subnetworks), where n is the order of the input matrix. These subnetworks can operate concurrently, so parallel and distributed processing can be achieved. In this way, the computational advantages over the existing sequential algorithms can be attained in real-time applications. The RNN defined in this paper is convenient for an implementation in an electronic circuit. The number of neurons in the neural network is the same as the number of elements in the output matrix, which represents the Drazin inverse. The difference between the proposed RNN and the existing ones for the Drazin inverse computation lies in their network architecture and dynamics. The conditions that ensure the stability of the defined RNN as well as its convergence toward the Drazin inverse are considered. In addition, illustrative examples and examples of application to the practical engineering problems are discussed to show the efficacy of the proposed neural network.

  14. Hyperchaos of four state autonomous system with three positive Lyapunov exponents

    International Nuclear Information System (INIS)

    Ge Zhengming; Yang, C-H.

    2009-01-01

    This Letter gives the results of numerical simulations of Quantum Cellular Neural Network (Quantum-CNN) autonomous system with four state variables. Three positive Lyapunov exponents confirm hyperchaotic nature of its dynamics

  15. Reduced-Order Modeling for Flutter/LCO Using Recurrent Artificial Neural Network

    Science.gov (United States)

    Yao, Weigang; Liou, Meng-Sing

    2012-01-01

    The present study demonstrates the efficacy of a recurrent artificial neural network to provide a high fidelity time-dependent nonlinear reduced-order model (ROM) for flutter/limit-cycle oscillation (LCO) modeling. An artificial neural network is a relatively straightforward nonlinear method for modeling an input-output relationship from a set of known data, for which we use the radial basis function (RBF) with its parameters determined through a training process. The resulting RBF neural network, however, is only static and is not yet adequate for an application to problems of dynamic nature. The recurrent neural network method [1] is applied to construct a reduced order model resulting from a series of high-fidelity time-dependent data of aero-elastic simulations. Once the RBF neural network ROM is constructed properly, an accurate approximate solution can be obtained at a fraction of the cost of a full-order computation. The method derived during the study has been validated for predicting nonlinear aerodynamic forces in transonic flow and is capable of accurate flutter/LCO simulations. The obtained results indicate that the present recurrent RBF neural network is accurate and efficient for nonlinear aero-elastic system analysis

  16. Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition

    OpenAIRE

    Francisco Javier Ordóñez; Daniel Roggen

    2016-01-01

    Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we pro...

  17. Deep Gate Recurrent Neural Network

    Science.gov (United States)

    2016-11-22

    and Fred Cummins. Learning to forget: Continual prediction with lstm . Neural computation, 12(10):2451–2471, 2000. Alex Graves. Generating sequences...DSGU) and Simple Gated Unit (SGU), which are structures for learning long-term dependencies. Compared to traditional Long Short-Term Memory ( LSTM ) and...Gated Recurrent Unit (GRU), both structures require fewer parameters and less computation time in sequence classification tasks. Unlike GRU and LSTM

  18. Evaluation of the Performance of Feedforward and Recurrent Neural Networks in Active Cancellation of Sound Noise

    OpenAIRE

    Mehrshad Salmasi; Homayoun Mahdavi-Nasab

    2012-01-01

    Active noise control is based on the destructive interference between the primary noise and generated noise from the secondary source. An antinoise of equal amplitude and opposite phase is generated and combined with the primary noise. In this paper, performance of the neural networks is evaluated in active cancellation of sound noise. For this reason, feedforward and recurrent neural networks are designed and trained. After training, performance of the feedforwrad and recurrent networks in n...

  19. Parameter estimation in space systems using recurrent neural networks

    Science.gov (United States)

    Parlos, Alexander G.; Atiya, Amir F.; Sunkel, John W.

    1991-01-01

    The identification of time-varying parameters encountered in space systems is addressed, using artificial neural systems. A hybrid feedforward/feedback neural network, namely a recurrent multilayer perception, is used as the model structure in the nonlinear system identification. The feedforward portion of the network architecture provides its well-known interpolation property, while through recurrency and cross-talk, the local information feedback enables representation of temporal variations in the system nonlinearities. The standard back-propagation-learning algorithm is modified and it is used for both the off-line and on-line supervised training of the proposed hybrid network. The performance of recurrent multilayer perceptron networks in identifying parameters of nonlinear dynamic systems is investigated by estimating the mass properties of a representative large spacecraft. The changes in the spacecraft inertia are predicted using a trained neural network, during two configurations corresponding to the early and late stages of the spacecraft on-orbit assembly sequence. The proposed on-line mass properties estimation capability offers encouraging results, though, further research is warranted for training and testing the predictive capabilities of these networks beyond nominal spacecraft operations.

  20. Robust lyapunov controller for uncertain systems

    KAUST Repository

    Laleg-Kirati, Taous-Meriem

    2017-02-23

    Various examples of systems and methods are provided for Lyapunov control for uncertain systems. In one example, a system includes a process plant and a robust Lyapunov controller configured to control an input of the process plant. The robust Lyapunov controller includes an inner closed loop Lyapunov controller and an outer closed loop error stabilizer. In another example, a method includes monitoring a system output of a process plant; generating an estimated system control input based upon a defined output reference; generating a system control input using the estimated system control input and a compensation term; and adjusting the process plant based upon the system control input to force the system output to track the defined output reference. An inner closed loop Lyapunov controller can generate the estimated system control input and an outer closed loop error stabilizer can generate the system control input.

  1. Convolutional neural networks for prostate cancer recurrence prediction

    Science.gov (United States)

    Kumar, Neeraj; Verma, Ruchika; Arora, Ashish; Kumar, Abhay; Gupta, Sanchit; Sethi, Amit; Gann, Peter H.

    2017-03-01

    Accurate prediction of the treatment outcome is important for cancer treatment planning. We present an approach to predict prostate cancer (PCa) recurrence after radical prostatectomy using tissue images. We used a cohort whose case vs. control (recurrent vs. non-recurrent) status had been determined using post-treatment follow up. Further, to aid the development of novel biomarkers of PCa recurrence, cases and controls were paired based on matching of other predictive clinical variables such as Gleason grade, stage, age, and race. For this cohort, tissue resection microarray with up to four cores per patient was available. The proposed approach is based on deep learning, and its novelty lies in the use of two separate convolutional neural networks (CNNs) - one to detect individual nuclei even in the crowded areas, and the other to classify them. To detect nuclear centers in an image, the first CNN predicts distance transform of the underlying (but unknown) multi-nuclear map from the input HE image. The second CNN classifies the patches centered at nuclear centers into those belonging to cases or controls. Voting across patches extracted from image(s) of a patient yields the probability of recurrence for the patient. The proposed approach gave 0.81 AUC for a sample of 30 recurrent cases and 30 non-recurrent controls, after being trained on an independent set of 80 case-controls pairs. If validated further, such an approach might help in choosing between a combination of treatment options such as active surveillance, radical prostatectomy, radiation, and hormone therapy. It can also generalize to the prediction of treatment outcomes in other cancers.

  2. Optimal Formation of Multirobot Systems Based on a Recurrent Neural Network.

    Science.gov (United States)

    Wang, Yunpeng; Cheng, Long; Hou, Zeng-Guang; Yu, Junzhi; Tan, Min

    2016-02-01

    The optimal formation problem of multirobot systems is solved by a recurrent neural network in this paper. The desired formation is described by the shape theory. This theory can generate a set of feasible formations that share the same relative relation among robots. An optimal formation means that finding one formation from the feasible formation set, which has the minimum distance to the initial formation of the multirobot system. Then, the formation problem is transformed into an optimization problem. In addition, the orientation, scale, and admissible range of the formation can also be considered as the constraints in the optimization problem. Furthermore, if all robots are identical, their positions in the system are exchangeable. Then, each robot does not necessarily move to one specific position in the formation. In this case, the optimal formation problem becomes a combinational optimization problem, whose optimal solution is very hard to obtain. Inspired by the penalty method, this combinational optimization problem can be approximately transformed into a convex optimization problem. Due to the involvement of the Euclidean norm in the distance, the objective function of these optimization problems are nonsmooth. To solve these nonsmooth optimization problems efficiently, a recurrent neural network approach is employed, owing to its parallel computation ability. Finally, some simulations and experiments are given to validate the effectiveness and efficiency of the proposed optimal formation approach.

  3. Recursive Bayesian recurrent neural networks for time-series modeling.

    Science.gov (United States)

    Mirikitani, Derrick T; Nikolaev, Nikolay

    2010-02-01

    This paper develops a probabilistic approach to recursive second-order training of recurrent neural networks (RNNs) for improved time-series modeling. A general recursive Bayesian Levenberg-Marquardt algorithm is derived to sequentially update the weights and the covariance (Hessian) matrix. The main strengths of the approach are a principled handling of the regularization hyperparameters that leads to better generalization, and stable numerical performance. The framework involves the adaptation of a noise hyperparameter and local weight prior hyperparameters, which represent the noise in the data and the uncertainties in the model parameters. Experimental investigations using artificial and real-world data sets show that RNNs equipped with the proposed approach outperform standard real-time recurrent learning and extended Kalman training algorithms for recurrent networks, as well as other contemporary nonlinear neural models, on time-series modeling.

  4. Analysis of surface ozone using a recurrent neural network.

    Science.gov (United States)

    Biancofiore, Fabio; Verdecchia, Marco; Di Carlo, Piero; Tomassetti, Barbara; Aruffo, Eleonora; Busilacchio, Marcella; Bianco, Sebastiano; Di Tommaso, Sinibaldo; Colangeli, Carlo

    2015-05-01

    Hourly concentrations of ozone (O₃) and nitrogen dioxide (NO₂) have been measured for 16 years, from 1998 to 2013, in a seaside town in central Italy. The seasonal trends of O₃ and NO₂ recorded in this period have been studied. Furthermore, we used the data collected during one year (2005), to define the characteristics of a multiple linear regression model and a neural network model. Both models are used to model the hourly O₃ concentration, using, two scenarios: 1) in the first as inputs, only meteorological parameters and 2) in the second adding photochemical parameters at those of the first scenario. In order to evaluate the performance of the model four statistical criteria are used: correlation coefficient, fractional bias, normalized mean squared error and a factor of two. All the criteria show that the neural network gives better results, compared to the regression model, in all the model scenarios. Predictions of O₃ have been carried out by many authors using a feed forward neural architecture. In this paper we show that a recurrent architecture significantly improves the performances of neural predictors. Using only the meteorological parameters as input, the recurrent architecture shows performance better than the multiple linear regression model that uses meteorological and photochemical data as input, making the neural network model with recurrent architecture a more useful tool in areas where only weather measurements are available. Finally, we used the neural network model to forecast the O₃ hourly concentrations 1, 3, 6, 12, 24 and 48 h ahead. The performances of the model in predicting O₃ levels are discussed. Emphasis is given to the possibility of using the neural network model in operational ways in areas where only meteorological data are available, in order to predict O₃ also in sites where it has not been measured yet. Copyright © 2015 Elsevier B.V. All rights reserved.

  5. Collective Lyapunov modes

    International Nuclear Information System (INIS)

    Takeuchi, Kazumasa A; Chaté, Hugues

    2013-01-01

    We show, using covariant Lyapunov vectors in addition to standard Lyapunov analysis, that there exists a set of collective Lyapunov modes in large chaotic systems exhibiting collective dynamics. Associated with delocalized Lyapunov vectors, they act collectively on the trajectory and hence characterize the instability of its collective dynamics. We further develop, for globally coupled systems, a connection between these collective modes and the Lyapunov modes in the corresponding Perron–Frobenius equation. We thereby address the fundamental question of the effective dimension of collective dynamics and discuss the extensivity of chaos in the presence of collective dynamics. This article is part of a special issue of Journal of Physics A: Mathematical and Theoretical devoted to ‘Lyapunov analysis: from dynamical systems theory to applications’. (paper)

  6. Estimating Ads’ Click through Rate with Recurrent Neural Network

    Directory of Open Access Journals (Sweden)

    Chen Qiao-Hong

    2016-01-01

    Full Text Available With the development of the Internet, online advertising spreads across every corner of the world, the ads' click through rate (CTR estimation is an important method to improve the online advertising revenue. Compared with the linear model, the nonlinear models can study much more complex relationships between a large number of nonlinear characteristics, so as to improve the accuracy of the estimation of the ads’ CTR. The recurrent neural network (RNN based on Long-Short Term Memory (LSTM is an improved model of the feedback neural network with ring structure. The model overcomes the problem of the gradient of the general RNN. Experiments show that the RNN based on LSTM exceeds the linear models, and it can effectively improve the estimation effect of the ads’ click through rate.

  7. Multistability and instability analysis of recurrent neural networks with time-varying delays.

    Science.gov (United States)

    Zhang, Fanghai; Zeng, Zhigang

    2018-01-01

    This paper provides new theoretical results on the multistability and instability analysis of recurrent neural networks with time-varying delays. It is shown that such n-neuronal recurrent neural networks have exactly [Formula: see text] equilibria, [Formula: see text] of which are locally exponentially stable and the others are unstable, where k 0 is a nonnegative integer such that k 0 ≤n. By using the combination method of two different divisions, recurrent neural networks can possess more dynamic properties. This method improves and extends the existing results in the literature. Finally, one numerical example is provided to show the superiority and effectiveness of the presented results. Copyright © 2017 Elsevier Ltd. All rights reserved.

  8. Supervised Sequence Labelling with Recurrent Neural Networks

    CERN Document Server

    Graves, Alex

    2012-01-01

    Supervised sequence labelling is a vital area of machine learning, encompassing tasks such as speech, handwriting and gesture recognition, protein secondary structure prediction and part-of-speech tagging. Recurrent neural networks are powerful sequence learning tools—robust to input noise and distortion, able to exploit long-range contextual information—that would seem ideally suited to such problems. However their role in large-scale sequence labelling systems has so far been auxiliary.    The goal of this book is a complete framework for classifying and transcribing sequential data with recurrent neural networks only. Three main innovations are introduced in order to realise this goal. Firstly, the connectionist temporal classification output layer allows the framework to be trained with unsegmented target sequences, such as phoneme-level speech transcriptions; this is in contrast to previous connectionist approaches, which were dependent on error-prone prior segmentation. Secondly, multidimensional...

  9. A one-layer recurrent neural network for constrained nonsmooth invex optimization.

    Science.gov (United States)

    Li, Guocheng; Yan, Zheng; Wang, Jun

    2014-02-01

    Invexity is an important notion in nonconvex optimization. In this paper, a one-layer recurrent neural network is proposed for solving constrained nonsmooth invex optimization problems, designed based on an exact penalty function method. It is proved herein that any state of the proposed neural network is globally convergent to the optimal solution set of constrained invex optimization problems, with a sufficiently large penalty parameter. In addition, any neural state is globally convergent to the unique optimal solution, provided that the objective function and constraint functions are pseudoconvex. Moreover, any neural state is globally convergent to the feasible region in finite time and stays there thereafter. The lower bounds of the penalty parameter and convergence time are also estimated. Two numerical examples are provided to illustrate the performances of the proposed neural network. Copyright © 2013 Elsevier Ltd. All rights reserved.

  10. Neural networks for tracking of unknown SISO discrete-time nonlinear dynamic systems.

    Science.gov (United States)

    Aftab, Muhammad Saleheen; Shafiq, Muhammad

    2015-11-01

    This article presents a Lyapunov function based neural network tracking (LNT) strategy for single-input, single-output (SISO) discrete-time nonlinear dynamic systems. The proposed LNT architecture is composed of two feedforward neural networks operating as controller and estimator. A Lyapunov function based back propagation learning algorithm is used for online adjustment of the controller and estimator parameters. The controller and estimator error convergence and closed-loop system stability analysis is performed by Lyapunov stability theory. Moreover, two simulation examples and one real-time experiment are investigated as case studies. The achieved results successfully validate the controller performance. Copyright © 2015 ISA. Published by Elsevier Ltd. All rights reserved.

  11. A non-penalty recurrent neural network for solving a class of constrained optimization problems.

    Science.gov (United States)

    Hosseini, Alireza

    2016-01-01

    In this paper, we explain a methodology to analyze convergence of some differential inclusion-based neural networks for solving nonsmooth optimization problems. For a general differential inclusion, we show that if its right hand-side set valued map satisfies some conditions, then solution trajectory of the differential inclusion converges to optimal solution set of its corresponding in optimization problem. Based on the obtained methodology, we introduce a new recurrent neural network for solving nonsmooth optimization problems. Objective function does not need to be convex on R(n) nor does the new neural network model require any penalty parameter. We compare our new method with some penalty-based and non-penalty based models. Moreover for differentiable cases, we implement circuit diagram of the new neural network. Copyright © 2015 Elsevier Ltd. All rights reserved.

  12. Representation of linguistic form and function in recurrent neural networks

    NARCIS (Netherlands)

    Kadar, Akos; Chrupala, Grzegorz; Alishahi, Afra

    2017-01-01

    We present novel methods for analyzing the activation patterns of recurrent neural networks from a linguistic point of view and explore the types of linguistic structure they learn. As a case study, we use a standard standalone language model, and a multi-task gated recurrent network architecture

  13. Study of hourly and daily solar irradiation forecast using diagonal recurrent wavelet neural networks

    International Nuclear Information System (INIS)

    Cao Jiacong; Lin Xingchun

    2008-01-01

    An accurate forecast of solar irradiation is required for various solar energy applications and environmental impact analyses in recent years. Comparatively, various irradiation forecast models based on artificial neural networks (ANN) perform much better in accuracy than many conventional prediction models. However, the forecast precision of most existing ANN based forecast models has not been satisfactory to researchers and engineers so far, and the generalization capability of these networks needs further improving. Combining the prominent dynamic properties of a recurrent neural network (RNN) with the enhanced ability of a wavelet neural network (WNN) in mapping nonlinear functions, a diagonal recurrent wavelet neural network (DRWNN) is newly established in this paper to perform fine forecasting of hourly and daily global solar irradiance. Some additional steps, e.g. applying historical information of cloud cover to sample data sets and the cloud cover from the weather forecast to network input, are adopted to help enhance the forecast precision. Besides, a specially scheduled two phase training algorithm is adopted. As examples, both hourly and daily irradiance forecasts are completed using sample data sets in Shanghai and Macau, and comparisons between irradiation models show that the DRWNN models are definitely more accurate

  14. Encoding sensory and motor patterns as time-invariant trajectories in recurrent neural networks.

    Science.gov (United States)

    Goudar, Vishwa; Buonomano, Dean V

    2018-03-14

    Much of the information the brain processes and stores is temporal in nature-a spoken word or a handwritten signature, for example, is defined by how it unfolds in time. However, it remains unclear how neural circuits encode complex time-varying patterns. We show that by tuning the weights of a recurrent neural network (RNN), it can recognize and then transcribe spoken digits. The model elucidates how neural dynamics in cortical networks may resolve three fundamental challenges: first, encode multiple time-varying sensory and motor patterns as stable neural trajectories; second, generalize across relevant spatial features; third, identify the same stimuli played at different speeds-we show that this temporal invariance emerges because the recurrent dynamics generate neural trajectories with appropriately modulated angular velocities. Together our results generate testable predictions as to how recurrent networks may use different mechanisms to generalize across the relevant spatial and temporal features of complex time-varying stimuli. © 2018, Goudar et al.

  15. EMG-Based Estimation of Limb Movement Using Deep Learning With Recurrent Convolutional Neural Networks.

    Science.gov (United States)

    Xia, Peng; Hu, Jie; Peng, Yinghong

    2017-10-25

    A novel model based on deep learning is proposed to estimate kinematic information for myoelectric control from multi-channel electromyogram (EMG) signals. The neural information of limb movement is embedded in EMG signals that are influenced by all kinds of factors. In order to overcome the negative effects of variability in signals, the proposed model employs the deep architecture combining convolutional neural networks (CNNs) and recurrent neural networks (RNNs). The EMG signals are transformed to time-frequency frames as the input to the model. The limb movement is estimated by the model that is trained with the gradient descent and backpropagation procedure. We tested the model for simultaneous and proportional estimation of limb movement in eight healthy subjects and compared it with support vector regression (SVR) and CNNs on the same data set. The experimental studies show that the proposed model has higher estimation accuracy and better robustness with respect to time. The combination of CNNs and RNNs can improve the model performance compared with using CNNs alone. The model of deep architecture is promising in EMG decoding and optimization of network structures can increase the accuracy and robustness. © 2017 International Center for Artificial Organs and Transplantation and Wiley Periodicals, Inc.

  16. Lyapunov-based control of limit cycle oscillations in uncertain aircraft systems

    Science.gov (United States)

    Bialy, Brendan

    Store-induced limit cycle oscillations (LCO) affect several fighter aircraft and is expected to remain an issue for next generation fighters. LCO arises from the interaction of aerodynamic and structural forces, however the primary contributor to the phenomenon is still unclear. The practical concerns regarding this phenomenon include whether or not ordnance can be safely released and the ability of the aircrew to perform mission-related tasks while in an LCO condition. The focus of this dissertation is the development of control strategies to suppress LCO in aircraft systems. The first contribution of this work (Chapter 2) is the development of a controller consisting of a continuous Robust Integral of the Sign of the Error (RISE) feedback term with a neural network (NN) feedforward term to suppress LCO behavior in an uncertain airfoil system. The second contribution of this work (Chapter 3) is the extension of the development in Chapter 2 to include actuator saturation. Suppression of LCO behavior is achieved through the implementation of an auxiliary error system that features hyperbolic functions and a saturated RISE feedback control structure. Due to the lack of clarity regarding the driving mechanism behind LCO, common practice in literature and in Chapters 2 and 3 is to replicate the symptoms of LCO by including nonlinearities in the wing structure, typically a nonlinear torsional stiffness. To improve the accuracy of the system model a partial differential equation (PDE) model of a flexible wing is derived (see Appendix F) using Hamilton's principle. Chapters 4 and 5 are focused on developing boundary control strategies for regulating the bending and twisting deformations of the derived model. The contribution of Chapter 4 is the construction of a backstepping-based boundary control strategy for a linear PDE model of an aircraft wing. The backstepping-based strategy transforms the original system to a exponentially stable system. A Lyapunov-based stability

  17. Large-Scale Recurrent Neural Network Based Modelling of Gene Regulatory Network Using Cuckoo Search-Flower Pollination Algorithm.

    Science.gov (United States)

    Mandal, Sudip; Khan, Abhinandan; Saha, Goutam; Pal, Rajat K

    2016-01-01

    The accurate prediction of genetic networks using computational tools is one of the greatest challenges in the postgenomic era. Recurrent Neural Network is one of the most popular but simple approaches to model the network dynamics from time-series microarray data. To date, it has been successfully applied to computationally derive small-scale artificial and real-world genetic networks with high accuracy. However, they underperformed for large-scale genetic networks. Here, a new methodology has been proposed where a hybrid Cuckoo Search-Flower Pollination Algorithm has been implemented with Recurrent Neural Network. Cuckoo Search is used to search the best combination of regulators. Moreover, Flower Pollination Algorithm is applied to optimize the model parameters of the Recurrent Neural Network formalism. Initially, the proposed method is tested on a benchmark large-scale artificial network for both noiseless and noisy data. The results obtained show that the proposed methodology is capable of increasing the inference of correct regulations and decreasing false regulations to a high degree. Secondly, the proposed methodology has been validated against the real-world dataset of the DNA SOS repair network of Escherichia coli. However, the proposed method sacrifices computational time complexity in both cases due to the hybrid optimization process.

  18. Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition

    Directory of Open Access Journals (Sweden)

    Francisco Javier Ordóñez

    2016-01-01

    Full Text Available Human activity recognition (HAR tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we propose a generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which: (i is suitable for multimodal wearable sensors; (ii can perform sensor fusion naturally; (iii does not require expert knowledge in designing features; and (iv explicitly models the temporal dynamics of feature activations. We evaluate our framework on two datasets, one of which has been used in a public activity recognition challenge. Our results show that our framework outperforms competing deep non-recurrent networks on the challenge dataset by 4% on average; outperforming some of the previous reported results by up to 9%. Our results show that the framework can be applied to homogeneous sensor modalities, but can also fuse multimodal sensors to improve performance. We characterise key architectural hyperparameters’ influence on performance to provide insights about their optimisation.

  19. Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition.

    Science.gov (United States)

    Ordóñez, Francisco Javier; Roggen, Daniel

    2016-01-18

    Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we propose a generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which: (i) is suitable for multimodal wearable sensors; (ii) can perform sensor fusion naturally; (iii) does not require expert knowledge in designing features; and (iv) explicitly models the temporal dynamics of feature activations. We evaluate our framework on two datasets, one of which has been used in a public activity recognition challenge. Our results show that our framework outperforms competing deep non-recurrent networks on the challenge dataset by 4% on average; outperforming some of the previous reported results by up to 9%. Our results show that the framework can be applied to homogeneous sensor modalities, but can also fuse multimodal sensors to improve performance. We characterise key architectural hyperparameters' influence on performance to provide insights about their optimisation.

  20. Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition

    Science.gov (United States)

    Ordóñez, Francisco Javier; Roggen, Daniel

    2016-01-01

    Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we propose a generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which: (i) is suitable for multimodal wearable sensors; (ii) can perform sensor fusion naturally; (iii) does not require expert knowledge in designing features; and (iv) explicitly models the temporal dynamics of feature activations. We evaluate our framework on two datasets, one of which has been used in a public activity recognition challenge. Our results show that our framework outperforms competing deep non-recurrent networks on the challenge dataset by 4% on average; outperforming some of the previous reported results by up to 9%. Our results show that the framework can be applied to homogeneous sensor modalities, but can also fuse multimodal sensors to improve performance. We characterise key architectural hyperparameters’ influence on performance to provide insights about their optimisation. PMID:26797612

  1. Local Dynamics in Trained Recurrent Neural Networks.

    Science.gov (United States)

    Rivkind, Alexander; Barak, Omri

    2017-06-23

    Learning a task induces connectivity changes in neural circuits, thereby changing their dynamics. To elucidate task-related neural dynamics, we study trained recurrent neural networks. We develop a mean field theory for reservoir computing networks trained to have multiple fixed point attractors. Our main result is that the dynamics of the network's output in the vicinity of attractors is governed by a low-order linear ordinary differential equation. The stability of the resulting equation can be assessed, predicting training success or failure. As a consequence, networks of rectified linear units and of sigmoidal nonlinearities are shown to have diametrically different properties when it comes to learning attractors. Furthermore, a characteristic time constant, which remains finite at the edge of chaos, offers an explanation of the network's output robustness in the presence of variability of the internal neural dynamics. Finally, the proposed theory predicts state-dependent frequency selectivity in the network response.

  2. Local Dynamics in Trained Recurrent Neural Networks

    Science.gov (United States)

    Rivkind, Alexander; Barak, Omri

    2017-06-01

    Learning a task induces connectivity changes in neural circuits, thereby changing their dynamics. To elucidate task-related neural dynamics, we study trained recurrent neural networks. We develop a mean field theory for reservoir computing networks trained to have multiple fixed point attractors. Our main result is that the dynamics of the network's output in the vicinity of attractors is governed by a low-order linear ordinary differential equation. The stability of the resulting equation can be assessed, predicting training success or failure. As a consequence, networks of rectified linear units and of sigmoidal nonlinearities are shown to have diametrically different properties when it comes to learning attractors. Furthermore, a characteristic time constant, which remains finite at the edge of chaos, offers an explanation of the network's output robustness in the presence of variability of the internal neural dynamics. Finally, the proposed theory predicts state-dependent frequency selectivity in the network response.

  3. Global exponential stability of inertial memristor-based neural networks with time-varying delays and impulses.

    Science.gov (United States)

    Zhang, Wei; Huang, Tingwen; He, Xing; Li, Chuandong

    2017-11-01

    In this study, we investigate the global exponential stability of inertial memristor-based neural networks with impulses and time-varying delays. We construct inertial memristor-based neural networks based on the characteristics of the inertial neural networks and memristor. Impulses with and without delays are considered when modeling the inertial neural networks simultaneously, which are of great practical significance in the current study. Some sufficient conditions are derived under the framework of the Lyapunov stability method, as well as an extended Halanay differential inequality and a new delay impulsive differential inequality, which depend on impulses with and without delays, in order to guarantee the global exponential stability of the inertial memristor-based neural networks. Finally, two numerical examples are provided to illustrate the efficiency of the proposed methods. Copyright © 2017 Elsevier Ltd. All rights reserved.

  4. Lyapunov Exponents

    CERN Document Server

    Crauel, Hans; Eckmann, Jean-Pierre

    1991-01-01

    Since the predecessor to this volume (LNM 1186, Eds. L. Arnold, V. Wihstutz)appeared in 1986, significant progress has been made in the theory and applications of Lyapunov exponents - one of the key concepts of dynamical systems - and in particular, pronounced shifts towards nonlinear and infinite-dimensional systems and engineering applications are observable. This volume opens with an introductory survey article (Arnold/Crauel) followed by 26 original (fully refereed) research papers, some of which have in part survey character. From the Contents: L. Arnold, H. Crauel: Random Dynamical Systems.- I.Ya. Goldscheid: Lyapunov exponents and asymptotic behaviour of the product of random matrices.- Y. Peres: Analytic dependence of Lyapunov exponents on transition probabilities.- O. Knill: The upper Lyapunov exponent of Sl (2, R) cocycles:Discontinuity and the problem of positivity.- Yu.D. Latushkin, A.M. Stepin: Linear skew-product flows and semigroups of weighted composition operators.- P. Baxendale: Invariant me...

  5. Synchronization stability of memristor-based complex-valued neural networks with time delays.

    Science.gov (United States)

    Liu, Dan; Zhu, Song; Ye, Er

    2017-12-01

    This paper focuses on the dynamical property of a class of memristor-based complex-valued neural networks (MCVNNs) with time delays. By constructing the appropriate Lyapunov functional and utilizing the inequality technique, sufficient conditions are proposed to guarantee exponential synchronization of the coupled systems based on drive-response concept. The proposed results are very easy to verify, and they also extend some previous related works on memristor-based real-valued neural networks. Meanwhile, the obtained sufficient conditions of this paper may be conducive to qualitative analysis of some complex-valued nonlinear delayed systems. A numerical example is given to demonstrate the effectiveness of our theoretical results. Copyright © 2017 Elsevier Ltd. All rights reserved.

  6. A recurrent neural model for proto-object based contour integration and figure-ground segregation.

    Science.gov (United States)

    Hu, Brian; Niebur, Ernst

    2017-12-01

    Visual processing of objects makes use of both feedforward and feedback streams of information. However, the nature of feedback signals is largely unknown, as is the identity of the neuronal populations in lower visual areas that receive them. Here, we develop a recurrent neural model to address these questions in the context of contour integration and figure-ground segregation. A key feature of our model is the use of grouping neurons whose activity represents tentative objects ("proto-objects") based on the integration of local feature information. Grouping neurons receive input from an organized set of local feature neurons, and project modulatory feedback to those same neurons. Additionally, inhibition at both the local feature level and the object representation level biases the interpretation of the visual scene in agreement with principles from Gestalt psychology. Our model explains several sets of neurophysiological results (Zhou et al. Journal of Neuroscience, 20(17), 6594-6611 2000; Qiu et al. Nature Neuroscience, 10(11), 1492-1499 2007; Chen et al. Neuron, 82(3), 682-694 2014), and makes testable predictions about the influence of neuronal feedback and attentional selection on neural responses across different visual areas. Our model also provides a framework for understanding how object-based attention is able to select both objects and the features associated with them.

  7. New zero-input overflow stability proofs based on Lyapunov theory

    NARCIS (Netherlands)

    Werter, M.J.; Ritzerfeld, J.H.F.

    1989-01-01

    The authors demonstrate some proofs of zero-input overflow-oscillation suppression in recursive digital filters. The proofs are based on the second method of Lyapunov. For second-order digital filters with complex conjugated poles, the state describes a trajectory in the phase plane, spiraling

  8. New prediction of chaotic time series based on local Lyapunov exponent

    International Nuclear Information System (INIS)

    Zhang Yong

    2013-01-01

    A new method of predicting chaotic time series is presented based on a local Lyapunov exponent, by quantitatively measuring the exponential rate of separation or attraction of two infinitely close trajectories in state space. After reconstructing state space from one-dimensional chaotic time series, neighboring multiple-state vectors of the predicting point are selected to deduce the prediction formula by using the definition of the local Lyapunov exponent. Numerical simulations are carried out to test its effectiveness and verify its higher precision over two older methods. The effects of the number of referential state vectors and added noise on forecasting accuracy are also studied numerically. (general)

  9. Master-slave exponential synchronization of delayed complex-valued memristor-based neural networks via impulsive control.

    Science.gov (United States)

    Li, Xiaofan; Fang, Jian-An; Li, Huiyuan

    2017-09-01

    This paper investigates master-slave exponential synchronization for a class of complex-valued memristor-based neural networks with time-varying delays via discontinuous impulsive control. Firstly, the master and slave complex-valued memristor-based neural networks with time-varying delays are translated to two real-valued memristor-based neural networks. Secondly, an impulsive control law is constructed and utilized to guarantee master-slave exponential synchronization of the neural networks. Thirdly, the master-slave synchronization problems are transformed into the stability problems of the master-slave error system. By employing linear matrix inequality (LMI) technique and constructing an appropriate Lyapunov-Krasovskii functional, some sufficient synchronization criteria are derived. Finally, a numerical simulation is provided to illustrate the effectiveness of the obtained theoretical results. Copyright © 2017 Elsevier Ltd. All rights reserved.

  10. Global exponential stability of reaction-diffusion recurrent neural networks with time-varying delays

    International Nuclear Information System (INIS)

    Liang Jinling; Cao Jinde

    2003-01-01

    Employing general Halanay inequality, we analyze the global exponential stability of a class of reaction-diffusion recurrent neural networks with time-varying delays. Several new sufficient conditions are obtained to ensure existence, uniqueness and global exponential stability of the equilibrium point of delayed reaction-diffusion recurrent neural networks. The results extend and improve the earlier publications. In addition, an example is given to show the effectiveness of the obtained result

  11. Optimization of recurrent neural networks for time series modeling

    DEFF Research Database (Denmark)

    Pedersen, Morten With

    1997-01-01

    The present thesis is about optimization of recurrent neural networks applied to time series modeling. In particular is considered fully recurrent networks working from only a single external input, one layer of nonlinear hidden units and a li near output unit applied to prediction of discrete time...... series. The overall objective s are to improve training by application of second-order methods and to improve generalization ability by architecture optimization accomplished by pruning. The major topics covered in the thesis are: 1. The problem of training recurrent networks is analyzed from a numerical...... of solution obtained as well as computation time required. 3. A theoretical definition of the generalization error for recurrent networks is provided. This definition justifies a commonly adopted approach for estimating generalization ability. 4. The viability of pruning recurrent networks by the Optimal...

  12. Synaptic plasticity in a recurrent neural network for versatile and adaptive behaviors of a walking robot

    DEFF Research Database (Denmark)

    Grinke, Eduard; Tetzlaff, Christian; Wörgötter, Florentin

    2015-01-01

    correlation-based learning with synaptic scaling is applied to adequately change the connections of the network. By doing so, we can effectively exploit neural dynamics (i.e., hysteresis effects and single attractors) in the network to generate different turning angles with short-term memory for a walking...... dynamics, plasticity, sensory feedback, and biomechanics. Generating such versatile and adaptive behaviors for a many degrees-of-freedom (DOFs) walking robot is a challenging task. Thus, in this study, we present a bio-inspired approach to solve this task. Specifically, the approach combines neural...... mechanisms with plasticity, exteroceptive sensory feedback, and biomechanics. The neural mechanisms consist of adaptive neural sensory processing and modular neural locomotion control. The sensory processing is based on a small recurrent neural network consisting of two fully connected neurons. Online...

  13. Relative Lyapunov Center Bifurcations

    DEFF Research Database (Denmark)

    Wulff, Claudia; Schilder, Frank

    2014-01-01

    Relative equilibria (REs) and relative periodic orbits (RPOs) are ubiquitous in symmetric Hamiltonian systems and occur, for example, in celestial mechanics, molecular dynamics, and rigid body motion. REs are equilibria, and RPOs are periodic orbits of the symmetry reduced system. Relative Lyapunov...... center bifurcations are bifurcations of RPOs from REs corresponding to Lyapunov center bifurcations of the symmetry reduced dynamics. In this paper we first prove a relative Lyapunov center theorem by combining recent results on the persistence of RPOs in Hamiltonian systems with a symmetric Lyapunov...... center theorem of Montaldi, Roberts, and Stewart. We then develop numerical methods for the detection of relative Lyapunov center bifurcations along branches of RPOs and for their computation. We apply our methods to Lagrangian REs of the N-body problem....

  14. Interpretation of correlated neural variability from models of feed-forward and recurrent circuits

    Science.gov (United States)

    2018-01-01

    Neural populations respond to the repeated presentations of a sensory stimulus with correlated variability. These correlations have been studied in detail, with respect to their mechanistic origin, as well as their influence on stimulus discrimination and on the performance of population codes. A number of theoretical studies have endeavored to link network architecture to the nature of the correlations in neural activity. Here, we contribute to this effort: in models of circuits of stochastic neurons, we elucidate the implications of various network architectures—recurrent connections, shared feed-forward projections, and shared gain fluctuations—on the stimulus dependence in correlations. Specifically, we derive mathematical relations that specify the dependence of population-averaged covariances on firing rates, for different network architectures. In turn, these relations can be used to analyze data on population activity. We examine recordings from neural populations in mouse auditory cortex. We find that a recurrent network model with random effective connections captures the observed statistics. Furthermore, using our circuit model, we investigate the relation between network parameters, correlations, and how well different stimuli can be discriminated from one another based on the population activity. As such, our approach allows us to relate properties of the neural circuit to information processing. PMID:29408930

  15. Interpretation of correlated neural variability from models of feed-forward and recurrent circuits.

    Directory of Open Access Journals (Sweden)

    Volker Pernice

    2018-02-01

    Full Text Available Neural populations respond to the repeated presentations of a sensory stimulus with correlated variability. These correlations have been studied in detail, with respect to their mechanistic origin, as well as their influence on stimulus discrimination and on the performance of population codes. A number of theoretical studies have endeavored to link network architecture to the nature of the correlations in neural activity. Here, we contribute to this effort: in models of circuits of stochastic neurons, we elucidate the implications of various network architectures-recurrent connections, shared feed-forward projections, and shared gain fluctuations-on the stimulus dependence in correlations. Specifically, we derive mathematical relations that specify the dependence of population-averaged covariances on firing rates, for different network architectures. In turn, these relations can be used to analyze data on population activity. We examine recordings from neural populations in mouse auditory cortex. We find that a recurrent network model with random effective connections captures the observed statistics. Furthermore, using our circuit model, we investigate the relation between network parameters, correlations, and how well different stimuli can be discriminated from one another based on the population activity. As such, our approach allows us to relate properties of the neural circuit to information processing.

  16. A recurrent neural network with ever changing synapses

    NARCIS (Netherlands)

    Heerema, M.; van Leeuwen, W.A.

    2000-01-01

    A recurrent neural network with noisy input is studied analytically, on the basis of a Discrete Time Master Equation. The latter is derived from a biologically realizable learning rule for the weights of the connections. In a numerical study it is found that the fixed points of the dynamics of the

  17. Stability Analysis of Interconnected Fuzzy Systems Using the Fuzzy Lyapunov Method

    Directory of Open Access Journals (Sweden)

    Ken Yeh

    2010-01-01

    Full Text Available The fuzzy Lyapunov method is investigated for use with a class of interconnected fuzzy systems. The interconnected fuzzy systems consist of J interconnected fuzzy subsystems, and the stability analysis is based on Lyapunov functions. Based on traditional Lyapunov stability theory, we further propose a fuzzy Lyapunov method for the stability analysis of interconnected fuzzy systems. The fuzzy Lyapunov function is defined in fuzzy blending quadratic Lyapunov functions. Some stability conditions are derived through the use of fuzzy Lyapunov functions to ensure that the interconnected fuzzy systems are asymptotically stable. Common solutions can be obtained by solving a set of linear matrix inequalities (LMIs that are numerically feasible. Finally, simulations are performed in order to verify the effectiveness of the proposed stability conditions in this paper.

  18. Stability results for stochastic delayed recurrent neural networks with discrete and distributed delays

    Science.gov (United States)

    Chen, Guiling; Li, Dingshi; Shi, Lin; van Gaans, Onno; Verduyn Lunel, Sjoerd

    2018-03-01

    We present new conditions for asymptotic stability and exponential stability of a class of stochastic recurrent neural networks with discrete and distributed time varying delays. Our approach is based on the method using fixed point theory, which do not resort to any Liapunov function or Liapunov functional. Our results neither require the boundedness, monotonicity and differentiability of the activation functions nor differentiability of the time varying delays. In particular, a class of neural networks without stochastic perturbations is also considered. Examples are given to illustrate our main results.

  19. Reinforcement Learning of Linking and Tracing Contours in Recurrent Neural Networks

    Science.gov (United States)

    Brosch, Tobias; Neumann, Heiko; Roelfsema, Pieter R.

    2015-01-01

    The processing of a visual stimulus can be subdivided into a number of stages. Upon stimulus presentation there is an early phase of feedforward processing where the visual information is propagated from lower to higher visual areas for the extraction of basic and complex stimulus features. This is followed by a later phase where horizontal connections within areas and feedback connections from higher areas back to lower areas come into play. In this later phase, image elements that are behaviorally relevant are grouped by Gestalt grouping rules and are labeled in the cortex with enhanced neuronal activity (object-based attention in psychology). Recent neurophysiological studies revealed that reward-based learning influences these recurrent grouping processes, but it is not well understood how rewards train recurrent circuits for perceptual organization. This paper examines the mechanisms for reward-based learning of new grouping rules. We derive a learning rule that can explain how rewards influence the information flow through feedforward, horizontal and feedback connections. We illustrate the efficiency with two tasks that have been used to study the neuronal correlates of perceptual organization in early visual cortex. The first task is called contour-integration and demands the integration of collinear contour elements into an elongated curve. We show how reward-based learning causes an enhancement of the representation of the to-be-grouped elements at early levels of a recurrent neural network, just as is observed in the visual cortex of monkeys. The second task is curve-tracing where the aim is to determine the endpoint of an elongated curve composed of connected image elements. If trained with the new learning rule, neural networks learn to propagate enhanced activity over the curve, in accordance with neurophysiological data. We close the paper with a number of model predictions that can be tested in future neurophysiological and computational studies

  20. Stability of dynamical systems on the role of monotonic and non-monotonic Lyapunov functions

    CERN Document Server

    Michel, Anthony N; Liu, Derong

    2015-01-01

    The second edition of this textbook provides a single source for the analysis of system models represented by continuous-time and discrete-time, finite-dimensional and infinite-dimensional, and continuous and discontinuous dynamical systems.  For these system models, it presents results which comprise the classical Lyapunov stability theory involving monotonic Lyapunov functions, as well as corresponding contemporary stability results involving non-monotonicLyapunov functions.Specific examples from several diverse areas are given to demonstrate the applicability of the developed theory to many important classes of systems, including digital control systems, nonlinear regulator systems, pulse-width-modulated feedback control systems, and artificial neural networks.   The authors cover the following four general topics:   -          Representation and modeling of dynamical systems of the types described above -          Presentation of Lyapunov and Lagrange stability theory for dynamical sy...

  1. Adaptive Sliding Mode Control of MEMS Gyroscope Based on Neural Network Approximation

    Directory of Open Access Journals (Sweden)

    Yuzheng Yang

    2014-01-01

    Full Text Available An adaptive sliding controller using radial basis function (RBF network to approximate the unknown system dynamics microelectromechanical systems (MEMS gyroscope sensor is proposed. Neural controller is proposed to approximate the unknown system model and sliding controller is employed to eliminate the approximation error and attenuate the model uncertainties and external disturbances. Online neural network (NN weight tuning algorithms, including correction terms, are designed based on Lyapunov stability theory, which can guarantee bounded tracking errors as well as bounded NN weights. The tracking error bound can be made arbitrarily small by increasing a certain feedback gain. Numerical simulation for a MEMS angular velocity sensor is investigated to verify the effectiveness of the proposed adaptive neural control scheme and demonstrate the satisfactory tracking performance and robustness.

  2. Online Sequence Training of Recurrent Neural Networks with Connectionist Temporal Classification

    OpenAIRE

    Hwang, Kyuyeon; Sung, Wonyong

    2015-01-01

    Connectionist temporal classification (CTC) based supervised sequence training of recurrent neural networks (RNNs) has shown great success in many machine learning areas including end-to-end speech and handwritten character recognition. For the CTC training, however, it is required to unroll (or unfold) the RNN by the length of an input sequence. This unrolling requires a lot of memory and hinders a small footprint implementation of online learning or adaptation. Furthermore, the length of tr...

  3. Firing rate dynamics in recurrent spiking neural networks with intrinsic and network heterogeneity.

    Science.gov (United States)

    Ly, Cheng

    2015-12-01

    Heterogeneity of neural attributes has recently gained a lot of attention and is increasing recognized as a crucial feature in neural processing. Despite its importance, this physiological feature has traditionally been neglected in theoretical studies of cortical neural networks. Thus, there is still a lot unknown about the consequences of cellular and circuit heterogeneity in spiking neural networks. In particular, combining network or synaptic heterogeneity and intrinsic heterogeneity has yet to be considered systematically despite the fact that both are known to exist and likely have significant roles in neural network dynamics. In a canonical recurrent spiking neural network model, we study how these two forms of heterogeneity lead to different distributions of excitatory firing rates. To analytically characterize how these types of heterogeneities affect the network, we employ a dimension reduction method that relies on a combination of Monte Carlo simulations and probability density function equations. We find that the relationship between intrinsic and network heterogeneity has a strong effect on the overall level of heterogeneity of the firing rates. Specifically, this relationship can lead to amplification or attenuation of firing rate heterogeneity, and these effects depend on whether the recurrent network is firing asynchronously or rhythmically firing. These observations are captured with the aforementioned reduction method, and furthermore simpler analytic descriptions based on this dimension reduction method are developed. The final analytic descriptions provide compact and descriptive formulas for how the relationship between intrinsic and network heterogeneity determines the firing rate heterogeneity dynamics in various settings.

  4. Exponential stability for stochastic delayed recurrent neural networks with mixed time-varying delays and impulses: the continuous-time case

    International Nuclear Information System (INIS)

    Karthik Raja, U; Leelamani, A; Raja, R; Samidurai, R

    2013-01-01

    In this paper, the exponential stability for a class of stochastic neural networks with time-varying delays and impulsive effects is considered. By constructing suitable Lyapunov functionals and by using the linear matrix inequality optimization approach, we obtain sufficient delay-dependent criteria to ensure the exponential stability of stochastic neural networks with time-varying delays and impulses. Two numerical examples with simulation results are provided to illustrate the effectiveness of the obtained results over those already existing in the literature. (paper)

  5. A one-layer recurrent neural network for constrained nonsmooth optimization.

    Science.gov (United States)

    Liu, Qingshan; Wang, Jun

    2011-10-01

    This paper presents a novel one-layer recurrent neural network modeled by means of a differential inclusion for solving nonsmooth optimization problems, in which the number of neurons in the proposed neural network is the same as the number of decision variables of optimization problems. Compared with existing neural networks for nonsmooth optimization problems, the global convexity condition on the objective functions and constraints is relaxed, which allows the objective functions and constraints to be nonconvex. It is proven that the state variables of the proposed neural network are convergent to optimal solutions if a single design parameter in the model is larger than a derived lower bound. Numerical examples with simulation results substantiate the effectiveness and illustrate the characteristics of the proposed neural network.

  6. Robust sliding mode control for uncertain servo system using friction observer and recurrent fuzzy neural networks

    International Nuclear Information System (INIS)

    Han, Seong Ik; Jeong, Chan Se; Yang, Soon Yong

    2012-01-01

    A robust positioning control scheme has been developed using friction parameter observer and recurrent fuzzy neural networks based on the sliding mode control. As a dynamic friction model, the LuGre model is adopted for handling friction compensation because it has been known to capture sufficiently the properties of a nonlinear dynamic friction. A developed friction parameter observer has a simple structure and also well estimates friction parameters of the LuGre friction model. In addition, an approximation method for the system uncertainty is developed using recurrent fuzzy neural networks technology to improve the precision positioning degree. Some simulation and experiment provide the verification on the performance of a proposed robust control scheme

  7. Robust sliding mode control for uncertain servo system using friction observer and recurrent fuzzy neural networks

    Energy Technology Data Exchange (ETDEWEB)

    Han, Seong Ik [Pusan National University, Busan (Korea, Republic of); Jeong, Chan Se; Yang, Soon Yong [University of Ulsan, Ulsan (Korea, Republic of)

    2012-04-15

    A robust positioning control scheme has been developed using friction parameter observer and recurrent fuzzy neural networks based on the sliding mode control. As a dynamic friction model, the LuGre model is adopted for handling friction compensation because it has been known to capture sufficiently the properties of a nonlinear dynamic friction. A developed friction parameter observer has a simple structure and also well estimates friction parameters of the LuGre friction model. In addition, an approximation method for the system uncertainty is developed using recurrent fuzzy neural networks technology to improve the precision positioning degree. Some simulation and experiment provide the verification on the performance of a proposed robust control scheme.

  8. A Recurrent Neural Network for Nonlinear Fractional Programming

    Directory of Open Access Journals (Sweden)

    Quan-Ju Zhang

    2012-01-01

    Full Text Available This paper presents a novel recurrent time continuous neural network model which performs nonlinear fractional optimization subject to interval constraints on each of the optimization variables. The network is proved to be complete in the sense that the set of optima of the objective function to be minimized with interval constraints coincides with the set of equilibria of the neural network. It is also shown that the network is primal and globally convergent in the sense that its trajectory cannot escape from the feasible region and will converge to an exact optimal solution for any initial point being chosen in the feasible interval region. Simulation results are given to demonstrate further the global convergence and good performance of the proposing neural network for nonlinear fractional programming problems with interval constraints.

  9. Brain Dynamics in Predicting Driving Fatigue Using a Recurrent Self-Evolving Fuzzy Neural Network.

    Science.gov (United States)

    Liu, Yu-Ting; Lin, Yang-Yin; Wu, Shang-Lin; Chuang, Chun-Hsiang; Lin, Chin-Teng

    2016-02-01

    This paper proposes a generalized prediction system called a recurrent self-evolving fuzzy neural network (RSEFNN) that employs an on-line gradient descent learning rule to address the electroencephalography (EEG) regression problem in brain dynamics for driving fatigue. The cognitive states of drivers significantly affect driving safety; in particular, fatigue driving, or drowsy driving, endangers both the individual and the public. For this reason, the development of brain-computer interfaces (BCIs) that can identify drowsy driving states is a crucial and urgent topic of study. Many EEG-based BCIs have been developed as artificial auxiliary systems for use in various practical applications because of the benefits of measuring EEG signals. In the literature, the efficacy of EEG-based BCIs in recognition tasks has been limited by low resolutions. The system proposed in this paper represents the first attempt to use the recurrent fuzzy neural network (RFNN) architecture to increase adaptability in realistic EEG applications to overcome this bottleneck. This paper further analyzes brain dynamics in a simulated car driving task in a virtual-reality environment. The proposed RSEFNN model is evaluated using the generalized cross-subject approach, and the results indicate that the RSEFNN is superior to competing models regardless of the use of recurrent or nonrecurrent structures.

  10. Singularity-Free Neural Control for the Exponential Trajectory Tracking in Multiple-Input Uncertain Systems with Unknown Deadzone Nonlinearities

    Directory of Open Access Journals (Sweden)

    J. Humberto Pérez-Cruz

    2014-01-01

    Full Text Available The trajectory tracking for a class of uncertain nonlinear systems in which the number of possible states is equal to the number of inputs and each input is preceded by an unknown symmetric deadzone is considered. The unknown dynamics is identified by means of a continuous time recurrent neural network in which the control singularity is conveniently avoided by guaranteeing the invertibility of the coupling matrix. Given this neural network-based mathematical model of the uncertain system, a singularity-free feedback linearization control law is developed in order to compel the system state to follow a reference trajectory. By means of Lyapunov-like analysis, the exponential convergence of the tracking error to a bounded zone can be proven. Likewise, the boundedness of all closed-loop signals can be guaranteed.

  11. pth moment exponential stability of stochastic memristor-based bidirectional associative memory (BAM) neural networks with time delays.

    Science.gov (United States)

    Wang, Fen; Chen, Yuanlong; Liu, Meichun

    2018-02-01

    Stochastic memristor-based bidirectional associative memory (BAM) neural networks with time delays play an increasingly important role in the design and implementation of neural network systems. Under the framework of Filippov solutions, the issues of the pth moment exponential stability of stochastic memristor-based BAM neural networks are investigated. By using the stochastic stability theory, Itô's differential formula and Young inequality, the criteria are derived. Meanwhile, with Lyapunov approach and Cauchy-Schwarz inequality, we derive some sufficient conditions for the mean square exponential stability of the above systems. The obtained results improve and extend previous works on memristor-based or usual neural networks dynamical systems. Four numerical examples are provided to illustrate the effectiveness of the proposed results. Copyright © 2017 Elsevier Ltd. All rights reserved.

  12. Different-Level Simultaneous Minimization Scheme for Fault Tolerance of Redundant Manipulator Aided with Discrete-Time Recurrent Neural Network.

    Science.gov (United States)

    Jin, Long; Liao, Bolin; Liu, Mei; Xiao, Lin; Guo, Dongsheng; Yan, Xiaogang

    2017-01-01

    By incorporating the physical constraints in joint space, a different-level simultaneous minimization scheme, which takes both the robot kinematics and robot dynamics into account, is presented and investigated for fault-tolerant motion planning of redundant manipulator in this paper. The scheme is reformulated as a quadratic program (QP) with equality and bound constraints, which is then solved by a discrete-time recurrent neural network. Simulative verifications based on a six-link planar redundant robot manipulator substantiate the efficacy and accuracy of the presented acceleration fault-tolerant scheme, the resultant QP and the corresponding discrete-time recurrent neural network.

  13. Tuning Recurrent Neural Networks for Recognizing Handwritten Arabic Words

    KAUST Repository

    Qaralleh, Esam

    2013-10-01

    Artificial neural networks have the abilities to learn by example and are capable of solving problems that are hard to solve using ordinary rule-based programming. They have many design parameters that affect their performance such as the number and sizes of the hidden layers. Large sizes are slow and small sizes are generally not accurate. Tuning the neural network size is a hard task because the design space is often large and training is often a long process. We use design of experiments techniques to tune the recurrent neural network used in an Arabic handwriting recognition system. We show that best results are achieved with three hidden layers and two subsampling layers. To tune the sizes of these five layers, we use fractional factorial experiment design to limit the number of experiments to a feasible number. Moreover, we replicate the experiment configuration multiple times to overcome the randomness in the training process. The accuracy and time measurements are analyzed and modeled. The two models are then used to locate network sizes that are on the Pareto optimal frontier. The approach described in this paper reduces the label error from 26.2% to 19.8%.

  14. Quantum synchronization in an optomechanical system based on Lyapunov control.

    Science.gov (United States)

    Li, Wenlin; Li, Chong; Song, Heshan

    2016-06-01

    We extend the concepts of quantum complete synchronization and phase synchronization, which were proposed in A. Mari et al., Phys. Rev. Lett. 111, 103605 (2013)PRLTAO0031-900710.1103/PhysRevLett.111.103605, to more widespread quantum generalized synchronization. Generalized synchronization can be considered a necessary condition or a more flexible derivative of complete synchronization, and its criterion and synchronization measure are proposed and analyzed in this paper. As examples, we consider two typical generalized synchronizations in a designed optomechanical system. Unlike the effort to construct a special coupling synchronization system, we purposefully design extra control fields based on Lyapunov control theory. We find that the Lyapunov function can adapt to more flexible control objectives, which is more suitable for generalized synchronization control, and the control fields can be achieved simply with a time-variant voltage. Finally, the existence of quantum entanglement in different generalized synchronizations is also discussed.

  15. H∞ state estimation of stochastic memristor-based neural networks with time-varying delays.

    Science.gov (United States)

    Bao, Haibo; Cao, Jinde; Kurths, Jürgen; Alsaedi, Ahmed; Ahmad, Bashir

    2018-03-01

    This paper addresses the problem of H ∞ state estimation for a class of stochastic memristor-based neural networks with time-varying delays. Under the framework of Filippov solution, the stochastic memristor-based neural networks are transformed into systems with interval parameters. The present paper is the first to investigate the H ∞ state estimation problem for continuous-time Itô-type stochastic memristor-based neural networks. By means of Lyapunov functionals and some stochastic technique, sufficient conditions are derived to ensure that the estimation error system is asymptotically stable in the mean square with a prescribed H ∞ performance. An explicit expression of the state estimator gain is given in terms of linear matrix inequalities (LMIs). Compared with other results, our results reduce control gain and control cost effectively. Finally, numerical simulations are provided to demonstrate the efficiency of the theoretical results. Copyright © 2018 Elsevier Ltd. All rights reserved.

  16. Fine-tuning and the stability of recurrent neural networks.

    Directory of Open Access Journals (Sweden)

    David MacNeil

    Full Text Available A central criticism of standard theoretical approaches to constructing stable, recurrent model networks is that the synaptic connection weights need to be finely-tuned. This criticism is severe because proposed rules for learning these weights have been shown to have various limitations to their biological plausibility. Hence it is unlikely that such rules are used to continuously fine-tune the network in vivo. We describe a learning rule that is able to tune synaptic weights in a biologically plausible manner. We demonstrate and test this rule in the context of the oculomotor integrator, showing that only known neural signals are needed to tune the weights. We demonstrate that the rule appropriately accounts for a wide variety of experimental results, and is robust under several kinds of perturbation. Furthermore, we show that the rule is able to achieve stability as good as or better than that provided by the linearly optimal weights often used in recurrent models of the integrator. Finally, we discuss how this rule can be generalized to tune a wide variety of recurrent attractor networks, such as those found in head direction and path integration systems, suggesting that it may be used to tune a wide variety of stable neural systems.

  17. A One-Layer Recurrent Neural Network for Real-Time Portfolio Optimization With Probability Criterion.

    Science.gov (United States)

    Liu, Qingshan; Dang, Chuangyin; Huang, Tingwen

    2013-02-01

    This paper presents a decision-making model described by a recurrent neural network for dynamic portfolio optimization. The portfolio-optimization problem is first converted into a constrained fractional programming problem. Since the objective function in the programming problem is not convex, the traditional optimization techniques are no longer applicable for solving this problem. Fortunately, the objective function in the fractional programming is pseudoconvex on the feasible region. It leads to a one-layer recurrent neural network modeled by means of a discontinuous dynamic system. To ensure the optimal solutions for portfolio optimization, the convergence of the proposed neural network is analyzed and proved. In fact, the neural network guarantees to get the optimal solutions for portfolio-investment advice if some mild conditions are satisfied. A numerical example with simulation results substantiates the effectiveness and illustrates the characteristics of the proposed neural network.

  18. Analysis of recurrent neural networks for short-term energy load forecasting

    Science.gov (United States)

    Di Persio, Luca; Honchar, Oleksandr

    2017-11-01

    Short-term forecasts have recently gained an increasing attention because of the rise of competitive electricity markets. In fact, short-terms forecast of possible future loads turn out to be fundamental to build efficient energy management strategies as well as to avoid energy wastage. Such type of challenges are difficult to tackle both from a theoretical and applied point of view. Latter tasks require sophisticated methods to manage multidimensional time series related to stochastic phenomena which are often highly interconnected. In the present work we first review novel approaches to energy load forecasting based on recurrent neural network, focusing our attention on long/short term memory architectures (LSTMs). Such type of artificial neural networks have been widely applied to problems dealing with sequential data such it happens, e.g., in socio-economics settings, for text recognition purposes, concerning video signals, etc., always showing their effectiveness to model complex temporal data. Moreover, we consider different novel variations of basic LSTMs, such as sequence-to-sequence approach and bidirectional LSTMs, aiming at providing effective models for energy load data. Last but not least, we test all the described algorithms on real energy load data showing not only that deep recurrent networks can be successfully applied to energy load forecasting, but also that this approach can be extended to other problems based on time series prediction.

  19. Bayesian model ensembling using meta-trained recurrent neural networks

    NARCIS (Netherlands)

    Ambrogioni, L.; Berezutskaya, Y.; Gü ç lü , U.; Borne, E.W.P. van den; Gü ç lü tü rk, Y.; Gerven, M.A.J. van; Maris, E.G.G.

    2017-01-01

    In this paper we demonstrate that a recurrent neural network meta-trained on an ensemble of arbitrary classification tasks can be used as an approximation of the Bayes optimal classifier. This result is obtained by relying on the framework of e-free approximate Bayesian inference, where the Bayesian

  20. Identification of Jets Containing b-Hadrons with Recurrent Neural Networks at the ATLAS Experiment

    CERN Multimedia

    CERN. Geneva

    2017-01-01

    A novel b-jet identification algorithm is constructed with a Recurrent Neural Network (RNN) at the ATLAS Experiment. This talk presents the expected performance of the RNN based b-tagging in simulated $t \\bar t$ events. The RNN based b-tagging processes properties of tracks associated to jets which are represented in sequences. In contrast to traditional impact-parameter-based b-tagging algorithms which assume the tracks of jets are independent from each other, RNN based b-tagging can exploit the spatial and kinematic correlations of tracks which are initiated from the same b-hadrons. The neural network nature of the tagging algorithm also allows the flexibility of extending input features to include more track properties than can be effectively used in traditional algorithms.

  1. Recurrent neural network based hybrid model for reconstructing gene regulatory network.

    Science.gov (United States)

    Raza, Khalid; Alam, Mansaf

    2016-10-01

    One of the exciting problems in systems biology research is to decipher how genome controls the development of complex biological system. The gene regulatory networks (GRNs) help in the identification of regulatory interactions between genes and offer fruitful information related to functional role of individual gene in a cellular system. Discovering GRNs lead to a wide range of applications, including identification of disease related pathways providing novel tentative drug targets, helps to predict disease response, and also assists in diagnosing various diseases including cancer. Reconstruction of GRNs from available biological data is still an open problem. This paper proposes a recurrent neural network (RNN) based model of GRN, hybridized with generalized extended Kalman filter for weight update in backpropagation through time training algorithm. The RNN is a complex neural network that gives a better settlement between biological closeness and mathematical flexibility to model GRN; and is also able to capture complex, non-linear and dynamic relationships among variables. Gene expression data are inherently noisy and Kalman filter performs well for estimation problem even in noisy data. Hence, we applied non-linear version of Kalman filter, known as generalized extended Kalman filter, for weight update during RNN training. The developed model has been tested on four benchmark networks such as DNA SOS repair network, IRMA network, and two synthetic networks from DREAM Challenge. We performed a comparison of our results with other state-of-the-art techniques which shows superiority of our proposed model. Further, 5% Gaussian noise has been induced in the dataset and result of the proposed model shows negligible effect of noise on results, demonstrating the noise tolerance capability of the model. Copyright © 2016 Elsevier Ltd. All rights reserved.

  2. Bi-directional LSTM Recurrent Neural Network for Chinese Word Segmentation

    OpenAIRE

    Yao, Yushi; Huang, Zheng

    2016-01-01

    Recurrent neural network(RNN) has been broadly applied to natural language processing(NLP) problems. This kind of neural network is designed for modeling sequential data and has been testified to be quite efficient in sequential tagging tasks. In this paper, we propose to use bi-directional RNN with long short-term memory(LSTM) units for Chinese word segmentation, which is a crucial preprocess task for modeling Chinese sentences and articles. Classical methods focus on designing and combining...

  3. The attractor recurrent neural network based on fuzzy functions: An effective model for the classification of lung abnormalities.

    Science.gov (United States)

    Khodabakhshi, Mohammad Bagher; Moradi, Mohammad Hassan

    2017-05-01

    The respiratory system dynamic is of high significance when it comes to the detection of lung abnormalities, which highlights the importance of presenting a reliable model for it. In this paper, we introduce a novel dynamic modelling method for the characterization of the lung sounds (LS), based on the attractor recurrent neural network (ARNN). The ARNN structure allows the development of an effective LS model. Additionally, it has the capability to reproduce the distinctive features of the lung sounds using its formed attractors. Furthermore, a novel ARNN topology based on fuzzy functions (FFs-ARNN) is developed. Given the utility of the recurrent quantification analysis (RQA) as a tool to assess the nature of complex systems, it was used to evaluate the performance of both the ARNN and the FFs-ARNN models. The experimental results demonstrate the effectiveness of the proposed approaches for multichannel LS analysis. In particular, a classification accuracy of 91% was achieved using FFs-ARNN with sequences of RQA features. Copyright © 2017 Elsevier Ltd. All rights reserved.

  4. Lyapunov based control of hybrid energy storage system in electric vehicles

    DEFF Research Database (Denmark)

    El Fadil, H.; Giri, F.; Guerrero, Josep M.

    2012-01-01

    This paper deals with a Lyapunov based control principle in a hybrid energy storage system for electric vehicle. The storage system consists on fuel cell (FC) as a main power source and a supercapacitor (SC) as an auxiliary power source. The power stage of energy conversion consists on a boost...

  5. Neural Network-Based Resistance Spot Welding Control and Quality Prediction

    Energy Technology Data Exchange (ETDEWEB)

    Allen, J.D., Jr.; Ivezic, N.D.; Zacharia, T.

    1999-07-10

    This paper describes the development and evaluation of neural network-based systems for industrial resistance spot welding process control and weld quality assessment. The developed systems utilize recurrent neural networks for process control and both recurrent networks and static networks for quality prediction. The first section describes a system capable of both welding process control and real-time weld quality assessment, The second describes the development and evaluation of a static neural network-based weld quality assessment system that relied on experimental design to limit the influence of environmental variability. Relevant data analysis methods are also discussed. The weld classifier resulting from the analysis successfldly balances predictive power and simplicity of interpretation. The results presented for both systems demonstrate clearly that neural networks can be employed to address two significant problems common to the resistance spot welding industry, control of the process itself, and non-destructive determination of resulting weld quality.

  6. Robust nonlinear autoregressive moving average model parameter estimation using stochastic recurrent artificial neural networks

    DEFF Research Database (Denmark)

    Chon, K H; Hoyer, D; Armoundas, A A

    1999-01-01

    In this study, we introduce a new approach for estimating linear and nonlinear stochastic autoregressive moving average (ARMA) model parameters, given a corrupt signal, using artificial recurrent neural networks. This new approach is a two-step approach in which the parameters of the deterministic...... part of the stochastic ARMA model are first estimated via a three-layer artificial neural network (deterministic estimation step) and then reestimated using the prediction error as one of the inputs to the artificial neural networks in an iterative algorithm (stochastic estimation step). The prediction...... error is obtained by subtracting the corrupt signal of the estimated ARMA model obtained via the deterministic estimation step from the system output response. We present computer simulation examples to show the efficacy of the proposed stochastic recurrent neural network approach in obtaining accurate...

  7. A One-Layer Recurrent Neural Network for Pseudoconvex Optimization Problems With Equality and Inequality Constraints.

    Science.gov (United States)

    Qin, Sitian; Yang, Xiudong; Xue, Xiaoping; Song, Jiahui

    2017-10-01

    Pseudoconvex optimization problem, as an important nonconvex optimization problem, plays an important role in scientific and engineering applications. In this paper, a recurrent one-layer neural network is proposed for solving the pseudoconvex optimization problem with equality and inequality constraints. It is proved that from any initial state, the state of the proposed neural network reaches the feasible region in finite time and stays there thereafter. It is also proved that the state of the proposed neural network is convergent to an optimal solution of the related problem. Compared with the related existing recurrent neural networks for the pseudoconvex optimization problems, the proposed neural network in this paper does not need the penalty parameters and has a better convergence. Meanwhile, the proposed neural network is used to solve three nonsmooth optimization problems, and we make some detailed comparisons with the known related conclusions. In the end, some numerical examples are provided to illustrate the effectiveness of the performance of the proposed neural network.

  8. Existence and global exponential stability of periodic solution of memristor-based BAM neural networks with time-varying delays.

    Science.gov (United States)

    Li, Hongfei; Jiang, Haijun; Hu, Cheng

    2016-03-01

    In this paper, we investigate a class of memristor-based BAM neural networks with time-varying delays. Under the framework of Filippov solutions, boundedness and ultimate boundedness of solutions of memristor-based BAM neural networks are guaranteed by Chain rule and inequalities technique. Moreover, a new method involving Yoshizawa-like theorem is favorably employed to acquire the existence of periodic solution. By applying the theory of set-valued maps and functional differential inclusions, an available Lyapunov functional and some new testable algebraic criteria are derived for ensuring the uniqueness and global exponential stability of periodic solution of memristor-based BAM neural networks. The obtained results expand and complement some previous work on memristor-based BAM neural networks. Finally, a numerical example is provided to show the applicability and effectiveness of our theoretical results. Copyright © 2015 Elsevier Ltd. All rights reserved.

  9. Analysis of Recurrent Analog Neural Networks

    Directory of Open Access Journals (Sweden)

    Z. Raida

    1998-06-01

    Full Text Available In this paper, an original rigorous analysis of recurrent analog neural networks, which are built from opamp neurons, is presented. The analysis, which comes from the approximate model of the operational amplifier, reveals causes of possible non-stable states and enables to determine convergence properties of the network. Results of the analysis are discussed in order to enable development of original robust and fast analog networks. In the analysis, the special attention is turned to the examination of the influence of real circuit elements and of the statistical parameters of processed signals to the parameters of the network.

  10. Bifurcation analysis on a generalized recurrent neural network with two interconnected three-neuron components

    International Nuclear Information System (INIS)

    Hajihosseini, Amirhossein; Maleki, Farzaneh; Rokni Lamooki, Gholam Reza

    2011-01-01

    Highlights: → We construct a recurrent neural network by generalizing a specific n-neuron network. → Several codimension 1 and 2 bifurcations take place in the newly constructed network. → The newly constructed network has higher capabilities to learn periodic signals. → The normal form theorem is applied to investigate dynamics of the network. → A series of bifurcation diagrams is given to support theoretical results. - Abstract: A class of recurrent neural networks is constructed by generalizing a specific class of n-neuron networks. It is shown that the newly constructed network experiences generic pitchfork and Hopf codimension one bifurcations. It is also proved that the emergence of generic Bogdanov-Takens, pitchfork-Hopf and Hopf-Hopf codimension two, and the degenerate Bogdanov-Takens bifurcation points in the parameter space is possible due to the intersections of codimension one bifurcation curves. The occurrence of bifurcations of higher codimensions significantly increases the capability of the newly constructed recurrent neural network to learn broader families of periodic signals.

  11. Lyapunov-Based Control Scheme for Single-Phase Grid-Connected PV Central Inverters

    NARCIS (Netherlands)

    Meza, C.; Biel, D.; Jeltsema, D.; Scherpen, J. M. A.

    A Lyapunov-based control scheme for single-phase single-stage grid-connected photovoltaic central inverters is presented. Besides rendering the closed-loop system globally stable, the designed controller is able to deal with the system uncertainty that depends on the solar irradiance. A laboratory

  12. Probing the basins of attraction of a recurrent neural network

    NARCIS (Netherlands)

    Heerema, M.; van Leeuwen, W.A.

    2000-01-01

    Analytical expressions for the weights $w_{ij}(b)$ of the connections of a recurrent neural network are found by taking explicitly into account basins of attraction, the size of which is characterized by a basin parameter $b$. It is shown that a network with $b \

  13. Predicting local field potentials with recurrent neural networks.

    Science.gov (United States)

    Kim, Louis; Harer, Jacob; Rangamani, Akshay; Moran, James; Parks, Philip D; Widge, Alik; Eskandar, Emad; Dougherty, Darin; Chin, Sang Peter

    2016-08-01

    We present a Recurrent Neural Network using LSTM (Long Short Term Memory) that is capable of modeling and predicting Local Field Potentials. We train and test the network on real data recorded from epilepsy patients. We construct networks that predict multi-channel LFPs for 1, 10, and 100 milliseconds forward in time. Our results show that prediction using LSTM outperforms regression when predicting 10 and 100 millisecond forward in time.

  14. Hysteretic recurrent neural networks: a tool for modeling hysteretic materials and systems

    International Nuclear Information System (INIS)

    Veeramani, Arun S; Crews, John H; Buckner, Gregory D

    2009-01-01

    This paper introduces a novel recurrent neural network, the hysteretic recurrent neural network (HRNN), that is ideally suited to modeling hysteretic materials and systems. This network incorporates a hysteretic neuron consisting of conjoined sigmoid activation functions. Although similar hysteretic neurons have been explored previously, the HRNN is unique in its utilization of simple recurrence to 'self-select' relevant activation functions. Furthermore, training is facilitated by placing the network weights on the output side, allowing standard backpropagation of error training algorithms to be used. We present two- and three-phase versions of the HRNN for modeling hysteretic materials with distinct phases. These models are experimentally validated using data collected from shape memory alloys and ferromagnetic materials. The results demonstrate the HRNN's ability to accurately generalize hysteretic behavior with a relatively small number of neurons. Additional benefits lie in the network's ability to identify statistical information concerning the macroscopic material by analyzing the weights of the individual neurons

  15. Attention-based Memory Selection Recurrent Network for Language Modeling

    OpenAIRE

    Liu, Da-Rong; Chuang, Shun-Po; Lee, Hung-yi

    2016-01-01

    Recurrent neural networks (RNNs) have achieved great success in language modeling. However, since the RNNs have fixed size of memory, their memory cannot store all the information about the words it have seen before in the sentence, and thus the useful long-term information may be ignored when predicting the next words. In this paper, we propose Attention-based Memory Selection Recurrent Network (AMSRN), in which the model can review the information stored in the memory at each previous time ...

  16. Encoding Time in Feedforward Trajectories of a Recurrent Neural Network Model.

    Science.gov (United States)

    Hardy, N F; Buonomano, Dean V

    2018-02-01

    Brain activity evolves through time, creating trajectories of activity that underlie sensorimotor processing, behavior, and learning and memory. Therefore, understanding the temporal nature of neural dynamics is essential to understanding brain function and behavior. In vivo studies have demonstrated that sequential transient activation of neurons can encode time. However, it remains unclear whether these patterns emerge from feedforward network architectures or from recurrent networks and, furthermore, what role network structure plays in timing. We address these issues using a recurrent neural network (RNN) model with distinct populations of excitatory and inhibitory units. Consistent with experimental data, a single RNN could autonomously produce multiple functionally feedforward trajectories, thus potentially encoding multiple timed motor patterns lasting up to several seconds. Importantly, the model accounted for Weber's law, a hallmark of timing behavior. Analysis of network connectivity revealed that efficiency-a measure of network interconnectedness-decreased as the number of stored trajectories increased. Additionally, the balance of excitation (E) and inhibition (I) shifted toward excitation during each unit's activation time, generating the prediction that observed sequential activity relies on dynamic control of the E/I balance. Our results establish for the first time that the same RNN can generate multiple functionally feedforward patterns of activity as a result of dynamic shifts in the E/I balance imposed by the connectome of the RNN. We conclude that recurrent network architectures account for sequential neural activity, as well as for a fundamental signature of timing behavior: Weber's law.

  17. Internal representation of task rules by recurrent dynamics: the importance of the diversity of neural responses

    Directory of Open Access Journals (Sweden)

    Mattia Rigotti

    2010-10-01

    Full Text Available Neural activity of behaving animals, especially in the prefrontal cortex, is highly heterogeneous, with selective responses to diverse aspects of the executed task. We propose a general model of recurrent neural networks that perform complex rule-based tasks, and we show that the diversity of neuronal responses plays a fundamental role when the behavioral responses are context dependent. Specifically, we found that when the inner mental states encoding the task rules are represented by stable patterns of neural activity (attractors of the neural dynamics, the neurons must be selective for combinations of sensory stimuli and inner mental states. Such mixed selectivity is easily obtained by neurons that connect with random synaptic strengths both to the recurrent network and to neurons encoding sensory inputs. The number of randomly connected neurons needed to solve a task is on average only three times as large as the number of neurons needed in a network designed ad hoc. Moreover, the number of needed neurons grows only linearly with the number of task-relevant events and mental states, provided that each neuron responds to a large proportion of events (dense/distributed coding. A biologically realistic implementation of the model captures several aspects of the activity recorded from monkeys performing context dependent tasks. Our findings explain the importance of the diversity of neural responses and provide us with simple and general principles for designing attractor neural networks that perform complex computation.

  18. DeepProbe: Information Directed Sequence Understanding and Chatbot Design via Recurrent Neural Networks

    OpenAIRE

    Yin, Zi; Chang, Keng-hao; Zhang, Ruofei

    2017-01-01

    Information extraction and user intention identification are central topics in modern query understanding and recommendation systems. In this paper, we propose DeepProbe, a generic information-directed interaction framework which is built around an attention-based sequence to sequence (seq2seq) recurrent neural network. DeepProbe can rephrase, evaluate, and even actively ask questions, leveraging the generative ability and likelihood estimation made possible by seq2seq models. DeepProbe makes...

  19. Bach in 2014: Music Composition with Recurrent Neural Network

    OpenAIRE

    Liu, I-Ting; Ramakrishnan, Bhiksha

    2014-01-01

    We propose a framework for computer music composition that uses resilient propagation (RProp) and long short term memory (LSTM) recurrent neural network. In this paper, we show that LSTM network learns the structure and characteristics of music pieces properly by demonstrating its ability to recreate music. We also show that predicting existing music using RProp outperforms Back propagation through time (BPTT).

  20. Predicting non-linear dynamics by stable local learning in a recurrent spiking neural network.

    Science.gov (United States)

    Gilra, Aditya; Gerstner, Wulfram

    2017-11-27

    The brain needs to predict how the body reacts to motor commands, but how a network of spiking neurons can learn non-linear body dynamics using local, online and stable learning rules is unclear. Here, we present a supervised learning scheme for the feedforward and recurrent connections in a network of heterogeneous spiking neurons. The error in the output is fed back through fixed random connections with a negative gain, causing the network to follow the desired dynamics. The rule for Feedback-based Online Local Learning Of Weights (FOLLOW) is local in the sense that weight changes depend on the presynaptic activity and the error signal projected onto the postsynaptic neuron. We provide examples of learning linear, non-linear and chaotic dynamics, as well as the dynamics of a two-link arm. Under reasonable approximations, we show, using the Lyapunov method, that FOLLOW learning is uniformly stable, with the error going to zero asymptotically.

  1. Containment control of networked autonomous underwater vehicles: A predictor-based neural DSC design.

    Science.gov (United States)

    Peng, Zhouhua; Wang, Dan; Wang, Wei; Liu, Lu

    2015-11-01

    This paper investigates the containment control problem of networked autonomous underwater vehicles in the presence of model uncertainty and unknown ocean disturbances. A predictor-based neural dynamic surface control design method is presented to develop the distributed adaptive containment controllers, under which the trajectories of follower vehicles nearly converge to the dynamic convex hull spanned by multiple reference trajectories over a directed network. Prediction errors, rather than tracking errors, are used to update the neural adaptation laws, which are independent of the tracking error dynamics, resulting in two time-scales to govern the entire system. The stability property of the closed-loop network is established via Lyapunov analysis, and transient property is quantified in terms of L2 norms of the derivatives of neural weights, which are shown to be smaller than the classical neural dynamic surface control approach. Comparative studies are given to show the substantial improvements of the proposed new method. Copyright © 2015 ISA. Published by Elsevier Ltd. All rights reserved.

  2. Using a multi-state recurrent neural network to optimize loading patterns in BWRs

    International Nuclear Information System (INIS)

    Ortiz, Juan Jose; Requena, Ignacio

    2004-01-01

    A Multi-State Recurrent Neural Network is used to optimize Loading Patterns (LP) in BWRs. We have proposed an energy function that depends on fuel assembly positions and their nuclear cross sections to carry out optimisation. Multi-State Recurrent Neural Networks creates LPs that satisfy the Radial Power Peaking Factor and maximize the effective multiplication factor at the Beginning of the Cycle, and also satisfy the Minimum Critical Power Ratio and Maximum Linear Heat Generation Rate at the End of the Cycle, thereby maximizing the effective multiplication factor. In order to evaluate the LPs, we have used a trained back-propagation neural network to predict the parameter values, instead of using a reactor core simulator, which saved considerable computation time in the search process. We applied this method to find optimal LPs for five cycles of Laguna Verde Nuclear Power Plant (LVNPP) in Mexico

  3. Deep Recurrent Neural Networks for Human Activity Recognition

    Directory of Open Access Journals (Sweden)

    Abdulmajid Murad

    2017-11-01

    Full Text Available Adopting deep learning methods for human activity recognition has been effective in extracting discriminative features from raw input sequences acquired from body-worn sensors. Although human movements are encoded in a sequence of successive samples in time, typical machine learning methods perform recognition tasks without exploiting the temporal correlations between input data samples. Convolutional neural networks (CNNs address this issue by using convolutions across a one-dimensional temporal sequence to capture dependencies among input data. However, the size of convolutional kernels restricts the captured range of dependencies between data samples. As a result, typical models are unadaptable to a wide range of activity-recognition configurations and require fixed-length input windows. In this paper, we propose the use of deep recurrent neural networks (DRNNs for building recognition models that are capable of capturing long-range dependencies in variable-length input sequences. We present unidirectional, bidirectional, and cascaded architectures based on long short-term memory (LSTM DRNNs and evaluate their effectiveness on miscellaneous benchmark datasets. Experimental results show that our proposed models outperform methods employing conventional machine learning, such as support vector machine (SVM and k-nearest neighbors (KNN. Additionally, the proposed models yield better performance than other deep learning techniques, such as deep believe networks (DBNs and CNNs.

  4. Deep Recurrent Neural Networks for Human Activity Recognition.

    Science.gov (United States)

    Murad, Abdulmajid; Pyun, Jae-Young

    2017-11-06

    Adopting deep learning methods for human activity recognition has been effective in extracting discriminative features from raw input sequences acquired from body-worn sensors. Although human movements are encoded in a sequence of successive samples in time, typical machine learning methods perform recognition tasks without exploiting the temporal correlations between input data samples. Convolutional neural networks (CNNs) address this issue by using convolutions across a one-dimensional temporal sequence to capture dependencies among input data. However, the size of convolutional kernels restricts the captured range of dependencies between data samples. As a result, typical models are unadaptable to a wide range of activity-recognition configurations and require fixed-length input windows. In this paper, we propose the use of deep recurrent neural networks (DRNNs) for building recognition models that are capable of capturing long-range dependencies in variable-length input sequences. We present unidirectional, bidirectional, and cascaded architectures based on long short-term memory (LSTM) DRNNs and evaluate their effectiveness on miscellaneous benchmark datasets. Experimental results show that our proposed models outperform methods employing conventional machine learning, such as support vector machine (SVM) and k-nearest neighbors (KNN). Additionally, the proposed models yield better performance than other deep learning techniques, such as deep believe networks (DBNs) and CNNs.

  5. From phonemes to images : levels of representation in a recurrent neural model of visually-grounded language learning

    NARCIS (Netherlands)

    Gelderloos, L.J.; Chrupala, Grzegorz

    2016-01-01

    We present a model of visually-grounded language learning based on stacked gated recurrent neural networks which learns to predict visual features given an image description in the form of a sequence of phonemes. The learning task resembles that faced by human language learners who need to discover

  6. Identification of serial number on bank card using recurrent neural network

    Science.gov (United States)

    Liu, Li; Huang, Linlin; Xue, Jian

    2018-04-01

    Identification of serial number on bank card has many applications. Due to the different number printing mode, complex background, distortion in shape, etc., it is quite challenging to achieve high identification accuracy. In this paper, we propose a method using Normalization-Cooperated Gradient Feature (NCGF) and Recurrent Neural Network (RNN) based on Long Short-Term Memory (LSTM) for serial number identification. The NCGF maps the gradient direction elements of original image to direction planes such that the RNN with direction planes as input can recognize numbers more accurately. Taking the advantages of NCGF and RNN, we get 90%digit string recognition accuracy.

  7. Precipitation Nowcast using Deep Recurrent Neural Network

    Science.gov (United States)

    Akbari Asanjan, A.; Yang, T.; Gao, X.; Hsu, K. L.; Sorooshian, S.

    2016-12-01

    An accurate precipitation nowcast (0-6 hours) with a fine temporal and spatial resolution has always been an important prerequisite for flood warning, streamflow prediction and risk management. Most of the popular approaches used for forecasting precipitation can be categorized into two groups. One type of precipitation forecast relies on numerical modeling of the physical dynamics of atmosphere and another is based on empirical and statistical regression models derived by local hydrologists or meteorologists. Given the recent advances in artificial intelligence, in this study a powerful Deep Recurrent Neural Network, termed as Long Short-Term Memory (LSTM) model, is creatively used to extract the patterns and forecast the spatial and temporal variability of Cloud Top Brightness Temperature (CTBT) observed from GOES satellite. Then, a 0-6 hours precipitation nowcast is produced using a Precipitation Estimation from Remote Sensing Information using Artificial Neural Network (PERSIANN) algorithm, in which the CTBT nowcast is used as the PERSIANN algorithm's raw inputs. Two case studies over the continental U.S. have been conducted that demonstrate the improvement of proposed approach as compared to a classical Feed Forward Neural Network and a couple simple regression models. The advantages and disadvantages of the proposed method are summarized with regard to its capability of pattern recognition through time, handling of vanishing gradient during model learning, and working with sparse data. The studies show that the LSTM model performs better than other methods, and it is able to learn the temporal evolution of the precipitation events through over 1000 time lags. The uniqueness of PERSIANN's algorithm enables an alternative precipitation nowcast approach as demonstrated in this study, in which the CTBT prediction is produced and used as the inputs for generating precipitation nowcast.

  8. Fuzzy wavelet plus a quantum neural network as a design base for power system stability enhancement.

    Science.gov (United States)

    Ganjefar, Soheil; Tofighi, Morteza; Karami, Hamidreza

    2015-11-01

    In this study, we introduce an indirect adaptive fuzzy wavelet neural controller (IAFWNC) as a power system stabilizer to damp inter-area modes of oscillations in a multi-machine power system. Quantum computing is an efficient method for improving the computational efficiency of neural networks, so we developed an identifier based on a quantum neural network (QNN) to train the IAFWNC in the proposed scheme. All of the controller parameters are tuned online based on the Lyapunov stability theory to guarantee the closed-loop stability. A two-machine, two-area power system equipped with a static synchronous series compensator as a series flexible ac transmission system was used to demonstrate the effectiveness of the proposed controller. The simulation and experimental results demonstrated that the proposed IAFWNC scheme can achieve favorable control performance. Copyright © 2015 Elsevier Ltd. All rights reserved.

  9. Some new results on stability and synchronization for delayed inertial neural networks based on non-reduced order method.

    Science.gov (United States)

    Li, Xuanying; Li, Xiaotong; Hu, Cheng

    2017-12-01

    In this paper, without transforming the second order inertial neural networks into the first order differential systems by some variable substitutions, asymptotic stability and synchronization for a class of delayed inertial neural networks are investigated. Firstly, a new Lyapunov functional is constructed to directly propose the asymptotic stability of the inertial neural networks, and some new stability criteria are derived by means of Barbalat Lemma. Additionally, by designing a new feedback control strategy, the asymptotic synchronization of the addressed inertial networks is studied and some effective conditions are obtained. To reduce the control cost, an adaptive control scheme is designed to realize the asymptotic synchronization. It is noted that the dynamical behaviors of inertial neural networks are directly analyzed in this paper by constructing some new Lyapunov functionals, this is totally different from the traditional reduced-order variable substitution method. Finally, some numerical simulations are given to demonstrate the effectiveness of the derived theoretical results. Copyright © 2017 Elsevier Ltd. All rights reserved.

  10. Large-Signal Lyapunov-Based Stability Analysis of DC/AC Inverters and Inverter-Based Microgrids

    Science.gov (United States)

    Kabalan, Mahmoud

    Microgrid stability studies have been largely based on small-signal linearization techniques. However, the validity and magnitude of the linearization domain is limited to small perturbations. Thus, there is a need to examine microgrids with large-signal nonlinear techniques to fully understand and examine their stability. Large-signal stability analysis can be accomplished by Lyapunov-based mathematical methods. These Lyapunov methods estimate the domain of asymptotic stability of the studied system. A survey of Lyapunov-based large-signal stability studies showed that few large-signal studies have been completed on either individual systems (dc/ac inverters, dc/dc rectifiers, etc.) or microgrids. The research presented in this thesis addresses the large-signal stability of droop-controlled dc/ac inverters and inverter-based microgrids. Dc/ac power electronic inverters allow microgrids to be technically feasible. Thus, as a prelude to examining the stability of microgrids, the research presented in Chapter 3 analyzes the stability of inverters. First, the 13 th order large-signal nonlinear model of a droop-controlled dc/ac inverter connected to an infinite bus is presented. The singular perturbation method is used to decompose the nonlinear model into 11th, 9th, 7th, 5th, 3rd and 1st order models. Each model ignores certain control or structural components of the full order model. The aim of the study is to understand the accuracy and validity of the reduced order models in replicating the performance of the full order nonlinear model. The performance of each model is studied in three different areas: time domain simulations, Lyapunov's indirect method and domain of attraction estimation. The work aims to present the best model to use in each of the three domains of study. Results show that certain reduced order models are capable of accurately reproducing the performance of the full order model while others can be used to gain insights into those three areas of

  11. Time-delay effects and simplified control fields in quantum Lyapunov control

    International Nuclear Information System (INIS)

    Yi, X X; Wu, S L; Wu, Chunfeng; Feng, X L; Oh, C H

    2011-01-01

    Lyapunov-based quantum control has the advantage that it is free from the measurement-induced decoherence and it includes the instantaneous information of the system in the control. The Lyapunov control is often confronted with time delay in the control fields and difficulty in practical implementations of the control. In this paper, we study the effect of time delay on the Lyapunov control and explore the possibility of replacing the control field with a pulse train or a bang-bang signal. The efficiency of the Lyapunov control is also presented through examining the convergence time of the system. These results suggest that the Lyapunov control is robust against time delay, easy to realize and effective for high-dimensional quantum systems.

  12. Prediction of Bladder Cancer Recurrences Using Artificial Neural Networks

    Science.gov (United States)

    Zulueta Guerrero, Ekaitz; Garay, Naiara Telleria; Lopez-Guede, Jose Manuel; Vilches, Borja Ayerdi; Iragorri, Eider Egilegor; Castaños, David Lecumberri; de La Hoz Rastrollo, Ana Belén; Peña, Carlos Pertusa

    Even if considerable advances have been made in the field of early diagnosis, there is no simple, cheap and non-invasive method that can be applied to the clinical monitorisation of bladder cancer patients. Moreover, bladder cancer recurrences or the reappearance of the tumour after its surgical resection cannot be predicted in the current clinical setting. In this study, Artificial Neural Networks (ANN) were used to assess how different combinations of classical clinical parameters (stage-grade and age) and two urinary markers (growth factor and pro-inflammatory mediator) could predict post surgical recurrences in bladder cancer patients. Different ANN methods, input parameter combinations and recurrence related output variables were used and the resulting positive and negative prediction rates compared. MultiLayer Perceptron (MLP) was selected as the most predictive model and urinary markers showed the highest sensitivity, predicting correctly 50% of the patients that would recur in a 2 year follow-up period.

  13. Stability analysis for cellular neural networks with variable delays

    International Nuclear Information System (INIS)

    Zhang Qiang; Wei Xiaopeng; Xu Jin

    2006-01-01

    Some sufficient conditions for the global exponential stability of cellular neural networks with variable delay are obtained by means of a method based on delay differential inequality. The method, which does not make use of Lyapunov functionals, is simple and effective for the stability analysis of neural networks with delay. Some previously established results in the literature are shown to be special cases of the presented result

  14. Lyapunov, attractors and exponents

    International Nuclear Information System (INIS)

    Oliveira, C.R. de.

    1987-01-01

    Based on the fundamental principles of statistical mechanics and ergodic theory a definition is given to atractor, as an invariant measure. Many results which reinforce this definition are demonstrated. Chaos is related to the presence of an atractor with entropy above zero. The role of Lyapunov exponents is analyzed. (A.C.A.S.) [pt

  15. Nonlinear Lyapunov-based boundary control of distributed heat transfer mechanisms in membrane distillation plant

    KAUST Repository

    Eleiwi, Fadi; Laleg-Kirati, Taous-Meriem

    2015-01-01

    This paper presents a nonlinear Lyapunov-based boundary control for the temperature difference of a membrane distillation boundary layers. The heat transfer mechanisms inside the process are modeled with a 2D advection-diffusion equation. The model

  16. Web server's reliability improvements using recurrent neural networks

    DEFF Research Database (Denmark)

    Madsen, Henrik; Albu, Rǎzvan-Daniel; Felea, Ioan

    2012-01-01

    In this paper we describe an interesting approach to error prediction illustrated by experimental results. The application consists of monitoring the activity for the web servers in order to collect the specific data. Predicting an error with severe consequences for the performance of a server (t...... usage, network usage and memory usage. We collect different data sets from monitoring the web server's activity and for each one we predict the server's reliability with the proposed recurrent neural network. © 2012 Taylor & Francis Group...

  17. A survey of quantum Lyapunov control methods.

    Science.gov (United States)

    Cong, Shuang; Meng, Fangfang

    2013-01-01

    The condition of a quantum Lyapunov-based control which can be well used in a closed quantum system is that the method can make the system convergent but not just stable. In the convergence study of the quantum Lyapunov control, two situations are classified: nondegenerate cases and degenerate cases. For these two situations, respectively, in this paper the target state is divided into four categories: the eigenstate, the mixed state which commutes with the internal Hamiltonian, the superposition state, and the mixed state which does not commute with the internal Hamiltonian. For these four categories, the quantum Lyapunov control methods for the closed quantum systems are summarized and analyzed. Particularly, the convergence of the control system to the different target states is reviewed, and how to make the convergence conditions be satisfied is summarized and analyzed.

  18. Passivity analysis of memristor-based impulsive inertial neural networks with time-varying delays.

    Science.gov (United States)

    Wan, Peng; Jian, Jigui

    2018-03-01

    This paper focuses on delay-dependent passivity analysis for a class of memristive impulsive inertial neural networks with time-varying delays. By choosing proper variable transformation, the memristive inertial neural networks can be rewritten as first-order differential equations. The memristive model presented here is regarded as a switching system rather than employing the theory of differential inclusion and set-value map. Based on matrix inequality and Lyapunov-Krasovskii functional method, several delay-dependent passivity conditions are obtained to ascertain the passivity of the addressed networks. In addition, the results obtained here contain those on the passivity for the addressed networks without impulse effects as special cases and can also be generalized to other neural networks with more complex pulse interference. Finally, one numerical example is presented to show the validity of the obtained results. Copyright © 2018 ISA. Published by Elsevier Ltd. All rights reserved.

  19. Global robust exponential stability analysis for interval recurrent neural networks

    International Nuclear Information System (INIS)

    Xu Shengyuan; Lam, James; Ho, Daniel W.C.; Zou Yun

    2004-01-01

    This Letter investigates the problem of robust global exponential stability analysis for interval recurrent neural networks (RNNs) via the linear matrix inequality (LMI) approach. The values of the time-invariant uncertain parameters are assumed to be bounded within given compact sets. An improved condition for the existence of a unique equilibrium point and its global exponential stability of RNNs with known parameters is proposed. Based on this, a sufficient condition for the global robust exponential stability for interval RNNs is obtained. Both of the conditions are expressed in terms of LMIs, which can be checked easily by various recently developed convex optimization algorithms. Examples are provided to demonstrate the reduced conservatism of the proposed exponential stability condition

  20. CloudScan - A Configuration-Free Invoice Analysis System Using Recurrent Neural Networks

    DEFF Research Database (Denmark)

    Palm, Rasmus Berg; Winther, Ole; Laws, Florian

    2017-01-01

    We present CloudScan; an invoice analysis system that requires zero configuration or upfront annotation. In contrast to previous work, CloudScan does not rely on templates of invoice layout, instead it learns a single global model of invoices that naturally generalizes to unseen invoice layouts....... The model is trained using data automatically extracted from end-user provided feedback. This automatic training data extraction removes the requirement for users to annotate the data precisely. We describe a recurrent neural network model that can capture long range context and compare it to a baseline...... logistic regression model corresponding to the current CloudScan production system. We train and evaluate the system on 8 important fields using a dataset of 326,471 invoices. The recurrent neural network and baseline model achieve 0.891 and 0.887 average F1 scores respectively on seen invoice layouts...

  1. Learning to Generate Sequences with Combination of Hebbian and Non-hebbian Plasticity in Recurrent Spiking Neural Networks.

    Science.gov (United States)

    Panda, Priyadarshini; Roy, Kaushik

    2017-01-01

    Synaptic Plasticity, the foundation for learning and memory formation in the human brain, manifests in various forms. Here, we combine the standard spike timing correlation based Hebbian plasticity with a non-Hebbian synaptic decay mechanism for training a recurrent spiking neural model to generate sequences. We show that inclusion of the adaptive decay of synaptic weights with standard STDP helps learn stable contextual dependencies between temporal sequences, while reducing the strong attractor states that emerge in recurrent models due to feedback loops. Furthermore, we show that the combined learning scheme suppresses the chaotic activity in the recurrent model substantially, thereby enhancing its' ability to generate sequences consistently even in the presence of perturbations.

  2. Encoding of phonology in a recurrent neural model of grounded speech

    NARCIS (Netherlands)

    Alishahi, Afra; Barking, Marie; Chrupala, Grzegorz; Levy, Roger; Specia, Lucia

    2017-01-01

    We study the representation and encoding of phonemes in a recurrent neural network model of grounded speech. We use a model which processes images and their spoken descriptions, and projects the visual and auditory representations into the same semantic space. We perform a number of analyses on how

  3. Detection of nonstationary transition to synchronized states of a neural network using recurrence analyses

    Science.gov (United States)

    Budzinski, R. C.; Boaretto, B. R. R.; Prado, T. L.; Lopes, S. R.

    2017-07-01

    We study the stability of asymptotic states displayed by a complex neural network. We focus on the loss of stability of a stationary state of networks using recurrence quantifiers as tools to diagnose local and global stabilities as well as the multistability of a coupled neural network. Numerical simulations of a neural network composed of 1024 neurons in a small-world connection scheme are performed using the model of Braun et al. [Int. J. Bifurcation Chaos 08, 881 (1998), 10.1142/S0218127498000681], which is a modified model from the Hodgkin-Huxley model [J. Phys. 117, 500 (1952)]. To validate the analyses, the results are compared with those produced by Kuramoto's order parameter [Chemical Oscillations, Waves, and Turbulence (Springer-Verlag, Berlin Heidelberg, 1984)]. We show that recurrence tools making use of just integrated signals provided by the networks, such as local field potential (LFP) (LFP signals) or mean field values bring new results on the understanding of neural behavior occurring before the synchronization states. In particular we show the occurrence of different stationary and nonstationarity asymptotic states.

  4. Classification of conductance traces with recurrent neural networks

    Science.gov (United States)

    Lauritzen, Kasper P.; Magyarkuti, András; Balogh, Zoltán; Halbritter, András; Solomon, Gemma C.

    2018-02-01

    We present a new automated method for structural classification of the traces obtained in break junction experiments. Using recurrent neural networks trained on the traces of minimal cross-sectional area in molecular dynamics simulations, we successfully separate the traces into two classes: point contact or nanowire. This is done without any assumptions about the expected features of each class. The trained neural network is applied to experimental break junction conductance traces, and it separates the classes as well as the previously used experimental methods. The effect of using partial conductance traces is explored, and we show that the method performs equally well using full or partial traces (as long as the trace just prior to breaking is included). When only the initial part of the trace is included, the results are still better than random chance. Finally, we show that the neural network classification method can be used to classify experimental conductance traces without using simulated results for training, but instead training the network on a few representative experimental traces. This offers a tool to recognize some characteristic motifs of the traces, which can be hard to find by simple data selection algorithms.

  5. Adaptive control of nonlinear in parameters chaotic system via Lyapunov exponents placement

    Energy Technology Data Exchange (ETDEWEB)

    Ayati, Moosa [Department of Electrical Engineering, K.N. Toosi University of Technology, Sayyed Khandan Bridge, Shariati Street, Tehran (Iran, Islamic Republic of)], E-mail: Ayati@dena.kntu.ac.ir; Khaki-Sedigh, Ali [Department of Electrical Engineering, K.N. Toosi University of Technology, Sayyed Khandan Bridge, Shariati Street, Tehran (Iran, Islamic Republic of)], E-mail: sedigh@kntu.ac.ir

    2009-08-30

    This paper proposes a new method for the adaptive control of nonlinear in parameters (NLP) chaotic systems. A method based on Lagrangian of a cost function is used to identify the parameters of the system. Estimation results are used to calculate the Lyapunov exponents adaptively. Finally, the Lyapunov exponents placement method is used to assign the desired Lyapunov exponents of the closed loop system.

  6. Adaptive control of nonlinear in parameters chaotic system via Lyapunov exponents placement

    International Nuclear Information System (INIS)

    Ayati, Moosa; Khaki-Sedigh, Ali

    2009-01-01

    This paper proposes a new method for the adaptive control of nonlinear in parameters (NLP) chaotic systems. A method based on Lagrangian of a cost function is used to identify the parameters of the system. Estimation results are used to calculate the Lyapunov exponents adaptively. Finally, the Lyapunov exponents placement method is used to assign the desired Lyapunov exponents of the closed loop system.

  7. A sampling approach to constructing Lyapunov functions for nonlinear continuous–time systems

    NARCIS (Netherlands)

    Bobiti, R.V.; Lazar, M.

    2016-01-01

    The problem of constructing a Lyapunov function for continuous-time nonlinear dynamical systems is tackled in this paper via a sampling-based approach. The main idea of the sampling-based method is to verify a Lyapunov-type inequality for a finite number of points (known state vectors) in the

  8. A Novel Recurrent Neural Network for Manipulator Control With Improved Noise Tolerance.

    Science.gov (United States)

    Li, Shuai; Wang, Huanqing; Rafique, Muhammad Usman

    2017-04-12

    In this paper, we propose a novel recurrent neural network to resolve the redundancy of manipulators for efficient kinematic control in the presence of noises in a polynomial type. Leveraging the high-order derivative properties of polynomial noises, a deliberately devised neural network is proposed to eliminate the impact of noises and recover the accurate tracking of desired trajectories in workspace. Rigorous analysis shows that the proposed neural law stabilizes the system dynamics and the position tracking error converges to zero in the presence of noises. Extensive simulations verify the theoretical results. Numerical comparisons show that existing dual neural solutions lose stability when exposed to large constant noises or time-varying noises. In contrast, the proposed approach works well and has a low tracking error comparable to noise-free situations.

  9. Using genetic programming to find Lyapunov functions

    NARCIS (Netherlands)

    Soute, I.A.C.; Molengraft, van de M.J.G.; Angelis, G.Z.; Ryan, C; Spector, L.

    2001-01-01

    In this paper Genetic Programming is used to find Lyapunov functions for (non)linear dif ferential equations of autonomous systems. As Lyapunov functions can be difficult to find, we use OP to make the decisions concerning the form of the Lyapunov function. As an e5cample two systems are taken to

  10. Multi-stability and almost periodic solutions of a class of recurrent neural networks

    International Nuclear Information System (INIS)

    Liu Yiguang; You Zhisheng

    2007-01-01

    This paper studies multi-stability, existence of almost periodic solutions of a class of recurrent neural networks with bounded activation functions. After introducing a sufficient condition insuring multi-stability, many criteria guaranteeing existence of almost periodic solutions are derived using Mawhin's coincidence degree theory. All the criteria are constructed without assuming the activation functions are smooth, monotonic or Lipschitz continuous, and that the networks contains periodic variables (such as periodic coefficients, periodic inputs or periodic activation functions), so all criteria can be easily extended to fit many concrete forms of neural networks such as Hopfield neural networks, or cellular neural networks, etc. Finally, all kinds of simulations are employed to illustrate the criteria

  11. Improving protein disorder prediction by deep bidirectional long short-term memory recurrent neural networks.

    Science.gov (United States)

    Hanson, Jack; Yang, Yuedong; Paliwal, Kuldip; Zhou, Yaoqi

    2017-03-01

    Capturing long-range interactions between structural but not sequence neighbors of proteins is a long-standing challenging problem in bioinformatics. Recently, long short-term memory (LSTM) networks have significantly improved the accuracy of speech and image classification problems by remembering useful past information in long sequential events. Here, we have implemented deep bidirectional LSTM recurrent neural networks in the problem of protein intrinsic disorder prediction. The new method, named SPOT-Disorder, has steadily improved over a similar method using a traditional, window-based neural network (SPINE-D) in all datasets tested without separate training on short and long disordered regions. Independent tests on four other datasets including the datasets from critical assessment of structure prediction (CASP) techniques and >10 000 annotated proteins from MobiDB, confirmed SPOT-Disorder as one of the best methods in disorder prediction. Moreover, initial studies indicate that the method is more accurate in predicting functional sites in disordered regions. These results highlight the usefulness combining LSTM with deep bidirectional recurrent neural networks in capturing non-local, long-range interactions for bioinformatics applications. SPOT-disorder is available as a web server and as a standalone program at: http://sparks-lab.org/server/SPOT-disorder/index.php . j.hanson@griffith.edu.au or yuedong.yang@griffith.edu.au or yaoqi.zhou@griffith.edu.au. Supplementary data is available at Bioinformatics online. © The Author 2016. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com

  12. Contemporary deep recurrent learning for recognition

    Science.gov (United States)

    Iftekharuddin, K. M.; Alam, M.; Vidyaratne, L.

    2017-05-01

    Large-scale feed-forward neural networks have seen intense application in many computer vision problems. However, these networks can get hefty and computationally intensive with increasing complexity of the task. Our work, for the first time in literature, introduces a Cellular Simultaneous Recurrent Network (CSRN) based hierarchical neural network for object detection. CSRN has shown to be more effective to solving complex tasks such as maze traversal and image processing when compared to generic feed forward networks. While deep neural networks (DNN) have exhibited excellent performance in object detection and recognition, such hierarchical structure has largely been absent in neural networks with recurrency. Further, our work introduces deep hierarchy in SRN for object recognition. The simultaneous recurrency results in an unfolding effect of the SRN through time, potentially enabling the design of an arbitrarily deep network. This paper shows experiments using face, facial expression and character recognition tasks using novel deep recurrent model and compares recognition performance with that of generic deep feed forward model. Finally, we demonstrate the flexibility of incorporating our proposed deep SRN based recognition framework in a humanoid robotic platform called NAO.

  13. Recurrent Neural Networks for Multivariate Time Series with Missing Values.

    Science.gov (United States)

    Che, Zhengping; Purushotham, Sanjay; Cho, Kyunghyun; Sontag, David; Liu, Yan

    2018-04-17

    Multivariate time series data in practical applications, such as health care, geoscience, and biology, are characterized by a variety of missing values. In time series prediction and other related tasks, it has been noted that missing values and their missing patterns are often correlated with the target labels, a.k.a., informative missingness. There is very limited work on exploiting the missing patterns for effective imputation and improving prediction performance. In this paper, we develop novel deep learning models, namely GRU-D, as one of the early attempts. GRU-D is based on Gated Recurrent Unit (GRU), a state-of-the-art recurrent neural network. It takes two representations of missing patterns, i.e., masking and time interval, and effectively incorporates them into a deep model architecture so that it not only captures the long-term temporal dependencies in time series, but also utilizes the missing patterns to achieve better prediction results. Experiments of time series classification tasks on real-world clinical datasets (MIMIC-III, PhysioNet) and synthetic datasets demonstrate that our models achieve state-of-the-art performance and provide useful insights for better understanding and utilization of missing values in time series analysis.

  14. A new wind power prediction method based on chaotic theory and Bernstein Neural Network

    International Nuclear Information System (INIS)

    Wang, Cong; Zhang, Hongli; Fan, Wenhui; Fan, Xiaochao

    2016-01-01

    The accuracy of wind power prediction is important for assessing the security and economy of the system operation when wind power connects to the grids. However, multiple factors cause a long delay and large errors in wind power prediction. Hence, efficient wind power forecasting approaches are still required for practical applications. In this paper, a new wind power forecasting method based on Chaos Theory and Bernstein Neural Network (BNN) is proposed. Firstly, the largest Lyapunov exponent as a judgment for wind power system's chaotic behavior is made. Secondly, Phase Space Reconstruction (PSR) is used to reconstruct the wind power series' phase space. Thirdly, the prediction model is constructed using the Bernstein polynomial and neural network. Finally, the weights and thresholds of the model are optimized by Primal Dual State Transition Algorithm (PDSTA). The practical hourly data of wind power generation in Xinjiang is used to test this forecaster. The proposed forecaster is compared with several current prominent research findings. Analytical results indicate that the forecasting error of PDSTA + BNN is 3.893% for 24 look-ahead hours, and has lower errors obtained compared with the other forecast methods discussed in this paper. The results of all cases studying confirm the validity of the new forecast method. - Highlights: • Lyapunov exponent is used to verify chaotic behavior of wind power series. • Phase Space Reconstruction is used to reconstruct chaotic wind power series. • A new Bernstein Neural Network to predict wind power series is proposed. • Primal dual state transition algorithm is chosen as the training strategy of BNN.

  15. Analysis of Multiple Structural Changes in Financial Contagion Based on the Largest Lyapunov Exponents

    Directory of Open Access Journals (Sweden)

    Rui Wang

    2014-01-01

    Full Text Available A modified multiple structural changes model is built to test structural breaks of the financial system based on calculating the largest Lyapunov exponents of the financial time series. Afterwards, the Lorenz system is used as a simulation example to inspect the new model. As the Lorenz system has strong nonlinearity, the verification results show that the new model has good capability in both finding the breakpoint and revealing the changes in nonlinear characteristics of the time series. The empirical study based on the model used daily data from the S&P 500 stock index during the global financial crisis from 2005 to 2012. The results provide four breakpoints of the period, which divide the contagion into four stages: stationary, local outbreak, global outbreak, and recovery period. An additional significant result is the obvious chaos characteristic difference in the largest Lyapunov exponents and the standard deviation at various stages, particularly at the local outbreak stage.

  16. LYAPUNOV-Based Sensor Failure Detection and Recovery for the Reverse Water Gas Shift Process

    Science.gov (United States)

    Haralambous, Michael G.

    2002-01-01

    Livingstone, a model-based AI software system, is planned for use in the autonomous fault diagnosis, reconfiguration, and control of the oxygen-producing reverse water gas shift (RWGS) process test-bed located in the Applied Chemistry Laboratory at KSC. In this report the RWGS process is first briefly described and an overview of Livingstone is given. Next, a Lyapunov-based approach for detecting and recovering from sensor failures, differing significantly from that used by Livingstone, is presented. In this new method, models used are in t e m of the defining differential equations of system components, thus differing from the qualitative, static models used by Livingstone. An easily computed scalar inequality constraint, expressed in terms of sensed system variables, is used to determine the existence of sensor failures. In the event of sensor failure, an observer/estimator is used for determining which sensors have failed. The theory underlying the new approach is developed. Finally, a recommendation is made to use the Lyapunov-based approach to complement the capability of Livingstone and to use this combination in the RWGS process.

  17. Centralized and decentralized global outer-synchronization of asymmetric recurrent time-varying neural network by data-sampling.

    Science.gov (United States)

    Lu, Wenlian; Zheng, Ren; Chen, Tianping

    2016-03-01

    In this paper, we discuss outer-synchronization of the asymmetrically connected recurrent time-varying neural networks. By using both centralized and decentralized discretization data sampling principles, we derive several sufficient conditions based on three vector norms to guarantee that the difference of any two trajectories starting from different initial values of the neural network converges to zero. The lower bounds of the common time intervals between data samples in centralized and decentralized principles are proved to be positive, which guarantees exclusion of Zeno behavior. A numerical example is provided to illustrate the efficiency of the theoretical results. Copyright © 2015 Elsevier Ltd. All rights reserved.

  18. Bilinear Approximate Model-Based Robust Lyapunov Control for Parabolic Distributed Collectors

    KAUST Repository

    Elmetennani, Shahrazed

    2016-11-09

    This brief addresses the control problem of distributed parabolic solar collectors in order to maintain the field outlet temperature around a desired level. The objective is to design an efficient controller to force the outlet fluid temperature to track a set reference despite the unpredictable varying working conditions. In this brief, a bilinear model-based robust Lyapunov control is proposed to achieve the control objectives with robustness to the environmental changes. The bilinear model is a reduced order approximate representation of the solar collector, which is derived from the hyperbolic distributed equation describing the heat transport dynamics by means of a dynamical Gaussian interpolation. Using the bilinear approximate model, a robust control strategy is designed applying Lyapunov stability theory combined with a phenomenological representation of the system in order to stabilize the tracking error. On the basis of the error analysis, simulation results show good performance of the proposed controller, in terms of tracking accuracy and convergence time, with limited measurement even under unfavorable working conditions. Furthermore, the presented work is of interest for a large category of dynamical systems knowing that the solar collector is representative of physical systems involving transport phenomena constrained by unknown external disturbances.

  19. Drawing and Recognizing Chinese Characters with Recurrent Neural Network.

    Science.gov (United States)

    Zhang, Xu-Yao; Yin, Fei; Zhang, Yan-Ming; Liu, Cheng-Lin; Bengio, Yoshua

    2018-04-01

    Recent deep learning based approaches have achieved great success on handwriting recognition. Chinese characters are among the most widely adopted writing systems in the world. Previous research has mainly focused on recognizing handwritten Chinese characters. However, recognition is only one aspect for understanding a language, another challenging and interesting task is to teach a machine to automatically write (pictographic) Chinese characters. In this paper, we propose a framework by using the recurrent neural network (RNN) as both a discriminative model for recognizing Chinese characters and a generative model for drawing (generating) Chinese characters. To recognize Chinese characters, previous methods usually adopt the convolutional neural network (CNN) models which require transforming the online handwriting trajectory into image-like representations. Instead, our RNN based approach is an end-to-end system which directly deals with the sequential structure and does not require any domain-specific knowledge. With the RNN system (combining an LSTM and GRU), state-of-the-art performance can be achieved on the ICDAR-2013 competition database. Furthermore, under the RNN framework, a conditional generative model with character embedding is proposed for automatically drawing recognizable Chinese characters. The generated characters (in vector format) are human-readable and also can be recognized by the discriminative RNN model with high accuracy. Experimental results verify the effectiveness of using RNNs as both generative and discriminative models for the tasks of drawing and recognizing Chinese characters.

  20. Simultaneous multichannel signal transfers via chaos in a recurrent neural network.

    Science.gov (United States)

    Soma, Ken-ichiro; Mori, Ryota; Sato, Ryuichi; Furumai, Noriyuki; Nara, Shigetoshi

    2015-05-01

    We propose neural network model that demonstrates the phenomenon of signal transfer between separated neuron groups via other chaotic neurons that show no apparent correlations with the input signal. The model is a recurrent neural network in which it is supposed that synchronous behavior between small groups of input and output neurons has been learned as fragments of high-dimensional memory patterns, and depletion of neural connections results in chaotic wandering dynamics. Computer experiments show that when a strong oscillatory signal is applied to an input group in the chaotic regime, the signal is successfully transferred to the corresponding output group, although no correlation is observed between the input signal and the intermediary neurons. Signal transfer is also observed when multiple signals are applied simultaneously to separate input groups belonging to different memory attractors. In this sense simultaneous multichannel communications are realized, and the chaotic neural dynamics acts as a signal transfer medium in which the signal appears to be hidden.

  1. Adaptive nonlinear control using input normalized neural networks

    International Nuclear Information System (INIS)

    Leeghim, Henzeh; Seo, In Ho; Bang, Hyo Choong

    2008-01-01

    An adaptive feedback linearization technique combined with the neural network is addressed to control uncertain nonlinear systems. The neural network-based adaptive control theory has been widely studied. However, the stability analysis of the closed-loop system with the neural network is rather complicated and difficult to understand, and sometimes unnecessary assumptions are involved. As a result, unnecessary assumptions for stability analysis are avoided by using the neural network with input normalization technique. The ultimate boundedness of the tracking error is simply proved by the Lyapunov stability theory. A new simple update law as an adaptive nonlinear control is derived by the simplification of the input normalized neural network assuming the variation of the uncertain term is sufficiently small

  2. Sequence-specific bias correction for RNA-seq data using recurrent neural networks.

    Science.gov (United States)

    Zhang, Yao-Zhong; Yamaguchi, Rui; Imoto, Seiya; Miyano, Satoru

    2017-01-25

    The recent success of deep learning techniques in machine learning and artificial intelligence has stimulated a great deal of interest among bioinformaticians, who now wish to bring the power of deep learning to bare on a host of bioinformatical problems. Deep learning is ideally suited for biological problems that require automatic or hierarchical feature representation for biological data when prior knowledge is limited. In this work, we address the sequence-specific bias correction problem for RNA-seq data redusing Recurrent Neural Networks (RNNs) to model nucleotide sequences without pre-determining sequence structures. The sequence-specific bias of a read is then calculated based on the sequence probabilities estimated by RNNs, and used in the estimation of gene abundance. We explore the application of two popular RNN recurrent units for this task and demonstrate that RNN-based approaches provide a flexible way to model nucleotide sequences without knowledge of predetermined sequence structures. Our experiments show that training a RNN-based nucleotide sequence model is efficient and RNN-based bias correction methods compare well with the-state-of-the-art sequence-specific bias correction method on the commonly used MAQC-III data set. RNNs provides an alternative and flexible way to calculate sequence-specific bias without explicitly pre-determining sequence structures.

  3. Toward a new task assignment and path evolution (TAPE) for missile defense system (MDS) using intelligent adaptive SOM with recurrent neural networks (RNNs).

    Science.gov (United States)

    Wang, Chi-Hsu; Chen, Chun-Yao; Hung, Kun-Neng

    2015-06-01

    In this paper, a new adaptive self-organizing map (SOM) with recurrent neural network (RNN) controller is proposed for task assignment and path evolution of missile defense system (MDS). We address the problem of N agents (defending missiles) and D targets (incoming missiles) in MDS. A new RNN controller is designed to force an agent (or defending missile) toward a target (or incoming missile), and a monitoring controller is also designed to reduce the error between RNN controller and ideal controller. A new SOM with RNN controller is then designed to dispatch agents to their corresponding targets by minimizing total damaging cost. This is actually an important application of the multiagent system. The SOM with RNN controller is the main controller. After task assignment, the weighting factors of our new SOM with RNN controller are activated to dispatch the agents toward their corresponding targets. Using the Lyapunov constraints, the weighting factors for the proposed SOM with RNN controller are updated to guarantee the stability of the path evolution (or planning) system. Excellent simulations are obtained using this new approach for MDS, which show that our RNN has the lowest average miss distance among the several techniques.

  4. Global stability of discrete-time recurrent neural networks with impulse effects

    International Nuclear Information System (INIS)

    Zhou, L; Li, C; Wan, J

    2008-01-01

    This paper formulates and studies a class of discrete-time recurrent neural networks with impulse effects. A stability criterion, which characterizes the effects of impulse and stability property of the corresponding impulse-free networks on the stability of the impulsive networks in an aggregate form, is established. Two simplified and numerically tractable criteria are also provided

  5. Sensorless control for permanent magnet synchronous motor using a neural network based adaptive estimator

    Science.gov (United States)

    Kwon, Chung-Jin; Kim, Sung-Joong; Han, Woo-Young; Min, Won-Kyoung

    2005-12-01

    The rotor position and speed estimation of permanent-magnet synchronous motor(PMSM) was dealt with. By measuring the phase voltages and currents of the PMSM drive, two diagonally recurrent neural network(DRNN) based observers, a neural current observer and a neural velocity observer were developed. DRNN which has self-feedback of the hidden neurons ensures that the outputs of DRNN contain the whole past information of the system even if the inputs of DRNN are only the present states and inputs of the system. Thus the structure of DRNN may be simpler than that of feedforward and fully recurrent neural networks. If the backpropagation method was used for the training of the DRNN the problem of slow convergence arise. In order to reduce this problem, recursive prediction error(RPE) based learning method for the DRNN was presented. The simulation results show that the proposed approach gives a good estimation of rotor speed and position, and RPE based training has requires a shorter computation time compared to backpropagation based training.

  6. Robust lyapunov controller for uncertain systems

    KAUST Repository

    Laleg-Kirati, Taous-Meriem; Elmetennani, Shahrazed

    2017-01-01

    Various examples of systems and methods are provided for Lyapunov control for uncertain systems. In one example, a system includes a process plant and a robust Lyapunov controller configured to control an input of the process plant. The robust

  7. Lyapunov Function Synthesis - Algorithm and Software

    DEFF Research Database (Denmark)

    Leth, Tobias; Sloth, Christoffer; Wisniewski, Rafal

    2016-01-01

    In this paper we introduce an algorithm for the synthesis of polynomial Lyapunov functions for polynomial vector fields. The Lyapunov function is a continuous piecewisepolynomial defined on simplices, which compose a collection of simplices. The algorithm is elaborated and crucial features are ex...

  8. Using deep recurrent neural network for direct beam solar irradiance cloud screening

    Science.gov (United States)

    Chen, Maosi; Davis, John M.; Liu, Chaoshun; Sun, Zhibin; Zempila, Melina Maria; Gao, Wei

    2017-09-01

    Cloud screening is an essential procedure for in-situ calibration and atmospheric properties retrieval on (UV-)MultiFilter Rotating Shadowband Radiometer [(UV-)MFRSR]. Previous study has explored a cloud screening algorithm for direct-beam (UV-)MFRSR voltage measurements based on the stability assumption on a long time period (typically a half day or a whole day). To design such an algorithm requires in-depth understanding of radiative transfer and delicate data manipulation. Recent rapid developments on deep neural network and computation hardware have opened a window for modeling complicated End-to-End systems with a standardized strategy. In this study, a multi-layer dynamic bidirectional recurrent neural network is built for determining the cloudiness on each time point with a 17-year training dataset and tested with another 1-year dataset. The dataset is the daily 3-minute cosine corrected voltages, airmasses, and the corresponding cloud/clear-sky labels at two stations of the USDA UV-B Monitoring and Research Program. The results show that the optimized neural network model (3-layer, 250 hidden units, and 80 epochs of training) has an overall test accuracy of 97.87% (97.56% for the Oklahoma site and 98.16% for the Hawaii site). Generally, the neural network model grasps the key concept of the original model to use data in the entire day rather than short nearby measurements to perform cloud screening. A scrutiny of the logits layer suggests that the neural network model automatically learns a way to calculate a quantity similar to total optical depth and finds an appropriate threshold for cloud screening.

  9. Minimal-Learning-Parameter Technique Based Adaptive Neural Sliding Mode Control of MEMS Gyroscope

    Directory of Open Access Journals (Sweden)

    Bin Xu

    2017-01-01

    Full Text Available This paper investigates an adaptive neural sliding mode controller for MEMS gyroscopes with minimal-learning-parameter technique. Considering the system uncertainty in dynamics, neural network is employed for approximation. Minimal-learning-parameter technique is constructed to decrease the number of update parameters, and in this way the computation burden is greatly reduced. Sliding mode control is designed to cancel the effect of time-varying disturbance. The closed-loop stability analysis is established via Lyapunov approach. Simulation results are presented to demonstrate the effectiveness of the method.

  10. Multiplex visibility graphs to investigate recurrent neural network dynamics

    Science.gov (United States)

    Bianchi, Filippo Maria; Livi, Lorenzo; Alippi, Cesare; Jenssen, Robert

    2017-03-01

    A recurrent neural network (RNN) is a universal approximator of dynamical systems, whose performance often depends on sensitive hyperparameters. Tuning them properly may be difficult and, typically, based on a trial-and-error approach. In this work, we adopt a graph-based framework to interpret and characterize internal dynamics of a class of RNNs called echo state networks (ESNs). We design principled unsupervised methods to derive hyperparameters configurations yielding maximal ESN performance, expressed in terms of prediction error and memory capacity. In particular, we propose to model time series generated by each neuron activations with a horizontal visibility graph, whose topological properties have been shown to be related to the underlying system dynamics. Successively, horizontal visibility graphs associated with all neurons become layers of a larger structure called a multiplex. We show that topological properties of such a multiplex reflect important features of ESN dynamics that can be used to guide the tuning of its hyperparamers. Results obtained on several benchmarks and a real-world dataset of telephone call data records show the effectiveness of the proposed methods.

  11. New baseline correction algorithm for text-line recognition with bidirectional recurrent neural networks

    Science.gov (United States)

    Morillot, Olivier; Likforman-Sulem, Laurence; Grosicki, Emmanuèle

    2013-04-01

    Many preprocessing techniques have been proposed for isolated word recognition. However, recently, recognition systems have dealt with text blocks and their compound text lines. In this paper, we propose a new preprocessing approach to efficiently correct baseline skew and fluctuations. Our approach is based on a sliding window within which the vertical position of the baseline is estimated. Segmentation of text lines into subparts is, thus, avoided. Experiments conducted on a large publicly available database (Rimes), with a BLSTM (bidirectional long short-term memory) recurrent neural network recognition system, show that our baseline correction approach highly improves performance.

  12. A general framework for global asymptotic stability analysis of delayed neural networks based on LMI approach

    International Nuclear Information System (INIS)

    Cao Jinde; Ho, Daniel W.C.

    2005-01-01

    In this paper, global asymptotic stability is discussed for neural networks with time-varying delay. Several new criteria in matrix inequality form are given to ascertain the uniqueness and global asymptotic stability of equilibrium point for neural networks with time-varying delay based on Lyapunov method and Linear Matrix Inequality (LMI) technique. The proposed LMI approach has the advantage of considering the difference of neuronal excitatory and inhibitory efforts, which is also computationally efficient as it can be solved numerically using recently developed interior-point algorithm. In addition, the proposed results generalize and improve previous works. The obtained criteria also combine two existing conditions into one generalized condition in matrix form. An illustrative example is also given to demonstrate the effectiveness of the proposed results

  13. Training the Recurrent neural network by the Fuzzy Min-Max algorithm for fault prediction

    International Nuclear Information System (INIS)

    Zemouri, Ryad; Racoceanu, Daniel; Zerhouni, Noureddine; Minca, Eugenia; Filip, Florin

    2009-01-01

    In this paper, we present a training technique of a Recurrent Radial Basis Function neural network for fault prediction. We use the Fuzzy Min-Max technique to initialize the k-center of the RRBF neural network. The k-means algorithm is then applied to calculate the centers that minimize the mean square error of the prediction task. The performances of the k-means algorithm are then boosted by the Fuzzy Min-Max technique.

  14. MATLAB Simulation of Gradient-Based Neural Network for Online Matrix Inversion

    Science.gov (United States)

    Zhang, Yunong; Chen, Ke; Ma, Weimu; Li, Xiao-Dong

    This paper investigates the simulation of a gradient-based recurrent neural network for online solution of the matrix-inverse problem. Several important techniques are employed as follows to simulate such a neural system. 1) Kronecker product of matrices is introduced to transform a matrix-differential-equation (MDE) to a vector-differential-equation (VDE); i.e., finally, a standard ordinary-differential-equation (ODE) is obtained. 2) MATLAB routine "ode45" is introduced to solve the transformed initial-value ODE problem. 3) In addition to various implementation errors, different kinds of activation functions are simulated to show the characteristics of such a neural network. Simulation results substantiate the theoretical analysis and efficacy of the gradient-based neural network for online constant matrix inversion.

  15. An Incremental Time-delay Neural Network for Dynamical Recurrent Associative Memory

    Institute of Scientific and Technical Information of China (English)

    2002-01-01

    An incremental time-delay neural network based on synapse growth, which is suitable for dynamic control and learning of autonomous robots, is proposed to improve the learning and retrieving performance of dynamical recurrent associative memory architecture. The model allows steady and continuous establishment of associative memory for spatio-temporal regularities and time series in discrete sequence of inputs. The inserted hidden units can be taken as the long-term memories that expand the capacity of network and sometimes may fade away under certain condition. Preliminary experiment has shown that this incremental network may be a promising approach to endow autonomous robots with the ability of adapting to new data without destroying the learned patterns. The system also benefits from its potential chaos character for emergence.

  16. Recurrent Neural Network Applications for Astronomical Time Series

    Science.gov (United States)

    Protopapas, Pavlos

    2017-06-01

    The benefits of good predictive models in astronomy lie in early event prediction systems and effective resource allocation. Current time series methods applicable to regular time series have not evolved to generalize for irregular time series. In this talk, I will describe two Recurrent Neural Network methods, Long Short-Term Memory (LSTM) and Echo State Networks (ESNs) for predicting irregular time series. Feature engineering along with a non-linear modeling proved to be an effective predictor. For noisy time series, the prediction is improved by training the network on error realizations using the error estimates from astronomical light curves. In addition to this, we propose a new neural network architecture to remove correlation from the residuals in order to improve prediction and compensate for the noisy data. Finally, I show how to set hyperparameters for a stable and performant solution correctly. In this work, we circumvent this obstacle by optimizing ESN hyperparameters using Bayesian optimization with Gaussian Process priors. This automates the tuning procedure, enabling users to employ the power of RNN without needing an in-depth understanding of the tuning procedure.

  17. A new neural network model for solving random interval linear programming problems.

    Science.gov (United States)

    Arjmandzadeh, Ziba; Safi, Mohammadreza; Nazemi, Alireza

    2017-05-01

    This paper presents a neural network model for solving random interval linear programming problems. The original problem involving random interval variable coefficients is first transformed into an equivalent convex second order cone programming problem. A neural network model is then constructed for solving the obtained convex second order cone problem. Employing Lyapunov function approach, it is also shown that the proposed neural network model is stable in the sense of Lyapunov and it is globally convergent to an exact satisfactory solution of the original problem. Several illustrative examples are solved in support of this technique. Copyright © 2017 Elsevier Ltd. All rights reserved.

  18. Real-Time Inverse Optimal Neural Control for Image Based Visual Servoing with Nonholonomic Mobile Robots

    Directory of Open Access Journals (Sweden)

    Carlos López-Franco

    2015-01-01

    Full Text Available We present an inverse optimal neural controller for a nonholonomic mobile robot with parameter uncertainties and unknown external disturbances. The neural controller is based on a discrete-time recurrent high order neural network (RHONN trained with an extended Kalman filter. The reference velocities for the neural controller are obtained with a visual sensor. The effectiveness of the proposed approach is tested by simulations and real-time experiments.

  19. Uniting Control Lyapunov and Control Barrier Functions

    NARCIS (Netherlands)

    Romdlony, Zakiyullah; Jayawardhana, Bayu

    2014-01-01

    In this paper, we propose a nonlinear control design for solving the problem of stabilization with guaranteed safety. The design is based on the merging of a Control Lyapunov Function and a Control Barrier Function. The proposed control method allows us to combine the design of a stabilizer based on

  20. Recurrent neural network for non-smooth convex optimization problems with application to the identification of genetic regulatory networks.

    Science.gov (United States)

    Cheng, Long; Hou, Zeng-Guang; Lin, Yingzi; Tan, Min; Zhang, Wenjun Chris; Wu, Fang-Xiang

    2011-05-01

    A recurrent neural network is proposed for solving the non-smooth convex optimization problem with the convex inequality and linear equality constraints. Since the objective function and inequality constraints may not be smooth, the Clarke's generalized gradients of the objective function and inequality constraints are employed to describe the dynamics of the proposed neural network. It is proved that the equilibrium point set of the proposed neural network is equivalent to the optimal solution of the original optimization problem by using the Lagrangian saddle-point theorem. Under weak conditions, the proposed neural network is proved to be stable, and the state of the neural network is convergent to one of its equilibrium points. Compared with the existing neural network models for non-smooth optimization problems, the proposed neural network can deal with a larger class of constraints and is not based on the penalty method. Finally, the proposed neural network is used to solve the identification problem of genetic regulatory networks, which can be transformed into a non-smooth convex optimization problem. The simulation results show the satisfactory identification accuracy, which demonstrates the effectiveness and efficiency of the proposed approach.

  1. Adaptive Backstepping-Based Neural Tracking Control for MIMO Nonlinear Switched Systems Subject to Input Delays.

    Science.gov (United States)

    Niu, Ben; Li, Lu

    2018-06-01

    This brief proposes a new neural-network (NN)-based adaptive output tracking control scheme for a class of disturbed multiple-input multiple-output uncertain nonlinear switched systems with input delays. By combining the universal approximation ability of radial basis function NNs and adaptive backstepping recursive design with an improved multiple Lyapunov function (MLF) scheme, a novel adaptive neural output tracking controller design method is presented for the switched system. The feature of the developed design is that different coordinate transformations are adopted to overcome the conservativeness caused by adopting a common coordinate transformation for all subsystems. It is shown that all the variables of the resulting closed-loop system are semiglobally uniformly ultimately bounded under a class of switching signals in the presence of MLF and that the system output can follow the desired reference signal. To demonstrate the practicability of the obtained result, an adaptive neural output tracking controller is designed for a mass-spring-damper system.

  2. A one-layer recurrent neural network for non-smooth convex optimization subject to linear inequality constraints

    International Nuclear Information System (INIS)

    Liu, Xiaolan; Zhou, Mi

    2016-01-01

    In this paper, a one-layer recurrent network is proposed for solving a non-smooth convex optimization subject to linear inequality constraints. Compared with the existing neural networks for optimization, the proposed neural network is capable of solving more general convex optimization with linear inequality constraints. The convergence of the state variables of the proposed neural network to achieve solution optimality is guaranteed as long as the designed parameters in the model are larger than the derived lower bounds.

  3. Hierarchical Recurrent Neural Hashing for Image Retrieval With Hierarchical Convolutional Features.

    Science.gov (United States)

    Lu, Xiaoqiang; Chen, Yaxiong; Li, Xuelong

    Hashing has been an important and effective technology in image retrieval due to its computational efficiency and fast search speed. The traditional hashing methods usually learn hash functions to obtain binary codes by exploiting hand-crafted features, which cannot optimally represent the information of the sample. Recently, deep learning methods can achieve better performance, since deep learning architectures can learn more effective image representation features. However, these methods only use semantic features to generate hash codes by shallow projection but ignore texture details. In this paper, we proposed a novel hashing method, namely hierarchical recurrent neural hashing (HRNH), to exploit hierarchical recurrent neural network to generate effective hash codes. There are three contributions of this paper. First, a deep hashing method is proposed to extensively exploit both spatial details and semantic information, in which, we leverage hierarchical convolutional features to construct image pyramid representation. Second, our proposed deep network can exploit directly convolutional feature maps as input to preserve the spatial structure of convolutional feature maps. Finally, we propose a new loss function that considers the quantization error of binarizing the continuous embeddings into the discrete binary codes, and simultaneously maintains the semantic similarity and balanceable property of hash codes. Experimental results on four widely used data sets demonstrate that the proposed HRNH can achieve superior performance over other state-of-the-art hashing methods.Hashing has been an important and effective technology in image retrieval due to its computational efficiency and fast search speed. The traditional hashing methods usually learn hash functions to obtain binary codes by exploiting hand-crafted features, which cannot optimally represent the information of the sample. Recently, deep learning methods can achieve better performance, since deep

  4. Neural-network hybrid control for antilock braking systems.

    Science.gov (United States)

    Lin, Chih-Min; Hsu, C F

    2003-01-01

    The antilock braking systems are designed to maximize wheel traction by preventing the wheels from locking during braking, while also maintaining adequate vehicle steerability; however, the performance is often degraded under harsh road conditions. In this paper, a hybrid control system with a recurrent neural network (RNN) observer is developed for antilock braking systems. This hybrid control system is comprised of an ideal controller and a compensation controller. The ideal controller, containing an RNN uncertainty observer, is the principal controller; and the compensation controller is a compensator for the difference between the system uncertainty and the estimated uncertainty. Since for dynamic response the RNN has capabilities superior to the feedforward NN, it is utilized for the uncertainty observer. The Taylor linearization technique is employed to increase the learning ability of the RNN. In addition, the on-line parameter adaptation laws are derived based on a Lyapunov function, so the stability of the system can be guaranteed. Simulations are performed to demonstrate the effectiveness of the proposed NN hybrid control system for antilock braking control under various road conditions.

  5. Dynamic training algorithm for dynamic neural networks

    International Nuclear Information System (INIS)

    Tan, Y.; Van Cauwenberghe, A.; Liu, Z.

    1996-01-01

    The widely used backpropagation algorithm for training neural networks based on the gradient descent has a significant drawback of slow convergence. A Gauss-Newton method based recursive least squares (RLS) type algorithm with dynamic error backpropagation is presented to speed-up the learning procedure of neural networks with local recurrent terms. Finally, simulation examples concerning the applications of the RLS type algorithm to identification of nonlinear processes using a local recurrent neural network are also included in this paper

  6. An automatic microseismic or acoustic emission arrival identification scheme with deep recurrent neural networks

    Science.gov (United States)

    Zheng, Jing; Lu, Jiren; Peng, Suping; Jiang, Tianqi

    2018-02-01

    The conventional arrival pick-up algorithms cannot avoid the manual modification of the parameters for the simultaneous identification of multiple events under different signal-to-noise ratios (SNRs). Therefore, in order to automatically obtain the arrivals of multiple events with high precision under different SNRs, in this study an algorithm was proposed which had the ability to pick up the arrival of microseismic or acoustic emission events based on deep recurrent neural networks. The arrival identification was performed using two important steps, which included a training phase and a testing phase. The training process was mathematically modelled by deep recurrent neural networks using Long Short-Term Memory architecture. During the testing phase, the learned weights were utilized to identify the arrivals through the microseismic/acoustic emission data sets. The data sets were obtained by rock physics experiments of the acoustic emission. In order to obtain the data sets under different SNRs, this study added random noise to the raw experiments' data sets. The results showed that the outcome of the proposed method was able to attain an above 80 per cent hit-rate at SNR 0 dB, and an approximately 70 per cent hit-rate at SNR -5 dB, with an absolute error in 10 sampling points. These results indicated that the proposed method had high selection precision and robustness.

  7. Neural network for solving convex quadratic bilevel programming problems.

    Science.gov (United States)

    He, Xing; Li, Chuandong; Huang, Tingwen; Li, Chaojie

    2014-03-01

    In this paper, using the idea of successive approximation, we propose a neural network to solve convex quadratic bilevel programming problems (CQBPPs), which is modeled by a nonautonomous differential inclusion. Different from the existing neural network for CQBPP, the model has the least number of state variables and simple structure. Based on the theory of nonsmooth analysis, differential inclusions and Lyapunov-like method, the limit equilibrium points sequence of the proposed neural networks can approximately converge to an optimal solution of CQBPP under certain conditions. Finally, simulation results on two numerical examples and the portfolio selection problem show the effectiveness and performance of the proposed neural network. Copyright © 2013 Elsevier Ltd. All rights reserved.

  8. A Velocity-Level Bi-Criteria Optimization Scheme for Coordinated Path Tracking of Dual Robot Manipulators Using Recurrent Neural Network.

    Science.gov (United States)

    Xiao, Lin; Zhang, Yongsheng; Liao, Bolin; Zhang, Zhijun; Ding, Lei; Jin, Long

    2017-01-01

    A dual-robot system is a robotic device composed of two robot arms. To eliminate the joint-angle drift and prevent the occurrence of high joint velocity, a velocity-level bi-criteria optimization scheme, which includes two criteria (i.e., the minimum velocity norm and the repetitive motion), is proposed and investigated for coordinated path tracking of dual robot manipulators. Specifically, to realize the coordinated path tracking of dual robot manipulators, two subschemes are first presented for the left and right robot manipulators. After that, such two subschemes are reformulated as two general quadratic programs (QPs), which can be formulated as one unified QP. A recurrent neural network (RNN) is thus presented to solve effectively the unified QP problem. At last, computer simulation results based on a dual three-link planar manipulator further validate the feasibility and the efficacy of the velocity-level optimization scheme for coordinated path tracking using the recurrent neural network.

  9. Identification of Jets Containing $b$-Hadrons with Recurrent Neural Networks at the ATLAS Experiment

    CERN Document Server

    The ATLAS collaboration

    2017-01-01

    A novel $b$-jet identification algorithm is constructed with a Recurrent Neural Network (RNN) at the ATLAS experiment at the CERN Large Hadron Collider. The RNN based $b$-tagging algorithm processes charged particle tracks associated to jets without reliance on secondary vertex finding, and can augment existing secondary-vertex based taggers. In contrast to traditional impact-parameter-based $b$-tagging algorithms which assume that tracks associated to jets are independent from each other, the RNN based $b$-tagging algorithm can exploit the spatial and kinematic correlations between tracks which are initiated from the same $b$-hadrons. This new approach also accommodates an extended set of input variables. This note presents the expected performance of the RNN based $b$-tagging algorithm in simulated $t \\bar t$ events at $\\sqrt{s}=13$ TeV.

  10. Energy-based Lyapunov functions for forced Hamiltonian systems with dissipation

    NARCIS (Netherlands)

    Maschke, Bernhard M.J.; Ortega, Romeo; Schaft, Arjan J. van der

    1998-01-01

    It is well known that the total energy is a suitable Lyapunov function to study the stability of the trivial equilibrium of an isolated standard Hamiltonian system. In many practical instances, however, the system is in interaction with its environment through some constant forcing terms. This gives

  11. An algorithm for constructing Lyapunov functions

    Directory of Open Access Journals (Sweden)

    Sigurdur Freyr Hafstein

    2007-08-01

    Full Text Available In this monograph we develop an algorithm for constructing Lyapunov functions for arbitrary switched dynamical systems $dot{mathbf{x}} = mathbf{f}_sigma(t,mathbf{x}$, possessing a uniformly asymptotically stable equilibrium. Let $dot{mathbf{x}}=mathbf{f}_p(t,mathbf{x}$, $pinmathcal{P}$, be the collection of the ODEs, to which the switched system corresponds. The number of the vector fields $mathbf{f}_p$ on the right-hand side of the differential equation is assumed to be finite and we assume that their components $f_{p,i}$ are $mathcal{C}^2$ functions and that we can give some bounds, not necessarily close, on their second-order partial derivatives. The inputs of the algorithm are solely a finite number of the function values of the vector fields $mathbf{f}_p$ and these bounds. The domain of the Lyapunov function constructed by the algorithm is only limited by the size of the equilibrium's region of attraction. Note, that the concept of a Lyapunov function for the arbitrary switched system $dot{mathbf{x}} = mathbf{f}_sigma(t,mathbf{x}$ is equivalent to the concept of a common Lyapunov function for the systems $dot{mathbf{x}}=mathbf{f}_p(t,mathbf{x}$, $pinmathcal{P}$, and that if $mathcal{P}$ contains exactly one element, then the switched system is just a usual ODE $dot{mathbf{x}}=mathbf{f}(t,mathbf{x}$. We give numerous examples of Lyapunov functions constructed by our method at the end of this monograph.

  12. Neural Network to Solve Concave Games

    OpenAIRE

    Liu, Zixin; Wang, Nengfa

    2014-01-01

    The issue on neural network method to solve concave games is concerned. Combined with variational inequality, Ky Fan inequality, and projection equation, concave games are transformed into a neural network model. On the basis of the Lyapunov stable theory, some stability results are also given. Finally, two classic games’ simulation results are given to illustrate the theoretical results.

  13. Cascaded bidirectional recurrent neural networks for protein secondary structure prediction.

    Science.gov (United States)

    Chen, Jinmiao; Chaudhari, Narendra

    2007-01-01

    Protein secondary structure (PSS) prediction is an important topic in bioinformatics. Our study on a large set of non-homologous proteins shows that long-range interactions commonly exist and negatively affect PSS prediction. Besides, we also reveal strong correlations between secondary structure (SS) elements. In order to take into account the long-range interactions and SS-SS correlations, we propose a novel prediction system based on cascaded bidirectional recurrent neural network (BRNN). We compare the cascaded BRNN against another two BRNN architectures, namely the original BRNN architecture used for speech recognition as well as Pollastri's BRNN that was proposed for PSS prediction. Our cascaded BRNN achieves an overall three state accuracy Q3 of 74.38\\%, and reaches a high Segment OVerlap (SOV) of 66.0455. It outperforms the original BRNN and Pollastri's BRNN in both Q3 and SOV. Specifically, it improves the SOV score by 4-6%.

  14. Robust stability of bidirectional associative memory neural networks with time delays

    Science.gov (United States)

    Park, Ju H.

    2006-01-01

    Based on the Lyapunov Krasovskii functionals combined with linear matrix inequality approach, a novel stability criterion is proposed for asymptotic stability of bidirectional associative memory neural networks with time delays. A novel delay-dependent stability criterion is given in terms of linear matrix inequalities, which can be solved easily by various optimization algorithms.

  15. Robust stability of bidirectional associative memory neural networks with time delays

    International Nuclear Information System (INIS)

    Park, Ju H.

    2006-01-01

    Based on the Lyapunov-Krasovskii functionals combined with linear matrix inequality approach, a novel stability criterion is proposed for asymptotic stability of bidirectional associative memory neural networks with time delays. A novel delay-dependent stability criterion is given in terms of linear matrix inequalities, which can be solved easily by various optimization algorithms

  16. Synchronising chaotic Chua's circuit using switching feedback control based on piecewise quadratic Lyapunov functions

    International Nuclear Information System (INIS)

    Hong-Bin, Zhang; Jian-Wei, Xia; Yong-Bin, Yu; Chuang-Yin, Dang

    2010-01-01

    This paper investigates the chaos synchronisation between two coupled chaotic Chua's circuits. The sufficient condition presented by linear matrix inequalities (LMIs) of global asymptotic synchronisation is attained based on piecewise quadratic Lyapunov functions. First, we obtain the piecewise linear differential inclusions (pwLDIs) model of synchronisation error dynamics, then we design a switching (piecewise-linear) feedback control law to stabilise it based on the piecewise quadratic Laypunov functions. Then we give some numerical simulations to demonstrate the effectiveness of our theoretical results

  17. The super-Turing computational power of plastic recurrent neural networks.

    Science.gov (United States)

    Cabessa, Jérémie; Siegelmann, Hava T

    2014-12-01

    We study the computational capabilities of a biologically inspired neural model where the synaptic weights, the connectivity pattern, and the number of neurons can evolve over time rather than stay static. Our study focuses on the mere concept of plasticity of the model so that the nature of the updates is assumed to be not constrained. In this context, we show that the so-called plastic recurrent neural networks (RNNs) are capable of the precise super-Turing computational power--as the static analog neural networks--irrespective of whether their synaptic weights are modeled by rational or real numbers, and moreover, irrespective of whether their patterns of plasticity are restricted to bi-valued updates or expressed by any other more general form of updating. Consequently, the incorporation of only bi-valued plastic capabilities in a basic model of RNNs suffices to break the Turing barrier and achieve the super-Turing level of computation. The consideration of more general mechanisms of architectural plasticity or of real synaptic weights does not further increase the capabilities of the networks. These results support the claim that the general mechanism of plasticity is crucially involved in the computational and dynamical capabilities of biological neural networks. They further show that the super-Turing level of computation reflects in a suitable way the capabilities of brain-like models of computation.

  18. Marginally Stable Triangular Recurrent Neural Network Architecture for Time Series Prediction.

    Science.gov (United States)

    Sivakumar, Seshadri; Sivakumar, Shyamala

    2017-09-25

    This paper introduces a discrete-time recurrent neural network architecture using triangular feedback weight matrices that allows a simplified approach to ensuring network and training stability. The triangular structure of the weight matrices is exploited to readily ensure that the eigenvalues of the feedback weight matrix represented by the block diagonal elements lie on the unit circle in the complex z-plane by updating these weights based on the differential of the angular error variable. Such placement of the eigenvalues together with the extended close interaction between state variables facilitated by the nondiagonal triangular elements, enhances the learning ability of the proposed architecture. Simulation results show that the proposed architecture is highly effective in time-series prediction tasks associated with nonlinear and chaotic dynamic systems with underlying oscillatory modes. This modular architecture with dual upper and lower triangular feedback weight matrices mimics fully recurrent network architectures, while maintaining learning stability with a simplified training process. While training, the block-diagonal weights (hence the eigenvalues) of the dual triangular matrices are constrained to the same values during weight updates aimed at minimizing the possibility of overfitting. The dual triangular architecture also exploits the benefit of parsing the input and selectively applying the parsed inputs to the two subnetworks to facilitate enhanced learning performance.

  19. Global exponential stability for discrete-time neural networks with variable delays

    International Nuclear Information System (INIS)

    Chen Wuhua; Lu Xiaomei; Liang Dongying

    2006-01-01

    This Letter provides new exponential stability criteria for discrete-time neural networks with variable delays. The main technique is to reduce exponential convergence estimation of the neural network solution to that of one component of the corresponding solution by constructing Lyapunov function based on M-matrix. By introducing the tuning parameter diagonal matrix, the delay-independent and delay-dependent exponential stability conditions have been unified in the same mathematical formula. The effectiveness of the new results are illustrated by three examples

  20. Lyapunov functionals and stability of stochastic functional differential equations

    CERN Document Server

    Shaikhet, Leonid

    2013-01-01

    Stability conditions for functional differential equations can be obtained using Lyapunov functionals. Lyapunov Functionals and Stability of Stochastic Functional Differential Equations describes the general method of construction of Lyapunov functionals to investigate the stability of differential equations with delays. This work continues and complements the author’s previous book Lyapunov Functionals and Stability of Stochastic Difference Equations, where this method is described for discrete- and continuous-time difference equations. The text begins with a description of the peculiarities of deterministic and stochastic functional differential equations. There follow basic definitions for stability theory of stochastic hereditary systems, and a formal procedure of Lyapunov functionals construction is presented. Stability investigation is conducted for stochastic linear and nonlinear differential equations with constant and distributed delays. The proposed method is used for stability investigation of di...

  1. Lyapunov-based Stability of Feedback Interconnections of Negative Imaginary Systems

    KAUST Repository

    Ghallab, Ahmed G.

    2017-10-19

    Feedback control systems using sensors and actuators such as piezoelectric sensors and actuators, micro-electro-mechanical systems (MEMS) sensors and opto-mechanical sensors, are allowing new advances in designing such high precision technologies. The negative imaginary control systems framework allows for robust control design for such high precision systems in the face of uncertainties due to unmodelled dynamics. The stability of the feedback interconnection of negative imaginary systems has been well established in the literature. However, the proofs of stability feedback interconnection which are used in some previous papers have a shortcoming due to a matrix inevitability issue. In this paper, we provide a new and correct Lyapunov-based proof of one such result and show that the result is still true.

  2. Lyapunov-based Stability of Feedback Interconnections of Negative Imaginary Systems

    KAUST Repository

    Ghallab, Ahmed G.; Mabrok, Mohamed; Petersen, Ian R.

    2017-01-01

    Feedback control systems using sensors and actuators such as piezoelectric sensors and actuators, micro-electro-mechanical systems (MEMS) sensors and opto-mechanical sensors, are allowing new advances in designing such high precision technologies. The negative imaginary control systems framework allows for robust control design for such high precision systems in the face of uncertainties due to unmodelled dynamics. The stability of the feedback interconnection of negative imaginary systems has been well established in the literature. However, the proofs of stability feedback interconnection which are used in some previous papers have a shortcoming due to a matrix inevitability issue. In this paper, we provide a new and correct Lyapunov-based proof of one such result and show that the result is still true.

  3. Neural Network Based Real-time Correction of Transducer Dynamic Errors

    Science.gov (United States)

    Roj, J.

    2013-12-01

    In order to carry out real-time dynamic error correction of transducers described by a linear differential equation, a novel recurrent neural network was developed. The network structure is based on solving this equation with respect to the input quantity when using the state variables. It is shown that such a real-time correction can be carried out using simple linear perceptrons. Due to the use of a neural technique, knowledge of the dynamic parameters of the transducer is not necessary. Theoretical considerations are illustrated by the results of simulation studies performed for the modeled second order transducer. The most important properties of the neural dynamic error correction, when emphasizing the fundamental advantages and disadvantages, are discussed.

  4. Covariant Lyapunov vectors

    International Nuclear Information System (INIS)

    Ginelli, Francesco; Politi, Antonio; Chaté, Hugues; Livi, Roberto

    2013-01-01

    Recent years have witnessed a growing interest in covariant Lyapunov vectors (CLVs) which span local intrinsic directions in the phase space of chaotic systems. Here, we review the basic results of ergodic theory, with a specific reference to the implications of Oseledets’ theorem for the properties of the CLVs. We then present a detailed description of a ‘dynamical’ algorithm to compute the CLVs and show that it generically converges exponentially in time. We also discuss its numerical performance and compare it with other algorithms presented in the literature. We finally illustrate how CLVs can be used to quantify deviations from hyperbolicity with reference to a dissipative system (a chain of Hénon maps) and a Hamiltonian model (a Fermi–Pasta–Ulam chain). This article is part of a special issue of Journal of Physics A: Mathematical and Theoretical devoted to ‘Lyapunov analysis: from dynamical systems theory to applications’. (paper)

  5. Hybrid Recurrent Laguerre-Orthogonal-Polynomial NN Control System Applied in V-Belt Continuously Variable Transmission System Using Particle Swarm Optimization

    Directory of Open Access Journals (Sweden)

    Chih-Hong Lin

    2015-01-01

    Full Text Available Because the V-belt continuously variable transmission (CVT system driven by permanent magnet synchronous motor (PMSM has much unknown nonlinear and time-varying characteristics, the better control performance design for the linear control design is a time consuming procedure. In order to overcome difficulties for design of the linear controllers, the hybrid recurrent Laguerre-orthogonal-polynomial neural network (NN control system which has online learning ability to respond to the system’s nonlinear and time-varying behaviors is proposed to control PMSM servo-driven V-belt CVT system under the occurrence of the lumped nonlinear load disturbances. The hybrid recurrent Laguerre-orthogonal-polynomial NN control system consists of an inspector control, a recurrent Laguerre-orthogonal-polynomial NN control with adaptive law, and a recouped control with estimated law. Moreover, the adaptive law of online parameters in the recurrent Laguerre-orthogonal-polynomial NN is derived using the Lyapunov stability theorem. Furthermore, the optimal learning rate of the parameters by means of modified particle swarm optimization (PSO is proposed to achieve fast convergence. Finally, to show the effectiveness of the proposed control scheme, comparative studies are demonstrated by experimental results.

  6. Recurrent Neural Networks to Correct Satellite Image Classification Maps

    Science.gov (United States)

    Maggiori, Emmanuel; Charpiat, Guillaume; Tarabalka, Yuliya; Alliez, Pierre

    2017-09-01

    While initially devised for image categorization, convolutional neural networks (CNNs) are being increasingly used for the pixelwise semantic labeling of images. However, the proper nature of the most common CNN architectures makes them good at recognizing but poor at localizing objects precisely. This problem is magnified in the context of aerial and satellite image labeling, where a spatially fine object outlining is of paramount importance. Different iterative enhancement algorithms have been presented in the literature to progressively improve the coarse CNN outputs, seeking to sharpen object boundaries around real image edges. However, one must carefully design, choose and tune such algorithms. Instead, our goal is to directly learn the iterative process itself. For this, we formulate a generic iterative enhancement process inspired from partial differential equations, and observe that it can be expressed as a recurrent neural network (RNN). Consequently, we train such a network from manually labeled data for our enhancement task. In a series of experiments we show that our RNN effectively learns an iterative process that significantly improves the quality of satellite image classification maps.

  7. Construction of a Smooth Lyapunov Function for the Robust and Exact Second-Order Differentiator

    Directory of Open Access Journals (Sweden)

    Tonametl Sanchez

    2016-01-01

    Full Text Available Differentiators play an important role in (continuous feedback control systems. In particular, the robust and exact second-order differentiator has shown some very interesting properties and it has been used successfully in sliding mode control, in spite of the lack of a Lyapunov based procedure to design its gains. As contribution of this paper, we provide a constructive method to determine a differentiable Lyapunov function for such a differentiator. Moreover, the Lyapunov function is used to provide a procedure to design the differentiator’s parameters. Also, some sets of such parameters are provided. The determination of the positive definiteness of the Lyapunov function and negative definiteness of its derivative is converted to the problem of solving a system of inequalities linear in the parameters of the Lyapunov function candidate and also linear in the gains of the differentiator, but bilinear in both.

  8. Very deep recurrent convolutional neural network for object recognition

    Science.gov (United States)

    Brahimi, Sourour; Ben Aoun, Najib; Ben Amar, Chokri

    2017-03-01

    In recent years, Computer vision has become a very active field. This field includes methods for processing, analyzing, and understanding images. The most challenging problems in computer vision are image classification and object recognition. This paper presents a new approach for object recognition task. This approach exploits the success of the Very Deep Convolutional Neural Network for object recognition. In fact, it improves the convolutional layers by adding recurrent connections. This proposed approach was evaluated on two object recognition benchmarks: Pascal VOC 2007 and CIFAR-10. The experimental results prove the efficiency of our method in comparison with the state of the art methods.

  9. Fixed-time synchronization of memristor-based BAM neural networks with time-varying discrete delay.

    Science.gov (United States)

    Chen, Chuan; Li, Lixiang; Peng, Haipeng; Yang, Yixian

    2017-12-01

    This paper is devoted to studying the fixed-time synchronization of memristor-based BAM neural networks (MBAMNNs) with discrete delay. Fixed-time synchronization means that synchronization can be achieved in a fixed time for any initial values of the considered systems. In the light of the double-layer structure of MBAMNNs, we design two similar feedback controllers. Based on Lyapunov stability theories, several criteria are established to guarantee that the drive and response MBAMNNs can realize synchronization in a fixed time. In particular, by changing the parameters of controllers, this fixed time can be adjusted to some desired value in advance, irrespective of the initial values of MBAMNNs. Numerical simulations are included to validate the derived results. Copyright © 2017 Elsevier Ltd. All rights reserved.

  10. Neural network-based distributed attitude coordination control for spacecraft formation flying with input saturation.

    Science.gov (United States)

    Zou, An-Min; Kumar, Krishna Dev

    2012-07-01

    This brief considers the attitude coordination control problem for spacecraft formation flying when only a subset of the group members has access to the common reference attitude. A quaternion-based distributed attitude coordination control scheme is proposed with consideration of the input saturation and with the aid of the sliding-mode observer, separation principle theorem, Chebyshev neural networks, smooth projection algorithm, and robust control technique. Using graph theory and a Lyapunov-based approach, it is shown that the distributed controller can guarantee the attitude of all spacecraft to converge to a common time-varying reference attitude when the reference attitude is available only to a portion of the group of spacecraft. Numerical simulations are presented to demonstrate the performance of the proposed distributed controller.

  11. Recurrent neural network-based modeling of gene regulatory network using elephant swarm water search algorithm.

    Science.gov (United States)

    Mandal, Sudip; Saha, Goutam; Pal, Rajat Kumar

    2017-08-01

    Correct inference of genetic regulations inside a cell from the biological database like time series microarray data is one of the greatest challenges in post genomic era for biologists and researchers. Recurrent Neural Network (RNN) is one of the most popular and simple approach to model the dynamics as well as to infer correct dependencies among genes. Inspired by the behavior of social elephants, we propose a new metaheuristic namely Elephant Swarm Water Search Algorithm (ESWSA) to infer Gene Regulatory Network (GRN). This algorithm is mainly based on the water search strategy of intelligent and social elephants during drought, utilizing the different types of communication techniques. Initially, the algorithm is tested against benchmark small and medium scale artificial genetic networks without and with presence of different noise levels and the efficiency was observed in term of parametric error, minimum fitness value, execution time, accuracy of prediction of true regulation, etc. Next, the proposed algorithm is tested against the real time gene expression data of Escherichia Coli SOS Network and results were also compared with others state of the art optimization methods. The experimental results suggest that ESWSA is very efficient for GRN inference problem and performs better than other methods in many ways.

  12. Particle Swarm Based Approach of a Real-Time Discrete Neural Identifier for Linear Induction Motors

    Directory of Open Access Journals (Sweden)

    Alma Y. Alanis

    2013-01-01

    Full Text Available This paper focusses on a discrete-time neural identifier applied to a linear induction motor (LIM model, whose model is assumed to be unknown. This neural identifier is robust in presence of external and internal uncertainties. The proposed scheme is based on a discrete-time recurrent high-order neural network (RHONN trained with a novel algorithm based on extended Kalman filter (EKF and particle swarm optimization (PSO, using an online series-parallel con…figuration. Real-time results are included in order to illustrate the applicability of the proposed scheme.

  13. Espectro de Lyapunov de un Oscilador Colpitts en Base Común

    Directory of Open Access Journals (Sweden)

    Camilo Andrés Florez

    2013-09-01

    Full Text Available En el presente documento se presenta la definición de exponentes de Lyapunov de un sistema autónomo no lineal de tiempo continuo y una técnica recomendada para medir dicho conjunto de exponentes (espectro, con la finalidad de detectar la existencia de ciclos límites o de caos en un circuito oscilador Colpitts implementado con un transistor BJT. A partir del modelo de Ebers-Möll del transistor BJT se derivaron las ecuaciones de estado que rigen al circuito, luego se adoptó un caso numérico de estudio, y mediante el uso de un programa de simulación matemática se aplicó la metodología propuesta para determinar el espectro de Lyapunov del oscilador. Los resultados obtenidos evidencian la existencia de caos para algunos conjuntos de valores de los parámetros del circuito.

  14. Globally exponential stability of neural network with constant and variable delays

    International Nuclear Information System (INIS)

    Zhao Weirui; Zhang Huanshui

    2006-01-01

    This Letter presents new sufficient conditions of globally exponential stability of neural networks with delays. We show that these results generalize recently published globally exponential stability results. In particular, several different globally exponential stability conditions in the literatures which were proved using different Lyapunov functionals are generalized and unified by using the same Lyapunov functional and the technique of inequality of integral. A comparison between our results and the previous results admits that our results establish a new set of stability criteria for delayed neural networks. Those conditions are less restrictive than those given in the earlier references

  15. Criticality meets learning: Criticality signatures in a self-organizing recurrent neural network.

    Science.gov (United States)

    Del Papa, Bruno; Priesemann, Viola; Triesch, Jochen

    2017-01-01

    Many experiments have suggested that the brain operates close to a critical state, based on signatures of criticality such as power-law distributed neuronal avalanches. In neural network models, criticality is a dynamical state that maximizes information processing capacities, e.g. sensitivity to input, dynamical range and storage capacity, which makes it a favorable candidate state for brain function. Although models that self-organize towards a critical state have been proposed, the relation between criticality signatures and learning is still unclear. Here, we investigate signatures of criticality in a self-organizing recurrent neural network (SORN). Investigating criticality in the SORN is of particular interest because it has not been developed to show criticality. Instead, the SORN has been shown to exhibit spatio-temporal pattern learning through a combination of neural plasticity mechanisms and it reproduces a number of biological findings on neural variability and the statistics and fluctuations of synaptic efficacies. We show that, after a transient, the SORN spontaneously self-organizes into a dynamical state that shows criticality signatures comparable to those found in experiments. The plasticity mechanisms are necessary to attain that dynamical state, but not to maintain it. Furthermore, onset of external input transiently changes the slope of the avalanche distributions - matching recent experimental findings. Interestingly, the membrane noise level necessary for the occurrence of the criticality signatures reduces the model's performance in simple learning tasks. Overall, our work shows that the biologically inspired plasticity and homeostasis mechanisms responsible for the SORN's spatio-temporal learning abilities can give rise to criticality signatures in its activity when driven by random input, but these break down under the structured input of short repeating sequences.

  16. Robust recurrent neural network modeling for software fault detection and correction prediction

    International Nuclear Information System (INIS)

    Hu, Q.P.; Xie, M.; Ng, S.H.; Levitin, G.

    2007-01-01

    Software fault detection and correction processes are related although different, and they should be studied together. A practical approach is to apply software reliability growth models to model fault detection, and fault correction process is assumed to be a delayed process. On the other hand, the artificial neural networks model, as a data-driven approach, tries to model these two processes together with no assumptions. Specifically, feedforward backpropagation networks have shown their advantages over analytical models in fault number predictions. In this paper, the following approach is explored. First, recurrent neural networks are applied to model these two processes together. Within this framework, a systematic networks configuration approach is developed with genetic algorithm according to the prediction performance. In order to provide robust predictions, an extra factor characterizing the dispersion of prediction repetitions is incorporated into the performance function. Comparisons with feedforward neural networks and analytical models are developed with respect to a real data set

  17. Folk music style modelling by recurrent neural networks with long short term memory units

    OpenAIRE

    Sturm, Bob; Santos, João Felipe; Korshunova, Iryna

    2015-01-01

    We demonstrate two generative models created by training a recurrent neural network (RNN) with three hidden layers of long short-term memory (LSTM) units. This extends past work in numerous directions, including training deeper models with nearly 24,000 high-level transcriptions of folk tunes. We discuss our on-going work.

  18. Neural feedback linearization adaptive control for affine nonlinear systems based on neural network estimator

    Directory of Open Access Journals (Sweden)

    Bahita Mohamed

    2011-01-01

    Full Text Available In this work, we introduce an adaptive neural network controller for a class of nonlinear systems. The approach uses two Radial Basis Functions, RBF networks. The first RBF network is used to approximate the ideal control law which cannot be implemented since the dynamics of the system are unknown. The second RBF network is used for on-line estimating the control gain which is a nonlinear and unknown function of the states. The updating laws for the combined estimator and controller are derived through Lyapunov analysis. Asymptotic stability is established with the tracking errors converging to a neighborhood of the origin. Finally, the proposed method is applied to control and stabilize the inverted pendulum system.

  19. Synaptic plasticity in a recurrent neural network for versatile and adaptive behaviors of a walking robot

    Directory of Open Access Journals (Sweden)

    Eduard eGrinke

    2015-10-01

    Full Text Available Walking animals, like insects, with little neural computing can effectively perform complex behaviors. They can walk around their environment, escape from corners/deadlocks, and avoid or climb over obstacles. While performing all these behaviors, they can also adapt their movements to deal with an unknown situation. As a consequence, they successfully navigate through their complex environment. The versatile and adaptive abilities are the result of an integration of several ingredients embedded in their sensorimotor loop. Biological studies reveal that the ingredients include neural dynamics, plasticity, sensory feedback, and biomechanics. Generating such versatile and adaptive behaviors for a walking robot is a challenging task. In this study, we present a bio-inspired approach to solve this task. Specifically, the approach combines neural mechanisms with plasticity, sensory feedback, and biomechanics. The neural mechanisms consist of adaptive neural sensory processing and modular neural locomotion control. The sensory processing is based on a small recurrent network consisting of two fully connected neurons. Online correlation-based learning with synaptic scaling is applied to adequately change the connections of the network. By doing so, we can effectively exploit neural dynamics (i.e., hysteresis effects and single attractors in the network to generate different turning angles with short-term memory for a biomechanical walking robot. The turning information is transmitted as descending steering signals to the locomotion control which translates the signals into motor actions. As a result, the robot can walk around and adapt its turning angle for avoiding obstacles in different situations as well as escaping from sharp corners or deadlocks. Using backbone joint control embedded in the locomotion control allows the robot to climb over small obstacles. Consequently, it can successfully explore and navigate in complex environments.

  20. A Lyapunov Function Based Remedial Action Screening Tool Using Real-Time Data

    Energy Technology Data Exchange (ETDEWEB)

    Mitra, Joydeep [Michigan State Univ., East Lansing, MI (United States); Ben-Idris, Mohammed [Univ. of Nevada, Reno, NV (United States); Faruque, Omar [Florida State Univ., Tallahassee, FL (United States); Backhaus, Scott [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Deb, Sidart [LCG Consulting, Los Altos, CA (United States)

    2016-03-30

    This report summarizes the outcome of a research project that comprised the development of a Lyapunov function based remedial action screening tool using real-time data (L-RAS). The L-RAS is an advanced computational tool that is intended to assist system operators in making real-time redispatch decisions to preserve power grid stability. The tool relies on screening contingencies using a homotopy method based on Lyapunov functions to avoid, to the extent possible, the use of time domain simulations. This enables transient stability evaluation at real-time speed without the use of massively parallel computational resources. The project combined the following components. 1. Development of a methodology for contingency screening using a homotopy method based on Lyapunov functions and real-time data. 2. Development of a methodology for recommending remedial actions based on the screening results. 3. Development of a visualization and operator interaction interface. 4. Testing of screening tool, validation of control actions, and demonstration of project outcomes on a representative real system simulated on a Real-Time Digital Simulator (RTDS) cluster. The project was led by Michigan State University (MSU), where the theoretical models including homotopy-based screening, trajectory correction using real-time data, and remedial action were developed and implemented in the form of research-grade software. Los Alamos National Laboratory (LANL) contributed to the development of energy margin sensitivity dynamics, which constituted a part of the remedial action portfolio. Florida State University (FSU) and Southern California Edison (SCE) developed a model of the SCE system that was implemented on FSU's RTDS cluster to simulate real-time data that was streamed over the internet to MSU where the L-RAS tool was executed and remedial actions were communicated back to FSU to execute stabilizing controls on the simulated system. LCG Consulting developed the visualization

  1. Synaptic plasticity in a recurrent neural network for versatile and adaptive behaviors of a walking robot.

    Science.gov (United States)

    Grinke, Eduard; Tetzlaff, Christian; Wörgötter, Florentin; Manoonpong, Poramate

    2015-01-01

    Walking animals, like insects, with little neural computing can effectively perform complex behaviors. For example, they can walk around their environment, escape from corners/deadlocks, and avoid or climb over obstacles. While performing all these behaviors, they can also adapt their movements to deal with an unknown situation. As a consequence, they successfully navigate through their complex environment. The versatile and adaptive abilities are the result of an integration of several ingredients embedded in their sensorimotor loop. Biological studies reveal that the ingredients include neural dynamics, plasticity, sensory feedback, and biomechanics. Generating such versatile and adaptive behaviors for a many degrees-of-freedom (DOFs) walking robot is a challenging task. Thus, in this study, we present a bio-inspired approach to solve this task. Specifically, the approach combines neural mechanisms with plasticity, exteroceptive sensory feedback, and biomechanics. The neural mechanisms consist of adaptive neural sensory processing and modular neural locomotion control. The sensory processing is based on a small recurrent neural network consisting of two fully connected neurons. Online correlation-based learning with synaptic scaling is applied to adequately change the connections of the network. By doing so, we can effectively exploit neural dynamics (i.e., hysteresis effects and single attractors) in the network to generate different turning angles with short-term memory for a walking robot. The turning information is transmitted as descending steering signals to the neural locomotion control which translates the signals into motor actions. As a result, the robot can walk around and adapt its turning angle for avoiding obstacles in different situations. The adaptation also enables the robot to effectively escape from sharp corners or deadlocks. Using backbone joint control embedded in the the locomotion control allows the robot to climb over small obstacles

  2. A statistical approach to estimate the LYAPUNOV spectrum in disc brake squeal

    Science.gov (United States)

    Oberst, S.; Lai, J. C. S.

    2015-01-01

    The estimation of squeal propensity of a brake system from the prediction of unstable vibration modes using the linear complex eigenvalue analysis (CEA) in the frequency domain has its fair share of successes and failures. While the CEA is almost standard practice for the automotive industry, time domain methods and the estimation of LYAPUNOV spectra have not received much attention in brake squeal analyses. One reason is the challenge in estimating the true LYAPUNOV exponents and their discrimination against spurious ones in experimental data. A novel method based on the application of the ECKMANN-RUELLE matrices is proposed here to estimate LYAPUNOV exponents by using noise in a statistical procedure. It is validated with respect to parameter variations and dimension estimates. By counting the number of non-overlapping confidence intervals for LYAPUNOV exponent distributions obtained by moving a window of increasing size over bootstrapped same-length estimates of an observation function, a dispersion measure's width is calculated and fed into a BAYESIAN beta-binomial model. Results obtained using this method for benchmark models of white and pink noise as well as the classical HENON map indicate that true LYAPUNOV exponents can be isolated from spurious ones with high confidence. The method is then applied to accelerometer and microphone data obtained from brake squeal tests. Estimated LYAPUNOV exponents indicate that the pad's out-of-plane vibration behaves quasi-periodically on the brink to chaos while the microphone's squeal signal remains periodic.

  3. Hall magnetohydrodynamics: Conservation laws and Lyapunov stability

    International Nuclear Information System (INIS)

    Holm, D.D.

    1987-01-01

    Hall electric fields produce circulating mass flow in confined ideal-fluid plasmas. The conservation laws, Hamiltonian structure, equilibrium state relations, and Lyapunov stability conditions are presented here for ideal Hall magnetohydrodynamics (HMHD) in two and three dimensions. The approach here is to use the remarkable array of nonlinear conservation laws for HMHD that follow from its Hamiltonian structure in order to construct explicit Lyapunov functionals for the HMHD equilibrium states. In this way, the Lyapunov stability analysis provides classes of HMHD equilibria that are stable and whose linearized initial-value problems are well posed (in the sense of possessing continuous dependence on initial conditions). Several examples are discussed in both two and three dimensions

  4. Neural processing of short-term recurrence in songbird vocal communication.

    Directory of Open Access Journals (Sweden)

    Gabriël J L Beckers

    Full Text Available BACKGROUND: Many situations involving animal communication are dominated by recurring, stereotyped signals. How do receivers optimally distinguish between frequently recurring signals and novel ones? Cortical auditory systems are known to be pre-attentively sensitive to short-term delivery statistics of artificial stimuli, but it is unknown if this phenomenon extends to the level of behaviorally relevant delivery patterns, such as those used during communication. METHODOLOGY/PRINCIPAL FINDINGS: We recorded and analyzed complete auditory scenes of spontaneously communicating zebra finch (Taeniopygia guttata pairs over a week-long period, and show that they can produce tens of thousands of short-range contact calls per day. Individual calls recur at time scales (median interval 1.5 s matching those at which mammalian sensory systems are sensitive to recent stimulus history. Next, we presented to anesthetized birds sequences of frequently recurring calls interspersed with rare ones, and recorded, in parallel, action and local field potential responses in the medio-caudal auditory forebrain at 32 unique sites. Variation in call recurrence rate over natural ranges leads to widespread and significant modulation in strength of neural responses. Such modulation is highly call-specific in secondary auditory areas, but not in the main thalamo-recipient, primary auditory area. CONCLUSIONS/SIGNIFICANCE: Our results support the hypothesis that pre-attentive neural sensitivity to short-term stimulus recurrence is involved in the analysis of auditory scenes at the level of delivery patterns of meaningful sounds. This may enable birds to efficiently and automatically distinguish frequently recurring vocalizations from other events in their auditory scene.

  5. A recurrent dynamic model for correspondence-based face recognition.

    Science.gov (United States)

    Wolfrum, Philipp; Wolff, Christian; Lücke, Jörg; von der Malsburg, Christoph

    2008-12-29

    Our aim here is to create a fully neural, functionally competitive, and correspondence-based model for invariant face recognition. By recurrently integrating information about feature similarities, spatial feature relations, and facial structure stored in memory, the system evaluates face identity ("what"-information) and face position ("where"-information) using explicit representations for both. The network consists of three functional layers of processing, (1) an input layer for image representation, (2) a middle layer for recurrent information integration, and (3) a gallery layer for memory storage. Each layer consists of cortical columns as functional building blocks that are modeled in accordance with recent experimental findings. In numerical simulations we apply the system to standard benchmark databases for face recognition. We find that recognition rates of our biologically inspired approach lie in the same range as recognition rates of recent and purely functionally motivated systems.

  6. Lyapunov matrices approach to the parametric optimization of time-delay systems

    Directory of Open Access Journals (Sweden)

    Duda Józef

    2015-09-01

    Full Text Available In the paper a Lyapunov matrices approach to the parametric optimization problem of time-delay systems with a P-controller is presented. The value of integral quadratic performance index of quality is equal to the value of Lyapunov functional for the initial function of the time-delay system. The Lyapunov functional is determined by means of the Lyapunov matrix

  7. Lyapunov-based decentralized control of a rougher flotation phenomenological simulator

    International Nuclear Information System (INIS)

    Benaskeur, A.R.; Desbiens, A.

    1999-01-01

    In this paper a new approach to decentralized control of linear two-by-two plants is presented. The novelty lies in the use of a modified control function of Lyapunov and the introduction of an integral action in each manipulated variable, to ensure zero tracking errors. An appropriate choice of the regulated errors, allows the elimination of the cross terms in the obtained backstepping-based multivariable controller. It will be proven that if the H ∞ -norm of the plant interaction quotient is less than one, the centralized controller can be split up into two independent scalar output feedback regulators. Under these conditions, the global stability and zero tracking errors will still be guaranteed. The developed scheme is successfully applied to the control of a rougher flotation phenomenological simulator. (author)

  8. Full spectrum of Lyapunov exponents in gauge field theories

    International Nuclear Information System (INIS)

    Biro, T.S.; Markum, H.; Pullirsch, R.

    2003-01-01

    Full text: Results are presented for the full spectrum of Lyapunov exponents of the compact U(1) gauge system in classical field theory. Instead of the determination of the largest Lyapunov exponent by the rescaling method we now use the monodromy matrix approach. The Lyapunov spectrum L i is expressed in terms of the eigenvalues Λ i of the monodromy matrix M. In the confinement phase the eigenvalues lie on either the real or on the imaginary axes. This is a nice illustration of a strange attractor of a chaotic system. Positive Lyapunov exponents eject the trajectories from oscillating orbits provided by the imaginary eigenvalues. Negative Lyapunov exponents attract the trajectories keeping them confined in the basin. Latest studies concern the time (in)dependence of the monodromy matrix. Further, we show that monopoles are created and annihilated in pairs as a function of real time in access to a fixed average monopole number. (author)

  9. A one-layer recurrent neural network for constrained pseudoconvex optimization and its application for dynamic portfolio optimization.

    Science.gov (United States)

    Liu, Qingshan; Guo, Zhishan; Wang, Jun

    2012-02-01

    In this paper, a one-layer recurrent neural network is proposed for solving pseudoconvex optimization problems subject to linear equality and bound constraints. Compared with the existing neural networks for optimization (e.g., the projection neural networks), the proposed neural network is capable of solving more general pseudoconvex optimization problems with equality and bound constraints. Moreover, it is capable of solving constrained fractional programming problems as a special case. The convergence of the state variables of the proposed neural network to achieve solution optimality is guaranteed as long as the designed parameters in the model are larger than the derived lower bounds. Numerical examples with simulation results illustrate the effectiveness and characteristics of the proposed neural network. In addition, an application for dynamic portfolio optimization is discussed. Copyright © 2011 Elsevier Ltd. All rights reserved.

  10. Global stability analysis of epidemiological models based on Volterra–Lyapunov stable matrices

    International Nuclear Information System (INIS)

    Liao Shu; Wang Jin

    2012-01-01

    Highlights: ► Global dynamics of high dimensional dynamical systems. ► A systematic approach for global stability analysis. ► Epidemiological models of environment-dependent diseases. - Abstract: In this paper, we study the global dynamics of a class of mathematical epidemiological models formulated by systems of differential equations. These models involve both human population and environmental component(s) and constitute high-dimensional nonlinear autonomous systems, for which the global asymptotic stability of the endemic equilibria has been a major challenge in analyzing the dynamics. By incorporating the theory of Volterra–Lyapunov stable matrices into the classical method of Lyapunov functions, we present an approach for global stability analysis and obtain new results on some three- and four-dimensional model systems. In addition, we conduct numerical simulation to verify the analytical results.

  11. Lyapunov analysis: from dynamical systems theory to applications

    Science.gov (United States)

    Cencini, Massimo; Ginelli, Francesco

    2013-06-01

    stable and unstable directions associated with Lyapunov exponents. Recently, novel and efficient algorithms for computing CLVs have become available, turning them into a practical tool for the study and characterization of chaotic dynamical systems, especially when many degrees of freedom are considered. Applications include—but are not limited to—the characterization of collective chaotic modes and the study of intermittency in shell models for turbulence. This section also includes a pedagogical discussion of CLVs in low dimensional Hamiltonian systems. Time series analysis. The computation of Lyapunov exponents from time series data is a very active field of research playing a crucial role in experimental studies, with applications ranging from nonlinear optics to neuroscience. The need to reconstruct a higher dimensional phase space from scalar or lower dimensional data poses fundamental issues, introducing spurious Lyapunov exponents and limiting the accuracy when quantities such as the Kaplan-Yorke-dimension or the Kolmogorov-Sinai entropy are concerned. Recent advances based on CLVs, however, may help in identifying such spurious exponents, increasing the accuracy of embedding techniques. Many-body systems. The study of many-body dynamical systems plays a fundamental role in the debate on the foundations of statistical physics. In this respect, the discovery of long wavelength tangent space modes associated with close-to-zero Lyapunov exponents, the so-called hydrodynamic Lyapunov modes, has raised hopes that a clear link could be established between hydrodynamic transport coefficients and microscopic dynamical properties. In this section, three contributions discuss Lyapunov analysis applied to many-body systems such as hard disks and lattices of interacting classical spins. Spatiotemporal chaos and hyperbolicity. Spatially extended chaotic systems pose a number of new issues when spatiotemporal chaos is considered, ranging from questions on its extensivity

  12. Lyapunov exponents and smooth ergodic theory

    CERN Document Server

    Barreira, Luis

    2001-01-01

    This book is a systematic introduction to smooth ergodic theory. The topics discussed include the general (abstract) theory of Lyapunov exponents and its applications to the stability theory of differential equations, stable manifold theory, absolute continuity, and the ergodic theory of dynamical systems with nonzero Lyapunov exponents (including geodesic flows). The authors consider several non-trivial examples of dynamical systems with nonzero Lyapunov exponents to illustrate some basic methods and ideas of the theory. This book is self-contained. The reader needs a basic knowledge of real analysis, measure theory, differential equations, and topology. The authors present basic concepts of smooth ergodic theory and provide complete proofs of the main results. They also state some more advanced results to give readers a broader view of smooth ergodic theory. This volume may be used by those nonexperts who wish to become familiar with the field.

  13. Stability analysis of polynomial fuzzy models via polynomial fuzzy Lyapunov functions

    OpenAIRE

    Bernal Reza, Miguel Ángel; Sala, Antonio; JAADARI, ABDELHAFIDH; Guerra, Thierry-Marie

    2011-01-01

    In this paper, the stability of continuous-time polynomial fuzzy models by means of a polynomial generalization of fuzzy Lyapunov functions is studied. Fuzzy Lyapunov functions have been fruitfully used in the literature for local analysis of Takagi-Sugeno models, a particular class of the polynomial fuzzy ones. Based on a recent Taylor-series approach which allows a polynomial fuzzy model to exactly represent a nonlinear model in a compact set of the state space, it is shown that a refinemen...

  14. Recurrent-neural-network-based identification of a cascade hydraulic actuator for closed-loop automotive power transmission control

    International Nuclear Information System (INIS)

    You, Seung Han; Hahn, Jin Oh

    2012-01-01

    By virtue of its ease of operation compared with its conventional manual counterpart, automatic transmissions are commonly used as automotive power transmission control system in today's passenger cars. In accordance with this trend, research efforts on closed-loop automatic transmission controls have been extensively carried out to improve ride quality and fuel economy. State-of-the-art power transmission control algorithms may have limitations in performance because they rely on the steady-state characteristics of the hydraulic actuator rather than fully exploit its dynamic characteristics. Since the ultimate viability of closed-loop power transmission control is dominated by precise pressure control at the level of hydraulic actuator, closed-loop control can potentially attain superior efficacy in case the hydraulic actuator can be easily incorporated into model-based observer/controller design. In this paper, we propose to use a recurrent neural network (RNN) to establish a nonlinear empirical model of a cascade hydraulic actuator in a passenger car automatic transmission, which has potential to be easily incorporated in designing observers and controllers. Experimental analysis is performed to grasp key system characteristics, based on which a nonlinear system identification procedure is carried out. Extensive experimental validation of the established model suggests that it has superb one-step-ahead prediction capability over appropriate frequency range, making it an attractive approach for model-based observer/controller design applications in automotive systems

  15. Optimizing Markovian modeling of chaotic systems with recurrent neural networks

    International Nuclear Information System (INIS)

    Cechin, Adelmo L.; Pechmann, Denise R.; Oliveira, Luiz P.L. de

    2008-01-01

    In this paper, we propose a methodology for optimizing the modeling of an one-dimensional chaotic time series with a Markov Chain. The model is extracted from a recurrent neural network trained for the attractor reconstructed from the data set. Each state of the obtained Markov Chain is a region of the reconstructed state space where the dynamics is approximated by a specific piecewise linear map, obtained from the network. The Markov Chain represents the dynamics of the time series in its statistical essence. An application to a time series resulted from Lorenz system is included

  16. Online Signature Verification using Recurrent Neural Network and Length-normalized Path Signature

    OpenAIRE

    Lai, Songxuan; Jin, Lianwen; Yang, Weixin

    2017-01-01

    Inspired by the great success of recurrent neural networks (RNNs) in sequential modeling, we introduce a novel RNN system to improve the performance of online signature verification. The training objective is to directly minimize intra-class variations and to push the distances between skilled forgeries and genuine samples above a given threshold. By back-propagating the training signals, our RNN network produced discriminative features with desired metrics. Additionally, we propose a novel d...

  17. Neural adaptive control for vibration suppression in composite fin-tip of aircraft.

    Science.gov (United States)

    Suresh, S; Kannan, N; Sundararajan, N; Saratchandran, P

    2008-06-01

    In this paper, we present a neural adaptive control scheme for active vibration suppression of a composite aircraft fin tip. The mathematical model of a composite aircraft fin tip is derived using the finite element approach. The finite element model is updated experimentally to reflect the natural frequencies and mode shapes very accurately. Piezo-electric actuators and sensors are placed at optimal locations such that the vibration suppression is a maximum. Model-reference direct adaptive neural network control scheme is proposed to force the vibration level within the minimum acceptable limit. In this scheme, Gaussian neural network with linear filters is used to approximate the inverse dynamics of the system and the parameters of the neural controller are estimated using Lyapunov based update law. In order to reduce the computational burden, which is critical for real-time applications, the number of hidden neurons is also estimated in the proposed scheme. The global asymptotic stability of the overall system is ensured using the principles of Lyapunov approach. Simulation studies are carried-out using sinusoidal force functions of varying frequency. Experimental results show that the proposed neural adaptive control scheme is capable of providing significant vibration suppression in the multiple bending modes of interest. The performance of the proposed scheme is better than the H(infinity) control scheme.

  18. Stability analysis of delayed Cohen-Grossberg BAM neural networks with impulses via nonsmooth analysis

    International Nuclear Information System (INIS)

    Wen Zhen; Sun Jitao

    2009-01-01

    In this paper, we investigate the existence and uniqueness of equilibrium point for delayed Cohen-Grossberg bidirectional associative memory (BAM) neural networks with impulses, based on nonsmooth analysis method. And we give the criteria of global exponential stability of the unique equilibrium point for the delayed BAM neural networks with impulses using Lyapunov method. The new sufficient condition generalizes and improves the previously known results. Finally, we present examples to illustrate that our results are effective.

  19. On global exponential stability of high-order neural networks with time-varying delays

    International Nuclear Information System (INIS)

    Zhang Baoyong; Xu Shengyuan; Li Yongmin; Chu Yuming

    2007-01-01

    This Letter investigates the problem of stability analysis for a class of high-order neural networks with time-varying delays. The delays are bounded but not necessarily differentiable. Based on the Lyapunov stability theory together with the linear matrix inequality (LMI) approach and the use of Halanay inequality, sufficient conditions guaranteeing the global exponential stability of the equilibrium point of the considered neural networks are presented. Two numerical examples are provided to demonstrate the effectiveness of the proposed stability criteria

  20. On global exponential stability of high-order neural networks with time-varying delays

    Energy Technology Data Exchange (ETDEWEB)

    Zhang Baoyong [School of Automation, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu (China)]. E-mail: baoyongzhang@yahoo.com.cn; Xu Shengyuan [School of Automation, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu (China)]. E-mail: syxu02@yahoo.com.cn; Li Yongmin [School of Automation, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu (China) and Department of Mathematics, Huzhou Teacher' s College, Huzhou 313000, Zhejiang (China)]. E-mail: ymlwww@163.com; Chu Yuming [Department of Mathematics, Huzhou Teacher' s College, Huzhou 313000, Zhejiang (China)

    2007-06-18

    This Letter investigates the problem of stability analysis for a class of high-order neural networks with time-varying delays. The delays are bounded but not necessarily differentiable. Based on the Lyapunov stability theory together with the linear matrix inequality (LMI) approach and the use of Halanay inequality, sufficient conditions guaranteeing the global exponential stability of the equilibrium point of the considered neural networks are presented. Two numerical examples are provided to demonstrate the effectiveness of the proposed stability criteria.

  1. arXiv The prototype of the HL-LHC magnets monitoring system based on Recurrent Neural Networks and adaptive quantization

    CERN Document Server

    Wielgosz, Maciej; Skoczeń, Andrzej

    This paper focuses on an examination of an applicability of Recurrent Neural Network models for detecting anomalous behavior of the CERN superconducting magnets. In order to conduct the experiments, the authors designed and implemented an adaptive signal quantization algorithm and a custom GRU-based detector and developed a method for the detector parameters selection. Three different datasets were used for testing the detector. Two artificially generated datasets were used to assess the raw performance of the system whereas the 231 MB dataset composed of the signals acquired from HiLumi magnets was intended for real-life experiments and model training. Several different setups of the developed anomaly detection system were evaluated and compared with state-of-the-art OC-SVM reference model operating on the same data. The OC-SVM model was equipped with a rich set of feature extractors accounting for a range of the input signal properties. It was determined in the course of the experiments that the detector, a...

  2. Impulsive effects on global asymptotic stability of delay BAM neural networks

    International Nuclear Information System (INIS)

    Chen Jun; Cui Baotong

    2008-01-01

    Based on the proper Lyapunov functions and the Jacobsthal liner inequality, some sufficient conditions are presented in this paper for global asymptotic stability of delay bidirectional associative memory neural networks with impulses. The obtained results are independently of the delay parameters and can be easily verified. Also, some remarks and an illustrative example are given to demonstrate the effectiveness of the obtained results

  3. Sequential neural models with stochastic layers

    DEFF Research Database (Denmark)

    Fraccaro, Marco; Sønderby, Søren Kaae; Paquet, Ulrich

    2016-01-01

    How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural...... generative model. The clear separation of deterministic and stochastic layers allows a structured variational inference network to track the factorization of the model's posterior distribution. By retaining both the nonlinear recursive structure of a recurrent neural network and averaging over...

  4. Neural network-based nonlinear model predictive control vs. linear quadratic gaussian control

    Science.gov (United States)

    Cho, C.; Vance, R.; Mardi, N.; Qian, Z.; Prisbrey, K.

    1997-01-01

    One problem with the application of neural networks to the multivariable control of mineral and extractive processes is determining whether and how to use them. The objective of this investigation was to compare neural network control to more conventional strategies and to determine if there are any advantages in using neural network control in terms of set-point tracking, rise time, settling time, disturbance rejection and other criteria. The procedure involved developing neural network controllers using both historical plant data and simulation models. Various control patterns were tried, including both inverse and direct neural network plant models. These were compared to state space controllers that are, by nature, linear. For grinding and leaching circuits, a nonlinear neural network-based model predictive control strategy was superior to a state space-based linear quadratic gaussian controller. The investigation pointed out the importance of incorporating state space into neural networks by making them recurrent, i.e., feeding certain output state variables into input nodes in the neural network. It was concluded that neural network controllers can have better disturbance rejection, set-point tracking, rise time, settling time and lower set-point overshoot, and it was also concluded that neural network controllers can be more reliable and easy to implement in complex, multivariable plants.

  5. Stability analysis for stochastic BAM nonlinear neural network with delays

    Science.gov (United States)

    Lv, Z. W.; Shu, H. S.; Wei, G. L.

    2008-02-01

    In this paper, stochastic bidirectional associative memory neural networks with constant or time-varying delays is considered. Based on a Lyapunov-Krasovskii functional and the stochastic stability analysis theory, we derive several sufficient conditions in order to guarantee the global asymptotically stable in the mean square. Our investigation shows that the stochastic bidirectional associative memory neural networks are globally asymptotically stable in the mean square if there are solutions to some linear matrix inequalities(LMIs). Hence, the global asymptotic stability of the stochastic bidirectional associative memory neural networks can be easily checked by the Matlab LMI toolbox. A numerical example is given to demonstrate the usefulness of the proposed global asymptotic stability criteria.

  6. Stability analysis for stochastic BAM nonlinear neural network with delays

    International Nuclear Information System (INIS)

    Lv, Z W; Shu, H S; Wei, G L

    2008-01-01

    In this paper, stochastic bidirectional associative memory neural networks with constant or time-varying delays is considered. Based on a Lyapunov-Krasovskii functional and the stochastic stability analysis theory, we derive several sufficient conditions in order to guarantee the global asymptotically stable in the mean square. Our investigation shows that the stochastic bidirectional associative memory neural networks are globally asymptotically stable in the mean square if there are solutions to some linear matrix inequalities(LMIs). Hence, the global asymptotic stability of the stochastic bidirectional associative memory neural networks can be easily checked by the Matlab LMI toolbox. A numerical example is given to demonstrate the usefulness of the proposed global asymptotic stability criteria

  7. Integrated built-in-test false and missed alarms reduction based on forward infinite impulse response & recurrent finite impulse response dynamic neural networks

    Science.gov (United States)

    Cui, Yiqian; Shi, Junyou; Wang, Zili

    2017-11-01

    Built-in tests (BITs) are widely used in mechanical systems to perform state identification, whereas the BIT false and missed alarms cause trouble to the operators or beneficiaries to make correct judgments. Artificial neural networks (ANN) are previously used for false and missed alarms identification, which has the features such as self-organizing and self-study. However, these ANN models generally do not incorporate the temporal effect of the bottom-level threshold comparison outputs and the historical temporal features are not fully considered. To improve the situation, this paper proposes a new integrated BIT design methodology by incorporating a novel type of dynamic neural networks (DNN) model. The new DNN model is termed as Forward IIR & Recurrent FIR DNN (FIRF-DNN), where its component neurons, network structures, and input/output relationships are discussed. The condition monitoring false and missed alarms reduction implementation scheme based on FIRF-DNN model is also illustrated, which is composed of three stages including model training, false and missed alarms detection, and false and missed alarms suppression. Finally, the proposed methodology is demonstrated in the application study and the experimental results are analyzed.

  8. A Lyapunov theory based UPFC controller for power flow control

    Energy Technology Data Exchange (ETDEWEB)

    Zangeneh, Ali; Kazemi, Ahad; Hajatipour, Majid; Jadid, Shahram [Center of Excellence for Power Systems Automation and Operation, Iran University of Science and Technology, Tehran (Iran)

    2009-09-15

    Unified power flow controller (UPFC) is the most comprehensive multivariable device among the FACTS controllers. Capability of power flow control is the most important responsibility of UPFC. According to high importance of power flow control in transmission lines, the proper controller should be robust against uncertainty and disturbance and also have suitable settling time. For this purpose, a new controller is designed based on the Lyapunov theory and its stability is also evaluated. The Main goal of this paper is to design a controller which enables a power system to track reference signals precisely and to be robust in the presence of uncertainty of system parameters and disturbances. The performance of the proposed controller is simulated on a two bus test system and compared with a conventional PI controller. The simulation results show the power and accuracy of the proposed controller. (author)

  9. Using Long-Short-Term-Memory Recurrent Neural Networks to Predict Aviation Engine Vibrations

    Science.gov (United States)

    ElSaid, AbdElRahman Ahmed

    This thesis examines building viable Recurrent Neural Networks (RNN) using Long Short Term Memory (LSTM) neurons to predict aircraft engine vibrations. The different networks are trained on a large database of flight data records obtained from an airline containing flights that suffered from excessive vibration. RNNs can provide a more generalizable and robust method for prediction over analytical calculations of engine vibration, as analytical calculations must be solved iteratively based on specific empirical engine parameters, and this database contains multiple types of engines. Further, LSTM RNNs provide a "memory" of the contribution of previous time series data which can further improve predictions of future vibration values. LSTM RNNs were used over traditional RNNs, as those suffer from vanishing/exploding gradients when trained with back propagation. The study managed to predict vibration values for 1, 5, 10, and 20 seconds in the future, with 2.84% 3.3%, 5.51% and 10.19% mean absolute error, respectively. These neural networks provide a promising means for the future development of warning systems so that suitable actions can be taken before the occurrence of excess vibration to avoid unfavorable situations during flight.

  10. Neural networks dynamic hysteresis model for piezoceramic actuator based on hysteresis operator of first-order differential equation

    International Nuclear Information System (INIS)

    Dang Xuanju; Tan Yonghong

    2005-01-01

    A new neural networks dynamic hysteresis model for piezoceramic actuator is proposed by combining the Preisach model with diagonal recurrent neural networks. The Preisach model is based on elementary rate-independent operators and is not suitable for modeling piezoceramic actuator across a wide frequency band because of the rate-dependent hysteresis characteristic of the piezoceramic actuator. The structure of the developed model is based on the structure of the Preisach model, in which the rate-independent relay hysteresis operators (cells) are replaced by the rate-dependent hysteresis operators of first-order differential equation. The diagonal recurrent neural networks being modified by an adjustable factor can be used to model the hysteresis behavior of the pizeoceramic actuator because its structure is similar to the structure of the modified Preisach model. Therefore, the proposed model not only possesses that of the Preisach model, but also can be used for describing its dynamic hysteresis behavior. Through the experimental results of both the approximation and the prediction, the effectiveness of the neural networks dynamic hysteresis model for the piezoceramic actuator is demonstrated

  11. Deep Recurrent Neural Networks for Supernovae Classification

    Science.gov (United States)

    Charnock, Tom; Moss, Adam

    2017-03-01

    We apply deep recurrent neural networks, which are capable of learning complex sequential information, to classify supernovae (code available at https://github.com/adammoss/supernovae). The observational time and filter fluxes are used as inputs to the network, but since the inputs are agnostic, additional data such as host galaxy information can also be included. Using the Supernovae Photometric Classification Challenge (SPCC) data, we find that deep networks are capable of learning about light curves, however the performance of the network is highly sensitive to the amount of training data. For a training size of 50% of the representational SPCC data set (around 104 supernovae) we obtain a type-Ia versus non-type-Ia classification accuracy of 94.7%, an area under the Receiver Operating Characteristic curve AUC of 0.986 and an SPCC figure-of-merit F 1 = 0.64. When using only the data for the early-epoch challenge defined by the SPCC, we achieve a classification accuracy of 93.1%, AUC of 0.977, and F 1 = 0.58, results almost as good as with the whole light curve. By employing bidirectional neural networks, we can acquire impressive classification results between supernovae types I, II and III at an accuracy of 90.4% and AUC of 0.974. We also apply a pre-trained model to obtain classification probabilities as a function of time and show that it can give early indications of supernovae type. Our method is competitive with existing algorithms and has applications for future large-scale photometric surveys.

  12. Globally exponential stability condition of a class of neural networks with time-varying delays

    International Nuclear Information System (INIS)

    Liao, T.-L.; Yan, J.-J.; Cheng, C.-J.; Hwang, C.-C.

    2005-01-01

    In this Letter, the globally exponential stability for a class of neural networks including Hopfield neural networks and cellular neural networks with time-varying delays is investigated. Based on the Lyapunov stability method, a novel and less conservative exponential stability condition is derived. The condition is delay-dependent and easily applied only by checking the Hamiltonian matrix with no eigenvalues on the imaginary axis instead of directly solving an algebraic Riccati equation. Furthermore, the exponential stability degree is more easily assigned than those reported in the literature. Some examples are given to demonstrate validity and excellence of the presented stability condition herein

  13. Intelligent fault diagnosis of rolling bearings using an improved deep recurrent neural network

    Science.gov (United States)

    Jiang, Hongkai; Li, Xingqiu; Shao, Haidong; Zhao, Ke

    2018-06-01

    Traditional intelligent fault diagnosis methods for rolling bearings heavily depend on manual feature extraction and feature selection. For this purpose, an intelligent deep learning method, named the improved deep recurrent neural network (DRNN), is proposed in this paper. Firstly, frequency spectrum sequences are used as inputs to reduce the input size and ensure good robustness. Secondly, DRNN is constructed by the stacks of the recurrent hidden layer to automatically extract the features from the input spectrum sequences. Thirdly, an adaptive learning rate is adopted to improve the training performance of the constructed DRNN. The proposed method is verified with experimental rolling bearing data, and the results confirm that the proposed method is more effective than traditional intelligent fault diagnosis methods.

  14. Training Excitatory-Inhibitory Recurrent Neural Networks for Cognitive Tasks: A Simple and Flexible Framework.

    Directory of Open Access Journals (Sweden)

    H Francis Song

    2016-02-01

    Full Text Available The ability to simultaneously record from large numbers of neurons in behaving animals has ushered in a new era for the study of the neural circuit mechanisms underlying cognitive functions. One promising approach to uncovering the dynamical and computational principles governing population responses is to analyze model recurrent neural networks (RNNs that have been optimized to perform the same tasks as behaving animals. Because the optimization of network parameters specifies the desired output but not the manner in which to achieve this output, "trained" networks serve as a source of mechanistic hypotheses and a testing ground for data analyses that link neural computation to behavior. Complete access to the activity and connectivity of the circuit, and the ability to manipulate them arbitrarily, make trained networks a convenient proxy for biological circuits and a valuable platform for theoretical investigation. However, existing RNNs lack basic biological features such as the distinction between excitatory and inhibitory units (Dale's principle, which are essential if RNNs are to provide insights into the operation of biological circuits. Moreover, trained networks can achieve the same behavioral performance but differ substantially in their structure and dynamics, highlighting the need for a simple and flexible framework for the exploratory training of RNNs. Here, we describe a framework for gradient descent-based training of excitatory-inhibitory RNNs that can incorporate a variety of biological knowledge. We provide an implementation based on the machine learning library Theano, whose automatic differentiation capabilities facilitate modifications and extensions. We validate this framework by applying it to well-known experimental paradigms such as perceptual decision-making, context-dependent integration, multisensory integration, parametric working memory, and motor sequence generation. Our results demonstrate the wide range of neural

  15. Training Excitatory-Inhibitory Recurrent Neural Networks for Cognitive Tasks: A Simple and Flexible Framework

    Science.gov (United States)

    Wang, Xiao-Jing

    2016-01-01

    The ability to simultaneously record from large numbers of neurons in behaving animals has ushered in a new era for the study of the neural circuit mechanisms underlying cognitive functions. One promising approach to uncovering the dynamical and computational principles governing population responses is to analyze model recurrent neural networks (RNNs) that have been optimized to perform the same tasks as behaving animals. Because the optimization of network parameters specifies the desired output but not the manner in which to achieve this output, “trained” networks serve as a source of mechanistic hypotheses and a testing ground for data analyses that link neural computation to behavior. Complete access to the activity and connectivity of the circuit, and the ability to manipulate them arbitrarily, make trained networks a convenient proxy for biological circuits and a valuable platform for theoretical investigation. However, existing RNNs lack basic biological features such as the distinction between excitatory and inhibitory units (Dale’s principle), which are essential if RNNs are to provide insights into the operation of biological circuits. Moreover, trained networks can achieve the same behavioral performance but differ substantially in their structure and dynamics, highlighting the need for a simple and flexible framework for the exploratory training of RNNs. Here, we describe a framework for gradient descent-based training of excitatory-inhibitory RNNs that can incorporate a variety of biological knowledge. We provide an implementation based on the machine learning library Theano, whose automatic differentiation capabilities facilitate modifications and extensions. We validate this framework by applying it to well-known experimental paradigms such as perceptual decision-making, context-dependent integration, multisensory integration, parametric working memory, and motor sequence generation. Our results demonstrate the wide range of neural activity

  16. Lyapunov functions for the fixed points of the Lorenz model

    International Nuclear Information System (INIS)

    Bakasov, A.A.; Govorkov, B.B. Jr.

    1992-11-01

    We have shown how the explicit Lyapunov functions can be constructed in the framework of a regular procedure suggested and completed by Lyapunov a century ago (''method of critical cases''). The method completely covers all practically encountering subtle cases of stability study for ordinary differential equations when the linear stability analysis fails. These subtle cases, ''the critical cases'', according to Lyapunov, include both bifurcations of solutions and solutions of systems with symmetry. Being properly specialized and actually powerful in case of ODE's, this Lyapunov's method is formulated in simple language and should attract a wide interest of the physical audience. The method leads to inevitable construction of the explicit Lyapunov function, takes automatically into account the Fredholm alternative and avoids infinite step calculations. Easy and apparent physical interpretation of the Lyapunov function as a potential or as a time-dependent entropy provides one with more details about the local dynamics of the system at non-equilibrium phase transition points. Another advantage is that this Lyapunov's method consists of a set of very detailed explicit prescriptions which allow one to easy programmize the method for a symbolic processor. The application of the Lyapunov theory for critical cases has been done in this work to the real Lorenz equations and it is shown, in particular, that increasing σ at the Hopf bifurcation point suppresses the contribution of one of the variables to the destabilization of the system. The relation of the method to contemporary methods and its place among them have been clearly and extensively discussed. Due to Appendices, the paper is self-contained and does not require from a reader to approach results published only in Russian. (author). 38 refs

  17. A Fuzzy Neural Network Based on Non-Euclidean Distance Clustering for Quality Index Model in Slashing Process

    Directory of Open Access Journals (Sweden)

    Yuxian Zhang

    2015-01-01

    Full Text Available The quality index model in slashing process is difficult to build by reason of the outliers and noise data from original data. To the above problem, a fuzzy neural network based on non-Euclidean distance clustering is proposed in which the input space is partitioned into many local regions by the fuzzy clustering based on non-Euclidean distance so that the computation complexity is decreased, and fuzzy rule number is determined by validity function based on both the separation and the compactness among clusterings. Then, the premise parameters and consequent parameters are trained by hybrid learning algorithm. The parameters identification is realized; meanwhile the convergence condition of consequent parameters is obtained by Lyapunov function. Finally, the proposed method is applied to build the quality index model in slashing process in which the experimental data come from the actual slashing process. The experiment results show that the proposed fuzzy neural network for quality index model has lower computation complexity and faster convergence time, comparing with GP-FNN, BPNN, and RBFNN.

  18. Improved Criteria on Delay-Dependent Stability for Discrete-Time Neural Networks with Interval Time-Varying Delays

    Directory of Open Access Journals (Sweden)

    O. M. Kwon

    2012-01-01

    Full Text Available The purpose of this paper is to investigate the delay-dependent stability analysis for discrete-time neural networks with interval time-varying delays. Based on Lyapunov method, improved delay-dependent criteria for the stability of the networks are derived in terms of linear matrix inequalities (LMIs by constructing a suitable Lyapunov-Krasovskii functional and utilizing reciprocally convex approach. Also, a new activation condition which has not been considered in the literature is proposed and utilized for derivation of stability criteria. Two numerical examples are given to illustrate the effectiveness of the proposed method.

  19. Some criteria for robust stability of Cohen-Grossberg neural networks with delays

    International Nuclear Information System (INIS)

    Xiong Weili; Xu Baoguo

    2008-01-01

    This paper considers the problem of robust stability of Cohen-Grossberg neural networks with time-varying delays. Based on the Lyapunov stability theory and linear matrix inequality (LMI) technique, some sufficient conditions are derived to ensure the global robust convergence of the equilibrium point. The proposed LMI conditions can be checked easily by recently developed algorithms solving LMIs. Comparisons between our results and previous results admits our results establish a new set of stability criteria for delayed Cohen-Grossberg neural networks. Numerical examples are given to illustrate the effectiveness of our results

  20. Anti-synchronization control of BAM memristive neural networks with multiple proportional delays and stochastic perturbations

    Science.gov (United States)

    Wang, Weiping; Yuan, Manman; Luo, Xiong; Liu, Linlin; Zhang, Yao

    2018-01-01

    Proportional delay is a class of unbounded time-varying delay. A class of bidirectional associative memory (BAM) memristive neural networks with multiple proportional delays is concerned in this paper. First, we propose the model of BAM memristive neural networks with multiple proportional delays and stochastic perturbations. Furthermore, by choosing suitable nonlinear variable transformations, the BAM memristive neural networks with multiple proportional delays can be transformed into the BAM memristive neural networks with constant delays. Based on the drive-response system concept, differential inclusions theory and Lyapunov stability theory, some anti-synchronization criteria are obtained. Finally, the effectiveness of proposed criteria are demonstrated through numerical examples.

  1. Switched-Observer-Based Adaptive Neural Control of MIMO Switched Nonlinear Systems With Unknown Control Gains.

    Science.gov (United States)

    Long, Lijun; Zhao, Jun

    2017-07-01

    In this paper, the problem of adaptive neural output-feedback control is addressed for a class of multi-input multioutput (MIMO) switched uncertain nonlinear systems with unknown control gains. Neural networks (NNs) are used to approximate unknown nonlinear functions. In order to avoid the conservativeness caused by adoption of a common observer for all subsystems, an MIMO NN switched observer is designed to estimate unmeasurable states. A new switched observer-based adaptive neural control technique for the problem studied is then provided by exploiting the classical average dwell time (ADT) method and the backstepping method and the Nussbaum gain technique. It effectively handles the obstacle about the coexistence of multiple Nussbaum-type function terms, and improves the classical ADT method, since the exponential decline property of Lyapunov functions for individual subsystems is no longer satisfied. It is shown that the technique proposed is able to guarantee semiglobal uniformly ultimately boundedness of all the signals in the closed-loop system under a class of switching signals with ADT, and the tracking errors converge to a small neighborhood of the origin. The effectiveness of the approach proposed is illustrated by its application to a two inverted pendulum system.

  2. Analysis and control of chaotic behavior in boost converter by ramp compensation based on Lyapunov exponents assignment: theoretical and experimental investigation

    International Nuclear Information System (INIS)

    Zamani, Najmeh; Ataei, Mohammad; Niroomand, Mehdi

    2015-01-01

    Highlights: • Applying nonlinear analysis of complex dynamics displayed by current-mode controlled boost converter. • The ramp compensation method is used to control bifurcation and chaos in these converters based on bifurcation diagram and Lyapunov exponents assignment. • A discrete-time iterative nonlinear mapping model has been derived by inserting the ramp compensation parameter in the dynamical equations of the system. • A design methodology for chaos control is provided in this converter based on Lyapunov exponents assignment in desired values theoretically by proper selection of compensator slope. • Practical results are provided to confirm the theoretical analysis and simulations. - Abstract: Nonlinear analysis of complex dynamics displayed by current mode dc–dc converter and idea of Lyapunov exponents assignment by ramp compensator in order to control chaotic behavior is proposed in this article. A discrete-time iterative nonlinear mapping model is derived. The occurrence of the complex behaviors of bifurcation and chaos generated by varying the circuit parameters are investigated through numerical analysis and software implementation of the circuit. Next, in order to control bifurcation and chaos in these converters, the ramp compensation method is used. By inserting the ramp compensation parameter in the dynamical equations of the system, these complex behaviors are examined theoretically and numerically as well. It is proved that through this method, the stable period-one operation of the converter can be extended. By evaluating the Lyapunov exponents (LEs) of the system, the impact of the slope on the location of LEs are determined analytically. This leads to a design methodology for control of chaos in this converter based on LEs assignment in desired values by proper selection of compensator slope. By developing an experimental set up, practical results are obtained to confirm the theoretical analysis and simulations.

  3. Deep recurrent neural network reveals a hierarchy of process memory during dynamic natural vision.

    Science.gov (United States)

    Shi, Junxing; Wen, Haiguang; Zhang, Yizhen; Han, Kuan; Liu, Zhongming

    2018-05-01

    The human visual cortex extracts both spatial and temporal visual features to support perception and guide behavior. Deep convolutional neural networks (CNNs) provide a computational framework to model cortical representation and organization for spatial visual processing, but unable to explain how the brain processes temporal information. To overcome this limitation, we extended a CNN by adding recurrent connections to different layers of the CNN to allow spatial representations to be remembered and accumulated over time. The extended model, or the recurrent neural network (RNN), embodied a hierarchical and distributed model of process memory as an integral part of visual processing. Unlike the CNN, the RNN learned spatiotemporal features from videos to enable action recognition. The RNN better predicted cortical responses to natural movie stimuli than the CNN, at all visual areas, especially those along the dorsal stream. As a fully observable model of visual processing, the RNN also revealed a cortical hierarchy of temporal receptive window, dynamics of process memory, and spatiotemporal representations. These results support the hypothesis of process memory, and demonstrate the potential of using the RNN for in-depth computational understanding of dynamic natural vision. © 2018 Wiley Periodicals, Inc.

  4. Forecasting energy market indices with recurrent neural networks: Case study of crude oil price fluctuations

    International Nuclear Information System (INIS)

    Wang, Jie; Wang, Jun

    2016-01-01

    In an attempt to improve the forecasting accuracy of crude oil price fluctuations, a new neural network architecture is established in this work which combines Multilayer perception and ERNN (Elman recurrent neural networks) with stochastic time effective function. ERNN is a time-varying predictive control system and is developed with the ability to keep memory of recent events in order to predict future output. The stochastic time effective function represents that the recent information has a stronger effect for the investors than the old information. With the established model the empirical research has a good performance in testing the predictive effects on four different time series indices. Compared to other models, the present model is possible to evaluate data from 1990s to today with extreme accuracy and speedy. The applied CID (complexity invariant distance) analysis and multiscale CID analysis, are provided as the new useful measures to evaluate a better predicting ability of the proposed model than other traditional models. - Highlights: • A new forecasting model is developed by a random Elman recurrent neural network. • The forecasting accuracy of crude oil price fluctuations is improved by the model. • The forecasting results of the proposed model are more accurate than compared models. • Two new distance analysis methods are applied to confirm the predicting results.

  5. Stochastic synchronization of coupled neural networks with intermittent control

    International Nuclear Information System (INIS)

    Yang Xinsong; Cao Jinde

    2009-01-01

    In this Letter, we study the exponential stochastic synchronization problem for coupled neural networks with stochastic noise perturbations. Based on Lyapunov stability theory, inequality techniques, the properties of Weiner process, and adding different intermittent controllers, several sufficient conditions are obtained to ensure exponential stochastic synchronization of coupled neural networks with or without coupling delays under stochastic perturbations. These stochastic synchronization criteria are expressed in terms of several lower-dimensional linear matrix inequalities (LMIs) and can be easily verified. Moreover, the results of this Letter are applicable to both directed and undirected weighted networks. A numerical example and its simulations are offered to show the effectiveness of our new results.

  6. The Multivariate Largest Lyapunov Exponent as an Age-Related Metric of Quiet Standing Balance

    Directory of Open Access Journals (Sweden)

    Kun Liu

    2015-01-01

    Full Text Available The largest Lyapunov exponent has been researched as a metric of the balance ability during human quiet standing. However, the sensitivity and accuracy of this measurement method are not good enough for clinical use. The present research proposes a metric of the human body’s standing balance ability based on the multivariate largest Lyapunov exponent which can quantify the human standing balance. The dynamic multivariate time series of ankle, knee, and hip were measured by multiple electrical goniometers. Thirty-six normal people of different ages participated in the test. With acquired data, the multivariate largest Lyapunov exponent was calculated. Finally, the results of the proposed approach were analysed and compared with the traditional method, for which the largest Lyapunov exponent and power spectral density from the centre of pressure were also calculated. The following conclusions can be obtained. The multivariate largest Lyapunov exponent has a higher degree of differentiation in differentiating balance in eyes-closed conditions. The MLLE value reflects the overall coordination between multisegment movements. Individuals of different ages can be distinguished by their MLLE values. The standing stability of human is reduced with the increment of age.

  7. Lyapunov vs. geometrical stability analysis of the Kepler and the restricted three body problems

    International Nuclear Information System (INIS)

    Yahalom, A.; Levitan, J.; Lewkowicz, M.; Horwitz, L.

    2011-01-01

    In this Letter we show that although the application of standard Lyapunov analysis predicts that completely integrable Kepler motion is unstable, the geometrical analysis of Horwitz et al. predicts the observed stability. This seems to us to provide evidence for both the incompleteness of the standard Lyapunov analysis and the strength of the geometrical analysis. Moreover, we apply this approach to the three body problem in which the third body is restricted to move on a circle of large radius which induces an adiabatic time dependent potential on the second body. This causes the second body to move in a very interesting and intricate but periodic trajectory; however, the standard Lyapunov analysis, as well as methods based on the parametric variation of curvature associated with the Jacobi metric, incorrectly predict chaotic behavior. The geometric approach predicts the correct stable motion in this case as well. - Highlights: → Lyapunov analysis predicts Kepler motion to be unstable. → Geometrical analysis predicts the observed stability. → Lyapunov analysis predicts chaotic behavior in restricted three body problem. → The geometric approach predicts the correct stable motion in restricted three body problem.

  8. Global exponential stability of BAM neural networks with time-varying delays: The discrete-time case

    Science.gov (United States)

    Raja, R.; Marshal Anthoni, S.

    2011-02-01

    This paper deals with the problem of stability analysis for a class of discrete-time bidirectional associative memory (BAM) neural networks with time-varying delays. By employing the Lyapunov functional and linear matrix inequality (LMI) approach, a new sufficient conditions is proposed for the global exponential stability of discrete-time BAM neural networks. The proposed LMI based results can be easily checked by LMI control toolbox. Moreover, an example is also provided to demonstrate the effectiveness of the proposed method.

  9. Stability of time-delay systems via Lyapunov functions

    Directory of Open Access Journals (Sweden)

    Carlos F. Alastruey

    2002-01-01

    Full Text Available In this paper, a Lyapunov function candidate is introduced for multivariable systems with inner delays, without assuming a priori stability for the nondelayed subsystem. By using this Lyapunov function, a controller is deduced. Such a controller utilizes an input–output description of the original system, a circumstance that facilitates practical applications of the proposed approach.

  10. Extended LaSalle's Invariance Principle for Full-Range Cellular Neural Networks

    Directory of Open Access Journals (Sweden)

    Mauro Di Marco

    2009-01-01

    Full Text Available In several relevant applications to the solution of signal processing tasks in real time, a cellular neural network (CNN is required to be convergent, that is, each solution should tend toward some equilibrium point. The paper develops a Lyapunov method, which is based on a generalized version of LaSalle's invariance principle, for studying convergence and stability of the differential inclusions modeling the dynamics of the full-range (FR model of CNNs. The applicability of the method is demonstrated by obtaining a rigorous proof of convergence for symmetric FR-CNNs. The proof, which is a direct consequence of the fact that a symmetric FR-CNN admits a strict Lyapunov function, is much more simple than the corresponding proof of convergence for symmetric standard CNNs.

  11. Relation Classification via Recurrent Neural Network

    OpenAIRE

    Zhang, Dongxu; Wang, Dong

    2015-01-01

    Deep learning has gained much success in sentence-level relation classification. For example, convolutional neural networks (CNN) have delivered competitive performance without much effort on feature engineering as the conventional pattern-based methods. Thus a lot of works have been produced based on CNN structures. However, a key issue that has not been well addressed by the CNN-based method is the lack of capability to learn temporal features, especially long-distance dependency between no...

  12. Modulational estimate for the maximal Lyapunov exponent in Fermi-Pasta-Ulam chains

    Science.gov (United States)

    Dauxois, Thierry; Ruffo, Stefano; Torcini, Alessandro

    1997-12-01

    In the framework of the Fermi-Pasta-Ulam (FPU) model, we show a simple method to give an accurate analytical estimation of the maximal Lyapunov exponent at high energy density. The method is based on the computation of the mean value of the modulational instability growth rates associated to unstable modes. Moreover, we show that the strong stochasticity threshold found in the β-FPU system is closely related to a transition in tangent space, the Lyapunov eigenvector being more localized in space at high energy.

  13. Adaptive Neural-Sliding Mode Control of Active Suspension System for Camera Stabilization

    Directory of Open Access Journals (Sweden)

    Feng Zhao

    2015-01-01

    Full Text Available The camera always suffers from image instability on the moving vehicle due to the unintentional vibrations caused by road roughness. This paper presents a novel adaptive neural network based on sliding mode control strategy to stabilize the image captured area of the camera. The purpose is to suppress vertical displacement of sprung mass with the application of active suspension system. Since the active suspension system has nonlinear and time varying characteristics, adaptive neural network (ANN is proposed to make the controller robustness against systematic uncertainties, which release the model-based requirement of the sliding model control, and the weighting matrix is adjusted online according to Lyapunov function. The control system consists of two loops. The outer loop is a position controller designed with sliding mode strategy, while the PID controller in the inner loop is to track the desired force. The closed loop stability and asymptotic convergence performance can be guaranteed on the basis of the Lyapunov stability theory. Finally, the simulation results show that the employed controller effectively suppresses the vibration of the camera and enhances the stabilization of the entire camera, where different excitations are considered to validate the system performance.

  14. Lyapunov exponent for aging process in induction motor

    Science.gov (United States)

    Bayram, Duygu; Ünnü, Sezen Yıdırım; Şeker, Serhat

    2012-09-01

    Nonlinear systems like electrical circuits and systems, mechanics, optics and even incidents in nature may pass through various bifurcations and steady states like equilibrium point, periodic, quasi-periodic, chaotic states. Although chaotic phenomena are widely observed in physical systems, it can not be predicted because of the nature of the system. On the other hand, it is known that, chaos is strictly dependent on initial conditions of the system [1-3]. There are several methods in order to define the chaos. Phase portraits, Poincaré maps, Lyapunov Exponents are the most common techniques. Lyapunov Exponents are the theoretical indicator of the chaos, named after the Russian mathematician Aleksandr Lyapunov (1857-1918). Lyapunov Exponents stand for the average exponential divergence or convergence of nearby system states, meaning estimating the quantitive measure of the chaotic attractor. Negative numbers of the exponents stand for a stable system whereas zero stands for quasi-periodic systems. On the other hand, at least if one of the exponents is positive, this situation is an indicator of the chaos. For estimating the exponents, the system should be modeled by differential equation but even in that case mathematical calculation of Lyapunov Exponents are not very practical and evaluation of these values requires a long signal duration [4-7]. For experimental data sets, it is not always possible to acquire the differential equations. There are several different methods in literature for determining the Lyapunov Exponents of the system [4, 5]. Induction motors are the most important tools for many industrial processes because they are cheap, robust, efficient and reliable. In order to have healthy processes in industrial applications, the conditions of the machines should be monitored and the different working conditions should be addressed correctly. To the best of our knowledge, researches related to Lyapunov exponents and electrical motors are mostly

  15. Lyapunov spectra of density fluctuations in TBR-1

    International Nuclear Information System (INIS)

    Oiwa, N.N.; Fidler-Ferrara, N.

    1993-01-01

    The results for the Lyapunov exponents associated with density fluctuations measured by Langmuir probes placed in the scrape-off layer of the Tokamak TBR-1 are reported. By a judicious use of the Sano-Sawada and Eckmann-Ruelle algorithms conclusive values for the positive Lyapunov exponents for most of the analysed signals are used showing evidences of chaotic behavior. (author)

  16. Word embeddings and recurrent neural networks based on Long-Short Term Memory nodes in supervised biomedical word sense disambiguation.

    Science.gov (United States)

    Jimeno Yepes, Antonio

    2017-09-01

    Word sense disambiguation helps identifying the proper sense of ambiguous words in text. With large terminologies such as the UMLS Metathesaurus ambiguities appear and highly effective disambiguation methods are required. Supervised learning algorithm methods are used as one of the approaches to perform disambiguation. Features extracted from the context of an ambiguous word are used to identify the proper sense of such a word. The type of features have an impact on machine learning methods, thus affect disambiguation performance. In this work, we have evaluated several types of features derived from the context of the ambiguous word and we have explored as well more global features derived from MEDLINE using word embeddings. Results show that word embeddings improve the performance of more traditional features and allow as well using recurrent neural network classifiers based on Long-Short Term Memory (LSTM) nodes. The combination of unigrams and word embeddings with an SVM sets a new state of the art performance with a macro accuracy of 95.97 in the MSH WSD data set. Copyright © 2017 Elsevier Inc. All rights reserved.

  17. A Recurrent Neural Network Approach to Rear Vehicle Detection Which Considered State Dependency

    Directory of Open Access Journals (Sweden)

    Kayichirou Inagaki

    2003-08-01

    Full Text Available Experimental vision-based detection often fails in cases when the acquired image quality is reduced by changing optical environments. In addition, the shape of vehicles in images that are taken from vision sensors change due to approaches by vehicle. Vehicle detection methods are required to perform successfully under these conditions. However, the conventional methods do not consider especially in rapidly varying by brightness conditions. We suggest a new detection method that compensates for those conditions in monocular vision-based vehicle detection. The suggested method employs a Recurrent Neural Network (RNN, which has been applied for spatiotemporal processing. The RNN is able to respond to consecutive scenes involving the target vehicle and can track the movements of the target by the effect of the past network states. The suggested method has a particularly beneficial effect in environments with sudden, extreme variations such as bright sunlight and shield. Finally, we demonstrate effectiveness by state-dependent of the RNN-based method by comparing its detection results with those of a Multi Layered Perceptron (MLP.

  18. Spatial Clockwork Recurrent Neural Network for Muscle Perimysium Segmentation.

    Science.gov (United States)

    Xie, Yuanpu; Zhang, Zizhao; Sapkota, Manish; Yang, Lin

    2016-10-01

    Accurate segmentation of perimysium plays an important role in early diagnosis of many muscle diseases because many diseases contain different perimysium inflammation. However, it remains as a challenging task due to the complex appearance of the perymisum morphology and its ambiguity to the background area. The muscle perimysium also exhibits strong structure spanned in the entire tissue, which makes it difficult for current local patch-based methods to capture this long-range context information. In this paper, we propose a novel spatial clockwork recurrent neural network (spatial CW-RNN) to address those issues. Specifically, we split the entire image into a set of non-overlapping image patches, and the semantic dependencies among them are modeled by the proposed spatial CW-RNN. Our method directly takes the 2D structure of the image into consideration and is capable of encoding the context information of the entire image into the local representation of each patch. Meanwhile, we leverage on the structured regression to assign one prediction mask rather than a single class label to each local patch, which enables both efficient training and testing. We extensively test our method for perimysium segmentation using digitized muscle microscopy images. Experimental results demonstrate the superiority of the novel spatial CW-RNN over other existing state of the arts.

  19. Global Exponential Stability of Delayed Cohen-Grossberg BAM Neural Networks with Impulses on Time Scales

    Directory of Open Access Journals (Sweden)

    Yongkun Li

    2009-01-01

    Full Text Available Based on the theory of calculus on time scales, the homeomorphism theory, Lyapunov functional method, and some analysis techniques, sufficient conditions are obtained for the existence, uniqueness, and global exponential stability of the equilibrium point of Cohen-Grossberg bidirectional associative memory (BAM neural networks with distributed delays and impulses on time scales. This is the first time applying the time-scale calculus theory to unify the discrete-time and continuous-time Cohen-Grossberg BAM neural network with impulses under the same framework.

  20. Hyperbolicity and integral expression of the Lyapunov exponents for linear cocycles

    Science.gov (United States)

    Dai, Xiongping

    Consider in this paper a linear skew-product system (θ,Θ) :T×W×R→W×R; (t,w,x)↦(tw,Θ(t,w)ṡx) where T=R or Z, and θ :(t,w)↦tw is a topological dynamical system on a compact metrizable space W, and where Θ(t,w)∈GL(n,R) satisfies the cocycle condition based on θ and is continuously differentiable in t if T=R. We show that 'semi λ-exponential dichotomy' of (θ,Θ) implies ' λ-exponential dichotomy.' Precisely, if Θ has no Lyapunov exponent λ and is almost uniformly λ-contracting along the λ-stable direction E(w;λ) and if dimE(w;λ) is constant a.e., then Θ is almost λ-exponentially dichotomous. To prove this, we first use Liao's spectrum theorem, which gives integral expression of the Lyapunov exponents, and then use the semi-uniform ergodic theorem by Sturman and Stark, which allows one to derive uniform estimates from nonuniform ones. As a consequence, we obtain the open-and-dense hyperbolicity of eventual GL(2,R)-cocycles based on a uniquely ergodic endomorphism, and of GL(2,R)-cocycles based on a uniquely ergodic equi-continuous endomorphism, respectively. On the other hand, in the sense of C-topology we obtain the density of SL(2,R)-cocycles having positive Lyapunov exponent based on a minimal subshift satisfying the Boshernitzan condition.

  1. Identification of a Typical CSTR Using Optimal Focused Time Lagged Recurrent Neural Network Model with Gamma Memory Filter

    OpenAIRE

    Naikwad, S. N.; Dudul, S. V.

    2009-01-01

    A focused time lagged recurrent neural network (FTLR NN) with gamma memory filter is designed to learn the subtle complex dynamics of a typical CSTR process. Continuous stirred tank reactor exhibits complex nonlinear operations where reaction is exothermic. It is noticed from literature review that process control of CSTR using neuro-fuzzy systems was attempted by many, but optimal neural network model for identification of CSTR process is not yet available. As CSTR process includes tempora...

  2. Lyapunov spectra and conjugate-pairing rule for confined atomic fluids

    DEFF Research Database (Denmark)

    Bernadi, Stefano; Todd, B.D.; Hansen, Jesper Schmidt

    2010-01-01

    In this work we present nonequilibrium molecular dynamics simulation results for the Lyapunov spectra of atomic fluids confined in narrow channels of the order of a few atomic diameters. We show the effect that realistic walls have on the Lyapunov spectra. All the degrees of freedom of the confin...... evolved Lyapunov vectors projected into a reduced dimensional phase space. We finally observe that the phase-space compression due to the thermostat remains confined into the wall region and does not significantly affect the purely Newtonian fluid region....

  3. A Three-Threshold Learning Rule Approaches the Maximal Capacity of Recurrent Neural Networks.

    Directory of Open Access Journals (Sweden)

    Alireza Alemi

    2015-08-01

    Full Text Available Understanding the theoretical foundations of how memories are encoded and retrieved in neural populations is a central challenge in neuroscience. A popular theoretical scenario for modeling memory function is the attractor neural network scenario, whose prototype is the Hopfield model. The model simplicity and the locality of the synaptic update rules come at the cost of a poor storage capacity, compared with the capacity achieved with perceptron learning algorithms. Here, by transforming the perceptron learning rule, we present an online learning rule for a recurrent neural network that achieves near-maximal storage capacity without an explicit supervisory error signal, relying only upon locally accessible information. The fully-connected network consists of excitatory binary neurons with plastic recurrent connections and non-plastic inhibitory feedback stabilizing the network dynamics; the memory patterns to be memorized are presented online as strong afferent currents, producing a bimodal distribution for the neuron synaptic inputs. Synapses corresponding to active inputs are modified as a function of the value of the local fields with respect to three thresholds. Above the highest threshold, and below the lowest threshold, no plasticity occurs. In between these two thresholds, potentiation/depression occurs when the local field is above/below an intermediate threshold. We simulated and analyzed a network of binary neurons implementing this rule and measured its storage capacity for different sizes of the basins of attraction. The storage capacity obtained through numerical simulations is shown to be close to the value predicted by analytical calculations. We also measured the dependence of capacity on the strength of external inputs. Finally, we quantified the statistics of the resulting synaptic connectivity matrix, and found that both the fraction of zero weight synapses and the degree of symmetry of the weight matrix increase with the

  4. Stability Analysis of Neural Networks-Based System Identification

    Directory of Open Access Journals (Sweden)

    Talel Korkobi

    2008-01-01

    Full Text Available This paper treats some problems related to nonlinear systems identification. A stability analysis neural network model for identifying nonlinear dynamic systems is presented. A constrained adaptive stable backpropagation updating law is presented and used in the proposed identification approach. The proposed backpropagation training algorithm is modified to obtain an adaptive learning rate guarantying convergence stability. The proposed learning rule is the backpropagation algorithm under the condition that the learning rate belongs to a specified range defining the stability domain. Satisfying such condition, unstable phenomena during the learning process are avoided. A Lyapunov analysis leads to the computation of the expression of a convenient adaptive learning rate verifying the convergence stability criteria. Finally, the elaborated training algorithm is applied in several simulations. The results confirm the effectiveness of the CSBP algorithm.

  5. Recurrent-neural-network-based Boolean factor analysis and its application to word clustering.

    Science.gov (United States)

    Frolov, Alexander A; Husek, Dusan; Polyakov, Pavel Yu

    2009-07-01

    The objective of this paper is to introduce a neural-network-based algorithm for word clustering as an extension of the neural-network-based Boolean factor analysis algorithm (Frolov , 2007). It is shown that this extended algorithm supports even the more complex model of signals that are supposed to be related to textual documents. It is hypothesized that every topic in textual data is characterized by a set of words which coherently appear in documents dedicated to a given topic. The appearance of each word in a document is coded by the activity of a particular neuron. In accordance with the Hebbian learning rule implemented in the network, sets of coherently appearing words (treated as factors) create tightly connected groups of neurons, hence, revealing them as attractors of the network dynamics. The found factors are eliminated from the network memory by the Hebbian unlearning rule facilitating the search of other factors. Topics related to the found sets of words can be identified based on the words' semantics. To make the method complete, a special technique based on a Bayesian procedure has been developed for the following purposes: first, to provide a complete description of factors in terms of component probability, and second, to enhance the accuracy of classification of signals to determine whether it contains the factor. Since it is assumed that every word may possibly contribute to several topics, the proposed method might be related to the method of fuzzy clustering. In this paper, we show that the results of Boolean factor analysis and fuzzy clustering are not contradictory, but complementary. To demonstrate the capabilities of this attempt, the method is applied to two types of textual data on neural networks in two different languages. The obtained topics and corresponding words are at a good level of agreement despite the fact that identical topics in Russian and English conferences contain different sets of keywords.

  6. Global robust asymptotical stability of multi-delayed interval neural networks: an LMI approach

    International Nuclear Information System (INIS)

    Li Chuandong; Liao Xiaofeng; Zhang Rong

    2004-01-01

    Based on the Lyapunov-Krasovskii stability theory for functional differential equations and the linear matrix inequality (LMI) technique, some delay-dependent criteria for interval neural networks (IDNN) with multiple time-varying delays are derived to guarantee global robust asymptotic stability. The main results are generalizations of some recent results reported in the literature. Numerical example is also given to show the effectiveness of our results

  7. Global exponential synchronization of inertial memristive neural networks with time-varying delay via nonlinear controller.

    Science.gov (United States)

    Gong, Shuqing; Yang, Shaofu; Guo, Zhenyuan; Huang, Tingwen

    2018-06-01

    The paper is concerned with the synchronization problem of inertial memristive neural networks with time-varying delay. First, by choosing a proper variable substitution, inertial memristive neural networks described by second-order differential equations can be transformed into first-order differential equations. Then, a novel controller with a linear diffusive term and discontinuous sign term is designed. By using the controller, the sufficient conditions for assuring the global exponential synchronization of the derive and response neural networks are derived based on Lyapunov stability theory and some inequality techniques. Finally, several numerical simulations are provided to substantiate the effectiveness of the theoretical results. Copyright © 2018 Elsevier Ltd. All rights reserved.

  8. OBSERVING LYAPUNOV EXPONENTS OF INFINITE-DIMENSIONAL DYNAMICAL SYSTEMS.

    Science.gov (United States)

    Ott, William; Rivas, Mauricio A; West, James

    2015-12-01

    Can Lyapunov exponents of infinite-dimensional dynamical systems be observed by projecting the dynamics into ℝ N using a 'typical' nonlinear projection map? We answer this question affirmatively by developing embedding theorems for compact invariant sets associated with C 1 maps on Hilbert spaces. Examples of such discrete-time dynamical systems include time- T maps and Poincaré return maps generated by the solution semigroups of evolution partial differential equations. We make every effort to place hypotheses on the projected dynamics rather than on the underlying infinite-dimensional dynamical system. In so doing, we adopt an empirical approach and formulate checkable conditions under which a Lyapunov exponent computed from experimental data will be a Lyapunov exponent of the infinite-dimensional dynamical system under study (provided the nonlinear projection map producing the data is typical in the sense of prevalence).

  9. On global stability criterion for neural networks with discrete and distributed delays

    International Nuclear Information System (INIS)

    Park, Ju H.

    2006-01-01

    Based on the Lyapunov functional stability analysis for differential equations and the linear matrix inequality (LMI) optimization approach, a new delay-dependent criterion for neural networks with discrete and distributed delays is derived to guarantee global asymptotic stability. The criterion is expressed in terms of LMIs, which can be solved easily by various convex optimization algorithms. Some numerical examples are given to show the effectiveness of proposed method

  10. A modular architecture for transparent computation in recurrent neural networks.

    Science.gov (United States)

    Carmantini, Giovanni S; Beim Graben, Peter; Desroches, Mathieu; Rodrigues, Serafim

    2017-01-01

    Computation is classically studied in terms of automata, formal languages and algorithms; yet, the relation between neural dynamics and symbolic representations and operations is still unclear in traditional eliminative connectionism. Therefore, we suggest a unique perspective on this central issue, to which we would like to refer as transparent connectionism, by proposing accounts of how symbolic computation can be implemented in neural substrates. In this study we first introduce a new model of dynamics on a symbolic space, the versatile shift, showing that it supports the real-time simulation of a range of automata. We then show that the Gödelization of versatile shifts defines nonlinear dynamical automata, dynamical systems evolving on a vectorial space. Finally, we present a mapping between nonlinear dynamical automata and recurrent artificial neural networks. The mapping defines an architecture characterized by its granular modularity, where data, symbolic operations and their control are not only distinguishable in activation space, but also spatially localizable in the network itself, while maintaining a distributed encoding of symbolic representations. The resulting networks simulate automata in real-time and are programmed directly, in the absence of network training. To discuss the unique characteristics of the architecture and their consequences, we present two examples: (i) the design of a Central Pattern Generator from a finite-state locomotive controller, and (ii) the creation of a network simulating a system of interactive automata that supports the parsing of garden-path sentences as investigated in psycholinguistics experiments. Copyright © 2016 Elsevier Ltd. All rights reserved.

  11. Discrete-time BAM neural networks with variable delays

    Science.gov (United States)

    Liu, Xin-Ge; Tang, Mei-Lan; Martin, Ralph; Liu, Xin-Bi

    2007-07-01

    This Letter deals with the global exponential stability of discrete-time bidirectional associative memory (BAM) neural networks with variable delays. Using a Lyapunov functional, and linear matrix inequality techniques (LMI), we derive a new delay-dependent exponential stability criterion for BAM neural networks with variable delays. As this criterion has no extra constraints on the variable delay functions, it can be applied to quite general BAM neural networks with a broad range of time delay functions. It is also easy to use in practice. An example is provided to illustrate the theoretical development.

  12. Discrete-time BAM neural networks with variable delays

    International Nuclear Information System (INIS)

    Liu Xinge; Tang Meilan; Martin, Ralph; Liu Xinbi

    2007-01-01

    This Letter deals with the global exponential stability of discrete-time bidirectional associative memory (BAM) neural networks with variable delays. Using a Lyapunov functional, and linear matrix inequality techniques (LMI), we derive a new delay-dependent exponential stability criterion for BAM neural networks with variable delays. As this criterion has no extra constraints on the variable delay functions, it can be applied to quite general BAM neural networks with a broad range of time delay functions. It is also easy to use in practice. An example is provided to illustrate the theoretical development

  13. Using recurrent neural network models for early detection of heart failure onset.

    Science.gov (United States)

    Choi, Edward; Schuetz, Andy; Stewart, Walter F; Sun, Jimeng

    2017-03-01

    We explored whether use of deep learning to model temporal relations among events in electronic health records (EHRs) would improve model performance in predicting initial diagnosis of heart failure (HF) compared to conventional methods that ignore temporality. Data were from a health system's EHR on 3884 incident HF cases and 28 903 controls, identified as primary care patients, between May 16, 2000, and May 23, 2013. Recurrent neural network (RNN) models using gated recurrent units (GRUs) were adapted to detect relations among time-stamped events (eg, disease diagnosis, medication orders, procedure orders, etc.) with a 12- to 18-month observation window of cases and controls. Model performance metrics were compared to regularized logistic regression, neural network, support vector machine, and K-nearest neighbor classifier approaches. Using a 12-month observation window, the area under the curve (AUC) for the RNN model was 0.777, compared to AUCs for logistic regression (0.747), multilayer perceptron (MLP) with 1 hidden layer (0.765), support vector machine (SVM) (0.743), and K-nearest neighbor (KNN) (0.730). When using an 18-month observation window, the AUC for the RNN model increased to 0.883 and was significantly higher than the 0.834 AUC for the best of the baseline methods (MLP). Deep learning models adapted to leverage temporal relations appear to improve performance of models for detection of incident heart failure with a short observation window of 12-18 months. © The Author 2016. Published by Oxford University Press on behalf of the American Medical Informatics Association.

  14. On the synchronization of neural networks containing time-varying delays and sector nonlinearity

    International Nuclear Information System (INIS)

    Yan, J.-J.; Lin, J.-S.; Hung, M.-L.; Liao, T.-L.

    2007-01-01

    We present a systematic design procedure for synchronization of neural networks subject to time-varying delays and sector nonlinearity in the control input. Based on the drive-response concept and the Lyapunov stability theorem, a memoryless decentralized control law is proposed which guarantees exponential synchronization even when input nonlinearity is present. The supplementary requirement that the time-derivative of time-varying delays must be smaller than one is released for the proposed control scheme. A four-dimensional Hopfield neural network with time-varying delays is presented as the illustrative example to demonstrate the effectiveness of the proposed synchronization scheme

  15. Using Covariant Lyapunov Vectors to Understand Spatiotemporal Chaos in Fluids

    Science.gov (United States)

    Paul, Mark; Xu, Mu; Barbish, Johnathon; Mukherjee, Saikat

    2017-11-01

    The spatiotemporal chaos of fluids present many difficult and fascinating challenges. Recent progress in computing covariant Lyapunov vectors for a variety of model systems has made it possible to probe fundamental ideas from dynamical systems theory including the degree of hyperbolicity, the fractal dimension, the dimension of the inertial manifold, and the decomposition of the dynamics into a finite number of physical modes and spurious modes. We are interested in building upon insights such as these for fluid systems. We first demonstrate the power of covariant Lyapunov vectors using a system of maps on a lattice with a nonlinear coupling. We then compute the covariant Lyapunov vectors for chaotic Rayleigh-Bénard convection for experimentally accessible conditions. We show that chaotic convection is non-hyperbolic and we quantify the spatiotemporal features of the spectrum of covariant Lyapunov vectors. NSF DMS-1622299 and DARPA/DSO Models, Dynamics, and Learning (MoDyL).

  16. Engine cylinder pressure reconstruction using crank kinematics and recurrently-trained neural networks

    Science.gov (United States)

    Bennett, C.; Dunne, J. F.; Trimby, S.; Richardson, D.

    2017-02-01

    A recurrent non-linear autoregressive with exogenous input (NARX) neural network is proposed, and a suitable fully-recurrent training methodology is adapted and tuned, for reconstructing cylinder pressure in multi-cylinder IC engines using measured crank kinematics. This type of indirect sensing is important for cost effective closed-loop combustion control and for On-Board Diagnostics. The challenge addressed is to accurately predict cylinder pressure traces within the cycle under generalisation conditions: i.e. using data not previously seen by the network during training. This involves direct construction and calibration of a suitable inverse crank dynamic model, which owing to singular behaviour at top-dead-centre (TDC), has proved difficult via physical model construction, calibration, and inversion. The NARX architecture is specialised and adapted to cylinder pressure reconstruction, using a fully-recurrent training methodology which is needed because the alternatives are too slow and unreliable for practical network training on production engines. The fully-recurrent Robust Adaptive Gradient Descent (RAGD) algorithm, is tuned initially using synthesised crank kinematics, and then tested on real engine data to assess the reconstruction capability. Real data is obtained from a 1.125 l, 3-cylinder, in-line, direct injection spark ignition (DISI) engine involving synchronised measurements of crank kinematics and cylinder pressure across a range of steady-state speed and load conditions. The paper shows that a RAGD-trained NARX network using both crank velocity and crank acceleration as input information, provides fast and robust training. By using the optimum epoch identified during RAGD training, acceptably accurate cylinder pressures, and especially accurate location-of-peak-pressure, can be reconstructed robustly under generalisation conditions, making it the most practical NARX configuration and recurrent training methodology for use on production engines.

  17. Lyapunov-based distributed control of the safety-factor profile in a tokamak plasma

    International Nuclear Information System (INIS)

    Bribiesca Argomedo, Federico; Witrant, Emmanuel; Prieur, Christophe; Brémond, Sylvain; Nouailletas, Rémy; Artaud, Jean-François

    2013-01-01

    A real-time model-based controller is developed for the tracking of the distributed safety-factor profile in a tokamak plasma. Using relevant physical models and simplifying assumptions, theoretical stability and robustness guarantees were obtained using a Lyapunov function. This approach considers the couplings between the poloidal flux diffusion equation, the time-varying temperature profiles and an independent total plasma current control. The actuator chosen for the safety-factor profile tracking is the lower hybrid current drive, although the results presented can be easily extended to any non-inductive current source. The performance and robustness of the proposed control law is evaluated with a physics-oriented simulation code on Tore Supra experimental test cases. (paper)

  18. Predicting Traffic Flow in Local Area Networks by the Largest Lyapunov Exponent

    Directory of Open Access Journals (Sweden)

    Yan Liu

    2016-01-01

    Full Text Available The dynamics of network traffic are complex and nonlinear, and chaotic behaviors and their prediction, which play an important role in local area networks (LANs, are studied in detail, using the largest Lyapunov exponent. With the introduction of phase space reconstruction based on the time sequence, the high-dimensional traffic is projected onto the low dimension reconstructed phase space, and a reduced dynamic system is obtained from the dynamic system viewpoint. Then, a numerical method for computing the largest Lyapunov exponent of the low-dimensional dynamic system is presented. Further, the longest predictable time, which is related to chaotic behaviors in the system, is studied using the largest Lyapunov exponent, and the Wolf method is used to predict the evolution of the traffic in a local area network by both Dot and Interval predictions, and a reliable result is obtained by the presented method. As the conclusion, the results show that the largest Lyapunov exponent can be used to describe the sensitivity of the trajectory in the reconstructed phase space to the initial values. Moreover, Dot Prediction can effectively predict the flow burst. The numerical simulation also shows that the presented method is feasible and efficient for predicting the complex dynamic behaviors in LAN traffic, especially for congestion and attack in networks, which are the main two complex phenomena behaving as chaos in networks.

  19. Artificial neural network and falls in community-dwellers: a new approach to identify the risk of recurrent falling?

    Science.gov (United States)

    Kabeshova, Anastasiia; Launay, Cyrille P; Gromov, Vasilii A; Annweiler, Cédric; Fantino, Bruno; Beauchet, Olivier

    2015-04-01

    Identification of the risk of recurrent falls is complex in older adults. The aim of this study was to examine the efficiency of 3 artificial neural networks (ANNs: multilayer perceptron [MLP], modified MLP, and neuroevolution of augmenting topologies [NEAT]) for the classification of recurrent fallers and nonrecurrent fallers using a set of clinical characteristics corresponding to risk factors of falls measured among community-dwelling older adults. Based on a cross-sectional design, 3289 community-dwelling volunteers aged 65 and older were recruited. Age, gender, body mass index (BMI), number of drugs daily taken, use of psychoactive drugs, diphosphonate, calcium, vitamin D supplements and walking aid, fear of falling, distance vision score, Timed Up and Go (TUG) score, lower-limb proprioception, handgrip strength, depressive symptoms, cognitive disorders, and history of falls were recorded. Participants were separated into 2 groups based on the number of falls that occurred over the past year: 0 or 1 fall and 2 or more falls. In addition, total population was separated into training and testing subgroups for ANN analysis. Among 3289 participants, 18.9% (n = 622) were recurrent fallers. NEAT, using 15 clinical characteristics (ie, use of walking aid, fear of falling, use of calcium, depression, use of vitamin D supplements, female, cognitive disorders, BMI 4, vision score 9 seconds, handgrip strength score ≤29 (N), and age ≥75 years), showed the best efficiency for identification of recurrent fallers, sensitivity (80.42%), specificity (92.54%), positive predictive value (84.38), negative predictive value (90.34), accuracy (88.39), and Cohen κ (0.74), compared with MLP and modified MLP. NEAT, using a set of 15 clinical characteristics, was an efficient ANN for the identification of recurrent fallers in older community-dwellers. Copyright © 2015 AMDA – The Society for Post-Acute and Long-Term Care Medicine. Published by Elsevier Inc. All rights reserved.

  20. Synchronization of chaotic systems and identification of nonlinear systems by using recurrent hierarchical type-2 fuzzy neural networks.

    Science.gov (United States)

    Mohammadzadeh, Ardashir; Ghaemi, Sehraneh

    2015-09-01

    This paper proposes a novel approach for training of proposed recurrent hierarchical interval type-2 fuzzy neural networks (RHT2FNN) based on the square-root cubature Kalman filters (SCKF). The SCKF algorithm is used to adjust the premise part of the type-2 FNN and the weights of defuzzification and the feedback weights. The recurrence property in the proposed network is the output feeding of each membership function to itself. The proposed RHT2FNN is employed in the sliding mode control scheme for the synchronization of chaotic systems. Unknown functions in the sliding mode control approach are estimated by RHT2FNN. Another application of the proposed RHT2FNN is the identification of dynamic nonlinear systems. The effectiveness of the proposed network and its learning algorithm is verified by several simulation examples. Furthermore, the universal approximation of RHT2FNNs is also shown. Copyright © 2015 ISA. Published by Elsevier Ltd. All rights reserved.

  1. Novel stability criteria for fuzzy Hopfield neural networks based on an improved homogeneous matrix polynomials technique

    International Nuclear Information System (INIS)

    Feng Yi-Fu; Zhang Qing-Ling; Feng De-Zhi

    2012-01-01

    The global stability problem of Takagi—Sugeno (T—S) fuzzy Hopfield neural networks (FHNNs) with time delays is investigated. Novel LMI-based stability criteria are obtained by using Lyapunov functional theory to guarantee the asymptotic stability of the FHNNs with less conservatism. Firstly, using both Finsler's lemma and an improved homogeneous matrix polynomial technique, and applying an affine parameter-dependent Lyapunov—Krasovskii functional, we obtain the convergent LMI-based stability criteria. Algebraic properties of the fuzzy membership functions in the unit simplex are considered in the process of stability analysis via the homogeneous matrix polynomials technique. Secondly, to further reduce the conservatism, a new right-hand-side slack variables introducing technique is also proposed in terms of LMIs, which is suitable to the homogeneous matrix polynomials setting. Finally, two illustrative examples are given to show the efficiency of the proposed approaches

  2. Heat conduction in one-dimensional chains and nonequilibrium Lyapunov spectrum

    International Nuclear Information System (INIS)

    Posch, H.A.; Hoover, W.G.

    1998-01-01

    We define and study the heat conductivity κ and the Lyapunov spectrum for a modified 'ding-a-ling' chain undergoing steady heat flow. Free and bound particles alternate along a chain. In the present work, we use a linear gravitational potential to bind all the even-numbered particles to their lattice sites. The chain is bounded by two stochastic heat reservoirs, one hot and one cold. The Fourier conductivity of the chain decreases smoothly to a finite large-system limit. Special treatment of satellite collisions with the stochastic boundaries is required to obtain Lyapunov spectra. The summed spectra are negative, and correspond to a relatively small contraction in phase space, with the formation of a multifractal strange attractor. The largest of the Lyapunov exponents for the ding-a-ling chain appears to converge to a limiting value with increasing chain length, so that the large-system Lyapunov spectrum has a finite limit. copyright 1998 The American Physical Society

  3. Distributed representations of action sequences in anterior cingulate cortex: A recurrent neural network approach.

    Science.gov (United States)

    Shahnazian, Danesh; Holroyd, Clay B

    2018-02-01

    Anterior cingulate cortex (ACC) has been the subject of intense debate over the past 2 decades, but its specific computational function remains controversial. Here we present a simple computational model of ACC that incorporates distributed representations across a network of interconnected processing units. Based on the proposal that ACC is concerned with the execution of extended, goal-directed action sequences, we trained a recurrent neural network to predict each successive step of several sequences associated with multiple tasks. In keeping with neurophysiological observations from nonhuman animals, the network yields distributed patterns of activity across ACC neurons that track the progression of each sequence, and in keeping with human neuroimaging data, the network produces discrepancy signals when any step of the sequence deviates from the predicted step. These simulations illustrate a novel approach for investigating ACC function.

  4. Passivity of memristive BAM neural networks with leakage and additive time-varying delays

    Science.gov (United States)

    Wang, Weiping; Wang, Meiqi; Luo, Xiong; Li, Lixiang; Zhao, Wenbing; Liu, Linlin; Ping, Yuan

    2018-02-01

    This paper investigates the passivity of memristive bidirectional associate memory neural networks (MBAMNNs) with leakage and additive time-varying delays. Based on some useful inequalities and appropriate Lyapunov-Krasovskii functionals (LKFs), several delay-dependent conditions for passivity performance are obtained in linear matrix inequalities (LMIs). Moreover, the leakage delays as well as additive delays are considered separately. Finally, numerical simulations are provided to demonstrate the feasibility of the theoretical results.

  5. Analysis of Human Standing Balance by Largest Lyapunov Exponent

    Directory of Open Access Journals (Sweden)

    Kun Liu

    2015-01-01

    Full Text Available The purpose of this research is to analyse the relationship between nonlinear dynamic character and individuals’ standing balance by the largest Lyapunov exponent, which is regarded as a metric for assessing standing balance. According to previous study, the largest Lyapunov exponent from centre of pressure time series could not well quantify the human balance ability. In this research, two improvements were made. Firstly, an external stimulus was applied to feet in the form of continuous horizontal sinusoidal motion by a moving platform. Secondly, a multiaccelerometer subsystem was adopted. Twenty healthy volunteers participated in this experiment. A new metric, coordinated largest Lyapunov exponent was proposed, which reflected the relationship of body segments by integrating multidimensional largest Lyapunov exponent values. By using this metric in actual standing performance under sinusoidal stimulus, an obvious relationship between the new metric and the actual balance ability was found in the majority of the subjects. These results show that the sinusoidal stimulus can make human balance characteristics more obvious, which is beneficial to assess balance, and balance is determined by the ability of coordinating all body segments.

  6. Recurrent-Neural-Network-Based Multivariable Adaptive Control for a Class of Nonlinear Dynamic Systems With Time-Varying Delay.

    Science.gov (United States)

    Hwang, Chih-Lyang; Jan, Chau

    2016-02-01

    At the beginning, an approximate nonlinear autoregressive moving average (NARMA) model is employed to represent a class of multivariable nonlinear dynamic systems with time-varying delay. It is known that the disadvantages of robust control for the NARMA model are as follows: 1) suitable control parameters for larger time delay are more sensitive to achieving desirable performance; 2) it only deals with bounded uncertainty; and 3) the nominal NARMA model must be learned in advance. Due to the dynamic feature of the NARMA model, a recurrent neural network (RNN) is online applied to learn it. However, the system performance becomes deteriorated due to the poor learning of the larger variation of system vector functions. In this situation, a simple network is employed to compensate the upper bound of the residue caused by the linear parameterization of the approximation error of RNN. An e -modification learning law with a projection for weight matrix is applied to guarantee its boundedness without persistent excitation. Under suitable conditions, the semiglobally ultimately bounded tracking with the boundedness of estimated weight matrix is obtained by the proposed RNN-based multivariable adaptive control. Finally, simulations are presented to verify the effectiveness and robustness of the proposed control.

  7. Recurrent Neural Network Model for Constructive Peptide Design.

    Science.gov (United States)

    Müller, Alex T; Hiss, Jan A; Schneider, Gisbert

    2018-02-26

    We present a generative long short-term memory (LSTM) recurrent neural network (RNN) for combinatorial de novo peptide design. RNN models capture patterns in sequential data and generate new data instances from the learned context. Amino acid sequences represent a suitable input for these machine-learning models. Generative models trained on peptide sequences could therefore facilitate the design of bespoke peptide libraries. We trained RNNs with LSTM units on pattern recognition of helical antimicrobial peptides and used the resulting model for de novo sequence generation. Of these sequences, 82% were predicted to be active antimicrobial peptides compared to 65% of randomly sampled sequences with the same amino acid distribution as the training set. The generated sequences also lie closer to the training data than manually designed amphipathic helices. The results of this study showcase the ability of LSTM RNNs to construct new amino acid sequences within the applicability domain of the model and motivate their prospective application to peptide and protein design without the need for the exhaustive enumeration of sequence libraries.

  8. Robust stability analysis of switched Hopfield neural networks with time-varying delay under uncertainty

    International Nuclear Information System (INIS)

    Huang He; Qu Yuzhong; Li Hanxiong

    2005-01-01

    With the development of intelligent control, switched systems have been widely studied. Here we try to introduce some ideas of the switched systems into the field of neural networks. In this Letter, a class of switched Hopfield neural networks with time-varying delay is investigated. The parametric uncertainty is considered and assumed to be norm bounded. Firstly, the mathematical model of the switched Hopfield neural networks is established in which a set of Hopfield neural networks are used as the individual subsystems and an arbitrary switching rule is assumed; Secondly, robust stability analysis for such switched Hopfield neural networks is addressed based on the Lyapunov-Krasovskii approach. Some criteria are given to guarantee the switched Hopfield neural networks to be globally exponentially stable for all admissible parametric uncertainties. These conditions are expressed in terms of some strict linear matrix inequalities (LMIs). Finally, a numerical example is provided to illustrate our results

  9. Lyapunov exponents

    CERN Document Server

    Barreira, Luís

    2017-01-01

    This book offers a self-contained introduction to the theory of Lyapunov exponents and its applications, mainly in connection with hyperbolicity, ergodic theory and multifractal analysis. It discusses the foundations and some of the main results and main techniques in the area, while also highlighting selected topics of current research interest. With the exception of a few basic results from ergodic theory and the thermodynamic formalism, all the results presented include detailed proofs. The book is intended for all researchers and graduate students specializing in dynamical systems who are looking for a comprehensive overview of the foundations of the theory and a sample of its applications.

  10. Global Robust Stability of Switched Interval Neural Networks with Discrete and Distributed Time-Varying Delays of Neural Type

    Directory of Open Access Journals (Sweden)

    Huaiqin Wu

    2012-01-01

    Full Text Available By combing the theories of the switched systems and the interval neural networks, the mathematics model of the switched interval neural networks with discrete and distributed time-varying delays of neural type is presented. A set of the interval parameter uncertainty neural networks with discrete and distributed time-varying delays of neural type are used as the individual subsystem, and an arbitrary switching rule is assumed to coordinate the switching between these networks. By applying the augmented Lyapunov-Krasovskii functional approach and linear matrix inequality (LMI techniques, a delay-dependent criterion is achieved to ensure to such switched interval neural networks to be globally asymptotically robustly stable in terms of LMIs. The unknown gain matrix is determined by solving this delay-dependent LMIs. Finally, an illustrative example is given to demonstrate the validity of the theoretical results.

  11. ? and ? nonquadratic stabilisation of discrete-time Takagi-Sugeno systems based on multi-instant fuzzy Lyapunov functions

    Science.gov (United States)

    Tognetti, Eduardo S.; Oliveira, Ricardo C. L. F.; Peres, Pedro L. D.

    2015-01-01

    The problem of state feedback control design for discrete-time Takagi-Sugeno (TS) (T-S) fuzzy systems is investigated in this paper. A Lyapunov function, which is quadratic in the state and presents a multi-polynomial dependence on the fuzzy weighting functions at the current and past instants of time, is proposed.This function contains, as particular cases, other previous Lyapunov functions already used in the literature, being able to provide less conservative conditions of control design for TS fuzzy systems. The structure of the proposed Lyapunov function also motivates the design of a new stabilising compensator for Takagi-Sugeno fuzzy systems. The main novelty of the proposed state feedback control law is that the gain is composed of matrices with multi-polynomial dependence on the fuzzy weighting functions at a set of past instants of time, including the current one. The conditions for the existence of a stabilising state feedback control law that minimises an upper bound to the ? or ? norms are given in terms of linear matrix inequalities. Numerical examples show that the approach can be less conservative and more efficient than other methods available in the literature.

  12. A new neural observer for an anaerobic bioreactor.

    Science.gov (United States)

    Belmonte-Izquierdo, R; Carlos-Hernandez, S; Sanchez, E N

    2010-02-01

    In this paper, a recurrent high order neural observer (RHONO) for anaerobic processes is proposed. The main objective is to estimate variables of methanogenesis: biomass, substrate and inorganic carbon in a completely stirred tank reactor (CSTR). The recurrent high order neural network (RHONN) structure is based on the hyperbolic tangent as activation function. The learning algorithm is based on an extended Kalman filter (EKF). The applicability of the proposed scheme is illustrated via simulation. A validation using real data from a lab scale process is included. Thus, this observer can be successfully implemented for control purposes.

  13. Identification of a Typical CSTR Using Optimal Focused Time Lagged Recurrent Neural Network Model with Gamma Memory Filter

    Directory of Open Access Journals (Sweden)

    S. N. Naikwad

    2009-01-01

    Full Text Available A focused time lagged recurrent neural network (FTLR NN with gamma memory filter is designed to learn the subtle complex dynamics of a typical CSTR process. Continuous stirred tank reactor exhibits complex nonlinear operations where reaction is exothermic. It is noticed from literature review that process control of CSTR using neuro-fuzzy systems was attempted by many, but optimal neural network model for identification of CSTR process is not yet available. As CSTR process includes temporal relationship in the input-output mappings, time lagged recurrent neural network is particularly used for identification purpose. The standard back propagation algorithm with momentum term has been proposed in this model. The various parameters like number of processing elements, number of hidden layers, training and testing percentage, learning rule and transfer function in hidden and output layer are investigated on the basis of performance measures like MSE, NMSE, and correlation coefficient on testing data set. Finally effects of different norms are tested along with variation in gamma memory filter. It is demonstrated that dynamic NN model has a remarkable system identification capability for the problems considered in this paper. Thus FTLR NN with gamma memory filter can be used to learn underlying highly nonlinear dynamics of the system, which is a major contribution of this paper.

  14. A Lyapunov Stability Theory-Based Control Strategy for Three-Level Shunt Active Power Filter

    Directory of Open Access Journals (Sweden)

    Yijia Cao

    2017-01-01

    Full Text Available The three-phase three-wire neutral-point-clamped shunt active power filter (NPC-SAPF, which most adopts classical closed-loop feedback control methods such as proportional-integral (PI, proportional-resonant (PR and repetitive control, can only output 1st–25th harmonic currents with 10–20 kHz switching frequency. The reason for this is that the controller design must make a compromise between system stability and harmonic current compensation ability under the condition of less than 20 kHz switching frequency. To broaden the bandwidth of the compensation current, a Lyapunov stability theory-based control strategy is presented in this paper for NPC-SAPF. The proposed control law is obtained by constructing the switching function on the basis of the mathematical model and the Lyapunov candidate function, which can avoid introducing closed-loop feedback control and keep the system globally asymptotically stable. By means of the proposed method, the NPC-SAPF has compensation ability for the 1st–50th harmonic currents, the total harmonic distortion (THD and each harmonic content of grid currents satisfy the requirements of IEEE Standard 519-2014. In order to verify the superiority of the proposed control strategy, stability conditions of the proposed strategy and the representative PR controllers are compared. The simulation results in MATLAB/Simulink (MathWorks, Natick, MA, USA and the experimental results obtained on a 6.6 kVA NPC-SAPF laboratory prototype validate the proposed control strategy.

  15. Impulsive Synchronization of Reaction-Diffusion Neural Networks With Mixed Delays and Its Application to Image Encryption.

    Science.gov (United States)

    Chen, Wu-Hua; Luo, Shixian; Zheng, Wei Xing

    2016-12-01

    This paper presents a new impulsive synchronization criterion of two identical reaction-diffusion neural networks with discrete and unbounded distributed delays. The new criterion is established by applying an impulse-time-dependent Lyapunov functional combined with the use of a new type of integral inequality for treating the reaction-diffusion terms. The impulse-time-dependent feature of the proposed Lyapunov functional can capture more hybrid dynamical behaviors of the impulsive reaction-diffusion neural networks than the conventional impulse-time-independent Lyapunov functions/functionals, while the new integral inequality, which is derived from Wirtinger's inequality, overcomes the conservatism introduced by the integral inequality used in the previous results. Numerical examples demonstrate the effectiveness of the proposed method. Later, the developed impulsive synchronization method is applied to build a spatiotemporal chaotic cryptosystem that can transmit an encrypted image. The experimental results verify that the proposed image-encrypting cryptosystem has the advantages of large key space and high security against some traditional attacks.

  16. Design of Connectivity Preserving Flocking Using Control Lyapunov Function

    OpenAIRE

    Erfianto, Bayu; Bambang, Riyanto T.; Hindersah, Hilwadi; Muchtadi-Alamsyah, Intan

    2016-01-01

    This paper investigates cooperative flocking control design with connectivity preserving mechanism. During flocking, interagent distance is measured to determine communication topology of the flocks. Then, cooperative flocking motion is built based on cooperative artificial potential field with connectivity preserving mechanism to achieve the common flocking objective. The flocking control input is then obtained by deriving cooperative artificial potential field using control Lyapunov functio...

  17. Distributed Adaptive Neural Control for Stochastic Nonlinear Multiagent Systems.

    Science.gov (United States)

    Wang, Fang; Chen, Bing; Lin, Chong; Li, Xuehua

    2016-11-14

    In this paper, a consensus tracking problem of nonlinear multiagent systems is investigated under a directed communication topology. All the followers are modeled by stochastic nonlinear systems in nonstrict feedback form, where nonlinearities and stochastic disturbance terms are totally unknown. Based on the structural characteristic of neural networks (in Lemma 4), a novel distributed adaptive neural control scheme is put forward. The raised control method not only effectively handles unknown nonlinearities in nonstrict feedback systems, but also copes with the interactions among agents and coupling terms. Based on the stochastic Lyapunov functional method, it is indicated that all the signals of the closed-loop system are bounded in probability and all followers' outputs are convergent to a neighborhood of the output of leader. At last, the efficiency of the control method is testified by a numerical example.

  18. Neural network-based optimal adaptive output feedback control of a helicopter UAV.

    Science.gov (United States)

    Nodland, David; Zargarzadeh, Hassan; Jagannathan, Sarangapani

    2013-07-01

    Helicopter unmanned aerial vehicles (UAVs) are widely used for both military and civilian operations. Because the helicopter UAVs are underactuated nonlinear mechanical systems, high-performance controller design for them presents a challenge. This paper introduces an optimal controller design via an output feedback for trajectory tracking of a helicopter UAV, using a neural network (NN). The output-feedback control system utilizes the backstepping methodology, employing kinematic and dynamic controllers and an NN observer. The online approximator-based dynamic controller learns the infinite-horizon Hamilton-Jacobi-Bellman equation in continuous time and calculates the corresponding optimal control input by minimizing a cost function, forward-in-time, without using the value and policy iterations. Optimal tracking is accomplished by using a single NN utilized for the cost function approximation. The overall closed-loop system stability is demonstrated using Lyapunov analysis. Finally, simulation results are provided to demonstrate the effectiveness of the proposed control design for trajectory tracking.

  19. A Heuristic Approach to Intra-Brain Communications Using Chaos in a Recurrent Neural Network Model

    Science.gov (United States)

    Soma, Ken-ichiro; Mori, Ryota; Sato, Ryuichi; Nara, Shigetoshi

    2011-09-01

    To approach functional roles of chaos in brain, a heuristic model to consider mechanisms of intra-brain communications is proposed. The key idea is to use chaos in firing pattern dynamics of a recurrent neural network consisting of birary state neurons, as propagation medium of pulse signals. Computer experiments and numerical methods are introduced to evaluate signal transport characteristics by calculating correlation functions between sending neurons and receiving neurons of pulse signals.

  20. Modeling the dynamics of the lead bismuth eutectic experimental accelerator driven system by an infinite impulse response locally recurrent neural network

    International Nuclear Information System (INIS)

    Zio, Enrico; Pedroni, Nicola; Broggi, Matteo; Golea, Lucia Roxana

    2009-01-01

    In this paper, an infinite impulse response locally recurrent neural network (IIR-LRNN) is employed for modelling the dynamics of the Lead Bismuth Eutectic eXperimental Accelerator Driven System (LBE-XADS). The network is trained by recursive back-propagation (RBP) and its ability in estimating transients is tested under various conditions. The results demonstrate the robustness of the locally recurrent scheme in the reconstruction of complex nonlinear dynamic relationships

  1. Dynamic analysis of stochastic bidirectional associative memory neural networks with delays

    International Nuclear Information System (INIS)

    Zhao Hongyong; Ding Nan

    2007-01-01

    In this paper, stochastic bidirectional associative memory neural networks model with delays is considered. By constructing Lyapunov functionals, and using stochastic analysis method and inequality technique, we give some sufficient criteria ensuring almost sure exponential stability, pth exponential stability and mean value exponential stability. The obtained criteria can be used as theoretic guidance to stabilize neural networks in practical applications when stochastic noise is taken into consideration

  2. Improved delay-dependent globally asymptotic stability of delayed uncertain recurrent neural networks with Markovian jumping parameters

    International Nuclear Information System (INIS)

    Yan, Ji; Bao-Tong, Cui

    2010-01-01

    In this paper, we have improved delay-dependent stability criteria for recurrent neural networks with a delay varying over a range and Markovian jumping parameters. The criteria improve over some previous ones in that they have fewer matrix variables yet less conservatism. In addition, a numerical example is provided to illustrate the applicability of the result using the linear matrix inequality toolbox in MATLAB. (general)

  3. Using LSTM recurrent neural networks for monitoring the LHC superconducting magnets

    Science.gov (United States)

    Wielgosz, Maciej; Skoczeń, Andrzej; Mertik, Matej

    2017-09-01

    The superconducting LHC magnets are coupled with an electronic monitoring system which records and analyzes voltage time series reflecting their performance. A currently used system is based on a range of preprogrammed triggers which launches protection procedures when a misbehavior of the magnets is detected. All the procedures used in the protection equipment were designed and implemented according to known working scenarios of the system and are updated and monitored by human operators. This paper proposes a novel approach to monitoring and fault protection of the Large Hadron Collider (LHC) superconducting magnets which employs state-of-the-art Deep Learning algorithms. Consequently, the authors of the paper decided to examine the performance of LSTM recurrent neural networks for modeling of voltage time series of the magnets. In order to address this challenging task different network architectures and hyper-parameters were used to achieve the best possible performance of the solution. The regression results were measured in terms of RMSE for different number of future steps and history length taken into account for the prediction. The best result of RMSE = 0 . 00104 was obtained for a network of 128 LSTM cells within the internal layer and 16 steps history buffer.

  4. Lyapunov Functions, Stationary Distributions, and Non-equilibrium Potential for Reaction Networks

    DEFF Research Database (Denmark)

    Anderson, David F; Craciun, Gheorghe; Gopalkrishnan, Manoj

    2015-01-01

    We consider the relationship between stationary distributions for stochastic models of reaction systems and Lyapunov functions for their deterministic counterparts. Specifically, we derive the well-known Lyapunov function of reaction network theory as a scaling limit of the non-equilibrium potent...

  5. A note on exponential convergence of neural networks with unbounded distributed delays

    Energy Technology Data Exchange (ETDEWEB)

    Chu Tianguang [Intelligent Control Laboratory, Center for Systems and Control, Department of Mechanics and Engineering Science, Peking University, Beijing 100871 (China)]. E-mail: chutg@pku.edu.cn; Yang Haifeng [Intelligent Control Laboratory, Center for Systems and Control, Department of Mechanics and Engineering Science, Peking University, Beijing 100871 (China)

    2007-12-15

    This note examines issues concerning global exponential convergence of neural networks with unbounded distributed delays. Sufficient conditions are derived by exploiting exponentially fading memory property of delay kernel functions. The method is based on comparison principle of delay differential equations and does not need the construction of any Lyapunov functionals. It is simple yet effective in deriving less conservative exponential convergence conditions and more detailed componentwise decay estimates. The results of this note and [Chu T. An exponential convergence estimate for analog neural networks with delay. Phys Lett A 2001;283:113-8] suggest a class of neural networks whose globally exponentially convergent dynamics is completely insensitive to a wide range of time delays from arbitrary bounded discrete type to certain unbounded distributed type. This is of practical interest in designing fast and reliable neural circuits. Finally, an open question is raised on the nature of delay kernels for attaining exponential convergence in an unbounded distributed delayed neural network.

  6. A note on exponential convergence of neural networks with unbounded distributed delays

    International Nuclear Information System (INIS)

    Chu Tianguang; Yang Haifeng

    2007-01-01

    This note examines issues concerning global exponential convergence of neural networks with unbounded distributed delays. Sufficient conditions are derived by exploiting exponentially fading memory property of delay kernel functions. The method is based on comparison principle of delay differential equations and does not need the construction of any Lyapunov functionals. It is simple yet effective in deriving less conservative exponential convergence conditions and more detailed componentwise decay estimates. The results of this note and [Chu T. An exponential convergence estimate for analog neural networks with delay. Phys Lett A 2001;283:113-8] suggest a class of neural networks whose globally exponentially convergent dynamics is completely insensitive to a wide range of time delays from arbitrary bounded discrete type to certain unbounded distributed type. This is of practical interest in designing fast and reliable neural circuits. Finally, an open question is raised on the nature of delay kernels for attaining exponential convergence in an unbounded distributed delayed neural network

  7. Finite-time convergent recurrent neural network with a hard-limiting activation function for constrained optimization with piecewise-linear objective functions.

    Science.gov (United States)

    Liu, Qingshan; Wang, Jun

    2011-04-01

    This paper presents a one-layer recurrent neural network for solving a class of constrained nonsmooth optimization problems with piecewise-linear objective functions. The proposed neural network is guaranteed to be globally convergent in finite time to the optimal solutions under a mild condition on a derived lower bound of a single gain parameter in the model. The number of neurons in the neural network is the same as the number of decision variables of the optimization problem. Compared with existing neural networks for optimization, the proposed neural network has a couple of salient features such as finite-time convergence and a low model complexity. Specific models for two important special cases, namely, linear programming and nonsmooth optimization, are also presented. In addition, applications to the shortest path problem and constrained least absolute deviation problem are discussed with simulation results to demonstrate the effectiveness and characteristics of the proposed neural network.

  8. Precision position control of servo systems using adaptive back-stepping and recurrent fuzzy neural networks

    International Nuclear Information System (INIS)

    Kim, Han Me; Kim, Jong Shik; Han, Seong Ik

    2009-01-01

    To improve position tracking performance of servo systems, a position tracking control using adaptive back-stepping control(ABSC) scheme and recurrent fuzzy neural networks(RFNN) is proposed. An adaptive rule of the ABSC based on system dynamics and dynamic friction model is also suggested to compensate nonlinear dynamic friction characteristics. However, it is difficult to reduce the position tracking error of servo systems by using only the ABSC scheme because of the system uncertainties which cannot be exactly identified during the modeling of servo systems. Therefore, in order to overcome system uncertainties and then to improve position tracking performance of servo systems, the RFNN technique is additionally applied to the servo system. The feasibility of the proposed control scheme for a servo system is validated through experiments. Experimental results show that the servo system with ABS controller based on the dual friction observer and RFNN including the reconstruction error estimator can achieve desired tracking performance and robustness

  9. Construction of Lyapunov Function for Dissipative Gyroscopic System

    International Nuclear Information System (INIS)

    Xu Wei; Ao Ping; Yuan Bo

    2011-01-01

    We introduce a force decomposition to construct a potential function in deterministic dynamics described by ordinary differential equations in the context of dissipative gyroscopic systems. Such a potential function serves as the corresponding Lyapunov function for the dynamics, hence it gives both quantitative and qualitative descriptions for stability of motion. As an example we apply our force decomposition to a four-dimensional dissipative gyroscopic system. We explicitly obtain the potential function for all parameter regimes in the linear limit, including those regimes where the Lyapunov function was previously believed not to exist. (general)

  10. International Neural Network Society Annual Meeting (1994) Held in San Diego, California on 5-9 June 1994. Volume 3.

    Science.gov (United States)

    1994-06-09

    PROBLEM BASED ON LEARNING IN THE RECURRENT RANDOM NEURAL NETWORK Jose AGUILAR EHEI. UFR de Mathematiques et d’Informatique. Universiti Rene Descartes 45...parallelisme optimal". PHD thesis. Rene Descartes University, Paris, France, 1992. 9. GELENBE, E. "Learning in the recurrent Random Neural Network", Neural

  11. Anisotropies in magnetic field evolution and local Lyapunov exponents

    International Nuclear Information System (INIS)

    Tang, X.Z.; Boozer, A.H.

    2000-01-01

    The natural occurrence of small scale structures and the extreme anisotropy in the evolution of a magnetic field embedded in a conducting flow is interpreted in terms of the properties of the local Lyapunov exponents along the various local characteristic (un)stable directions for the Lagrangian flow trajectories. The local Lyapunov exponents and the characteristic directions are functions of Lagrangian coordinates and time, which are completely determined once the flow field is specified. The characteristic directions that are associated with the spatial anisotropy of the problem, are prescribed in both Lagrangian and Eulerian frames. Coordinate transformation techniques are employed to relate the spatial distributions of the magnetic field, the induced current density, and the Lorentz force, which are usually followed in Eulerian frame, to those of the local Lyapunov exponents, which are naturally defined in Lagrangian coordinates

  12. Learning and retrieval behavior in recurrent neural networks with pre-synaptic dependent homeostatic plasticity

    Science.gov (United States)

    Mizusaki, Beatriz E. P.; Agnes, Everton J.; Erichsen, Rubem; Brunnet, Leonardo G.

    2017-08-01

    The plastic character of brain synapses is considered to be one of the foundations for the formation of memories. There are numerous kinds of such phenomenon currently described in the literature, but their role in the development of information pathways in neural networks with recurrent architectures is still not completely clear. In this paper we study the role of an activity-based process, called pre-synaptic dependent homeostatic scaling, in the organization of networks that yield precise-timed spiking patterns. It encodes spatio-temporal information in the synaptic weights as it associates a learned input with a specific response. We introduce a correlation measure to evaluate the precision of the spiking patterns and explore the effects of different inhibitory interactions and learning parameters. We find that large learning periods are important in order to improve the network learning capacity and discuss this ability in the presence of distinct inhibitory currents.

  13. Recurrent networks for wave forecasting

    Digital Repository Service at National Institute of Oceanography (India)

    Mandal, S.; Prabaharan, N.

    , merchant vessel routing, nearshore construction, etc. more efficiently and safely. This paper presents an application of the Artificial Neural Network, namely Backpropagation Recurrent Neural Network (BRNN) with rprop update algorithm for wave forecasting...

  14. An analysis of global robust stability of uncertain cellular neural networks with discrete and distributed delays

    International Nuclear Information System (INIS)

    Park, Ju H.

    2007-01-01

    This paper considers the robust stability analysis of cellular neural networks with discrete and distributed delays. Based on the Lyapunov stability theory and linear matrix inequality (LMI) technique, a novel stability criterion guaranteeing the global robust convergence of the equilibrium point is derived. The criterion can be solved easily by various convex optimization algorithms. An example is given to illustrate the usefulness of our results

  15. On the existence of polynomial Lyapunov functions for rationally stable vector fields

    DEFF Research Database (Denmark)

    Leth, Tobias; Wisniewski, Rafal; Sloth, Christoffer

    2018-01-01

    This paper proves the existence of polynomial Lyapunov functions for rationally stable vector fields. For practical purposes the existence of polynomial Lyapunov functions plays a significant role since polynomial Lyapunov functions can be found algorithmically. The paper extents an existing result...... on exponentially stable vector fields to the case of rational stability. For asymptotically stable vector fields a known counter example is investigated to exhibit the mechanisms responsible for the inability to extend the result further....

  16. Local Lyapunov exponents for dissipative continuous systems

    International Nuclear Information System (INIS)

    Grond, Florian; Diebner, Hans H.

    2005-01-01

    We analyze a recently proposed algorithm for computing Lyapunov exponents focusing on its capability to calculate reliable local values for chaotic attractors. The averaging process of local contributions to the global measure becomes interpretable, i.e. they are related to the local topological structure in phase space. We compare the algorithm with the commonly used Wolf algorithm by means of analyzing correlations between coordinates of the chaotic attractor and local values of the Lyapunov exponents. The correlations for the new algorithm turn out to be significantly stronger than those for the Wolf algorithm. Since the usage of scalar measures to capture complex structures can be questioned we discuss these entities along with a more phenomenological description of scatter plots

  17. Lyapunov stability analysis of magnetohydrodynamic plasma equilibria with axisymmetric toroidal flow

    International Nuclear Information System (INIS)

    Almaguer, J.A.; Hameiri, E.; Herrera, J.; Holm, D.D.

    1988-01-01

    Lyapunov stability conditions for ideal magnetohydrodynamic (MHD) plasmas with mass flow in axisymmetric toroidal geometry are determined in the Eulerian representation. Axisymmetric equilibrium solutions of ideal MHD are associated to critical points of a nonlinearly conserved Lyapunov functional consisting of the sum of the total energy and the following flux-weighted quantities: the circulation along field lines, the angular momentum, the toroidal flux, and the mass content within each flux tube. Conditions sufficient for Lyapunov stability of these equilibria against axisymmetric perturbations are found by taking advantage of the Hamiltonian formalism for ideal MHD. In particular [see Eq. (60)], it is sufficient for Lyapunov stability under linearized dynamics that an axisymmetric equilibrium be subsonic in the appropriate rotating frame, lie in the first elliptic regime of the Bernoulli--Grad--Shafranov (BGS) system of equations, and satisfy one additional, more complicated, condition. Effects of boundary conditions, nonlinearity, and three-dimensionality on MHD stability are also discussed

  18. Dynamical behaviors determined by the Lyapunov function in competitive Lotka-Volterra systems

    Science.gov (United States)

    Tang, Ying; Yuan, Ruoshi; Ma, Yian

    2013-01-01

    Dynamical behaviors of the competitive Lotka-Volterra system even for 3 species are not fully understood. In this paper, we study this problem from the perspective of the Lyapunov function. We construct explicitly the Lyapunov function using three examples of the competitive Lotka-Volterra system for the whole state space: (1) the general 2-species case, (2) a 3-species model, and (3) the model of May-Leonard. The basins of attraction for these examples are demonstrated, including cases with bistability and cyclical behavior. The first two examples are the generalized gradient system, where the energy dissipation may not follow the gradient of the Lyapunov function. In addition, under a new type of stochastic interpretation, the Lyapunov function also leads to the Boltzmann-Gibbs distribution on the final steady state when multiplicative noise is added.

  19. Global stability of stochastic high-order neural networks with discrete and distributed delays

    International Nuclear Information System (INIS)

    Wang Zidong; Fang Jianan; Liu Xiaohui

    2008-01-01

    High-order neural networks can be considered as an expansion of Hopfield neural networks, and have stronger approximation property, faster convergence rate, greater storage capacity, and higher fault tolerance than lower-order neural networks. In this paper, the global asymptotic stability analysis problem is considered for a class of stochastic high-order neural networks with discrete and distributed time-delays. Based on an Lyapunov-Krasovskii functional and the stochastic stability analysis theory, several sufficient conditions are derived, which guarantee the global asymptotic convergence of the equilibrium point in the mean square. It is shown that the stochastic high-order delayed neural networks under consideration are globally asymptotically stable in the mean square if two linear matrix inequalities (LMIs) are feasible, where the feasibility of LMIs can be readily checked by the Matlab LMI toolbox. It is also shown that the main results in this paper cover some recently published works. A numerical example is given to demonstrate the usefulness of the proposed global stability criteria

  20. Recurrent fuzzy neural network by using feedback error learning approaches for LFC in interconnected power system

    International Nuclear Information System (INIS)

    Sabahi, Kamel; Teshnehlab, Mohammad; Shoorhedeli, Mahdi Aliyari

    2009-01-01

    In this study, a new adaptive controller based on modified feedback error learning (FEL) approaches is proposed for load frequency control (LFC) problem. The FEL strategy consists of intelligent and conventional controllers in feedforward and feedback paths, respectively. In this strategy, a conventional feedback controller (CFC), i.e. proportional, integral and derivative (PID) controller, is essential to guarantee global asymptotic stability of the overall system; and an intelligent feedforward controller (INFC) is adopted to learn the inverse of the controlled system. Therefore, when the INFC learns the inverse of controlled system, the tracking of reference signal is done properly. Generally, the CFC is designed at nominal operating conditions of the system and, therefore, fails to provide the best control performance as well as global stability over a wide range of changes in the operating conditions of the system. So, in this study a supervised controller (SC), a lookup table based controller, is addressed for tuning of the CFC. During abrupt changes of the power system parameters, the SC adjusts the PID parameters according to these operating conditions. Moreover, for improving the performance of overall system, a recurrent fuzzy neural network (RFNN) is adopted in INFC instead of the conventional neural network, which was used in past studies. The proposed FEL controller has been compared with the conventional feedback error learning controller (CFEL) and the PID controller through some performance indices

  1. Distributed Recurrent Neural Forward Models with Neural Control for Complex Locomotion in Walking Robots

    DEFF Research Database (Denmark)

    Dasgupta, Sakyasingha; Goldschmidt, Dennis; Wörgötter, Florentin

    2015-01-01

    here, an artificial bio-inspired walking system which effectively combines biomechanics (in terms of the body and leg structures) with the underlying neural mechanisms. The neural mechanisms consist of (1) central pattern generator based control for generating basic rhythmic patterns and coordinated......Walking animals, like stick insects, cockroaches or ants, demonstrate a fascinating range of locomotive abilities and complex behaviors. The locomotive behaviors can consist of a variety of walking patterns along with adaptation that allow the animals to deal with changes in environmental...... conditions, like uneven terrains, gaps, obstacles etc. Biological study has revealed that such complex behaviors are a result of a combination of biomechanics and neural mechanism thus representing the true nature of embodied interactions. While the biomechanics helps maintain flexibility and sustain...

  2. Deep Recurrent Neural Networks for seizure detection and early seizure detection systems

    Energy Technology Data Exchange (ETDEWEB)

    Talathi, S. S. [Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)

    2017-06-05

    Epilepsy is common neurological diseases, affecting about 0.6-0.8 % of world population. Epileptic patients suffer from chronic unprovoked seizures, which can result in broad spectrum of debilitating medical and social consequences. Since seizures, in general, occur infrequently and are unpredictable, automated seizure detection systems are recommended to screen for seizures during long-term electroencephalogram (EEG) recordings. In addition, systems for early seizure detection can lead to the development of new types of intervention systems that are designed to control or shorten the duration of seizure events. In this article, we investigate the utility of recurrent neural networks (RNNs) in designing seizure detection and early seizure detection systems. We propose a deep learning framework via the use of Gated Recurrent Unit (GRU) RNNs for seizure detection. We use publicly available data in order to evaluate our method and demonstrate very promising evaluation results with overall accuracy close to 100 %. We also systematically investigate the application of our method for early seizure warning systems. Our method can detect about 98% of seizure events within the first 5 seconds of the overall epileptic seizure duration.

  3. Auto-Associative Recurrent Neural Networks and Long Term Dependencies in Novelty Detection for Audio Surveillance Applications

    Science.gov (United States)

    Rossi, A.; Montefoschi, F.; Rizzo, A.; Diligenti, M.; Festucci, C.

    2017-10-01

    Machine Learning applied to Automatic Audio Surveillance has been attracting increasing attention in recent years. In spite of several investigations based on a large number of different approaches, little attention had been paid to the environmental temporal evolution of the input signal. In this work, we propose an exploration in this direction comparing the temporal correlations extracted at the feature level with the one learned by a representational structure. To this aim we analysed the prediction performances of a Recurrent Neural Network architecture varying the length of the processed input sequence and the size of the time window used in the feature extraction. Results corroborated the hypothesis that sequential models work better when dealing with data characterized by temporal order. However, so far the optimization of the temporal dimension remains an open issue.

  4. Abstraction of continuous dynamical systems utilizing lyapunov functions

    DEFF Research Database (Denmark)

    Sloth, Christoffer; Wisniewski, Rafael

    2010-01-01

    This paper considers the development of a method for abstracting continuous dynamical systems by timed automata. The method is based on partitioning the state space of dynamical systems with invariant sets, which form cells representing locations of the timed automata. To enable verification...... of the dynamical system based on the abstraction, conditions for obtaining sound, complete, and refinable abstractions are set up. It is proposed to partition the state space utilizing sub-level sets of Lyapunov functions, since they are positive invariant sets. The existence of sound abstractions for Morse......-Smale systems and complete and refinable abstractions for linear systems are shown....

  5. Inertia theorems for operator Lyapunov inequalities

    NARCIS (Netherlands)

    Sasane, AJ; Curtain, RF

    2001-01-01

    We study operator Lyapunov inequalities and equations for which the infinitesimal generator is not necessarily stable, but it satisfies the spectrum decomposition assumption and it has at most finitely many unstable eigenvalues. Moreover, the input or output operators are not necessarily bounded,

  6. Coupled Heuristic Prediction of Long Lead-Time Accumulated Total Inflow of a Reservoir during Typhoons Using Deterministic Recurrent and Fuzzy Inference-Based Neural Network

    Directory of Open Access Journals (Sweden)

    Chien-Lin Huang

    2015-11-01

    Full Text Available This study applies Real-Time Recurrent Learning Neural Network (RTRLNN and Adaptive Network-based Fuzzy Inference System (ANFIS with novel heuristic techniques to develop an advanced prediction model of accumulated total inflow of a reservoir in order to solve the difficulties of future long lead-time highly varied uncertainty during typhoon attacks while using a real-time forecast. For promoting the temporal-spatial forecasted precision, the following original specialized heuristic inputs were coupled: observed-predicted inflow increase/decrease (OPIID rate, total precipitation, and duration from current time to the time of maximum precipitation and direct runoff ending (DRE. This study also investigated the temporal-spatial forecasted error feature to assess the feasibility of the developed models, and analyzed the output sensitivity of both single and combined heuristic inputs to determine whether the heuristic model is susceptible to the impact of future forecasted uncertainty/errors. Validation results showed that the long lead-time–predicted accuracy and stability of the RTRLNN-based accumulated total inflow model are better than that of the ANFIS-based model because of the real-time recurrent deterministic routing mechanism of RTRLNN. Simulations show that the RTRLNN-based model with coupled heuristic inputs (RTRLNN-CHI, average error percentage (AEP/average forecast lead-time (AFLT: 6.3%/49 h can achieve better prediction than the model with non-heuristic inputs (AEP of RTRLNN-NHI and ANFIS-NHI: 15.2%/31.8% because of the full consideration of real-time hydrological initial/boundary conditions. Besides, the RTRLNN-CHI model can promote the forecasted lead-time above 49 h with less than 10% of AEP which can overcome the previous forecasted limits of 6-h AFLT with above 20%–40% of AEP.

  7. Lyapunov stability of ideal compressible and incompressible fluid equilibria in three dimensions

    International Nuclear Information System (INIS)

    Holm, D.D.

    1985-08-01

    Linearized stability of ideal compressible and incompressible fluid equilibria in three dimensions is analyzed using Lyapunov's direct method. An action principle is given for the Eulerian and Lagrangian fluid descriptions and the family of constants of motion due to symmetry under fluid-particle relabelling is derived in the form of Ertel's theorem for each description. In an augmented Euleriah description, the steady equilibrium flows of these two fluids theories are identified as critical points of the conserved Lyapunov functionals defined by the sum, H + C, of the energy H, and the Ertel constants of motion, C. It turns out that unconditional linear Lyapunov stability of these flows in the norm provided by the second variation of H + C is precluded by vortex-particle stretching, even for otherwise shear-stable flows. Conditional Lyapunov stability of these flows is discussed. 24 refs

  8. Bayesian Recurrent Neural Network for Language Modeling.

    Science.gov (United States)

    Chien, Jen-Tzung; Ku, Yuan-Chu

    2016-02-01

    A language model (LM) is calculated as the probability of a word sequence that provides the solution to word prediction for a variety of information systems. A recurrent neural network (RNN) is powerful to learn the large-span dynamics of a word sequence in the continuous space. However, the training of the RNN-LM is an ill-posed problem because of too many parameters from a large dictionary size and a high-dimensional hidden layer. This paper presents a Bayesian approach to regularize the RNN-LM and apply it for continuous speech recognition. We aim to penalize the too complicated RNN-LM by compensating for the uncertainty of the estimated model parameters, which is represented by a Gaussian prior. The objective function in a Bayesian classification network is formed as the regularized cross-entropy error function. The regularized model is constructed not only by calculating the regularized parameters according to the maximum a posteriori criterion but also by estimating the Gaussian hyperparameter by maximizing the marginal likelihood. A rapid approximation to a Hessian matrix is developed to implement the Bayesian RNN-LM (BRNN-LM) by selecting a small set of salient outer-products. The proposed BRNN-LM achieves a sparser model than the RNN-LM. Experiments on different corpora show the robustness of system performance by applying the rapid BRNN-LM under different conditions.

  9. Adaptive exponential synchronization of delayed neural networks with reaction-diffusion terms

    International Nuclear Information System (INIS)

    Sheng Li; Yang Huizhong; Lou Xuyang

    2009-01-01

    This paper presents an exponential synchronization scheme for a class of neural networks with time-varying and distributed delays and reaction-diffusion terms. An adaptive synchronization controller is derived to achieve the exponential synchronization of the drive-response structure of neural networks by using the Lyapunov stability theory. At the same time, the update laws of parameters are proposed to guarantee the synchronization of delayed neural networks with all parameters unknown. It is shown that the approaches developed here extend and improve the ideas presented in recent literatures.

  10. Adaptive dynamic inversion robust control for BTT missile based on wavelet neural network

    Science.gov (United States)

    Li, Chuanfeng; Wang, Yongji; Deng, Zhixiang; Wu, Hao

    2009-10-01

    A new nonlinear control strategy incorporated the dynamic inversion method with wavelet neural networks is presented for the nonlinear coupling system of Bank-to-Turn(BTT) missile in reentry phase. The basic control law is designed by using the dynamic inversion feedback linearization method, and the online learning wavelet neural network is used to compensate the inversion error due to aerodynamic parameter errors, modeling imprecise and external disturbance in view of the time-frequency localization properties of wavelet transform. Weights adjusting laws are derived according to Lyapunov stability theory, which can guarantee the boundedness of all signals in the whole system. Furthermore, robust stability of the closed-loop system under this tracking law is proved. Finally, the six degree-of-freedom(6DOF) simulation results have shown that the attitude angles can track the anticipant command precisely under the circumstances of existing external disturbance and in the presence of parameter uncertainty. It means that the dependence on model by dynamic inversion method is reduced and the robustness of control system is enhanced by using wavelet neural network(WNN) to reconstruct inversion error on-line.

  11. A development of the direct Lyapunov method for the analysis of transient stability of a system of synchronous generators based on the determination of non- stable equilibria on a multidimensional sphere

    Directory of Open Access Journals (Sweden)

    A. V. Stepanov

    2014-01-01

    Full Text Available A development of the direct Lyapunov method for the analysis of transient stability of a system of synchronous generators based on the determination of non- stable equilibria on a multidimensional sphere.We consider the problem of transient stability analysis for a system of synchronous generators under the action of strong perturbations. The aim of our work is to develop methods to analyze a transient stability of the system of synchronous generators, which allow getting trustworthy results on reserve transient stability under different perturbations. For the analysis of transient stability, we use the direct Lyapunov method.One of the problems for this method application is to find the Lypunov function that well reflects the properties of a parallel system of synchronous generators. The most reliable results were obtained when the analysis of transient stability was performed with a Lyapunov function of energy type. Another problem for application of the direct Lyapunov method is to determine the critical value of the Lyapunov function, which requires finding the non-stable equilibria of the system. Determination of the non-stable equilibria requires studying the Lyapunov function in a multidimensional space in a neighborhood of a stable equilibrium for the post-breakdown system; this is a complicated non-linear problem.In the paper, we propose a method for determination of the non-stable equilibria on a multidimensional sphere. The method is based on a search of a minimum of the Lyapunov function on a multidimensional sphere the center of which is a stable equilibrium. Our method allows, comparing with the other, e.g., gradient methods, reliable finding a non-stable equilibrium and calculating the critical value. The reliability of our method is proved by numerical experiments. The developed methods and a program realized in a MATLAB package can be recommended for design of a post-breakdown control system of synchronous generators or as a

  12. Nonlinear Lyapunov-based boundary control of distributed heat transfer mechanisms in membrane distillation plant

    KAUST Repository

    Eleiwi, Fadi

    2015-07-01

    This paper presents a nonlinear Lyapunov-based boundary control for the temperature difference of a membrane distillation boundary layers. The heat transfer mechanisms inside the process are modeled with a 2D advection-diffusion equation. The model is semi-descretized in space, and a nonlinear state-space representation is provided. The control is designed to force the temperature difference along the membrane sides to track a desired reference asymptotically, and hence a desired flux would be generated. Certain constraints are put on the control law inputs to be within an economic range of energy supplies. The effect of the controller gain is discussed. Simulations with real process parameters for the model, and the controller are provided. © 2015 American Automatic Control Council.

  13. Almost Periodic Solution for Memristive Neural Networks with Time-Varying Delays

    Directory of Open Access Journals (Sweden)

    Huaiqin Wu

    2013-01-01

    Full Text Available This paper is concerned with the dynamical stability analysis for almost periodic solution of memristive neural networks with time-varying delays. Under the framework of Filippov solutions, by applying the inequality analysis techniques, the existence and asymptotically almost periodic behavior of solutions are discussed. Based on the differential inclusions theory and Lyapunov functional approach, the stability issues of almost periodic solution are investigated, and a sufficient condition for the existence, uniqueness, and global exponential stability of the almost periodic solution is established. Moreover, as a special case, the condition which ensures the global exponential stability of a unique periodic solution is also presented for the considered memristive neural networks. Two examples are given to illustrate the validity of the theoretical results.

  14. A novel delay-dependent criterion for delayed neural networks of neutral type

    International Nuclear Information System (INIS)

    Lee, S.M.; Kwon, O.M.; Park, Ju H.

    2010-01-01

    This Letter considers a robust stability analysis method for delayed neural networks of neutral type. By constructing a new Lyapunov functional, a novel delay-dependent criterion for the stability is derived in terms of LMIs (linear matrix inequalities). A less conservative stability criterion is derived by using nonlinear properties of the activation function of the neural networks. Two numerical examples are illustrated to show the effectiveness of the proposed method.

  15. Chaos Synchronization Using Adaptive Dynamic Neural Network Controller with Variable Learning Rates

    Directory of Open Access Journals (Sweden)

    Chih-Hong Kao

    2011-01-01

    Full Text Available This paper addresses the synchronization of chaotic gyros with unknown parameters and external disturbance via an adaptive dynamic neural network control (ADNNC system. The proposed ADNNC system is composed of a neural controller and a smooth compensator. The neural controller uses a dynamic RBF (DRBF network to online approximate an ideal controller. The DRBF network can create new hidden neurons online if the input data falls outside the hidden layer and prune the insignificant hidden neurons online if the hidden neuron is inappropriate. The smooth compensator is designed to compensate for the approximation error between the neural controller and the ideal controller. Moreover, the variable learning rates of the parameter adaptation laws are derived based on a discrete-type Lyapunov function to speed up the convergence rate of the tracking error. Finally, the simulation results which verified the chaotic behavior of two nonlinear identical chaotic gyros can be synchronized using the proposed ADNNC scheme.

  16. Adaptive Sliding Mode Control of Chaos in Permanent Magnet Synchronous Motor via Fuzzy Neural Networks

    Directory of Open Access Journals (Sweden)

    Tat-Bao-Thien Nguyen

    2014-01-01

    Full Text Available In this paper, based on fuzzy neural networks, we develop an adaptive sliding mode controller for chaos suppression and tracking control in a chaotic permanent magnet synchronous motor (PMSM drive system. The proposed controller consists of two parts. The first is an adaptive sliding mode controller which employs a fuzzy neural network to estimate the unknown nonlinear models for constructing the sliding mode controller. The second is a compensational controller which adaptively compensates estimation errors. For stability analysis, the Lyapunov synthesis approach is used to ensure the stability of controlled systems. Finally, simulation results are provided to verify the validity and superiority of the proposed method.

  17. Biological oscillations for learning walking coordination: dynamic recurrent neural network functionally models physiological central pattern generator.

    Science.gov (United States)

    Hoellinger, Thomas; Petieau, Mathieu; Duvinage, Matthieu; Castermans, Thierry; Seetharaman, Karthik; Cebolla, Ana-Maria; Bengoetxea, Ana; Ivanenko, Yuri; Dan, Bernard; Cheron, Guy

    2013-01-01

    The existence of dedicated neuronal modules such as those organized in the cerebral cortex, thalamus, basal ganglia, cerebellum, or spinal cord raises the question of how these functional modules are coordinated for appropriate motor behavior. Study of human locomotion offers an interesting field for addressing this central question. The coordination of the elevation of the 3 leg segments under a planar covariation rule (Borghese et al., 1996) was recently modeled (Barliya et al., 2009) by phase-adjusted simple oscillators shedding new light on the understanding of the central pattern generator (CPG) processing relevant oscillation signals. We describe the use of a dynamic recurrent neural network (DRNN) mimicking the natural oscillatory behavior of human locomotion for reproducing the planar covariation rule in both legs at different walking speeds. Neural network learning was based on sinusoid signals integrating frequency and amplitude features of the first three harmonics of the sagittal elevation angles of the thigh, shank, and foot of each lower limb. We verified the biological plausibility of the neural networks. Best results were obtained with oscillations extracted from the first three harmonics in comparison to oscillations outside the harmonic frequency peaks. Physiological replication steadily increased with the number of neuronal units from 1 to 80, where similarity index reached 0.99. Analysis of synaptic weighting showed that the proportion of inhibitory connections consistently increased with the number of neuronal units in the DRNN. This emerging property in the artificial neural networks resonates with recent advances in neurophysiology of inhibitory neurons that are involved in central nervous system oscillatory activities. The main message of this study is that this type of DRNN may offer a useful model of physiological central pattern generator for gaining insights in basic research and developing clinical applications.

  18. A loop-based neural architecture for structured behavior encoding and decoding.

    Science.gov (United States)

    Gisiger, Thomas; Boukadoum, Mounir

    2018-02-01

    We present a new type of artificial neural network that generalizes on anatomical and dynamical aspects of the mammal brain. Its main novelty lies in its topological structure which is built as an array of interacting elementary motifs shaped like loops. These loops come in various types and can implement functions such as gating, inhibitory or executive control, or encoding of task elements to name a few. Each loop features two sets of neurons and a control region, linked together by non-recurrent projections. The two neural sets do the bulk of the loop's computations while the control unit specifies the timing and the conditions under which the computations implemented by the loop are to be performed. By functionally linking many such loops together, a neural network is obtained that may perform complex cognitive computations. To demonstrate the potential offered by such a system, we present two neural network simulations. The first illustrates the structure and dynamics of a single loop implementing a simple gating mechanism. The second simulation shows how connecting four loops in series can produce neural activity patterns that are sufficient to pass a simplified delayed-response task. We also show that this network reproduces electrophysiological measurements gathered in various regions of the brain of monkeys performing similar tasks. We also demonstrate connections between this type of neural network and recurrent or long short-term memory network models, and suggest ways to generalize them for future artificial intelligence research. Copyright © 2017 Elsevier Ltd. All rights reserved.

  19. Recurrent neural network approach to quantum signal: coherent state restoration for continuous-variable quantum key distribution

    Science.gov (United States)

    Lu, Weizhao; Huang, Chunhui; Hou, Kun; Shi, Liting; Zhao, Huihui; Li, Zhengmei; Qiu, Jianfeng

    2018-05-01

    In continuous-variable quantum key distribution (CV-QKD), weak signal carrying information transmits from Alice to Bob; during this process it is easily influenced by unknown noise which reduces signal-to-noise ratio, and strongly impacts reliability and stability of the communication. Recurrent quantum neural network (RQNN) is an artificial neural network model which can perform stochastic filtering without any prior knowledge of the signal and noise. In this paper, a modified RQNN algorithm with expectation maximization algorithm is proposed to process the signal in CV-QKD, which follows the basic rule of quantum mechanics. After RQNN, noise power decreases about 15 dBm, coherent signal recognition rate of RQNN is 96%, quantum bit error rate (QBER) drops to 4%, which is 6.9% lower than original QBER, and channel capacity is notably enlarged.

  20. Simulating the dynamics of the neutron flux in a nuclear reactor by locally recurrent neural networks

    International Nuclear Information System (INIS)

    Cadini, F.; Zio, E.; Pedroni, N.

    2007-01-01

    In this paper, a locally recurrent neural network (LRNN) is employed for approximating the temporal evolution of a nonlinear dynamic system model of a simplified nuclear reactor. To this aim, an infinite impulse response multi-layer perceptron (IIR-MLP) is trained according to a recursive back-propagation (RBP) algorithm. The network nodes contain internal feedback paths and their connections are realized by means of IIR synaptic filters, which provide the LRNN with the necessary system state memory

  1. Periodic bidirectional associative memory neural networks with distributed delays

    Science.gov (United States)

    Chen, Anping; Huang, Lihong; Liu, Zhigang; Cao, Jinde

    2006-05-01

    Some sufficient conditions are obtained for the existence and global exponential stability of a periodic solution to the general bidirectional associative memory (BAM) neural networks with distributed delays by using the continuation theorem of Mawhin's coincidence degree theory and the Lyapunov functional method and the Young's inequality technique. These results are helpful for designing a globally exponentially stable and periodic oscillatory BAM neural network, and the conditions can be easily verified and be applied in practice. An example is also given to illustrate our results.

  2. Feature Set Evaluation for Offline Handwriting Recognition Systems: Application to the Recurrent Neural Network Model.

    Science.gov (United States)

    Chherawala, Youssouf; Roy, Partha Pratim; Cheriet, Mohamed

    2016-12-01

    The performance of handwriting recognition systems is dependent on the features extracted from the word image. A large body of features exists in the literature, but no method has yet been proposed to identify the most promising of these, other than a straightforward comparison based on the recognition rate. In this paper, we propose a framework for feature set evaluation based on a collaborative setting. We use a weighted vote combination of recurrent neural network (RNN) classifiers, each trained with a particular feature set. This combination is modeled in a probabilistic framework as a mixture model and two methods for weight estimation are described. The main contribution of this paper is to quantify the importance of feature sets through the combination weights, which reflect their strength and complementarity. We chose the RNN classifier because of its state-of-the-art performance. Also, we provide the first feature set benchmark for this classifier. We evaluated several feature sets on the IFN/ENIT and RIMES databases of Arabic and Latin script, respectively. The resulting combination model is competitive with state-of-the-art systems.

  3. Determining the Lyapunov Spectrum of Continuous-Time 1D and 2D Multiscroll Chaotic Oscillators via the Solution of m-PWL Variational Equations

    Directory of Open Access Journals (Sweden)

    Jesus Manuel Munoz-Pacheco

    2013-01-01

    Full Text Available An algorithm to compute the Lyapunov exponents of piecewise linear function-based multidirectional multiscroll chaotic oscillators is reported. Based on the m regions in the piecewise linear functions, the suggested algorithm determines the individual expansion rate of Lyapunov exponents from m-piecewise linear variational equations and their associated m-Jacobian matrices whose entries remain constant during all computation cycles. Additionally, by considering OpAmp-based chaotic oscillators, we study the impact of two analog design procedures on the magnitude of Lyapunov exponents. We focus on analyzing variations of both frequency bandwidth and voltage/current dynamic range of the chaotic signals at electronic system level. As a function of the design parameters, a renormalization factor is proposed to estimate correctly the Lyapunov spectrum. Numerical simulation results in a double-scroll type chaotic oscillator and complex chaotic oscillators generating multidirectional multiscroll chaotic attractors on phase space confirm the usefulness of the reported algorithm.

  4. Nonlinear generalized synchronization of chaotic systems by pure error dynamics and elaborate nondiagonal Lyapunov function

    International Nuclear Information System (INIS)

    Ge Zhengming; Chang Chingming

    2009-01-01

    By applying pure error dynamics and elaborate nondiagonal Lyapunov function, the nonlinear generalized synchronization is studied in this paper. Instead of current mixed error dynamics in which master state variables and slave state variables are presented, the nonlinear generalized synchronization can be obtained by pure error dynamics without auxiliary numerical simulation. The elaborate nondiagonal Lyapunov function is applied rather than current monotonous square sum Lyapunov function deeply weakening the powerfulness of Lyapunov direct method. Both autonomous and nonautonomous double Mathieu systems are used as examples with numerical simulations.

  5. Neural Based Orthogonal Data Fitting The EXIN Neural Networks

    CERN Document Server

    Cirrincione, Giansalvo

    2008-01-01

    Written by three leaders in the field of neural based algorithms, Neural Based Orthogonal Data Fitting proposes several neural networks, all endowed with a complete theory which not only explains their behavior, but also compares them with the existing neural and traditional algorithms. The algorithms are studied from different points of view, including: as a differential geometry problem, as a dynamic problem, as a stochastic problem, and as a numerical problem. All algorithms have also been analyzed on real time problems (large dimensional data matrices) and have shown accurate solutions. Wh

  6. Delay-Dependent Exponential Optimal Synchronization for Nonidentical Chaotic Systems via Neural-Network-Based Approach

    Directory of Open Access Journals (Sweden)

    Feng-Hsiag Hsiao

    2013-01-01

    Full Text Available A novel approach is presented to realize the optimal exponential synchronization of nonidentical multiple time-delay chaotic (MTDC systems via fuzzy control scheme. A neural-network (NN model is first constructed for the MTDC system. Then, a linear differential inclusion (LDI state-space representation is established for the dynamics of the NN model. Based on this LDI state-space representation, a delay-dependent exponential stability criterion of the error system derived in terms of Lyapunov's direct method is proposed to guarantee that the trajectories of the slave system can approach those of the master system. Subsequently, the stability condition of this criterion is reformulated into a linear matrix inequality (LMI. According to the LMI, a fuzzy controller is synthesized not only to realize the exponential synchronization but also to achieve the optimal performance by minimizing the disturbance attenuation level at the same time. Finally, a numerical example with simulations is given to demonstrate the effectiveness of our approach.

  7. Global asymptotic stability of Cohen-Grossberg neural networks with constant and variable delays

    International Nuclear Information System (INIS)

    Wu Wei; Cui Baotong; Huang Min

    2007-01-01

    Global asymptotic stability of Cohen-Grossberg neural networks with constant and variable delays is studied. Some sufficient conditions for the neural networks are proposed to guarantee the global asymptotic convergence by using different Lyapunov functionals. Our criteria represent an extension of the existing results in literatures. A comparison between our results and the previous results admits that our results establish a new set of stability criteria for delayed Cohen-Grossberg neural networks. Those conditions are less restrictive than those given in the earlier reference

  8. New results for global exponential synchronization in neural networks via functional differential inclusions.

    Science.gov (United States)

    Wang, Dongshu; Huang, Lihong; Tang, Longkun

    2015-08-01

    This paper is concerned with the synchronization dynamical behaviors for a class of delayed neural networks with discontinuous neuron activations. Continuous and discontinuous state feedback controller are designed such that the neural networks model can realize exponential complete synchronization in view of functional differential inclusions theory, Lyapunov functional method and inequality technique. The new proposed results here are very easy to verify and also applicable to neural networks with continuous activations. Finally, some numerical examples show the applicability and effectiveness of our main results.

  9. Issues in the use of neural networks in information retrieval

    CERN Document Server

    Iatan, Iuliana F

    2017-01-01

    This book highlights the ability of neural networks (NNs) to be excellent pattern matchers and their importance in information retrieval (IR), which is based on index term matching. The book defines a new NN-based method for learning image similarity and describes how to use fuzzy Gaussian neural networks to predict personality. It introduces the fuzzy Clifford Gaussian network, and two concurrent neural models: (1) concurrent fuzzy nonlinear perceptron modules, and (2) concurrent fuzzy Gaussian neural network modules. Furthermore, it explains the design of a new model of fuzzy nonlinear perceptron based on alpha level sets and describes a recurrent fuzzy neural network model with a learning algorithm based on the improved particle swarm optimization method.

  10. Self-Tuning Vibration Control of a Rotational Flexible Timoshenko Arm Using Neural Networks

    Directory of Open Access Journals (Sweden)

    Minoru Sasaki

    2012-01-01

    Full Text Available A self-tuning vibration control of a rotational flexible arm using neural networks is presented. To the self-tuning control system, the control scheme consists of gain tuning neural networks and a variable-gain feedback controller. The neural networks are trained so as to make the root moment zero. In the process, the neural networks learn the optimal gain of the feedback controller. The feedback controller is designed based on Lyapunov's direct method. The feedback control of the vibration of the flexible system is derived by considering the time rate of change of the total energy of the system. This approach has the advantage over the conventional methods in the respect that it allows one to deal directly with the system's partial differential equations without resorting to approximations. Numerical and experimental results for the vibration control of a rotational flexible arm are discussed. It verifies that the proposed control system is effective at controlling flexible dynamical systems.

  11. Statistical-mechanical formulation of Lyapunov exponents

    International Nuclear Information System (INIS)

    Tanase-Nicola, Sorin; Kurchan, Jorge

    2003-01-01

    We show how the Lyapunov exponents of a dynamic system can, in general, be expressed in terms of the free energy of a (non-Hermitian) quantum many-body problem. This puts their study as a problem of statistical mechanics, whose intuitive concepts and techniques of approximation can hence be borrowed

  12. Command Filtered Adaptive Fuzzy Neural Network Backstepping Control for Marine Power System

    Directory of Open Access Journals (Sweden)

    Xin Zhang

    2014-01-01

    Full Text Available In order to retrain chaotic oscillation of marine power system which is excited by periodic electromagnetism perturbation, a novel command-filtered adaptive fuzzy neural network backstepping control method is designed. First, the mathematical model of marine power system is established based on the two parallel nonlinear model. Then, main results of command-filtered adaptive fuzzy neural network backstepping control law are given. And the Lyapunov stability theory is applied to prove that the system can remain closed-loop asymptotically stable with this controller. Finally, simulation results indicate that the designed controller can suppress chaotic oscillation with fast convergence speed that makes the system return to the equilibrium point quickly; meanwhile, the parameter which induces chaotic oscillation can also be discriminated.

  13. Generalized direct Lyapunov method for the analysis of stability and attraction in general time systems

    International Nuclear Information System (INIS)

    Druzhinina, O V; Shestakov, A A

    2002-01-01

    A generalized direct Lyapunov method is put forward for the study of stability and attraction in general time systems of the following types: the classical dynamical system in the sense of Birkhoff, the general system in the sense of Zubov, the general system in the sense of Seibert, the general system with delay, and the general 'input-output' system. For such systems, with the help of generalized Lyapunov functions with respect to two filters, two quasifilters, or two filter bases, necessary and sufficient conditions for stability and attraction are obtained under minimal assumptions about the mathematical structure of the general system

  14. Thermodynamic stability of elementary chemical reactions proceeding at finite rates revisited using Lyapunov function analysis

    International Nuclear Information System (INIS)

    Burande, Chandrakant S.; Bhalekar, Anil A.

    2005-01-01

    The thermodynamic stability of a few representative elementary chemical reactions proceeding at finite rates has been investigated using the recently proposed thermodynamic Lyapunov function and following the steps of Lyapunov's second method (also termed as the direct method) of stability of motion. The thermodynamic Lyapunov function; L s , used herein is the excess rate of entropy production in the thermodynamic perturbation space, which thereby inherits the dictates of the second law of thermodynamics. This Lyapunov function is not the same as the excess entropy rate that one encounters in thermodynamic (irreversible) literature. The model chemical conversions studied in this presentation are A+B→v x X and A+B↔ν x X. For the sake of simplicity, the thermal effects of chemical reactions have been considered as not adding to the perturbation as our main aim was to demonstrate how one should use systematically the proposed thermodynamic Lyapunov function following the steps of Lyapunov's second method of stability of motion. The domains of thermodynamic stability under the constantly acting small disturbances, thermodynamic asymptotic stability and thermodynamic instability in these model systems get established

  15. New BFA Method Based on Attractor Neural Network and Likelihood Maximization

    Czech Academy of Sciences Publication Activity Database

    Frolov, A. A.; Húsek, Dušan; Polyakov, P.Y.; Snášel, V.

    2014-01-01

    Roč. 132, 20 May (2014), s. 14-29 ISSN 0925-2312 Grant - others:GA MŠk(CZ) ED1.1.00/02.0070; GA MŠk(CZ) EE.2.3.20.0073 Program:ED Institutional support: RVO:67985807 Keywords : recurrent neural network * associative memory * Hebbian learning rule * neural network application * data mining * statistics * Boolean factor analysis * information gain * dimension reduction * likelihood-maximization * bars problem Subject RIV: IN - Informatics, Computer Science Impact factor: 2.083, year: 2014

  16. Signal Processing and Neural Network Simulator

    Science.gov (United States)

    Tebbe, Dennis L.; Billhartz, Thomas J.; Doner, John R.; Kraft, Timothy T.

    1995-04-01

    The signal processing and neural network simulator (SPANNS) is a digital signal processing simulator with the capability to invoke neural networks into signal processing chains. This is a generic tool which will greatly facilitate the design and simulation of systems with embedded neural networks. The SPANNS is based on the Signal Processing WorkSystemTM (SPWTM), a commercial-off-the-shelf signal processing simulator. SPW provides a block diagram approach to constructing signal processing simulations. Neural network paradigms implemented in the SPANNS include Backpropagation, Kohonen Feature Map, Outstar, Fully Recurrent, Adaptive Resonance Theory 1, 2, & 3, and Brain State in a Box. The SPANNS was developed by integrating SAIC's Industrial Strength Neural Networks (ISNN) Software into SPW.

  17. The synaptic properties of cells define the hallmarks of interval timing in a recurrent neural network.

    Science.gov (United States)

    Pérez, Oswaldo; Merchant, Hugo

    2018-04-03

    Extensive research has described two key features of interval timing. The bias property is associated with accuracy and implies that time is overestimated for short intervals and underestimated for long intervals. The scalar property is linked to precision and states that the variability of interval estimates increases as a function of interval duration. The neural mechanisms behind these properties are not well understood. Here we implemented a recurrent neural network that mimics a cortical ensemble and includes cells that show paired-pulse facilitation and slow inhibitory synaptic currents. The network produces interval selective responses and reproduces both bias and scalar properties when a Bayesian decoder reads its activity. Notably, the interval-selectivity, timing accuracy, and precision of the network showed complex changes as a function of the decay time constants of the modeled synaptic properties and the level of background activity of the cells. These findings suggest that physiological values of the time constants for paired-pulse facilitation and GABAb, as well as the internal state of the network, determine the bias and scalar properties of interval timing. Significant Statement Timing is a fundamental element of complex behavior, including music and language. Temporal processing in a wide variety of contexts shows two primary features: time estimates exhibit a shift towards the mean (the bias property) and are more variable for longer intervals (the scalar property). We implemented a recurrent neural network that includes long-lasting synaptic currents, which can not only produce interval selective responses but also follow the bias and scalar properties. Interestingly, only physiological values of the time constants for paired-pulse facilitation and GABAb, as well as intermediate background activity within the network can reproduce the two key features of interval timing. Copyright © 2018 the authors.

  18. Delay-Dependent Stability Criterion for Bidirectional Associative Memory Neural Networks with Interval Time-Varying Delays

    Science.gov (United States)

    Park, Ju H.; Kwon, O. M.

    In the letter, the global asymptotic stability of bidirectional associative memory (BAM) neural networks with delays is investigated. The delay is assumed to be time-varying and belongs to a given interval. A novel stability criterion for the stability is presented based on the Lyapunov method. The criterion is represented in terms of linear matrix inequality (LMI), which can be solved easily by various optimization algorithms. Two numerical examples are illustrated to show the effectiveness of our new result.

  19. Automatic temporal segment detection via bilateral long short-term memory recurrent neural networks

    Science.gov (United States)

    Sun, Bo; Cao, Siming; He, Jun; Yu, Lejun; Li, Liandong

    2017-03-01

    Constrained by the physiology, the temporal factors associated with human behavior, irrespective of facial movement or body gesture, are described by four phases: neutral, onset, apex, and offset. Although they may benefit related recognition tasks, it is not easy to accurately detect such temporal segments. An automatic temporal segment detection framework using bilateral long short-term memory recurrent neural networks (BLSTM-RNN) to learn high-level temporal-spatial features, which synthesizes the local and global temporal-spatial information more efficiently, is presented. The framework is evaluated in detail over the face and body database (FABO). The comparison shows that the proposed framework outperforms state-of-the-art methods for solving the problem of temporal segment detection.

  20. Exponential stability of neural networks with asymmetric connection weights

    International Nuclear Information System (INIS)

    Yang Jinxiang; Zhong Shouming

    2007-01-01

    This paper investigates the exponential stability of a class of neural networks with asymmetric connection weights. By dividing the network state variables into various parts according to the characters of the neural networks, some new sufficient conditions of exponential stability are derived via constructing a Lyapunov function and using the method of the variation of constant. The new conditions are associated with the initial values and are described by some blocks of the interconnection matrix, and do not depend on other blocks. Examples are given to further illustrate the theory

  1. A hybrid model based on neural networks for biomedical relation extraction.

    Science.gov (United States)

    Zhang, Yijia; Lin, Hongfei; Yang, Zhihao; Wang, Jian; Zhang, Shaowu; Sun, Yuanyuan; Yang, Liang

    2018-05-01

    Biomedical relation extraction can automatically extract high-quality biomedical relations from biomedical texts, which is a vital step for the mining of biomedical knowledge hidden in the literature. Recurrent neural networks (RNNs) and convolutional neural networks (CNNs) are two major neural network models for biomedical relation extraction. Neural network-based methods for biomedical relation extraction typically focus on the sentence sequence and employ RNNs or CNNs to learn the latent features from sentence sequences separately. However, RNNs and CNNs have their own advantages for biomedical relation extraction. Combining RNNs and CNNs may improve biomedical relation extraction. In this paper, we present a hybrid model for the extraction of biomedical relations that combines RNNs and CNNs. First, the shortest dependency path (SDP) is generated based on the dependency graph of the candidate sentence. To make full use of the SDP, we divide the SDP into a dependency word sequence and a relation sequence. Then, RNNs and CNNs are employed to automatically learn the features from the sentence sequence and the dependency sequences, respectively. Finally, the output features of the RNNs and CNNs are combined to detect and extract biomedical relations. We evaluate our hybrid model using five public (protein-protein interaction) PPI corpora and a (drug-drug interaction) DDI corpus. The experimental results suggest that the advantages of RNNs and CNNs in biomedical relation extraction are complementary. Combining RNNs and CNNs can effectively boost biomedical relation extraction performance. Copyright © 2018 Elsevier Inc. All rights reserved.

  2. Piecewise quadratic Lyapunov functions for stability verification of approximate explicit MPC

    Directory of Open Access Journals (Sweden)

    Morten Hovd

    2010-04-01

    Full Text Available Explicit MPC of constrained linear systems is known to result in a piecewise affine controller and therefore also piecewise affine closed loop dynamics. The complexity of such analytic formulations of the control law can grow exponentially with the prediction horizon. The suboptimal solutions offer a trade-off in terms of complexity and several approaches can be found in the literature for the construction of approximate MPC laws. In the present paper a piecewise quadratic (PWQ Lyapunov function is used for the stability verification of an of approximate explicit Model Predictive Control (MPC. A novel relaxation method is proposed for the LMI criteria on the Lyapunov function design. This relaxation is applicable to the design of PWQ Lyapunov functions for discrete-time piecewise affine systems in general.

  3. Dynamic analysis of high-order Cohen-Grossberg neural networks with time delay

    International Nuclear Information System (INIS)

    Chen Zhang; Zhao Donghua; Ruan Jiong

    2007-01-01

    In this paper, a class of high-order Cohen-Grossberg neural networks with time delay is studied. Several sufficient conditions are obtained for global asymptotic stability and global exponential stability using Lyapunov and LMI method. Finally, two examples are given to illustrate the effectiveness of our method

  4. Critical behavior of the Lyapunov exponent in type-III intermittency

    Energy Technology Data Exchange (ETDEWEB)

    Alvarez-Llamoza, O. [Departamento de Fisica, FACYT, Universidad de Carabobo, Valencia (Venezuela); Centro de Fisica Fundamental, Grupo de Caos y Sistemas Complejos, Universidad de Los Andes, Merida 5251, Merida (Venezuela)], E-mail: llamoza@ula.ve; Cosenza, M.G. [Centro de Fisica Fundamental, Grupo de Caos y Sistemas Complejos, Universidad de Los Andes, Merida 5251, Merida (Venezuela); Ponce, G.A. [Departamento de Fisica, Universidad Nacional Autonoma de Honduras (Honduras); Departamento de Ciencias Naturales, Universidad Pedagogica Nacional Francisco Morazan, Tegucigalpa (Honduras)

    2008-04-15

    The critical behavior of the Lyapunov exponent near the transition to robust chaos via type-III intermittency is determined for a family of one-dimensional singular maps. Critical boundaries separating the region of robust chaos from the region where stable fixed points exist are calculated on the parameter space of the system. A critical exponent {beta} expressing the scaling of the Lyapunov exponent is calculated along the critical curve corresponding to the type-III intermittent transition to chaos. It is found that {beta} varies on the interval 0 {<=} {beta} < 1/2 as a function of the order of the singularity of the map. This contrasts with earlier predictions for the scaling behavior of the Lyapunov exponent in type-III intermittency. The variation of the critical exponent {beta} implies a continuous change in the nature of the transition to chaos via type-III intermittency, from a second-order, continuous transition to a first-order, discontinuous transition.

  5. Critical behavior of the Lyapunov exponent in type-III intermittency

    International Nuclear Information System (INIS)

    Alvarez-Llamoza, O.; Cosenza, M.G.; Ponce, G.A.

    2008-01-01

    The critical behavior of the Lyapunov exponent near the transition to robust chaos via type-III intermittency is determined for a family of one-dimensional singular maps. Critical boundaries separating the region of robust chaos from the region where stable fixed points exist are calculated on the parameter space of the system. A critical exponent β expressing the scaling of the Lyapunov exponent is calculated along the critical curve corresponding to the type-III intermittent transition to chaos. It is found that β varies on the interval 0 ≤ β < 1/2 as a function of the order of the singularity of the map. This contrasts with earlier predictions for the scaling behavior of the Lyapunov exponent in type-III intermittency. The variation of the critical exponent β implies a continuous change in the nature of the transition to chaos via type-III intermittency, from a second-order, continuous transition to a first-order, discontinuous transition

  6. Fast computation with spikes in a recurrent neural network

    International Nuclear Information System (INIS)

    Jin, Dezhe Z.; Seung, H. Sebastian

    2002-01-01

    Neural networks with recurrent connections are sometimes regarded as too slow at computation to serve as models of the brain. Here we analytically study a counterexample, a network consisting of N integrate-and-fire neurons with self excitation, all-to-all inhibition, instantaneous synaptic coupling, and constant external driving inputs. When the inhibition and/or excitation are large enough, the network performs a winner-take-all computation for all possible external inputs and initial states of the network. The computation is done very quickly: As soon as the winner spikes once, the computation is completed since no other neurons will spike. For some initial states, the winner is the first neuron to spike, and the computation is done at the first spike of the network. In general, there are M potential winners, corresponding to the top M external inputs. When the external inputs are close in magnitude, M tends to be larger. If M>1, the selection of the actual winner is strongly influenced by the initial states. If a special relation between the excitation and inhibition is satisfied, the network always selects the neuron with the maximum external input as the winner

  7. Nonlinear identification of process dynamics using neural networks

    International Nuclear Information System (INIS)

    Parlos, A.G.; Atiya, A.F.; Chong, K.T.

    1992-01-01

    In this paper the nonlinear identification of process dynamics encountered in nuclear power plant components is addressed, in an input-output sense, using artificial neural systems. A hybrid feedforward/feedback neural network, namely, a recurrent multilayer perceptron, is used as the model structure to be identified. The feedforward portion of the network architecture provides its well-known interpolation property, while through recurrency and cross-talk, the local information feedback enables representation of temporal variations in the system nonlinearities. The standard backpropagation learning algorithm is modified, and it is used for the supervised training of the proposed hybrid network. The performance of recurrent multilayer perceptron networks in identifying process dynamics is investigated via the case study of a U-tube steam generator. The response of representative steam generator is predicted using a neural network, and it is compared to the response obtained from a sophisticated computer model based on first principles. The transient responses compare well, although further research is warranted to determine the predictive capabilities of these networks during more severe operational transients and accident scenarios

  8. Regeneration cycle and the covariant Lyapunov vectors in a minimal wall turbulence.

    Science.gov (United States)

    Inubushi, Masanobu; Takehiro, Shin-ichi; Yamada, Michio

    2015-08-01

    Considering a wall turbulence as a chaotic dynamical system, we study regeneration cycles in a minimal wall turbulence from the viewpoint of orbital instability by employing the covariant Lyapunov analysis developed by [F. Ginelli et al. Phys. Rev. Lett. 99, 130601 (2007)]. We divide the regeneration cycle into two phases and characterize them with the local Lyapunov exponents and the covariant Lyapunov vectors of the Navier-Stokes turbulence. In particular, we show numerically that phase (i) is dominated by instabilities related to the sinuous mode and the streamwise vorticity, and there is no instability in phase (ii). Furthermore, we discuss a mechanism of the regeneration cycle, making use of an energy budget analysis.

  9. Delay-range-dependent exponential H∞ synchronization of a class of delayed neural networks

    International Nuclear Information System (INIS)

    Karimi, Hamid Reza; Maass, Peter

    2009-01-01

    This article aims to present a multiple delayed state-feedback control design for exponential H ∞ synchronization problem of a class of delayed neural networks with multiple time-varying discrete delays. On the basis of the drive-response concept and by introducing a descriptor technique and using Lyapunov-Krasovskii functional, new delay-range-dependent sufficient conditions for exponential H ∞ synchronization of the drive-response structure of neural networks are driven in terms of linear matrix inequalities (LMIs). The explicit expression of the controller gain matrices are parameterized based on the solvability conditions such that the drive system and the response system can be exponentially synchronized. A numerical example is included to illustrate the applicability of the proposed design method.

  10. Intercept Algorithm for Maneuvering Targets Based on Differential Geometry and Lyapunov Theory

    Directory of Open Access Journals (Sweden)

    Yunes Sh. ALQUDSI

    2018-03-01

    Full Text Available Nowadays, the homing guidance is utilized in the existed and under development air defense systems (ADS to effectively intercept the targets. The targets became smarter and capable to fly and maneuver professionally and the tendency to design missile with a small warhead became greater, then there is a pressure to produce a more precise and accurate missile guidance system based on intelligent algorithms to ensure effective interception of highly maneuverable targets. The aim of this paper is to present an intelligent guidance algorithm that effectively and precisely intercept the maneuverable and smart targets by virtue of the differential geometry (DG concepts. The intercept geometry and engagement kinematics, in addition to the direct intercept condition are developed and expressed in DG terms. The guidance algorithm is then developed by virtue of DG and Lyapunov theory. The study terminates with 2D engagement simulation with illustrative examples, to demonstrate that, the derived DG guidance algorithm is a generalized guidance approach and the well-known proportional navigation (PN guidance law is a subset of this approach.

  11. Interpretation of Recurrent Neural Networks

    DEFF Research Database (Denmark)

    Pedersen, Morten With; Larsen, Jan

    1997-01-01

    This paper addresses techniques for interpretation and characterization of trained recurrent nets for time series problems. In particular, we focus on assessment of effective memory and suggest an operational definition of memory. Further we discuss the evaluation of learning curves. Various nume...

  12. Exponential convergence for a class of delayed cellular neural networks with time-varying coefficients

    International Nuclear Information System (INIS)

    Liu Bingwen

    2008-01-01

    In this Letter, we consider a class of delayed cellular neural networks with time-varying coefficients. By applying Lyapunov functional method and differential inequality techniques, we establish new results to ensure that all solutions of the networks converge exponentially to zero point

  13. Concise Neural Nonaffine Control of Air-Breathing Hypersonic Vehicles Subject to Parametric Uncertainties

    Directory of Open Access Journals (Sweden)

    Xiangwei Bu

    2017-01-01

    Full Text Available In this paper, a novel simplified neural control strategy is proposed for the longitudinal dynamics of an air-breathing hypersonic vehicle (AHV directly using nonaffine models instead of affine ones. For the velocity dynamics, an adaptive neural controller is devised based on a minimal-learning parameter (MLP technique for the sake of decreasing computational loads. The altitude dynamics is rewritten as a pure feedback nonaffine formulation, for which a novel concise neural control approach is achieved without backstepping. The special contributions are that the control architecture is concise and the computational cost is low. Moreover, the exploited controller possesses good practicability since there is no need for affine models. The semiglobally uniformly ultimate boundedness of all the closed-loop system signals is guaranteed via Lyapunov stability theory. Finally, simulation results are presented to validate the effectiveness of the investigated control methodology in the presence of parametric uncertainties.

  14. Universality in chaos: Lyapunov spectrum and random matrix theory.

    Science.gov (United States)

    Hanada, Masanori; Shimada, Hidehiko; Tezuka, Masaki

    2018-02-01

    We propose the existence of a new universality in classical chaotic systems when the number of degrees of freedom is large: the statistical property of the Lyapunov spectrum is described by random matrix theory. We demonstrate it by studying the finite-time Lyapunov exponents of the matrix model of a stringy black hole and the mass-deformed models. The massless limit, which has a dual string theory interpretation, is special in that the universal behavior can be seen already at t=0, while in other cases it sets in at late time. The same pattern is demonstrated also in the product of random matrices.

  15. Generalized decompositions of dynamic systems and vector Lyapunov functions

    Science.gov (United States)

    Ikeda, M.; Siljak, D. D.

    1981-10-01

    The notion of decomposition is generalized to provide more freedom in constructing vector Lyapunov functions for stability analysis of nonlinear dynamic systems. A generalized decomposition is defined as a disjoint decomposition of a system which is obtained by expanding the state-space of a given system. An inclusion principle is formulated for the solutions of the expansion to include the solutions of the original system, so that stability of the expansion implies stability of the original system. Stability of the expansion can then be established by standard disjoint decompositions and vector Lyapunov functions. The applicability of the new approach is demonstrated using the Lotka-Volterra equations.

  16. Universality in chaos: Lyapunov spectrum and random matrix theory

    Science.gov (United States)

    Hanada, Masanori; Shimada, Hidehiko; Tezuka, Masaki

    2018-02-01

    We propose the existence of a new universality in classical chaotic systems when the number of degrees of freedom is large: the statistical property of the Lyapunov spectrum is described by random matrix theory. We demonstrate it by studying the finite-time Lyapunov exponents of the matrix model of a stringy black hole and the mass-deformed models. The massless limit, which has a dual string theory interpretation, is special in that the universal behavior can be seen already at t =0 , while in other cases it sets in at late time. The same pattern is demonstrated also in the product of random matrices.

  17. Generalized Projective Synchronization between Two Different Neural Networks with Mixed Time Delays

    Directory of Open Access Journals (Sweden)

    Xuefei Wu

    2012-01-01

    Full Text Available The generalized projective synchronization (GPS between two different neural networks with nonlinear coupling and mixed time delays is considered. Several kinds of nonlinear feedback controllers are designed to achieve GPS between two different such neural networks. Some results for GPS of these neural networks are proved theoretically by using the Lyapunov stability theory and the LaSalle invariance principle. Moreover, by comparison, we determine an optimal nonlinear controller from several ones and provide an adaptive update law for it. Computer simulations are provided to show the effectiveness and feasibility of the proposed methods.

  18. Image-Based Visual Servoing for Robotic Systems: A Nonlinear Lyapunov-Based Control Approach

    International Nuclear Information System (INIS)

    Dixon, Warren

    2003-01-01

    The objective of this project is to enable current and future EM robots with an increased ability to perceive and interact with unstructured and unknown environments through the use of camera-based visual servo controllers. The scientific goals of this research are to develop a new visual servo control methodology that: (1) adapts for the unknown camera calibration parameters (e.g., focal length, scaling factors, camera position, and orientation) and the physical parameters of the robotic system (e.g., mass, inertia, friction), (2) compensates for unknown depth information (extract 3D information from the 2D image), and (3) enables multi-uncalibrated cameras to be used as a means to provide a larger field-of-view. Nonlinear Lyapunov-based techniques in conjunction with results from projective geometry are being used to overcome the complex control issues and alleviate many of the restrictive assumptions that impact current visual servo controlled robotic systems. The potential relevance of this control methodology will be a plug-and-play visual servoing control module that can be utilized in conjunction with current technology such as feature extraction and recognition, to enable current EM robotic systems with the capabilities of increased accuracy, autonomy, and robustness, with a larger field of view (and hence a larger workspace). These capabilities will enable EM robots to significantly accelerate D and D operations by providing for improved robot autonomy and increased worker productivity, while also reducing the associated costs, removing the human operator from the hazardous environments, and reducing the burden and skill of the human operators

  19. Image-Based Visual Servoing for Robotic Systems: A Nonlinear Lyapunov-Based Control Approach

    International Nuclear Information System (INIS)

    Dixon, Warren

    2002-01-01

    The objective of this project is to enable current and future EM robots with an increased ability to perceive and interact with unstructured and unknown environments through the use of camera-based visual servo controlled robots. The scientific goals of this research are to develop a new visual servo control methodology that: (1) adapts for the unknown camera calibration parameters (e.g., focal length, scaling factors, camera position and orientation) and the physical parameters of the robotic system (e.g., mass, inertia, friction), (2) compensates for unknown depth information (extract 3D information from the 2D image), and (3) enables multi-uncalibrated cameras to be used as a means to provide a larger field-of-view. Nonlinear Lyapunov-based techniques are being used to overcome the complex control issues and alleviate many of the restrictive assumptions that impact current visual servo controlled robotic systems. The potential relevance of this control methodology will be a plug-and-play visual servoing control module that can be utilized in conjunction with current technology such as feature extraction and recognition, to enable current EM robotic systems with the capabilities of increased accuracy, autonomy, and robustness, with a larger field of view (and hence a larger workspace). These capabilities will enable EM robots to significantly accelerate D and D operations by providing for improved robot autonomy and increased worker productivity, while also reducing the associated costs, removing the human operator from the hazardous environments, and reducing the burden and skill of the human operators

  20. Robust stability of uncertain Markovian jumping Cohen-Grossberg neural networks with mixed time-varying delays

    International Nuclear Information System (INIS)

    Sheng Li; Yang Huizhong

    2009-01-01

    This paper considers the robust stability of a class of uncertain Markovian jumping Cohen-Grossberg neural networks (UMJCGNNs) with mixed time-varying delays. The parameter uncertainties are norm-bounded and the mixed time-varying delays comprise discrete and distributed time delays. Based on the Lyapunov stability theory and linear matrix inequality (LMI) technique, some robust stability conditions guaranteeing the global robust convergence of the equilibrium point are derived. An example is given to show the effectiveness of the proposed results.

  1. Neural network based adaptive control for nonlinear dynamic regimes

    Science.gov (United States)

    Shin, Yoonghyun

    Adaptive control designs using neural networks (NNs) based on dynamic inversion are investigated for aerospace vehicles which are operated at highly nonlinear dynamic regimes. NNs play a key role as the principal element of adaptation to approximately cancel the effect of inversion error, which subsequently improves robustness to parametric uncertainty and unmodeled dynamics in nonlinear regimes. An adaptive control scheme previously named 'composite model reference adaptive control' is further developed so that it can be applied to multi-input multi-output output feedback dynamic inversion. It can have adaptive elements in both the dynamic compensator (linear controller) part and/or in the conventional adaptive controller part, also utilizing state estimation information for NN adaptation. This methodology has more flexibility and thus hopefully greater potential than conventional adaptive designs for adaptive flight control in highly nonlinear flight regimes. The stability of the control system is proved through Lyapunov theorems, and validated with simulations. The control designs in this thesis also include the use of 'pseudo-control hedging' techniques which are introduced to prevent the NNs from attempting to adapt to various actuation nonlinearities such as actuator position and rate saturations. Control allocation is introduced for the case of redundant control effectors including thrust vectoring nozzles. A thorough comparison study of conventional and NN-based adaptive designs for a system under a limit cycle, wing-rock, is included in this research, and the NN-based adaptive control designs demonstrate their performances for two highly maneuverable aerial vehicles, NASA F-15 ACTIVE and FQM-117B unmanned aerial vehicle (UAV), operated under various nonlinearities and uncertainties.

  2. Lyapunov exponent of the random frequency oscillator: cumulant expansion approach

    International Nuclear Information System (INIS)

    Anteneodo, C; Vallejos, R O

    2010-01-01

    We consider a one-dimensional harmonic oscillator with a random frequency, focusing on both the standard and the generalized Lyapunov exponents, λ and λ* respectively. We discuss the numerical difficulties that arise in the numerical calculation of λ* in the case of strong intermittency. When the frequency corresponds to a Ornstein-Uhlenbeck process, we compute analytically λ* by using a cumulant expansion including up to the fourth order. Connections with the problem of finding an analytical estimate for the largest Lyapunov exponent of a many-body system with smooth interactions are discussed.

  3. Application of recurrent neural networks for drought projections in California

    Science.gov (United States)

    Le, J. A.; El-Askary, H. M.; Allali, M.; Struppa, D. C.

    2017-05-01

    We use recurrent neural networks (RNNs) to investigate the complex interactions between the long-term trend in dryness and a projected, short but intense, period of wetness due to the 2015-2016 El Niño. Although it was forecasted that this El Niño season would bring significant rainfall to the region, our long-term projections of the Palmer Z Index (PZI) showed a continuing drought trend, contrasting with the 1998-1999 El Niño event. RNN training considered PZI data during 1896-2006 that was validated against the 2006-2015 period to evaluate the potential of extreme precipitation forecast. We achieved a statistically significant correlation of 0.610 between forecasted and observed PZI on the validation set for a lead time of 1 month. This gives strong confidence to the forecasted precipitation indicator. The 2015-2016 El Niño season proved to be relatively weak as compared with the 1997-1998, with a peak PZI anomaly of 0.242 standard deviations below historical averages, continuing drought conditions.

  4. Global exponential stability for nonautonomous cellular neural networks with delays

    International Nuclear Information System (INIS)

    Zhang Qiang; Wei Xiaopeng; Xu Jin

    2006-01-01

    In this Letter, by utilizing Lyapunov functional method and Halanay inequalities, we analyze global exponential stability of nonautonomous cellular neural networks with delay. Several new sufficient conditions ensuring global exponential stability of the network are obtained. The results given here extend and improve the earlier publications. An example is given to demonstrate the effectiveness of the obtained results

  5. Análisis de estabilidad de controladores borrosos tipo Mamdani mediante el cálculo del exponente de Lyapunov

    Directory of Open Access Journals (Sweden)

    Leonardo-Alonso Martínez Rivera

    2015-10-01

    overcome this problem, we propose to employ the Lyapunov Exponent in order to determine whether fuzzy controllers are stable. The Lyapunov exponent is calculated through a numerical method on the time series obtained experimentally by having the fuzzy controller in closed loop with the plant dynamics. In this paper, the plant is the inverted pendulum, which is a benchmark plant to test complex control laws. Sixteen experiments were carried by modifying the rule base structure of Mamdani fuzzy controllers, which were also tested under normal and disturbed conditions. In all the cases, the Lyapunov Exponent is negative, indicating that the analyzed Mamdani controllers are indeed dissipative systems. Future applications on adaptive control are presented because the Lyapunov serves as a quantitative metric to determine controllers’ performance. Palabras clave: Estabilidad, Exponente de Lyapunov, Control Borroso, Bases de Reglas Mamdani, Series de Tiempo, Keywords: Stability Analysis, Lyapunov Exponent, Fuzzy Controllers, Mamdani Rule Base.

  6. Lyapunov exponent as a metric for assessing the dynamic content and predictability of large-eddy simulations

    Science.gov (United States)

    Nastac, Gabriel; Labahn, Jeffrey W.; Magri, Luca; Ihme, Matthias

    2017-09-01

    Metrics used to assess the quality of large-eddy simulations commonly rely on a statistical assessment of the solution. While these metrics are valuable, a dynamic measure is desirable to further characterize the ability of a numerical simulation for capturing dynamic processes inherent in turbulent flows. To address this issue, a dynamic metric based on the Lyapunov exponent is proposed which assesses the growth rate of the solution separation. This metric is applied to two turbulent flow configurations: forced homogeneous isotropic turbulence and a turbulent jet diffusion flame. First, it is shown that, despite the direct numerical simulation (DNS) and large-eddy simulation (LES) being high-dimensional dynamical systems with O (107) degrees of freedom, the separation growth rate qualitatively behaves like a lower-dimensional dynamical system, in which the dimension of the Lyapunov system is substantially smaller than the discretized dynamical system. Second, a grid refinement analysis of each configuration demonstrates that as the LES filter width approaches the smallest scales of the system the Lyapunov exponent asymptotically approaches a plateau. Third, a small perturbation is superimposed onto the initial conditions of each configuration, and the Lyapunov exponent is used to estimate the time required for divergence, thereby providing a direct assessment of the predictability time of simulations. By comparing inert and reacting flows, it is shown that combustion increases the predictability of the turbulent simulation as a result of the dilatation and increased viscosity by heat release. The predictability time is found to scale with the integral time scale in both the reacting and inert jet flows. Fourth, an analysis of the local Lyapunov exponent is performed to demonstrate that this metric can also determine flow-dependent properties, such as regions that are sensitive to small perturbations or conditions of large turbulence within the flow field. Finally

  7. Optimal multiple-information integration inherent in a ring neural network

    International Nuclear Information System (INIS)

    Takiyama, Ken

    2017-01-01

    Although several behavioral experiments have suggested that our neural system integrates multiple sources of information based on the certainty of each type of information in the manner of maximum-likelihood estimation, it is unclear how the maximum-likelihood estimation is implemented in our neural system. Here, I investigate the relationship between maximum-likelihood estimation and a widely used ring-type neural network model that is used as a model of visual, motor, or prefrontal cortices. Without any approximation or ansatz, I analytically demonstrate that the equilibrium of an order parameter in the neural network model exactly corresponds to the maximum-likelihood estimation when the strength of the symmetrical recurrent synaptic connectivity within a neural population is appropriately stronger than that of asymmetrical connectivity, that of local and external inputs, and that of symmetrical or asymmetrical connectivity between different neural populations. In this case, strengths of local and external inputs or those of symmetrical connectivity between different neural populations exactly correspond to the input certainty in maximum-likelihood estimation. Thus, my analysis suggests appropriately strong symmetrical recurrent connectivity as a possible candidate for implementing the maximum-likelihood estimation within our neural system. (paper)

  8. S4-1: Motion Detection Based on Recurrent Network Dynamics

    Directory of Open Access Journals (Sweden)

    Bart Krekelberg

    2012-10-01

    Full Text Available The detection of a sequence of events requires memory. The detection of visual motion is a well-studied example; there the memory allows the comparison of current with earlier visual input. This comparison results in an estimate of direction and speed of motion. The dominant model of motion detection in primates—the motion energy model—assumes that this memory resides in subclasses of cells with slower temporal dynamics. It is not clear, however, how such slow dynamics could arise. We used extracellularly recorded responses of neurons in the macaque middle temporal area to train an artificial neural network with recurrent connectivity. The trained network successfully reproduced the population response, and had many properties also found in the visual cortex (e.g., Gabor-like receptive fields, a hierarchy of simple and complex cells, motion opponency. When probed with reverse-correlation methods, the network's response was very similar to that of a feed-forward motion energy model, even though recurrent feedback is an essential part of its architecture. These findings show that a strongly recurrent network can masquerade as a feed-forward network. Moreover, they suggest a conceptually novel role for recurrent network connectivity: the creation of flexible temporal delays to implement short term memory and compute velocity.

  9. International Congress NONLINEAR DYNAMICAL ANALYSIS 2007 dedicated to the 150th Anniversary of Academician A. M. Lyapunov

    Science.gov (United States)

    2010-05-14

    Mikhailovich Lyapunov is discussed. Main attention is focused on the first Lyapunov method. LYAPUNOV BUNDLES IN CYCLIC FEEDBACK SYSTEMS WITH DELAYS George ...Lyapunov frequently discussed this problem with Henry Poincare (1854-1912) and George Darwin (1845 - 1912). They both considered the "pear-form" figure as... Cantor -type set. Neither can the existence of such systems be excluded. The results we present are discussed in a joint paper with K. Bjerkloev. МЕТОДЫ А.М

  10. Lyapunov functionals and stability of stochastic difference equations

    CERN Document Server

    Shaikhet, Leonid

    2011-01-01

    This book offers a general method of Lyapunov functional construction which lets researchers analyze the degree to which the stability properties of differential equations are preserved in their difference analogues. Includes examples from physical systems.

  11. Mixed H ∞ and Passive Projective Synchronization for Fractional Order Memristor-Based Neural Networks with Time-Delay and Parameter Uncertainty

    International Nuclear Information System (INIS)

    Song Xiao-Na; Song Shuai; Liu Lei-Po; Tejado Balsera, Inés

    2017-01-01

    This paper investigates the mixed H ∞ and passive projective synchronization problem for fractional-order (FO) memristor-based neural networks. Our aim is to design a controller such that, though the unavoidable phenomena of time-delay and parameter uncertainty are fully considered, the resulting closed-loop system is asymptotically stable with a mixed H ∞ and passive performance level. By combining active and adaptive control methods, a novel hybrid control strategy is designed, which can guarantee the robust stability of the closed-loop system and also ensure a mixed H ∞ and passive performance level. Via the application of FO Lyapunov stability theory, the projective synchronization conditions are addressed in terms of linear matrix inequality techniques. Finally, two simulation examples are given to illustrate the effectiveness of the proposed method. (paper)

  12. Synchronization of an Inertial Neural Network With Time-Varying Delays and Its Application to Secure Communication.

    Science.gov (United States)

    Lakshmanan, Shanmugam; Prakash, Mani; Lim, Chee Peng; Rakkiyappan, Rajan; Balasubramaniam, Pagavathigounder; Nahavandi, Saeid

    2018-01-01

    In this paper, synchronization of an inertial neural network with time-varying delays is investigated. Based on the variable transformation method, we transform the second-order differential equations into the first-order differential equations. Then, using suitable Lyapunov-Krasovskii functionals and Jensen's inequality, the synchronization criteria are established in terms of linear matrix inequalities. Moreover, a feedback controller is designed to attain synchronization between the master and slave models, and to ensure that the error model is globally asymptotically stable. Numerical examples and simulations are presented to indicate the effectiveness of the proposed method. Besides that, an image encryption algorithm is proposed based on the piecewise linear chaotic map and the chaotic inertial neural network. The chaotic signals obtained from the inertial neural network are utilized for the encryption process. Statistical analyses are provided to evaluate the effectiveness of the proposed encryption algorithm. The results ascertain that the proposed encryption algorithm is efficient and reliable for secure communication applications.

  13. Modeling of the pyruvate production with Escherichia coli: comparison of mechanistic and neural networks-based models.

    Science.gov (United States)

    Zelić, B; Bolf, N; Vasić-Racki, D

    2006-06-01

    Three different models: the unstructured mechanistic black-box model, the input-output neural network-based model and the externally recurrent neural network model were used to describe the pyruvate production process from glucose and acetate using the genetically modified Escherichia coli YYC202 ldhA::Kan strain. The experimental data were used from the recently described batch and fed-batch experiments [ Zelić B, Study of the process development for Escherichia coli-based pyruvate production. PhD Thesis, University of Zagreb, Faculty of Chemical Engineering and Technology, Zagreb, Croatia, July 2003. (In English); Zelić et al. Bioproc Biosyst Eng 26:249-258 (2004); Zelić et al. Eng Life Sci 3:299-305 (2003); Zelić et al Biotechnol Bioeng 85:638-646 (2004)]. The neural networks were built out of the experimental data obtained in the fed-batch pyruvate production experiments with the constant glucose feed rate. The model validation was performed using the experimental results obtained from the batch and fed-batch pyruvate production experiments with the constant acetate feed rate. Dynamics of the substrate and product concentration changes was estimated using two neural network-based models for biomass and pyruvate. It was shown that neural networks could be used for the modeling of complex microbial fermentation processes, even in conditions in which mechanistic unstructured models cannot be applied.

  14. On some properties of the discrete Lyapunov exponent

    International Nuclear Information System (INIS)

    Amigo, Jose M.; Kocarev, Ljupco; Szczepanski, Janusz

    2008-01-01

    One of the possible by-products of discrete chaos is the application of its tools, in particular of the discrete Lyapunov exponent, to cryptography. In this Letter we explore this question in a very general setting

  15. Modeling Broadband Microwave Structures by Artificial Neural Networks

    Directory of Open Access Journals (Sweden)

    V. Otevrel

    2004-06-01

    Full Text Available The paper describes the exploitation of feed-forward neural networksand recurrent neural networks for replacing full-wave numerical modelsof microwave structures in complex microwave design tools. Building aneural model, attention is turned to the modeling accuracy and to theefficiency of building a model. Dealing with the accuracy, we describea method of increasing it by successive completing a training set.Neural models are mutually compared in order to highlight theiradvantages and disadvantages. As a reference model for comparisons,approximations based on standard cubic splines are used. Neural modelsare used to replace both the time-domain numeric models and thefrequency-domain ones.

  16. Estabilización del Péndulo Invertido Sobre Dos Ruedas mediante el método de Lyapunov

    Directory of Open Access Journals (Sweden)

    O. Octavio Gutiérrez Frías

    2013-01-01

    Full Text Available Resumen: En este trabajo, se presenta un controlador no lineal para estabilizar el sistema Péndulo Invertido Sobre Dos Ruedas. Como primera etapa la estrategia de control, se basa en una linealización parcial por realimentación, para posteriormente proponer una función candidata de Lyapunov en combinación con el principio de invariancia de LaSalle con el fin de obtener el controlador esta- bilizador. El sistema en lazo cerrado obtenido es asintóticamente estable localmente alrededor del punto de equilibrio inestable, con un dominio de atracción calculable. Abstract: In this paper, a nonlinear controller is presented for the stabilization of the two wheels inverted pendulum. The control strategy is based on partial feedback linealization, in first stage and then a suitable function Lyapunov in conjunction with LaSalle's invariance principle is formed to obtain a stabilizing feedback controller. The obtained closed-loop system is locally asymptotically stable around its unstable equilibrium point, with a computable domain of attraction. Palabras clave: Sistema Subactuado, Péndulo Invertido Sobre Dos Ruedas, Método de Lyapunov, Control No Lineal, Keywords: Under Actuated System, Two Wheels Inverted Pendulum, Lyapunov Approach, Non-Linear Control

  17. Design of Connectivity Preserving Flocking Using Control Lyapunov Function

    Directory of Open Access Journals (Sweden)

    Bayu Erfianto

    2016-01-01

    Full Text Available This paper investigates cooperative flocking control design with connectivity preserving mechanism. During flocking, interagent distance is measured to determine communication topology of the flocks. Then, cooperative flocking motion is built based on cooperative artificial potential field with connectivity preserving mechanism to achieve the common flocking objective. The flocking control input is then obtained by deriving cooperative artificial potential field using control Lyapunov function. As a result, we prove that our flocking protocol establishes group stabilization and the communication topology of multiagent flocking is always connected.

  18. Satisfiability of logic programming based on radial basis function neural networks

    International Nuclear Information System (INIS)

    Hamadneh, Nawaf; Sathasivam, Saratha; Tilahun, Surafel Luleseged; Choon, Ong Hong

    2014-01-01

    In this paper, we propose a new technique to test the Satisfiability of propositional logic programming and quantified Boolean formula problem in radial basis function neural networks. For this purpose, we built radial basis function neural networks to represent the proportional logic which has exactly three variables in each clause. We used the Prey-predator algorithm to calculate the output weights of the neural networks, while the K-means clustering algorithm is used to determine the hidden parameters (the centers and the widths). Mean of the sum squared error function is used to measure the activity of the two algorithms. We applied the developed technique with the recurrent radial basis function neural networks to represent the quantified Boolean formulas. The new technique can be applied to solve many applications such as electronic circuits and NP-complete problems

  19. Satisfiability of logic programming based on radial basis function neural networks

    Energy Technology Data Exchange (ETDEWEB)

    Hamadneh, Nawaf; Sathasivam, Saratha; Tilahun, Surafel Luleseged; Choon, Ong Hong [School of Mathematical Sciences, Universiti Sains Malaysia, 11800 USM, Penang (Malaysia)

    2014-07-10

    In this paper, we propose a new technique to test the Satisfiability of propositional logic programming and quantified Boolean formula problem in radial basis function neural networks. For this purpose, we built radial basis function neural networks to represent the proportional logic which has exactly three variables in each clause. We used the Prey-predator algorithm to calculate the output weights of the neural networks, while the K-means clustering algorithm is used to determine the hidden parameters (the centers and the widths). Mean of the sum squared error function is used to measure the activity of the two algorithms. We applied the developed technique with the recurrent radial basis function neural networks to represent the quantified Boolean formulas. The new technique can be applied to solve many applications such as electronic circuits and NP-complete problems.

  20. LFNet: A Novel Bidirectional Recurrent Convolutional Neural Network for Light-Field Image Super-Resolution.

    Science.gov (United States)

    Wang, Yunlong; Liu, Fei; Zhang, Kunbo; Hou, Guangqi; Sun, Zhenan; Tan, Tieniu

    2018-09-01

    The low spatial resolution of light-field image poses significant difficulties in exploiting its advantage. To mitigate the dependency of accurate depth or disparity information as priors for light-field image super-resolution, we propose an implicitly multi-scale fusion scheme to accumulate contextual information from multiple scales for super-resolution reconstruction. The implicitly multi-scale fusion scheme is then incorporated into bidirectional recurrent convolutional neural network, which aims to iteratively model spatial relations between horizontally or vertically adjacent sub-aperture images of light-field data. Within the network, the recurrent convolutions are modified to be more effective and flexible in modeling the spatial correlations between neighboring views. A horizontal sub-network and a vertical sub-network of the same network structure are ensembled for final outputs via stacked generalization. Experimental results on synthetic and real-world data sets demonstrate that the proposed method outperforms other state-of-the-art methods by a large margin in peak signal-to-noise ratio and gray-scale structural similarity indexes, which also achieves superior quality for human visual systems. Furthermore, the proposed method can enhance the performance of light field applications such as depth estimation.