WorldWideScience

Sample records for continuous time markov

  1. The Limit Behaviour of Imprecise Continuous-Time Markov Chains

    Science.gov (United States)

    De Bock, Jasper

    2016-08-01

    We study the limit behaviour of a nonlinear differential equation whose solution is a superadditive generalisation of a stochastic matrix, prove convergence, and provide necessary and sufficient conditions for ergodicity. In the linear case, the solution of our differential equation is equal to the matrix exponential of an intensity matrix and can then be interpreted as the transition operator of a homogeneous continuous-time Markov chain. Similarly, in the generalised nonlinear case that we consider, the solution can be interpreted as the lower transition operator of a specific set of non-homogeneous continuous-time Markov chains, called an imprecise continuous-time Markov chain. In this context, our convergence result shows that for a fixed initial state, an imprecise continuous-time Markov chain always converges to a limiting distribution, and our ergodicity result provides a necessary and sufficient condition for this limiting distribution to be independent of the initial state.

  2. Continuous-time Markov decision processes theory and applications

    CERN Document Server

    Guo, Xianping

    2009-01-01

    This volume provides the first book entirely devoted to recent developments on the theory and applications of continuous-time Markov decision processes (MDPs). The MDPs presented here include most of the cases that arise in applications.

  3. Model checking conditional CSL for continuous-time Markov chains

    DEFF Research Database (Denmark)

    Gao, Yang; Xu, Ming; Zhan, Naijun;

    2013-01-01

    In this paper, we consider the model-checking problem of continuous-time Markov chains (CTMCs) with respect to conditional logic. To the end, we extend Continuous Stochastic Logic introduced in Aziz et al. (2000) [1] to Conditional Continuous Stochastic Logic (CCSL) by introducing a conditional...

  4. Ergodic degrees for continuous-time Markov chains

    Institute of Scientific and Technical Information of China (English)

    MAO; Yonghua

    2004-01-01

    This paper studies the existence of the higher orders deviation matrices for continuous time Markov chains by the moments for the hitting times. An estimate of the polynomial convergence rates for the transition matrix to the stationary measure is obtained. Finally, the explicit formulas for birth-death processes are presented.

  5. Efficient maximum likelihood parameterization of continuous-time Markov processes

    CERN Document Server

    McGibbon, Robert T

    2015-01-01

    Continuous-time Markov processes over finite state-spaces are widely used to model dynamical processes in many fields of natural and social science. Here, we introduce an maximum likelihood estimator for constructing such models from data observed at a finite time interval. This estimator is drastically more efficient than prior approaches, enables the calculation of deterministic confidence intervals in all model parameters, and can easily enforce important physical constraints on the models such as detailed balance. We demonstrate and discuss the advantages of these models over existing discrete-time Markov models for the analysis of molecular dynamics simulations.

  6. Parallel algorithms for simulating continuous time Markov chains

    Science.gov (United States)

    Nicol, David M.; Heidelberger, Philip

    1992-01-01

    We have previously shown that the mathematical technique of uniformization can serve as the basis of synchronization for the parallel simulation of continuous-time Markov chains. This paper reviews the basic method and compares five different methods based on uniformization, evaluating their strengths and weaknesses as a function of problem characteristics. The methods vary in their use of optimism, logical aggregation, communication management, and adaptivity. Performance evaluation is conducted on the Intel Touchstone Delta multiprocessor, using up to 256 processors.

  7. Efficient computation of time-bounded reachability probabilities in uniform continuous-time Markov decision processes

    NARCIS (Netherlands)

    Baier, Christel; Hermanns, H.; Katoen, Joost P.; Haverkort, Boudewijn R.H.M.

    2005-01-01

    A continuous-time Markov decision process (CTMDP) is a generalization of a continuous-time Markov chain in which both probabilistic and nondeterministic choices co-exist. This paper presents an efficient algorithm to compute the maximum (or minimum) probability to reach a set of goal states within a

  8. Nonequilibrium thermodynamic potentials for continuous-time Markov chains.

    Science.gov (United States)

    Verley, Gatien

    2016-01-01

    We connect the rare fluctuations of an equilibrium (EQ) process and the typical fluctuations of a nonequilibrium (NE) stationary process. In the framework of large deviation theory, this observation allows us to introduce NE thermodynamic potentials. For continuous-time Markov chains, we identify the relevant pairs of conjugated variables and propose two NE ensembles: one with fixed dynamics and fluctuating time-averaged variables, and another with fixed time-averaged variables, but a fluctuating dynamics. Accordingly, we show that NE processes are equivalent to conditioned EQ processes ensuring that NE potentials are Legendre dual. We find a variational principle satisfied by the NE potentials that reach their maximum in the NE stationary state and whose first derivatives produce the NE equations of state and second derivatives produce the NE Maxwell relations generalizing the Onsager reciprocity relations.

  9. Asymptotic Expansions of Backward Equations for Two-time-scale Markov Chains in Continuous Time

    Institute of Scientific and Technical Information of China (English)

    G Yin; Dung Tien Nguyen

    2009-01-01

    This work develops asymptotic expansions for solutions of systems of backward equations of timeinhomogeneons Markov chains in continuous time. Owing to the rapid progress in technology and the increasing complexity in modeling, the underlying Markov chains often have large state spaces, which make the computational tasks infeasible. To reduce the complexity, two-time-scale formulations are used. By introducing a small parameter ε> 0 and using suitable decomposition and aggregation procedures, it is formulated as a singular perturbation problem. Both Markov chains having recurrent states only and Markov chains including also transient states are treated. Under certain weak irreducibility and smoothness conditions of the generators, the desired asymptotic expansions are constructed. Then error bounds are obtained.

  10. Marked Continuous-Time Markov Chain Modelling of Burst Behaviour for Single Ion Channels

    Directory of Open Access Journals (Sweden)

    Frank G. Ball

    2007-01-01

    a continuous-time Markov chain with a finite-state space. We show how the use of marked continuous-time Markov chains can simplify the derivation of (i the distributions of several burst properties, including the total open time, the total charge transfer, and the number of openings in a burst, and (ii the form of these distributions when the underlying gating process is time reversible and in equilibrium.

  11. Physical time scale in kinetic Monte Carlo simulations of continuous-time Markov chains.

    Science.gov (United States)

    Serebrinsky, Santiago A

    2011-03-01

    We rigorously establish a physical time scale for a general class of kinetic Monte Carlo algorithms for the simulation of continuous-time Markov chains. This class of algorithms encompasses rejection-free (or BKL) and rejection (or "standard") algorithms. For rejection algorithms, it was formerly considered that the availability of a physical time scale (instead of Monte Carlo steps) was empirical, at best. Use of Monte Carlo steps as a time unit now becomes completely unnecessary.

  12. Explosion, implosion, and moments of passage times for continuous-time Markov chains: a semimartingale approach

    CERN Document Server

    Menshikov, Mikhail

    2012-01-01

    We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of occurrence of the phenomenon of explosion are also obtained. A new phenomenon of implosion is introduced and sharp conditions for its occurrence are proven. The general results are illustrated by treating models having a difficult behaviour even in discrete time.

  13. An Expectation Maximization Algorithm to Model Failure Times by Continuous-Time Markov Chains

    Directory of Open Access Journals (Sweden)

    Qihong Duan

    2010-01-01

    Full Text Available In many applications, the failure rate function may present a bathtub shape curve. In this paper, an expectation maximization algorithm is proposed to construct a suitable continuous-time Markov chain which models the failure time data by the first time reaching the absorbing state. Assume that a system is described by methods of supplementary variables, the device of stage, and so on. Given a data set, the maximum likelihood estimators of the initial distribution and the infinitesimal transition rates of the Markov chain can be obtained by our novel algorithm. Suppose that there are m transient states in the system and that there are n failure time data. The devised algorithm only needs to compute the exponential of m×m upper triangular matrices for O(nm2 times in each iteration. Finally, the algorithm is applied to two real data sets, which indicates the practicality and efficiency of our algorithm.

  14. Adaptive Continuous time Markov Chain Approximation Model to\\ud General Jump-Diffusions

    OpenAIRE

    Cerrato, Mario; Lo, Chia Chun; Skindilias, Konstantinos

    2011-01-01

    We propose a non-equidistant Q rate matrix formula and an adaptive numerical algorithm for a continuous time Markov chain to approximate jump-diffusions with affine or non-affine functional specifications. Our approach also accommodates state-dependent jump intensity and jump distribution, a flexibility that is very hard to achieve with other numerical methods. The Kologorov-Smirnov test shows that the proposed Markov chain transition density converges to the one given by the likelihood expan...

  15. Statistical Analysis of Notational AFL Data Using Continuous Time Markov Chains.

    Science.gov (United States)

    Meyer, Denny; Forbes, Don; Clarke, Stephen R

    2006-01-01

    Animal biologists commonly use continuous time Markov chain models to describe patterns of animal behaviour. In this paper we consider the use of these models for describing AFL football. In particular we test the assumptions for continuous time Markov chain models (CTMCs), with time, distance and speed values associated with each transition. Using a simple event categorisation it is found that a semi-Markov chain model is appropriate for this data. This validates the use of Markov Chains for future studies in which the outcomes of AFL matches are simulated. Key PointsA comparison of four AFL matches suggests similarity in terms of transition probabilities for events and the mean times, distances and speeds associated with each transition.The Markov assumption appears to be valid.However, the speed, time and distance distributions associated with each transition are not exponential suggesting that semi-Markov model can be used to model and simulate play.Team identified events and directions associated with transitions are required to develop the model into a tool for the prediction of match outcomes.

  16. Subgeometric Ergodicity Analysis of Continuous-Time Markov Chains under Random-Time State-Dependent Lyapunov Drift Conditions

    Directory of Open Access Journals (Sweden)

    Mokaedi V. Lekgari

    2014-01-01

    Full Text Available We investigate random-time state-dependent Foster-Lyapunov analysis on subgeometric rate ergodicity of continuous-time Markov chains (CTMCs. We are mainly concerned with making use of the available results on deterministic state-dependent drift conditions for CTMCs and on random-time state-dependent drift conditions for discrete-time Markov chains and transferring them to CTMCs.

  17. Generalization bounds of ERM-based learning processes for continuous-time Markov chains.

    Science.gov (United States)

    Zhang, Chao; Tao, Dacheng

    2012-12-01

    Many existing results on statistical learning theory are based on the assumption that samples are independently and identically distributed (i.i.d.). However, the assumption of i.i.d. samples is not suitable for practical application to problems in which samples are time dependent. In this paper, we are mainly concerned with the empirical risk minimization (ERM) based learning process for time-dependent samples drawn from a continuous-time Markov chain. This learning process covers many kinds of practical applications, e.g., the prediction for a time series and the estimation of channel state information. Thus, it is significant to study its theoretical properties including the generalization bound, the asymptotic convergence, and the rate of convergence. It is noteworthy that, since samples are time dependent in this learning process, the concerns of this paper cannot (at least straightforwardly) be addressed by existing methods developed under the sample i.i.d. assumption. We first develop a deviation inequality for a sequence of time-dependent samples drawn from a continuous-time Markov chain and present a symmetrization inequality for such a sequence. By using the resultant deviation inequality and symmetrization inequality, we then obtain the generalization bounds of the ERM-based learning process for time-dependent samples drawn from a continuous-time Markov chain. Finally, based on the resultant generalization bounds, we analyze the asymptotic convergence and the rate of convergence of the learning process.

  18. Summary statistics for end-point conditioned continuous-time Markov chains

    DEFF Research Database (Denmark)

    Hobolth, Asger; Jensen, Jens Ledet

    Continuous-time Markov chains are a widely used modelling tool. Applications include DNA sequence evolution, ion channel gating behavior and mathematical finance. We consider the problem of calculating properties of summary statistics (e.g. mean time spent in a state, mean number of jumps between...... two states and the distribution of the total number of jumps) for discretely observed continuous time Markov chains. Three alternative methods for calculating properties of summary statistics are described and the pros and cons of the methods are discussed. The methods are based on (i) an eigenvalue...... decomposition of the rate matrix, (ii) the uniformization method, and (iii) integrals of matrix exponentials. In particular we develop a framework that allows for analyses of rather general summary statistics using the uniformization method....

  19. An Efficient Finite Difference Method for Parameter Sensitivities of Continuous Time Markov Chains

    CERN Document Server

    Anderson, David F

    2011-01-01

    We present an efficient finite difference method for the computation of parameter sensitivities for a wide class of continuous time Markov chains. The motivating class of models, and the source of our examples, are the stochastic chemical kinetic models commonly used in the biosciences, though other natural application areas include population processes and queuing networks. The method is essentially derived by making effective use of the random time change representation of Kurtz, and is no harder to implement than any standard continuous time Markov chain algorithm, such as "Gillespie's algorithm" or the next reaction method. Further, the method is analytically tractable, and, for a given number of realizations of the stochastic process, produces an estimator with substantially lower variance than that obtained using other common methods. Therefore, the computational complexity required to solve a given problem is lowered greatly. In this work, we present the method together with the theoretical analysis de...

  20. Limit theorems for Markov processes indexed by continuous time Galton-Watson trees

    CERN Document Server

    Bansaye, Vincent; Marsalle, Laurence; Tran, Viet Chi

    2009-01-01

    We study the evolution of a particle system whose genealogy is given by a supercritical continuous time Galton-Watson tree. The particles move independently according to a Markov process and when a branching event occurs, the offspring locations depend on the position of the mother and the number of offspring. We prove a law of large numbers for the empirical measure of individuals alive at time $t$. This relies on a probabilistic interpretation of its intensity by mean of an auxiliary process. This latter has the same generator as the Markov process along the branches plus additional branching events, associated with jumps of accelerated rate and biased distribution. This comes from the fact that choosing an individual uniformly at time $t$ favors lineages with more branching events and larger offspring number. The central limit theorem is considered on a special case. Several examples are developed, including applications to splitting diffusions, cellular aging, branching L\\'evy processes and ancestral line...

  1. Fitting timeseries by continuous-time Markov chains: A quadratic programming approach

    Science.gov (United States)

    Crommelin, D. T.; Vanden-Eijnden, E.

    2006-09-01

    Construction of stochastic models that describe the effective dynamics of observables of interest is an useful instrument in various fields of application, such as physics, climate science, and finance. We present a new technique for the construction of such models. From the timeseries of an observable, we construct a discrete-in-time Markov chain and calculate the eigenspectrum of its transition probability (or stochastic) matrix. As a next step we aim to find the generator of a continuous-time Markov chain whose eigenspectrum resembles the observed eigenspectrum as closely as possible, using an appropriate norm. The generator is found by solving a minimization problem: the norm is chosen such that the object function is quadratic and convex, so that the minimization problem can be solved using quadratic programming techniques. The technique is illustrated on various toy problems as well as on datasets stemming from simulations of molecular dynamics and of atmospheric flows.

  2. Stylised facts of financial time series and hidden Markov models in continuous time

    DEFF Research Database (Denmark)

    Nystrup, Peter; Madsen, Henrik; Lindström, Erik

    2015-01-01

    Hidden Markov models are often applied in quantitative finance to capture the stylised facts of financial returns. They are usually discrete-time models and the number of states rarely exceeds two because of the quadratic increase in the number of parameters with the number of states. This paper...

  3. Comparison of methods for calculating conditional expectations of sufficient statistics for continuous time Markov chains

    DEFF Research Database (Denmark)

    Tataru, Paula Cristina; Hobolth, Asger

    2011-01-01

    past evolutionary events (exact times and types of changes) are unaccessible and the past must be inferred from DNA sequence data observed in the present. RESULTS: We describe and implement three algorithms for computing linear combinations of expected values of the sufficient statistics, conditioned......BACKGROUND: Continuous time Markov chains (CTMCs) is a widely used model for describing the evolution of DNA sequences on the nucleotide, amino acid or codon level. The sufficient statistics for CTMCs are the time spent in a state and the number of changes between any two states. In applications...

  4. Discounted continuous-time constrained Markov decision processes in Polish spaces

    CERN Document Server

    Guo, Xianping; 10.1214/10-AAP749

    2012-01-01

    This paper is devoted to studying constrained continuous-time Markov decision processes (MDPs) in the class of randomized policies depending on state histories. The transition rates may be unbounded, the reward and costs are admitted to be unbounded from above and from below, and the state and action spaces are Polish spaces. The optimality criterion to be maximized is the expected discounted rewards, and the constraints can be imposed on the expected discounted costs. First, we give conditions for the nonexplosion of underlying processes and the finiteness of the expected discounted rewards/costs. Second, using a technique of occupation measures, we prove that the constrained optimality of continuous-time MDPs can be transformed to an equivalent (optimality) problem over a class of probability measures. Based on the equivalent problem and a so-called $\\bar{w}$-weak convergence of probability measures developed in this paper, we show the existence of a constrained optimal policy. Third, by providing a linear ...

  5. Average Sample-path Optimality for Continuous-time Markov Decision Processes in Polish Spaces

    Institute of Scientific and Technical Information of China (English)

    Quan-xin ZHU

    2011-01-01

    In this paper we study the average sample-path cost (ASPC) problem for continuous-time Markov decision processes in Polish spaces.To the best of our knowledge,this paper is a first attempt to study the ASPC criterion on continuous-time MDPs with Polish state and action spaces.The corresponding transition rates are allowed to be unbounded,and the cost rates may have neither upper nor lower bounds.Under some mild hypotheses,we prove the existence of e (ε ≥ 0)-ASPC optimal stationary policies based on two different approaches:one is the “optimality equation” approach and the other is the “two optimality inequalities” approach.

  6. Continuous-Time Semi-Markov Models in Health Economic Decision Making: An Illustrative Example in Heart Failure Disease Management.

    Science.gov (United States)

    Cao, Qi; Buskens, Erik; Feenstra, Talitha; Jaarsma, Tiny; Hillege, Hans; Postmus, Douwe

    2016-01-01

    Continuous-time state transition models may end up having large unwieldy structures when trying to represent all relevant stages of clinical disease processes by means of a standard Markov model. In such situations, a more parsimonious, and therefore easier-to-grasp, model of a patient's disease progression can often be obtained by assuming that the future state transitions do not depend only on the present state (Markov assumption) but also on the past through time since entry in the present state. Despite that these so-called semi-Markov models are still relatively straightforward to specify and implement, they are not yet routinely applied in health economic evaluation to assess the cost-effectiveness of alternative interventions. To facilitate a better understanding of this type of model among applied health economic analysts, the first part of this article provides a detailed discussion of what the semi-Markov model entails and how such models can be specified in an intuitive way by adopting an approach called vertical modeling. In the second part of the article, we use this approach to construct a semi-Markov model for assessing the long-term cost-effectiveness of 3 disease management programs for heart failure. Compared with a standard Markov model with the same disease states, our proposed semi-Markov model fitted the observed data much better. When subsequently extrapolating beyond the clinical trial period, these relatively large differences in goodness-of-fit translated into almost a doubling in mean total cost and a 60-d decrease in mean survival time when using the Markov model instead of the semi-Markov model. For the disease process considered in our case study, the semi-Markov model thus provided a sensible balance between model parsimoniousness and computational complexity. © The Author(s) 2015.

  7. Continuous-time Markov chain-based flux analysis in metabolism.

    Science.gov (United States)

    Huo, Yunzhang; Ji, Ping

    2014-09-01

    Metabolic flux analysis (MFA), a key technology in bioinformatics, is an effective way of analyzing the entire metabolic system by measuring fluxes. Many existing MFA approaches are based on differential equations, which are complicated to be solved mathematically. So MFA requires some simple approaches to investigate metabolism further. In this article, we applied continuous-time Markov chain to MFA, called MMFA approach, and transformed the MFA problem into a set of quadratic equations by analyzing the transition probability of each carbon atom in the entire metabolic system. Unlike the other methods, MMFA analyzes the metabolic model only through the transition probability. This approach is very generic and it could be applied to any metabolic system if all the reaction mechanisms in the system are known. The results of the MMFA approach were compared with several chemical reaction equilibrium constants from early experiments by taking pentose phosphate pathway as an example.

  8. Comparison of methods for calculating conditional expectations of sufficient statistics for continuous time Markov chains

    Directory of Open Access Journals (Sweden)

    Tataru Paula

    2011-12-01

    Full Text Available Abstract Background Continuous time Markov chains (CTMCs is a widely used model for describing the evolution of DNA sequences on the nucleotide, amino acid or codon level. The sufficient statistics for CTMCs are the time spent in a state and the number of changes between any two states. In applications past evolutionary events (exact times and types of changes are unaccessible and the past must be inferred from DNA sequence data observed in the present. Results We describe and implement three algorithms for computing linear combinations of expected values of the sufficient statistics, conditioned on the end-points of the chain, and compare their performance with respect to accuracy and running time. The first algorithm is based on an eigenvalue decomposition of the rate matrix (EVD, the second on uniformization (UNI, and the third on integrals of matrix exponentials (EXPM. The implementation in R of the algorithms is available at http://www.birc.au.dk/~paula/. Conclusions We use two different models to analyze the accuracy and eight experiments to investigate the speed of the three algorithms. We find that they have similar accuracy and that EXPM is the slowest method. Furthermore we find that UNI is usually faster than EVD.

  9. Continuous-Time Semi-Markov Models in Health Economic Decision Making : An Illustrative Example in Heart Failure Disease Management

    NARCIS (Netherlands)

    Cao, Qi; Buskens, Erik; Feenstra, Talitha; Jaarsma, Tiny; Hillege, Hans; Postmus, Douwe

    2016-01-01

    Continuous-time state transition models may end up having large unwieldy structures when trying to represent all relevant stages of clinical disease processes by means of a standard Markov model. In such situations, a more parsimonious, and therefore easier-to-grasp, model of a patient's disease pro

  10. A Hybrid Secure Scheme for Wireless Sensor Networks against Timing Attacks Using Continuous-Time Markov Chain and Queueing Model

    Directory of Open Access Journals (Sweden)

    Tianhui Meng

    2016-09-01

    Full Text Available Wireless sensor networks (WSNs have recently gained popularity for a wide spectrum of applications. Monitoring tasks can be performed in various environments. This may be beneficial in many scenarios, but it certainly exhibits new challenges in terms of security due to increased data transmission over the wireless channel with potentially unknown threats. Among possible security issues are timing attacks, which are not prevented by traditional cryptographic security. Moreover, the limited energy and memory resources prohibit the use of complex security mechanisms in such systems. Therefore, balancing between security and the associated energy consumption becomes a crucial challenge. This paper proposes a secure scheme for WSNs while maintaining the requirement of the security-performance tradeoff. In order to proceed to a quantitative treatment of this problem, a hybrid continuous-time Markov chain (CTMC and queueing model are put forward, and the tradeoff analysis of the security and performance attributes is carried out. By extending and transforming this model, the mean time to security attributes failure is evaluated. Through tradeoff analysis, we show that our scheme can enhance the security of WSNs, and the optimal rekeying rate of the performance and security tradeoff can be obtained.

  11. Stochastic Model Checking Continuous Time Markov Process%随机模型检测连续时间Markov过程

    Institute of Scientific and Technical Information of China (English)

    钮俊; 曾国荪; 吕新荣; 徐畅

    2011-01-01

    The trustworthiness of a dynamic system includes the correctness of function and the satisfiability of per formance mainly. This paper proposed an approach to verify the function and performance of a system under considera tion integratedly. Continuous-time Markov decision process (CTMDP) is a model that contains some aspects such as probabilistic choice;stochastic timing and nondeterminacy; and it is the model by which we verify function properties and analyze performance properties uniformly. We can verify the functional and performance specifications by computing the reachability probabilities in the product CTMDP. We proved the correctness of our approach; and obtained our veri fication results by using model checker MRMC(Markov Reward Model Checker). The theoretical results show that model checking CTMDP model is necessary and the model checking approach is feasible.%功能正确和性能可满足是复杂系统可信要求非常重要的两个方面.从定性验证和定量分析相结合的角度,对复杂并发系统进行功能验证和性能分析,统一地评估系统是否可信.连续时间Markov决策过程CTMDP(Continuous-time Markov decision process)能够统一刻画复杂系统的概率选择、随机时间及不确定性等重要特征.提出用CTMDP作为系统定性验证和定量分析模型,将复杂系统的功能验证和性能分析转化为CTMDP中的可达概率求解,并证明验证过程的正确性,最终借助模型检测器MRMC(Markov Reward Model Checker)实现模型检测.理论分析表明,提出的针对CTMDP模型的验证需求是必要的,验证思路和方法具有可行性.

  12. Application of Stochastic Automata Networks for Creation of Continuous Time Markov Chain Models of Voltage Gating of Gap Junction Channels

    Directory of Open Access Journals (Sweden)

    Mindaugas Snipas

    2015-01-01

    Full Text Available The primary goal of this work was to study advantages of numerical methods used for the creation of continuous time Markov chain models (CTMC of voltage gating of gap junction (GJ channels composed of connexin protein. This task was accomplished by describing gating of GJs using the formalism of the stochastic automata networks (SANs, which allowed for very efficient building and storing of infinitesimal generator of the CTMC that allowed to produce matrices of the models containing a distinct block structure. All of that allowed us to develop efficient numerical methods for a steady-state solution of CTMC models. This allowed us to accelerate CPU time, which is necessary to solve CTMC models, ∼20 times.

  13. Application of stochastic automata networks for creation of continuous time Markov chain models of voltage gating of gap junction channels.

    Science.gov (United States)

    Snipas, Mindaugas; Pranevicius, Henrikas; Pranevicius, Mindaugas; Pranevicius, Osvaldas; Paulauskas, Nerijus; Bukauskas, Feliksas F

    2015-01-01

    The primary goal of this work was to study advantages of numerical methods used for the creation of continuous time Markov chain models (CTMC) of voltage gating of gap junction (GJ) channels composed of connexin protein. This task was accomplished by describing gating of GJs using the formalism of the stochastic automata networks (SANs), which allowed for very efficient building and storing of infinitesimal generator of the CTMC that allowed to produce matrices of the models containing a distinct block structure. All of that allowed us to develop efficient numerical methods for a steady-state solution of CTMC models. This allowed us to accelerate CPU time, which is necessary to solve CTMC models, ~20 times.

  14. Analysis of the non-Markov parameter in continuous-time signal processing.

    Science.gov (United States)

    Varghese, J J; Bellette, P A; Weegink, K J; Bradley, A P; Meehan, P A

    2014-02-01

    The use of statistical complexity metrics has yielded a number of successful methodologies to differentiate and identify signals from complex systems where the underlying dynamics cannot be calculated. The Mori-Zwanzig framework from statistical mechanics forms the basis for the generalized non-Markov parameter (NMP). The NMP has been used to successfully analyze signals in a diverse set of complex systems. In this paper we show that the Mori-Zwanzig framework masks an elegantly simple closed form of the first NMP, which, for C(1) smooth autocorrelation functions, is solely a function of the second moment (spread) and amplitude envelope of the measured power spectrum. We then show that the higher-order NMPs can be constructed in closed form in a modular fashion from the lower-order NMPs. These results provide an alternative, signal processing-based perspective to analyze the NMP, which does not require an understanding of the Mori-Zwanzig generating equations. We analyze the parametric sensitivity of the zero-frequency value of the first NMP, which has been used as a metric to discriminate between states in complex systems. Specifically, we develop closed-form expressions for three instructive systems: band-limited white noise, the output of white noise input to an idealized all-pole filter,f and a simple harmonic oscillator driven by white noise. Analysis of these systems shows a primary sensitivity to the decay rate of the tail of the power spectrum.

  15. Hybrid Discrete-Continuous Markov Decision Processes

    Science.gov (United States)

    Feng, Zhengzhu; Dearden, Richard; Meuleau, Nicholas; Washington, Rich

    2003-01-01

    This paper proposes a Markov decision process (MDP) model that features both discrete and continuous state variables. We extend previous work by Boyan and Littman on the mono-dimensional time-dependent MDP to multiple dimensions. We present the principle of lazy discretization, and piecewise constant and linear approximations of the model. Having to deal with several continuous dimensions raises several new problems that require new solutions. In the (piecewise) linear case, we use techniques from partially- observable MDPs (POMDPS) to represent value functions as sets of linear functions attached to different partitions of the state space.

  16. Using model-based proposals for fast parameter inference on discrete state space, continuous-time Markov processes.

    Science.gov (United States)

    Pooley, C M; Bishop, S C; Marion, G

    2015-06-06

    Bayesian statistics provides a framework for the integration of dynamic models with incomplete data to enable inference of model parameters and unobserved aspects of the system under study. An important class of dynamic models is discrete state space, continuous-time Markov processes (DCTMPs). Simulated via the Doob-Gillespie algorithm, these have been used to model systems ranging from chemistry to ecology to epidemiology. A new type of proposal, termed 'model-based proposal' (MBP), is developed for the efficient implementation of Bayesian inference in DCTMPs using Markov chain Monte Carlo (MCMC). This new method, which in principle can be applied to any DCTMP, is compared (using simple epidemiological SIS and SIR models as easy to follow exemplars) to a standard MCMC approach and a recently proposed particle MCMC (PMCMC) technique. When measurements are made on a single-state variable (e.g. the number of infected individuals in a population during an epidemic), model-based proposal MCMC (MBP-MCMC) is marginally faster than PMCMC (by a factor of 2-8 for the tests performed), and significantly faster than the standard MCMC scheme (by a factor of 400 at least). However, when model complexity increases and measurements are made on more than one state variable (e.g. simultaneously on the number of infected individuals in spatially separated subpopulations), MBP-MCMC is significantly faster than PMCMC (more than 100-fold for just four subpopulations) and this difference becomes increasingly large. © 2015 The Author(s) Published by the Royal Society. All rights reserved.

  17. Continuous time Markov chain approaches for analyzing transtheoretical models of health behavioral change: A case study and comparison of model estimations.

    Science.gov (United States)

    Ma, Junsheng; Chan, Wenyaw; Tilley, Barbara C

    2016-04-04

    Continuous time Markov chain models are frequently employed in medical research to study the disease progression but are rarely applied to the transtheoretical model, a psychosocial model widely used in the studies of health-related outcomes. The transtheoretical model often includes more than three states and conceptually allows for all possible instantaneous transitions (referred to as general continuous time Markov chain). This complicates the likelihood function because it involves calculating a matrix exponential that may not be simplified for general continuous time Markov chain models. We undertook a Bayesian approach wherein we numerically evaluated the likelihood using ordinary differential equation solvers available from thegnuscientific library. We compared our Bayesian approach with the maximum likelihood method implemented with theRpackageMSM Our simulation study showed that the Bayesian approach provided more accurate point and interval estimates than the maximum likelihood method, especially in complex continuous time Markov chain models with five states. When applied to data from a four-state transtheoretical model collected from a nutrition intervention study in the next step trial, we observed results consistent with the results of the simulation study. Specifically, the two approaches provided comparable point estimates and standard errors for most parameters, but the maximum likelihood offered substantially smaller standard errors for some parameters. Comparable estimates of the standard errors are obtainable from packageMSM, which works only when the model estimation algorithm converges. © The Author(s) 2016.

  18. General continuous-time Markov model of sequence evolution via insertions/deletions: are alignment probabilities factorable?

    Science.gov (United States)

    Ezawa, Kiyoshi

    2016-08-11

    Insertions and deletions (indels) account for more nucleotide differences between two related DNA sequences than substitutions do, and thus it is imperative to develop a stochastic evolutionary model that enables us to reliably calculate the probability of the sequence evolution through indel processes. Recently, indel probabilistic models are mostly based on either hidden Markov models (HMMs) or transducer theories, both of which give the indel component of the probability of a given sequence alignment as a product of either probabilities of column-to-column transitions or block-wise contributions along the alignment. However, it is not a priori clear how these models are related with any genuine stochastic evolutionary model, which describes the stochastic evolution of an entire sequence along the time-axis. Moreover, currently none of these models can fully accommodate biologically realistic features, such as overlapping indels, power-law indel-length distributions, and indel rate variation across regions. Here, we theoretically dissect the ab initio calculation of the probability of a given sequence alignment under a genuine stochastic evolutionary model, more specifically, a general continuous-time Markov model of the evolution of an entire sequence via insertions and deletions. Our model is a simple extension of the general "substitution/insertion/deletion (SID) model". Using the operator representation of indels and the technique of time-dependent perturbation theory, we express the ab initio probability as a summation over all alignment-consistent indel histories. Exploiting the equivalence relations between different indel histories, we find a "sufficient and nearly necessary" set of conditions under which the probability can be factorized into the product of an overall factor and the contributions from regions separated by gapless columns of the alignment, thus providing a sort of generalized HMM. The conditions distinguish evolutionary models with

  19. Automated recognition of bird song elements from continuous recordings using dynamic time warping and hidden Markov models: a comparative study.

    Science.gov (United States)

    Kogan, J A; Margoliash, D

    1998-04-01

    The performance of two techniques is compared for automated recognition of bird song units from continuous recordings. The advantages and limitations of dynamic time warping (DTW) and hidden Markov models (HMMs) are evaluated on a large database of male songs of zebra finches (Taeniopygia guttata) and indigo buntings (Passerina cyanea), which have different types of vocalizations and have been recorded under different laboratory conditions. Depending on the quality of recordings and complexity of song, the DTW-based technique gives excellent to satisfactory performance. Under challenging conditions such as noisy recordings or presence of confusing short-duration calls, good performance of the DTW-based technique requires careful selection of templates that may demand expert knowledge. Because HMMs are trained, equivalent or even better performance of HMMs can be achieved based only on segmentation and labeling of constituent vocalizations, albeit with many more training examples than DTW templates. One weakness in HMM performance is the misclassification of short-duration vocalizations or song units with more variable structure (e.g., some calls, and syllables of plastic songs). To address these and other limitations, new approaches for analyzing bird vocalizations are discussed.

  20. A continuous-time semi-markov bayesian belief network model for availability measure estimation of fault tolerant systems

    Directory of Open Access Journals (Sweden)

    Márcio das Chagas Moura

    2008-08-01

    Full Text Available In this work it is proposed a model for the assessment of availability measure of fault tolerant systems based on the integration of continuous time semi-Markov processes and Bayesian belief networks. This integration results in a hybrid stochastic model that is able to represent the dynamic characteristics of a system as well as to deal with cause-effect relationships among external factors such as environmental and operational conditions. The hybrid model also allows for uncertainty propagation on the system availability. It is also proposed a numerical procedure for the solution of the state probability equations of semi-Markov processes described in terms of transition rates. The numerical procedure is based on the application of Laplace transforms that are inverted by the Gauss quadrature method known as Gauss Legendre. The hybrid model and numerical procedure are illustrated by means of an example of application in the context of fault tolerant systems.Neste trabalho, é proposto um modelo baseado na integração entre processos semi-Markovianos e redes Bayesianas para avaliação da disponibilidade de sistemas tolerantes à falha. Esta integração resulta em um modelo estocástico híbrido o qual é capaz de representar as características dinâmicas de um sistema assim como tratar as relações de causa e efeito entre fatores externos tais como condições ambientais e operacionais. Além disso, o modelo híbrido permite avaliar a propagação de incerteza sobre a disponibilidade do sistema. É também proposto um procedimento numérico para a solução das equações de probabilidade de estado de processos semi-Markovianos descritos por taxas de transição. Tal procedimento numérico é baseado na aplicação de transformadas de Laplace que são invertidas pelo método de quadratura Gaussiana conhecido como Gauss Legendre. O modelo híbrido e procedimento numérico são ilustrados por meio de um exemplo de aplicação no contexto de

  1. Markov chains and mixing times

    CERN Document Server

    Levin, David A; Wilmer, Elizabeth L

    2009-01-01

    This book is an introduction to the modern approach to the theory of Markov chains. The main goal of this approach is to determine the rate of convergence of a Markov chain to the stationary distribution as a function of the size and geometry of the state space. The authors develop the key tools for estimating convergence times, including coupling, strong stationary times, and spectral methods. Whenever possible, probabilistic methods are emphasized. The book includes many examples and provides brief introductions to some central models of statistical mechanics. Also provided are accounts of r

  2. 基于 Markov 链互模拟的航天器发射任务可靠度模型%Mission reliability model of spacecraft launch based on bisimulation of continuous-time Markov processes

    Institute of Scientific and Technical Information of China (English)

    董学军; 武小悦; 陈英武

    2012-01-01

    状态空间复杂、多过程并发执行和子过程反复迭代的特点,使航天器发射工程实施全过程的任务可靠性评估难以量化.通过构建多个并发执行的时间连续的 Markov 链对航天器发射工程状态转移约束关系进行描述,采用互模拟时间等价关系简化航天器发射工程实施过程的状态空间,利用连续时间 Markov 链的概率转移特性进行建模与分析,得到了全系统、全过程的航天器发射任务可靠度模型.数值验证表明该模型可用于航天器发射任务工期推演、可靠度评估以及薄弱环节分析.%Characteristics of complex state space, multi-process concurrent execution and sub-processes iterative make mission reliability assessment for the whole process of spacecraft launch engineering implementation is difficult to quantify. Multiple concurrently executing continuous time Markov chain is constructed to describe state transition constraint relations of spacecraft launch engineering. The state space of the whole process of spacecraft launch engineering implementation is simplified by bisimulation equivalence relation. The model of mission reliability for spacecraft launch engineering is builded by continuous time Markov chain transfer probability characteristics. In this paper, the example applied results shows that the model is a feasible for decision-making demonstration of spacecraft launch project, evaluation of mission reliability and analysis of weak link.

  3. Model check of continue time Markov reward process with impulse reward%带动作回报的连续时间Markov回报过程验证

    Institute of Scientific and Technical Information of China (English)

    黄镇谨; 陆阳; 杨娟; 王智文

    2015-01-01

    为了能够更准确的表达不确定性复杂系统的时空验证,针对当前连续时间Markov回报过程( continue time markov reward decision process,CMRDP)验证中只考虑状态回报的问题,提出带动作回报的验证方法. 考虑添加了动作回报的空间性能约束,扩展现有的基于状态回报的连续时间Markov回报过程,用正则表达式表示验证属性的路径规范,扩展已有路径算子的表达能力. 给出带动作回报 CMRDP和路径规范的积模型,求解积模型在确定性策略下的诱导Markov回报模型( markov reward model,MRM) ,将CMRDP上的时空性能验证转换为MRM模型上的时空可达概率分析,并提出MRM中求解可达概率的算法. 实例分析表明,提出的验证思路和验证算法是可行的.%In order to express the verification of the uncertainty temporal and spatial properties of the complex sys-tems more accurately which include nondeterministic choices, continuous time Markov reward decision process based on state reward is extended by considering spatial properties with impulse reward and is adopted as verifica-tion model.The path formulas expressed by traditional path operator is replaced by regular expressions, which can express more comprehensive verification properties.Under deterministic schedulers, the induced Markov reward model of product model which is the product of continuous time Markov reward decision process with impulse reward and path formula is proposed.After that we reduce the problem of model checking for product model to the problem of computing the maximum time-and space-bound reachability probabilities of induced MRM and put forward a algo-rithm to solve this problem.The experiment results show that the model checking approach for CMRDP with im-pulse reward is feasible.

  4. Compositional Modeling and Minimization of Time-Inhomogeneous Markov Chains

    NARCIS (Netherlands)

    Han, T.; Katoen, J.P.; Mereacre, A.

    2008-01-01

    This paper presents a compositional framework for the modeling of interactive continuous-time Markov chains with time-dependent rates, a subclass of communicating piecewise deterministic Markov processes. A poly-time algorithm is presented for computing the coarsest quotient under strong bisimulatio

  5. Continuity Properties of Distances for Markov Processes

    DEFF Research Database (Denmark)

    Jaeger, Manfred; Mao, Hua; Larsen, Kim Guldstrand

    2014-01-01

    In this paper we investigate distance functions on finite state Markov processes that measure the behavioural similarity of non-bisimilar processes. We consider both probabilistic bisimilarity metrics, and trace-based distances derived from standard Lp and Kullback-Leibler distances. Two desirable...... continuity properties for such distances are identified. We then establish a number of results that show that these two properties are in conflict, and not simultaneously fulfilled by any of our candidate natural distance functions. An impossibility result is derived that explains to some extent...

  6. Error exponents for Neyman-Pearson detection of a continuous-time Gaussian Markov process from noisy irregular samples

    CERN Document Server

    Hachem, Walid; Roueff, Francois

    2009-01-01

    This paper addresses the detection of a stochastic process in noise from irregular samples. We consider two hypotheses. The \\emph{noise only} hypothesis amounts to model the observations as a sample of a i.i.d. Gaussian random variables (noise only). The \\emph{signal plus noise} hypothesis models the observations as the samples of a continuous time stationary Gaussian process (the signal) taken at known but random time-instants corrupted with an additive noise. Two binary tests are considered, depending on which assumptions is retained as the null hypothesis. Assuming that the signal is a linear combination of the solution of a multidimensional stochastic differential equation (SDE), it is shown that the minimum Type II error probability decreases exponentially in the number of samples when the False Alarm probability is fixed. This behavior is described by \\emph{error exponents} that are completely characterized. It turns out that they are related with the asymptotic behavior of the Kalman Filter in random s...

  7. Study of Disease Progression and Relevant Risk Factors in Diabetic Foot Patients Using a Multistate Continuous-Time Markov Chain Model.

    Science.gov (United States)

    Begun, Alexander; Morbach, Stephan; Rümenapf, Gerhard; Icks, Andrea

    2016-01-01

    The diabetic foot is a lifelong disease. The longer patients with diabetes and foot ulcers are observed, the higher the likelihood that they will develop comorbidities that adversely influence ulcer recurrence, amputation and survival (for example peripheral arterial disease, renal failure or ischaemic heart disease). The purpose of our study was to quantify person and limb-related disease progression and the time-dependent influence of any associated factors (present at baseline or appearing during observation) based on which effective prevention and/or treatment strategies could be developed. Using a nine-state continuous-time Markov chain model with time-dependent risk factors, all living patients were divided into eight groups based on their ulceration (previous or current) and previous amputation (none, minor or major) status. State nine is an absorbing state (death). If all transitions are fully observable, this model can be decomposed into eight submodels, which can be analyzed using the methods of survival analysis for competing risks. The dependencies of the risk factors (covariates) were included in the submodels using Cox-like regression. The transition intensities and relative risks for covariates were calculated from long-term data of patients with diabetic foot ulcers collected in a single specialized center in North-Rhine Westphalia (Germany). The detected estimates were in line with previously published, but scarce, data. Together with the interesting new results obtained, this indicates that the proposed model may be useful for studying disease progression in larger samples of patients with diabetic foot ulcers.

  8. Analysis of transtheoretical model of health behavioral changes in a nutrition intervention study--a continuous time Markov chain model with Bayesian approach.

    Science.gov (United States)

    Ma, Junsheng; Chan, Wenyaw; Tsai, Chu-Lin; Xiong, Momiao; Tilley, Barbara C

    2015-11-30

    Continuous time Markov chain (CTMC) models are often used to study the progression of chronic diseases in medical research but rarely applied to studies of the process of behavioral change. In studies of interventions to modify behaviors, a widely used psychosocial model is based on the transtheoretical model that often has more than three states (representing stages of change) and conceptually permits all possible instantaneous transitions. Very little attention is given to the study of the relationships between a CTMC model and associated covariates under the framework of transtheoretical model. We developed a Bayesian approach to evaluate the covariate effects on a CTMC model through a log-linear regression link. A simulation study of this approach showed that model parameters were accurately and precisely estimated. We analyzed an existing data set on stages of change in dietary intake from the Next Step Trial using the proposed method and the generalized multinomial logit model. We found that the generalized multinomial logit model was not suitable for these data because it ignores the unbalanced data structure and temporal correlation between successive measurements. Our analysis not only confirms that the nutrition intervention was effective but also provides information on how the intervention affected the transitions among the stages of change. We found that, compared with the control group, subjects in the intervention group, on average, spent substantively less time in the precontemplation stage and were more/less likely to move from an unhealthy/healthy state to a healthy/unhealthy state.

  9. Simulation from endpoint-conditioned, continuous-time Markov chains on a finite state space, with applications to molecular evolution

    DEFF Research Database (Denmark)

    Hobolth, Asger; Stone, Eric

    2009-01-01

    computational finance to human genetics and genomics. A common theme among these diverse applications is the need to simulate sample paths of a CTMC conditional on realized data that is discretely observed. Here we present a general solution to this sampling problem when the CTMC is defined on a discrete...... approaches: (1) modified rejection sampling, (2) direct sampling, and (3) uniformization. We then give analytical results for the complexity and efficiency of each method in terms of the instantaneous transition rate matrix Q of the CTMC, its beginning and ending states, and the length of sampling time T...

  10. Quantitative timed analysis of interactive Markov chains

    NARCIS (Netherlands)

    Guck, Dennis; Han, Tingting; Katoen, Joost-Pieter; Neuhausser, M.

    2012-01-01

    This paper presents new algorithms and accompanying tool support for analyzing interactive Markov chains (IMCs), a stochastic timed 1 1/2-player game in which delays are exponentially distributed. IMCs are compositional and act as semantic model for engineering formalisms such as AADL and dynamic fa

  11. Graphs: Associated Markov Chains

    OpenAIRE

    2012-01-01

    In this research paper, weighted / unweighted, directed / undirected graphs are associated with interesting Discrete Time Markov Chains (DTMCs) as well as Continuous Time Markov Chains (CTMCs). The equilibrium / transient behaviour of such Markov chains is studied. Also entropy dynamics (Shannon entropy) of certain structured Markov chains is investigated. Finally certain structured graphs and the associated Markov chains are studied.

  12. Markov Chain Modelling for Short-Term NDVI Time Series Forecasting

    Directory of Open Access Journals (Sweden)

    Stepčenko Artūrs

    2016-12-01

    Full Text Available In this paper, the NDVI time series forecasting model has been developed based on the use of discrete time, continuous state Markov chain of suitable order. The normalised difference vegetation index (NDVI is an indicator that describes the amount of chlorophyll (the green mass and shows the relative density and health of vegetation; therefore, it is an important variable for vegetation forecasting. A Markov chain is a stochastic process that consists of a state space. This stochastic process undergoes transitions from one state to another in the state space with some probabilities. A Markov chain forecast model is flexible in accommodating various forecast assumptions and structures. The present paper discusses the considerations and techniques in building a Markov chain forecast model at each step. Continuous state Markov chain model is analytically described. Finally, the application of the proposed Markov chain model is illustrated with reference to a set of NDVI time series data.

  13. 基于连续马尔可夫模型的时变电力系统自适应控制策略%Adaptive Control Strategy Based on Continuous Markov Model for Time-varying Power System

    Institute of Scientific and Technical Information of China (English)

    马静; 高翔; 李益楠; 邱扬

    2016-01-01

    An adaptive control strategy based on continuous Markov model for time-varying power system is proposed to deal with the large variation in operating condition of power system caused by the wind generator output fluctuation.First, Lyapunov functional containing a continuous Markov model of time-varying power system is developed and Dynkin lemma is used to deduce the robust stochastic stabilization theorem with H ∞ norm boundγ.On this basis,Schur complement lemma is used to deduce robust stochastic stabilization linear matrix inequalities (LMI) meeting decay rate γ and minimum variance constraints considering steady-state variance constraints of various operating conditions.Then it is turned into a minimization problem of linear objective function to obtain the sub-controllers.The operating condition is identified online with the stochastic gradient method and the weighted coefficient of each condition is determined to construct the adaptive control strategy.Time-domain simulation results show that the strategy has efficiently solved the problem that fixed fault sets cannot be matched with current operating condition reasonably.%针对风电机组出力波动等时变因素导致电力系统运行工况变化较大的问题,提出了一种基于连续马尔可夫(Markov)模型的时变电力系统自适应控制策略。首先,构建含连续 Markov 时变电力系统模型的 Lyapunov 泛函,利用 Dynkin 引理,推导出具有 H ∞范数界γ的鲁棒随机稳定性定理。在此基础上,利用 Schur 补引理,考虑系统各子工况下稳态方差约束,推导出满足干扰衰减度γ和最小方差约束的鲁棒随机稳定线性矩阵不等式(LMI),并将该 LMI 转化为线性目标函数的最小化问题求解各子控制器。然后,利用随机梯度法辨识系统工况,确定各子控制器加权系数,构造自适应控制策略。时域仿真表明,该控制策略有效避免了固定故障集无法与当前工况合理匹配的问题。

  14. ON MARKOV CHAINS IN SPACE-TIME RANDOM ENVIRONMENTS

    Institute of Scientific and Technical Information of China (English)

    Hu Dihe; Hu Xiaoyu

    2009-01-01

    In Section 1, the authors establish the models of two kinds of Markov chains in space-time random environments (MCSTRE and MCSTRE(+)) with Abstract state space. In Section 2, the authors construct a MCSTRE and a MCSTRE(+) by an initial distribution Ф and a random Markov kernel (RMK) p(γ). In Section 3, the authors establish several equivalence theorems on MCSTRE and MCSTRE(+). Finally, the authors give two very important examples of MCMSTRE, the random walk in spce-time random environment and the Markov branching chain in space-time random environment.

  15. Markov chains and semi-Markov models in time-to-event analysis.

    Science.gov (United States)

    Abner, Erin L; Charnigo, Richard J; Kryscio, Richard J

    2013-10-25

    A variety of statistical methods are available to investigators for analysis of time-to-event data, often referred to as survival analysis. Kaplan-Meier estimation and Cox proportional hazards regression are commonly employed tools but are not appropriate for all studies, particularly in the presence of competing risks and when multiple or recurrent outcomes are of interest. Markov chain models can accommodate censored data, competing risks (informative censoring), multiple outcomes, recurrent outcomes, frailty, and non-constant survival probabilities. Markov chain models, though often overlooked by investigators in time-to-event analysis, have long been used in clinical studies and have widespread application in other fields.

  16. Determining a Class of Markov Chains by Hitting Time

    Institute of Scientific and Technical Information of China (English)

    2001-01-01

    @@1 Introduction In many practical problems we often cannot observe the behavior of all states for a Markov chain (see [3-5]). A natural question is that from the observable data of a part of states, can one still obtain all statistical characteristics of the Markov chains. In this paper we give the positive answer for this question and prove the surprising result that the transition rate matrix of the birth-death chains with reflecting barriers and Markov chains on a star graph can be uniquely determined by the probability density functions (pdfs) of the sojourn times and the hitting times at a single special state. This result also suggest a new special type of statistics for Markov chains.

  17. Evaluation of Bayesian estimation of a hidden continuous-time Markov chain model with application to threshold violation in water-quality indicators

    Science.gov (United States)

    Deviney, Frank A.; Rice, Karen; Brown, Donald E.

    2012-01-01

    Natural resource managers require information concerning  the frequency, duration, and long-term probability of occurrence of water-quality indicator (WQI) violations of defined thresholds. The timing of these threshold crossings often is hidden from the observer, who is restricted to relatively infrequent observations. Here, a model for the hidden process is linked with a model for the observations, and the parameters describing duration, return period, and long-term probability of occurrence are estimated using Bayesian methods. A simulation experiment is performed to evaluate the approach under scenarios based on the equivalent of a total monitoring period of 5-30 years and an observation frequency of 1-50 observations per year. Given constant threshold crossing rate, accuracy and precision of parameter estimates increased with longer total monitoring period and more-frequent observations. Given fixed monitoring period and observation frequency, accuracy and precision of parameter estimates increased with longer times between threshold crossings. For most cases where the long-term probability of being in violation is greater than 0.10, it was determined that at least 600 observations are needed to achieve precise estimates.  An application of the approach is presented using 22 years of quasi-weekly observations of acid-neutralizing capacity from Deep Run, a stream in Shenandoah National Park, Virginia. The time series also was sub-sampled to simulate monthly and semi-monthly sampling protocols. Estimates of the long-term probability of violation were unbiased despite sampling frequency; however, the expected duration and return period were over-estimated using the sub-sampled time series with respect to the full quasi-weekly time series.

  18. Markov chain modeling of precipitation time series: Modeling waiting times between tipping bucket rain gauge tips

    DEFF Research Database (Denmark)

    Sørup, Hjalte Jomo Danielsen; Madsen, Henrik; Arnbjerg-Nielsen, Karsten

    2011-01-01

    A very fine temporal and volumetric resolution precipitation time series is modeled using Markov models. Both 1st and 2nd order Markov models as well as seasonal and diurnal models are investigated and evaluated using likelihood based techniques. The 2nd order Markov model is found to be insignif...

  19. Markov

    Directory of Open Access Journals (Sweden)

    Carlos Alejandro De Luna Ortega

    2006-01-01

    Full Text Available En este artículo se aborda el diseño de un reconocedor de voz, con el idioma español mexicano, del estado de Aguascalientes, de palabras aisladas, con dependencia del hablante y vocabulario pequeño, empleando Redes Neuronales Artificiales (ANN por sus siglas en inglés, Alineamiento Dinámico del Tiempo (DTW por sus siglas en inglés y Modelos Ocultos de Markov (HMM por sus siglas en inglés para la realización del algoritmo de reconocimiento.

  20. RANDOM TIMES TRANSFORMATION OF PROCESSES WITH MARKOV SKELETON%Markov 骨架过程的随机时变换

    Institute of Scientific and Technical Information of China (English)

    刘万荣; 刘再明; 侯振挺

    2000-01-01

    In this paper, random time transformations of processes with Markov skeleton are discussed. A class of random time transformations that transform a process with Markov skeleton into a process with Markov skeleton is given.%讨论了 Markov 骨架过程的随机时变换,给出了一类变换 Markov 骨架过程为Markov 骨架过程的随机时变换.

  1. A Single-Server Queue with Markov Modulated Service Times

    OpenAIRE

    1999-01-01

    We study an M/MMPP/1 queuing system, where the arrival process is Poisson and service requirements are Markov modulated. When the Markov Chain modulating service times has two states, we show that the distribution of the number-in-system is a superposition of two matrix-geometric series and provide a simple algorithm for computing the rate and coefficient matrices. These results hold for both finite and infinite waiting space systems and extend results obtained in Neuts [5] and Naoumov [4]. N...

  2. Clustering Multivariate Time Series Using Hidden Markov Models

    Directory of Open Access Journals (Sweden)

    Shima Ghassempour

    2014-03-01

    Full Text Available In this paper we describe an algorithm for clustering multivariate time series with variables taking both categorical and continuous values. Time series of this type are frequent in health care, where they represent the health trajectories of individuals. The problem is challenging because categorical variables make it difficult to define a meaningful distance between trajectories. We propose an approach based on Hidden Markov Models (HMMs, where we first map each trajectory into an HMM, then define a suitable distance between HMMs and finally proceed to cluster the HMMs with a method based on a distance matrix. We test our approach on a simulated, but realistic, data set of 1,255 trajectories of individuals of age 45 and over, on a synthetic validation set with known clustering structure, and on a smaller set of 268 trajectories extracted from the longitudinal Health and Retirement Survey. The proposed method can be implemented quite simply using standard packages in R and Matlab and may be a good candidate for solving the difficult problem of clustering multivariate time series with categorical variables using tools that do not require advanced statistic knowledge, and therefore are accessible to a wide range of researchers.

  3. On Markov Chains and Filtrations

    OpenAIRE

    1997-01-01

    In this paper we rederive some well known results for continuous time Markov processes that live on a finite state space.Martingale techniques are used throughout the paper. Special attention is paid to the construction of a continuous timeMarkov process, when we start from a discrete time Markov chain. The Markov property here holds with respect tofiltrations that need not be minimal.

  4. First passage times for Markov renewal processes and applications

    Institute of Scientific and Technical Information of China (English)

    2000-01-01

    This paper proposes a uniformly convergent algorithm for the joint transform of the first passage time and the first passage number of steps for general Markov renewal processes with any initial state probability vector. The uniformly convergent algorithm with arbitrarily prescribed error can be efficiently applied to compute busy periods, busy cycles, waiting times, sojourn times, and relevant indices of various generic queueing systems and queueing networks. This paper also conducts a numerical experiment to implement the proposed algorithm.

  5. Markov Model of Wind Power Time Series UsingBayesian Inference of Transition Matrix

    DEFF Research Database (Denmark)

    Chen, Peiyuan; Berthelsen, Kasper Klitgaard; Bak-Jensen, Birgitte

    2009-01-01

    This paper proposes to use Bayesian inference of transition matrix when developing a discrete Markov model of a wind speed/power time series and 95% credible interval for the model verification. The Dirichlet distribution is used as a conjugate prior for the transition matrix. Three discrete Markov...... models are compared, i.e. the basic Markov model, the Bayesian Markov model and the birth-and-death Markov model. The proposed Bayesian Markov model shows the best accuracy in modeling the autocorrelation of the wind power time series....

  6. Algebraic convergence for discrete-time ergodic Markov chains

    Institute of Scientific and Technical Information of China (English)

    MAO; Yonghua(毛永华)

    2003-01-01

    This paper studies the e-ergodicity for discrete-time recurrent Markov chains. It proves that thee-order deviation matrix exists and is finite if and only if the chain is (e + 2)-ergodic, and then the algebraicdecay rates of the n-step transition probability to the stationary distribution are obtained. The criteria fore-ergodicity are given in terms of existence of solution to an equation. The main results are illustrated by some examples.

  7. An Online Policy Gradient Algorithm for Markov Decision Processes with Continuous States and Actions.

    Science.gov (United States)

    Ma, Yao; Zhao, Tingting; Hatano, Kohei; Sugiyama, Masashi

    2016-03-01

    We consider the learning problem under an online Markov decision process (MDP) aimed at learning the time-dependent decision-making policy of an agent that minimizes the regret-the difference from the best fixed policy. The difficulty of online MDP learning is that the reward function changes over time. In this letter, we show that a simple online policy gradient algorithm achieves regret O(√T) for T steps under a certain concavity assumption and O(log T) under a strong concavity assumption. To the best of our knowledge, this is the first work to present an online MDP algorithm that can handle continuous state, action, and parameter spaces with guarantee. We also illustrate the behavior of the proposed online policy gradient method through experiments.

  8. Continuous-time signals

    CERN Document Server

    Shmaliy, Yuriy

    2006-01-01

    Gives a modern description of continuous-time deterministic signals Signal formation techniquesTime vs. frequency and frequency vs. time analysisCorrelation and energy analysisNarrowband signals and sampling.

  9. Exponential Stability for Neutral Stochastic Markov Systems With Time-Varying Delay and Its Applications.

    Science.gov (United States)

    Chen, Huabin; Shi, Peng; Lim, Cheng-Chew; Hu, Peng

    2016-06-01

    In this paper, the exponential stability in p th( p > 1 )-moment for neutral stochastic Markov systems with time-varying delay is studied. The derived stability conditions comprise two forms: 1) the delay-independent stability criteria which are obtained by establishing an integral inequality and 2) the delay-dependent stability criteria which are captured by using the theory of the functional differential equations. As its applications, the obtained stability results are used to investigate the exponential stability in p th( p > 1 )-moment for the neutral stochastic neural networks with time-varying delay and Markov switching, and the globally exponential adaptive synchronization for the neutral stochastic complex dynamical systems with time-varying delay and Markov switching, respectively. On the delay-independent criteria, sufficient conditions are given in terms of M -matrix and thus are easy to check. The delay-dependent criteria are presented in the forms of the algebraic inequalities, and the least upper bound of the time-varying delay is also provided. The primary advantages of these obtained results over some recent and similar works are that the differentiability or continuity of the delay function is not required, and that the difficulty stemming from the presence of the neutral item and the Markov switching is overcome. Three numerical examples are provided to examine the effectiveness and potential of the theoretic results obtained.

  10. MARKOV: A methodology for the solution of infinite time horizon MARKOV decision processes

    Science.gov (United States)

    Williams, B.K.

    1988-01-01

    Algorithms are described for determining optimal policies for finite state, finite action, infinite discrete time horizon Markov decision processes. Both value-improvement and policy-improvement techniques are used in the algorithms. Computing procedures are also described. The algorithms are appropriate for processes that are either finite or infinite, deterministic or stochastic, discounted or undiscounted, in any meaningful combination of these features. Computing procedures are described in terms of initial data processing, bound improvements, process reduction, and testing and solution. Application of the methodology is illustrated with an example involving natural resource management. Management implications of certain hypothesized relationships between mallard survival and harvest rates are addressed by applying the optimality procedures to mallard population models.

  11. Continuous Time Model Estimation

    OpenAIRE

    Carl Chiarella; Shenhuai Gao

    2004-01-01

    This paper introduces an easy to follow method for continuous time model estimation. It serves as an introduction on how to convert a state space model from continuous time to discrete time, how to decompose a hybrid stochastic model into a trend model plus a noise model, how to estimate the trend model by simulation, and how to calculate standard errors from estimation of the noise model. It also discusses the numerical difficulties involved in discrete time models that bring about the unit ...

  12. Process Algebra and Markov Chains

    NARCIS (Netherlands)

    Brinksma, Hendrik; Hermanns, H.; Brinksma, Hendrik; Hermanns, H.; Katoen, Joost P.

    This paper surveys and relates the basic concepts of process algebra and the modelling of continuous time Markov chains. It provides basic introductions to both fields, where we also study the Markov chains from an algebraic perspective, viz. that of Markov chain algebra. We then proceed to study

  13. Fluctuations in Markov Processes Time Symmetry and Martingale Approximation

    CERN Document Server

    Komorowski, Tomasz; Olla, Stefano

    2012-01-01

    The present volume contains the most advanced theories on the martingale approach to central limit theorems. Using the time symmetry properties of the Markov processes, the book develops the techniques that allow us to deal with infinite dimensional models that appear in statistical mechanics and engineering (interacting particle systems, homogenization in random environments, and diffusion in turbulent flows, to mention just a few applications). The first part contains a detailed exposition of the method, and can be used as a text for graduate courses. The second concerns application to exclu

  14. Resilient model approximation for Markov jump time-delay systems via reduced model with hierarchical Markov chains

    Science.gov (United States)

    Zhu, Yanzheng; Zhang, Lixian; Sreeram, Victor; Shammakh, Wafa; Ahmad, Bashir

    2016-10-01

    In this paper, the resilient model approximation problem for a class of discrete-time Markov jump time-delay systems with input sector-bounded nonlinearities is investigated. A linearised reduced-order model is determined with mode changes subject to domination by a hierarchical Markov chain containing two different nonhomogeneous Markov chains. Hence, the reduced-order model obtained not only reflects the dependence of the original systems but also model external influence that is related to the mode changes of the original system. Sufficient conditions formulated in terms of bilinear matrix inequalities for the existence of such models are established, such that the resulting error system is stochastically stable and has a guaranteed l2-l∞ error performance. A linear matrix inequalities optimisation coupled with line search is exploited to solve for the corresponding reduced-order systems. The potential and effectiveness of the developed theoretical results are demonstrated via a numerical example.

  15. Continuous Markov Random Fields for Robust Stereo Estimation

    CERN Document Server

    Yamaguchi, Koichiro; McAllester, David; Urtasun, Raquel

    2012-01-01

    In this paper we present a novel slanted-plane MRF model which reasons jointly about occlusion boundaries as well as depth. We formulate the problem as the one of inference in a hybrid MRF composed of both continuous (i.e., slanted 3D planes) and discrete (i.e., occlusion boundaries) random variables. This allows us to define potentials encoding the ownership of the pixels that compose the boundary between segments, as well as potentials encoding which junctions are physically possible. Our approach outperforms the state-of-the-art on Middlebury high resolution imagery as well as in the more challenging KITTI dataset, while being more efficient than existing slanted plane MRF-based methods, taking on average 2 minutes to perform inference on high resolution imagery.

  16. Semi-Markov processes

    CERN Document Server

    Grabski

    2014-01-01

    Semi-Markov Processes: Applications in System Reliability and Maintenance is a modern view of discrete state space and continuous time semi-Markov processes and their applications in reliability and maintenance. The book explains how to construct semi-Markov models and discusses the different reliability parameters and characteristics that can be obtained from those models. The book is a useful resource for mathematicians, engineering practitioners, and PhD and MSc students who want to understand the basic concepts and results of semi-Markov process theory. Clearly defines the properties and

  17. Long Memory of Financial Time Series and Hidden Markov Models with Time-Varying Parameters

    DEFF Research Database (Denmark)

    Nystrup, Peter; Madsen, Henrik; Lindström, Erik

    2016-01-01

    estimation approach that allows for the parameters of the estimated models to be time varying. It is shown that a two-state Gaussian hidden Markov model with time-varying parameters is able to reproduce the long memory of squared daily returns that was previously believed to be the most difficult fact...... to reproduce with a hidden Markov model. Capturing the time-varying behavior of the parameters also leads to improved one-step density forecasts. Finally, it is shown that the forecasting performance of the estimated models can be further improved using local smoothing to forecast the parameter variations....

  18. Long memory of financial time series and hidden Markov models with time-varying parameters

    DEFF Research Database (Denmark)

    Nystrup, Peter; Madsen, Henrik; Lindström, Erik

    facts have not been thoroughly examined. This paper presents an adaptive estimation approach that allows for the parameters of the estimated models to be time-varying. It is shown that a two-state Gaussian hidden Markov model with time-varying parameters is able to reproduce the long memory of squared...... daily returns that was previously believed to be the most difficult fact to reproduce with a hidden Markov model. Capturing the time-varying behavior of the parameters also leads to improved one-step predictions....

  19. Hidden Markov model tracking of continuous gravitational waves from a neutron star with wandering spin

    CERN Document Server

    Suvorova, S; Melatos, A; Moran, W; Evans, R J

    2016-01-01

    Gravitational wave searches for continuous-wave signals from neutron stars are especially challenging when the star's spin frequency is unknown a priori from electromagnetic observations and wanders stochastically under the action of internal (e.g. superfluid or magnetospheric) or external (e.g. accretion) torques. It is shown that frequency tracking by hidden Markov model (HMM) methods can be combined with existing maximum likelihood coherent matched filters like the F-statistic to surmount some of the challenges raised by spin wandering. Specifically it is found that, for an isolated, biaxial rotor whose spin frequency walks randomly, HMM tracking of the F-statistic output from coherent segments with duration T_drift = 10d over a total observation time of T_obs = 1yr can detect signals with wave strains h0 > 2e-26 at a noise level characteristic of the Advanced Laser Interferometer Gravitational Wave Observatory (Advanced LIGO). For a biaxial rotor with randomly walking spin in a binary orbit, whose orbital...

  20. Automatic detection of alpine rockslides in continuous seismic data using hidden Markov models

    Science.gov (United States)

    Dammeier, Franziska; Moore, Jeffrey R.; Hammer, Conny; Haslinger, Florian; Loew, Simon

    2016-02-01

    Data from continuously recording permanent seismic networks can contain information about rockslide occurrence and timing complementary to eyewitness observations and thus aid in construction of robust event catalogs. However, detecting infrequent rockslide signals within large volumes of continuous seismic waveform data remains challenging and often requires demanding manual intervention. We adapted an automatic classification method using hidden Markov models to detect rockslide signals in seismic data from two stations in central Switzerland. We first processed 21 known rockslides, with event volumes spanning 3 orders of magnitude and station event distances varying by 1 order of magnitude, which resulted in 13 and 19 successfully classified events at the two stations. Retraining the models to incorporate seismic noise from the day of the event improved the respective results to 16 and 19 successful classifications. The missed events generally had low signal-to-noise ratio and small to medium volumes. We then processed nearly 14 years of continuous seismic data from the same two stations to detect previously unknown events. After postprocessing, we classified 30 new events as rockslides, of which we could verify three through independent observation. In particular, the largest new event, with estimated volume of 500,000 m3, was not generally known within the Swiss landslide community, highlighting the importance of regional seismic data analysis even in densely populated mountainous regions. Our method can be easily implemented as part of existing earthquake monitoring systems, and with an average event detection rate of about two per month, manual verification would not significantly increase operational workload.

  1. a Continuous-Time Positive Linear System

    Directory of Open Access Journals (Sweden)

    Kyungsup Kim

    2013-01-01

    Full Text Available This paper discusses a computational method to construct positive realizations with sparse matrices for continuous-time positive linear systems with multiple complex poles. To construct a positive realization of a continuous-time system, we use a Markov sequence similar to the impulse response sequence that is used in the discrete-time case. The existence of the proposed positive realization can be analyzed with the concept of a polyhedral convex cone. We provide a constructive algorithm to compute positive realizations with sparse matrices of some positive systems under certain conditions. A sufficient condition for the existence of a positive realization, under which the proposed constructive algorithm works well, is analyzed.

  2. Hidden Markov model tracking of continuous gravitational waves from a neutron star with wandering spin

    Science.gov (United States)

    Suvorova, S.; Sun, L.; Melatos, A.; Moran, W.; Evans, R. J.

    2016-06-01

    Gravitational wave searches for continuous-wave signals from neutron stars are especially challenging when the star's spin frequency is unknown a priori from electromagnetic observations and wanders stochastically under the action of internal (e.g., superfluid or magnetospheric) or external (e.g., accretion) torques. It is shown that frequency tracking by hidden Markov model (HMM) methods can be combined with existing maximum likelihood coherent matched filters like the F -statistic to surmount some of the challenges raised by spin wandering. Specifically, it is found that, for an isolated, biaxial rotor whose spin frequency walks randomly, HMM tracking of the F -statistic output from coherent segments with duration Tdrift=10 d over a total observation time of Tobs=1 yr can detect signals with wave strains h0>2 ×10-26 at a noise level characteristic of the Advanced Laser Interferometer Gravitational Wave Observatory (Advanced LIGO). For a biaxial rotor with randomly walking spin in a binary orbit, whose orbital period and semimajor axis are known approximately from electromagnetic observations, HMM tracking of the Bessel-weighted F -statistic output can detect signals with h0>8 ×10-26. An efficient, recursive, HMM solver based on the Viterbi algorithm is demonstrated, which requires ˜103 CPU hours for a typical, broadband (0.5-kHz) search for the low-mass x-ray binary Scorpius X-1, including generation of the relevant F -statistic input. In a "realistic" observational scenario, Viterbi tracking successfully detects 41 out of 50 synthetic signals without spin wandering in stage I of the Scorpius X-1 Mock Data Challenge convened by the LIGO Scientific Collaboration down to a wave strain of h0=1.1 ×10-25, recovering the frequency with a root-mean-square accuracy of ≤4.3 ×10-3 Hz .

  3. Effective degree Markov-chain approach for discrete-time epidemic processes on uncorrelated networks.

    Science.gov (United States)

    Cai, Chao-Ran; Wu, Zhi-Xi; Guan, Jian-Yue

    2014-11-01

    Recently, Gómez et al. proposed a microscopic Markov-chain approach (MMCA) [S. Gómez, J. Gómez-Gardeñes, Y. Moreno, and A. Arenas, Phys. Rev. E 84, 036105 (2011)PLEEE81539-375510.1103/PhysRevE.84.036105] to the discrete-time susceptible-infected-susceptible (SIS) epidemic process and found that the epidemic prevalence obtained by this approach agrees well with that by simulations. However, we found that the approach cannot be straightforwardly extended to a susceptible-infected-recovered (SIR) epidemic process (due to its irreversible property), and the epidemic prevalences obtained by MMCA and Monte Carlo simulations do not match well when the infection probability is just slightly above the epidemic threshold. In this contribution we extend the effective degree Markov-chain approach, proposed for analyzing continuous-time epidemic processes [J. Lindquist, J. Ma, P. Driessche, and F. Willeboordse, J. Math. Biol. 62, 143 (2011)JMBLAJ0303-681210.1007/s00285-010-0331-2], to address discrete-time binary-state (SIS) or three-state (SIR) epidemic processes on uncorrelated complex networks. It is shown that the final epidemic size as well as the time series of infected individuals obtained from this approach agree very well with those by Monte Carlo simulations. Our results are robust to the change of different parameters, including the total population size, the infection probability, the recovery probability, the average degree, and the degree distribution of the underlying networks.

  4. CONSTRUCTION OF CONTINUOUS TIME MARKOVIAN ARRIVAL PROCESSES

    Institute of Scientific and Technical Information of China (English)

    Qi-Ming HE

    2010-01-01

    Markovian arrival processes were introduced by Neuts in 1979(Neuts 1979)and have been used extensively in the stochastic modeling of queueing,inventory,reliability,risk,and telecommunications systems.In this paper,we introduce a constructive approach to define continuous time Markovian arrival processes.The construction is based on Poisson processes,and is simple and intuitive.Such a construction makes it easy to interpret the parameters of Markovian arrival processes.The construction also makes it possible to establish rigorously basic equations,such as Kolmogorov differential equations,for Markovian arrival processes,using only elementary properties of exponential distributions and Poisson processes.In addition,the approach can be used to construct continuous time Markov chains with a finite number of states

  5. Hidden Markov Models for Time Series An Introduction Using R

    CERN Document Server

    Zucchini, Walter

    2009-01-01

    Illustrates the flexibility of HMMs as general-purpose models for time series data. This work presents an overview of HMMs for analyzing time series data, from continuous-valued, circular, and multivariate series to binary data, bounded and unbounded counts and categorical observations.

  6. Reduction techniques for discrete-time Markov chains on totally ordered state space using stochastic comparisons

    OpenAIRE

    2000-01-01

    We propose in this paper two methods to compute Markovian bounds for monotone functions of a discrete time homogeneous Markov chain evolving in a totally ordered state space. The main interest of such methods is to propose algorithms to simplify analysis of transient characteristics such as the output process of a queue, or sojourn time in a subset of states. Construction of bounds are based on two kinds of results: well-known results on stochastic comparison between Markov cha...

  7. Expectation propagation for continuous time stochastic processes

    Science.gov (United States)

    Cseke, Botond; Schnoerr, David; Opper, Manfred; Sanguinetti, Guido

    2016-12-01

    We consider the inverse problem of reconstructing the posterior measure over the trajectories of a diffusion process from discrete time observations and continuous time constraints. We cast the problem in a Bayesian framework and derive approximations to the posterior distributions of single time marginals using variational approximate inference, giving rise to an expectation propagation type algorithm. For non-linear diffusion processes, this is achieved by leveraging moment closure approximations. We then show how the approximation can be extended to a wide class of discrete-state Markov jump processes by making use of the chemical Langevin equation. Our empirical results show that the proposed method is computationally efficient and provides good approximations for these classes of inverse problems.

  8. Model Checking Interactive Markov Chains

    NARCIS (Netherlands)

    Neuhausser, M.; Zhang, Lijun; Esparza, J.; Majumdar, R.

    2010-01-01

    Hermanns has introduced interactive Markov chains (IMCs) which arise as an orthogonal extension of labelled transition systems and continuous-time Markov chains (CTMCs). IMCs enjoy nice compositional aggregation properties which help to minimize the state space incrementally. However, the model chec

  9. Applying Markov Chains for NDVI Time Series Forecasting of Latvian Regions

    Directory of Open Access Journals (Sweden)

    Stepchenko Arthur

    2015-12-01

    Full Text Available Time series of earth observation based estimates of vegetation inform about variations in vegetation at the scale of Latvia. A vegetation index is an indicator that describes the amount of chlorophyll (the green mass and shows the relative density and health of vegetation. NDVI index is an important variable for vegetation forecasting and management of various problems, such as climate change monitoring, energy usage monitoring, managing the consumption of natural resources, agricultural productivity monitoring, drought monitoring and forest fire detection. In this paper, we make a one-step-ahead prediction of 7-daily time series of NDVI index using Markov chains. The choice of a Markov chain is due to the fact that a Markov chain is a sequence of random variables where each variable is located in some state. And a Markov chain contains probabilities of moving from one state to other.

  10. A Simple Discrete Model of Brownian Motors: Time-periodic Markov Chains

    Science.gov (United States)

    Ge, Hao; Jiang, Da-Quan; Qian, Min

    2006-05-01

    In this paper, we consider periodically inhomogeneous Markov chains, which can be regarded as a simple version of physical model—Brownian motors. We introduce for them the concepts of periodical reversibility, detailed balance, entropy production rate and circulation distribution. We prove the equivalence of the following statements: The time-periodic Markov chain is periodically reversible; It is in detailed balance; Kolmogorov's cycle condition is satisfied; Its entropy production rate vanishes; Every circuit and its reversed circuit have the same circulation weight. Hence, in our model of Markov chains, the directed transport phenomenon of Brownian motors, i.e. the existence of net circulation, can occur only in nonequilibrium and irreversible systems. Moreover, we verify the large deviation property and the Gallavotti-Cohen fluctuation theorem of sample entropy production rates of the Markov chain.

  11. Markov Chains and Markov Processes

    OpenAIRE

    2016-01-01

    Markov chain, which was named after Andrew Markov is a mathematical system that transfers a state to another state. Many real world systems contain uncertainty. This study helps us to understand the basic idea of a Markov chain and how is been useful in our daily lives. For some times there had been suspense on distinct predictions and future existences. Also in different games there had been different expectations or results involved. That is the reason why we need Markov chains to predict o...

  12. Formal Reasoning About Finite-State Discrete-Time Markov Chains in HOL

    Institute of Scientific and Technical Information of China (English)

    Liya Liu; Osman Hasan; Sofiène Tahar

    2013-01-01

    Markov chains are extensively used in modeling different aspects of engineering and scientific systems,such as performance of algorithms and reliability of systems.Different techniques have been developed for analyzing Markovian models,for example,Markov Chain Monte Carlo based simulation,Markov Analyzer,and more recently probabilistic modelchecking.However,these techniques either do not guarantee accurate analysis or are not scalable.Higher-order-logic theorem proving is a formal method that has the ability to overcome the above mentioned limitations.However,it is not mature enough to handle all sorts of Markovian models.In this paper,we propose a formalization of Discrete-Time Markov Chain (DTMC) that facilitates formal reasoning about time-homogeneous finite-state discrete-time Markov chain.In particular,we provide a formal verification on some of its important properties,such as joint probabilities,Chapman-Kolmogorov equation,reversibility property,using higher-order logic.To demonstrate the usefulness of our work,we analyze two applications:a simplified binary communication channel and the Automatic Mail Quality Measurement protocol.

  13. Real time Markov chains: Wind states in anemometric data

    CERN Document Server

    Sanchez, P A; Jaramillo, O A

    2015-01-01

    The description of wind phenomena is frequently based on data obtained from anemometers, which usually report the wind speed and direction only in a horizontal plane. Such measurements are commonly used either to develop wind generation farms or to forecast weather conditions in a geographical region. Beyond these standard applications, the information contained in the data may be richer than expected and may lead to a better understanding of the wind dynamics in a geographical area. In this work we propose a statistical analysis based on the wind velocity vectors, which we propose may be grouped in "wind states" associated to binormal distribution functions. We found that the velocity plane defined by the anemometric velocity data may be used as a phase space, where a finite number of states may be found and sorted using standard clustering methods. The main result is a discretization technique useful to model the wind with Markov chains. We applied such ideas in anemometric data for two different sites in M...

  14. Performance Modeling of Communication Networks with Markov Chains

    CERN Document Server

    Mo, Jeonghoon

    2010-01-01

    This book is an introduction to Markov chain modeling with applications to communication networks. It begins with a general introduction to performance modeling in Chapter 1 where we introduce different performance models. We then introduce basic ideas of Markov chain modeling: Markov property, discrete time Markov chain (DTMe and continuous time Markov chain (CTMe. We also discuss how to find the steady state distributions from these Markov chains and how they can be used to compute the system performance metric. The solution methodologies include a balance equation technique, limiting probab

  15. A discrete time event-history approach to informative drop-out in mixed latent Markov models with covariates.

    Science.gov (United States)

    Bartolucci, Francesco; Farcomeni, Alessio

    2015-03-01

    Mixed latent Markov (MLM) models represent an important tool of analysis of longitudinal data when response variables are affected by time-fixed and time-varying unobserved heterogeneity, in which the latter is accounted for by a hidden Markov chain. In order to avoid bias when using a model of this type in the presence of informative drop-out, we propose an event-history (EH) extension of the latent Markov approach that may be used with multivariate longitudinal data, in which one or more outcomes of a different nature are observed at each time occasion. The EH component of the resulting model is referred to the interval-censored drop-out, and bias in MLM modeling is avoided by correlated random effects, included in the different model components, which follow common latent distributions. In order to perform maximum likelihood estimation of the proposed model by the expectation-maximization algorithm, we extend the usual forward-backward recursions of Baum and Welch. The algorithm has the same complexity as the one adopted in cases of non-informative drop-out. We illustrate the proposed approach through simulations and an application based on data coming from a medical study about primary biliary cirrhosis in which there are two outcomes of interest, one continuous and the other binary. © 2014, The International Biometric Society.

  16. Continuous Change Detection and Classification Using Hidden Markov Model: A Case Study for Monitoring Urban Encroachment onto Farmland in Beijing

    Directory of Open Access Journals (Sweden)

    Yuan Yuan

    2015-11-01

    Full Text Available In this paper, we propose a novel method to continuously monitor land cover change using satellite image time series, which can extract comprehensive change information including change time, location, and “from-to” information. This method is based on a hidden Markov model (HMM trained for each land cover class. Assuming a pixel’s initial class has been obtained, likelihoods of the corresponding model are calculated on incoming time series extracted with a temporal sliding window. By observing the likelihood change over the windows, land cover change can be precisely detected from the dramatic drop of likelihood. The established HMMs are then used for identifying the land cover class after the change. As a case study, the proposed method is applied to monitoring urban encroachment onto farmland in Beijing using 10-year MODIS time series from 2001 to 2010. The performance is evaluated on a validation set for different model structures and thresholds. Compared with other change detection methods, the proposed method shows superior change detection accuracy. In addition, it is also more computationally efficient.

  17. Multi-stream continuous hidden Markov models with application to landmine detection

    Science.gov (United States)

    Missaoui, Oualid; Frigui, Hichem; Gader, Paul

    2013-12-01

    We propose a multi-stream continuous hidden Markov model (MSCHMM) framework that can learn from multiple modalities. We assume that the feature space is partitioned into subspaces generated by different sources of information. In order to fuse the different modalities, the proposed MSCHMM introduces stream relevance weights. First, we modify the probability density function (pdf) that characterizes the standard continuous HMM to include state and component dependent stream relevance weights. The resulting pdf approximate is a linear combination of pdfs characterizing multiple modalities. Second, we formulate the CHMM objective function to allow for the simultaneous optimization of all model parameters including the relevance weights. Third, we generalize the maximum likelihood based Baum-Welch algorithm and the minimum classification error/gradient probabilistic descent (MCE/GPD) learning algorithms to include stream relevance weights. We propose two versions of the MSCHMM. The first one introduces the relevance weights at the state level while the second one introduces the weights at the component level. We illustrate the performance of the proposed MSCHMM structures using synthetic data sets. We also apply them to the problem of landmine detection using ground penetrating radar. We show that when the multiple sources of information are equally relevant across all training data, the performance of the proposed MSCHMM is comparable to the baseline CHMM. However, when the relevance of the sources varies, the MSCHMM outperforms the baseline CHMM because it can learn the optimal relevance weights. We also show that our approach outperforms existing multi-stream HMM because the latter one cannot optimize all model parameters simultaneously.

  18. Neural Network Based Finite-Time Stabilization for Discrete-Time Markov Jump Nonlinear Systems with Time Delays

    Directory of Open Access Journals (Sweden)

    Fei Chen

    2013-01-01

    Full Text Available This paper deals with the finite-time stabilization problem for discrete-time Markov jump nonlinear systems with time delays and norm-bounded exogenous disturbance. The nonlinearities in different jump modes are parameterized by neural networks. Subsequently, a linear difference inclusion state space representation for a class of neural networks is established. Based on this, sufficient conditions are derived in terms of linear matrix inequalities to guarantee stochastic finite-time boundedness and stochastic finite-time stabilization of the closed-loop system. A numerical example is illustrated to verify the efficiency of the proposed technique.

  19. Analysis of timed and long-run objectives for Markov automata

    NARCIS (Netherlands)

    Guck, Dennis; Hatefi, Hassan; Hermanns, Holger; Katoen, Joost-Pieter; Timmer, Mark

    2014-01-01

    Markov automata (MAs) extend labelled transition systems with random delays and probabilistic branching. Action-labelled transitions are instantaneous and yield a distribution over states, whereas timed transitions impose a random delay governed by an exponential distribution. MAs are thus a nondete

  20. Particle Markov Chain Monte Carlo Techniques of Unobserved Component Time Series Models Using Ox

    DEFF Research Database (Denmark)

    Nonejad, Nima

    This paper details Particle Markov chain Monte Carlo techniques for analysis of unobserved component time series models using several economic data sets. PMCMC combines the particle filter with the Metropolis-Hastings algorithm. Overall PMCMC provides a very compelling, computationally fast...

  1. Use of Markov Chains to Design an Agent Bidding Strategy for Continuous Double Auctions

    CERN Document Server

    Birmingham, W P; Park, S; 10.1613/jair.1466

    2011-01-01

    As computational agents are developed for increasingly complicated e-commerce applications, the complexity of the decisions they face demands advances in artificial intelligence techniques. For example, an agent representing a seller in an auction should try to maximize the seller?s profit by reasoning about a variety of possibly uncertain pieces of information, such as the maximum prices various buyers might be willing to pay, the possible prices being offered by competing sellers, the rules by which the auction operates, the dynamic arrival and matching of offers to buy and sell, and so on. A naive application of multiagent reasoning techniques would require the seller?s agent to explicitly model all of the other agents through an extended time horizon, rendering the problem intractable for many realistically-sized problems. We have instead devised a new strategy that an agent can use to determine its bid price based on a more tractable Markov chain model of the auction process. We have experimentally ident...

  2. Clarification of basic factorization identity is for the almost semi-continuous latticed Poisson processes on the Markov chain

    Directory of Open Access Journals (Sweden)

    Gerich M. S.

    2012-12-01

    Full Text Available Let ${xi(t, x(t}$ be a homogeneous semi-continuous lattice Poisson process on the Markov chain.The jumps of one sign are geometrically distributed, and jumps of the opposite sign are arbitrary latticed distribution. For a suchprocesses the relations for the components of two-sided matrix factorization are established.This relations define the moment genereting functions for extremumf of the process and their complements.

  3. Clarification of basic factorization identity is for the almost semi-continuous latticed Poisson processes on the Markov chain

    OpenAIRE

    2012-01-01

    Let ${xi(t), x(t)}$ be a homogeneous semi-continuous lattice Poisson process on the Markov chain.The jumps of one sign are geometrically distributed, and jumps of the opposite sign are arbitrary latticed distribution. For a suchprocesses the relations for the components of two-sided matrix factorization are established.This relations define the moment genereting functions for extremumf of the process and their complements.

  4. Markov chains theory and applications

    CERN Document Server

    Sericola, Bruno

    2013-01-01

    Markov chains are a fundamental class of stochastic processes. They are widely used to solve problems in a large number of domains such as operational research, computer science, communication networks and manufacturing systems. The success of Markov chains is mainly due to their simplicity of use, the large number of available theoretical results and the quality of algorithms developed for the numerical evaluation of many metrics of interest.The author presents the theory of both discrete-time and continuous-time homogeneous Markov chains. He carefully examines the explosion phenomenon, the

  5. Distributed synthesis in continuous time

    DEFF Research Database (Denmark)

    Hermanns, Holger; Krčál, Jan; Vester, Steen

    2016-01-01

    . Indeed, the explicit continuous time enables players to communicate their states by delaying synchronisation (which is unrestricted for non-urgent models). In general, the problems are undecidable already for two players in the quantitative case and three players in the qualitative case. The qualitative......We introduce a formalism modelling communication of distributed agents strictly in continuous-time. Within this framework, we study the problem of synthesising local strategies for individual agents such that a specified set of goal states is reached, or reached with at least a given probability....... The flow of time is modelled explicitly based on continuous-time randomness, with two natural implications: First, the non-determinism stemming from interleaving disappears. Second, when we restrict to a subclass of non-urgent models, the quantitative value problem for two players can be solved in EXPTIME...

  6. Nested Markov Compliance Class Model in the Presence of Time-Varying Noncompliance

    OpenAIRE

    Lin, Julia Y.; Ten Have, Thomas R.; ELLIOTT, MICHAEL R.

    2009-01-01

    We consider a Markov structure for partially unobserved time-varying compliance classes in the Imbens-Rubin (1997) compliance model framework. The context is a longitudinal randomized intervention study where subjects are randomized once at baseline, outcomes and patient adherence are measured at multiple follow-ups, and patient adherence to their randomized treatment could vary over time. We propose a nested latent compliance class model where we use time-invariant subject-specific complianc...

  7. Iterative convergence of passage-time densities in semi-Markov performance models

    OpenAIRE

    Bradley, J.T.; Wilson, H. J.

    2005-01-01

    Passage-time densities are important for the detailed performance analysis of distributed computer and communicating systems. We provide a proof and demonstration of a practical iterative algorithm for extracting complete passage-time densities from expressive semi-Markov systems. We end by showing its application to a distributed web-server cluster model of 15.9 million states. © 2004 Elsevier B.V. All rights reserved.

  8. A new approach to real-time reliability analysis of transmission system using fuzzy Markov model

    Energy Technology Data Exchange (ETDEWEB)

    Tanrioven, M.; Kocatepe, C. [University of Yildiz Technical, Istanbul (Turkey). Dept. of Electrical Engineering; Wu, Q.H.; Turner, D.R.; Wang, J. [Liverpool Univ. (United Kingdom). Dept. of Electrical Engineering and Economics

    2004-12-01

    To date the studies of power system reliability over a specified time period have used average values of the system transition rates in Markov techniques. [Singh C, Billinton R. System reliability modeling and evaluation. London: Hutchison Educational; 1977]. However, the level of power systems reliability varies from time to time due to weather conditions, power demand and random faults [Billinton R, Wojczynski E. Distributional variation of distribution system reliability indices. IEEE Trans Power Apparatus Systems 1985; PAS-104(11):3152-60]. It is essential to obtain an estimate of system reliability under all environmental and operating conditions. In this paper, fuzzy logic is used in the Markov model to describe both transition rates and temperature-based seasonal variations, which identifies multiple weather conditions such as normal, less stormy, very stormy, etc. A three-bus power system model is considered to determine the variation of system reliability in real-time, using this newly developed fuzzy Markov model (FMM). The results cover different aspects such as daily and monthly reliability changes during January and August. The reliability of the power transmission system is derived as a function of augmentation in peak load level. Finally the variation of the system reliability with weather conditions is determined. (author)

  9. Analysis of Streamline Separation at Infinity Using Time-Discrete Markov Chains.

    Science.gov (United States)

    Reich, W; Scheuermann, G

    2012-12-01

    Existing methods for analyzing separation of streamlines are often restricted to a finite time or a local area. In our paper we introduce a new method that complements them by allowing an infinite-time-evaluation of steady planar vector fields. Our algorithm unifies combinatorial and probabilistic methods and introduces the concept of separation in time-discrete Markov-Chains. We compute particle distributions instead of the streamlines of single particles. We encode the flow into a map and then into a transition matrix for each time direction. Finally, we compare the results of our grid-independent algorithm to the popular Finite-Time-Lyapunov-Exponents and discuss the discrepancies.

  10. A cyclic time-dependent Markov process to model daily patterns in wind turbine power production

    CERN Document Server

    Scholz, Teresa; Estanqueiro, Ana

    2013-01-01

    Wind energy is becoming a top contributor to the renewable energy mix, which raises potential reliability issues for the grid due to the fluctuating nature of its source. To achieve adequate reserve commitment and to promote market participation, it is necessary to provide models that can capture daily patterns in wind power production. This paper presents a cyclic inhomogeneous Markov process, which is based on a three-dimensional state-space (wind power, speed and direction). Each time-dependent transition probability is expressed as a Bernstein polynomial. The model parameters are estimated by solving a constrained optimization problem: The objective function combines two maximum likelihood estimators, one to ensure that the Markov process long-term behavior reproduces the data accurately and another to capture daily fluctuations. A convex formulation for the overall optimization problem is presented and its applicability demonstrated through the analysis of a case-study. The proposed model is capable of r...

  11. Regularity of harmonic functions for some Markov chains with unbounded range

    OpenAIRE

    2012-01-01

    We consider a class of continuous time Markov chains on $\\Z^d$. These chains are the discrete space analogue of Markov processes with jumps. Under some conditions, we show that harmonic functions associated with these Markov chains are H\\"{o}lder continuous.

  12. On characterizations of Metropolis type algorithms in continuous time

    OpenAIRE

    Diaconis, Persi; Miclo, Laurent

    2009-01-01

    International audience; In the continuous time framework, a new definition is proposed for the Metropolis algorithm $(\\wi X_t)_{t\\geq0}$ associated to an a priori given exploratory Markov process $( X_t)_{t\\geq0}$ and to a tarjet probability distribution $\\pi$. It should be the minimizer for the relative entropy of the trajectorial law of $(\\wi X_t)_{t\\in[0,T]}$ with respect to the law of $( X_t)_{t\\in[0,T]}$, when both processes start with $\\pi$ as initial law and when $\\pi$ is assumed to be...

  13. On the Imbedding Problem for Three-state Time Homogeneous Markov Chains with Coinciding Negative Eigenvalues

    CERN Document Server

    Chen, Yong

    2010-01-01

    For an indecomposable $3\\times 3$ stochastic matrix (i.e., 1-step transition probability matrix) with coinciding negative eigenvalues, a new necessary and sufficient condition of the imbedding problem for time homogeneous Markov chains is shown by means of an alternate parameterization of the transition rate matrix (i.e., intensity matrix, infinitesimal generator), which avoids calculating matrix logarithm or matrix square root. In addition, an implicit description of the imbedding problem for the $3\\times 3$ stochastic matrix in Johansen [J. Lond. Math. Soc., 8, 345-351. (1974)] is pointed out.

  14. Markov processes and controlled Markov chains

    CERN Document Server

    Filar, Jerzy; Chen, Anyue

    2002-01-01

    The general theory of stochastic processes and the more specialized theory of Markov processes evolved enormously in the second half of the last century. In parallel, the theory of controlled Markov chains (or Markov decision processes) was being pioneered by control engineers and operations researchers. Researchers in Markov processes and controlled Markov chains have been, for a long time, aware of the synergies between these two subject areas. However, this may be the first volume dedicated to highlighting these synergies and, almost certainly, it is the first volume that emphasizes the contributions of the vibrant and growing Chinese school of probability. The chapters that appear in this book reflect both the maturity and the vitality of modern day Markov processes and controlled Markov chains. They also will provide an opportunity to trace the connections that have emerged between the work done by members of the Chinese school of probability and the work done by the European, US, Central and South Ameri...

  15. Markov reward processes

    Science.gov (United States)

    Smith, R. M.

    1991-01-01

    Numerous applications in the area of computer system analysis can be effectively studied with Markov reward models. These models describe the behavior of the system with a continuous-time Markov chain, where a reward rate is associated with each state. In a reliability/availability model, upstates may have reward rate 1 and down states may have reward rate zero associated with them. In a queueing model, the number of jobs of certain type in a given state may be the reward rate attached to that state. In a combined model of performance and reliability, the reward rate of a state may be the computational capacity, or a related performance measure. Expected steady-state reward rate and expected instantaneous reward rate are clearly useful measures of the Markov reward model. More generally, the distribution of accumulated reward or time-averaged reward over a finite time interval may be determined from the solution of the Markov reward model. This information is of great practical significance in situations where the workload can be well characterized (deterministically, or by continuous functions e.g., distributions). The design process in the development of a computer system is an expensive and long term endeavor. For aerospace applications the reliability of the computer system is essential, as is the ability to complete critical workloads in a well defined real time interval. Consequently, effective modeling of such systems must take into account both performance and reliability. This fact motivates our use of Markov reward models to aid in the development and evaluation of fault tolerant computer systems.

  16. Entropy: The Markov Ordering Approach

    Directory of Open Access Journals (Sweden)

    Alexander N. Gorban

    2010-05-01

    Full Text Available The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant “additivity” properties: (i existence of a monotonic transformation which makes the functional additive with respect to the joining of independent systems and (ii existence of a monotonic transformation which makes the functional additive with respect to the partitioning of the space of states. All Lyapunov functionals for Markov chains which have properties (i and (ii are derived. We describe the most general ordering of the distribution space, with respect to which all continuous-time Markov processes are monotonic (the Markov order. The solution differs significantly from the ordering given by the inequality of entropy growth. For inference, this approach results in a convex compact set of conditionally “most random” distributions.

  17. Optimal State Estimation for Discrete-Time Markov Jump Systems with Missing Observations

    Directory of Open Access Journals (Sweden)

    Qing Sun

    2014-01-01

    Full Text Available This paper is concerned with the optimal linear estimation for a class of direct-time Markov jump systems with missing observations. An observer-based approach of fault detection and isolation (FDI is investigated as a detection mechanic of fault case. For systems with known information, a conditional prediction of observations is applied and fault observations are replaced and isolated; then, an FDI linear minimum mean square error estimation (LMMSE can be developed by comprehensive utilizing of the correct information offered by systems. A recursive equation of filtering based on the geometric arguments can be obtained. Meanwhile, a stability of the state estimator will be guaranteed under appropriate assumption.

  18. The limitations of discrete-time approaches to continuous-time contagion dynamics

    CERN Document Server

    Fennell, Peter G; Gleeson, James P

    2016-01-01

    Continuous-time Markov process models of contagions are widely studied, not least because of their utility in predicting the evolution of real-world contagions and in formulating control measures. It is often the case, however, that discrete-time approaches are employed to analyze such models or to simulate them numerically. In such cases, time is discretized into uniform steps and transition rates between states are replaced by transition probabilities. In this paper, we illustrate potential limitations to this approach. We show how discretizing time leads to a restriction on the values of the model parameters that can accurately be studied. We examine numerical simulation schemes employed in the literature, showing how synchronous-type updating schemes can bias discrete-time formalisms when compared against continuous-time formalisms. Event-based simulations, such as the Gillespie algorithm, are proposed as optimal simulation schemes both in terms of replicating the continuous-time process and computational...

  19. Travel cost inference from sparse, spatio-temporally correlated time series using markov models

    DEFF Research Database (Denmark)

    Yang, B.; Guo, C.; Jensen, C.S.

    2013-01-01

    of such time series offers insight into the underlying system and enables prediction of system behavior. While the techniques presented in the paper apply more generally, we consider the case of transportation systems and aim to predict travel cost from GPS tracking data from probe vehicles. Specifically, each......The monitoring of a system can yield a set of measurements that can be modeled as a collection of time series. These time series are often sparse, due to missing measurements, and spatiotemporally correlated, meaning that spatially close time series exhibit temporal correlation. The analysis...... road segment has an associated travel-cost time series, which is derived from GPS data. We use spatio-temporal hidden Markov models (STHMM) to model correlations among different traffic time series. We provide algorithms that are able to learn the parameters of an STHMM while contending...

  20. A compositional framework for Markov processes

    Science.gov (United States)

    Baez, John C.; Fong, Brendan; Pollard, Blake S.

    2016-03-01

    We define the concept of an "open" Markov process, or more precisely, continuous-time Markov chain, which is one where probability can flow in or out of certain states called "inputs" and "outputs." One can build up a Markov process from smaller open pieces. This process is formalized by making open Markov processes into the morphisms of a dagger compact category. We show that the behavior of a detailed balanced open Markov process is determined by a principle of minimum dissipation, closely related to Prigogine's principle of minimum entropy production. Using this fact, we set up a functor mapping open detailed balanced Markov processes to open circuits made of linear resistors. We also describe how to "black box" an open Markov process, obtaining the linear relation between input and output data that holds in any steady state, including nonequilibrium steady states with a nonzero flow of probability through the system. We prove that black boxing gives a symmetric monoidal dagger functor sending open detailed balanced Markov processes to Lagrangian relations between symplectic vector spaces. This allows us to compute the steady state behavior of an open detailed balanced Markov process from the behaviors of smaller pieces from which it is built. We relate this black box functor to a previously constructed black box functor for circuits.

  1. Travel cost inference from sparse, spatio-temporally correlated time series using markov models

    DEFF Research Database (Denmark)

    Yang, B.; Guo, C.; Jensen, C.S.

    2013-01-01

    of such time series offers insight into the underlying system and enables prediction of system behavior. While the techniques presented in the paper apply more generally, we consider the case of transportation systems and aim to predict travel cost from GPS tracking data from probe vehicles. Specifically, each...... road segment has an associated travel-cost time series, which is derived from GPS data. We use spatio-temporal hidden Markov models (STHMM) to model correlations among different traffic time series. We provide algorithms that are able to learn the parameters of an STHMM while contending...... with the sparsity, spatio-temporal correlation, and heterogeneity of the time series. Using the resulting STHMM, near future travel costs in the transportation network, e.g., travel time or greenhouse gas emissions, can be inferred, enabling a variety of routing services, e.g., eco-routing. Empirical studies...

  2. Finite-time stability and stabilization for stochastic markov jump systems with mode-dependent time delays.

    Science.gov (United States)

    Yan, Zhiguo; Song, Yunxia; Park, Ju H

    2017-05-01

    This paper is concerned with the problems of finite-time stability and stabilization for stochastic Markov systems with mode-dependent time-delays. In order to reduce conservatism, a mode-dependent approach is utilized. Based on the derived stability conditions, state-feedback controller and observer-based controller are designed, respectively. A new N-mode algorithm is given to obtain the maximum value of time-delay. Finally, an example is used to show the merit of the proposed results. Copyright © 2017 ISA. Published by Elsevier Ltd. All rights reserved.

  3. Variance-reduced simulation of lattice discrete-time Markov chains with applications in reaction networks

    Science.gov (United States)

    Maginnis, P. A.; West, M.; Dullerud, G. E.

    2016-10-01

    We propose an algorithm to accelerate Monte Carlo simulation for a broad class of stochastic processes. Specifically, the class of countable-state, discrete-time Markov chains driven by additive Poisson noise, or lattice discrete-time Markov chains. In particular, this class includes simulation of reaction networks via the tau-leaping algorithm. To produce the speedup, we simulate pairs of fair-draw trajectories that are negatively correlated. Thus, when averaged, these paths produce an unbiased Monte Carlo estimator that has reduced variance and, therefore, reduced error. Numerical results for three example systems included in this work demonstrate two to four orders of magnitude reduction of mean-square error. The numerical examples were chosen to illustrate different application areas and levels of system complexity. The areas are: gene expression (affine state-dependent rates), aerosol particle coagulation with emission and human immunodeficiency virus infection (both with nonlinear state-dependent rates). Our algorithm views the system dynamics as a "black-box", i.e., we only require control of pseudorandom number generator inputs. As a result, typical codes can be retrofitted with our algorithm using only minor changes. We prove several analytical results. Among these, we characterize the relationship of covariances between paths in the general nonlinear state-dependent intensity rates case, and we prove variance reduction of mean estimators in the special case of affine intensity rates.

  4. Controlling influenza disease: Comparison between discrete time Markov chain and deterministic model

    Science.gov (United States)

    Novkaniza, F.; Ivana, Aldila, D.

    2016-04-01

    Mathematical model of respiratory diseases spread with Discrete Time Markov Chain (DTMC) and deterministic approach for constant total population size are analyzed and compared in this article. Intervention of medical treatment and use of medical mask included in to the model as a constant parameter to controlling influenza spreads. Equilibrium points and basic reproductive ratio as the endemic criteria and it level set depend on some variable are given analytically and numerically as a results from deterministic model analysis. Assuming total of human population is constant from deterministic model, number of infected people also analyzed with Discrete Time Markov Chain (DTMC) model. Since Δt → 0, we could assume that total number of infected people might change only from i to i + 1, i - 1, or i. Approximation probability of an outbreak with gambler's ruin problem will be presented. We find that no matter value of basic reproductive ℛ0, either its larger than one or smaller than one, number of infection will always tends to 0 for t → ∞. Some numerical simulation to compare between deterministic and DTMC approach is given to give a better interpretation and a better understanding about the models results.

  5. Markov stochasticity coordinates

    Science.gov (United States)

    Eliazar, Iddo

    2017-01-01

    Markov dynamics constitute one of the most fundamental models of random motion between the states of a system of interest. Markov dynamics have diverse applications in many fields of science and engineering, and are particularly applicable in the context of random motion in networks. In this paper we present a two-dimensional gauging method of the randomness of Markov dynamics. The method-termed Markov Stochasticity Coordinates-is established, discussed, and exemplified. Also, the method is tweaked to quantify the stochasticity of the first-passage-times of Markov dynamics, and the socioeconomic equality and mobility in human societies.

  6. Markov stochasticity coordinates

    Energy Technology Data Exchange (ETDEWEB)

    Eliazar, Iddo, E-mail: iddo.eliazar@intel.com

    2017-01-15

    Markov dynamics constitute one of the most fundamental models of random motion between the states of a system of interest. Markov dynamics have diverse applications in many fields of science and engineering, and are particularly applicable in the context of random motion in networks. In this paper we present a two-dimensional gauging method of the randomness of Markov dynamics. The method–termed Markov Stochasticity Coordinates–is established, discussed, and exemplified. Also, the method is tweaked to quantify the stochasticity of the first-passage-times of Markov dynamics, and the socioeconomic equality and mobility in human societies.

  7. Continuous-time discrete-space models for animal movement

    Science.gov (United States)

    Hanks, Ephraim M.; Hooten, Mevin B.; Alldredge, Mat W.

    2015-01-01

    The processes influencing animal movement and resource selection are complex and varied. Past efforts to model behavioral changes over time used Bayesian statistical models with variable parameter space, such as reversible-jump Markov chain Monte Carlo approaches, which are computationally demanding and inaccessible to many practitioners. We present a continuous-time discrete-space (CTDS) model of animal movement that can be fit using standard generalized linear modeling (GLM) methods. This CTDS approach allows for the joint modeling of location-based as well as directional drivers of movement. Changing behavior over time is modeled using a varying-coefficient framework which maintains the computational simplicity of a GLM approach, and variable selection is accomplished using a group lasso penalty. We apply our approach to a study of two mountain lions (Puma concolor) in Colorado, USA.

  8. For Time-Continuous Optimisation

    DEFF Research Database (Denmark)

    Heinrich, Mary Katherine; Ayres, Phil

    2016-01-01

    Strategies for optimisation in design normatively assume an artefact end-point, disallowing continuous architecture that engages living systems, dynamic behaviour, and complex systems. In our Flora Robotica investigations of symbiotic plant-robot bio-hybrids, we re- quire computational tools...

  9. For Time-Continuous Optimisation

    DEFF Research Database (Denmark)

    Heinrich, Mary Katherine; Ayres, Phil

    2016-01-01

    Strategies for optimisation in design normatively assume an artefact end-point, disallowing continuous architecture that engages living systems, dynamic behaviour, and complex systems. In our Flora Robotica investigations of symbiotic plant-robot bio-hybrids, we re- quire computational tools...

  10. Quasi-stationary distributions of a pair of Markov chains related to time evolution of a DNA locus

    OpenAIRE

    Bobrowski, A.

    2004-01-01

    We consider a pair of Markov chains representing statistics of the Fisher-Wright-Moran model with mutations and drift. The chains have absorbing state at 0 and are related by the fact that some random time τ ago they were identical, evolving as a single Markov chain with values in {0,1,...}; from that time on they began to evolve independently, conditional on a state at the time of split, according to the same transition probabilities. The distribution of τ is a function ...

  11. A Discrete Time Markov Chain Model for High Throughput Bidirectional Fano Decoders

    CERN Document Server

    Xu, Ran; Morris, Kevin; Kocak, Taskin

    2010-01-01

    The bidirectional Fano algorithm (BFA) can achieve at least two times decoding throughput compared to the conventional unidirectional Fano algorithm (UFA). In this paper, bidirectional Fano decoding is examined from the queuing theory perspective. A Discrete Time Markov Chain (DTMC) is employed to model the BFA decoder with a finite input buffer. The relationship between the input data rate, the input buffer size and the clock speed of the BFA decoder is established. The DTMC based modelling can be used in designing a high throughput parallel BFA decoding system. It is shown that there is a tradeoff between the number of BFA decoders and the input buffer size, and an optimal input buffer size can be chosen to minimize the hardware complexity for a target decoding throughput in designing a high throughput parallel BFA decoding system.

  12. CONTINUITY OF DYNAMIC-SYSTEMS - THE CONTINUOUS-TIME CASE

    NARCIS (Netherlands)

    NIEUWENHUIS, JW; WILLEMS, JC

    1992-01-01

    The purpose of this paper is to study continuity of the parametrization of continuous-time linear time-invariant differential systems having a finite-dimensional state space. We show that convergence of the behavior of such systems corresponds to convergence of the coefficients of a set of associate

  13. Continuous Markovian Logics

    DEFF Research Database (Denmark)

    Mardare, Radu Iulian; Cardelli, Luca; Larsen, Kim Guldstrand

    2012-01-01

    Continuous Markovian Logic (CML) is a multimodal logic that expresses quantitative and qualitative properties of continuous-time labelled Markov processes with arbitrary (analytic) state-spaces, henceforth called continuous Markov processes (CMPs). The modalities of CML evaluate the rates of the ...

  14. Experimental optimization of a real time fed-batch fermentation process using Markov decision process.

    Science.gov (United States)

    Saucedo, V M; Karim, M N

    1997-07-20

    This article describes a methodology that implements a Markov decision process (MDP) optimization technique in a real time fed-batch experiment. Biological systems can be better modeled under the stochastic framework and MDP is shown to be a suitable technique for their optimization. A nonlinear input/output model is used to calculate the probability transitions. All elements of the MDP are identified according to physical parameters. Finally, this study compares the results obtained when optimizing ethanol production using the infinite horizon problem, with total expected discount policy, to previous experimental results aimed at optimizing ethanol production using a recombinant Escherichia coli fed-batch cultivation. (c) 1997 John Wiley & Sons, Inc. Biotechnol Bioeng 55: 317-327, 1997.

  15. Robust Estimation for Discrete Markov System with Time-Varying Delay and Missing Measurements

    Directory of Open Access Journals (Sweden)

    Jia You

    2013-01-01

    Full Text Available This paper addresses the ℋ∞ filtering problem for time-delayed Markov jump systems (MJSs with intermittent measurements. Within network environment, missing measurements are taken into account, since the communication channel is supposed to be imperfect. A Bernoulli process is utilized to describe the phenomenon of the missing measurements. The original system is transformed into an input-output form consisting of two interconnected subsystems. Based on scaled small gain (SSG theorem and proposed Lyapunov-Krasovskii functional (LKF, the scaled small gains of the subsystems are analyzed, respectively. New conditions for the existence of the ℋ∞ filters are established, and the corresponding ℋ∞ filter design scheme is proposed. Finally, a simulation example is provided to demonstrate the effectiveness of the proposed approach.

  16. Optimal control for unknown discrete-time nonlinear Markov jump systems using adaptive dynamic programming.

    Science.gov (United States)

    Zhong, Xiangnan; He, Haibo; Zhang, Huaguang; Wang, Zhanshan

    2014-12-01

    In this paper, we develop and analyze an optimal control method for a class of discrete-time nonlinear Markov jump systems (MJSs) with unknown system dynamics. Specifically, an identifier is established for the unknown systems to approximate system states, and an optimal control approach for nonlinear MJSs is developed to solve the Hamilton-Jacobi-Bellman equation based on the adaptive dynamic programming technique. We also develop detailed stability analysis of the control approach, including the convergence of the performance index function for nonlinear MJSs and the existence of the corresponding admissible control. Neural network techniques are used to approximate the proposed performance index function and the control law. To demonstrate the effectiveness of our approach, three simulation studies, one linear case, one nonlinear case, and one single link robot arm case, are used to validate the performance of the proposed optimal control method.

  17. KOMPARASI KINERJA FUZZY TIME SERIES DENGAN MODEL RANTAI MARKOV DALAM MERAMALKAN PRODUK DOMESTIK REGIONAL BRUTO BALI

    Directory of Open Access Journals (Sweden)

    I MADE ARYA ANTARA

    2015-02-01

    Full Text Available This paper aimed to elaborates and compares the performance of Fuzzy Time Series (FTS model with Markov Chain (MC model in forecasting the Gross Regional Domestic Product (GDRP of Bali Province.  Both methods were considered as forecasting methods in soft modeling domain.  The data used was quarterly data of Bali’s GDRP for year 1992 through 2013 from Indonesian Bureau of Statistic at Denpasar Office.  Inspite of using the original data, rate of change from two consecutive quarters was used to model. From the in-sample forecasting conducted, we got the Average Forecas­ting Error Rate (AFER for FTS dan MC models as much as 0,78 percent and 2,74 percent, respec­tively.  Based-on these findings, FTS outperformed MC in in-sample forecasting for GDRP of Bali’s data.

  18. The critical node problem in stochastic networks with discrete-time Markov chain

    Directory of Open Access Journals (Sweden)

    Gholam Hassan Shirdel

    2016-04-01

    Full Text Available The length of the stochastic shortest path is defined as the arrival probability from a source node to a destination node. The uncertainty of the network topology causes unstable connections between nodes. A discrete-time Markov chain is devised according to the uniform distribution of existing arcs where the arrival probability is computed as a finite transition probability from the initial state to the absorbing state. Two situations are assumed, departing from the current state to a new state, or waiting in the current state while expecting better conditions. Our goal is to contribute to determining the critical node in a stochastic network, where its absence results in the greatest decrease of the arrival probability. The proposed method is a simply application for analyzing the resistance of networks against congestion and provides some crucial information of the individual nodes. Finally, this is illustrated using networks of various topologies.

  19. Markov chains models, algorithms and applications

    CERN Document Server

    Ching, Wai-Ki; Ng, Michael K; Siu, Tak-Kuen

    2013-01-01

    This new edition of Markov Chains: Models, Algorithms and Applications has been completely reformatted as a text, complete with end-of-chapter exercises, a new focus on management science, new applications of the models, and new examples with applications in financial risk management and modeling of financial data.This book consists of eight chapters.  Chapter 1 gives a brief introduction to the classical theory on both discrete and continuous time Markov chains. The relationship between Markov chains of finite states and matrix theory will also be highlighted. Some classical iterative methods

  20. Likert pain score modeling: a Markov integer model and an autoregressive continuous model.

    Science.gov (United States)

    Plan, E L; Elshoff, J-P; Stockis, A; Sargentini-Maier, M L; Karlsson, M O

    2012-05-01

    Pain intensity is principally assessed using rating scales such as the 11-point Likert scale. In general, frequent pain assessments are serially correlated and underdispersed. The aim of this investigation was to develop population models adapted to fit the 11-point pain scale. Daily Likert scores were recorded over 18 weeks by 231 patients with neuropathic pain from a clinical trial placebo group. An integer model consisting of a truncated generalized Poisson (GP) distribution with Markovian transition probability inflation was implemented in NONMEM 7.1.0. It was compared to a logit-transformed autoregressive continuous model with correlated residual errors. In both models, the score baseline was estimated to be 6.2 and the placebo effect to be 19%. Developed models similarly retrieved consistent underlying features of the data and therefore correspond to platform models for drug effect detection. The integer model was complex but flexible, whereas the continuous model can more easily be developed, although requires longer runtimes.

  1. Evolutionary Trees can be Learned in Polynomial-Time in the Two-State General Markov Model

    DEFF Research Database (Denmark)

    Cryan, Mary; Goldberg, Leslie Ann; Goldberg, Paul Wilfred

    2001-01-01

    The j-state general Markov model of evolution (due to Steel) is a stochastic model concerned with the evolution of strings over an alphabet of size j. In particular, the two-state general Markov model of evolution generalizes the well-known Cavender-Farris-Neyman model of evolution by removing......--Farris--Neyman model provided that the target tree satisfies the additional restriction that all pairs of leaves have a sufficiently high probability of being the same. We show how to remove both restrictions and thereby obtain the first polynomial-time PAC-learning algorithm (in the sense of Kearns et al...... the symmetry restriction (which requires that the probability that a "0" turns into a "1" along an edge is the same as the probability that a "1" turns into a "0" along the edge). Farach and Kannan showed how to probably approximately correct (PAC)-learn Markov evolutionary trees in the Cavender...

  2. Evolutionary Trees can be Learned in Polynomial-Time in the Two-State General Markov Model

    DEFF Research Database (Denmark)

    Cryan, Mary; Goldberg, Leslie Ann; Goldberg, Paul Wilfred

    2001-01-01

    The j-state general Markov model of evolution (due to Steel) is a stochastic model concerned with the evolution of strings over an alphabet of size j. In particular, the two-state general Markov model of evolution generalizes the well-known Cavender-Farris-Neyman model of evolution by removing...... the symmetry restriction (which requires that the probability that a "0" turns into a "1" along an edge is the same as the probability that a "1" turns into a "0" along the edge). Farach and Kannan showed how to probably approximately correct (PAC)-learn Markov evolutionary trees in the Cavender......--Farris--Neyman model provided that the target tree satisfies the additional restriction that all pairs of leaves have a sufficiently high probability of being the same. We show how to remove both restrictions and thereby obtain the first polynomial-time PAC-learning algorithm (in the sense of Kearns et al...

  3. Extending Markov Automata with State and Action Rewards

    NARCIS (Netherlands)

    Guck, Dennis; Timmer, Mark; Blom, Stefan; Bertrand, N.; Bortolussi, L.

    2014-01-01

    This presentation introduces the Markov Reward Automaton (MRA), an extension of the Markov automaton that allows the modelling of systems incorporating rewards in addition to nondeterminism, discrete probabilistic choice and continuous stochastic timing. Our models support both rewards that are acqu

  4. Performance evaluation:= (process algebra + model checking) x Markov chains

    NARCIS (Netherlands)

    Hermanns, H.; Katoen, J.P.; Larsen, Kim G.; Nielsen, Mogens

    2001-01-01

    Markov chains are widely used in practice to determine system performance and reliability characteristics. The vast majority of applications considers continuous-time Markov chains (CTMCs). This tutorial paper shows how successful model specification and analysis techniques from concurrency theory c

  5. Real-time network security situation visualization and threat assessment based on semi-Markov process

    Science.gov (United States)

    Chen, Junhua

    2013-03-01

    To cope with a large amount of data in current sensed environments, decision aid tools should provide their understanding of situations in a time-efficient manner, so there is an increasing need for real-time network security situation awareness and threat assessment. In this study, the state transition model of vulnerability in the network based on semi-Markov process is proposed at first. Once events are triggered by an attacker's action or system response, the current states of the vulnerabilities are known. Then we calculate the transition probabilities of the vulnerability from the current state to security failure state. Furthermore in order to improve accuracy of our algorithms, we adjust the probabilities that they exploit the vulnerability according to the attacker's skill level. In the light of the preconditions and post-conditions of vulnerabilities in the network, attack graph is built to visualize security situation in real time. Subsequently, we predict attack path, recognize attack intention and estimate the impact through analysis of attack graph. These help administrators to insight into intrusion steps, determine security state and assess threat. Finally testing in a network shows that this method is reasonable and feasible, and can undertake tremendous analysis task to facilitate administrators' work.

  6. Polynomial Time Algorithms for Branching Markov Decision Processes and Probabilistic Min(Max) Polynomial Bellman Equations

    CERN Document Server

    Etessami, Kousha; Yannakakis, Mihalis

    2012-01-01

    We show that one can approximate the least fixed point solution for a multivariate system of monotone probabilistic max(min) polynomial equations, referred to as maxPPSs (and minPPSs, respectively), in time polynomial in both the encoding size of the system of equations and in log(1/epsilon), where epsilon > 0 is the desired additive error bound of the solution. (The model of computation is the standard Turing machine model.) We establish this result using a generalization of Newton's method which applies to maxPPSs and minPPSs, even though the underlying functions are only piecewise-differentiable. This generalizes our recent work which provided a P-time algorithm for purely probabilistic PPSs. These equations form the Bellman optimality equations for several important classes of infinite-state Markov Decision Processes (MDPs). Thus, as a corollary, we obtain the first polynomial time algorithms for computing to within arbitrary desired precision the optimal value vector for several classes of infinite-state...

  7. Hidden Markov induced Dynamic Bayesian Network for recovering time evolving gene regulatory networks

    Science.gov (United States)

    Zhu, Shijia; Wang, Yadong

    2015-12-01

    Dynamic Bayesian Networks (DBN) have been widely used to recover gene regulatory relationships from time-series data in computational systems biology. Its standard assumption is ‘stationarity’, and therefore, several research efforts have been recently proposed to relax this restriction. However, those methods suffer from three challenges: long running time, low accuracy and reliance on parameter settings. To address these problems, we propose a novel non-stationary DBN model by extending each hidden node of Hidden Markov Model into a DBN (called HMDBN), which properly handles the underlying time-evolving networks. Correspondingly, an improved structural EM algorithm is proposed to learn the HMDBN. It dramatically reduces searching space, thereby substantially improving computational efficiency. Additionally, we derived a novel generalized Bayesian Information Criterion under the non-stationary assumption (called BWBIC), which can help significantly improve the reconstruction accuracy and largely reduce over-fitting. Moreover, the re-estimation formulas for all parameters of our model are derived, enabling us to avoid reliance on parameter settings. Compared to the state-of-the-art methods, the experimental evaluation of our proposed method on both synthetic and real biological data demonstrates more stably high prediction accuracy and significantly improved computation efficiency, even with no prior knowledge and parameter settings.

  8. Hidden Markov induced Dynamic Bayesian Network for recovering time evolving gene regulatory networks.

    Science.gov (United States)

    Zhu, Shijia; Wang, Yadong

    2015-12-18

    Dynamic Bayesian Networks (DBN) have been widely used to recover gene regulatory relationships from time-series data in computational systems biology. Its standard assumption is 'stationarity', and therefore, several research efforts have been recently proposed to relax this restriction. However, those methods suffer from three challenges: long running time, low accuracy and reliance on parameter settings. To address these problems, we propose a novel non-stationary DBN model by extending each hidden node of Hidden Markov Model into a DBN (called HMDBN), which properly handles the underlying time-evolving networks. Correspondingly, an improved structural EM algorithm is proposed to learn the HMDBN. It dramatically reduces searching space, thereby substantially improving computational efficiency. Additionally, we derived a novel generalized Bayesian Information Criterion under the non-stationary assumption (called BWBIC), which can help significantly improve the reconstruction accuracy and largely reduce over-fitting. Moreover, the re-estimation formulas for all parameters of our model are derived, enabling us to avoid reliance on parameter settings. Compared to the state-of-the-art methods, the experimental evaluation of our proposed method on both synthetic and real biological data demonstrates more stably high prediction accuracy and significantly improved computation efficiency, even with no prior knowledge and parameter settings.

  9. Aperiodicity in one-way Markov cycles and repeat times of large earthquakes in faults

    CERN Document Server

    Tejedor, Alejandro; Pacheco, Amalio F

    2011-01-01

    A common use of Markov Chains is the simulation of the seismic cycle in a fault, i.e. as a renewal model for the repetition of its characteristic earthquakes. This representation is consistent with Reid's elastic rebound theory. Here it is proved that in {\\it any} one-way Markov cycle, the aperiodicity of the corresponding distribution of cycle lengths is always lower than one. This fact concurs with observations of large earthquakes in faults all over the world.

  10. Cooperation and Coordination Between Fuzzy Reinforcement Learning Agents in Continuous State Partially Observable Markov Decision Processes

    Science.gov (United States)

    Berenji, Hamid R.; Vengerov, David

    1999-01-01

    Successful operations of future multi-agent intelligent systems require efficient cooperation schemes between agents sharing learning experiences. We consider a pseudo-realistic world in which one or more opportunities appear and disappear in random locations. Agents use fuzzy reinforcement learning to learn which opportunities are most worthy of pursuing based on their promise rewards, expected lifetimes, path lengths and expected path costs. We show that this world is partially observable because the history of an agent influences the distribution of its future states. We consider a cooperation mechanism in which agents share experience by using and-updating one joint behavior policy. We also implement a coordination mechanism for allocating opportunities to different agents in the same world. Our results demonstrate that K cooperative agents each learning in a separate world over N time steps outperform K independent agents each learning in a separate world over K*N time steps, with this result becoming more pronounced as the degree of partial observability in the environment increases. We also show that cooperation between agents learning in the same world decreases performance with respect to independent agents. Since cooperation reduces diversity between agents, we conclude that diversity is a key parameter in the trade off between maximizing utility from cooperation when diversity is low and maximizing utility from competitive coordination when diversity is high.

  11. Markov bridges, bisection and variance reduction

    DEFF Research Database (Denmark)

    Asmussen, Søren; Hobolth, Asger

    Time-continuous Markov jump processes is a popular modelling tool in disciplines ranging from computational finance and operations research to human genetics and genomics. The data is often sampled at discrete points in time, and it can be useful to simulate sample paths between the datapoints....... In this paper we firstly consider the problem of generating sample paths from a continuous-time Markov chain conditioned on the endpoints using a new algorithm based on the idea of bisection. Secondly we study the potential of the bisection algorithm for variance reduction. In particular, examples are presented...... where the methods of stratification, importance sampling and quasi Monte Carlo are investigated....

  12. Detecting synchronization clusters in multivariate time series via coarse-graining of Markov chains.

    Science.gov (United States)

    Allefeld, Carsten; Bialonski, Stephan

    2007-12-01

    Synchronization cluster analysis is an approach to the detection of underlying structures in data sets of multivariate time series, starting from a matrix R of bivariate synchronization indices. A previous method utilized the eigenvectors of R for cluster identification, analogous to several recent attempts at group identification using eigenvectors of the correlation matrix. All of these approaches assumed a one-to-one correspondence of dominant eigenvectors and clusters, which has however been shown to be wrong in important cases. We clarify the usefulness of eigenvalue decomposition for synchronization cluster analysis by translating the problem into the language of stochastic processes, and derive an enhanced clustering method harnessing recent insights from the coarse-graining of finite-state Markov processes. We illustrate the operation of our method using a simulated system of coupled Lorenz oscillators, and we demonstrate its superior performance over the previous approach. Finally we investigate the question of robustness of the algorithm against small sample size, which is important with regard to field applications.

  13. Trajectory Learning for Robot Programming by Demonstration Using Hidden Markov Model and Dynamic Time Warping.

    Science.gov (United States)

    Vakanski, A; Mantegh, I; Irish, A; Janabi-Sharifi, F

    2012-08-01

    The main objective of this paper is to develop an efficient method for learning and reproduction of complex trajectories for robot programming by demonstration. Encoding of the demonstrated trajectories is performed with hidden Markov model, and generation of a generalized trajectory is achieved by using the concept of key points. Identification of the key points is based on significant changes in position and velocity in the demonstrated trajectories. The resulting sequences of trajectory key points are temporally aligned using the multidimensional dynamic time warping algorithm, and a generalized trajectory is obtained by smoothing spline interpolation of the clustered key points. The principal advantage of our proposed approach is utilization of the trajectory key points from all demonstrations for generation of a generalized trajectory. In addition, variability of the key points' clusters across the demonstrated set is employed for assigning weighting coefficients, resulting in a generalization procedure which accounts for the relevance of reproduction of different parts of the trajectories. The approach is verified experimentally for trajectories with two different levels of complexity.

  14. Continuous-Time Modeling with Spatial Dependence

    NARCIS (Netherlands)

    Oud, J.H.L.; Folmer, H.; Patuelli, R.; Nijkamp, P.

    2012-01-01

    (Spatial) panel data are routinely modeled in discrete time (DT). However, compelling arguments exist for continuous-time (CT) modeling of (spatial) panel data. Particularly, most social processes evolve in CT, so that statistical analysis in DT is an oversimplification, gives an incomplete

  15. Continuous-Time Modeling with Spatial Dependence

    NARCIS (Netherlands)

    Oud, J.; Folmer, H.; Patuelli, R.; Nijkamp, P.

    (Spatial) panel data are routinely modeled in discrete time (DT). However, compelling arguments exist for continuous-time (CT) modeling of (spatial) panel data. Particularly, most social processes evolve in CT, so that statistical analysis in DT is an oversimplification, gives an incomplete

  16. Introduction to the numerical solutions of Markov chains

    CERN Document Server

    Stewart, Williams J

    1994-01-01

    A cornerstone of applied probability, Markov chains can be used to help model how plants grow, chemicals react, and atoms diffuse - and applications are increasingly being found in such areas as engineering, computer science, economics, and education. To apply the techniques to real problems, however, it is necessary to understand how Markov chains can be solved numerically. In this book, the first to offer a systematic and detailed treatment of the numerical solution of Markov chains, William Stewart provides scientists on many levels with the power to put this theory to use in the actual world, where it has applications in areas as diverse as engineering, economics, and education. His efforts make for essential reading in a rapidly growing field. Here, Stewart explores all aspects of numerically computing solutions of Markov chains, especially when the state is huge. He provides extensive background to both discrete-time and continuous-time Markov chains and examines many different numerical computing metho...

  17. Time-patterns of antibiotic exposure in poultry production--a Markov chains exploratory study of nature and consequences.

    Science.gov (United States)

    Chauvin, C; Clement, C; Bruneau, M; Pommeret, D

    2007-07-16

    This article describes the use of Markov chains to explore the time-patterns of antimicrobial exposure in broiler poultry. The transition in antimicrobial exposure status (exposed/not exposed to an antimicrobial, with a distinction between exposures to the different antimicrobial classes) in extensive data collected in broiler chicken flocks from November 2003 onwards, was investigated. All Markov chains were first-order chains. Mortality rate, geographical location and slaughter semester were sources of heterogeneity between transition matrices. Transitions towards a 'no antimicrobial' exposure state were highly predominant, whatever the initial state. From a 'no antimicrobial' exposure state, the transition to beta-lactams was predominant among transitions to an antimicrobial exposure state. Transitions between antimicrobial classes were rare and variable. Switches between antimicrobial classes and repeats of a particular class were both observed. Application of Markov chains analysis to the database of the nation-wide antimicrobial resistance monitoring programme pointed out that transition probabilities between antimicrobial exposure states increased with the number of resistances in Escherichia coli strains.

  18. Markov Models in health care

    Directory of Open Access Journals (Sweden)

    Renato Cesar Sato

    2010-09-01

    Full Text Available Markov Chains provide support for problems involving decision on uncertainties through a continuous period of time. The greater availability and access to processing power through computers allow that these models can be used more often to represent clinical structures. Markov models consider the patients in a discrete state of health, and the events represent the transition from one state to another. The possibility of modeling repetitive events and time dependence of probabilities and utilities associated permits a more accurate representation of the evaluated clinical structure. These templates can be used for economic evaluation in health care taking into account the evaluation of costs and clinical outcomes, especially for evaluation of chronic diseases. This article provides a review of the use of modeling within the clinical context and the advantages of the possibility of including time for this type of study.

  19. Steady states of continuous-time open quantum walks

    Science.gov (United States)

    Liu, Chaobin; Balu, Radhakrishnan

    2017-07-01

    Continuous-time open quantum walks (CTOQW) are introduced as the formulation of quantum dynamical semigroups of trace-preserving and completely positive linear maps (or quantum Markov semigroups) on graphs. We show that a CTOQW always converges to a steady state regardless of the initial state when a graph is connected. When the graph is both connected and regular, it is shown that the steady state is the maximally mixed state. As shown by the examples in this article, the steady states of CTOQW can be very unusual and complicated even though the underlying graphs are simple. The examples demonstrate that the structure of a graph can affect quantum coherence in CTOQW through a long-time run. Precisely, the quantum coherence persists throughout the evolution of the CTOQW when the underlying topology is certain irregular graphs (such as a path or a star as shown in the examples). In contrast, the quantum coherence will eventually vanish from the open quantum system when the underlying topology is a regular graph (such as a cycle).

  20. Parameter Estimation in Continuous Time Domain

    Directory of Open Access Journals (Sweden)

    Gabriela M. ATANASIU

    2016-12-01

    Full Text Available This paper will aim to presents the applications of a continuous-time parameter estimation method for estimating structural parameters of a real bridge structure. For the purpose of illustrating this method two case studies of a bridge pile located in a highly seismic risk area are considered, for which the structural parameters for the mass, damping and stiffness are estimated. The estimation process is followed by the validation of the analytical results and comparison with them to the measurement data. Further benefits and applications for the continuous-time parameter estimation method in civil engineering are presented in the final part of this paper.

  1. Stochastic Games for Continuous-Time Jump Processes Under Finite-Horizon Payoff Criterion

    Energy Technology Data Exchange (ETDEWEB)

    Wei, Qingda, E-mail: weiqd@hqu.edu.cn [Huaqiao University, School of Economics and Finance (China); Chen, Xian, E-mail: chenxian@amss.ac.cn [Peking University, School of Mathematical Sciences (China)

    2016-10-15

    In this paper we study two-person nonzero-sum games for continuous-time jump processes with the randomized history-dependent strategies under the finite-horizon payoff criterion. The state space is countable, and the transition rates and payoff functions are allowed to be unbounded from above and from below. Under the suitable conditions, we introduce a new topology for the set of all randomized Markov multi-strategies and establish its compactness and metrizability. Then by constructing the approximating sequences of the transition rates and payoff functions, we show that the optimal value function for each player is a unique solution to the corresponding optimality equation and obtain the existence of a randomized Markov Nash equilibrium. Furthermore, we illustrate the applications of our main results with a controlled birth and death system.

  2. Time, physics, and the paradoxes of continuity

    CERN Document Server

    Steinberg, D A

    2003-01-01

    A recent article in this journal proposes a radical reformulation of classical and quantum dynamics based on a perceived deficiency in current definitions of time. The argument is incorrect but the errors highlight aspects of the foundations of mathematics and physics that are commonly confused and misunderstood. For this reason, the article provides an important and heuristic opportunity to reexamine the types of time and non-standard analysis. This paper will discuss the differences between physical time and experiential time and explain how an expanded system of real analysis containing infinitesimals can resolve the paradoxes of continuity without sacrificing the modern edifice of mathematical physics.

  3. Mixed continuous/discrete time modelling with exact time adjustments

    NARCIS (Netherlands)

    Rovers, K.C.; Kuper, Jan; van de Burgwal, M.D.; Kokkeler, Andre B.J.; Smit, Gerardus Johannes Maria

    2011-01-01

    Many systems interact with their physical environment. Design of such systems need a modelling and simulation tool which can deal with both the continuous and discrete aspects. However, most current tools are not adequately able to do so, as they implement both continuous and discrete time signals

  4. Continuous, real time microwave plasma element sensor

    Science.gov (United States)

    Woskov, Paul P.; Smatlak, Donna L.; Cohn, Daniel R.; Wittle, J. Kenneth; Titus, Charles H.; Surma, Jeffrey E.

    1995-01-01

    Microwave-induced plasma for continuous, real time trace element monitoring under harsh and variable conditions. The sensor includes a source of high power microwave energy and a shorted waveguide made of a microwave conductive, refractory material communicating with the source of the microwave energy to generate a plasma. The high power waveguide is constructed to be robust in a hot, hostile environment. It includes an aperture for the passage of gases to be analyzed and a spectrometer is connected to receive light from the plasma. Provision is made for real time in situ calibration. The spectrometer disperses the light, which is then analyzed by a computer. The sensor is capable of making continuous, real time quantitative measurements of desired elements, such as the heavy metals lead and mercury.

  5. Markov processes an introduction for physical scientists

    CERN Document Server

    Gillespie, Daniel T

    1991-01-01

    Markov process theory is basically an extension of ordinary calculus to accommodate functions whos time evolutions are not entirely deterministic. It is a subject that is becoming increasingly important for many fields of science. This book develops the single-variable theory of both continuous and jump Markov processes in a way that should appeal especially to physicists and chemists at the senior and graduate level.Key Features* A self-contained, prgamatic exposition of the needed elements of random variable theory* Logically integrated derviations of the Chapman-Kolmogorov e

  6. Waiting time distribution for continuous stochastic systems.

    Science.gov (United States)

    Gernert, Robert; Emary, Clive; Klapp, Sabine H L

    2014-12-01

    The waiting time distribution (WTD) is a common tool for analyzing discrete stochastic processes in classical and quantum systems. However, there are many physical examples where the dynamics is continuous and only approximately discrete, or where it is favourable to discuss the dynamics on a discretized and a continuous level in parallel. An example is the hindered motion of particles through potential landscapes with barriers. In the present paper we propose a consistent generalization of the WTD from the discrete case to situations where the particles perform continuous barrier crossing characterized by a finite duration. To this end, we introduce a recipe to calculate the WTD from the Fokker-Planck (Smoluchowski) equation. In contrast to the closely related first passage time distribution (FPTD), which is frequently used to describe continuous processes, the WTD contains information about the direction of motion. As an application, we consider the paradigmatic example of an overdamped particle diffusing through a washboard potential. To verify the approach and to elucidate its numerical implications, we compare the WTD defined via the Smoluchowski equation with data from direct simulation of the underlying Langevin equation and find full consistency provided that the jumps in the Langevin approach are defined properly. Moreover, for sufficiently large energy barriers, the WTD defined via the Smoluchowski equation becomes consistent with that resulting from the analytical solution of a (two-state) master equation model for the short-time dynamics developed previously by us [Phys. Rev. E 86, 061135 (2012)]. Thus, our approach "interpolates" between these two types of stochastic motion. We illustrate our approach for both symmetric systems and systems under constant force.

  7. Combined risk assessment of nonstationary monthly water quality based on Markov chain and time-varying copula.

    Science.gov (United States)

    Shi, Wei; Xia, Jun

    2017-02-01

    Water quality risk management is a global hot research linkage with the sustainable water resource development. Ammonium nitrogen (NH3-N) and permanganate index (CODMn) as the focus indicators in Huai River Basin, are selected to reveal their joint transition laws based on Markov theory. The time-varying moments model with either time or land cover index as explanatory variables is applied to build the time-varying marginal distributions of water quality time series. Time-varying copula model, which takes the non-stationarity in the marginal distribution and/or the time variation in dependence structure between water quality series into consideration, is constructed to describe a bivariate frequency analysis for NH3-N and CODMn series at the same monitoring gauge. The larger first-order Markov joint transition probability indicates water quality state Class Vw, Class IV and Class III will occur easily in the water body of Bengbu Sluice. Both marginal distribution and copula models are nonstationary, and the explanatory variable time yields better performance than land cover index in describing the non-stationarities in the marginal distributions. In modelling the dependence structure changes, time-varying copula has a better fitting performance than the copula with the constant or the time-trend dependence parameter. The largest synchronous encounter risk probability of NH3-N and CODMn simultaneously reaching Class V is 50.61%, while the asynchronous encounter risk probability is largest when NH3-N and CODMn is inferior to class V and class IV water quality standards, respectively.

  8. Modeling Uncertainty of Directed Movement via Markov Chains

    Directory of Open Access Journals (Sweden)

    YIN Zhangcai

    2015-10-01

    Full Text Available Probabilistic time geography (PTG is suggested as an extension of (classical time geography, in order to present the uncertainty of an agent located at the accessible position by probability. This may provide a quantitative basis for most likely finding an agent at a location. In recent years, PTG based on normal distribution or Brown bridge has been proposed, its variance, however, is irrelevant with the agent's speed or divergent with the increase of the speed; so they are difficult to take into account application pertinence and stability. In this paper, a new method is proposed to model PTG based on Markov chain. Firstly, a bidirectional conditions Markov chain is modeled, the limit of which, when the moving speed is large enough, can be regarded as the Brown bridge, thus has the characteristics of digital stability. Then, the directed movement is mapped to Markov chains. The essential part is to build step length, the state space and transfer matrix of Markov chain according to the space and time position of directional movement, movement speed information, to make sure the Markov chain related to the movement speed. Finally, calculating continuously the probability distribution of the directed movement at any time by the Markov chains, it can be get the possibility of an agent located at the accessible position. Experimental results show that, the variance based on Markov chains not only is related to speed, but also is tending towards stability with increasing the agent's maximum speed.

  9. Discrete- and continuous-time probabilistic models and algorithms for inferring neuronal UP and DOWN states.

    Science.gov (United States)

    Chen, Zhe; Vijayan, Sujith; Barbieri, Riccardo; Wilson, Matthew A; Brown, Emery N

    2009-07-01

    UP and DOWN states, the periodic fluctuations between increased and decreased spiking activity of a neuronal population, are a fundamental feature of cortical circuits. Understanding UP-DOWN state dynamics is important for understanding how these circuits represent and transmit information in the brain. To date, limited work has been done on characterizing the stochastic properties of UP-DOWN state dynamics. We present a set of Markov and semi-Markov discrete- and continuous-time probability models for estimating UP and DOWN states from multiunit neural spiking activity. We model multiunit neural spiking activity as a stochastic point process, modulated by the hidden (UP and DOWN) states and the ensemble spiking history. We estimate jointly the hidden states and the model parameters by maximum likelihood using an expectation-maximization (EM) algorithm and a Monte Carlo EM algorithm that uses reversible-jump Markov chain Monte Carlo sampling in the E-step. We apply our models and algorithms in the analysis of both simulated multiunit spiking activity and actual multi- unit spiking activity recorded from primary somatosensory cortex in a behaving rat during slow-wave sleep. Our approach provides a statistical characterization of UP-DOWN state dynamics that can serve as a basis for verifying and refining mechanistic descriptions of this process.

  10. A scaling analysis of a cat and mouse Markov chain

    NARCIS (Netherlands)

    Litvak, Nelly; Robert, Philippe

    2012-01-01

    If ($C_n$) a Markov chain on a discrete state space $S$, a Markov chain ($C_n, M_n$) on the product space $S \\times S$, the cat and mouse Markov chain, is constructed. The first coordinate of this Markov chain behaves like the original Markov chain and the second component changes only when both coo

  11. On Discrete Time Control of Continuous Time Systems

    DEFF Research Database (Denmark)

    Poulsen, Niels Kjølstad

    of Denmark. The focus in this paper is control of a continuous time system by means of a digital control. In this context the control signal can only change at sample instants and is constant between samples. The cost function do include the variations of output between samples....

  12. Continuous GPS Carrier-Phase Time Transfer

    Science.gov (United States)

    Yao, Jian

    Time transfer (TT) is the process of transmitting a timing signal from one place to another place. It has applications to the formation and realization of Coordinated Universal Time (UTC), telecommunications, electrical power grids, and even stock exchanges. TT is the actual bottleneck of the UTC formation and realization since the technology of atomic clocks is almost always ahead of that of TT. GPS carrier-phase time transfer (GPSCPTT), as a mainstream TT technique accepted by most national timing laboratories, has suffered from the day-boundary-discontinuity (day-BD) problem for many years. This makes us difficult to observe a remote Cesium fountain clock behavior even after a few days. We find that day-BD comes from the GPS code noise. The day-BD can be lowered by ˜40% if more satellite-clock information is provided and if a few GPS receivers at the same station are averaged. To completely eliminate day-BD, the RINEX-Shift (RS) and revised RS (RRS) algorithms have been designed. The RS/RRS result matches the two-way satellite time/frequency transfer (TWSTFT) result much better than the conventional GPSCPTT result. With the RS/RRS algorithm, we are able to observe a remote Cesium fountain after half a day. We also study the BD due to GPS data anomalies (anomaly-BD). A simple curve-fitting strategy can eliminate the anomaly-BD. Thus, we achieve continuous GPSCPTT after eliminating both day-BD and anomaly-BD.

  13. A First-Order Non-Homogeneous Markov Model for the Response of Spiking Neurons Stimulated by Small Phase-Continuous Signals

    CERN Document Server

    Tapson, J; van Schaik, A; Etienne-Cummings, R

    2008-01-01

    We present a first-order non-homogeneous Markov model for the interspike-interval density of a continuously stimulated spiking neuron. The model allows the conditional interspike-interval density and the stationary interspike-interval density to be expressed as products of two separate functions, one of which describes only the neuron characteristics, and the other of which describes only the signal characteristics. This allows the use of this model to predict the response when the underlying neuron model is not known or well determined. The approximation shows particularly clearly that signal autocorrelations and cross-correlations arise as natural features of the interspike-interval density, and are particularly clear for small signals and moderate noise. We show that this model simplifies the design of spiking neuron cross-correlation systems, and describe a four-neuron mutual inhibition network that generates a cross-correlation output for two input signals.

  14. Real-time Gaussian Markov random-field-based ground tracking for ground penetrating radar data

    Science.gov (United States)

    Bradbury, Kyle; Torrione, Peter A.; Collins, Leslie

    2009-05-01

    Current ground penetrating radar algorithms for landmine detection require accurate estimates of the location of the air/ground interface to maintain high levels of performance. However, the presence of surface clutter, natural soil roughness, and antenna motion lead to uncertainty in these estimates. Previous work on improving estimates of the location of the air/ground interface have focused on one-dimensional filtering techniques to localize the air/ground interface. In this work, we propose an algorithm for interface localization using a 2- D Gaussian Markov random field (GMRF). The GMRF provides a statistical model of the surface structure, which enables the application of statistical optimization techniques. In this work, the ground location is inferred using iterated conditional modes (ICM) optimization which maximizes the conditional pseudo-likelihood of the GMRF at a point, conditioned on its neighbors. To illustrate the efficacy of the proposed interface localization approach, pre-screener performance with and without the proposed ground localization algorithm is compared. We show that accurate localization of the air/ground interface provides the potential for future performance improvements.

  15. Continuous-Time Bilinear System Identification

    Science.gov (United States)

    Juang, Jer-Nan

    2003-01-01

    The objective of this paper is to describe a new method for identification of a continuous-time multi-input and multi-output bilinear system. The approach is to make judicious use of the linear-model properties of the bilinear system when subjected to a constant input. Two steps are required in the identification process. The first step is to use a set of pulse responses resulting from a constant input of one sample period to identify the state matrix, the output matrix, and the direct transmission matrix. The second step is to use another set of pulse responses with the same constant input over multiple sample periods to identify the input matrix and the coefficient matrices associated with the coupling terms between the state and the inputs. Numerical examples are given to illustrate the concept and the computational algorithm for the identification method.

  16. Greenhouse Modeling Using Continuous Timed Petri Nets

    Directory of Open Access Journals (Sweden)

    José Luis Tovany

    2013-01-01

    Full Text Available This paper presents a continuous timed Petri nets (ContPNs based greenhouse modeling methodology. The presented methodology is based on the definition of elementary ContPN modules which are designed to capture the components of a general energy and mass balance differential equation, like parts that are reducing or increasing variables, such as heat, CO2 concentration, and humidity. The semantics of ContPN is also extended in order to deal with variables depending on external greenhouse variables, such as solar radiation. Each external variable is represented by a place whose marking depends on an a priori known function, for instance, the solar radiation function of the greenhouse site, which can be obtained statistically. The modeling methodology is illustrated with a greenhouse modeling example.

  17. Continuous Time Group Discovery in Dynamic Graphs

    Energy Technology Data Exchange (ETDEWEB)

    Miller, K; Eliassi-Rad, T

    2010-11-04

    With the rise in availability and importance of graphs and networks, it has become increasingly important to have good models to describe their behavior. While much work has focused on modeling static graphs, we focus on group discovery in dynamic graphs. We adapt a dynamic extension of Latent Dirichlet Allocation to this task and demonstrate good performance on two datasets. Modeling relational data has become increasingly important in recent years. Much work has focused on static graphs - that is fixed graphs at a single point in time. Here we focus on the problem of modeling dynamic (i.e. time-evolving) graphs. We propose a scalable Bayesian approach for community discovery in dynamic graphs. Our approach is based on extensions of Latent Dirichlet Allocation (LDA). LDA is a latent variable model for topic modeling in text corpora. It was extended to deal with topic changes in discrete time and later in continuous time. These models were referred to as the discrete Dynamic Topic Model (dDTM) and the continuous Dynamic Topic Model (cDTM), respectively. When adapting these models to graphs, we take our inspiration from LDA-G and SSN-LDA, applications of LDA to static graphs that have been shown to effectively factor out community structure to explain link patterns in graphs. In this paper, we demonstrate how to adapt and apply the cDTM to the task of finding communities in dynamic networks. We use link prediction to measure the quality of the discovered community structure and apply it to two different relational datasets - DBLP author-keyword and CAIDA autonomous systems relationships. We also discuss a parallel implementation of this approach using Hadoop. In Section 2, we review LDA and LDA-G. In Section 3, we review the cDTM and introduce cDTMG, its adaptation to modeling dynamic graphs. We discuss inference for the cDTM-G and details of our parallel implementation in Section 4 and present its performance on two datasets in Section 5 before concluding in

  18. Statistical inference for critical continuous state and continuous time branching processes with immigration

    OpenAIRE

    2014-01-01

    We study asymptotic behavior of conditional least squares estimators for critical continuous state and continuous time branching processes with immigration based on discrete time (low frequency) observations.

  19. Real-Time Prediction of Gamers Behavior Using Variable Order Markov and Big Data Technology: A Case of Study

    Directory of Open Access Journals (Sweden)

    Alejandro Baldominos Gómez

    2016-03-01

    Full Text Available This paper presents the results and conclusions found when predicting the behavior of gamers in commercial videogames datasets. In particular, it uses Variable-Order Markov (VOM to build a probabilistic model that is able to use the historic behavior of gamers and to infer what will be their next actions. Being able to predict with accuracy the next user’s actions can be of special interest to learn from the behavior of gamers, to make them more engaged and to reduce churn rate. In order to support a big volume and velocity of data, the system is built on top of the Hadoop ecosystem, using HBase for real-time processing; and the prediction tool is provided as a service (SaaS and accessible through a RESTful API. The prediction system is evaluated using a case of study with two commercial videogames, attaining promising results with high prediction accuracies.

  20. On Markov parameters in system identification

    Science.gov (United States)

    Phan, Minh; Juang, Jer-Nan; Longman, Richard W.

    1991-01-01

    A detailed discussion of Markov parameters in system identification is given. Different forms of input-output representation of linear discrete-time systems are reviewed and discussed. Interpretation of sampled response data as Markov parameters is presented. Relations between the state-space model and particular linear difference models via the Markov parameters are formulated. A generalization of Markov parameters to observer and Kalman filter Markov parameters for system identification is explained. These extended Markov parameters play an important role in providing not only a state-space realization, but also an observer/Kalman filter for the system of interest.

  1. Language Emptiness of Continuous-Time Parametric Timed Automata

    DEFF Research Database (Denmark)

    Benes, Nikola; Bezdek, Peter; Larsen, Kim Guldstrand

    2015-01-01

    Parametric timed automata extend the standard timed automata with the possibility to use parameters in the clock guards. In general, if the parameters are real-valued, the problem of language emptiness of such automata is undecidable even for various restricted subclasses. We thus focus on the case...... of these clocks is compared with (an arbitrary number of) parameters, we show that the parametric language emptiness is decidable. The undecidability result tightens the bounds of a previous result which assumed six parameters, while the decidability result extends the existing approaches that deal with discrete......-time semantics only. To the best of our knowledge, this is the first positive result in the case of continuous-time and unbounded integer parameters, except for the rather simple case of single-clock automata....

  2. ADM-CLE approach for detecting slow variables in continuous time Markov chains and dynamic data

    CERN Document Server

    Cucuringu, Mihai

    2015-01-01

    A method for detecting intrinsic slow variables in high-dimensional stochastic chemical reaction networks is developed and analyzed. It combines anisotropic diffusion maps (ADM) with approximations based on the chemical Langevin equation (CLE). The resulting approach, called ADM-CLE, has the potential of being more efficient than the ADM method for a large class of chemical reaction systems, because it replaces the computationally most expensive step of ADM (running local short bursts of simulations) by using an approximation based on the CLE. The ADM-CLE approach can be used to estimate the stationary distribution of the detected slow variable, without any a-priori knowledge of it. If the conditional distribution of the fast variables can be obtained analytically, then the resulting ADM-CLE approach does not make any use of Monte Carlo simulations to estimate the distributions of both slow and fast variables.

  3. Contraction Integrated Semigroups and Their Application to Continuous-time Markov Chains

    Institute of Scientific and Technical Information of China (English)

    Yang Rong LI

    2003-01-01

    We introduce the notion of the contraction integrated semigroups and give the LumerPhillips characterization of the generator, and also the charaterazied generators of isometric integratedsemigroups. For their application, a necessary and sufficient condition for q-matrices Q generatinga contraction integrated semigroup is given, and a necessary and sufficient condition for a transitionfunction to be a Feller Reuter-Riley transition function is also given in terms of its q-matrix.

  4. ADM-CLE Approach for Detecting Slow Variables in Continuous Time Markov Chains and Dynamic Data

    Science.gov (United States)

    2015-04-01

    Max Plank ...v a ri a b le s (d) 0 50 100 150 200 0 0.2 0.4 0.6 0.8 1 Our recovered slow variables (1:201) J a c c a rd S im ila ri ty I n d e x Max matching in...250 300 0 0.2 0.4 0.6 0.8 1 Our recovered slow variables (1:314) J a c c a rd S im ila ri ty I n d e x Max matching in the Jaccard index matrix

  5. Efficient Approximation of Optimal Control for Markov Games

    DEFF Research Database (Denmark)

    Fearnley, John; Rabe, Markus; Schewe, Sven

    2011-01-01

    We study the time-bounded reachability problem for continuous-time Markov decision processes (CTMDPs) and games (CTMGs). Existing techniques for this problem use discretisation techniques to break time into discrete intervals, and optimal control is approximated for each interval separately...

  6. Behavioral Portfolio Selection in Continuous Time

    CERN Document Server

    Jin, Hanqing

    2007-01-01

    This paper formulates and studies a general continuous-time behavioral portfolio selection model under Kahneman and Tversky's (cumulative) prospect theory, featuring S-shaped utility (value) functions and probability distortions. Unlike the conventional expected utility maximization model, such a behavioral model could be easily mis-formulated (a.k.a. ill-posed) if its different components do not coordinate well with each other. Certain classes of an ill-posed model are identified. A systematic approach, which is fundamentally different from the ones employed for the utility model, is developed to solve a well-posed model, assuming a complete market and general It\\^o processes for asset prices. The optimal terminal wealth positions, derived in fairly explicit forms, possess surprisingly simple structure reminiscent of a gambling policy betting on a good state of the world while accepting a fixed, known loss in case of a bad one. An example with a two-piece CRRA utility is presented to illustrate the general r...

  7. Continuous-time random walk and parametric subordination in fractional diffusion

    Energy Technology Data Exchange (ETDEWEB)

    Gorenflo, Rudolf [Department of Mathematics and Informatics, Free University of Berlin, Arnimallee 3, D-14195 Berlin (Germany); Mainardi, Francesco [Department of Physics, University of Bologna and INFN, Via Irnerio 46, I-40126 Bologna (Italy)]. E-mail: mainardi@bo.infn.it; Vivoli, Alessandro [Department of Physics, University of Bologna and INFN, Via Irnerio 46, I-40126 Bologna (Italy)

    2007-10-15

    The well-scaled transition to the diffusion limit in the framework of the theory of continuous-time random walk (CTRW) is presented starting from its representation as an infinite series that points out the subordinated character of the CTRW itself. We treat the CTRW as a combination of a random walk on the axis of physical time with a random walk in space, both walks happening in discrete operational time. In the continuum limit, we obtain a (generally non-Markovian) diffusion process governed by a space-time fractional diffusion equation. The essential assumption is that the probabilities for waiting times and jump-widths behave asymptotically like powers with negative exponents related to the orders of the fractional derivatives. By what we call parametric subordination, applied to a combination of a Markov process with a positively oriented Levy process, we generate and display sample paths for some special cases.

  8. Measuring Years of Inactivity, Years in Retirement, Time to Retirement, and Age at Retirement Within the Markov Model

    Science.gov (United States)

    SKOOG, GARY R.; CIECKA, JAMES E.

    2010-01-01

    Retirement-related concepts are treated as random variables within Markov process models that capture multiple labor force entries and exits. The expected number of years spent outside of the labor force, expected years in retirement, and expected age at retirement are computed—all of which are of immense policy interest but have been heretofore reported with less precisely measured proxies. Expected age at retirement varies directly with a person’s age; but even younger people can expect to retire at ages substantially older than those commonly associated with retirement, such as age 60, 62, or 65. Between 1970 and 2003, men allocated most of their increase in life expectancy to increased time in retirement, but women allocated most of their increased life expectancy to labor force activity. Although people can exit and reenter the labor force at older ages, most 65-year-old men who are active in the labor force will not reenter after they eventually exit. At age 65, the probability that those who are inactive will reenter the labor force at some future time is .38 for men and .27 for women. Life expectancy at exact ages is decomposed into the sum of the expected time spent active and inactive in the labor force, and also as the sum of the expected time to labor force separation and time in retirement. PMID:20879680

  9. Two-boundary first exit time of Gauss-Markov processes for stochastic modeling of acto-myosin dynamics.

    Science.gov (United States)

    D'Onofrio, Giuseppe; Pirozzi, Enrica

    2017-05-01

    We consider a stochastic differential equation in a strip, with coefficients suitably chosen to describe the acto-myosin interaction subject to time-varying forces. By simulating trajectories of the stochastic dynamics via an Euler discretization-based algorithm, we fit experimental data and determine the values of involved parameters. The steps of the myosin are represented by the exit events from the strip. Motivated by these results, we propose a specific stochastic model based on the corresponding time-inhomogeneous Gauss-Markov and diffusion process evolving between two absorbing boundaries. We specify the mean and covariance functions of the stochastic modeling process taking into account time-dependent forces including the effect of an external load. We accurately determine the probability density function (pdf) of the first exit time (FET) from the strip by solving a system of two non singular second-type Volterra integral equations via a numerical quadrature. We provide numerical estimations of the mean of FET as approximations of the dwell-time of the proteins dynamics. The percentage of backward steps is given in agreement to experimental data. Numerical and simulation results are compared and discussed.

  10. SU-E-J-115: Using Markov Chain Modeling to Elucidate Patterns in Breast Cancer Metastasis Over Time and Space

    Energy Technology Data Exchange (ETDEWEB)

    Comen, E; Mason, J; Kuhn, P [The Scripps Research Institute, La Jolla, CA (United States); Nieva, J [Billings Clinic, Billings, Montana (United States); Newton, P [University of Southern California, Los Angeles, CA (United States); Norton, L; Venkatappa, N; Jochelson, M [Memorial Sloan-Kettering Cancer Center, NY, NY (United States)

    2014-06-01

    Purpose: Traditionally, breast cancer metastasis is described as a process wherein cancer cells spread from the breast to multiple organ systems via hematogenous and lymphatic routes. Mapping organ specific patterns of cancer spread over time is essential to understanding metastatic progression. In order to better predict sites of metastases, here we demonstrate modeling of the patterned migration of metastasis. Methods: We reviewed the clinical history of 453 breast cancer patients from Memorial Sloan Kettering Cancer Center who were non-metastatic at diagnosis but developed metastasis over time. We used the variables of organ site of metastases as well as time to create a Markov chain model of metastasis. We illustrate the probabilities of metastasis occurring at a given anatomic site together with the probability of spread to additional sites. Results: Based on the clinical histories of 453 breast cancer patients who developed metastasis, we have learned (i) how to create the Markov transition matrix governing the probabilities of cancer progression from site to site; (ii) how to create a systemic network diagram governing disease progression modeled as a random walk on a directed graph; (iii) how to classify metastatic sites as ‘sponges’ that tend to only receive cancer cells or ‘spreaders’ that receive and release them; (iv) how to model the time-scales of disease progression as a Weibull probability distribution function; (v) how to perform Monte Carlo simulations of disease progression; and (vi) how to interpret disease progression as an entropy-increasing stochastic process. Conclusion: Based on our modeling, metastatic spread may follow predictable pathways. Mapping metastasis not simply by organ site, but by function as either a ‘spreader’ or ‘sponge’ fundamentally reframes our understanding of metastatic processes. This model serves as a novel platform from which we may integrate the evolving genomic landscape that drives cancer

  11. Real-Time Landmine Detection with Ground-Penetrating Radar Using Discriminative and Adaptive Hidden Markov Models

    Directory of Open Access Journals (Sweden)

    Paul Gader

    2005-07-01

    Full Text Available We propose a real-time software system for landmine detection using ground-penetrating radar (GPR. The system includes an efficient and adaptive preprocessing component; a hidden Markov model- (HMM- based detector; a corrective training component; and an incremental update of the background model. The preprocessing is based on frequency-domain processing and performs ground-level alignment and background removal. The HMM detector is an improvement of a previously proposed system (baseline. It includes additional pre- and postprocessing steps to improve the time efficiency and enable real-time application. The corrective training component is used to adjust the initial model parameters to minimize the number of misclassification sequences. This component could be used offline, or online through feedback to adapt an initial model to specific sites and environments. The background update component adjusts the parameters of the background model to adapt it to each lane during testing. The proposed software system is applied to data acquired from three outdoor test sites at different geographic locations, using a state-of-the-art array GPR prototype. The first collection was used as training, and the other two (contain data from more than 1200 m2 of simulated dirt and gravel roads for testing. Our results indicate that, on average, the corrective training can improve the performance by about 10% for each site. For individual lanes, the performance gain can reach 50%.

  12. Nonlinear Markov Control Processes and Games

    Science.gov (United States)

    2012-11-15

    further research we indicated possible extensions to state spaces with nontrivial geometry, to the controlled nonlinear quantum dynamic semigroups and...space nonlinear Markov semigroup is a one-parameter semigroup of (possibly nonlinear) transformations of the unit simplex in n-dimensional Euclidean...certain mixing property of nonlinear transition probabilities. In case of the semigroup parametrized by continuous time one defines its generator as the

  13. Nonlinearly perturbed semi-Markov processes

    CERN Document Server

    Silvestrov, Dmitrii

    2017-01-01

    The book presents new methods of asymptotic analysis for nonlinearly perturbed semi-Markov processes with a finite phase space. These methods are based on special time-space screening procedures for sequential phase space reduction of semi-Markov processes combined with the systematical use of operational calculus for Laurent asymptotic expansions. Effective recurrent algorithms are composed for getting asymptotic expansions, without and with explicit upper bounds for remainders, for power moments of hitting times, stationary and conditional quasi-stationary distributions for nonlinearly perturbed semi-Markov processes. These results are illustrated by asymptotic expansions for birth-death-type semi-Markov processes, which play an important role in various applications. The book will be a useful contribution to the continuing intensive studies in the area. It is an essential reference for theoretical and applied researchers in the field of stochastic processes and their applications that will cont...

  14. Genomic relationships and speciation times of human, chimpanzee, and gorilla inferred from a coalescent hidden Markov model.

    Directory of Open Access Journals (Sweden)

    Asger Hobolth

    2007-02-01

    Full Text Available The genealogical relationship of human, chimpanzee, and gorilla varies along the genome. We develop a hidden Markov model (HMM that incorporates this variation and relate the model parameters to population genetics quantities such as speciation times and ancestral population sizes. Our HMM is an analytically tractable approximation to the coalescent process with recombination, and in simulations we see no apparent bias in the HMM estimates. We apply the HMM to four autosomal contiguous human-chimp-gorilla-orangutan alignments comprising a total of 1.9 million base pairs. We find a very recent speciation time of human-chimp (4.1 +/- 0.4 million years, and fairly large ancestral effective population sizes (65,000 +/- 30,000 for the human-chimp ancestor and 45,000 +/- 10,000 for the human-chimp-gorilla ancestor. Furthermore, around 50% of the human genome coalesces with chimpanzee after speciation with gorilla. We also consider 250,000 base pairs of X-chromosome alignments and find an effective population size much smaller than 75% of the autosomal effective population sizes. Finally, we find that the rate of transitions between different genealogies correlates well with the region-wide present-day human recombination rate, but does not correlate with the fine-scale recombination rates and recombination hot spots, suggesting that the latter are evolutionarily transient.

  15. Discounting Models for Outcomes over Continuous Time

    DEFF Research Database (Denmark)

    Harvey, Charles M.; Østerdal, Lars Peter

    Events that occur over a period of time can be described either as sequences of outcomes at discrete times or as functions of outcomes in an interval of time. This paper presents discounting models for events of the latter type. Conditions on preferences are shown to be satisfied if and only if t...

  16. REPRESENTATION OF ADDITIVE FUNCTIONALS AND LOCAL TIMES FOR JUMP MARKOV PROCESSES AND THEIR FUNCTIONAL LIMIT THEOREM

    Institute of Scientific and Technical Information of China (English)

    蒋义文; 刘禄勤

    2003-01-01

    The representation of additive functionals and local times for jump Markovprocesses are obtained. The results of uniformly functional moderate deviation and theirapplications to birth-death processes are also presented.

  17. 多维时间序列Granger因果图Markov性%Markov Properties of Multivariate Time Series Granger Causality Graphs

    Institute of Scientific and Technical Information of China (English)

    魏岳嵩

    2014-01-01

    文章利用Granger因果图表示多维时间变量序列间的因果关系,图中的顶点集由序列的各个分量组成,顶点间的有向边表示分量序列间的Granger因果关系,无向边表示分量间的同期因果关系。建立Granger因果图的p-分离准则,研究Granger因果图的Markov性。%In this paper,we use Granger causality graphs to present the causal relationships of sequence of variables of multivariate time series. In the graphs,the component series constitute the vertex set,while the di-rected edges denote the Granger causality relationships and undirected edges correspond to contemporaneous causality relationships. We construct the p-separated criterion and discuss the Markov properties of Grang-er causality graphs.

  18. The action operator for continuous time histories

    CERN Document Server

    Savvidou, K N

    1999-01-01

    We define the action operator for the History Projection Operator consistent histories theory, as the quantum analogue of the classical action functional, for the simple harmonic oscillator in one dimention. We conclude that the action operator is the generator of time transformations, and is associated with the two types of time-evolution of the standard quantum theory: the wave-packet reduction and the Heisenberg time-evolution. We construct corresponding classical histories and demonstrate the relevance with the quantum histories. Finally, we show the appearance of the action operator in the expression for the decoherence functional.

  19. Prognostics for Steam Generator Tube Rupture using Markov Chain model

    Energy Technology Data Exchange (ETDEWEB)

    Kim, Gibeom; Heo, Gyunyoung [Kyung Hee University, Yongin (Korea, Republic of); Kim, Hyeonmin [Korea Atomic Energy Research Institute, Daejeon (Korea, Republic of)

    2016-10-15

    This paper will describe the prognostics method for evaluating and forecasting the ageing effect and demonstrate the procedure of prognostics for the Steam Generator Tube Rupture (SGTR) accident. Authors will propose the data-driven method so called MCMC (Markov Chain Monte Carlo) which is preferred to the physical-model method in terms of flexibility and availability. Degradation data is represented as growth of burst probability over time. Markov chain model is performed based on transition probability of state. And the state must be discrete variable. Therefore, burst probability that is continuous variable have to be changed into discrete variable to apply Markov chain model to the degradation data. The Markov chain model which is one of prognostics methods was described and the pilot demonstration for a SGTR accident was performed as a case study. The Markov chain model is strong since it is possible to be performed without physical models as long as enough data are available. However, in the case of the discrete Markov chain used in this study, there must be loss of information while the given data is discretized and assigned to the finite number of states. In this process, original information might not be reflected on prediction sufficiently. This should be noted as the limitation of discrete models. Now we will be studying on other prognostics methods such as GPM (General Path Model) which is also data-driven method as well as the particle filer which belongs to physical-model method and conducting comparison analysis.

  20. Two-sided reflected Markov-modulated Brownian motion with applications to fluid queues and dividend payouts

    CERN Document Server

    D'Auria, Bernardo

    2011-01-01

    In this paper we study a reflected Markov-modulated Brownian motion with a two sided reflection in which the drift, diffusion coefficient and the two boundaries are (jointly) modulated by a finite state space irreducible continuous time Markov chain. The goal is to compute the stationary distribution of this Markov process, which in addition to the complication of having a stochastic boundary can also include jumps at state change epochs of the underlying Markov chain because of the boundary changes. We give the general theory and then specialize to the case where the underlying Markov chain has two states. Moreover, motivated by an application of optimal dividend strategies, we consider the case where the lower barrier is zero and the upper barrier is subject to control. In this case we generalized earlier results from the case of a reflected Brownian motion to the Markov modulated case.

  1. Lie algebra solution of population models based on time-inhomogeneous Markov chains

    CERN Document Server

    House, Thomas

    2011-01-01

    Many natural populations are well modelled through time-inhomogeneous stochastic processes. Such processes have been analysed in the physical sciences using a method based on Lie algebras, but this methodology is not widely used for models with ecological, medical and social applications. This paper presents the Lie algebraic method, and applies it to three biologically well motivated examples. The result of this is a solution form that is often highly computationally advantageous.

  2. Hierarchical Non-Emitting Markov Models

    CERN Document Server

    Ristad, E S; Ristad, Eric Sven; Thomas, Robert G.

    1998-01-01

    We describe a simple variant of the interpolated Markov model with non-emitting state transitions and prove that it is strictly more powerful than any Markov model. More importantly, the non-emitting model outperforms the classic interpolated model on the natural language texts under a wide range of experimental conditions, with only a modest increase in computational requirements. The non-emitting model is also much less prone to overfitting. Keywords: Markov model, interpolated Markov model, hidden Markov model, mixture modeling, non-emitting state transitions, state-conditional interpolation, statistical language model, discrete time series, Brown corpus, Wall Street Journal.

  3. FRACTAL DIMENSION RESULTS FOR CONTINUOUS TIME RANDOM WALKS.

    Science.gov (United States)

    Meerschaert, Mark M; Nane, Erkan; Xiao, Yimin

    2013-04-01

    Continuous time random walks impose random waiting times between particle jumps. This paper computes the fractal dimensions of their process limits, which represent particle traces in anomalous diffusion.

  4. Robust reliable H∞ control for discrete-time Markov jump linear systems with actuator failures

    Institute of Scientific and Technical Information of China (English)

    Chen Jiaorong; Liu Fei

    2008-01-01

    The robust reliable H∞ control problem for discrete-time Markovian jump systems with actuator failures is studied.A more practical model of actuator failures than outage is considered.Based on the state feedback method,the resulting closed-loop systems are reliable in that they remain robust stochastically stable and satisfy a certain level of Hex disturbance attenuation not only when all actuators are operational,but also in case of some actuator failures.The solvability condition of controllers can be equivalent to a feasibility problem of coupled linear matrix inequalities (LMIs).A numerical example is also given to illustrate the design procedures and their effectiveness.

  5. State Estimation for Time-Delay Systems with Markov Jump Parameters and Missing Measurements

    Directory of Open Access Journals (Sweden)

    Yushun Tan

    2014-01-01

    Full Text Available This paper is concerned with the state estimation problem for a class of time-delay systems with Markovian jump parameters and missing measurements, considering the fact that data missing may occur in the process of transmission and its failure rates are governed by random variables satisfying certain probabilistic distribution. By employing a new Lyapunov function and using the convexity property of the matrix inequality, a sufficient condition for the existence of the desired state estimator for Markovian jump systems with missing measurements can be achieved by solving some linear matrix inequalities, which can be easily facilitated by using the standard numerical software. Furthermore, the gain of state estimator can also be derived based on the known conditions. Finally, a numerical example is exploited to demonstrate the effectiveness of the proposed method.

  6. Markov chains with quasitoeplitz transition matrix: first zero hitting

    Directory of Open Access Journals (Sweden)

    Alexander M. Dukhovny

    1989-01-01

    Full Text Available This paper continues the investigation of Markov Chains with a quasitoeplitz transition matrix. Generating functions of first zero hitting probabilities and mean times are found by the solution of special Riemann boundary value problems on the unit circle. Duality is discussed.

  7. A Markov Model for Commen-Cause Failures

    DEFF Research Database (Denmark)

    Platz, Ole

    1984-01-01

    A continuous time four-state Markov chain is shown to cover several of the models that have been used for describing dependencies between failures of components in redundant systems. Among these are the models derived by Marshall and Olkin and by Freund and models for one-out-of-three and two......-out-of-three systems with identical components....

  8. On Markov operators and cones

    OpenAIRE

    Ivkovic, Stefan

    2015-01-01

    In this thesis we will consider Markov operators on cones . More precisely, we let X equipped with certain norm be a real Banach space, K in X be a closed, normal cone with nonempty interior, e in Int (K) be an order unit. A bounded, linear operator T from X into X is a Markov operator w.r.t. K and e if K is invariant under T and e is fixed by T. We consider then the adjoint of T, T* and homogeneous, discrete time Markov system given by u_k+1 = T*(u_k), k = 0,1,2 where u_0(x) is nonnegative f...

  9. Error Correction and Long Run Equilibrium in Continuous Time

    OpenAIRE

    1988-01-01

    This paper deals with error correction models (ECM's) and cointegrated systems that are formulated in continuous time. Problems of representation, identification, estimation and time aggregation are discussed. It is shown that every ECM in continuous time has a discrete time equivalent model in ECM format. Moreover, both models may be written as triangular systems with stationary errors. This formulation simplifies both the continuous and the discrete time ECM representations and it helps to ...

  10. 3D Markov Process for Traffic Flow Prediction in Real-Time

    Directory of Open Access Journals (Sweden)

    Eunjeong Ko

    2016-01-01

    Full Text Available Recently, the correct estimation of traffic flow has begun to be considered an essential component in intelligent transportation systems. In this paper, a new statistical method to predict traffic flows using time series analyses and geometric correlations is proposed. The novelty of the proposed method is two-fold: (1 a 3D heat map is designed to describe the traffic conditions between roads, which can effectively represent the correlations between spatially- and temporally-adjacent traffic states; and (2 the relationship between the adjacent roads on the spatiotemporal domain is represented by cliques in MRF and the clique parameters are obtained by example-based learning. In order to assess the validity of the proposed method, it is tested using data from expressway traffic that are provided by the Korean Expressway Corporation, and the performance of the proposed method is compared with existing approaches. The results demonstrate that the proposed method can predict traffic conditions with an accuracy of 85%, and this accuracy can be improved further.

  11. TR01: Time-continuous Sparse Imputation

    CERN Document Server

    Gemmeke, J F

    2009-01-01

    An effective way to increase the noise robustness of automatic speech recognition is to label noisy speech features as either reliable or unreliable (missing) prior to decoding, and to replace the missing ones by clean speech estimates. We present a novel method to obtain such clean speech estimates. Unlike previous imputation frameworks which work on a frame-by-frame basis, our method focuses on exploiting information from a large time-context. Using a sliding window approach, denoised speech representations are constructed using a sparse representation of the reliable features in an overcomplete basis of fixed-length exemplar fragments. We demonstrate the potential of our approach with experiments on the AURORA-2 connected digit database.

  12. Markov chains

    CERN Document Server

    Revuz, D

    1984-01-01

    This is the revised and augmented edition of a now classic book which is an introduction to sub-Markovian kernels on general measurable spaces and their associated homogeneous Markov chains. The first part, an expository text on the foundations of the subject, is intended for post-graduate students. A study of potential theory, the basic classification of chains according to their asymptotic behaviour and the celebrated Chacon-Ornstein theorem are examined in detail. The second part of the book is at a more advanced level and includes a treatment of random walks on general locally compact abelian groups. Further chapters develop renewal theory, an introduction to Martin boundary and the study of chains recurrent in the Harris sense. Finally, the last chapter deals with the construction of chains starting from a kernel satisfying some kind of maximum principle.

  13. Gearbox state recognition based on continuous hidden markov model%基于CHMM的齿轮箱状态识别研究

    Institute of Scientific and Technical Information of China (English)

    滕红智; 赵建民; 贾希胜; 张星辉; 王正军

    2012-01-01

    针对离散隐Markov模型(HMM)在状态识别中的不足,结合齿轮箱全寿命实验数据,研究了基于连续隐Markov模型(CHMM)的状态识别方法.建立了基于齿轮箱原始振动信号的CHMM状态识别框架,提出了基于K均值算法和交叉验证相结合的状态数优化方法,通过计算待确定观测数据的极大似然概率值来确定齿轮箱当前状态.结果表明,用原始振动信号作为CHMM的输入可以实现状态识别,验证了模型的有效性,为齿轮箱基于状态的维修提供了科学依据.%Combined with full lifetime test data of gearbox, state recognition based on continuous hidden markov model ( CHMM) was studied. The frame of state recognition based on CHMM using original vibration signal was established. Virtues and defects of existing classification methods classifying state in full life cycles were analyzed. State number optimization model was established based on K means and cross validation. Gearbox 's operating state was determined by calculating the maximum log-likelihood. The recognition results showed that the proposed method of state recognition based on CHMM using original vibration signals is feasible and effective.

  14. Stratification of the phase clouds and statistical effects of the non-Markovity in chaotic time series of human gait for healthy people and Parkinson patients

    Science.gov (United States)

    Yulmetyev, Renat; Demin, Sergey; Emelyanova, Natalya; Gafarov, Fail; Hänggi, Peter

    2003-03-01

    In this work we develop a new method of diagnosing the nervous system diseases and a new approach in studying human gait dynamics with the help of the theory of discrete non-Markov random processes (Phys. Rev. E 62 (5) (2000) 6178, Phys. Rev. E 64 (2001) 066132, Phys. Rev. E 65 (2002) 046107, Physica A 303 (2002) 427). The stratification of the phase clouds and the statistical non-Markov effects in the time series of the dynamics of human gait are considered. We carried out the comparative analysis of the data of four age groups of healthy people: children (from 3 to 10 year olds), teenagers (from 11 to 14 year olds), young people (from 21 up to 29 year olds), elderly persons (from 71 to 77 year olds) and Parkinson patients. The full data set are analyzed with the help of the phase portraits of the four dynamic variables, the power spectra of the initial time correlation function and the memory functions of junior orders, the three first points in the spectra of the statistical non-Markov parameter. The received results allow to define the predisposition of the probationers to deflections in the central nervous system caused by Parkinson's disease. We have found out distinct differences between the five submitted groups. On this basis we offer a new method of diagnostics and forecasting Parkinson's disease.

  15. Stratification of the phase clouds and statistical effects of the non-Markovity in chaotic time series of human gait for healthy people and Parkinson patients

    CERN Document Server

    Yulmetyev, R M; Emelyanova, N; Gafarov, F; Hänggi, P; Yulmetyev, Renat; Demin, Sergey; Emelyanova, Natalya; Gafarov, Fail; Hanggi, Peter

    2003-01-01

    In this work we develop a new method of diagnosing the nervous system diseases and a new approach in studying human gait dynamics with the help of the theory of discrete non-Markov random processes. The stratification of the phase clouds and the statistical non-Markov effects in the time series of the dynamics of human gait are considered. We carried out the comparative analysis of the data of four age groups of healthy people: children (from 3 to 10 year olds), teenagers (from 11 to 14 year oulds), young people (from 21 up to 29 year oulds), elderly persons (from 71 to 77 year olds) and Parkinson patients. The full data set are analyzed with the help of the phase portraits of the four dynamic variables, the power spectra of the initial time correlation function and the memory functions of junior orders, the three first points in the spectra of the statistical non-Markov parameter. The received results allow to define the predisposition of the probationers to deflections in the central nervous system caused b...

  16. Generalized Markov branching models

    OpenAIRE

    Li, Junping

    2005-01-01

    In this thesis, we first considered a modified Markov branching process incorporating both state-independent immigration and resurrection. After establishing the criteria for regularity and uniqueness, explicit expressions for the extinction probability and mean extinction time are presented. The criteria for recurrence and ergodicity are also established. In addition, an explicit expression for the equilibrium distribution is presented.\\ud \\ud We then moved on to investigate the basic proper...

  17. Generalized Markov branching models

    OpenAIRE

    Li, Junping

    2005-01-01

    In this thesis, we first considered a modified Markov branching process incorporating both state-independent immigration and resurrection. After establishing the criteria for regularity and uniqueness, explicit expressions for the extinction probability and mean extinction time are presented. The criteria for recurrence and ergodicity are also established. In addition, an explicit expression for the equilibrium distribution is presented. We then moved on to investigate the basic proper...

  18. Semigroups, boundary value problems and Markov processes

    CERN Document Server

    Taira, Kazuaki

    2014-01-01

    A careful and accessible exposition of functional analytic methods in stochastic analysis is provided in this book. It focuses on the interrelationship between three subjects in analysis: Markov processes, semi groups and elliptic boundary value problems. The author studies a general class of elliptic boundary value problems for second-order, Waldenfels integro-differential operators in partial differential equations and proves that this class of elliptic boundary value problems provides a general class of Feller semigroups in functional analysis. As an application, the author constructs a general class of Markov processes in probability in which a Markovian particle moves both by jumps and continuously in the state space until it 'dies' at the time when it reaches the set where the particle is definitely absorbed. Augmenting the 1st edition published in 2004, this edition includes four new chapters and eight re-worked and expanded chapters. It is amply illustrated and all chapters are rounded off with Notes ...

  19. Descriptive and predictive evaluation of high resolution Markov chain precipitation models

    DEFF Research Database (Denmark)

    Sørup, Hjalte Jomo Danielsen; Madsen, Henrik; Arnbjerg-Nielsen, Karsten

    2012-01-01

    . Continuous modelling of the Markov process proved attractive because of a marked decrease in the number of parameters. Inclusion of seasonality into the continuous Markov chain model proved difficult. Monte Carlo simulations with the models show that it is very difficult for all the model formulations...... to reproduce the time series on event level. Extreme events with short (10 min), medium (60 min) and long (12 h) durations were investigated because of their importance in urban hydrology. Both the descriptive likelihood based statistics and the predictive Monte Carlo simulation based statistics are valuable......A time series of tipping bucket recordings of very high temporal and volumetric resolution precipitation is modelled using Markov chain models. Both first and second‐order Markov models as well as seasonal and diurnal models are investigated and evaluated using likelihood based techniques...

  20. Continuous Markovian Logics

    DEFF Research Database (Denmark)

    Mardare, Radu Iulian; Cardelli, Luca; Larsen, Kim Guldstrand

    2012-01-01

    Continuous Markovian Logic (CML) is a multimodal logic that expresses quantitative and qualitative properties of continuous-time labelled Markov processes with arbitrary (analytic) state-spaces, henceforth called continuous Markov processes (CMPs). The modalities of CML evaluate the rates...... characterizes stochastic bisimilarity and it supports the definition of a quantified extension of the satisfiability relation that measures the "compatibility" between a model and a property. In this context, the metaproperties allows us to prove two robustness theorems for the logic stating that one can...

  1. SINGULARLY PERTURBED MARKOV DECISION PROCESSES WITH INCLUSION OF TRANSIENT STATES

    Institute of Scientific and Technical Information of China (English)

    R. H. Liu; Q. Zhang; G. Yin

    2001-01-01

    This paper is concerned with the continuous-time Markov decision processes (MDP) having weak and strong interactions. Using a hierarchical approach, the state space of the underlying Markov chain can be decomposed into several groups of recurrent states and a group of transient states resulting in a singularly perturbed MDP formulation.Instead of solving the original problem directly, a limit problem that is much simpler to handle is derived. On the basis of the optimal solution of the limit problem, nearly optimal decisions are constructed for the original problem. The asymptotic optimality of the constructed control is obtained; the rate of convergence is ascertained.

  2. Integration by Parts and Martingale Representation for a Markov Chain

    Directory of Open Access Journals (Sweden)

    Tak Kuen Siu

    2014-01-01

    Full Text Available Integration-by-parts formulas for functions of fundamental jump processes relating to a continuous-time, finite-state Markov chain are derived using Bismut's change of measures approach to Malliavin calculus. New expressions for the integrands in stochastic integrals corresponding to representations of martingales for the fundamental jump processes are derived using the integration-by-parts formulas. These results are then applied to hedge contingent claims in a Markov chain financial market, which provides a practical motivation for the developments of the integration-by-parts formulas and the martingale representations.

  3. Development of a Markov Model for Forecasting Continuation Rates for Enlisted Prior Service and Non-Prior Service Personnel in the Selective Marine Corps Reserve (SMCR)

    Science.gov (United States)

    2012-03-01

    Corps Officers” (master’s thesis, Naval Postgraduate School, 2007), 21. 35 Jeffery Sapp , “A Calculator Adaption of the Markov Chain Model for...a Class of the Civil Service.” Journal of the Royal Statistical Society 20, no. 1 (March 1971): 85–110. Sapp , Jeffrey K. “A Calculator Adaptation of

  4. STATISTICS OF A CLASS OF MARKOV CHAINS

    Institute of Scientific and Technical Information of China (English)

    DENG Yingchun; CAO Xianbing

    2004-01-01

    In this paper we prove that the distributions of their sojourn time and hitting time at one special state for random walks which are allowed to be finite or infinite and Markov chains on star-graphs with discrete time can uniquely determine the probability distribution of the whole chains. This result also suggests a new statistical method for Markov chains.

  5. Continuous Time Structural Equation Modeling with R Package ctsem

    Directory of Open Access Journals (Sweden)

    Charles C. Driver

    2017-04-01

    Full Text Available We introduce ctsem, an R package for continuous time structural equation modeling of panel (N > 1 and time series (N = 1 data, using full information maximum likelihood. Most dynamic models (e.g., cross-lagged panel models in the social and behavioural sciences are discrete time models. An assumption of discrete time models is that time intervals between measurements are equal, and that all subjects were assessed at the same intervals. Violations of this assumption are often ignored due to the difficulty of accounting for varying time intervals, therefore parameter estimates can be biased and the time course of effects becomes ambiguous. By using stochastic differential equations to estimate an underlying continuous process, continuous time models allow for any pattern of measurement occasions. By interfacing to OpenMx, ctsem combines the flexible specification of structural equation models with the enhanced data gathering opportunities and improved estimation of continuous time models. ctsem can estimate relationships over time for multiple latent processes, measured by multiple noisy indicators with varying time intervals between observations. Within and between effects are estimated simultaneously by modeling both observed covariates and unobserved heterogeneity. Exogenous shocks with different shapes, group differences, higher order diffusion effects and oscillating processes can all be simply modeled. We first introduce and define continuous time models, then show how to specify and estimate a range of continuous time models using ctsem.

  6. How to get an exact sample from a generic Markov chain and sample a random spanning tree from a directed graph, both within the cover time

    Energy Technology Data Exchange (ETDEWEB)

    Wilson, D.B.; Propp, J.G.

    1996-12-31

    This paper shows how to obtain unbiased samples from an unknown Markov chain by observing it for O(T{sub c}) steps, where T{sub c} is the cover time. This algorithm improves on several previous algorithms, and there is a matching lower bound. Using the techniques from the sampling algorithm, we also show how to sample random directed spanning trees from a weighted directed graph, with arcs directed to a root, and probability proportional to the product of the edge weights. This tree sampling algorithm runs within 18 cover times of the associated random walk, and is more generally applicable than the algorithm of Broder and Aldous.

  7. Metastability of exponentially perturbed Markov chains

    Institute of Scientific and Technical Information of China (English)

    陈大岳; 冯建峰; 钱敏平

    1996-01-01

    A family of irreducible Markov chains on a finite state space is considered as an exponential perturbation of a reducible Markov chain. This is a generalization of the Freidlin-Wentzell theory, motivated by studies of stochastic Ising models, neural network and simulated annealing. It is shown that the metastability is a universal feature for this wide class of Markov chains. The metastable states are simply those recurrent states of the reducible Markov chain. Higher level attractors, related attractive basins and their pyramidal structure are analysed. The logarithmic asymptotics of the hitting time of various sets are estimated. The hitting time over its mean converges in law to the unit exponential distribution.

  8. Multivariable identification of continuous-time fractional system

    OpenAIRE

    2009-01-01

    International audience; This paper presents two subspace-based methods, from the MOESP (MIMO output-error state space) family, for state-space identification of continuous-time fractional commensurate models from sampled input-output data. The methodology used in this paper involves a continuous-time fractional operator allowing to reformulate the problem so that the state-space matrices can be estimated with conventional discrete-time subspace techniques based on QR and singular value decomp...

  9. Learning Continuous Time Bayesian Network Classifiers Using MapReduce

    Directory of Open Access Journals (Sweden)

    Simone Villa

    2014-12-01

    Full Text Available Parameter and structural learning on continuous time Bayesian network classifiers are challenging tasks when you are dealing with big data. This paper describes an efficient scalable parallel algorithm for parameter and structural learning in the case of complete data using the MapReduce framework. Two popular instances of classifiers are analyzed, namely the continuous time naive Bayes and the continuous time tree augmented naive Bayes. Details of the proposed algorithm are presented using Hadoop, an open-source implementation of a distributed file system and the MapReduce framework for distributed data processing. Performance evaluation of the designed algorithm shows a robust parallel scaling.

  10. Stability of continuous-time quantum filters with measurement imperfections

    Science.gov (United States)

    Amini, H.; Pellegrini, C.; Rouchon, P.

    2014-07-01

    The fidelity between the state of a continuously observed quantum system and the state of its associated quantum filter, is shown to be always a submartingale. The observed system is assumed to be governed by a continuous-time Stochastic Master Equation (SME), driven simultaneously by Wiener and Poisson processes and that takes into account incompleteness and errors in measurements. This stability result is the continuous-time counterpart of a similar stability result already established for discrete-time quantum systems and where the measurement imperfections are modelled by a left stochastic matrix.

  11. Linear optimal control of continuous time chaotic systems.

    Science.gov (United States)

    Merat, Kaveh; Abbaszadeh Chekan, Jafar; Salarieh, Hassan; Alasty, Aria

    2014-07-01

    In this research study, chaos control of continuous time systems has been performed by using dynamic programming technique. In the first step by crossing the response orbits with a selected Poincare section and subsequently applying linear regression method, the continuous time system is converted to a discrete type. Then, by solving the Riccati equation a sub-optimal algorithm has been devised for the obtained discrete chaotic systems. In the next step, by implementing the acquired algorithm on the quantized continuous time system, the chaos has been suppressed in the Rossler and AFM systems as some case studies.

  12. An introduction to continuous-time stochastic processes theory, models, and applications to finance, biology, and medicine

    CERN Document Server

    Capasso, Vincenzo

    2015-01-01

    This textbook, now in its third edition, offers a rigorous and self-contained introduction to the theory of continuous-time stochastic processes, stochastic integrals, and stochastic differential equations. Expertly balancing theory and applications, the work features concrete examples of modeling real-world problems from biology, medicine, industrial applications, finance, and insurance using stochastic methods. No previous knowledge of stochastic processes is required. Key topics include: * Markov processes * Stochastic differential equations * Arbitrage-free markets and financial derivatives * Insurance risk * Population dynamics, and epidemics * Agent-based models New to the Third Edition: * Infinitely divisible distributions * Random measures * Levy processes * Fractional Brownian motion * Ergodic theory * Karhunen-Loeve expansion * Additional applications * Additional  exercises * Smoluchowski  approximation of  Langevin systems An Introduction to Continuous-Time Stochastic Processes, Third Editio...

  13. Quantum Markov fields on graphs

    OpenAIRE

    2009-01-01

    We introduce generalized quantum Markov states and generalized d-Markov chains which extend the notion quantum Markov chains on spin systems to that on $C^*$-algebras defined by general graphs. As examples of generalized d-Markov chains, we construct the entangled Markov fields on tree graphs. The concrete examples of generalized d-Markov chains on Cayley trees are also investigated.

  14. Integral-Value Models for Outcomes over Continuous Time

    DEFF Research Database (Denmark)

    Harvey, Charles M.; Østerdal, Lars Peter

    Models of preferences between outcomes over continuous time are important for individual, corporate, and social decision making, e.g., medical treatment, infrastructure development, and environmental regulation. This paper presents a foundation for such models. It shows that conditions...... on preferences between real- or vector-valued outcomes over continuous time are satisfied if and only if the preferences are represented by a value function having an integral form...

  15. Linear generalized synchronization of continuous-time chaotic systems

    Energy Technology Data Exchange (ETDEWEB)

    Lu Junguo E-mail: jglu@sjtu.edu.cn; Xi Yugeng

    2003-08-01

    This paper develops a general approach for constructing a response system to implement linear generalized synchronization (GS) with the drive continuous-time chaotic system. Some sufficient conditions of global asymptotic linear GS between the drive and response continuous-time chaotic systems are attained from rigorously modern control theory. Finally, we take Chua's circuit as an example for illustration and verification.

  16. 基于ℋ表示的时变随机Markov跳跃系统的能观性%Observability of time-varying stochastic Markov jump systems based onℋ-representation

    Institute of Scientific and Technical Information of China (English)

    盛立; 高明; 张维海

    2015-01-01

    研究时变连续和离散随机Markov跳跃系统(SMJSs)的能观性问题。基于ℋ表示方法将时变SMJSs转化为等价的时变线性系统,根据线性系统理论得到时变连续和离散SMJSs的能观性Gramian矩阵判据。数值仿真表明了所得结论的正确性。%The observability of time-varying continuous and discrete-time stochastic Markov jump systems(SMJSs) is investigated. Time-varying SMJSs are transformed into the equivalent time-varying linear systems based on the ℋ-representation method. Gramian matrix criteria for the observability of time-varying continuous and discrete-time SMJSs are derived based on the linear system theory. A numerical example is given to demonstrate the correctness of the obtained results.

  17. Application of continuous-time random walk to statistical arbitrage

    Directory of Open Access Journals (Sweden)

    Sergey Osmekhin

    2015-01-01

    Full Text Available An analytical statistical arbitrage strategy is proposed, where the distribution of the spread is modelled as a continuous-time random walk. Optimal boundaries, computed as a function of the mean and variance of the firstpassage time ofthe spread,maximises an objective function. The predictability of the trading strategy is analysed and contrasted for two forms of continuous-time random walk processes. We found that the waiting-time distribution has a significant impact on the prediction of the expected profit for intraday trading

  18. İlintili Görüntülerin Birleşik Markov Rastgele Alan Modeli ve Gerçek - Zamanlı Uygulanabilir En - İyileme Yaklaşımı = Random Markov Field of Coupled Images and a Relevant Real - Time Optimization Approach

    Directory of Open Access Journals (Sweden)

    Osman Nuri UÇAN

    2002-01-01

    Full Text Available This paper describes a Markov Random Field model for coupled range and confidence signals. Beamforming is a method used to bring a range image from backscattered echos of acoustic signals. Another information is confidence of signal which associated point by point with this range data. In the proposed algorithm, the range and confidence images are modeled as Markov Random Fields whose probability distributions are specified by a single energy function. The optimization of this model gives reconstructed range and restored confidence images and an approach to the optimization is suggested for the real-time implementation of this method.

  19. On Transaction-Cost Models in Continuous-Time Markets

    Directory of Open Access Journals (Sweden)

    Thomas Poufinas

    2015-04-01

    Full Text Available Transaction-cost models in continuous-time markets are considered. Given that investors decide to buy or sell at certain time instants, we study the existence of trading strategies that reach a certain final wealth level in continuous-time markets, under the assumption that transaction costs, built in certain recommended ways, have to be paid. Markets prove to behave in manners that resemble those of complete ones for a wide variety of transaction-cost types. The results are important, but not exclusively, for the pricing of options with transaction costs.

  20. Memory in linear recurrent neural networks in continuous time.

    Science.gov (United States)

    Hermans, Michiel; Schrauwen, Benjamin

    2010-04-01

    Reservoir Computing is a novel technique which employs recurrent neural networks while circumventing difficult training algorithms. A very recent trend in Reservoir Computing is the use of real physical dynamical systems as implementation platforms, rather than the customary digital emulations. Physical systems operate in continuous time, creating a fundamental difference with the classic discrete time definitions of Reservoir Computing. The specific goal of this paper is to study the memory properties of such systems, where we will limit ourselves to linear dynamics. We develop an analytical model which allows the calculation of the memory function for continuous time linear dynamical systems, which can be considered as networks of linear leaky integrator neurons. We then use this model to research memory properties for different types of reservoir. We start with random connection matrices with a shifted eigenvalue spectrum, which perform very poorly. Next, we transform two specific reservoir types, which are known to give good performance in discrete time, to the continuous time domain. Reservoirs based on uniform spreading of connection matrix eigenvalues on the unit disk in discrete time give much better memory properties than reservoirs with random connection matrices, where reservoirs based on orthogonal connection matrices in discrete time are very robust against noise and their memory properties can be tuned. The overall results found in this work yield important insights into how to design networks for continuous time.

  1. Coaction versus reciprocity in continuous-time models of cooperation

    NARCIS (Netherlands)

    van Doorn, G. Sander; Riebli, Thomas; Taborsky, Michael

    2014-01-01

    Cooperating animals frequently show closely coordinated behaviours organized by a continuous flow of information between interacting partners. Such real-time coaction is not captured by the iterated prisoner's dilemma and other discrete-time reciprocal cooperation games, which inherently feature a d

  2. Finding tree symmetries using continuous-time quantum walk

    Institute of Scientific and Technical Information of China (English)

    Wu Jun-Jie; Zhang Bai-Da; Tang Yu-Hua; Qiang Xiao-Gang; Wang Hui-Quan

    2013-01-01

    Quantum walk,the quantum counterpart of random walk,is an important model and widely studied to develop new quantum algorithms.This paper studies the relationship between the continuous-time quantum walk and the symmetry of a graph,especially that of a tree.Firstly,we prove in mathematics that the symmetry of a graph is highly related to quantum walk.Secondly,we propose an algorithm based on the continuous-time quantum walk to compute the symmetry of a tree.Our algorithm has better time complexity O(N3) than the current best algorithm.Finally,through testing three types of 10024 trees,we find that the symmetry of a tree can be found with an extremely high efficiency with the help of the continuous-time quantum walk.

  3. A continuous-time/discrete-time mixed audio-band sigma delta ADC

    Institute of Scientific and Technical Information of China (English)

    Liu Yan; Hua Siliang; Wang Donghui; Hou Chaohuan

    2011-01-01

    This paper introduces a mixed continuous-time/discrete-time, single-loop, fourth-order, 4-bit audioband sigma delta ADC that combines the benefits of continuous-time and discrete-time circuits, while mitigating the challenges associated with continuous-time design. Measurement results show that the peak SNR of this ADC reaches 100 dB and the total power consumption is less than 30 mW.

  4. Chernoff-Hoeffding Bounds for Markov Chains: Generalized and Simplified

    CERN Document Server

    Chung, Kai-Min; Liu, Zhenming; Mitzenmacher, Michael

    2012-01-01

    We prove the first Chernoff-Hoeffding bounds for general nonreversible finite-state Markov chains based on the standard L_1 (variation distance) mixing-time of the chain. Specifically, consider an ergodic Markov chain M and a weight function f: [n] -> [0,1] on the state space [n] of M with mean mu = E_{v = delta mu t ], is at most exp(-Omega(delta^2 mu t / T)) for 0 1. In fact, the bounds hold even if the weight functions f_i's for i in [t] are distinct, provided that all of them have the same mean mu. We also obtain a simplified proof for the Chernoff-Hoeffding bounds based on the spectral expansion lambda of M, which is the square root of the second largest eigenvalue (in absolute value) of M tilde{M}, where tilde{M} is the time-reversal Markov chain of M. We show that the probability Pr [ |X - mu t| >= delta mu t ] is at most exp(-Omega(delta^2 (1-lambda) mu t)) for 0 1. Both of our results extend to continuous-time Markov chains, and to the case where the walk starts from an arbitrary distribution x, at...

  5. A semi-Markov model for mitosis segmentation in time-lapse phase contrast microscopy image sequences of stem cell populations.

    Science.gov (United States)

    Liu, An-An; Li, Kang; Kanade, Takeo

    2012-02-01

    We propose a semi-Markov model trained in a max-margin learning framework for mitosis event segmentation in large-scale time-lapse phase contrast microscopy image sequences of stem cell populations. Our method consists of three steps. First, we apply a constrained optimization based microscopy image segmentation method that exploits phase contrast optics to extract candidate subsequences in the input image sequence that contains mitosis events. Then, we apply a max-margin hidden conditional random field (MM-HCRF) classifier learned from human-annotated mitotic and nonmitotic sequences to classify each candidate subsequence as a mitosis or not. Finally, a max-margin semi-Markov model (MM-SMM) trained on manually-segmented mitotic sequences is utilized to reinforce the mitosis classification results, and to further segment each mitosis into four predefined temporal stages. The proposed method outperforms the event-detection CRF model recently reported by Huh as well as several other competing methods in very challenging image sequences of multipolar-shaped C3H10T1/2 mesenchymal stem cells. For mitosis detection, an overall precision of 95.8% and a recall of 88.1% were achieved. For mitosis segmentation, the mean and standard deviation for the localization errors of the start and end points of all mitosis stages were well below 1 and 2 frames, respectively. In particular, an overall temporal location error of 0.73 ± 1.29 frames was achieved for locating daughter cell birth events.

  6. An Approximate Markov Model for the Wright-Fisher Diffusion and Its Application to Time Series Data.

    Science.gov (United States)

    Ferrer-Admetlla, Anna; Leuenberger, Christoph; Jensen, Jeffrey D; Wegmann, Daniel

    2016-06-01

    The joint and accurate inference of selection and demography from genetic data is considered a particularly challenging question in population genetics, since both process may lead to very similar patterns of genetic diversity. However, additional information for disentangling these effects may be obtained by observing changes in allele frequencies over multiple time points. Such data are common in experimental evolution studies, as well as in the comparison of ancient and contemporary samples. Leveraging this information, however, has been computationally challenging, particularly when considering multilocus data sets. To overcome these issues, we introduce a novel, discrete approximation for diffusion processes, termed mean transition time approximation, which preserves the long-term behavior of the underlying continuous diffusion process. We then derive this approximation for the particular case of inferring selection and demography from time series data under the classic Wright-Fisher model and demonstrate that our approximation is well suited to describe allele trajectories through time, even when only a few states are used. We then develop a Bayesian inference approach to jointly infer the population size and locus-specific selection coefficients with high accuracy and further extend this model to also infer the rates of sequencing errors and mutations. We finally apply our approach to recent experimental data on the evolution of drug resistance in influenza virus, identifying likely targets of selection and finding evidence for much larger viral population sizes than previously reported.

  7. Continuous-Time System Identification of a Smoking Cessation Intervention.

    Science.gov (United States)

    Timms, Kevin P; Rivera, Daniel E; Collins, Linda M; Piper, Megan E

    2014-01-01

    Cigarette smoking is a major global public health issue and the leading cause of preventable death in the United States. Toward a goal of designing better smoking cessation treatments, system identification techniques are applied to intervention data to describe smoking cessation as a process of behavior change. System identification problems that draw from two modeling paradigms in quantitative psychology (statistical mediation and self-regulation) are considered, consisting of a series of continuous-time estimation problems. A continuous-time dynamic modeling approach is employed to describe the response of craving and smoking rates during a quit attempt, as captured in data from a smoking cessation clinical trial. The use of continuous-time models provide benefits of parsimony, ease of interpretation, and the opportunity to work with uneven or missing data.

  8. Continuous-time system identification of a smoking cessation intervention

    Science.gov (United States)

    Timms, Kevin P.; Rivera, Daniel E.; Collins, Linda M.; Piper, Megan E.

    2014-07-01

    Cigarette smoking is a major global public health issue and the leading cause of preventable death in the United States. Toward a goal of designing better smoking cessation treatments, system identification techniques are applied to intervention data to describe smoking cessation as a process of behaviour change. System identification problems that draw from two modelling paradigms in quantitative psychology (statistical mediation and self-regulation) are considered, consisting of a series of continuous-time estimation problems. A continuous-time dynamic modelling approach is employed to describe the response of craving and smoking rates during a quit attempt, as captured in data from a smoking cessation clinical trial. The use of continuous-time models provide benefits of parsimony, ease of interpretation, and the opportunity to work with uneven or missing data.

  9. Pseudo-Hermitian continuous-time quantum walks

    Energy Technology Data Exchange (ETDEWEB)

    Salimi, S; Sorouri, A, E-mail: shsalimi@uok.ac.i, E-mail: a.sorouri@uok.ac.i [Department of Physics, University of Kurdistan, PO Box 66177-15175, Sanandaj (Iran, Islamic Republic of)

    2010-07-09

    In this paper we present a model exhibiting a new type of continuous-time quantum walk (as a quantum-mechanical transport process) on networks, which is described by a non-Hermitian Hamiltonian possessing a real spectrum. We call it pseudo-Hermitian continuous-time quantum walk. We introduce a method to obtain the probability distribution of walk on any vertex and then study a specific system. We observe that the probability distribution on certain vertices increases compared to that of the Hermitian case. This formalism makes the transport process faster and can be useful for search algorithms.

  10. High frequency sampling of a continuous-time ARMA process

    CERN Document Server

    Brockwell, Peter J; Klüppelberg, Claudia

    2011-01-01

    Continuous-time autoregressive moving average (CARMA) processes have recently been used widely in the modeling of non-uniformly spaced data and as a tool for dealing with high-frequency data of the form $Y_{n\\Delta}, n=0,1,2,...$, where $\\Delta$ is small and positive. Such data occur in many fields of application, particularly in finance and the study of turbulence. This paper is concerned with the characteristics of the process $(Y_{n\\Delta})_{n\\in\\bbz}$, when $\\Delta$ is small and the underlying continuous-time process $(Y_t)_{t\\in\\bbr}$ is a specified CARMA process.

  11. Neuromorphic Continuous-Time State Space Pole Placement Adaptive Control

    Institute of Scientific and Technical Information of China (English)

    卢钊; 孙明伟

    2003-01-01

    A neuromorphic continuous-time state space pole assignment adaptive controller is proposed, which is particularly appropriate for controlling a large-scale time-variant state-space model due to the parallely distributed nature of neurocomputing. In our approach, Hopfield neural network is exploited to identify the parameters of a continuous-time state-space model, and a dedicated recurrent neural network is designed to compute pole placement feedback control law in real time. Thus the identification and the control computation are incorporated in the closed-loop, adaptive, real-time control system. The merit of this approach is that the neural networks converge to their solutions very quickly and simultaneously.

  12. Hidden Markov models estimation and control

    CERN Document Server

    Elliott, Robert J; Moore, John B

    1995-01-01

    As more applications are found, interest in Hidden Markov Models continues to grow. Following comments and feedback from colleagues, students and other working with Hidden Markov Models the corrected 3rd printing of this volume contains clarifications, improvements and some new material, including results on smoothing for linear Gaussian dynamics. In Chapter 2 the derivation of the basic filters related to the Markov chain are each presented explicitly, rather than as special cases of one general filter. Furthermore, equations for smoothed estimates are given. The dynamics for the Kalman filte

  13. H∞ Filtering for Discrete Markov Jump Singular Systems with Mode-Dependent Time Delay Based on T-S Fuzzy Model

    Directory of Open Access Journals (Sweden)

    Cheng Gong

    2014-01-01

    Full Text Available This paper investigates the H∞ filtering problem of discrete singular Markov jump systems (SMJSs with mode-dependent time delay based on T-S fuzzy model. First, by Lyapunov-Krasovskii functional approach, a delay-dependent sufficient condition on H∞-disturbance attenuation is presented, in which both stability and prescribed H∞ performance are required to be achieved for the filtering-error systems. Then, based on the condition, the delay-dependent H∞ filter design scheme for SMJSs with mode-dependent time delay based on T-S fuzzy model is developed in term of linear matrix inequality (LMI. Finally, an example is given to illustrate the effectiveness of the result.

  14. Narrow Artificial Intelligence with Machine Learning for Real-Time Estimation of a Mobile Agent’s Location Using Hidden Markov Models

    Directory of Open Access Journals (Sweden)

    Cédric Beaulac

    2017-01-01

    Full Text Available We propose to use a supervised machine learning technique to track the location of a mobile agent in real time. Hidden Markov Models are used to build artificial intelligence that estimates the unknown position of a mobile target moving in a defined environment. This narrow artificial intelligence performs two distinct tasks. First, it provides real-time estimation of the mobile agent’s position using the forward algorithm. Second, it uses the Baum–Welch algorithm as a statistical learning tool to gain knowledge of the mobile target. Finally, an experimental environment is proposed, namely, a video game that we use to test our artificial intelligence. We present statistical and graphical results to illustrate the efficiency of our method.

  15. Input-Output Approach to Control for Fuzzy Markov Jump Systems With Time-Varying Delays and Uncertain Packet Dropout Rate.

    Science.gov (United States)

    Zhang, Lixian; Ning, Zepeng; Shi, Peng

    2015-11-01

    This paper is concerned with H∞ control problem for a class of discrete-time Takagi-Sugeno fuzzy Markov jump systems with time-varying delays under unreliable communication links. It is assumed that the data transmission between the plant and the controller are subject to randomly occurred packet dropouts satisfying Bernoulli distribution and the dropout rate is uncertain. Based on a fuzzy-basis-dependent and mode-dependent Lyapunov function, the existence conditions of the desired H∞ state-feedback controllers are derived by employing the scaled small gain theorem such that the closed-loop system is stochastically stable and achieves a guaranteed H∞ performance. The gains of the controllers are constructed by solving a set of linear matrix inequalities. Finally, a practical example of robot arm is provided to illustrate the performance of the proposed approach.

  16. Exponential and Strong Ergodicity for Markov Processes with an Application to Queues

    Institute of Scientific and Technical Information of China (English)

    Yuanyuan LIU; Zhenting HOU

    2008-01-01

    For an ergodic continuous-time Markov process with a particular state in its space, the authors provide the necessary and sufficient conditions for exponential and strongerg odicity in terms of the moments of the first hitting time on the state. An application to the queue length process of M/G/1 queue with multiple vacations is given.

  17. Quantum Markov Chain Mixing and Dissipative Engineering

    DEFF Research Database (Denmark)

    Kastoryano, Michael James

    2012-01-01

    This thesis is the fruit of investigations on the extension of ideas of Markov chain mixing to the quantum setting, and its application to problems of dissipative engineering. A Markov chain describes a statistical process where the probability of future events depends only on the state...... of the system at the present point in time, but not on the history of events. Very many important processes in nature are of this type, therefore a good understanding of their behaviour has turned out to be very fruitful for science. Markov chains always have a non-empty set of limiting distributions...... (stationary states). The aim of Markov chain mixing is to obtain (upper and/or lower) bounds on the number of steps it takes for the Markov chain to reach a stationary state. The natural quantum extensions of these notions are density matrices and quantum channels. We set out to develop a general mathematical...

  18. Incomplete Continuous-time Securities Markets with Stochastic Income Volatility

    DEFF Research Database (Denmark)

    Christensen, Peter Ove; Larsen, Kasper

    2014-01-01

    We derive closed-form solutions for the equilibrium interest rate and market price of risk processes in an incomplete continuous-time market with uncertainty generated by Brownian motions. The economy has a finite number of heterogeneous exponential utility investors, who receive partially...... equilibrium displays both lower interest rates and higher risk premia compared to the equilibrium in an otherwise identical complete market....

  19. A mean-variance frontier in discrete and continuous time

    NARCIS (Netherlands)

    Bekker, Paul A.

    2004-01-01

    The paper presents a mean-variance frontier based on dynamic frictionless investment strategies in continuous time. The result applies to a finite number of risky assets whose price process is given by multivariate geometric Brownian motion with deterministically varying coefficients. The derivation

  20. Integral-Value Models for Outcomes over Continuous Time

    DEFF Research Database (Denmark)

    Harvey, Charles M.; Østerdal, Lars Peter

    Models of preferences between outcomes over continuous time are important for individual, corporate, and social decision making, e.g., medical treatment, infrastructure development, and environmental regulation. This paper presents a foundation for such models. It shows that conditions on prefere...

  1. Continuous Time Portfolio Selection under Conditional Capital at Risk

    Directory of Open Access Journals (Sweden)

    Gordana Dmitrasinovic-Vidovic

    2010-01-01

    Full Text Available Portfolio optimization with respect to different risk measures is of interest to both practitioners and academics. For there to be a well-defined optimal portfolio, it is important that the risk measure be coherent and quasiconvex with respect to the proportion invested in risky assets. In this paper we investigate one such measure—conditional capital at risk—and find the optimal strategies under this measure, in the Black-Scholes continuous time setting, with time dependent coefficients.

  2. Bayesian Markov-Chain-Monte-Carlo inversion of time-lapse crosshole GPR data to characterize the vadose zone at the Arrenaes Site, Denmark

    DEFF Research Database (Denmark)

    Scholer, Marie; Irving, James; Zibar, Majken Caroline Looms

    2012-01-01

    We examined to what extent time-lapse crosshole ground-penetrating radar traveltimes, measured during a forced infiltration experiment at the Arreneas field site in Denmark, could help to quantify vadose zone hydraulic properties and their corresponding uncertainties using a Bayesian Markov......-chain-Monte-Carlo inversion approach with different priors. The ground-penetrating radar (GPR) geophysical method has the potential to provide valuable information on the hydraulic properties of the vadose zone because of its strong sensitivity to soil water content. In particular, recent evidence has suggested...... that the stochastic inversion of crosshole GPR traveltime data can allow for a significant reduction in uncertainty regarding subsurface van Genuchten–Mualem (VGM) parameters. Much of the previous work on the stochastic estimation of VGM parameters from crosshole GPR data has considered the case of steady...

  3. Anomalous diffusion in correlated continuous time random walks

    Energy Technology Data Exchange (ETDEWEB)

    Tejedor, Vincent; Metzler, Ralf, E-mail: metz@ph.tum.d [Physics Department T30 g, Technical University of Munich, 85747 Garching (Germany)

    2010-02-26

    We demonstrate that continuous time random walks in which successive waiting times are correlated by Gaussian statistics lead to anomalous diffusion with the mean squared displacement (r{sup 2}(t)) {approx_equal} t{sup 2/3}. Long-ranged correlations of the waiting times with a power-law exponent alpha (0 < alpha <= 2) give rise to subdiffusion of the form (r{sup 2}(t)) {approx_equal} t{sup {alpha}/(1+{alpha})}. In contrast, correlations in the jump lengths are shown to produce superdiffusion. We show that in both cases weak ergodicity breaking occurs. Our results are in excellent agreement with simulations. (fast track communication)

  4. Continuous Time Random Walks for the Evolution of Lagrangian Velocities

    CERN Document Server

    Dentz, Marco; Comolli, Alessandro; Borgne, Tanguy Le; Lester, Daniel R

    2016-01-01

    We develop a continuous time random walk (CTRW) approach for the evolution of Lagrangian velocities in steady heterogeneous flows based on a stochastic relaxation process for the streamwise particle velocities. This approach describes persistence of velocities over a characteristic spatial scale, unlike classical random walk methods, which model persistence over a characteristic time scale. We first establish the relation between Eulerian and Lagrangian velocities for both equidistant and isochrone sampling along streamlines, under transient and stationary conditions. Based on this, we develop a space continuous CTRW approach for the spatial and temporal dynamics of Lagrangian velocities. While classical CTRW formulations have non-stationary Lagrangian velocity statistics, the proposed approach quantifies the evolution of the Lagrangian velocity statistics under both stationary and non-stationary conditions. We provide explicit expressions for the Lagrangian velocity statistics, and determine the behaviors of...

  5. Incomplete Continuous-Time Securities Markets with Stochastic Income Volatility

    DEFF Research Database (Denmark)

    Christensen, Peter Ove; Larsen, Kasper

    and can trade continuously on a finite time interval in a money market account and a single risky security. Besides establishing the existence of an equilibrium, our main result shows that if the investors' unspanned income has stochastic counter-cyclical volatility, the resulting equilibrium can display......In an incomplete continuous-time securities market governed by Brownian motions, we derive closed-form solutions for the equilibrium risk-free rate and equity premium processes. The economy has a finite number of heterogeneous exponential utility investors, who receive partially unspanned income...... both lower risk-free rates and higher risk premia relative to the Pareto efficient equilibrium in an otherwise identical complete market. Consequently, our model can simultaneously help explaining the risk-free rate and equity premium puzzles....

  6. Incomplete Continuous-Time Securities Markets with Stochastic Income Volatility

    DEFF Research Database (Denmark)

    Christensen, Peter Ove; Larsen, Kasper

    and can trade continuously on a finite time interval in a money market account and a single risky security. Besides establishing the existence of an equilibrium, our main result shows that if the investors' unspanned income has stochastic counter-cyclical volatility, the resulting equilibrium can display......In an incomplete continuous-time securities market governed by Brownian motions, we derive closed-form solutions for the equilibrium risk-free rate and equity premium processes. The economy has a finite number of heterogeneous exponential utility investors, who receive partially unspanned income...... both lower risk-free rates and higher risk premia relative to the Pareto efficient equilibrium in an otherwise identical complete market. Consequently, our model can simultaneously help explaining the risk-free rate and equity premium puzzles....

  7. The average rate of change for continuous time models.

    Science.gov (United States)

    Kelley, Ken

    2009-05-01

    The average rate of change (ARC) is a concept that has been misunderstood in the applied longitudinal data analysis literature, where the slope from the straight-line change model is often thought of as though it were the ARC. The present article clarifies the concept of ARC and shows unequivocally the mathematical definition and meaning of ARC when measurement is continuous across time. It is shown that the slope from the straight-line change model generally is not equal to the ARC. General equations are presented for two measures of discrepancy when the slope from the straight-line change model is used to estimate the ARC in the case of continuous time for any model linear in its parameters, and for three useful models nonlinear in their parameters.

  8. Introducing the Dimensional Continuous Space-Time Theory

    Science.gov (United States)

    Martini, Luiz Cesar

    2013-04-01

    This article is an introduction to a new theory. The name of the theory is justified by the dimensional description of the continuous space-time of the matter, energy and empty space, that gathers all the real things that exists in the universe. The theory presents itself as the consolidation of the classical, quantum and relativity theories. A basic equation that describes the formation of the Universe, relating time, space, matter, energy and movement, is deduced. The four fundamentals physics constants, light speed in empty space, gravitational constant, Boltzmann's constant and Planck's constant and also the fundamentals particles mass, the electrical charges, the energies, the empty space and time are also obtained from this basic equation. This theory provides a new vision of the Big-Bang and how the galaxies, stars, black holes and planets were formed. Based on it, is possible to have a perfect comprehension of the duality between wave-particle, which is an intrinsic characteristic of the matter and energy. It will be possible to comprehend the formation of orbitals and get the equationing of atomics orbits. It presents a singular comprehension of the mass relativity, length and time. It is demonstrated that the continuous space-time is tridimensional, inelastic and temporally instantaneous, eliminating the possibility of spatial fold, slot space, worm hole, time travels and parallel universes. It is shown that many concepts, like dark matter and strong forces, that hypothetically keep the cohesion of the atomics nucleons, are without sense.

  9. Continuous-Time Delta-Sigma Modulators for Wireless Communication

    OpenAIRE

    Andersson, Mattias

    2014-01-01

    The ever increasing data rates in wireless communication require analog to digital converters (ADCs) with greater requirements on speed and accuracy, while being power efficient to prolong battery life. This dissertation contains an introduction to the field and five papers that focus on the continuous-time (CT) Delta-Sigma modulator (DSM) as ADC. Paper I analyses the performance degradation of dynamic nonlinearity in the feedback DAC of the DSM, caused by Vth mismatch in the current-s...

  10. Continuous Time Quantum Monte Carlo simulation of Kondo shuttling

    Science.gov (United States)

    Zhang, Peng; Assaad, Fakher; Jarrell, Mark

    2010-03-01

    The Kondo shuttling problem is investigated by using the Continuous Time Quantum Monte Carlo method in both the anti-adiabatic limit φTK and the intermediate regime φ˜TK, where φ is the phonon modulation frequency and TK is the Kondo temperature. We investigate the potential emergence of Kondo effect or Kondo breakdown as a function of the phonon modulation frequency and electron-phonon coupling. This research is supported by grant OISE-0952300.

  11. Monte Carlo methods in continuous time for lattice Hamiltonians

    CERN Document Server

    Huffman, Emilie

    2016-01-01

    We solve a variety of sign problems for models in lattice field theory using the Hamiltonian formulation, including Yukawa models and simple lattice gauge theories. The solutions emerge naturally in continuous time and use the dual representation for the bosonic fields. These solutions allow us to construct quantum Monte Carlo methods for these problems. The methods could provide an alternative approach to understanding non-perturbative dynamics of some lattice field theories.

  12. Continuous time limits of the Utterance Selection Model

    CERN Document Server

    Michaud, Jérôme

    2016-01-01

    In this paper, we derive new continuous time limits of the Utterance Selection Model (USM) for language change (Baxter et al., Phys. Rev. E {\\bf 73}, 046118, 2006). This is motivated by the fact that the Fokker-Planck continuous time limit derived in the original version of the USM is only valid for a small range range of parameters. We investigate the consequences of relaxing these constraints on parameters. Using the normal approximation of the multinomial approximation, we derive a new continuous time limit of the USM in the form of a weak-noise stochastic differential equation. We argue that this weak noise, not captured by the Kramers-Moyal expansion, can not be neglected. We then propose a coarse-graining procedure, which takes the form of a stochastic version of the \\emph{heterogeneous mean field} approximation. This approximation groups the behaviour of nodes of same degree, reducing the complexity of the problem. With the help of this approximation, we study in detail two simple families of networks:...

  13. Fields From Markov Chains

    DEFF Research Database (Denmark)

    Justesen, Jørn

    2005-01-01

    A simple construction of two-dimensional (2-D) fields is presented. Rows and columns are outcomes of the same Markov chain. The entropy can be calculated explicitly.......A simple construction of two-dimensional (2-D) fields is presented. Rows and columns are outcomes of the same Markov chain. The entropy can be calculated explicitly....

  14. Decisive Markov Chains

    CERN Document Server

    Abdulla, Parosh Aziz; Mayr, Richard

    2007-01-01

    We consider qualitative and quantitative verification problems for infinite-state Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which F can no longer be reached. While all finite Markov chains are trivially decisive (for every set F), this also holds for many classes of infinite Markov chains. Infinite Markov chains which contain a finite attractor are decisive w.r.t. every set F. In particular, this holds for probabilistic lossy channel systems (PLCS). Furthermore, all globally coarse Markov chains are decisive. This class includes probabilistic vector addition systems (PVASS) and probabilistic noisy Turing machines (PNTM). We consider both safety and liveness problems for decisive Markov chains, i.e., the probabilities that a given set of states F is eventually reached or reached infinitely often, respectively. 1. We express the qualitative problems in abstract terms for decisive Markov chains, and show...

  15. Study of Simplification of Markov Model for Analyzing System Dependability

    Energy Technology Data Exchange (ETDEWEB)

    Son, Gwang Seop; Kim, Dong Hoon; Choi, Jong Gyun [Korea Atomic Energy Research Institute, Daejeon (Korea, Republic of)

    2015-05-15

    In this paper, we introduce the simplification methodology of the Markov model for analyzing system dependability using system failure rate concept. This system failure rate is the probability that the system is failed or unavailable given that the system was as good as at this time. Using this parameter, the Markov model of sub system can be replaced to the system failure rate and then this parameter just is considered in the Markov model of whole system. In this paper, we proposed the method to simplify the Markov model in complex system architecture. We define the system failure rate and using this parameter, the Markov model of system could be simplified.

  16. Turbulent pair dispersion as a continuous-time random walk

    CERN Document Server

    Thalabard, Simon; Bec, Jeremie

    2014-01-01

    The phenomenology of turbulent relative dispersion is revisited. A heuristic scenario is proposed, in which pairs of tracers undergo a succession of independent ballistic separations during time intervals whose lengths fluctuate. This approach suggests that the logarithm of the distance between tracers self-averages and performs a continuous-time random walk. This leads to specific predictions for the probability distribution of separations, that differ from those obtained using scale-dependent eddy-diffusivity models (e.g. in the framework of Richardson's approach). Such predictions are tested against high-resolution simulations and shed new lights on the explosive separation between tracers.

  17. Dynamical continuous time random Lévy flights

    Science.gov (United States)

    Liu, Jian; Chen, Xiaosong

    2016-03-01

    The Lévy flights' diffusive behavior is studied within the framework of the dynamical continuous time random walk (DCTRW) method, while the nonlinear friction is introduced in each step. Through the DCTRW method, Lévy random walker in each step flies by obeying the Newton's Second Law while the nonlinear friction f(v) = - γ0v - γ2v3 being considered instead of Stokes friction. It is shown that after introducing the nonlinear friction, the superdiffusive Lévy flights converges, behaves localization phenomenon with long time limit, but for the Lévy index μ = 2 case, it is still Brownian motion.

  18. Coaction versus reciprocity in continuous-time models of cooperation.

    Science.gov (United States)

    van Doorn, G Sander; Riebli, Thomas; Taborsky, Michael

    2014-09-07

    Cooperating animals frequently show closely coordinated behaviours organized by a continuous flow of information between interacting partners. Such real-time coaction is not captured by the iterated prisoner's dilemma and other discrete-time reciprocal cooperation games, which inherently feature a delay in information exchange. Here, we study the evolution of cooperation when individuals can dynamically respond to each other's actions. We develop continuous-time analogues of iterated-game models and describe their dynamics in terms of two variables, the propensity of individuals to initiate cooperation (altruism) and their tendency to mirror their partner's actions (coordination). These components of cooperation stabilize at an evolutionary equilibrium or show oscillations, depending on the chosen payoff parameters. Unlike reciprocal altruism, cooperation by coaction does not require that those willing to initiate cooperation pay in advance for uncertain future benefits. Correspondingly, we show that introducing a delay to information transfer between players is equivalent to increasing the cost of cooperation. Cooperative coaction can therefore evolve much more easily than reciprocal cooperation. When delays entirely prevent coordination, we recover results from the discrete-time alternating prisoner's dilemma, indicating that coaction and reciprocity are connected by a continuum of opportunities for real-time information exchange.

  19. Nonuniform Markov models

    CERN Document Server

    Ristad, E S; Ristad, Eric Sven; Thomas, Robert G.

    1996-01-01

    A statistical language model assigns probability to strings of arbitrary length. Unfortunately, it is not possible to gather reliable statistics on strings of arbitrary length from a finite corpus. Therefore, a statistical language model must decide that each symbol in a string depends on at most a small, finite number of other symbols in the string. In this report we propose a new way to model conditional independence in Markov models. The central feature of our nonuniform Markov model is that it makes predictions of varying lengths using contexts of varying lengths. Experiments on the Wall Street Journal reveal that the nonuniform model performs slightly better than the classic interpolated Markov model. This result is somewhat remarkable because both models contain identical numbers of parameters whose values are estimated in a similar manner. The only difference between the two models is how they combine the statistics of longer and shorter strings. Keywords: nonuniform Markov model, interpolated Markov m...

  20. Minimal Markov Models

    CERN Document Server

    Gonzalez-Lopez, Jesus E Garcia Veronica A

    2010-01-01

    In this work we introduce a new and richer class of finite order Markov chain models and address the following model selection problem: find the Markov model with the minimal set of parameters (minimal Markov model) which is necessary to represent a source as a Markov chain of finite order. Let us call $M$ the order of the chain and $A$ the finite alphabet, to determine the minimal Markov model, we define an equivalence relation on the state space $A^{M}$, such that all the sequences of size $M$ with the same transition probabilities are put in the same category. In this way we have one set of $(|A|-1)$ transition probabilities for each category, obtaining a model with a minimal number of parameters. We show that the model can be selected consistently using the Bayesian information criterion.

  1. Continuous time limits of the utterance selection model

    Science.gov (United States)

    Michaud, Jérôme

    2017-02-01

    In this paper we derive alternative continuous time limits of the utterance selection model (USM) for language change [G. J. Baxter et al., Phys. Rev. E 73, 046118 (2006), 10.1103/PhysRevE.73.046118]. This is motivated by the fact that the Fokker-Planck continuous time limit derived in the original version of the USM is only valid for a small range of parameters. We investigate the consequences of relaxing these constraints on parameters. Using the normal approximation of the multinomial approximation, we derive a continuous time limit of the USM in the form of a weak-noise stochastic differential equation. We argue that this weak noise, not captured by the Kramers-Moyal expansion, cannot be neglected. We then propose a coarse-graining procedure, which takes the form of a stochastic version of the heterogeneous mean field approximation. This approximation groups the behavior of nodes of the same degree, reducing the complexity of the problem. With the help of this approximation, we study in detail two simple families of networks: the regular networks and the star-shaped networks. The analysis reveals and quantifies a finite-size effect of the dynamics. If we increase the size of the network by keeping all the other parameters constant, we transition from a state where conventions emerge to a state where no convention emerges. Furthermore, we show that the degree of a node acts as a time scale. For heterogeneous networks such as star-shaped networks, the time scale difference can become very large, leading to a noisier behavior of highly connected nodes.

  2. Pattern Recognition Technology Based on Continuous Hidden Markov Model and its Application%基于连续隐马尔柯夫模型的模式识别技术及其应用

    Institute of Scientific and Technical Information of China (English)

    刘伯高

    2015-01-01

    对利用基因算法训练连续隐马尔柯夫模型的语音识别的具体算法进行系统的研究;然后基于该语音识别技术对深圳市司法局社区矫正声纹识别系统进行详细设计。该系统上线后的运行结果表明,利用基因算法训练连续隐马尔柯夫模型的语音识别算法的识别速度较快同时具有较高的识别率。基于模式识别技术的司法社区矫正声纹识别系统建设在我国司法系统目前尚处于起步阶段,推广和建设司法社区矫正声纹识别系统具有重要的现实意义。%Systematic research was done on the specific algorithm for speech recognition in using genetic algorithm to train continuous hidden Markov mode.Then the detailed design of Voiceprint Recognition System of Community Correction Objects in the Shenzhen City Bureau of Justice has been done based on the speech recognition technology.The system run-ning results show that the recognition rate of recognition algorithm using genetic algorithm to train continuous hidden Mark-ov model is faster and has a higher rate of recognition.Construction of voiceprint recognition system of judicial community correction objects based on pattern recognition is still in the junior stage in our judicial system,and promotion and the con-struction of voiceprint recognition system of judicial community correction objects have the important practical significance.

  3. Nonparametric model validations for hidden Markov models with applications in financial econometrics.

    Science.gov (United States)

    Zhao, Zhibiao

    2011-06-01

    We address the nonparametric model validation problem for hidden Markov models with partially observable variables and hidden states. We achieve this goal by constructing a nonparametric simultaneous confidence envelope for transition density function of the observable variables and checking whether the parametric density estimate is contained within such an envelope. Our specification test procedure is motivated by a functional connection between the transition density of the observable variables and the Markov transition kernel of the hidden states. Our approach is applicable for continuous time diffusion models, stochastic volatility models, nonlinear time series models, and models with market microstructure noise.

  4. Statistical inference for 2-type doubly symmetric critical irreducible continuous state and continuous time branching processes with immigration

    OpenAIRE

    2014-01-01

    We study asymptotic behavior of conditional least squares estimators for 2-type doubly symmetric critical irreducible continuous state and continuous time branching processes with immigration based on discrete time (low frequency) observations.

  5. Behavior of an Almost Semicontinuous Poisson Process on a Markov Chain Upon Attainment of A Level

    CERN Document Server

    Karnaukh, Ievgen

    2011-01-01

    We consider the almost semi-continuous processes defined on a finite Markov chain. The representation of the moment generating functions for the absolute maximum after achievement positive level and for the recovery time are obtained. Modified processes with two-step rate of negative jumps are investigated.

  6. Average, sensitive and Blackwell-optimal policies in denumerable Markov decision chains with unbounded rewards

    NARCIS (Netherlands)

    R. Dekker (Rommert); A. Hordijk (Arie)

    1988-01-01

    textabstractIn this paper we consider a (discrete-time) Markov decision chain with a denumerabloe state space and compact action sets and we assume that for all states the rewards and transition probabilities depend continuously on the actions. The first objective of this paper is to develop an anal

  7. The invariant measure of homogeneous Markov processes in the quarter-plane: Representation in geometric terms

    NARCIS (Netherlands)

    Chen, Yanting; Boucherie, Richard J.; Goseling, Jasper

    2011-01-01

    We consider the invariant measure of a homogeneous continuous-time Markov process in the quarter-plane. The basic solutions of the global balance equation are the geometric distributions. We first show that the invariant measure can not be a finite linear combination of basic geometric distributions

  8. Distinct timing mechanisms produce discrete and continuous movements.

    Directory of Open Access Journals (Sweden)

    Raoul Huys

    2008-04-01

    Full Text Available The differentiation of discrete and continuous movement is one of the pillars of motor behavior classification. Discrete movements have a definite beginning and end, whereas continuous movements do not have such discriminable end points. In the past decade there has been vigorous debate whether this classification implies different control processes. This debate up until the present has been empirically based. Here, we present an unambiguous non-empirical classification based on theorems in dynamical system theory that sets discrete and continuous movements apart. Through computational simulations of representative modes of each class and topological analysis of the flow in state space, we show that distinct control mechanisms underwrite discrete and fast rhythmic movements. In particular, we demonstrate that discrete movements require a time keeper while fast rhythmic movements do not. We validate our computational findings experimentally using a behavioral paradigm in which human participants performed finger flexion-extension movements at various movement paces and under different instructions. Our results demonstrate that the human motor system employs different timing control mechanisms (presumably via differential recruitment of neural subsystems to accomplish varying behavioral functions such as speed constraints.

  9. Educating Families on Real Time Continuous Glucose Monitoring

    Science.gov (United States)

    Messer, Laurel; Ruedy, Katrina; Xing, Dongyuan; Coffey, Julie; Englert, Kimberly; Caswell, Kimberly; Ives, Brett

    2013-01-01

    Purpose The purpose of this article is to describe the process of educating families and children with type 1 diabetes on real time continuous glucose monitoring (RT-CGM) and to note the similarities and differences of training patients using continuous subcutaneous insulin infusion (CSII) versus multiple daily injections (MDI). Methods A total of 30 CSII participants and 27 MDI participants were educated using the Navigator RT-CGM in a clinical trial. Time spent with families for visits and calls was tracked and compared between patient groups. The Diabetes Research in Children Network (DirecNet) educators were surveyed to assess the most crucial, time intensive, and difficult educational concepts related to CGM. Results Of the 27 MDI families, an average of 9.6 hours was spent on protocol-prescribed visits and calls (not measured in CSII) and 2 hours on participant-initiated contacts over 3 months. MDI families required an average of 5.4 more phone contacts over 3 months than CSII families. According to the DirecNet educators, lag time and calibrations were the most crucial teaching concepts for successful RT-CGM use. The most time was spent on teaching technical aspects, troubleshooting, and insulin dosing. The most unanticipated difficulties were skin problems including irritation and the sensor not adhering well. Conclusion Educators who teach RT-CGM should emphasize lag time and calibration techniques, technical device training, and sensor insertion. Follow-up focus should include insulin dosing adjustments and skin issues. The time and effort required to introduce RT-CGM provided an opportunity for the diabetes educators to reemphasize good diabetes care practices and promote self-awareness and autonomy to patients and families. PMID:19244568

  10. On the Total Variation Distance of Semi-Markov Chains

    DEFF Research Database (Denmark)

    Bacci, Giorgio; Bacci, Giovanni; Larsen, Kim Guldstrand

    2015-01-01

    linear real-time specifications. Specifically, we prove that the total variation between two SMCs coincides with the maximal difference w.r.t. the likelihood of satisfying arbitrary MTL formulas or omega-languages recognized by timed automata. Computing this distance (i.e., solving its threshold problem......Semi-Markov chains (SMCs) are continuous-time probabilistic transition systems where the residence time on states is governed by generic distributions on the positive real line. This paper shows the tight relation between the total variation distance on SMCs and their model checking problem over...

  11. A continuous-time neural model for sequential action.

    Science.gov (United States)

    Kachergis, George; Wyatte, Dean; O'Reilly, Randall C; de Kleijn, Roy; Hommel, Bernhard

    2014-11-01

    Action selection, planning and execution are continuous processes that evolve over time, responding to perceptual feedback as well as evolving top-down constraints. Existing models of routine sequential action (e.g. coffee- or pancake-making) generally fall into one of two classes: hierarchical models that include hand-built task representations, or heterarchical models that must learn to represent hierarchy via temporal context, but thus far lack goal-orientedness. We present a biologically motivated model of the latter class that, because it is situated in the Leabra neural architecture, affords an opportunity to include both unsupervised and goal-directed learning mechanisms. Moreover, we embed this neurocomputational model in the theoretical framework of the theory of event coding (TEC), which posits that actions and perceptions share a common representation with bidirectional associations between the two. Thus, in this view, not only does perception select actions (along with task context), but actions are also used to generate perceptions (i.e. intended effects). We propose a neural model that implements TEC to carry out sequential action control in hierarchically structured tasks such as coffee-making. Unlike traditional feedforward discrete-time neural network models, which use static percepts to generate static outputs, our biological model accepts continuous-time inputs and likewise generates non-stationary outputs, making short-timescale dynamic predictions.

  12. A Monte Carlo study of time-aggregation in continuous-time and discrete-time parametric hazard models.

    NARCIS (Netherlands)

    Hofstede, ter F.; Wedel, M.

    1998-01-01

    This study investigates the effects of time aggregation in discrete and continuous-time hazard models. A Monte Carlo study is conducted in which data are generated according to various continuous and discrete-time processes, and aggregated into daily, weekly and monthly intervals. These data are

  13. Speed and entropy of an interacting continuous time quantum walk

    CERN Document Server

    De Falco, D; Falco, Diego de; Tamascelli, Dario

    2006-01-01

    We present some dynamic and entropic considerations about the evolution of a continuous time quantum walk implementing the clock of an autonomous machine. On a simple model, we study in quite explicit terms the Lindblad evolution of the clocked subsystem, relating the evolution of its entropy to the spreading of the wave packet of the clock. We explore possible ways of reducing the generation of entropy in the clocked subsystem, as it amounts to a deficit in the probability of finding the target state of the computation. We are thus lead to examine the benefits of abandoning some classical prejudice about how a clocking mechanism should operate.

  14. STUDY ON CONTINUOUS-TIME HEDGING PROBLEM IN INCOMPLETE MARKETS

    Institute of Scientific and Technical Information of China (English)

    刘海龙; 吴冲锋

    2002-01-01

    This paper extended the continuous-time dynamic-hedging theorem for the incomplete markets of Bertsimas, Kogan and Lo's to the case in which riskless interest rate is not zero. The theorem was then proved with the stochastic dynamic programming theory, by constructing a self-financing dynamic strategy that best approximates an arbitrary payoff function in the mean-squared sense. When the riskless interest rate is zero, our optimal hedging strategy coincides with the results of Bertsimas, Kogan and Lo,i.e. their results are special cases of ours.

  15. Continuous-time quantum walks on multilayer dendrimer networks

    Science.gov (United States)

    Galiceanu, Mircea; Strunz, Walter T.

    2016-08-01

    We consider continuous-time quantum walks (CTQWs) on multilayer dendrimer networks (MDs) and their application to quantum transport. A detailed study of properties of CTQWs is presented and transport efficiency is determined in terms of the exact and average return probabilities. The latter depends only on the eigenvalues of the connectivity matrix, which even for very large structures allows a complete analytical solution for this particular choice of network. In the case of MDs we observe an interplay between strong localization effects, due to the dendrimer topology, and good efficiency from the linear segments. We show that quantum transport is enhanced by interconnecting more layers of dendrimers.

  16. Measuring and modelling occupancy time in NHS continuing healthcare

    Directory of Open Access Journals (Sweden)

    Millard Peter H

    2011-06-01

    Full Text Available Abstract Background Due to increasing demand and financial constraints, NHS continuing healthcare systems seek to find better ways of forecasting demand and budgeting for care. This paper investigates two areas of concern, namely, how long existing patients stay in service and the number of patients that are likely to be still in care after a period of time. Methods An anonymised dataset containing information for all funded admissions to placement and home care in the NHS continuing healthcare system was provided by 26 (out of 31 London primary care trusts. The data related to 11289 patients staying in placement and home care between 1 April 2005 and 31 May 2008 were first analysed. Using a methodology based on length of stay (LoS modelling, we captured the distribution of LoS of patients to estimate the probability of a patient staying in care over a period of time. Using the estimated probabilities we forecasted the number of patients that are likely to be still in care after a period of time (e.g. monthly. Results We noticed that within the NHS continuing healthcare system there are three main categories of patients. Some patients are discharged after a short stay (few days, some others staying for few months and the third category of patients staying for a long period of time (years. Some variations in proportions of discharge and transition between types of care as well as between care groups (e.g. palliative, functional mental health were observed. A close agreement of the observed and the expected numbers of patients suggests a good prediction model. Conclusions The model was tested for care groups within the NHS continuing healthcare system in London to support Primary Care Trusts in budget planning and improve their responsiveness to meet the increasing demand under limited availability of resources. Its applicability can be extended to other types of care, such as hospital care and re-ablement. Further work will be geared towards

  17. Finite time convergent learning law for continuous neural networks.

    Science.gov (United States)

    Chairez, Isaac

    2014-02-01

    This paper addresses the design of a discontinuous finite time convergent learning law for neural networks with continuous dynamics. The neural network was used here to obtain a non-parametric model for uncertain systems described by a set of ordinary differential equations. The source of uncertainties was the presence of some external perturbations and poor knowledge of the nonlinear function describing the system dynamics. A new adaptive algorithm based on discontinuous algorithms was used to adjust the weights of the neural network. The adaptive algorithm was derived by means of a non-standard Lyapunov function that is lower semi-continuous and differentiable in almost the whole space. A compensator term was included in the identifier to reject some specific perturbations using a nonlinear robust algorithm. Two numerical examples demonstrated the improvements achieved by the learning algorithm introduced in this paper compared to classical schemes with continuous learning methods. The first one dealt with a benchmark problem used in the paper to explain how the discontinuous learning law works. The second one used the methane production model to show the benefits in engineering applications of the learning law proposed in this paper.

  18. MARKOV CHAIN MODELING OF PERFORMANCE DEGRADATION OF PHOTOVOLTAIC SYSTEM

    Directory of Open Access Journals (Sweden)

    E. Suresh Kumar

    2012-01-01

    Full Text Available Modern probability theory studies chance processes for which theknowledge of previous outcomes influence predictions for future experiments. In principle, when a sequence of chance experiments, all of the past outcomes could influence the predictions for the next experiment. In Markov chain type of chance, the outcome of a given experiment can affect the outcome of the next experiment. The system state changes with time and the state X and time t are two random variables. Each of these variables can be either continuous or discrete. Various degradation on photovoltaic (PV systems can be viewed as different Markov states and further degradation can be treated as the outcome of the present state. The PV system is treated as a discrete state continuous time system with four possible outcomes, namely, s1 : Good condition, s2 : System with partial degradation failures and fully operational, s3 : System with major faults and partially working and hence partial output power, s4 : System completely fails. The calculation of the reliability of the photovoltaic system is complicated since the system have elements or subsystems exhibiting dependent failures and involving repair and standby operations. Markov model is a better technique that has much appeal and works well when failure hazards and repair hazards are constant. The usual practice of reliability analysis techniques include FMEA((failure mode and effect analysis, Parts count analysis, RBD ( reliability block diagram , FTA( fault tree analysis etc. These are logical, boolean and block diagram approaches and never accounts the environmental degradation on the performance of the system. This is too relevant in the case of PV systems which are operated under harsh environmental conditions. This paper is an insight into the degradation of performance of PV systems and presenting a Markov model of the system by means of the different states and transitions between these states.

  19. Verification of Continuous Dynamical Systems by Timed Automata

    DEFF Research Database (Denmark)

    Sloth, Christoffer; Wisniewski, Rafael

    2011-01-01

    This paper presents a method for abstracting continuous dynamical systems by timed automata. The abstraction is based on partitioning the state space of a dynamical system using positive invariant sets, which form cells that represent locations of a timed automaton. The abstraction is intended...... to enable formal verification of temporal properties of dynamical systems without simulating any system trajectory, which is currently not possible. Therefore, conditions for obtaining sound, complete, and refinable abstractions are set up. The novelty of the method is the partitioning of the state space...... of the verification process. The proposed abstraction is applied to two examples, which illustrate how sound and complete abstractions are generated and the type of specification we can check. Finally, an example shows how the compositionality of the abstraction can be used to analyze a high-dimensional system....

  20. Correlated continuous time random walk and option pricing

    Science.gov (United States)

    Lv, Longjin; Xiao, Jianbin; Fan, Liangzhong; Ren, Fuyao

    2016-04-01

    In this paper, we study a correlated continuous time random walk (CCTRW) with averaged waiting time, whose probability density function (PDF) is proved to follow stretched Gaussian distribution. Then, we apply this process into option pricing problem. Supposing the price of the underlying is driven by this CCTRW, we find this model captures the subdiffusive characteristic of financial markets. By using the mean self-financing hedging strategy, we obtain the closed-form pricing formulas for a European option with and without transaction costs, respectively. At last, comparing the obtained model with the classical Black-Scholes model, we find the price obtained in this paper is higher than that obtained from the Black-Scholes model. A empirical analysis is also introduced to confirm the obtained results can fit the real data well.

  1. Fuzzy Markov chains: uncertain probabilities

    OpenAIRE

    2002-01-01

    We consider finite Markov chains where there are uncertainties in some of the transition probabilities. These uncertainties are modeled by fuzzy numbers. Using a restricted fuzzy matrix multiplication we investigate the properties of regular, and absorbing, fuzzy Markov chains and show that the basic properties of these classical Markov chains generalize to fuzzy Markov chains.

  2. Timing skills and expertise: discrete and continuous timed movements among musicians and athletes

    Science.gov (United States)

    Braun Janzen, Thenille; Thompson, William Forde; Ammirante, Paolo; Ranvaud, Ronald

    2014-01-01

    Introduction: Movement-based expertise relies on precise timing of movements and the capacity to predict the timing of events. Music performance involves discrete rhythmic actions that adhere to regular cycles of timed events, whereas many sports involve continuous movements that are not timed in a cyclical manner. It has been proposed that the precision of discrete movements relies on event timing (clock mechanism), whereas continuous movements are controlled by emergent timing. We examined whether movement-based expertise influences the timing mode adopted to maintain precise rhythmic actions. Materials and Method: Timing precision was evaluated in musicians, athletes and control participants. Discrete and continuous movements were assessed using finger-tapping and circle-drawing tasks, respectively, based on the synchronization-continuation paradigm. In Experiment 1, no auditory feedback was provided in the continuation phase of the trials, whereas in Experiment 2 every action triggered a feedback tone. Results: Analysis of precision in the continuation phase indicated that athletes performed significantly better than musicians and controls in the circle-drawing task, whereas musicians were more precise than controls in the finger tapping task. Interestingly, musicians were also more precise than controls in the circle-drawing task. Results also showed that the timing mode adopted was dependent on expertise and the presence of auditory feedback. Discussion: Results showed that movement-based expertise is associated with enhanced timing, but these effects depend on the nature of the training. Expertise was found to influence the timing strategy adopted to maintain precise rhythmic movements, suggesting that event and emergent timing mechanisms are not strictly tied to specific tasks, but can both be adopted to achieve precise timing. PMID:25566154

  3. Timing skills and expertise: discrete and continuous timed movements among musicians and athletes

    Directory of Open Access Journals (Sweden)

    Thenille eBraun Janzen

    2014-12-01

    Full Text Available Introduction: Movement-based expertise relies on precise timing of movements and the capacity to predict the timing of events. Music performance involves discrete rhythmic actions that adhere to regular cycles of timed events, whereas many sports involve continuous movements that are not timed in a cyclical manner. It has been proposed that the precision of discrete movements relies on event timing (clock mechanism, whereas continuous movements are controlled by emergent timing. We examined whether movement-based expertise influences the timing mode adopted to maintain precise rhythmic actions. Materials and Method: Timing precision was evaluated in musicians, athletes and control participants. Discrete and continuous movements were assessed using finger-tapping and circle-drawing tasks, respectively, based on the synchronization-continuation paradigm. In Experiment 1, no auditory feedback was provided in the continuation phase of the trials, whereas in Experiment 2 every action triggered a feedback tone. Results: Analysis of precision in the continuation phase indicated that athletes performed significantly better than musicians and controls in the circle-drawing task, whereas musicians were more precise than controls in the finger tapping task. Interestingly, musicians were also more precise than controls in the circle-drawing task. Results also showed that the timing mode adopted was dependent on expertise and the presence of auditory feedback. Discussion: Results showed that movement-based expertise is associated with enhanced timing, but these effects depend on the nature of the training. Expertise was found to influence the timing strategy adopted to maintain precise rhythmic movements, suggesting that event and emergent timing mechanisms are not strictly tied to specific tasks, but can both be adopted to achieve precise timing.

  4. Markov Chain Ontology Analysis (MCOA)

    Science.gov (United States)

    2012-01-01

    Background Biomedical ontologies have become an increasingly critical lens through which researchers analyze the genomic, clinical and bibliographic data that fuels scientific research. Of particular relevance are methods, such as enrichment analysis, that quantify the importance of ontology classes relative to a collection of domain data. Current analytical techniques, however, remain limited in their ability to handle many important types of structural complexity encountered in real biological systems including class overlaps, continuously valued data, inter-instance relationships, non-hierarchical relationships between classes, semantic distance and sparse data. Results In this paper, we describe a methodology called Markov Chain Ontology Analysis (MCOA) and illustrate its use through a MCOA-based enrichment analysis application based on a generative model of gene activation. MCOA models the classes in an ontology, the instances from an associated dataset and all directional inter-class, class-to-instance and inter-instance relationships as a single finite ergodic Markov chain. The adjusted transition probability matrix for this Markov chain enables the calculation of eigenvector values that quantify the importance of each ontology class relative to other classes and the associated data set members. On both controlled Gene Ontology (GO) data sets created with Escherichia coli, Drosophila melanogaster and Homo sapiens annotations and real gene expression data extracted from the Gene Expression Omnibus (GEO), the MCOA enrichment analysis approach provides the best performance of comparable state-of-the-art methods. Conclusion A methodology based on Markov chain models and network analytic metrics can help detect the relevant signal within large, highly interdependent and noisy data sets and, for applications such as enrichment analysis, has been shown to generate superior performance on both real and simulated data relative to existing state-of-the-art approaches

  5. Markov information sources

    Science.gov (United States)

    Massey, J. L.

    1975-01-01

    A regular Markov source is defined as the output of a deterministic, but noisy, channel driven by the state sequence of a regular finite-state Markov chain. The rate of such a source is the per letter uncertainty of its digits. The well-known result that the rate of a unifilar regular Markov source is easily calculable is demonstrated, where unifilarity means that the present state of the Markov chain and the next output of the deterministic channel uniquely determine the next state. At present, there is no known method to calculate the rate of a nonunifilar source. Two tentative approaches to this unsolved problem are given, namely source identical twins and the master-slave source, which appear to shed some light on the question of rate calculation for a nonunifilar source.

  6. Markov information sources

    Science.gov (United States)

    Massey, J. L.

    1975-01-01

    A regular Markov source is defined as the output of a deterministic, but noisy, channel driven by the state sequence of a regular finite-state Markov chain. The rate of such a source is the per letter uncertainty of its digits. The well-known result that the rate of a unifilar regular Markov source is easily calculable is demonstrated, where unifilarity means that the present state of the Markov chain and the next output of the deterministic channel uniquely determine the next state. At present, there is no known method to calculate the rate of a nonunifilar source. Two tentative approaches to this unsolved problem are given, namely source identical twins and the master-slave source, which appear to shed some light on the question of rate calculation for a nonunifilar source.

  7. Stochastic Monotone Markov Integrated Semigroups%随机单调Markov积分半群

    Institute of Scientific and Technical Information of China (English)

    文兴易; 李扬荣

    2009-01-01

    In this paper we discuss the relationship between monotone of Markov integrated semigroups and transition functions,which are closely linked with each other for continuous time Markov chains.By this a necessary and sufficient condition for the minimal q-semigroup to be stochastic monotone is given in terms of their q-matrix only.%讨论了Markov积分半群的单调性和转移函数的单调性的等价性,并得到最小的Q半群是单调的充要条件.

  8. Algorithmic analysis of the maximum level length in general-block two-dimensional Markov processes

    Directory of Open Access Journals (Sweden)

    2006-01-01

    Full Text Available Two-dimensional continuous-time Markov chains (CTMCs are useful tools for studying stochastic models such as queueing, inventory, and production systems. Of particular interest in this paper is the distribution of the maximal level visited in a busy period because this descriptor provides an excellent measure of the system congestion. We present an algorithmic analysis for the computation of its distribution which is valid for Markov chains with general-block structure. For a multiserver batch arrival queue with retrials and negative arrivals, we exploit the underlying internal block structure and present numerical examples that reveal some interesting facts of the system.

  9. Markov and semi-Markov processes as a failure rate

    Science.gov (United States)

    Grabski, Franciszek

    2016-06-01

    In this paper the reliability function is defined by the stochastic failure rate process with a non negative and right continuous trajectories. Equations for the conditional reliability functions of an object, under assumption that the failure rate is a semi-Markov process with an at most countable state space are derived. A proper theorem is presented. The linear systems of equations for the appropriate Laplace transforms allow to find the reliability functions for the alternating, the Poisson and the Furry-Yule failure rate processes.

  10. Observation uncertainty in reversible Markov chains.

    Science.gov (United States)

    Metzner, Philipp; Weber, Marcus; Schütte, Christof

    2010-09-01

    In many applications one is interested in finding a simplified model which captures the essential dynamical behavior of a real life process. If the essential dynamics can be assumed to be (approximately) memoryless then a reasonable choice for a model is a Markov model whose parameters are estimated by means of Bayesian inference from an observed time series. We propose an efficient Monte Carlo Markov chain framework to assess the uncertainty of the Markov model and related observables. The derived Gibbs sampler allows for sampling distributions of transition matrices subject to reversibility and/or sparsity constraints. The performance of the suggested sampling scheme is demonstrated and discussed for a variety of model examples. The uncertainty analysis of functions of the Markov model under investigation is discussed in application to the identification of conformations of the trialanine molecule via Robust Perron Cluster Analysis (PCCA+) .

  11. Markov and mixed models with applications

    DEFF Research Database (Denmark)

    Mortensen, Stig Bousgaard

    This thesis deals with mathematical and statistical models with focus on applications in pharmacokinetic and pharmacodynamic (PK/PD) modelling. These models are today an important aspect of the drug development in the pharmaceutical industry and continued research in statistical methodology within...... as a deterministic mean value using ordinary differential equations to which a random error is added. This thesis explores methods based on stochastic differential equations (SDEs) to extend the models to more adequately describe both true random biological variations and also variations due to unknown...... the individual in almost any thinkable way. This project focuses on measuring the eects on sleep in both humans and animals. The sleep process is usually analyzed by categorizing small time segments into a number of sleep states and this can be modelled using a Markov process. For this purpose new methods...

  12. A Markov Chain Model for Contagion

    Directory of Open Access Journals (Sweden)

    Angelos Dassios

    2014-11-01

    Full Text Available We introduce a bivariate Markov chain counting process with contagion for modelling the clustering arrival of loss claims with delayed settlement for an insurance company. It is a general continuous-time model framework that also has the potential to be applicable to modelling the clustering arrival of events, such as jumps, bankruptcies, crises and catastrophes in finance, insurance and economics with both internal contagion risk and external common risk. Key distributional properties, such as the moments and probability generating functions, for this process are derived. Some special cases with explicit results and numerical examples and the motivation for further actuarial applications are also discussed. The model can be considered a generalisation of the dynamic contagion process introduced by Dassios and Zhao (2011.

  13. Boundary value problems and Markov processes

    CERN Document Server

    Taira, Kazuaki

    1991-01-01

    Focussing on the interrelations of the subjects of Markov processes, analytic semigroups and elliptic boundary value problems, this monograph provides a careful and accessible exposition of functional methods in stochastic analysis. The author studies a class of boundary value problems for second-order elliptic differential operators which includes as particular cases the Dirichlet and Neumann problems, and proves that this class of boundary value problems provides a new example of analytic semigroups both in the Lp topology and in the topology of uniform convergence. As an application, one can construct analytic semigroups corresponding to the diffusion phenomenon of a Markovian particle moving continuously in the state space until it "dies", at which time it reaches the set where the absorption phenomenon occurs. A class of initial-boundary value problems for semilinear parabolic differential equations is also considered. This monograph will appeal to both advanced students and researchers as an introductio...

  14. Universal Denoising of Discrete-time Continuous-Amplitude Signals

    CERN Document Server

    Sivaramakrishnan, Kamakshi

    2008-01-01

    We consider the problem of reconstructing a discrete-time signal (sequence) with continuous-valued components corrupted by a known memoryless channel. When performance is measured using a per-symbol loss function satisfying mild regularity conditions, we develop a sequence of denoisers that, although independent of the distribution of the underlying `clean' sequence, is universally optimal in the limit of large sequence length. This sequence of denoisers is universal in the sense of performing as well as any sliding window denoising scheme which may be optimized for the underlying clean signal. Our results are initially developed in a ``semi-stochastic'' setting, where the noiseless signal is an unknown individual sequence, and the only source of randomness is due to the channel noise. It is subsequently shown that in the fully stochastic setting, where the noiseless sequence is a stationary stochastic process, our schemes universally attain optimum performance. The proposed schemes draw from nonparametric de...

  15. Continuous-time quantum Monte Carlo using worm sampling

    Science.gov (United States)

    Gunacker, P.; Wallerberger, M.; Gull, E.; Hausoel, A.; Sangiovanni, G.; Held, K.

    2015-10-01

    We present a worm sampling method for calculating one- and two-particle Green's functions using continuous-time quantum Monte Carlo simulations in the hybridization expansion (CT-HYB). Instead of measuring Green's functions by removing hybridization lines from partition function configurations, as in conventional CT-HYB, the worm algorithm directly samples the Green's function. We show that worm sampling is necessary to obtain general two-particle Green's functions which are not of density-density type and that it improves the sampling efficiency when approaching the atomic limit. Such two-particle Green's functions are needed to compute off-diagonal elements of susceptibilities and occur in diagrammatic extensions of the dynamical mean-field theory and in efficient estimators for the single-particle self-energy.

  16. Time will tell: resource continuity bolsters ecosystem services.

    Science.gov (United States)

    Schellhorn, Nancy A; Gagic, Vesna; Bommarco, Riccardo

    2015-09-01

    A common suggestion to support ecosystem services to agriculture provided by mobile organisms is to increase the amount of natural and seminatural habitat in the landscape. This might, however, be inefficient, and demands for agricultural products limit the feasibility of converting arable land into natural habitat. To develop more targeted means to promote ecosystem services, we need a solid understanding of the limitations to population growth for service-providing organisms. We propose a research agenda that identifies resource bottlenecks and interruptions over time to key beneficial organisms, emphasising their resulting population dynamics. Targeted measures that secure the continuity of resources throughout the life cycle of service-providing organisms are likely to effectively increase the stock, flow, and stability of ecosystem services.

  17. Real time continuous wavelet transform implementation on a DSP processor.

    Science.gov (United States)

    Patil, S; Abel, E W

    2009-01-01

    The continuous wavelet transform (CWT) is an effective tool when the emphasis is on the analysis of non-stationary signals and on localization and characterization of singularities in signals. We have used the B-spline based CWT, the Lipschitz Exponent (LE) and measures derived from it to detect and quantify the singularity characteristics of biomedical signals. In this article, a real-time implementation of a B-spline based CWT on a digital signal processor is presented, with the aim of providing quantitative information about the signal to a clinician as it is being recorded. A recursive algorithm implementation was shown to be too slow for real-time implementation so a parallel algorithm was considered. The use of a parallel algorithm involves redundancy in calculations at the boundary points. An optimization of numerical computation to remove redundancy in calculation was carried out. A formula has been derived to give an exact operation count for any integer scale m and any B-spline of order n (for the case where n is odd) to calculate the CWT for both the original and the optimized parallel methods. Experimental results show that the optimized method is 20-28% faster than the original method. As an example of applying this optimized method, a real-time implementation of the CWT with LE postprocessing has been achieved for an EMG Interference Pattern signal sampled at 50 kHz.

  18. Stochastic calculus for uncoupled continuous-time random walks.

    Science.gov (United States)

    Germano, Guido; Politi, Mauro; Scalas, Enrico; Schilling, René L

    2009-06-01

    The continuous-time random walk (CTRW) is a pure-jump stochastic process with several applications not only in physics but also in insurance, finance, and economics. A definition is given for a class of stochastic integrals driven by a CTRW, which includes the Itō and Stratonovich cases. An uncoupled CTRW with zero-mean jumps is a martingale. It is proved that, as a consequence of the martingale transform theorem, if the CTRW is a martingale, the Itō integral is a martingale too. It is shown how the definition of the stochastic integrals can be used to easily compute them by Monte Carlo simulation. The relations between a CTRW, its quadratic variation, its Stratonovich integral, and its Itō integral are highlighted by numerical calculations when the jumps in space of the CTRW have a symmetric Lévy alpha -stable distribution and its waiting times have a one-parameter Mittag-Leffler distribution. Remarkably, these distributions have fat tails and an unbounded quadratic variation. In the diffusive limit of vanishing scale parameters, the probability density of this kind of CTRW satisfies the space-time fractional diffusion equation (FDE) or more in general the fractional Fokker-Planck equation, which generalizes the standard diffusion equation, solved by the probability density of the Wiener process, and thus provides a phenomenologic model of anomalous diffusion. We also provide an analytic expression for the quadratic variation of the stochastic process described by the FDE and check it by Monte Carlo.

  19. Continuous time of flight measurements in a Lissajous configuration

    Science.gov (United States)

    Dobos, G.; Hárs, G.

    2017-01-01

    Short pulses used by traditional time-of-flight mass spectrometers limit their duty cycle, pose space-charge issues, and require high speed detectors and electronics. The motivation behind the invention of continuous time of flight mass spectrometers was to mitigate these problems, by increasing the number of ions reaching the detector and eliminating the need for fast data acquisition systems. The most crucial components of these spectrometers are their modulators: they determine both the maximal modulation frequency and the modulation depth. Through these parameters they limit the achievable mass resolution and signal-to-noise ratio. In this paper, a new kind of setup is presented which modulates the beam by deflecting it in two perpendicular directions and collects ions on a position sensitive detector. Such an Lissajous time of flight spectrometer achieves modulation without the use of slits or apertures, making it possible for all ions to reach the detector, thereby increasing the transmission and signal-to-noise ratio. In this paper, we provide the mathematical description of the system, discuss its properties, and present a practical demonstration of the principle.

  20. Norm convergence of continuous-time polynomial multiple ergodic averages

    CERN Document Server

    Austin, Tim

    2011-01-01

    For a jointly measurable probability-preserving action \\tau:\\bbR^D\\curvearrowright (X,\\mu) and a tuple of polynomial maps p_i:\\bbR\\to \\bbR^D, i=1,2,...,k, the multiple ergodic averages \\frac{1}{T}\\int_0^T (f_1\\circ \\tau^{p_1(t)})(f_2\\circ\\tau^{p_2(t)})... (f_k\\circ\\tau^{p_k(t)})\\,\\d t converge in L^2(\\mu) as T \\to \\infty for any f_1,f_2,...,f_k \\in L^\\infty(\\mu). This confirms the continuous-time analog of the conjectured norm convergence of discrete polynomial multiple ergodic averages, which in is its original formulation remains open in most cases. A proof of convergence can be given based on the idea of passing up to a sated extension of (X,\\mu,\\tau) in order to find simple characteristic factors, similarly to the recent development of this idea for the study of related discrete-time averages, together with a new inductive scheme on tuples of polynomials. The new induction scheme becomes available upon changing the time variable in the above integral by some fractional power, and provides an alternative t...

  1. Optimal periodic orbits of continuous time chaotic systems

    Science.gov (United States)

    Yang; Hunt; Ott

    2000-08-01

    In previous work [B. R. Hunt and E. Ott, Phys. Rev. Lett. 76, 2254 (1996); Phys. Rev. E 54, 328, (1996)], based on numerical experiments and analysis, it was conjectured that the optimal orbit selected from all possible orbits on a chaotic attractor is "typically" a periodic orbit of low period. By an optimal orbit we mean the orbit that yields the largest value of a time average of a given smooth "performance" function of the system state. Thus optimality is defined with respect to the given performance function. (The study of optimal orbits is of interest in at least three contexts: controlling chaos, embedding of low-dimensional attractors of high-dimensional dynamical systems in low-dimensional measurement spaces, and bubbling bifurcations of synchronized chaotic systems.) Here we extend this previous work. In particular, the previous work was for discrete time dynamical systems, and here we shall consider continuous time systems (flows). An essential difference for flows is that chaotic attractors can have embedded within them, not only unstable periodic orbits, but also unstable steady states, and we find that optimality can often occur on steady states. We also shed further light on the sense in which optimality is "typically" achieved at low period. In particular, we find that, as a system parameter is tuned to be closer to a crisis of the chaotic attractor, optimality may occur at higher period.

  2. L2 Orthogonal Space Time Code for Continuous Phase Modulation

    CERN Document Server

    Hesse, Matthias; Deneire, Luc

    2008-01-01

    To combine the high power efficiency of Continuous Phase Modulation (CPM) with either high spectral efficiency or enhanced performance in low Signal to Noise conditions, some authors have proposed to introduce CPM in a MIMO frame, by using Space Time Codes (STC). In this paper, we address the code design problem of Space Time Block Codes combined with CPM and introduce a new design criterion based on L2 orthogonality. This L2 orthogonality condition, with the help of simplifying assumption, leads, in the 2x2 case, to a new family of codes. These codes generalize the Wang and Xia code, which was based on pointwise orthogonality. Simulations indicate that the new codes achieve full diversity and a slightly better coding gain. Moreover, one of the codes can be interpreted as two antennas fed by two conventional CPMs using the same data but with different alphabet sets. Inspection of these alphabet sets lead also to a simple explanation of the (small) spectrum broadening of Space Time Coded CPM.

  3. Time limit and time at VO2max' during a continuous and an intermittent run.

    Science.gov (United States)

    Demarie, S; Koralsztein, J P; Billat, V

    2000-06-01

    The purpose of this study was to verify, by track field tests, whether sub-elite runners (n=15) could (i) reach their VO2max while running at v50%delta, i.e. midway between the speed associated with lactate threshold (vLAT) and that associated with maximal aerobic power (vVO2max), and (ii) if an intermittent exercise provokes a maximal and/or supra maximal oxygen consumption longer than a continuous one. Within three days, subjects underwent a multistage incremental test during which their vVO2max and vLAT were determined; they then performed two additional testing sessions, where continuous and intermittent running exercises at v50%delta were performed up to exhaustion. Subject's gas exchange and heart rate were continuously recorded by means of a telemetric apparatus. Blood samples were taken from fingertip and analysed for blood lactate concentration. In the continuous and the intermittent tests peak VO2 exceeded VO2max values, as determined during the incremental test. However in the intermittent exercise, peak VO2, time to exhaustion and time at VO2max reached significantly higher values, while blood lactate accumulation showed significantly lower values than in the continuous one. The v50%delta is sufficient to stimulate VO2max in both intermittent and continuous running. The intermittent exercise results better than the continuous one in increasing maximal aerobic power, allowing longer time at VO2max and obtaining higher peak VO2 with lower lactate accumulation.

  4. A Hidden Markov Models Approach for Crop Classification: Linking Crop Phenology to Time Series of Multi-Sensor Remote Sensing Data

    Directory of Open Access Journals (Sweden)

    Sofia Siachalou

    2015-03-01

    Full Text Available Vegetation monitoring and mapping based on multi-temporal imagery has recently received much attention due to the plethora of medium-high spatial resolution satellites and the improved classification accuracies attained compared to uni-temporal approaches. Efficient image processing strategies are needed to exploit the phenological information present in temporal image sequences and to limit data redundancy and computational complexity. Within this framework, we implement the theory of Hidden Markov Models in crop classification, based on the time-series analysis of phenological states, inferred by a sequence of remote sensing observations. More specifically, we model the dynamics of vegetation over an agricultural area of Greece, characterized by spatio-temporal heterogeneity and small-sized fields, using RapidEye and Landsat ETM+ imagery. In addition, the classification performance of image sequences with variable spatial and temporal characteristics is evaluated and compared. The classification model considering one RapidEye and four pan-sharpened Landsat ETM+ images was found superior, resulting in a conditional kappa from 0.77 to 0.94 per class and an overall accuracy of 89.7%. The results highlight the potential of the method for operational crop mapping in Euro-Mediterranean areas and provide some hints for optimal image acquisition windows regarding major crop types in Greece.

  5. Digital decoupling controller design for multiple time-delay continuous-time transfer function matrices

    Science.gov (United States)

    Xie, L. B.; Wu, C. Y.; Shieh, L. S.; Tsai, J. S. H.

    2015-03-01

    This paper presents an extended adjoint decoupling method to conduct the digital decoupling controller design for the continuous-time transfer function matrices with multiple (integer/fractional) time delays in both the denominator and the numerator matrix. First, based on the sampled unit-step response data of the afore-mentioned multiple time-delay system, the conventional balanced model-reduction method is utilised to construct an approximated discrete-time model of the original (known/unknown) multiple time-delay continuous-time transfer function matrix. Then, a digital decoupling controller is designed by utilising the extended adjoint decoupling method together with the conventional discrete-time root-locus method. An illustrative example is given to demonstrate the effectiveness of the proposed method.

  6. Optimal timing of influenza vaccination in patients with human immunodeficiency virus: a Markov cohort model based on serial study participant hemoagglutination inhibition titers.

    Science.gov (United States)

    Werker, Gregory R; Sharif, Behnam; Sun, Huiying; Cooper, Curtis; Bansback, Nick; Anis, Aslam H

    2014-02-03

    Seasonal influenza vaccination offers one of the best population-level protections against influenza-like illness (ILI). For most people, a single dose prior to the flu season offers adequate immunogenicity. HIV+ patients, however, tend to exhibit a shorter period of clinical protection, and therefore may not retain immunogenicity for the entire season. Building on the work of Nosyk et al. (2011) that determined a single dose is the optimal dosing strategy for HIV+ patients, we investigate the optimal time to administer this vaccination. Using data from the "single dose" treatment arm of an RCT conducted at 12 CIHR Canadian HIV Trials Network sites we estimated semimonthly clinical seroprotection levels for a cohort (N=93) based on HAI titer levels. These estimates were combined with CDC attack rate data for the three main strains of seasonal influenza to estimate instances of ILI over different vaccination timing strategies. Using bootstrap resampling of the cohort, nine years of CDC data, and parameter distributions, we developed a Markov cohort model that included probabilistic sensitivity analysis. Cost, quality adjusted life-years (QALYs), and net monetary benefits are presented for each timing strategy. The beginning of December is the optimal time for HIV+ patients to receive the seasonal influenza vaccine. Assuming a willingness-to-pay threshold of $50,000, the net monetary benefit associated with a Dec 1 vaccination date is $19,501.49 and the annual QALY was 0.833744. Our results support a policy of administering the seasonal influenza vaccination for this population in the middle of November or beginning of December, assuming nothing is know about the upcoming flu season. But because the difference in between this strategy and the CDC guideline is small-12 deaths averted per year and a savings of $60 million across the HIV+ population in the US-more research is needed concerning strategies for subpopulations. Copyright © 2013 Elsevier Ltd. All rights

  7. A continuous time random walk model for Darcy-scale anomalous transport in heterogeneous porous media.

    Science.gov (United States)

    Comolli, Alessandro; Hakoun, Vivien; Dentz, Marco

    2017-04-01

    Achieving the understanding of the process of solute transport in heterogeneous porous media is of crucial importance for several environmental and social purposes, ranging from aquifers contamination and remediation, to risk assessment in nuclear waste repositories. The complexity of this aim is mainly ascribable to the heterogeneity of natural media, which can be observed at all the scales of interest, from pore scale to catchment scale. In fact, the intrinsic heterogeneity of porous media is responsible for the arising of the well-known non-Fickian footprints of transport, including heavy-tailed breakthrough curves, non-Gaussian spatial density profiles and the non-linear growth of the mean squared displacement. Several studies investigated the processes through which heterogeneity impacts the transport properties, which include local modifications to the advective-dispersive motion of solutes, mass exchanges between some mobile and immobile phases (e.g. sorption/desorption reactions or diffusion into solid matrix) and spatial correlation of the flow field. In the last decades, the continuous time random walk (CTRW) model has often been used to describe solute transport in heterogenous conditions and to quantify the impact of point heterogeneity, spatial correlation and mass transfer on the average transport properties [1]. Open issues regarding this approach are the possibility to relate measurable properties of the medium to the parameters of the model, as well as its capability to provide predictive information. In a recent work [2] the authors have shed new light on understanding the relationship between Lagrangian and Eulerian dynamics as well as on their evolution from arbitrary initial conditions. On the basis of these results, we derive a CTRW model for the description of Darcy-scale transport in d-dimensional media characterized by spatially random permeability fields. The CTRW approach models particle velocities as a spatial Markov process, which is

  8. From Discrete-Time Models to Continuous-Time, Asynchronous Models of Financial Markets

    NARCIS (Netherlands)

    K. Boer-Sorban (Katalin); U. Kaymak (Uzay); J. Spiering (Jaap)

    2006-01-01

    textabstractMost agent-based simulation models of financial markets are discrete-time in nature. In this paper, we investigate to what degree such models are extensible to continuous-time, asynchronous modelling of financial markets. We study the behaviour of a learning market maker in a market with

  9. From Discrete-Time Models to Continuous-Time, Asynchronous Models of Financial Markets

    NARCIS (Netherlands)

    K. Boer-Sorban (Katalin); U. Kaymak (Uzay); J. Spiering (Jaap)

    2006-01-01

    textabstractMost agent-based simulation models of financial markets are discrete-time in nature. In this paper, we investigate to what degree such models are extensible to continuous-time, asynchronous modelling of financial markets. We study the behaviour of a learning market maker in a market with

  10. Estimating Divergence Time and Ancestral Effective Population Size of Bornean and Sumatran Orangutan Subspecies Using a Coalescent Hidden Markov Model

    DEFF Research Database (Denmark)

    Mailund, Thomas; Dutheil, Julien; Hobolth, Asger

    2011-01-01

    event has occurred to split them apart. The size of these segments of constant divergence depends on the recombination rate, but also on the speciation time, the effective population size of the ancestral population, as well as demographic effects and selection. Thus, inference of these parameters may......ue to genetic variation in the ancestor of two populations or two species, the divergence time for DNA sequences from two populations is variable along the genome. Within genomic segments all bases will share the same divergence—because they share a most recent common ancestor—when no recombination...

  11. Probabilistic forecasting of wind power at the minute time-scale with Markov-switching autoregressive models

    DEFF Research Database (Denmark)

    Pinson, Pierre; Madsen, Henrik

    2008-01-01

    Better modelling and forecasting of very short-term power fluctuations at large offshore wind farms may significantly enhance control and management strategies of their power output. The paper introduces a new methodology for modelling and forecasting such very short-term fluctuations. The proposed...... consists in 1-step ahead forecasting exercise on time-series of wind generation with a time resolution of 10 minute. The quality of the introduced forecasting methodology and its interest for better understanding power fluctuations are finally discussed....

  12. Bayesian Markov chain Monte Carlo Inversion of Time-Lapse Geophysical Data To Characterize the Vadose Zone

    DEFF Research Database (Denmark)

    Scholer, Marie; Irving, James; Zibar, Majken Caroline Looms;

    Geophysical methods have the potential to provide valuable information on hydrological properties in the unsaturated zone. In particular, time-lapse geophysical data, when coupled with a hydrological model and inverted stochastically, may allow for the effective estimation of subsurface hydraulic...

  13. The One-Dimensional Distribution and Construction of Semi-Markov Processes%半马氏过程的一维分布及构造

    Institute of Scientific and Technical Information of China (English)

    唐荣; 郭先平; 刘再明

    2008-01-01

    In this paper, we obtain the transition probability of jump chain of semi-Markov process, the distribution of sojourn time and one-dimensional distribution of semi-Markov process.Furthermore, the semi-Markov process X(t, ω) is constructed from the semi-Markov matrix and it is proved that two definitions of semi-Markov process are equivalent.

  14. Continuous-wave laser particle conditioning: Thresholds and time scales

    Science.gov (United States)

    Brown, Andrew; Ogloza, Albert; Olson, Kyle; Talghader, Joseph

    2017-03-01

    The optical absorption of contaminants on high reflectivity mirrors was measured using photo thermal common-path interferometry before and after exposure to high power continuous-wave laser light. The contaminants were micron-sized graphite flakes on hafnia-silica distributed Bragg reflectors illuminated by a ytterbium-doped fiber laser. After one-second periods of exposure, the mirrors demonstrated reduced absorption for irradiances as low as 11 kW cm-2 and had an obvious threshold near 20 kW cm-2. Final absorption values were reduced by up to 90% of their initial value for irradiances of 92 kW cm-2. For shorter pulses at 34 kW cm-2, a minimum exposure time required to begin absorption reduction was found between 100 μs and 200 μs, with particles reaching their final minimum absorption value within 300 ms. Microscope images of the surface showed agglomerated particles fragmenting with some being removed completely, probably by evaporation for exposures between 200 μs to 10 ms. Exposures of 100 ms and longer left behind a thin semi-transparent residue, covering much of the conditioned area. An order of magnitude estimate of the time necessary to begin altering the surface contaminants (also known as "conditioning") indicates about 200 μs seconds at 34 kW cm-2, based on heating an average carbon particle to its sublimation temperature including energy loss to thermal contact and radiation. This estimation is close to the observed exposure time required to begin absorption reduction.

  15. A Directed Continuous Time Random Walk Model with Jump Length Depending on Waiting Time

    Directory of Open Access Journals (Sweden)

    Long Shi

    2014-01-01

    Full Text Available In continuum one-dimensional space, a coupled directed continuous time random walk model is proposed, where the random walker jumps toward one direction and the waiting time between jumps affects the subsequent jump. In the proposed model, the Laplace-Laplace transform of the probability density function P(x,t of finding the walker at position x at time t is completely determined by the Laplace transform of the probability density function φ(t of the waiting time. In terms of the probability density function of the waiting time in the Laplace domain, the limit distribution of the random process and the corresponding evolving equations are derived.

  16. Transient analysis of some rewarded Markov models using randomization with quasistationarity detection

    OpenAIRE

    Carrasco, Juan A.

    2004-01-01

    Rewarded homogeneous continuous-time Markov chain (CTMC) models can be used to analyze performance, dependability and performability attributes of computer and telecommunication systems. In this paper, we consider rewarded CTMC models with a reward structure including reward rates associated with states and two measures summarizing the behavior in time of the resulting reward rate random variable: the expected transient reward rate at time t and the expected averaged reward rate in the tim...

  17. Phasic Triplet Markov Chains.

    Science.gov (United States)

    El Yazid Boudaren, Mohamed; Monfrini, Emmanuel; Pieczynski, Wojciech; Aïssani, Amar

    2014-11-01

    Hidden Markov chains have been shown to be inadequate for data modeling under some complex conditions. In this work, we address the problem of statistical modeling of phenomena involving two heterogeneous system states. Such phenomena may arise in biology or communications, among other fields. Namely, we consider that a sequence of meaningful words is to be searched within a whole observation that also contains arbitrary one-by-one symbols. Moreover, a word may be interrupted at some site to be carried on later. Applying plain hidden Markov chains to such data, while ignoring their specificity, yields unsatisfactory results. The Phasic triplet Markov chain, proposed in this paper, overcomes this difficulty by means of an auxiliary underlying process in accordance with the triplet Markov chains theory. Related Bayesian restoration techniques and parameters estimation procedures according to the new model are then described. Finally, to assess the performance of the proposed model against the conventional hidden Markov chain model, experiments are conducted on synthetic and real data.

  18. Continuous-time cross-phase modulation and quantum computation

    CERN Document Server

    Shapiro, J H; Razavi, Mohsen; Shapiro, Jeffrey H.

    2006-01-01

    The weak nonlinear Kerr interaction between single photons and intense laser fields has been recently proposed as a basis for distributed optics-based solutions to few-qubit applications in quantum communication and computation. Here, we analyze the above Kerr interaction by employing a continuous-time multi-mode model for the input/output fields to/from the nonlinear medium. In contrast to previous single-mode treatments of this problem, our analysis takes into account the full temporal content of the free-field input beams as well as the non-instantaneous response of the medium. The main implication of this model, in which the cross-Kerr phase shift on one input is proportional to the photon flux of the other input, is the existence of phase noise terms at the output. We show that these phase noise terms will degrade the performance of the parity gate proposed by Munro, Nemoto, and Spiller [New J. Phys. 7, 137 (2005)].

  19. [Decision analysis in radiology using Markov models].

    Science.gov (United States)

    Golder, W

    2000-01-01

    Markov models (Multistate transition models) are mathematical tools to simulate a cohort of individuals followed over time to assess the prognosis resulting from different strategies. They are applied on the assumption that persons are in one of a finite number of states of health (Markov states). Each condition is given a transition probability as well as an incremental value. Probabilities may be chosen constant or varying over time due to predefined rules. Time horizon is divided into equal increments (Markov cycles). The model calculates quality-adjusted life expectancy employing real-life units and values and summing up the length of time spent in each health state adjusted for objective outcomes and subjective appraisal. This sort of modeling prognosis for a given patient is analogous to utility in common decision trees. Markov models can be evaluated by matrix algebra, probabilistic cohort simulation and Monte Carlo simulation. They have been applied to assess the relative benefits and risks of a limited number of diagnostic and therapeutic procedures in radiology. More interventions should be submitted to Markov analyses in order to elucidate their cost-effectiveness.

  20. Anticontrol of chaos in continuous-time systems via time-delay feedback.

    Science.gov (United States)

    Wang, Xiao Fan; Chen, Guanrong; Yu, Xinghuo

    2000-12-01

    In this paper, a systematic design approach based on time-delay feedback is developed for anticontrol of chaos in a continuous-time system. This anticontrol method can drive a finite-dimensional, continuous-time, autonomous system from nonchaotic to chaotic, and can also enhance the existing chaos of an originally chaotic system. Asymptotic analysis is used to establish an approximate relationship between a time-delay differential equation and a discrete map. Anticontrol of chaos is then accomplished based on this relationship and the differential-geometry control theory. Several examples are given to verify the effectiveness of the methodology and to illustrate the systematic design procedure. (c) 2000 American Institute of Physics.

  1. Dynamics of Weeds in the Soil Seed Bank: A Hidden Markov Model to Estimate Life History Traits from Standing Plant Time Series.

    Science.gov (United States)

    Borgy, Benjamin; Reboud, Xavier; Peyrard, Nathalie; Sabbadin, Régis; Gaba, Sabrina

    2015-01-01

    Predicting the population dynamics of annual plants is a challenge due to their hidden seed banks in the field. However, such predictions are highly valuable for determining management strategies, specifically in agricultural landscapes. In agroecosystems, most weed seeds survive during unfavourable seasons and persist for several years in the seed bank. This causes difficulties in making accurate predictions of weed population dynamics and life history traits (LHT). Consequently, it is very difficult to identify management strategies that limit both weed populations and species diversity. In this article, we present a method of assessing weed population dynamics from both standing plant time series data and an unknown seed bank. We use a Hidden Markov Model (HMM) to obtain estimates of over 3,080 botanical records for three major LHT: seed survival in the soil, plant establishment (including post-emergence mortality), and seed production of 18 common weed species. Maximum likelihood and Bayesian approaches were complementarily used to estimate LHT values. The results showed that the LHT provided by the HMM enabled fairly accurate estimates of weed populations in different crops. There was a positive correlation between estimated germination rates and an index of the specialisation to the crop type (IndVal). The relationships between estimated LHTs and that between the estimated LHTs and the ecological characteristics of weeds provided insights into weed strategies. For example, a common strategy to cope with agricultural practices in several weeds was to produce less seeds and increase germination rates. This knowledge, especially of LHT for each type of crop, should provide valuable information for developing sustainable weed management strategies.

  2. Markov Chain: A Predictive Model for Manpower Planning ...

    African Journals Online (AJOL)

    ADOWIE PERE

    numerous previous studies have applied Markov chain models in describing title or level promotions .... is one of the most crucial, complex and continuing ... computational tools that will enable administrators to ... random variables. ,.... ,.

  3. A new Markov Binomial distribution

    OpenAIRE

    Leda D. Minkova; Omey, Edward

    2011-01-01

    In this paper, we introduce a two state homogeneous Markov chain and define a geometric distribution related to this Markov chain. We define also the negative binomial distribution similar to the classical case and call it NB related to interrupted Markov chain. The new binomial distribution is related to the interrupted Markov chain. Some characterization properties of the Geometric distributions are given. Recursion formulas and probability mass functions for the NB distribution and the new...

  4. Robust passive filtering for continuous-time polytopic uncertain time-delay systems

    Institute of Scientific and Technical Information of China (English)

    LU Ling-ling; DUAN Guang-ren; WU Ai-guo

    2008-01-01

    To obtain a stable and proper linear filter to make the filtering error system robustly and strictly passive,the problem of full-order robust passive filtering for continuous-time polytopie uncertain time-delay systems was investigated.A criterion for the passivity of time-delay systems was firstly provided in terms of linear matrix inequalities(LMI).Then an LMI sufficient condition for the existence of a robust filter was established and a design procedure was proposed for this type of systems.A numerical example demonstrated the feasibility of the filtering design procedure.

  5. Partially Hidden Markov Models

    DEFF Research Database (Denmark)

    Forchhammer, Søren Otto; Rissanen, Jorma

    1996-01-01

    Partially Hidden Markov Models (PHMM) are introduced. They differ from the ordinary HMM's in that both the transition probabilities of the hidden states and the output probabilities are conditioned on past observations. As an illustration they are applied to black and white image compression wher...

  6. On Weak Markov's Principle

    DEFF Research Database (Denmark)

    Kohlenbach, Ulrich Wilhelm

    2002-01-01

    We show that the so-called weak Markov's principle (WMP) which states that every pseudo-positive real number is positive is underivable in E-HA + AC. Since allows one to formalize (atl eastl arge parts of) Bishop's constructive mathematics, this makes it unlikely that WMP can be proved within the...

  7. A Noether Theorem for Markov Processes

    CERN Document Server

    Baez, John C

    2012-01-01

    Noether's theorem links the symmetries of a quantum system with its conserved quantities, and is a cornerstone of quantum mechanics. Here we prove a version of Noether's theorem for Markov processes. In quantum mechanics, an observable commutes with the Hamiltonian if and only if its expected value remains constant in time for every state. For Markov processes that no longer holds, but an observable commutes with the Hamiltonian if and only if both its expected value and standard deviation are constant in time for every state.

  8. Entropy production fluctuations of finite Markov chains

    Science.gov (United States)

    Jiang, Da-Quan; Qian, Min; Zhang, Fu-Xi

    2003-09-01

    For almost every trajectory segment over a finite time span of a finite Markov chain with any given initial distribution, the logarithm of the ratio of its probability to that of its time-reversal converges exponentially to the entropy production rate of the Markov chain. The large deviation rate function has a symmetry of Gallavotti-Cohen type, which is called the fluctuation theorem. Moreover, similar symmetries also hold for the rate functions of the joint distributions of general observables and the logarithmic probability ratio.

  9. Lagging/Leading Coupled Continuous Time Random Walks, Renewal Times and their Joint Limits

    CERN Document Server

    Straka, Peter

    2010-01-01

    Subordinating a random walk to a renewal process yields a continuous time random walk (CTRW) model for diffusion, including the possibility of anomalous diffusion. Transition densities of scaling limits of power law CTRWs have been shown to solve fractional Fokker-Planck equations. We consider limits of sequences of CTRWs which arise when both waiting times and jumps are taken from an infinitesimal triangular array. We identify two different limit processes $X_t$ and $Y_t$ when waiting times precede or follow jumps, respectively. In the limiting procedure, we keep track of the renewal times of the CTRWs and hence find two more limit processes. Finally, we calculate the joint law of all four limit processes evaluated at a fixed time $t$.

  10. Time-frequency representation of musical rhythm by continuous wavelets

    NARCIS (Netherlands)

    Smith, L.M.; Honing, H.

    2008-01-01

    A method is described that exhaustively represents the periodicities created by a musical rhythm. The continuous wavelet transform is used to decompose an interval representation of a musical rhythm into a hierarchy of short-term frequencies. This reveals the temporal relationships between events ov

  11. Estimating an Activity Driven Hidden Markov Model

    OpenAIRE

    Meyer, David A.; Shakeel, Asif

    2015-01-01

    We define a Hidden Markov Model (HMM) in which each hidden state has time-dependent $\\textit{activity levels}$ that drive transitions and emissions, and show how to estimate its parameters. Our construction is motivated by the problem of inferring human mobility on sub-daily time scales from, for example, mobile phone records.

  12. Robust Continuous-time Generalized Predictive Control for Large Time-delay System

    Institute of Scientific and Technical Information of China (English)

    WEI Huan; PAN Li-deng; ZHEN Xin-ping

    2008-01-01

    A simple delay-predictive continuous-time generalized predictive controller with filter (F - SDCGPC) is proposed. By using modified predictive output signal and cost function, the delay compensator is incorporated in the control law with observer structure, and a filter is added for enhancing robustness. The design of filter does not affect the nominal set-point response, and it is more flexible than the design of observer polynomial. The analysis and simulation results show that the F - SDCGPC has better robustness than the observer structure without filter when large time-delay error is considered.

  13. Finite-frequency model reduction of continuous-time switched linear systems with average dwell time

    Science.gov (United States)

    Ding, Da-Wei; Du, Xin

    2016-11-01

    This paper deals with the model reduction problem of continuous-time switched linear systems with finite-frequency input signals. The objective of the paper is to propose a finite-frequency model reduction method for such systems. A finite-frequency ? performance index is first defined in frequency domain, and then a finite-frequency performance analysis condition is derived by Parseval's theorem. Combined with the average dwell time approach, sufficient conditions for the existence of exponentially stable reduced-order models are derived. An algorithm is proposed to construct the desired reduced-order models. The effectiveness of the proposed method is illustrated by a numerical example.

  14. Quantum Markov Chain Mixing and Dissipative Engineering

    DEFF Research Database (Denmark)

    Kastoryano, Michael James

    2012-01-01

    This thesis is the fruit of investigations on the extension of ideas of Markov chain mixing to the quantum setting, and its application to problems of dissipative engineering. A Markov chain describes a statistical process where the probability of future events depends only on the state of the sy......This thesis is the fruit of investigations on the extension of ideas of Markov chain mixing to the quantum setting, and its application to problems of dissipative engineering. A Markov chain describes a statistical process where the probability of future events depends only on the state...... of the system at the present point in time, but not on the history of events. Very many important processes in nature are of this type, therefore a good understanding of their behaviour has turned out to be very fruitful for science. Markov chains always have a non-empty set of limiting distributions....... Finally, we consider three independent tasks of dissipative engineering: dissipatively preparing a maximally entangled state of two atoms trapped in an optical cavity, dissipative preparation of graph states, and dissipative quantum computing construction....

  15. Continuous time modelling of dynamical spatial lattice data observed at sparsely distributed times

    DEFF Research Database (Denmark)

    Rasmussen, Jakob Gulddahl; Møller, Jesper

    2007-01-01

    Summary. We consider statistical and computational aspects of simulation-based Bayesian inference for a spatial-temporal model based on a multivariate point process which is only observed at sparsely distributed times. The point processes are indexed by the sites of a spatial lattice, and they ex......Summary. We consider statistical and computational aspects of simulation-based Bayesian inference for a spatial-temporal model based on a multivariate point process which is only observed at sparsely distributed times. The point processes are indexed by the sites of a spatial lattice......, and they exhibit spatial interaction. For specificity we consider a particular dynamical spatial lattice data set which has previously been analysed by a discrete time model involving unknown normalizing constants. We discuss the advantages and disadvantages of using continuous time processes compared...

  16. TIME INCONSISTENCY AND REPUTATION IN MONETARY POLICY: A STRATEGIC MODELLING IN CONTINUOUS TIME

    Institute of Scientific and Technical Information of China (English)

    Li Jingyuan; Tian Guoqiang

    2008-01-01

    This article develops a model to examine the equilibrium behavior of the time inconsistency problem in a continuous time economy with stochastic and endogenized dis-tortion. First, the authors introduce the notion of sequentially rational equilibrium, and show that the time inconsistency problem may be solved with trigger reputation strategies for stochastic setting. The conditions for the existence of sequentially rational equilibrium are provided. Then, the concept of sequentially rational stochastically stable equilibrium is introduced. The authors compare the relative stability between the cooperative behavior and uncooperative behavior, and show that the cooperative equilibrium in this monetary policy game is a sequentially rational stochastically stable equilibrium and the uncooper-ative equilibrium is sequentially rational stochastically unstable equilibrium. In the long run, the zero inflation monetary policies are inherently more stable than the discretion rules, and once established, they tend to persist for longer periods of the time.

  17. Research on soft sensing method based on continuous hidden Markov model in fermentation process%基于连续隐Markov模型的发酵过程软测量方法

    Institute of Scientific and Technical Information of China (English)

    刘国海; 江兴科; 梅从立

    2011-01-01

    针对生物发酵过程中一些生物参量难以用仪表进行在线检测的问题,提出一种基于连续隐Markov模型(CHMM)的发酵过程软测量建模方法.为减少建模过程的计算量,提出了改进最小分类误差准则,用于CHMM软测量模型参数估计.为避免软测量结果在发酵过程监测与控制实际应用中存在的盲目性,提出了在线评价软测量结果可靠性的可信度评价指标.实验结果表明了所提出方法的有效性以及可信度评价指标的实际意义.%A soft sensing modeling method based on continuous hidden Markov model(CHMM) is developed to deal with the problem that some biologic variables cannot be measured directly online in fermentation process.In order to reduce the computation quantity of modeling process,improved minimum classification error criteria is used to train the CHMMbased soft sensor.Meanwhile,a soft sensing credibility evaluation index is proposed to avoid blindness problem during the practical application of soft sensing result to monitoring in fermentation process.The testing result shows the effectiveness of the proposed method and the practical significance of the credibility evaluation index.

  18. Parallel Markov chain Monte Carlo simulations.

    Science.gov (United States)

    Ren, Ruichao; Orkoulas, G

    2007-06-07

    With strict detailed balance, parallel Monte Carlo simulation through domain decomposition cannot be validated with conventional Markov chain theory, which describes an intrinsically serial stochastic process. In this work, the parallel version of Markov chain theory and its role in accelerating Monte Carlo simulations via cluster computing is explored. It is shown that sequential updating is the key to improving efficiency in parallel simulations through domain decomposition. A parallel scheme is proposed to reduce interprocessor communication or synchronization, which slows down parallel simulation with increasing number of processors. Parallel simulation results for the two-dimensional lattice gas model show substantial reduction of simulation time for systems of moderate and large size.

  19. H∞ deconvolution filter design for time-delay linear continuous-time systems

    Institute of Scientific and Technical Information of China (English)

    2006-01-01

    Proposes an H∞ deconvolution design for time-delay linear continuous-time systems. We first analyze the general structure and innovation structure of the H∞ deconvolution filter. The deconvolution filter with innovation structure is made up of an output observer and a linear mapping, where the latter reflects the internal connection between the unknown input signal and the output estimate error. Based on the bounded real lemma,a time domain design approach and a sufficient condition for the existence of deconvolution filter are presented.The parameterization of the deconvolution filter can be completed by solving a Riccati equation. The proposed method is useful for the case that does not require statistical information about disturbances. At last, a numerical example is given to demonstrate the performance of the proposed filter.

  20. Discrete-Time Approximation for Nonlinear Continuous Systems with Time Delays

    Directory of Open Access Journals (Sweden)

    Bemri H’mida

    2016-05-01

    Full Text Available This paper is concerned with the discretization of nonlinear continuous time delay systems. Our approach is based on Taylor-Lie series. The main idea aims to minimize the effect of the delay and neglects the importance of nonlinear parameter by the linearization of the system study in an attempt to make its handling and easier programming as possible. We investigate a new method based on the development of new theoretical methods for the time discretization of nonlinear systems with time delay .The performance of these proposed discretization methods was validated by doing the numerical simulation using a nonlinear system with state delay. Some illustrative examples are given to show the effectiveness of the obtained results.

  1. Cluster Observations of Non-Time Continuous Magnetosonic Waves

    Science.gov (United States)

    Walker, Simon N.; Demekhov, Andrei G.; Boardsen, Scott A.; Ganushkina, Natalia Y.; Sibeck, David G.; Balikhin, Michael A.

    2016-01-01

    Equatorial magnetosonic waves are normally observed as temporally continuous sets of emissions lasting from minutes to hours. Recent observations, however, have shown that this is not always the case. Using Cluster data, this study identifies two distinct forms of these non temporally continuous use missions. The first, referred to as rising tone emissions, are characterized by the systematic onset of wave activity at increasing proton gyroharmonic frequencies. Sets of harmonic emissions (emission elements)are observed to occur periodically in the region +/- 10 off the geomagnetic equator. The sweep rate of these emissions maximizes at the geomagnetic equator. In addition, the ellipticity and propagation direction also change systematically as Cluster crosses the geomagnetic equator. It is shown that the observed frequency sweep rate is unlikely to result from the sideband instability related to nonlinear trapping of suprathermal protons in the wave field. The second form of emissions is characterized by the simultaneous onset of activity across a range of harmonic frequencies. These waves are observed at irregular intervals. Their occurrence correlates with changes in the spacecraft potential, a measurement that is used as a proxy for electron density. Thus, these waves appear to be trapped within regions of localized enhancement of the electron density.

  2. Maintaining Visual Attention over Time: Effects of Object Continuity

    Directory of Open Access Journals (Sweden)

    Katsumi Watanabe

    2011-05-01

    Full Text Available Maintaining visual attention in dynamic environments is an important function of the visual system. The visual system appears to modulate attentional state to identify a target embedded in a rapid sequence of nontargets; typically, target identification gets better as the number of preceding items increases and is maintained at this increased level. We examined the temporal characteristics of the visual system that contribute to maintaining attentional state. Inserting one-second temporal gap in the sequence reset the elevated attentional state. However, the results also suggested that the attentional state was maintained as long as the sequence was interpreted as a single event (i.e., the object continuity was maintained. It has been proposed that ‘object files’ keep track of visual items and accumulate information as they move and change. It is also known that objects' features (shape and/or color affect the distribution of attention within objects. In a separate set of experiments, we examined whether the change detection of feature-location binding in moving visual objects would affect the attentional distribution within objects by using a probe-detection paradigm. The results showed that the performance of irregularity detection did not influence the attentional distribution within objects. This finding implies that attentional distribution within objects occurs independent of object continuity.

  3. Delays and the Capacity of Continuous-time Channels

    CERN Document Server

    Khanna, Sanjeev

    2011-01-01

    Any physical channel of communication offers two potential reasons why its capacity (the number of bits it can transmit in a unit of time) might be unbounded: (1) Infinitely many choices of signal strength at any given instant of time, and (2) Infinitely many instances of time at which signals may be sent. However channel noise cancels out the potential unboundedness of the first aspect, leaving typical channels with only a finite capacity per instant of time. The latter source of infinity seems less studied. A potential source of unreliability that might restrict the capacity also from the second aspect is delay: Signals transmitted by the sender at a given point of time may not be received with a predictable delay at the receiving end. Here we examine this source of uncertainty by considering a simple discrete model of delay errors. In our model the communicating parties get to subdivide time as microscopically finely as they wish, but still have to cope with communication delays that are macroscopic and va...

  4. Epidemic spreading in interconnected networks: a continuous time approach

    CERN Document Server

    de Arruda, Guilherme Ferraz; Peixoto, Tiago P; Rodrigues, Francisco A; Moreno, Yamir

    2015-01-01

    We present a continuous formulation of epidemic spreading on interconnected networks using a tensorial notation, extending the models of monoplex networks to this context. We derive analytical expressions for the epidemic threshold on the SIS and SIR dynamics, as well as upper and lower bounds for the steady-state. Using the quasi-stationary state (QS) method we show the emergence of two or more phase transitions, and propose an analytical and numerical analysis based on the inverse participation ratio. Furthermore, when mapping the critical epidemic dynamics to the eigenvalue problem, we observe a characteristic transition in the eigenvalue spectra of the supra-contact tensor as a function of the ratio of spreading rates: If the spreading rate within each layer is comparable to the rate across layers, the individual spectra of each layer merge with the coupling between layers. Our formalism provides a mathematical approach to epidemic spreading in multiplex systems and our results reinforce the importance of...

  5. Causal inference for continuous-time processes when covariates are observed only at discrete times

    CERN Document Server

    Zhang, Mingyuan; Small, Dylan S; 10.1214/10-AOS830

    2011-01-01

    Most of the work on the structural nested model and g-estimation for causal inference in longitudinal data assumes a discrete-time underlying data generating process. However, in some observational studies, it is more reasonable to assume that the data are generated from a continuous-time process and are only observable at discrete time points. When these circumstances arise, the sequential randomization assumption in the observed discrete-time data, which is essential in justifying discrete-time g-estimation, may not be reasonable. Under a deterministic model, we discuss other useful assumptions that guarantee the consistency of discrete-time g-estimation. In more general cases, when those assumptions are violated, we propose a controlling-the-future method that performs at least as well as g-estimation in most scenarios and which provides consistent estimation in some cases where g-estimation is severely inconsistent. We apply the methods discussed in this paper to simulated data, as well as to a data set c...

  6. Efficient Modelling and Generation of Markov Automata

    NARCIS (Netherlands)

    Timmer, Mark; Katoen, Joost-Pieter; Pol, van de Jaco; Stoelinga, Mariëlle; Koutny, M.; Ulidowski, I.

    2012-01-01

    This paper introduces a framework for the efficient modelling and generation of Markov automata. It consists of (1) the data-rich process-algebraic language MAPA, allowing concise modelling of systems with nondeterminism, probability and Markovian timing; (2) a restricted form of the language, the M

  7. One-Counter Markov Decision Processes

    NARCIS (Netherlands)

    Brazdil, T.; Brozek, V.; Etessami, K.; Kucera, A.; Wojtczak, D.K.; Charikar, M.

    2010-01-01

    We study the computational complexity of central analysis problems for One-Counter Markov Decision Processes (OC-MDPs), a class of finitely-presented, countable-state MDPs. OC-MDPs are equivalent to a controlled extension of (discrete-time) Quasi-Birth-Death processes (QBDs), a stochastic model stud

  8. A Martingale Decomposition of Discrete Markov Chains

    DEFF Research Database (Denmark)

    Hansen, Peter Reinhard

    We consider a multivariate time series whose increments are given from a homogeneous Markov chain. We show that the martingale component of this process can be extracted by a filtering method and establish the corresponding martingale decomposition in closed-form. This representation is useful...

  9. Real-Time Continuous Response Spectra Exceedance Calculation

    Science.gov (United States)

    Vernon, Frank; Harvey, Danny; Lindquist, Kent; Franke, Mathias

    2017-04-01

    A novel approach is presented for near real-time earthquake alarms for critical structures at distributed locations using real-time estimation of response spectra obtained from near free-field motions. Influential studies dating back to the 1980s identified spectral response acceleration as a key ground motion characteristic that correlates well with observed damage in structures. Thus, monitoring and reporting on exceedance of spectra-based thresholds are useful tools for assessing the potential for damage to facilities or multi-structure campuses based on input ground motions only. With as little as one strong-motion station per site, this scalable approach can provide rapid alarms on the damage status of remote towns, critical infrastructure (e.g., hospitals, schools) and points of interests (e.g., bridges) for a very large number of locations enabling better rapid decision making during critical and difficult immediate post-earthquake response actions. Real-time calculation of PSA exceedance and alarm dissemination are enabled with Bighorn, a module included in the Antelope software package that combines real-time spectral monitoring and alarm capabilities with a robust built-in web display server. Examples of response spectra from several M 5 events recorded by the ANZA seismic network in southern California will be presented.

  10. Robust dissipative filtering for continuous-time polytopic uncertain time-delay systems

    Institute of Scientific and Technical Information of China (English)

    LV Ling-ling; DUAN Guang-ren; WU Ai-guo

    2010-01-01

    This paper focuses on the problem of dissipative filtering for linear continuous-time polytopic uncertain time-delay systems.To obtain a stable and proper linear filter such that the filtering error system is strictly dissipative for all admissible uncertainties,a new dissipativity criterion which realizes separation between the Lyapunov matrices and the system dynamic matrices is firstly provided in terms of linear matrix inequalities (LMI).Then an LMI sufficient condition for the existence of a robust filter is established and a design procedure is proposed for this type of systems.One numerical example demonstrates less conservativeness of the proposed criterion,the other numerical example illustrates the validity of the proposed filter design.

  11. Lifting—A nonreversible Markov chain Monte Carlo algorithm

    Science.gov (United States)

    Vucelja, Marija

    2016-12-01

    Markov chain Monte Carlo algorithms are invaluable tools for exploring stationary properties of physical systems, especially in situations where direct sampling is unfeasible. Common implementations of Monte Carlo algorithms employ reversible Markov chains. Reversible chains obey detailed balance and thus ensure that the system will eventually relax to equilibrium, though detailed balance is not necessary for convergence to equilibrium. We review nonreversible Markov chains, which violate detailed balance and yet still relax to a given target stationary distribution. In particular cases, nonreversible Markov chains are substantially better at sampling than the conventional reversible Markov chains with up to a square root improvement in the convergence time to the steady state. One kind of nonreversible Markov chain is constructed from the reversible ones by enlarging the state space and by modifying and adding extra transition rates to create non-reversible moves. Because of the augmentation of the state space, such chains are often referred to as lifted Markov Chains. We illustrate the use of lifted Markov chains for efficient sampling on several examples. The examples include sampling on a ring, sampling on a torus, the Ising model on a complete graph, and the one-dimensional Ising model. We also provide a pseudocode implementation, review related work, and discuss the applicability of such methods.

  12. Determining expected behaviour of fraudsters for a continuous audit system

    Directory of Open Access Journals (Sweden)

    Mathew A. Thomas

    2012-06-01

    Full Text Available This research attempts to determine the behaviour of fraudsters in a continuous audit system where the fraudsters have multiple options for committing fraud. The system is modelled as a Continuous Time Markov Chain where the state changes are caused by the fraudster's actions. The model uses a dynamic game with probabilistic transitions to determine the expected behaviour of the fraudster.

  13. Ageing first passage time density in continuous time random walks and quenched energy landscapes

    Science.gov (United States)

    Krüsemann, Henning; Godec, Aljaž; Metzler, Ralf

    2015-07-01

    We study the first passage dynamics of an ageing stochastic process in the continuous time random walk (CTRW) framework. In such CTRW processes the test particle performs a random walk, in which successive steps are separated by random waiting times distributed in terms of the waiting time probability density function \\psi (t)≃ {t}-1-α (0≤slant α ≤slant 2). An ageing stochastic process is defined by the explicit dependence of its dynamic quantities on the ageing time ta, the time elapsed between its preparation and the start of the observation. Subdiffusive ageing CTRWs with 0\\lt α \\lt 1 describe systems such as charge carriers in amorphous semiconducters, tracer dispersion in geological and biological systems, or the dynamics of blinking quantum dots. We derive the exact forms of the first passage time density for an ageing subdiffusive CTRW in the semi-infinite, confined, and biased case, finding different scaling regimes for weakly, intermediately, and strongly aged systems: these regimes, with different scaling laws, are also found when the scaling exponent is in the range 1\\lt α \\lt 2, for sufficiently long ta. We compare our results with the ageing motion of a test particle in a quenched energy landscape. We test our theoretical results in the quenched landscape against simulations: only when the bias is strong enough, the correlations from returning to previously visited sites become insignificant and the results approach the ageing CTRW results. With small bias or without bias, the ageing effects disappear and a change in the exponent compared to the case of a completely annealed landscape can be found, reflecting the build-up of correlations in the quenched landscape.

  14. Prostate tumor alignment and continuous, real-time adaptive radiation therapy using electromagnetic fiducials: clinical and cost-utility analyses.

    Science.gov (United States)

    Quigley, Martin M; Mate, Timothy P; Sylvester, John E

    2009-01-01

    To evaluate the accuracy, utility, and cost effectiveness of a new electromagnetic patient positioning and continuous, real-time monitoring system, which uses permanently implanted resonant transponders in the target (Calypso 4D Localization System and Beacon transponders, Seattle, WA) to continuously monitor tumor location and movement during external beam radiation therapy of the prostate. This clinical trial studied 43 patients at 5 sites. All patients were implanted with 3 transponders each. In 41 patients, the system was used for initial alignment at each therapy session. Thirty-five patients had continuous monitoring during their radiation treatment. Over 1,000 alignment comparisons were made to a commercially available kV X-ray positioning system (BrainLAB ExacTrac, Munich, Germany). Using decision analysis and Markov processes, the outcomes of patients were simulated over a 5-year period and measured in terms of costs from a payer's perspective and quality-adjusted life years (QALYs). All patients had satisfactory transponder implantations for monitoring purposes. In over 75% of the treatment sessions, the correction to conventional positioning (laser and tattoos) directed by an electromagnetic patient positioning and monitoring system was greater than 5 mm. Ninety-seven percent (34/35) of the patients who underwent continuous monitoring had target motion that exceeded preset limits at some point during the course of their radiation therapy. Exceeding preset thresholds resulted in user intervention at least once during the therapy in 80% of the patients (28/35). Compared with localization using ultrasound, electronic portal imaging devices (EPID), or computed tomography (CT), localization with the electromagnetic patient positioning and monitoring system yielded superior gains in QALYs at comparable costs. Most patients positioned with conventional tattoos and lasers for prostate radiation therapy were found by use of the electromagnetic patient positioning

  15. Continuous-time analog filter in CMOS nanoscale era

    Science.gov (United States)

    Baschirotto, A.; De Matteis, M.; Pezzotta, A.; D'Amico, S.

    2014-04-01

    Analog filters are key blocks in analog signal processing. They are widely employed in many systems, like wireless transceivers, detectors read-out, sensors interfaces, etc. The IC technology choice for such systems is mainly dictated by the requirements of high speed and low power consumption of digital circuits. This pushed an impressive movement towards scaled technology and this has important consequences on the analog circuits design. The impact of technology scaling down to nanometre scale on analog filters design is here investigated. For instance, supply voltage reduction in analog filters limits circuits design solutions and could result in higher power consumption. Moreover, at the same time, innovative systems push analog filters to get higher and higher operation frequencies, due to the increasing bandwidth/speed requirements. Recent solutions for efficient low-voltage and high frequency analog filters in nanometre technology are described.

  16. Time lagged ordinal partition networks for capturing dynamics of continuous dynamical systems

    Energy Technology Data Exchange (ETDEWEB)

    McCullough, Michael; Iu, Herbert Ho-Ching [School of Electrical and Electronic Engineering, The University of Western Australia, Crawley WA 6009 (Australia); Small, Michael; Stemler, Thomas [School of Mathematics and Statistics, The University of Western Australia, Crawley WA 6009 (Australia)

    2015-05-15

    We investigate a generalised version of the recently proposed ordinal partition time series to network transformation algorithm. First, we introduce a fixed time lag for the elements of each partition that is selected using techniques from traditional time delay embedding. The resulting partitions define regions in the embedding phase space that are mapped to nodes in the network space. Edges are allocated between nodes based on temporal succession thus creating a Markov chain representation of the time series. We then apply this new transformation algorithm to time series generated by the Rössler system and find that periodic dynamics translate to ring structures whereas chaotic time series translate to band or tube-like structures—thereby indicating that our algorithm generates networks whose structure is sensitive to system dynamics. Furthermore, we demonstrate that simple network measures including the mean out degree and variance of out degrees can track changes in the dynamical behaviour in a manner comparable to the largest Lyapunov exponent. We also apply the same analysis to experimental time series generated by a diode resonator circuit and show that the network size, mean shortest path length, and network diameter are highly sensitive to the interior crisis captured in this particular data set.

  17. DOUBLE-MARKOV RISK MODEL

    Institute of Scientific and Technical Information of China (English)

    Xiaoyun MO; Jieming ZHOU; Hui OU; Xiangqun YANG

    2013-01-01

    Given a new Double-Markov risk model DM =(μ,Q,v,H; Y,Z) and Double-Markov risk process U ={U(t),t ≥ 0}.The ruin or survival problem is addressed.Equations which the survival probability satisfied and the formulas of calculating survival probability are obtained.Recursion formulas of calculating the survival probability and analytic expression of recursion items are obtained.The conclusions are expressed by Q matrix for a Markov chain and transition probabilities for another Markov Chain.

  18. Towards continuous and real-time attention monitoring at work: reaction time versus brain response.

    Science.gov (United States)

    Mijović, Pavle; Ković, Vanja; De Vos, Maarten; Mačužić, Ivan; Todorović, Petar; Jeremić, Branislav; Gligorijević, Ivan

    2017-02-01

    Continuous and objective measurement of the user attention state still represents a major challenge in the ergonomics research. Recently available wearable electroencephalography (EEG) opens new opportunities for objective and continuous evaluation of operators' attention, which may provide a new paradigm in ergonomics. In this study, wearable EEG was recorded during simulated assembly operation, with the aim to analyse P300 event-related potential component, which provides reliable information on attention processing. In parallel, reaction times (RTs) were recorded and the correlation between these two attention-related modalities was investigated. Negative correlation between P300 amplitudes and RTs has been observed on the group level (p attention monitoring in ergonomics research. On the other hand, no significant correlation between RTs and P300 latency was found on group, neither on individual level. Practitioner Summary: Ergonomic studies of assembly operations mainly investigated physical aspects, while mental states of the assemblers were not sufficiently addressed. Presented study aims at attention tracking, using realistic workplace replica. It is shown that drops in attention could be successfully traced only by direct brainwave observation, using wireless electroencephalographic measurements.

  19. Improved Continuous-Time Higher Harmonic Control Using Hinfinity Methods

    Science.gov (United States)

    Fan, Frank H.

    The helicopter is a versatile aircraft that can take-off and land vertically, hover efficiently, and maneuver in confined space. This versatility is enabled by the main rotor, which also causes undesired harmonic vibration during operation. This unwanted vibration has a negative impact on the practicality of the helicopter and also increases its operational cost. Passive control techniques have been applied to helicopter vibration suppression, but these methods are generally heavy and are not robust to changes in operating conditions. Feedback control offers the advantages of robustness and potentially higher performance over passive control techniques, and amongst the various feedback schemes, Shaw's higher harmonic control algorithm has been shown to be an effective method for attenuating harmonic disturbance in helicopters. In this thesis, the higher harmonic disturbance algorithm is further developed to achieve improved performance. One goal in this thesis is to determine the importance of periodicity in the helicopter rotor dynamics for control synthesis. Based on the analysis of wind tunnel data and simulation results, we conclude the helicopter rotor can be modeled reasonably well as linear and time-invariant for control design purposes. Modeling the helicopter rotor as linear time-invariant allows us to apply linear control theory concepts to the higher harmonic control problem. Another goal in this thesis is to find the limits of performance in harmonic disturbance rejection. To achieve this goal, we first define the metrics to measure the performance of the controller in terms of response speed and robustness to changes in the plant dynamics. The performance metrics are incorporated into an Hinfinity control problem. For a given plant, the resulting Hinfinity controller achieves the maximum performance, thus allowing us to identify the performance limitation in harmonic disturbance rejection. However, the Hinfinity controllers are of high order, and may

  20. Shift ergodicity for stationary Markov processes

    Institute of Scientific and Technical Information of China (English)

    东金文

    2001-01-01

    In this paper shift ergodicity and related topics are studied for certain stationary processes. We first present a simple proof of the conclusion that every stationary Markov process is a generalized convex combination of stationary ergodic Markov processes. A direct consequence is that a stationary distribution of a Markov process is extremal if and only if the corresponding stationary Markov process is time ergodic and every stationary distribution is a generalized convex combination of such extremal ones. We then consider space ergodicity for spin flip particle systems. We prove space shift ergodicity and mixing for certain extremal invariant measures for a class of spin systems, in which most of the typical models, such as the Voter Models and the Contact Models, are included. As a consequence of these results we see that for such systems, under each of those extremal invariant measures, the space and time means of an observable coincide, an important phenomenon in statistical physics. Our results provide partial answers to certain interesting problems in spin systems.

  1. Time-sampled versus continuous-time reporting for measuring incidence.

    Science.gov (United States)

    McNamee, Roseanne; Chen, Yiqun; Hussey, Louise; Agius, Raymond

    2010-05-01

    Accuracy of incidence estimates may be affected by biases that depend on frequency of approach to reporters and reporting window length. A time-sampling strategy enables infrequent approaches with short windows but has never been evaluated. A randomized crossover trial compared incidence estimates of work-related diseases using time-sampled versus continuous-time reporting. Physicians were randomly allocated either to report every month (12/12) in 2004 and for 1 randomly chosen month (1/12) in 2005, or to the reverse sequence. Numbers of new cases of work-related disease reported per reporter per month for 1/12 and 12/12 reporting periods were compared. Response rates were high (87%). Withdrawal from the study was higher under 12/12 reporting. The rate ratio for 1/12 versus 12/12 reporting was 1.26 (95% confidence interval = 1.11-1.42). Rates declined gradually in the 12/12 groups over the year, consistent with reporting fatigue. Increased frequency of data collection may reduce incidence estimates.

  2. Fisher information and asymptotic normality in system identification for quantum Markov chains

    CERN Document Server

    Guta, Madalin

    2010-01-01

    This paper deals with the problem of estimating the coupling constant $\\theta$ of a mixing quantum Markov chain with finite dimensional quantum systems. For a repeated measurement on the chain's output we show that the outcomes' time average has an asymptotically normal (Gaussian) distribution, and we give the explicit expressions of its mean and variance. This provides a simple estimator of $\\theta$ with computable classical Fisher information which can be optimized over different choices of measurements. We then show that the output itself is asymptotically normal, i.e. at time $n$ the joint output states with parameters of the form $\\theta_{0}+u/\\sqrt{n}$ look like a one dimensional family of coherent states with displacement $u\\sqrt{F/2}$, where $F$ is the quantum Fisher information of the output. These results are quantum extensions of theorems from the theory of classical Markov chains, and further extensions to continuous time dynamics and multiple parameters are in preparation.

  3. Stochastic continuous time neurite branching models with tree and segment dependent rates

    NARCIS (Netherlands)

    van Elburg, Ronald A. J.

    2011-01-01

    In this paper we introduce a continuous time stochastic neurite branching model closely related to the discrete time stochastic BES-model. The discrete time BES-model is underlying current attempts to simulate cortical development, but is difficult to analyze. The new continuous time formulation fac

  4. The algebra of the general Markov model on phylogenetic trees and networks.

    Science.gov (United States)

    Sumner, J G; Holland, B R; Jarvis, P D

    2012-04-01

    It is known that the Kimura 3ST model of sequence evolution on phylogenetic trees can be extended quite naturally to arbitrary split systems. However, this extension relies heavily on mathematical peculiarities of the associated Hadamard transformation, and providing an analogous augmentation of the general Markov model has thus far been elusive. In this paper, we rectify this shortcoming by showing how to extend the general Markov model on trees to include incompatible edges; and even further to more general network models. This is achieved by exploring the algebra of the generators of the continuous-time Markov chain together with the “splitting” operator that generates the branching process on phylogenetic trees. For simplicity, we proceed by discussing the two state case and then show that our results are easily extended to more states with little complication. Intriguingly, upon restriction of the two state general Markov model to the parameter space of the binary symmetric model, our extension is indistinguishable from the Hadamard approach only on trees; as soon as any incompatible splits are introduced the two approaches give rise to differing probability distributions with disparate structure. Through exploration of a simple example, we give an argument that our extension to more general networks has desirable properties that the previous approaches do not share. In particular, our construction allows for convergent evolution of previously divergent lineages; a property that is of significant interest for biological applications.

  5. Revisiting Causality in Markov Chains

    CERN Document Server

    Shojaee, Abbas

    2016-01-01

    Identifying causal relationships is a key premise of scientific research. The growth of observational data in different disciplines along with the availability of machine learning methods offers the possibility of using an empirical approach to identifying potential causal relationships, to deepen our understandings of causal behavior and to build theories accordingly. Conventional methods of causality inference from observational data require a considerable length of time series data to capture cause-effect relationship. We find that potential causal relationships can be inferred from the composition of one step transition rates to and from an event. Also known as Markov chain, one step transition rates are a commonly available resource in different scientific disciplines. Here we introduce a simple, effective and computationally efficient method that we termed 'Causality Inference using Composition of Transitions CICT' to reveal causal structure with high accuracy. We characterize the differences in causes,...

  6. Renewal characterization of Markov modulated Poisson processes

    Directory of Open Access Journals (Sweden)

    Marcel F. Neuts

    1989-01-01

    Full Text Available A Markov Modulated Poisson Process (MMPP M(t defined on a Markov chain J(t is a pure jump process where jumps of M(t occur according to a Poisson process with intensity λi whenever the Markov chain J(t is in state i. M(t is called strongly renewal (SR if M(t is a renewal process for an arbitrary initial probability vector of J(t with full support on P={i:λi>0}. M(t is called weakly renewal (WR if there exists an initial probability vector of J(t such that the resulting MMPP is a renewal process. The purpose of this paper is to develop general characterization theorems for the class SR and some sufficiency theorems for the class WR in terms of the first passage times of the bivariate Markov chain [J(t,M(t]. Relevance to the lumpability of J(t is also studied.

  7. Noise can speed convergence in Markov chains.

    Science.gov (United States)

    Franzke, Brandon; Kosko, Bart

    2011-10-01

    A new theorem shows that noise can speed convergence to equilibrium in discrete finite-state Markov chains. The noise applies to the state density and helps the Markov chain explore improbable regions of the state space. The theorem ensures that a stochastic-resonance noise benefit exists for states that obey a vector-norm inequality. Such noise leads to faster convergence because the noise reduces the norm components. A corollary shows that a noise benefit still occurs if the system states obey an alternate norm inequality. This leads to a noise-benefit algorithm that requires knowledge of the steady state. An alternative blind algorithm uses only past state information to achieve a weaker noise benefit. Simulations illustrate the predicted noise benefits in three well-known Markov models. The first model is a two-parameter Ehrenfest diffusion model that shows how noise benefits can occur in the class of birth-death processes. The second model is a Wright-Fisher model of genotype drift in population genetics. The third model is a chemical reaction network of zeolite crystallization. A fourth simulation shows a convergence rate increase of 64% for states that satisfy the theorem and an increase of 53% for states that satisfy the corollary. A final simulation shows that even suboptimal noise can speed convergence if the noise applies over successive time cycles. Noise benefits tend to be sharpest in Markov models that do not converge quickly and that do not have strong absorbing states.

  8. Spectral Analysis of Markov Chains

    OpenAIRE

    2007-01-01

    The paper deals with the problem of a statistical analysis of Markov chains connected with the spectral density. We present the expressions for the function of spectral density. These expressions may be used to estimate the parameter of the Markov chain.

  9. Bayesian posterior distributions without Markov chains.

    Science.gov (United States)

    Cole, Stephen R; Chu, Haitao; Greenland, Sander; Hamra, Ghassan; Richardson, David B

    2012-03-01

    Bayesian posterior parameter distributions are often simulated using Markov chain Monte Carlo (MCMC) methods. However, MCMC methods are not always necessary and do not help the uninitiated understand Bayesian inference. As a bridge to understanding Bayesian inference, the authors illustrate a transparent rejection sampling method. In example 1, they illustrate rejection sampling using 36 cases and 198 controls from a case-control study (1976-1983) assessing the relation between residential exposure to magnetic fields and the development of childhood cancer. Results from rejection sampling (odds ratio (OR) = 1.69, 95% posterior interval (PI): 0.57, 5.00) were similar to MCMC results (OR = 1.69, 95% PI: 0.58, 4.95) and approximations from data-augmentation priors (OR = 1.74, 95% PI: 0.60, 5.06). In example 2, the authors apply rejection sampling to a cohort study of 315 human immunodeficiency virus seroconverters (1984-1998) to assess the relation between viral load after infection and 5-year incidence of acquired immunodeficiency syndrome, adjusting for (continuous) age at seroconversion and race. In this more complex example, rejection sampling required a notably longer run time than MCMC sampling but remained feasible and again yielded similar results. The transparency of the proposed approach comes at a price of being less broadly applicable than MCMC.

  10. Boundary value problems and Markov processes

    CERN Document Server

    Taira, Kazuaki

    2009-01-01

    This volume is devoted to a thorough and accessible exposition on the functional analytic approach to the problem of construction of Markov processes with Ventcel' boundary conditions in probability theory. Analytically, a Markovian particle in a domain of Euclidean space is governed by an integro-differential operator, called a Waldenfels operator, in the interior of the domain, and it obeys a boundary condition, called the Ventcel' boundary condition, on the boundary of the domain. Probabilistically, a Markovian particle moves both by jumps and continuously in the state space and it obeys the Ventcel' boundary condition, which consists of six terms corresponding to the diffusion along the boundary, the absorption phenomenon, the reflection phenomenon, the sticking (or viscosity) phenomenon, the jump phenomenon on the boundary, and the inward jump phenomenon from the boundary. In particular, second-order elliptic differential operators are called diffusion operators and describe analytically strong Markov pr...

  11. Jump Markov models and transition state theory: the quasi-stationary distribution approach.

    Science.gov (United States)

    Di Gesù, Giacomo; Lelièvre, Tony; Le Peutrec, Dorian; Nectoux, Boris

    2016-12-22

    We are interested in the connection between a metastable continuous state space Markov process (satisfying e.g. the Langevin or overdamped Langevin equation) and a jump Markov process in a discrete state space. More precisely, we use the notion of quasi-stationary distribution within a metastable state for the continuous state space Markov process to parametrize the exit event from the state. This approach is useful to analyze and justify methods which use the jump Markov process underlying a metastable dynamics as a support to efficiently sample the state-to-state dynamics (accelerated dynamics techniques). Moreover, it is possible by this approach to quantify the error on the exit event when the parametrization of the jump Markov model is based on the Eyring-Kramers formula. This therefore provides a mathematical framework to justify the use of transition state theory and the Eyring-Kramers formula to build kinetic Monte Carlo or Markov state models.

  12. Jump Markov models and transition state theory: the Quasi-Stationary Distribution approach

    CERN Document Server

    Di Gesù, Giacomo; Peutrec, Dorian Le; Nectoux, Boris

    2016-01-01

    We are interested in the connection between a metastable continuous state space Markov process (satisfying e.g. the Langevin or overdamped Langevin equation) and a jump Markov process in a discrete state space. More precisely, we use the notion of quasi-stationary distribution within a metastable state for the continuous state space Markov process to parametrize the exit event from the state. This approach is useful to analyze and justify methods which use the jump Markov process underlying a metastable dynamics as a support to efficiently sample the state-to-state dynamics (accelerated dynamics techniques). Moreover, it is possible by this approach to quantify the error on the exit event when the parametrization of the jump Markov model is based on the Eyring-Kramers formula. This therefore provides a mathematical framework to justify the use of transition state theory and the Eyring-Kramers formula to build kinetic Monte Carlo or Markov state models.

  13. Consensus of high-order continuous-time multi-agent systems with time-delays and switching topologies

    Institute of Scientific and Technical Information of China (English)

    Yang Tan; Jin Yue-Hui; Wang Wei; Shi Ying-Jing

    2011-01-01

    Consensus problems of high-order continuous-time multi-agent systems with time-delays and switching topologies are studied. The motivation of this work is to extend second-order continuous-time multi-agent systems from the literature. It is shown that consensus can be reached with arbitrarily bounded time-delays even though the communication topology might not have spanning trees. A numerical example is included to show the theoretical results.

  14. Hidden semi-Markov Model based earthquake classification system using Weighted Finite-State Transducers

    Directory of Open Access Journals (Sweden)

    M. Beyreuther

    2011-02-01

    Full Text Available Automatic earthquake detection and classification is required for efficient analysis of large seismic datasets. Such techniques are particularly important now because access to measures of ground motion is nearly unlimited and the target waveforms (earthquakes are often hard to detect and classify. Here, we propose to use models from speech synthesis which extend the double stochastic models from speech recognition by integrating a more realistic duration of the target waveforms. The method, which has general applicability, is applied to earthquake detection and classification. First, we generate characteristic functions from the time-series. The Hidden semi-Markov Models are estimated from the characteristic functions and Weighted Finite-State Transducers are constructed for the classification. We test our scheme on one month of continuous seismic data, which corresponds to 370 151 classifications, showing that incorporating the time dependency explicitly in the models significantly improves the results compared to Hidden Markov Models.

  15. Modelling the Heterogeneous Markov Attrition Process .

    Directory of Open Access Journals (Sweden)

    Jau Yeu Menq

    1993-01-01

    Full Text Available A model for heterogeneous dynamics combat as a continuos-time Markov process has been studied, and on account of the special form of its infinitesimal generator, recursive algorithms are derived to compute the important characteristics of the combat, such as the combat time distribution, expected value and variance, and the probability of winning and expected survivors. Numerical results are also presented. This approach can also be used to consider initial contact forces of both sides as random variables.

  16. A Markov chain Monte Carlo Expectation Maximization Algorithm for Statistical Analysis of DNA Sequence Evolution with Neighbor-Dependent Substitution Rates

    DEFF Research Database (Denmark)

    Hobolth, Asger

    2008-01-01

    The evolution of DNA sequences can be described by discrete state continuous time Markov processes on a phylogenetic tree. We consider neighbor-dependent evolutionary models where the instantaneous rate of substitution at a site depends on the states of the neighboring sites. Neighbor-dependent s...

  17. Theory of eigenvalues for periodic non-stationary Markov processes: the Kolmogorov operator and its applications

    Energy Technology Data Exchange (ETDEWEB)

    Caceres, Manuel O [Abdus Salam International Centre for Theoretical Physics, Strada Costiera, 11-34014 Trieste (Italy); Lobos, Alejandro M [Centro Atomico Bariloche, Instituto Balseiro, CNEA, Universidad Nacional de Cuyo, and CONICET, Av E Bustillo Km 9.5, 8400 Bariloche (Argentina)

    2006-02-17

    We present an eigenvalue theory to study the stochastic dynamics of non-stationary time-periodic Markov processes. The analysis is carried out by solving an integral operator of the Fredholm type, i.e. considering complex-valued functions fulfilling the Kolmogorov compatibility condition. We show that the asymptotic behaviour of the stochastic process is characterized by the smaller time-scale associated with the spectrum of the Kolmogorov operator. The presence of time-periodic elements in the evolution equation for the semigroup leads to a Floquet analysis. The first non-trivial Kolmogorov's eigenvalue is interpreted from a physical point of view. This non-trivial characteristic time-scale strongly depends on the interplay between the stochastic behaviour of the process and the time-periodic structure of the Fokker-Planck equation for continuous processes, or the periodically modulated master equation for discrete Markov processes. We present pedagogical examples in a finite-dimensional vector space to calculate the Kolmogorov characteristic time-scale for discrete Markov processes.

  18. On the Markov-dependent risk model with tax

    Institute of Scientific and Technical Information of China (English)

    PENG Xing-chun; WANG Wen-yuan; HU Yi-jun

    2015-01-01

    In this paper we consider the Markov-dependent risk model with tax payments in which the claim occurrence, the claim amount as well as the tax rate are controlled by an irreducible discrete-time Markov chain. Systems of integro-diff erential equations satisfied by the expected discounted tax payments and the non-ruin probability in terms of the ruin probabilities under the Markov-dependent risk model without tax are established. The analytical solutions of the systems of integro-diff erential equations are also obtained by the iteration method.

  19. Continuous-time performance limitations for overshoot and resulted tracking measures

    CERN Document Server

    wenczel, rob

    2011-01-01

    A dual formulation for the problem of determining absolute performance limitations on overshoot, undershoot, maximum amplitude and fluctuation minimization for continuous-time feedback systems is constructed. Determining, for example, the minimum possible overshoot attainable by all possible stabilizing controllers is an optimization task that cannot be expressed as a minimum-norm problem. It is this fact, coupled with the continuous-time rather than discrete-time formulation, that makes these problems challenging. We extend previous results to include more general reference functions, and derive new results (in continuous time) on the influence of pole/zero locations on achievable time-domain performance.

  20. Markov chain aggregation and its applications to combinatorial reaction networks.

    Science.gov (United States)

    Ganguly, Arnab; Petrov, Tatjana; Koeppl, Heinz

    2014-09-01

    We consider a continuous-time Markov chain (CTMC) whose state space is partitioned into aggregates, and each aggregate is assigned a probability measure. A sufficient condition for defining a CTMC over the aggregates is presented as a variant of weak lumpability, which also characterizes that the measure over the original process can be recovered from that of the aggregated one. We show how the applicability of de-aggregation depends on the initial distribution. The application section is devoted to illustrate how the developed theory aids in reducing CTMC models of biochemical systems particularly in connection to protein-protein interactions. We assume that the model is written by a biologist in form of site-graph-rewrite rules. Site-graph-rewrite rules compactly express that, often, only a local context of a protein (instead of a full molecular species) needs to be in a certain configuration in order to trigger a reaction event. This observation leads to suitable aggregate Markov chains with smaller state spaces, thereby providing sufficient reduction in computational complexity. This is further exemplified in two case studies: simple unbounded polymerization and early EGFR/insulin crosstalk.

  1. The application of Markov decision process in restaurant delivery robot

    Science.gov (United States)

    Wang, Yong; Hu, Zhen; Wang, Ying

    2017-05-01

    As the restaurant delivery robot is often in a dynamic and complex environment, including the chairs inadvertently moved to the channel and customers coming and going. The traditional path planning algorithm is not very ideal. To solve this problem, this paper proposes the Markov dynamic state immediate reward (MDR) path planning algorithm according to the traditional Markov decision process. First of all, it uses MDR to plan a global path, then navigates along this path. When the sensor detects there is no obstructions in front state, increase its immediate state reward value; when the sensor detects there is an obstacle in front, plan a global path that can avoid obstacle with the current position as the new starting point and reduce its state immediate reward value. This continues until the target is reached. When the robot learns for a period of time, it can avoid those places where obstacles are often present when planning the path. By analyzing the simulation experiment, the algorithm has achieved good results in the global path planning under the dynamic environment.

  2. The effect of real-time continuous glucose monitoring in pregnant women with diabetes

    DEFF Research Database (Denmark)

    Secher, Anna L; Ringholm, Lene; Damm, Peter;

    2013-01-01

    To assess whether intermittent real-time continuous glucose monitoring (CGM) improves glycemic control and pregnancy outcome in unselected women with pregestational diabetes.......To assess whether intermittent real-time continuous glucose monitoring (CGM) improves glycemic control and pregnancy outcome in unselected women with pregestational diabetes....

  3. Markov chain Monte Carlo simulation for Bayesian Hidden Markov Models

    Science.gov (United States)

    Chan, Lay Guat; Ibrahim, Adriana Irawati Nur Binti

    2016-10-01

    A hidden Markov model (HMM) is a mixture model which has a Markov chain with finite states as its mixing distribution. HMMs have been applied to a variety of fields, such as speech and face recognitions. The main purpose of this study is to investigate the Bayesian approach to HMMs. Using this approach, we can simulate from the parameters' posterior distribution using some Markov chain Monte Carlo (MCMC) sampling methods. HMMs seem to be useful, but there are some limitations. Therefore, by using the Mixture of Dirichlet processes Hidden Markov Model (MDPHMM) based on Yau et. al (2011), we hope to overcome these limitations. We shall conduct a simulation study using MCMC methods to investigate the performance of this model.

  4. Compressing redundant information in Markov chains

    OpenAIRE

    2006-01-01

    Given a strongly stationary Markov chain and a finite set of stopping rules, we prove the existence of a polynomial algorithm which projects the Markov chain onto a minimal Markov chain without redundant information. Markov complexity is hence defined and tested on some classical problems.

  5. Correlated Node Behavior Model based on Semi Markov Process for MANETS

    CERN Document Server

    Azni, A H; Noh, Zul Azri Muhamad; Basari, Abd Samad Hasan; Hussin, Burairah

    2012-01-01

    This paper introduces a new model for node behavior namely Correlated Node Behavior Model which is an extension of Node Behavior Model. The model adopts semi Markov process in continuous time which clusters the node that has correlation. The key parameter of the process is determined by five probabilistic parameters based on the Markovian model. Computed from the transition probabilities of the semi-Markov process, the node correlation impact on network survivability and resilience can be measure quantitatively. From the result, the quantitative analysis of correlated node behavior on the survivability is obtained through mathematical description, and the effectiveness and rationality of the proposed model are verified through numerical analysis. The analytical results show that the effect from correlated failure nodes on network survivability is much severer than other misbehaviors.

  6. Markov chain model helps predict pitting corrosion depth and rate in underground pipelines

    Energy Technology Data Exchange (ETDEWEB)

    Caleyo, F.; Velazquez, J.C.; Hallen, J. M. [ESIQIE, Instituto Politecnico Nacional, Mexico D. F. (Mexico); Esquivel-Amezcua, A. [PEMEX PEP Region Sur, Villahermosa, Tabasco (Mexico); Valor, A. [Universidad de la Habana, Vedado, La Habana (Cuba)

    2010-07-01

    Recent reports place pipeline corrosion costs in North America at seven billion dollars per year. Pitting corrosion causes the higher percentage of failures among other corrosion mechanisms. This has motivated multiple modelling studies to be focused on corrosion pitting of underground pipelines. In this study, a continuous-time, non-homogenous pure birth Markov chain serves to model external pitting corrosion in buried pipelines. The analytical solution of Kolmogorov's forward equations for this type of Markov process gives the transition probability function in a discrete space of pit depths. The transition probability function can be completely identified by making a correlation between the stochastic pit depth mean and the deterministic mean obtained experimentally. The model proposed in this study can be applied to pitting corrosion data from repeated in-line pipeline inspections. Case studies presented in this work show how pipeline inspection and maintenance planning can be improved by using the proposed Markovian model for pitting corrosion.

  7. Continuous-time operational modal analysis in the presence of harmonic disturbances—The multivariate case

    Science.gov (United States)

    Pintelon, R.; Peeters, B.; Guillaume, P.

    2010-01-01

    Recently [R. Pintelon, B. Peeters, P. Guillaume, Continuous-time operational modal analysis in the presence of harmonic disturbances, Mechanical Systems and Signal Processing 22 (5) (2008) 1017-1035] a single-output algorithm for continuous-time operational modal analysis in the presence of harmonic disturbances with time-varying frequency has been developed. This paper extends the results of Pintelon, et al. [Continuous-time operational modal analysis in the presence of harmonic disturbances, Mechanical Systems and Signal Processing 22 (5) (2008) 1017-1035] to multi-output signals. The statistical performance of the proposed maximum likelihood estimator is illustrated on simulations and real helicopter data.

  8. Markov Chains and Chemical Processes

    Science.gov (United States)

    Miller, P. J.

    1972-01-01

    Views as important the relating of abstract ideas of modern mathematics now being taught in the schools to situations encountered in the sciences. Describes use of matrices and Markov chains to study first-order processes. (Author/DF)

  9. Bibliometric Application of Markov Chains.

    Science.gov (United States)

    Pao, Miranda Lee; McCreery, Laurie

    1986-01-01

    A rudimentary description of Markov Chains is presented in order to introduce its use to describe and to predict authors' movements among subareas of the discipline of ethnomusicology. Other possible applications are suggested. (Author)

  10. Soundness of Timed-Arc Workflow Nets in Discrete and Continuous-Time Semantics

    DEFF Research Database (Denmark)

    Mateo, Jose Antonio; Srba, Jiri; Sørensen, Mathias Grund

    2015-01-01

    Analysis of workflow processes with quantitative aspectslike timing is of interest in numerous time-critical applications. We suggest a workflow model based on timed-arc Petri nets and studythe foundational problems of soundness and strong (time-bounded) soundness.We first consider the discrete-t...

  11. A Multilayer Hidden Markov Models-Based Method for Human-Robot Interaction

    Directory of Open Access Journals (Sweden)

    Chongben Tao

    2013-01-01

    Full Text Available To achieve Human-Robot Interaction (HRI by using gestures, a continuous gesture recognition approach based on Multilayer Hidden Markov Models (MHMMs is proposed, which consists of two parts. One part is gesture spotting and segment module, the other part is continuous gesture recognition module. Firstly, a Kinect sensor is used to capture 3D acceleration and 3D angular velocity data of hand gestures. And then, a Feed-forward Neural Networks (FNNs and a threshold criterion are used for gesture spotting and segment, respectively. Afterwards, the segmented gesture signals are respectively preprocessed and vector symbolized by a sliding window and a K-means clustering method. Finally, symbolized data are sent into Lower Hidden Markov Models (LHMMs to identify individual gestures, and then, a Bayesian filter with sequential constraints among gestures in Upper Hidden Markov Models (UHMMs is used to correct recognition errors created in LHMMs. Five predefined gestures are used to interact with a Kinect mobile robot in experiments. The experimental results show that the proposed method not only has good effectiveness and accuracy, but also has favorable real-time performance.

  12. Markov processes, semigroups and generators

    CERN Document Server

    Kolokoltsov, Vassili N

    2011-01-01

    This work offers a highly useful, well developed reference on Markov processes, the universal model for random processes and evolutions. The wide range of applications, in exact sciences as well as in other areas like social studies, require a volume that offers a refresher on fundamentals before conveying the Markov processes and examples for applications. This work does just that, and with the necessary mathematical rigor.

  13. On nonlinear Markov chain Monte Carlo

    CERN Document Server

    Andrieu, Christophe; Doucet, Arnaud; Del Moral, Pierre; 10.3150/10-BEJ307

    2011-01-01

    Let $\\mathscr{P}(E)$ be the space of probability measures on a measurable space $(E,\\mathcal{E})$. In this paper we introduce a class of nonlinear Markov chain Monte Carlo (MCMC) methods for simulating from a probability measure $\\pi\\in\\mathscr{P}(E)$. Nonlinear Markov kernels (see [Feynman--Kac Formulae: Genealogical and Interacting Particle Systems with Applications (2004) Springer]) $K:\\mathscr{P}(E)\\times E\\rightarrow\\mathscr{P}(E)$ can be constructed to, in some sense, improve over MCMC methods. However, such nonlinear kernels cannot be simulated exactly, so approximations of the nonlinear kernels are constructed using auxiliary or potentially self-interacting chains. Several nonlinear kernels are presented and it is demonstrated that, under some conditions, the associated approximations exhibit a strong law of large numbers; our proof technique is via the Poisson equation and Foster--Lyapunov conditions. We investigate the performance of our approximations with some simulations.

  14. Weak Markov Processes as Linear Systems

    CERN Document Server

    Gohm, Rolf

    2012-01-01

    A noncommutative Fornasini-Marchesini system (a multi-variable version of a linear system) can be realized within a weak Markov process (a model for quantum evolution). For a discrete time parameter this is worked out systematically as a theory of representations of structure maps of a system by a weak process. We introduce subprocesses and quotient processes which can be described naturally by a suitable category of weak processes. A corresponding notion of cascade for processes induces a represented cascade of systems. We study the control theoretic notion of observability which turns out to be particularly interesting in connection with a cascade structure. As an application we gain new insights into stationary Markov chains where observability for the system is closely related to asymptotic completeness in the scattering theory of the chain. This motivates a general definition of asymptotic completeness in the category of weak processes.

  15. Stochastic Dynamics through Hierarchically Embedded Markov Chains

    Science.gov (United States)

    Vasconcelos, Vítor V.; Santos, Fernando P.; Santos, Francisco C.; Pacheco, Jorge M.

    2017-02-01

    Studying dynamical phenomena in finite populations often involves Markov processes of significant mathematical and/or computational complexity, which rapidly becomes prohibitive with increasing population size or an increasing number of individual configuration states. Here, we develop a framework that allows us to define a hierarchy of approximations to the stationary distribution of general systems that can be described as discrete Markov processes with time invariant transition probabilities and (possibly) a large number of states. This results in an efficient method for studying social and biological communities in the presence of stochastic effects—such as mutations in evolutionary dynamics and a random exploration of choices in social systems—including situations where the dynamics encompasses the existence of stable polymorphic configurations, thus overcoming the limitations of existing methods. The present formalism is shown to be general in scope, widely applicable, and of relevance to a variety of interdisciplinary problems.

  16. A Graph-Algorithmic Approach for the Study of Metastability in Markov Chains

    Science.gov (United States)

    Gan, Tingyue; Cameron, Maria

    2017-01-01

    Large continuous-time Markov chains with exponentially small transition rates arise in modeling complex systems in physics, chemistry, and biology. We propose a constructive graph-algorithmic approach to determine the sequence of critical timescales at which the qualitative behavior of a given Markov chain changes, and give an effective description of the dynamics on each of them. This approach is valid for both time-reversible and time-irreversible Markov processes, with or without symmetry. Central to this approach are two graph algorithms, Algorithm 1 and Algorithm 2, for obtaining the sequences of the critical timescales and the hierarchies of Typical Transition Graphs or T-graphs indicating the most likely transitions in the system without and with symmetry, respectively. The sequence of critical timescales includes the subsequence of the reciprocals of the real parts of eigenvalues. Under a certain assumption, we prove sharp asymptotic estimates for eigenvalues (including pre-factors) and show how one can extract them from the output of Algorithm 1. We discuss the relationship between Algorithms 1 and 2 and explain how one needs to interpret the output of Algorithm 1 if it is applied in the case with symmetry instead of Algorithm 2. Finally, we analyze an example motivated by R. D. Astumian's model of the dynamics of kinesin, a molecular motor, by means of Algorithm 2.

  17. A Markov Chain Monte Carlo Method for Simulation of Wind and Solar Power Time Series%风光发电功率时间序列模拟的MCMC方法

    Institute of Scientific and Technical Information of China (English)

    罗钢; 石东源; 陈金富; 吴小珊

    2014-01-01

    准确、合理地构建间歇性电源的发电功率模型对于电力系统的仿真分析与计算具有重要意义。提出了一种风光发电功率时间序列模拟的单变量与多变量马尔科夫链蒙特卡罗(Markov chain Monte Carlo,MCMC)仿真方法。该模型针对风电场与光伏电站等多种类型的间歇性电源,构建发电功率时间序列的马尔科夫链,采用Gibbs抽样技术实现了单变量或多变量的时间序列模拟。不仅全面地分析了不同类型间歇性电源马尔科夫过程的特征与影响因素,并且在 MCMC方法中考虑了多变量之间的相互联系,使模型能够适应多组间歇性电源彼此间存在相关性的情形。对德国2家电力公司控制区域内的风电场、光伏电站进行仿真模拟,通过统计特征参数的对比分析,验证了所提模型的有效性。%Constructing a model of generated power for intermittent power source accurately and reasonably is of great significance for power system simulation and analysis. A single- and multi-variable Markov chain Monte Carlo (MCMC) method to simulate time series of wind and photovoltaic (PV) power is proposed. Aiming to many types of intermittent power source such as wind farms and PV generation stations, the proposed model constructs Markov chain for time series of generated power and utilizing Gibbs sampling the single- or multi-variable time series simulation is realized. Not only the features and impacting factors of Markov processes for different types of intermittent power sources are overall analyzed, and in the MCMC method the interrelation among multi-variables is considered, thus the proposed model can adapt to the condition that there are interrelations among multi sets of intermittent power sources. The wind farms and PV power plants located in the control areas of two power companies in Germany are simulated, and through the comparative analysis on statistical feature parameters the validity of

  18. CMOS continuous-time adaptive equalizers for high-speed serial links

    CERN Document Server

    Gimeno Gasca, Cecilia; Aldea Chagoyen, Concepción

    2015-01-01

    This book introduces readers to the design of adaptive equalization solutions integrated in standard CMOS technology for high-speed serial links. Since continuous-time equalizers offer various advantages as an alternative to discrete-time equalizers at multi-gigabit rates, this book provides a detailed description of continuous-time adaptive equalizers design - both at transistor and system levels-, their main characteristics and performances. The authors begin with a complete review and analysis of the state of the art of equalizers for wireline applications, describing why they are necessary, their types, and their main applications. Next, theoretical fundamentals of continuous-time adaptive equalizers are explored. Then, new structures are proposed to implement the different building blocks of the adaptive equalizer: line equalizer, loop-filters, power comparator, etc.  The authors demonstrate the design of a complete low-power, low-voltage, high-speed, continuous-time adaptive equalizer. Finally, a cost-...

  19. Implementation of Time-Scale Transformation Based on Continuous Wavelet Theory

    Institute of Scientific and Technical Information of China (English)

    2000-01-01

    The basic objective of time-scale transformation is to compress or expand the signal in time field while keeping the same spectral properties.This paper presents two methods to derive time-scale transformation formula based on continuous wavelet transform.For an arbitrary given square-integrable function f(t),g(t) = f(t/λ) is derived by continuous wavelet transform and its inverse transform.The result shows that time-scale transformation may be obtained through the modification of the time-scale of wavelet function filter using equivalent substitution. The paper demonstrates the result by theoretic derivations and experimental simulation.

  20. Strong Feller Continuity of Feller Processes and Semigroups

    OpenAIRE

    Schilling, René L.; WANG, JIAN

    2010-01-01

    We study two equivalent characterizations of the strong Feller property for a Markov process and of the associated sub-Markovian semigroup. One is described in terms of locally uniform absolute continuity, whereas the other uses local Orlicz-ultracontractivity. These criteria generalize many existing results on strong Feller continuity and seem to be more natural for Feller processes. By establishing the estimates of the first exit time from balls, we also investigate the continuity of harmon...

  1. Combination Load Forecast with Time-varying Weights Based on Markov Chain%马尔科夫链在电力负荷组合预测中的应用

    Institute of Scientific and Technical Information of China (English)

    李敏; 江辉; 黄银华; 宋小明

    2011-01-01

    In view of the specific utilization scope and condition of each single forecasting model, a novel combinatorial forecasting algorithm with nonnegative time-varying based on Markov chain is proposed in this paper.Firstly, Markov chain is used to fit the law of status probability distribution of these filtered models, and then the estimating problem of the one-step status probabilities transition matrix is translated into constrained multivariate self-regression analysis model. Secondly, the combination weights of these filtered models are determined through the estimation of the one-step status probabilities transition matrix and the distribution of status probability. Results of calculation examples show that the forecasting results by the proposed model is accurate and the proposed method is practicable.%针对单一模型都有其特定的适用范围和条件,文中提出了一种基于马尔科夫链拟合的非负时变权重组合预测算法.该算法通过马尔科夫链对筛选出模型的状态概率分布变化规律进行拟合,将一步转移概率矩阵的估计问题转化为多元约束自回归模型,然后利用一步转移概率矩阵的估计和初始状态概率分布来确定组合权重.实例表明,该方法计算量小、精确度高、具有实用性.

  2. Solution Estimates for Semilinear Difference-Delay Equations with Continuous Time

    Directory of Open Access Journals (Sweden)

    Michael Gil'

    2007-01-01

    Full Text Available We consider semilinear difference-delay equations with continuous time in a Euclidean space. Estimates are found for the solutions. Such estimates are then applied to obtain the stability and boundedness criteria for solutions.

  3. Growth of Preferential Attachment Random Graphs Via Continuous-Time Branching Processes

    Indian Academy of Sciences (India)

    Krishna B Athreya; Arka P Ghosh; Sunder Sethuraman

    2008-08-01

    Some growth asymptotics of a version of `preferential attachment’ random graphs are studied through an embedding into a continuous-time branching scheme. These results complement and extend previous work in the literature.

  4. Continuous-time model identification and state estimation using non-uniformly sampled data

    OpenAIRE

    2009-01-01

    This contribution reviews theory, algorithms, and validation results for system identification of continuous-time state-space models from finite input-output sequences. The algorithms developed are autoregressive methods, methods of subspace-based model identification and stochastic realization adapted to the continuous-time context. The resulting model can be decomposed into an input-output model and a stochastic innovations model. Using the Riccati equation, we have designed a procedure to ...

  5. Estimation in continuous-time stochastic| volatility models using nonlinear filters

    DEFF Research Database (Denmark)

    Nielsen, Jan Nygaard; Vestergaard, M.; Madsen, Henrik

    2000-01-01

    Presents a correction to the authorship of the article 'Estimation in Continuous-Time Stochastic Volatility Models Using Nonlinear Filters,' published in the periodical 'International Journal of Theoretical and Applied Finance,' Vol. 3, No. 2., pp. 279-308.......Presents a correction to the authorship of the article 'Estimation in Continuous-Time Stochastic Volatility Models Using Nonlinear Filters,' published in the periodical 'International Journal of Theoretical and Applied Finance,' Vol. 3, No. 2., pp. 279-308....

  6. Improving GOOGLE'S Cartographer 3d Mapping by Continuous-Time Slam

    Science.gov (United States)

    Nüchter, A.; Bleier, M.; Schauer, J.; Janotta, P.

    2017-02-01

    This paper shows how to use the result of Google's SLAM solution, called Cartographer, to bootstrap our continuous-time SLAM algorithm. The presented approach optimizes the consistency of the global point cloud, and thus improves on Google's results. We use the algorithms and data from Google as input for our continuous-time SLAM software. We also successfully applied our software to a similar backpack system which delivers consistent 3D point clouds even in absence of an IMU.

  7. From Continuous-Time Design to Sampled-Data Design of Nonlinear Observers

    OpenAIRE

    Karafyllis, Iasson; Kravaris, Costas

    2008-01-01

    In this work, a sampled-data nonlinear observer is designed using a continuous-time design coupled with an inter-sample output predictor. The proposed sampled-data observer is a hybrid system. It is shown that under certain conditions, the robustness properties of the continuous-time design are inherited by the sampled-data design, as long as the sampling period is not too large. The approach is applied to linear systems and to triangular globally Lipschitz systems.

  8. Application of Markov Chains to Stock Trends

    Directory of Open Access Journals (Sweden)

    Kevin J. Doubleday

    2011-01-01

    Full Text Available Problem statement: Modeling of the Dow Jones Industrial Average is frequently attempted in order to determine trading strategies with maximum payoff. Changes in the DJIA are important since movements may affect both individuals and corporations profoundly. Previous work showed that modeling a market as a random walk was valid and that a market may be viewed as having the Markov property. Approach: The aim of this research was to determine the relationship between a diverse portfolio of stocks and the market as a whole. To that end, the DJIA was analyzed using a discrete time stochastic model, namely a Markov Chain. Two models were highlighted, where the DJIA was considered as being in a state of (1 gain or loss and (2 small, moderate, or large gain or loss. A portfolio of five stocks was then considered and two models of the portfolio much the same as those for the DJIA. These models were used to obtain transitional probabilities and steady state probabilities. Results: Our results indicated that the portfolio behaved similarly to the entire DJIA, both in the simple model and the partitioned model. Conclusion: When treated as a Markov process, the entire market was useful in gauging how a diverse portfolio of stocks might behave. Future work may include different classifications of states to refine the transition matrices.

  9. NonMarkov Ito Processes with 1- state memory

    Science.gov (United States)

    McCauley, Joseph L.

    2010-08-01

    A Markov process, by definition, cannot depend on any previous state other than the last observed state. An Ito process implies the Fokker-Planck and Kolmogorov backward time partial differential eqns. for transition densities, which in turn imply the Chapman-Kolmogorov eqn., but without requiring the Markov condition. We present a class of Ito process superficially resembling Markov processes, but with 1-state memory. In finance, such processes would obey the efficient market hypothesis up through the level of pair correlations. These stochastic processes have been mislabeled in recent literature as 'nonlinear Markov processes'. Inspired by Doob and Feller, who pointed out that the ChapmanKolmogorov eqn. is not restricted to Markov processes, we exhibit a Gaussian Ito transition density with 1-state memory in the drift coefficient that satisfies both of Kolmogorov's partial differential eqns. and also the Chapman-Kolmogorov eqn. In addition, we show that three of the examples from McKean's seminal 1966 paper are also nonMarkov Ito processes. Last, we show that the transition density of the generalized Black-Scholes type partial differential eqn. describes a martingale, and satisfies the ChapmanKolmogorov eqn. This leads to the shortest-known proof that the Green function of the Black-Scholes eqn. with variable diffusion coefficient provides the so-called martingale measure of option pricing.

  10. Minimal state space realisation of continuous-time linear time-variant input-output models

    Science.gov (United States)

    Goos, J.; Pintelon, R.

    2016-04-01

    In the linear time-invariant (LTI) framework, the transformation from an input-output equation into state space representation is well understood. Several canonical forms exist that realise the same dynamic behaviour. If the coefficients become time-varying however, the LTI transformation no longer holds. We prove by induction that there exists a closed-form expression for the observability canonical state space model, using binomial coefficients.

  11. Subspace identification for continuous-time errors-in-variables model from sampled data

    Institute of Scientific and Technical Information of China (English)

    Ping WU; Chun-jie YANG; Zhi-huan SONG

    2009-01-01

    We study the subspace identification for the continuous-time errors-in-variables model from sampled data. First, the filtering approach is applied to handle the time-derivative problem inherent in continuous-time identification. The generalized Poisson moment functional is focused. A total least squares equation based on this filtering approach is derived. Inspired by the idea of discrete-time subspace identification based on principal component analysis, we develop two algorithms to deliver consistent estimates for the continuous-time errors-in-variables model by introducing two different instrumental variables. Order determination and other instrumental variables are discussed. The usefulness of the proposed algorithms is illustrated through numerical simulation.

  12. Dynamic Radioactive Source for Evaluating and Demonstrating Time-dependent Performance of Continuous Air Monitors.

    Science.gov (United States)

    McLean, Thomas D; Moore, Murray E; Justus, Alan L; Hudston, Jonathan A; Barbé, Benoît

    2016-11-01

    Evaluation of continuous air monitors in the presence of a plutonium aerosol is time intensive, expensive, and requires a specialized facility. The Radiation Protection Services Group at Los Alamos National Laboratory has designed a Dynamic Radioactive Source, intended to replace plutonium aerosol challenge testing. The Dynamic Radioactive Source is small enough to be inserted into the sampler filter chamber of a typical continuous air monitor. Time-dependent radioactivity is introduced from electroplated sources for real-time testing of a continuous air monitor where a mechanical wristwatch motor rotates a mask above an alpha-emitting electroplated disk source. The mask is attached to the watch's minute hand, and as it rotates, more of the underlying source is revealed. The measured alpha activity increases with time, simulating the arrival of airborne radioactive particulates at the air sampler inlet. The Dynamic Radioactive Source allows the temporal behavior of puff and chronic release conditions to be mimicked without the need for radioactive aerosols. The new system is configurable to different continuous air monitor designs and provides an in-house testing capability (benchtop compatible). It is a repeatable and reusable system and does not contaminate the tested air monitor. Test benefits include direct user control, realistic (plutonium) aerosol spectra, and iterative development of continuous air monitor alarm algorithms. Data obtained using the Dynamic Radioactive Source has been used to elucidate alarm algorithms and to compare the response time of two commercial continuous air monitors.

  13. Synchronization of coupled noisy oscillators: Coarse graining from continuous to discrete phases

    Science.gov (United States)

    Escaff, Daniel; Rosas, Alexandre; Toral, Raúl; Lindenberg, Katja

    2016-11-01

    The theoretical description of synchronization phenomena often relies on coupled units of continuous time noisy Markov chains with a small number of states in each unit. It is frequently assumed, either explicitly or implicitly, that coupled discrete-state noisy Markov units can be used to model mathematically more complex coupled noisy continuous phase oscillators. In this work we explore conditions that justify this assumption by coarse graining continuous phase units. In particular, we determine the minimum number of states necessary to justify this correspondence for Kuramoto-like oscillators.

  14. Social security as Markov equilibrium in OLG models: A note

    DEFF Research Database (Denmark)

    Gonzalez Eiras, Martin

    2011-01-01

    I refine and extend the Markov perfect equilibrium of the social security policy game in Forni (2005) for the special case of logarithmic utility. Under the restriction that the policy function be continuous, instead of differentiable, the equilibrium is globally well defined and its dynamics...

  15. Fluctuation Theorems for Entropy Production and Heat Dissipation in Periodically Driven Markov Chains

    CERN Document Server

    Shargel, Benjamin Hertz

    2009-01-01

    Asymptotic fluctuation theorems are statements of a Gallavotti-Cohen symmetry in the rate function of either the time-averaged entropy production or heat dissipation of a process. Such theorems have been proved for various general classes of continuous-time deterministic and stochastic processes, but always under the assumption that the forces driving the system are time independent, and often relying on the existence of a limiting ergodic distribution. In this paper we extend the asymptotic fluctuation theorem for the first time to inhomogeneous continuous-time processes without a stationary distribution, considering specifically a finite state Markov chain driven by periodic transition rates. We find that for both entropy production and heat dissipation, the usual Gallavotti-Cohen symmetry of the rate function is generalized to an analogous relation between the rate functions of the original process and its corresponding backward process, in which the trajectory and the driving protocol have been time-rever...

  16. Stochastic model of milk homogenization process using Markov's chain

    OpenAIRE

    A. A. Khvostov; R. S. Sumina; G. I. Kotov; Ivanov, A. V.

    2016-01-01

    The process of development of a mathematical model of the process of homogenization of dairy products is considered in the work. The theory of Markov's chains was used in the development of the mathematical model, Markov's chain with discrete states and continuous parameter for which the homogenisation pressure is taken, being the basis for the model structure. Machine realization of the model is implemented in the medium of structural modeling MathWorks Simulink™. Identification of the model...

  17. Exploratory Study for Continuous-time Parameter Estimation of Ankle Dynamics

    Science.gov (United States)

    Kukreja, Sunil L.; Boyle, Richard D.

    2014-01-01

    Recently, a parallel pathway model to describe ankle dynamics was proposed. This model provides a relationship between ankle angle and net ankle torque as the sum of a linear and nonlinear contribution. A technique to identify parameters of this model in discrete-time has been developed. However, these parameters are a nonlinear combination of the continuous-time physiology, making insight into the underlying physiology impossible. The stable and accurate estimation of continuous-time parameters is critical for accurate disease modeling, clinical diagnosis, robotic control strategies, development of optimal exercise protocols for longterm space exploration, sports medicine, etc. This paper explores the development of a system identification technique to estimate the continuous-time parameters of ankle dynamics. The effectiveness of this approach is assessed via simulation of a continuous-time model of ankle dynamics with typical parameters found in clinical studies. The results show that although this technique improves estimates, it does not provide robust estimates of continuous-time parameters of ankle dynamics. Due to this we conclude that alternative modeling strategies and more advanced estimation techniques be considered for future work.

  18. Real-time assessment of critical quality attributes of a continuous granulation process.

    Science.gov (United States)

    Fonteyne, Margot; Vercruysse, Jurgen; Díaz, Damián Córdoba; Gildemyn, Delphine; Vervaet, Chris; Remon, Jean Paul; De Beer, Thomas

    2013-02-01

    There exists the intention to shift pharmaceutical manufacturing of solid dosage forms from traditional batch production towards continuous production. The currently applied conventional quality control systems, based on sampling and time-consuming off-line analyses in analytical laboratories, would annul the advantages of continuous processing. It is clear that real-time quality assessment and control is indispensable for continuous production. This manuscript evaluates strengths and weaknesses of several complementary Process Analytical Technology (PAT) tools implemented in a continuous wet granulation process, which is part of a fully continuous from powder-to-tablet production line. The use of Raman and NIR-spectroscopy and a particle size distribution analyzer is evaluated for the real-time monitoring of critical parameters during the continuous wet agglomeration of an anhydrous theophylline- lactose blend. The solid state characteristics and particle size of the granules were analyzed in real-time and the critical process parameters influencing these granule characteristics were identified. The temperature of the granulator barrel, the amount of granulation liquid added and, to a lesser extent, the powder feed rate were the parameters influencing the solid state of the active pharmaceutical ingredient (API). A higher barrel temperature and a higher powder feed rate, resulted in larger granules.

  19. Continuous review inventory models under time value of money and crashable lead time consideration

    Directory of Open Access Journals (Sweden)

    Hung Kuo-Chen

    2011-01-01

    Full Text Available A stock is an asset if it can react to economic and seasonal influences in the management of the current assets. The financial manager must calculate the input of funds to the stock intelligently and the amount of money cycled through stocks, taking into account the time factors in the future. The purpose of this paper is to propose an inventory model considering issues of crash cost and current value. The sensitivity analysis of each parameter, in this research, differs from the traditional approach. We utilize a course of deduction with sound mathematics to develop several lemmas and one theorem to estimate optimal solutions. This study first tries to find the optimal order quantity at all lengths of lead time with components crashed at their minimum duration. Second, a simple method to locate the optimal solution unlike traditional sensitivity analysis is developed. Finally, some numerical examples are given to illustrate all lemmas and the theorem in the solution algorithm.

  20. Markov Networks in Evolutionary Computation

    CERN Document Server

    Shakya, Siddhartha

    2012-01-01

    Markov networks and other probabilistic graphical modes have recently received an upsurge in attention from Evolutionary computation community, particularly in the area of Estimation of distribution algorithms (EDAs).  EDAs have arisen as one of the most successful experiences in the application of machine learning methods in optimization, mainly due to their efficiency to solve complex real-world optimization problems and their suitability for theoretical analysis. This book focuses on the different steps involved in the conception, implementation and application of EDAs that use Markov networks, and undirected models in general. It can serve as a general introduction to EDAs but covers also an important current void in the study of these algorithms by explaining the specificities and benefits of modeling optimization problems by means of undirected probabilistic models. All major developments to date in the progressive introduction of Markov networks based EDAs are reviewed in the book. Hot current researc...

  1. Mixing and decoherence in continuous-time quantum walks on long-range interacting cycles

    Energy Technology Data Exchange (ETDEWEB)

    Salimi, S; Radgohar, R [Faculty of Science, Department of Physics, University of Kurdistan, Pasdaran Ave., Sanandaj (Iran, Islamic Republic of)], E-mail: shsalimi@uok.ac.ir, E-mail: r.radgohar@uok.ac.ir

    2009-11-27

    We study the effect of small decoherence in continuous-time quantum walks on long-range interacting cycles, which are constructed by connecting all the two nodes of distance m on the cycle graph. In our investigation, each node is continuously monitored by an individual point contact, which induces the decoherence process. We obtain the analytical probability distribution and the mixing time upper bound. Our results show that, for small rates of decoherence, the mixing time upper bound is independent of distance parameter m and is proportional to inverse of decoherence rate.

  2. Cooperation in an Infinite-Choice Continuous-Time Prisoner's Dilemma.

    Science.gov (United States)

    Feeley, Thomas H.; Tutzauer, Frank; Young, Melissa J.; Rosenfeld, Heather L.

    1997-01-01

    The Prisoner's Dilemma (PD) game demonstrates how cooperative or competitive choices influence decision making between two people or groups. A study of 48 college students tested an infinite-choice, continuous-time version of the PD. Results indicated that oscillatory cooperation was the predominant over-time behavior, that players matched…

  3. Effect of delay time on the generation of chaos in continuous systems

    CERN Document Server

    Berezowski, Marek

    2016-01-01

    The present study deals with a theoretical analysis of the effect of delay time of energy transport upon the generation of complex dynamics in continuous physical system. The importance of this time for the presence of quasi - periodicity and chaos in a reactor is demonstrated. The considerations are preceded by the analysis of one - dimensional mathematical model.

  4. Chaotification of polynomial continuous-time systems and rational normal forms

    Energy Technology Data Exchange (ETDEWEB)

    Starkov, Konstantin E-mail: konst@citedi.mxkonstarkov@hotmail.com; Chen Guanrong E-mail: eegchen@cityu.edu.hk

    2004-11-01

    In this paper we study the chaotification problem of polynomial continuous-time systems in a semiglobal setting. Our results are based on the computation of rational normal forms and time-delay anticontroller design. As examples, the Roessler system, some Sprott systems and the Lorenz system are considered.

  5. Identification of linear continuous-time system using wavelet modulating filters

    Institute of Scientific and Technical Information of China (English)

    贺尚红; 钟掘

    2004-01-01

    An approach to identification of linear continuous-time system is studied with modulating functions. Based on wavelet analysis theory, the multi-resolution modulating functions are designed, and the corresponding filters have been analyzed. Using linear modulating filters, we can obtain an identification model that is parameterized directly in continuous-time model parameters. By applying the results from discrete-time model identification to the obtained identification model, a continuous-time estimation method is developed. Considering the accuracy of parameter estimates, an instrumental variable(V) method is proposed, and the design of modulating integral filter is discussed. The relationship between the accuracy of identification and the parameter of modulating filter is investigated, and some points about designing Gaussian wavelet modulating function are outlined. Finally, a simulation study is also included to verify the theoretical results.

  6. Monotone measures of ergodicity for Markov chains

    Directory of Open Access Journals (Sweden)

    J. Keilson

    1998-01-01

    Full Text Available The following paper, first written in 1974, was never published other than as part of an internal research series. Its lack of publication is unrelated to the merits of the paper and the paper is of current importance by virtue of its relation to the relaxation time. A systematic discussion is provided of the approach of a finite Markov chain to ergodicity by proving the monotonicity of an important set of norms, each measures of egodicity, whether or not time reversibility is present. The paper is of particular interest because the discussion of the relaxation time of a finite Markov chain [2] has only been clean for time reversible chains, a small subset of the chains of interest. This restriction is not present here. Indeed, a new relaxation time quoted quantifies the relaxation time for all finite ergodic chains (cf. the discussion of Q1(t below Equation (1.7]. This relaxation time was developed by Keilson with A. Roy in his thesis [6], yet to be published.

  7. Markov chain Monte Carlo based analysis of post-translationally modified VDAC gating kinetics.

    Science.gov (United States)

    Tewari, Shivendra G; Zhou, Yifan; Otto, Bradley J; Dash, Ranjan K; Kwok, Wai-Meng; Beard, Daniel A

    2014-01-01

    The voltage-dependent anion channel (VDAC) is the main conduit for permeation of solutes (including nucleotides and metabolites) of up to 5 kDa across the mitochondrial outer membrane (MOM). Recent studies suggest that VDAC activity is regulated via post-translational modifications (PTMs). Yet the nature and effect of these modifications is not understood. Herein, single channel currents of wild-type, nitrosated, and phosphorylated VDAC are analyzed using a generalized continuous-time Markov chain Monte Carlo (MCMC) method. This developed method describes three distinct conducting states (open, half-open, and closed) of VDAC activity. Lipid bilayer experiments are also performed to record single VDAC activity under un-phosphorylated and phosphorylated conditions, and are analyzed using the developed stochastic search method. Experimental data show significant alteration in VDAC gating kinetics and conductance as a result of PTMs. The effect of PTMs on VDAC kinetics is captured in the parameters associated with the identified Markov model. Stationary distributions of the Markov model suggest that nitrosation of VDAC not only decreased its conductance but also significantly locked VDAC in a closed state. On the other hand, stationary distributions of the model associated with un-phosphorylated and phosphorylated VDAC suggest a reversal in channel conformation from relatively closed state to an open state. Model analyses of the nitrosated data suggest that faster reaction of nitric oxide with Cys-127 thiol group might be responsible for the biphasic effect of nitric oxide on basal VDAC conductance.

  8. Dependence of the crossing time on the sequence length in the continuous-time mutation-selection model

    Energy Technology Data Exchange (ETDEWEB)

    Gill, Wonpyong [Pusan National University, Busan (Korea, Republic of)

    2010-08-15

    The dependence of the crossing time on the sequence length in the coupled and the decoupled continuous-time mutation-selection models in an asymmetric sharply-peaked landscape with a positive asymmetric parameter, r, was examined for a fixed extension parameter, E, which is defined as the average Hamming distance from the optimal allele of the initial quasispecies divided by the sequence length. Two versions of the coupled mutation-selection model, the continuous-time version and discrete-time version, were found to have the same boundary between the deterministic and the stochastic regions, which is different from the boundary between the deterministic and the stochastic regions in the decoupled continuous-time mutation-selection model. The maximum sequence length for a finite population that can evolve through the fitness barrier, e.g., within 10{sup 6} generations in the decoupled continuous-time mutation-selection model, increased by approximately eight sequence elements with increasing population size by a factor of a thousand when E = 0.1 and r = 0.1. The crossing time for a finite population in the decoupled model in the stochastic region was shorter than the crossing time for a finite population in the coupled model, and the maximum evolvable sequence length for a finite population in the decoupled model was longer than the maximum evolvable sequence length for a finite population in the coupled model. This suggests that a mutation allowed at any time during the life cycle might be more effective than a mutation allowed only at reproduction events when a finite population transits to a higher fitness peak through the fitness barrier in an asymmetric sharply-peaked landscape.

  9. Toward Continuous GPS Carrier-Phase Time Transfer: Eliminating the Time Discontinuity at an Anomaly.

    Science.gov (United States)

    Yao, Jian; Levine, Judah; Weiss, Marc

    2015-01-01

    The wide application of Global Positioning System (GPS) carrier-phase (CP) time transfer is limited by the problem of boundary discontinuity (BD). The discontinuity has two categories. One is "day boundary discontinuity," which has been studied extensively and can be solved by multiple methods [1-8]. The other category of discontinuity, called "anomaly boundary discontinuity (anomaly-BD)," comes from a GPS data anomaly. The anomaly can be a data gap (i.e., missing data), a GPS measurement error (i.e., bad data), or a cycle slip. Initial study of the anomaly-BD shows that we can fix the discontinuity if the anomaly lasts no more than 20 min, using the polynomial curve-fitting strategy to repair the anomaly [9]. However, sometimes, the data anomaly lasts longer than 20 min. Thus, a better curve-fitting strategy is in need. Besides, a cycle slip, as another type of data anomaly, can occur and lead to an anomaly-BD. To solve these problems, this paper proposes a new strategy, i.e., the satellite-clock-aided curve fitting strategy with the function of cycle slip detection. Basically, this new strategy applies the satellite clock correction to the GPS data. After that, we do the polynomial curve fitting for the code and phase data, as before. Our study shows that the phase-data residual is only ~3 mm for all GPS satellites. The new strategy also detects and finds the number of cycle slips by searching the minimum curve-fitting residual. Extensive examples show that this new strategy enables us to repair up to a 40-min GPS data anomaly, regardless of whether the anomaly is due to a data gap, a cycle slip, or a combination of the two. We also find that interference of the GPS signal, known as "jamming", can possibly lead to a time-transfer error, and that this new strategy can compensate for jamming outages. Thus, the new strategy can eliminate the impact of jamming on time transfer. As a whole, we greatly improve the robustness of the GPS CP time transfer.

  10. Markov Models for Handwriting Recognition

    CERN Document Server

    Plotz, Thomas

    2011-01-01

    Since their first inception, automatic reading systems have evolved substantially, yet the recognition of handwriting remains an open research problem due to its substantial variation in appearance. With the introduction of Markovian models to the field, a promising modeling and recognition paradigm was established for automatic handwriting recognition. However, no standard procedures for building Markov model-based recognizers have yet been established. This text provides a comprehensive overview of the application of Markov models in the field of handwriting recognition, covering both hidden

  11. Markov Random Field Surface Reconstruction

    DEFF Research Database (Denmark)

    Paulsen, Rasmus Reinhold; Bærentzen, Jakob Andreas; Larsen, Rasmus

    2010-01-01

    ) and knowledge about data (the observation model) in an orthogonal fashion. Local models that account for both scene-specific knowledge and physical properties of the scanning device are described. Furthermore, how the optimal distance field can be computed is demonstrated using conjugate gradients, sparse......A method for implicit surface reconstruction is proposed. The novelty in this paper is the adaption of Markov Random Field regularization of a distance field. The Markov Random Field formulation allows us to integrate both knowledge about the type of surface we wish to reconstruct (the prior...

  12. Hidden Semi Markov Models for Multiple Observation Sequences: The mhsmm Package for R

    DEFF Research Database (Denmark)

    O'Connell, Jarad Michael; Højsgaard, Søren

    2011-01-01

    This paper describes the R package mhsmm which implements estimation and prediction methods for hidden Markov and semi-Markov models for multiple observation sequences. Such techniques are of interest when observed data is thought to be dependent on some unobserved (or hidden) state. Hidden Markov...... models only allow a geometrically distributed sojourn time in a given state, while hidden semi-Markov models extend this by allowing an arbitrary sojourn distribution. We demonstrate the software with simulation examples and an application involving the modelling of the ovarian cycle of dairy cows...

  13. Continuous Time Models of Interest Rate: Testing the Mexican Data (1998-2006)

    OpenAIRE

    Jose Luis de la Cruz; Elizabeth Ortega.

    2007-01-01

    Distinct parametric models in continuous time for the interest rates are tested by means of a comparative analysis of the implied parametric and nonparametric densities. In this research the statistic developed by Ait-Sahalia (1996a) has been applied to the Mexican CETES (28 days) interest rate in the period 1998-2006. With this technique, the discrete approximation to the continuous model is unnecessary even when the data are discrete. The results allow to affirm that the models of interest ...

  14. Improved H_∞ filtering for Markov jumping linear systems with non-accessible mode information

    Institute of Scientific and Technical Information of China (English)

    GUO YaFeng; LI ShaoYuan

    2009-01-01

    This paper is concerned with the H_∞ filtering problems for both continuous-and discrete-time Markov jumping linear systems (MJLS) with non-accessible mode Information.A new design method is proposed,which greatly reduces the overdesign Introduced in the derivation process.The desired filters can be obtained from the solution of convex optimization problems in terms of linear matrix inequalities (LMIs),which can be solved via efficient interior-point algorithms.Numerical examples are provided to Illustrate the advantages of the proposed approach.

  15. A Markov chain technique for determining the acquisition behavior of a digital tracking loop

    Science.gov (United States)

    Chadwick, H. D.

    1972-01-01

    An iterative procedure is presented for determining the acquisition behavior of discrete or digital implementations of a tracking loop. The technique is based on the theory of Markov chains and provides the cumulative probability of acquisition in the loop as a function of time in the presence of noise and a given set of initial condition probabilities. A digital second-order tracking loop to be used in the Viking command receiver for continuous tracking of the command subcarrier phase was analyzed using this technique, and the results agree closely with experimental data.

  16. On the impact of information delay on location-based relaying: a markov modeling approach

    DEFF Research Database (Denmark)

    Nielsen, Jimmy Jessen; Olsen, Rasmus Løvenstein; Madsen, Tatiana Kozlova;

    2012-01-01

    For centralized selection of communication relays, the necessary decision information needs to be collected from the mobile nodes by the access point (centralized decision point). In mobile scenarios, the required information collection and forwarding delays will affect the reliability...... of the collected information and hence will influence the performance of the relay selection method. This paper analyzes this influence in the decision process for the example of a mobile location-based relay selection approach using a continuous time Markov chain model. The model is used to obtain optimal relay...

  17. 长春和新乡雨衰时间序列的马尔科夫链模拟%Rain attenuation time series simulation by Markov chain about Changchun and Xinxiang areas

    Institute of Scientific and Technical Information of China (English)

    李磊; 杨瑞科; 赵振维

    2012-01-01

    Based on the N-State Markov chain model established by Markov theory,the rain attenuation time series about Changchun and Xinxiang areas are simulated and the complementary cumulative distributions obtained by the simulatived and measured data at these areas are compared.The probability distribution about 50 simulative rain attenuation time series are compiled in statistics.The results agree with the results predicted by ITU-R model at 12.5 GHz and 92°Eorbital position of the geostationary satellite.Hence,the usability of this model is validated at partial areas in China.This research establishes a basis for the development of the rain fade mitigation techniques in communication and radar systems at Ku and above Ku band.%基于马尔科夫理论建立的N阶马尔科夫链模型,模拟了长春和新乡地区的降雨衰减时间序列,比较了长春和新乡地区单个模拟和实测雨衰时间序列的概率分布;分别统计了长春和新乡地区50组模拟雨衰时间序列的百分概率分布,并与国际电信联盟无线电通信研究组(ITU-R)提供的卫星轨道位置为92°E、频率为12.5GHz在线极化情况下长春和新乡雨区不同降雨衰减值下的时间百分概率进行了比较,一致性很好,从而验证了N阶马尔可夫链模型在中国部分地区的可用性。模拟结果对我国在Ku及以上频段通信卫星的抗衰落技术的发展具有重要的应用价值。

  18. Nonlinear continuous-time generalized predictive control of solar power plant

    Directory of Open Access Journals (Sweden)

    Khoukhi Billal

    2015-01-01

    Full Text Available This paper presents an application of nonlinear continuous-time generalized predictive control (GPC to the distributed collector field of a solar power plant. The major characteristic of a solar power plant is that the primary energy source, solar radiation, cannot be manipulated. Solar radiation varies throughout the day, causing changes in plant dynamics and strong perturbations in the process. A brief description of the solar power plant and its simulator is given. After that, basic concepts of predictive control and continuous-time generalized predictive control are introduced. A new control strategy, named nonlinear continuous-time generalized predictive control (NCGPC, is then derived to control the process. The simulation results show that the NCGPC gives a greater flexibility to achieve performance goals and better perturbation rejection than classical control.

  19. On a Result for Finite Markov Chains

    Science.gov (United States)

    Kulathinal, Sangita; Ghosh, Lagnojita

    2006-01-01

    In an undergraduate course on stochastic processes, Markov chains are discussed in great detail. Textbooks on stochastic processes provide interesting properties of finite Markov chains. This note discusses one such property regarding the number of steps in which a state is reachable or accessible from another state in a finite Markov chain with M…

  20. Consistency and Refinement for Interval Markov Chains

    DEFF Research Database (Denmark)

    Delahaye, Benoit; Larsen, Kim Guldstrand; Legay, Axel;

    2012-01-01

    Interval Markov Chains (IMC), or Markov Chains with probability intervals in the transition matrix, are the base of a classic specification theory for probabilistic systems [18]. The standard semantics of IMCs assigns to a specification the set of all Markov Chains that satisfy its interval...

  1. Assessment of the accuracy of real-time continuous glucose monitoring system and its correlated factors

    Institute of Scientific and Technical Information of China (English)

    洛佩

    2014-01-01

    Objective To assess the factors that influence the accuracy of real-time continuous glucose monitoring system(RT-CGM).Methods A total of 79 diabetic patients wore RT-CGM for three days continuously while calibrating by interphalangeal glucose values 4-8 times a day.We counted matching rate of interphalangeal glucose values and RT-CGM probe value,and analyzed correlation of the matching rate with MAGE,SDBG,MBG,AUC10,AUC3.9,and NGE by Pearson correlation analysis and multiple linear

  2. Real-time, continuous water-quality monitoring in Indiana and Kentucky

    Science.gov (United States)

    Shoda, Megan E.; Lathrop, Timothy R.; Risch, Martin R.

    2015-01-01

    Water-quality “super” gages (also known as “sentry” gages) provide real-time, continuous measurements of the physical and chemical characteristics of stream water at or near selected U.S. Geological Survey (USGS) streamgages in Indiana and Kentucky. A super gage includes streamflow and water-quality instrumentation and representative stream sample collection for laboratory analysis. USGS scientists can use statistical surrogate models to relate instrument values to analyzed chemical concentrations at a super gage. Real-time, continuous and laboratory-analyzed concentration and load data are publicly accessible on USGS Web pages.

  3. A radial basis function network approach for the computation of inverse continuous time variant functions.

    Science.gov (United States)

    Mayorga, René V; Carrera, Jonathan

    2007-06-01

    This Paper presents an efficient approach for the fast computation of inverse continuous time variant functions with the proper use of Radial Basis Function Networks (RBFNs). The approach is based on implementing RBFNs for computing inverse continuous time variant functions via an overall damped least squares solution that includes a novel null space vector for singularities prevention. The singularities avoidance null space vector is derived from developing a sufficiency condition for singularities prevention that conduces to establish some characterizing matrices and an associated performance index.

  4. Correlations of the Time Dependent Signal and the State of a Continuously Monitored Quantum System

    Science.gov (United States)

    Foroozani, N.; Naghiloo, M.; Tan, D.; Mølmer, K.; Murch, K. W.

    2016-03-01

    In quantum physics, measurements give random results and yield a corresponding random backaction on the state of the system subject to measurement. If a quantum system is probed continuously over time, its state evolves along a stochastic quantum trajectory. To investigate the characteristic properties of such dynamics, we perform weak continuous measurements on a superconducting qubit that is driven to undergo Rabi oscillations. From the data we observe a number of striking temporal correlations within the time dependent signals and the quantum trajectories of the qubit, and we discuss their explanation in terms of quantum measurement and photodetection theory.

  5. Confluence reduction for Markov automata

    NARCIS (Netherlands)

    Timmer, Mark; Pol, van de Jaco; Stoelinga, Mariëlle

    2013-01-01

    Markov automata are a novel formalism for specifying systems exhibiting nondeterminism, probabilistic choices and Markovian rates. Recently, the process algebra MAPA was introduced to efficiently model such systems. As always, the state space explosion threatens the analysability of the models gener

  6. Relative survival multistate Markov model.

    Science.gov (United States)

    Huszti, Ella; Abrahamowicz, Michal; Alioum, Ahmadou; Binquet, Christine; Quantin, Catherine

    2012-02-10

    Prognostic studies often have to deal with two important challenges: (i) separating effects of predictions on different 'competing' events and (ii) uncertainty about cause of death. Multistate Markov models permit multivariable analyses of competing risks of, for example, mortality versus disease recurrence. On the other hand, relative survival methods help estimate disease-specific mortality risks even in the absence of data on causes of death. In this paper, we propose a new Markov relative survival (MRS) model that attempts to combine these two methodologies. Our MRS model extends the existing multistate Markov piecewise constant intensities model to relative survival modeling. The intensity of transitions leading to death in the MRS model is modeled as the sum of an estimable excess hazard of mortality from the disease of interest and an 'offset' defined as the expected hazard of all-cause 'natural' mortality obtained from relevant life-tables. We evaluate the new MRS model through simulations, with a design based on registry-based prognostic studies of colon cancer. Simulation results show almost unbiased estimates of prognostic factor effects for the MRS model. We also applied the new MRS model to reassess the role of prognostic factors for mortality in a study of colorectal cancer. The MRS model considerably reduces the bias observed with the conventional Markov model that does not permit accounting for unknown causes of death, especially if the 'true' effects of a prognostic factor on the two types of mortality differ substantially.

  7. Quadratic Variation by Markov Chains

    DEFF Research Database (Denmark)

    Hansen, Peter Reinhard; Horel, Guillaume

    We introduce a novel estimator of the quadratic variation that is based on the the- ory of Markov chains. The estimator is motivated by some general results concerning filtering contaminated semimartingales. Specifically, we show that filtering can in prin- ciple remove the effects of market...

  8. Dynamic system evolution and markov chain approximation

    Directory of Open Access Journals (Sweden)

    Roderick V. Nicholas Melnik

    1998-01-01

    Full Text Available In this paper computational aspects of the mathematical modelling of dynamic system evolution have been considered as a problem in information theory. The construction of mathematical models is treated as a decision making process with limited available information.The solution of the problem is associated with a computational model based on heuristics of a Markov Chain in a discrete space–time of events. A stable approximation of the chain has been derived and the limiting cases are discussed. An intrinsic interconnection of constructive, sequential, and evolutionary approaches in related optimization problems provides new challenges for future work.

  9. Markov chains with quasitoeplitz transition matrix: applications

    OpenAIRE

    1990-01-01

    Application problems are investigated for the Markov chains with quasitoeplitz transition matrix. Generating functions of transient and steady state probabilities, first zero hitting probabilities and mean times are found for various particular cases, corresponding to some known patterns of feedback ( “warm-up,” “switch at threshold” etc.), Level depending dams and queue-depending queueing systems of both M/G/1 and MI/G/1 types with arbitrary random sizes of arriving and departing groups are ...

  10. When Dread Risks Are More Dreadful than Continuous Risks: Comparing Cumulative Population Losses over Time.

    Directory of Open Access Journals (Sweden)

    Nicolai Bodemer

    Full Text Available People show higher sensitivity to dread risks, rare events that kill many people at once, compared with continuous risks, relatively frequent events that kill many people over a longer period of time. The different reaction to dread risks is often considered a bias: If the continuous risk causes the same number of fatalities, it should not be perceived as less dreadful. We test the hypothesis that a dread risk may have a stronger negative impact on the cumulative population size over time in comparison with a continuous risk causing the same number of fatalities. This difference should be particularly strong when the risky event affects children and young adults who would have produced future offspring if they had survived longer. We conducted a series of simulations, with varying assumptions about population size, population growth, age group affected by risky event, and the underlying demographic model. Results show that dread risks affect the population more severely over time than continuous risks that cause the same number of fatalities, suggesting that fearing a dread risk more than a continuous risk is an ecologically rational strategy.

  11. ANALYSING ACCEPTANCE SAMPLING PLANS BY MARKOV CHAINS

    Directory of Open Access Journals (Sweden)

    Mohammad Mirabi

    2012-01-01

    Full Text Available

    ENGLISH ABSTRACT: In this research, a Markov analysis of acceptance sampling plans in a single stage and in two stages is proposed, based on the quality of the items inspected. In a stage of this policy, if the number of defective items in a sample of inspected items is more than the upper threshold, the batch is rejected. However, the batch is accepted if the number of defective items is less than the lower threshold. Nonetheless, when the number of defective items falls between the upper and lower thresholds, the decision-making process continues to inspect the items and collect further samples. The primary objective is to determine the optimal values of the upper and lower thresholds using a Markov process to minimise the total cost associated with a batch acceptance policy. A solution method is presented, along with a numerical demonstration of the application of the proposed methodology.

    AFRIKAANSE OPSOMMING: In hierdie navorsing word ’n Markov-ontleding gedoen van aannamemonsternemingsplanne wat plaasvind in ’n enkele stap of in twee stappe na gelang van die kwaliteit van die items wat geïnspekteer word. Indien die eerste monster toon dat die aantal defektiewe items ’n boonste grens oorskry, word die lot afgekeur. Indien die eerste monster toon dat die aantal defektiewe items minder is as ’n onderste grens, word die lot aanvaar. Indien die eerste monster toon dat die aantal defektiewe items in die gebied tussen die boonste en onderste grense lê, word die besluitnemingsproses voortgesit en verdere monsters word geneem. Die primêre doel is om die optimale waardes van die booonste en onderste grense te bepaal deur gebruik te maak van ’n Markov-proses sodat die totale koste verbonde aan die proses geminimiseer kan word. ’n Oplossing word daarna voorgehou tesame met ’n numeriese voorbeeld van die toepassing van die voorgestelde oplossing.

  12. Epidemiological characterization of diabetic patients on therapy with continuous subcutaneous insulin infusion and continuous glucose monitoring in real time.

    Science.gov (United States)

    Aristizábal, Natalia; Ramírez, Alex; Hincapié-García, Jaime; Laiton, Estefany; Aristizábal, Carolina; Cuesta, Diana; Monsalve, Claudia; Hincapié, Gloria; Zapata, Eliana; Abad, Verónica; Delgado, Maria-Rocio; Torres, José-Luis; Palacio, Andrés; Botero, José

    2015-11-01

    To describe baseline characteristics of diabetic patients who were started on insulin pump and real time continuous glucose monitor (CSII-rtCGM) in a specialized center in Medellin, Colombia. All patients with diabetes with complete data who were started on CSII-rtCGM between February 2010 and May 2014 were included. This is a descriptive analysis of the sociodemographic and clinical characteristics. 141 of 174 patients attending the clinic were included. 90,1% had type 1diabetes (T1D). The average age of T1D patients at the beginning of therapy was 31,4 years (SD 14,1). 75.8% of patients had normal weight (BMI30). The median duration of T1D was 13 years (P25-P75=10.7-22.0). 14,2% of the patients were admitted at least once in the year preceding the start of CSII-rtCGM because of diabetes related complications. Mean A1c was 8.6%±1.46%. The main reasons for starting CSII-rtCGM were: poor glycemic control (50.2%); frequent hypoglycemia, nocturnal hypoglycemia, hypoglycemia related to exercise, asymptomatic hypoglycemia (30.2%); severe hypoglycemia (16.44%) and dawn phenomena (3.1%). Baseline characteristics of patients included in this study who were started on CSII-rtCGM are similar to those reported in the literature. The Clinic starts CSII-rtCGM mainly in T1D patients with poor glycemic control, frequent or severe hypoglycemia despite being on basal/bolus therapy. Copyright © 2015 SEEN. Published by Elsevier España, S.L.U. All rights reserved.

  13. MARKOV GRAPHS OF ONE–DIMENSIONAL DYNAMICAL SYSTEMS AND THEIR DISCRETE ANALOGUES AND THEIR DISCRETE ANALOGUES

    Directory of Open Access Journals (Sweden)

    SERGIY KOZERENKO

    2016-04-01

    Full Text Available One feature of the famous Sharkovsky’s theorem is that it can be proved using digraphs of a special type (the so–called Markov graphs. The most general definition assigns a Markov graph to every continuous map from the topological graph to itself. We show that this definition is too broad, i.e. every finite digraph can be viewed as a Markov graph of some one–dimensional dynamical system on a tree. We therefore consider discrete analogues of Markov graphs for vertex maps on combinatorial trees and characterize all maps on trees whose discrete Markov graphs are of the following types: complete, complete bipartite, the disjoint union of cycles, with every arc being a loop.

  14. Role of Ito's lemma in sampling pinned diffusion paths in the continuous-time limit

    Science.gov (United States)

    Malsom, P. J.; Pinski, F. J.

    2016-10-01

    We consider pinned diffusion paths that are explored by a particle moving via a conservative force while being in thermal equilibrium with its surroundings. To probe rare transitions, we use the Onsager-Machlup (OM) functional as a path probability distribution function for transition paths that are constrained to start and stop at predesignated points in different energy basins after a fixed time. The OM theory is based on a discrete-time version of Brownian dynamics, and thus it possesses a finite number of time steps. Here we explore the continuous-time limit where the number of time steps, and hence the dimensionality, becomes infinite. In this regime, the OM functional has been commonly regularized by using the Ito-Girsanov change of measure. This regularized form can then be used as a basis of a numerical algorithm to probe transition paths. In doing so, time again is discretized, progressing in fixed increments. When sampling paths, we find that numerical schemes based on this regularized continuous-time limit can fail catastrophically in describing the path of a particle moving in a potential with multiple wells. The origin of this behavior is traced to numerical instabilities in the discrete version of the continuous-time path measure that are not present in the infinite-dimensional limit. These instabilities arise because of the difficulty of satisfying, in finite dimensions, the conditions imposed by Ito's lemma that was an essential ingredient in the derivation of the regularized continuous-time measure. As an important consequence of this analysis, we conclude that the most probable diffusion path is not a physical entity because the thermodynamic action is effectively flat and cannot be minimized.

  15. Quantum cooling and squeezing of a levitating nanosphere via time-continuous measurements

    Science.gov (United States)

    Genoni, Marco G.; Zhang, Jinglei; Millen, James; Barker, Peter F.; Serafini, Alessio

    2015-07-01

    With the purpose of controlling the steady state of a dielectric nanosphere levitated within an optical cavity, we study its conditional dynamics under simultaneous sideband cooling and additional time-continuous measurement of either the output cavity mode or the nanosphere’s position. We find that the average phonon number, purity and quantum squeezing of the steady-states can all be made more non-classical through the addition of time-continuous measurement. We predict that the continuous monitoring of the system, together with Markovian feedback, allows one to stabilize the dynamics for any value of the laser frequency driving the cavity. By considering state of the art values of the experimental parameters, we prove that one can in principle obtain a non-classical (squeezed) steady-state with an average phonon number {n}{ph}≈ 0.5.

  16. Quantum states with continuous spectrum for a general time-dependent oscillator

    Indian Academy of Sciences (India)

    Jeong-Ryeol Choi

    2005-08-01

    We investigated quantum states with continuous spectrum for a general time-dependent oscillator using invariant operator and unitary transformation methods together. The form of the transformed invariant operator by a unitary operator is the same as the Hamiltonian of the simple harmonic oscillator: $\\hat{I'} = \\hat{p}^{2}/2 + ^{2}\\hat{q}^{2}/2$. The fact that 2 of the transformed invariant operator is constant enabled us to investigate the system separately for three cases, where 2 > 0, 2 < 0, and 2 = 0. The eigenstates of the system are discrete for 2 > 0. On the other hand, for 2 ≤ 0, the eigenstates are continuous. The time-dependent oscillators whose spectra of the wave function are continuous are not oscillatory. The wave function for 2 < 0 is expressed in terms of the parabolic cylinder function. We applied our theory to the driven harmonic oscillator with strongly pulsating mass.

  17. Continuous-Time Discrete-Distribution Theory for Activity-Driven Networks

    Science.gov (United States)

    Zino, Lorenzo; Rizzo, Alessandro; Porfiri, Maurizio

    2016-11-01

    Activity-driven networks are a powerful paradigm to study epidemic spreading over time-varying networks. Despite significant advances, most of the current understanding relies on discrete-time computer simulations, in which each node is assigned an activity potential from a continuous distribution. Here, we establish a continuous-time discrete-distribution framework toward an analytical treatment of the epidemic spreading, from its onset to the endemic equilibrium. In the thermodynamic limit, we derive a nonlinear dynamical system to accurately model the epidemic spreading and leverage techniques from the fields of differential inclusions and adaptive estimation to inform short- and long-term predictions. We demonstrate our framework through the analysis of two real-world case studies, exemplifying different physical phenomena and time scales.

  18. A flexible parametric approach for estimating continuous-time inverse probability of treatment and censoring weights.

    Science.gov (United States)

    Saarela, Olli; Liu, Zhihui Amy

    2016-10-15

    Marginal structural Cox models are used for quantifying marginal treatment effects on outcome event hazard function. Such models are estimated using inverse probability of treatment and censoring (IPTC) weighting, which properly accounts for the impact of time-dependent confounders, avoiding conditioning on factors on the causal pathway. To estimate the IPTC weights, the treatment assignment mechanism is conventionally modeled in discrete time. While this is natural in situations where treatment information is recorded at scheduled follow-up visits, in other contexts, the events specifying the treatment history can be modeled in continuous time using the tools of event history analysis. This is particularly the case for treatment procedures, such as surgeries. In this paper, we propose a novel approach for flexible parametric estimation of continuous-time IPTC weights and illustrate it in assessing the relationship between metastasectomy and mortality in metastatic renal cell carcinoma patients. Copyright © 2016 John Wiley & Sons, Ltd.

  19. Consensus of Continuous-Time Multiagent Systems with General Linear Dynamics and Nonuniform Sampling

    Directory of Open Access Journals (Sweden)

    Yanping Gao

    2013-01-01

    Full Text Available This paper studies the consensus problem of multiple agents with general linear continuous-time dynamics. It is assumed that the information transmission among agents is intermittent; namely, each agent can only obtain the information of other agents at some discrete times, where the discrete time intervals may not be equal. Some sufficient conditions for consensus in the cases of state feedback and static output feedback are established, and it is shown that if the controller gain and the upper bound of discrete time intervals satisfy certain linear matrix inequality, then consensus can be reached. Simulations are performed to validate the theoretical results.

  20. Continuous-Discrete Time Prediction-Error Identification Relevant for Linear Model Predictive Control

    DEFF Research Database (Denmark)

    Jørgensen, John Bagterp; Jørgensen, Sten Bay

    2007-01-01

    model is realized from a continuous-discrete-time linear stochastic system specified using transfer functions with time-delays. It is argued that the prediction-error criterion should be selected such that it is compatible with the objective function of the predictive controller in which the model......A Prediction-error-method tailored for model based predictive control is presented. The prediction-error method studied are based on predictions using the Kalman filter and Kalman predictors for a linear discrete-time stochastic state space model. The linear discrete-time stochastic state space...

  1. Inventory Model (Q, R) With Period of Grace, Quadratic Backorder Cost and Continuous Lead Time

    OpenAIRE

    Dr. Martin Osawaru Omorodion

    2015-01-01

    The paper considers the simple economic order model where the period of grace is operating, the lead time is continuous and the backorder cost is quadratic. The lead time follows a gamma distribution. The expected backorder cost per cycle is derived and averaged over all the states of the lead time L. Next we obtain the expected on hand inventory. The Lead time is taken as a Normal variate. The expected backorder costs are, derived after which the expected on hand inventory is derived. The...

  2. A continuous-time Bayesian network reliability modeling and analysis framework

    NARCIS (Netherlands)

    Boudali, H.; Dugan, J.B.

    2006-01-01

    We present a continuous-time Bayesian network (CTBN) framework for dynamic systems reliability modeling and analysis. Dynamic systems exhibit complex behaviors and interactions between their components; where not only the combination of failure events matters, but so does the sequence ordering of th

  3. Boiteux's solution to the shifting-peak problem and the equilibrium price density in continuous time

    NARCIS (Netherlands)

    Horsley, A.; Wrobel, A.J.

    2002-01-01

    Bewley's condition on production sets, imposed to ensure the existence of an equilibrium price density when L∞ is the commodity space, is weakened to allow applications to continuous-time problems, and especially to peak-load pricing when the users' utility and production functions are Mackey contin

  4. Exploring Continuity of Care in Patients with Alcohol Use Disorders Using Time-Variant Measures

    NARCIS (Netherlands)

    S.C. de Vries (Sjoerd); A.I. Wierdsma (André)

    2009-01-01

    textabstractBackground/Aims: We used time-variant measures of continuity of care to study fluctuations in long-term treatment use by patients with alcohol-related disorders. Methods: Data on service use were extracted from the Psychiatric Case Register for the Rotterdam Region, The Netherlands. Cont

  5. Continuous-Time Models, Realized Volatilities, and Testable Distributional Implications for Daily Stock Returns

    DEFF Research Database (Denmark)

    Andersen, Torben G.; Bollerslev, Tim; Frederiksen, Per Houmann

    We provide an empirical framework for assessing the distributional properties of daily specu- lative returns within the context of the continuous-time modeling paradigm traditionally used in asset pricing finance. Our approach builds directly on recently developed realized variation measures and ...

  6. Continuous-time Identification of Exponential-Affine Term Structure Models

    NARCIS (Netherlands)

    Arianto Wibowo, A.W.

    2006-01-01

    This thesis addresses the problem of parameter estimation of the exponentialaffine class of models, which is a class of multi-factor models for the short rate. We propose a continuous-time maximum likelihood estimation method to estimate the parameters of a short rate model, given set of

  7. Critical flicker frequency and continuous reaction times for the diagnosis of minimal hepatic encephalopathy

    DEFF Research Database (Denmark)

    Lauridsen, Mette Enok Munk; Jepsen, Peter; Vilstrup, Hendrik

    2011-01-01

    -to-perform reproducible bedside methods: the critical flicker frequency (CFF) and continuous reaction times (CRT) tests. A CFF ... appropriately to a sensory stimulus. The choice of test depends on the information needed in the clinical and scientific care and study of the patients....

  8. Variational space-time (dis)continuous Galerkin method for linear free surface waves

    NARCIS (Netherlands)

    Ambati, V.R.; Vegt, van der J.J.W.; Bokhove, O.

    2008-01-01

    A new variational (dis)continuous Galerkin finite element method is presented for the linear free surface gravity water wave equations. We formulate the space-time finite element discretization based on a variational formulation analogous to Luke's variational principle. The linear algebraic system

  9. Computation of non-monotonic Lyapunov functions for continuous-time systems

    Science.gov (United States)

    Li, Huijuan; Liu, AnPing

    2017-09-01

    In this paper, we propose two methods to compute non-monotonic Lyapunov functions for continuous-time systems which are asymptotically stable. The first method is to solve a linear optimization problem on a compact and bounded set. The proposed linear programming based algorithm delivers a CPA1

  10. Space-time continuous models of swarm robotic systems supporting global-to-local programming

    CERN Document Server

    Hamann, Heiko; Nakamura, Yoshihiko

    2010-01-01

    Space-Time Continuous Models of Swarm Robotic Systems presents a control-algorithm-based model that predicts the behavior of large self-organizing robot groups, or robot swarms. Readers will find an extensive look into the interdisciplinary research field of swarm robotics.

  11. A continuous-time Bayesian network reliability modeling and analysis framework

    NARCIS (Netherlands)

    Boudali, H.; Dugan, J.B.

    2006-01-01

    We present a continuous-time Bayesian network (CTBN) framework for dynamic systems reliability modeling and analysis. Dynamic systems exhibit complex behaviors and interactions between their components; where not only the combination of failure events matters, but so does the sequence ordering of th

  12. Accuracy evaluation of a new real-time continuous glucose monitoring algorithm in hypoglycemia

    DEFF Research Database (Denmark)

    Mahmoudi, Zeinab; Jensen, Morten Hasselstrøm; Johansen, Mette Dencker;

    2014-01-01

    UNLABELLED: Abstract Background: The purpose of this study was to evaluate the performance of a new continuous glucose monitoring (CGM) calibration algorithm and to compare it with the Guardian(®) REAL-Time (RT) (Medtronic Diabetes, Northridge, CA) calibration algorithm in hypoglycemia. SUBJECTS...

  13. Spectrogram analysis of selected tremor signals using short-time Fourier transform and continuous wavelet transform

    Energy Technology Data Exchange (ETDEWEB)

    Bartosch, T. [Erlanger-Nuernberg Univ., Erlanger (Germany). Lehrstul fuer Nachrichtentechnik I; Seidl, D. [Seismologisches Zentralobservatorium Graefenberg, Erlanegen (Greece). Bundesanstalt fuer Geiwissenschaften und Rohstoffe

    1999-06-01

    Among a variety of spectrogram methods short-time Fourier transform (STFT) and continuous wavelet transform (CWT) were selected to analyse transients in non-stationary signals. Depending on the properties of the tremor signals from the volcanos Mt. Stromboli, Mt. Semeru and Mt. Pinatubo were analyzed using both methods. The CWT can also be used to extend the definition of coherency into a time-varying coherency spectrogram. An example is given using array data from the volcano Mt. Stromboli (Italy).

  14. Real-time, continuous-wave terahertz imaging by a pyroelectric camera

    Institute of Scientific and Technical Information of China (English)

    Jun Yang; Shuangchen Ruan; Min Zhang

    2008-01-01

    Real-time, continuous-wave terahertz (THz) imaging is demonstrated. A 1.89-THz optically-pumped farinfrared laser is used as the illumination source, and a 124 × 124 element room-temperature pyroelectric camera is adopted as the detector. With this setup, THz images through various wrapping materials are shown. The results show that this imaging system has the potential applications in real-time mail and security inspection.

  15. A ’Smart’ Molecular Sieve Oxygen Concentrator with Continuous Cycle Time Adjustment.

    Science.gov (United States)

    1996-04-01

    A ’smart’ molecular sieve oxygen concentrator (MSOC) is controlled by a set of computer algorithms . The ’smart’ system automatically adjusts...determine if concentrator performance could be controlled by computer algorithms which continuously adjust concentrator cycle time. A two-bed... Computer algorithms or decision process were developed which allowed the software to control concentrator cycle time. Step changes in product flow from 5

  16. Variational space-time (dis)continuous Galerkin method for nonlinear free surface waves

    OpenAIRE

    Gagarina, E; Vegt, van der, N.F.A.; Ambati, V.R.; Bokhove, O.

    2013-01-01

    A new variational finite element method is developed for nonlinear free surface gravity water waves. This method also handles waves generated by a wave maker. Its formulation stems from Miles' variational principle for water waves together with a space-time finite element discretization that is continuous in space and discontinuous in time. The key features of this formulation are: (i) a discrete variational approach that gives rise to conservation of discrete energy and phase space and prese...

  17. Detectability of Granger causality for subsampled continuous-time neurophysiological processes.

    Science.gov (United States)

    Barnett, Lionel; Seth, Anil K

    2017-01-01

    Granger causality is well established within the neurosciences for inference of directed functional connectivity from neurophysiological data. These data usually consist of time series which subsample a continuous-time biophysiological process. While it is well known that subsampling can lead to imputation of spurious causal connections where none exist, less is known about the effects of subsampling on the ability to reliably detect causal connections which do exist. We present a theoretical analysis of the effects of subsampling on Granger-causal inference. Neurophysiological processes typically feature signal propagation delays on multiple time scales; accordingly, we base our analysis on a distributed-lag, continuous-time stochastic model, and consider Granger causality in continuous time at finite prediction horizons. Via exact analytical solutions, we identify relationships among sampling frequency, underlying causal time scales and detectability of causalities. We reveal complex interactions between the time scale(s) of neural signal propagation and sampling frequency. We demonstrate that detectability decays exponentially as the sample time interval increases beyond causal delay times, identify detectability "black spots" and "sweet spots", and show that downsampling may potentially improve detectability. We also demonstrate that the invariance of Granger causality under causal, invertible filtering fails at finite prediction horizons, with particular implications for inference of Granger causality from fMRI data. Our analysis emphasises that sampling rates for causal analysis of neurophysiological time series should be informed by domain-specific time scales, and that state-space modelling should be preferred to purely autoregressive modelling. On the basis of a very general model that captures the structure of neurophysiological processes, we are able to help identify confounds, and offer practical insights, for successful detection of causal connectivity

  18. The effect of large decoherence on mixing time in continuous-time quantum walks on long-range interacting cycles

    Energy Technology Data Exchange (ETDEWEB)

    Salimi, S; Radgohar, R, E-mail: shsalimi@uok.ac.i, E-mail: r.radgohar@uok.ac.i [Faculty of Science, Department of Physics, University of Kurdistan, Pasdaran Ave, Sanandaj (Iran, Islamic Republic of)

    2010-01-28

    In this paper, we consider decoherence in continuous-time quantum walks on long-range interacting cycles (LRICs), which are the extensions of the cycle graphs. For this purpose, we use Gurvitz's model and assume that every node is monitored by the corresponding point-contact induced by the decoherence process. Then, we focus on large rates of decoherence and calculate the probability distribution analytically and obtain the lower and upper bounds of the mixing time. Our results prove that the mixing time is proportional to the rate of decoherence and the inverse of the square of the distance parameter (m). This shows that the mixing time decreases with increasing range of interaction. Also, what we obtain for m = 0 is in agreement with Fedichkin, Solenov and Tamon's results [48] for cycle, and we see that the mixing time of CTQWs on cycle improves with adding interacting edges.

  19. Irregular-Time Bayesian Networks

    CERN Document Server

    Ramati, Michael

    2012-01-01

    In many fields observations are performed irregularly along time, due to either measurement limitations or lack of a constant immanent rate. While discrete-time Markov models (as Dynamic Bayesian Networks) introduce either inefficient computation or an information loss to reasoning about such processes, continuous-time Markov models assume either a discrete state space (as Continuous-Time Bayesian Networks), or a flat continuous state space (as stochastic dif- ferential equations). To address these problems, we present a new modeling class called Irregular-Time Bayesian Networks (ITBNs), generalizing Dynamic Bayesian Networks, allowing substantially more compact representations, and increasing the expressivity of the temporal dynamics. In addition, a globally optimal solution is guaranteed when learning temporal systems, provided that they are fully observed at the same irregularly spaced time-points, and a semiparametric subclass of ITBNs is introduced to allow further adaptation to the irregular nature of t...

  20. Time reversal of continuous-wave, monochromatic signals in elastic media

    Energy Technology Data Exchange (ETDEWEB)

    Anderson, Brian E [Los Alamos National Laboratory; Guyer, Robert A [Los Alamos National Laboratory; Ulrich, Timothy J [Los Alamos National Laboratory; Johnson, Paul A [Los Alamos National Laboratory

    2009-01-01

    Experimental observations of spatial focusing of continuous-wave, steady-state elastic waves in a reverberant elastic cavity using time reversal are reported here. Spatially localized focusing is achieved when multiple channels are employed, while a single channel does not yield such focusing. The amplitude of the energy at the focal location increases as the square of the number of channels used, while the amplitude elsewhere in the medium increases proportionally with the number of channels used. The observation is important in the context of imaging in solid laboratory samples as well as problems involving continuous-wave signals in Earth.