Continuous-time Markov decision processes theory and applications
Guo, Xianping
2009-01-01
This volume provides the first book entirely devoted to recent developments on the theory and applications of continuous-time Markov decision processes (MDPs). The MDPs presented here include most of the cases that arise in applications.
Model checking conditional CSL for continuous-time Markov chains
DEFF Research Database (Denmark)
Gao, Yang; Xu, Ming; Zhan, Naijun
2013-01-01
In this paper, we consider the model-checking problem of continuous-time Markov chains (CTMCs) with respect to conditional logic. To the end, we extend Continuous Stochastic Logic introduced in Aziz et al. (2000) [1] to Conditional Continuous Stochastic Logic (CCSL) by introducing a conditional...
The deviation matrix of a continuous-time Markov chain
Coolen-Schrijner, P.; van Doorn, E.A.
2001-01-01
The deviation matrix of an ergodic, continuous-time Markov chain with transition probability matrix $P(.)$ and ergodic matrix $\\Pi$ is the matrix $D \\equiv \\int_0^{\\infty} (P(t)-\\Pi)dt$. We give conditions for $D$ to exist and discuss properties and a representation of $D$. The deviation matrix of a
The deviation matrix of a continuous-time Markov chain
Coolen-Schrijner, Pauline; van Doorn, Erik A.
2002-01-01
he deviation matrix of an ergodic, continuous-time Markov chain with transition probability matrix $P(.)$ and ergodic matrix $\\Pi$ is the matrix $D \\equiv \\int_0^{\\infty} (P(t)-\\Pi)dt$. We give conditions for $D$ to exist and discuss properties and a representation of $D$. The deviation matrix of a
Parallel algorithms for simulating continuous time Markov chains
Nicol, David M.; Heidelberger, Philip
1992-01-01
We have previously shown that the mathematical technique of uniformization can serve as the basis of synchronization for the parallel simulation of continuous-time Markov chains. This paper reviews the basic method and compares five different methods based on uniformization, evaluating their strengths and weaknesses as a function of problem characteristics. The methods vary in their use of optimism, logical aggregation, communication management, and adaptivity. Performance evaluation is conducted on the Intel Touchstone Delta multiprocessor, using up to 256 processors.
Nonequilibrium thermodynamic potentials for continuous-time Markov chains.
Verley, Gatien
2016-01-01
We connect the rare fluctuations of an equilibrium (EQ) process and the typical fluctuations of a nonequilibrium (NE) stationary process. In the framework of large deviation theory, this observation allows us to introduce NE thermodynamic potentials. For continuous-time Markov chains, we identify the relevant pairs of conjugated variables and propose two NE ensembles: one with fixed dynamics and fluctuating time-averaged variables, and another with fixed time-averaged variables, but a fluctuating dynamics. Accordingly, we show that NE processes are equivalent to conditioned EQ processes ensuring that NE potentials are Legendre dual. We find a variational principle satisfied by the NE potentials that reach their maximum in the NE stationary state and whose first derivatives produce the NE equations of state and second derivatives produce the NE Maxwell relations generalizing the Onsager reciprocity relations.
STATISTICAL ANALYSIS OF NOTATIONAL AFL DATA USING CONTINUOUS TIME MARKOV CHAINS
Directory of Open Access Journals (Sweden)
Denny Meyer
2006-12-01
Full Text Available Animal biologists commonly use continuous time Markov chain models to describe patterns of animal behaviour. In this paper we consider the use of these models for describing AFL football. In particular we test the assumptions for continuous time Markov chain models (CTMCs, with time, distance and speed values associated with each transition. Using a simple event categorisation it is found that a semi-Markov chain model is appropriate for this data. This validates the use of Markov Chains for future studies in which the outcomes of AFL matches are simulated
Computing continuous-time Markov chains as transformers of unbounded observables
DEFF Research Database (Denmark)
Danos, Vincent; Heindel, Tobias; Garnier, Ilias
2017-01-01
The paper studies continuous-time Markov chains (CTMCs) as transformers of real-valued functions on their state space, considered as generalised predicates and called observables. Markov chains are assumed to take values in a countable state space S; observables f: S → ℝ may be unbounded...
Physical time scale in kinetic Monte Carlo simulations of continuous-time Markov chains.
Serebrinsky, Santiago A
2011-03-01
We rigorously establish a physical time scale for a general class of kinetic Monte Carlo algorithms for the simulation of continuous-time Markov chains. This class of algorithms encompasses rejection-free (or BKL) and rejection (or "standard") algorithms. For rejection algorithms, it was formerly considered that the availability of a physical time scale (instead of Monte Carlo steps) was empirical, at best. Use of Monte Carlo steps as a time unit now becomes completely unnecessary.
An Expectation Maximization Algorithm to Model Failure Times by Continuous-Time Markov Chains
Directory of Open Access Journals (Sweden)
Qihong Duan
2010-01-01
Full Text Available In many applications, the failure rate function may present a bathtub shape curve. In this paper, an expectation maximization algorithm is proposed to construct a suitable continuous-time Markov chain which models the failure time data by the first time reaching the absorbing state. Assume that a system is described by methods of supplementary variables, the device of stage, and so on. Given a data set, the maximum likelihood estimators of the initial distribution and the infinitesimal transition rates of the Markov chain can be obtained by our novel algorithm. Suppose that there are m transient states in the system and that there are n failure time data. The devised algorithm only needs to compute the exponential of m×m upper triangular matrices for O(nm2 times in each iteration. Finally, the algorithm is applied to two real data sets, which indicates the practicality and efficiency of our algorithm.
van Rosmalen, Joost; Toy, Mehlika; O'Mahony, James F
2013-08-01
Markov models are a simple and powerful tool for analyzing the health and economic effects of health care interventions. These models are usually evaluated in discrete time using cohort analysis. The use of discrete time assumes that changes in health states occur only at the end of a cycle period. Discrete-time Markov models only approximate the process of disease progression, as clinical events typically occur in continuous time. The approximation can yield biased cost-effectiveness estimates for Markov models with long cycle periods and if no half-cycle correction is made. The purpose of this article is to present an overview of methods for evaluating Markov models in continuous time. These methods use mathematical results from stochastic process theory and control theory. The methods are illustrated using an applied example on the cost-effectiveness of antiviral therapy for chronic hepatitis B. The main result is a mathematical solution for the expected time spent in each state in a continuous-time Markov model. It is shown how this solution can account for age-dependent transition rates and discounting of costs and health effects, and how the concept of tunnel states can be used to account for transition rates that depend on the time spent in a state. The applied example shows that the continuous-time model yields more accurate results than the discrete-time model but does not require much computation time and is easily implemented. In conclusion, continuous-time Markov models are a feasible alternative to cohort analysis and can offer several theoretical and practical advantages.
Directory of Open Access Journals (Sweden)
Mokaedi V. Lekgari
2014-01-01
Full Text Available We investigate random-time state-dependent Foster-Lyapunov analysis on subgeometric rate ergodicity of continuous-time Markov chains (CTMCs. We are mainly concerned with making use of the available results on deterministic state-dependent drift conditions for CTMCs and on random-time state-dependent drift conditions for discrete-time Markov chains and transferring them to CTMCs.
Generalization bounds of ERM-based learning processes for continuous-time Markov chains.
Zhang, Chao; Tao, Dacheng
2012-12-01
Many existing results on statistical learning theory are based on the assumption that samples are independently and identically distributed (i.i.d.). However, the assumption of i.i.d. samples is not suitable for practical application to problems in which samples are time dependent. In this paper, we are mainly concerned with the empirical risk minimization (ERM) based learning process for time-dependent samples drawn from a continuous-time Markov chain. This learning process covers many kinds of practical applications, e.g., the prediction for a time series and the estimation of channel state information. Thus, it is significant to study its theoretical properties including the generalization bound, the asymptotic convergence, and the rate of convergence. It is noteworthy that, since samples are time dependent in this learning process, the concerns of this paper cannot (at least straightforwardly) be addressed by existing methods developed under the sample i.i.d. assumption. We first develop a deviation inequality for a sequence of time-dependent samples drawn from a continuous-time Markov chain and present a symmetrization inequality for such a sequence. By using the resultant deviation inequality and symmetrization inequality, we then obtain the generalization bounds of the ERM-based learning process for time-dependent samples drawn from a continuous-time Markov chain. Finally, based on the resultant generalization bounds, we analyze the asymptotic convergence and the rate of convergence of the learning process.
Summary statistics for end-point conditioned continuous-time Markov chains
DEFF Research Database (Denmark)
Hobolth, Asger; Jensen, Jens Ledet
Continuous-time Markov chains are a widely used modelling tool. Applications include DNA sequence evolution, ion channel gating behavior and mathematical finance. We consider the problem of calculating properties of summary statistics (e.g. mean time spent in a state, mean number of jumps between...... two states and the distribution of the total number of jumps) for discretely observed continuous time Markov chains. Three alternative methods for calculating properties of summary statistics are described and the pros and cons of the methods are discussed. The methods are based on (i) an eigenvalue...... decomposition of the rate matrix, (ii) the uniformization method, and (iii) integrals of matrix exponentials. In particular we develop a framework that allows for analyses of rather general summary statistics using the uniformization method....
Fitting timeseries by continuous-time Markov chains: A quadratic programming approach
International Nuclear Information System (INIS)
Crommelin, D.T.; Vanden-Eijnden, E.
2006-01-01
Construction of stochastic models that describe the effective dynamics of observables of interest is an useful instrument in various fields of application, such as physics, climate science, and finance. We present a new technique for the construction of such models. From the timeseries of an observable, we construct a discrete-in-time Markov chain and calculate the eigenspectrum of its transition probability (or stochastic) matrix. As a next step we aim to find the generator of a continuous-time Markov chain whose eigenspectrum resembles the observed eigenspectrum as closely as possible, using an appropriate norm. The generator is found by solving a minimization problem: the norm is chosen such that the object function is quadratic and convex, so that the minimization problem can be solved using quadratic programming techniques. The technique is illustrated on various toy problems as well as on datasets stemming from simulations of molecular dynamics and of atmospheric flows
Impulsive Control for Continuous-Time Markov Decision Processes: A Linear Programming Approach
Energy Technology Data Exchange (ETDEWEB)
Dufour, F., E-mail: dufour@math.u-bordeaux1.fr [Bordeaux INP, IMB, UMR CNRS 5251 (France); Piunovskiy, A. B., E-mail: piunov@liv.ac.uk [University of Liverpool, Department of Mathematical Sciences (United Kingdom)
2016-08-15
In this paper, we investigate an optimization problem for continuous-time Markov decision processes with both impulsive and continuous controls. We consider the so-called constrained problem where the objective of the controller is to minimize a total expected discounted optimality criterion associated with a cost rate function while keeping other performance criteria of the same form, but associated with different cost rate functions, below some given bounds. Our model allows multiple impulses at the same time moment. The main objective of this work is to study the associated linear program defined on a space of measures including the occupation measures of the controlled process and to provide sufficient conditions to ensure the existence of an optimal control.
DEFF Research Database (Denmark)
Tataru, Paula Cristina; Hobolth, Asger
2011-01-01
past evolutionary events (exact times and types of changes) are unaccessible and the past must be inferred from DNA sequence data observed in the present. RESULTS: We describe and implement three algorithms for computing linear combinations of expected values of the sufficient statistics, conditioned......BACKGROUND: Continuous time Markov chains (CTMCs) is a widely used model for describing the evolution of DNA sequences on the nucleotide, amino acid or codon level. The sufficient statistics for CTMCs are the time spent in a state and the number of changes between any two states. In applications...... of the algorithms is available at www.birc.au.dk/~paula/. CONCLUSIONS: We use two different models to analyze the accuracy and eight experiments to investigate the speed of the three algorithms. We find that they have similar accuracy and that EXPM is the slowest method. Furthermore we find that UNI is usually...
A joint logistic regression and covariate-adjusted continuous-time Markov chain model.
Rubin, Maria Laura; Chan, Wenyaw; Yamal, Jose-Miguel; Robertson, Claudia Sue
2017-12-10
The use of longitudinal measurements to predict a categorical outcome is an increasingly common goal in research studies. Joint models are commonly used to describe two or more models simultaneously by considering the correlated nature of their outcomes and the random error present in the longitudinal measurements. However, there is limited research on joint models with longitudinal predictors and categorical cross-sectional outcomes. Perhaps the most challenging task is how to model the longitudinal predictor process such that it represents the true biological mechanism that dictates the association with the categorical response. We propose a joint logistic regression and Markov chain model to describe a binary cross-sectional response, where the unobserved transition rates of a two-state continuous-time Markov chain are included as covariates. We use the method of maximum likelihood to estimate the parameters of our model. In a simulation study, coverage probabilities of about 95%, standard deviations close to standard errors, and low biases for the parameter values show that our estimation method is adequate. We apply the proposed joint model to a dataset of patients with traumatic brain injury to describe and predict a 6-month outcome based on physiological data collected post-injury and admission characteristics. Our analysis indicates that the information provided by physiological changes over time may help improve prediction of long-term functional status of these severely ill subjects. Copyright © 2017 John Wiley & Sons, Ltd. Copyright © 2017 John Wiley & Sons, Ltd.
Cao, Qi; Buskens, Erik; Feenstra, Talitha; Jaarsma, Tiny; Hillege, Hans; Postmus, Douwe
2016-01-01
Continuous-time state transition models may end up having large unwieldy structures when trying to represent all relevant stages of clinical disease processes by means of a standard Markov model. In such situations, a more parsimonious, and therefore easier-to-grasp, model of a patient's disease progression can often be obtained by assuming that the future state transitions do not depend only on the present state (Markov assumption) but also on the past through time since entry in the present state. Despite that these so-called semi-Markov models are still relatively straightforward to specify and implement, they are not yet routinely applied in health economic evaluation to assess the cost-effectiveness of alternative interventions. To facilitate a better understanding of this type of model among applied health economic analysts, the first part of this article provides a detailed discussion of what the semi-Markov model entails and how such models can be specified in an intuitive way by adopting an approach called vertical modeling. In the second part of the article, we use this approach to construct a semi-Markov model for assessing the long-term cost-effectiveness of 3 disease management programs for heart failure. Compared with a standard Markov model with the same disease states, our proposed semi-Markov model fitted the observed data much better. When subsequently extrapolating beyond the clinical trial period, these relatively large differences in goodness-of-fit translated into almost a doubling in mean total cost and a 60-d decrease in mean survival time when using the Markov model instead of the semi-Markov model. For the disease process considered in our case study, the semi-Markov model thus provided a sensible balance between model parsimoniousness and computational complexity. © The Author(s) 2015.
Fitting and interpreting continuous-time latent Markov models for panel data.
Lange, Jane M; Minin, Vladimir N
2013-11-20
Multistate models characterize disease processes within an individual. Clinical studies often observe the disease status of individuals at discrete time points, making exact times of transitions between disease states unknown. Such panel data pose considerable modeling challenges. Assuming the disease process progresses accordingly, a standard continuous-time Markov chain (CTMC) yields tractable likelihoods, but the assumption of exponential sojourn time distributions is typically unrealistic. More flexible semi-Markov models permit generic sojourn distributions yet yield intractable likelihoods for panel data in the presence of reversible transitions. One attractive alternative is to assume that the disease process is characterized by an underlying latent CTMC, with multiple latent states mapping to each disease state. These models retain analytic tractability due to the CTMC framework but allow for flexible, duration-dependent disease state sojourn distributions. We have developed a robust and efficient expectation-maximization algorithm in this context. Our complete data state space consists of the observed data and the underlying latent trajectory, yielding computationally efficient expectation and maximization steps. Our algorithm outperforms alternative methods measured in terms of time to convergence and robustness. We also examine the frequentist performance of latent CTMC point and interval estimates of disease process functionals based on simulated data. The performance of estimates depends on time, functional, and data-generating scenario. Finally, we illustrate the interpretive power of latent CTMC models for describing disease processes on a dataset of lung transplant patients. We hope our work will encourage wider use of these models in the biomedical setting. Copyright © 2013 John Wiley & Sons, Ltd.
A toolbox for safety instrumented system evaluation based on improved continuous-time Markov chain
Wardana, Awang N. I.; Kurniady, Rahman; Pambudi, Galih; Purnama, Jaka; Suryopratomo, Kutut
2017-08-01
Safety instrumented system (SIS) is designed to restore a plant into a safe condition when pre-hazardous event is occur. It has a vital role especially in process industries. A SIS shall be meet with safety requirement specifications. To confirm it, SIS shall be evaluated. Typically, the evaluation is calculated by hand. This paper presents a toolbox for SIS evaluation. It is developed based on improved continuous-time Markov chain. The toolbox supports to detailed approach of evaluation. This paper also illustrates an industrial application of the toolbox to evaluate arch burner safety system of primary reformer. The results of the case study demonstrates that the toolbox can be used to evaluate industrial SIS in detail and to plan the maintenance strategy.
Relative entropy and waiting time for continuous-time Markov processes
Chazottes, J.R.; Giardinà, C.; Redig, F.H.J.
2006-01-01
For discrete-time stochastic processes, there is a close connection between return (resp. waiting) times and entropy (resp. relative entropy). Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one needs a reference measure on
Tataru, Paula; Hobolth, Asger
2011-12-05
Continuous time Markov chains (CTMCs) is a widely used model for describing the evolution of DNA sequences on the nucleotide, amino acid or codon level. The sufficient statistics for CTMCs are the time spent in a state and the number of changes between any two states. In applications past evolutionary events (exact times and types of changes) are unaccessible and the past must be inferred from DNA sequence data observed in the present. We describe and implement three algorithms for computing linear combinations of expected values of the sufficient statistics, conditioned on the end-points of the chain, and compare their performance with respect to accuracy and running time. The first algorithm is based on an eigenvalue decomposition of the rate matrix (EVD), the second on uniformization (UNI), and the third on integrals of matrix exponentials (EXPM). The implementation in R of the algorithms is available at http://www.birc.au.dk/~paula/. We use two different models to analyze the accuracy and eight experiments to investigate the speed of the three algorithms. We find that they have similar accuracy and that EXPM is the slowest method. Furthermore we find that UNI is usually faster than EVD.
Directory of Open Access Journals (Sweden)
Tataru Paula
2011-12-01
Full Text Available Abstract Background Continuous time Markov chains (CTMCs is a widely used model for describing the evolution of DNA sequences on the nucleotide, amino acid or codon level. The sufficient statistics for CTMCs are the time spent in a state and the number of changes between any two states. In applications past evolutionary events (exact times and types of changes are unaccessible and the past must be inferred from DNA sequence data observed in the present. Results We describe and implement three algorithms for computing linear combinations of expected values of the sufficient statistics, conditioned on the end-points of the chain, and compare their performance with respect to accuracy and running time. The first algorithm is based on an eigenvalue decomposition of the rate matrix (EVD, the second on uniformization (UNI, and the third on integrals of matrix exponentials (EXPM. The implementation in R of the algorithms is available at http://www.birc.au.dk/~paula/. Conclusions We use two different models to analyze the accuracy and eight experiments to investigate the speed of the three algorithms. We find that they have similar accuracy and that EXPM is the slowest method. Furthermore we find that UNI is usually faster than EVD.
Cao, Qi; Buskens, Erik; Feenstra, Talitha; Jaarsma, Tiny; Hillege, Hans; Postmus, Douwe
Continuous-time state transition models may end up having large unwieldy structures when trying to represent all relevant stages of clinical disease processes by means of a standard Markov model. In such situations, a more parsimonious, and therefore easier-to-grasp, model of a patient's disease
Stylised facts of financial time series and hidden Markov models in continuous time
DEFF Research Database (Denmark)
Nystrup, Peter; Madsen, Henrik; Lindström, Erik
2015-01-01
presents an extension to continuous time where it is possible to increase the number of states with a linear rather than quadratic growth in the number of parameters. The possibility of increasing the number of states leads to a better fit to both the distributional and temporal properties of daily returns....
International Nuclear Information System (INIS)
Younmyoung Lee; Kunjai Lee
1995-01-01
A model using continuous time Markov process for nuclide transport of decay chain of arbitrary length in the fractured rock medium has been developed. Considering the fracture in the rock matrix as a finite number of compartments, the transition probability for nuclide from the transition intensity between and out of the compartments is represented utilizing Chapman-Kolmogorov equation, with which the expectation and the variance of nuclide distribution for the fractured rock medium could be obtained. A comparison between continuous time Markov process model and available analytical solutions for the nuclide transport of three decay chains without rock matrix diffusion has been made showing comparatively good agreement. Fittings with experimental breakthrough curves obtained with nonsorbing materials such as NaLS and uranine in the artificial fractured rock are also made. (author)
Meng, Tianhui; Li, Xiaofan; Zhang, Sha; Zhao, Yubin
2016-09-28
Wireless sensor networks (WSNs) have recently gained popularity for a wide spectrum of applications. Monitoring tasks can be performed in various environments. This may be beneficial in many scenarios, but it certainly exhibits new challenges in terms of security due to increased data transmission over the wireless channel with potentially unknown threats. Among possible security issues are timing attacks, which are not prevented by traditional cryptographic security. Moreover, the limited energy and memory resources prohibit the use of complex security mechanisms in such systems. Therefore, balancing between security and the associated energy consumption becomes a crucial challenge. This paper proposes a secure scheme for WSNs while maintaining the requirement of the security-performance tradeoff. In order to proceed to a quantitative treatment of this problem, a hybrid continuous-time Markov chain (CTMC) and queueing model are put forward, and the tradeoff analysis of the security and performance attributes is carried out. By extending and transforming this model, the mean time to security attributes failure is evaluated. Through tradeoff analysis, we show that our scheme can enhance the security of WSNs, and the optimal rekeying rate of the performance and security tradeoff can be obtained.
International Nuclear Information System (INIS)
Lee, Youn Myoung
1995-02-01
As a newly approaching model, a stochastic model using continuous time Markov process for nuclide decay chain transport of arbitrary length in the fractured porous rock medium has been proposed, by which the need for solving a set of partial differential equations corresponding to various sets of side conditions can be avoided. Once the single planar fracture in the rock matrix is represented by a series of finite number of compartments having region wise constant parameter values in them, the medium is continuous in view of various processes associated with nuclide transport but discrete in medium space and such geologic system is assumed to have Markov property, since the Markov process requires that only the present value of the time dependent random variable be known to determine the future value of random variable, nuclide transport in the medium can then be modeled as a continuous time Markov process. Processes that are involved in nuclide transport are advective transport due to groundwater flow, diffusion into the rock matrix, adsorption onto the wall of the fracture and within the pores in the rock matrix, and radioactive decay chain. The transition probabilities for nuclide from the transition intensities between and out of the compartments are represented utilizing Chapman-Kolmogorov equation, through which the expectation and the variance of nuclide distribution for each compartment or the fractured rock medium can be obtained. Some comparisons between Markov process model developed in this work and available analytical solutions for one-dimensional layered porous medium, fractured medium with rock matrix diffusion, and porous medium considering three member nuclide decay chain without rock matrix diffusion have been made showing comparatively good agreement for all cases. To verify the model developed in this work another comparative study was also made by fitting the experimental data obtained with NaLS and uranine running in the artificial fractured
The Green-Kubo formula for general Markov processes with a continuous time parameter
International Nuclear Information System (INIS)
Yang Fengxia; Liu Yong; Chen Yong
2010-01-01
For general Markov processes, the Green-Kubo formula is shown to be valid under a mild condition. A class of stochastic evolution equations on a separable Hilbert space and three typical infinite systems of locally interacting diffusions on Z d (irreversible in most cases) are shown to satisfy the Green-Kubo formula, and the Einstein relations for these stochastic evolution equations are shown explicitly as a corollary.
Directory of Open Access Journals (Sweden)
Mindaugas Snipas
2015-01-01
Full Text Available The primary goal of this work was to study advantages of numerical methods used for the creation of continuous time Markov chain models (CTMC of voltage gating of gap junction (GJ channels composed of connexin protein. This task was accomplished by describing gating of GJs using the formalism of the stochastic automata networks (SANs, which allowed for very efficient building and storing of infinitesimal generator of the CTMC that allowed to produce matrices of the models containing a distinct block structure. All of that allowed us to develop efficient numerical methods for a steady-state solution of CTMC models. This allowed us to accelerate CPU time, which is necessary to solve CTMC models, ∼20 times.
Pranevicius, Henrikas; Pranevicius, Mindaugas; Pranevicius, Osvaldas; Bukauskas, Feliksas F.
2015-01-01
The primary goal of this work was to study advantages of numerical methods used for the creation of continuous time Markov chain models (CTMC) of voltage gating of gap junction (GJ) channels composed of connexin protein. This task was accomplished by describing gating of GJs using the formalism of the stochastic automata networks (SANs), which allowed for very efficient building and storing of infinitesimal generator of the CTMC that allowed to produce matrices of the models containing a distinct block structure. All of that allowed us to develop efficient numerical methods for a steady-state solution of CTMC models. This allowed us to accelerate CPU time, which is necessary to solve CTMC models, ∼20 times. PMID:25705700
Energy Technology Data Exchange (ETDEWEB)
Chen Yong; Chen Xi; Qian Minping [School of Mathematical Sciences, Peking University, Beijing 100871 (China)
2006-03-17
A general form of the Green-Kubo formula, which describes the fluctuations pertaining to all the steady states whether equilibrium or non-equilibrium, for a system driven by a finite Markov chain with continuous time (briefly, MC) {l_brace}{xi}{sub t}{r_brace}, is shown. The equivalence of different forms of the Green-Kubo formula is exploited. We also look at the differences in terms of the autocorrelation function and the fluctuation spectrum between the equilibrium state and the non-equilibrium steady state. Also, if the MC is in the non-equilibrium steady state, we can always find a complex function {psi}, such that the fluctuation spectrum of {l_brace}{phi}({xi}{sub t}){r_brace} is non-monotonous in [0, + {infinity})
International Nuclear Information System (INIS)
Chen Yong; Chen Xi; Qian Minping
2006-01-01
A general form of the Green-Kubo formula, which describes the fluctuations pertaining to all the steady states whether equilibrium or non-equilibrium, for a system driven by a finite Markov chain with continuous time (briefly, MC) {ξ t }, is shown. The equivalence of different forms of the Green-Kubo formula is exploited. We also look at the differences in terms of the autocorrelation function and the fluctuation spectrum between the equilibrium state and the non-equilibrium steady state. Also, if the MC is in the non-equilibrium steady state, we can always find a complex function ψ, such that the fluctuation spectrum of {φ(ξ t )} is non-monotonous in [0, + ∞)
Hobolth, Asger; Stone, Eric A
2009-09-01
Analyses of serially-sampled data often begin with the assumption that the observations represent discrete samples from a latent continuous-time stochastic process. The continuous-time Markov chain (CTMC) is one such generative model whose popularity extends to a variety of disciplines ranging from computational finance to human genetics and genomics. A common theme among these diverse applications is the need to simulate sample paths of a CTMC conditional on realized data that is discretely observed. Here we present a general solution to this sampling problem when the CTMC is defined on a discrete and finite state space. Specifically, we consider the generation of sample paths, including intermediate states and times of transition, from a CTMC whose beginning and ending states are known across a time interval of length T. We first unify the literature through a discussion of the three predominant approaches: (1) modified rejection sampling, (2) direct sampling, and (3) uniformization. We then give analytical results for the complexity and efficiency of each method in terms of the instantaneous transition rate matrix Q of the CTMC, its beginning and ending states, and the length of sampling time T. In doing so, we show that no method dominates the others across all model specifications, and we give explicit proof of which method prevails for any given Q, T, and endpoints. Finally, we introduce and compare three applications of CTMCs to demonstrate the pitfalls of choosing an inefficient sampler.
Markov chains and mixing times
Levin, David A; Wilmer, Elizabeth L
2009-01-01
This book is an introduction to the modern approach to the theory of Markov chains. The main goal of this approach is to determine the rate of convergence of a Markov chain to the stationary distribution as a function of the size and geometry of the state space. The authors develop the key tools for estimating convergence times, including coupling, strong stationary times, and spectral methods. Whenever possible, probabilistic methods are emphasized. The book includes many examples and provides brief introductions to some central models of statistical mechanics. Also provided are accounts of r
Markov chains and mixing times
Levin, David A
2017-01-01
Markov Chains and Mixing Times is a magical book, managing to be both friendly and deep. It gently introduces probabilistic techniques so that an outsider can follow. At the same time, it is the first book covering the geometric theory of Markov chains and has much that will be new to experts. It is certainly THE book that I will use to teach from. I recommend it to all comers, an amazing achievement. -Persi Diaconis, Mary V. Sunseri Professor of Statistics and Mathematics, Stanford University Mixing times are an active research topic within many fields from statistical physics to the theory of algorithms, as well as having intrinsic interest within mathematical probability and exploiting discrete analogs of important geometry concepts. The first edition became an instant classic, being accessible to advanced undergraduates and yet bringing readers close to current research frontiers. This second edition adds chapters on monotone chains, the exclusion process and hitting time parameters. Having both exercises...
Timed Comparisons of Semi-Markov Processes
DEFF Research Database (Denmark)
Pedersen, Mathias Ruggaard; Larsen, Kim Guldstrand; Bacci, Giorgio
2018-01-01
-Markov processes, and investigate the question of how to compare two semi-Markov processes with respect to their time-dependent behaviour. To this end, we introduce the relation of being “faster than” between processes and study its algorithmic complexity. Through a connection to probabilistic automata we obtain...
Continuity Properties of Distances for Markov Processes
DEFF Research Database (Denmark)
Jaeger, Manfred; Mao, Hua; Larsen, Kim Guldstrand
2014-01-01
In this paper we investigate distance functions on finite state Markov processes that measure the behavioural similarity of non-bisimilar processes. We consider both probabilistic bisimilarity metrics, and trace-based distances derived from standard Lp and Kullback-Leibler distances. Two desirable...
Ma, Junsheng; Chan, Wenyaw; Tsai, Chu-Lin; Xiong, Momiao; Tilley, Barbara C
2015-11-30
Continuous time Markov chain (CTMC) models are often used to study the progression of chronic diseases in medical research but rarely applied to studies of the process of behavioral change. In studies of interventions to modify behaviors, a widely used psychosocial model is based on the transtheoretical model that often has more than three states (representing stages of change) and conceptually permits all possible instantaneous transitions. Very little attention is given to the study of the relationships between a CTMC model and associated covariates under the framework of transtheoretical model. We developed a Bayesian approach to evaluate the covariate effects on a CTMC model through a log-linear regression link. A simulation study of this approach showed that model parameters were accurately and precisely estimated. We analyzed an existing data set on stages of change in dietary intake from the Next Step Trial using the proposed method and the generalized multinomial logit model. We found that the generalized multinomial logit model was not suitable for these data because it ignores the unbalanced data structure and temporal correlation between successive measurements. Our analysis not only confirms that the nutrition intervention was effective but also provides information on how the intervention affected the transitions among the stages of change. We found that, compared with the control group, subjects in the intervention group, on average, spent substantively less time in the precontemplation stage and were more/less likely to move from an unhealthy/healthy state to a healthy/unhealthy state. Copyright © 2015 John Wiley & Sons, Ltd.
Markov Trends in Macroeconomic Time Series
R. Paap (Richard)
1997-01-01
textabstractMany macroeconomic time series are characterised by long periods of positive growth, expansion periods, and short periods of negative growth, recessions. A popular model to describe this phenomenon is the Markov trend, which is a stochastic segmented trend where the slope depends on the
Song, Youngseok; Ishikawa, Hiroshi; Wu, Mengfei; Liu, Yu-Ying; Lucy, Katie A; Lavinsky, Fabio; Liu, Mengling; Wollstein, Gadi; Schuman, Joel S
2018-03-20
Previously, we introduced a state-based 2-dimensional continuous-time hidden Markov model (2D CT HMM) to model the pattern of detected glaucoma changes using structural and functional information simultaneously. The purpose of this study was to evaluate the detected glaucoma change prediction performance of the model in a real clinical setting using a retrospective longitudinal dataset. Longitudinal, retrospective study. One hundred thirty-four eyes from 134 participants diagnosed with glaucoma or as glaucoma suspects (average follow-up, 4.4±1.2 years; average number of visits, 7.1±1.8). A 2D CT HMM model was trained using OCT (Cirrus HD-OCT; Zeiss, Dublin, CA) average circumpapillary retinal nerve fiber layer (cRNFL) thickness and visual field index (VFI) or mean deviation (MD; Humphrey Field Analyzer; Zeiss). The model was trained using a subset of the data (107 of 134 eyes [80%]) including all visits except for the last visit, which was used to test the prediction performance (training set). Additionally, the remaining 27 eyes were used for secondary performance testing as an independent group (validation set). The 2D CT HMM predicts 1 of 4 possible detected state changes based on 1 input state. Prediction accuracy was assessed as the percentage of correct prediction against the patient's actual recorded state. In addition, deviations of the predicted long-term detected change paths from the actual detected change paths were measured. Baseline mean ± standard deviation age was 61.9±11.4 years, VFI was 90.7±17.4, MD was -3.50±6.04 dB, and cRNFL thickness was 74.9±12.2 μm. The accuracy of detected glaucoma change prediction using the training set was comparable with the validation set (57.0% and 68.0%, respectively). Prediction deviation from the actual detected change path showed stability throughout patient follow-up. The 2D CT HMM demonstrated promising prediction performance in detecting glaucoma change performance in a simulated clinical setting
Modeling nonhomogeneous Markov processes via time transformation.
Hubbard, R A; Inoue, L Y T; Fann, J R
2008-09-01
Longitudinal studies are a powerful tool for characterizing the course of chronic disease. These studies are usually carried out with subjects observed at periodic visits giving rise to panel data. Under this observation scheme the exact times of disease state transitions and sequence of disease states visited are unknown and Markov process models are often used to describe disease progression. Most applications of Markov process models rely on the assumption of time homogeneity, that is, that the transition rates are constant over time. This assumption is not satisfied when transition rates depend on time from the process origin. However, limited statistical tools are available for dealing with nonhomogeneity. We propose models in which the time scale of a nonhomogeneous Markov process is transformed to an operational time scale on which the process is homogeneous. We develop a method for jointly estimating the time transformation and the transition intensity matrix for the time transformed homogeneous process. We assess maximum likelihood estimation using the Fisher scoring algorithm via simulation studies and compare performance of our method to homogeneous and piecewise homogeneous models. We apply our methodology to a study of delirium progression in a cohort of stem cell transplantation recipients and show that our method identifies temporal trends in delirium incidence and recovery.
Markov Chain Modelling for Short-Term NDVI Time Series Forecasting
Directory of Open Access Journals (Sweden)
Stepčenko Artūrs
2016-12-01
Full Text Available In this paper, the NDVI time series forecasting model has been developed based on the use of discrete time, continuous state Markov chain of suitable order. The normalised difference vegetation index (NDVI is an indicator that describes the amount of chlorophyll (the green mass and shows the relative density and health of vegetation; therefore, it is an important variable for vegetation forecasting. A Markov chain is a stochastic process that consists of a state space. This stochastic process undergoes transitions from one state to another in the state space with some probabilities. A Markov chain forecast model is flexible in accommodating various forecast assumptions and structures. The present paper discusses the considerations and techniques in building a Markov chain forecast model at each step. Continuous state Markov chain model is analytically described. Finally, the application of the proposed Markov chain model is illustrated with reference to a set of NDVI time series data.
Harmonic spectral components in time sequences of Markov correlated events
Mazzetti, Piero; Carbone, Anna
2017-07-01
The paper concerns the analysis of the conditions allowing time sequences of Markov correlated events give rise to a line power spectrum having a relevant physical interest. It is found that by specializing the Markov matrix in order to represent closed loop sequences of events with arbitrary distribution, generated in a steady physical condition, a large set of line spectra, covering all possible frequency values, is obtained. The amplitude of the spectral lines is given by a matrix equation based on a generalized Markov matrix involving the Fourier transform of the distribution functions representing the time intervals between successive events of the sequence. The paper is a complement of a previous work where a general expression for the continuous power spectrum was given. In that case the Markov matrix was left in a more general form, thus preventing the possibility of finding line spectra of physical interest. The present extension is also suggested by the interest of explaining the emergence of a broad set of waves found in the electro and magneto-encephalograms, whose frequency ranges from 0.5 to about 40Hz, in terms of the effects produced by chains of firing neurons within the complex neural network of the brain. An original model based on synchronized closed loop sequences of firing neurons is proposed, and a few numerical simulations are reported as an application of the above cited equation.
Absolute continuity of the distribution of some Markov geometric series
Institute of Scientific and Technical Information of China (English)
Ai-hua; FAN; Ji-hong; ZHANG
2007-01-01
Let (∈n)≥0 be the Markov chain of two states with respect to the probability measure of the maximal entropy on the subshift space ∑A defined by Fibonacci incident matrix A.We consider the measure μλ of the probability distribution of the random series ∑∞n=0 εnλn (0 ＜λ＜ 1).It is proved that μλ is singular if λ∈ (0,√5-1/2) and that μλ is absolutely continuous for almost all λ∈ (√5-1/2,0.739).
Recursive smoothers for hidden discrete-time Markov chains
Directory of Open Access Journals (Sweden)
Lakhdar Aggoun
2005-01-01
Full Text Available We consider a discrete-time Markov chain observed through another Markov chain. The proposed model extends models discussed by Elliott et al. (1995. We propose improved recursive formulae to update smoothed estimates of processes related to the model. These recursive estimates are used to update the parameter of the model via the expectation maximization (EM algorithm.
Optimal Time-Abstract Schedulers for CTMDPs and Markov Games
Directory of Open Access Journals (Sweden)
Markus Rabe
2010-06-01
Full Text Available We study time-bounded reachability in continuous-time Markov decision processes for time-abstract scheduler classes. Such reachability problems play a paramount role in dependability analysis and the modelling of manufacturing and queueing systems. Consequently, their analysis has been studied intensively, and techniques for the approximation of optimal control are well understood. From a mathematical point of view, however, the question of approximation is secondary compared to the fundamental question whether or not optimal control exists. We demonstrate the existence of optimal schedulers for the time-abstract scheduler classes for all CTMDPs. Our proof is constructive: We show how to compute optimal time-abstract strategies with finite memory. It turns out that these optimal schedulers have an amazingly simple structure---they converge to an easy-to-compute memoryless scheduling policy after a finite number of steps. Finally, we show that our argument can easily be lifted to Markov games: We show that both players have a likewise simple optimal strategy in these more general structures.
Markov chains and semi-Markov models in time-to-event analysis.
Abner, Erin L; Charnigo, Richard J; Kryscio, Richard J
2013-10-25
A variety of statistical methods are available to investigators for analysis of time-to-event data, often referred to as survival analysis. Kaplan-Meier estimation and Cox proportional hazards regression are commonly employed tools but are not appropriate for all studies, particularly in the presence of competing risks and when multiple or recurrent outcomes are of interest. Markov chain models can accommodate censored data, competing risks (informative censoring), multiple outcomes, recurrent outcomes, frailty, and non-constant survival probabilities. Markov chain models, though often overlooked by investigators in time-to-event analysis, have long been used in clinical studies and have widespread application in other fields.
Sampling rare fluctuations of discrete-time Markov chains
Whitelam, Stephen
2018-03-01
We describe a simple method that can be used to sample the rare fluctuations of discrete-time Markov chains. We focus on the case of Markov chains with well-defined steady-state measures, and derive expressions for the large-deviation rate functions (and upper bounds on such functions) for dynamical quantities extensive in the length of the Markov chain. We illustrate the method using a series of simple examples, and use it to study the fluctuations of a lattice-based model of active matter that can undergo motility-induced phase separation.
Deviney, Frank A.; Rice, Karen; Brown, Donald E.
2012-01-01
Natural resource managers require information concerning the frequency, duration, and long-term probability of occurrence of water-quality indicator (WQI) violations of defined thresholds. The timing of these threshold crossings often is hidden from the observer, who is restricted to relatively infrequent observations. Here, a model for the hidden process is linked with a model for the observations, and the parameters describing duration, return period, and long-term probability of occurrence are estimated using Bayesian methods. A simulation experiment is performed to evaluate the approach under scenarios based on the equivalent of a total monitoring period of 5-30 years and an observation frequency of 1-50 observations per year. Given constant threshold crossing rate, accuracy and precision of parameter estimates increased with longer total monitoring period and more-frequent observations. Given fixed monitoring period and observation frequency, accuracy and precision of parameter estimates increased with longer times between threshold crossings. For most cases where the long-term probability of being in violation is greater than 0.10, it was determined that at least 600 observations are needed to achieve precise estimates. An application of the approach is presented using 22 years of quasi-weekly observations of acid-neutralizing capacity from Deep Run, a stream in Shenandoah National Park, Virginia. The time series also was sub-sampled to simulate monthly and semi-monthly sampling protocols. Estimates of the long-term probability of violation were unbiased despite sampling frequency; however, the expected duration and return period were over-estimated using the sub-sampled time series with respect to the full quasi-weekly time series.
Segmenting Continuous Motions with Hidden Semi-markov Models and Gaussian Processes
Directory of Open Access Journals (Sweden)
Tomoaki Nakamura
2017-12-01
Full Text Available Humans divide perceived continuous information into segments to facilitate recognition. For example, humans can segment speech waves into recognizable morphemes. Analogously, continuous motions are segmented into recognizable unit actions. People can divide continuous information into segments without using explicit segment points. This capacity for unsupervised segmentation is also useful for robots, because it enables them to flexibly learn languages, gestures, and actions. In this paper, we propose a Gaussian process-hidden semi-Markov model (GP-HSMM that can divide continuous time series data into segments in an unsupervised manner. Our proposed method consists of a generative model based on the hidden semi-Markov model (HSMM, the emission distributions of which are Gaussian processes (GPs. Continuous time series data is generated by connecting segments generated by the GP. Segmentation can be achieved by using forward filtering-backward sampling to estimate the model's parameters, including the lengths and classes of the segments. In an experiment using the CMU motion capture dataset, we tested GP-HSMM with motion capture data containing simple exercise motions; the results of this experiment showed that the proposed GP-HSMM was comparable with other methods. We also conducted an experiment using karate motion capture data, which is more complex than exercise motion capture data; in this experiment, the segmentation accuracy of GP-HSMM was 0.92, which outperformed other methods.
Mission reliability of semi-Markov systems under generalized operational time requirements
International Nuclear Information System (INIS)
Wu, Xiaoyue; Hillston, Jane
2015-01-01
Mission reliability of a system depends on specific criteria for mission success. To evaluate the mission reliability of some mission systems that do not need to work normally for the whole mission time, two types of mission reliability for such systems are studied. The first type corresponds to the mission requirement that the system must remain operational continuously for a minimum time within the given mission time interval, while the second corresponds to the mission requirement that the total operational time of the system within the mission time window must be greater than a given value. Based on Markov renewal properties, matrix integral equations are derived for semi-Markov systems. Numerical algorithms and a simulation procedure are provided for both types of mission reliability. Two examples are used for illustration purposes. One is a one-unit repairable Markov system, and the other is a cold standby semi-Markov system consisting of two components. By the proposed approaches, the mission reliability of systems with time redundancy can be more precisely estimated to avoid possible unnecessary redundancy of system resources. - Highlights: • Two types of mission reliability under generalized requirements are defined. • Equations for both types of reliability are derived for semi-Markov systems. • Numerical methods are given for solving both types of reliability. • Simulation procedure is given for estimating both types of reliability. • Verification of the numerical methods is given by the results of simulation
Book Review: "Hidden Markov Models for Time Series: An ...
African Journals Online (AJOL)
Hidden Markov Models for Time Series: An Introduction using R. by Walter Zucchini and Iain L. MacDonald. Chapman & Hall (CRC Press), 2009. Full Text: EMAIL FULL TEXT EMAIL FULL TEXT · DOWNLOAD FULL TEXT DOWNLOAD FULL TEXT · http://dx.doi.org/10.4314/saaj.v10i1.61717 · AJOL African Journals Online.
Clustering Multivariate Time Series Using Hidden Markov Models
Directory of Open Access Journals (Sweden)
Shima Ghassempour
2014-03-01
Full Text Available In this paper we describe an algorithm for clustering multivariate time series with variables taking both categorical and continuous values. Time series of this type are frequent in health care, where they represent the health trajectories of individuals. The problem is challenging because categorical variables make it difficult to define a meaningful distance between trajectories. We propose an approach based on Hidden Markov Models (HMMs, where we first map each trajectory into an HMM, then define a suitable distance between HMMs and finally proceed to cluster the HMMs with a method based on a distance matrix. We test our approach on a simulated, but realistic, data set of 1,255 trajectories of individuals of age 45 and over, on a synthetic validation set with known clustering structure, and on a smaller set of 268 trajectories extracted from the longitudinal Health and Retirement Survey. The proposed method can be implemented quite simply using standard packages in R and Matlab and may be a good candidate for solving the difficult problem of clustering multivariate time series with categorical variables using tools that do not require advanced statistic knowledge, and therefore are accessible to a wide range of researchers.
International Nuclear Information System (INIS)
Costa, O. L. V.; Dufour, F.
2011-01-01
This paper deals with the expected discounted continuous control of piecewise deterministic Markov processes (PDMP’s) using a singular perturbation approach for dealing with rapidly oscillating parameters. The state space of the PDMP is written as the product of a finite set and a subset of the Euclidean space ℝ n . The discrete part of the state, called the regime, characterizes the mode of operation of the physical system under consideration, and is supposed to have a fast (associated to a small parameter ε>0) and a slow behavior. By using a similar approach as developed in Yin and Zhang (Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach, Applications of Mathematics, vol. 37, Springer, New York, 1998, Chaps. 1 and 3) the idea in this paper is to reduce the number of regimes by considering an averaged model in which the regimes within the same class are aggregated through the quasi-stationary distribution so that the different states in this class are replaced by a single one. The main goal is to show that the value function of the control problem for the system driven by the perturbed Markov chain converges to the value function of this limit control problem as ε goes to zero. This convergence is obtained by, roughly speaking, showing that the infimum and supremum limits of the value functions satisfy two optimality inequalities as ε goes to zero. This enables us to show the result by invoking a uniqueness argument, without needing any kind of Lipschitz continuity condition.
Adiabatic condition and the quantum hitting time of Markov chains
International Nuclear Information System (INIS)
Krovi, Hari; Ozols, Maris; Roland, Jeremie
2010-01-01
We present an adiabatic quantum algorithm for the abstract problem of searching marked vertices in a graph, or spatial search. Given a random walk (or Markov chain) P on a graph with a set of unknown marked vertices, one can define a related absorbing walk P ' where outgoing transitions from marked vertices are replaced by self-loops. We build a Hamiltonian H(s) from the interpolated Markov chain P(s)=(1-s)P+sP ' and use it in an adiabatic quantum algorithm to drive an initial superposition over all vertices to a superposition over marked vertices. The adiabatic condition implies that, for any reversible Markov chain and any set of marked vertices, the running time of the adiabatic algorithm is given by the square root of the classical hitting time. This algorithm therefore demonstrates a novel connection between the adiabatic condition and the classical notion of hitting time of a random walk. It also significantly extends the scope of previous quantum algorithms for this problem, which could only obtain a full quadratic speedup for state-transitive reversible Markov chains with a unique marked vertex.
Pemodelan Markov Switching Dengan Time-varying Transition Probability
Savitri, Anggita Puri; Warsito, Budi; Rahmawati, Rita
2016-01-01
Exchange rate or currency is an economic variable which reflects country's state of economy. It fluctuates over time because of its ability to switch the condition or regime caused by economic and political factors. The changes in the exchange rate are depreciation and appreciation. Therefore, it could be modeled using Markov Switching with Time-Varying Transition Probability which observe the conditional changes and use information variable. From this model, time-varying transition probabili...
Sun, Wei; Ding, Wei; Yan, Huifang; Duan, Shunli
2018-06-01
Shoe-mounted pedestrian navigation systems based on micro inertial sensors rely on zero velocity updates to correct their positioning errors in time, which effectively makes determining the zero velocity interval play a key role during normal walking. However, as walking gaits are complicated, and vary from person to person, it is difficult to detect walking gaits with a fixed threshold method. This paper proposes a pedestrian gait classification method based on a hidden Markov model. Pedestrian gait data are collected with a micro inertial measurement unit installed at the instep. On the basis of analyzing the characteristics of the pedestrian walk, a single direction angular rate gyro output is used to classify gait features. The angular rate data are modeled into a univariate Gaussian mixture model with three components, and a four-state left–right continuous hidden Markov model (CHMM) is designed to classify the normal walking gait. The model parameters are trained and optimized using the Baum–Welch algorithm and then the sliding window Viterbi algorithm is used to decode the gait. Walking data are collected through eight subjects walking along the same route at three different speeds; the leave-one-subject-out cross validation method is conducted to test the model. Experimental results show that the proposed algorithm can accurately detect different walking gaits of zero velocity interval. The location experiment shows that the precision of CHMM-based pedestrian navigation improved by 40% when compared to the angular rate threshold method.
Continuous strong Markov processes in dimension one a stochastic calculus approach
Assing, Sigurd
1998-01-01
The book presents an in-depth study of arbitrary one-dimensional continuous strong Markov processes using methods of stochastic calculus. Departing from the classical approaches, a unified investigation of regular as well as arbitrary non-regular diffusions is provided. A general construction method for such processes, based on a generalization of the concept of a perfect additive functional, is developed. The intrinsic decomposition of a continuous strong Markov semimartingale is discovered. The book also investigates relations to stochastic differential equations and fundamental examples of irregular diffusions.
Sebastian, Tunny; Jeyaseelan, Visalakshi; Jeyaseelan, Lakshmanan; Anandan, Shalini; George, Sebastian; Bangdiwala, Shrikant I
2018-01-01
Hidden Markov models are stochastic models in which the observations are assumed to follow a mixture distribution, but the parameters of the components are governed by a Markov chain which is unobservable. The issues related to the estimation of Poisson-hidden Markov models in which the observations are coming from mixture of Poisson distributions and the parameters of the component Poisson distributions are governed by an m-state Markov chain with an unknown transition probability matrix are explained here. These methods were applied to the data on Vibrio cholerae counts reported every month for 11-year span at Christian Medical College, Vellore, India. Using Viterbi algorithm, the best estimate of the state sequence was obtained and hence the transition probability matrix. The mean passage time between the states were estimated. The 95% confidence interval for the mean passage time was estimated via Monte Carlo simulation. The three hidden states of the estimated Markov chain are labelled as 'Low', 'Moderate' and 'High' with the mean counts of 1.4, 6.6 and 20.2 and the estimated average duration of stay of 3, 3 and 4 months, respectively. Environmental risk factors were studied using Markov ordinal logistic regression analysis. No significant association was found between disease severity levels and climate components.
Process Algebra and Markov Chains
Brinksma, Hendrik; Hermanns, H.; Brinksma, Hendrik; Hermanns, H.; Katoen, Joost P.
This paper surveys and relates the basic concepts of process algebra and the modelling of continuous time Markov chains. It provides basic introductions to both fields, where we also study the Markov chains from an algebraic perspective, viz. that of Markov chain algebra. We then proceed to study
Process algebra and Markov chains
Brinksma, E.; Hermanns, H.; Brinksma, E.; Hermanns, H.; Katoen, J.P.
2001-01-01
This paper surveys and relates the basic concepts of process algebra and the modelling of continuous time Markov chains. It provides basic introductions to both fields, where we also study the Markov chains from an algebraic perspective, viz. that of Markov chain algebra. We then proceed to study
Solution of the Markov chain for the dead time problem
International Nuclear Information System (INIS)
Degweker, S.B.
1997-01-01
A method for solving the equation for the Markov chain, describing the effect of a non-extendible dead time on the statistics of time correlated pulses, is discussed. The equation, which was derived in an earlier paper, describes a non-linear process and is not amenable to exact solution. The present method consists of representing the probability generating function as a factorial cumulant expansion and neglecting factorial cumulants beyond the second. This results in a closed set of non-linear equations for the factorial moments. Stationary solutions of these equations, which are of interest for calculating the count rate, are obtained iteratively. The method is applied to the variable dead time counter technique for estimation of system parameters in passive neutron assay of Pu and reactor noise analysis. Comparisons of results by this method with Monte Carlo calculations are presented. (author)
The exit-time problem for a Markov jump process
Burch, N.; D'Elia, M.; Lehoucq, R. B.
2014-12-01
The purpose of this paper is to consider the exit-time problem for a finite-range Markov jump process, i.e, the distance the particle can jump is bounded independent of its location. Such jump diffusions are expedient models for anomalous transport exhibiting super-diffusion or nonstandard normal diffusion. We refer to the associated deterministic equation as a volume-constrained nonlocal diffusion equation. The volume constraint is the nonlocal analogue of a boundary condition necessary to demonstrate that the nonlocal diffusion equation is well-posed and is consistent with the jump process. A critical aspect of the analysis is a variational formulation and a recently developed nonlocal vector calculus. This calculus allows us to pose nonlocal backward and forward Kolmogorov equations, the former equation granting the various moments of the exit-time distribution.
Grabski
2014-01-01
Semi-Markov Processes: Applications in System Reliability and Maintenance is a modern view of discrete state space and continuous time semi-Markov processes and their applications in reliability and maintenance. The book explains how to construct semi-Markov models and discusses the different reliability parameters and characteristics that can be obtained from those models. The book is a useful resource for mathematicians, engineering practitioners, and PhD and MSc students who want to understand the basic concepts and results of semi-Markov process theory. Clearly defines the properties and
Long memory of financial time series and hidden Markov models with time-varying parameters
DEFF Research Database (Denmark)
Nystrup, Peter; Madsen, Henrik; Lindström, Erik
Hidden Markov models are often used to capture stylized facts of daily returns and to infer the hidden state of financial markets. Previous studies have found that the estimated models change over time, but the implications of the time-varying behavior for the ability to reproduce the stylized...... facts have not been thoroughly examined. This paper presents an adaptive estimation approach that allows for the parameters of the estimated models to be time-varying. It is shown that a two-state Gaussian hidden Markov model with time-varying parameters is able to reproduce the long memory of squared...... daily returns that was previously believed to be the most difficult fact to reproduce with a hidden Markov model. Capturing the time-varying behavior of the parameters also leads to improved one-step predictions....
Long Memory of Financial Time Series and Hidden Markov Models with Time-Varying Parameters
DEFF Research Database (Denmark)
Nystrup, Peter; Madsen, Henrik; Lindström, Erik
2016-01-01
Hidden Markov models are often used to model daily returns and to infer the hidden state of financial markets. Previous studies have found that the estimated models change over time, but the implications of the time-varying behavior have not been thoroughly examined. This paper presents an adaptive...... to reproduce with a hidden Markov model. Capturing the time-varying behavior of the parameters also leads to improved one-step density forecasts. Finally, it is shown that the forecasting performance of the estimated models can be further improved using local smoothing to forecast the parameter variations....
Rate Reduction for State-labelled Markov Chains with Upper Time-bounded CSL Requirements
Directory of Open Access Journals (Sweden)
Bharath Siva Kumar Tati
2016-07-01
Full Text Available This paper presents algorithms for identifying and reducing a dedicated set of controllable transition rates of a state-labelled continuous-time Markov chain model. The purpose of the reduction is to make states to satisfy a given requirement, specified as a CSL upper time-bounded Until formula. We distinguish two different cases, depending on the type of probability bound. A natural partitioning of the state space allows us to develop possible solutions, leading to simple algorithms for both cases.
Utilization of two web-based continuing education courses evaluated by Markov chain model.
Tian, Hao; Lin, Jin-Mann S; Reeves, William C
2012-01-01
To evaluate the web structure of two web-based continuing education courses, identify problems and assess the effects of web site modifications. Markov chain models were built from 2008 web usage data to evaluate the courses' web structure and navigation patterns. The web site was then modified to resolve identified design issues and the improvement in user activity over the subsequent 12 months was quantitatively evaluated. Web navigation paths were collected between 2008 and 2010. The probability of navigating from one web page to another was analyzed. The continuing education courses' sequential structure design was clearly reflected in the resulting actual web usage models, and none of the skip transitions provided was heavily used. The web navigation patterns of the two different continuing education courses were similar. Two possible design flaws were identified and fixed in only one of the two courses. Over the following 12 months, the drop-out rate in the modified course significantly decreased from 41% to 35%, but remained unchanged in the unmodified course. The web improvement effects were further verified via a second-order Markov chain model. The results imply that differences in web content have less impact than web structure design on how learners navigate through continuing education courses. Evaluation of user navigation can help identify web design flaws and guide modifications. This study showed that Markov chain models provide a valuable tool to evaluate web-based education courses. Both the results and techniques in this study would be very useful for public health education and research specialists.
a Continuous-Time Positive Linear System
Directory of Open Access Journals (Sweden)
Kyungsup Kim
2013-01-01
Full Text Available This paper discusses a computational method to construct positive realizations with sparse matrices for continuous-time positive linear systems with multiple complex poles. To construct a positive realization of a continuous-time system, we use a Markov sequence similar to the impulse response sequence that is used in the discrete-time case. The existence of the proposed positive realization can be analyzed with the concept of a polyhedral convex cone. We provide a constructive algorithm to compute positive realizations with sparse matrices of some positive systems under certain conditions. A sufficient condition for the existence of a positive realization, under which the proposed constructive algorithm works well, is analyzed.
International Nuclear Information System (INIS)
Lemons, Don S.
2012-01-01
We develop a Markov process theory of charged particle scattering from stationary, transverse, magnetic waves. We examine approximations that lead to quasilinear theory, in particular the resonant diffusion approximation. We find that, when appropriate, the resonant diffusion approximation simplifies the result of the weak turbulence approximation without significant further restricting the regime of applicability. We also explore a theory generated by expanding drift and diffusion rates in terms of a presumed small correlation time. This small correlation time expansion leads to results valid for relatively small pitch angle and large wave energy density - a regime that may govern pitch angle scattering of high-energy electrons into the geomagnetic loss cone.
Cai, Chao-Ran; Wu, Zhi-Xi; Guan, Jian-Yue
2014-11-01
Recently, Gómez et al. proposed a microscopic Markov-chain approach (MMCA) [S. Gómez, J. Gómez-Gardeñes, Y. Moreno, and A. Arenas, Phys. Rev. E 84, 036105 (2011)PLEEE81539-375510.1103/PhysRevE.84.036105] to the discrete-time susceptible-infected-susceptible (SIS) epidemic process and found that the epidemic prevalence obtained by this approach agrees well with that by simulations. However, we found that the approach cannot be straightforwardly extended to a susceptible-infected-recovered (SIR) epidemic process (due to its irreversible property), and the epidemic prevalences obtained by MMCA and Monte Carlo simulations do not match well when the infection probability is just slightly above the epidemic threshold. In this contribution we extend the effective degree Markov-chain approach, proposed for analyzing continuous-time epidemic processes [J. Lindquist, J. Ma, P. Driessche, and F. Willeboordse, J. Math. Biol. 62, 143 (2011)JMBLAJ0303-681210.1007/s00285-010-0331-2], to address discrete-time binary-state (SIS) or three-state (SIR) epidemic processes on uncorrelated complex networks. It is shown that the final epidemic size as well as the time series of infected individuals obtained from this approach agree very well with those by Monte Carlo simulations. Our results are robust to the change of different parameters, including the total population size, the infection probability, the recovery probability, the average degree, and the degree distribution of the underlying networks.
Maximum Kolmogorov-Sinai Entropy Versus Minimum Mixing Time in Markov Chains
Mihelich, M.; Dubrulle, B.; Paillard, D.; Kral, Q.; Faranda, D.
2018-01-01
We establish a link between the maximization of Kolmogorov Sinai entropy (KSE) and the minimization of the mixing time for general Markov chains. Since the maximisation of KSE is analytical and easier to compute in general than mixing time, this link provides a new faster method to approximate the minimum mixing time dynamics. It could be interesting in computer sciences and statistical physics, for computations that use random walks on graphs that can be represented as Markov chains.
Hidden Markov Models for Time Series An Introduction Using R
Zucchini, Walter
2009-01-01
Illustrates the flexibility of HMMs as general-purpose models for time series data. This work presents an overview of HMMs for analyzing time series data, from continuous-valued, circular, and multivariate series to binary data, bounded and unbounded counts and categorical observations.
Mapping of uncertainty relations between continuous and discrete time.
Chiuchiù, Davide; Pigolotti, Simone
2018-03-01
Lower bounds on fluctuations of thermodynamic currents depend on the nature of time, discrete or continuous. To understand the physical reason, we compare current fluctuations in discrete-time Markov chains and continuous-time master equations. We prove that current fluctuations in the master equations are always more likely, due to random timings of transitions. This comparison leads to a mapping of the moments of a current between discrete and continuous time. We exploit this mapping to obtain uncertainty bounds. Our results reduce the quests for uncertainty bounds in discrete and continuous time to a single problem.
Weber, Juliane; Zachow, Christopher; Witthaut, Dirk
2018-03-01
Wind power generation exhibits a strong temporal variability, which is crucial for system integration in highly renewable power systems. Different methods exist to simulate wind power generation but they often cannot represent the crucial temporal fluctuations properly. We apply the concept of additive binary Markov chains to model a wind generation time series consisting of two states: periods of high and low wind generation. The only input parameter for this model is the empirical autocorrelation function. The two-state model is readily extended to stochastically reproduce the actual generation per period. To evaluate the additive binary Markov chain method, we introduce a coarse model of the electric power system to derive backup and storage needs. We find that the temporal correlations of wind power generation, the backup need as a function of the storage capacity, and the resting time distribution of high and low wind events for different shares of wind generation can be reconstructed.
Weber, Juliane; Zachow, Christopher; Witthaut, Dirk
2018-03-01
Wind power generation exhibits a strong temporal variability, which is crucial for system integration in highly renewable power systems. Different methods exist to simulate wind power generation but they often cannot represent the crucial temporal fluctuations properly. We apply the concept of additive binary Markov chains to model a wind generation time series consisting of two states: periods of high and low wind generation. The only input parameter for this model is the empirical autocorrelation function. The two-state model is readily extended to stochastically reproduce the actual generation per period. To evaluate the additive binary Markov chain method, we introduce a coarse model of the electric power system to derive backup and storage needs. We find that the temporal correlations of wind power generation, the backup need as a function of the storage capacity, and the resting time distribution of high and low wind events for different shares of wind generation can be reconstructed.
Expectation propagation for continuous time stochastic processes
International Nuclear Information System (INIS)
Cseke, Botond; Schnoerr, David; Sanguinetti, Guido; Opper, Manfred
2016-01-01
We consider the inverse problem of reconstructing the posterior measure over the trajectories of a diffusion process from discrete time observations and continuous time constraints. We cast the problem in a Bayesian framework and derive approximations to the posterior distributions of single time marginals using variational approximate inference, giving rise to an expectation propagation type algorithm. For non-linear diffusion processes, this is achieved by leveraging moment closure approximations. We then show how the approximation can be extended to a wide class of discrete-state Markov jump processes by making use of the chemical Langevin equation. Our empirical results show that the proposed method is computationally efficient and provides good approximations for these classes of inverse problems. (paper)
Masuyama, Hiroyuki
2014-01-01
In this paper we study the augmented truncation of discrete-time block-monotone Markov chains under geometric drift conditions. We first present a bound for the total variation distance between the stationary distributions of an original Markov chain and its augmented truncation. We also obtain such error bounds for more general cases, where an original Markov chain itself is not necessarily block monotone but is blockwise dominated by a block-monotone Markov chain. Finally,...
Applying Markov Chains for NDVI Time Series Forecasting of Latvian Regions
Directory of Open Access Journals (Sweden)
Stepchenko Arthur
2015-12-01
Full Text Available Time series of earth observation based estimates of vegetation inform about variations in vegetation at the scale of Latvia. A vegetation index is an indicator that describes the amount of chlorophyll (the green mass and shows the relative density and health of vegetation. NDVI index is an important variable for vegetation forecasting and management of various problems, such as climate change monitoring, energy usage monitoring, managing the consumption of natural resources, agricultural productivity monitoring, drought monitoring and forest fire detection. In this paper, we make a one-step-ahead prediction of 7-daily time series of NDVI index using Markov chains. The choice of a Markov chain is due to the fact that a Markov chain is a sequence of random variables where each variable is located in some state. And a Markov chain contains probabilities of moving from one state to other.
Multi-category micro-milling tool wear monitoring with continuous hidden Markov models
Zhu, Kunpeng; Wong, Yoke San; Hong, Geok Soon
2009-02-01
In-process monitoring of tool conditions is important in micro-machining due to the high precision requirement and high tool wear rate. Tool condition monitoring in micro-machining poses new challenges compared to conventional machining. In this paper, a multi-category classification approach is proposed for tool flank wear state identification in micro-milling. Continuous Hidden Markov models (HMMs) are adapted for modeling of the tool wear process in micro-milling, and estimation of the tool wear state given the cutting force features. For a noise-robust approach, the HMM outputs are connected via a medium filter to minimize the tool state before entry into the next state due to high noise level. A detailed study on the selection of HMM structures for tool condition monitoring (TCM) is presented. Case studies on the tool state estimation in the micro-milling of pure copper and steel demonstrate the effectiveness and potential of these methods.
Kirkwood, James R
2015-01-01
Review of ProbabilityShort HistoryReview of Basic Probability DefinitionsSome Common Probability DistributionsProperties of a Probability DistributionProperties of the Expected ValueExpected Value of a Random Variable with Common DistributionsGenerating FunctionsMoment Generating FunctionsExercisesDiscrete-Time, Finite-State Markov ChainsIntroductionNotationTransition MatricesDirected Graphs: Examples of Markov ChainsRandom Walk with Reflecting BoundariesGamblerâ€™s RuinEhrenfest ModelCentral Problem of Markov ChainsCondition to Ensure a Unique Equilibrium StateFinding the Equilibrium StateTransient and Recurrent StatesIndicator FunctionsPerron-Frobenius TheoremAbsorbing Markov ChainsMean First Passage TimeMean Recurrence Time and the Equilibrium StateFundamental Matrix for Regular Markov ChainsDividing a Markov Chain into Equivalence ClassesPeriodic Markov ChainsReducible Markov ChainsSummaryExercisesDiscrete-Time, Infinite-State Markov ChainsRenewal ProcessesDelayed Renewal ProcessesEquilibrium State f...
Hidden Markov model tracking of continuous gravitational waves from young supernova remnants
Sun, L.; Melatos, A.; Suvorova, S.; Moran, W.; Evans, R. J.
2018-02-01
Searches for persistent gravitational radiation from nonpulsating neutron stars in young supernova remnants are computationally challenging because of rapid stellar braking. We describe a practical, efficient, semicoherent search based on a hidden Markov model tracking scheme, solved by the Viterbi algorithm, combined with a maximum likelihood matched filter, the F statistic. The scheme is well suited to analyzing data from advanced detectors like the Advanced Laser Interferometer Gravitational Wave Observatory (Advanced LIGO). It can track rapid phase evolution from secular stellar braking and stochastic timing noise torques simultaneously without searching second- and higher-order derivatives of the signal frequency, providing an economical alternative to stack-slide-based semicoherent algorithms. One implementation tracks the signal frequency alone. A second implementation tracks the signal frequency and its first time derivative. It improves the sensitivity by a factor of a few upon the first implementation, but the cost increases by 2 to 3 orders of magnitude.
Extending Markov Automata with State and Action Rewards
Guck, Dennis; Timmer, Mark; Blom, Stefan; Bertrand, N.; Bortolussi, L.
This presentation introduces the Markov Reward Automaton (MRA), an extension of the Markov automaton that allows the modelling of systems incorporating rewards in addition to nondeterminism, discrete probabilistic choice and continuous stochastic timing. Our models support both rewards that are
Markov Chains and Markov Processes
Ogunbayo, Segun
2016-01-01
Markov chain, which was named after Andrew Markov is a mathematical system that transfers a state to another state. Many real world systems contain uncertainty. This study helps us to understand the basic idea of a Markov chain and how is been useful in our daily lives. For some times there had been suspense on distinct predictions and future existences. Also in different games there had been different expectations or results involved. That is the reason why we need Markov chains to predict o...
Basic problems and solution methods for two-dimensional continuous 3 × 3 order hidden Markov model
International Nuclear Information System (INIS)
Wang, Guo-gang; Tang, Gui-jin; Gan, Zong-liang; Cui, Zi-guan; Zhu, Xiu-chang
2016-01-01
A novel model referred to as two-dimensional continuous 3 × 3 order hidden Markov model is put forward to avoid the disadvantages of the classical hypothesis of two-dimensional continuous hidden Markov model. This paper presents three equivalent definitions of the model, in which the state transition probability relies on not only immediate horizontal and vertical states but also immediate diagonal state, and in which the probability density of the observation relies on not only current state but also immediate horizontal and vertical states. The paper focuses on the three basic problems of the model, namely probability density calculation, parameters estimation and path backtracking. Some algorithms solving the questions are theoretically derived, by exploiting the idea that the sequences of states on rows or columns of the model can be viewed as states of a one-dimensional continuous 1 × 2 order hidden Markov model. Simulation results further demonstrate the performance of the algorithms. Because there are more statistical characteristics in the structure of the proposed new model, it can more accurately describe some practical problems, as compared to two-dimensional continuous hidden Markov model.
Markov Chain Model with Catastrophe to Determine Mean Time to Default of Credit Risky Assets
Dharmaraja, Selvamuthu; Pasricha, Puneet; Tardelli, Paola
2017-11-01
This article deals with the problem of probabilistic prediction of the time distance to default for a firm. To model the credit risk, the dynamics of an asset is described as a function of a homogeneous discrete time Markov chain subject to a catastrophe, the default. The behaviour of the Markov chain is investigated and the mean time to the default is expressed in a closed form. The methodology to estimate the parameters is given. Numerical results are provided to illustrate the applicability of the proposed model on real data and their analysis is discussed.
Applying Mean-Field Approximation to Continuous Time Markov Chains
Kolesnichenko, A.V.; Senni, Valerio; Pourranjabar, Alireza; Remke, A.K.I.; Stoelinga, M.I.A.
2014-01-01
The mean-field analysis technique is used to perform analysis of a system with a large number of components to determine the emergent deterministic behaviour and how this behaviour modifies when its parameters are perturbed. The computer science performance modelling and analysis community has found
Mapping absorption processes onto a Markov chain, conserving the mean first passage time
International Nuclear Information System (INIS)
Biswas, Katja
2013-01-01
The dynamics of a multidimensional system is projected onto a discrete state master equation using the transition rates W(k → k′; t, t + dt) between a set of states {k} represented by the regions {ζ k } in phase or discrete state space. Depending on the dynamics Γ i (t) of the original process and the choice of ζ k , the discretized process can be Markovian or non-Markovian. For absorption processes, it is shown that irrespective of these properties of the projection, a master equation with time-independent transition rates W-bar (k→k ' ) can be obtained, which conserves the total occupation time of the partitions of the phase or discrete state space of the original process. An expression for the transition probabilities p-bar (k ' |k) is derived based on either time-discrete measurements {t i } with variable time stepping Δ (i+1)i = t i+1 − t i or the theoretical knowledge at continuous times t. This allows computational methods of absorbing Markov chains to be used to obtain the mean first passage time (MFPT) of the system. To illustrate this approach, the procedure is applied to obtain the MFPT for the overdamped Brownian motion of particles subject to a system with dichotomous noise and the escape from an entropic barrier. The high accuracy of the simulation results confirms with the theory. (paper)
A Bayesian method for construction of Markov models to describe dynamics on various time-scales.
Rains, Emily K; Andersen, Hans C
2010-10-14
The dynamics of many biological processes of interest, such as the folding of a protein, are slow and complicated enough that a single molecular dynamics simulation trajectory of the entire process is difficult to obtain in any reasonable amount of time. Moreover, one such simulation may not be sufficient to develop an understanding of the mechanism of the process, and multiple simulations may be necessary. One approach to circumvent this computational barrier is the use of Markov state models. These models are useful because they can be constructed using data from a large number of shorter simulations instead of a single long simulation. This paper presents a new Bayesian method for the construction of Markov models from simulation data. A Markov model is specified by (τ,P,T), where τ is the mesoscopic time step, P is a partition of configuration space into mesostates, and T is an N(P)×N(P) transition rate matrix for transitions between the mesostates in one mesoscopic time step, where N(P) is the number of mesostates in P. The method presented here is different from previous Bayesian methods in several ways. (1) The method uses Bayesian analysis to determine the partition as well as the transition probabilities. (2) The method allows the construction of a Markov model for any chosen mesoscopic time-scale τ. (3) It constructs Markov models for which the diagonal elements of T are all equal to or greater than 0.5. Such a model will be called a "consistent mesoscopic Markov model" (CMMM). Such models have important advantages for providing an understanding of the dynamics on a mesoscopic time-scale. The Bayesian method uses simulation data to find a posterior probability distribution for (P,T) for any chosen τ. This distribution can be regarded as the Bayesian probability that the kinetics observed in the atomistic simulation data on the mesoscopic time-scale τ was generated by the CMMM specified by (P,T). An optimization algorithm is used to find the most
QRS complex detection based on continuous density hidden Markov models using univariate observations
Sotelo, S.; Arenas, W.; Altuve, M.
2018-04-01
In the electrocardiogram (ECG), the detection of QRS complexes is a fundamental step in the ECG signal processing chain since it allows the determination of other characteristics waves of the ECG and provides information about heart rate variability. In this work, an automatic QRS complex detector based on continuous density hidden Markov models (HMM) is proposed. HMM were trained using univariate observation sequences taken either from QRS complexes or their derivatives. The detection approach is based on the log-likelihood comparison of the observation sequence with a fixed threshold. A sliding window was used to obtain the observation sequence to be evaluated by the model. The threshold was optimized by receiver operating characteristic curves. Sensitivity (Sen), specificity (Spc) and F1 score were used to evaluate the detection performance. The approach was validated using ECG recordings from the MIT-BIH Arrhythmia database. A 6-fold cross-validation shows that the best detection performance was achieved with 2 states HMM trained with QRS complexes sequences (Sen = 0.668, Spc = 0.360 and F1 = 0.309). We concluded that these univariate sequences provide enough information to characterize the QRS complex dynamics from HMM. Future works are directed to the use of multivariate observations to increase the detection performance.
Nonparametric Estimation of Interval Reliability for Discrete-Time Semi-Markov Systems
DEFF Research Database (Denmark)
Georgiadis, Stylianos; Limnios, Nikolaos
2016-01-01
In this article, we consider a repairable discrete-time semi-Markov system with finite state space. The measure of the interval reliability is given as the probability of the system being operational over a given finite-length time interval. A nonparametric estimator is proposed for the interval...
Quasi-stationary distributions for reducible absorbing Markov chains in discrete time
van Doorn, Erik A.; Pollett, P.K.
2009-01-01
We consider discrete-time Markov chains with one coffin state and a finite set $S$ of transient states, and are interested in the limiting behaviour of such a chain as time $n \\to \\infty,$ conditional on survival up to $n$. It is known that, when $S$ is irreducible, the limiting conditional
On the record process of time-reversible spectrally-negative Markov additive processes
J. Ivanovs; M.R.H. Mandjes (Michel)
2009-01-01
htmlabstractWe study the record process of a spectrally-negative Markov additive process (MAP). Assuming time-reversibility, a number of key quantities can be given explicitly. It is shown how these key quantities can be used when analyzing the distribution of the all-time maximum attained by MAPs
Road maintenance optimization through a discrete-time semi-Markov decision process
International Nuclear Information System (INIS)
Zhang Xueqing; Gao Hui
2012-01-01
Optimization models are necessary for efficient and cost-effective maintenance of a road network. In this regard, road deterioration is commonly modeled as a discrete-time Markov process such that an optimal maintenance policy can be obtained based on the Markov decision process, or as a renewal process such that an optimal maintenance policy can be obtained based on the renewal theory. However, the discrete-time Markov process cannot capture the real time at which the state transits while the renewal process considers only one state and one maintenance action. In this paper, road deterioration is modeled as a semi-Markov process in which the state transition has the Markov property and the holding time in each state is assumed to follow a discrete Weibull distribution. Based on this semi-Markov process, linear programming models are formulated for both infinite and finite planning horizons in order to derive optimal maintenance policies to minimize the life-cycle cost of a road network. A hypothetical road network is used to illustrate the application of the proposed optimization models. The results indicate that these linear programming models are practical for the maintenance of a road network having a large number of road segments and that they are convenient to incorporate various constraints on the decision process, for example, performance requirements and available budgets. Although the optimal maintenance policies obtained for the road network are randomized stationary policies, the extent of this randomness in decision making is limited. The maintenance actions are deterministic for most states and the randomness in selecting actions occurs only for a few states.
Performance Modeling of Communication Networks with Markov Chains
Mo, Jeonghoon
2010-01-01
This book is an introduction to Markov chain modeling with applications to communication networks. It begins with a general introduction to performance modeling in Chapter 1 where we introduce different performance models. We then introduce basic ideas of Markov chain modeling: Markov property, discrete time Markov chain (DTMe and continuous time Markov chain (CTMe. We also discuss how to find the steady state distributions from these Markov chains and how they can be used to compute the system performance metric. The solution methodologies include a balance equation technique, limiting probab
Directory of Open Access Journals (Sweden)
Yuan Yuan
2015-11-01
Full Text Available In this paper, we propose a novel method to continuously monitor land cover change using satellite image time series, which can extract comprehensive change information including change time, location, and “from-to” information. This method is based on a hidden Markov model (HMM trained for each land cover class. Assuming a pixel’s initial class has been obtained, likelihoods of the corresponding model are calculated on incoming time series extracted with a temporal sliding window. By observing the likelihood change over the windows, land cover change can be precisely detected from the dramatic drop of likelihood. The established HMMs are then used for identifying the land cover class after the change. As a case study, the proposed method is applied to monitoring urban encroachment onto farmland in Beijing using 10-year MODIS time series from 2001 to 2010. The performance is evaluated on a validation set for different model structures and thresholds. Compared with other change detection methods, the proposed method shows superior change detection accuracy. In addition, it is also more computationally efficient.
Regeneration and general Markov chains
Directory of Open Access Journals (Sweden)
Vladimir V. Kalashnikov
1994-01-01
Full Text Available Ergodicity, continuity, finite approximations and rare visits of general Markov chains are investigated. The obtained results permit further quantitative analysis of characteristics, such as, rates of convergence, continuity (measured as a distance between perturbed and non-perturbed characteristics, deviations between Markov chains, accuracy of approximations and bounds on the distribution function of the first visit time to a chosen subset, etc. The underlying techniques use the embedding of the general Markov chain into a wide sense regenerative process with the help of splitting construction.
Directory of Open Access Journals (Sweden)
Fei Chen
2013-01-01
Full Text Available This paper deals with the finite-time stabilization problem for discrete-time Markov jump nonlinear systems with time delays and norm-bounded exogenous disturbance. The nonlinearities in different jump modes are parameterized by neural networks. Subsequently, a linear difference inclusion state space representation for a class of neural networks is established. Based on this, sufficient conditions are derived in terms of linear matrix inequalities to guarantee stochastic finite-time boundedness and stochastic finite-time stabilization of the closed-loop system. A numerical example is illustrated to verify the efficiency of the proposed technique.
Directory of Open Access Journals (Sweden)
Gerich M. S.
2012-12-01
Full Text Available Let ${xi(t, x(t}$ be a homogeneous semi-continuous lattice Poisson process on the Markov chain.The jumps of one sign are geometrically distributed, and jumps of the opposite sign are arbitrary latticed distribution. For a suchprocesses the relations for the components of two-sided matrix factorization are established.This relations define the moment genereting functions for extremumf of the process and their complements.
Energy Technology Data Exchange (ETDEWEB)
Beyreuther, Moritz; Wassermann, Joachim [Department of Earth and Environmental Sciences (Geophys. Observatory), Ludwig Maximilians Universitaet Muenchen, D-80333 (Germany); Carniel, Roberto [Dipartimento di Georisorse e Territorio Universitat Degli Studi di Udine, I-33100 (Italy)], E-mail: roberto.carniel@uniud.it
2008-10-01
A possible interaction of (volcano-) tectonic earthquakes with the continuous seismic noise recorded in the volcanic island of Tenerife was recently suggested, but existing catalogues seem to be far from being self consistent, calling for the development of automatic detection and classification algorithms. In this work we propose the adoption of a methodology based on Hidden Markov Models (HMMs), widely used already in other fields, such as speech classification.
Markov chains theory and applications
Sericola, Bruno
2013-01-01
Markov chains are a fundamental class of stochastic processes. They are widely used to solve problems in a large number of domains such as operational research, computer science, communication networks and manufacturing systems. The success of Markov chains is mainly due to their simplicity of use, the large number of available theoretical results and the quality of algorithms developed for the numerical evaluation of many metrics of interest.The author presents the theory of both discrete-time and continuous-time homogeneous Markov chains. He carefully examines the explosion phenomenon, the
Distributed synthesis in continuous time
DEFF Research Database (Denmark)
Hermanns, Holger; Krčál, Jan; Vester, Steen
2016-01-01
We introduce a formalism modelling communication of distributed agents strictly in continuous-time. Within this framework, we study the problem of synthesising local strategies for individual agents such that a specified set of goal states is reached, or reached with at least a given probability....... The flow of time is modelled explicitly based on continuous-time randomness, with two natural implications: First, the non-determinism stemming from interleaving disappears. Second, when we restrict to a subclass of non-urgent models, the quantitative value problem for two players can be solved in EXPTIME....... Indeed, the explicit continuous time enables players to communicate their states by delaying synchronisation (which is unrestricted for non-urgent models). In general, the problems are undecidable already for two players in the quantitative case and three players in the qualitative case. The qualitative...
Verification of Open Interactive Markov Chains
Brazdil, Tomas; Hermanns, Holger; Krcal, Jan; Kretinsky, Jan; Rehak, Vojtech
2012-01-01
Interactive Markov chains (IMC) are compositional behavioral models extending both labeled transition systems and continuous-time Markov chains. IMC pair modeling convenience - owed to compositionality properties - with effective verification algorithms and tools - owed to Markov properties. Thus far however, IMC verification did not consider compositionality properties, but considered closed systems. This paper discusses the evaluation of IMC in an open and thus compositional interpretation....
A new Markov-chain-related statistical approach for modelling synthetic wind power time series
International Nuclear Information System (INIS)
Pesch, T; Hake, J F; Schröders, S; Allelein, H J
2015-01-01
The integration of rising shares of volatile wind power in the generation mix is a major challenge for the future energy system. To address the uncertainties involved in wind power generation, models analysing and simulating the stochastic nature of this energy source are becoming increasingly important. One statistical approach that has been frequently used in the literature is the Markov chain approach. Recently, the method was identified as being of limited use for generating wind time series with time steps shorter than 15–40 min as it is not capable of reproducing the autocorrelation characteristics accurately. This paper presents a new Markov-chain-related statistical approach that is capable of solving this problem by introducing a variable second lag. Furthermore, additional features are presented that allow for the further adjustment of the generated synthetic time series. The influences of the model parameter settings are examined by meaningful parameter variations. The suitability of the approach is demonstrated by an application analysis with the example of the wind feed-in in Germany. It shows that—in contrast to conventional Markov chain approaches—the generated synthetic time series do not systematically underestimate the required storage capacity to balance wind power fluctuation. (paper)
Analysis of Streamline Separation at Infinity Using Time-Discrete Markov Chains.
Reich, W; Scheuermann, G
2012-12-01
Existing methods for analyzing separation of streamlines are often restricted to a finite time or a local area. In our paper we introduce a new method that complements them by allowing an infinite-time-evaluation of steady planar vector fields. Our algorithm unifies combinatorial and probabilistic methods and introduces the concept of separation in time-discrete Markov-Chains. We compute particle distributions instead of the streamlines of single particles. We encode the flow into a map and then into a transition matrix for each time direction. Finally, we compare the results of our grid-independent algorithm to the popular Finite-Time-Lyapunov-Exponents and discuss the discrepancies.
Predicting hepatitis B monthly incidence rates using weighted Markov chains and time series methods.
Shahdoust, Maryam; Sadeghifar, Majid; Poorolajal, Jalal; Javanrooh, Niloofar; Amini, Payam
2015-01-01
Hepatitis B (HB) is a major global mortality. Accurately predicting the trend of the disease can provide an appropriate view to make health policy disease prevention. This paper aimed to apply three different to predict monthly incidence rates of HB. This historical cohort study was conducted on the HB incidence data of Hamadan Province, the west of Iran, from 2004 to 2012. Weighted Markov Chain (WMC) method based on Markov chain theory and two time series models including Holt Exponential Smoothing (HES) and SARIMA were applied on the data. The results of different applied methods were compared to correct percentages of predicted incidence rates. The monthly incidence rates were clustered into two clusters as state of Markov chain. The correct predicted percentage of the first and second clusters for WMC, HES and SARIMA methods was (100, 0), (84, 67) and (79, 47) respectively. The overall incidence rate of HBV is estimated to decrease over time. The comparison of results of the three models indicated that in respect to existing seasonality trend and non-stationarity, the HES had the most accurate prediction of the incidence rates.
Li, Yue; Jha, Devesh K; Ray, Asok; Wettergren, Thomas A; Yue Li; Jha, Devesh K; Ray, Asok; Wettergren, Thomas A; Wettergren, Thomas A; Li, Yue; Ray, Asok; Jha, Devesh K
2018-06-01
This paper presents information-theoretic performance analysis of passive sensor networks for detection of moving targets. The proposed method falls largely under the category of data-level information fusion in sensor networks. To this end, a measure of information contribution for sensors is formulated in a symbolic dynamics framework. The network information state is approximately represented as the largest principal component of the time series collected across the network. To quantify each sensor's contribution for generation of the information content, Markov machine models as well as x-Markov (pronounced as cross-Markov) machine models, conditioned on the network information state, are constructed; the difference between the conditional entropies of these machines is then treated as an approximate measure of information contribution by the respective sensors. The x-Markov models represent the conditional temporal statistics given the network information state. The proposed method has been validated on experimental data collected from a local area network of passive sensors for target detection, where the statistical characteristics of environmental disturbances are similar to those of the target signal in the sense of time scale and texture. A distinctive feature of the proposed algorithm is that the network decisions are independent of the behavior and identity of the individual sensors, which is desirable from computational perspectives. Results are presented to demonstrate the proposed method's efficacy to correctly identify the presence of a target with very low false-alarm rates. The performance of the underlying algorithm is compared with that of a recent data-driven, feature-level information fusion algorithm. It is shown that the proposed algorithm outperforms the other algorithm.
Memetic Approaches for Optimizing Hidden Markov Models: A Case Study in Time Series Prediction
Bui, Lam Thu; Barlow, Michael
We propose a methodology for employing memetics (local search) within the framework of evolutionary algorithms to optimize parameters of hidden markov models. With this proposal, the rate and frequency of using local search are automatically changed over time either at a population or individual level. At the population level, we allow the rate of using local search to decay over time to zero (at the final generation). At the individual level, each individual is equipped with information of when it will do local search and for how long. This information evolves over time alongside the main elements of the chromosome representing the individual.
Sinitskiy, Anton V.; Pande, Vijay S.
2018-01-01
Markov state models (MSMs) have been widely used to analyze computer simulations of various biomolecular systems. They can capture conformational transitions much slower than an average or maximal length of a single molecular dynamics (MD) trajectory from the set of trajectories used to build the MSM. A rule of thumb claiming that the slowest implicit time scale captured by an MSM should be comparable by the order of magnitude to the aggregate duration of all MD trajectories used to build this MSM has been known in the field. However, this rule has never been formally proved. In this work, we present analytical results for the slowest time scale in several types of MSMs, supporting the above rule. We conclude that the slowest implicit time scale equals the product of the aggregate sampling and four factors that quantify: (1) how much statistics on the conformational transitions corresponding to the longest implicit time scale is available, (2) how good the sampling of the destination Markov state is, (3) the gain in statistics from using a sliding window for counting transitions between Markov states, and (4) a bias in the estimate of the implicit time scale arising from finite sampling of the conformational transitions. We demonstrate that in many practically important cases all these four factors are on the order of unity, and we analyze possible scenarios that could lead to their significant deviation from unity. Overall, we provide for the first time analytical results on the slowest time scales captured by MSMs. These results can guide further practical applications of MSMs to biomolecular dynamics and allow for higher computational efficiency of simulations.
Masuyama, Hiroyuki
2015-01-01
This paper studies the last-column-block-augmented northwest-corner truncation (LC-block-augmented truncation, for short) of discrete-time block-monotone Markov chains under subgeometric drift conditions. The main result of this paper is to present an upper bound for the total variation distance between the stationary probability vectors of a block-monotone Markov chain and its LC-block-augmented truncation. The main result is extended to Markov chains that themselves may not be block monoton...
Directory of Open Access Journals (Sweden)
Li Qiu
2013-01-01
unified Markov jump model. The random time delays and packet dropouts existed in feedback communication link are modeled by two independent Markov chains; the resulting closed-loop system is described by a new Markovian jump linear system (MJLS with Markov delays. Sufficient conditions of the stochastic stability for NCSs is obtained by constructing a novel Lyapunov functional, and the mode-dependent output feedback controller design method is presented based on linear matrix inequality (LMI technique. A numerical example is given to illustrate the effectiveness of the proposed method.
Markov processes and controlled Markov chains
Filar, Jerzy; Chen, Anyue
2002-01-01
The general theory of stochastic processes and the more specialized theory of Markov processes evolved enormously in the second half of the last century. In parallel, the theory of controlled Markov chains (or Markov decision processes) was being pioneered by control engineers and operations researchers. Researchers in Markov processes and controlled Markov chains have been, for a long time, aware of the synergies between these two subject areas. However, this may be the first volume dedicated to highlighting these synergies and, almost certainly, it is the first volume that emphasizes the contributions of the vibrant and growing Chinese school of probability. The chapters that appear in this book reflect both the maturity and the vitality of modern day Markov processes and controlled Markov chains. They also will provide an opportunity to trace the connections that have emerged between the work done by members of the Chinese school of probability and the work done by the European, US, Central and South Ameri...
Particle Markov Chain Monte Carlo Techniques of Unobserved Component Time Series Models Using Ox
DEFF Research Database (Denmark)
Nonejad, Nima
This paper details Particle Markov chain Monte Carlo techniques for analysis of unobserved component time series models using several economic data sets. PMCMC combines the particle filter with the Metropolis-Hastings algorithm. Overall PMCMC provides a very compelling, computationally fast...... and efficient framework for estimation. These advantages are used to for instance estimate stochastic volatility models with leverage effect or with Student-t distributed errors. We also model changing time series characteristics of the US inflation rate by considering a heteroskedastic ARFIMA model where...
Saddlepoint expansions for sums of Markov dependent variables on a continuous state space
DEFF Research Database (Denmark)
Jensen, J.L.
1991-01-01
Based on the conjugate kernel studied in Iscoe et al. (1985) we derive saddlepoint expansions for either the density or distribution function of a sum f(X1)+...+f(Xn), where the Xi's constitute a Markov chain. The chain is assumed to satisfy a strong recurrence condition which makes the results...... here very similar to the classical results for i.i.d. variables. In particular we establish also conditions under which the expansions hold uniformly over the range of the saddlepoint. Expansions are also derived for sums of the form f(X1, X0)+f(X2, X1)+...+f(Xn, Xn-1) although the uniformity result...
Chemical Continuous Time Random Walks
Aquino, T.; Dentz, M.
2017-12-01
Traditional methods for modeling solute transport through heterogeneous media employ Eulerian schemes to solve for solute concentration. More recently, Lagrangian methods have removed the need for spatial discretization through the use of Monte Carlo implementations of Langevin equations for solute particle motions. While there have been recent advances in modeling chemically reactive transport with recourse to Lagrangian methods, these remain less developed than their Eulerian counterparts, and many open problems such as efficient convergence and reconstruction of the concentration field remain. We explore a different avenue and consider the question: In heterogeneous chemically reactive systems, is it possible to describe the evolution of macroscopic reactant concentrations without explicitly resolving the spatial transport? Traditional Kinetic Monte Carlo methods, such as the Gillespie algorithm, model chemical reactions as random walks in particle number space, without the introduction of spatial coordinates. The inter-reaction times are exponentially distributed under the assumption that the system is well mixed. In real systems, transport limitations lead to incomplete mixing and decreased reaction efficiency. We introduce an arbitrary inter-reaction time distribution, which may account for the impact of incomplete mixing. This process defines an inhomogeneous continuous time random walk in particle number space, from which we derive a generalized chemical Master equation and formulate a generalized Gillespie algorithm. We then determine the modified chemical rate laws for different inter-reaction time distributions. We trace Michaelis-Menten-type kinetics back to finite-mean delay times, and predict time-nonlocal macroscopic reaction kinetics as a consequence of broadly distributed delays. Non-Markovian kinetics exhibit weak ergodicity breaking and show key features of reactions under local non-equilibrium.
Optimal State Estimation for Discrete-Time Markov Jump Systems with Missing Observations
Directory of Open Access Journals (Sweden)
Qing Sun
2014-01-01
Full Text Available This paper is concerned with the optimal linear estimation for a class of direct-time Markov jump systems with missing observations. An observer-based approach of fault detection and isolation (FDI is investigated as a detection mechanic of fault case. For systems with known information, a conditional prediction of observations is applied and fault observations are replaced and isolated; then, an FDI linear minimum mean square error estimation (LMMSE can be developed by comprehensive utilizing of the correct information offered by systems. A recursive equation of filtering based on the geometric arguments can be obtained. Meanwhile, a stability of the state estimator will be guaranteed under appropriate assumption.
International Nuclear Information System (INIS)
Gao Zhong-Ke; Hu Li-Dan; Jin Ning-De
2013-01-01
We generate a directed weighted complex network by a method based on Markov transition probability to represent an experimental two-phase flow. We first systematically carry out gas—liquid two-phase flow experiments for measuring the time series of flow signals. Then we construct directed weighted complex networks from various time series in terms of a network generation method based on Markov transition probability. We find that the generated network inherits the main features of the time series in the network structure. In particular, the networks from time series with different dynamics exhibit distinct topological properties. Finally, we construct two-phase flow directed weighted networks from experimental signals and associate the dynamic behavior of gas-liquid two-phase flow with the topological statistics of the generated networks. The results suggest that the topological statistics of two-phase flow networks allow quantitative characterization of the dynamic flow behavior in the transitions among different gas—liquid flow patterns. (general)
Katoen, Joost P.; Maneesh Khattri, M.; Zapreev, I.S.; Zapreev, I.S.
2005-01-01
This short tool paper introduces MRMC, a model checker for discrete-time and continuous-time Markov reward models. It supports reward extensions of PCTL and CSL, and allows for the automated verification of properties concerning long-run and instantaneous rewards as well as cumulative rewards. In
First and second order Markov chain models for synthetic generation of wind speed time series
International Nuclear Information System (INIS)
Shamshad, A.; Bawadi, M.A.; Wan Hussin, W.M.A.; Majid, T.A.; Sanusi, S.A.M.
2005-01-01
Hourly wind speed time series data of two meteorological stations in Malaysia have been used for stochastic generation of wind speed data using the transition matrix approach of the Markov chain process. The transition probability matrices have been formed using two different approaches: the first approach involves the use of the first order transition probability matrix of a Markov chain, and the second involves the use of a second order transition probability matrix that uses the current and preceding values to describe the next wind speed value. The algorithm to generate the wind speed time series from the transition probability matrices is described. Uniform random number generators have been used for transition between successive time states and within state wind speed values. The ability of each approach to retain the statistical properties of the generated speed is compared with the observed ones. The main statistical properties used for this purpose are mean, standard deviation, median, percentiles, Weibull distribution parameters, autocorrelations and spectral density of wind speed values. The comparison of the observed wind speed and the synthetically generated ones shows that the statistical characteristics are satisfactorily preserved
Travel Cost Inference from Sparse, Spatio-Temporally Correlated Time Series Using Markov Models
DEFF Research Database (Denmark)
Yang, Bin; Guo, Chenjuan; Jensen, Christian S.
2013-01-01
of such time series offers insight into the underlying system and enables prediction of system behavior. While the techniques presented in the paper apply more generally, we consider the case of transportation systems and aim to predict travel cost from GPS tracking data from probe vehicles. Specifically, each...... road segment has an associated travel-cost time series, which is derived from GPS data. We use spatio-temporal hidden Markov models (STHMM) to model correlations among different traffic time series. We provide algorithms that are able to learn the parameters of an STHMM while contending...... with the sparsity, spatio-temporal correlation, and heterogeneity of the time series. Using the resulting STHMM, near future travel costs in the transportation network, e.g., travel time or greenhouse gas emissions, can be inferred, enabling a variety of routing services, e.g., eco-routing. Empirical studies...
Model Checking Markov Reward Models with Impulse Rewards
Cloth, Lucia; Katoen, Joost-Pieter; Khattri, Maneesh; Pulungan, Reza; Bondavalli, Andrea; Haverkort, Boudewijn; Tang, Dong
This paper considers model checking of Markov reward models (MRMs), continuous-time Markov chains with state rewards as well as impulse rewards. The reward extension of the logic CSL (Continuous Stochastic Logic) is interpreted over such MRMs, and two numerical algorithms are provided to check the
A Modularized Efficient Framework for Non-Markov Time Series Estimation
Schamberg, Gabriel; Ba, Demba; Coleman, Todd P.
2018-06-01
We present a compartmentalized approach to finding the maximum a-posteriori (MAP) estimate of a latent time series that obeys a dynamic stochastic model and is observed through noisy measurements. We specifically consider modern signal processing problems with non-Markov signal dynamics (e.g. group sparsity) and/or non-Gaussian measurement models (e.g. point process observation models used in neuroscience). Through the use of auxiliary variables in the MAP estimation problem, we show that a consensus formulation of the alternating direction method of multipliers (ADMM) enables iteratively computing separate estimates based on the likelihood and prior and subsequently "averaging" them in an appropriate sense using a Kalman smoother. As such, this can be applied to a broad class of problem settings and only requires modular adjustments when interchanging various aspects of the statistical model. Under broad log-concavity assumptions, we show that the separate estimation problems are convex optimization problems and that the iterative algorithm converges to the MAP estimate. As such, this framework can capture non-Markov latent time series models and non-Gaussian measurement models. We provide example applications involving (i) group-sparsity priors, within the context of electrophysiologic specrotemporal estimation, and (ii) non-Gaussian measurement models, within the context of dynamic analyses of learning with neural spiking and behavioral observations.
Computing Fault-Containment Times of Self-Stabilizing Algorithms Using Lumped Markov Chains
Directory of Open Access Journals (Sweden)
Volker Turau
2018-05-01
Full Text Available The analysis of self-stabilizing algorithms is often limited to the worst case stabilization time starting from an arbitrary state, i.e., a state resulting from a sequence of faults. Considering the fact that these algorithms are intended to provide fault tolerance in the long run, this is not the most relevant metric. A common situation is that a running system is an a legitimate state when hit by a single fault. This event has a much higher probability than multiple concurrent faults. Therefore, the worst case time to recover from a single fault is more relevant than the recovery time from a large number of faults. This paper presents techniques to derive upper bounds for the mean time to recover from a single fault for self-stabilizing algorithms based on Markov chains in combination with lumping. To illustrate the applicability of the techniques they are applied to a new self-stabilizing coloring algorithm.
A high-fidelity weather time series generator using the Markov Chain process on a piecewise level
Hersvik, K.; Endrerud, O.-E. V.
2017-12-01
A method is developed for generating a set of unique weather time-series based on an existing weather series. The method allows statistically valid weather variations to take place within repeated simulations of offshore operations. The numerous generated time series need to share the same statistical qualities as the original time series. Statistical qualities here refer mainly to the distribution of weather windows available for work, including durations and frequencies of such weather windows, and seasonal characteristics. The method is based on the Markov chain process. The core new development lies in how the Markov Process is used, specifically by joining small pieces of random length time series together rather than joining individual weather states, each from a single time step, which is a common solution found in the literature. This new Markov model shows favorable characteristics with respect to the requirements set forth and all aspects of the validation performed.
Hidden Markov Item Response Theory Models for Responses and Response Times.
Molenaar, Dylan; Oberski, Daniel; Vermunt, Jeroen; De Boeck, Paul
2016-01-01
Current approaches to model responses and response times to psychometric tests solely focus on between-subject differences in speed and ability. Within subjects, speed and ability are assumed to be constants. Violations of this assumption are generally absorbed in the residual of the model. As a result, within-subject departures from the between-subject speed and ability level remain undetected. These departures may be of interest to the researcher as they reflect differences in the response processes adopted on the items of a test. In this article, we propose a dynamic approach for responses and response times based on hidden Markov modeling to account for within-subject differences in responses and response times. A simulation study is conducted to demonstrate acceptable parameter recovery and acceptable performance of various fit indices in distinguishing between different models. In addition, both a confirmatory and an exploratory application are presented to demonstrate the practical value of the modeling approach.
Markov stochasticity coordinates
International Nuclear Information System (INIS)
Eliazar, Iddo
2017-01-01
Markov dynamics constitute one of the most fundamental models of random motion between the states of a system of interest. Markov dynamics have diverse applications in many fields of science and engineering, and are particularly applicable in the context of random motion in networks. In this paper we present a two-dimensional gauging method of the randomness of Markov dynamics. The method–termed Markov Stochasticity Coordinates–is established, discussed, and exemplified. Also, the method is tweaked to quantify the stochasticity of the first-passage-times of Markov dynamics, and the socioeconomic equality and mobility in human societies.
Markov stochasticity coordinates
Energy Technology Data Exchange (ETDEWEB)
Eliazar, Iddo, E-mail: iddo.eliazar@intel.com
2017-01-15
Markov dynamics constitute one of the most fundamental models of random motion between the states of a system of interest. Markov dynamics have diverse applications in many fields of science and engineering, and are particularly applicable in the context of random motion in networks. In this paper we present a two-dimensional gauging method of the randomness of Markov dynamics. The method–termed Markov Stochasticity Coordinates–is established, discussed, and exemplified. Also, the method is tweaked to quantify the stochasticity of the first-passage-times of Markov dynamics, and the socioeconomic equality and mobility in human societies.
For Time-Continuous Optimisation
DEFF Research Database (Denmark)
Heinrich, Mary Katherine; Ayres, Phil
2016-01-01
Strategies for optimisation in design normatively assume an artefact end-point, disallowing continuous architecture that engages living systems, dynamic behaviour, and complex systems. In our Flora Robotica investigations of symbiotic plant-robot bio-hybrids, we re- quire computational tools...
Robust Estimation for Discrete Markov System with Time-Varying Delay and Missing Measurements
Directory of Open Access Journals (Sweden)
Jia You
2013-01-01
Full Text Available This paper addresses the ℋ∞ filtering problem for time-delayed Markov jump systems (MJSs with intermittent measurements. Within network environment, missing measurements are taken into account, since the communication channel is supposed to be imperfect. A Bernoulli process is utilized to describe the phenomenon of the missing measurements. The original system is transformed into an input-output form consisting of two interconnected subsystems. Based on scaled small gain (SSG theorem and proposed Lyapunov-Krasovskii functional (LKF, the scaled small gains of the subsystems are analyzed, respectively. New conditions for the existence of the ℋ∞ filters are established, and the corresponding ℋ∞ filter design scheme is proposed. Finally, a simulation example is provided to demonstrate the effectiveness of the proposed approach.
Time series segmentation: a new approach based on Genetic Algorithm and Hidden Markov Model
Toreti, A.; Kuglitsch, F. G.; Xoplaki, E.; Luterbacher, J.
2009-04-01
The subdivision of a time series into homogeneous segments has been performed using various methods applied to different disciplines. In climatology, for example, it is accompanied by the well-known homogenization problem and the detection of artificial change points. In this context, we present a new method (GAMM) based on Hidden Markov Model (HMM) and Genetic Algorithm (GA), applicable to series of independent observations (and easily adaptable to autoregressive processes). A left-to-right hidden Markov model, estimating the parameters and the best-state sequence, respectively, with the Baum-Welch and Viterbi algorithms, was applied. In order to avoid the well-known dependence of the Baum-Welch algorithm on the initial condition, a Genetic Algorithm was developed. This algorithm is characterized by mutation, elitism and a crossover procedure implemented with some restrictive rules. Moreover the function to be minimized was derived following the approach of Kehagias (2004), i.e. it is the so-called complete log-likelihood. The number of states was determined applying a two-fold cross-validation procedure (Celeux and Durand, 2008). Being aware that the last issue is complex, and it influences all the analysis, a Multi Response Permutation Procedure (MRPP; Mielke et al., 1981) was inserted. It tests the model with K+1 states (where K is the state number of the best model) if its likelihood is close to K-state model. Finally, an evaluation of the GAMM performances, applied as a break detection method in the field of climate time series homogenization, is shown. 1. G. Celeux and J.B. Durand, Comput Stat 2008. 2. A. Kehagias, Stoch Envir Res 2004. 3. P.W. Mielke, K.J. Berry, G.W. Brier, Monthly Wea Rev 1981.
Markov chains models, algorithms and applications
Ching, Wai-Ki; Ng, Michael K; Siu, Tak-Kuen
2013-01-01
This new edition of Markov Chains: Models, Algorithms and Applications has been completely reformatted as a text, complete with end-of-chapter exercises, a new focus on management science, new applications of the models, and new examples with applications in financial risk management and modeling of financial data.This book consists of eight chapters. Chapter 1 gives a brief introduction to the classical theory on both discrete and continuous time Markov chains. The relationship between Markov chains of finite states and matrix theory will also be highlighted. Some classical iterative methods
Stifter, Cynthia A.; Rovine, Michael
2015-01-01
The focus of the present longitudinal study, to examine mother-infant interaction during the administration of immunizations at 2 and 6?months of age, used hidden Markov modelling, a time series approach that produces latent states to describe how mothers and infants work together to bring the infant to a soothed state. Results revealed a…
Hossen, Jakir; Jacobs, Eddie L.; Chari, Srikant
2015-07-01
Linear pyroelectric array sensors have enabled useful classifications of objects such as humans and animals to be performed with relatively low-cost hardware in border and perimeter security applications. Ongoing research has sought to improve the performance of these sensors through signal processing algorithms. In the research presented here, we introduce the use of hidden Markov tree (HMT) models for object recognition in images generated by linear pyroelectric sensors. HMTs are trained to statistically model the wavelet features of individual objects through an expectation-maximization learning process. Human versus animal classification for a test object is made by evaluating its wavelet features against the trained HMTs using the maximum-likelihood criterion. The classification performance of this approach is compared to two other techniques; a texture, shape, and spectral component features (TSSF) based classifier and a speeded-up robust feature (SURF) classifier. The evaluation indicates that among the three techniques, the wavelet-based HMT model works well, is robust, and has improved classification performance compared to a SURF-based algorithm in equivalent computation time. When compared to the TSSF-based classifier, the HMT model has a slightly degraded performance but almost an order of magnitude improvement in computation time enabling real-time implementation.
A new look at the robust control of discrete-time Markov jump linear systems
Todorov, M. G.; Fragoso, M. D.
2016-03-01
In this paper, we make a foray in the role played by a set of four operators on the study of robust H2 and mixed H2/H∞ control problems for discrete-time Markov jump linear systems. These operators appear in the study of mean square stability for this class of systems. By means of new linear matrix inequality (LMI) characterisations of controllers, which include slack variables that, to some extent, separate the robustness and performance objectives, we introduce four alternative approaches to the design of controllers which are robustly stabilising and at the same time provide a guaranteed level of H2 performance. Since each operator provides a different degree of conservatism, the results are unified in the form of an iterative LMI technique for designing robust H2 controllers, whose convergence is attained in a finite number of steps. The method yields a new way of computing mixed H2/H∞ controllers, whose conservatism decreases with iteration. Two numerical examples illustrate the applicability of the proposed results for the control of a small unmanned aerial vehicle, and for an underactuated robotic arm.
Sweeting, M J; Farewell, V T; De Angelis, D
2010-05-20
In many chronic diseases it is important to understand the rate at which patients progress from infection through a series of defined disease states to a clinical outcome, e.g. cirrhosis in hepatitis C virus (HCV)-infected individuals or AIDS in HIV-infected individuals. Typically data are obtained from longitudinal studies, which often are observational in nature, and where disease state is observed only at selected examinations throughout follow-up. Transition times between disease states are therefore interval censored. Multi-state Markov models are commonly used to analyze such data, but rely on the assumption that the examination times are non-informative, and hence the examination process is ignorable in a likelihood-based analysis. In this paper we develop a Markov model that relaxes this assumption through the premise that the examination process is ignorable only after conditioning on a more regularly observed auxiliary variable. This situation arises in a study of HCV disease progression, where liver biopsies (the examinations) are sparse, irregular, and potentially informative with respect to the transition times. We use additional information on liver function tests (LFTs), commonly collected throughout follow-up, to inform current disease state and to assume an ignorable examination process. The model developed has a similar structure to a hidden Markov model and accommodates both the series of LFT measurements and the partially latent series of disease states. We show through simulation how this model compares with the commonly used ignorable Markov model, and a Markov model that assumes the examination process is non-ignorable. Copyright 2010 John Wiley & Sons, Ltd.
Directory of Open Access Journals (Sweden)
Surafel Luleseged Tilahun
2017-01-01
Full Text Available Traffic congestion is one of the main issues in the study of transportation planning and management. It creates different problems including environmental pollution and health problem and incurs a cost which is increasing through years. One-third of this congestion is created by cars searching for parking places. Drivers may be aware that parking places are fully occupied but will drive around hoping that a parking place may become vacant. Opportunistic services, involving learning, predicting, and exploiting Internet of Things scenarios, are able to adapt to dynamic unforeseen situations and have the potential to ease parking search issues. Hence, in this paper, a cooperative dynamic prediction mechanism between multiple agents for parking space availability in the neighborhood, integrating foreseen and unforeseen events and adapting for long-term changes, is proposed. An agent in each parking place will use a dynamic and time varying Markov chain to predict the parking availability and these agents will communicate to produce the parking availability prediction in the whole neighborhood. Furthermore, a learning approach is proposed where the system can adapt to different changes in the parking demand including long-term changes. Simulation results, using synthesized data based on an actual parking lot data from a shopping mall in Geneva, show that the proposed model is promising based on the learning accuracy with service adaptation and performance in different cases.
Timing of bariatric surgery for severely obese adolescents: a Markov decision-analysis.
Stroud, Andrea M; Parker, Devin; Croitoru, Daniel P
2016-05-01
Although controversial, bariatric surgery is increasingly being performed in adolescents. We developed a model to simulate the effect of timing of gastric bypass in obese adolescents on quantity and quality of life. A Markov state-transition model was constructed comparing two treatment strategies: gastric bypass surgery at age 16 versus delayed surgery in adulthood. The model simulated a hypothetical cohort of adolescents with body mass index of 45kg/m(2). Model inputs were derived from current literature. The main outcome measure was quality and quantity of life, measured using quality-adjusted life-years (QALYs). For females, early gastric bypass surgery was favored by 2.02 QALYs compared to delaying surgery until age 35 (48.91 vs. 46.89 QALYs). The benefit was even greater for males, where early surgery was favored by 2.9 QALYs (48.30 vs. 45.40 QALYs). The absolute benefit of surgery at age 16 increased; the later surgery was delayed into adulthood. Sensitivity analyses demonstrated that adult surgery was favored only when the values for adverse events were unrealistically high. In our model, early gastric bypass in obese adolescents improved both quality and quantity of life. These findings are useful for surgeons and pediatricians when counseling adolescents considering weight loss surgery. Copyright © 2016 Elsevier Inc. All rights reserved.
Markov bridges, bisection and variance reduction
DEFF Research Database (Denmark)
Asmussen, Søren; Hobolth, Asger
. In this paper we firstly consider the problem of generating sample paths from a continuous-time Markov chain conditioned on the endpoints using a new algorithm based on the idea of bisection. Secondly we study the potential of the bisection algorithm for variance reduction. In particular, examples are presented......Time-continuous Markov jump processes is a popular modelling tool in disciplines ranging from computational finance and operations research to human genetics and genomics. The data is often sampled at discrete points in time, and it can be useful to simulate sample paths between the datapoints...
Model Checking Infinite-State Markov Chains
Remke, Anne Katharina Ingrid; Haverkort, Boudewijn R.H.M.; Cloth, L.
2004-01-01
In this paper algorithms for model checking CSL (continuous stochastic logic) against infinite-state continuous-time Markov chains of so-called quasi birth-death type are developed. In doing so we extend the applicability of CSL model checking beyond the recently proposed case for finite-state
Lismawati, Eka; Respatiwulan; Widyaningsih, Purnami
2017-06-01
The SIS epidemic model describes the pattern of disease spread with characteristics that recovered individuals can be infected more than once. The number of susceptible and infected individuals every time follows the discrete time Markov process. It can be represented by the discrete time Markov chains (DTMC) SIS. The DTMC SIS epidemic model can be developed for two pathogens in two patches. The aims of this paper are to reconstruct and to apply the DTMC SIS epidemic model with two pathogens in two patches. The model was presented as transition probabilities. The application of the model obtain that the number of susceptible individuals decreases while the number of infected individuals increases for each pathogen in each patch.
Discrete-time semi-Markov modeling of human papillomavirus persistence
Mitchell, C. E.; Hudgens, M. G.; King, C. C.; Cu-Uvin, S.; Lo, Y.; Rompalo, A.; Sobel, J.; Smith, J. S.
2011-01-01
Multi-state modeling is often employed to describe the progression of a disease process. In epidemiological studies of certain diseases, the disease state is typically only observed at periodic clinical visits, producing incomplete longitudinal data. In this paper we consider fitting semi-Markov models to estimate the persistence of human papillomavirus (HPV) type-specific infection in studies where the status of HPV type(s) is assessed periodically. Simulation study results are presented indicating the semi-Markov estimator is more accurate than an estimator currently used in the HPV literature. The methods are illustrated using data from the HIV Epidemiology Research Study (HERS). PMID:21538985
DEFF Research Database (Denmark)
Xu, Shenzhi; Ai, Xiaomeng; Fang, Jiakun
2017-01-01
Photovoltaic (PV) power generation has made considerable developments in recent years. But its intermittent and volatility of its output has seriously affected the security operation of the power system. In order to better understand the PV generation and provide sufficient data support...... for analysis the impacts, a novel generation method for PV power time series combining decomposition technique and Markov chain theory is presented in this paper. It digs important factors from historical data from existing PV plants and then reproduce new data with similar patterns. In detail, the proposed...... method first decomposes the PV power time series into ideal output curve, amplitude parameter series and random fluctuating component three parts. Then generating daily ideal output curve by the extraction of typical daily data, amplitude parameter series based on the Markov chain Monte Carlo (MCMC...
Continuity of Local Time: An applied perspective
Ramirez, Jorge M.; Waymire, Edward C.; Thomann, Enrique A.
2015-01-01
Continuity of local time for Brownian motion ranks among the most notable mathematical results in the theory of stochastic processes. This article addresses its implications from the point of view of applications. In particular an extension of previous results on an explicit role of continuity of (natural) local time is obtained for applications to recent classes of problems in physics, biology and finance involving discontinuities in a dispersion coefficient. The main theorem and its corolla...
On Probabilistic Automata in Continuous Time
DEFF Research Database (Denmark)
Eisentraut, Christian; Hermanns, Holger; Zhang, Lijun
2010-01-01
We develop a compositional behavioural model that integrates a variation of probabilistic automata into a conservative extension of interactive Markov chains. The model is rich enough to embody the semantics of generalised stochastic Petri nets. We define strong and weak bisimulations and discuss...
The Markov chain method for solving dead time problems in the space dependent model of reactor noise
International Nuclear Information System (INIS)
Degweker, S.B.
1997-01-01
The discrete time Markov chain approach for deriving the statistics of time-correlated pulses, in the presence of a non-extending dead time, is extended to include the effect of space energy distribution of the neutron field. Equations for the singlet and doublet densities of follower neutrons are derived by neglecting correlations beyond the second order. These equations are solved by the modal method. It is shown that in the unimodal approximation, the equations reduce to the point model equations with suitably defined parameters. (author)
Introduction to the numerical solutions of Markov chains
Stewart, Williams J
1994-01-01
A cornerstone of applied probability, Markov chains can be used to help model how plants grow, chemicals react, and atoms diffuse - and applications are increasingly being found in such areas as engineering, computer science, economics, and education. To apply the techniques to real problems, however, it is necessary to understand how Markov chains can be solved numerically. In this book, the first to offer a systematic and detailed treatment of the numerical solution of Markov chains, William Stewart provides scientists on many levels with the power to put this theory to use in the actual world, where it has applications in areas as diverse as engineering, economics, and education. His efforts make for essential reading in a rapidly growing field. Here, Stewart explores all aspects of numerically computing solutions of Markov chains, especially when the state is huge. He provides extensive background to both discrete-time and continuous-time Markov chains and examines many different numerical computing metho...
DEFF Research Database (Denmark)
Scholer, Marie; Irving, James; Zibar, Majken Caroline Looms
Geophysical methods have the potential to provide valuable information on hydrological properties in the unsaturated zone. In particular, time-lapse geophysical data, when coupled with a hydrological model and inverted stochastically, may allow for the effective estimation of subsurface hydraulic...... parameters and their corresponding uncertainties. In this study, we use a Bayesian Markov-chain-Monte-Carlo (MCMC) inversion approach to investigate how much information regarding vadose zone hydraulic properties can be retrieved from time-lapse crosshole GPR data collected at the Arrenaes field site...
Parameter Estimation in Continuous Time Domain
Directory of Open Access Journals (Sweden)
Gabriela M. ATANASIU
2016-12-01
Full Text Available This paper will aim to presents the applications of a continuous-time parameter estimation method for estimating structural parameters of a real bridge structure. For the purpose of illustrating this method two case studies of a bridge pile located in a highly seismic risk area are considered, for which the structural parameters for the mass, damping and stiffness are estimated. The estimation process is followed by the validation of the analytical results and comparison with them to the measurement data. Further benefits and applications for the continuous-time parameter estimation method in civil engineering are presented in the final part of this paper.
Figueiredo, Danilo Zucolli; Costa, Oswaldo Luiz do Valle
2017-10-01
This paper deals with the H2 optimal control problem of discrete-time Markov jump linear systems (MJLS) considering the case in which the Markov chain takes values in a general Borel space ?. It is assumed that the controller has access only to an output variable and to the jump parameter. The goal, in this case, is to design a dynamic Markov jump controller such that the H2-norm of the closed-loop system is minimised. It is shown that the H2-norm can be written as the sum of two H2-norms, such that one of them does not depend on the control, and the other one is obtained from the optimal filter for an infinite-horizon filtering problem. This result can be seen as a separation principle for MJLS with Markov chain in a Borel space ? considering the infinite time horizon case.
Stochastic Games for Continuous-Time Jump Processes Under Finite-Horizon Payoff Criterion
Energy Technology Data Exchange (ETDEWEB)
Wei, Qingda, E-mail: weiqd@hqu.edu.cn [Huaqiao University, School of Economics and Finance (China); Chen, Xian, E-mail: chenxian@amss.ac.cn [Peking University, School of Mathematical Sciences (China)
2016-10-15
In this paper we study two-person nonzero-sum games for continuous-time jump processes with the randomized history-dependent strategies under the finite-horizon payoff criterion. The state space is countable, and the transition rates and payoff functions are allowed to be unbounded from above and from below. Under the suitable conditions, we introduce a new topology for the set of all randomized Markov multi-strategies and establish its compactness and metrizability. Then by constructing the approximating sequences of the transition rates and payoff functions, we show that the optimal value function for each player is a unique solution to the corresponding optimality equation and obtain the existence of a randomized Markov Nash equilibrium. Furthermore, we illustrate the applications of our main results with a controlled birth and death system.
Stochastic Games for Continuous-Time Jump Processes Under Finite-Horizon Payoff Criterion
International Nuclear Information System (INIS)
Wei, Qingda; Chen, Xian
2016-01-01
In this paper we study two-person nonzero-sum games for continuous-time jump processes with the randomized history-dependent strategies under the finite-horizon payoff criterion. The state space is countable, and the transition rates and payoff functions are allowed to be unbounded from above and from below. Under the suitable conditions, we introduce a new topology for the set of all randomized Markov multi-strategies and establish its compactness and metrizability. Then by constructing the approximating sequences of the transition rates and payoff functions, we show that the optimal value function for each player is a unique solution to the corresponding optimality equation and obtain the existence of a randomized Markov Nash equilibrium. Furthermore, we illustrate the applications of our main results with a controlled birth and death system.
Bisimulation and Simulation Relations for Markov Chains
Baier, Christel; Hermanns, H.; Katoen, Joost P.; Wolf, Verena; Aceto, L.; Gordon, A.
2006-01-01
Formal notions of bisimulation and simulation relation play a central role for any kind of process algebra. This short paper sketches the main concepts for bisimulation and simulation relations for probabilistic systems, modelled by discrete- or continuous-time Markov chains.
Time, physics, and the paradoxes of continuity
Steinberg, D A
2003-01-01
A recent article in this journal proposes a radical reformulation of classical and quantum dynamics based on a perceived deficiency in current definitions of time. The argument is incorrect but the errors highlight aspects of the foundations of mathematics and physics that are commonly confused and misunderstood. For this reason, the article provides an important and heuristic opportunity to reexamine the types of time and non-standard analysis. This paper will discuss the differences between physical time and experiential time and explain how an expanded system of real analysis containing infinitesimals can resolve the paradoxes of continuity without sacrificing the modern edifice of mathematical physics.
Barbu, Vlad
2008-01-01
Semi-Markov processes are much more general and better adapted to applications than the Markov ones because sojourn times in any state can be arbitrarily distributed, as opposed to the geometrically distributed sojourn time in the Markov case. This book concerns with the estimation of discrete-time semi-Markov and hidden semi-Markov processes
Continuous Time Dynamic Contraflow Models and Algorithms
Directory of Open Access Journals (Sweden)
Urmila Pyakurel
2016-01-01
Full Text Available The research on evacuation planning problem is promoted by the very challenging emergency issues due to large scale natural or man-created disasters. It is the process of shifting the maximum number of evacuees from the disastrous areas to the safe destinations as quickly and efficiently as possible. Contraflow is a widely accepted model for good solution of evacuation planning problem. It increases the outbound road capacity by reversing the direction of roads towards the safe destination. The continuous dynamic contraflow problem sends the maximum number of flow as a flow rate from the source to the sink in every moment of time unit. We propose the mathematical model for the continuous dynamic contraflow problem. We present efficient algorithms to solve the maximum continuous dynamic contraflow and quickest continuous contraflow problems on single source single sink arbitrary networks and continuous earliest arrival contraflow problem on single source single sink series-parallel networks with undefined supply and demand. We also introduce an approximation solution for continuous earliest arrival contraflow problem on two-terminal arbitrary networks.
Time-delay analyzer with continuous discretization
International Nuclear Information System (INIS)
Bayatyan, G.L.; Darbinyan, K.T.; Mkrtchyan, K.K.; Stepanyan, S.S.
1988-01-01
A time-delay analyzer is described which when triggered by a start pulse of adjustable duration performs continuous discretization of the analyzed signal within nearly 22 ns time intervals, the recording in a memory unit with following slow read-out of the information to the computer and its processing. The time-delay analyzer consists of four CAMAC-VECTOR systems of unit width. With its help one can separate comparatively short, small-amplitude rare signals against the background of quasistationary noise processes. 4 refs.; 3 figs
Path probabilities of continuous time random walks
International Nuclear Information System (INIS)
Eule, Stephan; Friedrich, Rudolf
2014-01-01
Employing the path integral formulation of a broad class of anomalous diffusion processes, we derive the exact relations for the path probability densities of these processes. In particular, we obtain a closed analytical solution for the path probability distribution of a Continuous Time Random Walk (CTRW) process. This solution is given in terms of its waiting time distribution and short time propagator of the corresponding random walk as a solution of a Dyson equation. Applying our analytical solution we derive generalized Feynman–Kac formulae. (paper)
Directory of Open Access Journals (Sweden)
Rathinasamy Sakthivel
2018-01-01
Full Text Available The problem of robust nonfragile synchronization is investigated in this paper for a class of complex dynamical networks subject to semi-Markov jumping outer coupling, time-varying coupling delay, randomly occurring gain variation, and stochastic noise over a desired finite-time interval. In particular, the network topology is assumed to follow a semi-Markov process such that it may switch from one to another at different instants. In this paper, the random gain variation is represented by a stochastic variable that is assumed to satisfy the Bernoulli distribution with white sequences. Based on these hypotheses and the Lyapunov-Krasovskii stability theory, a new finite-time stochastic synchronization criterion is established for the considered network in terms of linear matrix inequalities. Moreover, the control design parameters that guarantee the required criterion are computed by solving a set of linear matrix inequality constraints. An illustrative example is finally given to show the effectiveness and advantages of the developed analytical results.
Interaction-aided continuous time quantum search
International Nuclear Information System (INIS)
Bae, Joonwoo; Kwon, Younghun; Baek, Inchan; Yoon, Dalsun
2005-01-01
The continuous quantum search algorithm (based on the Farhi-Gutmann Hamiltonian evolution) is known to be analogous to the Grover (or discrete time quantum) algorithm. Any errors introduced in Grover algorithm are fatal to its success. In the same way the Farhi-Gutmann Hamiltonian algorithm has a severe difficulty when the Hamiltonian is perturbed. In this letter we will show that the interaction term in quantum search Hamiltonian (actually which is in the generalized quantum search Hamiltonian) can save the perturbed Farhi-Gutmann Hamiltonian that should otherwise fail. We note that this fact is quite remarkable since it implies that introduction of interaction can be a way to correct some errors on the continuous time quantum search
Markov processes an introduction for physical scientists
Gillespie, Daniel T
1991-01-01
Markov process theory is basically an extension of ordinary calculus to accommodate functions whos time evolutions are not entirely deterministic. It is a subject that is becoming increasingly important for many fields of science. This book develops the single-variable theory of both continuous and jump Markov processes in a way that should appeal especially to physicists and chemists at the senior and graduate level.Key Features* A self-contained, prgamatic exposition of the needed elements of random variable theory* Logically integrated derviations of the Chapman-Kolmogorov e
Shi, Wei; Xia, Jun
2017-02-01
Water quality risk management is a global hot research linkage with the sustainable water resource development. Ammonium nitrogen (NH 3 -N) and permanganate index (COD Mn ) as the focus indicators in Huai River Basin, are selected to reveal their joint transition laws based on Markov theory. The time-varying moments model with either time or land cover index as explanatory variables is applied to build the time-varying marginal distributions of water quality time series. Time-varying copula model, which takes the non-stationarity in the marginal distribution and/or the time variation in dependence structure between water quality series into consideration, is constructed to describe a bivariate frequency analysis for NH 3 -N and COD Mn series at the same monitoring gauge. The larger first-order Markov joint transition probability indicates water quality state Class V w , Class IV and Class III will occur easily in the water body of Bengbu Sluice. Both marginal distribution and copula models are nonstationary, and the explanatory variable time yields better performance than land cover index in describing the non-stationarities in the marginal distributions. In modelling the dependence structure changes, time-varying copula has a better fitting performance than the copula with the constant or the time-trend dependence parameter. The largest synchronous encounter risk probability of NH 3 -N and COD Mn simultaneously reaching Class V is 50.61%, while the asynchronous encounter risk probability is largest when NH 3 -N and COD Mn is inferior to class V and class IV water quality standards, respectively.
Speech Silicon: An FPGA Architecture for Real-Time Hidden Markov-Model-Based Speech Recognition
Directory of Open Access Journals (Sweden)
Schuster Jeffrey
2006-01-01
Full Text Available This paper examines the design of an FPGA-based system-on-a-chip capable of performing continuous speech recognition on medium sized vocabularies in real time. Through the creation of three dedicated pipelines, one for each of the major operations in the system, we were able to maximize the throughput of the system while simultaneously minimizing the number of pipeline stalls in the system. Further, by implementing a token-passing scheme between the later stages of the system, the complexity of the control was greatly reduced and the amount of active data present in the system at any time was minimized. Additionally, through in-depth analysis of the SPHINX 3 large vocabulary continuous speech recognition engine, we were able to design models that could be efficiently benchmarked against a known software platform. These results, combined with the ability to reprogram the system for different recognition tasks, serve to create a system capable of performing real-time speech recognition in a vast array of environments.
Speech Silicon: An FPGA Architecture for Real-Time Hidden Markov-Model-Based Speech Recognition
Directory of Open Access Journals (Sweden)
Alex K. Jones
2006-11-01
Full Text Available This paper examines the design of an FPGA-based system-on-a-chip capable of performing continuous speech recognition on medium sized vocabularies in real time. Through the creation of three dedicated pipelines, one for each of the major operations in the system, we were able to maximize the throughput of the system while simultaneously minimizing the number of pipeline stalls in the system. Further, by implementing a token-passing scheme between the later stages of the system, the complexity of the control was greatly reduced and the amount of active data present in the system at any time was minimized. Additionally, through in-depth analysis of the SPHINX 3 large vocabulary continuous speech recognition engine, we were able to design models that could be efficiently benchmarked against a known software platform. These results, combined with the ability to reprogram the system for different recognition tasks, serve to create a system capable of performing real-time speech recognition in a vast array of environments.
Modeling Uncertainty of Directed Movement via Markov Chains
Directory of Open Access Journals (Sweden)
YIN Zhangcai
2015-10-01
Full Text Available Probabilistic time geography (PTG is suggested as an extension of (classical time geography, in order to present the uncertainty of an agent located at the accessible position by probability. This may provide a quantitative basis for most likely finding an agent at a location. In recent years, PTG based on normal distribution or Brown bridge has been proposed, its variance, however, is irrelevant with the agent's speed or divergent with the increase of the speed; so they are difficult to take into account application pertinence and stability. In this paper, a new method is proposed to model PTG based on Markov chain. Firstly, a bidirectional conditions Markov chain is modeled, the limit of which, when the moving speed is large enough, can be regarded as the Brown bridge, thus has the characteristics of digital stability. Then, the directed movement is mapped to Markov chains. The essential part is to build step length, the state space and transfer matrix of Markov chain according to the space and time position of directional movement, movement speed information, to make sure the Markov chain related to the movement speed. Finally, calculating continuously the probability distribution of the directed movement at any time by the Markov chains, it can be get the possibility of an agent located at the accessible position. Experimental results show that, the variance based on Markov chains not only is related to speed, but also is tending towards stability with increasing the agent's maximum speed.
Markov Chain Models for the Stochastic Modeling of Pitting Corrosion
Valor, A.; Caleyo, F.; Alfonso, L.; Velázquez, J. C.; Hallen, J. M.
2013-01-01
The stochastic nature of pitting corrosion of metallic structures has been widely recognized. It is assumed that this kind of deterioration retains no memory of the past, so only the current state of the damage influences its future development. This characteristic allows pitting corrosion to be categorized as a Markov process. In this paper, two different models of pitting corrosion, developed using Markov chains, are presented. Firstly, a continuous-time, nonhomogeneous linear growth (pure ...
Language Emptiness of Continuous-Time Parametric Timed Automata
DEFF Research Database (Denmark)
Benes, Nikola; Bezdek, Peter; Larsen, Kim Guldstrand
2015-01-01
Parametric timed automata extend the standard timed automata with the possibility to use parameters in the clock guards. In general, if the parameters are real-valued, the problem of language emptiness of such automata is undecidable even for various restricted subclasses. We thus focus on the case...... where parameters are assumed to be integer-valued, while the time still remains continuous. On the one hand, we show that the problem remains undecidable for parametric timed automata with three clocks and one parameter. On the other hand, for the case with arbitrary many clocks where only one......-time semantics only. To the best of our knowledge, this is the first positive result in the case of continuous-time and unbounded integer parameters, except for the rather simple case of single-clock automata....
DEFF Research Database (Denmark)
Pinson, Pierre; Madsen, Henrik
2008-01-01
Better modelling and forecasting of very short-term power fluctuations at large offshore wind farms may significantly enhance control and management strategies of their power output. The paper introduces a new methodology for modelling and forecasting such very short-term fluctuations. The proposed...... consists in 1-step ahead forecasting exercise on time-series of wind generation with a time resolution of 10 minute. The quality of the introduced forecasting methodology and its interest for better understanding power fluctuations are finally discussed....... methodology is based on a Markov-switching autoregressive model with time-varying coefficients. An advantage of the method is that one can easily derive full predictive densities. The quality of this methodology is demonstrated from the test case of 2 large offshore wind farms in Denmark. The exercise...
Yuan, Y.; Meng, Y.; Chen, Y. X.; Jiang, C.; Yue, A. Z.
2018-04-01
In this study, we proposed a method to map urban encroachment onto farmland using satellite image time series (SITS) based on the hierarchical hidden Markov model (HHMM). In this method, the farmland change process is decomposed into three hierarchical levels, i.e., the land cover level, the vegetation phenology level, and the SITS level. Then a three-level HHMM is constructed to model the multi-level semantic structure of farmland change process. Once the HHMM is established, a change from farmland to built-up could be detected by inferring the underlying state sequence that is most likely to generate the input time series. The performance of the method is evaluated on MODIS time series in Beijing. Results on both simulated and real datasets demonstrate that our method improves the change detection accuracy compared with the HMM-based method.
Rahman, P. A.; D'K Novikova Freyre Shavier, G.
2018-03-01
This scientific paper is devoted to the analysis of the mean time to data loss of redundant disk arrays RAID-6 with alternation of data considering different failure rates of disks both in normal state of the disk array and in degraded and rebuild states, and also nonzero time of the disk replacement. The reliability model developed by the authors on the basis of the Markov chain and obtained calculation formula for estimation of the mean time to data loss (MTTDL) of the RAID-6 disk arrays are also presented. At last, the technique of estimation of the initial reliability parameters and examples of calculation of the MTTDL of the RAID-6 disk arrays for the different numbers of disks are also given.
Herrera, C.; Nava, F. A.; Lomnitz, C.
2006-08-01
A previous work introduced a new method for seismic hazard evaluation in a system (a geographic area with distinct, but related seismogenic regions) based on modeling the transition probabilities of states (patterns of presence or absence of seismicity, with magnitude greater or equal to a threshold magnitude Mr, in the regions of the system, during a time interval Δt) as a Markov chain. Application of this direct method to the Japan area gave very good results. Given that the most important limitation of the direct method is the relative scarcity of large magnitude events, we decided to explore the possibility that seismicity with magnitude M ≥ Mmr contains information about the future occurrence of earthquakes with M ≥ Mmr > Mmr. This mixed Markov chain method estimates the probabilities of occurrence of a system state for M ≥ MMr on the basis of the observed state for M ≥ Mmr in the previous Δt. Application of the mixed method to the area of Japan gives better hazard estimations than the direct method; in particular for large earthquakes. As part of this study, the problem of performance evaluation of hazard estimation methods is addressed, leading to the use of grading functions.
Stifter, Cynthia A; Rovine, Michael
2015-01-01
The focus of the present longitudinal study, to examine mother-infant interaction during the administration of immunizations at two and six months of age, used hidden Markov modeling, a time series approach that produces latent states to describe how mothers and infants work together to bring the infant to a soothed state. Results revealed a 4-state model for the dyadic responses to a two-month inoculation whereas a 6-state model best described the dyadic process at six months. Two of the states at two months and three of the states at six months suggested a progression from high intensity crying to no crying with parents using vestibular and auditory soothing methods. The use of feeding and/or pacifying to soothe the infant characterized one two-month state and two six-month states. These data indicate that with maturation and experience, the mother-infant dyad is becoming more organized around the soothing interaction. Using hidden Markov modeling to describe individual differences, as well as normative processes, is also presented and discussed.
A scaling analysis of a cat and mouse Markov chain
Litvak, Nelli; Robert, Philippe
2012-01-01
If ($C_n$) a Markov chain on a discrete state space $S$, a Markov chain ($C_n, M_n$) on the product space $S \\times S$, the cat and mouse Markov chain, is constructed. The first coordinate of this Markov chain behaves like the original Markov chain and the second component changes only when both
Fast-slow asymptotics for a Markov chain model of fast sodium current
Starý, Tomáš; Biktashev, Vadim N.
2017-09-01
We explore the feasibility of using fast-slow asymptotics to eliminate the computational stiffness of discrete-state, continuous-time deterministic Markov chain models of ionic channels underlying cardiac excitability. We focus on a Markov chain model of fast sodium current, and investigate its asymptotic behaviour with respect to small parameters identified in different ways.
A continuous time Cournot duopoly with delays
International Nuclear Information System (INIS)
Gori, Luca; Guerrini, Luca; Sodini, Mauro
2015-01-01
This paper extends the classical repeated duopoly model with quantity-setting firms of Bischi et al. (1998) by assuming that production of goods is subject to some gestation lags but exchanges take place continuously in the market. The model is expressed in the form of differential equations with discrete delays. By using some recent mathematical techniques and numerical experiments, results show some dynamic phenomena that cannot be observed when delays are absent. In addition, depending on the extent of time delays and inertia, synchronisation failure can arise even in the event of homogeneous firms.
Stochastic volatility of volatility in continuous time
DEFF Research Database (Denmark)
Barndorff-Nielsen, Ole; Veraart, Almut
This paper introduces the concept of stochastic volatility of volatility in continuous time and, hence, extends standard stochastic volatility (SV) models to allow for an additional source of randomness associated with greater variability in the data. We discuss how stochastic volatility...... of volatility can be defined both non-parametrically, where we link it to the quadratic variation of the stochastic variance process, and parametrically, where we propose two new SV models which allow for stochastic volatility of volatility. In addition, we show that volatility of volatility can be estimated...
Heterogeneous continuous-time random walks
Grebenkov, Denis S.; Tupikina, Liubov
2018-01-01
We introduce a heterogeneous continuous-time random walk (HCTRW) model as a versatile analytical formalism for studying and modeling diffusion processes in heterogeneous structures, such as porous or disordered media, multiscale or crowded environments, weighted graphs or networks. We derive the exact form of the propagator and investigate the effects of spatiotemporal heterogeneities onto the diffusive dynamics via the spectral properties of the generalized transition matrix. In particular, we show how the distribution of first-passage times changes due to local and global heterogeneities of the medium. The HCTRW formalism offers a unified mathematical language to address various diffusion-reaction problems, with numerous applications in material sciences, physics, chemistry, biology, and social sciences.
Irreversible Local Markov Chains with Rapid Convergence towards Equilibrium
Kapfer, Sebastian C.; Krauth, Werner
2017-12-01
We study the continuous one-dimensional hard-sphere model and present irreversible local Markov chains that mix on faster time scales than the reversible heat bath or Metropolis algorithms. The mixing time scales appear to fall into two distinct universality classes, both faster than for reversible local Markov chains. The event-chain algorithm, the infinitesimal limit of one of these Markov chains, belongs to the class presenting the fastest decay. For the lattice-gas limit of the hard-sphere model, reversible local Markov chains correspond to the symmetric simple exclusion process (SEP) with periodic boundary conditions. The two universality classes for irreversible Markov chains are realized by the totally asymmetric SEP (TASEP), and by a faster variant (lifted TASEP) that we propose here. We discuss how our irreversible hard-sphere Markov chains generalize to arbitrary repulsive pair interactions and carry over to higher dimensions through the concept of lifted Markov chains and the recently introduced factorized Metropolis acceptance rule.
International Nuclear Information System (INIS)
Degweker, S.B.
1996-01-01
The problem of deriving the statistics of time-correlated detector pulses in the presence of a non-extendible dead time is studied by constructing a Markov Chain to describe the process. Expressions for the transition matrix are derived for problems in the passive neutron assay of Pu and (zero-power) reactor noise. Perturbative and numerical solutions of the master equations are discussed for a simple problem in the passive neutron assay of Pu. Expressions for the mean count rate and variance in a given interval are derived. (Author)
Directory of Open Access Journals (Sweden)
Alejandro Baldominos Gómez
2016-03-01
Full Text Available This paper presents the results and conclusions found when predicting the behavior of gamers in commercial videogames datasets. In particular, it uses Variable-Order Markov (VOM to build a probabilistic model that is able to use the historic behavior of gamers and to infer what will be their next actions. Being able to predict with accuracy the next user’s actions can be of special interest to learn from the behavior of gamers, to make them more engaged and to reduce churn rate. In order to support a big volume and velocity of data, the system is built on top of the Hadoop ecosystem, using HBase for real-time processing; and the prediction tool is provided as a service (SaaS and accessible through a RESTful API. The prediction system is evaluated using a case of study with two commercial videogames, attaining promising results with high prediction accuracies.
Solan, Eilon; Vieille, Nicolas
2015-01-01
We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the transition matrix. We define a new closeness relation between transition matrices, and use graph-theoretic techniques, in contrast with the matrix analysis techniques previously used.
Efficient Approximation of Optimal Control for Markov Games
DEFF Research Database (Denmark)
Fearnley, John; Rabe, Markus; Schewe, Sven
2011-01-01
We study the time-bounded reachability problem for continuous-time Markov decision processes (CTMDPs) and games (CTMGs). Existing techniques for this problem use discretisation techniques to break time into discrete intervals, and optimal control is approximated for each interval separately...
D'Onofrio, Giuseppe; Pirozzi, Enrica
2017-05-01
We consider a stochastic differential equation in a strip, with coefficients suitably chosen to describe the acto-myosin interaction subject to time-varying forces. By simulating trajectories of the stochastic dynamics via an Euler discretization-based algorithm, we fit experimental data and determine the values of involved parameters. The steps of the myosin are represented by the exit events from the strip. Motivated by these results, we propose a specific stochastic model based on the corresponding time-inhomogeneous Gauss-Markov and diffusion process evolving between two absorbing boundaries. We specify the mean and covariance functions of the stochastic modeling process taking into account time-dependent forces including the effect of an external load. We accurately determine the probability density function (pdf) of the first exit time (FET) from the strip by solving a system of two non singular second-type Volterra integral equations via a numerical quadrature. We provide numerical estimations of the mean of FET as approximations of the dwell-time of the proteins dynamics. The percentage of backward steps is given in agreement to experimental data. Numerical and simulation results are compared and discussed.
International Nuclear Information System (INIS)
Minjarez-Sosa, J. Adolfo; Luque-Vasquez, Fernando
2008-01-01
This paper deals with two person zero-sum semi-Markov games with a possibly unbounded payoff function, under a discounted payoff criterion. Assuming that the distribution of the holding times H is unknown for one of the players, we combine suitable methods of statistical estimation of H with control procedures to construct an asymptotically discount optimal pair of strategies
Hartfiel, Darald J
1998-01-01
In this study extending classical Markov chain theory to handle fluctuating transition matrices, the author develops a theory of Markov set-chains and provides numerous examples showing how that theory can be applied. Chapters are concluded with a discussion of related research. Readers who can benefit from this monograph are those interested in, or involved with, systems whose data is imprecise or that fluctuate with time. A background equivalent to a course in linear algebra and one in probability theory should be sufficient.
DEFF Research Database (Denmark)
Larsen, Christian; Turkensteen, Marcel
2014-01-01
be a two-dimensional area or a one-dimensional line structure (corresponding to e.g. a major traffic artery). The expected travel distances across a given number of retailers can now be estimated analytically, using results from the field of continuous approxim ation for two-dimensional areas, or using our...
Discrete time and continuous time dynamic mean-variance analysis
Reiss, Ariane
1999-01-01
Contrary to static mean-variance analysis, very few papers have dealt with dynamic mean-variance analysis. Here, the mean-variance efficient self-financing portfolio strategy is derived for n risky assets in discrete and continuous time. In the discrete setting, the resulting portfolio is mean-variance efficient in a dynamic sense. It is shown that the optimal strategy for n risky assets may be dominated if the expected terminal wealth is constrained to exactly attain a certain goal instead o...
Energy Technology Data Exchange (ETDEWEB)
Comen, E; Mason, J; Kuhn, P [The Scripps Research Institute, La Jolla, CA (United States); Nieva, J [Billings Clinic, Billings, Montana (United States); Newton, P [University of Southern California, Los Angeles, CA (United States); Norton, L; Venkatappa, N; Jochelson, M [Memorial Sloan-Kettering Cancer Center, NY, NY (United States)
2014-06-01
Purpose: Traditionally, breast cancer metastasis is described as a process wherein cancer cells spread from the breast to multiple organ systems via hematogenous and lymphatic routes. Mapping organ specific patterns of cancer spread over time is essential to understanding metastatic progression. In order to better predict sites of metastases, here we demonstrate modeling of the patterned migration of metastasis. Methods: We reviewed the clinical history of 453 breast cancer patients from Memorial Sloan Kettering Cancer Center who were non-metastatic at diagnosis but developed metastasis over time. We used the variables of organ site of metastases as well as time to create a Markov chain model of metastasis. We illustrate the probabilities of metastasis occurring at a given anatomic site together with the probability of spread to additional sites. Results: Based on the clinical histories of 453 breast cancer patients who developed metastasis, we have learned (i) how to create the Markov transition matrix governing the probabilities of cancer progression from site to site; (ii) how to create a systemic network diagram governing disease progression modeled as a random walk on a directed graph; (iii) how to classify metastatic sites as ‘sponges’ that tend to only receive cancer cells or ‘spreaders’ that receive and release them; (iv) how to model the time-scales of disease progression as a Weibull probability distribution function; (v) how to perform Monte Carlo simulations of disease progression; and (vi) how to interpret disease progression as an entropy-increasing stochastic process. Conclusion: Based on our modeling, metastatic spread may follow predictable pathways. Mapping metastasis not simply by organ site, but by function as either a ‘spreader’ or ‘sponge’ fundamentally reframes our understanding of metastatic processes. This model serves as a novel platform from which we may integrate the evolving genomic landscape that drives cancer
International Nuclear Information System (INIS)
Comen, E; Mason, J; Kuhn, P; Nieva, J; Newton, P; Norton, L; Venkatappa, N; Jochelson, M
2014-01-01
Purpose: Traditionally, breast cancer metastasis is described as a process wherein cancer cells spread from the breast to multiple organ systems via hematogenous and lymphatic routes. Mapping organ specific patterns of cancer spread over time is essential to understanding metastatic progression. In order to better predict sites of metastases, here we demonstrate modeling of the patterned migration of metastasis. Methods: We reviewed the clinical history of 453 breast cancer patients from Memorial Sloan Kettering Cancer Center who were non-metastatic at diagnosis but developed metastasis over time. We used the variables of organ site of metastases as well as time to create a Markov chain model of metastasis. We illustrate the probabilities of metastasis occurring at a given anatomic site together with the probability of spread to additional sites. Results: Based on the clinical histories of 453 breast cancer patients who developed metastasis, we have learned (i) how to create the Markov transition matrix governing the probabilities of cancer progression from site to site; (ii) how to create a systemic network diagram governing disease progression modeled as a random walk on a directed graph; (iii) how to classify metastatic sites as ‘sponges’ that tend to only receive cancer cells or ‘spreaders’ that receive and release them; (iv) how to model the time-scales of disease progression as a Weibull probability distribution function; (v) how to perform Monte Carlo simulations of disease progression; and (vi) how to interpret disease progression as an entropy-increasing stochastic process. Conclusion: Based on our modeling, metastatic spread may follow predictable pathways. Mapping metastasis not simply by organ site, but by function as either a ‘spreader’ or ‘sponge’ fundamentally reframes our understanding of metastatic processes. This model serves as a novel platform from which we may integrate the evolving genomic landscape that drives cancer
Composable Markov Building Blocks
Evers, S.; Fokkinga, M.M.; Apers, Peter M.G.; Prade, H.; Subrahmanian, V.S.
2007-01-01
In situations where disjunct parts of the same process are described by their own first-order Markov models and only one model applies at a time (activity in one model coincides with non-activity in the other models), these models can be joined together into one. Under certain conditions, nearly all
Composable Markov Building Blocks
Evers, S.; Fokkinga, M.M.; Apers, Peter M.G.
2007-01-01
In situations where disjunct parts of the same process are described by their own first-order Markov models, these models can be joined together under the constraint that there can only be one activity at a time, i.e. the activities of one model coincide with non-activity in the other models. Under
Matsunaga, Y.; Sugita, Y.
2018-06-01
A data-driven modeling scheme is proposed for conformational dynamics of biomolecules based on molecular dynamics (MD) simulations and experimental measurements. In this scheme, an initial Markov State Model (MSM) is constructed from MD simulation trajectories, and then, the MSM parameters are refined using experimental measurements through machine learning techniques. The second step can reduce the bias of MD simulation results due to inaccurate force-field parameters. Either time-series trajectories or ensemble-averaged data are available as a training data set in the scheme. Using a coarse-grained model of a dye-labeled polyproline-20, we compare the performance of machine learning estimations from the two types of training data sets. Machine learning from time-series data could provide the equilibrium populations of conformational states as well as their transition probabilities. It estimates hidden conformational states in more robust ways compared to that from ensemble-averaged data although there are limitations in estimating the transition probabilities between minor states. We discuss how to use the machine learning scheme for various experimental measurements including single-molecule time-series trajectories.
Directory of Open Access Journals (Sweden)
Ho KC
2005-01-01
Full Text Available We propose a real-time software system for landmine detection using ground-penetrating radar (GPR. The system includes an efficient and adaptive preprocessing component; a hidden Markov model- (HMM- based detector; a corrective training component; and an incremental update of the background model. The preprocessing is based on frequency-domain processing and performs ground-level alignment and background removal. The HMM detector is an improvement of a previously proposed system (baseline. It includes additional pre- and postprocessing steps to improve the time efficiency and enable real-time application. The corrective training component is used to adjust the initial model parameters to minimize the number of misclassification sequences. This component could be used offline, or online through feedback to adapt an initial model to specific sites and environments. The background update component adjusts the parameters of the background model to adapt it to each lane during testing. The proposed software system is applied to data acquired from three outdoor test sites at different geographic locations, using a state-of-the-art array GPR prototype. The first collection was used as training, and the other two (contain data from more than 1200 m of simulated dirt and gravel roads for testing. Our results indicate that, on average, the corrective training can improve the performance by about 10% for each site. For individual lanes, the performance gain can reach 50%.
Nonlinearly perturbed semi-Markov processes
Silvestrov, Dmitrii
2017-01-01
The book presents new methods of asymptotic analysis for nonlinearly perturbed semi-Markov processes with a finite phase space. These methods are based on special time-space screening procedures for sequential phase space reduction of semi-Markov processes combined with the systematical use of operational calculus for Laurent asymptotic expansions. Effective recurrent algorithms are composed for getting asymptotic expansions, without and with explicit upper bounds for remainders, for power moments of hitting times, stationary and conditional quasi-stationary distributions for nonlinearly perturbed semi-Markov processes. These results are illustrated by asymptotic expansions for birth-death-type semi-Markov processes, which play an important role in various applications. The book will be a useful contribution to the continuing intensive studies in the area. It is an essential reference for theoretical and applied researchers in the field of stochastic processes and their applications that will cont...
On the Total Variation Distance of Semi-Markov Chains
DEFF Research Database (Denmark)
Bacci, Giorgio; Bacci, Giovanni; Larsen, Kim Guldstrand
2015-01-01
Semi-Markov chains (SMCs) are continuous-time probabilistic transition systems where the residence time on states is governed by generic distributions on the positive real line. This paper shows the tight relation between the total variation distance on SMCs and their model checking problem over...
Discounting Models for Outcomes over Continuous Time
DEFF Research Database (Denmark)
Harvey, Charles M.; Østerdal, Lars Peter
Events that occur over a period of time can be described either as sequences of outcomes at discrete times or as functions of outcomes in an interval of time. This paper presents discounting models for events of the latter type. Conditions on preferences are shown to be satisfied if and only if t...... if the preferences are represented by a function that is an integral of a discounting function times a scale defined on outcomes at instants of time....
Hierarchical Hidden Markov Models for Multivariate Integer-Valued Time-Series
DEFF Research Database (Denmark)
Catania, Leopoldo; Di Mari, Roberto
2018-01-01
We propose a new flexible dynamic model for multivariate nonnegative integer-valued time-series. Observations are assumed to depend on the realization of two additional unobserved integer-valued stochastic variables which control for the time-and cross-dependence of the data. An Expectation......-Maximization algorithm for maximum likelihood estimation of the model's parameters is derived. We provide conditional and unconditional (cross)-moments implied by the model, as well as the limiting distribution of the series. A Monte Carlo experiment investigates the finite sample properties of our estimation...
Remembering the time: a continuous clock.
Lewis, Penelope A; Miall, R Chris
2006-09-01
The neural mechanisms for time measurement are currently a subject of much debate. This article argues that our brains can measure time using the same dorsolateral prefrontal cells that are known to be involved in working memory. Evidence for this is: (1) the dorsolateral prefrontal cortex is integral to both cognitive timing and working memory; (2) both behavioural processes are modulated by dopamine and disrupted by manipulation of dopaminergic projections to the dorsolateral prefrontal cortex; (3) the neurons in question ramp their activity in a temporally predictable way during both types of processing; and (4) this ramping activity is modulated by dopamine. The dual involvement of these prefrontal neurons in working memory and cognitive timing supports a view of the prefrontal cortex as a multipurpose processor recruited by a wide variety of tasks.
Prognostics for Steam Generator Tube Rupture using Markov Chain model
International Nuclear Information System (INIS)
Kim, Gibeom; Heo, Gyunyoung; Kim, Hyeonmin
2016-01-01
This paper will describe the prognostics method for evaluating and forecasting the ageing effect and demonstrate the procedure of prognostics for the Steam Generator Tube Rupture (SGTR) accident. Authors will propose the data-driven method so called MCMC (Markov Chain Monte Carlo) which is preferred to the physical-model method in terms of flexibility and availability. Degradation data is represented as growth of burst probability over time. Markov chain model is performed based on transition probability of state. And the state must be discrete variable. Therefore, burst probability that is continuous variable have to be changed into discrete variable to apply Markov chain model to the degradation data. The Markov chain model which is one of prognostics methods was described and the pilot demonstration for a SGTR accident was performed as a case study. The Markov chain model is strong since it is possible to be performed without physical models as long as enough data are available. However, in the case of the discrete Markov chain used in this study, there must be loss of information while the given data is discretized and assigned to the finite number of states. In this process, original information might not be reflected on prediction sufficiently. This should be noted as the limitation of discrete models. Now we will be studying on other prognostics methods such as GPM (General Path Model) which is also data-driven method as well as the particle filer which belongs to physical-model method and conducting comparison analysis
3D Markov Process for Traffic Flow Prediction in Real-Time
Directory of Open Access Journals (Sweden)
Eunjeong Ko
2016-01-01
Full Text Available Recently, the correct estimation of traffic flow has begun to be considered an essential component in intelligent transportation systems. In this paper, a new statistical method to predict traffic flows using time series analyses and geometric correlations is proposed. The novelty of the proposed method is two-fold: (1 a 3D heat map is designed to describe the traffic conditions between roads, which can effectively represent the correlations between spatially- and temporally-adjacent traffic states; and (2 the relationship between the adjacent roads on the spatiotemporal domain is represented by cliques in MRF and the clique parameters are obtained by example-based learning. In order to assess the validity of the proposed method, it is tested using data from expressway traffic that are provided by the Korean Expressway Corporation, and the performance of the proposed method is compared with existing approaches. The results demonstrate that the proposed method can predict traffic conditions with an accuracy of 85%, and this accuracy can be improved further.
Structure and Randomness of Continuous-Time, Discrete-Event Processes
Marzen, Sarah E.; Crutchfield, James P.
2017-10-01
Loosely speaking, the Shannon entropy rate is used to gauge a stochastic process' intrinsic randomness; the statistical complexity gives the cost of predicting the process. We calculate, for the first time, the entropy rate and statistical complexity of stochastic processes generated by finite unifilar hidden semi-Markov models—memoryful, state-dependent versions of renewal processes. Calculating these quantities requires introducing novel mathematical objects (ɛ -machines of hidden semi-Markov processes) and new information-theoretic methods to stochastic processes.
Energy Technology Data Exchange (ETDEWEB)
Frank, T D [Center for the Ecological Study of Perception and Action, Department of Psychology, University of Connecticut, 406 Babbidge Road, Storrs, CT 06269 (United States)
2008-07-18
We discuss nonlinear Markov processes defined on discrete time points and discrete state spaces using Markov chains. In this context, special attention is paid to the distinction between linear and nonlinear Markov processes. We illustrate that the Chapman-Kolmogorov equation holds for nonlinear Markov processes by a winner-takes-all model for social conformity. (fast track communication)
International Nuclear Information System (INIS)
Frank, T D
2008-01-01
We discuss nonlinear Markov processes defined on discrete time points and discrete state spaces using Markov chains. In this context, special attention is paid to the distinction between linear and nonlinear Markov processes. We illustrate that the Chapman-Kolmogorov equation holds for nonlinear Markov processes by a winner-takes-all model for social conformity. (fast track communication)
A Markov Model for Commen-Cause Failures
DEFF Research Database (Denmark)
Platz, Ole
1984-01-01
A continuous time four-state Markov chain is shown to cover several of the models that have been used for describing dependencies between failures of components in redundant systems. Among these are the models derived by Marshall and Olkin and by Freund and models for one-out-of-three and two...
Power plant reliability calculation with Markov chain models
International Nuclear Information System (INIS)
Senegacnik, A.; Tuma, M.
1998-01-01
In the paper power plant operation is modelled using continuous time Markov chains with discrete state space. The model is used to compute the power plant reliability and the importance and influence of individual states, as well as the transition probabilities between states. For comparison the model is fitted to data for coal and nuclear power plants recorded over several years. (orig.) [de
Markov chains with quasitoeplitz transition matrix: first zero hitting
Directory of Open Access Journals (Sweden)
Alexander M. Dukhovny
1989-01-01
Full Text Available This paper continues the investigation of Markov Chains with a quasitoeplitz transition matrix. Generating functions of first zero hitting probabilities and mean times are found by the solution of special Riemann boundary value problems on the unit circle. Duality is discussed.
Generalized Markov branching models
Li, Junping
2005-01-01
In this thesis, we first considered a modified Markov branching process incorporating both state-independent immigration and resurrection. After establishing the criteria for regularity and uniqueness, explicit expressions for the extinction probability and mean extinction time are presented. The criteria for recurrence and ergodicity are also established. In addition, an explicit expression for the equilibrium distribution is presented.\\ud \\ud We then moved on to investigate the basic proper...
Pemodelan Markov Switching Autoregressive
Ariyani, Fiqria Devi; Warsito, Budi; Yasin, Hasbi
2014-01-01
Transition from depreciation to appreciation of exchange rate is one of regime switching that ignored by classic time series model, such as ARIMA, ARCH, or GARCH. Therefore, economic variables are modeled by Markov Switching Autoregressive (MSAR) which consider the regime switching. MLE is not applicable to parameters estimation because regime is an unobservable variable. So that filtering and smoothing process are applied to see the regime probabilities of observation. Using this model, tran...
Reviving Markov processes and applications
International Nuclear Information System (INIS)
Cai, H.
1988-01-01
In this dissertation we study a procedure which restarts a Markov process when the process is killed by some arbitrary multiplicative functional. The regenerative nature of this revival procedure is characterized through a Markov renewal equation. An interesting duality between the revival procedure and the classical killing operation is found. Under the condition that the multiplicative functional possesses an intensity, the generators of the revival process can be written down explicitly. An intimate connection is also found between the perturbation of the sample path of a Markov process and the perturbation of a generator (in Kato's sense). The applications of the theory include the study of the processes like piecewise-deterministic Markov process, virtual waiting time process and the first entrance decomposition (taboo probability)
Markov chain modelling of pitting corrosion in underground pipelines
Energy Technology Data Exchange (ETDEWEB)
Caleyo, F. [Departamento de Ingenieri' a Metalurgica, ESIQIE, IPN, UPALM Edif. 7, Zacatenco, Mexico D. F. 07738 (Mexico)], E-mail: fcaleyo@gmail.com; Velazquez, J.C. [Departamento de Ingenieri' a Metalurgica, ESIQIE, IPN, UPALM Edif. 7, Zacatenco, Mexico D. F. 07738 (Mexico); Valor, A. [Facultad de Fisica, Universidad de La Habana, San Lazaro y L, Vedado, 10400 La Habana (Cuba); Hallen, J.M. [Departamento de Ingenieri' a Metalurgica, ESIQIE, IPN, UPALM Edif. 7, Zacatenco, Mexico D. F. 07738 (Mexico)
2009-09-15
A continuous-time, non-homogenous linear growth (pure birth) Markov process has been used to model external pitting corrosion in underground pipelines. The closed form solution of Kolmogorov's forward equations for this type of Markov process is used to describe the transition probability function in a discrete pit depth space. The identification of the transition probability function can be achieved by correlating the stochastic pit depth mean with the deterministic mean obtained experimentally. Monte-Carlo simulations previously reported have been used to predict the time evolution of the mean value of the pit depth distribution for different soil textural classes. The simulated distributions have been used to create an empirical Markov chain-based stochastic model for predicting the evolution of pitting corrosion depth and rate distributions from the observed properties of the soil. The proposed model has also been applied to pitting corrosion data from pipeline repeated in-line inspections and laboratory immersion experiments.
Markov chain modelling of pitting corrosion in underground pipelines
International Nuclear Information System (INIS)
Caleyo, F.; Velazquez, J.C.; Valor, A.; Hallen, J.M.
2009-01-01
A continuous-time, non-homogenous linear growth (pure birth) Markov process has been used to model external pitting corrosion in underground pipelines. The closed form solution of Kolmogorov's forward equations for this type of Markov process is used to describe the transition probability function in a discrete pit depth space. The identification of the transition probability function can be achieved by correlating the stochastic pit depth mean with the deterministic mean obtained experimentally. Monte-Carlo simulations previously reported have been used to predict the time evolution of the mean value of the pit depth distribution for different soil textural classes. The simulated distributions have been used to create an empirical Markov chain-based stochastic model for predicting the evolution of pitting corrosion depth and rate distributions from the observed properties of the soil. The proposed model has also been applied to pitting corrosion data from pipeline repeated in-line inspections and laboratory immersion experiments.
Yan, Huaicheng; Zhang, Hao; Yang, Fuwen; Zhan, Xisheng; Peng, Chen
2017-08-18
This paper is concerned with the guaranteed cost control problem for a class of Markov jump discrete-time neural networks (NNs) with event-triggered mechanism, asynchronous jumping, and fading channels. The Markov jump NNs are introduced to be close to reality, where the modes of the NNs and guaranteed cost controller are determined by two mutually independent Markov chains. The asynchronous phenomenon is considered, which increases the difficulty of designing required mode-dependent controller. The event-triggered mechanism is designed by comparing the relative measurement error with the last triggered state at the process of data transmission, which is used to eliminate dispensable transmission and reduce the networked energy consumption. In addition, the signal fading is considered for the effect of signal reflection and shadow in wireless networks, which is modeled by the novel Rice fading models. Some novel sufficient conditions are obtained to guarantee that the closed-loop system reaches a specified cost value under the designed jumping state feedback control law in terms of linear matrix inequalities. Finally, some simulation results are provided to illustrate the effectiveness of the proposed method.
Detecting Faults By Use Of Hidden Markov Models
Smyth, Padhraic J.
1995-01-01
Frequency of false alarms reduced. Faults in complicated dynamic system (e.g., antenna-aiming system, telecommunication network, or human heart) detected automatically by method of automated, continuous monitoring. Obtains time-series data by sampling multiple sensor outputs at discrete intervals of t and processes data via algorithm determining whether system in normal or faulty state. Algorithm implements, among other things, hidden first-order temporal Markov model of states of system. Mathematical model of dynamics of system not needed. Present method is "prior" method mentioned in "Improved Hidden-Markov-Model Method of Detecting Faults" (NPO-18982).
Analyzing the profit-loss sharing contracts with Markov model
Directory of Open Access Journals (Sweden)
Imam Wahyudi
2016-12-01
Full Text Available The purpose of this paper is to examine how to use first order Markov chain to build a reliable monitoring system for the profit-loss sharing based contracts (PLS as the mode of financing contracts in Islamic bank with censored continuous-time observations. The paper adopts the longitudinal analysis with the first order Markov chain framework. Laplace transform was used with homogenous continuous time assumption, from discretized generator matrix, to generate the transition matrix. Various metrics, i.e.: eigenvalue and eigenvector were used to test the first order Markov chain assumption. Cox semi parametric model was used also to analyze the momentum and waiting time effect as non-Markov behavior. The result shows that first order Markov chain is powerful as a monitoring tool for Islamic banks. We find that waiting time negatively affected present rating downgrade (upgrade significantly. Likewise, momentum covariate showed negative effect. Finally, the result confirms that different origin rating have different movement behavior. The paper explores the potential of Markov chain framework as a risk management tool for Islamic banks. It provides valuable insight and integrative model for banks to manage their borrower accounts. This model can be developed to be a powerful early warning system to identify which borrower needs to be monitored intensively. Ultimately, this model could potentially increase the efficiency, productivity and competitiveness of Islamic banks in Indonesia. The analysis used only rating data. Further study should be able to give additional information about the determinant factors of rating movement of the borrowers by incorporating various factors such as contract-related factors, bank-related factors, borrower-related factors and macroeconomic factors.
Continuous-time quantum walks on star graphs
International Nuclear Information System (INIS)
Salimi, S.
2009-01-01
In this paper, we investigate continuous-time quantum walk on star graphs. It is shown that quantum central limit theorem for a continuous-time quantum walk on star graphs for N-fold star power graph, which are invariant under the quantum component of adjacency matrix, converges to continuous-time quantum walk on K 2 graphs (complete graph with two vertices) and the probability of observing walk tends to the uniform distribution.
Markov Chain Models for the Stochastic Modeling of Pitting Corrosion
Directory of Open Access Journals (Sweden)
A. Valor
2013-01-01
Full Text Available The stochastic nature of pitting corrosion of metallic structures has been widely recognized. It is assumed that this kind of deterioration retains no memory of the past, so only the current state of the damage influences its future development. This characteristic allows pitting corrosion to be categorized as a Markov process. In this paper, two different models of pitting corrosion, developed using Markov chains, are presented. Firstly, a continuous-time, nonhomogeneous linear growth (pure birth Markov process is used to model external pitting corrosion in underground pipelines. A closed-form solution of the system of Kolmogorov's forward equations is used to describe the transition probability function in a discrete pit depth space. The transition probability function is identified by correlating the stochastic pit depth mean with the empirical deterministic mean. In the second model, the distribution of maximum pit depths in a pitting experiment is successfully modeled after the combination of two stochastic processes: pit initiation and pit growth. Pit generation is modeled as a nonhomogeneous Poisson process, in which induction time is simulated as the realization of a Weibull process. Pit growth is simulated using a nonhomogeneous Markov process. An analytical solution of Kolmogorov's system of equations is also found for the transition probabilities from the first Markov state. Extreme value statistics is employed to find the distribution of maximum pit depths.
DEFF Research Database (Denmark)
Madsen, Line Meldgaard; Fiandaca, Gianluca; Auken, Esben
2017-01-01
The application of time-domain induced polarization (TDIP) is increasing with advances in acquisition techniques, data processing and spectral inversion schemes. An inversion of TDIP data for the spectral Cole-Cole parameters is a non-linear problem, but by applying a 1-D Markov Chain Monte Carlo......-shaped probability distributions with a single maximum, show that the Cole-Cole parameters can be resolved from TDIP data if an acquisition range above two decades in time is applied. Linear correlations between the Cole-Cole parameters are observed and by decreasing the acquisitions ranges, the correlations...
Spectral methods for quantum Markov chains
Energy Technology Data Exchange (ETDEWEB)
Szehr, Oleg
2014-05-08
The aim of this project is to contribute to our understanding of quantum time evolutions, whereby we focus on quantum Markov chains. The latter constitute a natural generalization of the ubiquitous concept of a classical Markov chain to describe evolutions of quantum mechanical systems. We contribute to the theory of such processes by introducing novel methods that allow us to relate the eigenvalue spectrum of the transition map to convergence as well as stability properties of the Markov chain.
Spectral methods for quantum Markov chains
International Nuclear Information System (INIS)
Szehr, Oleg
2014-01-01
The aim of this project is to contribute to our understanding of quantum time evolutions, whereby we focus on quantum Markov chains. The latter constitute a natural generalization of the ubiquitous concept of a classical Markov chain to describe evolutions of quantum mechanical systems. We contribute to the theory of such processes by introducing novel methods that allow us to relate the eigenvalue spectrum of the transition map to convergence as well as stability properties of the Markov chain.
Continuous Time Structural Equation Modeling with R Package ctsem
Directory of Open Access Journals (Sweden)
Charles C. Driver
2017-04-01
Full Text Available We introduce ctsem, an R package for continuous time structural equation modeling of panel (N > 1 and time series (N = 1 data, using full information maximum likelihood. Most dynamic models (e.g., cross-lagged panel models in the social and behavioural sciences are discrete time models. An assumption of discrete time models is that time intervals between measurements are equal, and that all subjects were assessed at the same intervals. Violations of this assumption are often ignored due to the difficulty of accounting for varying time intervals, therefore parameter estimates can be biased and the time course of effects becomes ambiguous. By using stochastic differential equations to estimate an underlying continuous process, continuous time models allow for any pattern of measurement occasions. By interfacing to OpenMx, ctsem combines the flexible specification of structural equation models with the enhanced data gathering opportunities and improved estimation of continuous time models. ctsem can estimate relationships over time for multiple latent processes, measured by multiple noisy indicators with varying time intervals between observations. Within and between effects are estimated simultaneously by modeling both observed covariates and unobserved heterogeneity. Exogenous shocks with different shapes, group differences, higher order diffusion effects and oscillating processes can all be simply modeled. We first introduce and define continuous time models, then show how to specify and estimate a range of continuous time models using ctsem.
A continuous-time control model on production planning network ...
African Journals Online (AJOL)
A continuous-time control model on production planning network. DEA Omorogbe, MIU Okunsebor. Abstract. In this paper, we give a slightly detailed review of Graves and Hollywood model on constant inventory tactical planning model for a job shop. The limitations of this model are pointed out and a continuous time ...
Continuous Time Modeling of the Cross-Lagged Panel Design
Oud, J.H.L.
2002-01-01
Since Newton (1642-1727) continuous time modeling by means of differential equations is the standard approach of dynamic phenomena in natural science. It is argued that most processes in behavioral science also unfold in continuous time and should be analyzed accordingly. After dealing with the
Volchenkov, Dima; Dawin, Jean René
A system for using dice to compose music randomly is known as the musical dice game. The discrete time MIDI models of 804 pieces of classical music written by 29 composers have been encoded into the transition matrices and studied by Markov chains. Contrary to human languages, entropy dominates over redundancy, in the musical dice games based on the compositions of classical music. The maximum complexity is achieved on the blocks consisting of just a few notes (8 notes, for the musical dice games generated over Bach's compositions). First passage times to notes can be used to resolve tonality and feature a composer.
Markov Chain Models for Stochastic Behavior in Resonance Overlap Regions
McCarthy, Morgan; Quillen, Alice
2018-01-01
We aim to predict lifetimes of particles in chaotic zoneswhere resonances overlap. A continuous-time Markov chain model isconstructed using mean motion resonance libration timescales toestimate transition times between resonances. The model is applied todiffusion in the co-rotation region of a planet. For particles begunat low eccentricity, the model is effective for early diffusion, butnot at later time when particles experience close encounters to the planet.
Alejandro Baldominos Gómez; Esperanza Albacete; Ignacio Merrero; Yago Saez
2016-01-01
This paper presents the results and conclusions found when predicting the behavior of gamers in commercial videogames datasets. In particular, it uses Variable-Order Markov (VOM) to build a probabilistic model that is able to use the historic behavior of gamers and to infer what will be their next actions. Being able to predict with accuracy the next user's actions can be of special interest to learn from the behavior of gamers, to make them more engaged and to reduce churn rate. In order to ...
Stability of continuous-time quantum filters with measurement imperfections
Amini, H.; Pellegrini, C.; Rouchon, P.
2014-07-01
The fidelity between the state of a continuously observed quantum system and the state of its associated quantum filter, is shown to be always a submartingale. The observed system is assumed to be governed by a continuous-time Stochastic Master Equation (SME), driven simultaneously by Wiener and Poisson processes and that takes into account incompleteness and errors in measurements. This stability result is the continuous-time counterpart of a similar stability result already established for discrete-time quantum systems and where the measurement imperfections are modelled by a left stochastic matrix.
Integral-Value Models for Outcomes over Continuous Time
DEFF Research Database (Denmark)
Harvey, Charles M.; Østerdal, Lars Peter
Models of preferences between outcomes over continuous time are important for individual, corporate, and social decision making, e.g., medical treatment, infrastructure development, and environmental regulation. This paper presents a foundation for such models. It shows that conditions on prefere...... on preferences between real- or vector-valued outcomes over continuous time are satisfied if and only if the preferences are represented by a value function having an integral form......Models of preferences between outcomes over continuous time are important for individual, corporate, and social decision making, e.g., medical treatment, infrastructure development, and environmental regulation. This paper presents a foundation for such models. It shows that conditions...
Abdulla, Parosh Aziz; Henda, Noomene Ben; Mayr, Richard
2007-01-01
We consider qualitative and quantitative verification problems for infinite-state Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which F can no longer be reached. While all finite Markov chains are trivially decisive (for every set F), this also holds for many classes of infinite Markov chains. Infinite Markov chains which contain a finite attractor are decisive w.r.t. every set F. In part...
Capasso, Vincenzo
2015-01-01
This textbook, now in its third edition, offers a rigorous and self-contained introduction to the theory of continuous-time stochastic processes, stochastic integrals, and stochastic differential equations. Expertly balancing theory and applications, the work features concrete examples of modeling real-world problems from biology, medicine, industrial applications, finance, and insurance using stochastic methods. No previous knowledge of stochastic processes is required. Key topics include: * Markov processes * Stochastic differential equations * Arbitrage-free markets and financial derivatives * Insurance risk * Population dynamics, and epidemics * Agent-based models New to the Third Edition: * Infinitely divisible distributions * Random measures * Levy processes * Fractional Brownian motion * Ergodic theory * Karhunen-Loeve expansion * Additional applications * Additional exercises * Smoluchowski approximation of Langevin systems An Introduction to Continuous-Time Stochastic Processes, Third Editio...
A continuous time formulation of the Regge calculus
International Nuclear Information System (INIS)
Brewin, Leo
1988-01-01
A complete continuous time formulation of the Regge calculus is presented by developing the associated continuous time Regge action. It is shown that the time constraint is, by way of the Bianchi identities conserved by the evolution equations. This analysis leads to an explicit first integral for each of the evolution equations. The dynamical equations of the theory are therefore reduced to a set of first-order differential equations. In this formalism the time constraints reduce to a simple sum of the integration constants. This result is unique to the Regge calculus-there does not appear to be a complete set of first integrals available for the vacuum Einstein equations. (author)
Application of continuous-time random walk to statistical arbitrage
Directory of Open Access Journals (Sweden)
Sergey Osmekhin
2015-01-01
Full Text Available An analytical statistical arbitrage strategy is proposed, where the distribution of the spread is modelled as a continuous-time random walk. Optimal boundaries, computed as a function of the mean and variance of the firstpassage time ofthe spread,maximises an objective function. The predictability of the trading strategy is analysed and contrasted for two forms of continuous-time random walk processes. We found that the waiting-time distribution has a significant impact on the prediction of the expected profit for intraday trading
Continuous-time quantum random walks require discrete space
International Nuclear Information System (INIS)
Manouchehri, K; Wang, J B
2007-01-01
Quantum random walks are shown to have non-intuitive dynamics which makes them an attractive area of study for devising quantum algorithms for long-standing open problems as well as those arising in the field of quantum computing. In the case of continuous-time quantum random walks, such peculiar dynamics can arise from simple evolution operators closely resembling the quantum free-wave propagator. We investigate the divergence of quantum walk dynamics from the free-wave evolution and show that, in order for continuous-time quantum walks to display their characteristic propagation, the state space must be discrete. This behavior rules out many continuous quantum systems as possible candidates for implementing continuous-time quantum random walks
Continuous-time quantum random walks require discrete space
Manouchehri, K.; Wang, J. B.
2007-11-01
Quantum random walks are shown to have non-intuitive dynamics which makes them an attractive area of study for devising quantum algorithms for long-standing open problems as well as those arising in the field of quantum computing. In the case of continuous-time quantum random walks, such peculiar dynamics can arise from simple evolution operators closely resembling the quantum free-wave propagator. We investigate the divergence of quantum walk dynamics from the free-wave evolution and show that, in order for continuous-time quantum walks to display their characteristic propagation, the state space must be discrete. This behavior rules out many continuous quantum systems as possible candidates for implementing continuous-time quantum random walks.
Directory of Open Access Journals (Sweden)
Yanfeng Wang
2017-01-01
Full Text Available This paper investigates the observer-based controller design problem for a class of nonlinear networked control systems with random time-delays. The nonlinearity is assumed to satisfy a global Lipschitz condition and two dependent Markov chains are employed to describe the time-delay from sensor to controller (S-C delay and the time-delay from controller to actuator (C-A delay, respectively. The transition probabilities of S-C delay and C-A delay are both assumed to be partly inaccessible. Sufficient conditions on the stochastic stability for the closed-loop systems are obtained by constructing proper Lyapunov functional. The methods of calculating the controller and the observer gain matrix are also given. Two numerical examples are used to illustrate the effectiveness of the proposed method.
Monte Carlo methods for the reliability analysis of Markov systems
International Nuclear Information System (INIS)
Buslik, A.J.
1985-01-01
This paper presents Monte Carlo methods for the reliability analysis of Markov systems. Markov models are useful in treating dependencies between components. The present paper shows how the adjoint Monte Carlo method for the continuous time Markov process can be derived from the method for the discrete-time Markov process by a limiting process. The straightforward extensions to the treatment of mean unavailability (over a time interval) are given. System unavailabilities can also be estimated; this is done by making the system failed states absorbing, and not permitting repair from them. A forward Monte Carlo method is presented in which the weighting functions are related to the adjoint function. In particular, if the exact adjoint function is known then weighting factors can be constructed such that the exact answer can be obtained with a single Monte Carlo trial. Of course, if the exact adjoint function is known, there is no need to perform the Monte Carlo calculation. However, the formulation is useful since it gives insight into choices of the weight factors which will reduce the variance of the estimator
On Transaction-Cost Models in Continuous-Time Markets
Directory of Open Access Journals (Sweden)
Thomas Poufinas
2015-04-01
Full Text Available Transaction-cost models in continuous-time markets are considered. Given that investors decide to buy or sell at certain time instants, we study the existence of trading strategies that reach a certain final wealth level in continuous-time markets, under the assumption that transaction costs, built in certain recommended ways, have to be paid. Markets prove to behave in manners that resemble those of complete ones for a wide variety of transaction-cost types. The results are important, but not exclusively, for the pricing of options with transaction costs.
Continuous time modeling of panel data by means of SEM
Oud, J.H.L.; Delsing, M.J.M.H.; Montfort, C.A.G.M.; Oud, J.H.L.; Satorra, A.
2010-01-01
After a brief history of continuous time modeling and its implementation in panel analysis by means of structural equation modeling (SEM), the problems of discrete time modeling are discussed in detail. This is done by means of the popular cross-lagged panel design. Next, the exact discrete model
Identification of continuous-time systems from samples of input ...
Indian Academy of Sciences (India)
Abstract. This paper presents an introductory survey of the methods that have been developed for identification of continuous-time systems from samples of input±output data. The two basic approaches may be described as (i) the indirect method, where first a discrete-time model is estimated from the sampled data and then ...
A continuous-time/discrete-time mixed audio-band sigma delta ADC
International Nuclear Information System (INIS)
Liu Yan; Hua Siliang; Wang Donghui; Hou Chaohuan
2011-01-01
This paper introduces a mixed continuous-time/discrete-time, single-loop, fourth-order, 4-bit audio-band sigma delta ADC that combines the benefits of continuous-time and discrete-time circuits, while mitigating the challenges associated with continuous-time design. Measurement results show that the peak SNR of this ADC reaches 100 dB and the total power consumption is less than 30 mW. (semiconductor integrated circuits)
Directory of Open Access Journals (Sweden)
Cédric Beaulac
2017-01-01
Full Text Available We propose to use a supervised machine learning technique to track the location of a mobile agent in real time. Hidden Markov Models are used to build artificial intelligence that estimates the unknown position of a mobile target moving in a defined environment. This narrow artificial intelligence performs two distinct tasks. First, it provides real-time estimation of the mobile agent’s position using the forward algorithm. Second, it uses the Baum–Welch algorithm as a statistical learning tool to gain knowledge of the mobile target. Finally, an experimental environment is proposed, namely, a video game that we use to test our artificial intelligence. We present statistical and graphical results to illustrate the efficiency of our method.
Directory of Open Access Journals (Sweden)
Cheng Gong
2014-01-01
Full Text Available This paper investigates the H∞ filtering problem of discrete singular Markov jump systems (SMJSs with mode-dependent time delay based on T-S fuzzy model. First, by Lyapunov-Krasovskii functional approach, a delay-dependent sufficient condition on H∞-disturbance attenuation is presented, in which both stability and prescribed H∞ performance are required to be achieved for the filtering-error systems. Then, based on the condition, the delay-dependent H∞ filter design scheme for SMJSs with mode-dependent time delay based on T-S fuzzy model is developed in term of linear matrix inequality (LMI. Finally, an example is given to illustrate the effectiveness of the result.
DEFF Research Database (Denmark)
Scholer, Marie; Irving, James; Zibar, Majken Caroline Looms
2012-01-01
We examined to what extent time-lapse crosshole ground-penetrating radar traveltimes, measured during a forced infiltration experiment at the Arreneas field site in Denmark, could help to quantify vadose zone hydraulic properties and their corresponding uncertainties using a Bayesian Markov...... distributions compared with the corresponding priors, which in turn significantly improves knowledge of soil hydraulic properties. Overall, the results obtained clearly demonstrate the value of the information contained in time-lapse GPR data for characterizing vadose zone dynamics.......-chain-Monte-Carlo inversion approach with different priors. The ground-penetrating radar (GPR) geophysical method has the potential to provide valuable information on the hydraulic properties of the vadose zone because of its strong sensitivity to soil water content. In particular, recent evidence has suggested...
Pseudo-Hermitian continuous-time quantum walks
Energy Technology Data Exchange (ETDEWEB)
Salimi, S; Sorouri, A, E-mail: shsalimi@uok.ac.i, E-mail: a.sorouri@uok.ac.i [Department of Physics, University of Kurdistan, PO Box 66177-15175, Sanandaj (Iran, Islamic Republic of)
2010-07-09
In this paper we present a model exhibiting a new type of continuous-time quantum walk (as a quantum-mechanical transport process) on networks, which is described by a non-Hermitian Hamiltonian possessing a real spectrum. We call it pseudo-Hermitian continuous-time quantum walk. We introduce a method to obtain the probability distribution of walk on any vertex and then study a specific system. We observe that the probability distribution on certain vertices increases compared to that of the Hermitian case. This formalism makes the transport process faster and can be useful for search algorithms.
Quantum Markov Chain Mixing and Dissipative Engineering
DEFF Research Database (Denmark)
Kastoryano, Michael James
2012-01-01
This thesis is the fruit of investigations on the extension of ideas of Markov chain mixing to the quantum setting, and its application to problems of dissipative engineering. A Markov chain describes a statistical process where the probability of future events depends only on the state...... of the system at the present point in time, but not on the history of events. Very many important processes in nature are of this type, therefore a good understanding of their behaviour has turned out to be very fruitful for science. Markov chains always have a non-empty set of limiting distributions...... (stationary states). The aim of Markov chain mixing is to obtain (upper and/or lower) bounds on the number of steps it takes for the Markov chain to reach a stationary state. The natural quantum extensions of these notions are density matrices and quantum channels. We set out to develop a general mathematical...
Incomplete Continuous-time Securities Markets with Stochastic Income Volatility
DEFF Research Database (Denmark)
Christensen, Peter Ove; Larsen, Kasper
2014-01-01
We derive closed-form solutions for the equilibrium interest rate and market price of risk processes in an incomplete continuous-time market with uncertainty generated by Brownian motions. The economy has a finite number of heterogeneous exponential utility investors, who receive partially...
Modeling of water treatment plant using timed continuous Petri nets
Nurul Fuady Adhalia, H.; Subiono, Adzkiya, Dieky
2017-08-01
Petri nets represent graphically certain conditions and rules. In this paper, we construct a model of the Water Treatment Plant (WTP) using timed continuous Petri nets. Specifically, we consider that (1) the water pump always active and (2) the water source is always available. After obtaining the model, the flow through the transitions and token conservation laws are calculated.
Incomplete Continuous-Time Securities Markets with Stochastic Income Volatility
DEFF Research Database (Denmark)
Christensen, Peter Ove; Larsen, Kasper
In an incomplete continuous-time securities market governed by Brownian motions, we derive closed-form solutions for the equilibrium risk-free rate and equity premium processes. The economy has a finite number of heterogeneous exponential utility investors, who receive partially unspanned income ...
Asymptotic absolute continuity for perturbed time-dependent ...
Indian Academy of Sciences (India)
R. Narasimhan (Krishtel eMaging) 1461 1996 Oct 15 13:05:22
We study the notion of asymptotic velocity for a class of perturbed time- ... for Mathematical Physics and Stochastics, funded by a grant from the Danish National Research Foun- .... Using (2.4) we can readily continue α(t) to the whole half-axis.
Noise Simulation of Continuous-Time ΣΔ Modulators
International Nuclear Information System (INIS)
Arias, J.; Quintanilla, L.; Bisbal, D.; San Pablo, J.; Enriquez, L.; Vicente, J.; Barbolla, J.
2005-01-01
In this work, an approach for the simulation of the effect of noise sources in the performance of continuous-time ΔΣ modulators is presented. Electrical noise including thermal noise, 1/f noise and clock jitter are included in a simulation program and their impact on the system performance is analyzed
A mean-variance frontier in discrete and continuous time
Bekker, Paul A.
2004-01-01
The paper presents a mean-variance frontier based on dynamic frictionless investment strategies in continuous time. The result applies to a finite number of risky assets whose price process is given by multivariate geometric Brownian motion with deterministically varying coefficients. The derivation
Stability and the structure of continuous-time economic models
Nieuwenhuis, H.J.; Schoonbeek, L.
In this paper we investigate the relationship between the stability of macroeconomic, or macroeconometric, continuous-time models and the structure of the matrices appearing in these models. In particular, we concentrate on dominant-diagonal structures. We derive general stability results for models
A mean-variance frontier in discrete and continuous time
Bekker, Paul A.
2004-01-01
The paper presents a mean-variance frontier based on dynamic frictionless investment strategies in continuous time. The result applies to a finite number of risky assets whose price process is given by multivariate geometric Brownian motion with deterministically varying coefficients. The derivation is based on the solution for the frontier in discrete time. Using the same multiperiod framework as Li and Ng (2000), I provide an alternative derivation and an alternative formulation of the solu...
Continuous Time Portfolio Selection under Conditional Capital at Risk
Directory of Open Access Journals (Sweden)
Gordana Dmitrasinovic-Vidovic
2010-01-01
Full Text Available Portfolio optimization with respect to different risk measures is of interest to both practitioners and academics. For there to be a well-defined optimal portfolio, it is important that the risk measure be coherent and quasiconvex with respect to the proportion invested in risky assets. In this paper we investigate one such measure—conditional capital at risk—and find the optimal strategies under this measure, in the Black-Scholes continuous time setting, with time dependent coefficients.
Approximating Markov Chains: What and why
International Nuclear Information System (INIS)
Pincus, S.
1996-01-01
Much of the current study of dynamical systems is focused on geometry (e.g., chaos and bifurcations) and ergodic theory. Yet dynamical systems were originally motivated by an attempt to open-quote open-quote solve,close-quote close-quote or at least understand, a discrete-time analogue of differential equations. As such, numerical, analytical solution techniques for dynamical systems would seem desirable. We discuss an approach that provides such techniques, the approximation of dynamical systems by suitable finite state Markov Chains. Steady state distributions for these Markov Chains, a straightforward calculation, will converge to the true dynamical system steady state distribution, with appropriate limit theorems indicated. Thus (i) approximation by a computable, linear map holds the promise of vastly faster steady state solutions for nonlinear, multidimensional differential equations; (ii) the solution procedure is unaffected by the presence or absence of a probability density function for the attractor, entirely skirting singularity, fractal/multifractal, and renormalization considerations. The theoretical machinery underpinning this development also implies that under very general conditions, steady state measures are weakly continuous with control parameter evolution. This means that even though a system may change periodicity, or become chaotic in its limiting behavior, such statistical parameters as the mean, standard deviation, and tail probabilities change continuously, not abruptly with system evolution. copyright 1996 American Institute of Physics
Deep Brain Stimulation, Continuity over Time, and the True Self.
Nyholm, Sven; O'Neill, Elizabeth
2016-10-01
One of the topics that often comes up in ethical discussions of deep brain stimulation (DBS) is the question of what impact DBS has, or might have, on the patient's self. This is often understood as a question of whether DBS poses a threat to personal identity, which is typically understood as having to do with psychological and/or narrative continuity over time. In this article, we argue that the discussion of whether DBS is a threat to continuity over time is too narrow. There are other questions concerning DBS and the self that are overlooked in discussions exclusively focusing on psychological and/or narrative continuity. For example, it is also important to investigate whether DBS might sometimes have a positive (e.g., a rehabilitating) effect on the patient's self. To widen the discussion of DBS, so as to make it encompass a broader range of considerations that bear on DBS's impact on the self, we identify six features of the commonly used concept of a person's "true self." We apply these six features to the relation between DBS and the self. And we end with a brief discussion of the role DBS might play in treating otherwise treatment-refractory anorexia nervosa. This further highlights the importance of discussing both continuity over time and the notion of the true self.
A stochastic surplus production model in continuous time
DEFF Research Database (Denmark)
Pedersen, Martin Wæver; Berg, Casper Willestofte
2017-01-01
surplus production model in continuous time (SPiCT), which in addition to stock dynamics also models the dynamics of the fisheries. This enables error in the catch process to be reflected in the uncertainty of estimated model parameters and management quantities. Benefits of the continuous-time state......Surplus production modelling has a long history as a method for managing data-limited fish stocks. Recent advancements have cast surplus production models as state-space models that separate random variability of stock dynamics from error in observed indices of biomass. We present a stochastic......-space model formulation include the ability to provide estimates of exploitable biomass and fishing mortality at any point in time from data sampled at arbitrary and possibly irregular intervals. We show in a simulation that the ability to analyse subannual data can increase the effective sample size...
Continuous real-time water information: an important Kansas resource
Loving, Brian L.; Putnam, James E.; Turk, Donita M.
2014-01-01
Continuous real-time information on streams, lakes, and groundwater is an important Kansas resource that can safeguard lives and property, and ensure adequate water resources for a healthy State economy. The U.S. Geological Survey (USGS) operates approximately 230 water-monitoring stations at Kansas streams, lakes, and groundwater sites. Most of these stations are funded cooperatively in partnerships with local, tribal, State, or other Federal agencies. The USGS real-time water-monitoring network provides long-term, accurate, and objective information that meets the needs of many customers. Whether the customer is a water-management or water-quality agency, an emergency planner, a power or navigational official, a farmer, a canoeist, or a fisherman, all can benefit from the continuous real-time water information gathered by the USGS.
Martingale Regressions for a Continuous Time Model of Exchange Rates
Guo, Zi-Yi
2017-01-01
One of the daunting problems in international finance is the weak explanatory power of existing theories of the nominal exchange rates, the so-called “foreign exchange rate determination puzzle”. We propose a continuous-time model to study the impact of order flow on foreign exchange rates. The model is estimated by a newly developed econometric tool based on a time-change sampling from calendar to volatility time. The estimation results indicate that the effect of order flow on exchange rate...
Verification of Continuous Dynamical Systems by Timed Automata
DEFF Research Database (Denmark)
Sloth, Christoffer; Wisniewski, Rafael
2011-01-01
This paper presents a method for abstracting continuous dynamical systems by timed automata. The abstraction is based on partitioning the state space of a dynamical system using positive invariant sets, which form cells that represent locations of a timed automaton. The abstraction is intended......, which is generated utilizing sub-level sets of Lyapunov functions, as they are positive invariant sets. It is shown that this partition generates sound and complete abstractions. Furthermore, the complete abstractions can be composed of multiple timed automata, allowing parallelization...
Coupled continuous time-random walks in quenched random environment
Magdziarz, M.; Szczotka, W.
2018-02-01
We introduce a coupled continuous-time random walk with coupling which is characteristic for Lévy walks. Additionally we assume that the walker moves in a quenched random environment, i.e. the site disorder at each lattice point is fixed in time. We analyze the scaling limit of such a random walk. We show that for large times the behaviour of the analyzed process is exactly the same as in the case of uncoupled quenched trap model for Lévy flights.
Correlating defect density with growth time in continuous graphene films.
Kang, Cheong; Jung, Da Hee; Nam, Ji Eun; Lee, Jin Seok
2014-12-01
We report that graphene flakes and films which were synthesized by copper-catalyzed atmospheric pressure chemical vapor deposition (APCVD) method using a mixture of Ar, H2, and CH4 gases. It was found that variations in the reaction parameters, such as reaction temperature, annealing time, and growth time, influenced the domain size of as-grown graphene. Besides, the reaction parameters influenced the number of layers, degree of defects and uniformity of the graphene films. The increase in growth temperature and annealing time tends to accelerate the graphene growth rate and increase the diffusion length, respectively, thereby increasing the average size of graphene domains. In addition, we confirmed that the number of pinholes reduced with increase in the growth time. Micro-Raman analysis of the as-grown graphene films confirmed that the continuous graphene monolayer film with low defects and high uniformity could be obtained with prolonged reaction time, under the appropriate annealing time and growth temperature.
DEFF Research Database (Denmark)
Justesen, Jørn
2005-01-01
A simple construction of two-dimensional (2-D) fields is presented. Rows and columns are outcomes of the same Markov chain. The entropy can be calculated explicitly.......A simple construction of two-dimensional (2-D) fields is presented. Rows and columns are outcomes of the same Markov chain. The entropy can be calculated explicitly....
DEFF Research Database (Denmark)
Hobolth, Asger; Stone, Eric
2009-01-01
computational finance to human genetics and genomics. A common theme among these diverse applications is the need to simulate sample paths of a CTMC conditional on realized data that is discretely observed. Here we present a general solution to this sampling problem when the CTMC is defined on a discrete....... In doing so, we show that no method dominates the others across all model specifications, and we give explicit proof of which method prevails for any given Q, T, and endpoints. Finally, we introduce and compare three applications of CTMCs to demonstrate the pitfalls of choosing an inefficient sampler....
Continuous time Boolean modeling for biological signaling: application of Gillespie algorithm.
Stoll, Gautier; Viara, Eric; Barillot, Emmanuel; Calzone, Laurence
2012-08-29
Mathematical modeling is used as a Systems Biology tool to answer biological questions, and more precisely, to validate a network that describes biological observations and predict the effect of perturbations. This article presents an algorithm for modeling biological networks in a discrete framework with continuous time. There exist two major types of mathematical modeling approaches: (1) quantitative modeling, representing various chemical species concentrations by real numbers, mainly based on differential equations and chemical kinetics formalism; (2) and qualitative modeling, representing chemical species concentrations or activities by a finite set of discrete values. Both approaches answer particular (and often different) biological questions. Qualitative modeling approach permits a simple and less detailed description of the biological systems, efficiently describes stable state identification but remains inconvenient in describing the transient kinetics leading to these states. In this context, time is represented by discrete steps. Quantitative modeling, on the other hand, can describe more accurately the dynamical behavior of biological processes as it follows the evolution of concentration or activities of chemical species as a function of time, but requires an important amount of information on the parameters difficult to find in the literature. Here, we propose a modeling framework based on a qualitative approach that is intrinsically continuous in time. The algorithm presented in this article fills the gap between qualitative and quantitative modeling. It is based on continuous time Markov process applied on a Boolean state space. In order to describe the temporal evolution of the biological process we wish to model, we explicitly specify the transition rates for each node. For that purpose, we built a language that can be seen as a generalization of Boolean equations. Mathematically, this approach can be translated in a set of ordinary differential
Continuous equilibrium scores: factoring in the time before a fall.
Wood, Scott J; Reschke, Millard F; Owen Black, F
2012-07-01
The equilibrium (EQ) score commonly used in computerized dynamic posturography is normalized between 0 and 100, with falls assigned a score of 0. The resulting mixed discrete-continuous distribution limits certain statistical analyses and treats all trials with falls equally. We propose a simple modification of the formula in which peak-to-peak sway data from trials with falls is scaled according the percent of the trial completed to derive a continuous equilibrium (cEQ) score. The cEQ scores for trials without falls remain unchanged from the original methodology. The cEQ factors in the time before a fall and results in a continuous variable retaining the central tendencies of the original EQ distribution. A random set of 5315 Sensory Organization Test trials were pooled that included 81 falls. A comparison of the original and cEQ distributions and their rank ordering demonstrated that trials with falls continue to constitute the lower range of scores with the cEQ methodology. The area under the receiver operating characteristic curve (0.997) demonstrates that the cEQ retained near-perfect discrimination between trials with and without falls. We conclude that the cEQ score provides the ability to discriminate between ballistic falls from falls that occur later in the trial. This approach of incorporating time and sway magnitude can be easily extended to enhance other balance tests that include fall data or incomplete trials. Copyright © 2012 Elsevier B.V. All rights reserved.
Introducing the Dimensional Continuous Space-Time Theory
International Nuclear Information System (INIS)
Martini, Luiz Cesar
2013-01-01
This article is an introduction to a new theory. The name of the theory is justified by the dimensional description of the continuous space-time of the matter, energy and empty space, that gathers all the real things that exists in the universe. The theory presents itself as the consolidation of the classical, quantum and relativity theories. A basic equation that describes the formation of the Universe, relating time, space, matter, energy and movement, is deduced. The four fundamentals physics constants, light speed in empty space, gravitational constant, Boltzmann's constant and Planck's constant and also the fundamentals particles mass, the electrical charges, the energies, the empty space and time are also obtained from this basic equation. This theory provides a new vision of the Big-Bang and how the galaxies, stars, black holes and planets were formed. Based on it, is possible to have a perfect comprehension of the duality between wave-particle, which is an intrinsic characteristic of the matter and energy. It will be possible to comprehend the formation of orbitals and get the equationing of atomics orbits. It presents a singular comprehension of the mass relativity, length and time. It is demonstrated that the continuous space-time is tridimensional, inelastic and temporally instantaneous, eliminating the possibility of spatial fold, slot space, worm hole, time travels and parallel universes. It is shown that many concepts, like dark matter and strong forces, that hypothetically keep the cohesion of the atomics nucleons, are without sense.
Continuous-Time Symmetric Hopfield Nets are Computationally Universal
Czech Academy of Sciences Publication Activity Database
Šíma, Jiří; Orponen, P.
2003-01-01
Roč. 15, č. 3 (2003), s. 693-733 ISSN 0899-7667 R&D Projects: GA AV ČR IAB2030007; GA ČR GA201/02/1456 Institutional research plan: AV0Z1030915 Keywords : continuous-time Hopfield network * Liapunov function * analog computation * computational power * Turing universality Subject RIV: BA - General Mathematics Impact factor: 2.747, year: 2003
Estimation of Continuous Time Models in Economics: an Overview
Clifford R. Wymer
2009-01-01
The dynamics of economic behaviour is often developed in theory as a continuous time system. Rigorous estimation and testing of such systems, and the analysis of some aspects of their properties, is of particular importance in distinguishing between competing hypotheses and the resulting models. The consequences for the international economy during the past eighteen months of failures in the financial sector, and particularly the banking sector, make it essential that the dynamics of financia...
International Nuclear Information System (INIS)
Liang Jinling; Cao Jinde
2004-01-01
First, convergence of continuous-time Bidirectional Associative Memory (BAM) neural networks are studied. By using Lyapunov functionals and some analysis technique, the delay-independent sufficient conditions are obtained for the networks to converge exponentially toward the equilibrium associated with the constant input sources. Second, discrete-time analogues of the continuous-time BAM networks are formulated and studied. It is shown that the convergence characteristics of the continuous-time systems are preserved by the discrete-time analogues without any restriction imposed on the uniform discretionary step size. An illustrative example is given to demonstrate the effectiveness of the obtained results
The problem with time in mixed continuous/discrete time modelling
Rovers, K.C.; Kuper, Jan; Smit, Gerardus Johannes Maria
The design of cyber-physical systems requires the use of mixed continuous time and discrete time models. Current modelling tools have problems with time transformations (such as a time delay) or multi-rate systems. We will present a novel approach that implements signals as functions of time,
Markov chains analytic and Monte Carlo computations
Graham, Carl
2014-01-01
Markov Chains: Analytic and Monte Carlo Computations introduces the main notions related to Markov chains and provides explanations on how to characterize, simulate, and recognize them. Starting with basic notions, this book leads progressively to advanced and recent topics in the field, allowing the reader to master the main aspects of the classical theory. This book also features: Numerous exercises with solutions as well as extended case studies.A detailed and rigorous presentation of Markov chains with discrete time and state space.An appendix presenting probabilistic notions that are nec
Dynamical continuous time random Lévy flights
Liu, Jian; Chen, Xiaosong
2016-03-01
The Lévy flights' diffusive behavior is studied within the framework of the dynamical continuous time random walk (DCTRW) method, while the nonlinear friction is introduced in each step. Through the DCTRW method, Lévy random walker in each step flies by obeying the Newton's Second Law while the nonlinear friction f(v) = - γ0v - γ2v3 being considered instead of Stokes friction. It is shown that after introducing the nonlinear friction, the superdiffusive Lévy flights converges, behaves localization phenomenon with long time limit, but for the Lévy index μ = 2 case, it is still Brownian motion.
Infinite time interval backward stochastic differential equations with continuous coefficients.
Zong, Zhaojun; Hu, Feng
2016-01-01
In this paper, we study the existence theorem for [Formula: see text] [Formula: see text] solutions to a class of 1-dimensional infinite time interval backward stochastic differential equations (BSDEs) under the conditions that the coefficients are continuous and have linear growths. We also obtain the existence of a minimal solution. Furthermore, we study the existence and uniqueness theorem for [Formula: see text] [Formula: see text] solutions of infinite time interval BSDEs with non-uniformly Lipschitz coefficients. It should be pointed out that the assumptions of this result is weaker than that of Theorem 3.1 in Zong (Turkish J Math 37:704-718, 2013).
Quantum trajectories and measurements in continuous time. The diffusive case
International Nuclear Information System (INIS)
Barchielli, Alberto; Gregoratti, Matteo
2009-01-01
continuous time for quantum systems. The two-level atom is again used to introduce and study an example of feedback based on the observed output. (orig.)
Coaction versus reciprocity in continuous-time models of cooperation.
van Doorn, G Sander; Riebli, Thomas; Taborsky, Michael
2014-09-07
Cooperating animals frequently show closely coordinated behaviours organized by a continuous flow of information between interacting partners. Such real-time coaction is not captured by the iterated prisoner's dilemma and other discrete-time reciprocal cooperation games, which inherently feature a delay in information exchange. Here, we study the evolution of cooperation when individuals can dynamically respond to each other's actions. We develop continuous-time analogues of iterated-game models and describe their dynamics in terms of two variables, the propensity of individuals to initiate cooperation (altruism) and their tendency to mirror their partner's actions (coordination). These components of cooperation stabilize at an evolutionary equilibrium or show oscillations, depending on the chosen payoff parameters. Unlike reciprocal altruism, cooperation by coaction does not require that those willing to initiate cooperation pay in advance for uncertain future benefits. Correspondingly, we show that introducing a delay to information transfer between players is equivalent to increasing the cost of cooperation. Cooperative coaction can therefore evolve much more easily than reciprocal cooperation. When delays entirely prevent coordination, we recover results from the discrete-time alternating prisoner's dilemma, indicating that coaction and reciprocity are connected by a continuum of opportunities for real-time information exchange. Copyright © 2014 Elsevier Ltd. All rights reserved.
Zhao, Zhibiao
2011-06-01
We address the nonparametric model validation problem for hidden Markov models with partially observable variables and hidden states. We achieve this goal by constructing a nonparametric simultaneous confidence envelope for transition density function of the observable variables and checking whether the parametric density estimate is contained within such an envelope. Our specification test procedure is motivated by a functional connection between the transition density of the observable variables and the Markov transition kernel of the hidden states. Our approach is applicable for continuous time diffusion models, stochastic volatility models, nonlinear time series models, and models with market microstructure noise.
Semi-Markov Arnason-Schwarz models.
King, Ruth; Langrock, Roland
2016-06-01
We consider multi-state capture-recapture-recovery data where observed individuals are recorded in a set of possible discrete states. Traditionally, the Arnason-Schwarz model has been fitted to such data where the state process is modeled as a first-order Markov chain, though second-order models have also been proposed and fitted to data. However, low-order Markov models may not accurately represent the underlying biology. For example, specifying a (time-independent) first-order Markov process involves the assumption that the dwell time in each state (i.e., the duration of a stay in a given state) has a geometric distribution, and hence that the modal dwell time is one. Specifying time-dependent or higher-order processes provides additional flexibility, but at the expense of a potentially significant number of additional model parameters. We extend the Arnason-Schwarz model by specifying a semi-Markov model for the state process, where the dwell-time distribution is specified more generally, using, for example, a shifted Poisson or negative binomial distribution. A state expansion technique is applied in order to represent the resulting semi-Markov Arnason-Schwarz model in terms of a simpler and computationally tractable hidden Markov model. Semi-Markov Arnason-Schwarz models come with only a very modest increase in the number of parameters, yet permit a significantly more flexible state process. Model selection can be performed using standard procedures, and in particular via the use of information criteria. The semi-Markov approach allows for important biological inference to be drawn on the underlying state process, for example, on the times spent in the different states. The feasibility of the approach is demonstrated in a simulation study, before being applied to real data corresponding to house finches where the states correspond to the presence or absence of conjunctivitis. © 2015, The International Biometric Society.
Markov models for digraph panel data : Monte Carlo-based derivative estimation
Schweinberger, Michael; Snijders, Tom A. B.
2007-01-01
A parametric, continuous-time Markov model for digraph panel data is considered. The parameter is estimated by the method of moments. A convenient method for estimating the variance-covariance matrix of the moment estimator relies on the delta method, requiring the Jacobian matrix-that is, the
CSL Model Checking Algorithms for Infinite-state Structured Markov chains
Remke, Anne Katharina Ingrid; Haverkort, Boudewijn R.H.M.; Raskin, J.-F.; Thiagarajan, P.S.
2007-01-01
Jackson queueing networks (JQNs) are a very general class of queueing networks that find their application in a variety of settings. The state space of the continuous-time Markov chain (CTMC) that underlies such a JQN, is highly structured, however, of infinite size in as many dimensions as there
The space-time model according to dimensional continuous space-time theory
International Nuclear Information System (INIS)
Martini, Luiz Cesar
2014-01-01
This article results from the Dimensional Continuous Space-Time Theory for which the introductory theoretician was presented in [1]. A theoretical model of the Continuous Space-Time is presented. The wave equation of time into absolutely stationary empty space referential will be described in detail. The complex time, that is the time fixed on the infinite phase time speed referential, is deduced from the New View of Relativity Theory that is being submitted simultaneously with this article in this congress. Finally considering the inseparable Space-Time is presented the duality equation wave-particle.
Distinct timing mechanisms produce discrete and continuous movements.
Directory of Open Access Journals (Sweden)
Raoul Huys
2008-04-01
Full Text Available The differentiation of discrete and continuous movement is one of the pillars of motor behavior classification. Discrete movements have a definite beginning and end, whereas continuous movements do not have such discriminable end points. In the past decade there has been vigorous debate whether this classification implies different control processes. This debate up until the present has been empirically based. Here, we present an unambiguous non-empirical classification based on theorems in dynamical system theory that sets discrete and continuous movements apart. Through computational simulations of representative modes of each class and topological analysis of the flow in state space, we show that distinct control mechanisms underwrite discrete and fast rhythmic movements. In particular, we demonstrate that discrete movements require a time keeper while fast rhythmic movements do not. We validate our computational findings experimentally using a behavioral paradigm in which human participants performed finger flexion-extension movements at various movement paces and under different instructions. Our results demonstrate that the human motor system employs different timing control mechanisms (presumably via differential recruitment of neural subsystems to accomplish varying behavioral functions such as speed constraints.
Algebraic decay in self-similar Markov chains
International Nuclear Information System (INIS)
Hanson, J.D.; Cary, J.R.; Meiss, J.D.
1985-01-01
A continuous-time Markov chain is used to model motion in the neighborhood of a critical invariant circle for a Hamiltonian map. States in the infinite chain represent successive rational approximants to the frequency of the invariant circle. For the case of a noble frequency, the chain is self-similar and the nonlinear integral equation for the first passage time distribution is solved exactly. The asymptotic distribution is a power law times a function periodic in the logarithm of the time. For parameters relevant to the critical noble circle, the decay proceeds as t/sup -4.05/
Algebraic decay in self-similar Markov chains
International Nuclear Information System (INIS)
Hanson, J.D.; Cary, J.R.; Meiss, J.D.
1984-10-01
A continuous time Markov chain is used to model motion in the neighborhood of a critical noble invariant circle in an area-preserving map. States in the infinite chain represent successive rational approximants to the frequency of the invariant circle. The nonlinear integral equation for the first passage time distribution is solved exactly. The asymptotic distribution is a power law times a function periodic in the logarithm of the time. For parameters relevant to Hamiltonian systems the decay proceeds as t -4 05
Stochastic modeling of pitting corrosion in underground pipelines using Markov chains
Energy Technology Data Exchange (ETDEWEB)
Velazquez, J.C.; Caleyo, F.; Hallen, J.M.; Araujo, J.E. [Instituto Politecnico Nacional (IPN), Mexico D.F. (Mexico). Escuela Superior de Ingenieria Quimica e Industrias Extractivas (ESIQIE); Valor, A. [Universidad de La Habana, La Habana (Cuba)
2009-07-01
A non-homogenous, linear growth (pure birth) Markov process, with discrete states in continuous time, has been used to model external pitting corrosion in underground pipelines. The transition probability function for the pit depth is obtained from the analytical solution of the forward Kolmogorov equations for this process. The parameters of the transition probability function between depth states can be identified from the observed time evolution of the mean of the pit depth distribution. Monte Carlo simulations were used to predict the time evolution of the mean value of the pit depth distribution in soils with different physicochemical characteristics. The simulated distributions have been used to create an empirical Markov-chain-based stochastic model for predicting the evolution of pitting corrosion from the observed properties of the soil in contact with the pipeline. Real- life case studies, involving simulated and measured pit depth distributions are presented to illustrate the application of the proposed Markov chains model. (author)
A continuous-time neural model for sequential action.
Kachergis, George; Wyatte, Dean; O'Reilly, Randall C; de Kleijn, Roy; Hommel, Bernhard
2014-11-05
Action selection, planning and execution are continuous processes that evolve over time, responding to perceptual feedback as well as evolving top-down constraints. Existing models of routine sequential action (e.g. coffee- or pancake-making) generally fall into one of two classes: hierarchical models that include hand-built task representations, or heterarchical models that must learn to represent hierarchy via temporal context, but thus far lack goal-orientedness. We present a biologically motivated model of the latter class that, because it is situated in the Leabra neural architecture, affords an opportunity to include both unsupervised and goal-directed learning mechanisms. Moreover, we embed this neurocomputational model in the theoretical framework of the theory of event coding (TEC), which posits that actions and perceptions share a common representation with bidirectional associations between the two. Thus, in this view, not only does perception select actions (along with task context), but actions are also used to generate perceptions (i.e. intended effects). We propose a neural model that implements TEC to carry out sequential action control in hierarchically structured tasks such as coffee-making. Unlike traditional feedforward discrete-time neural network models, which use static percepts to generate static outputs, our biological model accepts continuous-time inputs and likewise generates non-stationary outputs, making short-timescale dynamic predictions. © 2014 The Author(s) Published by the Royal Society. All rights reserved.
Recommender engine for continuous-time quantum Monte Carlo methods
Huang, Li; Yang, Yi-feng; Wang, Lei
2017-03-01
Recommender systems play an essential role in the modern business world. They recommend favorable items such as books, movies, and search queries to users based on their past preferences. Applying similar ideas and techniques to Monte Carlo simulations of physical systems boosts their efficiency without sacrificing accuracy. Exploiting the quantum to classical mapping inherent in the continuous-time quantum Monte Carlo methods, we construct a classical molecular gas model to reproduce the quantum distributions. We then utilize powerful molecular simulation techniques to propose efficient quantum Monte Carlo updates. The recommender engine approach provides a general way to speed up the quantum impurity solvers.
Continuous-time quantum walks on multilayer dendrimer networks
Galiceanu, Mircea; Strunz, Walter T.
2016-08-01
We consider continuous-time quantum walks (CTQWs) on multilayer dendrimer networks (MDs) and their application to quantum transport. A detailed study of properties of CTQWs is presented and transport efficiency is determined in terms of the exact and average return probabilities. The latter depends only on the eigenvalues of the connectivity matrix, which even for very large structures allows a complete analytical solution for this particular choice of network. In the case of MDs we observe an interplay between strong localization effects, due to the dendrimer topology, and good efficiency from the linear segments. We show that quantum transport is enhanced by interconnecting more layers of dendrimers.
Measuring and modelling occupancy time in NHS continuing healthcare
Directory of Open Access Journals (Sweden)
Millard Peter H
2011-06-01
Full Text Available Abstract Background Due to increasing demand and financial constraints, NHS continuing healthcare systems seek to find better ways of forecasting demand and budgeting for care. This paper investigates two areas of concern, namely, how long existing patients stay in service and the number of patients that are likely to be still in care after a period of time. Methods An anonymised dataset containing information for all funded admissions to placement and home care in the NHS continuing healthcare system was provided by 26 (out of 31 London primary care trusts. The data related to 11289 patients staying in placement and home care between 1 April 2005 and 31 May 2008 were first analysed. Using a methodology based on length of stay (LoS modelling, we captured the distribution of LoS of patients to estimate the probability of a patient staying in care over a period of time. Using the estimated probabilities we forecasted the number of patients that are likely to be still in care after a period of time (e.g. monthly. Results We noticed that within the NHS continuing healthcare system there are three main categories of patients. Some patients are discharged after a short stay (few days, some others staying for few months and the third category of patients staying for a long period of time (years. Some variations in proportions of discharge and transition between types of care as well as between care groups (e.g. palliative, functional mental health were observed. A close agreement of the observed and the expected numbers of patients suggests a good prediction model. Conclusions The model was tested for care groups within the NHS continuing healthcare system in London to support Primary Care Trusts in budget planning and improve their responsiveness to meet the increasing demand under limited availability of resources. Its applicability can be extended to other types of care, such as hospital care and re-ablement. Further work will be geared towards
Price discovery in a continuous-time setting
DEFF Research Database (Denmark)
Dias, Gustavo Fruet; Fernandes, Marcelo; Scherrer, Cristina
We formulate a continuous-time price discovery model in which the price discovery measure varies (stochastically) at daily frequency. We estimate daily measures of price discovery using a kernel-based OLS estimator instead of running separate daily VECM regressions as standard in the literature. We...... show that our estimator is not only consistent, but also outperforms the standard daily VECM in finite samples. We illustrate our theoretical findings by studying the price discovery process of 10 actively traded stocks in the U.S. from 2007 to 2013....
Discrete and continuous time dynamic mean-variance analysis
Reiss, Ariane
1999-01-01
Contrary to static mean-variance analysis, very few papers have dealt with dynamic mean-variance analysis. Here, the mean-variance efficient self-financing portfolio strategy is derived for n risky assets in discrete and continuous time. In the discrete setting, the resulting portfolio is mean-variance efficient in a dynamic sense. It is shown that the optimal strategy for n risky assets may be dominated if the expected terminal wealth is constrained to exactly attain a certain goal instead o...
Markov Chain Ontology Analysis (MCOA).
Frost, H Robert; McCray, Alexa T
2012-02-03
Biomedical ontologies have become an increasingly critical lens through which researchers analyze the genomic, clinical and bibliographic data that fuels scientific research. Of particular relevance are methods, such as enrichment analysis, that quantify the importance of ontology classes relative to a collection of domain data. Current analytical techniques, however, remain limited in their ability to handle many important types of structural complexity encountered in real biological systems including class overlaps, continuously valued data, inter-instance relationships, non-hierarchical relationships between classes, semantic distance and sparse data. In this paper, we describe a methodology called Markov Chain Ontology Analysis (MCOA) and illustrate its use through a MCOA-based enrichment analysis application based on a generative model of gene activation. MCOA models the classes in an ontology, the instances from an associated dataset and all directional inter-class, class-to-instance and inter-instance relationships as a single finite ergodic Markov chain. The adjusted transition probability matrix for this Markov chain enables the calculation of eigenvector values that quantify the importance of each ontology class relative to other classes and the associated data set members. On both controlled Gene Ontology (GO) data sets created with Escherichia coli, Drosophila melanogaster and Homo sapiens annotations and real gene expression data extracted from the Gene Expression Omnibus (GEO), the MCOA enrichment analysis approach provides the best performance of comparable state-of-the-art methods. A methodology based on Markov chain models and network analytic metrics can help detect the relevant signal within large, highly interdependent and noisy data sets and, for applications such as enrichment analysis, has been shown to generate superior performance on both real and simulated data relative to existing state-of-the-art approaches.
Finite time convergent learning law for continuous neural networks.
Chairez, Isaac
2014-02-01
This paper addresses the design of a discontinuous finite time convergent learning law for neural networks with continuous dynamics. The neural network was used here to obtain a non-parametric model for uncertain systems described by a set of ordinary differential equations. The source of uncertainties was the presence of some external perturbations and poor knowledge of the nonlinear function describing the system dynamics. A new adaptive algorithm based on discontinuous algorithms was used to adjust the weights of the neural network. The adaptive algorithm was derived by means of a non-standard Lyapunov function that is lower semi-continuous and differentiable in almost the whole space. A compensator term was included in the identifier to reject some specific perturbations using a nonlinear robust algorithm. Two numerical examples demonstrated the improvements achieved by the learning algorithm introduced in this paper compared to classical schemes with continuous learning methods. The first one dealt with a benchmark problem used in the paper to explain how the discontinuous learning law works. The second one used the methane production model to show the benefits in engineering applications of the learning law proposed in this paper. Copyright © 2013 Elsevier Ltd. All rights reserved.
Browne, William J; Steele, Fiona; Golalizadeh, Mousa; Green, Martin J
2009-06-01
We consider the application of Markov chain Monte Carlo (MCMC) estimation methods to random-effects models and in particular the family of discrete time survival models. Survival models can be used in many situations in the medical and social sciences and we illustrate their use through two examples that differ in terms of both substantive area and data structure. A multilevel discrete time survival analysis involves expanding the data set so that the model can be cast as a standard multilevel binary response model. For such models it has been shown that MCMC methods have advantages in terms of reducing estimate bias. However, the data expansion results in very large data sets for which MCMC estimation is often slow and can produce chains that exhibit poor mixing. Any way of improving the mixing will result in both speeding up the methods and more confidence in the estimates that are produced. The MCMC methodological literature is full of alternative algorithms designed to improve mixing of chains and we describe three reparameterization techniques that are easy to implement in available software. We consider two examples of multilevel survival analysis: incidence of mastitis in dairy cattle and contraceptive use dynamics in Indonesia. For each application we show where the reparameterization techniques can be used and assess their performance.
Hofstede, ter F.; Wedel, M.
1998-01-01
This study investigates the effects of time aggregation in discrete and continuous-time hazard models. A Monte Carlo study is conducted in which data are generated according to various continuous and discrete-time processes, and aggregated into daily, weekly and monthly intervals. These data are
janssen, Anja; Segers, Johan
2013-01-01
The extremes of a univariate Markov chain with regularly varying stationary marginal distribution and asymptotically linear behavior are known to exhibit a multiplicative random walk structure called the tail chain. In this paper we extend this fact to Markov chains with multivariate regularly varying marginal distributions in Rd. We analyze both the forward and the backward tail process and show that they mutually determine each other through a kind of adjoint relation. In ...
Correlated continuous time random walk and option pricing
Lv, Longjin; Xiao, Jianbin; Fan, Liangzhong; Ren, Fuyao
2016-04-01
In this paper, we study a correlated continuous time random walk (CCTRW) with averaged waiting time, whose probability density function (PDF) is proved to follow stretched Gaussian distribution. Then, we apply this process into option pricing problem. Supposing the price of the underlying is driven by this CCTRW, we find this model captures the subdiffusive characteristic of financial markets. By using the mean self-financing hedging strategy, we obtain the closed-form pricing formulas for a European option with and without transaction costs, respectively. At last, comparing the obtained model with the classical Black-Scholes model, we find the price obtained in this paper is higher than that obtained from the Black-Scholes model. A empirical analysis is also introduced to confirm the obtained results can fit the real data well.
Observation uncertainty in reversible Markov chains.
Metzner, Philipp; Weber, Marcus; Schütte, Christof
2010-09-01
In many applications one is interested in finding a simplified model which captures the essential dynamical behavior of a real life process. If the essential dynamics can be assumed to be (approximately) memoryless then a reasonable choice for a model is a Markov model whose parameters are estimated by means of Bayesian inference from an observed time series. We propose an efficient Monte Carlo Markov chain framework to assess the uncertainty of the Markov model and related observables. The derived Gibbs sampler allows for sampling distributions of transition matrices subject to reversibility and/or sparsity constraints. The performance of the suggested sampling scheme is demonstrated and discussed for a variety of model examples. The uncertainty analysis of functions of the Markov model under investigation is discussed in application to the identification of conformations of the trialanine molecule via Robust Perron Cluster Analysis (PCCA+) .
On properties of continuous-time random walks with non-Poissonian jump-times
International Nuclear Information System (INIS)
Villarroel, Javier; Montero, Miquel
2009-01-01
The usual development of the continuous-time random walk (CTRW) proceeds by assuming that the present is one of the jumping times. Under this restrictive assumption integral equations for the propagator and mean escape times have been derived. We generalize these results to the case when the present is an arbitrary time by recourse to renewal theory. The case of Erlang distributed times is analyzed in detail. Several concrete examples are considered.
Directory of Open Access Journals (Sweden)
Sofia Siachalou
2015-03-01
Full Text Available Vegetation monitoring and mapping based on multi-temporal imagery has recently received much attention due to the plethora of medium-high spatial resolution satellites and the improved classification accuracies attained compared to uni-temporal approaches. Efficient image processing strategies are needed to exploit the phenological information present in temporal image sequences and to limit data redundancy and computational complexity. Within this framework, we implement the theory of Hidden Markov Models in crop classification, based on the time-series analysis of phenological states, inferred by a sequence of remote sensing observations. More specifically, we model the dynamics of vegetation over an agricultural area of Greece, characterized by spatio-temporal heterogeneity and small-sized fields, using RapidEye and Landsat ETM+ imagery. In addition, the classification performance of image sequences with variable spatial and temporal characteristics is evaluated and compared. The classification model considering one RapidEye and four pan-sharpened Landsat ETM+ images was found superior, resulting in a conditional kappa from 0.77 to 0.94 per class and an overall accuracy of 89.7%. The results highlight the potential of the method for operational crop mapping in Euro-Mediterranean areas and provide some hints for optimal image acquisition windows regarding major crop types in Greece.
A Markov Chain Model for Contagion
Directory of Open Access Journals (Sweden)
Angelos Dassios
2014-11-01
Full Text Available We introduce a bivariate Markov chain counting process with contagion for modelling the clustering arrival of loss claims with delayed settlement for an insurance company. It is a general continuous-time model framework that also has the potential to be applicable to modelling the clustering arrival of events, such as jumps, bankruptcies, crises and catastrophes in finance, insurance and economics with both internal contagion risk and external common risk. Key distributional properties, such as the moments and probability generating functions, for this process are derived. Some special cases with explicit results and numerical examples and the motivation for further actuarial applications are also discussed. The model can be considered a generalisation of the dynamic contagion process introduced by Dassios and Zhao (2011.
Stencil method: a Markov model for transport in porous media
Delgoshaie, A. H.; Tchelepi, H.; Jenny, P.
2016-12-01
In porous media the transport of fluid is dominated by flow-field heterogeneity resulting from the underlying transmissibility field. Since the transmissibility is highly uncertain, many realizations of a geological model are used to describe the statistics of the transport phenomena in a Monte Carlo framework. One possible way to avoid the high computational cost of physics-based Monte Carlo simulations is to model the velocity field as a Markov process and use Markov Chain Monte Carlo. In previous works multiple Markov models for discrete velocity processes have been proposed. These models can be divided into two general classes of Markov models in time and Markov models in space. Both of these choices have been shown to be effective to some extent. However some studies have suggested that the Markov property cannot be confirmed for a temporal Markov process; Therefore there is not a consensus about the validity and value of Markov models in time. Moreover, previous spacial Markov models have only been used for modeling transport on structured networks and can not be readily applied to model transport in unstructured networks. In this work we propose a novel approach for constructing a Markov model in time (stencil method) for a discrete velocity process. The results form the stencil method are compared to previously proposed spacial Markov models for structured networks. The stencil method is also applied to unstructured networks and can successfully describe the dispersion of particles in this setting. Our conclusion is that both temporal Markov models and spacial Markov models for discrete velocity processes can be valid for a range of model parameters. Moreover, we show that the stencil model can be more efficient in many practical settings and is suited to model dispersion both on structured and unstructured networks.
Derivation of Markov processes that violate detailed balance
Lee, Julian
2018-03-01
Time-reversal symmetry of the microscopic laws dictates that the equilibrium distribution of a stochastic process must obey the condition of detailed balance. However, cyclic Markov processes that do not admit equilibrium distributions with detailed balance are often used to model systems driven out of equilibrium by external agents. I show that for a Markov model without detailed balance, an extended Markov model can be constructed, which explicitly includes the degrees of freedom for the driving agent and satisfies the detailed balance condition. The original cyclic Markov model for the driven system is then recovered as an approximation at early times by summing over the degrees of freedom for the driving agent. I also show that the widely accepted expression for the entropy production in a cyclic Markov model is actually a time derivative of an entropy component in the extended model. Further, I present an analytic expression for the entropy component that is hidden in the cyclic Markov model.
Continuous-time quantum Monte Carlo impurity solvers
Gull, Emanuel; Werner, Philipp; Fuchs, Sebastian; Surer, Brigitte; Pruschke, Thomas; Troyer, Matthias
2011-04-01
Continuous-time quantum Monte Carlo impurity solvers are algorithms that sample the partition function of an impurity model using diagrammatic Monte Carlo techniques. The present paper describes codes that implement the interaction expansion algorithm originally developed by Rubtsov, Savkin, and Lichtenstein, as well as the hybridization expansion method developed by Werner, Millis, Troyer, et al. These impurity solvers are part of the ALPS-DMFT application package and are accompanied by an implementation of dynamical mean-field self-consistency equations for (single orbital single site) dynamical mean-field problems with arbitrary densities of states. Program summaryProgram title: dmft Catalogue identifier: AEIL_v1_0 Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEIL_v1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: ALPS LIBRARY LICENSE version 1.1 No. of lines in distributed program, including test data, etc.: 899 806 No. of bytes in distributed program, including test data, etc.: 32 153 916 Distribution format: tar.gz Programming language: C++ Operating system: The ALPS libraries have been tested on the following platforms and compilers: Linux with GNU Compiler Collection (g++ version 3.1 and higher), and Intel C++ Compiler (icc version 7.0 and higher) MacOS X with GNU Compiler (g++ Apple-version 3.1, 3.3 and 4.0) IBM AIX with Visual Age C++ (xlC version 6.0) and GNU (g++ version 3.1 and higher) compilers Compaq Tru64 UNIX with Compq C++ Compiler (cxx) SGI IRIX with MIPSpro C++ Compiler (CC) HP-UX with HP C++ Compiler (aCC) Windows with Cygwin or coLinux platforms and GNU Compiler Collection (g++ version 3.1 and higher) RAM: 10 MB-1 GB Classification: 7.3 External routines: ALPS [1], BLAS/LAPACK, HDF5 Nature of problem: (See [2].) Quantum impurity models describe an atom or molecule embedded in a host material with which it can exchange electrons. They are basic to nanoscience as
Stochastic calculus for uncoupled continuous-time random walks.
Germano, Guido; Politi, Mauro; Scalas, Enrico; Schilling, René L
2009-06-01
The continuous-time random walk (CTRW) is a pure-jump stochastic process with several applications not only in physics but also in insurance, finance, and economics. A definition is given for a class of stochastic integrals driven by a CTRW, which includes the Itō and Stratonovich cases. An uncoupled CTRW with zero-mean jumps is a martingale. It is proved that, as a consequence of the martingale transform theorem, if the CTRW is a martingale, the Itō integral is a martingale too. It is shown how the definition of the stochastic integrals can be used to easily compute them by Monte Carlo simulation. The relations between a CTRW, its quadratic variation, its Stratonovich integral, and its Itō integral are highlighted by numerical calculations when the jumps in space of the CTRW have a symmetric Lévy alpha -stable distribution and its waiting times have a one-parameter Mittag-Leffler distribution. Remarkably, these distributions have fat tails and an unbounded quadratic variation. In the diffusive limit of vanishing scale parameters, the probability density of this kind of CTRW satisfies the space-time fractional diffusion equation (FDE) or more in general the fractional Fokker-Planck equation, which generalizes the standard diffusion equation, solved by the probability density of the Wiener process, and thus provides a phenomenologic model of anomalous diffusion. We also provide an analytic expression for the quadratic variation of the stochastic process described by the FDE and check it by Monte Carlo.
Effect of hydraulic retention time on continuous biocatalytic calcification reactor
International Nuclear Information System (INIS)
Isik, Mustafa; Altas, Levent; Kurmac, Yakup; Ozcan, Samet; Oruc, Ozcan
2010-01-01
High calcium concentrations in the wastewaters are problematic, because they lead to clogging of pipelines, boilers and heat exchangers through scaling (as carbonate, sulfate or phosphate precipitates), or malfunctioning of aerobic and anaerobic reactors. As a remedy to this problem, the industry typically uses chemical crystallization reactors which are efficient but often require complex monitoring and control and, as a drawback, can give rise to highly alkaline effluents. Biomineralization are emerging as alternative mechanisms for the removal of calcium from aqueous environments. Biocatalytic calcification reactors (BCR) utilize microbial urea hydrolysis by bacteria for the removal of calcium, as calcite, from industrial wastewater. Hydraulic retention time (HRT) effect on calcium removal was studied with a continuous feed BCR reactor treating a simulated pulp paper wastewater. Study showed that HRT is important parameter and HRT of 5-6 h is optimum for calcium removal from calcium-rich wastewaters.
A Continuous-Time Model for Valuing Foreign Exchange Options
Directory of Open Access Journals (Sweden)
James J. Kung
2013-01-01
Full Text Available This paper makes use of stochastic calculus to develop a continuous-time model for valuing European options on foreign exchange (FX when both domestic and foreign spot rates follow a generalized Wiener process. Using the dollar/euro exchange rate as input for parameter estimation and employing our FX option model as a yardstick, we find that the traditional Garman-Kohlhagen FX option model, which assumes constant spot rates, values incorrectly calls and puts for different values of the ratio of exchange rate to exercise price. Specifically, it undervalues calls when the ratio is between 0.70 and 1.08, and it overvalues calls when the ratio is between 1.18 and 1.30, whereas it overvalues puts when the ratio is between 0.70 and 0.82, and it undervalues puts when the ratio is between 0.86 and 1.30.
DEFF Research Database (Denmark)
Mailund, Thomas; Dutheil, Julien; Hobolth, Asger
2011-01-01
event has occurred to split them apart. The size of these segments of constant divergence depends on the recombination rate, but also on the speciation time, the effective population size of the ancestral population, as well as demographic effects and selection. Thus, inference of these parameters may......, and the ancestral effective population size. The model is efficient enough to allow inference on whole-genome data sets. We first investigate the power and consistency of the model with coalescent simulations and then apply it to the whole-genome sequences of the two orangutan sub-species, Bornean (P. p. pygmaeus......) and Sumatran (P. p. abelii) orangutans from the Orangutan Genome Project. We estimate the speciation time between the two sub-species to be thousand years ago and the effective population size of the ancestral orangutan species to be , consistent with recent results based on smaller data sets. We also report...
Time limit and time at VO2max' during a continuous and an intermittent run.
Demarie, S; Koralsztein, J P; Billat, V
2000-06-01
The purpose of this study was to verify, by track field tests, whether sub-elite runners (n=15) could (i) reach their VO2max while running at v50%delta, i.e. midway between the speed associated with lactate threshold (vLAT) and that associated with maximal aerobic power (vVO2max), and (ii) if an intermittent exercise provokes a maximal and/or supra maximal oxygen consumption longer than a continuous one. Within three days, subjects underwent a multistage incremental test during which their vVO2max and vLAT were determined; they then performed two additional testing sessions, where continuous and intermittent running exercises at v50%delta were performed up to exhaustion. Subject's gas exchange and heart rate were continuously recorded by means of a telemetric apparatus. Blood samples were taken from fingertip and analysed for blood lactate concentration. In the continuous and the intermittent tests peak VO2 exceeded VO2max values, as determined during the incremental test. However in the intermittent exercise, peak VO2, time to exhaustion and time at VO2max reached significantly higher values, while blood lactate accumulation showed significantly lower values than in the continuous one. The v50%delta is sufficient to stimulate VO2max in both intermittent and continuous running. The intermittent exercise results better than the continuous one in increasing maximal aerobic power, allowing longer time at VO2max and obtaining higher peak VO2 with lower lactate accumulation.
Madsen, Line Meldgaard; Fiandaca, Gianluca; Auken, Esben; Christiansen, Anders Vest
2017-12-01
The application of time-domain induced polarization (TDIP) is increasing with advances in acquisition techniques, data processing and spectral inversion schemes. An inversion of TDIP data for the spectral Cole-Cole parameters is a non-linear problem, but by applying a 1-D Markov Chain Monte Carlo (MCMC) inversion algorithm, a full non-linear uncertainty analysis of the parameters and the parameter correlations can be accessed. This is essential to understand to what degree the spectral Cole-Cole parameters can be resolved from TDIP data. MCMC inversions of synthetic TDIP data, which show bell-shaped probability distributions with a single maximum, show that the Cole-Cole parameters can be resolved from TDIP data if an acquisition range above two decades in time is applied. Linear correlations between the Cole-Cole parameters are observed and by decreasing the acquisitions ranges, the correlations increase and become non-linear. It is further investigated how waveform and parameter values influence the resolution of the Cole-Cole parameters. A limiting factor is the value of the frequency exponent, C. As C decreases, the resolution of all the Cole-Cole parameters decreases and the results become increasingly non-linear. While the values of the time constant, τ, must be in the acquisition range to resolve the parameters well, the choice between a 50 per cent and a 100 per cent duty cycle for the current injection does not have an influence on the parameter resolution. The limits of resolution and linearity are also studied in a comparison between the MCMC and a linearized gradient-based inversion approach. The two methods are consistent for resolved models, but the linearized approach tends to underestimate the uncertainties for poorly resolved parameters due to the corresponding non-linear features. Finally, an MCMC inversion of 1-D field data verifies that spectral Cole-Cole parameters can also be resolved from TD field measurements.
DEFF Research Database (Denmark)
Mardare, Radu Iulian; Cardelli, Luca; Larsen, Kim Guldstrand
2012-01-01
Continuous Markovian Logic (CML) is a multimodal logic that expresses quantitative and qualitative properties of continuous-time labelled Markov processes with arbitrary (analytic) state-spaces, henceforth called continuous Markov processes (CMPs). The modalities of CML evaluate the rates...... of the exponentially distributed random variables that characterize the duration of the labeled transitions of a CMP. In this paper we present weak and strong complete axiomatizations for CML and prove a series of metaproperties, including the finite model property and the construction of canonical models. CML...... characterizes stochastic bisimilarity and it supports the definition of a quantified extension of the satisfiability relation that measures the "compatibility" between a model and a property. In this context, the metaproperties allows us to prove two robustness theorems for the logic stating that one can...
Comolli, Alessandro; Hakoun, Vivien; Dentz, Marco
2017-04-01
Achieving the understanding of the process of solute transport in heterogeneous porous media is of crucial importance for several environmental and social purposes, ranging from aquifers contamination and remediation, to risk assessment in nuclear waste repositories. The complexity of this aim is mainly ascribable to the heterogeneity of natural media, which can be observed at all the scales of interest, from pore scale to catchment scale. In fact, the intrinsic heterogeneity of porous media is responsible for the arising of the well-known non-Fickian footprints of transport, including heavy-tailed breakthrough curves, non-Gaussian spatial density profiles and the non-linear growth of the mean squared displacement. Several studies investigated the processes through which heterogeneity impacts the transport properties, which include local modifications to the advective-dispersive motion of solutes, mass exchanges between some mobile and immobile phases (e.g. sorption/desorption reactions or diffusion into solid matrix) and spatial correlation of the flow field. In the last decades, the continuous time random walk (CTRW) model has often been used to describe solute transport in heterogenous conditions and to quantify the impact of point heterogeneity, spatial correlation and mass transfer on the average transport properties [1]. Open issues regarding this approach are the possibility to relate measurable properties of the medium to the parameters of the model, as well as its capability to provide predictive information. In a recent work [2] the authors have shed new light on understanding the relationship between Lagrangian and Eulerian dynamics as well as on their evolution from arbitrary initial conditions. On the basis of these results, we derive a CTRW model for the description of Darcy-scale transport in d-dimensional media characterized by spatially random permeability fields. The CTRW approach models particle velocities as a spatial Markov process, which is
Continuous data recording on fast real-time systems
Energy Technology Data Exchange (ETDEWEB)
Zabeo, L., E-mail: lzabeo@jet.u [Euratom-CCFE, Culham Science Centre, Abingdon, Oxon OX14 3DB (United Kingdom); Sartori, F. [Euratom-CCFE, Culham Science Centre, Abingdon, Oxon OX14 3DB (United Kingdom); Neto, A. [Associacao Euratom-IST, Instituto de Plasmas e Fusao Nuclear, Av. Rovisco Pais, 1049-001 Lisboa (Portugal); Piccolo, F. [Euratom-CCFE, Culham Science Centre, Abingdon, Oxon OX14 3DB (United Kingdom); Alves, D. [Associacao Euratom-IST, Instituto de Plasmas e Fusao Nuclear, Av. Rovisco Pais, 1049-001 Lisboa (Portugal); Vitelli, R. [Dipartimento di Informatica, Sistemi e Produzione, Universita di Roma, Tor Vergata, Via del Politecnico, 1-00133 Roma (Italy); Barbalace, A. [Euratom-ENEA Association, Consorzio RFX, 35127 Padova (Italy); De Tommasi, G. [Associazione EURATOM/ENEA/CREATE, Universita di Napoli Federico II, Napoli (Italy)
2010-07-15
The PCU-Project launched for the enhancement of the vertical stabilisation system at JET required the design of a new real-time control system with the challenging specifications of 2Gops and a cycle time of 50 {mu}s. The RTAI based architecture running on an x86 multi-core processor technology demonstrated to be the best platform for meeting the high requirements. Moreover, on this architecture thanks to the smart allocation of the interrupts it was possible to demonstrate simultaneous data streaming at 50 MBs on Ethernet while handling a real-time 100 kHz interrupt source with a maximum jitter of just 3 {mu}s. Because of the memory limitation imposed by 32 bit version Linux running in kernel mode, the RTAI-based new controller allows a maximum practical data storage of 800 MB per pulse. While this amount of data can be accepted for JET normal operation it posed some limitations in the debugging and commissioning of the system. In order to increase the capability of the data acquisition of the system we have designed a mechanism that allows continuous full bandwidth (56 MB/s) data streaming from the real-time task (running in kernel mode) to either a data collector (running in user mode) or an external data acquisition server. The exploited architecture involves a peer to peer mechanisms where the sender running in RTAI kernel mode broadcasts large chunks of data using UDP packets, implemented using the 'fcomm' RTAI extension , to a receiver that will store the data. The paper will present the results of the initial RTAI operating system tests, the design of the streaming architecture and the first experimental results.
From discrete-time models to continuous-time, asynchronous modeling of financial markets
Boer, Katalin; Kaymak, Uzay; Spiering, Jaap
2007-01-01
Most agent-based simulation models of financial markets are discrete-time in nature. In this paper, we investigate to what degree such models are extensible to continuous-time, asynchronous modeling of financial markets. We study the behavior of a learning market maker in a market with information
From Discrete-Time Models to Continuous-Time, Asynchronous Models of Financial Markets
K. Boer-Sorban (Katalin); U. Kaymak (Uzay); J. Spiering (Jaap)
2006-01-01
textabstractMost agent-based simulation models of financial markets are discrete-time in nature. In this paper, we investigate to what degree such models are extensible to continuous-time, asynchronous modelling of financial markets. We study the behaviour of a learning market maker in a market with
Verhoeven, Ronald; Dalmau Codina, Ramon; Prats Menéndez, Xavier; de Gelder, Nico
2014-01-01
1 Abstract In this paper an initial implementation of a real - time aircraft trajectory optimization algorithm is presented . The aircraft trajectory for descent and approach is computed for minimum use of thrust and speed brake in support of a “green” continuous descent and approach flight operation, while complying with ATC time constraints for maintaining runway throughput and co...
Chaos and unpredictability in evolution of cooperation in continuous time
You, Taekho; Kwon, Minji; Jo, Hang-Hyun; Jung, Woo-Sung; Baek, Seung Ki
2017-12-01
Cooperators benefit others with paying costs. Evolution of cooperation crucially depends on the cost-benefit ratio of cooperation, denoted as c . In this work, we investigate the infinitely repeated prisoner's dilemma for various values of c with four of the representative memory-one strategies, i.e., unconditional cooperation, unconditional defection, tit-for-tat, and win-stay-lose-shift. We consider replicator dynamics which deterministically describes how the fraction of each strategy evolves over time in an infinite-sized well-mixed population in the presence of implementation error and mutation among the four strategies. Our finding is that this three-dimensional continuous-time dynamics exhibits chaos through a bifurcation sequence similar to that of a logistic map as c varies. If mutation occurs with rate μ ≪1 , the position of the bifurcation sequence on the c axis is numerically found to scale as μ0.1, and such sensitivity to μ suggests that mutation may have nonperturbative effects on evolutionary paths. It demonstrates how the microscopic randomness of the mutation process can be amplified to macroscopic unpredictability by evolutionary dynamics.
El Yazid Boudaren, Mohamed; Monfrini, Emmanuel; Pieczynski, Wojciech; Aïssani, Amar
2014-11-01
Hidden Markov chains have been shown to be inadequate for data modeling under some complex conditions. In this work, we address the problem of statistical modeling of phenomena involving two heterogeneous system states. Such phenomena may arise in biology or communications, among other fields. Namely, we consider that a sequence of meaningful words is to be searched within a whole observation that also contains arbitrary one-by-one symbols. Moreover, a word may be interrupted at some site to be carried on later. Applying plain hidden Markov chains to such data, while ignoring their specificity, yields unsatisfactory results. The Phasic triplet Markov chain, proposed in this paper, overcomes this difficulty by means of an auxiliary underlying process in accordance with the triplet Markov chains theory. Related Bayesian restoration techniques and parameters estimation procedures according to the new model are then described. Finally, to assess the performance of the proposed model against the conventional hidden Markov chain model, experiments are conducted on synthetic and real data.
Canonical Structure and Orthogonality of Forces and Currents in Irreversible Markov Chains
Kaiser, Marcus; Jack, Robert L.; Zimmer, Johannes
2018-03-01
We discuss a canonical structure that provides a unifying description of dynamical large deviations for irreversible finite state Markov chains (continuous time), Onsager theory, and Macroscopic Fluctuation Theory (MFT). For Markov chains, this theory involves a non-linear relation between probability currents and their conjugate forces. Within this framework, we show how the forces can be split into two components, which are orthogonal to each other, in a generalised sense. This splitting allows a decomposition of the pathwise rate function into three terms, which have physical interpretations in terms of dissipation and convergence to equilibrium. Similar decompositions hold for rate functions at level 2 and level 2.5. These results clarify how bounds on entropy production and fluctuation theorems emerge from the underlying dynamical rules. We discuss how these results for Markov chains are related to similar structures within MFT, which describes hydrodynamic limits of such microscopic models.
Corbett-Detig, Russell; Nielsen, Rasmus
2017-01-01
Admixture-the mixing of genomes from divergent populations-is increasingly appreciated as a central process in evolution. To characterize and quantify patterns of admixture across the genome, a number of methods have been developed for local ancestry inference. However, existing approaches have a number of shortcomings. First, all local ancestry inference methods require some prior assumption about the expected ancestry tract lengths. Second, existing methods generally require genotypes, which is not feasible to obtain for many next-generation sequencing projects. Third, many methods assume samples are diploid, however a wide variety of sequencing applications will fail to meet this assumption. To address these issues, we introduce a novel hidden Markov model for estimating local ancestry that models the read pileup data, rather than genotypes, is generalized to arbitrary ploidy, and can estimate the time since admixture during local ancestry inference. We demonstrate that our method can simultaneously estimate the time since admixture and local ancestry with good accuracy, and that it performs well on samples of high ploidy-i.e. 100 or more chromosomes. As this method is very general, we expect it will be useful for local ancestry inference in a wider variety of populations than what previously has been possible. We then applied our method to pooled sequencing data derived from populations of Drosophila melanogaster on an ancestry cline on the east coast of North America. We find that regions of local recombination rates are negatively correlated with the proportion of African ancestry, suggesting that selection against foreign ancestry is the least efficient in low recombination regions. Finally we show that clinal outlier loci are enriched for genes associated with gene regulatory functions, consistent with a role of regulatory evolution in ecological adaptation of admixed D. melanogaster populations. Our results illustrate the potential of local ancestry
Continuous Fine-Fault Estimation with Real-Time GNSS
Norford, B. B.; Melbourne, T. I.; Szeliga, W. M.; Santillan, V. M.; Scrivner, C.; Senko, J.; Larsen, D.
2017-12-01
Thousands of real-time telemetered GNSS stations operate throughout the circum-Pacific that may be used for rapid earthquake characterization and estimation of local tsunami excitation. We report on the development of a GNSS-based finite-fault inversion system that continuously estimates slip using real-time GNSS position streams from the Cascadia subduction zone and which is being expanded throughout the circum-Pacific. The system uses 1 Hz precise point position streams computed in the ITRF14 reference frame using clock and satellite orbit corrections from the IGS. The software is implemented as seven independent modules that filter time series using Kalman filters, trigger and estimate coseismic offsets, invert for slip using a non-negative least squares method developed by Lawson and Hanson (1974) and elastic half-space Green's Functions developed by Okada (1985), smooth the results temporally and spatially, and write the resulting streams of time-dependent slip to a RabbitMQ messaging server for use by downstream modules such as tsunami excitation modules. Additional fault models can be easily added to the system for other circum-Pacific subduction zones as additional real-time GNSS data become available. The system is currently being tested using data from well-recorded earthquakes including the 2011 Tohoku earthquake, the 2010 Maule earthquake, the 2015 Illapel earthquake, the 2003 Tokachi-oki earthquake, the 2014 Iquique earthquake, the 2010 Mentawai earthquake, the 2016 Kaikoura earthquake, the 2016 Ecuador earthquake, the 2015 Gorkha earthquake, and others. Test data will be fed to the system and the resultant earthquake characterizations will be compared with published earthquake parameters. Seismic events will be assumed to occur on major faults, so, for example, only the San Andreas fault will be considered in Southern California, while the hundreds of other faults in the region will be ignored. Rake will be constrained along each subfault to be
Time inconsistency and reputation in monetary policy: a strategic model in continuous time
Li, Jingyuan; Tian, Guoqiang
2005-01-01
This article develops a model to examine the equilibrium behavior of the time inconsistency problem in a continuous time economy with stochastic and endogenized dis- tortion. First, the authors introduce the notion of sequentially rational equilibrium, and show that the time inconsistency problem may be solved with trigger reputation strategies for stochastic setting. The conditions for the existence of sequentially rational equilibrium are provided. Then, the concept of sequen...
Model Checking Structured Infinite Markov Chains
Remke, Anne Katharina Ingrid
2008-01-01
In the past probabilistic model checking hast mostly been restricted to finite state models. This thesis explores the possibilities of model checking with continuous stochastic logic (CSL) on infinite-state Markov chains. We present an in-depth treatment of model checking algorithms for two special
Markov process of muscle motors
International Nuclear Information System (INIS)
Kondratiev, Yu; Pechersky, E; Pirogov, S
2008-01-01
We study a Markov random process describing muscle molecular motor behaviour. Every motor is either bound up with a thin filament or unbound. In the bound state the motor creates a force proportional to its displacement from the neutral position. In both states the motor spends an exponential time depending on the state. The thin filament moves at a velocity proportional to the average of all displacements of all motors. We assume that the time which a motor stays in the bound state does not depend on its displacement. Then one can find an exact solution of a nonlinear equation appearing in the limit of an infinite number of motors
Continuous time quantum random walks in free space
Eichelkraut, Toni; Vetter, Christian; Perez-Leija, Armando; Christodoulides, Demetrios; Szameit, Alexander
2014-05-01
We show theoretically and experimentally that two-dimensional continuous time coherent random walks are possible in free space, that is, in the absence of any external potential, by properly tailoring the associated initial wave function. These effects are experimentally demonstrated using classical paraxial light. Evidently, the usage of classical beams to explore the dynamics of point-like quantum particles is possible since both phenomena are mathematically equivalent. This in turn makes our approach suitable for the realization of random walks using different quantum particles, including electrons and photons. To study the spatial evolution of a wavefunction theoretically, we consider the one-dimensional paraxial wave equation (i∂z +1/2 ∂x2) Ψ = 0 . Starting with the initially localized wavefunction Ψ (x , 0) = exp [ -x2 / 2σ2 ] J0 (αx) , one can show that the evolution of such Gaussian-apodized Bessel envelopes within a region of validity resembles the probability pattern of a quantum walker traversing a uniform lattice. In order to generate the desired input-field in our experimental setting we shape the amplitude and phase of a collimated light beam originating from a classical HeNe-Laser (633 nm) utilizing a spatial light modulator.
Anomalous transport in turbulent plasmas and continuous time random walks
International Nuclear Information System (INIS)
Balescu, R.
1995-01-01
The possibility of a model of anomalous transport problems in a turbulent plasma by a purely stochastic process is investigated. The theory of continuous time random walks (CTRW's) is briefly reviewed. It is shown that a particular class, called the standard long tail CTRW's is of special interest for the description of subdiffusive transport. Its evolution is described by a non-Markovian diffusion equation that is constructed in such a way as to yield exact values for all the moments of the density profile. The concept of a CTRW model is compared to an exact solution of a simple test problem: transport of charged particles in a fluctuating magnetic field in the limit of infinite perpendicular correlation length. Although the well-known behavior of the mean square displacement proportional to t 1/2 is easily recovered, the exact density profile cannot be modeled by a CTRW. However, the quasilinear approximation of the kinetic equation has the form of a non-Markovian diffusion equation and can thus be generated by a CTRW
Inverse Ising problem in continuous time: A latent variable approach
Donner, Christian; Opper, Manfred
2017-12-01
We consider the inverse Ising problem: the inference of network couplings from observed spin trajectories for a model with continuous time Glauber dynamics. By introducing two sets of auxiliary latent random variables we render the likelihood into a form which allows for simple iterative inference algorithms with analytical updates. The variables are (1) Poisson variables to linearize an exponential term which is typical for point process likelihoods and (2) Pólya-Gamma variables, which make the likelihood quadratic in the coupling parameters. Using the augmented likelihood, we derive an expectation-maximization (EM) algorithm to obtain the maximum likelihood estimate of network parameters. Using a third set of latent variables we extend the EM algorithm to sparse couplings via L1 regularization. Finally, we develop an efficient approximate Bayesian inference algorithm using a variational approach. We demonstrate the performance of our algorithms on data simulated from an Ising model. For data which are simulated from a more biologically plausible network with spiking neurons, we show that the Ising model captures well the low order statistics of the data and how the Ising couplings are related to the underlying synaptic structure of the simulated network.
Continuous-time quantum algorithms for unstructured problems
International Nuclear Information System (INIS)
Hen, Itay
2014-01-01
We consider a family of unstructured optimization problems, for which we propose a method for constructing analogue, continuous-time (not necessarily adiabatic) quantum algorithms that are faster than their classical counterparts. In this family of problems, which we refer to as ‘scrambled input’ problems, one has to find a minimum-cost configuration of a given integer-valued n-bit black-box function whose input values have been scrambled in some unknown way. Special cases within this set of problems are Grover’s search problem of finding a marked item in an unstructured database, certain random energy models, and the functions of the Deutsch–Josza problem. We consider a couple of examples in detail. In the first, we provide an O(1) deterministic analogue quantum algorithm to solve the seminal problem of Deutsch and Josza, in which one has to determine whether an n-bit boolean function is constant (gives 0 on all inputs or 1 on all inputs) or balanced (returns 0 on half the input states and 1 on the other half). We also study one variant of the random energy model, and show that, as one might expect, its minimum energy configuration can be found quadratically faster with a quantum adiabatic algorithm than with classical algorithms. (paper)
Markov and mixed models with applications
DEFF Research Database (Denmark)
Mortensen, Stig Bousgaard
This thesis deals with mathematical and statistical models with focus on applications in pharmacokinetic and pharmacodynamic (PK/PD) modelling. These models are today an important aspect of the drug development in the pharmaceutical industry and continued research in statistical methodology within...... or uncontrollable factors in an individual. Modelling using SDEs also provides new tools for estimation of unknown inputs to a system and is illustrated with an application to estimation of insulin secretion rates in diabetic patients. Models for the eect of a drug is a broader area since drugs may affect...... for non-parametric estimation of Markov processes are proposed to give a detailed description of the sleep process during the night. Statistically the Markov models considered for sleep states are closely related to the PK models based on SDEs as both models share the Markov property. When the models...
Confluence reduction for Markov automata
Timmer, Mark; Katoen, Joost P.; van de Pol, Jaco; Stoelinga, Mariëlle Ida Antoinette
2016-01-01
Markov automata are a novel formalism for specifying systems exhibiting nondeterminism, probabilistic choices and Markovian rates. As expected, the state space explosion threatens the analysability of these models. We therefore introduce confluence reduction for Markov automata, a powerful reduction
Simulating continuous-time Hamiltonian dynamics by way of a discrete-time quantum walk
International Nuclear Information System (INIS)
Schmitz, A.T.; Schwalm, W.A.
2016-01-01
Much effort has been made to connect the continuous-time and discrete-time quantum walks. We present a method for making that connection for a general graph Hamiltonian on a bigraph. Furthermore, such a scheme may be adapted for simulating discretized quantum models on a quantum computer. A coin operator is found for the discrete-time quantum walk which exhibits the same dynamics as the continuous-time evolution. Given the spectral decomposition of the graph Hamiltonian and certain restrictions, the discrete-time evolution is solved for explicitly and understood at or near important values of the parameters. Finally, this scheme is connected to past results for the 1D chain. - Highlights: • A discrete-time quantum walk is purposed which approximates a continuous-time quantum walk. • The purposed quantum walk could be used to simulate Hamiltonian dynamics on a quantum computer. • Given the spectra decomposition of the Hamiltonian, the quantum walk is solved explicitly. • The method is demonstrated and connected to previous work done on the 1D chain.
Time-aggregation effects on the baseline of continuous-time and discrete-time hazard models
ter Hofstede, F.; Wedel, M.
In this study we reinvestigate the effect of time-aggregation for discrete- and continuous-time hazard models. We reanalyze the results of a previous Monte Carlo study by ter Hofstede and Wedel (1998), in which the effects of time-aggregation on the parameter estimates of hazard models were
Anticontrol of chaos in continuous-time systems via time-delay feedback.
Wang, Xiao Fan; Chen, Guanrong; Yu, Xinghuo
2000-12-01
In this paper, a systematic design approach based on time-delay feedback is developed for anticontrol of chaos in a continuous-time system. This anticontrol method can drive a finite-dimensional, continuous-time, autonomous system from nonchaotic to chaotic, and can also enhance the existing chaos of an originally chaotic system. Asymptotic analysis is used to establish an approximate relationship between a time-delay differential equation and a discrete map. Anticontrol of chaos is then accomplished based on this relationship and the differential-geometry control theory. Several examples are given to verify the effectiveness of the methodology and to illustrate the systematic design procedure. (c) 2000 American Institute of Physics.
Finite-Time Stability and Controller Design of Continuous-Time Polynomial Fuzzy Systems
Directory of Open Access Journals (Sweden)
Xiaoxing Chen
2017-01-01
Full Text Available Finite-time stability and stabilization problem is first investigated for continuous-time polynomial fuzzy systems. The concept of finite-time stability and stabilization is given for polynomial fuzzy systems based on the idea of classical references. A sum-of-squares- (SOS- based approach is used to obtain the finite-time stability and stabilization conditions, which include some classical results as special cases. The proposed conditions can be solved with the help of powerful Matlab toolbox SOSTOOLS and a semidefinite-program (SDP solver. Finally, two numerical examples and one practical example are employed to illustrate the validity and effectiveness of the provided conditions.
Continuous time modelling of dynamical spatial lattice data observed at sparsely distributed times
DEFF Research Database (Denmark)
Rasmussen, Jakob Gulddahl; Møller, Jesper
2007-01-01
Summary. We consider statistical and computational aspects of simulation-based Bayesian inference for a spatial-temporal model based on a multivariate point process which is only observed at sparsely distributed times. The point processes are indexed by the sites of a spatial lattice......, and they exhibit spatial interaction. For specificity we consider a particular dynamical spatial lattice data set which has previously been analysed by a discrete time model involving unknown normalizing constants. We discuss the advantages and disadvantages of using continuous time processes compared...... with discrete time processes in the setting of the present paper as well as other spatial-temporal situations....
Finite-Time H∞ Filtering for Linear Continuous Time-Varying Systems with Uncertain Observations
Directory of Open Access Journals (Sweden)
Huihong Zhao
2012-01-01
Full Text Available This paper is concerned with the finite-time H∞ filtering problem for linear continuous time-varying systems with uncertain observations and ℒ2-norm bounded noise. The design of finite-time H∞ filter is equivalent to the problem that a certain indefinite quadratic form has a minimum and the filter is such that the minimum is positive. The quadratic form is related to a Krein state-space model according to the Krein space linear estimation theory. By using the projection theory in Krein space, the finite-time H∞ filtering problem is solved. A numerical example is given to illustrate the performance of the H∞ filter.
Voelkle, Manuel C; Oud, Johan H L
2013-02-01
When designing longitudinal studies, researchers often aim at equal intervals. In practice, however, this goal is hardly ever met, with different time intervals between assessment waves and different time intervals between individuals being more the rule than the exception. One of the reasons for the introduction of continuous time models by means of structural equation modelling has been to deal with irregularly spaced assessment waves (e.g., Oud & Delsing, 2010). In the present paper we extend the approach to individually varying time intervals for oscillating and non-oscillating processes. In addition, we show not only that equal intervals are unnecessary but also that it can be advantageous to use unequal sampling intervals, in particular when the sampling rate is low. Two examples are provided to support our arguments. In the first example we compare a continuous time model of a bivariate coupled process with varying time intervals to a standard discrete time model to illustrate the importance of accounting for the exact time intervals. In the second example the effect of different sampling intervals on estimating a damped linear oscillator is investigated by means of a Monte Carlo simulation. We conclude that it is important to account for individually varying time intervals, and encourage researchers to conceive of longitudinal studies with different time intervals within and between individuals as an opportunity rather than a problem. © 2012 The British Psychological Society.
Global dissipativity of continuous-time recurrent neural networks with time delay
International Nuclear Information System (INIS)
Liao Xiaoxin; Wang Jun
2003-01-01
This paper addresses the global dissipativity of a general class of continuous-time recurrent neural networks. First, the concepts of global dissipation and global exponential dissipation are defined and elaborated. Next, the sets of global dissipativity and global exponentially dissipativity are characterized using the parameters of recurrent neural network models. In particular, it is shown that the Hopfield network and cellular neural networks with or without time delays are dissipative systems
Echocardiography as an indication of continuous-time cardiac quiescence
Wick, C. A.; Auffermann, W. F.; Shah, A. J.; Inan, O. T.; Bhatti, P. T.; Tridandapani, S.
2016-07-01
Cardiac computed tomography (CT) angiography using prospective gating requires that data be acquired during intervals of minimal cardiac motion to obtain diagnostic images of the coronary vessels free of motion artifacts. This work is intended to assess B-mode echocardiography as a continuous-time indication of these quiescent periods to determine if echocardiography can be used as a cost-efficient, non-ionizing modality to develop new prospective gating techniques for cardiac CT. These new prospective gating approaches will not be based on echocardiography itself but on CT-compatible modalities derived from the mechanics of the heart (e.g. seismocardiography and impedance cardiography), unlike the current standard electrocardiogram. To this end, echocardiography and retrospectively-gated CT data were obtained from ten patients with varied cardiac conditions. CT reconstructions were made throughout the cardiac cycle. Motion of the interventricular septum (IVS) was calculated from both echocardiography and CT reconstructions using correlation-based, deviation techniques. The IVS was chosen because it (1) is visible in echocardiography images, whereas the coronary vessels generally are not, and (2) has been shown to be a suitable indicator of cardiac quiescence. Quiescent phases were calculated as the minima of IVS motion and CT volumes were reconstructed for these phases. The diagnostic quality of the CT reconstructions from phases calculated from echocardiography and CT data was graded on a four-point Likert scale by a board-certified radiologist fellowship-trained in cardiothoracic radiology. Using a Wilcoxon signed-rank test, no significant difference in the diagnostic quality of the coronary vessels was found between CT volumes reconstructed from echocardiography- and CT-selected phases. Additionally, there was a correlation of 0.956 between the echocardiography- and CT-selected phases. This initial work suggests that B-mode echocardiography can be used as a
Indian Academy of Sciences (India)
be obtained as a limiting value of a sample path of a suitable ... makes a mathematical model of chance and deals with the problem by .... Is the Markov chain aperiodic? It is! Here is how you can see it. Suppose that after you do the cut, you hold the top half in your right hand, and the bottom half in your left. Then there.
Indian Academy of Sciences (India)
Home; Journals; Resonance – Journal of Science Education; Volume 7; Issue 3. Markov Chain Monte Carlo - Examples. Arnab Chakraborty. General Article Volume 7 Issue 3 March 2002 pp 25-34. Fulltext. Click here to view fulltext PDF. Permanent link: https://www.ias.ac.in/article/fulltext/reso/007/03/0025-0034. Keywords.
Partially Hidden Markov Models
DEFF Research Database (Denmark)
Forchhammer, Søren Otto; Rissanen, Jorma
1996-01-01
Partially Hidden Markov Models (PHMM) are introduced. They differ from the ordinary HMM's in that both the transition probabilities of the hidden states and the output probabilities are conditioned on past observations. As an illustration they are applied to black and white image compression where...
Wang, Lianzhen; Pei, Yulong
2014-09-01
This real road driving study was conducted to investigate the effects of driving time and rest time on the driving performance and recovery of commercial coach drivers. Thirty-three commercial coach drivers participated in the study, and were divided into three groups according to driving time: (a) 2 h, (b) 3 h, and (c) 4 h. The Stanford Sleepiness Scale (SSS) was used to assess the subjective fatigue level of the drivers. One-way ANOVA was employed to analyze the variation in driving performance. The statistical analysis revealed that driving time had a significant effect on the subjective fatigue and driving performance measures among the three groups. After 2 h of driving, both the subjective fatigue and driving performance measures began to deteriorate. After 4 h of driving, all of the driving performance indicators changed significantly except for depth perception. A certain amount of rest time eliminated the negative effects of fatigue. A 15-minute rest allowed drivers to recover from a two-hour driving task. This needed to be prolonged to 30 min for driving tasks of 3 to 4 h of continuous driving. Drivers' attention, reactions, operating ability, and perceptions are all affected in turn after over 2 h of continuous driving. Drivers should take a certain amount of rest to recover from the fatigue effects before they continue driving. Copyright © 2014 National Safety Council and Elsevier Ltd. All rights reserved.
Giordano, J O; Kalantari, A S; Fricke, P M; Wiltbank, M C; Cabrera, V E
2012-09-01
Our objective was to compare the economic and reproductive performance of programs combining timed artificial insemination (TAI) and different levels of AI after estrus detection (ED) using a daily Markov-chain model. A dairy herd was modeled with every cow following daily probabilistic events of aging, replacement, mortality, pregnancy, pregnancy loss, and calving. The probability of pregnancy depended on the combination of probability of insemination and conception rate (CR). All nonpregnant cows had a probability of pregnancy between the end of the voluntary waiting period and days in milk cutoff for AI. After the cutoff, cows were labeled as do not breed and replaced when milk production was below a minimum milk threshold. A similar model was created to represent a replacement heifer herd to simulate and adjust the supply and demand of replacements. The net value (NV) of a program was the sum of milk income over feed cost, replacement and mortality cost, income from newborns, and reproductive costs. The model was used to compare the NV of 19 programs. One program used 100% TAI (42% CR for first TAI and 30% for second-and-later services), whereas the other programs combined TAI with ED. The proportion of cows receiving AI after ED for the combined programs ranged from 30 to 80%, with levels of CR of 25, 30, and 35%. As the proportion of cows receiving AI after ED increased, the CR of cows receiving TAI decreased. The combined programs with CR of 35% for cows receiving AI after ED had the greatest NV and reproductive performance at all levels of ED. The program using 100% TAI had greater NV and better reproductive performance than all programs with 25% CR after ED inseminations, whereas it had very similar performance to combined programs with up to 60% of cows receiving AI after ED and 30% CR. The factor with the greatest relative contribution to the differences among programs was income over feed cost, followed by replacement and reproductive costs. Adjusting
Occupation times and ergodicity breaking in biased continuous time random walks
International Nuclear Information System (INIS)
Bel, Golan; Barkai, Eli
2005-01-01
Continuous time random walk (CTRW) models are widely used to model diffusion in condensed matter. There are two classes of such models, distinguished by the convergence or divergence of the mean waiting time. Systems with finite average sojourn time are ergodic and thus Boltzmann-Gibbs statistics can be applied. We investigate the statistical properties of CTRW models with infinite average sojourn time; in particular, the occupation time probability density function is obtained. It is shown that in the non-ergodic phase the distribution of the occupation time of the particle on a given lattice point exhibits bimodal U or trimodal W shape, related to the arcsine law. The key points are as follows. (a) In a CTRW with finite or infinite mean waiting time, the distribution of the number of visits on a lattice point is determined by the probability that a member of an ensemble of particles in equilibrium occupies the lattice point. (b) The asymmetry parameter of the probability distribution function of occupation times is related to the Boltzmann probability and to the partition function. (c) The ensemble average is given by Boltzmann-Gibbs statistics for either finite or infinite mean sojourn time, when detailed balance conditions hold. (d) A non-ergodic generalization of the Boltzmann-Gibbs statistical mechanics for systems with infinite mean sojourn time is found
Inhomogeneous Markov Models for Describing Driving Patterns
DEFF Research Database (Denmark)
Iversen, Emil Banning; Møller, Jan K.; Morales, Juan Miguel
2017-01-01
. Specifically, an inhomogeneous Markov model that captures the diurnal variation in the use of a vehicle is presented. The model is defined by the time-varying probabilities of starting and ending a trip, and is justified due to the uncertainty associated with the use of the vehicle. The model is fitted to data...... collected from the actual utilization of a vehicle. Inhomogeneous Markov models imply a large number of parameters. The number of parameters in the proposed model is reduced using B-splines....
Inhomogeneous Markov Models for Describing Driving Patterns
DEFF Research Database (Denmark)
Iversen, Jan Emil Banning; Møller, Jan Kloppenborg; Morales González, Juan Miguel
. Specically, an inhomogeneous Markov model that captures the diurnal variation in the use of a vehicle is presented. The model is dened by the time-varying probabilities of starting and ending a trip and is justied due to the uncertainty associated with the use of the vehicle. The model is tted to data...... collected from the actual utilization of a vehicle. Inhomogeneous Markov models imply a large number of parameters. The number of parameters in the proposed model is reduced using B-splines....
Ragain, Stephen; Ugander, Johan
2016-01-01
As datasets capturing human choices grow in richness and scale---particularly in online domains---there is an increasing need for choice models that escape traditional choice-theoretic axioms such as regularity, stochastic transitivity, and Luce's choice axiom. In this work we introduce the Pairwise Choice Markov Chain (PCMC) model of discrete choice, an inferentially tractable model that does not assume any of the above axioms while still satisfying the foundational axiom of uniform expansio...
Distinguishing Hidden Markov Chains
Kiefer, Stefan; Sistla, A. Prasad
2015-01-01
Hidden Markov Chains (HMCs) are commonly used mathematical models of probabilistic systems. They are employed in various fields such as speech recognition, signal processing, and biological sequence analysis. We consider the problem of distinguishing two given HMCs based on an observation sequence that one of the HMCs generates. More precisely, given two HMCs and an observation sequence, a distinguishing algorithm is expected to identify the HMC that generates the observation sequence. Two HM...
Fannes, Mark; Wouters, Jeroen
2012-01-01
We study a quantum process that can be considered as a quantum analogue for the classical Markov process. We specifically construct a version of these processes for free Fermions. For such free Fermionic processes we calculate the entropy density. This can be done either directly using Szeg\\"o's theorem for asymptotic densities of functions of Toeplitz matrices, or through an extension of said theorem to rates of functions, which we present in this article.
Approximate quantum Markov chains
Sutter, David
2018-01-01
This book is an introduction to quantum Markov chains and explains how this concept is connected to the question of how well a lost quantum mechanical system can be recovered from a correlated subsystem. To achieve this goal, we strengthen the data-processing inequality such that it reveals a statement about the reconstruction of lost information. The main difficulty in order to understand the behavior of quantum Markov chains arises from the fact that quantum mechanical operators do not commute in general. As a result we start by explaining two techniques of how to deal with non-commuting matrices: the spectral pinching method and complex interpolation theory. Once the reader is familiar with these techniques a novel inequality is presented that extends the celebrated Golden-Thompson inequality to arbitrarily many matrices. This inequality is the key ingredient in understanding approximate quantum Markov chains and it answers a question from matrix analysis that was open since 1973, i.e., if Lieb's triple ma...
A relation between non-Markov and Markov processes
International Nuclear Information System (INIS)
Hara, H.
1980-01-01
With the aid of a transformation technique, it is shown that some memory effects in the non-Markov processes can be eliminated. In other words, some non-Markov processes are rewritten in a form obtained by the random walk process; the Markov process. To this end, two model processes which have some memory or correlation in the random walk process are introduced. An explanation of the memory in the processes is given. (orig.)
Cluster Observations of Non-Time Continuous Magnetosonic Waves
Walker, Simon N.; Demekhov, Andrei G.; Boardsen, Scott A.; Ganushkina, Natalia Y.; Sibeck, David G.; Balikhin, Michael A.
2016-01-01
Equatorial magnetosonic waves are normally observed as temporally continuous sets of emissions lasting from minutes to hours. Recent observations, however, have shown that this is not always the case. Using Cluster data, this study identifies two distinct forms of these non temporally continuous use missions. The first, referred to as rising tone emissions, are characterized by the systematic onset of wave activity at increasing proton gyroharmonic frequencies. Sets of harmonic emissions (emission elements)are observed to occur periodically in the region +/- 10 off the geomagnetic equator. The sweep rate of these emissions maximizes at the geomagnetic equator. In addition, the ellipticity and propagation direction also change systematically as Cluster crosses the geomagnetic equator. It is shown that the observed frequency sweep rate is unlikely to result from the sideband instability related to nonlinear trapping of suprathermal protons in the wave field. The second form of emissions is characterized by the simultaneous onset of activity across a range of harmonic frequencies. These waves are observed at irregular intervals. Their occurrence correlates with changes in the spacecraft potential, a measurement that is used as a proxy for electron density. Thus, these waves appear to be trapped within regions of localized enhancement of the electron density.
The explicit form of the rate function for semi-Markov processes and its contractions
Sughiyama, Yuki; Kobayashi, Testuya J.
2018-03-01
We derive the explicit form of the rate function for semi-Markov processes. Here, the ‘random time change trick’ plays an essential role. Also, by exploiting the contraction principle of large deviation theory to the explicit form, we show that the fluctuation theorem (Gallavotti-Cohen symmetry) holds for semi-Markov cases. Furthermore, we elucidate that our rate function is an extension of the level 2.5 rate function for Markov processes to semi-Markov cases.
A Martingale Decomposition of Discrete Markov Chains
DEFF Research Database (Denmark)
Hansen, Peter Reinhard
We consider a multivariate time series whose increments are given from a homogeneous Markov chain. We show that the martingale component of this process can be extracted by a filtering method and establish the corresponding martingale decomposition in closed-form. This representation is useful fo...
Efficient Modelling and Generation of Markov Automata
Koutny, M.; Timmer, Mark; Ulidowski, I.; Katoen, Joost P.; van de Pol, Jan Cornelis; Stoelinga, Mariëlle Ida Antoinette
This paper introduces a framework for the efficient modelling and generation of Markov automata. It consists of (1) the data-rich process-algebraic language MAPA, allowing concise modelling of systems with nondeterminism, probability and Markovian timing; (2) a restricted form of the language, the
Course Development Cycle Time: A Framework for Continuous Process Improvement.
Lake, Erinn
2003-01-01
Details Edinboro University's efforts to reduce the extended cycle time required to develop new courses and programs. Describes a collaborative process improvement framework, illustrated data findings, the team's recommendations for improvement, and the outcomes of those recommendations. (EV)
Belief Bisimulation for Hidden Markov Models Logical Characterisation and Decision Algorithm
DEFF Research Database (Denmark)
Jansen, David N.; Nielson, Flemming; Zhang, Lijun
2012-01-01
This paper establishes connections between logical equivalences and bisimulation relations for hidden Markov models (HMM). Both standard and belief state bisimulations are considered. We also present decision algorithms for the bisimilarities. For standard bisimilarity, an extension of the usual...... partition refinement algorithm is enough. Belief bisimilarity, being a relation on the continuous space of belief states, cannot be described directly. Instead, we show how to generate a linear equation system in time cubic in the number of states....
Mallak, Saed
1996-01-01
Ankara : Department of Mathematics and Institute of Engineering and Sciences of Bilkent University, 1996. Thesis (Master's) -- Bilkent University, 1996. Includes bibliographical references leaves leaf 29 In thi.s work, we studierl the Ergodicilv of Non-Stationary .Markov chains. We gave several e.xainples with different cases. We proved that given a sec[uence of Markov chains such that the limit of this sec|uence is an Ergodic Markov chain, then the limit of the combination ...
An automated quasi-continuous capillary refill timing device
International Nuclear Information System (INIS)
Blaxter, L L; Morris, D E; Crowe, J A; Hayes-Gill, B R; Henry, C; Hill, S; Sharkey, D; Vyas, H
2016-01-01
Capillary refill time (CRT) is a simple means of cardiovascular assessment which is widely used in clinical care. Currently, CRT is measured through manual assessment of the time taken for skin tone to return to normal colour following blanching of the skin surface. There is evidence to suggest that manually assessed CRT is subject to bias from ambient light conditions, a lack of standardisation of both blanching time and manually applied pressure, subjectiveness of return to normal colour, and variability in the manual assessment of time. We present a novel automated system for CRT measurement, incorporating three components: a non-invasive adhesive sensor incorporating a pneumatic actuator, a diffuse multi-wavelength reflectance measurement device, and a temperature sensor; a battery operated datalogger unit containing a self contained pneumatic supply; and PC based data analysis software for the extraction of refill time, patient skin surface temperature, and sensor signal quality. Through standardisation of the test, it is hoped that some of the shortcomings of manual CRT can be overcome. In addition, an automated system will facilitate easier integration of CRT into electronic record keeping and clinical monitoring or scoring systems, as well as reducing demands on clinicians. Summary analysis of volunteer (n = 30) automated CRT datasets are presented, from 15 healthy adults and 15 healthy children (aged from 5 to 15 years), as their arms were cooled from ambient temperature to 5°C. A more detailed analysis of two typical datasets is also presented, demonstrating that the response of automated CRT to cooling matches that of previously published studies. (paper)
Mental time travel: a case for evolutionary continuity.
Corballis, Michael C
2013-01-01
In humans, hippocampal activity responds to the imagining of past or future events. In rats, hippocampal activity is tied to particular locations in a maze, occurs after the animal has been in the maze, and sometimes corresponds to locations the animal did not actually visit. This suggests that mental time travel has neurophysiological underpinnings that go far back in evolution, and may not be, as some (including myself) have claimed, unique to humans. Copyright © 2012 Elsevier Ltd. All rights reserved.
Continuous radon measurements in schools: time variations and related parameters
International Nuclear Information System (INIS)
Giovani, C.; Cappelletto, C.; Garavaglia, M.; Pividore, S.; Villalta, R.
2004-01-01
Some results are reported of observations made within a four-year survey, during different seasons and in different conditions of school building use. Natural radon variations (day-night cycles, seasonal and temperature dependent variations etc..) and artificial ones (opening of windows, weekends and vacations, deployment of air conditioning or heating systems. etc.) were investigated as parameters affecting time dependent radon concentrations. (P.A.)
Markov Chain Monte Carlo Methods
Indian Academy of Sciences (India)
Keywords. Markov chain; state space; stationary transition probability; stationary distribution; irreducibility; aperiodicity; stationarity; M-H algorithm; proposal distribution; acceptance probability; image processing; Gibbs sampler.
Real-time continuous nitrate monitoring in Illinois in 2013
Warner, Kelly L.; Terrio, Paul J.; Straub, Timothy D.; Roseboom, Donald; Johnson, Gary P.
2013-01-01
Many sources contribute to the nitrogen found in surface water in Illinois. Illinois is located in the most productive agricultural area in the country, and nitrogen fertilizer is commonly used to maximize corn production in this area. Additionally, septic/wastewater systems, industrial emissions, and lawn fertilizer are common sources of nitrogen in urban areas of Illinois. In agricultural areas, the use of fertilizer has increased grain production to meet the needs of a growing population, but also has resulted in increases in nitrogen concentrations in many streams and aquifers (Dubrovsky and others, 2010). The urban sources can increase nitrogen concentrations, too. The Federal limit for nitrate nitrogen in water that is safe to drink is 10 milligrams per liter (mg/L) (http://water.epa.gov/drink/contaminants/basicinformation/nitrate.cfm, accessed on May 24, 2013). In addition to the concern with nitrate nitrogen in drinking water, nitrogen, along with phosphorus, is an aquatic concern because it feeds the intensive growth of algae that are responsible for the hypoxic zone in the Gulf of Mexico. The largest nitrogen flux to the waters feeding the Gulf of Mexico is from Illinois (Alexander and others, 2008). Most studies of nitrogen in surface water and groundwater include samples for nitrate nitrogen collected weekly or monthly, but nitrate concentrations can change rapidly and these discrete samples may not capture rapid changes in nitrate concentrations that can affect human and aquatic health. Continuous monitoring for nitrate could inform scientists and water-resource managers of these changes and provide information on the transport of nitrate in surface water and groundwater.
On almost-periodic points of a topological Markov chain
International Nuclear Information System (INIS)
Bogatyi, Semeon A; Redkozubov, Vadim V
2012-01-01
We prove that a transitive topological Markov chain has almost-periodic points of all D-periods. Moreover, every D-period is realized by continuously many distinct minimal sets. We give a simple constructive proof of the result which asserts that any transitive topological Markov chain has periodic points of almost all periods, and study the structure of the finite set of positive integers that are not periods.
DEFF Research Database (Denmark)
Kohlenbach, Ulrich Wilhelm
2002-01-01
We show that the so-called weak Markov's principle (WMP) which states that every pseudo-positive real number is positive is underivable in E-HA + AC. Since allows one to formalize (atl eastl arge parts of) Bishop's constructive mathematics, this makes it unlikely that WMP can be proved within...... the framework of Bishop-style mathematics (which has been open for about 20 years). The underivability even holds if the ine.ective schema of full comprehension (in all types) for negated formulas (in particular for -free formulas) is added, which allows one to derive the law of excluded middle...
Continuous-time random walks on networks with vertex- and time-dependent forcing.
Angstmann, C N; Donnelly, I C; Henry, B I; Langlands, T A M
2013-08-01
We have investigated the transport of particles moving as random walks on the vertices of a network, subject to vertex- and time-dependent forcing. We have derived the generalized master equations for this transport using continuous time random walks, characterized by jump and waiting time densities, as the underlying stochastic process. The forcing is incorporated through a vertex- and time-dependent bias in the jump densities governing the random walking particles. As a particular case, we consider particle forcing proportional to the concentration of particles on adjacent vertices, analogous to self-chemotactic attraction in a spatial continuum. Our algebraic and numerical studies of this system reveal an interesting pair-aggregation pattern formation in which the steady state is composed of a high concentration of particles on a small number of isolated pairs of adjacent vertices. The steady states do not exhibit this pair aggregation if the transport is random on the vertices, i.e., without forcing. The manifestation of pair aggregation on a transport network may thus be a signature of self-chemotactic-like forcing.
Markov and semi-Markov switching linear mixed models used to identify forest tree growth components.
Chaubert-Pereira, Florence; Guédon, Yann; Lavergne, Christian; Trottier, Catherine
2010-09-01
Tree growth is assumed to be mainly the result of three components: (i) an endogenous component assumed to be structured as a succession of roughly stationary phases separated by marked change points that are asynchronous among individuals, (ii) a time-varying environmental component assumed to take the form of synchronous fluctuations among individuals, and (iii) an individual component corresponding mainly to the local environment of each tree. To identify and characterize these three components, we propose to use semi-Markov switching linear mixed models, i.e., models that combine linear mixed models in a semi-Markovian manner. The underlying semi-Markov chain represents the succession of growth phases and their lengths (endogenous component) whereas the linear mixed models attached to each state of the underlying semi-Markov chain represent-in the corresponding growth phase-both the influence of time-varying climatic covariates (environmental component) as fixed effects, and interindividual heterogeneity (individual component) as random effects. In this article, we address the estimation of Markov and semi-Markov switching linear mixed models in a general framework. We propose a Monte Carlo expectation-maximization like algorithm whose iterations decompose into three steps: (i) sampling of state sequences given random effects, (ii) prediction of random effects given state sequences, and (iii) maximization. The proposed statistical modeling approach is illustrated by the analysis of successive annual shoots along Corsican pine trunks influenced by climatic covariates. © 2009, The International Biometric Society.
Chen, Xiaofeng; Song, Qiankun; Li, Zhongshan; Zhao, Zhenjiang; Liu, Yurong
2018-07-01
This paper addresses the problem of stability for continuous-time and discrete-time quaternion-valued neural networks (QVNNs) with linear threshold neurons. Applying the semidiscretization technique to the continuous-time QVNNs, the discrete-time analogs are obtained, which preserve the dynamical characteristics of their continuous-time counterparts. Via the plural decomposition method of quaternion, homeomorphic mapping theorem, as well as Lyapunov theorem, some sufficient conditions on the existence, uniqueness, and global asymptotical stability of the equilibrium point are derived for the continuous-time QVNNs and their discrete-time analogs, respectively. Furthermore, a uniform sufficient condition on the existence, uniqueness, and global asymptotical stability of the equilibrium point is obtained for both continuous-time QVNNs and their discrete-time version. Finally, two numerical examples are provided to substantiate the effectiveness of the proposed results.
The algebra of the general Markov model on phylogenetic trees and networks.
Sumner, J G; Holland, B R; Jarvis, P D
2012-04-01
It is known that the Kimura 3ST model of sequence evolution on phylogenetic trees can be extended quite naturally to arbitrary split systems. However, this extension relies heavily on mathematical peculiarities of the associated Hadamard transformation, and providing an analogous augmentation of the general Markov model has thus far been elusive. In this paper, we rectify this shortcoming by showing how to extend the general Markov model on trees to include incompatible edges; and even further to more general network models. This is achieved by exploring the algebra of the generators of the continuous-time Markov chain together with the “splitting” operator that generates the branching process on phylogenetic trees. For simplicity, we proceed by discussing the two state case and then show that our results are easily extended to more states with little complication. Intriguingly, upon restriction of the two state general Markov model to the parameter space of the binary symmetric model, our extension is indistinguishable from the Hadamard approach only on trees; as soon as any incompatible splits are introduced the two approaches give rise to differing probability distributions with disparate structure. Through exploration of a simple example, we give an argument that our extension to more general networks has desirable properties that the previous approaches do not share. In particular, our construction allows for convergent evolution of previously divergent lineages; a property that is of significant interest for biological applications.
DEFF Research Database (Denmark)
Hobolth, Asger
2008-01-01
-dimensional integrals required in the EM algorithm are estimated using MCMC sampling. The MCMC sampler requires simulation of sample paths from a continuous time Markov process, conditional on the beginning and ending states and the paths of the neighboring sites. An exact path sampling algorithm is developed......The evolution of DNA sequences can be described by discrete state continuous time Markov processes on a phylogenetic tree. We consider neighbor-dependent evolutionary models where the instantaneous rate of substitution at a site depends on the states of the neighboring sites. Neighbor......-dependent substitution models are analytically intractable and must be analyzed using either approximate or simulation-based methods. We describe statistical inference of neighbor-dependent models using a Markov chain Monte Carlo expectation maximization (MCMC-EM) algorithm. In the MCMC-EM algorithm, the high...
Nonlinear Markov processes: Deterministic case
International Nuclear Information System (INIS)
Frank, T.D.
2008-01-01
Deterministic Markov processes that exhibit nonlinear transition mechanisms for probability densities are studied. In this context, the following issues are addressed: Markov property, conditional probability densities, propagation of probability densities, multistability in terms of multiple stationary distributions, stability analysis of stationary distributions, and basin of attraction of stationary distribution
Renewal characterization of Markov modulated Poisson processes
Directory of Open Access Journals (Sweden)
Marcel F. Neuts
1989-01-01
Full Text Available A Markov Modulated Poisson Process (MMPP M(t defined on a Markov chain J(t is a pure jump process where jumps of M(t occur according to a Poisson process with intensity λi whenever the Markov chain J(t is in state i. M(t is called strongly renewal (SR if M(t is a renewal process for an arbitrary initial probability vector of J(t with full support on P={i:λi>0}. M(t is called weakly renewal (WR if there exists an initial probability vector of J(t such that the resulting MMPP is a renewal process. The purpose of this paper is to develop general characterization theorems for the class SR and some sufficiency theorems for the class WR in terms of the first passage times of the bivariate Markov chain [J(t,M(t]. Relevance to the lumpability of J(t is also studied.
Noise can speed convergence in Markov chains.
Franzke, Brandon; Kosko, Bart
2011-10-01
A new theorem shows that noise can speed convergence to equilibrium in discrete finite-state Markov chains. The noise applies to the state density and helps the Markov chain explore improbable regions of the state space. The theorem ensures that a stochastic-resonance noise benefit exists for states that obey a vector-norm inequality. Such noise leads to faster convergence because the noise reduces the norm components. A corollary shows that a noise benefit still occurs if the system states obey an alternate norm inequality. This leads to a noise-benefit algorithm that requires knowledge of the steady state. An alternative blind algorithm uses only past state information to achieve a weaker noise benefit. Simulations illustrate the predicted noise benefits in three well-known Markov models. The first model is a two-parameter Ehrenfest diffusion model that shows how noise benefits can occur in the class of birth-death processes. The second model is a Wright-Fisher model of genotype drift in population genetics. The third model is a chemical reaction network of zeolite crystallization. A fourth simulation shows a convergence rate increase of 64% for states that satisfy the theorem and an increase of 53% for states that satisfy the corollary. A final simulation shows that even suboptimal noise can speed convergence if the noise applies over successive time cycles. Noise benefits tend to be sharpest in Markov models that do not converge quickly and that do not have strong absorbing states.
28 CFR 301.204 - Continuation of lost-time wages.
2010-07-01
... 28 Judicial Administration 2 2010-07-01 2010-07-01 false Continuation of lost-time wages. 301.204... ACCIDENT COMPENSATION Lost-Time Wages § 301.204 Continuation of lost-time wages. (a) Once approved, the inmate shall receive lost-time wages until the inmate: (1) Is released; (2) Is transferred to another...
International Nuclear Information System (INIS)
Floriani, Elena; Lima, Ricardo; Ourrad, Ouerdia; Spinelli, Lionel
2016-01-01
Highlights: • The flux through a Markov chain of a conserved quantity (mass) is studied. • Mass is supplied by an external source and ends in the absorbing states of the chain. • Meaningful for modeling open systems whose dynamics has a Markov property. • The analytical expression of mass distribution is given for a constant source. • The expression of mass distribution is given for periodic or random sources. - Abstract: In this paper we study the flux through a finite Markov chain of a quantity, that we will call mass, which moves through the states of the chain according to the Markov transition probabilities. Mass is supplied by an external source and accumulates in the absorbing states of the chain. We believe that studying how this conserved quantity evolves through the transient (non-absorbing) states of the chain could be useful for the modelization of open systems whose dynamics has a Markov property.
Bayesian posterior distributions without Markov chains.
Cole, Stephen R; Chu, Haitao; Greenland, Sander; Hamra, Ghassan; Richardson, David B
2012-03-01
Bayesian posterior parameter distributions are often simulated using Markov chain Monte Carlo (MCMC) methods. However, MCMC methods are not always necessary and do not help the uninitiated understand Bayesian inference. As a bridge to understanding Bayesian inference, the authors illustrate a transparent rejection sampling method. In example 1, they illustrate rejection sampling using 36 cases and 198 controls from a case-control study (1976-1983) assessing the relation between residential exposure to magnetic fields and the development of childhood cancer. Results from rejection sampling (odds ratio (OR) = 1.69, 95% posterior interval (PI): 0.57, 5.00) were similar to MCMC results (OR = 1.69, 95% PI: 0.58, 4.95) and approximations from data-augmentation priors (OR = 1.74, 95% PI: 0.60, 5.06). In example 2, the authors apply rejection sampling to a cohort study of 315 human immunodeficiency virus seroconverters (1984-1998) to assess the relation between viral load after infection and 5-year incidence of acquired immunodeficiency syndrome, adjusting for (continuous) age at seroconversion and race. In this more complex example, rejection sampling required a notably longer run time than MCMC sampling but remained feasible and again yielded similar results. The transparency of the proposed approach comes at a price of being less broadly applicable than MCMC.
Markov's theorem and algorithmically non-recognizable combinatorial manifolds
International Nuclear Information System (INIS)
Shtan'ko, M A
2004-01-01
We prove the theorem of Markov on the existence of an algorithmically non-recognizable combinatorial n-dimensional manifold for every n≥4. We construct for the first time a concrete manifold which is algorithmically non-recognizable. A strengthened form of Markov's theorem is proved using the combinatorial methods of regular neighbourhoods and handle theory. The proofs coincide for all n≥4. We use Borisov's group with insoluble word problem. It has two generators and twelve relations. The use of this group forms the base for proving the strengthened form of Markov's theorem
Accurate Lithium-ion battery parameter estimation with continuous-time system identification methods
International Nuclear Information System (INIS)
Xia, Bing; Zhao, Xin; Callafon, Raymond de; Garnier, Hugues; Nguyen, Truong; Mi, Chris
2016-01-01
Highlights: • Continuous-time system identification is applied in Lithium-ion battery modeling. • Continuous-time and discrete-time identification methods are compared in detail. • The instrumental variable method is employed to further improve the estimation. • Simulations and experiments validate the advantages of continuous-time methods. - Abstract: The modeling of Lithium-ion batteries usually utilizes discrete-time system identification methods to estimate parameters of discrete models. However, in real applications, there is a fundamental limitation of the discrete-time methods in dealing with sensitivity when the system is stiff and the storage resolutions are limited. To overcome this problem, this paper adopts direct continuous-time system identification methods to estimate the parameters of equivalent circuit models for Lithium-ion batteries. Compared with discrete-time system identification methods, the continuous-time system identification methods provide more accurate estimates to both fast and slow dynamics in battery systems and are less sensitive to disturbances. A case of a 2"n"d-order equivalent circuit model is studied which shows that the continuous-time estimates are more robust to high sampling rates, measurement noises and rounding errors. In addition, the estimation by the conventional continuous-time least squares method is further improved in the case of noisy output measurement by introducing the instrumental variable method. Simulation and experiment results validate the analysis and demonstrate the advantages of the continuous-time system identification methods in battery applications.
DEFF Research Database (Denmark)
Hobolth, Asger
2008-01-01
The evolution of DNA sequences can be described by discrete state continuous time Markov processes on a phylogenetic tree. We consider neighbor-dependent evolutionary models where the instantaneous rate of substitution at a site depends on the states of the neighboring sites. Neighbor...
Lithofacies cyclicity determination in the guaduas formation (Colombia using Markov chains
Directory of Open Access Journals (Sweden)
Jorge Eliecer Mariño Martinez
2016-07-01
Full Text Available Statistical embedded Markov Chain processes were used to analyze facies transitions and to determine the stacking pattern of the lithofacies of the Guaduas Formation. Twelve Lithofacies were found and characterized based on lithology and sedimentary structures in four stratigraphic sections. The findings were compared with a previous assemblage of lithofacies, interpretations of sedimentary environments, and depositional systems. As a result, four depositional Systems were established. Through the statistical analyses of facies transitions it was found that tidal facies are prevalent in the Socota section, especially in the upper part, whereas in the Sogamoso, Umbita and Peñas de Sutatausa sections, fluvial facies are prevalent in the upper part of the sections, and follow a regressive sequence with more continental deposits around the upper part of the sections. For each of these sections the Markov Chain transition matrices illustrates a strong interaction between tidal facies and fluvial facies, specially in the Peñas de Sutatausa matrix, where facies 6, made up of tidal deposits, appears several times. From the facies model and Markov Chain analyses, it is evident that the Guaduas Formation is a cyclic sequence in which the Markov facies repetitions are consistent with the lithofacies analyses conducted in previous stratigraphic studies. The results reveal that the Markov Chain statistical process can be used to predict stratigraphy in order to correlate contiguous geologically unexplored areas in the Guaduas Formation, where much work relating to correlation and the continuity of coal beds has yet to be done. Determinacion de la ciclicidad de las facies en la formacion Guaduas (Colombia usando las cadenas de Markov Resumen Se utilizaron los procesos estadísticos de las cadenas de Markov para analizar las transiciones de facies y para determinar el patrón de apilamiento de las litofacies de la formación Guaduas. Se encontraron y
International Nuclear Information System (INIS)
Hirschmann, H.
1983-06-01
The consequences of the basic assumptions of the semi-Markov process as defined from a homogeneous renewal process with a stationary Markov condition are reviewed. The notion of the semi-Markov process is generalized by its redefinition from a nonstationary Markov renewal process. For both the nongeneralized and the generalized case a representation of the first order conditional state probabilities is derived in terms of the transition probabilities of the Markov renewal process. Some useful calculation rules (regeneration rules) are derived for the conditional state probabilities of the semi-Markov process. Compared to the semi-Markov process in its usual definition the generalized process allows the analysis of a larger class of systems. For instance systems with arbitrarily distributed lifetimes of their components can be described. There is also a chance to describe systems which are modified during time by forces or manipulations from outside. (Auth.)
Quadratic Variation by Markov Chains
DEFF Research Database (Denmark)
Hansen, Peter Reinhard; Horel, Guillaume
We introduce a novel estimator of the quadratic variation that is based on the the- ory of Markov chains. The estimator is motivated by some general results concerning filtering contaminated semimartingales. Specifically, we show that filtering can in prin- ciple remove the effects of market...... microstructure noise in a general framework where little is assumed about the noise. For the practical implementation, we adopt the dis- crete Markov chain model that is well suited for the analysis of financial high-frequency prices. The Markov chain framework facilitates simple expressions and elegant analyti...
The application of Markov decision process in restaurant delivery robot
Wang, Yong; Hu, Zhen; Wang, Ying
2017-05-01
As the restaurant delivery robot is often in a dynamic and complex environment, including the chairs inadvertently moved to the channel and customers coming and going. The traditional path planning algorithm is not very ideal. To solve this problem, this paper proposes the Markov dynamic state immediate reward (MDR) path planning algorithm according to the traditional Markov decision process. First of all, it uses MDR to plan a global path, then navigates along this path. When the sensor detects there is no obstructions in front state, increase its immediate state reward value; when the sensor detects there is an obstacle in front, plan a global path that can avoid obstacle with the current position as the new starting point and reduce its state immediate reward value. This continues until the target is reached. When the robot learns for a period of time, it can avoid those places where obstacles are often present when planning the path. By analyzing the simulation experiment, the algorithm has achieved good results in the global path planning under the dynamic environment.
Markov chain aggregation and its applications to combinatorial reaction networks.
Ganguly, Arnab; Petrov, Tatjana; Koeppl, Heinz
2014-09-01
We consider a continuous-time Markov chain (CTMC) whose state space is partitioned into aggregates, and each aggregate is assigned a probability measure. A sufficient condition for defining a CTMC over the aggregates is presented as a variant of weak lumpability, which also characterizes that the measure over the original process can be recovered from that of the aggregated one. We show how the applicability of de-aggregation depends on the initial distribution. The application section is devoted to illustrate how the developed theory aids in reducing CTMC models of biochemical systems particularly in connection to protein-protein interactions. We assume that the model is written by a biologist in form of site-graph-rewrite rules. Site-graph-rewrite rules compactly express that, often, only a local context of a protein (instead of a full molecular species) needs to be in a certain configuration in order to trigger a reaction event. This observation leads to suitable aggregate Markov chains with smaller state spaces, thereby providing sufficient reduction in computational complexity. This is further exemplified in two case studies: simple unbounded polymerization and early EGFR/insulin crosstalk.
Sumner, Jeremy G; Taylor, Amelia; Holland, Barbara R; Jarvis, Peter D
2017-12-01
Recently there has been renewed interest in phylogenetic inference methods based on phylogenetic invariants, alongside the related Markov invariants. Broadly speaking, both these approaches give rise to polynomial functions of sequence site patterns that, in expectation value, either vanish for particular evolutionary trees (in the case of phylogenetic invariants) or have well understood transformation properties (in the case of Markov invariants). While both approaches have been valued for their intrinsic mathematical interest, it is not clear how they relate to each other, and to what extent they can be used as practical tools for inference of phylogenetic trees. In this paper, by focusing on the special case of binary sequence data and quartets of taxa, we are able to view these two different polynomial-based approaches within a common framework. To motivate the discussion, we present three desirable statistical properties that we argue any invariant-based phylogenetic method should satisfy: (1) sensible behaviour under reordering of input sequences; (2) stability as the taxa evolve independently according to a Markov process; and (3) explicit dependence on the assumption of a continuous-time process. Motivated by these statistical properties, we develop and explore several new phylogenetic inference methods. In particular, we develop a statistically bias-corrected version of the Markov invariants approach which satisfies all three properties. We also extend previous work by showing that the phylogenetic invariants can be implemented in such a way as to satisfy property (3). A simulation study shows that, in comparison to other methods, our new proposed approach based on bias-corrected Markov invariants is extremely powerful for phylogenetic inference. The binary case is of particular theoretical interest as-in this case only-the Markov invariants can be expressed as linear combinations of the phylogenetic invariants. A wider implication of this is that, for
Markov Chain Monte Carlo Methods
Indian Academy of Sciences (India)
Systat Software Asia-Pacific. Ltd., in Bangalore, where the technical work for the development of the statistical software Systat takes ... In Part 4, we discuss some applications of the Markov ... one can construct the joint probability distribution of.
Confluence reduction for Markov automata
Timmer, Mark; van de Pol, Jan Cornelis; Stoelinga, Mariëlle Ida Antoinette
Markov automata are a novel formalism for specifying systems exhibiting nondeterminism, probabilistic choices and Markovian rates. Recently, the process algebra MAPA was introduced to efficiently model such systems. As always, the state space explosion threatens the analysability of the models
Confluence Reduction for Markov Automata
Timmer, Mark; van de Pol, Jan Cornelis; Stoelinga, Mariëlle Ida Antoinette; Braberman, Victor; Fribourg, Laurent
Markov automata are a novel formalism for specifying systems exhibiting nondeterminism, probabilistic choices and Markovian rates. Recently, the process algebra MAPA was introduced to efficiently model such systems. As always, the state space explosion threatens the analysability of the models
Stochastic demand patterns for Markov service facilities with neutral and active periods
International Nuclear Information System (INIS)
Csenki, Attila
2009-01-01
In an earlier paper, a closed form expression was obtained for the joint interval reliability of a Markov system with a partitioned state space S=U union D, i.e. for the probability that the system will reside in the set of up states U throughout the union of some specific disjoint time intervals I l =[θ l ,θ l +ζ l ],l=1,...,k. The deterministic time intervals I l formed a demand pattern specifying the desired active periods. In the present paper, we admit stochastic demand patterns by assuming that the lengths of the active periods, ζ l , as well as the lengths of the neutral periods, θ l -(θ l-1 +ζ l-1 ), are random. We explore two mechanisms for modelling random demand: (1) by alternating renewal processes; (2) by sojourn times of some continuous time Markov chain with a partitioned state space. The first construction results in an expression in terms of a revised version of the moment generating functions of the sojourns of the alternating renewal process. The second construction involves the probability that a Markov chain follows certain patterns of visits to some groups of states and yields an expression using Kronecker matrix operations. The model of a small computer system is analysed to exemplify the ideas
Integrating Continuous-Time and Discrete-Event Concepts in Process Modelling, Simulation and Control
Beek, van D.A.; Gordijn, S.H.F.; Rooda, J.E.; Ertas, A.
1995-01-01
Currently, modelling of systems in the process industry requires the use of different specification languages for the specification of the discrete-event and continuous-time subsystems. In this way, models are restricted to individual subsystems of either a continuous-time or discrete-event nature.
A Multistep Extending Truncation Method towards Model Construction of Infinite-State Markov Chains
Directory of Open Access Journals (Sweden)
Kemin Wang
2014-01-01
Full Text Available The model checking of Infinite-State Continuous Time Markov Chains will inevitably encounter the state explosion problem when constructing the CTMCs model; our method is to get a truncated model of the infinite one; to get a sufficient truncated model to meet the model checking of Continuous Stochastic Logic based system properties, we propose a multistep extending advanced truncation method towards model construction of CTMCs and implement it in the INFAMY model checker; the experiment results show that our method is effective.
Markov chain model helps predict pitting corrosion depth and rate in underground pipelines
Energy Technology Data Exchange (ETDEWEB)
Caleyo, F.; Velazquez, J.C.; Hallen, J. M. [ESIQIE, Instituto Politecnico Nacional, Mexico D. F. (Mexico); Esquivel-Amezcua, A. [PEMEX PEP Region Sur, Villahermosa, Tabasco (Mexico); Valor, A. [Universidad de la Habana, Vedado, La Habana (Cuba)
2010-07-01
Recent reports place pipeline corrosion costs in North America at seven billion dollars per year. Pitting corrosion causes the higher percentage of failures among other corrosion mechanisms. This has motivated multiple modelling studies to be focused on corrosion pitting of underground pipelines. In this study, a continuous-time, non-homogenous pure birth Markov chain serves to model external pitting corrosion in buried pipelines. The analytical solution of Kolmogorov's forward equations for this type of Markov process gives the transition probability function in a discrete space of pit depths. The transition probability function can be completely identified by making a correlation between the stochastic pit depth mean and the deterministic mean obtained experimentally. The model proposed in this study can be applied to pitting corrosion data from repeated in-line pipeline inspections. Case studies presented in this work show how pipeline inspection and maintenance planning can be improved by using the proposed Markovian model for pitting corrosion.
Classification of customer lifetime value models using Markov chain
Permana, Dony; Pasaribu, Udjianna S.; Indratno, Sapto W.; Suprayogi
2017-10-01
A firm’s potential reward in future time from a customer can be determined by customer lifetime value (CLV). There are some mathematic methods to calculate it. One method is using Markov chain stochastic model. Here, a customer is assumed through some states. Transition inter the states follow Markovian properties. If we are given some states for a customer and the relationships inter states, then we can make some Markov models to describe the properties of the customer. As Markov models, CLV is defined as a vector contains CLV for a customer in the first state. In this paper we make a classification of Markov Models to calculate CLV. Start from two states of customer model, we make develop in many states models. The development a model is based on weaknesses in previous model. Some last models can be expected to describe how real characters of customers in a firm.
International Nuclear Information System (INIS)
Huo Haifeng; Li Wantong
2009-01-01
This paper is concerned with the global stability characteristics of a system of equations modelling the dynamics of continuous-time bidirectional associative memory neural networks with impulses. Sufficient conditions which guarantee the existence of a unique equilibrium and its exponential stability of the networks are obtained. For the goal of computation, discrete-time analogues of the corresponding continuous-time bidirectional associative memory neural networks with impulses are also formulated and studied. Our results show that the above continuous-time and discrete-time systems with impulses preserve the dynamics of the networks without impulses when we make some modifications and impose some additional conditions on the systems, the convergence characteristics dynamics of the networks are preserved by both continuous-time and discrete-time systems with some restriction imposed on the impulse effect.
Stochastic Dynamics through Hierarchically Embedded Markov Chains.
Vasconcelos, Vítor V; Santos, Fernando P; Santos, Francisco C; Pacheco, Jorge M
2017-02-03
Studying dynamical phenomena in finite populations often involves Markov processes of significant mathematical and/or computational complexity, which rapidly becomes prohibitive with increasing population size or an increasing number of individual configuration states. Here, we develop a framework that allows us to define a hierarchy of approximations to the stationary distribution of general systems that can be described as discrete Markov processes with time invariant transition probabilities and (possibly) a large number of states. This results in an efficient method for studying social and biological communities in the presence of stochastic effects-such as mutations in evolutionary dynamics and a random exploration of choices in social systems-including situations where the dynamics encompasses the existence of stable polymorphic configurations, thus overcoming the limitations of existing methods. The present formalism is shown to be general in scope, widely applicable, and of relevance to a variety of interdisciplinary problems.
Energy Technology Data Exchange (ETDEWEB)
Balan, I.
2005-05-01
This work presents the implementation of the Adjoint Sensitivity Analysis Procedure (ASAP) for the Continuous Time, Discrete Space Markov chains (CTMC), as an alternative to the other computational expensive methods. In order to develop this procedure as an end product in reliability studies, the reliability of the physical systems is analyzed using a coupled Fault-Tree - Markov chain technique, i.e. the abstraction of the physical system is performed using as the high level interface the Fault-Tree and afterwards this one is automatically converted into a Markov chain. The resulting differential equations based on the Markov chain model are solved in order to evaluate the system reliability. Further sensitivity analyses using ASAP applied to CTMC equations are performed to study the influence of uncertainties in input data to the reliability measures and to get the confidence in the final reliability results. The methods to generate the Markov chain and the ASAP for the Markov chain equations have been implemented into the new computer code system QUEFT/MARKOMAGS/MCADJSEN for reliability and sensitivity analysis of physical systems. The validation of this code system has been carried out by using simple problems for which analytical solutions can be obtained. Typical sensitivity results show that the numerical solution using ASAP is robust, stable and accurate. The method and the code system developed during this work can be used further as an efficient and flexible tool to evaluate the sensitivities of reliability measures for any physical system analyzed using the Markov chain. Reliability and sensitivity analyses using these methods have been performed during this work for the IFMIF Accelerator System Facilities. The reliability studies using Markov chain have been concentrated around the availability of the main subsystems of this complex physical system for a typical mission time. The sensitivity studies for two typical responses using ASAP have been
Perturbation approach to scaled type Markov renewal processes with infinite mean
Pajor-Gyulai, Zsolt; Szász, Domokos
2010-01-01
Scaled type Markov renewal processes generalize classical renewal processes: renewal times come from a one parameter family of probability laws and the sequence of the parameters is the trajectory of an ergodic Markov chain. Our primary interest here is the asymptotic distribution of the Markovian parameter at time t \\to \\infty. The limit, of course, depends on the stationary distribution of the Markov chain. The results, however, are essentially different depending on whether the expectation...
A Multilayer Hidden Markov Models-Based Method for Human-Robot Interaction
Directory of Open Access Journals (Sweden)
Chongben Tao
2013-01-01
Full Text Available To achieve Human-Robot Interaction (HRI by using gestures, a continuous gesture recognition approach based on Multilayer Hidden Markov Models (MHMMs is proposed, which consists of two parts. One part is gesture spotting and segment module, the other part is continuous gesture recognition module. Firstly, a Kinect sensor is used to capture 3D acceleration and 3D angular velocity data of hand gestures. And then, a Feed-forward Neural Networks (FNNs and a threshold criterion are used for gesture spotting and segment, respectively. Afterwards, the segmented gesture signals are respectively preprocessed and vector symbolized by a sliding window and a K-means clustering method. Finally, symbolized data are sent into Lower Hidden Markov Models (LHMMs to identify individual gestures, and then, a Bayesian filter with sequential constraints among gestures in Upper Hidden Markov Models (UHMMs is used to correct recognition errors created in LHMMs. Five predefined gestures are used to interact with a Kinect mobile robot in experiments. The experimental results show that the proposed method not only has good effectiveness and accuracy, but also has favorable real-time performance.
Soundness of Timed-Arc Workflow Nets in Discrete and Continuous-Time Semantics
DEFF Research Database (Denmark)
Mateo, Jose Antonio; Srba, Jiri; Sørensen, Mathias Grund
2015-01-01
Analysis of workflow processes with quantitative aspectslike timing is of interest in numerous time-critical applications. We suggest a workflow model based on timed-arc Petri nets and studythe foundational problems of soundness and strong (time-bounded) soundness.We first consider the discrete-t...
A Graph-Algorithmic Approach for the Study of Metastability in Markov Chains
Gan, Tingyue; Cameron, Maria
2017-06-01
Large continuous-time Markov chains with exponentially small transition rates arise in modeling complex systems in physics, chemistry, and biology. We propose a constructive graph-algorithmic approach to determine the sequence of critical timescales at which the qualitative behavior of a given Markov chain changes, and give an effective description of the dynamics on each of them. This approach is valid for both time-reversible and time-irreversible Markov processes, with or without symmetry. Central to this approach are two graph algorithms, Algorithm 1 and Algorithm 2, for obtaining the sequences of the critical timescales and the hierarchies of Typical Transition Graphs or T-graphs indicating the most likely transitions in the system without and with symmetry, respectively. The sequence of critical timescales includes the subsequence of the reciprocals of the real parts of eigenvalues. Under a certain assumption, we prove sharp asymptotic estimates for eigenvalues (including pre-factors) and show how one can extract them from the output of Algorithm 1. We discuss the relationship between Algorithms 1 and 2 and explain how one needs to interpret the output of Algorithm 1 if it is applied in the case with symmetry instead of Algorithm 2. Finally, we analyze an example motivated by R. D. Astumian's model of the dynamics of kinesin, a molecular motor, by means of Algorithm 2.
Elastic LiDAR Fusion: Dense Map-Centric Continuous-Time SLAM
Park, Chanoh; Moghadam, Peyman; Kim, Soohwan; Elfes, Alberto; Fookes, Clinton; Sridharan, Sridha
2017-01-01
The concept of continuous-time trajectory representation has brought increased accuracy and efficiency to multi-modal sensor fusion in modern SLAM. However, regardless of these advantages, its offline property caused by the requirement of global batch optimization is critically hindering its relevance for real-time and life-long applications. In this paper, we present a dense map-centric SLAM method based on a continuous-time trajectory to cope with this problem. The proposed system locally f...
Dalmau Codina, Ramon; Prats Menéndez, Xavier
2017-01-01
Continuous descent operations with controlled times of arrival at one or several metering fixes could enable environmentally friendly procedures without compromising terminal airspace capacity. This paper focuses on controlled time of arrival updates once the descent has been already initiated, assessing the feasible time window (and associated fuel consumption) of continuous descent operations requiring neither thrust nor speed-brake usage along the whole descent (i.e. only elevator control ...
Abbas, Ismail; Rovira, Joan; Casanovas, Josep
2007-05-01
The patient recruitment process of clinical trials is an essential element which needs to be designed properly. In this paper we describe different simulation models under continuous and discrete time assumptions for the design of recruitment in clinical trials. The results of hypothetical examples of clinical trial recruitments are presented. The recruitment time is calculated and the number of recruited patients is quantified for a given time and probability of recruitment. The expected delay and the effective recruitment durations are estimated using both continuous and discrete time modeling. The proposed type of Monte Carlo simulation Markov models will enable optimization of the recruitment process and the estimation and the calibration of its parameters to aid the proposed clinical trials. A continuous time simulation may minimize the duration of the recruitment and, consequently, the total duration of the trial.
Carl Chiarella; Chih-Ying Hsiao
2005-01-01
This paper considers an asset allocation strategy over a finite period under investment uncertainty and short-sale constraints as a continuous time stochastic control problem. Investment uncertainty is characterised by a stochastic interest rate and inflation risk. If there are no short-sale constraints, the optimal asset allocation strategy can be solved analytically. We consider several kinds of short-sale constraints and employ the backward Markov chain approximation method to explore the ...
CMOS continuous-time adaptive equalizers for high-speed serial links
Gimeno Gasca, Cecilia; Aldea Chagoyen, Concepción
2015-01-01
This book introduces readers to the design of adaptive equalization solutions integrated in standard CMOS technology for high-speed serial links. Since continuous-time equalizers offer various advantages as an alternative to discrete-time equalizers at multi-gigabit rates, this book provides a detailed description of continuous-time adaptive equalizers design - both at transistor and system levels-, their main characteristics and performances. The authors begin with a complete review and analysis of the state of the art of equalizers for wireline applications, describing why they are necessary, their types, and their main applications. Next, theoretical fundamentals of continuous-time adaptive equalizers are explored. Then, new structures are proposed to implement the different building blocks of the adaptive equalizer: line equalizer, loop-filters, power comparator, etc. The authors demonstrate the design of a complete low-power, low-voltage, high-speed, continuous-time adaptive equalizer. Finally, a cost-...
Elliott, Thomas J.; Gu, Mile
2018-03-01
Continuous-time stochastic processes pervade everyday experience, and the simulation of models of these processes is of great utility. Classical models of systems operating in continuous-time must typically track an unbounded amount of information about past behaviour, even for relatively simple models, enforcing limits on precision due to the finite memory of the machine. However, quantum machines can require less information about the past than even their optimal classical counterparts to simulate the future of discrete-time processes, and we demonstrate that this advantage extends to the continuous-time regime. Moreover, we show that this reduction in the memory requirement can be unboundedly large, allowing for arbitrary precision even with a finite quantum memory. We provide a systematic method for finding superior quantum constructions, and a protocol for analogue simulation of continuous-time renewal processes with a quantum machine.
Markov Processes in Image Processing
Petrov, E. P.; Kharina, N. L.
2018-05-01
Digital images are used as an information carrier in different sciences and technologies. The aspiration to increase the number of bits in the image pixels for the purpose of obtaining more information is observed. In the paper, some methods of compression and contour detection on the basis of two-dimensional Markov chain are offered. Increasing the number of bits on the image pixels will allow one to allocate fine object details more precisely, but it significantly complicates image processing. The methods of image processing do not concede by the efficiency to well-known analogues, but surpass them in processing speed. An image is separated into binary images, and processing is carried out in parallel with each without an increase in speed, when increasing the number of bits on the image pixels. One more advantage of methods is the low consumption of energy resources. Only logical procedures are used and there are no computing operations. The methods can be useful in processing images of any class and assignment in processing systems with a limited time and energy resources.
NonMarkov Ito Processes with 1- state memory
McCauley, Joseph L.
2010-08-01
A Markov process, by definition, cannot depend on any previous state other than the last observed state. An Ito process implies the Fokker-Planck and Kolmogorov backward time partial differential eqns. for transition densities, which in turn imply the Chapman-Kolmogorov eqn., but without requiring the Markov condition. We present a class of Ito process superficially resembling Markov processes, but with 1-state memory. In finance, such processes would obey the efficient market hypothesis up through the level of pair correlations. These stochastic processes have been mislabeled in recent literature as 'nonlinear Markov processes'. Inspired by Doob and Feller, who pointed out that the ChapmanKolmogorov eqn. is not restricted to Markov processes, we exhibit a Gaussian Ito transition density with 1-state memory in the drift coefficient that satisfies both of Kolmogorov's partial differential eqns. and also the Chapman-Kolmogorov eqn. In addition, we show that three of the examples from McKean's seminal 1966 paper are also nonMarkov Ito processes. Last, we show that the transition density of the generalized Black-Scholes type partial differential eqn. describes a martingale, and satisfies the ChapmanKolmogorov eqn. This leads to the shortest-known proof that the Green function of the Black-Scholes eqn. with variable diffusion coefficient provides the so-called martingale measure of option pricing.
English, Thomas
2005-01-01
A standard tool of reliability analysis used at NASA-JSC is the event tree. An event tree is simply a probability tree, with the probabilities determining the next step through the tree specified at each node. The nodal probabilities are determined by a reliability study of the physical system at work for a particular node. The reliability study performed at a node is typically referred to as a fault tree analysis, with the potential of a fault tree existing.for each node on the event tree. When examining an event tree it is obvious why the event tree/fault tree approach has been adopted. Typical event trees are quite complex in nature, and the event tree/fault tree approach provides a systematic and organized approach to reliability analysis. The purpose of this study was two fold. Firstly, we wanted to explore the possibility that a semi-Markov process can create dependencies between sojourn times (the times it takes to transition from one state to the next) that can decrease the uncertainty when estimating time to failures. Using a generalized semi-Markov model, we studied a four element reliability model and were able to demonstrate such sojourn time dependencies. Secondly, we wanted to study the use of semi-Markov processes to introduce a time variable into the event tree diagrams that are commonly developed in PRA (Probabilistic Risk Assessment) analyses. Event tree end states which change with time are more representative of failure scenarios than are the usual static probability-derived end states.
Markov-modulated infinite-server queues driven by a common background process
Mandjes , Michel; De Turck , Koen
2016-01-01
International audience; This paper studies a system with multiple infinite-server queues which are modulated by a common background process. If this background process, being modeled as a finite-state continuous-time Markov chain, is in state j, then the arrival rate into the i-th queue is λi,j, whereas the service times of customers present in this queue are exponentially distributed with mean µ −1 i,j ; at each of the individual queues all customers present are served in parallel (thus refl...
Growth of preferential attachment random graphs via continuous ...
Indian Academy of Sciences (India)
Preferential attachment processes have a long history dating back at least to Yule ... We remark that some connections to branching and continuous-time Markov ..... convenience, we provide a short proof of Lemma 2.1 in the general form in ...
Delsing, M.J.M.H.; Oud, J.H.L.; Bruyn, E.E.J. De
2005-01-01
In family research, bidirectional influences between the family and the individual are usually analyzed in discrete time. Results from discrete time analysis, however, have been shown to be highly dependent on the length of the observation interval. Continuous time analysis using stochastic
A New Approach to Rational Discrete-Time Approximations to Continuous-Time Fractional-Order Systems
Matos , Carlos; Ortigueira , Manuel ,
2012-01-01
Part 10: Signal Processing; International audience; In this paper a new approach to rational discrete-time approximations to continuous fractional-order systems of the form 1/(sα+p) is proposed. We will show that such fractional-order LTI system can be decomposed into sub-systems. One has the classic behavior and the other is similar to a Finite Impulse Response (FIR) system. The conversion from continuous-time to discrete-time systems will be done using the Laplace transform inversion integr...
Tornadoes and related damage costs: statistical modeling with a semi-Markov approach
Corini, Chiara; D'Amico, Guglielmo; Petroni, Filippo; Prattico, Flavio; Manca, Raimondo
2015-01-01
We propose a statistical approach to tornadoes modeling for predicting and simulating occurrences of tornadoes and accumulated cost distributions over a time interval. This is achieved by modeling the tornadoes intensity, measured with the Fujita scale, as a stochastic process. Since the Fujita scale divides tornadoes intensity into six states, it is possible to model the tornadoes intensity by using Markov and semi-Markov models. We demonstrate that the semi-Markov approach is able to reprod...
Social security as Markov equilibrium in OLG models: A note
DEFF Research Database (Denmark)
Gonzalez Eiras, Martin
2011-01-01
I refine and extend the Markov perfect equilibrium of the social security policy game in Forni (2005) for the special case of logarithmic utility. Under the restriction that the policy function be continuous, instead of differentiable, the equilibrium is globally well defined and its dynamics...
Performability assessment by model checking of Markov reward models
Baier, Christel; Cloth, L.; Haverkort, Boudewijn R.H.M.; Hermanns, H.; Katoen, Joost P.
2010-01-01
This paper describes efficient procedures for model checking Markov reward models, that allow us to evaluate, among others, the performability of computer-communication systems. We present the logic CSRL (Continuous Stochastic Reward Logic) to specify performability measures. It provides flexibility
On Characterisation of Markov Processes Via Martingale Problems
Indian Academy of Sciences (India)
This extension is used to improve on a criterion for a probability measure to be invariant for the semigroup associated with the Markov process. We also give examples of martingale problems that are well-posed in the class of solutions which are continuous in probability but for which no r.c.l.l. solution exists.
Hardware solution for continuous time-resolved burst detection of single molecules in flow
Wahl, Michael; Erdmann, Rainer; Lauritsen, Kristian; Rahn, Hans-Juergen
1998-04-01
Time Correlated Single Photon Counting (TCSPC) is a valuable tool for Single Molecule Detection (SMD). However, existing TCSPC systems did not support continuous data collection and processing as is desirable for applications such as SMD for e.g. DNA-sequencing in a liquid flow. First attempts at using existing instrumentation in this kind of operation mode required additional routing hardware to switch between several memory banks and were not truly continuous. We have designed a hard- and software system to perform continuous real-time TCSPC based upon a modern solid state Time to Digital Converter (TDC). Short dead times of the fully digital TDC design combined with fast Field Programmable Gay Array logic permit a continuous data throughput as high as 3 Mcounts/sec. The histogramming time may be set as short as 100 microsecond(s) . Every histogram or every single fluorescence photon can be real-time tagged at 200 ns resolution in addition to recording its arrival time relative to the excitation pulse. Continuous switching between memory banks permits concurrent histogramming and data read-out. The instrument provides a time resolution of 60 ps and up to 4096 histogram channels. The overall instrument response function in combination with a low cost picosecond diode laser and an inexpensive photomultiplier tube was found to be 180 ps and well sufficient to measure sub-nanosecond fluorescence lifetimes.
Maximizing Entropy over Markov Processes
DEFF Research Database (Denmark)
Biondi, Fabrizio; Legay, Axel; Nielsen, Bo Friis
2013-01-01
The channel capacity of a deterministic system with confidential data is an upper bound on the amount of bits of data an attacker can learn from the system. We encode all possible attacks to a system using a probabilistic specification, an Interval Markov Chain. Then the channel capacity...... as a reward function, a polynomial algorithm to verify the existence of an system maximizing entropy among those respecting a specification, a procedure for the maximization of reward functions over Interval Markov Chains and its application to synthesize an implementation maximizing entropy. We show how...... to use Interval Markov Chains to model abstractions of deterministic systems with confidential data, and use the above results to compute their channel capacity. These results are a foundation for ongoing work on computing channel capacity for abstractions of programs derived from code....
Maximizing entropy over Markov processes
DEFF Research Database (Denmark)
Biondi, Fabrizio; Legay, Axel; Nielsen, Bo Friis
2014-01-01
The channel capacity of a deterministic system with confidential data is an upper bound on the amount of bits of data an attacker can learn from the system. We encode all possible attacks to a system using a probabilistic specification, an Interval Markov Chain. Then the channel capacity...... as a reward function, a polynomial algorithm to verify the existence of a system maximizing entropy among those respecting a specification, a procedure for the maximization of reward functions over Interval Markov Chains and its application to synthesize an implementation maximizing entropy. We show how...... to use Interval Markov Chains to model abstractions of deterministic systems with confidential data, and use the above results to compute their channel capacity. These results are a foundation for ongoing work on computing channel capacity for abstractions of programs derived from code. © 2014 Elsevier...
Markov Networks in Evolutionary Computation
Shakya, Siddhartha
2012-01-01
Markov networks and other probabilistic graphical modes have recently received an upsurge in attention from Evolutionary computation community, particularly in the area of Estimation of distribution algorithms (EDAs). EDAs have arisen as one of the most successful experiences in the application of machine learning methods in optimization, mainly due to their efficiency to solve complex real-world optimization problems and their suitability for theoretical analysis. This book focuses on the different steps involved in the conception, implementation and application of EDAs that use Markov networks, and undirected models in general. It can serve as a general introduction to EDAs but covers also an important current void in the study of these algorithms by explaining the specificities and benefits of modeling optimization problems by means of undirected probabilistic models. All major developments to date in the progressive introduction of Markov networks based EDAs are reviewed in the book. Hot current researc...
Continuous and Discrete-Time Optimal Controls for an Isolated Signalized Intersection
Directory of Open Access Journals (Sweden)
Jiyuan Tan
2017-01-01
Full Text Available A classical control problem for an isolated oversaturated intersection is revisited with a focus on the optimal control policy to minimize total delay. The difference and connection between existing continuous-time planning models and recently proposed discrete-time planning models are studied. A gradient descent algorithm is proposed to convert the optimal control plan of the continuous-time model to the plan of the discrete-time model in many cases. Analytic proof and numerical tests for the algorithm are also presented. The findings shed light on the links between two kinds of models.
Markov Chain Analysis of Musical Dice Games
Volchenkov, D.; Dawin, J. R.
2012-07-01
A system for using dice to compose music randomly is known as the musical dice game. The discrete time MIDI models of 804 pieces of classical music written by 29 composers have been encoded into the transition matrices and studied by Markov chains. Contrary to human languages, entropy dominates over redundancy, in the musical dice games based on the compositions of classical music. The maximum complexity is achieved on the blocks consisting of just a few notes (8 notes, for the musical dice games generated over Bach's compositions). First passage times to notes can be used to resolve tonality and feature a composer.
Markov Models for Handwriting Recognition
Plotz, Thomas
2011-01-01
Since their first inception, automatic reading systems have evolved substantially, yet the recognition of handwriting remains an open research problem due to its substantial variation in appearance. With the introduction of Markovian models to the field, a promising modeling and recognition paradigm was established for automatic handwriting recognition. However, no standard procedures for building Markov model-based recognizers have yet been established. This text provides a comprehensive overview of the application of Markov models in the field of handwriting recognition, covering both hidden
Monotone measures of ergodicity for Markov chains
Directory of Open Access Journals (Sweden)
J. Keilson
1998-01-01
Full Text Available The following paper, first written in 1974, was never published other than as part of an internal research series. Its lack of publication is unrelated to the merits of the paper and the paper is of current importance by virtue of its relation to the relaxation time. A systematic discussion is provided of the approach of a finite Markov chain to ergodicity by proving the monotonicity of an important set of norms, each measures of egodicity, whether or not time reversibility is present. The paper is of particular interest because the discussion of the relaxation time of a finite Markov chain [2] has only been clean for time reversible chains, a small subset of the chains of interest. This restriction is not present here. Indeed, a new relaxation time quoted quantifies the relaxation time for all finite ergodic chains (cf. the discussion of Q1(t below Equation (1.7]. This relaxation time was developed by Keilson with A. Roy in his thesis [6], yet to be published.
Constructing Dynamic Event Trees from Markov Models
International Nuclear Information System (INIS)
Paolo Bucci; Jason Kirschenbaum; Tunc Aldemir; Curtis Smith; Ted Wood
2006-01-01
In the probabilistic risk assessment (PRA) of process plants, Markov models can be used to model accurately the complex dynamic interactions between plant physical process variables (e.g., temperature, pressure, etc.) and the instrumentation and control system that monitors and manages the process. One limitation of this approach that has prevented its use in nuclear power plant PRAs is the difficulty of integrating the results of a Markov analysis into an existing PRA. In this paper, we explore a new approach to the generation of failure scenarios and their compilation into dynamic event trees from a Markov model of the system. These event trees can be integrated into an existing PRA using software tools such as SAPHIRE. To implement our approach, we first construct a discrete-time Markov chain modeling the system of interest by: (a) partitioning the process variable state space into magnitude intervals (cells), (b) using analytical equations or a system simulator to determine the transition probabilities between the cells through the cell-to-cell mapping technique, and, (c) using given failure/repair data for all the components of interest. The Markov transition matrix thus generated can be thought of as a process model describing the stochastic dynamic behavior of the finite-state system. We can therefore search the state space starting from a set of initial states to explore all possible paths to failure (scenarios) with associated probabilities. We can also construct event trees of arbitrary depth by tracing paths from a chosen initiating event and recording the following events while keeping track of the probabilities associated with each branch in the tree. As an example of our approach, we use the simple level control system often used as benchmark in the literature with one process variable (liquid level in a tank), and three control units: a drain unit and two supply units. Each unit includes a separate level sensor to observe the liquid level in the tank
The Markov process admits a consistent steady-state thermodynamic formalism
Peng, Liangrong; Zhu, Yi; Hong, Liu
2018-01-01
The search for a unified formulation for describing various non-equilibrium processes is a central task of modern non-equilibrium thermodynamics. In this paper, a novel steady-state thermodynamic formalism was established for general Markov processes described by the Chapman-Kolmogorov equation. Furthermore, corresponding formalisms of steady-state thermodynamics for the master equation and Fokker-Planck equation could be rigorously derived in mathematics. To be concrete, we proved that (1) in the limit of continuous time, the steady-state thermodynamic formalism for the Chapman-Kolmogorov equation fully agrees with that for the master equation; (2) a similar one-to-one correspondence could be established rigorously between the master equation and Fokker-Planck equation in the limit of large system size; (3) when a Markov process is restrained to one-step jump, the steady-state thermodynamic formalism for the Fokker-Planck equation with discrete state variables also goes to that for master equations, as the discretization step gets smaller and smaller. Our analysis indicated that general Markov processes admit a unified and self-consistent non-equilibrium steady-state thermodynamic formalism, regardless of underlying detailed models.
[Design and implementation of real-time continuous glucose monitoring instrument].
Huang, Yonghong; Liu, Hongying; Tian, Senfu; Jia, Ziru; Wang, Zi; Pi, Xitian
2017-12-01
Real-time continuous glucose monitoring can help diabetics to control blood sugar levels within the normal range. However, in the process of practical monitoring, the output of real-time continuous glucose monitoring system is susceptible to glucose sensor and environment noise, which will influence the measurement accuracy of the system. Aiming at this problem, a dual-calibration algorithm for the moving-window double-layer filtering algorithm combined with real-time self-compensation calibration algorithm is proposed in this paper, which can realize the signal drift compensation for current data. And a real-time continuous glucose monitoring instrument based on this study was designed. This real-time continuous glucose monitoring instrument consisted of an adjustable excitation voltage module, a current-voltage converter module, a microprocessor and a wireless transceiver module. For portability, the size of the device was only 40 mm × 30 mm × 5 mm and its weight was only 30 g. In addition, a communication command code algorithm was designed to ensure the security and integrity of data transmission in this study. Results of experiments in vitro showed that current detection of the device worked effectively. A 5-hour monitoring of blood glucose level in vivo showed that the device could continuously monitor blood glucose in real time. The relative error of monitoring results of the designed device ranged from 2.22% to 7.17% when comparing to a portable blood meter.
Exploratory Study for Continuous-time Parameter Estimation of Ankle Dynamics
Kukreja, Sunil L.; Boyle, Richard D.
2014-01-01
Recently, a parallel pathway model to describe ankle dynamics was proposed. This model provides a relationship between ankle angle and net ankle torque as the sum of a linear and nonlinear contribution. A technique to identify parameters of this model in discrete-time has been developed. However, these parameters are a nonlinear combination of the continuous-time physiology, making insight into the underlying physiology impossible. The stable and accurate estimation of continuous-time parameters is critical for accurate disease modeling, clinical diagnosis, robotic control strategies, development of optimal exercise protocols for longterm space exploration, sports medicine, etc. This paper explores the development of a system identification technique to estimate the continuous-time parameters of ankle dynamics. The effectiveness of this approach is assessed via simulation of a continuous-time model of ankle dynamics with typical parameters found in clinical studies. The results show that although this technique improves estimates, it does not provide robust estimates of continuous-time parameters of ankle dynamics. Due to this we conclude that alternative modeling strategies and more advanced estimation techniques be considered for future work.
Toward Continuous GPS Carrier-Phase Time Transfer: Eliminating the Time Discontinuity at an Anomaly.
Yao, Jian; Levine, Judah; Weiss, Marc
2015-01-01
The wide application of Global Positioning System (GPS) carrier-phase (CP) time transfer is limited by the problem of boundary discontinuity (BD). The discontinuity has two categories. One is "day boundary discontinuity," which has been studied extensively and can be solved by multiple methods [1-8]. The other category of discontinuity, called "anomaly boundary discontinuity (anomaly-BD)," comes from a GPS data anomaly. The anomaly can be a data gap (i.e., missing data), a GPS measurement error (i.e., bad data), or a cycle slip. Initial study of the anomaly-BD shows that we can fix the discontinuity if the anomaly lasts no more than 20 min, using the polynomial curve-fitting strategy to repair the anomaly [9]. However, sometimes, the data anomaly lasts longer than 20 min. Thus, a better curve-fitting strategy is in need. Besides, a cycle slip, as another type of data anomaly, can occur and lead to an anomaly-BD. To solve these problems, this paper proposes a new strategy, i.e., the satellite-clock-aided curve fitting strategy with the function of cycle slip detection. Basically, this new strategy applies the satellite clock correction to the GPS data. After that, we do the polynomial curve fitting for the code and phase data, as before. Our study shows that the phase-data residual is only ~3 mm for all GPS satellites. The new strategy also detects and finds the number of cycle slips by searching the minimum curve-fitting residual. Extensive examples show that this new strategy enables us to repair up to a 40-min GPS data anomaly, regardless of whether the anomaly is due to a data gap, a cycle slip, or a combination of the two. We also find that interference of the GPS signal, known as "jamming", can possibly lead to a time-transfer error, and that this new strategy can compensate for jamming outages. Thus, the new strategy can eliminate the impact of jamming on time transfer. As a whole, we greatly improve the robustness of the GPS CP time transfer.
Measurement of average continuous-time structure of a bond and ...
African Journals Online (AJOL)
The expected continuous-time structure of a bond and bond's interest rate risk in an investment settings was studied. We determined the expected number of years an investor or manager will wait until the stock comes to maturity. The expected principal amount to be paid back per stock at time 't' was determined, while ...
Lyapunov stability robust analysis and robustness design for linear continuous-time systems
Luo, J.S.; Johnson, A.; Bosch, van den P.P.J.
1995-01-01
The linear continuous-time systems to be discussed are described by state space models with structured time-varying uncertainties. First, the explicit maximal perturbation bound for maintaining quadratic Lyapunov stability of the closed-loop systems is presented. Then, a robust design method is
Continuous-time random walk as a guide to fractional Schroedinger equation
International Nuclear Information System (INIS)
Lenzi, E. K.; Ribeiro, H. V.; Mukai, H.; Mendes, R. S.
2010-01-01
We argue that the continuous-time random walk approach may be a useful guide to extend the Schroedinger equation in order to incorporate nonlocal effects, avoiding the inconsistencies raised by Jeng et al. [J. Math. Phys. 51, 062102 (2010)]. As an application, we work out a free particle in a half space, obtaining the time dependent solution by considering an arbitrary initial condition.
Robust model predictive control for constrained continuous-time nonlinear systems
Sun, Tairen; Pan, Yongping; Zhang, Jun; Yu, Haoyong
2018-02-01
In this paper, a robust model predictive control (MPC) is designed for a class of constrained continuous-time nonlinear systems with bounded additive disturbances. The robust MPC consists of a nonlinear feedback control and a continuous-time model-based dual-mode MPC. The nonlinear feedback control guarantees the actual trajectory being contained in a tube centred at the nominal trajectory. The dual-mode MPC is designed to ensure asymptotic convergence of the nominal trajectory to zero. This paper extends current results on discrete-time model-based tube MPC and linear system model-based tube MPC to continuous-time nonlinear model-based tube MPC. The feasibility and robustness of the proposed robust MPC have been demonstrated by theoretical analysis and applications to a cart-damper springer system and a one-link robot manipulator.
Consistency and refinement for Interval Markov Chains
DEFF Research Database (Denmark)
Delahaye, Benoit; Larsen, Kim Guldstrand; Legay, Axel
2012-01-01
Interval Markov Chains (IMC), or Markov Chains with probability intervals in the transition matrix, are the base of a classic specification theory for probabilistic systems [18]. The standard semantics of IMCs assigns to a specification the set of all Markov Chains that satisfy its interval...
Adaptive Partially Hidden Markov Models
DEFF Research Database (Denmark)
Forchhammer, Søren Otto; Rasmussen, Tage
1996-01-01
Partially Hidden Markov Models (PHMM) have recently been introduced. The transition and emission probabilities are conditioned on the past. In this report, the PHMM is extended with a multiple token version. The different versions of the PHMM are applied to bi-level image coding....
Markov Decision Processes in Practice
Boucherie, Richardus J.; van Dijk, N.M.
2017-01-01
It is over 30 years ago since D.J. White started his series of surveys on practical applications of Markov decision processes (MDP), over 20 years after the phenomenal book by Martin Puterman on the theory of MDP, and over 10 years since Eugene A. Feinberg and Adam Shwartz published their Handbook
Directory of Open Access Journals (Sweden)
Tao Wang
2013-01-01
Full Text Available To obtain reliable transient auditory evoked potentials (AEPs from EEGs recorded using high stimulus rate (HSR paradigm, it is critical to design the stimulus sequences of appropriate frequency properties. Traditionally, the individual stimulus events in a stimulus sequence occur only at discrete time points dependent on the sampling frequency of the recording system and the duration of stimulus sequence. This dependency likely causes the implementation of suboptimal stimulus sequences, sacrificing the reliability of resulting AEPs. In this paper, we explicate the use of continuous-time stimulus sequence for HSR paradigm, which is independent of the discrete electroencephalogram (EEG recording system. We employ simulation studies to examine the applicability of the continuous-time stimulus sequences and the impacts of sampling frequency on AEPs in traditional studies using discrete-time design. Results from these studies show that the continuous-time sequences can offer better frequency properties and improve the reliability of recovered AEPs. Furthermore, we find that the errors in the recovered AEPs depend critically on the sampling frequencies of experimental systems, and their relationship can be fitted using a reciprocal function. As such, our study contributes to the literature by demonstrating the applicability and advantages of continuous-time stimulus sequences for HSR paradigm and by revealing the relationship between the reliability of AEPs and sampling frequencies of the experimental systems when discrete-time stimulus sequences are used in traditional manner for the HSR paradigm.
Local and global dynamics of Ramsey model: From continuous to discrete time.
Guzowska, Malgorzata; Michetti, Elisabetta
2018-05-01
The choice of time as a discrete or continuous variable may radically affect equilibrium stability in an endogenous growth model with durable consumption. In the continuous-time Ramsey model [F. P. Ramsey, Econ. J. 38(152), 543-559 (1928)], the steady state is locally saddle-path stable with monotonic convergence. However, in the discrete-time version, the steady state may be unstable or saddle-path stable with monotonic or oscillatory convergence or periodic solutions [see R.-A. Dana et al., Handbook on Optimal Growth 1 (Springer, 2006) and G. Sorger, Working Paper No. 1505 (2015)]. When this occurs, the discrete-time counterpart of the continuous-time model is not consistent with the initial framework. In order to obtain a discrete-time Ramsey model preserving the main properties of the continuous-time counterpart, we use a general backward and forward discretisation as initially proposed by Bosi and Ragot [Theor. Econ. Lett. 2(1), 10-15 (2012)]. The main result of the study here presented is that, with this hybrid discretisation method, fixed points and local dynamics do not change. For what it concerns global dynamics, i.e., long-run behavior for initial conditions taken on the state space, we mainly perform numerical analysis with the main scope of comparing both qualitative and quantitative evolution of the two systems, also varying some parameters of interest.
Optimization of hospital ward resources with patient relocation using Markov chain modeling
DEFF Research Database (Denmark)
Andersen, Anders Reenberg; Nielsen, Bo Friis; Reinhardt, Line Blander
2017-01-01
available to the hospital. Patient flow is modeled using a homogeneous continuous-time Markov chain and optimization is conducted using a local search heuristic. Our model accounts for patient relocation, which has not been done analytically in literature with similar scope. The study objective is to ensure...... are distributed. Furthermore, our heuristic is found to efficiently derive the optimal solution. Applying our model to the hospital case, we found that relocation of daily arrivals can be reduced by 11.7% by re-distributing beds that are already available to the hospital....
Description of quantum-mechanical motion by using the formalism of non-Markov stochastic process
International Nuclear Information System (INIS)
Skorobogatov, G.A.; Svertilov, S.I.
1999-01-01
The principle possibilities of mathematical modeling of quantum mechanical motion by the theory of a real stochastic processes is considered. The set of equations corresponding to the simplest case of a two-level system undergoing transitions under the influence of electromagnetic field are obtained. It is shown that quantum-mechanical processes are purely discrete processes of non-Markovian type. They are continuous processes in the space of probability amplitudes and posses the properties of quantum Markovity. The formulation of quantum mechanics in terms of the theory of stochastic processes is necessary for its generalization on small space-time intervals [ru
Energy Technology Data Exchange (ETDEWEB)
Dufour, F., E-mail: dufour@math.u-bordeaux1.fr [Institut de Mathématiques de Bordeaux, INRIA Bordeaux Sud Ouest, Team: CQFD, and IMB (France); Prieto-Rumeau, T., E-mail: tprieto@ccia.uned.es [UNED, Department of Statistics and Operations Research (Spain)
2016-08-15
We consider a discrete-time constrained discounted Markov decision process (MDP) with Borel state and action spaces, compact action sets, and lower semi-continuous cost functions. We introduce a set of hypotheses related to a positive weight function which allow us to consider cost functions that might not be bounded below by a constant, and which imply the solvability of the linear programming formulation of the constrained MDP. In particular, we establish the existence of a constrained optimal stationary policy. Our results are illustrated with an application to a fishery management problem.
Decoding LDPC Convolutional Codes on Markov Channels
Directory of Open Access Journals (Sweden)
Kashyap Manohar
2008-01-01
Full Text Available Abstract This paper describes a pipelined iterative technique for joint decoding and channel state estimation of LDPC convolutional codes over Markov channels. Example designs are presented for the Gilbert-Elliott discrete channel model. We also compare the performance and complexity of our algorithm against joint decoding and state estimation of conventional LDPC block codes. Complexity analysis reveals that our pipelined algorithm reduces the number of operations per time step compared to LDPC block codes, at the expense of increased memory and latency. This tradeoff is favorable for low-power applications.
Decoding LDPC Convolutional Codes on Markov Channels
Directory of Open Access Journals (Sweden)
Chris Winstead
2008-04-01
Full Text Available This paper describes a pipelined iterative technique for joint decoding and channel state estimation of LDPC convolutional codes over Markov channels. Example designs are presented for the Gilbert-Elliott discrete channel model. We also compare the performance and complexity of our algorithm against joint decoding and state estimation of conventional LDPC block codes. Complexity analysis reveals that our pipelined algorithm reduces the number of operations per time step compared to LDPC block codes, at the expense of increased memory and latency. This tradeoff is favorable for low-power applications.
ANALYSING ACCEPTANCE SAMPLING PLANS BY MARKOV CHAINS
Directory of Open Access Journals (Sweden)
Mohammad Mirabi
2012-01-01
Full Text Available
ENGLISH ABSTRACT: In this research, a Markov analysis of acceptance sampling plans in a single stage and in two stages is proposed, based on the quality of the items inspected. In a stage of this policy, if the number of defective items in a sample of inspected items is more than the upper threshold, the batch is rejected. However, the batch is accepted if the number of defective items is less than the lower threshold. Nonetheless, when the number of defective items falls between the upper and lower thresholds, the decision-making process continues to inspect the items and collect further samples. The primary objective is to determine the optimal values of the upper and lower thresholds using a Markov process to minimise the total cost associated with a batch acceptance policy. A solution method is presented, along with a numerical demonstration of the application of the proposed methodology.
AFRIKAANSE OPSOMMING: In hierdie navorsing word ’n Markov-ontleding gedoen van aannamemonsternemingsplanne wat plaasvind in ’n enkele stap of in twee stappe na gelang van die kwaliteit van die items wat geïnspekteer word. Indien die eerste monster toon dat die aantal defektiewe items ’n boonste grens oorskry, word die lot afgekeur. Indien die eerste monster toon dat die aantal defektiewe items minder is as ’n onderste grens, word die lot aanvaar. Indien die eerste monster toon dat die aantal defektiewe items in die gebied tussen die boonste en onderste grense lê, word die besluitnemingsproses voortgesit en verdere monsters word geneem. Die primêre doel is om die optimale waardes van die booonste en onderste grense te bepaal deur gebruik te maak van ’n Markov-proses sodat die totale koste verbonde aan die proses geminimiseer kan word. ’n Oplossing word daarna voorgehou tesame met ’n numeriese voorbeeld van die toepassing van die voorgestelde oplossing.
The spectral method and ergodic theorems for general Markov chains
International Nuclear Information System (INIS)
Nagaev, S V
2015-01-01
We study the ergodic properties of Markov chains with an arbitrary state space and prove a geometric ergodic theorem. The method of the proof is new: it may be described as an operator method. Our main result is an ergodic theorem for Harris-Markov chains in the case when the return time to some fixed set has finite expectation. Our conditions for the transition function are more general than those used by Athreya-Ney and Nummelin. Unlike them, we impose restrictions not on the original transition function but on the transition function of an embedded Markov chain constructed from the return times to the fixed set mentioned above. The proof uses the spectral theory of linear operators on a Banach space
An Application of Graph Theory in Markov Chains Reliability Analysis
Directory of Open Access Journals (Sweden)
Pavel Skalny
2014-01-01
Full Text Available The paper presents reliability analysis which was realized for an industrial company. The aim of the paper is to present the usage of discrete time Markov chains and the flow in network approach. Discrete Markov chains a well-known method of stochastic modelling describes the issue. The method is suitable for many systems occurring in practice where we can easily distinguish various amount of states. Markov chains are used to describe transitions between the states of the process. The industrial process is described as a graph network. The maximal flow in the network corresponds to the production. The Ford-Fulkerson algorithm is used to quantify the production for each state. The combination of both methods are utilized to quantify the expected value of the amount of manufactured products for the given time period.
Optimization of Modulator and Circuits for Low Power Continuous-Time Delta-Sigma ADC
DEFF Research Database (Denmark)
Marker-Villumsen, Niels; Bruun, Erik
2014-01-01
This paper presents a new optimization method for achieving a minimum current consumption in a continuous-time Delta-Sigma analog-to-digital converter (ADC). The method is applied to a continuous-time modulator realised with active-RC integrators and with a folded-cascode operational transconduc...... levels are swept. Based on the results of the circuit analysis, for each modulator combination the summed current consumption of the 1st integrator and quantizer of the ADC is determined. By also sweeping the partitioning of the noise power for the different circuit parts, the optimum modulator...
Directory of Open Access Journals (Sweden)
SERGIY KOZERENKO
2016-04-01
Full Text Available One feature of the famous Sharkovsky’s theorem is that it can be proved using digraphs of a special type (the so–called Markov graphs. The most general definition assigns a Markov graph to every continuous map from the topological graph to itself. We show that this definition is too broad, i.e. every finite digraph can be viewed as a Markov graph of some one–dimensional dynamical system on a tree. We therefore consider discrete analogues of Markov graphs for vertex maps on combinatorial trees and characterize all maps on trees whose discrete Markov graphs are of the following types: complete, complete bipartite, the disjoint union of cycles, with every arc being a loop.
Portfolio Optimization in a Semi-Markov Modulated Market
International Nuclear Information System (INIS)
Ghosh, Mrinal K.; Goswami, Anindya; Kumar, Suresh K.
2009-01-01
We address a portfolio optimization problem in a semi-Markov modulated market. We study both the terminal expected utility optimization on finite time horizon and the risk-sensitive portfolio optimization on finite and infinite time horizon. We obtain optimal portfolios in relevant cases. A numerical procedure is also developed to compute the optimal expected terminal utility for finite horizon problem
Aristizábal, Natalia; Ramírez, Alex; Hincapié-García, Jaime; Laiton, Estefany; Aristizábal, Carolina; Cuesta, Diana; Monsalve, Claudia; Hincapié, Gloria; Zapata, Eliana; Abad, Verónica; Delgado, Maria-Rocio; Torres, José-Luis; Palacio, Andrés; Botero, José
2015-11-01
To describe baseline characteristics of diabetic patients who were started on insulin pump and real time continuous glucose monitor (CSII-rtCGM) in a specialized center in Medellin, Colombia. All patients with diabetes with complete data who were started on CSII-rtCGM between February 2010 and May 2014 were included. This is a descriptive analysis of the sociodemographic and clinical characteristics. 141 of 174 patients attending the clinic were included. 90,1% had type 1diabetes (T1D). The average age of T1D patients at the beginning of therapy was 31,4 years (SD 14,1). 75.8% of patients had normal weight (BMI30). The median duration of T1D was 13 years (P25-P75=10.7-22.0). 14,2% of the patients were admitted at least once in the year preceding the start of CSII-rtCGM because of diabetes related complications. Mean A1c was 8.6%±1.46%. The main reasons for starting CSII-rtCGM were: poor glycemic control (50.2%); frequent hypoglycemia, nocturnal hypoglycemia, hypoglycemia related to exercise, asymptomatic hypoglycemia (30.2%); severe hypoglycemia (16.44%) and dawn phenomena (3.1%). Baseline characteristics of patients included in this study who were started on CSII-rtCGM are similar to those reported in the literature. The Clinic starts CSII-rtCGM mainly in T1D patients with poor glycemic control, frequent or severe hypoglycemia despite being on basal/bolus therapy. Copyright © 2015 SEEN. Published by Elsevier España, S.L.U. All rights reserved.
Xia, Li C; Ai, Dongmei; Cram, Jacob A; Liang, Xiaoyi; Fuhrman, Jed A; Sun, Fengzhu
2015-09-21
Local trend (i.e. shape) analysis of time series data reveals co-changing patterns in dynamics of biological systems. However, slow permutation procedures to evaluate the statistical significance of local trend scores have limited its applications to high-throughput time series data analysis, e.g., data from the next generation sequencing technology based studies. By extending the theories for the tail probability of the range of sum of Markovian random variables, we propose formulae for approximating the statistical significance of local trend scores. Using simulations and real data, we show that the approximate p-value is close to that obtained using a large number of permutations (starting at time points >20 with no delay and >30 with delay of at most three time steps) in that the non-zero decimals of the p-values obtained by the approximation and the permutations are mostly the same when the approximate p-value is less than 0.05. In addition, the approximate p-value is slightly larger than that based on permutations making hypothesis testing based on the approximate p-value conservative. The approximation enables efficient calculation of p-values for pairwise local trend analysis, making large scale all-versus-all comparisons possible. We also propose a hybrid approach by integrating the approximation and permutations to obtain accurate p-values for significantly associated pairs. We further demonstrate its use with the analysis of the Polymouth Marine Laboratory (PML) microbial community time series from high-throughput sequencing data and found interesting organism co-occurrence dynamic patterns. The software tool is integrated into the eLSA software package that now provides accelerated local trend and similarity analysis pipelines for time series data. The package is freely available from the eLSA website: http://bitbucket.org/charade/elsa.
Descriptive and predictive evaluation of high resolution Markov chain precipitation models
DEFF Research Database (Denmark)
Sørup, Hjalte Jomo Danielsen; Madsen, Henrik; Arnbjerg-Nielsen, Karsten
2012-01-01
A time series of tipping bucket recordings of very high temporal and volumetric resolution precipitation is modelled using Markov chain models. Both first and second‐order Markov models as well as seasonal and diurnal models are investigated and evaluated using likelihood based techniques. The fi...
Prediction of inspection intervals using the Markov analysis
International Nuclear Information System (INIS)
Rea, R.; Arellano, J.
2005-01-01
To solve the unmanageable number of states of Markov of systems that have a great number of components, it is intends a modification to the method of Markov, denominated Markov truncated analysis, in which is assumed that it is worthless the dependence among faults of components. With it the number of states is increased in a lineal way (not exponential) with the number of components of the system, simplifying the analysis vastly. As example, the proposed method was applied to the system HPCS of the CLV considering its 18 main components. It thinks about that each component can take three states: operational, with hidden fault and with revealed fault. Additionally, it takes into account the configuration of the system HPCS by means of a block diagram of dependability to estimate their unavailability at level system. The results of the model here proposed are compared with other methods and approaches used to simplify the Markov analysis. It also intends the modification of the intervals of inspection of three components of the system HPCS. This finishes with base in the developed Markov model and in the maximum time allowed by the code ASME (NUREG-1482) to inspect components of systems that are in reservation in nuclear power plants. (Author)
Benchmarking of a Markov multizone model of contaminant transport.
Jones, Rachael M; Nicas, Mark
2014-10-01
A Markov chain model previously applied to the simulation of advection and diffusion process of gaseous contaminants is extended to three-dimensional transport of particulates in indoor environments. The model framework and assumptions are described. The performance of the Markov model is benchmarked against simple conventional models of contaminant transport. The Markov model is able to replicate elutriation predictions of particle deposition with distance from a point source, and the stirred settling of respirable particles. Comparisons with turbulent eddy diffusion models indicate that the Markov model exhibits numerical diffusion in the first seconds after release, but over time accurately predicts mean lateral dispersion. The Markov model exhibits some instability with grid length aspect when turbulence is incorporated by way of the turbulent diffusion coefficient, and advection is present. However, the magnitude of prediction error may be tolerable for some applications and can be avoided by incorporating turbulence by way of fluctuating velocity (e.g. turbulence intensity). © The Author 2014. Published by Oxford University Press on behalf of the British Occupational Hygiene Society.
Quantum cooling and squeezing of a levitating nanosphere via time-continuous measurements
Genoni, Marco G.; Zhang, Jinglei; Millen, James; Barker, Peter F.; Serafini, Alessio
2015-07-01
With the purpose of controlling the steady state of a dielectric nanosphere levitated within an optical cavity, we study its conditional dynamics under simultaneous sideband cooling and additional time-continuous measurement of either the output cavity mode or the nanosphere’s position. We find that the average phonon number, purity and quantum squeezing of the steady-states can all be made more non-classical through the addition of time-continuous measurement. We predict that the continuous monitoring of the system, together with Markovian feedback, allows one to stabilize the dynamics for any value of the laser frequency driving the cavity. By considering state of the art values of the experimental parameters, we prove that one can in principle obtain a non-classical (squeezed) steady-state with an average phonon number {n}{ph}≈ 0.5.
Continuous-Time Random Walk with multi-step memory: an application to market dynamics
Gubiec, Tomasz; Kutner, Ryszard
2017-11-01
An extended version of the Continuous-Time Random Walk (CTRW) model with memory is herein developed. This memory involves the dependence between arbitrary number of successive jumps of the process while waiting times between jumps are considered as i.i.d. random variables. This dependence was established analyzing empirical histograms for the stochastic process of a single share price on a market within the high frequency time scale. Then, it was justified theoretically by considering bid-ask bounce mechanism containing some delay characteristic for any double-auction market. Our model appeared exactly analytically solvable. Therefore, it enables a direct comparison of its predictions with their empirical counterparts, for instance, with empirical velocity autocorrelation function. Thus, the present research significantly extends capabilities of the CTRW formalism. Contribution to the Topical Issue "Continuous Time Random Walk Still Trendy: Fifty-year History, Current State and Outlook", edited by Ryszard Kutner and Jaume Masoliver.
Directory of Open Access Journals (Sweden)
Lammers Jan-Willem J
2007-07-01
Full Text Available Abstract Background In order to accurately distinguish gaps of varying length in drug treatment for chronic conditions from discontinuation without resuming therapy, short-term observation does not suffice. Thus, the use of inhalation corticosteroids (ICS in the long-term, during a ten-year period is investigated. To describe medication use as a continuum, taking into account the timeliness and consistency of refilling, a Markov model is proposed. Methods Patients, that filled at least one prescription in 1993, were selected from the PHARMO medical record linkage system (RLS containing >95% prescription dispensings per patient originating from community pharmacy records of 6 medium-sized cities in the Netherlands. The probabilities of continuous use, the refilling of at least one ICS prescription in each year of follow-up, and medication free periods were assessed by Markov analysis. Stratified analysis according to new use was performed. Results The transition probabilities of the refilling of at least one ICS prescription in the subsequent year of follow-up, were assessed for each year of follow-up and for the total study period. The change of transition probabilities in time was evaluated, e.g. the probability of continuing ICS use of starters in the first two years (51% of follow-up increased to more than 70% in the following years. The probabilities of different patterns of medication use were assessed: continuous use (7.7%, cumulative medication gaps (1–8 years 69.1% and discontinuing (23.2% during ten-year follow-up for new users. New users had lower probability of continuous use (7.7% and more variability in ICS refill patterns than previous users (56%. Conclusion In addition to well-established methods in epidemiology to ascertain compliance and persistence, a Markov model could be useful to further specify the variety of possible patterns of medication use within the continuum of adherence. This Markov model describes variation in
Wang, Jun; Liang, Jin-Rong; Lv, Long-Jin; Qiu, Wei-Yuan; Ren, Fu-Yao
2012-02-01
In this paper, we study the problem of continuous time option pricing with transaction costs by using the homogeneous subdiffusive fractional Brownian motion (HFBM) Z(t)=X(Sα(t)), 0transaction costs of replicating strategies. We also give the total transaction costs.
Bayesian inference and the analytic continuation of imaginary-time quantum Monte Carlo data
International Nuclear Information System (INIS)
Gubernatis, J.E.; Bonca, J.; Jarrell, M.
1995-01-01
We present brief description of how methods of Bayesian inference are used to obtain real frequency information by the analytic continuation of imaginary-time quantum Monte Carlo data. We present the procedure we used, which is due to R. K. Bryan, and summarize several bottleneck issues
Directory of Open Access Journals (Sweden)
Wilson S
2015-01-01
Full Text Available Scott Wilson,1,2 Andrea Bowyer,3 Stephen B Harrap4 1Department of Renal Medicine, The Alfred Hospital, 2Baker IDI, Melbourne, 3Department of Anaesthesia, Royal Melbourne Hospital, 4University of Melbourne, Parkville, VIC, Australia Abstract: The clinical characterization of cardiovascular dynamics during hemodialysis (HD has important pathophysiological implications in terms of diagnostic, cardiovascular risk assessment, and treatment efficacy perspectives. Currently the diagnosis of significant intradialytic systolic blood pressure (SBP changes among HD patients is imprecise and opportunistic, reliant upon the presence of hypotensive symptoms in conjunction with coincident but isolated noninvasive brachial cuff blood pressure (NIBP readings. Considering hemodynamic variables as a time series makes a continuous recording approach more desirable than intermittent measures; however, in the clinical environment, the data signal is susceptible to corruption due to both impulsive and Gaussian-type noise. Signal preprocessing is an attractive solution to this problem. Prospectively collected continuous noninvasive SBP data over the short-break intradialytic period in ten patients was preprocessed using a novel median hybrid filter (MHF algorithm and compared with 50 time-coincident pairs of intradialytic NIBP measures from routine HD practice. The median hybrid preprocessing technique for continuously acquired cardiovascular data yielded a dynamic regression without significant noise and artifact, suitable for high-level profiling of time-dependent SBP behavior. Signal accuracy is highly comparable with standard NIBP measurement, with the added clinical benefit of dynamic real-time hemodynamic information. Keywords: continuous monitoring, blood pressure
A sixth-order continuous-time bandpass sigma-delta modulator for digital radio IF
Engelen, van J.A.E.P.; Plassche, van de R.J.; Stikvoort, E.F.; Venes, A.G.W.
1999-01-01
This paper presents a sixth-order continuous-time bandpass sigma-delta modulator (SDM) for analog-to-digital conversion of intermediate-frequency signals. An important aspect in the design of this SDM is the stability analysis using the describing function method. The key to the analysis is the
DEFF Research Database (Denmark)
Lauridsen, Mette Munk; Grønbæk, Henning; Næser, Esben
2012-01-01
Abstract Minimal hepatic encephalopathy (MHE) is a metabolic brain disorder occurring in patients with liver cirrhosis. MHE lessens a patient's quality of life, but is treatable when identified. The continuous reaction times (CRT) method is used in screening for MHE. Gender and age effects...
Continuous performance test assessed with time-domain functional near infrared spectroscopy
Torricelli, Alessandro; Contini, Davide; Spinelli, Lorenzo; Caffini, Matteo; Butti, Michele; Baselli, Giuseppe; Bianchi, Anna M.; Bardoni, Alessandra; Cerutti, Sergio; Cubeddu, Rinaldo
2007-07-01
A time-domain fNIRS multichannel system was used in a sustained attention protocol (continuous performance test) to study activation of the prefrontal cortex. Preliminary results on volounteers show significant activation (decrease in deoxy-hemoglobin and increase in oxy-hemoglobin) in both left and right prefrontal cortex.
Exploring Continuity of Care in Patients with Alcohol Use Disorders Using Time-Variant Measures
S.C. de Vries (Sjoerd); A.I. Wierdsma (André)
2008-01-01
textabstractBackground/Aims: We used time-variant measures of continuity of care to study fluctuations in long-term treatment use by patients with alcohol-related disorders. Methods: Data on service use were extracted from the Psychiatric Case Register for the Rotterdam Region, The Netherlands.
Continuous relaxation time spectrum of α-process in glass-like B2O3
International Nuclear Information System (INIS)
Bartenev, G.M.; Lomovskij, V.A.
1991-01-01
α-process of relaxation of glass-like B 2 O 3 was investigated in a wide temperature range. Continuous spectrum of relaxation times H(τ) for this process was constructed, using data of dynamic methods of investigation. It is shown that increase of temperature of α-process investigation leads to change of glass-like BaO 3 structure in such a way, that H(τ) spectrum tends to the maxwell one with a unit relaxation time
Measuring patient-centered medical home access and continuity in clinics with part-time clinicians.
Rosland, Ann-Marie; Krein, Sarah L; Kim, Hyunglin Myra; Greenstone, Clinton L; Tremblay, Adam; Ratz, David; Saffar, Darcy; Kerr, Eve A
2015-05-01
Common patient-centered medical home (PCMH) performance measures value access to a single primary care provider (PCP), which may have unintended consequences for clinics that rely on part-time PCPs and team-based care. Retrospective analysis of 110,454 primary care visits from 2 Veterans Health Administration clinics from 2010 to 2012. Multi-level models examined associations between PCP availability in clinic, and performance on access and continuity measures. Patient experiences with access and continuity were compared using 2012 patient survey data (N = 2881). Patients of PCPs with fewer half-day clinic sessions per week were significantly less likely to get a requested same-day appointment with their usual PCP (predicted probability 17% for PCPs with 2 sessions/week, 20% for 5 sessions/week, and 26% for 10 sessions/week). Among requests that did not result in a same-day appointment with the usual PCP, there were no significant differences in same-day access to a different PCP, or access within 2 to 7 days with patients' usual PCP. Overall, patients had >92% continuity with their usual PCP at the hospital-based site regardless of PCP sessions/week. Patients of full-time PCPs reported timely appointments for urgent needs more often than patients of part-time PCPs (82% vs 71%; P Part-time PCP performance appeared worse when using measures focused on same-day access to patients' usual PCP. However, clinic-level same-day access, same-week access to the usual PCP, and overall continuity were similar for patients of part-time and full-time PCPs. Measures of in-person access to a usual PCP do not capture alternate access approaches encouraged by PCMH, and often used by part-time providers, such as team-based or non-face-to-face care.
Global stabilization of linear continuous time-varying systems with bounded controls
International Nuclear Information System (INIS)
Phat, V.N.
2004-08-01
This paper deals with the problem of global stabilization of a class of linear continuous time-varying systems with bounded controls. Based on the controllability of the nominal system, a sufficient condition for the global stabilizability is proposed without solving any Riccati differential equation. Moreover, we give sufficient conditions for the robust stabilizability of perturbation/uncertain linear time-varying systems with bounded controls. (author)
Energy Technology Data Exchange (ETDEWEB)
Bartosch, T. [Erlanger-Nuernberg Univ., Erlanger (Germany). Lehrstul fuer Nachrichtentechnik I; Seidl, D. [Seismologisches Zentralobservatorium Graefenberg, Erlanegen (Greece). Bundesanstalt fuer Geiwissenschaften und Rohstoffe
1999-06-01
Among a variety of spectrogram methods short-time Fourier transform (STFT) and continuous wavelet transform (CWT) were selected to analyse transients in non-stationary signals. Depending on the properties of the tremor signals from the volcanos Mt. Stromboli, Mt. Semeru and Mt. Pinatubo were analyzed using both methods. The CWT can also be used to extend the definition of coherency into a time-varying coherency spectrogram. An example is given using array data from the volcano Mt. Stromboli (Italy).
Period, epoch, and prediction errors of ephemerides from continuous sets of timing measurements
Deeg, H. J.
2015-06-01
Space missions such as Kepler and CoRoT have led to large numbers of eclipse or transit measurements in nearly continuous time series. This paper shows how to obtain the period error in such measurements from a basic linear least-squares fit, and how to correctly derive the timing error in the prediction of future transit or eclipse events. Assuming strict periodicity, a formula for the period error of these time series is derived, σP = σT (12 / (N3-N))1 / 2, where σP is the period error, σT the timing error of a single measurement, and N the number of measurements. Compared to the iterative method for period error estimation by Mighell & Plavchan (2013), this much simpler formula leads to smaller period errors, whose correctness has been verified through simulations. For the prediction of times of future periodic events, usual linear ephemeris were epoch errors are quoted for the first time measurement, are prone to an overestimation of the error of that prediction. This may be avoided by a correction for the duration of the time series. An alternative is the derivation of ephemerides whose reference epoch and epoch error are given for the centre of the time series. For long continuous or near-continuous time series whose acquisition is completed, such central epochs should be the preferred way for the quotation of linear ephemerides. While this work was motivated from the analysis of eclipse timing measures in space-based light curves, it should be applicable to any other problem with an uninterrupted sequence of discrete timings for which the determination of a zero point, of a constant period and of the associated errors is needed.
Detectability of Granger causality for subsampled continuous-time neurophysiological processes.
Barnett, Lionel; Seth, Anil K
2017-01-01
Granger causality is well established within the neurosciences for inference of directed functional connectivity from neurophysiological data. These data usually consist of time series which subsample a continuous-time biophysiological process. While it is well known that subsampling can lead to imputation of spurious causal connections where none exist, less is known about the effects of subsampling on the ability to reliably detect causal connections which do exist. We present a theoretical analysis of the effects of subsampling on Granger-causal inference. Neurophysiological processes typically feature signal propagation delays on multiple time scales; accordingly, we base our analysis on a distributed-lag, continuous-time stochastic model, and consider Granger causality in continuous time at finite prediction horizons. Via exact analytical solutions, we identify relationships among sampling frequency, underlying causal time scales and detectability of causalities. We reveal complex interactions between the time scale(s) of neural signal propagation and sampling frequency. We demonstrate that detectability decays exponentially as the sample time interval increases beyond causal delay times, identify detectability "black spots" and "sweet spots", and show that downsampling may potentially improve detectability. We also demonstrate that the invariance of Granger causality under causal, invertible filtering fails at finite prediction horizons, with particular implications for inference of Granger causality from fMRI data. Our analysis emphasises that sampling rates for causal analysis of neurophysiological time series should be informed by domain-specific time scales, and that state-space modelling should be preferred to purely autoregressive modelling. On the basis of a very general model that captures the structure of neurophysiological processes, we are able to help identify confounds, and offer practical insights, for successful detection of causal connectivity
Energy Technology Data Exchange (ETDEWEB)
Salimi, S; Radgohar, R, E-mail: shsalimi@uok.ac.i, E-mail: r.radgohar@uok.ac.i [Faculty of Science, Department of Physics, University of Kurdistan, Pasdaran Ave, Sanandaj (Iran, Islamic Republic of)
2010-01-28
In this paper, we consider decoherence in continuous-time quantum walks on long-range interacting cycles (LRICs), which are the extensions of the cycle graphs. For this purpose, we use Gurvitz's model and assume that every node is monitored by the corresponding point-contact induced by the decoherence process. Then, we focus on large rates of decoherence and calculate the probability distribution analytically and obtain the lower and upper bounds of the mixing time. Our results prove that the mixing time is proportional to the rate of decoherence and the inverse of the square of the distance parameter (m). This shows that the mixing time decreases with increasing range of interaction. Also, what we obtain for m = 0 is in agreement with Fedichkin, Solenov and Tamon's results [48] for cycle, and we see that the mixing time of CTQWs on cycle improves with adding interacting edges.
Energy Technology Data Exchange (ETDEWEB)
Rea, R.; Arellano, J. [IIE, Calle Reforma 113, Col. Palmira, Cuernavaca, Morelos (Mexico)]. e-mail: rrea@iie.org.mx
2005-07-01
To solve the unmanageable number of states of Markov of systems that have a great number of components, it is intends a modification to the method of Markov, denominated Markov truncated analysis, in which is assumed that it is worthless the dependence among faults of components. With it the number of states is increased in a lineal way (not exponential) with the number of components of the system, simplifying the analysis vastly. As example, the proposed method was applied to the system HPCS of the CLV considering its 18 main components. It thinks about that each component can take three states: operational, with hidden fault and with revealed fault. Additionally, it takes into account the configuration of the system HPCS by means of a block diagram of dependability to estimate their unavailability at level system. The results of the model here proposed are compared with other methods and approaches used to simplify the Markov analysis. It also intends the modification of the intervals of inspection of three components of the system HPCS. This finishes with base in the developed Markov model and in the maximum time allowed by the code ASME (NUREG-1482) to inspect components of systems that are in reservation in nuclear power plants. (Author)
Martingales and Markov chains solved exercises and elements of theory
Baldi, Paolo; Priouret, Pierre
2002-01-01
CONDITIONAL EXPECTATIONSIntroductionDefinition and First PropertiesConditional Expectations and Conditional LawsExercisesSolutionsSTOCHASTIC PROCESSESGeneral FactsStopping TimesExercisesSolutionsMARTINGALESFirst DefinitionsFirst PropertiesThe Stopping TheoremMaximal InequalitiesSquare Integral MartingalesConvergence TheoremsRegular MartingalesExercisesProblemsSolutionsMARKOV CHAINSTransition Matrices, Markov ChainsConstruction and ExistenceComputations on the Canonical ChainPotential OperatorsPassage ProblemsRecurrence, TransienceRecurrent Irreducible ChainsPeriodicityExercisesProblemsSolution
Dynamic modeling of presence of occupants using inhomogeneous Markov chains
DEFF Research Database (Denmark)
Andersen, Philip Hvidthøft Delff; Iversen, Anne; Madsen, Henrik
2014-01-01
on time of day, and by use of a filter of the observations it is able to capture per-employee sequence dynamics. Simulations using this method are compared with simulations using homogeneous Markov chains and show far better ability to reproduce key properties of the data. The method is based...... on inhomogeneous Markov chains with where the transition probabilities are estimated using generalized linear models with polynomials, B-splines, and a filter of passed observations as inputs. For treating the dispersion of the data series, a hierarchical model structure is used where one model is for low presence...
Learning of temporal motor patterns: An analysis of continuous vs. reset timing
Directory of Open Access Journals (Sweden)
Rodrigo eLaje
2011-10-01
Full Text Available Our ability to generate well-timed sequences of movements is critical to an array of behaviors, including the ability to play a musical instrument or a video game. Here we address two questions relating to timing with the goal of better understanding the neural mechanisms underlying temporal processing. First, how does accuracy and variance change over the course of learning of complex spatiotemporal patterns? Second, is the timing of sequential responses most consistent with starting and stopping an internal timer at each interval or with continuous timing?To address these questions we used a psychophysical task in which subjects learned to reproduce a sequence of finger taps in the correct order and at the correct times—much like playing a melody at the piano. This task allowed us to calculate the variance of the responses at different time points using data from the same trials. Our results show that while standard Weber’s law is clearly violated, variance does increase as a function of time squared, as expected according to the generalized form of Weber’s law—which separates the source of variance into time-dependent and time-independent components. Over the course of learning, both the time-independent variance and the coefficient of the time-dependent term decrease. Our analyses also suggest that timing of sequential events does not rely on the resetting of an internal timer at each event.We describe and interpret our results in the context of computer simulations that capture some of our psychophysical findings. Specifically, we show that continuous timing, as opposed to reset timing, is expected from population clock models in which timing emerges from the internal dynamics of recurrent neural networks.
Markov processes characterization and convergence
Ethier, Stewart N
2009-01-01
The Wiley-Interscience Paperback Series consists of selected books that have been made more accessible to consumers in an effort to increase global appeal and general circulation. With these new unabridged softcover volumes, Wiley hopes to extend the lives of these works by making them available to future generations of statisticians, mathematicians, and scientists."[A]nyone who works with Markov processes whose state space is uncountably infinite will need this most impressive book as a guide and reference."-American Scientist"There is no question but that space should immediately be reserved for [this] book on the library shelf. Those who aspire to mastery of the contents should also reserve a large number of long winter evenings."-Zentralblatt f?r Mathematik und ihre Grenzgebiete/Mathematics Abstracts"Ethier and Kurtz have produced an excellent treatment of the modern theory of Markov processes that [is] useful both as a reference work and as a graduate textbook."-Journal of Statistical PhysicsMarkov Proce...
Smartphone-based Continuous Blood Pressure Measurement Using Pulse Transit Time.
Gholamhosseini, Hamid; Meintjes, Andries; Baig, Mirza; Linden, Maria
2016-01-01
The increasing availability of low cost and easy to use personalized medical monitoring devices has opened the door for new and innovative methods of health monitoring to emerge. Cuff-less and continuous methods of measuring blood pressure are particularly attractive as blood pressure is one of the most important measurements of long term cardiovascular health. Current methods of noninvasive blood pressure measurement are based on inflation and deflation of a cuff with some effects on arteries where blood pressure is being measured. This inflation can also cause patient discomfort and alter the measurement results. In this work, a mobile application was developed to collate the PhotoPlethysmoGramm (PPG) waveform provided by a pulse oximeter and the electrocardiogram (ECG) for calculating the pulse transit time. This information is then indirectly related to the user's systolic blood pressure. The developed application successfully connects to the PPG and ECG monitoring devices using Bluetooth wireless connection and stores the data onto an online server. The pulse transit time is estimated in real time and the user's systolic blood pressure can be estimated after the system has been calibrated. The synchronization between the two devices was found to pose a challenge to this method of continuous blood pressure monitoring. However, the implemented continuous blood pressure monitoring system effectively serves as a proof of concept. This combined with the massive benefits that an accurate and robust continuous blood pressure monitoring system would provide indicates that it is certainly worthwhile to further develop this system.
Park, K. C.; Belvin, W. Keith
1990-01-01
A general form for the first-order representation of the continuous second-order linear structural-dynamics equations is introduced to derive a corresponding form of first-order continuous Kalman filtering equations. Time integration of the resulting equations is carried out via a set of linear multistep integration formulas. It is shown that a judicious combined selection of computational paths and the undetermined matrices introduced in the general form of the first-order linear structural systems leads to a class of second-order discrete Kalman filtering equations involving only symmetric sparse N x N solution matrices.
Optimal batch production strategies under continuous price decrease and time discounting
Directory of Open Access Journals (Sweden)
Mandal S.
2007-01-01
Full Text Available Single price discount in unit cost for bulk purchasing is quite common in reality as well as in inventory literature. However, in today's high-tech industries such as personal computers and mobile industries, continuous decrease in unit cost is a regular phenomenon. In the present paper, an attempt has been made to investigate the effects of continuous price decrease and time-value of money on optimal decisions for inventoried goods having time-dependent demand and production rates. The proposed models are developed over a finite time horizon considering both shortages and without shortages in inventory. Numerical examples are taken to illustrate the developed models and to examine the sensitivity of model parameters.
Directory of Open Access Journals (Sweden)
Songlin Wo
2018-01-01
Full Text Available Singular systems arise in a great deal of domains of engineering and can be used to solve problems which are more difficult and more extensive than regular systems to solve. Therefore, in this paper, the definition of finite-time robust H∞ control for uncertain linear continuous-time singular systems is presented. The problem we address is to design a robust state feedback controller which can deal with the singular system with time-varying norm-bounded exogenous disturbance, such that the singular system is finite-time robust bounded (FTRB with disturbance attenuation γ. Sufficient conditions for the existence of solutions to this problem are obtained in terms of linear matrix equalities (LMIs. When these LMIs are feasible, the desired robust controller is given. A detailed solving method is proposed for the restricted linear matrix inequalities. Finally, examples are given to show the validity of the methodology.
Continuous-Time Mean-Variance Portfolio Selection with Random Horizon
International Nuclear Information System (INIS)
Yu, Zhiyong
2013-01-01
This paper examines the continuous-time mean-variance optimal portfolio selection problem with random market parameters and random time horizon. Treating this problem as a linearly constrained stochastic linear-quadratic optimal control problem, I explicitly derive the efficient portfolios and efficient frontier in closed forms based on the solutions of two backward stochastic differential equations. Some related issues such as a minimum variance portfolio and a mutual fund theorem are also addressed. All the results are markedly different from those in the problem with deterministic exit time. A key part of my analysis involves proving the global solvability of a stochastic Riccati equation, which is interesting in its own right
Continuous-Time Mean-Variance Portfolio Selection with Random Horizon
Energy Technology Data Exchange (ETDEWEB)
Yu, Zhiyong, E-mail: yuzhiyong@sdu.edu.cn [Shandong University, School of Mathematics (China)
2013-12-15
This paper examines the continuous-time mean-variance optimal portfolio selection problem with random market parameters and random time horizon. Treating this problem as a linearly constrained stochastic linear-quadratic optimal control problem, I explicitly derive the efficient portfolios and efficient frontier in closed forms based on the solutions of two backward stochastic differential equations. Some related issues such as a minimum variance portfolio and a mutual fund theorem are also addressed. All the results are markedly different from those in the problem with deterministic exit time. A key part of my analysis involves proving the global solvability of a stochastic Riccati equation, which is interesting in its own right.
Continuous-variable quantum computing in optical time-frequency modes using quantum memories.
Humphreys, Peter C; Kolthammer, W Steven; Nunn, Joshua; Barbieri, Marco; Datta, Animesh; Walmsley, Ian A
2014-09-26
We develop a scheme for time-frequency encoded continuous-variable cluster-state quantum computing using quantum memories. In particular, we propose a method to produce, manipulate, and measure two-dimensional cluster states in a single spatial mode by exploiting the intrinsic time-frequency selectivity of Raman quantum memories. Time-frequency encoding enables the scheme to be extremely compact, requiring a number of memories that are a linear function of only the number of different frequencies in which the computational state is encoded, independent of its temporal duration. We therefore show that quantum memories can be a powerful component for scalable photonic quantum information processing architectures.
Bi-Criteria System Optimum Traffic Assignment in Networks With Continuous Value of Time
Directory of Open Access Journals (Sweden)
Xin Wang
2013-04-01
Full Text Available For an elastic demand transportation network with continuously distributed value of time, the system disutility can be measured either in time units or in cost units. The user equilibrium model and the system optimization model are each formulated in two different criteria. The conditions required for making the system optimum link flow pattern equivalent to the user equilibrium link flow pattern are derived. Furthermore, a bi-objective model has been developed which minimizes simultaneously the system travel time and the system travel cost. The existence of a pricing scheme with anonymous link tolls which can decentralize a Pareto system optimum into the user equilibrium has been investigated.
CONTINUITY OF THE MEANINGS AND FORMS OF PATRIOTISM IN THE CONTEXT OF SOCIAL TIME STUDY
Directory of Open Access Journals (Sweden)
Olga Valerjevna Kashirina
2017-06-01
Full Text Available Purpose. The work objective is to identify the focus of the meanings’ continuity and forms of patriotism in patriotic choice as the frame meaning of main life strategy that each civilized subject has- an individual, a social community of any size. The choice truthfulness is defined by presence of the meaning time continuity and approach of its structure to «the right rate». Methodology. The problem analysis is carried out on the basis of transdisciplinary dialectical and trialectical method of distinction and meaning-making with respect to intellectual technology of civilized and noospheric patriotism continuity. Results. The article regards to the continuity of meanings and forms of patriotism in the context of social time study and searches for the solution to the problem of patriotism in three lines: 1 as the problem of civilized patriotism of Great and Small Motherland, 2 as the problem of noospheric patriotism, 3 as the problem of the continuity of the meanings between them. It highlights the solution flexibility of patriotism problem that is related to the fact that social time study considers patriotism as the culture phenomenon that has the dialectical «nature of existence», and at the same time, it has three way model of civilized reality «existence» meanings – entirety of present, continuity of past and reasonability of future. The article says that the dynamic balance of meanings of civilized and noospheric patriotism in the identity culture of a civilized subject making the culture of his/her behavior and activity provides formation and stability of moral and spiritual immunity that appears by virtue of them in the semantic field of patriotism. Practical implications. The practical implication of the research is in its usability to work out courses on philosophy, culture philosophy, etc. Social time study theory can be realized in teaching practice of the new course unit «The basics of social time study» as a humanity
Astrand, Elaine
2018-06-01
Working memory (WM), crucial for successful behavioral performance in most of our everyday activities, holds a central role in goal-directed behavior. As task demands increase, inducing higher WM load, maintaining successful behavioral performance requires the brain to work at the higher end of its capacity. Because it is depending on both external and internal factors, individual WM load likely varies in a continuous fashion. The feasibility to extract such a continuous measure in time that correlates to behavioral performance during a working memory task remains unsolved. Multivariate pattern decoding was used to test whether a decoder constructed from two discrete levels of WM load can generalize to produce a continuous measure that predicts task performance. Specifically, a linear regression with L2-regularization was chosen with input features from EEG oscillatory activity recorded from healthy participants while performing the n-back task, [Formula: see text]. The feasibility to extract a continuous time-resolved measure that correlates positively to trial-by-trial working memory task performance is demonstrated (r = 0.47, p performance before action (r = 0.49, p < 0.05). We show that the extracted continuous measure enables to study the temporal dynamics of the complex activation pattern of WM encoding during the n-back task. Specifically, temporally precise contributions of different spectral features are observed which extends previous findings of traditional univariate approaches. These results constitute an important contribution towards a wide range of applications in the field of cognitive brain-machine interfaces. Monitoring mental processes related to attention and WM load to reduce the risk of committing errors in high-risk environments could potentially prevent many devastating consequences or using the continuous measure as neurofeedback opens up new possibilities to develop novel rehabilitation techniques for
A new continuous-time formulation for scheduling crude oil operations
International Nuclear Information System (INIS)
Reddy, P. Chandra Prakash; Karimi, I.A.; Srinivasan, R.
2004-01-01
In today's competitive business climate characterized by uncertain oil markets, responding effectively and speedily to market forces, while maintaining reliable operations, is crucial to a refinery's bottom line. Optimal crude oil scheduling enables cost reduction by using cheaper crudes intelligently, minimizing crude changeovers, and avoiding ship demurrage. So far, only discrete-time formulations have stood up to the challenge of this important, nonlinear problem. A continuous-time formulation would portend numerous advantages, however, existing work in this area has just begun to scratch the surface. In this paper, we present the first complete continuous-time mixed integer linear programming (MILP) formulation for the short-term scheduling of operations in a refinery that receives crude from very large crude carriers via a high-volume single buoy mooring pipeline. This novel formulation accounts for real-world operational practices. We use an iterative algorithm to eliminate the crude composition discrepancy that has proven to be the Achilles heel for existing formulations. While it does not guarantee global optimality, the algorithm needs only MILP solutions and obtains excellent maximum-profit schedules for industrial problems with up to 7 days of scheduling horizon. We also report the first comparison of discrete- vs. continuous-time formulations for this complex problem. (Author)
Optimal control of nonlinear continuous-time systems in strict-feedback form.
Zargarzadeh, Hassan; Dierks, Travis; Jagannathan, Sarangapani
2015-10-01
This paper proposes a novel optimal tracking control scheme for nonlinear continuous-time systems in strict-feedback form with uncertain dynamics. The optimal tracking problem is transformed into an equivalent optimal regulation problem through a feedforward adaptive control input that is generated by modifying the standard backstepping technique. Subsequently, a neural network-based optimal control scheme is introduced to estimate the cost, or value function, over an infinite horizon for the resulting nonlinear continuous-time systems in affine form when the internal dynamics are unknown. The estimated cost function is then used to obtain the optimal feedback control input; therefore, the overall optimal control input for the nonlinear continuous-time system in strict-feedback form includes the feedforward plus the optimal feedback terms. It is shown that the estimated cost function minimizes the Hamilton-Jacobi-Bellman estimation error in a forward-in-time manner without using any value or policy iterations. Finally, optimal output feedback control is introduced through the design of a suitable observer. Lyapunov theory is utilized to show the overall stability of the proposed schemes without requiring an initial admissible controller. Simulation examples are provided to validate the theoretical results.
Wilson, Scott; Bowyer, Andrea; Harrap, Stephen B
2015-01-01
The clinical characterization of cardiovascular dynamics during hemodialysis (HD) has important pathophysiological implications in terms of diagnostic, cardiovascular risk assessment, and treatment efficacy perspectives. Currently the diagnosis of significant intradialytic systolic blood pressure (SBP) changes among HD patients is imprecise and opportunistic, reliant upon the presence of hypotensive symptoms in conjunction with coincident but isolated noninvasive brachial cuff blood pressure (NIBP) readings. Considering hemodynamic variables as a time series makes a continuous recording approach more desirable than intermittent measures; however, in the clinical environment, the data signal is susceptible to corruption due to both impulsive and Gaussian-type noise. Signal preprocessing is an attractive solution to this problem. Prospectively collected continuous noninvasive SBP data over the short-break intradialytic period in ten patients was preprocessed using a novel median hybrid filter (MHF) algorithm and compared with 50 time-coincident pairs of intradialytic NIBP measures from routine HD practice. The median hybrid preprocessing technique for continuously acquired cardiovascular data yielded a dynamic regression without significant noise and artifact, suitable for high-level profiling of time-dependent SBP behavior. Signal accuracy is highly comparable with standard NIBP measurement, with the added clinical benefit of dynamic real-time hemodynamic information.
Neyman, Markov processes and survival analysis.
Yang, Grace
2013-07-01
J. Neyman used stochastic processes extensively in his applied work. One example is the Fix and Neyman (F-N) competing risks model (1951) that uses finite homogeneous Markov processes to analyse clinical trials with breast cancer patients. We revisit the F-N model, and compare it with the Kaplan-Meier (K-M) formulation for right censored data. The comparison offers a way to generalize the K-M formulation to include risks of recovery and relapses in the calculation of a patient's survival probability. The generalization is to extend the F-N model to a nonhomogeneous Markov process. Closed-form solutions of the survival probability are available in special cases of the nonhomogeneous processes, like the popular multiple decrement model (including the K-M model) and Chiang's staging model, but these models do not consider recovery and relapses while the F-N model does. An analysis of sero-epidemiology current status data with recurrent events is illustrated. Fix and Neyman used Neyman's RBAN (regular best asymptotic normal) estimates for the risks, and provided a numerical example showing the importance of considering both the survival probability and the length of time of a patient living a normal life in the evaluation of clinical trials. The said extension would result in a complicated model and it is unlikely to find analytical closed-form solutions for survival analysis. With ever increasing computing power, numerical methods offer a viable way of investigating the problem.
Markov transitions and the propagation of chaos
International Nuclear Information System (INIS)
Gottlieb, A.
1998-01-01
The propagation of chaos is a central concept of kinetic theory that serves to relate the equations of Boltzmann and Vlasov to the dynamics of many-particle systems. Propagation of chaos means that molecular chaos, i.e., the stochastic independence of two random particles in a many-particle system, persists in time, as the number of particles tends to infinity. We establish a necessary and sufficient condition for a family of general n-particle Markov processes to propagate chaos. This condition is expressed in terms of the Markov transition functions associated to the n-particle processes, and it amounts to saying that chaos of random initial states propagates if it propagates for pure initial states. Our proof of this result relies on the weak convergence approach to the study of chaos due to Sztitman and Tanaka. We assume that the space in which the particles live is homomorphic to a complete and separable metric space so that we may invoke Prohorov's theorem in our proof. We also show that, if the particles can be in only finitely many states, then molecular chaos implies that the specific entropies in the n-particle distributions converge to the entropy of the limiting single-particle distribution
Asymptotic evolution of quantum Markov chains
Energy Technology Data Exchange (ETDEWEB)
Novotny, Jaroslav [FNSPE, CTU in Prague, 115 19 Praha 1 - Stare Mesto (Czech Republic); Alber, Gernot [Institut fuer Angewandte Physik, Technische Universitaet Darmstadt, D-64289 Darmstadt (Germany)
2012-07-01
The iterated quantum operations, so called quantum Markov chains, play an important role in various branches of physics. They constitute basis for many discrete models capable to explore fundamental physical problems, such as the approach to thermal equilibrium, or the asymptotic dynamics of macroscopic physical systems far from thermal equilibrium. On the other hand, in the more applied area of quantum technology they also describe general characteristic properties of quantum networks or they can describe different quantum protocols in the presence of decoherence. A particularly, an interesting aspect of these quantum Markov chains is their asymptotic dynamics and its characteristic features. We demonstrate there is always a vector subspace (typically low-dimensional) of so-called attractors on which the resulting superoperator governing the iterative time evolution of quantum states can be diagonalized and in which the asymptotic quantum dynamics takes place. As the main result interesting algebraic relations are presented for this set of attractors which allow to specify their dual basis and to determine them in a convenient way. Based on this general theory we show some generalizations concerning the theory of fixed points or asymptotic evolution of random quantum operations.
Embedding a State Space Model Into a Markov Decision Process
DEFF Research Database (Denmark)
Nielsen, Lars Relund; Jørgensen, Erik; Højsgaard, Søren
2011-01-01
In agriculture Markov decision processes (MDPs) with finite state and action space are often used to model sequential decision making over time. For instance, states in the process represent possible levels of traits of the animal and transition probabilities are based on biological models...
Critical Age-Dependent Branching Markov Processes and their ...
Indian Academy of Sciences (India)
This paper studies: (i) the long-time behaviour of the empirical distribution of age and normalized position of an age-dependent critical branching Markov process conditioned on non-extinction; and (ii) the super-process limit of a sequence of age-dependent critical branching Brownian motions.
Markov Stochastic Technique to Determine Galactic Cosmic Ray ...
Indian Academy of Sciences (India)
A new numerical model of particle propagation in the Galaxy has been developed, which allows the study of cosmic-ray production and propagation in 2D. The model has been used to solve cosmic ray diffusive transport equation with a complete network of nuclear interactions using the time backward Markov stochastic ...
A theoretical Markov chain model for evaluating correctional ...
African Journals Online (AJOL)
In this paper a stochastic method is applied in the study of the long time effect of confinement in a correctional institution on the behaviour of a person with criminal tendencies. The approach used is Markov chain, which uses past history to predict the state of a system in the future. A model is developed for comparing the ...
Efficient Modelling and Generation of Markov Automata (extended version)
Timmer, Mark; Katoen, Joost P.; van de Pol, Jan Cornelis; Stoelinga, Mariëlle Ida Antoinette
2012-01-01
This paper introduces a framework for the efficient modelling and generation of Markov automata. It consists of (1) the data-rich process-algebraic language MAPA, allowing concise modelling of systems with nondeterminism, probability and Markovian timing; (2) a restricted form of the language, the
From Brownian Dynamics to Markov Chain: An Ion Channel Example
Chen, Wan
2014-02-27
A discrete rate theory for multi-ion channels is presented, in which the continuous dynamics of ion diffusion is reduced to transitions between Markovian discrete states. In an open channel, the ion permeation process involves three types of events: an ion entering the channel, an ion escaping from the channel, or an ion hopping between different energy minima in the channel. The continuous dynamics leads to a hierarchy of Fokker-Planck equations, indexed by channel occupancy. From these the mean escape times and splitting probabilities (denoting from which side an ion has escaped) can be calculated. By equating these with the corresponding expressions from the Markov model, one can determine the Markovian transition rates. The theory is illustrated with a two-ion one-well channel. The stationary probability of states is compared with that from both Brownian dynamics simulation and the hierarchical Fokker-Planck equations. The conductivity of the channel is also studied, and the optimal geometry maximizing ion flux is computed. © 2014 Society for Industrial and Applied Mathematics.