WorldWideScience

Sample records for adaptive markov chain

  1. Bayesian adaptive Markov chain Monte Carlo estimation of genetic parameters.

    Mathew, B; Bauer, A M; Koistinen, P; Reetz, T C; Léon, J; Sillanpää, M J

    2012-10-01

    Accurate and fast estimation of genetic parameters that underlie quantitative traits using mixed linear models with additive and dominance effects is of great importance in both natural and breeding populations. Here, we propose a new fast adaptive Markov chain Monte Carlo (MCMC) sampling algorithm for the estimation of genetic parameters in the linear mixed model with several random effects. In the learning phase of our algorithm, we use the hybrid Gibbs sampler to learn the covariance structure of the variance components. In the second phase of the algorithm, we use this covariance structure to formulate an effective proposal distribution for a Metropolis-Hastings algorithm, which uses a likelihood function in which the random effects have been integrated out. Compared with the hybrid Gibbs sampler, the new algorithm had better mixing properties and was approximately twice as fast to run. Our new algorithm was able to detect different modes in the posterior distribution. In addition, the posterior mode estimates from the adaptive MCMC method were close to the REML (residual maximum likelihood) estimates. Moreover, our exponential prior for inverse variance components was vague and enabled the estimated mode of the posterior variance to be practically zero, which was in agreement with the support from the likelihood (in the case of no dominance). The method performance is illustrated using simulated data sets with replicates and field data in barley.

  2. Strong Approximations of Martingale Vectors and Their Applications in Markov-Chain Adaptive Designs

    Li-xin Zhang

    2004-01-01

    The strong approximations of a class of Rd-valued martingales are considered. The conditions usedin this paper are easier to check than those used in [3] and [9]. As an application, the strong approximation ofa class of non-homogenous Markov chains is established, and the asymptotic properties are established for themulti-treatment Markov chain adaptive designs in clinical trials.

  3. Graphs: Associated Markov Chains

    2012-01-01

    In this research paper, weighted / unweighted, directed / undirected graphs are associated with interesting Discrete Time Markov Chains (DTMCs) as well as Continuous Time Markov Chains (CTMCs). The equilibrium / transient behaviour of such Markov chains is studied. Also entropy dynamics (Shannon entropy) of certain structured Markov chains is investigated. Finally certain structured graphs and the associated Markov chains are studied.

  4. Adaptive Continuous time Markov Chain Approximation Model to\\ud General Jump-Diffusions

    Cerrato, Mario; Lo, Chia Chun; Skindilias, Konstantinos

    2011-01-01

    We propose a non-equidistant Q rate matrix formula and an adaptive numerical algorithm for a continuous time Markov chain to approximate jump-diffusions with affine or non-affine functional specifications. Our approach also accommodates state-dependent jump intensity and jump distribution, a flexibility that is very hard to achieve with other numerical methods. The Kologorov-Smirnov test shows that the proposed Markov chain transition density converges to the one given by the likelihood expan...

  5. Markov Chains and Markov Processes

    2016-01-01

    Markov chain, which was named after Andrew Markov is a mathematical system that transfers a state to another state. Many real world systems contain uncertainty. This study helps us to understand the basic idea of a Markov chain and how is been useful in our daily lives. For some times there had been suspense on distinct predictions and future existences. Also in different games there had been different expectations or results involved. That is the reason why we need Markov chains to predict o...

  6. Markov Chain Analysis of Cumulative Step-Size Adaptation on a Linear Constrained Problem.

    Chotard, Alexandre; Auger, Anne; Hansen, Nikolaus

    2015-01-01

    This paper analyzes a (1, λ)-Evolution Strategy, a randomized comparison-based adaptive search algorithm optimizing a linear function with a linear constraint. The algorithm uses resampling to handle the constraint. Two cases are investigated: first, the case where the step-size is constant, and second, the case where the step-size is adapted using cumulative step-size adaptation. We exhibit for each case a Markov chain describing the behavior of the algorithm. Stability of the chain implies, by applying a law of large numbers, either convergence or divergence of the algorithm. Divergence is the desired behavior. In the constant step-size case, we show stability of the Markov chain and prove the divergence of the algorithm. In the cumulative step-size adaptation case, we prove stability of the Markov chain in the simplified case where the cumulation parameter equals 1, and discuss steps to obtain similar results for the full (default) algorithm where the cumulation parameter is smaller than 1. The stability of the Markov chain allows us to deduce geometric divergence or convergence, depending on the dimension, constraint angle, population size, and damping parameter, at a rate that we estimate. Our results complement previous studies where stability was assumed.

  7. Adaptive Markov chain Monte Carlo forward projection for statistical analysis in epidemic modelling of human papillomavirus.

    Korostil, Igor A; Peters, Gareth W; Cornebise, Julien; Regan, David G

    2013-05-20

    A Bayesian statistical model and estimation methodology based on forward projection adaptive Markov chain Monte Carlo is developed in order to perform the calibration of a high-dimensional nonlinear system of ordinary differential equations representing an epidemic model for human papillomavirus types 6 and 11 (HPV-6, HPV-11). The model is compartmental and involves stratification by age, gender and sexual-activity group. Developing this model and a means to calibrate it efficiently is relevant because HPV is a very multi-typed and common sexually transmitted infection with more than 100 types currently known. The two types studied in this paper, types 6 and 11, are causing about 90% of anogenital warts. We extend the development of a sexual mixing matrix on the basis of a formulation first suggested by Garnett and Anderson, frequently used to model sexually transmitted infections. In particular, we consider a stochastic mixing matrix framework that allows us to jointly estimate unknown attributes and parameters of the mixing matrix along with the parameters involved in the calibration of the HPV epidemic model. This matrix describes the sexual interactions between members of the population under study and relies on several quantities that are a priori unknown. The Bayesian model developed allows one to estimate jointly the HPV-6 and HPV-11 epidemic model parameters as well as unknown sexual mixing matrix parameters related to assortativity. Finally, we explore the ability of an extension to the class of adaptive Markov chain Monte Carlo algorithms to incorporate a forward projection strategy for the ordinary differential equation state trajectories. Efficient exploration of the Bayesian posterior distribution developed for the ordinary differential equation parameters provides a challenge for any Markov chain sampling methodology, hence the interest in adaptive Markov chain methods. We conclude with simulation studies on synthetic and recent actual data.

  8. Decisive Markov Chains

    Abdulla, Parosh Aziz; Mayr, Richard

    2007-01-01

    We consider qualitative and quantitative verification problems for infinite-state Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which F can no longer be reached. While all finite Markov chains are trivially decisive (for every set F), this also holds for many classes of infinite Markov chains. Infinite Markov chains which contain a finite attractor are decisive w.r.t. every set F. In particular, this holds for probabilistic lossy channel systems (PLCS). Furthermore, all globally coarse Markov chains are decisive. This class includes probabilistic vector addition systems (PVASS) and probabilistic noisy Turing machines (PNTM). We consider both safety and liveness problems for decisive Markov chains, i.e., the probabilities that a given set of states F is eventually reached or reached infinitely often, respectively. 1. We express the qualitative problems in abstract terms for decisive Markov chains, and show...

  9. Fields From Markov Chains

    Justesen, Jørn

    2005-01-01

    A simple construction of two-dimensional (2-D) fields is presented. Rows and columns are outcomes of the same Markov chain. The entropy can be calculated explicitly.......A simple construction of two-dimensional (2-D) fields is presented. Rows and columns are outcomes of the same Markov chain. The entropy can be calculated explicitly....

  10. Fuzzy Markov chains: uncertain probabilities

    2002-01-01

    We consider finite Markov chains where there are uncertainties in some of the transition probabilities. These uncertainties are modeled by fuzzy numbers. Using a restricted fuzzy matrix multiplication we investigate the properties of regular, and absorbing, fuzzy Markov chains and show that the basic properties of these classical Markov chains generalize to fuzzy Markov chains.

  11. Adaptive relaxation for the steady-state analysis of Markov chains

    Horton, Graham

    1994-01-01

    We consider a variant of the well-known Gauss-Seidel method for the solution of Markov chains in steady state. Whereas the standard algorithm visits each state exactly once per iteration in a predetermined order, the alternative approach uses a dynamic strategy. A set of states to be visited is maintained which can grow and shrink as the computation progresses. In this manner, we hope to concentrate the computational work in those areas of the chain in which maximum improvement in the solution can be achieved. We consider the adaptive approach both as a solver in its own right and as a relaxation method within the multi-level algorithm. Experimental results show significant computational savings in both cases.

  12. Markov processes and controlled Markov chains

    Filar, Jerzy; Chen, Anyue

    2002-01-01

    The general theory of stochastic processes and the more specialized theory of Markov processes evolved enormously in the second half of the last century. In parallel, the theory of controlled Markov chains (or Markov decision processes) was being pioneered by control engineers and operations researchers. Researchers in Markov processes and controlled Markov chains have been, for a long time, aware of the synergies between these two subject areas. However, this may be the first volume dedicated to highlighting these synergies and, almost certainly, it is the first volume that emphasizes the contributions of the vibrant and growing Chinese school of probability. The chapters that appear in this book reflect both the maturity and the vitality of modern day Markov processes and controlled Markov chains. They also will provide an opportunity to trace the connections that have emerged between the work done by members of the Chinese school of probability and the work done by the European, US, Central and South Ameri...

  13. Phasic Triplet Markov Chains.

    El Yazid Boudaren, Mohamed; Monfrini, Emmanuel; Pieczynski, Wojciech; Aïssani, Amar

    2014-11-01

    Hidden Markov chains have been shown to be inadequate for data modeling under some complex conditions. In this work, we address the problem of statistical modeling of phenomena involving two heterogeneous system states. Such phenomena may arise in biology or communications, among other fields. Namely, we consider that a sequence of meaningful words is to be searched within a whole observation that also contains arbitrary one-by-one symbols. Moreover, a word may be interrupted at some site to be carried on later. Applying plain hidden Markov chains to such data, while ignoring their specificity, yields unsatisfactory results. The Phasic triplet Markov chain, proposed in this paper, overcomes this difficulty by means of an auxiliary underlying process in accordance with the triplet Markov chains theory. Related Bayesian restoration techniques and parameters estimation procedures according to the new model are then described. Finally, to assess the performance of the proposed model against the conventional hidden Markov chain model, experiments are conducted on synthetic and real data.

  14. Accelerating Markov chain Monte Carlo simulation by differential evolution with self-adaptive randomized subspace sampling

    Vrugt, J.A.; Braak, ter C.J.F.; Diks, C.G.H.; Robinson, B.A.; Hyman, J.M.; Higdon, D.

    2009-01-01

    Markov chain Monte Carlo (MCMC) methods have found widespread use in many fields of study to estimate the average properties of complex systems, and for posterior inference in a Bayesian framework. Existing theory and experiments prove convergence of well-constructed MCMC schemes to the appropriate

  15. Accelerating Markov chain Monte Carlo simulation by differential evolution with self-adaptive randomized subspace sampling

    Vrugt, J.A.; Braak, C.J.F.; Diks, C.G.H.; Robinson, B.A.; Hyman, J.M.; Higdon, D.

    2009-01-01

    Markov chain Monte Carlo (MCMC) methods have found widespread use in many fields of study to estimate the average properties of complex systems, and for posterior inference in a Bayesian framework. Existing theory and experiments prove convergence of well constructed MCMC schemes to the appropriate

  16. On Markov Chains and Filtrations

    1997-01-01

    In this paper we rederive some well known results for continuous time Markov processes that live on a finite state space.Martingale techniques are used throughout the paper. Special attention is paid to the construction of a continuous timeMarkov process, when we start from a discrete time Markov chain. The Markov property here holds with respect tofiltrations that need not be minimal.

  17. Spectral Analysis of Markov Chains

    2007-01-01

    The paper deals with the problem of a statistical analysis of Markov chains connected with the spectral density. We present the expressions for the function of spectral density. These expressions may be used to estimate the parameter of the Markov chain.

  18. Markov chains theory and applications

    Sericola, Bruno

    2013-01-01

    Markov chains are a fundamental class of stochastic processes. They are widely used to solve problems in a large number of domains such as operational research, computer science, communication networks and manufacturing systems. The success of Markov chains is mainly due to their simplicity of use, the large number of available theoretical results and the quality of algorithms developed for the numerical evaluation of many metrics of interest.The author presents the theory of both discrete-time and continuous-time homogeneous Markov chains. He carefully examines the explosion phenomenon, the

  19. Bibliometric Application of Markov Chains.

    Pao, Miranda Lee; McCreery, Laurie

    1986-01-01

    A rudimentary description of Markov Chains is presented in order to introduce its use to describe and to predict authors' movements among subareas of the discipline of ethnomusicology. Other possible applications are suggested. (Author)

  20. Markov Chains and Chemical Processes

    Miller, P. J.

    1972-01-01

    Views as important the relating of abstract ideas of modern mathematics now being taught in the schools to situations encountered in the sciences. Describes use of matrices and Markov chains to study first-order processes. (Author/DF)

  1. Compressing redundant information in Markov chains

    2006-01-01

    Given a strongly stationary Markov chain and a finite set of stopping rules, we prove the existence of a polynomial algorithm which projects the Markov chain onto a minimal Markov chain without redundant information. Markov complexity is hence defined and tested on some classical problems.

  2. Markov chains and mixing times

    Levin, David A; Wilmer, Elizabeth L

    2009-01-01

    This book is an introduction to the modern approach to the theory of Markov chains. The main goal of this approach is to determine the rate of convergence of a Markov chain to the stationary distribution as a function of the size and geometry of the state space. The authors develop the key tools for estimating convergence times, including coupling, strong stationary times, and spectral methods. Whenever possible, probabilistic methods are emphasized. The book includes many examples and provides brief introductions to some central models of statistical mechanics. Also provided are accounts of r

  3. DREAM(D: an adaptive markov chain monte carlo simulation algorithm to solve discrete, noncontinuous, posterior parameter estimation problems

    J. A. Vrugt

    2011-04-01

    Full Text Available Formal and informal Bayesian approaches are increasingly being used to treat forcing, model structural, parameter and calibration data uncertainty, and summarize hydrologic prediction uncertainty. This requires posterior sampling methods that approximate the (evolving posterior distribution. We recently introduced the DiffeRential Evolution Adaptive Metropolis (DREAM algorithm, an adaptive Markov Chain Monte Carlo (MCMC method that is especially designed to solve complex, high-dimensional and multimodal posterior probability density functions. The method runs multiple chains in parallel, and maintains detailed balance and ergodicity. Here, I present the latest algorithmic developments, and introduce a discrete sampling variant of DREAM that samples the parameter space at fixed points. The development of this new code, DREAM(D, has been inspired by the existing class of integer optimization problems, and emerging class of experimental design problems. Such non-continuous parameter estimation problems are of considerable theoretical and practical interest. The theory developed herein is applicable to DREAM(ZS (Vrugt et al., 2011 and MT-DREAM(ZS (Laloy and Vrugt, 2011 as well. Two case studies involving a sudoku puzzle and rainfall – runoff model calibration problem are used to illustrate DREAM(D.

  4. Quadratic Variation by Markov Chains

    Hansen, Peter Reinhard; Horel, Guillaume

    We introduce a novel estimator of the quadratic variation that is based on the the- ory of Markov chains. The estimator is motivated by some general results concerning filtering contaminated semimartingales. Specifically, we show that filtering can in prin- ciple remove the effects of market...

  5. On a Result for Finite Markov Chains

    Kulathinal, Sangita; Ghosh, Lagnojita

    2006-01-01

    In an undergraduate course on stochastic processes, Markov chains are discussed in great detail. Textbooks on stochastic processes provide interesting properties of finite Markov chains. This note discusses one such property regarding the number of steps in which a state is reachable or accessible from another state in a finite Markov chain with M…

  6. Consistency and Refinement for Interval Markov Chains

    Delahaye, Benoit; Larsen, Kim Guldstrand; Legay, Axel;

    2012-01-01

    Interval Markov Chains (IMC), or Markov Chains with probability intervals in the transition matrix, are the base of a classic specification theory for probabilistic systems [18]. The standard semantics of IMCs assigns to a specification the set of all Markov Chains that satisfy its interval...

  7. STATISTICS OF A CLASS OF MARKOV CHAINS

    DENG Yingchun; CAO Xianbing

    2004-01-01

    In this paper we prove that the distributions of their sojourn time and hitting time at one special state for random walks which are allowed to be finite or infinite and Markov chains on star-graphs with discrete time can uniquely determine the probability distribution of the whole chains. This result also suggests a new statistical method for Markov chains.

  8. Metastability of exponentially perturbed Markov chains

    陈大岳; 冯建峰; 钱敏平

    1996-01-01

    A family of irreducible Markov chains on a finite state space is considered as an exponential perturbation of a reducible Markov chain. This is a generalization of the Freidlin-Wentzell theory, motivated by studies of stochastic Ising models, neural network and simulated annealing. It is shown that the metastability is a universal feature for this wide class of Markov chains. The metastable states are simply those recurrent states of the reducible Markov chain. Higher level attractors, related attractive basins and their pyramidal structure are analysed. The logarithmic asymptotics of the hitting time of various sets are estimated. The hitting time over its mean converges in law to the unit exponential distribution.

  9. A scaling analysis of a cat and mouse Markov chain

    Litvak, Nelly; Robert, Philippe

    2012-01-01

    If ($C_n$) a Markov chain on a discrete state space $S$, a Markov chain ($C_n, M_n$) on the product space $S \\times S$, the cat and mouse Markov chain, is constructed. The first coordinate of this Markov chain behaves like the original Markov chain and the second component changes only when both coo

  10. Stochastic seismic tomography by interacting Markov chains

    Bottero, Alexis; Gesret, Alexandrine; Romary, Thomas; Noble, Mark; Maisons, Christophe

    2016-10-01

    Markov chain Monte Carlo sampling methods are widely used for non-linear Bayesian inversion where no analytical expression for the forward relation between data and model parameters is available. Contrary to the linear(ized) approaches, they naturally allow to evaluate the uncertainties on the model found. Nevertheless their use is problematic in high-dimensional model spaces especially when the computational cost of the forward problem is significant and/or the a posteriori distribution is multimodal. In this case, the chain can stay stuck in one of the modes and hence not provide an exhaustive sampling of the distribution of interest. We present here a still relatively unknown algorithm that allows interaction between several Markov chains at different temperatures. These interactions (based on importance resampling) ensure a robust sampling of any posterior distribution and thus provide a way to efficiently tackle complex fully non-linear inverse problems. The algorithm is easy to implement and is well adapted to run on parallel supercomputers. In this paper, the algorithm is first introduced and applied to a synthetic multimodal distribution in order to demonstrate its robustness and efficiency compared to a simulated annealing method. It is then applied in the framework of first arrival traveltime seismic tomography on real data recorded in the context of hydraulic fracturing. To carry out this study a wavelet-based adaptive model parametrization has been used. This allows to integrate the a priori information provided by sonic logs and to reduce optimally the dimension of the problem.

  11. Stochastic seismic tomography by interacting Markov chains

    Bottero, Alexis; Gesret, Alexandrine; Romary, Thomas; Noble, Mark; Maisons, Christophe

    2016-07-01

    Markov chain Monte Carlo sampling methods are widely used for non-linear Bayesian inversion where no analytical expression for the forward relation between data and model parameters is available. Contrary to the linear(ized) approaches they naturally allow to evaluate the uncertainties on the model found. Nevertheless their use is problematic in high dimensional model spaces especially when the computational cost of the forward problem is significant and/or the a posteriori distribution is multimodal. In this case the chain can stay stuck in one of the modes and hence not provide an exhaustive sampling of the distribution of interest. We present here a still relatively unknown algorithm that allows interaction between several Markov chains at different temperatures. These interactions (based on Importance Resampling) ensure a robust sampling of any posterior distribution and thus provide a way to efficiently tackle complex fully non linear inverse problems. The algorithm is easy to implement and is well adapted to run on parallel supercomputers. In this paper the algorithm is first introduced and applied to a synthetic multimodal distribution in order to demonstrate its robustness and efficiency compared to a Simulated Annealing method. It is then applied in the framework of first arrival traveltime seismic tomography on real data recorded in the context of hydraulic fracturing. To carry out this study a wavelet based adaptive model parametrization has been used. This allows to integrate the a priori information provided by sonic logs and to reduce optimally the dimension of the problem.

  12. Uncertainty quantification for Markov chain models.

    Meidani, Hadi; Ghanem, Roger

    2012-12-01

    Transition probabilities serve to parameterize Markov chains and control their evolution and associated decisions and controls. Uncertainties in these parameters can be associated with inherent fluctuations in the medium through which a chain evolves, or with insufficient data such that the inferential value of the chain is jeopardized. The behavior of Markov chains associated with such uncertainties is described using a probabilistic model for the transition matrices. The principle of maximum entropy is used to characterize the probability measure of the transition rates. The formalism is demonstrated on a Markov chain describing the spread of disease, and a number of quantities of interest, pertaining to different aspects of decision-making, are investigated.

  13. Using Games to Teach Markov Chains

    Johnson, Roger W.

    2003-01-01

    Games are promoted as examples for classroom discussion of stationary Markov chains. In a game context Markov chain terminology and results are made concrete, interesting, and entertaining. Game length for several-player games such as "Hi Ho! Cherry-O" and "Chutes and Ladders" is investigated and new, simple formulas are given. Slight…

  14. Revisiting Causality in Markov Chains

    Shojaee, Abbas

    2016-01-01

    Identifying causal relationships is a key premise of scientific research. The growth of observational data in different disciplines along with the availability of machine learning methods offers the possibility of using an empirical approach to identifying potential causal relationships, to deepen our understandings of causal behavior and to build theories accordingly. Conventional methods of causality inference from observational data require a considerable length of time series data to capture cause-effect relationship. We find that potential causal relationships can be inferred from the composition of one step transition rates to and from an event. Also known as Markov chain, one step transition rates are a commonly available resource in different scientific disciplines. Here we introduce a simple, effective and computationally efficient method that we termed 'Causality Inference using Composition of Transitions CICT' to reveal causal structure with high accuracy. We characterize the differences in causes,...

  15. Quantum Markov Chain Mixing and Dissipative Engineering

    Kastoryano, Michael James

    2012-01-01

    This thesis is the fruit of investigations on the extension of ideas of Markov chain mixing to the quantum setting, and its application to problems of dissipative engineering. A Markov chain describes a statistical process where the probability of future events depends only on the state...... of the system at the present point in time, but not on the history of events. Very many important processes in nature are of this type, therefore a good understanding of their behaviour has turned out to be very fruitful for science. Markov chains always have a non-empty set of limiting distributions...... (stationary states). The aim of Markov chain mixing is to obtain (upper and/or lower) bounds on the number of steps it takes for the Markov chain to reach a stationary state. The natural quantum extensions of these notions are density matrices and quantum channels. We set out to develop a general mathematical...

  16. Entropy Rate for Hidden Markov Chains with rare transitions

    2010-01-01

    We consider Hidden Markov Chains obtained by passing a Markov Chain with rare transitions through a noisy memoryless channel. We obtain asymptotic estimates for the entropy of the resulting Hidden Markov Chain as the transition rate is reduced to zero.

  17. Markov chain Monte Carlo simulation for Bayesian Hidden Markov Models

    Chan, Lay Guat; Ibrahim, Adriana Irawati Nur Binti

    2016-10-01

    A hidden Markov model (HMM) is a mixture model which has a Markov chain with finite states as its mixing distribution. HMMs have been applied to a variety of fields, such as speech and face recognitions. The main purpose of this study is to investigate the Bayesian approach to HMMs. Using this approach, we can simulate from the parameters' posterior distribution using some Markov chain Monte Carlo (MCMC) sampling methods. HMMs seem to be useful, but there are some limitations. Therefore, by using the Mixture of Dirichlet processes Hidden Markov Model (MDPHMM) based on Yau et. al (2011), we hope to overcome these limitations. We shall conduct a simulation study using MCMC methods to investigate the performance of this model.

  18. Markov chains models, algorithms and applications

    Ching, Wai-Ki; Ng, Michael K; Siu, Tak-Kuen

    2013-01-01

    This new edition of Markov Chains: Models, Algorithms and Applications has been completely reformatted as a text, complete with end-of-chapter exercises, a new focus on management science, new applications of the models, and new examples with applications in financial risk management and modeling of financial data.This book consists of eight chapters.  Chapter 1 gives a brief introduction to the classical theory on both discrete and continuous time Markov chains. The relationship between Markov chains of finite states and matrix theory will also be highlighted. Some classical iterative methods

  19. Generalized crested products of Markov chains

    D'Angeli, Daniele

    2010-01-01

    We define a finite Markov chain, called generalized crested product, which naturally appears as a generalization of the first crested product of Markov chains. A complete spectral analysis is developed and the $k$-step transition probability is given. It is important to remark that this Markov chain describes a more general version of the classical Ehrenfest diffusion model. As a particular case, one gets a generalization of the classical Insect Markov chain defined on the ultrametric space. Finally, an interpretation in terms of representation group theory is given, by showing the correspondence between the spectral decomposition of the generalized crested product and the Gelfand pairs associated with the generalized wreath product of permutation groups.

  20. Markov chains and decision processes for engineers and managers

    Sheskin, Theodore J

    2010-01-01

    Markov Chain Structure and ModelsHistorical NoteStates and TransitionsModel of the WeatherRandom WalksEstimating Transition ProbabilitiesMultiple-Step Transition ProbabilitiesState Probabilities after Multiple StepsClassification of StatesMarkov Chain StructureMarkov Chain ModelsProblemsReferencesRegular Markov ChainsSteady State ProbabilitiesFirst Passage to a Target StateProblemsReferencesReducible Markov ChainsCanonical Form of the Transition MatrixTh

  1. A scaling analysis of a cat and mouse Markov chain

    Litvak, Nelly; Robert, Philippe

    2009-01-01

    Motivated by an original on-line page-ranking algorithm, starting from an arbitrary Markov chain $(C_n)$ on a discrete state space ${\\cal S}$, a Markov chain $(C_n,M_n)$ on the product space ${\\cal S}^2$, the cat and mouse Markov chain, is constructed. The first coordinate of this Markov

  2. State Truncation for Large Markov Chains

    JIANGLetian; XUGuozhi; ZHANGHao; YINGRendong

    2003-01-01

    One of the main issues to apply the Markov modeling method to reliability and availability analysis is the challenge called largeness, I.e., the explosive number of states, for a system with a large number of components.One method to quickly calculate the reliability of a sys-tem is to neglect ‘unimportant’ states in the Markov chain model. In this paper, based on a Markov model that is widely used in practical systems, a criterion of state trun-cation is presented.

  3. Cosmological Markov Chain Monte Carlo simulation with Cmbeasy

    Müller, C M

    2004-01-01

    We introduce a Markov Chain Monte Carlo simulation and data analysis package for the cosmological computation package Cmbeasy. We have taken special care in implementing an adaptive step algorithm for the Markov Chain Monte Carlo in order to improve convergence. Data analysis routines are provided which allow to test models of the Universe against up-to-date measurements of the Cosmic Microwave Background, Supernovae Ia and Large Scale Structure. The observational data is provided with the software for convenient usage. The package is publicly available as part of the Cmbeasy software at www.cmbeasy.org.

  4. Quantum Markov Chain Mixing and Dissipative Engineering

    Kastoryano, Michael James

    2012-01-01

    This thesis is the fruit of investigations on the extension of ideas of Markov chain mixing to the quantum setting, and its application to problems of dissipative engineering. A Markov chain describes a statistical process where the probability of future events depends only on the state of the sy......This thesis is the fruit of investigations on the extension of ideas of Markov chain mixing to the quantum setting, and its application to problems of dissipative engineering. A Markov chain describes a statistical process where the probability of future events depends only on the state...... framework for studying quantum Markov chain mixing. We introduce two new distance measures into the quantum setting; the quantum $\\chi^2$-divergence and Hilbert's projective metric. With the help of these distance measures, we are able to derive some basic bounds on the the mixing times of quantum channels....... Finally, we consider three independent tasks of dissipative engineering: dissipatively preparing a maximally entangled state of two atoms trapped in an optical cavity, dissipative preparation of graph states, and dissipative quantum computing construction....

  5. Markov chains for testing redundant software

    White, Allan L.; Sjogren, Jon A.

    1988-01-01

    A preliminary design for a validation experiment has been developed that addresses several problems unique to assuring the extremely high quality of multiple-version programs in process-control software. The procedure uses Markov chains to model the error states of the multiple version programs. The programs are observed during simulated process-control testing, and estimates are obtained for the transition probabilities between the states of the Markov chain. The experimental Markov chain model is then expanded into a reliability model that takes into account the inertia of the system being controlled. The reliability of the multiple version software is computed from this reliability model at a given confidence level using confidence intervals obtained for the transition probabilities during the experiment. An example demonstrating the method is provided.

  6. Parallel Markov chain Monte Carlo simulations.

    Ren, Ruichao; Orkoulas, G

    2007-06-07

    With strict detailed balance, parallel Monte Carlo simulation through domain decomposition cannot be validated with conventional Markov chain theory, which describes an intrinsically serial stochastic process. In this work, the parallel version of Markov chain theory and its role in accelerating Monte Carlo simulations via cluster computing is explored. It is shown that sequential updating is the key to improving efficiency in parallel simulations through domain decomposition. A parallel scheme is proposed to reduce interprocessor communication or synchronization, which slows down parallel simulation with increasing number of processors. Parallel simulation results for the two-dimensional lattice gas model show substantial reduction of simulation time for systems of moderate and large size.

  7. Entropy production fluctuations of finite Markov chains

    Jiang, Da-Quan; Qian, Min; Zhang, Fu-Xi

    2003-09-01

    For almost every trajectory segment over a finite time span of a finite Markov chain with any given initial distribution, the logarithm of the ratio of its probability to that of its time-reversal converges exponentially to the entropy production rate of the Markov chain. The large deviation rate function has a symmetry of Gallavotti-Cohen type, which is called the fluctuation theorem. Moreover, similar symmetries also hold for the rate functions of the joint distributions of general observables and the logarithmic probability ratio.

  8. Observation uncertainty in reversible Markov chains.

    Metzner, Philipp; Weber, Marcus; Schütte, Christof

    2010-09-01

    In many applications one is interested in finding a simplified model which captures the essential dynamical behavior of a real life process. If the essential dynamics can be assumed to be (approximately) memoryless then a reasonable choice for a model is a Markov model whose parameters are estimated by means of Bayesian inference from an observed time series. We propose an efficient Monte Carlo Markov chain framework to assess the uncertainty of the Markov model and related observables. The derived Gibbs sampler allows for sampling distributions of transition matrices subject to reversibility and/or sparsity constraints. The performance of the suggested sampling scheme is demonstrated and discussed for a variety of model examples. The uncertainty analysis of functions of the Markov model under investigation is discussed in application to the identification of conformations of the trialanine molecule via Robust Perron Cluster Analysis (PCCA+) .

  9. Markov Chain Approximations to Singular Stable-like Processes

    2012-01-01

    We consider the Markov chain approximations for singular stable-like processes. First we obtain properties of some Markov chains. Then we construct the approximating Markov chains and give a necessary condition for weak convergence of these chains to singular stable-like processes.

  10. Performance Modeling of Communication Networks with Markov Chains

    Mo, Jeonghoon

    2010-01-01

    This book is an introduction to Markov chain modeling with applications to communication networks. It begins with a general introduction to performance modeling in Chapter 1 where we introduce different performance models. We then introduce basic ideas of Markov chain modeling: Markov property, discrete time Markov chain (DTMe and continuous time Markov chain (CTMe. We also discuss how to find the steady state distributions from these Markov chains and how they can be used to compute the system performance metric. The solution methodologies include a balance equation technique, limiting probab

  11. Quantitative timed analysis of interactive Markov chains

    Guck, Dennis; Han, Tingting; Katoen, Joost-Pieter; Neuhausser, M.

    2012-01-01

    This paper presents new algorithms and accompanying tool support for analyzing interactive Markov chains (IMCs), a stochastic timed 1 1/2-player game in which delays are exponentially distributed. IMCs are compositional and act as semantic model for engineering formalisms such as AADL and dynamic fa

  12. A Martingale Decomposition of Discrete Markov Chains

    Hansen, Peter Reinhard

    We consider a multivariate time series whose increments are given from a homogeneous Markov chain. We show that the martingale component of this process can be extracted by a filtering method and establish the corresponding martingale decomposition in closed-form. This representation is useful fo...

  13. Document Ranking Based upon Markov Chains.

    Danilowicz, Czeslaw; Balinski, Jaroslaw

    2001-01-01

    Considers how the order of documents in information retrieval responses are determined and introduces a method that uses a probabilistic model of a document set where documents are regarded as states of a Markov chain and where transition probabilities are directly proportional to similarities between documents. (Author/LRW)

  14. Markov Chain Estimation of Avian Seasonal Fecundity

    To explore the consequences of modeling decisions on inference about avian seasonal fecundity we generalize previous Markov chain (MC) models of avian nest success to formulate two different MC models of avian seasonal fecundity that represent two different ways to model renestin...

  15. Multi-dimensional quasitoeplitz Markov chains

    Alexander N. Dudin

    1999-01-01

    Full Text Available This paper deals with multi-dimensional quasitoeplitz Markov chains. We establish a sufficient equilibrium condition and derive a functional matrix equation for the corresponding vector-generating function, whose solution is given algorithmically. The results are demonstrated in the form of examples and applications in queues with BMAP-input, which operate in synchronous random environment.

  16. Markov chains with quasitoeplitz transition matrix

    Alexander M. Dukhovny

    1989-01-01

    Full Text Available This paper investigates a class of Markov chains which are frequently encountered in various applications (e.g. queueing systems, dams and inventories with feedback. Generating functions of transient and steady state probabilities are found by solving a special Riemann boundary value problem on the unit circle. A criterion of ergodicity is established.

  17. Noise can speed convergence in Markov chains.

    Franzke, Brandon; Kosko, Bart

    2011-10-01

    A new theorem shows that noise can speed convergence to equilibrium in discrete finite-state Markov chains. The noise applies to the state density and helps the Markov chain explore improbable regions of the state space. The theorem ensures that a stochastic-resonance noise benefit exists for states that obey a vector-norm inequality. Such noise leads to faster convergence because the noise reduces the norm components. A corollary shows that a noise benefit still occurs if the system states obey an alternate norm inequality. This leads to a noise-benefit algorithm that requires knowledge of the steady state. An alternative blind algorithm uses only past state information to achieve a weaker noise benefit. Simulations illustrate the predicted noise benefits in three well-known Markov models. The first model is a two-parameter Ehrenfest diffusion model that shows how noise benefits can occur in the class of birth-death processes. The second model is a Wright-Fisher model of genotype drift in population genetics. The third model is a chemical reaction network of zeolite crystallization. A fourth simulation shows a convergence rate increase of 64% for states that satisfy the theorem and an increase of 53% for states that satisfy the corollary. A final simulation shows that even suboptimal noise can speed convergence if the noise applies over successive time cycles. Noise benefits tend to be sharpest in Markov models that do not converge quickly and that do not have strong absorbing states.

  18. Variable context Markov chains for HIV protease cleavage site prediction.

    Oğul, Hasan

    2009-06-01

    Deciphering the knowledge of HIV protease specificity and developing computational tools for detecting its cleavage sites in protein polypeptide chain are very desirable for designing efficient and specific chemical inhibitors to prevent acquired immunodeficiency syndrome. In this study, we developed a generative model based on a generalization of variable order Markov chains (VOMC) for peptide sequences and adapted the model for prediction of their cleavability by certain proteases. The new method, called variable context Markov chains (VCMC), attempts to identify the context equivalence based on the evolutionary similarities between individual amino acids. It was applied for HIV-1 protease cleavage site prediction problem and shown to outperform existing methods in terms of prediction accuracy on a common dataset. In general, the method is a promising tool for prediction of cleavage sites of all proteases and encouraged to be used for any kind of peptide classification problem as well.

  19. Parallel algorithms for simulating continuous time Markov chains

    Nicol, David M.; Heidelberger, Philip

    1992-01-01

    We have previously shown that the mathematical technique of uniformization can serve as the basis of synchronization for the parallel simulation of continuous-time Markov chains. This paper reviews the basic method and compares five different methods based on uniformization, evaluating their strengths and weaknesses as a function of problem characteristics. The methods vary in their use of optimism, logical aggregation, communication management, and adaptivity. Performance evaluation is conducted on the Intel Touchstone Delta multiprocessor, using up to 256 processors.

  20. On nonlinear Markov chain Monte Carlo

    Andrieu, Christophe; Doucet, Arnaud; Del Moral, Pierre; 10.3150/10-BEJ307

    2011-01-01

    Let $\\mathscr{P}(E)$ be the space of probability measures on a measurable space $(E,\\mathcal{E})$. In this paper we introduce a class of nonlinear Markov chain Monte Carlo (MCMC) methods for simulating from a probability measure $\\pi\\in\\mathscr{P}(E)$. Nonlinear Markov kernels (see [Feynman--Kac Formulae: Genealogical and Interacting Particle Systems with Applications (2004) Springer]) $K:\\mathscr{P}(E)\\times E\\rightarrow\\mathscr{P}(E)$ can be constructed to, in some sense, improve over MCMC methods. However, such nonlinear kernels cannot be simulated exactly, so approximations of the nonlinear kernels are constructed using auxiliary or potentially self-interacting chains. Several nonlinear kernels are presented and it is demonstrated that, under some conditions, the associated approximations exhibit a strong law of large numbers; our proof technique is via the Poisson equation and Foster--Lyapunov conditions. We investigate the performance of our approximations with some simulations.

  1. MARKOV CHAIN PORTFOLIO LIQUIDITY OPTIMIZATION MODEL

    Eder Oliveira Abensur

    2014-05-01

    Full Text Available The international financial crisis of September 2008 and May 2010 showed the importance of liquidity as an attribute to be considered in portfolio decisions. This study proposes an optimization model based on available public data, using Markov chain and Genetic Algorithms concepts as it considers the classic duality of risk versus return and incorporating liquidity costs. The work intends to propose a multi-criterion non-linear optimization model using liquidity based on a Markov chain. The non-linear model was tested using Genetic Algorithms with twenty five Brazilian stocks from 2007 to 2009. The results suggest that this is an innovative development methodology and useful for developing an efficient and realistic financial portfolio, as it considers many attributes such as risk, return and liquidity.

  2. Handbook of Markov chain Monte Carlo

    Brooks, Steve

    2011-01-01

    ""Handbook of Markov Chain Monte Carlo"" brings together the major advances that have occurred in recent years while incorporating enough introductory material for new users of MCMC. Along with thorough coverage of the theoretical foundations and algorithmic and computational methodology, this comprehensive handbook includes substantial realistic case studies from a variety of disciplines. These case studies demonstrate the application of MCMC methods and serve as a series of templates for the construction, implementation, and choice of MCMC methodology.

  3. Numerical methods in Markov chain modeling

    Philippe, Bernard; Saad, Youcef; Stewart, William J.

    1989-01-01

    Several methods for computing stationary probability distributions of Markov chains are described and compared. The main linear algebra problem consists of computing an eigenvector of a sparse, usually nonsymmetric, matrix associated with a known eigenvalue. It can also be cast as a problem of solving a homogeneous singular linear system. Several methods based on combinations of Krylov subspace techniques are presented. The performance of these methods on some realistic problems are compared.

  4. Temperature scaling method for Markov chains.

    Crosby, Lonnie D; Windus, Theresa L

    2009-01-22

    The use of ab initio potentials in Monte Carlo simulations aimed at investigating the nucleation kinetics of water clusters is complicated by the computational expense of the potential energy determinations. Furthermore, the common desire to investigate the temperature dependence of kinetic properties leads to an urgent need to reduce the expense of performing simulations at many different temperatures. A method is detailed that allows a Markov chain (obtained via Monte Carlo) at one temperature to be scaled to other temperatures of interest without the need to perform additional large simulations. This Markov chain temperature-scaling (TeS) can be generally applied to simulations geared for numerous applications. This paper shows the quality of results which can be obtained by TeS and the possible quantities which may be extracted from scaled Markov chains. Results are obtained for a 1-D analytical potential for which the exact solutions are known. Also, this method is applied to water clusters consisting of between 2 and 5 monomers, using Dynamical Nucleation Theory to determine the evaporation rate constant for monomer loss. Although ab initio potentials are not utilized in this paper, the benefit of this method is made apparent by using the Dang-Chang polarizable classical potential for water to obtain statistical properties at various temperatures.

  5. Dynamic system evolution and markov chain approximation

    Roderick V. Nicholas Melnik

    1998-01-01

    Full Text Available In this paper computational aspects of the mathematical modelling of dynamic system evolution have been considered as a problem in information theory. The construction of mathematical models is treated as a decision making process with limited available information.The solution of the problem is associated with a computational model based on heuristics of a Markov Chain in a discrete space–time of events. A stable approximation of the chain has been derived and the limiting cases are discussed. An intrinsic interconnection of constructive, sequential, and evolutionary approaches in related optimization problems provides new challenges for future work.

  6. POISSON LIMIT THEOREM FOR COUNTABLE MARKOV CHAINS IN MARKOVIAN ENVIRONMENTS

    方大凡; 王汉兴; 唐矛宁

    2003-01-01

    A countable Markov chain in a Markovian environment is considered. A Poisson limit theorem for the chain recurring to small cylindrical sets is mainly achieved. In order to prove this theorem, the entropy function h is introduced and the Shannon-McMillan-Breiman theorem for the Markov chain in a Markovian environment is shown. It' s well-known that a Markov process in a Markovian environment is generally not a standard Markov chain, so an example of Poisson approximation for a process which is not a Markov process is given. On the other hand, when the environmental process degenerates to a constant sequence, a Poisson limit theorem for countable Markov chains, which is the generalization of Pitskel's result for finite Markov chains is obtained.

  7. Monotone measures of ergodicity for Markov chains

    J. Keilson

    1998-01-01

    Full Text Available The following paper, first written in 1974, was never published other than as part of an internal research series. Its lack of publication is unrelated to the merits of the paper and the paper is of current importance by virtue of its relation to the relaxation time. A systematic discussion is provided of the approach of a finite Markov chain to ergodicity by proving the monotonicity of an important set of norms, each measures of egodicity, whether or not time reversibility is present. The paper is of particular interest because the discussion of the relaxation time of a finite Markov chain [2] has only been clean for time reversible chains, a small subset of the chains of interest. This restriction is not present here. Indeed, a new relaxation time quoted quantifies the relaxation time for all finite ergodic chains (cf. the discussion of Q1(t below Equation (1.7]. This relaxation time was developed by Keilson with A. Roy in his thesis [6], yet to be published.

  8. Growth and dissolution of macromolecular Markov chains

    Gaspard, Pierre

    2016-01-01

    The kinetics and thermodynamics of free living copolymerization are studied for processes with rates depending on k monomeric units of the macromolecular chain behind the unit that is attached or detached. In this case, the sequence of monomeric units in the growing copolymer is a kth-order Markov chain. In the regime of steady growth, the statistical properties of the sequence are determined analytically in terms of the attachment and detachment rates. In this way, the mean growth velocity as well as the thermodynamic entropy production and the sequence disorder can be calculated systematically. These different properties are also investigated in the regime of depolymerization where the macromolecular chain is dissolved by the surrounding solution. In this regime, the entropy production is shown to satisfy Landauer's principle.

  9. Growth and Dissolution of Macromolecular Markov Chains

    Gaspard, Pierre

    2016-07-01

    The kinetics and thermodynamics of free living copolymerization are studied for processes with rates depending on k monomeric units of the macromolecular chain behind the unit that is attached or detached. In this case, the sequence of monomeric units in the growing copolymer is a kth-order Markov chain. In the regime of steady growth, the statistical properties of the sequence are determined analytically in terms of the attachment and detachment rates. In this way, the mean growth velocity as well as the thermodynamic entropy production and the sequence disorder can be calculated systematically. These different properties are also investigated in the regime of depolymerization where the macromolecular chain is dissolved by the surrounding solution. In this regime, the entropy production is shown to satisfy Landauer's principle.

  10. Application of Markov Chains to Stock Trends

    Kevin J. Doubleday

    2011-01-01

    Full Text Available Problem statement: Modeling of the Dow Jones Industrial Average is frequently attempted in order to determine trading strategies with maximum payoff. Changes in the DJIA are important since movements may affect both individuals and corporations profoundly. Previous work showed that modeling a market as a random walk was valid and that a market may be viewed as having the Markov property. Approach: The aim of this research was to determine the relationship between a diverse portfolio of stocks and the market as a whole. To that end, the DJIA was analyzed using a discrete time stochastic model, namely a Markov Chain. Two models were highlighted, where the DJIA was considered as being in a state of (1 gain or loss and (2 small, moderate, or large gain or loss. A portfolio of five stocks was then considered and two models of the portfolio much the same as those for the DJIA. These models were used to obtain transitional probabilities and steady state probabilities. Results: Our results indicated that the portfolio behaved similarly to the entire DJIA, both in the simple model and the partitioned model. Conclusion: When treated as a Markov process, the entire market was useful in gauging how a diverse portfolio of stocks might behave. Future work may include different classifications of states to refine the transition matrices.

  11. Markov chains with quasitoeplitz transition matrix: applications

    1990-01-01

    Application problems are investigated for the Markov chains with quasitoeplitz transition matrix. Generating functions of transient and steady state probabilities, first zero hitting probabilities and mean times are found for various particular cases, corresponding to some known patterns of feedback ( “warm-up,” “switch at threshold” etc.), Level depending dams and queue-depending queueing systems of both M/G/1 and MI/G/1 types with arbitrary random sizes of arriving and departing groups are ...

  12. Dynamic Bandwidth Provisioning Using Markov Chain Based on RSVP

    2013-09-01

    Cambridge University Press,2008. [20] P. Bremaud, Markov Chains : Gibbs Fields, Monte Carlo Simulation and Queues, New York, NY, Springer Science...is successful.  Qualnet, a simulation platform for the wireless environment is used to simulate the algorithm (integration of Markov chain ...in Qualnet, the simulation platform used. 16 THIS PAGE INTENTIONALLY LEFT BLANK 17 III. GENERAL DISCUSSION OF MARKOV CHAIN ALGORITHM AND RSVP

  13. Remarks on a monotone Markov chain

    P. Todorovic

    1987-01-01

    Full Text Available In applications, considerations on stochastic models often involve a Markov chain {ζn}0∞ with state space in R+, and a transition probability Q. For each x  R+ the support of Q(x,. is [0,x]. This implies that ζ0≥ζ1≥…. Under certain regularity assumptions on Q we show that Qn(x,Bu→1 as n→∞ for all u>0 and that 1−Qn(x,Bu≤[1−Q(x,Bu]n where Bu=[0,u. Set τ0=max{k;ζk=ζ0}, τn=max{k;ζk=ζτn−1+1} and write Xn=ζτn−1+1, Tn=τn−τn−1. We investigate some properties of the imbedded Markov chain {Xn}0∞ and of {Tn}0∞. We determine all the marginal distributions of {Tn}0∞ and show that it is asymptotically stationary and that it possesses a monotonicity property. We also prove that under some mild regularity assumptions on β(x=1−Q(x,Bx, ∑1n(Ti−a/bn→dZ∼N(0,1.

  14. Variable length Markov chains and dynamical sources

    Cénac, Peggy; Paccaut, Frédéric; Pouyanne, Nicolas

    2010-01-01

    Infinite random sequences of letters can be viewed as stochastic chains or as strings produced by a source, in the sense of information theory. The relationship between Variable Length Markov Chains (VLMC) and probabilistic dynamical sources is studied. We establish a probabilistic frame for context trees and VLMC and we prove that any VLMC is a dynamical source for which we explicitly build the mapping. On two examples, the ``comb'' and the ``bamboo blossom'', we find a necessary and sufficient condition for the existence and the unicity of a stationary probability measure for the VLMC. These two examples are detailed in order to provide the associated Dirichlet series as well as the generating functions of word occurrences.

  15. Lifting—A nonreversible Markov chain Monte Carlo algorithm

    Vucelja, Marija

    2016-12-01

    Markov chain Monte Carlo algorithms are invaluable tools for exploring stationary properties of physical systems, especially in situations where direct sampling is unfeasible. Common implementations of Monte Carlo algorithms employ reversible Markov chains. Reversible chains obey detailed balance and thus ensure that the system will eventually relax to equilibrium, though detailed balance is not necessary for convergence to equilibrium. We review nonreversible Markov chains, which violate detailed balance and yet still relax to a given target stationary distribution. In particular cases, nonreversible Markov chains are substantially better at sampling than the conventional reversible Markov chains with up to a square root improvement in the convergence time to the steady state. One kind of nonreversible Markov chain is constructed from the reversible ones by enlarging the state space and by modifying and adding extra transition rates to create non-reversible moves. Because of the augmentation of the state space, such chains are often referred to as lifted Markov Chains. We illustrate the use of lifted Markov chains for efficient sampling on several examples. The examples include sampling on a ring, sampling on a torus, the Ising model on a complete graph, and the one-dimensional Ising model. We also provide a pseudocode implementation, review related work, and discuss the applicability of such methods.

  16. Performance evaluation:= (process algebra + model checking) x Markov chains

    Hermanns, H.; Katoen, J.P.; Larsen, Kim G.; Nielsen, Mogens

    2001-01-01

    Markov chains are widely used in practice to determine system performance and reliability characteristics. The vast majority of applications considers continuous-time Markov chains (CTMCs). This tutorial paper shows how successful model specification and analysis techniques from concurrency theory c

  17. Series Expansions for Finite-State Markov Chains

    Heidergott, Bernd; Hordijk, Arie; Uitert, van Miranda

    2005-01-01

    This paper provides series expansions of the stationary distribution of a finite Markov chain. This leads to an efficient numerical algorithm for computing the stationary distribution of a finite Markov chain. Numerical examples are given to illustrate the performance of the algorithm.

  18. First hitting probabilities for semi markov chains and estimation

    Georgiadis, Stylianos

    2017-01-01

    We first consider a stochastic system described by an absorbing semi-Markov chain with finite state space and we introduce the absorption probability to a class of recurrent states. Afterwards, we study the first hitting probability to a subset of states for an irreducible semi-Markov chain...

  19. Multivariate Markov chain modeling for stock markets

    Maskawa, Jun-ichi

    2003-06-01

    We study a multivariate Markov chain model as a stochastic model of the price changes of portfolios in the framework of the mean field approximation. The time series of price changes are coded into the sequences of up and down spins according to their signs. We start with the discussion for small portfolios consisting of two stock issues. The generalization of our model to arbitrary size of portfolio is constructed by a recurrence relation. The resultant form of the joint probability of the stationary state coincides with Gibbs measure assigned to each configuration of spin glass model. Through the analysis of actual portfolios, it has been shown that the synchronization of the direction of the price changes is well described by the model.

  20. Markov chain approach to identifying Wiener systems

    ZHAO WenXiao; CHEN HanFu

    2012-01-01

    Identification of the Wiener system composed of an infinite impulse response (IIR) linear subsystem followed by a static nonlinearity is considered.The recursive estimates for unknown coefficients of the linear subsystem and for the values of the nonlinear function at any fixed points are given by the stochastic approximation algorithms with expanding truncations (SAAWET).With the help of properties of the Markov chain connected with the linear subsystem,all estimates derived in the paper are proved to be strongly consistent.In comparison with the existing results on the topic,the method presented in the paper simplifies the convergence analysis and requires weaker conditions.A numerical example is given,and the simulation results are consistent with the theoretical analysis.

  1. Compressed classification learning with Markov chain samples.

    Cao, Feilong; Dai, Tenghui; Zhang, Yongquan; Tan, Yuanpeng

    2014-02-01

    In this article, we address the problem of compressed classification learning. A generalization bound of the support vector machines (SVMs) compressed classification algorithm with uniformly ergodic Markov chain samples is established. This bound indicates that the accuracy of the SVM classifier in the compressed domain is close to that of the best classifier in the data domain. In a sense, the fact that the compressed learning can avoid the curse of dimensionality in the learning process is shown. In addition, we show that compressed classification learning reduces the learning time at the price of decreasing the classification accuracy, but the decrement can be controlled. The numerical experiments further verify the results claimed in this article.

  2. A Markov Chain Model for Contagion

    Angelos Dassios

    2014-11-01

    Full Text Available We introduce a bivariate Markov chain counting process with contagion for modelling the clustering arrival of loss claims with delayed settlement for an insurance company. It is a general continuous-time model framework that also has the potential to be applicable to modelling the clustering arrival of events, such as jumps, bankruptcies, crises and catastrophes in finance, insurance and economics with both internal contagion risk and external common risk. Key distributional properties, such as the moments and probability generating functions, for this process are derived. Some special cases with explicit results and numerical examples and the motivation for further actuarial applications are also discussed. The model can be considered a generalisation of the dynamic contagion process introduced by Dassios and Zhao (2011.

  3. NONLINEAR EXPECTATIONS AND NONLINEAR MARKOV CHAINS

    PENG SHIGE

    2005-01-01

    This paper deals with nonlinear expectations. The author obtains a nonlinear generalization of the well-known Kolmogorov's consistent theorem and then use it to construct filtration-consistent nonlinear expectations via nonlinear Markov chains. Compared to the author's previous results, i.e., the theory of g-expectations introduced via BSDE on a probability space, the present framework is not based on a given probability measure. Many fully nonlinear and singular situations are covered. The induced topology is a natural generalization of Lp-norms and L∞-norm in linear situations.The author also obtains the existence and uniqueness result of BSDE under this new framework and develops a nonlinear type of von Neumann-Morgenstern representation theorem to utilities and present dynamic risk measures.

  4. Modeling Uncertainty of Directed Movement via Markov Chains

    YIN Zhangcai

    2015-10-01

    Full Text Available Probabilistic time geography (PTG is suggested as an extension of (classical time geography, in order to present the uncertainty of an agent located at the accessible position by probability. This may provide a quantitative basis for most likely finding an agent at a location. In recent years, PTG based on normal distribution or Brown bridge has been proposed, its variance, however, is irrelevant with the agent's speed or divergent with the increase of the speed; so they are difficult to take into account application pertinence and stability. In this paper, a new method is proposed to model PTG based on Markov chain. Firstly, a bidirectional conditions Markov chain is modeled, the limit of which, when the moving speed is large enough, can be regarded as the Brown bridge, thus has the characteristics of digital stability. Then, the directed movement is mapped to Markov chains. The essential part is to build step length, the state space and transfer matrix of Markov chain according to the space and time position of directional movement, movement speed information, to make sure the Markov chain related to the movement speed. Finally, calculating continuously the probability distribution of the directed movement at any time by the Markov chains, it can be get the possibility of an agent located at the accessible position. Experimental results show that, the variance based on Markov chains not only is related to speed, but also is tending towards stability with increasing the agent's maximum speed.

  5. Maximally reliable Markov chains under energy constraints.

    Escola, Sean; Eisele, Michael; Miller, Kenneth; Paninski, Liam

    2009-07-01

    Signal-to-noise ratios in physical systems can be significantly degraded if the outputs of the systems are highly variable. Biological processes for which highly stereotyped signal generations are necessary features appear to have reduced their signal variabilities by employing multiple processing steps. To better understand why this multistep cascade structure might be desirable, we prove that the reliability of a signal generated by a multistate system with no memory (i.e., a Markov chain) is maximal if and only if the system topology is such that the process steps irreversibly through each state, with transition rates chosen such that an equal fraction of the total signal is generated in each state. Furthermore, our result indicates that by increasing the number of states, it is possible to arbitrarily increase the reliability of the system. In a physical system, however, an energy cost is associated with maintaining irreversible transitions, and this cost increases with the number of such transitions (i.e., the number of states). Thus, an infinite-length chain, which would be perfectly reliable, is infeasible. To model the effects of energy demands on the maximally reliable solution, we numerically optimize the topology under two distinct energy functions that penalize either irreversible transitions or incommunicability between states, respectively. In both cases, the solutions are essentially irreversible linear chains, but with upper bounds on the number of states set by the amount of available energy. We therefore conclude that a physical system for which signal reliability is important should employ a linear architecture, with the number of states (and thus the reliability) determined by the intrinsic energy constraints of the system.

  6. Markov Chain-based Degree Distributions of Evolving Networks

    Xiang Xing KONG; Zhen Ting HOU; Ding Hua SHI; Quan Rong CHEN; Qing Gui ZHAO

    2012-01-01

    In this paper,we study a class of stochastic processes,called evolving network Markov chains,in evolving networks. Our approach is to transform the degree distribution problem of an evolving network to a corresponding problem of evolving network Markov chains.We investigate the evolving network Markov chains,thereby obtaining some exact formulas as well as a precise criterion for determining whether the steady degree distribution of the evolving network is a power-law or not.With this new method,we finally obtain a rigorous,exact and unified solution of the steady degree distribution of the evolving network.

  7. Information-Geometric Markov Chain Monte Carlo Methods Using Diffusions

    Samuel Livingstone

    2014-06-01

    Full Text Available Recent work incorporating geometric ideas in Markov chain Monte Carlo is reviewed in order to highlight these advances and their possible application in a range of domains beyond statistics. A full exposition of Markov chains and their use in Monte Carlo simulation for statistical inference and molecular dynamics is provided, with particular emphasis on methods based on Langevin diffusions. After this, geometric concepts in Markov chain Monte Carlo are introduced. A full derivation of the Langevin diffusion on a Riemannian manifold is given, together with a discussion of the appropriate Riemannian metric choice for different problems. A survey of applications is provided, and some open questions are discussed.

  8. Determining a Class of Markov Chains by Hitting Time

    2001-01-01

    @@1 Introduction In many practical problems we often cannot observe the behavior of all states for a Markov chain (see [3-5]). A natural question is that from the observable data of a part of states, can one still obtain all statistical characteristics of the Markov chains. In this paper we give the positive answer for this question and prove the surprising result that the transition rate matrix of the birth-death chains with reflecting barriers and Markov chains on a star graph can be uniquely determined by the probability density functions (pdfs) of the sojourn times and the hitting times at a single special state. This result also suggest a new special type of statistics for Markov chains.

  9. Compositional Modeling and Minimization of Time-Inhomogeneous Markov Chains

    Han, T.; Katoen, J.P.; Mereacre, A.

    2008-01-01

    This paper presents a compositional framework for the modeling of interactive continuous-time Markov chains with time-dependent rates, a subclass of communicating piecewise deterministic Markov processes. A poly-time algorithm is presented for computing the coarsest quotient under strong bisimulatio

  10. Bayesian posterior distributions without Markov chains.

    Cole, Stephen R; Chu, Haitao; Greenland, Sander; Hamra, Ghassan; Richardson, David B

    2012-03-01

    Bayesian posterior parameter distributions are often simulated using Markov chain Monte Carlo (MCMC) methods. However, MCMC methods are not always necessary and do not help the uninitiated understand Bayesian inference. As a bridge to understanding Bayesian inference, the authors illustrate a transparent rejection sampling method. In example 1, they illustrate rejection sampling using 36 cases and 198 controls from a case-control study (1976-1983) assessing the relation between residential exposure to magnetic fields and the development of childhood cancer. Results from rejection sampling (odds ratio (OR) = 1.69, 95% posterior interval (PI): 0.57, 5.00) were similar to MCMC results (OR = 1.69, 95% PI: 0.58, 4.95) and approximations from data-augmentation priors (OR = 1.74, 95% PI: 0.60, 5.06). In example 2, the authors apply rejection sampling to a cohort study of 315 human immunodeficiency virus seroconverters (1984-1998) to assess the relation between viral load after infection and 5-year incidence of acquired immunodeficiency syndrome, adjusting for (continuous) age at seroconversion and race. In this more complex example, rejection sampling required a notably longer run time than MCMC sampling but remained feasible and again yielded similar results. The transparency of the proposed approach comes at a price of being less broadly applicable than MCMC.

  11. The Laplace Functional and Moments for Markov Branching Chains in Random Environments

    HU Di-he; ZHANG Shu-lin

    2005-01-01

    The concepts of random Markov matrix, Markov branching chain in random environment (MBCRE) and Laplace functional of Markov branching chain in random environment (LFMBCRE) are introduced. The properties of LFMBCRE and the explicit formulas of moments of MBCRE are given.

  12. On the Markov Chain Monte Carlo (MCMC) method

    Rajeeva L Karandikar

    2006-04-01

    Markov Chain Monte Carlo (MCMC) is a popular method used to generate samples from arbitrary distributions, which may be specified indirectly. In this article, we give an introduction to this method along with some examples.

  13. The Limit Behaviour of Imprecise Continuous-Time Markov Chains

    De Bock, Jasper

    2016-08-01

    We study the limit behaviour of a nonlinear differential equation whose solution is a superadditive generalisation of a stochastic matrix, prove convergence, and provide necessary and sufficient conditions for ergodicity. In the linear case, the solution of our differential equation is equal to the matrix exponential of an intensity matrix and can then be interpreted as the transition operator of a homogeneous continuous-time Markov chain. Similarly, in the generalised nonlinear case that we consider, the solution can be interpreted as the lower transition operator of a specific set of non-homogeneous continuous-time Markov chains, called an imprecise continuous-time Markov chain. In this context, our convergence result shows that for a fixed initial state, an imprecise continuous-time Markov chain always converges to a limiting distribution, and our ergodicity result provides a necessary and sufficient condition for this limiting distribution to be independent of the initial state.

  14. ON MARKOV CHAINS IN SPACE-TIME RANDOM ENVIRONMENTS

    Hu Dihe; Hu Xiaoyu

    2009-01-01

    In Section 1, the authors establish the models of two kinds of Markov chains in space-time random environments (MCSTRE and MCSTRE(+)) with Abstract state space. In Section 2, the authors construct a MCSTRE and a MCSTRE(+) by an initial distribution Ф and a random Markov kernel (RMK) p(γ). In Section 3, the authors establish several equivalence theorems on MCSTRE and MCSTRE(+). Finally, the authors give two very important examples of MCMSTRE, the random walk in spce-time random environment and the Markov branching chain in space-time random environment.

  15. Reversible jump Markov chain Monte Carlo for deconvolution.

    Kang, Dongwoo; Verotta, Davide

    2007-06-01

    To solve the problem of estimating an unknown input function to a linear time invariant system we propose an adaptive non-parametric method based on reversible jump Markov chain Monte Carlo (RJMCMC). We use piecewise polynomial functions (splines) to represent the input function. The RJMCMC algorithm allows the exploration of a large space of competing models, in our case the collection of splines corresponding to alternative positions of breakpoints, and it is based on the specification of transition probabilities between the models. RJMCMC determines: the number and the position of the breakpoints, and the coefficients determining the shape of the spline, as well as the corresponding posterior distribution of breakpoints, number of breakpoints, coefficients and arbitrary statistics of interest associated with the estimation problem. Simulation studies show that the RJMCMC method can obtain accurate reconstructions of complex input functions, and obtains better results compared with standard non-parametric deconvolution methods. Applications to real data are also reported.

  16. Infinitely dimensional control Markov branching chains in random environments

    2006-01-01

    First of all we introduce the concepts of infinitely dimensional control Markov branching chains in random environments (β-MBCRE) and prove the existence of such chains, then we introduce the concepts of conditional generating functionals and random Markov transition functions of such chains and investigate their branching property. Base on these concepts we calculate the moments of the β-MBCRE and obtain the main results of this paper such as extinction probabilities, polarization and proliferation rate. Finally we discuss the classification ofβ-MBCRE according to the different standards.

  17. Confronting uncertainty in model-based geostatistics using Markov Chain Monte Carlo simulation

    Minasny, B.; Vrugt, J.A.; McBratney, A.B.

    2011-01-01

    This paper demonstrates for the first time the use of Markov Chain Monte Carlo (MCMC) simulation for parameter inference in model-based soil geostatistics. We implemented the recently developed DiffeRential Evolution Adaptive Metropolis (DREAM) algorithm to jointly summarize the posterior distributi

  18. The skew product Markov chain%Markov-双链

    张王月; 张金洪; 邹健

    2001-01-01

    就随机环境下的 Markov-链,介绍了 Markov-双链的构造,并证明了以已给 P(θ )为转移概率的 Markov-双链的存在性 .当环境空间和状态空间均可数时,希望通过对 Markov-双链的研究,进而实现对随机环境下的 Markov-链的研究 .%The process of the construction of the skew product Markov Chain on the basic of the Markov Chain in random environments is introduced and the existence of the skew product Markov Chain is proven, taking the given P(θ ) as the transition probability. The Markov Chain in random environments by studying the skew product Markov chain , can be studied when the environment space and the state space are countable.

  19. Weighted Markov Chain Based Aggregation of Bio-molecule Orderings.

    Sengupta, Debarka; Maulik, Ujjwal; Bandyopadhyay, Sanghamitra

    2012-01-31

    The scope and effectiveness of rank aggregation have already been established in contemporary bioinformatics research. Rank aggregation helps in meta analysis of putative results collected from different analytic or experimental sources. For example, we often receive considerably differing ranked lists of genes or microRNAs from various target prediction algorithms or microarray studies. Sometimes combining them all, in some sense, yields more effective ordering of the set of objects. Also, assigning a certain level of confidence to each source of ranking is a natural demand of aggregation. Assignment of weights to the sources of orderings can be performed by experts. Several rank aggregation approaches like those based on Markov chains (MC), evolutionary algorithms etc., exist in the literature. Markov chains, in general are faster than the evolutionary approaches. Unlike the evolutionary computing approaches Markov chains have not been used for weighted aggregation scenarios. This is because of the absence of a formal framework of weighted Markov chain. In this article we propose the use of a modified version of MC4 (one of the Markov chains proposed by Dwork et al., 2001), followed by the weighted analog of local Kemenization for performing rank aggregation, where the sources of rankings can be prioritized by an expert.

  20. Markov chains and semi-Markov models in time-to-event analysis.

    Abner, Erin L; Charnigo, Richard J; Kryscio, Richard J

    2013-10-25

    A variety of statistical methods are available to investigators for analysis of time-to-event data, often referred to as survival analysis. Kaplan-Meier estimation and Cox proportional hazards regression are commonly employed tools but are not appropriate for all studies, particularly in the presence of competing risks and when multiple or recurrent outcomes are of interest. Markov chain models can accommodate censored data, competing risks (informative censoring), multiple outcomes, recurrent outcomes, frailty, and non-constant survival probabilities. Markov chain models, though often overlooked by investigators in time-to-event analysis, have long been used in clinical studies and have widespread application in other fields.

  1. Intrusion detection based on system calls and homogeneous Markov chains

    Tian Xinguang; Duan Miyi; Sun Chunlai; Li Wenfa

    2008-01-01

    A novel method for detecting anomalous program behavior is presented, which is applicable to hostbased intrusion detection systems that monitor system call activities. The method constructs a homogeneous Markov chain model to characterize the normal behavior of a privileged program, and associates the states of the Markov chain with the unique system calls in the training data. At the detection stage, the probabilities that the Markov chain model supports the system call sequences generated by the program are computed. A low probability indicates an anomalous sequence that may result from intrusive activities. Then a decision rule based on the number of anomalous sequences in a locality frame is adopted to classify the program's behavior. The method gives attention to both computational efficiency and detection accuracy, and is especially suitable for on-line detection. It has been applied to practical host-based intrusion detection systems.

  2. Markov chain solution of photon multiple scattering through turbid slabs.

    Lin, Ying; Northrop, William F; Li, Xuesong

    2016-11-14

    This work introduces a Markov Chain solution to model photon multiple scattering through turbid slabs via anisotropic scattering process, i.e., Mie scattering. Results show that the proposed Markov Chain model agree with commonly used Monte Carlo simulation for various mediums such as medium with non-uniform phase functions and absorbing medium. The proposed Markov Chain solution method successfully converts the complex multiple scattering problem with practical phase functions into a matrix form and solves transmitted/reflected photon angular distributions by matrix multiplications. Such characteristics would potentially allow practical inversions by matrix manipulation or stochastic algorithms where widely applied stochastic methods such as Monte Carlo simulations usually fail, and thus enable practical diagnostics reconstructions such as medical diagnosis, spray analysis, and atmosphere sciences.

  3. Efficient Generation of Random Bits from Finite State Markov Chains

    Zhou, Hongchao

    2010-01-01

    The problem of random number generation from an uncorrelated random source (of unknown probability distribution) dates back to von Neumann's 1951 work. Elias (1972) generalized von Neumann's scheme and showed how to achieve optimal efficiency in unbiased random bits generation. Hence, a natural question is what if the sources are correlated? Both Elias and Samuelson proposed methods for generating unbiased random bits in the case of correlated sources (of unknown probability distribution), specifically, they considered finite Markov chains. However, their proposed methods are not efficient or have implementation difficulties. Blum (1986) devised an algorithm for efficiently generating random bits from degree-2 finite Markov chains in expected linear time, however, his beautiful method is still far from optimality on information-efficiency. In this paper, we generalize Blum's algorithm to arbitrary degree finite Markov chains and combine it with Elias's method for efficient generation of unbiased bits. As a re...

  4. Dynamical Systems Based Non Equilibrium Statistical Mechanics for Markov Chains

    Prevost, Mireille

    We introduce an abstract framework concerning non-equilibrium statistical mechanics in the specific context of Markov chains. This framework encompasses both the Evans-Searles and the Gallavotti-Cohen fluctuation theorems. To support and expand on these concepts, several results are proven, among which a central limit theorem and a large deviation principle. The interest for Markov chains is twofold. First, they model a great variety of physical systems. Secondly, their simplicity allows for an easy introduction to an otherwise complicated field encompassing the statistical mechanics of Anosov and Axiom A diffeomorphisms. We give two examples relating the present framework to physical cases modelled by Markov chains. One of these concerns chemical reactions and links key concepts from the framework to their well known physical counterpart.

  5. Stochastic Dynamics through Hierarchically Embedded Markov Chains

    Vasconcelos, Vítor V.; Santos, Fernando P.; Santos, Francisco C.; Pacheco, Jorge M.

    2017-02-01

    Studying dynamical phenomena in finite populations often involves Markov processes of significant mathematical and/or computational complexity, which rapidly becomes prohibitive with increasing population size or an increasing number of individual configuration states. Here, we develop a framework that allows us to define a hierarchy of approximations to the stationary distribution of general systems that can be described as discrete Markov processes with time invariant transition probabilities and (possibly) a large number of states. This results in an efficient method for studying social and biological communities in the presence of stochastic effects—such as mutations in evolutionary dynamics and a random exploration of choices in social systems—including situations where the dynamics encompasses the existence of stable polymorphic configurations, thus overcoming the limitations of existing methods. The present formalism is shown to be general in scope, widely applicable, and of relevance to a variety of interdisciplinary problems.

  6. Markov Chains For Testing Redundant Software

    White, Allan L.; Sjogren, Jon A.

    1990-01-01

    Preliminary design developed for validation experiment that addresses problems unique to assuring extremely high quality of multiple-version programs in process-control software. Approach takes into account inertia of controlled system in sense it takes more than one failure of control program to cause controlled system to fail. Verification procedure consists of two steps: experimentation (numerical simulation) and computation, with Markov model for each step.

  7. Stochastic model of milk homogenization process using Markov's chain

    A. A. Khvostov; R. S. Sumina; G. I. Kotov; Ivanov, A. V.

    2016-01-01

    The process of development of a mathematical model of the process of homogenization of dairy products is considered in the work. The theory of Markov's chains was used in the development of the mathematical model, Markov's chain with discrete states and continuous parameter for which the homogenisation pressure is taken, being the basis for the model structure. Machine realization of the model is implemented in the medium of structural modeling MathWorks Simulink™. Identification of the model...

  8. Dynamic modeling of presence of occupants using inhomogeneous Markov chains

    Andersen, Philip Hvidthøft Delff; Iversen, Anne; Madsen, Henrik;

    2014-01-01

    on time of day, and by use of a filter of the observations it is able to capture per-employee sequence dynamics. Simulations using this method are compared with simulations using homogeneous Markov chains and show far better ability to reproduce key properties of the data. The method is based...... on inhomogeneous Markov chains with where the transition probabilities are estimated using generalized linear models with polynomials, B-splines, and a filter of passed observations as inputs. For treating the dispersion of the data series, a hierarchical model structure is used where one model is for low presence...

  9. Chernoff-Hoeffding Bounds for Markov Chains: Generalized and Simplified

    Chung, Kai-Min; Liu, Zhenming; Mitzenmacher, Michael

    2012-01-01

    We prove the first Chernoff-Hoeffding bounds for general nonreversible finite-state Markov chains based on the standard L_1 (variation distance) mixing-time of the chain. Specifically, consider an ergodic Markov chain M and a weight function f: [n] -> [0,1] on the state space [n] of M with mean mu = E_{v = delta mu t ], is at most exp(-Omega(delta^2 mu t / T)) for 0 1. In fact, the bounds hold even if the weight functions f_i's for i in [t] are distinct, provided that all of them have the same mean mu. We also obtain a simplified proof for the Chernoff-Hoeffding bounds based on the spectral expansion lambda of M, which is the square root of the second largest eigenvalue (in absolute value) of M tilde{M}, where tilde{M} is the time-reversal Markov chain of M. We show that the probability Pr [ |X - mu t| >= delta mu t ] is at most exp(-Omega(delta^2 (1-lambda) mu t)) for 0 1. Both of our results extend to continuous-time Markov chains, and to the case where the walk starts from an arbitrary distribution x, at...

  10. Influence of credit scoring on the dynamics of Markov chain

    Galina, Timofeeva

    2015-11-01

    Markov processes are widely used to model the dynamics of a credit portfolio and forecast the portfolio risk and profitability. In the Markov chain model the loan portfolio is divided into several groups with different quality, which determined by presence of indebtedness and its terms. It is proposed that dynamics of portfolio shares is described by a multistage controlled system. The article outlines mathematical formalization of controls which reflect the actions of the bank's management in order to improve the loan portfolio quality. The most important control is the organization of approval procedure of loan applications. The credit scoring is studied as a control affecting to the dynamic system. Different formalizations of "good" and "bad" consumers are proposed in connection with the Markov chain model.

  11. Ergodic degrees for continuous-time Markov chains

    MAO; Yonghua

    2004-01-01

    This paper studies the existence of the higher orders deviation matrices for continuous time Markov chains by the moments for the hitting times. An estimate of the polynomial convergence rates for the transition matrix to the stationary measure is obtained. Finally, the explicit formulas for birth-death processes are presented.

  12. BLOCKWISE PERTURBATION THEORY FOR 2 × 2 BLOCK MARKOV CHAINS

    Jun-gong Xue; Wei-guo Gao

    2000-01-01

    Let P be a transition matrix of a Markov chain and be of the form The stationary distribution πT is partitioned conformally in the form (π1T, π2T).This paper establish the relative error bound in πiT (i = 1, 2) when each block Pij get a small relative perturbation.

  13. Markov chain for estimating human mitochondrial DNA mutation pattern

    Vantika, Sandy; Pasaribu, Udjianna S.

    2015-12-01

    The Markov chain was proposed to estimate the human mitochondrial DNA mutation pattern. One DNA sequence was taken randomly from 100 sequences in Genbank. The nucleotide transition matrix and mutation transition matrix were estimated from this sequence. We determined whether the states (mutation/normal) are recurrent or transient. The results showed that both of them are recurrent.

  14. Exact goodness-of-fit tests for Markov chains.

    Besag, J; Mondal, D

    2013-06-01

    Goodness-of-fit tests are useful in assessing whether a statistical model is consistent with available data. However, the usual χ² asymptotics often fail, either because of the paucity of the data or because a nonstandard test statistic is of interest. In this article, we describe exact goodness-of-fit tests for first- and higher order Markov chains, with particular attention given to time-reversible ones. The tests are obtained by conditioning on the sufficient statistics for the transition probabilities and are implemented by simple Monte Carlo sampling or by Markov chain Monte Carlo. They apply both to single and to multiple sequences and allow a free choice of test statistic. Three examples are given. The first concerns multiple sequences of dry and wet January days for the years 1948-1983 at Snoqualmie Falls, Washington State, and suggests that standard analysis may be misleading. The second one is for a four-state DNA sequence and lends support to the original conclusion that a second-order Markov chain provides an adequate fit to the data. The last one is six-state atomistic data arising in molecular conformational dynamics simulation of solvated alanine dipeptide and points to strong evidence against a first-order reversible Markov chain at 6 picosecond time steps.

  15. Operations and support cost modeling using Markov chains

    Unal, Resit

    1989-01-01

    Systems for future missions will be selected with life cycle costs (LCC) as a primary evaluation criterion. This reflects the current realization that only systems which are considered affordable will be built in the future due to the national budget constaints. Such an environment calls for innovative cost modeling techniques which address all of the phases a space system goes through during its life cycle, namely: design and development, fabrication, operations and support; and retirement. A significant portion of the LCC for reusable systems are generated during the operations and support phase (OS). Typically, OS costs can account for 60 to 80 percent of the total LCC. Clearly, OS costs are wholly determined or at least strongly influenced by decisions made during the design and development phases of the project. As a result OS costs need to be considered and estimated early in the conceptual phase. To be effective, an OS cost estimating model needs to account for actual instead of ideal processes by associating cost elements with probabilities. One approach that may be suitable for OS cost modeling is the use of the Markov Chain Process. Markov chains are an important method of probabilistic analysis for operations research analysts but they are rarely used for life cycle cost analysis. This research effort evaluates the use of Markov Chains in LCC analysis by developing OS cost model for a hypothetical reusable space transportation vehicle (HSTV) and suggests further uses of the Markov Chain process as a design-aid tool.

  16. Students' Progress throughout Examination Process as a Markov Chain

    Hlavatý, Robert; Dömeová, Ludmila

    2014-01-01

    The paper is focused on students of Mathematical methods in economics at the Czech university of life sciences (CULS) in Prague. The idea is to create a model of students' progress throughout the whole course using the Markov chain approach. Each student has to go through various stages of the course requirements where his success depends on the…

  17. Using Markov Chain Analyses in Counselor Education Research

    Duys, David K.; Headrick, Todd C.

    2004-01-01

    This study examined the efficacy of an infrequently used statistical analysis in counselor education research. A Markov chain analysis was used to examine hypothesized differences between students' use of counseling skills in an introductory course. Thirty graduate students participated in the study. Independent raters identified the microskills…

  18. Exploring Mass Perception with Markov Chain Monte Carlo

    Cohen, Andrew L.; Ross, Michael G.

    2009-01-01

    Several previous studies have examined the ability to judge the relative mass of objects in idealized collisions. With a newly developed technique of psychological Markov chain Monte Carlo sampling (A. N. Sanborn & T. L. Griffiths, 2008), this work explores participants; perceptions of different collision mass ratios. The results reveal…

  19. Building Higher-Order Markov Chain Models with EXCEL

    Ching, Wai-Ki; Fung, Eric S.; Ng, Michael K.

    2004-01-01

    Categorical data sequences occur in many applications such as forecasting, data mining and bioinformatics. In this note, we present higher-order Markov chain models for modelling categorical data sequences with an efficient algorithm for solving the model parameters. The algorithm can be implemented easily in a Microsoft EXCEL worksheet. We give a…

  20. Model checking conditional CSL for continuous-time Markov chains

    Gao, Yang; Xu, Ming; Zhan, Naijun;

    2013-01-01

    In this paper, we consider the model-checking problem of continuous-time Markov chains (CTMCs) with respect to conditional logic. To the end, we extend Continuous Stochastic Logic introduced in Aziz et al. (2000) [1] to Conditional Continuous Stochastic Logic (CCSL) by introducing a conditional...

  1. Complete Axiomatization for the Bisimilarity Distance on Markov Chains

    Bacci, Giorgio; Bacci, Giovanni; Larsen, Kim Guldstrand;

    2016-01-01

    In this paper we propose a complete axiomatization of the bisimilarity distance of Desharnais et al. for the class of finite labelled Markov chains. Our axiomatization is given in the style of a quantitative extension of equational logic recently proposed by Mardare, Panangaden, and Plotkin (LICS...

  2. On the Total Variation Distance of Semi-Markov Chains

    Bacci, Giorgio; Bacci, Giovanni; Larsen, Kim Guldstrand;

    2015-01-01

    Semi-Markov chains (SMCs) are continuous-time probabilistic transition systems where the residence time on states is governed by generic distributions on the positive real line. This paper shows the tight relation between the total variation distance on SMCs and their model checking problem over...

  3. Converging from Branching to Linear Metrics on Markov Chains

    Bacci, Giorgio; Bacci, Giovanni; Larsen, Kim Guldstrand;

    2015-01-01

    We study the strong and strutter trace distances on Markov chains (MCs). Our interest in these metrics is motivated by their relation to the probabilistic LTL-model checking problem: we prove that they correspond to the maximal differences in the probability of satisfying the same LTL and LTL...

  4. Markov chains with quasitoeplitz transition matrix: first zero hitting

    Alexander M. Dukhovny

    1989-01-01

    Full Text Available This paper continues the investigation of Markov Chains with a quasitoeplitz transition matrix. Generating functions of first zero hitting probabilities and mean times are found by the solution of special Riemann boundary value problems on the unit circle. Duality is discussed.

  5. Recombination Processes and Nonlinear Markov Chains.

    Pirogov, Sergey; Rybko, Alexander; Kalinina, Anastasia; Gelfand, Mikhail

    2016-09-01

    Bacteria are known to exchange genetic information by horizontal gene transfer. Since the frequency of homologous recombination depends on the similarity between the recombining segments, several studies examined whether this could lead to the emergence of subspecies. Most of them simulated fixed-size Wright-Fisher populations, in which the genetic drift should be taken into account. Here, we use nonlinear Markov processes to describe a bacterial population evolving under mutation and recombination. We consider a population structure as a probability measure on the space of genomes. This approach implies the infinite population size limit, and thus, the genetic drift is not assumed. We prove that under these conditions, the emergence of subspecies is impossible.

  6. Algebraic convergence for discrete-time ergodic Markov chains

    MAO; Yonghua(毛永华)

    2003-01-01

    This paper studies the e-ergodicity for discrete-time recurrent Markov chains. It proves that thee-order deviation matrix exists and is finite if and only if the chain is (e + 2)-ergodic, and then the algebraicdecay rates of the n-step transition probability to the stationary distribution are obtained. The criteria fore-ergodicity are given in terms of existence of solution to an equation. The main results are illustrated by some examples.

  7. A New Multivariate Markov Chain Model for Adding a New Categorical Data Sequence

    2014-01-01

    We propose a new multivariate Markov chain model for adding a new categorical data sequence. The number of the parameters in the new multivariate Markov chain model is only (3s) less than ((s+1)2) the number of the parameters in the former multivariate Markov chain model. Numerical experiments demonstrate the benefits of the new multivariate Markov chain model on saving computational resources.

  8. Markov Chain for Reuse Strategies of Product Families

    LUO Jia; JIANG Lan

    2007-01-01

    A methodology is presented to plan reuse strategies of common modules in a product family by using the concepts of function degradation, reliability, function requirement, cost and life time. Markov chain model is employed to predict function degradation and reliability. A utility model is used to evaluate the preference between used modules and new modules. An example of cascading-requirment product family illustrates the main ideas of our work. The Markov models are used effectively to predict function degradation and reliability. Utility theory is helpful to evaluate the reuse options of common modules.

  9. Markov chain aggregation for agent-based models

    Banisch, Sven

    2016-01-01

    This self-contained text develops a Markov chain approach that makes the rigorous analysis of a class of microscopic models that specify the dynamics of complex systems at the individual level possible. It presents a general framework of aggregation in agent-based and related computational models, one which makes use of lumpability and information theory in order to link the micro and macro levels of observation. The starting point is a microscopic Markov chain description of the dynamical process in complete correspondence with the dynamical behavior of the agent-based model (ABM), which is obtained by considering the set of all possible agent configurations as the state space of a huge Markov chain. An explicit formal representation of a resulting “micro-chain” including microscopic transition rates is derived for a class of models by using the random mapping representation of a Markov process. The type of probability distribution used to implement the stochastic part of the model, which defines the upd...

  10. Fuzzy Markov random fields versus chains for multispectral image segmentation.

    Salzenstein, Fabien; Collet, Christophe

    2006-11-01

    This paper deals with a comparison of recent statistical models based on fuzzy Markov random fields and chains for multispectral image segmentation. The fuzzy scheme takes into account discrete and continuous classes which model the imprecision of the hidden data. In this framework, we assume the dependence between bands and we express the general model for the covariance matrix. A fuzzy Markov chain model is developed in an unsupervised way. This method is compared with the fuzzy Markovian field model previously proposed by one of the authors. The segmentation task is processed with Bayesian tools, such as the well-known MPM (Mode of Posterior Marginals) criterion. Our goal is to compare the robustness and rapidity for both methods (fuzzy Markov fields versus fuzzy Markov chains). Indeed, such fuzzy-based procedures seem to be a good answer, e.g., for astronomical observations when the patterns present diffuse structures. Moreover, these approaches allow us to process missing data in one or several spectral bands which correspond to specific situations in astronomy. To validate both models, we perform and compare the segmentation on synthetic images and raw multispectral astronomical data.

  11. Regularity of harmonic functions for some Markov chains with unbounded range

    2012-01-01

    We consider a class of continuous time Markov chains on $\\Z^d$. These chains are the discrete space analogue of Markov processes with jumps. Under some conditions, we show that harmonic functions associated with these Markov chains are H\\"{o}lder continuous.

  12. Occupancy distributions in Markov chains via Doeblin's ergodicity coefficient

    Chestnut, Stephen

    2010-01-01

    We apply Doeblin's ergodicity coefficient as a computational tool to approximate the occupancy distribution of a set of states in a homogeneous but possibly non-stationary finite Markov chain. Our approximation is based on new properties satisfied by this coefficient, which allow us to approximate a chain of duration n by independent and short-lived realizations of an auxiliary homogeneous Markov chain of duration of order ln(n). Our approximation may be particularly useful when exact calculations via first-step methods or transfer matrices are impractical, and asymptotic approximations may not be yet reliable. Our findings may find applications to pattern problems in Markovian and non-Markovian sequences that are treatable via embedding techniques.

  13. Asymptotics of Markov Kernels and the Tail Chain

    Resnick, Sidney I

    2011-01-01

    An asymptotic model for extreme behavior of certain Markov chains is the "tail chain". Generally taking the form of a multiplicative random walk, it is useful in deriving extremal characteristics such as point process limits. We place this model in a more general context, formulated in terms of extreme value theory for transition kernels, and extend it by formalizing the distinction between extreme and non-extreme states. We make the link between the update function and transition kernel forms considered in previous work, and we show that the tail chain model leads to a multivariate regular variation property of the finite-dimensional distributions under assumptions on the marginal tails alone.

  14. Markov chain Monte Carlo inference for Markov jump processes via the linear noise approximation.

    Stathopoulos, Vassilios; Girolami, Mark A

    2013-02-13

    Bayesian analysis for Markov jump processes (MJPs) is a non-trivial and challenging problem. Although exact inference is theoretically possible, it is computationally demanding, thus its applicability is limited to a small class of problems. In this paper, we describe the application of Riemann manifold Markov chain Monte Carlo (MCMC) methods using an approximation to the likelihood of the MJP that is valid when the system modelled is near its thermodynamic limit. The proposed approach is both statistically and computationally efficient whereas the convergence rate and mixing of the chains allow for fast MCMC inference. The methodology is evaluated using numerical simulations on two problems from chemical kinetics and one from systems biology.

  15. An Overview of Markov Chain Methods for the Study of Stage-Sequential Developmental Processes

    Kapland, David

    2008-01-01

    This article presents an overview of quantitative methodologies for the study of stage-sequential development based on extensions of Markov chain modeling. Four methods are presented that exemplify the flexibility of this approach: the manifest Markov model, the latent Markov model, latent transition analysis, and the mixture latent Markov model.…

  16. On Dirichlet eigenvectors for neutral two-dimensional Markov chains

    Champagnat, Nicolas; Miclo, Laurent

    2012-01-01

    We consider a general class of discrete, two-dimensional Markov chains modeling the dynamics of a population with two types, without mutation or immigration, and neutral in the sense that type has no influence on each individual's birth or death parameters. We prove that all the eigenvectors of the corresponding transition matrix or infinitesimal generator \\Pi\\ can be expressed as the product of "universal" polynomials of two variables, depending on each type's size but not on the specific transitions of the dynamics, and functions depending only on the total population size. These eigenvectors appear to be Dirichlet eigenvectors for \\Pi\\ on the complement of triangular subdomains, and as a consequence the corresponding eigenvalues are ordered in a specific way. As an application, we study the quasistationary behavior of finite, nearly neutral, two-dimensional Markov chains, absorbed in the sense that 0 is an absorbing state for each component of the process.

  17. Quantile estimation for a non-geometric ergodic Markov chain

    Ramirez-Nafarrate, Adrian; Muñoz, David F.

    2013-10-01

    Simulation has been successfully used for estimating performance measures (e.g. mean, variance and quantiles) of complex systems, such as queueing and inventory systems. However, parameter estimation using simulation may be a difficult task under some conditions. In this paper, we present a counterexample for which traditional simulation methods do not allow us to estimate the accuracy of the point estimators for the mean and risk performance measures for steady-state. The counterexample is based on a Markov chain with continuous state space and non-geometric ergodicity. The simulation of this Markov chain shows that neither multiple replications nor batch-based methodologies can produce asymptotically valid confidence intervals for the point estimators.

  18. Statistical significance test for transition matrices of atmospheric Markov chains

    Vautard, Robert; Mo, Kingtse C.; Ghil, Michael

    1990-01-01

    Low-frequency variability of large-scale atmospheric dynamics can be represented schematically by a Markov chain of multiple flow regimes. This Markov chain contains useful information for the long-range forecaster, provided that the statistical significance of the associated transition matrix can be reliably tested. Monte Carlo simulation yields a very reliable significance test for the elements of this matrix. The results of this test agree with previously used empirical formulae when each cluster of maps identified as a distinct flow regime is sufficiently large and when they all contain a comparable number of maps. Monte Carlo simulation provides a more reliable way to test the statistical significance of transitions to and from small clusters. It can determine the most likely transitions, as well as the most unlikely ones, with a prescribed level of statistical significance.

  19. Maximum Entropy Estimation of Transition Probabilities of Reversible Markov Chains

    Erik Van der Straeten

    2009-11-01

    Full Text Available In this paper, we develop a general theory for the estimation of the transition probabilities of reversible Markov chains using the maximum entropy principle. A broad range of physical models can be studied within this approach. We use one-dimensional classical spin systems to illustrate the theoretical ideas. The examples studied in this paper are: the Ising model, the Potts model and the Blume-Emery-Griffiths model.

  20. Classification of Markov chains describing the evolution of random strings

    Gairat, A. S.; Malyshev, V. A.; Men'shikov, M. V.; Pelikh, K. D.

    1995-04-01

    Contents §1. Introduction §2. Linear equation for average times: the case d = 1 Associated branching process The simplest queue §3. Fundamental non-linear equation: the case d > 1Minimal solution of the equation F(p) = p §4. The main criterion. The case d > 1 §5. Classification of Markov chains by means of a minimal solution §6. Appendix Bibliography

  1. Space system operations and support cost analysis using Markov chains

    Unal, Resit; Dean, Edwin B.; Moore, Arlene A.; Fairbairn, Robert E.

    1990-01-01

    This paper evaluates the use of Markov chain process in probabilistic life cycle cost analysis and suggests further uses of the process as a design aid tool. A methodology is developed for estimating operations and support cost and expected life for reusable space transportation systems. Application of the methodology is demonstrated for the case of a hypothetical space transportation vehicle. A sensitivity analysis is carried out to explore the effects of uncertainty in key model inputs.

  2. Recurrence and invariant measure of Markov chains in double-infinite random environments

    XING; Xiusan

    2001-01-01

    [1]Cogburn, R., Markov chains in random environments: The case of Markovian environments, Ann. Probab., 1980, 8(3): 908—916.[2]Cogburn, R., The ergodic theory of Markov chains in random environments, Z. W., 1984, 66(2): 109—128.[3]Orey, S., Markov chains with stochastically stationary transition probabilities, Ann. Probab., 1991, 19(3): 907—928.[4]Li Yingqiu, Some notes of Markov chains in Markov environments, Advances in Mathematics(in Chinese), 1999, 28(4): 358—360.

  3. Markov chain decision model for urinary incontinence procedures.

    Kumar, Sameer; Ghildayal, Nidhi; Ghildayal, Neha

    2017-03-13

    Purpose Urinary incontinence (UI) is a common chronic health condition, a problem specifically among elderly women that impacts quality of life negatively. However, UI is usually viewed as likely result of old age, and as such is generally not evaluated or even managed appropriately. Many treatments are available to manage incontinence, such as bladder training and numerous surgical procedures such as Burch colposuspension and Sling for UI which have high success rates. The purpose of this paper is to analyze which of these popular surgical procedures for UI is effective. Design/methodology/approach This research employs randomized, prospective studies to obtain robust cost and utility data used in the Markov chain decision model for examining which of these surgical interventions is more effective in treating women with stress UI based on two measures: number of quality adjusted life years (QALY) and cost per QALY. Treeage Pro Healthcare software was employed in Markov decision analysis. Findings Results showed the Sling procedure is a more effective surgical intervention than the Burch. However, if a utility greater than certain utility value, for which both procedures are equally effective, is assigned to persistent incontinence, the Burch procedure is more effective than the Sling procedure. Originality/value This paper demonstrates the efficacy of a Markov chain decision modeling approach to study the comparative effectiveness analysis of available treatments for patients with UI, an important public health issue, widely prevalent among elderly women in developed and developing countries. This research also improves upon other analyses using a Markov chain decision modeling process to analyze various strategies for treating UI.

  4. Markov chain evaluation of acute postoperative pain transition states.

    Tighe, Patrick J; Bzdega, Matthew; Fillingim, Roger B; Rashidi, Parisa; Aytug, Haldun

    2016-03-01

    Previous investigations on acute postoperative pain dynamicity have focused on daily pain assessments, and so were unable to examine intraday variations in acute pain intensity. We analyzed 476,108 postoperative acute pain intensity ratings, which were clinically documented on postoperative days 1 to 7 from 8346 surgical patients using Markov chain modeling to describe how patients are likely to transition from one pain state to another in a probabilistic fashion. The Markov chain was found to be irreducible and positive recurrent, with no absorbing states. Transition probabilities ranged from 0.0031, for the transition from state 10 to state 1, to 0.69 for the transition from state 0 to state 0. The greatest density of transitions was noted in the diagonal region of the transition matrix, suggesting that patients were generally most likely to transition to the same pain state as their current state. There were also slightly increased probability densities in transitioning to a state of asleep or 0 from the current state. An examination of the number of steps required to traverse from a particular first pain score to a target state suggested that overall, fewer steps were required to reach a state of 0 (range 6.1-8.8 steps) or asleep (range 9.1-11) than were required to reach a mild pain intensity state. Our results suggest that using Markov chains is a feasible method for describing probabilistic postoperative pain trajectories, pointing toward the possibility of using Markov decision processes to model sequential interactions between pain intensity ratings, and postoperative analgesic interventions.

  5. Inferring animal densities from tracking data using Markov chains.

    Hal Whitehead

    Full Text Available The distributions and relative densities of species are keys to ecology. Large amounts of tracking data are being collected on a wide variety of animal species using several methods, especially electronic tags that record location. These tracking data are effectively used for many purposes, but generally provide biased measures of distribution, because the starts of the tracks are not randomly distributed among the locations used by the animals. We introduce a simple Markov-chain method that produces unbiased measures of relative density from tracking data. The density estimates can be over a geographical grid, and/or relative to environmental measures. The method assumes that the tracked animals are a random subset of the population in respect to how they move through the habitat cells, and that the movements of the animals among the habitat cells form a time-homogenous Markov chain. We illustrate the method using simulated data as well as real data on the movements of sperm whales. The simulations illustrate the bias introduced when the initial tracking locations are not randomly distributed, as well as the lack of bias when the Markov method is used. We believe that this method will be important in giving unbiased estimates of density from the growing corpus of animal tracking data.

  6. Inferring animal densities from tracking data using Markov chains.

    Whitehead, Hal; Jonsen, Ian D

    2013-01-01

    The distributions and relative densities of species are keys to ecology. Large amounts of tracking data are being collected on a wide variety of animal species using several methods, especially electronic tags that record location. These tracking data are effectively used for many purposes, but generally provide biased measures of distribution, because the starts of the tracks are not randomly distributed among the locations used by the animals. We introduce a simple Markov-chain method that produces unbiased measures of relative density from tracking data. The density estimates can be over a geographical grid, and/or relative to environmental measures. The method assumes that the tracked animals are a random subset of the population in respect to how they move through the habitat cells, and that the movements of the animals among the habitat cells form a time-homogenous Markov chain. We illustrate the method using simulated data as well as real data on the movements of sperm whales. The simulations illustrate the bias introduced when the initial tracking locations are not randomly distributed, as well as the lack of bias when the Markov method is used. We believe that this method will be important in giving unbiased estimates of density from the growing corpus of animal tracking data.

  7. Dynamic temperature selection for parallel-tempering in Markov chain Monte Carlo simulations

    Vousden, Will; Mandel, Ilya

    2015-01-01

    Modern problems in astronomical Bayesian inference require efficient methods for sampling from complex, high-dimensional, often multi-modal probability distributions. Most popular methods, such as Markov chain Monte Carlo sampling, perform poorly on strongly multi-modal probability distributions, rarely jumping between modes or settling on just one mode without finding others. Parallel tempering addresses this problem by sampling simultaneously with separate Markov chains from tempered versions of the target distribution with reduced contrast levels. Gaps between modes can be traversed at higher temperatures, while individual modes can be efficiently explored at lower temperatures. In this paper, we investigate how one might choose the ladder of temperatures to achieve lower autocorrelation time for the sampler (and therefore more efficient sampling). In particular, we present a simple, easily-implemented algorithm for dynamically adapting the temperature configuration of a sampler while sampling in order to ...

  8. DREAM(D: an adaptive Markov Chain Monte Carlo simulation algorithm to solve discrete, noncontinuous, and combinatorial posterior parameter estimation problems

    C. J. F. Ter Braak

    2011-12-01

    Full Text Available Formal and informal Bayesian approaches have found widespread implementation and use in environmental modeling to summarize parameter and predictive uncertainty. Successful implementation of these methods relies heavily on the availability of efficient sampling methods that approximate, as closely and consistently as possible the (evolving posterior target distribution. Much of this work has focused on continuous variables that can take on any value within their prior defined ranges. Here, we introduce theory and concepts of a discrete sampling method that resolves the parameter space at fixed points. This new code, entitled DREAM(D uses the recently developed DREAM algorithm (Vrugt et al., 2008, 2009a, b as its main building block but implements two novel proposal distributions to help solve discrete and combinatorial optimization problems. This novel MCMC sampler maintains detailed balance and ergodicity, and is especially designed to resolve the emerging class of optimal experimental design problems. Three different case studies involving a Sudoku puzzle, soil water retention curve, and rainfall – runoff model calibration problem are used to benchmark the performance of DREAM(D. The theory and concepts developed herein can be easily integrated into other (adaptive MCMC algorithms.

  9. SDI and Markov Chains for Regional Drought Characteristics

    Chen-Feng Yeh

    2015-08-01

    Full Text Available In recent years, global climate change has altered precipitation patterns, causing uneven spatial and temporal distribution of precipitation that gradually induces precipitation polarization phenomena. Taiwan is located in the subtropical climate zone, with distinct wet and dry seasons, which makes the polarization phenomenon more obvious; this has also led to a large difference between river flows during the wet and dry seasons, which is significantly influenced by precipitation, resulting in hydrological drought. Therefore, to effectively address the growing issue of water shortages, it is necessary to explore and assess the drought characteristics of river systems. In this study, the drought characteristics of northern Taiwan were studied using the streamflow drought index (SDI and Markov chains. Analysis results showed that the year 2002 was a turning point for drought severity in both the Lanyang River and Yilan River basins; the severity of rain events in the Lanyang River basin increased after 2002, and the severity of drought events in the Yilan River basin exhibited a gradual upward trend. In the study of drought severity, analysis results from periods of three months (November to January and six months (November to April have shown significant drought characteristics. In addition, analysis of drought occurrence probabilities using the method of Markov chains has shown that the occurrence probabilities of drought events are higher in the Lanyang River basin than in the Yilan River basin; particularly for extreme events, the occurrence probability of an extreme drought event is 20.6% during the dry season (November to April in the Lanyang River basin, and 3.4% in the Yilan River basin. This study shows that for analysis of drought/wet occurrence probabilities, the results obtained for the drought frequency and occurrence probability using short-term data with the method of Markov chains can be used to predict the long-term occurrence

  10. Markov Chain Modelling for Short-Term NDVI Time Series Forecasting

    Stepčenko Artūrs

    2016-12-01

    Full Text Available In this paper, the NDVI time series forecasting model has been developed based on the use of discrete time, continuous state Markov chain of suitable order. The normalised difference vegetation index (NDVI is an indicator that describes the amount of chlorophyll (the green mass and shows the relative density and health of vegetation; therefore, it is an important variable for vegetation forecasting. A Markov chain is a stochastic process that consists of a state space. This stochastic process undergoes transitions from one state to another in the state space with some probabilities. A Markov chain forecast model is flexible in accommodating various forecast assumptions and structures. The present paper discusses the considerations and techniques in building a Markov chain forecast model at each step. Continuous state Markov chain model is analytically described. Finally, the application of the proposed Markov chain model is illustrated with reference to a set of NDVI time series data.

  11. Prediction of primate splice site using inhomogeneous Markov chain and neural network.

    Liu, Libin; Ho, Yee-Kin; Yau, Stephen

    2007-07-01

    The inhomogeneous Markov chain model is used to discriminate acceptor and donor sites in genomic DNA sequences. It outperforms statistical methods such as homogeneous Markov chain model, higher order Markov chain and interpolated Markov chain models, and machine-learning methods such as k-nearest neighbor and support vector machine as well. Besides its high accuracy, another advantage of inhomogeneous Markov chain model is its simplicity in computation. In the three states system (acceptor, donor, and neither), the inhomogeneous Markov chain model is combined with a three-layer feed forward neural network. Using this combined system 3175 primate splice-junction gene sequences have been tested, with a prediction accuracy of greater than 98%.

  12. A Markov chain representation of the multiple testing problem.

    Cabras, Stefano

    2016-03-16

    The problem of multiple hypothesis testing can be represented as a Markov process where a new alternative hypothesis is accepted in accordance with its relative evidence to the currently accepted one. This virtual and not formally observed process provides the most probable set of non null hypotheses given the data; it plays the same role as Markov Chain Monte Carlo in approximating a posterior distribution. To apply this representation and obtain the posterior probabilities over all alternative hypotheses, it is enough to have, for each test, barely defined Bayes Factors, e.g. Bayes Factors obtained up to an unknown constant. Such Bayes Factors may either arise from using default and improper priors or from calibrating p-values with respect to their corresponding Bayes Factor lower bound. Both sources of evidence are used to form a Markov transition kernel on the space of hypotheses. The approach leads to easy interpretable results and involves very simple formulas suitable to analyze large datasets as those arising from gene expression data (microarray or RNA-seq experiments).

  13. Exact Markov chains versus diffusion theory for haploid random mating.

    Tyvand, Peder A; Thorvaldsen, Steinar

    2010-05-01

    Exact discrete Markov chains are applied to the Wright-Fisher model and the Moran model of haploid random mating. Selection and mutations are neglected. At each discrete value of time t there is a given number n of diploid monoecious organisms. The evolution of the population distribution is given in diffusion variables, to compare the two models of random mating with their common diffusion limit. Only the Moran model converges uniformly to the diffusion limit near the boundary. The Wright-Fisher model allows the population size to change with the generations. Diffusion theory tends to under-predict the loss of genetic information when a population enters a bottleneck.

  14. Improvement of Fuzzy Image Contrast Enhancement Using Simulated Ergodic Fuzzy Markov Chains

    Behrouz Fathi-Vajargah

    2014-01-01

    Full Text Available This paper presents a novel fuzzy enhancement technique using simulated ergodic fuzzy Markov chains for low contrast brain magnetic resonance imaging (MRI. The fuzzy image contrast enhancement is proposed by weighted fuzzy expected value. The membership values are then modified to enhance the image using ergodic fuzzy Markov chains. The qualitative performance of the proposed method is compared to another method in which ergodic fuzzy Markov chains are not considered. The proposed method produces better quality image.

  15. MARKOV CHAIN MODELING OF PERFORMANCE DEGRADATION OF PHOTOVOLTAIC SYSTEM

    E. Suresh Kumar

    2012-01-01

    Full Text Available Modern probability theory studies chance processes for which theknowledge of previous outcomes influence predictions for future experiments. In principle, when a sequence of chance experiments, all of the past outcomes could influence the predictions for the next experiment. In Markov chain type of chance, the outcome of a given experiment can affect the outcome of the next experiment. The system state changes with time and the state X and time t are two random variables. Each of these variables can be either continuous or discrete. Various degradation on photovoltaic (PV systems can be viewed as different Markov states and further degradation can be treated as the outcome of the present state. The PV system is treated as a discrete state continuous time system with four possible outcomes, namely, s1 : Good condition, s2 : System with partial degradation failures and fully operational, s3 : System with major faults and partially working and hence partial output power, s4 : System completely fails. The calculation of the reliability of the photovoltaic system is complicated since the system have elements or subsystems exhibiting dependent failures and involving repair and standby operations. Markov model is a better technique that has much appeal and works well when failure hazards and repair hazards are constant. The usual practice of reliability analysis techniques include FMEA((failure mode and effect analysis, Parts count analysis, RBD ( reliability block diagram , FTA( fault tree analysis etc. These are logical, boolean and block diagram approaches and never accounts the environmental degradation on the performance of the system. This is too relevant in the case of PV systems which are operated under harsh environmental conditions. This paper is an insight into the degradation of performance of PV systems and presenting a Markov model of the system by means of the different states and transitions between these states.

  16. Radiative transfer calculated from a Markov chain formalism

    Esposito, L. W.; House, L. L.

    1978-01-01

    The theory of Markov chains is used to formulate the radiative transport problem in a general way by modeling the successive interactions of a photon as a stochastic process. Under the minimal requirement that the stochastic process is a Markov chain, the determination of the diffuse reflection or transmission from a scattering atmosphere is equivalent to the solution of a system of linear equations. This treatment is mathematically equivalent to, and thus has many of the advantages of, Monte Carlo methods, but can be considerably more rapid than Monte Carlo algorithms for numerical calculations in particular applications. We have verified the speed and accuracy of this formalism for the standard problem of finding the intensity of scattered light from a homogeneous plane-parallel atmosphere with an arbitrary phase function for scattering. Accurate results over a wide range of parameters were obtained with computation times comparable to those of a standard 'doubling' routine. The generality of this formalism thus allows fast, direct solutions to problems that were previously soluble only by Monte Carlo methods. Some comparisons are made with respect to integral equation methods.

  17. On Markov Chains Induced by Partitioned Transition Probability Matrices

    Thomas KAIJSER

    2011-01-01

    Let S be a denumerable state space and let P be a transition probability matrix on S. If a denumerable set M of nonnegative matrices is such that the sum of the matrices is equal to P, then we call M a partition of P. Let K denote the set of probability vectors on S. With every partition M of P we can associate a transition probability function PM on K defined in such a way that if p ∈ K and M ∈ M are such that ‖pM‖ > 0, then, with probability ‖pM‖, the vector p is transferred to the vector pM/‖pM‖. Here ‖· ‖ denotes the l1-norm. In this paper we investigate the convergence in distribution for Markov chains generated by transition probability functions induced by partitions of transition probability matrices. The main motivation for this investigation is the application of the convergence results obtained to filtering processes of partially observed Markov chains with denumerable state space.

  18. Markov chain aggregation and its applications to combinatorial reaction networks.

    Ganguly, Arnab; Petrov, Tatjana; Koeppl, Heinz

    2014-09-01

    We consider a continuous-time Markov chain (CTMC) whose state space is partitioned into aggregates, and each aggregate is assigned a probability measure. A sufficient condition for defining a CTMC over the aggregates is presented as a variant of weak lumpability, which also characterizes that the measure over the original process can be recovered from that of the aggregated one. We show how the applicability of de-aggregation depends on the initial distribution. The application section is devoted to illustrate how the developed theory aids in reducing CTMC models of biochemical systems particularly in connection to protein-protein interactions. We assume that the model is written by a biologist in form of site-graph-rewrite rules. Site-graph-rewrite rules compactly express that, often, only a local context of a protein (instead of a full molecular species) needs to be in a certain configuration in order to trigger a reaction event. This observation leads to suitable aggregate Markov chains with smaller state spaces, thereby providing sufficient reduction in computational complexity. This is further exemplified in two case studies: simple unbounded polymerization and early EGFR/insulin crosstalk.

  19. A Markov chain model for reliability growth and decay

    Siegrist, K.

    1982-01-01

    A mathematical model is developed to describe a complex system undergoing a sequence of trials in which there is interaction between the internal states of the system and the outcomes of the trials. For example, the model might describe a system undergoing testing that is redesigned after each failure. The basic assumptions for the model are that the state of the system after a trial depends probabilistically only on the state before the trial and on the outcome of the trial and that the outcome of a trial depends probabilistically only on the state of the system before the trial. It is shown that under these basic assumptions, the successive states form a Markov chain and the successive states and outcomes jointly form a Markov chain. General results are obtained for the transition probabilities, steady-state distributions, etc. A special case studied in detail describes a system that has two possible state ('repaired' and 'unrepaired') undergoing trials that have three possible outcomes ('inherent failure', 'assignable-cause' 'failure' and 'success'). For this model, the reliability function is computed explicitly and an optimal repair policy is obtained.

  20. Optimized nested Markov chain Monte Carlo sampling: theory

    Coe, Joshua D [Los Alamos National Laboratory; Shaw, M Sam [Los Alamos National Laboratory; Sewell, Thomas D [U. MISSOURI

    2009-01-01

    Metropolis Monte Carlo sampling of a reference potential is used to build a Markov chain in the isothermal-isobaric ensemble. At the endpoints of the chain, the energy is reevaluated at a different level of approximation (the 'full' energy) and a composite move encompassing all of the intervening steps is accepted on the basis of a modified Metropolis criterion. By manipulating the thermodynamic variables characterizing the reference system we maximize the average acceptance probability of composite moves, lengthening significantly the random walk made between consecutive evaluations of the full energy at a fixed acceptance probability. This provides maximally decorrelated samples of the full potential, thereby lowering the total number required to build ensemble averages of a given variance. The efficiency of the method is illustrated using model potentials appropriate to molecular fluids at high pressure. Implications for ab initio or density functional theory (DFT) treatment are discussed.

  1. Large deviations for Markov chains in the positive quadrant

    Borovkov, A. A.; Mogul'skii, A. A.

    2001-10-01

    The paper deals with so-called N-partially space-homogeneous time-homogeneous Markov chains X(y,n), n=0,1,2,\\dots, X(y,0)=y, in the positive quadrant \\mathbb R^{2+}=\\{x=(x_2,x_2):x_1\\geqslant0,\\ x_2\\geqslant0\\}. These Markov chains are characterized by the following property of the transition probabilities P(y,A)=\\mathsf P(X(y,1)\\in A): for some N\\geqslant 0 the measure P(y,dx) depends only on x_2, y_2, and x_1-y_1 in the domain x_1>N, y_1>N, and only on x_1, y_1, and x_2-y_2 in the domain x_2>N, y_2>N. For such chains the asymptotic behaviour of \\displaystyle \\ln\\mathsf P\\Bigl(\\frac 1sX(y,n)\\in B\\Bigr), \\qquad \\ln\\mathsf P\\bigl(X(y,n)\\in x+B\\bigr) is found for a fixed set B as s\\to\\infty, \\vert x\\vert\\to\\infty, and n\\to\\infty. Some other conditions on the growth of parameters are also considered, for example, \\vert x-y\\vert\\to\\infty, \\vert y\\vert\\to\\infty. A study is made of the structure of the most probable trajectories, which give the main contribution to this asymptotics, and a number of other results pertaining to the topic are established. Similar results are obtained for the narrower class of 0-partially homogeneous ergodic chains under less restrictive moment conditions on the transition probabilities P(y,dx). Moreover, exact asymptotic expressions for the probabilities \\mathsf P(X(0,n)\\in x+B) are found for 0-partially homogeneous ergodic chains under some additional conditions. The interest in partially homogeneous Markov chains in positive octants is due to the mathematical aspects (new and interesting problems arise in the framework of general large deviation theory) as well as applied issues, for such chains prove to be quite accurate mathematical models for numerous basic types of queueing and communication networks such as the widely known Jackson networks, polling systems, or communication networks associated with the ALOHA algorithm. There is a vast literature dealing with the analysis of these objects. The

  2. Modeling and computing of stock index forecasting based on neural network and Markov chain.

    Dai, Yonghui; Han, Dongmei; Dai, Weihui

    2014-01-01

    The stock index reflects the fluctuation of the stock market. For a long time, there have been a lot of researches on the forecast of stock index. However, the traditional method is limited to achieving an ideal precision in the dynamic market due to the influences of many factors such as the economic situation, policy changes, and emergency events. Therefore, the approach based on adaptive modeling and conditional probability transfer causes the new attention of researchers. This paper presents a new forecast method by the combination of improved back-propagation (BP) neural network and Markov chain, as well as its modeling and computing technology. This method includes initial forecasting by improved BP neural network, division of Markov state region, computing of the state transition probability matrix, and the prediction adjustment. Results of the empirical study show that this method can achieve high accuracy in the stock index prediction, and it could provide a good reference for the investment in stock market.

  3. Perfect Sampling of Markov Chains with Piecewise Homogeneous Events

    Bušić, Ana; Pin, Furcy

    2010-01-01

    Perfect sampling is a technique that uses coupling arguments to provide a sample from the stationary distribution of a Markov chain in a finite time without ever computing the distribution. This technique is very efficient if all the events in the system have monotonicity property. However, in the general (non-monotone) case, this technique needs to consider the whole state space, which limits its application only to chains with a state space of small cardinality. We propose here a new approach for the general case that only needs to consider two trajectories. Instead of the original chain, we use two bounding processes (envelopes) and we show that, whenever they couple, one obtains a sample under the stationary distribution of the original chain. We show that this new approach is particularly effective when the state space can be partitioned into pieces where envelopes can be easily computed. We further show that most Markovian queueing networks have this property and we propose efficient algorithms for some...

  4. A Markov Chain Estimator of Multivariate Volatility from High Frequency Data

    Hansen, Peter Reinhard; Horel, Guillaume; Lunde, Asger;

    We introduce a multivariate estimator of financial volatility that is based on the theory of Markov chains. The Markov chain framework takes advantage of the discreteness of high-frequency returns. We study the finite sample properties of the estimation in a simulation study and apply...

  5. Rate Function of Large Deviation for a Class of Nonhomogeneous Markov Chains on Supercritical Percolation Network

    Zhong Hao XU; Dong HAN

    2011-01-01

    We model an epidemic with a class of nonhomogeneous Markov chains on the supercritical percolation network on Zd.The large deviations law for the Markov chain is given.Explicit expression of the rate function for large deviation is obtained.

  6. A note on asymptotic expansions for Markov chains using operator theory

    Jensen, J.L.

    1987-01-01

    We consider asymptotic expansions for sums Sn on the form Sn = fhook0(X0) + fhook(X1, X0) + ... + fhook(Xn, Xn-1), where Xi is a Markov chain. Under different ergodicity conditions on the Markov chain and certain conditional moment conditions on fhook(Xi, Xi-1), a simple representation...

  7. Almost Sure Central Limit Theorem for Partial Sums of Markov Chain

    Guangming ZHUANG; Zuoxiang PENG; Zhongquan TAN

    2012-01-01

    The authors prove an almost sure central limit theorem for partial sums based on an irreducible and positive recurrent Markov chain using logarithmic means,which realizes the extension of the almost sure central limit theorem for partial sums from an i.i.d.sequence of random variables to a Markov chain.

  8. Grey Markov chain and its application in drift prediction model of FOGs

    Fan Chunling; Jin Zhihua; Tian Weifeng; Qian Feng

    2005-01-01

    A novel grey Markov chain predictive model is discussed to reduce drift influence on the output of fiber optical gyroscopes (FOGs) and to improve FOGs' measurement precision. The proposed method possesses advantages of grey model and Markov chain. It makes good use of dynamic modeling idea of the grey model to predict general trend of original data. Then according to the trend, states are divided so that it can overcome the disadvantage of high computational cost of state transition probability matrix in Markov chain. Moreover, the presented approach expands the applied scope of the grey model and makes it be fit for prediction of random data with bigger fluctuation. The numerical results of real drift data from a certain type FOG verify the effectiveness of the proposed grey Markov chain model powerfully. The Markov chain is also investigated to provide a comparison with the grey Markov chain model. It is shown that the hybrid grey Markov chain prediction model has higher modeling precision than Markov chain itself, which prove this proposed method is very applicable and effective.

  9. Bayesian seismic tomography by parallel interacting Markov chains

    Gesret, Alexandrine; Bottero, Alexis; Romary, Thomas; Noble, Mark; Desassis, Nicolas

    2014-05-01

    The velocity field estimated by first arrival traveltime tomography is commonly used as a starting point for further seismological, mineralogical, tectonic or similar analysis. In order to interpret quantitatively the results, the tomography uncertainty values as well as their spatial distribution are required. The estimated velocity model is obtained through inverse modeling by minimizing an objective function that compares observed and computed traveltimes. This step is often performed by gradient-based optimization algorithms. The major drawback of such local optimization schemes, beyond the possibility of being trapped in a local minimum, is that they do not account for the multiple possible solutions of the inverse problem. They are therefore unable to assess the uncertainties linked to the solution. Within a Bayesian (probabilistic) framework, solving the tomography inverse problem aims at estimating the posterior probability density function of velocity model using a global sampling algorithm. Markov chains Monte-Carlo (MCMC) methods are known to produce samples of virtually any distribution. In such a Bayesian inversion, the total number of simulations we can afford is highly related to the computational cost of the forward model. Although fast algorithms have been recently developed for computing first arrival traveltimes of seismic waves, the complete browsing of the posterior distribution of velocity model is hardly performed, especially when it is high dimensional and/or multimodal. In the latter case, the chain may even stay stuck in one of the modes. In order to improve the mixing properties of classical single MCMC, we propose to make interact several Markov chains at different temperatures. This method can make efficient use of large CPU clusters, without increasing the global computational cost with respect to classical MCMC and is therefore particularly suited for Bayesian inversion. The exchanges between the chains allow a precise sampling of the

  10. Logics and Models for Stochastic Analysis Beyond Markov Chains

    Zeng, Kebin

    form of discrete PH distributions as computational vehicle on measuring the performance of concurrent wireless sensor networks. Secondly, choosing stochastic process algebras as a widely accepted formalism, we study the compositionality of continuous PH distributions in order to support modelling......Within the last twenty years, logics and models for stochastic analysis of information systems have been widely studied in both theory and practice. The quantitative properties, such as performance and reliability, are evaluated over discrete–time and continuous–time Markov chains. This thesis...... kernels. The most appealing feature is that all the components before and after compositions are secured to have a minimal state space representation. To support quantitative verification, we also propose stochastic model checking algorithms to our problem....

  11. Analysis of Users Web Browsing Behavior Using Markov chain Model

    Diwakar Shukla

    2011-03-01

    Full Text Available In present days of growing information technology, many browsers available for surfing and web mining. A user has option to use any of them at a time to mine out the desired website. Every browser has pre-defined level of popularity and reputation in the market. This paper considers the setup of only two browsers in a computer system and a user prefers to any one, if fails, switches to the other one .The behavior of user is modeled through Markov chain procedure and transition probabilities are calculated. The quitting to browsing is treated as a parameter of variation over the popularity. Graphical study is performed to explain the inter relationship between user behavior parameters and browser market popularity parameters. If rate of a company is lowest in terms of browser failure and lowest in terms of quitting probability then company enjoys better popularity and larger user proportion

  12. Kinetics and thermodynamics of first-order Markov chain copolymerization

    Gaspard, P.; Andrieux, D.

    2014-07-01

    We report a theoretical study of stochastic processes modeling the growth of first-order Markov copolymers, as well as the reversed reaction of depolymerization. These processes are ruled by kinetic equations describing both the attachment and detachment of monomers. Exact solutions are obtained for these kinetic equations in the steady regimes of multicomponent copolymerization and depolymerization. Thermodynamic equilibrium is identified as the state at which the growth velocity is vanishing on average and where detailed balance is satisfied. Away from equilibrium, the analytical expression of the thermodynamic entropy production is deduced in terms of the Shannon disorder per monomer in the copolymer sequence. The Mayo-Lewis equation is recovered in the fully irreversible growth regime. The theory also applies to Bernoullian chains in the case where the attachment and detachment rates only depend on the reacting monomer.

  13. Efficient Word Alignment with Markov Chain Monte Carlo

    Östling Robert

    2016-10-01

    Full Text Available We present EFMARAL, a new system for efficient and accurate word alignment using a Bayesian model with Markov Chain Monte Carlo (MCMC inference. Through careful selection of data structures and model architecture we are able to surpass the fast_align system, commonly used for performance-critical word alignment, both in computational efficiency and alignment accuracy. Our evaluation shows that a phrase-based statistical machine translation (SMT system produces translations of higher quality when using word alignments from EFMARAL than from fast_align, and that translation quality is on par with what is obtained using GIZA++, a tool requiring orders of magnitude more processing time. More generally we hope to convince the reader that Monte Carlo sampling, rather than being viewed as a slow method of last resort, should actually be the method of choice for the SMT practitioner and others interested in word alignment.

  14. Markov Chain Monte-Carlo Models of Starburst Clusters

    Melnick, Jorge

    2015-01-01

    There are a number of stochastic effects that must be considered when comparing models to observations of starburst clusters: the IMF is never fully populated; the stars can never be strictly coeval; stars rotate and their photometric properties depend on orientation; a significant fraction of massive stars are in interacting binaries; and the extinction varies from star to star. The probability distributions of each of these effects are not a priori known, but must be extracted from the observations. Markov Chain Monte-Carlo methods appear to provide the best statistical approach. Here I present an example of stochastic age effects upon the upper mass limit of the IMF of the Arches cluster as derived from near-IR photometry.

  15. Markov Chain Monte Carlo Bayesian Learning for Neural Networks

    Goodrich, Michael S.

    2011-01-01

    Conventional training methods for neural networks involve starting al a random location in the solution space of the network weights, navigating an error hyper surface to reach a minimum, and sometime stochastic based techniques (e.g., genetic algorithms) to avoid entrapment in a local minimum. It is further typically necessary to preprocess the data (e.g., normalization) to keep the training algorithm on course. Conversely, Bayesian based learning is an epistemological approach concerned with formally updating the plausibility of competing candidate hypotheses thereby obtaining a posterior distribution for the network weights conditioned on the available data and a prior distribution. In this paper, we developed a powerful methodology for estimating the full residual uncertainty in network weights and therefore network predictions by using a modified Jeffery's prior combined with a Metropolis Markov Chain Monte Carlo method.

  16. On the Multilevel Solution Algorithm for Markov Chains

    Horton, Graham

    1997-01-01

    We discuss the recently introduced multilevel algorithm for the steady-state solution of Markov chains. The method is based on an aggregation principle which is well established in the literature and features a multiplicative coarse-level correction. Recursive application of the aggregation principle, which uses an operator-dependent coarsening, yields a multi-level method which has been shown experimentally to give results significantly faster than the typical methods currently in use. When cast as a multigrid-like method, the algorithm is seen to be a Galerkin-Full Approximation Scheme with a solution-dependent prolongation operator. Special properties of this prolongation lead to the cancellation of the computationally intensive terms of the coarse-level equations.

  17. Nonequilibrium thermodynamic potentials for continuous-time Markov chains.

    Verley, Gatien

    2016-01-01

    We connect the rare fluctuations of an equilibrium (EQ) process and the typical fluctuations of a nonequilibrium (NE) stationary process. In the framework of large deviation theory, this observation allows us to introduce NE thermodynamic potentials. For continuous-time Markov chains, we identify the relevant pairs of conjugated variables and propose two NE ensembles: one with fixed dynamics and fluctuating time-averaged variables, and another with fixed time-averaged variables, but a fluctuating dynamics. Accordingly, we show that NE processes are equivalent to conditioned EQ processes ensuring that NE potentials are Legendre dual. We find a variational principle satisfied by the NE potentials that reach their maximum in the NE stationary state and whose first derivatives produce the NE equations of state and second derivatives produce the NE Maxwell relations generalizing the Onsager reciprocity relations.

  18. HYDRA: a Java library for Markov Chain Monte Carlo

    Gregory R. Warnes

    2002-03-01

    Full Text Available Hydra is an open-source, platform-neutral library for performing Markov Chain Monte Carlo. It implements the logic of standard MCMC samplers within a framework designed to be easy to use, extend, and integrate with other software tools. In this paper, we describe the problem that motivated our work, outline our goals for the Hydra pro ject, and describe the current features of the Hydra library. We then provide a step-by-step example of using Hydra to simulate from a mixture model drawn from cancer genetics, first using a variable-at-a-time Metropolis sampler and then a Normal Kernel Coupler. We conclude with a discussion of future directions for Hydra.

  19. On the multi-level solution algorithm for Markov chains

    Horton, G. [Univ. of Erlangen, Nuernberg (Germany)

    1996-12-31

    We discuss the recently introduced multi-level algorithm for the steady-state solution of Markov chains. The method is based on the aggregation principle, which is well established in the literature. Recursive application of the aggregation yields a multi-level method which has been shown experimentally to give results significantly faster than the methods currently in use. The algorithm can be reformulated as an algebraic multigrid scheme of Galerkin-full approximation type. The uniqueness of the scheme stems from its solution-dependent prolongation operator which permits significant computational savings in the evaluation of certain terms. This paper describes the modeling of computer systems to derive information on performance, measured typically as job throughput or component utilization, and availability, defined as the proportion of time a system is able to perform a certain function in the presence of component failures and possibly also repairs.

  20. The Importance Analysis of Use Case Map with Markov Chains

    Feng, Yaping

    2010-01-01

    UCMs (Use Case Maps) model describes functional requirements and high-level designs with causal paths superimposed on a structure of components. It could provide useful resources for software acceptance testing. However until now statistical testing technologies for large scale software is not considered yet in UCMs model. Thus if one applies UCMs model to a large scale software using traditional coverage based exhaustive tasting, then it requires too much costs for the quality assurance. Therefore this paper proposes an importance analysis of UCMs model with Markov chains. With this approach not only highly frequently used usage scenarios but also important objects such as components, responsibilities, stubs and plugins can also be identified from UCMs specifications. Therefore careful analysis, design, implementation and efficient testing could be possible with the importance of scenarios and objects during the full software life cycle. Consequently product reliability can be obtained with low costs. This p...

  1. Projection methods for the numerical solution of Markov chain models

    Saad, Youcef

    1989-01-01

    Projection methods for computing stationary probability distributions for Markov chain models are presented. A general projection method is a method which seeks an approximation from a subspace of small dimension to the original problem. Thus, the original matrix problem of size N is approximated by one of dimension m, typically much smaller than N. A particularly successful class of methods based on this principle is that of Krylov subspace methods which utilize subspaces of the form span(v,av,...,A(exp m-1)v). These methods are effective in solving linear systems and eigenvalue problems (Lanczos, Arnoldi,...) as well as nonlinear equations. They can be combined with more traditional iterative methods such as successive overrelaxation, symmetric successive overrelaxation, or with incomplete factorization methods to enhance convergence.

  2. Denker-Sato type Markov chains and Harnack inequality

    Deng, Qi-Rong; Wang, Xiang-Yang

    2015-10-01

    In ([DS1], [DS2], [DS3]), Denker and Sato studied a Markov chain on the finite words space of the Sierpinski gasket (SG). They showed that the Martin boundary is homeomorphic to the SG. Recently, Lau and Wang (2015 Math. Z. 280 401-20) showed that the homeomorphism holds for an iterated function system with the open set condition provided that the transition probability on the finite words space is of DS-type. In this work, we continue studying this kind of transition probability on the unit interval. Using matrix expressions, we obtain a formula to calculate the Green function. By the ergodic arguments for non-negative matrices, we find that the Martin boundary is homeomorphic to the unit interval or the union of the unit interval and a countable set. This gives a good illustration for the results in Lau and Wang (2015 Math. Z. 280 401-20).

  3. Recurrence and invariant measure of Markov chains in double-infinite random environments

    2001-01-01

    The concepts of π-irreduciblity, recurrence and transience are introduced into the research field of Markov chains in random environments.That a π-irreducible chain must be either recurrent or transient is proved, a criterion is shown for recurrent Markov chains in double-infinite random environments, the existence of invariant measure of π-irreducible chains in double-infinite environments is discussed,and then Orey's open-questions are partially answered.

  4. Stochastic model of milk homogenization process using Markov's chain

    A. A. Khvostov

    2016-01-01

    Full Text Available The process of development of a mathematical model of the process of homogenization of dairy products is considered in the work. The theory of Markov's chains was used in the development of the mathematical model, Markov's chain with discrete states and continuous parameter for which the homogenisation pressure is taken, being the basis for the model structure. Machine realization of the model is implemented in the medium of structural modeling MathWorks Simulink™. Identification of the model parameters was carried out by minimizing the standard deviation calculated from the experimental data for each fraction of dairy products fat phase. As the set of experimental data processing results of the micrographic images of fat globules of whole milk samples distribution which were subjected to homogenization at different pressures were used. Pattern Search method was used as optimization method with the Latin Hypercube search algorithm from Global Optimization Тoolbox library. The accuracy of calculations averaged over all fractions of 0.88% (the relative share of units, the maximum relative error was 3.7% with the homogenization pressure of 30 MPa, which may be due to the very abrupt change in properties from the original milk in the particle size distribution at the beginning of the homogenization process and the lack of experimental data at homogenization pressures of below the specified value. The mathematical model proposed allows to calculate the profile of volume and mass distribution of the fat phase (fat globules in the product, depending on the homogenization pressure and can be used in the laboratory and research of dairy products composition, as well as in the calculation, design and modeling of the process equipment of the dairy industry enterprises.

  5. Accelerating Monte Carlo Markov chains with proxy and error models

    Josset, Laureline; Demyanov, Vasily; Elsheikh, Ahmed H.; Lunati, Ivan

    2015-12-01

    In groundwater modeling, Monte Carlo Markov Chain (MCMC) simulations are often used to calibrate aquifer parameters and propagate the uncertainty to the quantity of interest (e.g., pollutant concentration). However, this approach requires a large number of flow simulations and incurs high computational cost, which prevents a systematic evaluation of the uncertainty in the presence of complex physical processes. To avoid this computational bottleneck, we propose to use an approximate model (proxy) to predict the response of the exact model. Here, we use a proxy that entails a very simplified description of the physics with respect to the detailed physics described by the "exact" model. The error model accounts for the simplification of the physical process; and it is trained on a learning set of realizations, for which both the proxy and exact responses are computed. First, the key features of the set of curves are extracted using functional principal component analysis; then, a regression model is built to characterize the relationship between the curves. The performance of the proposed approach is evaluated on the Imperial College Fault model. We show that the joint use of the proxy and the error model to infer the model parameters in a two-stage MCMC set-up allows longer chains at a comparable computational cost. Unnecessary evaluations of the exact responses are avoided through a preliminary evaluation of the proposal made on the basis of the corrected proxy response. The error model trained on the learning set is crucial to provide a sufficiently accurate prediction of the exact response and guide the chains to the low misfit regions. The proposed methodology can be extended to multiple-chain algorithms or other Bayesian inference methods. Moreover, FPCA is not limited to the specific presented application and offers a general framework to build error models.

  6. Resilient model approximation for Markov jump time-delay systems via reduced model with hierarchical Markov chains

    Zhu, Yanzheng; Zhang, Lixian; Sreeram, Victor; Shammakh, Wafa; Ahmad, Bashir

    2016-10-01

    In this paper, the resilient model approximation problem for a class of discrete-time Markov jump time-delay systems with input sector-bounded nonlinearities is investigated. A linearised reduced-order model is determined with mode changes subject to domination by a hierarchical Markov chain containing two different nonhomogeneous Markov chains. Hence, the reduced-order model obtained not only reflects the dependence of the original systems but also model external influence that is related to the mode changes of the original system. Sufficient conditions formulated in terms of bilinear matrix inequalities for the existence of such models are established, such that the resulting error system is stochastically stable and has a guaranteed l2-l∞ error performance. A linear matrix inequalities optimisation coupled with line search is exploited to solve for the corresponding reduced-order systems. The potential and effectiveness of the developed theoretical results are demonstrated via a numerical example.

  7. Analysing grouping of nucleotides in DNA sequences using lumped processes constructed from Markov chains.

    Guédon, Yann; d'Aubenton-Carafa, Yves; Thermes, Claude

    2006-03-01

    The most commonly used models for analysing local dependencies in DNA sequences are (high-order) Markov chains. Incorporating knowledge relative to the possible grouping of the nucleotides enables to define dedicated sub-classes of Markov chains. The problem of formulating lumpability hypotheses for a Markov chain is therefore addressed. In the classical approach to lumpability, this problem can be formulated as the determination of an appropriate state space (smaller than the original state space) such that the lumped chain defined on this state space retains the Markov property. We propose a different perspective on lumpability where the state space is fixed and the partitioning of this state space is represented by a one-to-many probabilistic function within a two-level stochastic process. Three nested classes of lumped processes can be defined in this way as sub-classes of first-order Markov chains. These lumped processes enable parsimonious reparameterizations of Markov chains that help to reveal relevant partitions of the state space. Characterizations of the lumped processes on the original transition probability matrix are derived. Different model selection methods relying either on hypothesis testing or on penalized log-likelihood criteria are presented as well as extensions to lumped processes constructed from high-order Markov chains. The relevance of the proposed approach to lumpability is illustrated by the analysis of DNA sequences. In particular, the use of lumped processes enables to highlight differences between intronic sequences and gene untranslated region sequences.

  8. Prediction of Synchrostate Transitions in EEG Signals Using Markov Chain Models

    Jamal, Wasifa; Oprescu, Ioana-Anastasia; Maharatna, Koushik

    2014-01-01

    This paper proposes a stochastic model using the concept of Markov chains for the inter-state transitions of the millisecond order quasi-stable phase synchronized patterns or synchrostates, found in multi-channel Electroencephalogram (EEG) signals. First and second order transition probability matrices are estimated for Markov chain modelling from 100 trials of 128-channel EEG signals during two different face perception tasks. Prediction accuracies with such finite Markov chain models for synchrostate transition are also compared, under a data-partitioning based cross-validation scheme.

  9. Large Deviations for Empirical Measures of Not Necessarily Irreducible Countable Markov Chains with Arbitrary Initial Measures

    Yi Wen JIANG; Li Ming WU

    2005-01-01

    All known results on large deviations of occupation measures of Markov processes are based on the assumption of (essential) irreducibility. In this paper we establish the weak* large deviation principle of occupation measures for any countable Markov chain with arbitrary initial measures. The new rate function that we obtain is not convex and depends on the initial measure, contrary to the (essentially) irreducible case.

  10. Real time Markov chains: Wind states in anemometric data

    Sanchez, P A; Jaramillo, O A

    2015-01-01

    The description of wind phenomena is frequently based on data obtained from anemometers, which usually report the wind speed and direction only in a horizontal plane. Such measurements are commonly used either to develop wind generation farms or to forecast weather conditions in a geographical region. Beyond these standard applications, the information contained in the data may be richer than expected and may lead to a better understanding of the wind dynamics in a geographical area. In this work we propose a statistical analysis based on the wind velocity vectors, which we propose may be grouped in "wind states" associated to binormal distribution functions. We found that the velocity plane defined by the anemometric velocity data may be used as a phase space, where a finite number of states may be found and sorted using standard clustering methods. The main result is a discretization technique useful to model the wind with Markov chains. We applied such ideas in anemometric data for two different sites in M...

  11. Seriation in paleontological data using markov chain Monte Carlo methods.

    Kai Puolamäki

    2006-02-01

    Full Text Available Given a collection of fossil sites with data about the taxa that occur in each site, the task in biochronology is to find good estimates for the ages or ordering of sites. We describe a full probabilistic model for fossil data. The parameters of the model are natural: the ordering of the sites, the origination and extinction times for each taxon, and the probabilities of different types of errors. We show that the posterior distributions of these parameters can be estimated reliably by using Markov chain Monte Carlo techniques. The posterior distributions of the model parameters can be used to answer many different questions about the data, including seriation (finding the best ordering of the sites and outlier detection. We demonstrate the usefulness of the model and estimation method on synthetic data and on real data on large late Cenozoic mammals. As an example, for the sites with large number of occurrences of common genera, our methods give orderings, whose correlation with geochronologic ages is 0.95.

  12. Quantum mixing of Markov chains for special distributions

    Dunjko, V.; Briegel, H. J.

    2015-07-01

    The preparation of the stationary distribution of irreducible, time-reversible Markov chains (MCs) is a fundamental building block in many heuristic approaches to algorithmically hard problems. It has been conjectured that quantum analogs of classical mixing processes may offer a generic quadratic speed-up in realizing such stationary distributions. Such a speed-up would also imply a speed-up of a broad family of heuristic algorithms. However, a true quadratic speed up has thus far only been demonstrated for special classes of MCs. These results often presuppose a regular structure of the underlying graph of the MC, and also a regularity in the transition probabilities. In this work, we demonstrate a true quadratic speed-up for a class of MCs where the restriction is only on the form of the stationary distribution, rather than directly on the MC structure itself. In particular, we show efficient mixing can be achieved when it is known beforehand that the distribution is monotonically decreasing relative to a known order on the state space. Following this, we show that our approach extends to a wider class of distributions, where only a fraction of the shape of the distribution is known to be monotonic. Our approach is built on the Szegedy-type quantization of transition operators.

  13. Markov chain Monte Carlo: an introduction for epidemiologists.

    Hamra, Ghassan; MacLehose, Richard; Richardson, David

    2013-04-01

    Markov Chain Monte Carlo (MCMC) methods are increasingly popular among epidemiologists. The reason for this may in part be that MCMC offers an appealing approach to handling some difficult types of analyses. Additionally, MCMC methods are those most commonly used for Bayesian analysis. However, epidemiologists are still largely unfamiliar with MCMC. They may lack familiarity either with he implementation of MCMC or with interpretation of the resultant output. As with tutorials outlining the calculus behind maximum likelihood in previous decades, a simple description of the machinery of MCMC is needed. We provide an introduction to conducting analyses with MCMC, and show that, given the same data and under certain model specifications, the results of an MCMC simulation match those of methods based on standard maximum-likelihood estimation (MLE). In addition, we highlight examples of instances in which MCMC approaches to data analysis provide a clear advantage over MLE. We hope that this brief tutorial will encourage epidemiologists to consider MCMC approaches as part of their analytic tool-kit.

  14. Searching for efficient Markov chain Monte Carlo proposal kernels.

    Yang, Ziheng; Rodríguez, Carlos E

    2013-11-26

    Markov chain Monte Carlo (MCMC) or the Metropolis-Hastings algorithm is a simulation algorithm that has made modern Bayesian statistical inference possible. Nevertheless, the efficiency of different Metropolis-Hastings proposal kernels has rarely been studied except for the Gaussian proposal. Here we propose a unique class of Bactrian kernels, which avoid proposing values that are very close to the current value, and compare their efficiency with a number of proposals for simulating different target distributions, with efficiency measured by the asymptotic variance of a parameter estimate. The uniform kernel is found to be more efficient than the Gaussian kernel, whereas the Bactrian kernel is even better. When optimal scales are used for both, the Bactrian kernel is at least 50% more efficient than the Gaussian. Implementation in a Bayesian program for molecular clock dating confirms the general applicability of our results to generic MCMC algorithms. Our results refute a previous claim that all proposals had nearly identical performance and will prompt further research into efficient MCMC proposals.

  15. Compound extremes in a changing climate - a Markov chain approach

    Sedlmeier, Katrin; Mieruch, Sebastian; Schädler, Gerd; Kottmeier, Christoph

    2016-11-01

    Studies using climate models and observed trends indicate that extreme weather has changed and may continue to change in the future. The potential impact of extreme events such as heat waves or droughts depends not only on their number of occurrences but also on "how these extremes occur", i.e., the interplay and succession of the events. These quantities are quite unexplored, for past changes as well as for future changes and call for sophisticated methods of analysis. To address this issue, we use Markov chains for the analysis of the dynamics and succession of multivariate or compound extreme events. We apply the method to observational data (1951-2010) and an ensemble of regional climate simulations for central Europe (1971-2000, 2021-2050) for two types of compound extremes, heavy precipitation and cold in winter and hot and dry days in summer. We identify three regions in Europe, which turned out to be likely susceptible to a future change in the succession of heavy precipitation and cold in winter, including a region in southwestern France, northern Germany and in Russia around Moscow. A change in the succession of hot and dry days in summer can be expected for regions in Spain and Bulgaria. The susceptibility to a dynamic change of hot and dry extremes in the Russian region will probably decrease.

  16. Finding and testing network communities by lumped Markov chains.

    Piccardi, Carlo

    2011-01-01

    Identifying communities (or clusters), namely groups of nodes with comparatively strong internal connectivity, is a fundamental task for deeply understanding the structure and function of a network. Yet, there is a lack of formal criteria for defining communities and for testing their significance. We propose a sharp definition that is based on a quality threshold. By means of a lumped Markov chain model of a random walker, a quality measure called "persistence probability" is associated to a cluster, which is then defined as an "α-community" if such a probability is not smaller than α. Consistently, a partition composed of α-communities is an "α-partition." These definitions turn out to be very effective for finding and testing communities. If a set of candidate partitions is available, setting the desired α-level allows one to immediately select the α-partition with the finest decomposition. Simultaneously, the persistence probabilities quantify the quality of each single community. Given its ability in individually assessing each single cluster, this approach can also disclose single well-defined communities even in networks that overall do not possess a definite clusterized structure.

  17. Threshold partitioning of sparse matrices and applications to Markov chains

    Choi, Hwajeong; Szyld, D.B. [Temple Univ., Philadelphia, PA (United States)

    1996-12-31

    It is well known that the order of the variables and equations of a large, sparse linear system influences the performance of classical iterative methods. In particular if, after a symmetric permutation, the blocks in the diagonal have more nonzeros, classical block methods have a faster asymptotic rate of convergence. In this paper, different ordering and partitioning algorithms for sparse matrices are presented. They are modifications of PABLO. In the new algorithms, in addition to the location of the nonzeros, the values of the entries are taken into account. The matrix resulting after the symmetric permutation has dense blocks along the diagonal, and small entries in the off-diagonal blocks. Parameters can be easily adjusted to obtain, for example, denser blocks, or blocks with elements of larger magnitude. In particular, when the matrices represent Markov chains, the permuted matrices are well suited for block iterative methods that find the corresponding probability distribution. Applications to three types of methods are explored: (1) Classical block methods, such as Block Gauss Seidel. (2) Preconditioned GMRES, where a block diagonal preconditioner is used. (3) Iterative aggregation method (also called aggregation/disaggregation) where the partition obtained from the ordering algorithm with certain parameters is used as an aggregation scheme. In all three cases, experiments are presented which illustrate the performance of the methods with the new orderings. The complexity of the new algorithms is linear in the number of nonzeros and the order of the matrix, and thus adding little computational effort to the overall solution.

  18. Asteroid mass estimation using Markov-Chain Monte Carlo techniques

    Siltala, Lauri; Granvik, Mikael

    2016-10-01

    Estimates for asteroid masses are based on their gravitational perturbations on the orbits of other objects such as Mars, spacecraft, or other asteroids and/or their satellites. In the case of asteroid-asteroid perturbations, this leads to a 13-dimensional inverse problem where the aim is to derive the mass of the perturbing asteroid and six orbital elements for both the perturbing asteroid and the test asteroid using astrometric observations. We have developed and implemented three different mass estimation algorithms utilizing asteroid-asteroid perturbations into the OpenOrb asteroid-orbit-computation software: the very rough 'marching' approximation, in which the asteroid orbits are fixed at a given epoch, reducing the problem to a one-dimensional estimation of the mass, an implementation of the Nelder-Mead simplex method, and most significantly, a Markov-Chain Monte Carlo (MCMC) approach. We will introduce each of these algorithms with particular focus on the MCMC algorithm, and present example results for both synthetic and real data. Our results agree with the published mass estimates, but suggest that the published uncertainties may be misleading as a consequence of using linearized mass-estimation methods. Finally, we discuss remaining challenges with the algorithms as well as future plans, particularly in connection with ESA's Gaia mission.

  19. An efficient interpolation technique for jump proposals in reversible-jump Markov chain Monte Carlo calculations.

    Farr, W M; Mandel, I; Stevens, D

    2015-06-01

    Selection among alternative theoretical models given an observed dataset is an important challenge in many areas of physics and astronomy. Reversible-jump Markov chain Monte Carlo (RJMCMC) is an extremely powerful technique for performing Bayesian model selection, but it suffers from a fundamental difficulty and it requires jumps between model parameter spaces, but cannot efficiently explore both parameter spaces at once. Thus, a naive jump between parameter spaces is unlikely to be accepted in the Markov chain Monte Carlo (MCMC) algorithm and convergence is correspondingly slow. Here, we demonstrate an interpolation technique that uses samples from single-model MCMCs to propose intermodel jumps from an approximation to the single-model posterior of the target parameter space. The interpolation technique, based on a kD-tree data structure, is adaptive and efficient in modest dimensionality. We show that our technique leads to improved convergence over naive jumps in an RJMCMC, and compare it to other proposals in the literature to improve the convergence of RJMCMCs. We also demonstrate the use of the same interpolation technique as a way to construct efficient 'global' proposal distributions for single-model MCMCs without prior knowledge of the structure of the posterior distribution, and discuss improvements that permit the method to be used in higher dimensional spaces efficiently.

  20. Simplification of irreversible Markov chains by removal of states with fast leaving rates.

    Jia, Chen

    2016-07-07

    In the recent work of Ullah et al. (2012a), the authors developed an effective method to simplify reversible Markov chains by removal of states with low equilibrium occupancies. In this paper, we extend this result to irreversible Markov chains. We show that an irreversible chain can be simplified by removal of states with fast leaving rates. Moreover, we reveal that the irreversibility of the chain will always decrease after model simplification. This suggests that although model simplification can retain almost all the dynamic information of the chain, it will lose some thermodynamic information as a trade-off. Examples from biology are also given to illustrate the main results of this paper.

  1. Asymptotic Expansions of Backward Equations for Two-time-scale Markov Chains in Continuous Time

    G Yin; Dung Tien Nguyen

    2009-01-01

    This work develops asymptotic expansions for solutions of systems of backward equations of timeinhomogeneons Markov chains in continuous time. Owing to the rapid progress in technology and the increasing complexity in modeling, the underlying Markov chains often have large state spaces, which make the computational tasks infeasible. To reduce the complexity, two-time-scale formulations are used. By introducing a small parameter ε> 0 and using suitable decomposition and aggregation procedures, it is formulated as a singular perturbation problem. Both Markov chains having recurrent states only and Markov chains including also transient states are treated. Under certain weak irreducibility and smoothness conditions of the generators, the desired asymptotic expansions are constructed. Then error bounds are obtained.

  2. Maps of sparse Markov chains efficiently reveal community structure in network flows with memory

    Persson, Christian; Edler, Daniel; Rosvall, Martin

    2016-01-01

    To better understand the flows of ideas or information through social and biological systems, researchers develop maps that reveal important patterns in network flows. In practice, network flow models have implied memoryless first-order Markov chains, but recently researchers have introduced higher-order Markov chain models with memory to capture patterns in multi-step pathways. Higher-order models are particularly important for effectively revealing actual, overlapping community structure, but higher-order Markov chain models suffer from the curse of dimensionality: their vast parameter spaces require exponentially increasing data to avoid overfitting and therefore make mapping inefficient already for moderate-sized systems. To overcome this problem, we introduce an efficient cross-validated mapping approach based on network flows modeled by sparse Markov chains. To illustrate our approach, we present a map of citation flows in science with research fields that overlap in multidisciplinary journals. Compared...

  3. A comparison of strategies for Markov chain Monte Carlo computation in quantitative genetics

    Waagepetersen, Rasmus; Ibánez-Escriche, Noelia; Sorensen, Daniel

    2008-01-01

    In quantitative genetics, Markov chain Monte Carlo (MCMC) methods are indispensable for statistical inference in non-standard models like generalized linear models with genetic random effects or models with genetically structured variance heterogeneity. A particular challenge for MCMC applications...

  4. Technical manual for basic version of the Markov chain nest productivity model (MCnest)

    The Markov Chain Nest Productivity Model (or MCnest) integrates existing toxicity information from three standardized avian toxicity tests with information on species life history and the timing of pesticide applications relative to the timing of avian breeding seasons to quantit...

  5. User’s manual for basic version of MCnest Markov chain nest productivity model

    The Markov Chain Nest Productivity Model (or MCnest) integrates existing toxicity information from three standardized avian toxicity tests with information on species life history and the timing of pesticide applications relative to the timing of avian breeding seasons to quantit...

  6. Generation of intervention strategy for a genetic regulatory network represented by a family of Markov Chains.

    Berlow, Noah; Pal, Ranadip

    2011-01-01

    Genetic Regulatory Networks (GRNs) are frequently modeled as Markov Chains providing the transition probabilities of moving from one state of the network to another. The inverse problem of inference of the Markov Chain from noisy and limited experimental data is an ill posed problem and often generates multiple model possibilities instead of a unique one. In this article, we address the issue of intervention in a genetic regulatory network represented by a family of Markov Chains. The purpose of intervention is to alter the steady state probability distribution of the GRN as the steady states are considered to be representative of the phenotypes. We consider robust stationary control policies with best expected behavior. The extreme computational complexity involved in search of robust stationary control policies is mitigated by using a sequential approach to control policy generation and utilizing computationally efficient techniques for updating the stationary probability distribution of a Markov chain following a rank one perturbation.

  7. Applying Markov Chains for NDVI Time Series Forecasting of Latvian Regions

    Stepchenko Arthur

    2015-12-01

    Full Text Available Time series of earth observation based estimates of vegetation inform about variations in vegetation at the scale of Latvia. A vegetation index is an indicator that describes the amount of chlorophyll (the green mass and shows the relative density and health of vegetation. NDVI index is an important variable for vegetation forecasting and management of various problems, such as climate change monitoring, energy usage monitoring, managing the consumption of natural resources, agricultural productivity monitoring, drought monitoring and forest fire detection. In this paper, we make a one-step-ahead prediction of 7-daily time series of NDVI index using Markov chains. The choice of a Markov chain is due to the fact that a Markov chain is a sequence of random variables where each variable is located in some state. And a Markov chain contains probabilities of moving from one state to other.

  8. A Simple Discrete Model of Brownian Motors: Time-periodic Markov Chains

    Ge, Hao; Jiang, Da-Quan; Qian, Min

    2006-05-01

    In this paper, we consider periodically inhomogeneous Markov chains, which can be regarded as a simple version of physical model—Brownian motors. We introduce for them the concepts of periodical reversibility, detailed balance, entropy production rate and circulation distribution. We prove the equivalence of the following statements: The time-periodic Markov chain is periodically reversible; It is in detailed balance; Kolmogorov's cycle condition is satisfied; Its entropy production rate vanishes; Every circuit and its reversed circuit have the same circulation weight. Hence, in our model of Markov chains, the directed transport phenomenon of Brownian motors, i.e. the existence of net circulation, can occur only in nonequilibrium and irreversible systems. Moreover, we verify the large deviation property and the Gallavotti-Cohen fluctuation theorem of sample entropy production rates of the Markov chain.

  9. Marked Continuous-Time Markov Chain Modelling of Burst Behaviour for Single Ion Channels

    Frank G. Ball

    2007-01-01

    a continuous-time Markov chain with a finite-state space. We show how the use of marked continuous-time Markov chains can simplify the derivation of (i the distributions of several burst properties, including the total open time, the total charge transfer, and the number of openings in a burst, and (ii the form of these distributions when the underlying gating process is time reversible and in equilibrium.

  10. THE TRANSITION PROBABILITY MATRIX OF A MARKOV CHAIN MODEL IN AN ATM NETWORK

    YUE Dequan; ZHANG Huachen; TU Fengsheng

    2003-01-01

    In this paper we consider a Markov chain model in an ATM network, which has been studied by Dag and Stavrakakis. On the basis of the iterative formulas obtained by Dag and Stavrakakis, we obtain the explicit analytical expression of the transition probability matrix. It is very simple to calculate the transition probabilities of the Markov chain by these expressions. In addition, we obtain some results about the structure of the transition probability matrix, which are helpful in numerical calculation and theoretical analysis.

  11. Markov Chain Computation for Homogeneous and Non-homogeneous Data: MARCH 1.1 Users Guide

    Andre Berchtold

    2001-03-01

    Full Text Available MARCH is a free software for the computation of different types of Markovian models including homogeneous Markov Chains, Hidden Markov Models (HMMs and Double Chain Markov Models (DCMMs. The main characteristic of this software is the implementation of a powerful optimization method for HMMs and DCMMs combining a genetic algorithm with the standard Baum-Welch procedure. MARCH is distributed as a set of Matlab functions running under Matlab 5 or higher on any computing platform. A PC Windows version running independently from Matlab is also available.

  12. Dynamic temperature selection for parallel tempering in Markov chain Monte Carlo simulations

    Vousden, W. D.; Farr, W. M.; Mandel, I.

    2016-01-01

    Modern problems in astronomical Bayesian inference require efficient methods for sampling from complex, high-dimensional, often multimodal probability distributions. Most popular methods, such as MCMC sampling, perform poorly on strongly multimodal probability distributions, rarely jumping between modes or settling on just one mode without finding others. Parallel tempering addresses this problem by sampling simultaneously with separate Markov chains from tempered versions of the target distribution with reduced contrast levels. Gaps between modes can be traversed at higher temperatures, while individual modes can be efficiently explored at lower temperatures. In this paper, we investigate how one might choose the ladder of temperatures to achieve more efficient sampling, as measured by the autocorrelation time of the sampler. In particular, we present a simple, easily implemented algorithm for dynamically adapting the temperature configuration of a sampler while sampling. This algorithm dynamically adjusts the temperature spacing to achieve a uniform rate of exchanges between chains at neighbouring temperatures. We compare the algorithm to conventional geometric temperature configurations on a number of test distributions and on an astrophysical inference problem, reporting efficiency gains by a factor of 1.2-2.5 over a well-chosen geometric temperature configuration and by a factor of 1.5-5 over a poorly chosen configuration. On all of these problems, a sampler using the dynamical adaptations to achieve uniform acceptance ratios between neighbouring chains outperforms one that does not.

  13. Modeling and Computing of Stock Index Forecasting Based on Neural Network and Markov Chain

    Yonghui Dai

    2014-01-01

    Full Text Available The stock index reflects the fluctuation of the stock market. For a long time, there have been a lot of researches on the forecast of stock index. However, the traditional method is limited to achieving an ideal precision in the dynamic market due to the influences of many factors such as the economic situation, policy changes, and emergency events. Therefore, the approach based on adaptive modeling and conditional probability transfer causes the new attention of researchers. This paper presents a new forecast method by the combination of improved back-propagation (BP neural network and Markov chain, as well as its modeling and computing technology. This method includes initial forecasting by improved BP neural network, division of Markov state region, computing of the state transition probability matrix, and the prediction adjustment. Results of the empirical study show that this method can achieve high accuracy in the stock index prediction, and it could provide a good reference for the investment in stock market.

  14. Saccade selection when reward probability is dynamically manipulated using Markov chains.

    Nummela, Samuel U; Lovejoy, Lee P; Krauzlis, Richard J

    2008-05-01

    Markov chains (stochastic processes where probabilities are assigned based on the previous outcome) are commonly used to examine the transitions between behavioral states, such as those that occur during foraging or social interactions. However, relatively little is known about how well primates can incorporate knowledge about Markov chains into their behavior. Saccadic eye movements are an example of a simple behavior influenced by information about probability, and thus are good candidates for testing whether subjects can learn Markov chains. In addition, when investigating the influence of probability on saccade target selection, the use of Markov chains could provide an alternative method that avoids confounds present in other task designs. To investigate these possibilities, we evaluated human behavior on a task in which stimulus reward probabilities were assigned using a Markov chain. On each trial, the subject selected one of four identical stimuli by saccade; after selection, feedback indicated the rewarded stimulus. Each session consisted of 200-600 trials, and on some sessions, the reward magnitude varied. On sessions with a uniform reward, subjects (n = 6) learned to select stimuli at a frequency close to reward probability, which is similar to human behavior on matching or probability classification tasks. When informed that a Markov chain assigned reward probabilities, subjects (n = 3) learned to select the greatest reward probability more often, bringing them close to behavior that maximizes reward. On sessions where reward magnitude varied across stimuli, subjects (n = 6) demonstrated preferences for both greater reward probability and greater reward magnitude, resulting in a preference for greater expected value (the product of reward probability and magnitude). These results demonstrate that Markov chains can be used to dynamically assign probabilities that are rapidly exploited by human subjects during saccade target selection.

  15. Statistical Analysis of Notational AFL Data Using Continuous Time Markov Chains.

    Meyer, Denny; Forbes, Don; Clarke, Stephen R

    2006-01-01

    Animal biologists commonly use continuous time Markov chain models to describe patterns of animal behaviour. In this paper we consider the use of these models for describing AFL football. In particular we test the assumptions for continuous time Markov chain models (CTMCs), with time, distance and speed values associated with each transition. Using a simple event categorisation it is found that a semi-Markov chain model is appropriate for this data. This validates the use of Markov Chains for future studies in which the outcomes of AFL matches are simulated. Key PointsA comparison of four AFL matches suggests similarity in terms of transition probabilities for events and the mean times, distances and speeds associated with each transition.The Markov assumption appears to be valid.However, the speed, time and distance distributions associated with each transition are not exponential suggesting that semi-Markov model can be used to model and simulate play.Team identified events and directions associated with transitions are required to develop the model into a tool for the prediction of match outcomes.

  16. Strong Stationary Duality for M\\"obius Monotone Markov Chains: Unreliable Networks

    Lorek, Pawel

    2011-01-01

    For Markov chains with a partially ordered finite state space we show strong stationary duality under the condition of M\\"obius monotonicity of the chain. We show relations of M\\"obius monotonicity to other definitions of monotone chains. We give examples of dual chains in this context which have transitions only upwards. We illustrate general theory by an analysis of nonsymmetric random walks on the cube with an application to networks of queues.

  17. Descriptive and predictive evaluation of high resolution Markov chain precipitation models

    Sørup, Hjalte Jomo Danielsen; Madsen, Henrik; Arnbjerg-Nielsen, Karsten

    2012-01-01

    . Continuous modelling of the Markov process proved attractive because of a marked decrease in the number of parameters. Inclusion of seasonality into the continuous Markov chain model proved difficult. Monte Carlo simulations with the models show that it is very difficult for all the model formulations...... to reproduce the time series on event level. Extreme events with short (10 min), medium (60 min) and long (12 h) durations were investigated because of their importance in urban hydrology. Both the descriptive likelihood based statistics and the predictive Monte Carlo simulation based statistics are valuable......A time series of tipping bucket recordings of very high temporal and volumetric resolution precipitation is modelled using Markov chain models. Both first and second‐order Markov models as well as seasonal and diurnal models are investigated and evaluated using likelihood based techniques...

  18. Markov chain order estimation with parametric significance tests of conditional mutual information

    Papapetrou, Maria

    2015-01-01

    Besides the different approaches suggested in the literature, accurate estimation of the order of a Markov chain from a given symbol sequence is an open issue, especially when the order is moderately large. Here, parametric significance tests of conditional mutual information (CMI) of increasing order $m$, $I_c(m)$, on a symbol sequence are conducted for increasing orders $m$ in order to estimate the true order $L$ of the underlying Markov chain. CMI of order $m$ is the mutual information of two variables in the Markov chain being $m$ time steps apart, conditioning on the intermediate variables of the chain. The null distribution of CMI is approximated with a normal and gamma distribution deriving analytic expressions of their parameters, and a gamma distribution deriving its parameters from the mean and variance of the normal distribution. The accuracy of order estimation is assessed with the three parametric tests, and the parametric tests are compared to the randomization significance test and other known ...

  19. A sufficient condition for the subexponential asymptotics of GI/G/1-type Markov chains with queueing applications

    2013-01-01

    The main contribution of this paper is to present a new sufficient condition for the subexponential asymptotics of the stationary distribution of a GI/GI/1-type Markov chain without jumps from level "infinity" to level zero. For simplicity, we call such Markov chains {\\it GI/GI/1-type Markov chains without disasters} because they are often used to analyze semi-Markovian queues without "disasters", which are negative customers who remove all the customers in the system (including themselves) o...

  20. STRONG LAW OF LARGE NUMBERS AND ASYMPTOTIC EQUIPARTITION PROPERTY FOR NONSYMMETRIC MARKOV CHAIN FIELDS ON CAYLEY TREES

    Bao Zhenhua; Ye Zhongxing

    2007-01-01

    Some strong laws of large numbers for the frequencies of occurrence of states and ordered couples of states for nonsymmetric Markov chain fields (NSMC) on Cayley trees are studied. In the proof, a new technique for the study of strong limit theorems of Markov chains is extended to the case of Markov chain fields. The asymptotic equipartition properties with almost everywhere (a.e.) convergence for NSMC on Cayley trees are obtained.

  1. Strong Law of Large Numbers for Countable Markov Chains Indexed by an Infinite Tree with Uniformly Bounded Degree

    Bao Wang

    2014-01-01

    Full Text Available We study the strong law of large numbers for the frequencies of occurrence of states and ordered couples of states for countable Markov chains indexed by an infinite tree with uniformly bounded degree, which extends the corresponding results of countable Markov chains indexed by a Cayley tree and generalizes the relative results of finite Markov chains indexed by a uniformly bounded tree.

  2. Enhancement of Markov chain model by integrating exponential smoothing: A case study on Muslims marriage and divorce

    Jamaluddin, Fadhilah; Rahim, Rahela Abdul

    2015-12-01

    Markov Chain has been introduced since the 1913 for the purpose of studying the flow of data for a consecutive number of years of the data and also forecasting. The important feature in Markov Chain is obtaining the accurate Transition Probability Matrix (TPM). However to obtain the suitable TPM is hard especially in involving long-term modeling due to unavailability of data. This paper aims to enhance the classical Markov Chain by introducing Exponential Smoothing technique in developing the appropriate TPM.

  3. Reduction techniques for discrete-time Markov chains on totally ordered state space using stochastic comparisons

    2000-01-01

    We propose in this paper two methods to compute Markovian bounds for monotone functions of a discrete time homogeneous Markov chain evolving in a totally ordered state space. The main interest of such methods is to propose algorithms to simplify analysis of transient characteristics such as the output process of a queue, or sojourn time in a subset of states. Construction of bounds are based on two kinds of results: well-known results on stochastic comparison between Markov cha...

  4. Predictive glycoengineering of biosimilars using a Markov chain glycosylation model

    Spahn, Philipp N.; Hansen, Anders Holmgaard; Kol, Stefan;

    2016-01-01

    biogenesis. This usually implies that costly and time-consuming experimentation is required for clone identification and optimization of biosimilar glycosylation. Here, we describe a computational method that utilizes a Markov model of glycosylation to predict optimal glycoengineering strategies to obtain...

  5. Reliability analysis and prediction of mixed mode load using Markov Chain Model

    Nikabdullah, N. [Department of Mechanical and Materials Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia and Institute of Space Science (ANGKASA), Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia (Malaysia); Singh, S. S. K.; Alebrahim, R.; Azizi, M. A. [Department of Mechanical and Materials Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia (Malaysia); K, Elwaleed A. [Institute of Space Science (ANGKASA), Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia (Malaysia); Noorani, M. S. M. [School of Mathematical Sciences, Faculty of Science and Technology, Universiti Kebangsaan Malaysia (Malaysia)

    2014-06-19

    The aim of this paper is to present the reliability analysis and prediction of mixed mode loading by using a simple two state Markov Chain Model for an automotive crankshaft. The reliability analysis and prediction for any automotive component or structure is important for analyzing and measuring the failure to increase the design life, eliminate or reduce the likelihood of failures and safety risk. The mechanical failures of the crankshaft are due of high bending and torsion stress concentration from high cycle and low rotating bending and torsional stress. The Markov Chain was used to model the two states based on the probability of failure due to bending and torsion stress. In most investigations it revealed that bending stress is much serve than torsional stress, therefore the probability criteria for the bending state would be higher compared to the torsion state. A statistical comparison between the developed Markov Chain Model and field data was done to observe the percentage of error. The reliability analysis and prediction was derived and illustrated from the Markov Chain Model were shown in the Weibull probability and cumulative distribution function, hazard rate and reliability curve and the bathtub curve. It can be concluded that Markov Chain Model has the ability to generate near similar data with minimal percentage of error and for a practical application; the proposed model provides a good accuracy in determining the reliability for the crankshaft under mixed mode loading.

  6. Markov Chain-Like Quantum Biological Modeling of Mutations, Aging, and Evolution

    Ivan B. Djordjevic

    2015-08-01

    Full Text Available Recent evidence suggests that quantum mechanics is relevant in photosynthesis, magnetoreception, enzymatic catalytic reactions, olfactory reception, photoreception, genetics, electron-transfer in proteins, and evolution; to mention few. In our recent paper published in Life, we have derived the operator-sum representation of a biological channel based on codon basekets, and determined the quantum channel model suitable for study of the quantum biological channel capacity. However, this model is essentially memoryless and it is not able to properly model the propagation of mutation errors in time, the process of aging, and evolution of genetic information through generations. To solve for these problems, we propose novel quantum mechanical models to accurately describe the process of creation spontaneous, induced, and adaptive mutations and their propagation in time. Different biological channel models with memory, proposed in this paper, include: (i Markovian classical model, (ii Markovian-like quantum model, and (iii hybrid quantum-classical model. We then apply these models in a study of aging and evolution of quantum biological channel capacity through generations. We also discuss key differences of these models with respect to a multilevel symmetric channel-based Markovian model and a Kimura model-based Markovian process. These models are quite general and applicable to many open problems in biology, not only biological channel capacity, which is the main focus of the paper. We will show that the famous quantum Master equation approach, commonly used to describe different biological processes, is just the first-order approximation of the proposed quantum Markov chain-like model, when the observation interval tends to zero. One of the important implications of this model is that the aging phenotype becomes determined by different underlying transition probabilities in both programmed and random (damage Markov chain-like models of aging, which

  7. Markov Chain-Like Quantum Biological Modeling of Mutations, Aging, and Evolution.

    Djordjevic, Ivan B

    2015-08-24

    Recent evidence suggests that quantum mechanics is relevant in photosynthesis, magnetoreception, enzymatic catalytic reactions, olfactory reception, photoreception, genetics, electron-transfer in proteins, and evolution; to mention few. In our recent paper published in Life, we have derived the operator-sum representation of a biological channel based on codon basekets, and determined the quantum channel model suitable for study of the quantum biological channel capacity. However, this model is essentially memoryless and it is not able to properly model the propagation of mutation errors in time, the process of aging, and evolution of genetic information through generations. To solve for these problems, we propose novel quantum mechanical models to accurately describe the process of creation spontaneous, induced, and adaptive mutations and their propagation in time. Different biological channel models with memory, proposed in this paper, include: (i) Markovian classical model, (ii) Markovian-like quantum model, and (iii) hybrid quantum-classical model. We then apply these models in a study of aging and evolution of quantum biological channel capacity through generations. We also discuss key differences of these models with respect to a multilevel symmetric channel-based Markovian model and a Kimura model-based Markovian process. These models are quite general and applicable to many open problems in biology, not only biological channel capacity, which is the main focus of the paper. We will show that the famous quantum Master equation approach, commonly used to describe different biological processes, is just the first-order approximation of the proposed quantum Markov chain-like model, when the observation interval tends to zero. One of the important implications of this model is that the aging phenotype becomes determined by different underlying transition probabilities in both programmed and random (damage) Markov chain-like models of aging, which are mutually

  8. Marathon: An Open Source Software Library for the Analysis of Markov-Chain Monte Carlo Algorithms.

    Rechner, Steffen; Berger, Annabell

    2016-01-01

    We present the software library marathon, which is designed to support the analysis of sampling algorithms that are based on the Markov-Chain Monte Carlo principle. The main application of this library is the computation of properties of so-called state graphs, which represent the structure of Markov chains. We demonstrate applications and the usefulness of marathon by investigating the quality of several bounding methods on four well-known Markov chains for sampling perfect matchings and bipartite graphs. In a set of experiments, we compute the total mixing time and several of its bounds for a large number of input instances. We find that the upper bound gained by the famous canonical path method is often several magnitudes larger than the total mixing time and deteriorates with growing input size. In contrast, the spectral bound is found to be a precise approximation of the total mixing time.

  9. Counting of oligomers in sequences generated by markov chains for DNA motif discovery.

    Shan, Gao; Zheng, Wei-Mou

    2009-02-01

    By means of the technique of the imbedded Markov chain, an efficient algorithm is proposed to exactly calculate first, second moments of word counts and the probability for a word to occur at least once in random texts generated by a Markov chain. A generating function is introduced directly from the imbedded Markov chain to derive asymptotic approximations for the problem. Two Z-scores, one based on the number of sequences with hits and the other on the total number of word hits in a set of sequences, are examined for discovery of motifs on a set of promoter sequences extracted from A. thaliana genome. Source code is available at http://www.itp.ac.cn/zheng/oligo.c.

  10. Weighted Markov chains for forecasting and analysis in Incidence of infectious diseases in jiangsu Province, China.

    Peng, Zhihang; Bao, Changjun; Zhao, Yang; Yi, Honggang; Xia, Letian; Yu, Hao; Shen, Hongbing; Chen, Feng

    2010-05-01

    This paper first applies the sequential cluster method to set up the classification standard of infectious disease incidence state based on the fact that there are many uncertainty characteristics in the incidence course. Then the paper presents a weighted Markov chain, a method which is used to predict the future incidence state. This method assumes the standardized self-coefficients as weights based on the special characteristics of infectious disease incidence being a dependent stochastic variable. It also analyzes the characteristics of infectious diseases incidence via the Markov chain Monte Carlo method to make the long-term benefit of decision optimal. Our method is successfully validated using existing incidents data of infectious diseases in Jiangsu Province. In summation, this paper proposes ways to improve the accuracy of the weighted Markov chain, specifically in the field of infection epidemiology.

  11. Bayesian analysis of non-homogeneous Markov chains: application to mental health data.

    Sung, Minje; Soyer, Refik; Nhan, Nguyen

    2007-07-10

    In this paper we present a formal treatment of non-homogeneous Markov chains by introducing a hierarchical Bayesian framework. Our work is motivated by the analysis of correlated categorical data which arise in assessment of psychiatric treatment programs. In our development, we introduce a Markovian structure to describe the non-homogeneity of transition patterns. In doing so, we introduce a logistic regression set-up for Markov chains and incorporate covariates in our model. We present a Bayesian model using Markov chain Monte Carlo methods and develop inference procedures to address issues encountered in the analyses of data from psychiatric treatment programs. Our model and inference procedures are implemented to some real data from a psychiatric treatment study.

  12. Formal Reasoning About Finite-State Discrete-Time Markov Chains in HOL

    Liya Liu; Osman Hasan; Sofiène Tahar

    2013-01-01

    Markov chains are extensively used in modeling different aspects of engineering and scientific systems,such as performance of algorithms and reliability of systems.Different techniques have been developed for analyzing Markovian models,for example,Markov Chain Monte Carlo based simulation,Markov Analyzer,and more recently probabilistic modelchecking.However,these techniques either do not guarantee accurate analysis or are not scalable.Higher-order-logic theorem proving is a formal method that has the ability to overcome the above mentioned limitations.However,it is not mature enough to handle all sorts of Markovian models.In this paper,we propose a formalization of Discrete-Time Markov Chain (DTMC) that facilitates formal reasoning about time-homogeneous finite-state discrete-time Markov chain.In particular,we provide a formal verification on some of its important properties,such as joint probabilities,Chapman-Kolmogorov equation,reversibility property,using higher-order logic.To demonstrate the usefulness of our work,we analyze two applications:a simplified binary communication channel and the Automatic Mail Quality Measurement protocol.

  13. Markov

    Carlos Alejandro De Luna Ortega

    2006-01-01

    Full Text Available En este artículo se aborda el diseño de un reconocedor de voz, con el idioma español mexicano, del estado de Aguascalientes, de palabras aisladas, con dependencia del hablante y vocabulario pequeño, empleando Redes Neuronales Artificiales (ANN por sus siglas en inglés, Alineamiento Dinámico del Tiempo (DTW por sus siglas en inglés y Modelos Ocultos de Markov (HMM por sus siglas en inglés para la realización del algoritmo de reconocimiento.

  14. Strong, Weak and Branching Bisimulation for Transition Systems and Markov Reward Chains: A Unifying Matrix Approach

    Nikola Trčka

    2009-12-01

    Full Text Available We first study labeled transition systems with explicit successful termination. We establish the notions of strong, weak, and branching bisimulation in terms of boolean matrix theory, introducing thus a novel and powerful algebraic apparatus. Next we consider Markov reward chains which are standardly presented in real matrix theory. By interpreting the obtained matrix conditions for bisimulations in this setting, we automatically obtain the definitions of strong, weak, and branching bisimulation for Markov reward chains. The obtained strong and weak bisimulations are shown to coincide with some existing notions, while the obtained branching bisimulation is new, but its usefulness is questionable.

  15. THE CONSTRUCTION OF MULTITYPE CANONICAL MARKOV BRANCHING CHAINS IN RANDOM ENVIRONMENTS

    2006-01-01

    The investigation for branching processes has a long history by their strong physics background, but only a few authors have investigated the branching processes in random environments. First of all, the author introduces the concepts of the multitype canonical Markov branching chain in random environment (CMBCRE) and multitype Markov branching chain in random environment (MBCRE) and proved that CMBCRE must be MBCRE, and any MBCRE must be equivalent to another CMBCRE in distribution. The main results of this article are the construction of CMBCRE and some of its probability properties.

  16. Strong, Weak and Branching Bisimulation for Transition Systems and Markov Reward Chains: A Unifying Matrix Approach

    Trčka, Nikola

    2009-01-01

    We first study labeled transition systems with explicit successful termination. We establish the notions of strong, weak, and branching bisimulation in terms of boolean matrix theory, introducing thus a novel and powerful algebraic apparatus. Next we consider Markov reward chains which are standardly presented in real matrix theory. By interpreting the obtained matrix conditions for bisimulations in this setting, we automatically obtain the definitions of strong, weak, and branching bisimulation for Markov reward chains. The obtained strong and weak bisimulations are shown to coincide with some existing notions, while the obtained branching bisimulation is new, but its usefulness is questionable.

  17. Integration within the Felsenstein equation for improved Markov chain Monte Carlo methods in population genetics

    Hey, Jody; Nielsen, Rasmus

    2007-01-01

    Carlo methods, have been developed to find approximate solutions. Here, we describe an approach in which Markov chain Monte Carlo simulations are used to integrate over the space of genealogies, whereas other parameters are integrated out analytically. The result is an approximation to the full joint......In 1988, Felsenstein described a framework for assessing the likelihood of a genetic data set in which all of the possible genealogical histories of the data are considered, each in proportion to their probability. Although not analytically solvable, several approaches, including Markov chain Monte...

  18. 2nd International Workshop on the Numerical Solution of Markov Chains

    1995-01-01

    Computations with Markov Chains presents the edited and reviewed proceedings of the Second International Workshop on the Numerical Solution of Markov Chains, held January 16--18, 1995, in Raleigh, North Carolina. New developments of particular interest include recent work on stability and conditioning, Krylov subspace-based methods for transient solutions, quadratic convergent procedures for matrix geometric problems, further analysis of the GTH algorithm, the arrival of stochastic automata networks at the forefront of modelling stratagems, and more. An authoritative overview of the field for applied probabilists, numerical analysts and systems modelers, including computer scientists and engineers.

  19. Improvement of Markov Chain Model for Occurrence Degree Prediction of Myzus persicae (Sulzer) and Its Application

    REN Guangwei; WANG Xiufang; WANG Xinwei; ZHOU Xiansheng; DONG Xiaowei

    2008-01-01

    For long-term prediction of occurrence degree of tobacco aphid Myzus persicae (Sulzer), Markov chain method was used to establish prediction model for occurrence degree of tobacco aphid. With 4 levels of occurrence degree, Markov chain model was established based on the data in 1987-2004. The results indicated that the accuracy for total prediction in 2005-2007 and the back prediction in 1987-2004 reached 88.89% and 85.12%, respectively. The method is simple and feasible for long-term prediction of occurrence degree of tobacco aphid.

  20. A Cost-Effective Smoothed Multigrid with Modified Neighborhood-Based Aggregation for Markov Chains

    Zhao-Li Shen

    2015-01-01

    Full Text Available Smoothed aggregation multigrid method is considered for computing stationary distributions of Markov chains. A judgement which determines whether to implement the whole aggregation procedure is proposed. Through this strategy, a large amount of time in the aggregation procedure is saved without affecting the convergence behavior. Besides this, we explain the shortage and irrationality of the Neighborhood-Based aggregation which is commonly used in multigrid methods. Then a modified version is presented to remedy and improve it. Numerical experiments on some typical Markov chain problems are reported to illustrate the performance of these methods.

  1. First and second order semi-Markov chains for wind speed modeling

    D'Amico, Guglielmo; Prattico, Flavio

    2012-01-01

    The increasing interest in renewable energy, particularly in wind, has given rise to the necessity of accurate models for the generation of good synthetic wind speed data. Markov chains are often used with this purpose but better models are needed to reproduce the statistical properties of wind speed data. We downloaded a database, freely available from the web, in which are included wind speed data taken from L.S.I. -Lastem station (Italy) and sampled every 10 minutes. With the aim of reproducing the statistical properties of this data we propose the use of three semi-Markov models. We generate synthetic time series for wind speed by means of Monte Carlo simulations. The time lagged autocorrelation is then used to compare statistical properties of the proposed models with those of real data and also with a synthetic time series generated though a simple Markov chain.

  2. On dynamic selection of households for direct marketing based on Markov chain models with memory

    Otter, Pieter W.

    2007-01-01

    A simple, dynamic selection procedure is proposed, based on conditional, expected profits using Markov chain models with memory. The method is easy to apply, only frequencies and mean values have to be calculated or estimated. The method is empirically illustrated using a data set from a charitable

  3. Experimental Simulations of Extreme Precipitation Based on the Multi-Status Markov Chain Model

    DING Yuguo; ZHANG Jinling; JIANG Zhihong

    2010-01-01

    A multi-status Markov chain model is proposed to produce daily rainrall, and based on which extreme rainfall is simulated with the generalized Pareto distribution (GPD). The simulated daily rainfall shows high precision at most stations, especially in pluvial regions of East China. The analysis reveals that the multi-status Markov chain model excels the bi-status Markov chain model in simulating climatic features of extreme rainfall. Results from the selected six stations demonstrate excellent simulations in the following aspects: standard deviation of monthly precipitation, daily maximum precipitation, the monthly mean rainfall days, standard deviation of daily precipitation and mean daily precipitation, which are proved to be consistent with the observations. A comparative study involving 78 stations in East China also reveals good consistency in monthly mean rainfall days and mean daily maximum rainfall, except mean daily rainfall. Simulation results at the above 6 stations have shown satisfactory fitting capability of the extreme precipitation GPD method. Good analogy is also found between simulation and observation in threshold and return values. As the errors of the threshold decrease, so do the differences between the return and real values. All the above demonstrates the applicability of the Markov chain model to extreme rainfall simulations.

  4. An NCME Instructional Module on Estimating Item Response Theory Models Using Markov Chain Monte Carlo Methods

    Kim, Jee-Seon; Bolt, Daniel M.

    2007-01-01

    The purpose of this ITEMS module is to provide an introduction to Markov chain Monte Carlo (MCMC) estimation for item response models. A brief description of Bayesian inference is followed by an overview of the various facets of MCMC algorithms, including discussion of prior specification, sampling procedures, and methods for evaluating chain…

  5. Behavior of an Almost Semicontinuous Poisson Process on a Markov Chain Upon Attainment of A Level

    Karnaukh, Ievgen

    2011-01-01

    We consider the almost semi-continuous processes defined on a finite Markov chain. The representation of the moment generating functions for the absolute maximum after achievement positive level and for the recovery time are obtained. Modified processes with two-step rate of negative jumps are investigated.

  6. Some strong limit theorems for nonhomogeneous Markov chains indexed by controlled trees

    Weicai Peng

    2016-02-01

    Full Text Available Abstract In this paper, a kind of infinite, local finite tree T, named a controlled tree, is introduced. Some strong limit properties, such as the strong law of large numbers and the asymptotic equipartition property, for nonhomogeneous Markov chains indexed by T, are established. The outcomes are the generalizations of some well-known results.

  7. Treatment of input uncertainty in hydrologic modeling: Doing hydrology backward with Markov chain Monte Carlo simulation

    Vrugt, J.A.; Braak, ter C.J.F.; Clark, M.P.; Hyman, J.M.; Robinson, B.A.

    2008-01-01

    There is increasing consensus in the hydrologic literature that an appropriate framework for streamflow forecasting and simulation should include explicit recognition of forcing and parameter and model structural error. This paper presents a novel Markov chain Monte Carlo (MCMC) sampler, entitled di

  8. Teaching Markov Chain Monte Carlo: Revealing the Basic Ideas behind the Algorithm

    Stewart, Wayne; Stewart, Sepideh

    2014-01-01

    For many scientists, researchers and students Markov chain Monte Carlo (MCMC) simulation is an important and necessary tool to perform Bayesian analyses. The simulation is often presented as a mathematical algorithm and then translated into an appropriate computer program. However, this can result in overlooking the fundamental and deeper…

  9. Average, sensitive and Blackwell-optimal policies in denumerable Markov decision chains with unbounded rewards

    R. Dekker (Rommert); A. Hordijk (Arie)

    1988-01-01

    textabstractIn this paper we consider a (discrete-time) Markov decision chain with a denumerabloe state space and compact action sets and we assume that for all states the rewards and transition probabilities depend continuously on the actions. The first objective of this paper is to develop an anal

  10. Exponential integrators for a Markov chain model of the fast sodium channel of cardiomyocytes.

    Starý, Tomás; Biktashev, Vadim N

    2015-04-01

    The modern Markov chain models of ionic channels in excitable membranes are numerically stiff. The popular numerical methods for these models require very small time steps to ensure stability. Our objective is to formulate and test two methods addressing this issue, so that the timestep can be chosen based on accuracy rather than stability. Both proposed methods extend Rush-Larsen technique, which was originally developed to Hogdkin-Huxley type gate models. One method, "matrix Rush-Larsen" (MRL) uses a matrix reformulation of the Rush-Larsen scheme, where the matrix exponentials are calculated using precomputed tables of eigenvalues and eigenvectors. The other, "hybrid operator splitting" (HOS) method exploits asymptotic properties of a particular Markov chain model, allowing explicit analytical expressions for the substeps. We test both methods on the Clancy and Rudy (2002) I(Na)Markov chain model. With precomputed tables for functions of the transmembrane voltage, both methods are comparable to the forward Euler method in accuracy and computational cost, but allow longer time steps without numerical instability. We conclude that both methods are of practical interest. MRL requires more computations than HOS, but is formulated in general terms which can be readily extended to other Markov chain channel models, whereas the utility of HOS depends on the asymptotic properties of a particular model. The significance of the methods is that they allow a considerable speed-up of large-scale computations of cardiac excitation models by increasing the time step, while maintaining acceptable accuracy and preserving numerical stability.

  11. An Evaluation of a Markov Chain Monte Carlo Method for the Rasch Model.

    Kim, Seock-Ho

    2001-01-01

    Examined the accuracy of the Gibbs sampling Markov chain Monte Carlo procedure for estimating item and person (theta) parameters in the one-parameter logistic model. Analyzed four empirical datasets using the Gibbs sampling, conditional maximum likelihood, marginal maximum likelihood, and joint maximum likelihood methods. Discusses the conditions…

  12. Markov chain conditions for admissibility in estimation problems with quadratic loss

    Eaton, M.L.

    1999-01-01

    Consider the problem of estimating a parametric function when the loss is quadratic. Given an improper prior distribution, there is a formal Bayes estimator for the parametric function. Associated with the estimation problem and the improper prior is a symmetric Markov chain. It is shown that if the

  13. Avian life history profiles for use in the Markov chain nest productivity model (MCnest)

    The Markov Chain nest productivity model, or MCnest, quantitatively estimates the effects of pesticides or other toxic chemicals on annual reproductive success of avian species (Bennett and Etterson 2013, Etterson and Bennett 2013). The Basic Version of MCnest was developed as a...

  14. A Markov Chain Monte Carlo Approach to Confirmatory Item Factor Analysis

    Edwards, Michael C.

    2010-01-01

    Item factor analysis has a rich tradition in both the structural equation modeling and item response theory frameworks. The goal of this paper is to demonstrate a novel combination of various Markov chain Monte Carlo (MCMC) estimation routines to estimate parameters of a wide variety of confirmatory item factor analysis models. Further, I show…

  15. Particle Markov Chain Monte Carlo Techniques of Unobserved Component Time Series Models Using Ox

    Nonejad, Nima

    This paper details Particle Markov chain Monte Carlo techniques for analysis of unobserved component time series models using several economic data sets. PMCMC combines the particle filter with the Metropolis-Hastings algorithm. Overall PMCMC provides a very compelling, computationally fast...

  16. Treatment-based Markov chain models clarify mechanisms of invasion in an invaded grassland community.

    Nelis, Lisa Castillo; Wootton, J Timothy

    2010-02-22

    What are the relative roles of mechanisms underlying plant responses in grassland communities invaded by both plants and mammals? What type of community can we expect in the future given current or novel conditions? We address these questions by comparing Markov chain community models among treatments from a field experiment on invasive species on Robinson Crusoe Island, Chile. Because of seed dispersal, grazing and disturbance, we predicted that the exotic European rabbit (Oryctolagus cuniculus) facilitates epizoochorous exotic plants (plants with seeds that stick to the skin an animal) at the expense of native plants. To test our hypothesis, we crossed rabbit exclosure treatments with disturbance treatments, and sampled the plant community in permanent plots over 3 years. We then estimated Markov chain model transition probabilities and found significant differences among treatments. As hypothesized, this modelling revealed that exotic plants survive better in disturbed areas, while natives prefer no rabbits or disturbance. Surprisingly, rabbits negatively affect epizoochorous plants. Markov chain dynamics indicate that an overall replacement of native plants by exotic plants is underway. Using a treatment-based approach to multi-species Markov chain models allowed us to examine the changes in the importance of mechanisms in response to experimental impacts on communities.

  17. Markov Chain Monte Carlo Estimation of Item Parameters for the Generalized Graded Unfolding Model

    de la Torre, Jimmy; Stark, Stephen; Chernyshenko, Oleksandr S.

    2006-01-01

    The authors present a Markov Chain Monte Carlo (MCMC) parameter estimation procedure for the generalized graded unfolding model (GGUM) and compare it to the marginal maximum likelihood (MML) approach implemented in the GGUM2000 computer program, using simulated and real personality data. In the simulation study, test length, number of response…

  18. Learning Bayesian network classifiers for credit scoring using Markov Chain Monte Carlo search

    Baesens, B.; Egmont-Petersen, M.; Castelo, R.; Vanthienen, J.

    2002-01-01

    In this paper, we will evaluate the power and usefulness of Bayesian network classifiers for credit scoring. Various types of Bayesian network classifiers will be evaluated and contrasted including unrestricted Bayesian network classifiers learnt using Markov Chain Monte Carlo (MCMC) search. The exp

  19. The Fracture Mechanical Markov Chain Fatigue Model Compared with Empirical Data

    Gansted, L.; Brincker, Rune; Hansen, Lars Pilegaard

    The applicability of the FMF-model (Fracture Mechanical Markov Chain Fatigue Model) introduced in Gansted, L., R. Brincker and L. Pilegaard Hansen (1991) is tested by simulations and compared with empirical data. Two sets of data have been used, the Virkler data (aluminium alloy) and data...... established at the Laboratory of Structural Engineering at Aalborg University, the AUC-data, (mild steel). The model, which is based on the assumption, that the crack propagation process can be described by a discrete Space Markov theory, is applicable to constant as well as random loading. It is shown...

  20. First and second order semi-Markov chains for wind speed modeling

    Prattico, F.; Petroni, F.; D'Amico, G.

    2012-04-01

    The increasing interest in renewable energy leads scientific research to find a better way to recover most of the available energy. Particularly, the maximum energy recoverable from wind is equal to 59.3% of that available (Betz law) at a specific pitch angle and when the ratio between the wind speed in output and in input is equal to 1/3. The pitch angle is the angle formed between the airfoil of the blade of the wind turbine and the wind direction. Old turbine and a lot of that actually marketed, in fact, have always the same invariant geometry of the airfoil. This causes that wind turbines will work with an efficiency that is lower than 59.3%. New generation wind turbines, instead, have a system to variate the pitch angle by rotating the blades. This system able the wind turbines to recover, at different wind speed, always the maximum energy, working in Betz limit at different speed ratios. A powerful system control of the pitch angle allows the wind turbine to recover better the energy in transient regime. A good stochastic model for wind speed is then needed to help both the optimization of turbine design and to assist the system control to predict the value of the wind speed to positioning the blades quickly and correctly. The possibility to have synthetic data of wind speed is a powerful instrument to assist designer to verify the structures of the wind turbines or to estimate the energy recoverable from a specific site. To generate synthetic data, Markov chains of first or higher order are often used [1,2,3]. In particular in [3] is presented a comparison between a first-order Markov chain and a second-order Markov chain. A similar work, but only for the first-order Markov chain, is conduced by [2], presenting the probability transition matrix and comparing the energy spectral density and autocorrelation of real and synthetic wind speed data. A tentative to modeling and to join speed and direction of wind is presented in [1], by using two models, first

  1. Understanding for convergence monitoring for probabilistic risk assessment based on Markov Chain Monte Carlo Simulation

    Kim, Joo Yeon; Jang, Han Ki; Jang, Sol Ah; Park, Tae Jin [Korean Association for Radiation Application, Seoul (Korea, Republic of)

    2014-04-15

    There is a question that the simulation actually leads to draws from its target distribution and the most basic one is whether such Markov chains can always be constructed and all chain values sampled from them. The problem to be solved is the determination of how large this iteration should be to achieve the target distribution. This problem can be answered as convergence monitoring. In this paper, two widely used methods, such as autocorrelation and potential scale reduction factor (PSRF) in MCMC are characterized. There is no general agreement on the subject of the convergence. Although it is generally agreed that running n parallel chains in practice is computationally inefficient and unnecessary, running multiple parallel chains is generally applied for the convergence monitoring due to easy implementation. The main debate is the number of parallel chains needed. If the convergence properties of the chain are well understood then clearly a single chain suffices. Therefore, autocorrelation using single chain and multiple parallel ones are tried and their results then compared with each other in this study. And, the following question is answered from the two convergence results: Have the Markov chain realizations for achieved the target distribution?.

  2. An Approach for Customer Satisfaction: Evaluation, Validation and Modeling By A Markov Chain

    Amina EL KEBBAJ

    2015-01-01

    Full Text Available The main objective of this work is to develop a practical approach to improve customer satisfaction, which is generally regarded as the pillar of customer loyalty to the company. Today, customer satisfaction is a major challenge. In fact, listening to the customer, anticipating and properly managing his claims are stone keys and fundamental values for the company. From a perspective of the quality of the product, skills, and mostly, the service provided to the customer, it is essential for organizations to differentiate themselves, especially in a more competitive world, in order to ensure a higher level of customer satisfaction. Ignoring or not taking into account customer satisfaction can have harmful consequences on both the economic performances and the organization’s image. For that, it is crucial to develop new methods and have new approaches to the problematic customer dissatisfaction, by improving the services quality provided to the costumer. This work describes a simple and practical approach for modeling customer satisfaction for organizations in order to reduce the level of dissatisfaction using the decisional prediction that allows companies to anticipate and adapt to future changes in their environment to be able to make the best decisions to ensure their sustainability. This approach respects the constraints of the organization and eliminates any action that can lead to loss of customers and degradation of the image of the organization. The problem is mathematically modeled by a Markov chain and practical examples to illustrate the work are given.

  3. An Efficient Interpolation Technique for Jump Proposals in Reversible-Jump Markov Chain Monte Carlo Calculations

    Farr, Will M

    2011-01-01

    Selection among alternative theoretical models given an observed data set is an important challenge in many areas of physics and astronomy. Reversible-jump Markov chain Monte Carlo (RJMCMC) is an extremely powerful technique for performing Bayesian model selection, but it suffers from a fundamental difficulty: it requires jumps between model parameter spaces, but cannot retain a memory of the favored locations in more than one parameter space at a time. Thus, a naive jump between parameter spaces is unlikely to be accepted in the MCMC algorithm and convergence is correspondingly slow. Here we demonstrate an interpolation technique that uses samples from single-model MCMCs to propose inter-model jumps from an approximation to the single-model posterior of the target parameter space. The interpolation technique, based on a kD-tree data structure, is adaptive and efficient in arbitrary dimensions. We show that our technique leads to dramatically improved convergence over naive jumps in an RJMCMC, and compare it ...

  4. From Brownian Dynamics to Markov Chain: An Ion Channel Example

    Chen, Wan

    2014-02-27

    A discrete rate theory for multi-ion channels is presented, in which the continuous dynamics of ion diffusion is reduced to transitions between Markovian discrete states. In an open channel, the ion permeation process involves three types of events: an ion entering the channel, an ion escaping from the channel, or an ion hopping between different energy minima in the channel. The continuous dynamics leads to a hierarchy of Fokker-Planck equations, indexed by channel occupancy. From these the mean escape times and splitting probabilities (denoting from which side an ion has escaped) can be calculated. By equating these with the corresponding expressions from the Markov model, one can determine the Markovian transition rates. The theory is illustrated with a two-ion one-well channel. The stationary probability of states is compared with that from both Brownian dynamics simulation and the hierarchical Fokker-Planck equations. The conductivity of the channel is also studied, and the optimal geometry maximizing ion flux is computed. © 2014 Society for Industrial and Applied Mathematics.

  5. From Brownian Dynamics to Markov Chain: an Ion Channel Example

    Chen, Wan; Chapman, S Jonathan

    2012-01-01

    A discrete rate theory for general multi-ion channels is presented, in which the continuous dynamics of ion diffusion is reduced to transitions between Markovian discrete states. In an open channel, the ion permeation process involves three types of events: an ion entering the channel, an ion escaping from the channel, or an ion hopping between different energy minima in the channel. The continuous dynamics leads to a hierarchy of Fokker-Planck equations, indexed by channel occupancy. From these the mean escape times and splitting probabilities (denoting from which side an ion has escaped) can be calculated. By equating these with the corresponding expressions from the Markov model the Markovian transition rates can be determined. The theory is illustrated with a two-ion one-well channel. The stationary probability of states is compared with that from both Brownian dynamics simulation and the hierarchical Fokker-Planck equations. The conductivity of the channel is also studied, and the optimal geometry maximi...

  6. Detecting memory and structure in human navigation patterns using Markov chain models of varying order.

    Singer, Philipp; Helic, Denis; Taraghi, Behnam; Strohmaier, Markus

    2014-01-01

    One of the most frequently used models for understanding human navigation on the Web is the Markov chain model, where Web pages are represented as states and hyperlinks as probabilities of navigating from one page to another. Predominantly, human navigation on the Web has been thought to satisfy the memoryless Markov property stating that the next page a user visits only depends on her current page and not on previously visited ones. This idea has found its way in numerous applications such as Google's PageRank algorithm and others. Recently, new studies suggested that human navigation may better be modeled using higher order Markov chain models, i.e., the next page depends on a longer history of past clicks. Yet, this finding is preliminary and does not account for the higher complexity of higher order Markov chain models which is why the memoryless model is still widely used. In this work we thoroughly present a diverse array of advanced inference methods for determining the appropriate Markov chain order. We highlight strengths and weaknesses of each method and apply them for investigating memory and structure of human navigation on the Web. Our experiments reveal that the complexity of higher order models grows faster than their utility, and thus we confirm that the memoryless model represents a quite practical model for human navigation on a page level. However, when we expand our analysis to a topical level, where we abstract away from specific page transitions to transitions between topics, we find that the memoryless assumption is violated and specific regularities can be observed. We report results from experiments with two types of navigational datasets (goal-oriented vs. free form) and observe interesting structural differences that make a strong argument for more contextual studies of human navigation in future work.

  7. Generation of solar radiation values by using Markov chains; Generacion de valores de radiacion usando cadenas de Markov

    Adaro, Jorge; Cesari, Daniela; Lema, Alba; Galimberti, Pablo [Universidad Nacional de Rio Cuarto (Argentina). Facultad de Ingenieria]. E-mail: aadaro@ing.unrc.edu.ar

    2000-07-01

    The objective of the present work is to adopt a methodology that allows to generate sequences of values of global solar radiation. It is carried out a preliminary study on the generation of radiation sequence a concept of Chains of Markov. For it is analyzed it the readiness of data and it is investigated about the possibility of using such a methodology calculating values of indexes of clarity previously. With data of available radiation and provided the National Meteorological Service for Rio Cuarto, the preliminary study is carried out the preliminary study looking for to validated the pattern to the effects of being able to transfer the use of the methodology in other regions. (author)

  8. Fisher information and asymptotic normality in system identification for quantum Markov chains

    Guta, Madalin

    2010-01-01

    This paper deals with the problem of estimating the coupling constant $\\theta$ of a mixing quantum Markov chain with finite dimensional quantum systems. For a repeated measurement on the chain's output we show that the outcomes' time average has an asymptotically normal (Gaussian) distribution, and we give the explicit expressions of its mean and variance. This provides a simple estimator of $\\theta$ with computable classical Fisher information which can be optimized over different choices of measurements. We then show that the output itself is asymptotically normal, i.e. at time $n$ the joint output states with parameters of the form $\\theta_{0}+u/\\sqrt{n}$ look like a one dimensional family of coherent states with displacement $u\\sqrt{F/2}$, where $F$ is the quantum Fisher information of the output. These results are quantum extensions of theorems from the theory of classical Markov chains, and further extensions to continuous time dynamics and multiple parameters are in preparation.

  9. Uncovering and testing the fuzzy clusters based on lumped Markov chain in complex network.

    Jing, Fan; Jianbin, Xie; Jinlong, Wang; Jinshuai, Qu

    2013-01-01

    Identifying clusters, namely groups of nodes with comparatively strong internal connectivity, is a fundamental task for deeply understanding the structure and function of a network. By means of a lumped Markov chain model of a random walker, we propose two novel ways of inferring the lumped markov transition matrix. Furthermore, some useful results are proposed based on the analysis of the properties of the lumped Markov process. To find the best partition of complex networks, a novel framework including two algorithms for network partition based on the optimal lumped Markovian dynamics is derived to solve this problem. The algorithms are constructed to minimize the objective function under this framework. It is demonstrated by the simulation experiments that our algorithms can efficiently determine the probabilities with which a node belongs to different clusters during the learning process and naturally supports the fuzzy partition. Moreover, they are successfully applied to real-world network, including the social interactions between members of a karate club.

  10. Quantum Markov fields on graphs

    2009-01-01

    We introduce generalized quantum Markov states and generalized d-Markov chains which extend the notion quantum Markov chains on spin systems to that on $C^*$-algebras defined by general graphs. As examples of generalized d-Markov chains, we construct the entangled Markov fields on tree graphs. The concrete examples of generalized d-Markov chains on Cayley trees are also investigated.

  11. STRONG LAW OF LARGE NUMBERS AND SHANNON-MCMILLAN THEOREM FOR MARKOV CHAINS FIELD ON CAYLEY TREE

    杨卫国; 刘文

    2001-01-01

    This paper studies the strong law of large numbers and the Shannom-McMillan theorem for Markov chains field on Cayley tree. The authors first prove the strong law of large number on the frequencies of states and orderd couples of states for Markov chains field on Cayley tree. Then they prove thc Shannon-McMillan theorem with a.e. convergence for Markov chains field on Cayley tree. In the proof, a new technique in the study the strong limit theorem in probability theory is applied.

  12. Modeling Urban Expansion in Bangkok Metropolitan Region Using Demographic–Economic Data through Cellular Automata-Markov Chain and Multi-Layer Perceptron-Markov Chain Models

    Chudech Losiri

    2016-07-01

    Full Text Available Urban expansion is considered as one of the most important problems in several developing countries. Bangkok Metropolitan Region (BMR is the urbanized and agglomerated area of Bangkok Metropolis (BM and its vicinity, which confronts the expansion problem from the center of the city. Landsat images of 1988, 1993, 1998, 2003, 2008, and 2011 were used to detect the land use and land cover (LULC changes. The demographic and economic data together with corresponding maps were used to determine the driving factors for land conversions. This study applied Cellular Automata-Markov Chain (CA-MC and Multi-Layer Perceptron-Markov Chain (MLP-MC to model LULC and urban expansions. The performance of the CA-MC and MLP-MC yielded more than 90% overall accuracy to predict the LULC, especially the MLP-MC method. Further, the annual population and economic growth rates were considered to produce the land demand for the LULC in 2014 and 2035 using the statistical extrapolation and system dynamics (SD. It was evident that the simulated map in 2014 resulting from the SD yielded the highest accuracy. Therefore, this study applied the SD method to generate the land demand for simulating LULC in 2035. The outcome showed that urban occupied the land around a half of the BMR.

  13. Fitting timeseries by continuous-time Markov chains: A quadratic programming approach

    Crommelin, D. T.; Vanden-Eijnden, E.

    2006-09-01

    Construction of stochastic models that describe the effective dynamics of observables of interest is an useful instrument in various fields of application, such as physics, climate science, and finance. We present a new technique for the construction of such models. From the timeseries of an observable, we construct a discrete-in-time Markov chain and calculate the eigenspectrum of its transition probability (or stochastic) matrix. As a next step we aim to find the generator of a continuous-time Markov chain whose eigenspectrum resembles the observed eigenspectrum as closely as possible, using an appropriate norm. The generator is found by solving a minimization problem: the norm is chosen such that the object function is quadratic and convex, so that the minimization problem can be solved using quadratic programming techniques. The technique is illustrated on various toy problems as well as on datasets stemming from simulations of molecular dynamics and of atmospheric flows.

  14. Inverting OII 83.4 nm Dayglow Profiles Using Markov Chain Radiative Transfer

    Geddes, George; Finn, Susanna C; Cook, Timothy; Chakrabarti, Supriya

    2016-01-01

    Emission profiles of the resonantly scattered OII~83.4~nm triplet can in principle be used to estimate \\(\\mathrm{O}^+\\) density profiles in the F2 region of the ionosphere. Given the emission source profile, solution of this inverse problem is possible, but requires significant computation. The traditional Feautrier solution to the radiative transfer problem requires many iterations to converge, making it time-consuming to compute. A Markov chain approach to the problem produces similar results by directly constructing a matrix that maps the source emission rate to an effective emission rate which includes scattering to all orders. The Markov chain approach presented here yields faster results and therefore can be used to perform the \\(\\mathrm{O}^+\\) density retrieval with higher resolution than would otherwise be possible.

  15. Risk Minimization for Insurance Products via F-Doubly Stochastic Markov Chains

    Francesca Biagini

    2016-07-01

    Full Text Available We study risk-minimization for a large class of insurance contracts. Given that the individual progress in time of visiting an insurance policy’s states follows an F -doubly stochastic Markov chain, we describe different state-dependent types of insurance benefits. These cover single payments at maturity, annuity-type payments and payments at the time of a transition. Based on the intensity of the F -doubly stochastic Markov chain, we provide the Galtchouk-Kunita-Watanabe decomposition for a general insurance contract and specify risk-minimizing strategies in a Brownian financial market setting. The results are further illustrated explicitly within an affine structure for the intensity.

  16. An Efficient Finite Difference Method for Parameter Sensitivities of Continuous Time Markov Chains

    Anderson, David F

    2011-01-01

    We present an efficient finite difference method for the computation of parameter sensitivities for a wide class of continuous time Markov chains. The motivating class of models, and the source of our examples, are the stochastic chemical kinetic models commonly used in the biosciences, though other natural application areas include population processes and queuing networks. The method is essentially derived by making effective use of the random time change representation of Kurtz, and is no harder to implement than any standard continuous time Markov chain algorithm, such as "Gillespie's algorithm" or the next reaction method. Further, the method is analytically tractable, and, for a given number of realizations of the stochastic process, produces an estimator with substantially lower variance than that obtained using other common methods. Therefore, the computational complexity required to solve a given problem is lowered greatly. In this work, we present the method together with the theoretical analysis de...

  17. MC3: Multi-core Markov-chain Monte Carlo code

    Cubillos, Patricio; Harrington, Joseph; Lust, Nate; Foster, AJ; Stemm, Madison; Loredo, Tom; Stevenson, Kevin; Campo, Chris; Hardin, Matt; Hardy, Ryan

    2016-10-01

    MC3 (Multi-core Markov-chain Monte Carlo) is a Bayesian statistics tool that can be executed from the shell prompt or interactively through the Python interpreter with single- or multiple-CPU parallel computing. It offers Markov-chain Monte Carlo (MCMC) posterior-distribution sampling for several algorithms, Levenberg-Marquardt least-squares optimization, and uniform non-informative, Jeffreys non-informative, or Gaussian-informative priors. MC3 can share the same value among multiple parameters and fix the value of parameters to constant values, and offers Gelman-Rubin convergence testing and correlated-noise estimation with time-averaging or wavelet-based likelihood estimation methods.

  18. AN IMPROVED MARKOV CHAIN MONTE CARLO METHOD FOR MIMO ITERATIVE DETECTION AND DECODING

    Han Xiang; Wei Jibo

    2008-01-01

    Recently, a new soft-in soft-out detection algorithm based on the Markov Chain Monte Carlo (MCMC) simulation technique for Multiple-Input Multiple-Output (MIMO) systems is proposed,which is shown to perform significantly better than their sphere decoding counterparts with relatively low complexity. However, the MCMC simulator is likely to get trapped in a fixed state when the channel SNR is high, thus lots of repetitive samples are observed and the accuracy of A Posteriori Probability (APP) estimation deteriorates. To solve this problem, an improved version of MCMC simulator, named forced-dispersed MCMC algorithm is proposed. Based on the a posteriori variance of each bit, the Gibbs sampler is monitored. Once the trapped state is detected, the sample is dispersed intentionally according to the a posteriori variance. Extensive simulation shows that, compared with the existing solution, the proposed algorithm enables the markov chain to travel more states, which ensures a near-optimal performance.

  19. Inverting OII 83.4 nm dayglow profiles using Markov chain radiative transfer

    Geddes, George; Douglas, Ewan; Finn, Susanna C.; Cook, Timothy; Chakrabarti, Supriya

    2016-11-01

    Emission profiles of the resonantly scattered OII 83.4 nm triplet can in principle be used to estimate O+ density profiles in the F2 region of the ionosphere. Given the emission source profile, solution of this inverse problem is possible but requires significant computation. The traditional Feautrier solution to the radiative transfer problem requires many iterations to converge, making it time consuming to compute. A Markov chain approach to the problem produces similar results by directly constructing a matrix that maps the source emission rate to an effective emission rate which includes scattering to all orders. The Markov chain approach presented here yields faster results and therefore can be used to perform the O+ density retrieval with higher resolution than would otherwise be possible.

  20. An 'adding' algorithm for the Markov chain formalism for radiation transfer

    Esposito, L. W.

    1979-01-01

    An adding algorithm is presented, that extends the Markov chain method and considers a preceding calculation as a single state of a new Markov chain. This method takes advantage of the description of the radiation transport as a stochastic process. Successive application of this procedure makes calculation possible for any optical depth without increasing the size of the linear system used. It is determined that the time required for the algorithm is comparable to that for a doubling calculation for homogeneous atmospheres. For an inhomogeneous atmosphere the new method is considerably faster than the standard adding routine. It is concluded that the algorithm is efficient, accurate, and suitable for smaller computers in calculating the diffuse intensity scattered by an inhomogeneous planetary atmosphere.

  1. Inferring Markov chains: Bayesian estimation, model comparison, entropy rate, and out-of-class modeling.

    Strelioff, Christopher C; Crutchfield, James P; Hübler, Alfred W

    2007-07-01

    Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer kth order Markov chains, for arbitrary k , from finite data by applying Bayesian methods to both parameter estimation and model-order selection. Extending existing results for multinomial models of discrete data, we connect inference to statistical mechanics through information-theoretic (type theory) techniques. We establish a direct relationship between Bayesian evidence and the partition function which allows for straightforward calculation of the expectation and variance of the conditional relative entropy and the source entropy rate. Finally, we introduce a method that uses finite data-size scaling with model-order comparison to infer the structure of out-of-class processes.

  2. Overshoot in biological systems modelled by Markov chains: a non-equilibrium dynamic phenomenon.

    Jia, Chen; Qian, Minping; Jiang, Daquan

    2014-08-01

    A number of biological systems can be modelled by Markov chains. Recently, there has been an increasing concern about when biological systems modelled by Markov chains will perform a dynamic phenomenon called overshoot. In this study, the authors found that the steady-state behaviour of the system will have a great effect on the occurrence of overshoot. They showed that overshoot in general cannot occur in systems that will finally approach an equilibrium steady state. They further classified overshoot into two types, named as simple overshoot and oscillating overshoot. They showed that except for extreme cases, oscillating overshoot will occur if the system is far from equilibrium. All these results clearly show that overshoot is a non-equilibrium dynamic phenomenon with energy consumption. In addition, the main result in this study is validated with real experimental data.

  3. Summary statistics for end-point conditioned continuous-time Markov chains

    Hobolth, Asger; Jensen, Jens Ledet

    Continuous-time Markov chains are a widely used modelling tool. Applications include DNA sequence evolution, ion channel gating behavior and mathematical finance. We consider the problem of calculating properties of summary statistics (e.g. mean time spent in a state, mean number of jumps between...... two states and the distribution of the total number of jumps) for discretely observed continuous time Markov chains. Three alternative methods for calculating properties of summary statistics are described and the pros and cons of the methods are discussed. The methods are based on (i) an eigenvalue...... decomposition of the rate matrix, (ii) the uniformization method, and (iii) integrals of matrix exponentials. In particular we develop a framework that allows for analyses of rather general summary statistics using the uniformization method....

  4. An Expectation Maximization Algorithm to Model Failure Times by Continuous-Time Markov Chains

    Qihong Duan

    2010-01-01

    Full Text Available In many applications, the failure rate function may present a bathtub shape curve. In this paper, an expectation maximization algorithm is proposed to construct a suitable continuous-time Markov chain which models the failure time data by the first time reaching the absorbing state. Assume that a system is described by methods of supplementary variables, the device of stage, and so on. Given a data set, the maximum likelihood estimators of the initial distribution and the infinitesimal transition rates of the Markov chain can be obtained by our novel algorithm. Suppose that there are m transient states in the system and that there are n failure time data. The devised algorithm only needs to compute the exponential of m×m upper triangular matrices for O(nm2 times in each iteration. Finally, the algorithm is applied to two real data sets, which indicates the practicality and efficiency of our algorithm.

  5. Explosion, implosion, and moments of passage times for continuous-time Markov chains: a semimartingale approach

    Menshikov, Mikhail

    2012-01-01

    We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of occurrence of the phenomenon of explosion are also obtained. A new phenomenon of implosion is introduced and sharp conditions for its occurrence are proven. The general results are illustrated by treating models having a difficult behaviour even in discrete time.

  6. Evaluating Stationary Distribution of the Binary GA Markov Chain in Special Cases

    Mitavskiy, Boris S.; Cannings, Chris

    2008-01-01

    The evolutionary algorithm stochastic process is well-known to be Markovian. These have been under investigation in much of the theoretical evolutionary computing research. When mutation rate is positive, the Markov chain modeling an evolutionary algorithm is irreducible and, therefore, has a unique stationary distribution, yet, rather little is known about the stationary distribution. On the other hand, knowing the stationary distribution may provide some information abo...

  7. A Stochastic Markov Chain Model to Describe Lung Cancer Growth and Metastasis

    Newton, Paul K.; Jeremy Mason; Kelly Bethel; Bazhenova, Lyudmila A.; Jorge Nieva; Peter Kuhn

    2012-01-01

    A stochastic Markov chain model for metastatic progression is developed for primary lung cancer based on a network construction of metastatic sites with dynamics modeled as an ensemble of random walkers on the network. We calculate a transition matrix, with entries (transition probabilities) interpreted as random variables, and use it to construct a circular bi-directional network of primary and metastatic locations based on postmortem tissue analysis of 3827 autopsies on untreated patients d...

  8. A Markov chain model of a polling system with parameter regeneration

    2007-01-01

    International audience; We study a model of a polling system i.e.\\ a collection of $d$ queues with a single server that switches from queue to queue. The service time distribution and arrival rates change randomly every time a queue is emptied. This model is mapped to a mathematically equivalent model of a random walk with random choice of transition probabilities, a model which is of independent interest. All our results are obtained using methods from the constructive theory of Markov chain...

  9. THE DECOMPOSITION OF STATE SPACE FOR MARKOV CHAIN IN RANDOM ENVIRONMENT

    Hu Dihe

    2005-01-01

    This paper is a continuation of [8] and [9]. The author obtains the decomposition of state space χof an Markov chain in random environment by making use of the results in [8] and [9], gives three examples, random walk in random environment, renewal process in random environment and queue process in random environment, and obtains the decompositions of the state spaces of these three special examples.

  10. Cosmological constraints on generalized Chaplygin gas model: Markov Chain Monte Carlo approach

    Xu, Lixin; Lu, Jianbo

    2010-01-01

    We use the Markov Chain Monte Carlo method to investigate a global constraints on the generalized Chaplygin gas (GCG) model as the unification of dark matter and dark energy from the latest observational data: the Constitution dataset of type supernovae Ia (SNIa), the observational Hubble data (OHD), the cluster X-ray gas mass fraction, the baryon acoustic oscillation (BAO), and the cosmic microwave background (CMB) data. In a non-flat universe, the constraint results for GCG model are, $\\Ome...

  11. Convergence rates for reversible Markov chains without the assumption of nonnegative definite matrices

    2010-01-01

    Explicit convergence rates in geometric and strong ergodicity for denumerable discrete time Markov chains with general reversible transition matrices are obtained in terms of the geometric moments or uniform moments of the hitting times to a fixed point.Another way by Lyapunov’s drift conditions is also used to derive these convergence rates.As a typical example,the discrete time birth-death process(random walk) is studied and the explicit criteria for geometric ergodicity are presented.

  12. Physical time scale in kinetic Monte Carlo simulations of continuous-time Markov chains.

    Serebrinsky, Santiago A

    2011-03-01

    We rigorously establish a physical time scale for a general class of kinetic Monte Carlo algorithms for the simulation of continuous-time Markov chains. This class of algorithms encompasses rejection-free (or BKL) and rejection (or "standard") algorithms. For rejection algorithms, it was formerly considered that the availability of a physical time scale (instead of Monte Carlo steps) was empirical, at best. Use of Monte Carlo steps as a time unit now becomes completely unnecessary.

  13. A new fuzzy Monte Carlo method for solving SLAE with ergodic fuzzy Markov chains

    Maryam Gharehdaghi

    2015-05-01

    Full Text Available In this paper we introduce a new fuzzy Monte Carlo method for solving system of linear algebraic equations (SLAE over the possibility theory and max-min algebra. To solve the SLAE, we first define a fuzzy estimator and prove that this is an unbiased estimator of the solution. To prove unbiasedness, we apply the ergodic fuzzy Markov chains. This new approach works even for cases with coefficients matrix with a norm greater than one.

  14. Generalization bounds of ERM-based learning processes for continuous-time Markov chains.

    Zhang, Chao; Tao, Dacheng

    2012-12-01

    Many existing results on statistical learning theory are based on the assumption that samples are independently and identically distributed (i.i.d.). However, the assumption of i.i.d. samples is not suitable for practical application to problems in which samples are time dependent. In this paper, we are mainly concerned with the empirical risk minimization (ERM) based learning process for time-dependent samples drawn from a continuous-time Markov chain. This learning process covers many kinds of practical applications, e.g., the prediction for a time series and the estimation of channel state information. Thus, it is significant to study its theoretical properties including the generalization bound, the asymptotic convergence, and the rate of convergence. It is noteworthy that, since samples are time dependent in this learning process, the concerns of this paper cannot (at least straightforwardly) be addressed by existing methods developed under the sample i.i.d. assumption. We first develop a deviation inequality for a sequence of time-dependent samples drawn from a continuous-time Markov chain and present a symmetrization inequality for such a sequence. By using the resultant deviation inequality and symmetrization inequality, we then obtain the generalization bounds of the ERM-based learning process for time-dependent samples drawn from a continuous-time Markov chain. Finally, based on the resultant generalization bounds, we analyze the asymptotic convergence and the rate of convergence of the learning process.

  15. Bayesian clustering of DNA sequences using Markov chains and a stochastic partition model.

    Jääskinen, Väinö; Parkkinen, Ville; Cheng, Lu; Corander, Jukka

    2014-02-01

    In many biological applications it is necessary to cluster DNA sequences into groups that represent underlying organismal units, such as named species or genera. In metagenomics this grouping needs typically to be achieved on the basis of relatively short sequences which contain different types of errors, making the use of a statistical modeling approach desirable. Here we introduce a novel method for this purpose by developing a stochastic partition model that clusters Markov chains of a given order. The model is based on a Dirichlet process prior and we use conjugate priors for the Markov chain parameters which enables an analytical expression for comparing the marginal likelihoods of any two partitions. To find a good candidate for the posterior mode in the partition space, we use a hybrid computational approach which combines the EM-algorithm with a greedy search. This is demonstrated to be faster and yield highly accurate results compared to earlier suggested clustering methods for the metagenomics application. Our model is fairly generic and could also be used for clustering of other types of sequence data for which Markov chains provide a reasonable way to compress information, as illustrated by experiments on shotgun sequence type data from an Escherichia coli strain.

  16. Characterization of the rat exploratory behavior in the elevated plus-maze with Markov chains.

    Tejada, Julián; Bosco, Geraldine G; Morato, Silvio; Roque, Antonio C

    2010-11-30

    The elevated plus-maze is an animal model of anxiety used to study the effect of different drugs on the behavior of the animal. It consists of a plus-shaped maze with two open and two closed arms elevated 50cm from the floor. The standard measures used to characterize exploratory behavior in the elevated plus-maze are the time spent and the number of entries in the open arms. In this work, we use Markov chains to characterize the exploratory behavior of the rat in the elevated plus-maze under three different conditions: normal and under the effects of anxiogenic and anxiolytic drugs. The spatial structure of the elevated plus-maze is divided into squares, which are associated with states of a Markov chain. By counting the frequencies of transitions between states during 5-min sessions in the elevated plus-maze, we constructed stochastic matrices for the three conditions studied. The stochastic matrices show specific patterns, which correspond to the observed behaviors of the rat under the three different conditions. For the control group, the stochastic matrix shows a clear preference for places in the closed arms. This preference is enhanced for the anxiogenic group. For the anxiolytic group, the stochastic matrix shows a pattern similar to a random walk. Our results suggest that Markov chains can be used together with the standard measures to characterize the rat behavior in the elevated plus-maze.

  17. A Markov chain Monte Carlo method family in incomplete data analysis

    Vasić Vladimir V.

    2003-01-01

    Full Text Available A Markov chain Monte Carlo method family is a collection of techniques for pseudorandom draws out of probability distribution function. In recent years, these techniques have been the subject of intensive interest of many statisticians. Roughly speaking, the essence of a Markov chain Monte Carlo method family is generating one or more values of a random variable Z, which is usually multidimensional. Let P(Z = f(Z denote a density function of a random variable Z, which we will refer to as a target distribution. Instead of sampling directly from the distribution f, we will generate [Z(1, Z(2..., Z(t,... ], in which each value is, in a way, dependant upon the previous value and where the stationary distribution will be a target distribution. For a sufficient value of t, Z(t will be approximately random sampling of the distribution f. A Markov chain Monte Carlo method family is useful when direct sampling is difficult, but when sampling of each value is not.

  18. State space orderings for Gauss-Seidel in Markov chains revisited

    Dayar, T. [Bilkent Univ., Ankara (Turkey)

    1996-12-31

    Symmetric state space orderings of a Markov chain may be used to reduce the magnitude of the subdominant eigenvalue of the (Gauss-Seidel) iteration matrix. Orderings that maximize the elemental mass or the number of nonzero elements in the dominant term of the Gauss-Seidel splitting (that is, the term approximating the coefficient matrix) do not necessarily converge faster. An ordering of a Markov chain that satisfies Property-R is semi-convergent. On the other hand, there are semi-convergent symmetric state space orderings that do not satisfy Property-R. For a given ordering, a simple approach for checking Property-R is shown. An algorithm that orders the states of a Markov chain so as to increase the likelihood of satisfying Property-R is presented. The computational complexity of the ordering algorithm is less than that of a single Gauss-Seidel iteration (for sparse matrices). In doing all this, the aim is to gain an insight for faster converging orderings. Results from a variety of applications improve the confidence in the algorithm.

  19. Subgeometric Ergodicity Analysis of Continuous-Time Markov Chains under Random-Time State-Dependent Lyapunov Drift Conditions

    Mokaedi V. Lekgari

    2014-01-01

    Full Text Available We investigate random-time state-dependent Foster-Lyapunov analysis on subgeometric rate ergodicity of continuous-time Markov chains (CTMCs. We are mainly concerned with making use of the available results on deterministic state-dependent drift conditions for CTMCs and on random-time state-dependent drift conditions for discrete-time Markov chains and transferring them to CTMCs.

  20. The combinational structure of non-homogeneous Markov chains with countable states

    A. Mukherjea

    1983-01-01

    Full Text Available Let P(s,t denote a non-homogeneous continuous parameter Markov chain with countable state space E and parameter space [a,b], −∞0}. It is shown in this paper that R(s,t is reflexive, transitive, and independent of (s,t, schains in the case when E is infinite.

  1. Predicting transient particle transport in enclosed environments with the combined computational fluid dynamics and Markov chain method.

    Chen, C; Lin, C-H; Long, Z; Chen, Q

    2014-02-01

    To quickly obtain information about airborne infectious disease transmission in enclosed environments is critical in reducing the infection risk to the occupants. This study developed a combined computational fluid dynamics (CFD) and Markov chain method for quickly predicting transient particle transport in enclosed environments. The method first calculated a transition probability matrix using CFD simulations. Next, the Markov chain technique was applied to calculate the transient particle concentration distributions. This investigation used three cases, particle transport in an isothermal clean room, an office with an underfloor air distribution system, and the first-class cabin of an MD-82 airliner, to validate the combined CFD and Markov chain method. The general trends of the particle concentrations vs. time predicted by the Markov chain method agreed with the CFD simulations for these cases. The proposed Markov chain method can provide faster-than-real-time information about particle transport in enclosed environments. Furthermore, for a fixed airflow field, when the source location is changed, the Markov chain method can be used to avoid recalculation of the particle transport equation and thus reduce computing costs.

  2. Simple Markov-chain algorithms for generating bipartite graphs and tournaments

    Kannan, R.; Vempala, S. [Carnegie Mellon Univ., Pittsburgh, PA (United States); Tetali, P. [Georgia Institute of Technology, Atlanta, GA (United States)

    1997-06-01

    We consider two problems: randomly generating labeled bipartite graphs with a given degree sequence and randomly generating labeled tournaments with a given score sequence. We analyze simple Markov chains for both problems. For the first problem, we cannot prove that our chain is rapidly mixing in general, but in the (near-) regular case, i.e. when all the degrees are (almost) equal, we give a proof of rapid mixing. Our methods also apply to the corresponding problem for general (nonbipartite) regular graphs which was studied earlier by several researchers. One significant difference in our approach is that our chain has one state for every graph (or bipartite graph) with the given degree sequence; in particular, there are no auxiliary states as in the chain used by Jerrum and Sinclair. For the problem of generating tournaments, we are able to prove that our Markov chain on tournaments is rapidly mixing, if the score sequence is near-regular. The proof techniques we use for the two problems are similar.

  3. Exact likelihood-free Markov chain Monte Carlo for elliptically contoured distributions.

    Muchmore, Patrick; Marjoram, Paul

    2015-08-01

    Recent results in Markov chain Monte Carlo (MCMC) show that a chain based on an unbiased estimator of the likelihood can have a stationary distribution identical to that of a chain based on exact likelihood calculations. In this paper we develop such an estimator for elliptically contoured distributions, a large family of distributions that includes and generalizes the multivariate normal. We then show how this estimator, combined with pseudorandom realizations of an elliptically contoured distribution, can be used to run MCMC in a way that replicates the stationary distribution of a likelihood based chain, but does not require explicit likelihood calculations. Because many elliptically contoured distributions do not have closed form densities, our simulation based approach enables exact MCMC based inference in a range of cases where previously it was impossible.

  4. A Graph-Algorithmic Approach for the Study of Metastability in Markov Chains

    Gan, Tingyue; Cameron, Maria

    2017-01-01

    Large continuous-time Markov chains with exponentially small transition rates arise in modeling complex systems in physics, chemistry, and biology. We propose a constructive graph-algorithmic approach to determine the sequence of critical timescales at which the qualitative behavior of a given Markov chain changes, and give an effective description of the dynamics on each of them. This approach is valid for both time-reversible and time-irreversible Markov processes, with or without symmetry. Central to this approach are two graph algorithms, Algorithm 1 and Algorithm 2, for obtaining the sequences of the critical timescales and the hierarchies of Typical Transition Graphs or T-graphs indicating the most likely transitions in the system without and with symmetry, respectively. The sequence of critical timescales includes the subsequence of the reciprocals of the real parts of eigenvalues. Under a certain assumption, we prove sharp asymptotic estimates for eigenvalues (including pre-factors) and show how one can extract them from the output of Algorithm 1. We discuss the relationship between Algorithms 1 and 2 and explain how one needs to interpret the output of Algorithm 1 if it is applied in the case with symmetry instead of Algorithm 2. Finally, we analyze an example motivated by R. D. Astumian's model of the dynamics of kinesin, a molecular motor, by means of Algorithm 2.

  5. Some Aspects of Modeling Dependence in Copula-based Markov chains

    Longla, Martial

    2011-01-01

    Dependence coefficients have been widely studied for Markov processes defined by a set of transition probabilities and an initial distribution. This work clarifies some aspects of the theory of dependence structure of Markov chains generated by copulas that are useful in time series econometrics and other applied fields. The main aim of this paper is to clarify the relationship between the notions of geometric ergodicity and geometric {\\rho}-mixing; namely, to point out that for a large number of well known copulas, such as Clayton, Gumbel or Student, these notions are equivalent. Some of the results published in the last years appear to be redundant if one takes into account this fact. We apply this equivalence to show that any mixture of Clayton, Gumbel or Student copulas generate both geometrically ergodic and geometric {\\rho}-mixing stationary Markov chains, answering in this way an open question in the literature. We shall also point out that a sufficient condition for {\\rho}-mixing, used in the literatu...

  6. Animal vocal sequences: not the Markov chains we thought they were.

    Kershenbaum, Arik; Bowles, Ann E; Freeberg, Todd M; Jin, Dezhe Z; Lameira, Adriano R; Bohn, Kirsten

    2014-10-07

    Many animals produce vocal sequences that appear complex. Most researchers assume that these sequences are well characterized as Markov chains (i.e. that the probability of a particular vocal element can be calculated from the history of only a finite number of preceding elements). However, this assumption has never been explicitly tested. Furthermore, it is unclear how language could evolve in a single step from a Markovian origin, as is frequently assumed, as no intermediate forms have been found between animal communication and human language. Here, we assess whether animal taxa produce vocal sequences that are better described by Markov chains, or by non-Markovian dynamics such as the 'renewal process' (RP), characterized by a strong tendency to repeat elements. We examined vocal sequences of seven taxa: Bengalese finches Lonchura striata domestica, Carolina chickadees Poecile carolinensis, free-tailed bats Tadarida brasiliensis, rock hyraxes Procavia capensis, pilot whales Globicephala macrorhynchus, killer whales Orcinus orca and orangutans Pongo spp. The vocal systems of most of these species are more consistent with a non-Markovian RP than with the Markovian models traditionally assumed. Our data suggest that non-Markovian vocal sequences may be more common than Markov sequences, which must be taken into account when evaluating alternative hypotheses for the evolution of signalling complexity, and perhaps human language origins.

  7. Markov chain model helps predict pitting corrosion depth and rate in underground pipelines

    Caleyo, F.; Velazquez, J.C.; Hallen, J. M. [ESIQIE, Instituto Politecnico Nacional, Mexico D. F. (Mexico); Esquivel-Amezcua, A. [PEMEX PEP Region Sur, Villahermosa, Tabasco (Mexico); Valor, A. [Universidad de la Habana, Vedado, La Habana (Cuba)

    2010-07-01

    Recent reports place pipeline corrosion costs in North America at seven billion dollars per year. Pitting corrosion causes the higher percentage of failures among other corrosion mechanisms. This has motivated multiple modelling studies to be focused on corrosion pitting of underground pipelines. In this study, a continuous-time, non-homogenous pure birth Markov chain serves to model external pitting corrosion in buried pipelines. The analytical solution of Kolmogorov's forward equations for this type of Markov process gives the transition probability function in a discrete space of pit depths. The transition probability function can be completely identified by making a correlation between the stochastic pit depth mean and the deterministic mean obtained experimentally. The model proposed in this study can be applied to pitting corrosion data from repeated in-line pipeline inspections. Case studies presented in this work show how pipeline inspection and maintenance planning can be improved by using the proposed Markovian model for pitting corrosion.

  8. Analysis on the Spatial-Temporal Dynamics of Financial Agglomeration with Markov Chain Approach in China

    Weimin Chen

    2014-01-01

    Full Text Available The standard approach to studying financial industrial agglomeration is to construct measures of the degree of agglomeration within financial industry. But such measures often fail to exploit the convergence or divergence of financial agglomeration. In this paper, we apply Markov chain approach to diagnose the convergence of financial agglomeration in China based on the location quotient coefficients across the provincial regions over 1993–2011. The estimation of Markov transition probability matrix offers more detailed insights into the mechanics of financial agglomeration evolution process in China during the research period. The results show that the spatial evolution of financial agglomeration changes faster in the period of 2003–2011 than that in the period of 1993–2002. Furthermore, there exists a very uneven financial development patterns, but there is regional convergence for financial agglomeration in China.

  9. Ancestry inference in complex admixtures via variable-length Markov chain linkage models.

    Rodriguez, Jesse M; Bercovici, Sivan; Elmore, Megan; Batzoglou, Serafim

    2013-03-01

    Inferring the ancestral origin of chromosomal segments in admixed individuals is key for genetic applications, ranging from analyzing population demographics and history, to mapping disease genes. Previous methods addressed ancestry inference by using either weak models of linkage disequilibrium, or large models that make explicit use of ancestral haplotypes. In this paper we introduce ALLOY, an efficient method that incorporates generalized, but highly expressive, linkage disequilibrium models. ALLOY applies a factorial hidden Markov model to capture the parallel process producing the maternal and paternal admixed haplotypes, and models the background linkage disequilibrium in the ancestral populations via an inhomogeneous variable-length Markov chain. We test ALLOY in a broad range of scenarios ranging from recent to ancient admixtures with up to four ancestral populations. We show that ALLOY outperforms the previous state of the art, and is robust to uncertainties in model parameters.

  10. Markov chain modeling of evolution of strains in reinforced concrete flexural beams

    Anoop, M. B.

    2012-09-01

    Full Text Available From the analysis of experimentally observed variations in surface strains with loading in reinforced concrete beams, it is noted that there is a need to consider the evolution of strains (with loading as a stochastic process. Use of Markov Chains for modeling stochastic evolution of strains with loading in reinforced concrete flexural beams is studied in this paper. A simple, yet practically useful, bi-level homogeneous Gaussian Markov Chain (BLHGMC model is proposed for determining the state of strain in reinforced concrete beams. The BLHGMC model will be useful for predicting behavior/response of reinforced concrete beams leading to more rational design.A través del análisis de la evolución de la deformación superficial observada experimentalmente en vigas de hormigón armado al entrar en carga, se constata que dicho proceso debe considerarse estocástico. En este trabajo se estudia la utilización de cadenas de Markov para modelizar la evolución estocástica de la deformación de vigas flexotraccionadas. Se propone, para establecer el estado de deformación de estas, un modelo con distribución gaussiana tipo cadena de Markov homogénea de dos niveles (BLHGMC por sus siglas en inglés, cuyo empleo resulta sencillo y práctico. Se comprueba la utilidad del modelo BLHGMC para prever el comportamiento de estos elementos, lo que determina a su vez una mayor racionalidad a la hora de su cálculo y diseño

  11. Performance evaluation of Warshall algorithm and dynamic programming for Markov chain in local sequence alignment.

    Khan, Mohammad Ibrahim; Kamal, Md Sarwar

    2015-03-01

    Markov Chain is very effective in prediction basically in long data set. In DNA sequencing it is always very important to find the existence of certain nucleotides based on the previous history of the data set. We imposed the Chapman Kolmogorov equation to accomplish the task of Markov Chain. Chapman Kolmogorov equation is the key to help the address the proper places of the DNA chain and this is very powerful tools in mathematics as well as in any other prediction based research. It incorporates the score of DNA sequences calculated by various techniques. Our research utilize the fundamentals of Warshall Algorithm (WA) and Dynamic Programming (DP) to measures the score of DNA segments. The outcomes of the experiment are that Warshall Algorithm is good for small DNA sequences on the other hand Dynamic Programming are good for long DNA sequences. On the top of above findings, it is very important to measure the risk factors of local sequencing during the matching of local sequence alignments whatever the length.

  12. Markov chain Monte Carlo based analysis of post-translationally modified VDAC gating kinetics.

    Tewari, Shivendra G; Zhou, Yifan; Otto, Bradley J; Dash, Ranjan K; Kwok, Wai-Meng; Beard, Daniel A

    2014-01-01

    The voltage-dependent anion channel (VDAC) is the main conduit for permeation of solutes (including nucleotides and metabolites) of up to 5 kDa across the mitochondrial outer membrane (MOM). Recent studies suggest that VDAC activity is regulated via post-translational modifications (PTMs). Yet the nature and effect of these modifications is not understood. Herein, single channel currents of wild-type, nitrosated, and phosphorylated VDAC are analyzed using a generalized continuous-time Markov chain Monte Carlo (MCMC) method. This developed method describes three distinct conducting states (open, half-open, and closed) of VDAC activity. Lipid bilayer experiments are also performed to record single VDAC activity under un-phosphorylated and phosphorylated conditions, and are analyzed using the developed stochastic search method. Experimental data show significant alteration in VDAC gating kinetics and conductance as a result of PTMs. The effect of PTMs on VDAC kinetics is captured in the parameters associated with the identified Markov model. Stationary distributions of the Markov model suggest that nitrosation of VDAC not only decreased its conductance but also significantly locked VDAC in a closed state. On the other hand, stationary distributions of the model associated with un-phosphorylated and phosphorylated VDAC suggest a reversal in channel conformation from relatively closed state to an open state. Model analyses of the nitrosated data suggest that faster reaction of nitric oxide with Cys-127 thiol group might be responsible for the biphasic effect of nitric oxide on basal VDAC conductance.

  13. Reliability measures for indexed semi-Markov chains applied to wind energy production

    D'Amico, Guglielmo; Prattico, Flavio

    2013-01-01

    The computation of the dependability measures is a crucial point in the planning and development of a wind farm. In this paper we address the issue of energy production by wind turbine by using an indexed semi-Markov chain as a model of wind speed. We present the mathematical model, we describe the data and technical characteristics of a commercial wind turbine (Aircon HAWT-10kW). We show how to compute some of the main dependability measures such as reliability, availability and maintainability functions. We compare the results of the model with real energy production obtained from data available in the Lastem station (Italy) and sampled every 10 minutes.

  14. Markov chain Monte Carlo methods for statistical analysis of RF photonic devices.

    Piels, Molly; Zibar, Darko

    2016-02-08

    The microwave reflection coefficient is commonly used to characterize the impedance of high-speed optoelectronic devices. Error and uncertainty in equivalent circuit parameters measured using this data are systematically evaluated. The commonly used nonlinear least-squares method for estimating uncertainty is shown to give unsatisfactory and incorrect results due to the nonlinear relationship between the circuit parameters and the measured data. Markov chain Monte Carlo methods are shown to provide superior results, both for individual devices and for assessing within-die variation.

  15. A new method based on Markov chains for deriving SB2 orbits directly from their spectra

    Salomon, J -B; Guillout, P; Halbwachs, J -L; Arenou, F; Famaey, B; Lebreton, Y; Mazeh, T; Pourbaix, D; Tal-Or, L

    2012-01-01

    We present a new method to derive orbital elements of double-lined spectroscopic binaries (SB2). The aim is to have accurate orbital parameters of a selection of SB2 in order to prepare the exploitation of astrometric Gaia observations. Combined with our results, they should allow one to measure the mass of each star with a precision of better than 1%. The new method presented here consists of using the spectra at all epochs simultaneously to derive the orbital elements without templates. It is based on a Markov chain including a new method for disentangling the spectra.

  16. Network reconstruction by stationary distribution data of Markov chains based on correlation analysis

    He, Zhe; Wang, Bing-Hong

    2014-01-01

    We propose a new method for network reconstruction by the stationary distribution data of Markov chains on this network. Our method has the merits that: the data we need are much few than most method and need not defer to the time order, and we do not need the input data. We define some criterions to measure the efficacy and the simulation results on several networks, including computer-generated networks and real networks, indicate our method works well. The method consist of two procedures, fist, reconstruct degree sequence, second, reconstruct the network(or edges). And we test the efficacy of each procedure.

  17. Combined constraints on modified Chaplygin gas model from cosmological observed data: Markov Chain Monte Carlo approach

    Lu, Jianbo; Xu, Lixin; Wu, Yabo; Liu, Molin

    2011-01-01

    We use the Markov Chain Monte Carlo method to investigate a global constraints on the modified Chaplygin gas (MCG) model as the unification of dark matter and dark energy from the latest observational data: the Union2 dataset of type supernovae Ia (SNIa), the observational Hubble data (OHD), the cluster X-ray gas mass fraction, the baryon acoustic oscillation (BAO), and the cosmic microwave background (CMB) data. In a flat universe, the constraint results for MCG model are, $\\Omega_{b}h^{2}=0...

  18. Exponential approximation for the nearly critical Galton-Watson process and occupation times of Markov chains

    Peköz, E

    2010-01-01

    In this article we provide new applications for exponential approximation using the framework of Pek\\"oz and R\\"ollin (in press), which is based on Stein's method. We give error bounds for the nearly critical Galton-Watson process conditioned on non-extinction, and for the occupation times of Markov chains; for the latter, in particular, we give a new exponential approximation rate for the number of revisits to the origin for general two dimensional random walk, also known as the Erd\\H{o}s-Taylor theorem.

  19. Testing Homogeneity of Mixture of Skew-normal Distributions Via Markov Chain Monte Carlo Simulation

    Rahman Farnoosh Morteza Ebrahimi

    2015-05-01

    Full Text Available The main purpose of this study is to intoduce an optimal penalty function for testing homogeneity of finite mixture of skew-normal distribution based on Markov Chain Monte Carlo (MCMC simulation. In the present study the penalty function is considered as a parametric function in term of parameter of mixture models and a Baysian approach is employed to estimating the parameters of model. In order to examine the efficiency of the present study in comparison with the previous approaches, some simulation studies are presented.

  20. Uncertainty assessment of integrated distributed hydrological models using GLUE with Markov chain Monte Carlo sampling

    Blasone, Roberta-Serena; Madsen, Henrik; Rosbjerg, Dan

    2008-01-01

    uncertainty estimation (GLUE) procedure based on Markov chain Monte Carlo sampling is applied in order to improve the performance of the methodology in estimating parameters and posterior output distributions. The description of the spatial variations of the hydrological processes is accounted for by defining...... the identifiability of the parameters and results in satisfactory multi-variable simulations and uncertainty estimates. However, the parameter uncertainty alone cannot explain the total uncertainty at all the sites, due to limitations in the distributed data included in the model calibration. The study also indicates...

  1. A multi-level solution algorithm for steady-state Markov chains

    Horton, Graham; Leutenegger, Scott T.

    1993-01-01

    A new iterative algorithm, the multi-level algorithm, for the numerical solution of steady state Markov chains is presented. The method utilizes a set of recursively coarsened representations of the original system to achieve accelerated convergence. It is motivated by multigrid methods, which are widely used for fast solution of partial differential equations. Initial results of numerical experiments are reported, showing significant reductions in computation time, often an order of magnitude or more, relative to the Gauss-Seidel and optimal SOR algorithms for a variety of test problems. The multi-level method is compared and contrasted with the iterative aggregation-disaggregation algorithm of Takahashi.

  2. A Markov chain technique for determining the acquisition behavior of a digital tracking loop

    Chadwick, H. D.

    1972-01-01

    An iterative procedure is presented for determining the acquisition behavior of discrete or digital implementations of a tracking loop. The technique is based on the theory of Markov chains and provides the cumulative probability of acquisition in the loop as a function of time in the presence of noise and a given set of initial condition probabilities. A digital second-order tracking loop to be used in the Viking command receiver for continuous tracking of the command subcarrier phase was analyzed using this technique, and the results agree closely with experimental data.

  3. Analysis of Streamline Separation at Infinity Using Time-Discrete Markov Chains.

    Reich, W; Scheuermann, G

    2012-12-01

    Existing methods for analyzing separation of streamlines are often restricted to a finite time or a local area. In our paper we introduce a new method that complements them by allowing an infinite-time-evaluation of steady planar vector fields. Our algorithm unifies combinatorial and probabilistic methods and introduces the concept of separation in time-discrete Markov-Chains. We compute particle distributions instead of the streamlines of single particles. We encode the flow into a map and then into a transition matrix for each time direction. Finally, we compare the results of our grid-independent algorithm to the popular Finite-Time-Lyapunov-Exponents and discuss the discrepancies.

  4. On the Imbedding Problem for Three-state Time Homogeneous Markov Chains with Coinciding Negative Eigenvalues

    Chen, Yong

    2010-01-01

    For an indecomposable $3\\times 3$ stochastic matrix (i.e., 1-step transition probability matrix) with coinciding negative eigenvalues, a new necessary and sufficient condition of the imbedding problem for time homogeneous Markov chains is shown by means of an alternate parameterization of the transition rate matrix (i.e., intensity matrix, infinitesimal generator), which avoids calculating matrix logarithm or matrix square root. In addition, an implicit description of the imbedding problem for the $3\\times 3$ stochastic matrix in Johansen [J. Lond. Math. Soc., 8, 345-351. (1974)] is pointed out.

  5. De-interlacing using nonlocal costs and Markov-chain-based estimation of interpolation methods.

    Vedadi, Farhang; Shirani, Shahram

    2013-04-01

    A new method of de-interlacing is proposed. De-interlacing is revisited as the problem of assigning a sequence of interpolation methods (interpolators) to a sequence of missing pixels of an interlaced frame (field). With this assumption, our de-interlacing algorithm (de-interlacer), undergoes transitions from one interpolation method to another, as it moves from one missing pixel position to the horizontally adjacent missing pixel position in a missing row of a field. We assume a discrete countable-state Markov-chain model on the sequence of interpolators (Markov-chain states) which are selected from a user-defined set of candidate interpolators. An estimation of the optimum sequence of interpolators with the aforementioned Markov-chain model requires the definition of an efficient cost function as well as a global optimization technique. Our algorithm introduces for the first time using a nonlocal cost (NLC) scheme. The proposed algorithm uses the NLC to not only measure the fitness of an interpolator at a missing pixel position, but also to derive an approximation for transition matrix (TM) of the Markov-chain of interpolators. The TM in our algorithm is a frame-variate matrix, i.e., the algorithm updates the TM for each frame automatically. The algorithm finally uses a Viterbi algorithm to find the global optimum sequence of interpolators given the cost function defined and neighboring original pixels in hand. Next, we introduce a new MAP-based formulation for the estimation of the sequence of interpolators this time not by estimating the best sequence of interpolators but by successive estimations of the best interpolator at each missing pixel using Forward-Backward algorithm. Simulation results prove that, while competitive with each other on different test sequences, the proposed methods (one using Viterbi and the other Forward-Backward algorithm) are superior to state-of-the-art de-interlacing algorithms proposed recently. Finally, we propose motion

  6. Comparison of methods for calculating conditional expectations of sufficient statistics for continuous time Markov chains

    Tataru, Paula Cristina; Hobolth, Asger

    2011-01-01

    past evolutionary events (exact times and types of changes) are unaccessible and the past must be inferred from DNA sequence data observed in the present. RESULTS: We describe and implement three algorithms for computing linear combinations of expected values of the sufficient statistics, conditioned......BACKGROUND: Continuous time Markov chains (CTMCs) is a widely used model for describing the evolution of DNA sequences on the nucleotide, amino acid or codon level. The sufficient statistics for CTMCs are the time spent in a state and the number of changes between any two states. In applications...

  7. Fitting Spectral Energy Distributions of AGN - A Markov Chain Monte Carlo Approach

    Rivera, Gabriela Calistro; Hennawi, Joseph F; Hogg, David W

    2014-01-01

    We present AGNfitter: a Markov Chain Monte Carlo algorithm developed to fit the spectral energy distributions (SEDs) of active galactic nuclei (AGN) with different physical models of AGN components. This code is well suited to determine in a robust way multiple parameters and their uncertainties, which quantify the physical processes responsible for the panchromatic nature of active galaxies and quasars. We describe the technicalities of the code and test its capabilities in the context of X-ray selected obscured AGN using multiwavelength data from the XMM-COSMOS survey.

  8. Monte Carlo estimation of total variation distance of Markov chains on large spaces, with application to phylogenetics.

    Herbei, Radu; Kubatko, Laura

    2013-03-26

    Markov chains are widely used for modeling in many areas of molecular biology and genetics. As the complexity of such models advances, it becomes increasingly important to assess the rate at which a Markov chain converges to its stationary distribution in order to carry out accurate inference. A common measure of convergence to the stationary distribution is the total variation distance, but this measure can be difficult to compute when the state space of the chain is large. We propose a Monte Carlo method to estimate the total variation distance that can be applied in this situation, and we demonstrate how the method can be efficiently implemented by taking advantage of GPU computing techniques. We apply the method to two Markov chains on the space of phylogenetic trees, and discuss the implications of our findings for the development of algorithms for phylogenetic inference.

  9. Quasi-stationary distributions of a pair of Markov chains related to time evolution of a DNA locus

    Bobrowski, A.

    2004-01-01

    We consider a pair of Markov chains representing statistics of the Fisher-Wright-Moran model with mutations and drift. The chains have absorbing state at 0 and are related by the fact that some random time τ ago they were identical, evolving as a single Markov chain with values in {0,1,...}; from that time on they began to evolve independently, conditional on a state at the time of split, according to the same transition probabilities. The distribution of τ is a function ...

  10. A GM (1, 1 Markov Chain-Based Aeroengine Performance Degradation Forecast Approach Using Exhaust Gas Temperature

    Ning-bo Zhao

    2014-01-01

    Full Text Available Performance degradation forecast technology for quantitatively assessing degradation states of aeroengine using exhaust gas temperature is an important technology in the aeroengine health management. In this paper, a GM (1, 1 Markov chain-based approach is introduced to forecast exhaust gas temperature by taking the advantages of GM (1, 1 model in time series and the advantages of Markov chain model in dealing with highly nonlinear and stochastic data caused by uncertain factors. In this approach, firstly, the GM (1, 1 model is used to forecast the trend by using limited data samples. Then, Markov chain model is integrated into GM (1, 1 model in order to enhance the forecast performance, which can solve the influence of random fluctuation data on forecasting accuracy and achieving an accurate estimate of the nonlinear forecast. As an example, the historical monitoring data of exhaust gas temperature from CFM56 aeroengine of China Southern is used to verify the forecast performance of the GM (1, 1 Markov chain model. The results show that the GM (1, 1 Markov chain model is able to forecast exhaust gas temperature accurately, which can effectively reflect the random fluctuation characteristics of exhaust gas temperature changes over time.

  11. A new grey forecasting model based on BP neural network and Markov chain

    2007-01-01

    A new grey forecasting model based on BP neural network and Markov chain was proposed. In order to combine the grey forecasting model with neural network, an important theorem that the grey differential equation is equivalent to the time response model, was proved by analyzing the features of grey forecasting model(GM(1,1)). Based on this, the differential equation parameters were included in the network when the BP neural network was constructed, and the neural network was trained by extracting samples from grey system's known data. When BP network was converged, the whitened grey differential equation parameters were extracted and then the grey neural network forecasting model (GNNM(1,1)) was built. In order to reduce stochastic phenomenon in GNNM(1,1), the state transition probability between two states was defined and the Markov transition matrix was established by building the residual sequences between grey forecasting and actual value. Thus, the new grey forecasting model(MNNGM(1,1)) was proposed by combining Markov chain with GNNM(1,1). Based on the above discussion, three different approaches were put forward for forecasting China electricity demands. By comparing GM(1,1) and GNNM(1,1) with the proposed model, the results indicate that the absolute mean error of MNNGM(1,1) is about 0.4 times of GNNM(1,1) and 0.2 times of GM(1,1), and the mean square error of MNNGM(1,1) is about 0.25 times of GNNM(1,1) and 0.1 times of GM(1,1).

  12. Markov Chain Modelling of Reliability Analysis and Prediction under Mixed Mode Loading

    SINGH Salvinder; ABDULLAH Shahrum; NIK MOHAMED Nik Abdullah; MOHD NOORANI Mohd Salmi

    2015-01-01

    The reliability assessment for an automobile crankshaft provides an important understanding in dealing with the design life of the component in order to eliminate or reduce the likelihood of failure and safety risks. The failures of the crankshafts are considered as a catastrophic failure that leads towards a severe failure of the engine block and its other connecting subcomponents. The reliability of an automotive crankshaft under mixed mode loading using the Markov Chain Model is studied. The Markov Chain is modelled by using a two-state condition to represent the bending and torsion loads that would occur on the crankshaft. The automotive crankshaft represents a good case study of a component under mixed mode loading due to the rotating bending and torsion stresses. An estimation of the Weibull shape parameter is used to obtain the probability density function, cumulative distribution function, hazard and reliability rate functions, the bathtub curve and the mean time to failure. The various properties of the shape parameter is used to model the failure characteristic through the bathtub curve is shown. Likewise, an understanding of the patterns posed by the hazard rate onto the component can be used to improve the design and increase the life cycle based on the reliability and dependability of the component. The proposed reliability assessment provides an accurate, efficient, fast and cost effective reliability analysis in contrast to costly and lengthy experimental techniques.

  13. Controlling influenza disease: Comparison between discrete time Markov chain and deterministic model

    Novkaniza, F.; Ivana, Aldila, D.

    2016-04-01

    Mathematical model of respiratory diseases spread with Discrete Time Markov Chain (DTMC) and deterministic approach for constant total population size are analyzed and compared in this article. Intervention of medical treatment and use of medical mask included in to the model as a constant parameter to controlling influenza spreads. Equilibrium points and basic reproductive ratio as the endemic criteria and it level set depend on some variable are given analytically and numerically as a results from deterministic model analysis. Assuming total of human population is constant from deterministic model, number of infected people also analyzed with Discrete Time Markov Chain (DTMC) model. Since Δt → 0, we could assume that total number of infected people might change only from i to i + 1, i - 1, or i. Approximation probability of an outbreak with gambler's ruin problem will be presented. We find that no matter value of basic reproductive ℛ0, either its larger than one or smaller than one, number of infection will always tends to 0 for t → ∞. Some numerical simulation to compare between deterministic and DTMC approach is given to give a better interpretation and a better understanding about the models results.

  14. Variance-reduced simulation of lattice discrete-time Markov chains with applications in reaction networks

    Maginnis, P. A.; West, M.; Dullerud, G. E.

    2016-10-01

    We propose an algorithm to accelerate Monte Carlo simulation for a broad class of stochastic processes. Specifically, the class of countable-state, discrete-time Markov chains driven by additive Poisson noise, or lattice discrete-time Markov chains. In particular, this class includes simulation of reaction networks via the tau-leaping algorithm. To produce the speedup, we simulate pairs of fair-draw trajectories that are negatively correlated. Thus, when averaged, these paths produce an unbiased Monte Carlo estimator that has reduced variance and, therefore, reduced error. Numerical results for three example systems included in this work demonstrate two to four orders of magnitude reduction of mean-square error. The numerical examples were chosen to illustrate different application areas and levels of system complexity. The areas are: gene expression (affine state-dependent rates), aerosol particle coagulation with emission and human immunodeficiency virus infection (both with nonlinear state-dependent rates). Our algorithm views the system dynamics as a "black-box", i.e., we only require control of pseudorandom number generator inputs. As a result, typical codes can be retrofitted with our algorithm using only minor changes. We prove several analytical results. Among these, we characterize the relationship of covariances between paths in the general nonlinear state-dependent intensity rates case, and we prove variance reduction of mean estimators in the special case of affine intensity rates.

  15. Testing and Evaluation for Web Usability Based on Extended Markov Chain Model

    MAO Cheng-ying; LU Yan-sheng

    2004-01-01

    As the increasing popularity and complexity of Web applications and the emergence of their new characteristics, the testing and maintenance of large, complex Web applications are becoming more complex and difficult.Web applications generally contain lots of pages and are used by enormous users.Statistical testing is an effective way of ensuring their quality.Web usage can be accurately described by Markov chain which has been proved to be an ideal model for software statistical testing.The results of unit testing can be utilized in the latter stages, which is an important strategy for bottom-to-top integration testing, and the other improvement of extended Markov chain model (EMM) is to present the error type vector which is treated as a part of page node.This paper also proposes the algorithm for generating test cases of usage paths.Finally, optional usage reliability evaluation methods and an incremental usability regression testing model for testing and evaluation are presented.

  16. Population synthesis of radio and gamma-ray millisecond pulsars using Markov Chain Monte Carlo techniques

    Gonthier, Peter L.; Koh, Yew-Meng; Kust Harding, Alice

    2016-04-01

    We present preliminary results of a new population synthesis of millisecond pulsars (MSP) from the Galactic disk using Markov Chain Monte Carlo techniques to better understand the model parameter space. We include empirical radio and gamma-ray luminosity models that are dependent on the pulsar period and period derivative with freely varying exponents. The magnitudes of the model luminosities are adjusted to reproduce the number of MSPs detected by a group of thirteen radio surveys as well as the MSP birth rate in the Galaxy and the number of MSPs detected by Fermi. We explore various high-energy emission geometries like the slot gap, outer gap, two pole caustic and pair starved polar cap models. The parameters associated with the birth distributions for the mass accretion rate, magnetic field, and period distributions are well constrained. With the set of four free parameters, we employ Markov Chain Monte Carlo simulations to explore the model parameter space. We present preliminary comparisons of the simulated and detected distributions of radio and gamma-ray pulsar characteristics. We estimate the contribution of MSPs to the diffuse gamma-ray background with a special focus on the Galactic Center.We express our gratitude for the generous support of the National Science Foundation (RUI: AST-1009731), Fermi Guest Investigator Program and the NASA Astrophysics Theory and Fundamental Program (NNX09AQ71G).

  17. Markov-chain model of classified atomistic transition states for discrete kinetic Monte Carlo simulations.

    Numazawa, Satoshi; Smith, Roger

    2011-10-01

    Classical harmonic transition state theory is considered and applied in discrete lattice cells with hierarchical transition levels. The scheme is then used to determine transitions that can be applied in a lattice-based kinetic Monte Carlo (KMC) atomistic simulation model. The model results in an effective reduction of KMC simulation steps by utilizing a classification scheme of transition levels for thermally activated atomistic diffusion processes. Thermally activated atomistic movements are considered as local transition events constrained in potential energy wells over certain local time periods. These processes are represented by Markov chains of multidimensional Boolean valued functions in three-dimensional lattice space. The events inhibited by the barriers under a certain level are regarded as thermal fluctuations of the canonical ensemble and accepted freely. Consequently, the fluctuating system evolution process is implemented as a Markov chain of equivalence class objects. It is shown that the process can be characterized by the acceptance of metastable local transitions. The method is applied to a problem of Au and Ag cluster growth on a rippled surface. The simulation predicts the existence of a morphology-dependent transition time limit from a local metastable to stable state for subsequent cluster growth by accretion. Excellent agreement with observed experimental results is obtained.

  18. Markov Chain Monte Carlo simulation for projection of end stage renal disease patients in Greece.

    Rodina-Theocharaki, A; Bliznakova, K; Pallikarakis, N

    2012-07-01

    End stage renal disease (ESRD) treatment methods are considered to be among the most expensive procedures for chronic conditions worldwide which also have severe impact on patients' quality of life. During the last decade, Greece has been among the countries with the highest incidence and prevalence, while at the same time with the lowest kidney transplantation rates. Predicting future patients' number on Renal Replacement Therapy (RRT) is essential for health care providers in order to achieve more effective resource management. In this study a Markov Chain Monte Carlo (MCMC) simulation is presented for predicting the future number of ESRD patients for the period 2009-2020 in Greece. The MCMC model comprises Monte Carlo sampling techniques applied on probability distributions of the constructed Markov Chain. The model predicts that there will be 15,147 prevalent patients on RRT in Greece by 2020. Additionally, a cost-effectiveness analysis was performed on a scenario of gradually reducing the hemodialysis patients in favor of increasing the transplantation number by 2020. The proposed scenario showed net savings of 86.54 million Euros for the period 2009-2020 compared to the base-case prediction.

  19. Detection of dispersed short tandem repeats using reversible jump Markov chain Monte Carlo.

    Liang, Tong; Fan, Xiaodan; Li, Qiwei; Li, Shuo-Yen R

    2012-10-01

    Tandem repeats occur frequently in biological sequences. They are important for studying genome evolution and human disease. A number of methods have been designed to detect a single tandem repeat in a sliding window. In this article, we focus on the case that an unknown number of tandem repeat segments of the same pattern are dispersively distributed in a sequence. We construct a probabilistic generative model for the tandem repeats, where the sequence pattern is represented by a motif matrix. A Bayesian approach is adopted to compute this model. Markov chain Monte Carlo (MCMC) algorithms are used to explore the posterior distribution as an effort to infer both the motif matrix of tandem repeats and the location of repeat segments. Reversible jump Markov chain Monte Carlo (RJMCMC) algorithms are used to address the transdimensional model selection problem raised by the variable number of repeat segments. Experiments on both synthetic data and real data show that this new approach is powerful in detecting dispersed short tandem repeats. As far as we know, it is the first work to adopt RJMCMC algorithms in the detection of tandem repeats.

  20. Predicting seasonal fate of phenanthrene in aquatic environment with a Markov chain.

    Sun, Caiyun; Ma, Qiyun; Zhang, Jiquan; Zhou, Mo; Chen, Yanan

    2016-08-01

    Phenanthrene (Phe) with carcinogenicity is ubiquitous in the environment, especially in aquatic environment; its toxicity is greater. To help determine toxicity risk and remediation strategies, this study predicted seasonal fate of Phe in aquatic environment. Candidate mechanisms including biodegradation, sorption, desorption, photodegradation, hydrolysis and volatility were studied; the results for experiments under simulated conditions for normal, wet and dry seasons in the Yinma River Basin indicated that biodegradation in sediment, sorption, desorption, and volatility were important pathways for elimination of Phe from aquatic environment and showed seasonal variations. A microcosm which was used to mimic sediment/water system was set up to illustrate seasonal distribution and transport of Phe. A Markov chain was applied to predict seasonal fate of Phe in air/water/sediment environment, the predicted results were perfectly agreed with results of microcosm experiments. Predicted results with a Markov chain suggested that volatility and biodegradation in sediment were main elimination pathways, and contributions of elimination pathways showed seasonal variations; Phe was eliminated from water and sediment to negligible levels over around 250 h in August and over 1000 h in May; in November, Phe was eliminated from water to a negligible level while about 31 % of Phe amount still remained in sediment over 1000 h.

  1. Comparing variational Bayes with Markov chain Monte Carlo for Bayesian computation in neuroimaging.

    Nathoo, F S; Lesperance, M L; Lawson, A B; Dean, C B

    2013-08-01

    In this article, we consider methods for Bayesian computation within the context of brain imaging studies. In such studies, the complexity of the resulting data often necessitates the use of sophisticated statistical models; however, the large size of these data can pose significant challenges for model fitting. We focus specifically on the neuroelectromagnetic inverse problem in electroencephalography, which involves estimating the neural activity within the brain from electrode-level data measured across the scalp. The relationship between the observed scalp-level data and the unobserved neural activity can be represented through an underdetermined dynamic linear model, and we discuss Bayesian computation for such models, where parameters represent the unknown neural sources of interest. We review the inverse problem and discuss variational approximations for fitting hierarchical models in this context. While variational methods have been widely adopted for model fitting in neuroimaging, they have received very little attention in the statistical literature, where Markov chain Monte Carlo is often used. We derive variational approximations for fitting two models: a simple distributed source model and a more complex spatiotemporal mixture model. We compare the approximations to Markov chain Monte Carlo using both synthetic data as well as through the analysis of a real electroencephalography dataset examining the evoked response related to face perception. The computational advantages of the variational method are demonstrated and the accuracy associated with the resulting approximations are clarified.

  2. Inferring Sequential Order of Somatic Mutations during Tumorgenesis based on Markov Chain Model.

    Kang, Hao; Cho, Kwang-Hyun; Zhang, Xiaohua Douglas; Zeng, Tao; Chen, Luonan

    2015-01-01

    Tumors are developed and worsen with the accumulated mutations on DNA sequences during tumorigenesis. Identifying the temporal order of gene mutations in cancer initiation and development is a challenging topic. It not only provides a new insight into the study of tumorigenesis at the level of genome sequences but also is an effective tool for early diagnosis of tumors and preventive medicine. In this paper, we develop a novel method to accurately estimate the sequential order of gene mutations during tumorigenesis from genome sequencing data based on Markov chain model as TOMC (Temporal Order based on Markov Chain), and also provide a new criterion to further infer the order of samples or patients, which can characterize the severity or stage of the disease. We applied our method to the analysis of tumors based on several high-throughput datasets. Specifically, first, we revealed that tumor suppressor genes (TSG) tend to be mutated ahead of oncogenes, which are considered as important events for key functional loss and gain during tumorigenesis. Second, the comparisons of various methods demonstrated that our approach has clear advantages over the existing methods due to the consideration on the effect of mutation dependence among genes, such as co-mutation. Third and most important, our method is able to deduce the ordinal sequence of patients or samples to quantitatively characterize their severity of tumors. Therefore, our work provides a new way to quantitatively understand the development and progression of tumorigenesis based on high throughput sequencing data.

  3. Spreaders and sponges define metastasis in lung cancer: a Markov chain Monte Carlo mathematical model.

    Newton, Paul K; Mason, Jeremy; Bethel, Kelly; Bazhenova, Lyudmila; Nieva, Jorge; Norton, Larry; Kuhn, Peter

    2013-05-01

    The classic view of metastatic cancer progression is that it is a unidirectional process initiated at the primary tumor site, progressing to variably distant metastatic sites in a fairly predictable, although not perfectly understood, fashion. A Markov chain Monte Carlo mathematical approach can determine a pathway diagram that classifies metastatic tumors as "spreaders" or "sponges" and orders the timescales of progression from site to site. In light of recent experimental evidence highlighting the potential significance of self-seeding of primary tumors, we use a Markov chain Monte Carlo (MCMC) approach, based on large autopsy data sets, to quantify the stochastic, systemic, and often multidirectional aspects of cancer progression. We quantify three types of multidirectional mechanisms of progression: (i) self-seeding of the primary tumor, (ii) reseeding of the primary tumor from a metastatic site (primary reseeding), and (iii) reseeding of metastatic tumors (metastasis reseeding). The model shows that the combined characteristics of the primary and the first metastatic site to which it spreads largely determine the future pathways and timescales of systemic disease.

  4. Effective degree Markov-chain approach for discrete-time epidemic processes on uncorrelated networks.

    Cai, Chao-Ran; Wu, Zhi-Xi; Guan, Jian-Yue

    2014-11-01

    Recently, Gómez et al. proposed a microscopic Markov-chain approach (MMCA) [S. Gómez, J. Gómez-Gardeñes, Y. Moreno, and A. Arenas, Phys. Rev. E 84, 036105 (2011)PLEEE81539-375510.1103/PhysRevE.84.036105] to the discrete-time susceptible-infected-susceptible (SIS) epidemic process and found that the epidemic prevalence obtained by this approach agrees well with that by simulations. However, we found that the approach cannot be straightforwardly extended to a susceptible-infected-recovered (SIR) epidemic process (due to its irreversible property), and the epidemic prevalences obtained by MMCA and Monte Carlo simulations do not match well when the infection probability is just slightly above the epidemic threshold. In this contribution we extend the effective degree Markov-chain approach, proposed for analyzing continuous-time epidemic processes [J. Lindquist, J. Ma, P. Driessche, and F. Willeboordse, J. Math. Biol. 62, 143 (2011)JMBLAJ0303-681210.1007/s00285-010-0331-2], to address discrete-time binary-state (SIS) or three-state (SIR) epidemic processes on uncorrelated complex networks. It is shown that the final epidemic size as well as the time series of infected individuals obtained from this approach agree very well with those by Monte Carlo simulations. Our results are robust to the change of different parameters, including the total population size, the infection probability, the recovery probability, the average degree, and the degree distribution of the underlying networks.

  5. MBMC: An Effective Markov Chain Approach for Binning Metagenomic Reads from Environmental Shotgun Sequencing Projects.

    Wang, Ying; Hu, Haiyan; Li, Xiaoman

    2016-08-01

    Metagenomics is a next-generation omics field currently impacting postgenomic life sciences and medicine. Binning metagenomic reads is essential for the understanding of microbial function, compositions, and interactions in given environments. Despite the existence of dozens of computational methods for metagenomic read binning, it is still very challenging to bin reads. This is especially true for reads from unknown species, from species with similar abundance, and/or from low-abundance species in environmental samples. In this study, we developed a novel taxonomy-dependent and alignment-free approach called MBMC (Metagenomic Binning by Markov Chains). Different from all existing methods, MBMC bins reads by measuring the similarity of reads to the trained Markov chains for different taxa instead of directly comparing reads with known genomic sequences. By testing on more than 24 simulated and experimental datasets with species of similar abundance, species of low abundance, and/or unknown species, we report here that MBMC reliably grouped reads from different species into separate bins. Compared with four existing approaches, we demonstrated that the performance of MBMC was comparable with existing approaches when binning reads from sequenced species, and superior to existing approaches when binning reads from unknown species. MBMC is a pivotal tool for binning metagenomic reads in the current era of Big Data and postgenomic integrative biology. The MBMC software can be freely downloaded at http://hulab.ucf.edu/research/projects/metagenomics/MBMC.html .

  6. Motif finding in DNA sequences based on skipping nonconserved positions in background Markov chains.

    Zhao, Xiaoyan; Sze, Sing-Hoi

    2011-05-01

    One strategy to identify transcription factor binding sites is through motif finding in upstream DNA sequences of potentially co-regulated genes. Despite extensive efforts, none of the existing algorithms perform very well. We consider a string representation that allows arbitrary ignored positions within the nonconserved portion of single motifs, and use O(2(l)) Markov chains to model the background distributions of motifs of length l while skipping these positions within each Markov chain. By focusing initially on positions that have fixed nucleotides to define core occurrences, we develop an algorithm to identify motifs of moderate lengths. We compare the performance of our algorithm to other motif finding algorithms on a few benchmark data sets, and show that significant improvement in accuracy can be obtained when the sites are sufficiently conserved within a given sample, while comparable performance is obtained when the site conservation rate is low. A software program (PosMotif ) and detailed results are available online at http://faculty.cse.tamu.edu/shsze/posmotif.

  7. Trans-dimensional matched-field geoacoustic inversion with hierarchical error models and interacting Markov chains.

    Dettmer, Jan; Dosso, Stan E

    2012-10-01

    This paper develops a trans-dimensional approach to matched-field geoacoustic inversion, including interacting Markov chains to improve efficiency and an autoregressive model to account for correlated errors. The trans-dimensional approach and hierarchical seabed model allows inversion without assuming any particular parametrization by relaxing model specification to a range of plausible seabed models (e.g., in this case, the number of sediment layers is an unknown parameter). Data errors are addressed by sampling statistical error-distribution parameters, including correlated errors (covariance), by applying a hierarchical autoregressive error model. The well-known difficulty of low acceptance rates for trans-dimensional jumps is addressed with interacting Markov chains, resulting in a substantial increase in efficiency. The trans-dimensional seabed model and the hierarchical error model relax the degree of prior assumptions required in the inversion, resulting in substantially improved (more realistic) uncertainty estimates and a more automated algorithm. In particular, the approach gives seabed parameter uncertainty estimates that account for uncertainty due to prior model choice (layering and data error statistics). The approach is applied to data measured on a vertical array in the Mediterranean Sea.

  8. Conditional probability Markov chain simulation based reliability analysis method for nonnormal variables

    2010-01-01

    Based on fast Markov chain simulation for generating the samples distributed in failure region and saddlepoint approximation(SA) technique,an efficient reliability analysis method is presented to evaluate the small failure probability of non-linear limit state function(LSF) with non-normal variables.In the presented method,the failure probability of the non-linear LSF is transformed into a product of the failure probability of the introduced linear LSF and a feature ratio factor.The introduced linear LSF which approximately has the same maximum likelihood points as the non-linear LSF is constructed and its failure probability can be calculated by SA technique.The feature ratio factor,which can be evaluated on the basis of multiplicative rule of probability,exhibits the relation between the failure probability of the non-linear LSF and that of the linear LSF,and it can be fast computed by utilizing the Markov chain algorithm to directly simulate the samples distributed in the failure regions of the non-linear LSF and those of the linear LSF.Moreover,the expectation and variance of the failure probability estimate are derived.The results of several examples demonstrate that the presented method has wide applicability,can be easily implemented,and possesses high precision and high efficiency.

  9. Gallavotti-Cohen-Type symmetry related to cycle decompositions for Markov chains and biochemical applications

    Faggionato, A

    2010-01-01

    We slightly extend the fluctuation theorem obtained in \\cite{LS} for sums of generators, considering continuous-time Markov chains on a finite state space whose underlying graph has multiple edges and no loop. This extended frame is suited when analyzing chemical systems. As simple corollary we derive in a different method the fluctuation theorem of D. Andrieux and P. Gaspard for the fluxes along the chords associated to a fundamental set of oriented cycles \\cite{AG2}. We associate to each random trajectory an oriented cycle on the graph and we decompose it in terms of a basis of oriented cycles. We prove a fluctuation theorem for the coefficients in this decomposition. The resulting fluctuation theorem involves the cycle affinities, which in many real systems correspond to the macroscopic forces. In addition, the above decomposition is useful when analyzing the large deviations of additive functionals of the Markov chain. As example of application, in a very general context we derive a fluctuation relation f...

  10. Gallavotti-Cohen-Type Symmetry Related to Cycle Decompositions for Markov Chains and Biochemical Applications

    Faggionato, Alessandra; di Pietro, Daniele

    2011-04-01

    We slightly extend the fluctuation theorem obtained in (Lebowitz and Spohn in J. Stat. Phys. 95:333-365, 1999) for sums of generators, considering continuous-time Markov chains on a finite state space whose underlying graph has multiple edges and no loop. This extended frame is suited when analyzing chemical systems. As simple corollary we derive by a different method the fluctuation theorem of D. Andrieux and P. Gaspard for the fluxes along the chords associated to a fundamental set of oriented cycles (Andrieux and Gaspard in J. Stat. Phys. 127:107-131, 2007). We associate to each random trajectory an oriented cycle on the graph and we decompose it in terms of a basis of oriented cycles. We prove a fluctuation theorem for the coefficients in this decomposition. The resulting fluctuation theorem involves the cycle affinities, which in many real systems correspond to the macroscopic forces. In addition, the above decomposition is useful when analyzing the large deviations of additive functionals of the Markov chain. As example of application, in a very general context we derive a fluctuation relation for the mechanical and chemical currents of a molecular motor moving along a periodic filament.

  11. Estimating the ratios of the stationary distribution values for Markov chains modeling evolutionary algorithms.

    Mitavskiy, Boris; Cannings, Chris

    2009-01-01

    The evolutionary algorithm stochastic process is well-known to be Markovian. These have been under investigation in much of the theoretical evolutionary computing research. When the mutation rate is positive, the Markov chain modeling of an evolutionary algorithm is irreducible and, therefore, has a unique stationary distribution. Rather little is known about the stationary distribution. In fact, the only quantitative facts established so far tell us that the stationary distributions of Markov chains modeling evolutionary algorithms concentrate on uniform populations (i.e., those populations consisting of a repeated copy of the same individual). At the same time, knowing the stationary distribution may provide some information about the expected time it takes for the algorithm to reach a certain solution, assessment of the biases due to recombination and selection, and is of importance in population genetics to assess what is called a "genetic load" (see the introduction for more details). In the recent joint works of the first author, some bounds have been established on the rates at which the stationary distribution concentrates on the uniform populations. The primary tool used in these papers is the "quotient construction" method. It turns out that the quotient construction method can be exploited to derive much more informative bounds on ratios of the stationary distribution values of various subsets of the state space. In fact, some of the bounds obtained in the current work are expressed in terms of the parameters involved in all the three main stages of an evolutionary algorithm: namely, selection, recombination, and mutation.

  12. Entropy and long-range memory in random symbolic additive Markov chains

    Melnik, S. S.; Usatenko, O. V.

    2016-06-01

    The goal of this paper is to develop an estimate for the entropy of random symbolic sequences with elements belonging to a finite alphabet. As a plausible model, we use the high-order additive stationary ergodic Markov chain with long-range memory. Supposing that the correlations between random elements of the chain are weak, we express the conditional entropy of the sequence by means of the symbolic pair correlation function. We also examine an algorithm for estimating the conditional entropy of finite symbolic sequences. We show that the entropy contains two contributions, i.e., the correlation and the fluctuation. The obtained analytical results are used for numerical evaluation of the entropy of written English texts and DNA nucleotide sequences. The developed theory opens the way for constructing a more consistent and sophisticated approach to describe the systems with strong short-range and weak long-range memory.

  13. Bayesian Inference for LISA Pathfinder using Markov Chain Monte Carlo Methods

    Ferraioli, Luigi; Plagnol, Eric

    2012-01-01

    We present a parameter estimation procedure based on a Bayesian framework by applying a Markov Chain Monte Carlo algorithm to the calibration of the dynamical parameters of a space based gravitational wave detector. The method is based on the Metropolis-Hastings algorithm and a two-stage annealing treatment in order to ensure an effective exploration of the parameter space at the beginning of the chain. We compare two versions of the algorithm with an application to a LISA Pathfinder data analysis problem. The two algorithms share the same heating strategy but with one moving in coordinate directions using proposals from a multivariate Gaussian distribution, while the other uses the natural logarithm of some parameters and proposes jumps in the eigen-space of the Fisher Information matrix. The algorithm proposing jumps in the eigen-space of the Fisher Information matrix demonstrates a higher acceptance rate and a slightly better convergence towards the equilibrium parameter distributions in the application to...

  14. Entropy and long-range memory in random symbolic additive Markov chains.

    Melnik, S S; Usatenko, O V

    2016-06-01

    The goal of this paper is to develop an estimate for the entropy of random symbolic sequences with elements belonging to a finite alphabet. As a plausible model, we use the high-order additive stationary ergodic Markov chain with long-range memory. Supposing that the correlations between random elements of the chain are weak, we express the conditional entropy of the sequence by means of the symbolic pair correlation function. We also examine an algorithm for estimating the conditional entropy of finite symbolic sequences. We show that the entropy contains two contributions, i.e., the correlation and the fluctuation. The obtained analytical results are used for numerical evaluation of the entropy of written English texts and DNA nucleotide sequences. The developed theory opens the way for constructing a more consistent and sophisticated approach to describe the systems with strong short-range and weak long-range memory.

  15. On the utility of the multi-level algorithm for the solution of nearly completely decomposable Markov chains

    Leutenegger, Scott T.; Horton, Graham

    1994-01-01

    Recently the Multi-Level algorithm was introduced as a general purpose solver for the solution of steady state Markov chains. In this paper, we consider the performance of the Multi-Level algorithm for solving Nearly Completely Decomposable (NCD) Markov chains, for which special-purpose iteractive aggregation/disaggregation algorithms such as the Koury-McAllister-Stewart (KMS) method have been developed that can exploit the decomposability of the the Markov chain. We present experimental results indicating that the general-purpose Multi-Level algorithm is competitive, and can be significantly faster than the special-purpose KMS algorithm when Gauss-Seidel and Gaussian Elimination are used for solving the individual blocks.

  16. Optimal clinical trial design based on a dichotomous Markov-chain mixed-effect sleep model.

    Steven Ernest, C; Nyberg, Joakim; Karlsson, Mats O; Hooker, Andrew C

    2014-12-01

    D-optimal designs for discrete-type responses have been derived using generalized linear mixed models, simulation based methods and analytical approximations for computing the fisher information matrix (FIM) of non-linear mixed effect models with homogeneous probabilities over time. In this work, D-optimal designs using an analytical approximation of the FIM for a dichotomous, non-homogeneous, Markov-chain phase advanced sleep non-linear mixed effect model was investigated. The non-linear mixed effect model consisted of transition probabilities of dichotomous sleep data estimated as logistic functions using piecewise linear functions. Theoretical linear and nonlinear dose effects were added to the transition probabilities to modify the probability of being in either sleep stage. D-optimal designs were computed by determining an analytical approximation the FIM for each Markov component (one where the previous state was awake and another where the previous state was asleep). Each Markov component FIM was weighted either equally or by the average probability of response being awake or asleep over the night and summed to derive the total FIM (FIM(total)). The reference designs were placebo, 0.1, 1-, 6-, 10- and 20-mg dosing for a 2- to 6-way crossover study in six dosing groups. Optimized design variables were dose and number of subjects in each dose group. The designs were validated using stochastic simulation/re-estimation (SSE). Contrary to expectations, the predicted parameter uncertainty obtained via FIM(total) was larger than the uncertainty in parameter estimates computed by SSE. Nevertheless, the D-optimal designs decreased the uncertainty of parameter estimates relative to the reference designs. Additionally, the improvement for the D-optimal designs were more pronounced using SSE than predicted via FIM(total). Through the use of an approximate analytic solution and weighting schemes, the FIM(total) for a non-homogeneous, dichotomous Markov-chain phase

  17. A Markov Chain-based quantitative study of angular distribution of photons through turbid slabs via isotropic light scattering

    Li, Xuesong; Northrop, William F.

    2016-04-01

    This paper describes a quantitative approach to approximate multiple scattering through an isotropic turbid slab based on Markov Chain theorem. There is an increasing need to utilize multiple scattering for optical diagnostic purposes; however, existing methods are either inaccurate or computationally expensive. Here, we develop a novel Markov Chain approximation approach to solve multiple scattering angular distribution (AD) that can accurately calculate AD while significantly reducing computational cost compared to Monte Carlo simulation. We expect this work to stimulate ongoing multiple scattering research and deterministic reconstruction algorithm development with AD measurements.

  18. Stochastic-shielding approximation of Markov chains and its application to efficiently simulate random ion-channel gating.

    Schmandt, Nicolaus T; Galán, Roberto F

    2012-09-14

    Markov chains provide realistic models of numerous stochastic processes in nature. We demonstrate that in any Markov chain, the change in occupation number in state A is correlated to the change in occupation number in state B if and only if A and B are directly connected. This implies that if we are only interested in state A, fluctuations in B may be replaced with their mean if state B is not directly connected to A, which shortens computing time considerably. We show the accuracy and efficacy of our approximation theoretically and in simulations of stochastic ion-channel gating in neurons.

  19. The Study of Reinforcement Learning for Traffic Self-Adaptive Control under Multiagent Markov Game Environment

    Lun-Hui Xu

    2013-01-01

    Full Text Available Urban traffic self-adaptive control problem is dynamic and uncertain, so the states of traffic environment are hard to be observed. Efficient agent which controls a single intersection can be discovered automatically via multiagent reinforcement learning. However, in the majority of the previous works on this approach, each agent needed perfect observed information when interacting with the environment and learned individually with less efficient coordination. This study casts traffic self-adaptive control as a multiagent Markov game problem. The design employs traffic signal control agent (TSCA for each signalized intersection that coordinates with neighboring TSCAs. A mathematical model for TSCAs’ interaction is built based on nonzero-sum markov game which has been applied to let TSCAs learn how to cooperate. A multiagent Markov game reinforcement learning approach is constructed on the basis of single-agent Q-learning. This method lets each TSCA learn to update its Q-values under the joint actions and imperfect information. The convergence of the proposed algorithm is analyzed theoretically. The simulation results show that the proposed method is convergent and effective in realistic traffic self-adaptive control setting.

  20. Markov chains at the interface of combinatorics, computing, and statistical physics

    Streib, Amanda Pascoe

    The fields of statistical physics, discrete probability, combinatorics, and theoretical computer science have converged around efforts to understand random structures and algorithms. Recent activity in the interface of these fields has enabled tremendous breakthroughs in each domain and has supplied a new set of techniques for researchers approaching related problems. This thesis makes progress on several problems in this interface whose solutions all build on insights from multiple disciplinary perspectives. First, we consider a dynamic growth process arising in the context of DNA-based self-assembly. The assembly process can be modeled as a simple Markov chain. We prove that the chain is rapidly mixing for large enough bias in regions of Zd. The proof uses a geometric distance function and a variant of path coupling in order to handle distances that can be exponentially large. We also provide the first results in the case of fluctuating bias, where the bias can vary depending on the location of the tile, which arises in the nanotechnology application. Moreover, we use intuition from statistical physics to construct a choice of the biases for which the Markov chain Mmon requires exponential time to converge. Second, we consider a related problem regarding the convergence rate of biased permutations that arises in the context of self-organizing lists. The Markov chain Mnn in this case is a nearest-neighbor chain that allows adjacent transpositions, and the rate of these exchanges is governed by various input parameters. It was conjectured that the chain is always rapidly mixing when the inversion probabilities are positively biased, i.e., we put nearest neighbor pair x chain Mmon was known to have connections to a simplified version of this biased card-shuffling. We provide new connections between Mnn and Mmon by using simple combinatorial bijections, and we prove that Mnn is always rapidly mixing for two general classes of positively biased { pxy}. More

  1. Retail Banking Loan Portfolio Equilibrium Mix : A Markov Chain Model Analysis

    V. Thyagarajan

    2005-01-01

    Full Text Available The variance analysis of actual loan sanctions with the non-documented method of loan allocation of the selected retail bank, over a period of 24 months, revealed that there is a scope to improve their income earnings. Realizing its importance Markov Chain Market Share model was applied to inter temporal data of loan disbursements of the selected bank. By applying Estimate Transition Matrix, scope for probability of loan switching among its types was calculated to suggest the probable mix of loan portfolio. From the results it was suggested that the loan proportions among various types were as follows: Housing (32.0 %, Others (28.1 %, Business (20.0 % and Education (19.7 %. These proportions can be taken as guideline percentage within the government norms for the priority sector. Simulation studies were also done to calculate the expected income of interest using Markov proportions and compared with the actual interest earnings to prove the superiority of the model.

  2. Reliability Measures of Second-Order Semi-Markov Chain Applied to Wind Energy Production

    Guglielmo D'Amico

    2013-01-01

    Full Text Available We consider the problem of wind energy production by using a second-order semi-Markov chain in state and duration as a model of wind speed. The model used in this paper is based on our previous work where we have shown the ability of second-order semi-Markov process in reproducing statistical features of wind speed. Here we briefly present the mathematical model and describe the data and technical characteristics of a commercial wind turbine (Aircon HAWT-10 kW. We show how, by using our model, it is possible to compute some of the main dependability measures such as reliability, availability, and maintainability functions. We compare, by means of Monte Carlo simulations, the results of the model with real energy production obtained from data available in the Lastem station (Italy and sampled every 10 minutes. The computation of the dependability measures is a crucial point in the planning and development of a wind farm. Through our model, we show how the values of this quantity can be obtained both analytically and computationally.

  3. Consensus protocol for heterogeneous multi-agent systems:A Markov chain approach

    Zhu Shan-Ying; Chen Cai-Lian; Guan Xin-Ping

    2013-01-01

    This paper deals with the consensus problem for heterogeneous multi-agent systems.Different from most existing consensus protocols,we consider the consensus seeking of two types of agents,namely,active agents and passive agents.The objective is to directly control the active agents such that the states of all the agents would achieve consensus.In order to obtain a computational approach,we subtly introduce an appropriate Markov chain to cast the heterogeneous systems into a unified framework.Such a framework is helpful for tackling the constraints from passive agents.Furthermore,a sufficient and necessary condition is established to guarantee the consensus in heterogeneous multi-agent systems.Finally,simulation results are provided to verify the theoretical analysis and the effectiveness of the proposed protocol.

  4. Study of behavior and determination of customer lifetime value(CLV) using Markov chain model

    Permana, Dony; Indratno, Sapto Wahyu; Pasaribu, Udjianna S.

    2014-03-01

    Customer Lifetime Value or CLV is a restriction on interactive marketing to help a company in arranging financial for the marketing of new customer acquisition and customer retention. Additionally CLV can be able to segment customers for financial arrangements. Stochastic models for the fairly new CLV used a Markov chain. In this model customer retention probability and new customer acquisition probability play an important role. This model is originally introduced by Pfeifer and Carraway in 2000 [1]. They introduced several CLV models, one of them only involves customer and former customer. In this paper we expand the model by adding the assumption of the transition from former customer to customer. In the proposed model, the CLV value is higher than the CLV value obtained by Pfeifer and Caraway model. But our model still requires a longer convergence time.

  5. First Passage Probability Estimation of Wind Turbines by Markov Chain Monte Carlo

    Sichani, Mahdi Teimouri; Nielsen, Søren R.K.

    2013-01-01

    Markov Chain Monte Carlo simulation has received considerable attention within the past decade as reportedly one of the most powerful techniques for the first passage probability estimation of dynamic systems. A very popular method in this direction capable of estimating probability of rare events...... with low computation cost is the subset simulation (SS). The idea of the method is to break a rare event into a sequence of more probable events which are easy to be estimated based on the conditional simulation techniques. Recently, two algorithms have been proposed in order to increase the efficiency...... of the method by modifying the conditional sampler. In this paper, applicability of the original SS is compared to the recently introduced modifications of the method on a wind turbine model. The model incorporates a PID pitch controller which aims at keeping the rotational speed of the wind turbine rotor equal...

  6. A MATLAB Package for Markov Chain Monte Carlo with a Multi-Unidimensional IRT Model

    Yanyan Sheng

    2008-11-01

    Full Text Available Unidimensional item response theory (IRT models are useful when each item is designed to measure some facet of a unified latent trait. In practical applications, items are not necessarily measuring the same underlying trait, and hence the more general multi-unidimensional model should be considered. This paper provides the requisite information and description of software that implements the Gibbs sampler for such models with two item parameters and a normal ogive form. The software developed is written in the MATLAB package IRTmu2no. The package is flexible enough to allow a user the choice to simulate binary response data with multiple dimensions, set the number of total or burn-in iterations, specify starting values or prior distributions for model parameters, check convergence of the Markov chain, as well as obtain Bayesian fit statistics. Illustrative examples are provided to demonstrate and validate the use of the software package.

  7. Towards Analyzing Crossover Operators in Evolutionary Search via General Markov Chain Switching Theorem

    Yu, Yang; Zhou, Zhi-Hua

    2011-01-01

    Evolutionary algorithms (EAs), simulating the evolution process of natural species, are used to solve optimization problems. Crossover (also called recombination), originated from simulating the chromosome exchange phenomena in zoogamy reproduction, is widely employed in EAs to generate offspring solutions, of which the effectiveness has been examined empirically in applications. However, due to the irregularity of crossover operators and the complicated interactions to mutation, crossover operators are hard to analyze and thus have few theoretical results. Therefore, analyzing crossover not only helps in understanding EAs, but also helps in developing novel techniques for analyzing sophisticated metaheuristic algorithms. In this paper, we derive the General Markov Chain Switching Theorem (GMCST) to facilitate theoretical studies of crossover-enabled EAs. The theorem allows us to analyze the running time of a sophisticated EA from an easy-to-analyze EA. Using this tool, we analyze EAs with several crossover o...

  8. A Markov Chain Model for the Analysis of Round-Robin Scheduling Scheme

    Shukla, D; Singhai, Rahul; Agarwal, R K

    2010-01-01

    In the literature of Round-Robin scheduling scheme, each job is processed, one after the another after giving a fix quantum. In case of First-come first-served, each process is executed, if the previously arrived processed is completed. Both these scheduling schemes are used in this paper as its special cases. A Markov chain model is used to compare several scheduling schemes of the class. An index measure is defined to compare the model based efficiency of different scheduling schemes. One scheduling scheme which is the mixture of FIFO and round robin is found efficient in terms of model based study. The system simulation procedure is used to derive the conclusion of the content

  9. Data Model Approach And Markov Chain Based Analysis Of Multi-Level Queue Scheduling

    Diwakar Shukla

    2010-01-01

    Full Text Available There are many CPU scheduling algorithms inliterature like FIFO, Round Robin, Shortest-Job-First and so on.The Multilevel-Queue-Scheduling is superior to these due to itsbetter management of a variety of processes. In this paper, aMarkov chain model is used for a general setup of Multilevelqueue-scheduling and the scheduler is assumed to performrandom movement on queue over the quantum of time.Performance of scheduling is examined through a rowdependent data model. It is found that with increasing value of αand d, the chance of system going over the waiting state reduces.At some of the interesting combinations of α and d, it diminishesto zero, thereby, provides us some clue regarding better choice ofqueues over others for high priority jobs. It is found that ifqueue priorities are added in the scheduling intelligently thenbetter performance could be obtained. Data model helpschoosing appropriate preferences.

  10. A methodology for stochastic analysis of share prices as Markov chains with finite states.

    Mettle, Felix Okoe; Quaye, Enoch Nii Boi; Laryea, Ravenhill Adjetey

    2014-01-01

    Price volatilities make stock investments risky, leaving investors in critical position when uncertain decision is made. To improve investor evaluation confidence on exchange markets, while not using time series methodology, we specify equity price change as a stochastic process assumed to possess Markov dependency with respective state transition probabilities matrices following the identified state pace (i.e. decrease, stable or increase). We established that identified states communicate, and that the chains are aperiodic and ergodic thus possessing limiting distributions. We developed a methodology for determining expected mean return time for stock price increases and also establish criteria for improving investment decision based on highest transition probabilities, lowest mean return time and highest limiting distributions. We further developed an R algorithm for running the methodology introduced. The established methodology is applied to selected equities from Ghana Stock Exchange weekly trading data.

  11. Unifying Markov Chain Approach for Disease and Rumor Spreading in Complex Networks

    de Arruda, Guilherme Ferraz; Rodriiguez, Pablo Martin; Cozzo, Emanuele; Moreno, Yamir

    2016-01-01

    Spreading processes are ubiquitous in natural and artificial systems. They can be studied via a plethora of models, depending on the specific details of the phenomena under study. Disease contagion and rumor spreading are among the most important of these processes due to their practical relevance. However, despite the similarities between them, current models address both spreading dynamics separately. In this paper, we propose a general information spreading model that is based on discrete time Markov chains. The model includes all the transitions that are plausible for both a disease contagion process and rumor propagation. We show that our model not only covers the traditional spreading schemes, but that it also contains some features relevant in social dynamics, such as apathy, forgetting, and lost/recovering of interest. The model is evaluated analytically to obtain the spreading thresholds and the early time dynamical behavior for the contact and reactive processes in several scenarios. Comparison with...

  12. Irreversible Markov chain Monte Carlo algorithm for self-avoiding walk

    Hu, Hao; Chen, Xiaosong; Deng, Youjin

    2017-02-01

    We formulate an irreversible Markov chain Monte Carlo algorithm for the self-avoiding walk (SAW), which violates the detailed balance condition and satisfies the balance condition. Its performance improves significantly compared to that of the Berretti-Sokal algorithm, which is a variant of the Metropolis-Hastings method. The gained efficiency increases with spatial dimension (D), from approximately 10 times in 2D to approximately 40 times in 5D. We simulate the SAW on a 5D hypercubic lattice with periodic boundary conditions, for a linear system with a size up to L = 128, and confirm that as for the 5D Ising model, the finite-size scaling of the SAW is governed by renormalized exponents, v* = 2/ d and γ/ v* = d/2. The critical point is determined, which is approximately 8 times more precise than the best available estimate.

  13. ASSESSING CONVERGENCE OF THE MARKOV CHAIN MONTE CARLO METHOD IN MULTIVARIATE CASE

    Daniel Furtado Ferreira

    2012-01-01

    Full Text Available The formal convergence diagnosis of the Markov Chain Monte Carlo (MCMC is made using univariate and multivariate criteria. In 1998, a multivariate extension of the univariate criterion of multiple sequences was proposed. However, due to some problems of that multivariate criterion, an alternative form of calculation was proposed in addition to the two new alternatives for multivariate convergence criteria. In this study, two models were used, one related to time series with two interventions and ARMA (2, 2 error and another related to a trivariate normal distribution, considering three different cases for the covariance matrix. In both the cases, the Gibbs sampler and the proposed criteria to monitor the convergence were used. Results revealed the proposed criteria to be adequate, besides being easy to implement.

  14. Continuous-time Markov chain-based flux analysis in metabolism.

    Huo, Yunzhang; Ji, Ping

    2014-09-01

    Metabolic flux analysis (MFA), a key technology in bioinformatics, is an effective way of analyzing the entire metabolic system by measuring fluxes. Many existing MFA approaches are based on differential equations, which are complicated to be solved mathematically. So MFA requires some simple approaches to investigate metabolism further. In this article, we applied continuous-time Markov chain to MFA, called MMFA approach, and transformed the MFA problem into a set of quadratic equations by analyzing the transition probability of each carbon atom in the entire metabolic system. Unlike the other methods, MMFA analyzes the metabolic model only through the transition probability. This approach is very generic and it could be applied to any metabolic system if all the reaction mechanisms in the system are known. The results of the MMFA approach were compared with several chemical reaction equilibrium constants from early experiments by taking pentose phosphate pathway as an example.

  15. On the reliability of NMR relaxation data analyses: a Markov Chain Monte Carlo approach.

    Abergel, Daniel; Volpato, Andrea; Coutant, Eloi P; Polimeno, Antonino

    2014-09-01

    The analysis of NMR relaxation data is revisited along the lines of a Bayesian approach. Using a Markov Chain Monte Carlo strategy of data fitting, we investigate conditions under which relaxation data can be effectively interpreted in terms of internal dynamics. The limitations to the extraction of kinetic parameters that characterize internal dynamics are analyzed, and we show that extracting characteristic time scales shorter than a few tens of ps is very unlikely. However, using MCMC methods, reliable estimates of the marginal probability distributions and estimators (average, standard deviations, etc.) can still be obtained for subsets of the model parameters. Thus, unlike more conventional strategies of data analysis, the method avoids a model selection process. In addition, it indicates what information may be extracted from the data, but also what cannot.

  16. Of bugs and birds: Markov Chain Monte Carlo for hierarchical modeling in wildlife research

    Link, W.A.; Cam, E.; Nichols, J.D.; Cooch, E.G.

    2002-01-01

    Markov chain Monte Carlo (MCMC) is a statistical innovation that allows researchers to fit far more complex models to data than is feasible using conventional methods. Despite its widespread use in a variety of scientific fields, MCMC appears to be underutilized in wildlife applications. This may be due to a misconception that MCMC requires the adoption of a subjective Bayesian analysis, or perhaps simply to its lack of familiarity among wildlife researchers. We introduce the basic ideas of MCMC and software BUGS (Bayesian inference using Gibbs sampling), stressing that a simple and satisfactory intuition for MCMC does not require extraordinary mathematical sophistication. We illustrate the use of MCMC with an analysis of the association between latent factors governing individual heterogeneity in breeding and survival rates of kittiwakes (Rissa tridactyla). We conclude with a discussion of the importance of individual heterogeneity for understanding population dynamics and designing management plans.

  17. Markov chain algorithms: a template for building future robust low-power systems.

    Deka, Biplab; Birklykke, Alex A; Duwe, Henry; Mansinghka, Vikash K; Kumar, Rakesh

    2014-06-28

    Although computational systems are looking towards post CMOS devices in the pursuit of lower power, the expected inherent unreliability of such devices makes it difficult to design robust systems without additional power overheads for guaranteeing robustness. As such, algorithmic structures with inherent ability to tolerate computational errors are of significant interest. We propose to cast applications as stochastic algorithms based on Markov chains (MCs) as such algorithms are both sufficiently general and tolerant to transition errors. We show with four example applications--Boolean satisfiability, sorting, low-density parity-check decoding and clustering-how applications can be cast as MC algorithms. Using algorithmic fault injection techniques, we demonstrate the robustness of these implementations to transition errors with high error rates. Based on these results, we make a case for using MCs as an algorithmic template for future robust low-power systems.

  18. ProMT: effective human promoter prediction using Markov chain model based on DNA structural properties.

    Xiong, Dapeng; Liu, Rongjie; Xiao, Fen; Gao, Xieping

    2014-12-01

    The core promoters play significant and extensive roles for the initiation and regulation of DNA transcription. The identification of core promoters is one of the most challenging problems yet. Due to the diverse nature of core promoters, the results obtained through existing computational approaches are not satisfactory. None of them considered the potential influence on performance of predictive approach resulted by the interference between neighboring TSSs in TSS clusters. In this paper, we sufficiently considered this main factor and proposed an approach to locate potential TSS clusters according to the correlation of regional profiles of DNA and TSS clusters. On this basis, we further presented a novel computational approach (ProMT) for promoter prediction using Markov chain model and predictive TSS clusters based on structural properties of DNA. Extensive experiments demonstrated that ProMT can significantly improve the predictive performance. Therefore, considering interference between neighboring TSSs is essential for a wider range of promoter prediction.

  19. Simplification of reversible Markov chains by removal of states with low equilibrium occupancy.

    Ullah, Ghanim; Bruno, William J; Pearson, John E

    2012-10-21

    We present a practical method for simplifying Markov chains on a potentially large state space when detailed balance holds. A simple and transparent technique is introduced to remove states with low equilibrium occupancy. The resulting system has fewer parameters. The resulting effective rates between the remaining nodes give dynamics identical to the original system's except on very fast timescales. This procedure amounts to using separation of timescales to neglect small capacitance nodes in a network of resistors and capacitors. We illustrate the technique by simplifying various reaction networks, including transforming an acyclic four-node network to a three-node cyclic network. For a reaction step in which a ligand binds, the law of mass action implies a forward rate proportional to ligand concentration. The effective rates in the simplified network are found to be rational functions of ligand concentration.

  20. Markov Chain Monte Carlo (MCMC) methods for parameter estimation of a novel hybrid redundant robot

    Wang Yongbo, E-mail: yongbo.wang@hotmail.com [Laboratory of Intelligent Machine, Lappeenranta University of Technology, FIN-53851 Lappeenranta (Finland); Wu Huapeng; Handroos, Heikki [Laboratory of Intelligent Machine, Lappeenranta University of Technology, FIN-53851 Lappeenranta (Finland)

    2011-10-15

    This paper presents a statistical method for the calibration of a redundantly actuated hybrid serial-parallel robot IWR (Intersector Welding Robot). The robot under study will be used to carry out welding, machining, and remote handing for the assembly of vacuum vessel of International Thermonuclear Experimental Reactor (ITER). The robot has ten degrees of freedom (DOF), among which six DOF are contributed by the parallel mechanism and the rest are from the serial mechanism. In this paper, a kinematic error model which involves 54 unknown geometrical error parameters is developed for the proposed robot. Based on this error model, the mean values of the unknown parameters are statistically analyzed and estimated by means of Markov Chain Monte Carlo (MCMC) approach. The computer simulation is conducted by introducing random geometric errors and measurement poses which represent the corresponding real physical behaviors. The simulation results of the marginal posterior distributions of the estimated model parameters indicate that our method is reliable and robust.

  1. Mapping systematic errors in helium abundance determinations using Markov Chain Monte Carlo

    Aver, Erik; Skillman, Evan D

    2010-01-01

    Monte Carlo techniques have been used to evaluate the statistical and systematic uncertainties in the helium abundances derived from extragalactic H~II regions. The helium abundance is sensitive to several physical parameters associated with the H~II region. In this work, we introduce Markov Chain Monte Carlo (MCMC) methods to efficiently explore the parameter space and determine the helium abundance, the physical parameters, and the uncertainties derived from observations of metal poor nebulae. Experiments with synthetic data show that the MCMC method is superior to previous implementations (based on flux perturbation) in that it is not affected by biases due to non-physical parameter space. The MCMC analysis allows a detailed exploration of degeneracies, and, in particular, a false minimum that occurs at large values of optical depth in the He~I emission lines. We demonstrate that introducing the electron temperature derived from the [O~III] emission lines as a prior, in a very conservative manner, produces...

  2. Soft Uncoupling of Markov Chains for Permeable Language Distinction: A New Algorithm

    Nock, Richard; Nielsen, Frank; Henry, Claudia

    2008-01-01

    Without prior knowledge, distinguishing different languages may be a hard task, especially when their borders are permeable. We develop an extension of spectral clustering -- a powerful unsupervised classification toolbox -- that is shown to resolve accurately the task of soft language distinction. At the heart of our approach, we replace the usual hard membership assignment of spectral clustering by a soft, probabilistic assignment, which also presents the advantage to bypass a well-known complexity bottleneck of the method. Furthermore, our approach relies on a novel, convenient construction of a Markov chain out of a corpus. Extensive experiments with a readily available system clearly display the potential of the method, which brings a visually appealing soft distinction of languages that may define altogether a whole corpus.

  3. Fluctuation Theorems for Entropy Production and Heat Dissipation in Periodically Driven Markov Chains

    Shargel, Benjamin Hertz

    2009-01-01

    Asymptotic fluctuation theorems are statements of a Gallavotti-Cohen symmetry in the rate function of either the time-averaged entropy production or heat dissipation of a process. Such theorems have been proved for various general classes of continuous-time deterministic and stochastic processes, but always under the assumption that the forces driving the system are time independent, and often relying on the existence of a limiting ergodic distribution. In this paper we extend the asymptotic fluctuation theorem for the first time to inhomogeneous continuous-time processes without a stationary distribution, considering specifically a finite state Markov chain driven by periodic transition rates. We find that for both entropy production and heat dissipation, the usual Gallavotti-Cohen symmetry of the rate function is generalized to an analogous relation between the rate functions of the original process and its corresponding backward process, in which the trajectory and the driving protocol have been time-rever...

  4. Byzantine Fault Tolerance In The Distributed Environment Using Markov Chain Technique

    R. Kalaivani

    2015-02-01

    Full Text Available ABSTRACT The abstract of this paper is to tolerate the byzantine fault by providing the predefined constraints of the Nodes in the distributed environment. The nodes in the distributed environment automatically generated their constraints using Markov chain. The distributed environment predefined constraints and the member nodes predefined constraints can be updated periodically. According to this update if the member nodes predefined constraints may not matches with the distributed system predefined constraints then using Breadth First Search technique the membership service discards the service of the node in the distributed environment . The new node having constraints wants to communicate with the distributed environment. These constraints can be compared with the distributed system constraints using probability of random matching technique.

  5. A Discrete Time Markov Chain Model for High Throughput Bidirectional Fano Decoders

    Xu, Ran; Morris, Kevin; Kocak, Taskin

    2010-01-01

    The bidirectional Fano algorithm (BFA) can achieve at least two times decoding throughput compared to the conventional unidirectional Fano algorithm (UFA). In this paper, bidirectional Fano decoding is examined from the queuing theory perspective. A Discrete Time Markov Chain (DTMC) is employed to model the BFA decoder with a finite input buffer. The relationship between the input data rate, the input buffer size and the clock speed of the BFA decoder is established. The DTMC based modelling can be used in designing a high throughput parallel BFA decoding system. It is shown that there is a tradeoff between the number of BFA decoders and the input buffer size, and an optimal input buffer size can be chosen to minimize the hardware complexity for a target decoding throughput in designing a high throughput parallel BFA decoding system.

  6. Corruption of accuracy and efficiency of Markov chain Monte Carlo simulation by inaccurate numerical implementation of conceptual hydrologic models

    Schoups, G.; Vrugt, J. A.; Fenicia, F.; van de Giesen, N. C.

    2010-10-01

    Conceptual rainfall-runoff models have traditionally been applied without paying much attention to numerical errors induced by temporal integration of water balance dynamics. Reliance on first-order, explicit, fixed-step integration methods leads to computationally cheap simulation models that are easy to implement. Computational speed is especially desirable for estimating parameter and predictive uncertainty using Markov chain Monte Carlo (MCMC) methods. Confirming earlier work of Kavetski et al. (2003), we show here that the computational speed of first-order, explicit, fixed-step integration methods comes at a cost: for a case study with a spatially lumped conceptual rainfall-runoff model, it introduces artificial bimodality in the marginal posterior parameter distributions, which is not present in numerically accurate implementations of the same model. The resulting effects on MCMC simulation include (1) inconsistent estimates of posterior parameter and predictive distributions, (2) poor performance and slow convergence of the MCMC algorithm, and (3) unreliable convergence diagnosis using the Gelman-Rubin statistic. We studied several alternative numerical implementations to remedy these problems, including various adaptive-step finite difference schemes and an operator splitting method. Our results show that adaptive-step, second-order methods, based on either explicit finite differencing or operator splitting with analytical integration, provide the best alternative for accurate and efficient MCMC simulation. Fixed-step or adaptive-step implicit methods may also be used for increased accuracy, but they cannot match the efficiency of adaptive-step explicit finite differencing or operator splitting. Of the latter two, explicit finite differencing is more generally applicable and is preferred if the individual hydrologic flux laws cannot be integrated analytically, as the splitting method then loses its advantage.

  7. A Markov Chain Model for Changes in Users’ Assessment of Search Results

    Zhitomirsky-Geffet, Maayan; Bar-Ilan, Judit; Levene, Mark

    2016-01-01

    Previous research shows that users tend to change their assessment of search results over time. This is a first study that investigates the factors and reasons for these changes, and describes a stochastic model of user behaviour that may explain these changes. In particular, we hypothesise that most of the changes are local, i.e. between results with similar or close relevance to the query, and thus belong to the same”coarse” relevance category. According to the theory of coarse beliefs and categorical thinking, humans tend to divide the range of values under consideration into coarse categories, and are thus able to distinguish only between cross-category values but not within them. To test this hypothesis we conducted five experiments with about 120 subjects divided into 3 groups. Each student in every group was asked to rank and assign relevance scores to the same set of search results over two or three rounds, with a period of three to nine weeks between each round. The subjects of the last three-round experiment were then exposed to the differences in their judgements and were asked to explain them. We make use of a Markov chain model to measure change in users’ judgments between the different rounds. The Markov chain demonstrates that the changes converge, and that a majority of the changes are local to a neighbouring relevance category. We found that most of the subjects were satisfied with their changes, and did not perceive them as mistakes but rather as a legitimate phenomenon, since they believe that time has influenced their relevance assessment. Both our quantitative analysis and user comments support the hypothesis of the existence of coarse relevance categories resulting from categorical thinking in the context of user evaluation of search results. PMID:27171426

  8. A Markov Chain Model for Changes in Users' Assessment of Search Results.

    Maayan Zhitomirsky-Geffet

    Full Text Available Previous research shows that users tend to change their assessment of search results over time. This is a first study that investigates the factors and reasons for these changes, and describes a stochastic model of user behaviour that may explain these changes. In particular, we hypothesise that most of the changes are local, i.e. between results with similar or close relevance to the query, and thus belong to the same"coarse" relevance category. According to the theory of coarse beliefs and categorical thinking, humans tend to divide the range of values under consideration into coarse categories, and are thus able to distinguish only between cross-category values but not within them. To test this hypothesis we conducted five experiments with about 120 subjects divided into 3 groups. Each student in every group was asked to rank and assign relevance scores to the same set of search results over two or three rounds, with a period of three to nine weeks between each round. The subjects of the last three-round experiment were then exposed to the differences in their judgements and were asked to explain them. We make use of a Markov chain model to measure change in users' judgments between the different rounds. The Markov chain demonstrates that the changes converge, and that a majority of the changes are local to a neighbouring relevance category. We found that most of the subjects were satisfied with their changes, and did not perceive them as mistakes but rather as a legitimate phenomenon, since they believe that time has influenced their relevance assessment. Both our quantitative analysis and user comments support the hypothesis of the existence of coarse relevance categories resulting from categorical thinking in the context of user evaluation of search results.

  9. A Markov Chain Model for Changes in Users' Assessment of Search Results.

    Zhitomirsky-Geffet, Maayan; Bar-Ilan, Judit; Levene, Mark

    2016-01-01

    Previous research shows that users tend to change their assessment of search results over time. This is a first study that investigates the factors and reasons for these changes, and describes a stochastic model of user behaviour that may explain these changes. In particular, we hypothesise that most of the changes are local, i.e. between results with similar or close relevance to the query, and thus belong to the same"coarse" relevance category. According to the theory of coarse beliefs and categorical thinking, humans tend to divide the range of values under consideration into coarse categories, and are thus able to distinguish only between cross-category values but not within them. To test this hypothesis we conducted five experiments with about 120 subjects divided into 3 groups. Each student in every group was asked to rank and assign relevance scores to the same set of search results over two or three rounds, with a period of three to nine weeks between each round. The subjects of the last three-round experiment were then exposed to the differences in their judgements and were asked to explain them. We make use of a Markov chain model to measure change in users' judgments between the different rounds. The Markov chain demonstrates that the changes converge, and that a majority of the changes are local to a neighbouring relevance category. We found that most of the subjects were satisfied with their changes, and did not perceive them as mistakes but rather as a legitimate phenomenon, since they believe that time has influenced their relevance assessment. Both our quantitative analysis and user comments support the hypothesis of the existence of coarse relevance categories resulting from categorical thinking in the context of user evaluation of search results.

  10. Effects of tour boats on dolphin activity examined with sensitivity analysis of Markov chains.

    Dans, Silvana Laura; Degrati, Mariana; Pedraza, Susana Noemí; Crespo, Enrique Alberto

    2012-08-01

    In Patagonia, Argentina, watching dolphins, especially dusky dolphins (Lagenorhynchus obscurus), is a new tourist activity. Feeding time decreases and time to return to feeding after feeding is abandoned and time it takes a group of dolphins to feed increase in the presence of boats. Such effects on feeding behavior may exert energetic costs on dolphins and thus reduce an individual's survival and reproductive capacity or maybe associated with shifts in distribution. We sought to predict which behavioral changes modify the activity pattern of dolphins the most. We modeled behavioral sequences of dusky dolphins with Markov chains. We calculated transition probabilities from one activity to another and arranged them in a stochastic matrix model. The proportion of time dolphins dedicated to a given activity (activity budget) and the time it took a dolphin to resume that activity after it had been abandoned (recurrence time) were calculated. We used a sensitivity analysis of Markov chains to calculate the sensitivity of the time budget and the activity-resumption time to changes in behavioral transition probabilities. Feeding-time budget was most sensitive to changes in the probability of dolphins switching from traveling to feeding behavior and of maintaining feeding behavior. Thus, an increase in these probabilities would be associated with the largest reduction in the time dedicated to feeding. A reduction in the probability of changing from traveling to feeding would also be associated with the largest increases in the time it takes dolphins to resume feeding. To approach dolphins when they are traveling would not affect behavior less because presence of the boat may keep dolphins from returning to feeding. Our results may help operators of dolphin-watching vessels minimize negative effects on dolphins.

  11. Enhancing multi-objective evolutionary algorithm performance with Markov Chain Monte Carlo

    Shafii, M.; Vrugt, J. A.; Tolson, B.; Matott, L. S.

    2009-12-01

    Multi-Objective Evolutionary Algorithms (MOEAs) have emerged as successful optimization routines to solve complex and large-scale multi-objective model calibration problems. However, a common draw-back of these methods is that they require a relatively high number of function evaluations to produce an accurate approximation of Pareto front. This requirement can translate into incredibly large computational costs in hydrologic model calibration problems. Most research efforts to address this computational burden are focused on introducing or improving the operators applied in the MOEAs structure. However, population initialization, usually done through Random Sampling (RS) or Latin Hypercube Sampling (LHS), can also affect the searching efficiency and the quality of MOEA results. This study presents a novel approach to generate initial population of a MOEA (i.e. NSGA-II) by applying a Markov Chain Monte Carlo (MCMC) sampler. The basis of MCMC methods is a Markov chain generating a random walk through the search space, using a formal likelihood function to sample the high-probability-density regions of the parameter space. Therefore, these solutions, when used as initial population, are capable of carrying quite valuable information into MOEAs process. Instead of running the MCMC sampler (i.e. DREAM) to convergence, it is applied for a relatively small and fixed number of function evaluations. The MCMC samples are then processed to identify and archive the non-dominated solutions and this archive is used as NSGA-II’s initial population. In order to analyze the applicability of this approach, it is used for a number of benchmark mathematical problems, as well as multi-objective calibration of a rainfall-runoff model (HYMOD). Initial results show promising MOEA improvement when it is initialized with an MCMC based initial population. Results will be presented that comprehensively compares MOEA results with and without an MCMC based initial population in terms of the

  12. Strong law of large numbers for Markov chains indexed by an infinite tree with uniformly bounded degree

    HUANG HuiLin; YANG WeiGuo

    2008-01-01

    In this paper, we study the strong law of large numbers and Shannon-McMillan (S-M) theorem for Markov chains indexed by an infinite tree with uniformly bounded degree. The results generalize the analogous results on a homogeneous tree.

  13. Strong law of large numbers for Markov chains indexed by an infinite tree with uniformly bounded degree

    2008-01-01

    In this paper,we study the strong law of large numbers and Shannon-McMillan (S-M) theorem for Markov chains indexed by an infinite tree with uniformly bounded degree.The results generalize the analogous results on a homogeneous tree.

  14. SEMI-BLIND CHANNEL ESTIMATION OF MULTIPLE-INPUT/MULTIPLE-OUTPUT SYSTEMS BASED ON MARKOV CHAIN MONTE CARLO METHODS

    Jiang Wei; Xiang Haige

    2004-01-01

    This paper addresses the issues of channel estimation in a Multiple-Input/Multiple-Output (MIMO) system. Markov Chain Monte Carlo (MCMC) method is employed to jointly estimate the Channel State Information (CSI) and the transmitted signals. The deduced algorithms can work well under circumstances of low Signal-to-Noise Ratio (SNR). Simulation results are presented to demonstrate their effectiveness.

  15. Recovery of Graded Response Model Parameters: A Comparison of Marginal Maximum Likelihood and Markov Chain Monte Carlo Estimation

    Kieftenbeld, Vincent; Natesan, Prathiba

    2012-01-01

    Markov chain Monte Carlo (MCMC) methods enable a fully Bayesian approach to parameter estimation of item response models. In this simulation study, the authors compared the recovery of graded response model parameters using marginal maximum likelihood (MML) and Gibbs sampling (MCMC) under various latent trait distributions, test lengths, and…

  16. Parallel GPGPU Evaluation of Small Angle X-ray Scattering Profiles in a Markov Chain Monte Carlo Framework

    Antonov, Lubomir Dimitrov; Andreetta, Christian; Hamelryck, Thomas Wim

    2013-01-01

    Inference of protein structure from experimental data is of crucial interest in science, medicine and biotechnology. Low-resolution methods, such as small angle X-ray scattering (SAXS), play a major role in investigating important biological questions regarding the structure of proteins in soluti......, and implements a caching procedure employed in the partial forward model evaluations within a Markov chain Monte Carlo framework....

  17. Assessing the Progress and the Underlying Nature of the Flows of Doctoral and Master Degree Candidates Using Absorbing Markov Chains

    Nicholls, Miles G.

    2007-01-01

    In this paper, absorbing markov chains are used to analyse the flows of higher degree by research candidates (doctoral and master) within an Australian faculty of business. The candidates are analysed according to whether they are full time or part time. The need for such analysis stemmed from what appeared to be a rather poor completion rate (as…

  18. Invited commentary: Lost in estimation--searching for alternatives to markov chains to fit complex Bayesian models.

    Molitor, John

    2012-03-01

    Bayesian methods have seen an increase in popularity in a wide variety of scientific fields, including epidemiology. One of the main reasons for their widespread application is the power of the Markov chain Monte Carlo (MCMC) techniques generally used to fit these models. As a result, researchers often implicitly associate Bayesian models with MCMC estimation procedures. However, Bayesian models do not always require Markov-chain-based methods for parameter estimation. This is important, as MCMC estimation methods, while generally quite powerful, are complex and computationally expensive and suffer from convergence problems related to the manner in which they generate correlated samples used to estimate probability distributions for parameters of interest. In this issue of the Journal, Cole et al. (Am J Epidemiol. 2012;175(5):368-375) present an interesting paper that discusses non-Markov-chain-based approaches to fitting Bayesian models. These methods, though limited, can overcome some of the problems associated with MCMC techniques and promise to provide simpler approaches to fitting Bayesian models. Applied researchers will find these estimation approaches intuitively appealing and will gain a deeper understanding of Bayesian models through their use. However, readers should be aware that other non-Markov-chain-based methods are currently in active development and have been widely published in other fields.

  19. Simulation from endpoint-conditioned, continuous-time Markov chains on a finite state space, with applications to molecular evolution

    Hobolth, Asger; Stone, Eric

    2009-01-01

    Analyses of serially-sampled data often begin with the assumption that the observations represent discrete samples from a latent continuous-time stochastic process. The continuous-time Markov chain (CTMC) is one such generative model whose popularity extends to a variety of disciplines ranging fr...

  20. Averaging over fast variables in the fluid limit for Markov chains: Application to the supermarket model with memory

    2013-01-01

    We set out a general procedure which allows the approximation of certain Markov chains by the solutions of differential equations. The chains considered have some components which oscillate rapidly and randomly, while others are close to deterministic. The limiting dynamics are obtained by averaging the drift of the latter with respect to a local equilibrium distribution of the former. Some general estimates are proved under a uniform mixing condition on the fast variable which give explicit ...

  1. Markov chain modeling of precipitation time series: Modeling waiting times between tipping bucket rain gauge tips

    Sørup, Hjalte Jomo Danielsen; Madsen, Henrik; Arnbjerg-Nielsen, Karsten

    2011-01-01

    A very fine temporal and volumetric resolution precipitation time series is modeled using Markov models. Both 1st and 2nd order Markov models as well as seasonal and diurnal models are investigated and evaluated using likelihood based techniques. The 2nd order Markov model is found to be insignif...

  2. Multi-Physics Markov Chain Monte Carlo Methods for Subsurface Flows

    Rigelo, J.; Ginting, V.; Rahunanthan, A.; Pereira, F.

    2014-12-01

    For CO2 sequestration in deep saline aquifers, contaminant transport in subsurface, and oil or gas recovery, we often need to forecast flow patterns. Subsurface characterization is a critical and challenging step in flow forecasting. To characterize subsurface properties we establish a statistical description of the subsurface properties that are conditioned to existing dynamic and static data. A Markov Chain Monte Carlo (MCMC) algorithm is used in a Bayesian statistical description to reconstruct the spatial distribution of rock permeability and porosity. The MCMC algorithm requires repeatedly solving a set of nonlinear partial differential equations describing displacement of fluids in porous media for different values of permeability and porosity. The time needed for the generation of a reliable MCMC chain using the algorithm can be too long to be practical for flow forecasting. In this work we develop fast and effective computational methods for generating MCMC chains in the Bayesian framework for the subsurface characterization. Our strategy consists of constructing a family of computationally inexpensive preconditioners based on simpler physics as well as on surrogate models such that the number of fine-grid simulations is drastically reduced in the generated MCMC chains. In particular, we introduce a huff-puff technique as screening step in a three-stage multi-physics MCMC algorithm to reduce the number of expensive final stage simulations. The huff-puff technique in the algorithm enables a better characterization of subsurface near wells. We assess the quality of the proposed multi-physics MCMC methods by considering Monte Carlo simulations for forecasting oil production in an oil reservoir.

  3. Comparison of methods for calculating conditional expectations of sufficient statistics for continuous time Markov chains

    Tataru Paula

    2011-12-01

    Full Text Available Abstract Background Continuous time Markov chains (CTMCs is a widely used model for describing the evolution of DNA sequences on the nucleotide, amino acid or codon level. The sufficient statistics for CTMCs are the time spent in a state and the number of changes between any two states. In applications past evolutionary events (exact times and types of changes are unaccessible and the past must be inferred from DNA sequence data observed in the present. Results We describe and implement three algorithms for computing linear combinations of expected values of the sufficient statistics, conditioned on the end-points of the chain, and compare their performance with respect to accuracy and running time. The first algorithm is based on an eigenvalue decomposition of the rate matrix (EVD, the second on uniformization (UNI, and the third on integrals of matrix exponentials (EXPM. The implementation in R of the algorithms is available at http://www.birc.au.dk/~paula/. Conclusions We use two different models to analyze the accuracy and eight experiments to investigate the speed of the three algorithms. We find that they have similar accuracy and that EXPM is the slowest method. Furthermore we find that UNI is usually faster than EVD.

  4. Using Markov chains to predict the natural progression of diabetic retinopathy

    Priyanka; Srikanth

    2015-01-01

    AIM: To study the natural progression of diabetic retinopathy in patients with type 2 diabetes.METHODS: This was an observational study of 153 cases with type 2 diabetes from 2010 to 2013. The state of patient was noted at end of each year and transition matrices were developed to model movement between years. Patients who progressed to severe non-proliferative diabetic retinopathy(NPDR) were treated.Markov Chains and Chi-square test were used for statistical analysis.RESULTS: We modelled the transition of 153 patients from NPDR to blindness on an annual basis. At the end of year 3, we compared results from the Markov model versus actual data. The results from Chi-square test confirmed that there was statistically no significant difference(P =0.70) which provided assurance that the model was robust to estimate mean sojourn times. The key finding was that a patient entering the system in mild NPDR state is expected to stay in that state for 5y followed by 1.07 y in moderate NPDR, be in the severe NPDR state for 1.33 y before moving into PDR for roughly8 y. It is therefore expected that such a patient entering the model in a state of mild NPDR will enter blindness after 15.29 y.CONCLUSION: Patients stay for long time periods in mild NPDR before transitioning into moderate NPDR.However, they move rapidly from moderate NPDR to proliferative diabetic retinopathy(PDR) and stay in that state for long periods before transitioning into blindness.

  5. Three types of fuzzy Markov chain predication models%三种模糊Markov链状预测模型

    郭嗣琮; 余岚

    2011-01-01

    为了解决具有不确定信息的Markov链状预测问题,给出了“模糊状态—精确观测数据”、“精确状态—模糊观测数据”和“模糊状态—模糊观测数据”三类模糊Markov链状预测的模型,涵盖了具有不确定信息的Markov链状预测的各种形式,系统研究了三种模型下的状态转移概率确定方法与预测过程。该项工作使得模糊Markov链状预测模型问题趋于完善,为深入研究模糊Markov过程以及其它特殊的模糊随机过程提供了思路。%In order to solve Markov chain prediction problem with uncertain information, this paper presents three types of fuzzy Markov chain predication models including "fuzzy state - accurate observation data" model, "accurate status - fuzzy observation data" model and "fuzzy state - fuzzy observation data" model. They cover various forms of Markov chain predication with uncertain information. This study systematically investigates the method for determining the state transition probability and the prediction process under the proposed three models. This study greatly improves the fuzzy Markov chain forecasting models and provides a guideline for further in-depth study on fuzzy Markov process and other special fuzzy stochastic processes.

  6. Fuzzy hidden Markov chains segmentation for volume determination and quantitation in PET

    Hatt, M [INSERM U650, Laboratoire du Traitement de l' Information Medicale (LaTIM), CHU Morvan, Bat 2bis (I3S), 5 avenue Foch, Brest, 29609 (France); Lamare, F [INSERM U650, Laboratoire du Traitement de l' Information Medicale (LaTIM), CHU Morvan, Bat 2bis (I3S), 5 avenue Foch, Brest, 29609, (France); Boussion, N [INSERM U650, Laboratoire du Traitement de l' Information Medicale (LaTIM), CHU Morvan, Bat 2bis (I3S), 5 avenue Foch, Brest, 29609 (France); Turzo, A [INSERM U650, Laboratoire du Traitement de l' Information Medicale (LaTIM), CHU Morvan, Bat 2bis (I3S), 5 avenue Foch, Brest, 29609 (France); Collet, C [Ecole Nationale Superieure de Physique de Strasbourg (ENSPS), ULP, Strasbourg, F-67000 (France); Salzenstein, F [Institut d' Electronique du Solide et des Systemes (InESS), ULP, Strasbourg, F-67000 (France); Roux, C [INSERM U650, Laboratoire du Traitement de l' Information Medicale (LaTIM), CHU Morvan, Bat 2bis (I3S), 5 avenue Foch, Brest, 29609 (France); Jarritt, P [Medical Physics Agency, Royal Victoria Hospital, Belfast (United Kingdom); Carson, K [Medical Physics Agency, Royal Victoria Hospital, Belfast (United Kingdom); Rest, C Cheze-Le [INSERM U650, Laboratoire du Traitement de l' Information Medicale (LaTIM), CHU Morvan, Bat 2bis (I3S), 5 avenue Foch, Brest, 29609 (France); Visvikis, D [INSERM U650, Laboratoire du Traitement de l' Information Medicale (LaTIM), CHU Morvan, Bat 2bis (I3S), 5 avenue Foch, Brest, 29609 (France)

    2007-07-21

    Accurate volume of interest (VOI) estimation in PET is crucial in different oncology applications such as response to therapy evaluation and radiotherapy treatment planning. The objective of our study was to evaluate the performance of the proposed algorithm for automatic lesion volume delineation; namely the fuzzy hidden Markov chains (FHMC), with that of current state of the art in clinical practice threshold based techniques. As the classical hidden Markov chain (HMC) algorithm, FHMC takes into account noise, voxel intensity and spatial correlation, in order to classify a voxel as background or functional VOI. However the novelty of the fuzzy model consists of the inclusion of an estimation of imprecision, which should subsequently lead to a better modelling of the 'fuzzy' nature of the object of interest boundaries in emission tomography data. The performance of the algorithms has been assessed on both simulated and acquired datasets of the IEC phantom, covering a large range of spherical lesion sizes (from 10 to 37 mm), contrast ratios (4:1 and 8:1) and image noise levels. Both lesion activity recovery and VOI determination tasks were assessed in reconstructed images using two different voxel sizes (8 mm{sup 3} and 64 mm{sup 3}). In order to account for both the functional volume location and its size, the concept of % classification errors was introduced in the evaluation of volume segmentation using the simulated datasets. Results reveal that FHMC performs substantially better than the threshold based methodology for functional volume determination or activity concentration recovery considering a contrast ratio of 4:1 and lesion sizes of <28 mm. Furthermore differences between classification and volume estimation errors evaluated were smaller for the segmented volumes provided by the FHMC algorithm. Finally, the performance of the automatic algorithms was less susceptible to image noise levels in comparison to the threshold based techniques. The

  7. Measuring and partitioning the high-order linkage disequilibrium by multiple order Markov chains.

    Kim, Yunjung; Feng, Sheng; Zeng, Zhao-Bang

    2008-05-01

    A map of the background levels of disequilibrium between nearby markers can be useful for association mapping studies. In order to assess the background levels of linkage disequilibrium (LD), multilocus LD measures are more advantageous than pairwise LD measures because the combined analysis of pairwise LD measures is not adequate to detect simultaneous allele associations among multiple markers. Various multilocus LD measures based on haplotypes have been proposed. However, most of these measures provide a single index of association among multiple markers and does not reveal the complex patterns and different levels of LD structure. In this paper, we employ non-homogeneous, multiple order Markov Chain models as a statistical framework to measure and partition the LD among multiple markers into components due to different orders of marker associations. Using a sliding window of multiple markers on phased haplotype data, we compute corresponding likelihoods for different Markov Chain (MC) orders in each window. The log-likelihood difference between the lowest MC order model (MC0) and the highest MC order model in each window is used as a measure of the total LD or the overall deviation from the gametic equilibrium for the window. Then, we partition the total LD into lower order disequilibria and estimate the effects from two-, three-, and higher order disequilibria. The relationship between different orders of LD and the log-likelihood difference involving two different orders of MC models are explored. By applying our method to the phased haplotype data in the ENCODE regions of the HapMap project, we are able to identify high/low multilocus LD regions. Our results reveal that the most LD in the HapMap data is attributed to the LD between adjacent pairs of markers across the whole region. LD between adjacent pairs of markers appears to be more significant in high multilocus LD regions than in low multilocus LD regions. We also find that as the multilocus total LD

  8. Time-patterns of antibiotic exposure in poultry production--a Markov chains exploratory study of nature and consequences.

    Chauvin, C; Clement, C; Bruneau, M; Pommeret, D

    2007-07-16

    This article describes the use of Markov chains to explore the time-patterns of antimicrobial exposure in broiler poultry. The transition in antimicrobial exposure status (exposed/not exposed to an antimicrobial, with a distinction between exposures to the different antimicrobial classes) in extensive data collected in broiler chicken flocks from November 2003 onwards, was investigated. All Markov chains were first-order chains. Mortality rate, geographical location and slaughter semester were sources of heterogeneity between transition matrices. Transitions towards a 'no antimicrobial' exposure state were highly predominant, whatever the initial state. From a 'no antimicrobial' exposure state, the transition to beta-lactams was predominant among transitions to an antimicrobial exposure state. Transitions between antimicrobial classes were rare and variable. Switches between antimicrobial classes and repeats of a particular class were both observed. Application of Markov chains analysis to the database of the nation-wide antimicrobial resistance monitoring programme pointed out that transition probabilities between antimicrobial exposure states increased with the number of resistances in Escherichia coli strains.

  9. Dynamical Models for NGC 6503 using a Markov Chain Monte Carlo Technique

    Puglielli, David; Courteau, Stéphane

    2010-01-01

    We use Bayesian statistics and Markov chain Monte Carlo (MCMC) techniques to construct dynamical models for the spiral galaxy NGC 6503. The constraints include surface brightness profiles which display a Freeman Type II structure; HI and ionized gas rotation curves; the stellar rotation, which is nearly coincident with the ionized gas curve; and the line of sight stellar dispersion, with a sigma-drop at the centre. The galaxy models consist of a Sersic bulge, an exponential disc with an optional inner truncation and a cosmologically motivated dark halo. The Bayesian/MCMC technique yields the joint posterior probability distribution function for the input parameters. We examine several interpretations of the data: the Type II surface brightness profile may be due to dust extinction, to an inner truncated disc or to a ring of bright stars; and we test separate fits to the gas and stellar rotation curves to determine if the gas traces the gravitational potential. We test each of these scenarios for bar stability...

  10. Study on the Calculation Models of Bus Delay at Bays Using Queueing Theory and Markov Chain

    Feng Sun

    2015-01-01

    Full Text Available Traffic congestion at bus bays has decreased the service efficiency of public transit seriously in China, so it is crucial to systematically study its theory and methods. However, the existing studies lack theoretical model on computing efficiency. Therefore, the calculation models of bus delay at bays are studied. Firstly, the process that buses are delayed at bays is analyzed, and it was found that the delay can be divided into entering delay and exiting delay. Secondly, the queueing models of bus bays are formed, and the equilibrium distribution functions are proposed by applying the embedded Markov chain to the traditional model of queuing theory in the steady state; then the calculation models of entering delay are derived at bays. Thirdly, the exiting delay is studied by using the queueing theory and the gap acceptance theory. Finally, the proposed models are validated using field-measured data, and then the influencing factors are discussed. With these models the delay is easily assessed knowing the characteristics of the dwell time distribution and traffic volume at the curb lane in different locations and different periods. It can provide basis for the efficiency evaluation of bus bays.

  11. Use of Markov Chains to Design an Agent Bidding Strategy for Continuous Double Auctions

    Birmingham, W P; Park, S; 10.1613/jair.1466

    2011-01-01

    As computational agents are developed for increasingly complicated e-commerce applications, the complexity of the decisions they face demands advances in artificial intelligence techniques. For example, an agent representing a seller in an auction should try to maximize the seller?s profit by reasoning about a variety of possibly uncertain pieces of information, such as the maximum prices various buyers might be willing to pay, the possible prices being offered by competing sellers, the rules by which the auction operates, the dynamic arrival and matching of offers to buy and sell, and so on. A naive application of multiagent reasoning techniques would require the seller?s agent to explicitly model all of the other agents through an extended time horizon, rendering the problem intractable for many realistically-sized problems. We have instead devised a new strategy that an agent can use to determine its bid price based on a more tractable Markov chain model of the auction process. We have experimentally ident...

  12. Improving Markov Chain Monte Carlo algorithms in LISA Pathfinder Data Analysis

    Karnesis, N.; Nofrarias, M.; Sopuerta, C. F.; Lobo, A.

    2012-06-01

    The LISA Pathfinder mission (LPF) aims to test key technologies for the future LISA mission. The LISA Technology Package (LTP) on-board LPF will consist of an exhaustive suite of experiments and its outcome will be crucial for the future detection of gravitational waves. In order to achieve maximum sensitivity, we need to have an understanding of every instrument on-board and parametrize the properties of the underlying noise models. The Data Analysis team has developed algorithms for parameter estimation of the system. A very promising one implemented for LISA Pathfinder data analysis is the Markov Chain Monte Carlo. A series of experiments are going to take place during flight operations and each experiment is going to provide us with essential information for the next in the sequence. Therefore, it is a priority to optimize and improve our tools available for data analysis during the mission. Using a Bayesian framework analysis allows us to apply prior knowledge for each experiment, which means that we can efficiently use our prior estimates for the parameters, making the method more accurate and significantly faster. This, together with other algorithm improvements, will lead us to our main goal, which is no other than creating a robust and reliable tool for parameter estimation during the LPF mission.

  13. Markov chain formalism for generalized radiative transfer in a plane-parallel medium, accounting for polarization

    Xu, Feng; Davis, Anthony B.; Diner, David J.

    2016-11-01

    A Markov chain formalism is developed for computing the transport of polarized radiation according to Generalized Radiative Transfer (GRT) theory, which was developed recently to account for unresolved random fluctuations of scattering particle density and can also be applied to unresolved spectral variability of gaseous absorption as an improvement over the standard correlated-k method. Using Gamma distribution to describe the probability density function of the extinction or absorption coefficient, a shape parameter a that quantifies the variability is introduced, defined as the mean extinction or absorption coefficient squared divided by its variance. It controls the decay rate of a power-law transmission that replaces the usual exponential Beer-Lambert-Bouguer law. Exponential transmission, hence classic RT, is recovered when a→∞. The new approach is verified to high accuracy against numerical benchmark results obtained with a custom Monte Carlo method. For a<∞, angular reciprocity is violated to a degree that increases with the spatial variability, as observed for finite portions of real-world cloudy scenes. While the degree of linear polarization in liquid water cloudbows, supernumerary bows, and glories is affected by spatial heterogeneity, the positions in scattering angle of these features are relatively unchanged. As a result, a single-scattering model based on the assumption of subpixel homogeneity can still be used to derive droplet size distributions from polarimetric measurements of extended stratocumulus clouds.

  14. Study on the Calculation Models of Bus Delay at Bays Using Queueing Theory and Markov Chain

    Sun, Li; Sun, Shao-wei; Wang, Dian-hai

    2015-01-01

    Traffic congestion at bus bays has decreased the service efficiency of public transit seriously in China, so it is crucial to systematically study its theory and methods. However, the existing studies lack theoretical model on computing efficiency. Therefore, the calculation models of bus delay at bays are studied. Firstly, the process that buses are delayed at bays is analyzed, and it was found that the delay can be divided into entering delay and exiting delay. Secondly, the queueing models of bus bays are formed, and the equilibrium distribution functions are proposed by applying the embedded Markov chain to the traditional model of queuing theory in the steady state; then the calculation models of entering delay are derived at bays. Thirdly, the exiting delay is studied by using the queueing theory and the gap acceptance theory. Finally, the proposed models are validated using field-measured data, and then the influencing factors are discussed. With these models the delay is easily assessed knowing the characteristics of the dwell time distribution and traffic volume at the curb lane in different locations and different periods. It can provide basis for the efficiency evaluation of bus bays. PMID:25759720

  15. Study on the calculation models of bus delay at bays using queueing theory and Markov chain.

    Sun, Feng; Sun, Li; Sun, Shao-Wei; Wang, Dian-Hai

    2015-01-01

    Traffic congestion at bus bays has decreased the service efficiency of public transit seriously in China, so it is crucial to systematically study its theory and methods. However, the existing studies lack theoretical model on computing efficiency. Therefore, the calculation models of bus delay at bays are studied. Firstly, the process that buses are delayed at bays is analyzed, and it was found that the delay can be divided into entering delay and exiting delay. Secondly, the queueing models of bus bays are formed, and the equilibrium distribution functions are proposed by applying the embedded Markov chain to the traditional model of queuing theory in the steady state; then the calculation models of entering delay are derived at bays. Thirdly, the exiting delay is studied by using the queueing theory and the gap acceptance theory. Finally, the proposed models are validated using field-measured data, and then the influencing factors are discussed. With these models the delay is easily assessed knowing the characteristics of the dwell time distribution and traffic volume at the curb lane in different locations and different periods. It can provide basis for the efficiency evaluation of bus bays.

  16. Performance evaluation of railway blocking system based on markov chain and queuing theory

    Guo, Jin; Chen, Hongxia; Yang, Yang

    2005-12-01

    Railway blocking system is the system with the high demanding of real-time performance. Firstly, the tasks and the time limits, which had to be handled for the blocking system, were introduced. The FCFS and the Markov chain were used to set the model for it. By analyzing the performance of the system with the FCFS model found out that it was not satisfied to the real-time performance. Secondly, NPPR model to evaluate the software real-time performance of the blocking processor was proposed. By evaluation, analysis and comparison, the results indicate that the NPPR model is prevail over the model of (M/M/1): (N/N/FCFS) in real-time performance. And the priorities of the tasks in the system should be given according to their time limit. With the principle of (M/M/1): (N/N/NPPR), if the priority was given to the tasks properly, the satisfied real-time performance will be gotten. The models were tested in forms software and the satisfied result has been gotten in practice.

  17. Sanov and central limit theorems for output statistics of quantum Markov chains

    Horssen, Merlijn van, E-mail: merlijn.vanhorssen@nottingham.ac.uk [School of Physics and Astronomy, University of Nottingham, Nottingham NG7 2RD (United Kingdom); Guţă, Mădălin, E-mail: madalin.guta@nottingham.ac.uk [School of Mathematical Sciences, University of Nottingham, Nottingham NG7 2RD (United Kingdom)

    2015-02-15

    In this paper, we consider the statistics of repeated measurements on the output of a quantum Markov chain. We establish a large deviations result analogous to Sanov’s theorem for the multi-site empirical measure associated to finite sequences of consecutive outcomes of a classical stochastic process. Our result relies on the construction of an extended quantum transition operator (which keeps track of previous outcomes) in terms of which we compute moment generating functions, and whose spectral radius is related to the large deviations rate function. As a corollary to this, we obtain a central limit theorem for the empirical measure. Such higher level statistics may be used to uncover critical behaviour such as dynamical phase transitions, which are not captured by lower level statistics such as the sample mean. As a step in this direction, we give an example of a finite system whose level-1 (empirical mean) rate function is independent of a model parameter while the level-2 (empirical measure) rate is not.

  18. A Markov chain model for image ranking system in social networks

    Zin, Thi Thi; Tin, Pyke; Toriu, Takashi; Hama, Hiromitsu

    2014-03-01

    In today world, different kinds of networks such as social, technological, business and etc. exist. All of the networks are similar in terms of distributions, continuously growing and expanding in large scale. Among them, many social networks such as Facebook, Twitter, Flickr and many others provides a powerful abstraction of the structure and dynamics of diverse kinds of inter personal connection and interaction. Generally, the social network contents are created and consumed by the influences of all different social navigation paths that lead to the contents. Therefore, identifying important and user relevant refined structures such as visual information or communities become major factors in modern decision making world. Moreover, the traditional method of information ranking systems cannot be successful due to their lack of taking into account the properties of navigation paths driven by social connections. In this paper, we propose a novel image ranking system in social networks by using the social data relational graphs from social media platform jointly with visual data to improve the relevance between returned images and user intentions (i.e., social relevance). Specifically, we propose a Markov chain based Social-Visual Ranking algorithm by taking social relevance into account. By using some extensive experiments, we demonstrated the significant and effectiveness of the proposed social-visual ranking method.

  19. Empirical Markov Chain Monte Carlo Bayesian analysis of fMRI data.

    de Pasquale, F; Del Gratta, C; Romani, G L

    2008-08-01

    In this work an Empirical Markov Chain Monte Carlo Bayesian approach to analyse fMRI data is proposed. The Bayesian framework is appealing since complex models can be adopted in the analysis both for the image and noise model. Here, the noise autocorrelation is taken into account by adopting an AutoRegressive model of order one and a versatile non-linear model is assumed for the task-related activation. Model parameters include the noise variance and autocorrelation, activation amplitudes and the hemodynamic response function parameters. These are estimated at each voxel from samples of the Posterior Distribution. Prior information is included by means of a 4D spatio-temporal model for the interaction between neighbouring voxels in space and time. The results show that this model can provide smooth estimates from low SNR data while important spatial structures in the data can be preserved. A simulation study is presented in which the accuracy and bias of the estimates are addressed. Furthermore, some results on convergence diagnostic of the adopted algorithm are presented. To validate the proposed approach a comparison of the results with those from a standard GLM analysis, spatial filtering techniques and a Variational Bayes approach is provided. This comparison shows that our approach outperforms the classical analysis and is consistent with other Bayesian techniques. This is investigated further by means of the Bayes Factors and the analysis of the residuals. The proposed approach applied to Blocked Design and Event Related datasets produced reliable maps of activation.

  20. Smart pilot points using reversible-jump Markov-chain Monte Carlo

    Jiménez, S.; Mariethoz, G.; Brauchler, R.; Bayer, P.

    2016-05-01

    Pilot points are typical means for calibration of highly parameterized numerical models. We propose a novel procedure based on estimating not only the pilot point values, but also their number and suitable locations. This is accomplished by a trans-dimensional Bayesian inversion procedure that also allows for uncertainty quantification. The utilized algorithm, reversible-jump Markov-Chain Monte Carlo (RJ-MCMC), is computationally demanding and this challenges the application for model calibration. We present a solution for fast, approximate simulation through the application of a Bayesian inversion. A fast pathfinding algorithm is used to estimate tracer travel times instead of doing a full transport simulation. This approach extracts the information from measured breakthrough curves, which is crucial for the reconstruction of aquifer heterogeneity. As a result, the "smart pilot points" can be tuned during thousands of rapid model evaluations. This is demonstrated for both a synthetic and a field application. For the selected synthetic layered aquifer, two different hydrofacies are reconstructed. For the field investigation, multiple fluorescent tracers were injected in different well screens in a shallow alluvial aquifer and monitored in a tomographic source-receiver configuration. With the new inversion procedure, a sand layer was identified and reconstructed with a high spatial resolution in 3-D. The sand layer was successfully validated through additional slug tests at the site. The promising results encourage further applications in hydrogeological model calibration, especially for cases with simulation of transport.

  1. Estimating stepwise debromination pathways of polybrominated diphenyl ethers with an analogue Markov Chain Monte Carlo algorithm.

    Zou, Yonghong; Christensen, Erik R; Zheng, Wei; Wei, Hua; Li, An

    2014-11-01

    A stochastic process was developed to simulate the stepwise debromination pathways for polybrominated diphenyl ethers (PBDEs). The stochastic process uses an analogue Markov Chain Monte Carlo (AMCMC) algorithm to generate PBDE debromination profiles. The acceptance or rejection of the randomly drawn stepwise debromination reactions was determined by a maximum likelihood function. The experimental observations at certain time points were used as target profiles; therefore, the stochastic processes are capable of presenting the effects of reaction conditions on the selection of debromination pathways. The application of the model is illustrated by adopting the experimental results of decabromodiphenyl ether (BDE209) in hexane exposed to sunlight. Inferences that were not obvious from experimental data were suggested by model simulations. For example, BDE206 has much higher accumulation at the first 30 min of sunlight exposure. By contrast, model simulation suggests that, BDE206 and BDE207 had comparable yields from BDE209. The reason for the higher BDE206 level is that BDE207 has the highest depletion in producing octa products. Compared to a previous version of the stochastic model based on stochastic reaction sequences (SRS), the AMCMC approach was determined to be more efficient and robust. Due to the feature of only requiring experimental observations as input, the AMCMC model is expected to be applicable to a wide range of PBDE debromination processes, e.g. microbial, photolytic, or joint effects in natural environments.

  2. PHAISTOS: a framework for Markov chain Monte Carlo simulation and inference of protein structure.

    Boomsma, Wouter; Frellsen, Jes; Harder, Tim; Bottaro, Sandro; Johansson, Kristoffer E; Tian, Pengfei; Stovgaard, Kasper; Andreetta, Christian; Olsson, Simon; Valentin, Jan B; Antonov, Lubomir D; Christensen, Anders S; Borg, Mikael; Jensen, Jan H; Lindorff-Larsen, Kresten; Ferkinghoff-Borg, Jesper; Hamelryck, Thomas

    2013-07-15

    We present a new software framework for Markov chain Monte Carlo sampling for simulation, prediction, and inference of protein structure. The software package contains implementations of recent advances in Monte Carlo methodology, such as efficient local updates and sampling from probabilistic models of local protein structure. These models form a probabilistic alternative to the widely used fragment and rotamer libraries. Combined with an easily extendible software architecture, this makes PHAISTOS well suited for Bayesian inference of protein structure from sequence and/or experimental data. Currently, two force-fields are available within the framework: PROFASI and OPLS-AA/L, the latter including the generalized Born surface area solvent model. A flexible command-line and configuration-file interface allows users quickly to set up simulations with the desired configuration. PHAISTOS is released under the GNU General Public License v3.0. Source code and documentation are freely available from http://phaistos.sourceforge.net. The software is implemented in C++ and has been tested on Linux and OSX platforms.

  3. Ascertainment correction for Markov chain Monte Carlo segregation and linkage analysis of a quantitative trait.

    Ma, Jianzhong; Amos, Christopher I; Warwick Daw, E

    2007-09-01

    Although extended pedigrees are often sampled through probands with extreme levels of a quantitative trait, Markov chain Monte Carlo (MCMC) methods for segregation and linkage analysis have not been able to perform ascertainment corrections. Further, the extent to which ascertainment of pedigrees leads to biases in the estimation of segregation and linkage parameters has not been previously studied for MCMC procedures. In this paper, we studied these issues with a Bayesian MCMC approach for joint segregation and linkage analysis, as implemented in the package Loki. We first simulated pedigrees ascertained through individuals with extreme values of a quantitative trait in spirit of the sequential sampling theory of Cannings and Thompson [Cannings and Thompson [1977] Clin. Genet. 12:208-212]. Using our simulated data, we detected no bias in estimates of the trait locus location. However, in addition to allele frequencies, when the ascertainment threshold was higher than or close to the true value of the highest genotypic mean, bias was also found in the estimation of this parameter. When there were multiple trait loci, this bias destroyed the additivity of the effects of the trait loci, and caused biases in the estimation all genotypic means when a purely additive model was used for analyzing the data. To account for pedigree ascertainment with sequential sampling, we developed a Bayesian ascertainment approach and implemented Metropolis-Hastings updates in the MCMC samplers used in Loki. Ascertainment correction greatly reduced biases in parameter estimates. Our method is designed for multiple, but a fixed number of trait loci.

  4. Predicting protein subcellular locations using hierarchical ensemble of Bayesian classifiers based on Markov chains

    Eils Roland

    2006-06-01

    Full Text Available Abstract Background The subcellular location of a protein is closely related to its function. It would be worthwhile to develop a method to predict the subcellular location for a given protein when only the amino acid sequence of the protein is known. Although many efforts have been made to predict subcellular location from sequence information only, there is the need for further research to improve the accuracy of prediction. Results A novel method called HensBC is introduced to predict protein subcellular location. HensBC is a recursive algorithm which constructs a hierarchical ensemble of classifiers. The classifiers used are Bayesian classifiers based on Markov chain models. We tested our method on six various datasets; among them are Gram-negative bacteria dataset, data for discriminating outer membrane proteins and apoptosis proteins dataset. We observed that our method can predict the subcellular location with high accuracy. Another advantage of the proposed method is that it can improve the accuracy of the prediction of some classes with few sequences in training and is therefore useful for datasets with imbalanced distribution of classes. Conclusion This study introduces an algorithm which uses only the primary sequence of a protein to predict its subcellular location. The proposed recursive scheme represents an interesting methodology for learning and combining classifiers. The method is computationally efficient and competitive with the previously reported approaches in terms of prediction accuracies as empirical results indicate. The code for the software is available upon request.

  5. Two-state Markov-chain Poisson nature of individual cellphone call statistics

    Jiang, Zhi-Qiang; Xie, Wen-Jie; Li, Ming-Xia; Zhou, Wei-Xing; Sornette, Didier

    2016-07-01

    Unfolding the burst patterns in human activities and social interactions is a very important issue especially for understanding the spreading of disease and information and the formation of groups and organizations. Here, we conduct an in-depth study of the temporal patterns of cellphone conversation activities of 73 339 anonymous cellphone users, whose inter-call durations are Weibull distributed. We find that the individual call events exhibit a pattern of bursts, that high activity periods are alternated with low activity periods. In both periods, the number of calls are exponentially distributed for individuals, but power-law distributed for the population. Together with the exponential distributions of inter-call durations within bursts and of the intervals between consecutive bursts, we demonstrate that the individual call activities are driven by two independent Poisson processes, which can be combined within a minimal model in terms of a two-state first-order Markov chain, giving significant fits for nearly half of the individuals. By measuring directly the distributions of call rates across the population, which exhibit power-law tails, we purport the existence of power-law distributions, via the ‘superposition of distributions’ mechanism. Our findings shed light on the origins of bursty patterns in other human activities.

  6. Exact Markov chain and approximate diffusion solution for haploid genetic drift with one-way mutation.

    Hössjer, Ola; Tyvand, Peder A; Miloh, Touvia

    2016-02-01

    The classical Kimura solution of the diffusion equation is investigated for a haploid random mating (Wright-Fisher) model, with one-way mutations and initial-value specified by the founder population. The validity of the transient diffusion solution is checked by exact Markov chain computations, using a Jordan decomposition of the transition matrix. The conclusion is that the one-way diffusion model mostly works well, although the rate of convergence depends on the initial allele frequency and the mutation rate. The diffusion approximation is poor for mutation rates so low that the non-fixation boundary is regular. When this happens we perturb the diffusion solution around the non-fixation boundary and obtain a more accurate approximation that takes quasi-fixation of the mutant allele into account. The main application is to quantify how fast a specific genetic variant of the infinite alleles model is lost. We also discuss extensions of the quasi-fixation approach to other models with small mutation rates.

  7. Determining coding CpG islands by identifying regions significant for pattern statistics on Markov chains.

    Singer, Meromit; Engström, Alexander; Schönhuth, Alexander; Pachter, Lior

    2011-09-23

    Recent experimental and computational work confirms that CpGs can be unmethylated inside coding exons, thereby showing that codons may be subjected to both genomic and epigenomic constraint. It is therefore of interest to identify coding CpG islands (CCGIs) that are regions inside exons enriched for CpGs. The difficulty in identifying such islands is that coding exons exhibit sequence biases determined by codon usage and constraints that must be taken into account. We present a method for finding CCGIs that showcases a novel approach we have developed for identifying regions of interest that are significant (with respect to a Markov chain) for the counts of any pattern. Our method begins with the exact computation of tail probabilities for the number of CpGs in all regions contained in coding exons, and then applies a greedy algorithm for selecting islands from among the regions. We show that the greedy algorithm provably optimizes a biologically motivated criterion for selecting islands while controlling the false discovery rate. We applied this approach to the human genome (hg18) and annotated CpG islands in coding exons. The statistical criterion we apply to evaluating islands reduces the number of false positives in existing annotations, while our approach to defining islands reveals significant numbers of undiscovered CCGIs in coding exons. Many of these appear to be examples of functional epigenetic specialization in coding exons.

  8. An informational transition in conditioned Markov chains: Applied to genetics and evolution.

    Zhao, Lei; Lascoux, Martin; Waxman, David

    2016-08-07

    In this work we assume that we have some knowledge about the state of a population at two known times, when the dynamics is governed by a Markov chain such as a Wright-Fisher model. Such knowledge could be obtained, for example, from observations made on ancient and contemporary DNA, or during laboratory experiments involving long term evolution. A natural assumption is that the behaviour of the population, between observations, is related to (or constrained by) what was actually observed. The present work shows that this assumption has limited validity. When the time interval between observations is larger than a characteristic value, which is a property of the population under consideration, there is a range of intermediate times where the behaviour of the population has reduced or no dependence on what was observed and an equilibrium-like distribution applies. Thus, for example, if the frequency of an allele is observed at two different times, then for a large enough time interval between observations, the population has reduced or no dependence on the two observed frequencies for a range of intermediate times. Given observations of a population at two times, we provide a general theoretical analysis of the behaviour of the population at all intermediate times, and determine an expression for the characteristic time interval, beyond which the observations do not constrain the population's behaviour over a range of intermediate times. The findings of this work relate to what can be meaningfully inferred about a population at intermediate times, given knowledge of terminal states.

  9. Prediction of small molecule binding property of protein domains with Bayesian classifiers based on Markov chains.

    Bulashevska, Alla; Stein, Martin; Jackson, David; Eils, Roland

    2009-12-01

    Accurate computational methods that can help to predict biological function of a protein from its sequence are of great interest to research biologists and pharmaceutical companies. One approach to assume the function of proteins is to predict the interactions between proteins and other molecules. In this work, we propose a machine learning method that uses a primary sequence of a domain to predict its propensity for interaction with small molecules. By curating the Pfam database with respect to the small molecule binding ability of its component domains, we have constructed a dataset of small molecule binding and non-binding domains. This dataset was then used as training set to learn a Bayesian classifier, which should distinguish members of each class. The domain sequences of both classes are modelled with Markov chains. In a Jack-knife test, our classification procedure achieved the predictive accuracies of 77.2% and 66.7% for binding and non-binding classes respectively. We demonstrate the applicability of our classifier by using it to identify previously unknown small molecule binding domains. Our predictions are available as supplementary material and can provide very useful information to drug discovery specialists. Given the ubiquitous and essential role small molecules play in biological processes, our method is important for identifying pharmaceutically relevant components of complete proteomes. The software is available from the author upon request.

  10. Solving inverse problem for Markov chain model of customer lifetime value using flower pollination algorithm

    Al-Ma'shumah, Fathimah; Permana, Dony; Sidarto, Kuntjoro Adji

    2015-12-01

    Customer Lifetime Value is an important and useful concept in marketing. One of its benefits is to help a company for budgeting marketing expenditure for customer acquisition and customer retention. Many mathematical models have been introduced to calculate CLV considering the customer retention/migration classification scheme. A fairly new class of these models which will be described in this paper uses Markov Chain Models (MCM). This class of models has the major advantage for its flexibility to be modified to several different cases/classification schemes. In this model, the probabilities of customer retention and acquisition play an important role. From Pfeifer and Carraway, 2000, the final formula of CLV obtained from MCM usually contains nonlinear form of the transition probability matrix. This nonlinearity makes the inverse problem of CLV difficult to solve. This paper aims to solve this inverse problem, yielding the approximate transition probabilities for the customers, by applying metaheuristic optimization algorithm developed by Yang, 2013, Flower Pollination Algorithm. The major interpretation of obtaining the transition probabilities are to set goals for marketing teams in keeping the relative frequencies of customer acquisition and customer retention.

  11. Performance evaluation of corrosion-affected reinforced concrete bridge girders using Markov chains with fuzzy states

    M B ANOOP; K BALAJI RAO

    2016-08-01

    A methodology for performance evaluation of reinforced concrete bridge girders in corrosive environments is proposed. The methodology uses the concept of performability and considers both serviceability- and ultimate-limit states. The serviceability limit states are defined based on the degree of cracking (characterized by crack width) in the girder due to chloride induced corrosion of reinforcement, and the ultimate limit states are defined based on the flexural load carrying capacity of the girder (characterized in terms of rating factor using the load and resistance factor rating method). The condition of the bridge girder is specified by the assignment of a condition state from a set of predefined condition states. Generally, the classification of condition states is linguistic, while the condition states are considered to be mutually exclusive and collectivelyexhaustive. In the present study, the condition states of the bridge girder are also represented by fuzzy sets to consider the ambiguities arising due to the linguistic classification of condition states. A non-homogeneous Markov chain (MC) model is used for modeling the condition state evolution of the bridge girder with time. The usefulness of the proposed methodology is demonstrated through a case study of a severely distressed beam of the Rocky Point Viaduct. The results obtained using the proposed approach are compared with those obtained using conventional MC model. It is noted that the use of MC with fuzzy states leads to conservative decision making for the problem considered in the case study.

  12. Phase Transitions for Quantum Markov Chains Associated with Ising Type Models on a Cayley Tree

    Mukhamedov, Farrukh; Barhoumi, Abdessatar; Souissi, Abdessatar

    2016-05-01

    The main aim of the present paper is to prove the existence of a phase transition in quantum Markov chain (QMC) scheme for the Ising type models on a Cayley tree. Note that this kind of models do not have one-dimensional analogous, i.e. the considered model persists only on trees. In this paper, we provide a more general construction of forward QMC. In that construction, a QMC is defined as a weak limit of finite volume states with boundary conditions, i.e. QMC depends on the boundary conditions. Our main result states the existence of a phase transition for the Ising model with competing interactions on a Cayley tree of order two. By the phase transition we mean the existence of two distinct QMC which are not quasi-equivalent and their supports do not overlap. We also study some algebraic property of the disordered phase of the model, which is a new phenomena even in a classical setting.

  13. On stochastic error and computational efficiency of the Markov Chain Monte Carlo method

    Li, Jun

    2014-01-01

    In Markov Chain Monte Carlo (MCMC) simulations, thermal equilibria quantities are estimated by ensemble average over a sample set containing a large number of correlated samples. These samples are selected in accordance with the probability distribution function, known from the partition function of equilibrium state. As the stochastic error of the simulation results is significant, it is desirable to understand the variance of the estimation by ensemble average, which depends on the sample size (i.e., the total number of samples in the set) and the sampling interval (i.e., cycle number between two consecutive samples). Although large sample sizes reduce the variance, they increase the computational cost of the simulation. For a given CPU time, the sample size can be reduced greatly by increasing the sampling interval, while having the corresponding increase in variance be negligible if the original sampling interval is very small. In this work, we report a few general rules that relate the variance with the sample size and the sampling interval. These results are observed and confirmed numerically. These variance rules are derived for theMCMCmethod but are also valid for the correlated samples obtained using other Monte Carlo methods. The main contribution of this work includes the theoretical proof of these numerical observations and the set of assumptions that lead to them. © 2014 Global-Science Press.

  14. Improving Hydrologic Data Assimilation by a Multivariate Particle Filter-Markov Chain Monte Carlo

    Yan, H.; DeChant, C. M.; Moradkhani, H.

    2014-12-01

    Data assimilation (DA) is a popular method for merging information from multiple sources (i.e. models and remotely sensing), leading to improved hydrologic prediction. With the increasing availability of satellite observations (such as soil moisture) in recent years, DA is emerging in operational forecast systems. Although these techniques have seen widespread application, developmental research has continued to further refine their effectiveness. This presentation will examine potential improvements to the Particle Filter (PF) through the inclusion of multivariate correlation structures. Applications of the PF typically rely on univariate DA schemes (such as assimilating the outlet observed discharge), and multivariate schemes generally ignore the spatial correlation of the observations. In this study, a multivariate DA scheme is proposed by introducing geostatistics into the newly developed particle filter with Markov chain Monte Carlo (PF-MCMC) method. This new method is assessed by a case study over one of the basin with natural hydrologic process in Model Parameter Estimation Experiment (MOPEX), located in Arizona. The multivariate PF-MCMC method is used to assimilate the Advanced Scatterometer (ASCAT) grid (12.5 km) soil moisture retrievals and the observed streamflow in five gages (four inlet and one outlet gages) into the Sacramento Soil Moisture Accounting (SAC-SMA) model for the same scale (12.5 km), leading to greater skill in hydrologic predictions.

  15. Fluctuation Theorems for Entropy Production and Heat Dissipation in Periodically Driven Markov Chains

    Shargel, Benjamin Hertz; Chou, Tom

    2009-10-01

    Asymptotic fluctuation theorems are statements of a Gallavotti-Cohen symmetry in the rate function of either the time-averaged entropy production or heat dissipation of a process. Such theorems have been proved for various general classes of continuous-time deterministic and stochastic processes, but always under the assumption that the forces driving the system are time independent, and often relying on the existence of a limiting ergodic distribution. In this paper we extend the asymptotic fluctuation theorem for the first time to inhomogeneous continuous-time processes without a stationary distribution, considering specifically a finite state Markov chain driven by periodic transition rates. We find that for both entropy production and heat dissipation, the usual Gallavotti-Cohen symmetry of the rate function is generalized to an analogous relation between the rate functions of the original process and its corresponding backward process, in which the trajectory and the driving protocol have been time-reversed. The effect is that spontaneous positive fluctuations in the long time average of each quantity in the forward process are exponentially more likely than spontaneous negative fluctuations in the backward process, and vice-versa, revealing that the distributions of fluctuations in universes in which time moves forward and backward are related. As an additional result, the asymptotic time-averaged entropy production is obtained as the integral of a periodic entropy production rate that generalizes the constant rate pertaining to homogeneous dynamics.

  16. FINITE MARKOV CHAINS IN THE MODEL REPRESENTATION OF THE HUMAN OPERATOR ACTIVITY IN QUASI-FUNCTIONAL ENVIRONMENT

    M. V. Serzhantova

    2016-05-01

    Full Text Available Subject of Research. We analyze the problems of finite Markov chains apparatus application for simulating a human operator activity in the quasi-static functional environment. It is shown that the functional environment stochastic nature is generated by a factor of interval character of human operator properties. Method. The problem is solved in the class of regular (recurrent finite Markov chains with three states of the human operator: with a favorable, median and unfavorable combination of the values of mathematical model parameters of the human operator in a quasi-static functional environment. The finite Markov chain is designed taking into account the factors of human operator tiredness and interval character of parameters of the model representation of his properties. The device is based on the usage of mathematical approximation of the standard curve of the human operator activity performance during work shift. The standard curve of the human operator activity performance is based on the extensive research experience of functional activity of the human operator with the help of photos of the day, his action timing and ergonomic generalizations. Main Results. The apparatus of regular finite Markov chains gave the possibility to evaluate correctly the human operator activity performance in a quasi-static functional environment with the use of the main information component of these chains as a vector of final probabilities. In addition, we managed to build an algorithmic basis for estimating the stationary time (time study for transit of human operator from arbitrary initial functional state into a state corresponding to a vector of final probabilities for a used chain after it reaches the final state based on the analysis of the eigenvalues spectrum of the matrix of transition probabilities for a regular (recurrent finite Markov chain. Practical Relevance. Obtained theoretical results are confirmed by illustrative examples, which

  17. An open Markov chain scheme model for a credit consumption portfolio fed by ARIMA and SARMA processes

    Esquível, Manuel L.; Fernandes, José Moniz; Guerreiro, Gracinda R.

    2016-06-01

    We introduce a schematic formalism for the time evolution of a random population entering some set of classes and such that each member of the population evolves among these classes according to a scheme based on a Markov chain model. We consider that the flow of incoming members is modeled by a time series and we detail the time series structure of the elements in each of the classes. We present a practical application to data from a credit portfolio of a Cape Verdian bank; after modeling the entering population in two different ways - namely as an ARIMA process and as a deterministic sigmoid type trend plus a SARMA process for the residues - we simulate the behavior of the population and compare the results. We get that the second method is more accurate in describing the behavior of the populations when compared to the observed values in a direct simulation of the Markov chain.

  18. On The Transition Probabilities for the Fuzzy States of a Fuzzy Markov Chain

    J.Earnest Lazarus Piriyakumar

    2015-12-01

    Full Text Available In this paper the theory of fuzzy logic is mixed with the theory of Markov systems and the abstraction of a Markov system with fuzzy states introduced. The notions such as fuzzy transient, fuzzy recurrent etc., were introduced. The results based on these notions are introduced.

  19. An optimal energy management system for islanded Microgrids based on multi-period artificial bee colony combined with Markov Chain

    Marzband, Mousa; Azarinejadian, Fatemeh; Savaghebi, Mehdi

    2017-01-01

    neural network combined with a Markov chain (ANN-MC) approach is used to predict nondispatchable power generation and load demand considering uncertainties. Furthermore, other capabilities such as extendibility, reliability, and flexibility are examined about the proposed approach......., the DR magnitude, the duration, and the minimum cost of energy. In this paper, a multiperiod artificial bee colony optimization algorithm is implemented for economic dispatch considering generation, storage, and responsive load offers. The better performance of the proposed algorithm is shown...

  20. A Markov chain Monte Carlo (MCMC) methodology with bootstrap percentile estimates for predicting presidential election results in Ghana

    Nortey, Ezekiel N. N.; Ansah-Narh, Theophilus; Asah-Asante, Richard; Minkah, Richard

    2015-01-01

    Although, there exists numerous literature on the procedure for forecasting or predicting election results, in Ghana only opinion poll strategies have been used. To fill this gap, the paper develops Markov chain models for forecasting the 2016 presidential election results at the Regional, Zonal (i.e. Savannah, Coastal and Forest) and the National levels using past presidential election results of Ghana. The methodology develops a model for prediction of the 2016 presidential election results...

  1. Clarification of basic factorization identity is for the almost semi-continuous latticed Poisson processes on the Markov chain

    Gerich M. S.

    2012-12-01

    Full Text Available Let ${xi(t, x(t}$ be a homogeneous semi-continuous lattice Poisson process on the Markov chain.The jumps of one sign are geometrically distributed, and jumps of the opposite sign are arbitrary latticed distribution. For a suchprocesses the relations for the components of two-sided matrix factorization are established.This relations define the moment genereting functions for extremumf of the process and their complements.

  2. Test of The Weak Form Efficient Market Hypothesis for The Istanbul Stock Exchange By Markov Chains Methodology

    KILIÇ, Öğr.Gör.Dr. Süleyman Bilgin

    2013-01-01

    In this study Markov chain methodology is used to test whether or not the daily returns of the Istanbul Stock Exchange ISE 100 index follows a martingale random walk process If the Weak Form Efficient Market Hypothesis EMH holds in any stock market stocks prices or returns follow a random walk process The random walk theory asserts that price movements will not follow any patterns or trends and that past price movements cannot be used to predict future price movements hence technic...

  3. Clarification of basic factorization identity is for the almost semi-continuous latticed Poisson processes on the Markov chain

    2012-01-01

    Let ${xi(t), x(t)}$ be a homogeneous semi-continuous lattice Poisson process on the Markov chain.The jumps of one sign are geometrically distributed, and jumps of the opposite sign are arbitrary latticed distribution. For a suchprocesses the relations for the components of two-sided matrix factorization are established.This relations define the moment genereting functions for extremumf of the process and their complements.

  4. The cluster index of regularly varying sequences with applications to limit theory for functions of multivariate Markov chains

    Mikosch, Thomas Valentin; Wintenberger, Olivier

    2014-01-01

    We introduce the cluster index of a multivariate stationary sequence and characterize the index in terms of the spectral tail process. This index plays a major role in limit theory for partial sums of sequences. We illustrate the use of the cluster index by characterizing infinite variance stable...... limit distributions and precise large deviation results for sums of multivariate functions acting on a stationary Markov chain under a drift condition....

  5. Multinomial Logistic Functions in Markov Chain Models of Sleep Architecture: Internal and External Validation and Covariate Analysis

    Bizzotto, Roberto; zamuner, stefano; Mezzalana, Enrica; De Nicolao, Giuseppe; Gomeni, Roberto; Hooker, Andrew C; Karlsson, Mats O.

    2011-01-01

    Mixed-effect Markov chain models have been recently proposed to characterize the time course of transition probabilities between sleep stages in insomniac patients. The most recent one, based on multinomial logistic functions, was used as a base to develop a final model combining the strengths of the existing ones. This final model was validated on placebo data applying also new diagnostic methods and then used for the inclusion of potential age, gender, and BMI effects. Internal validation w...

  6. A MAP-based image interpolation method via Viterbi decoding of Markov chains of interpolation functions.

    Vedadi, Farhang; Shirani, Shahram

    2014-01-01

    A new method of image resolution up-conversion (image interpolation) based on maximum a posteriori sequence estimation is proposed. Instead of making a hard decision about the value of each missing pixel, we estimate the missing pixels in groups. At each missing pixel of the high resolution (HR) image, we consider an ensemble of candidate interpolation methods (interpolation functions). The interpolation functions are interpreted as states of a Markov model. In other words, the proposed method undergoes state transitions from one missing pixel position to the next. Accordingly, the interpolation problem is translated to the problem of estimating the optimal sequence of interpolation functions corresponding to the sequence of missing HR pixel positions. We derive a parameter-free probabilistic model for this to-be-estimated sequence of interpolation functions. Then, we solve the estimation problem using a trellis representation and the Viterbi algorithm. Using directional interpolation functions and sequence estimation techniques, we classify the new algorithm as an adaptive directional interpolation using soft-decision estimation techniques. Experimental results show that the proposed algorithm yields images with higher or comparable peak signal-to-noise ratios compared with some benchmark interpolation methods in the literature while being efficient in terms of implementation and complexity considerations.

  7. Derivation of a Markov state model of the dynamics of a protein-like chain immersed in an implicit solvent.

    Schofield, Jeremy; Bayat, Hanif

    2014-09-07

    A Markov state model of the dynamics of a protein-like chain immersed in an implicit hard sphere solvent is derived from first principles for a system of monomers that interact via discontinuous potentials designed to account for local structure and bonding in a coarse-grained sense. The model is based on the assumption that the implicit solvent interacts on a fast time scale with the monomers of the chain compared to the time scale for structural rearrangements of the chain and provides sufficient friction so that the motion of monomers is governed by the Smoluchowski equation. A microscopic theory for the dynamics of the system is developed that reduces to a Markovian model of the kinetics under well-defined conditions. Microscopic expressions for the rate constants that appear in the Markov state model are analyzed and expressed in terms of a temperature-dependent linear combination of escape rates that themselves are independent of temperature. Excellent agreement is demonstrated between the theoretical predictions of the escape rates and those obtained through simulation of a stochastic model of the dynamics of bond formation. Finally, the Markov model is studied by analyzing the eigenvalues and eigenvectors of the matrix of transition rates, and the equilibration process for a simple helix-forming system from an ensemble of initially extended configurations to mainly folded configurations is investigated as a function of temperature for a number of different chain lengths. For short chains, the relaxation is primarily single-exponential and becomes independent of temperature in the low-temperature regime. The profile is more complicated for longer chains, where multi-exponential relaxation behavior is seen at intermediate temperatures followed by a low temperature regime in which the folding becomes rapid and single exponential. It is demonstrated that the behavior of the equilibration profile as the temperature is lowered can be understood in terms of the

  8. Additive N-step Markov chains as prototype model of symbolic stochastic dynamical systems with long-range correlations

    Mayzelis, Z.A. [Department of Physics, Kharkov National University, 4 Svoboda Sq., Kharkov 61077 (Ukraine); Apostolov, S.S. [Department of Physics, Kharkov National University, 4 Svoboda Sq., Kharkov 61077 (Ukraine); Melnyk, S.S. [A. Ya. Usikov Institute for Radiophysics and Electronics, Ukrainian Academy of Science, 12 Proskura Street, 61085 Kharkov (Ukraine); Usatenko, O.V. [A. Ya. Usikov Institute for Radiophysics and Electronics, Ukrainian Academy of Science, 12 Proskura Street, 61085 Kharkov (Ukraine)]. E-mail: usatenko@ire.kharkov.ua; Yampol' skii, V.A. [A. Ya. Usikov Institute for Radiophysics and Electronics, Ukrainian Academy of Science, 12 Proskura Street, 61085 Kharkov (Ukraine)

    2007-10-15

    A theory of symbolic dynamic systems with long-range correlations based on the consideration of the binary N-step Markov chains developed earlier in Phys Rev Lett 2003;90:110601 is generalized to the biased case (non-equal numbers of zeros and unities in the chain). In the model, the conditional probability that the ith symbol in the chain equals zero (or unity) is a linear function of the number of unities (zeros) among the preceding N symbols. The correlation and distribution functions as well as the variance of number of symbols in the words of arbitrary length L are obtained analytically and verified by numerical simulations. A self-similarity of the studied stochastic process is revealed and the similarity group transformation of the chain parameters is presented. The diffusion Fokker-Planck equation governing the distribution function of the L-words is explored. If the persistent correlations are not extremely strong, the distribution function is shown to be the Gaussian with the variance being nonlinearly dependent on L. An equation connecting the memory and correlation function of the additive Markov chain is presented. This equation allows reconstructing a memory function using a correlation function of the system. Effectiveness and robustness of the proposed method is demonstrated by simple model examples. Memory functions of concrete coarse-grained literary texts are found and their universal power-law behavior at long distances is revealed.

  9. Modelling heterotachy in phylogenetic inference by reversible-jump Markov chain Monte Carlo.

    Pagel, Mark; Meade, Andrew

    2008-12-27

    The rate at which a given site in a gene sequence alignment evolves over time may vary. This phenomenon--known as heterotachy--can bias or distort phylogenetic trees inferred from models of sequence evolution that assume rates of evolution are constant. Here, we describe a phylogenetic mixture model designed to accommodate heterotachy. The method sums the likelihood of the data at each site over more than one set of branch lengths on the same tree topology. A branch-length set that is best for one site may differ from the branch-length set that is best for some other site, thereby allowing different sites to have different rates of change throughout the tree. Because rate variation may not be present in all branches, we use a reversible-jump Markov chain Monte Carlo algorithm to identify those branches in which reliable amounts of heterotachy occur. We implement the method in combination with our 'pattern-heterogeneity' mixture model, applying it to simulated data and five published datasets. We find that complex evolutionary signals of heterotachy are routinely present over and above variation in the rate or pattern of evolution across sites, that the reversible-jump method requires far fewer parameters than conventional mixture models to describe it, and serves to identify the regions of the tree in which heterotachy is most pronounced. The reversible-jump procedure also removes the need for a posteriori tests of 'significance' such as the Akaike or Bayesian information criterion tests, or Bayes factors. Heterotachy has important consequences for the correct reconstruction of phylogenies as well as for tests of hypotheses that rely on accurate branch-length information. These include molecular clocks, analyses of tempo and mode of evolution, comparative studies and ancestral state reconstruction. The model is available from the authors' website, and can be used for the analysis of both nucleotide and morphological data.

  10. A Markov chain Monte Carlo with Gibbs sampling approach to anisotropic receiver function forward modeling

    Wirth, Erin A.; Long, Maureen D.; Moriarty, John C.

    2017-01-01

    Teleseismic receiver functions contain information regarding Earth structure beneath a seismic station. P-to-SV converted phases are often used to characterize crustal and upper-mantle discontinuities and isotropic velocity structures. More recently, P-to-SH converted energy has been used to interrogate the orientation of anisotropy at depth, as well as the geometry of dipping interfaces. Many studies use a trial-and-error forward modeling approach for the interpretation of receiver functions, generating synthetic receiver functions from a user-defined input model of Earth structure and amending this model until it matches major features in the actual data. While often successful, such an approach makes it impossible to explore model space in a systematic and robust manner, which is especially important given that solutions are likely non-unique. Here, we present a Markov chain Monte Carlo algorithm with Gibbs sampling for the interpretation of anisotropic receiver functions. Synthetic examples are used to test the viability of the algorithm, suggesting that it works well for models with a reasonable number of free parameters (<˜20). Additionally, the synthetic tests illustrate that certain parameters are well constrained by receiver function data, while others are subject to severe trade-offs-an important implication for studies that attempt to interpret Earth structure based on receiver function data. Finally, we apply our algorithm to receiver function data from station WCI in the central United States. We find evidence for a change in anisotropic structure at mid-lithospheric depths, consistent with previous work that used a grid search approach to model receiver function data at this station. Forward modeling of receiver functions using model space search algorithms, such as the one presented here, provide a meaningful framework for interrogating Earth structure from receiver function data.

  11. Input estimation for drug discovery using optimal control and Markov chain Monte Carlo approaches.

    Trägårdh, Magnus; Chappell, Michael J; Ahnmark, Andrea; Lindén, Daniel; Evans, Neil D; Gennemark, Peter

    2016-04-01

    Input estimation is employed in cases where it is desirable to recover the form of an input function which cannot be directly observed and for which there is no model for the generating process. In pharmacokinetic and pharmacodynamic modelling, input estimation in linear systems (deconvolution) is well established, while the nonlinear case is largely unexplored. In this paper, a rigorous definition of the input-estimation problem is given, and the choices involved in terms of modelling assumptions and estimation algorithms are discussed. In particular, the paper covers Maximum a Posteriori estimates using techniques from optimal control theory, and full Bayesian estimation using Markov Chain Monte Carlo (MCMC) approaches. These techniques are implemented using the optimisation software CasADi, and applied to two example problems: one where the oral absorption rate and bioavailability of the drug eflornithine are estimated using pharmacokinetic data from rats, and one where energy intake is estimated from body-mass measurements of mice exposed to monoclonal antibodies targeting the fibroblast growth factor receptor (FGFR) 1c. The results from the analysis are used to highlight the strengths and weaknesses of the methods used when applied to sparsely sampled data. The presented methods for optimal control are fast and robust, and can be recommended for use in drug discovery. The MCMC-based methods can have long running times and require more expertise from the user. The rigorous definition together with the illustrative examples and suggestions for software serve as a highly promising starting point for application of input-estimation methods to problems in drug discovery.

  12. A Markov Chain Monte Carlo Approach to Estimate AIDS after HIV Infection.

    Apenteng, Ofosuhene O; Ismail, Noor Azina

    2015-01-01

    The spread of human immunodeficiency virus (HIV) infection and the resulting acquired immune deficiency syndrome (AIDS) is a major health concern in many parts of the world, and mathematical models are commonly applied to understand the spread of the HIV epidemic. To understand the spread of HIV and AIDS cases and their parameters in a given population, it is necessary to develop a theoretical framework that takes into account realistic factors. The current study used this framework to assess the interaction between individuals who developed AIDS after HIV infection and individuals who did not develop AIDS after HIV infection (pre-AIDS). We first investigated how probabilistic parameters affect the model in terms of the HIV and AIDS population over a period of time. We observed that there is a critical threshold parameter, R0, which determines the behavior of the model. If R0 ≤ 1, there is a unique disease-free equilibrium; if R0 1, the disease-free equilibrium is unstable. We also show how a Markov chain Monte Carlo (MCMC) approach could be used as a supplement to forecast the numbers of reported HIV and AIDS cases. An approach using a Monte Carlo analysis is illustrated to understand the impact of model-based predictions in light of uncertain parameters on the spread of HIV. Finally, to examine this framework and demonstrate how it works, a case study was performed of reported HIV and AIDS cases from an annual data set in Malaysia, and then we compared how these approaches complement each other. We conclude that HIV disease in Malaysia shows epidemic behavior, especially in the context of understanding and predicting emerging cases of HIV and AIDS.

  13. Discovering beaten paths in collaborative ontology-engineering projects using Markov chains.

    Walk, Simon; Singer, Philipp; Strohmaier, Markus; Tudorache, Tania; Musen, Mark A; Noy, Natalya F

    2014-10-01

    Biomedical taxonomies, thesauri and ontologies in the form of the International Classification of Diseases as a taxonomy or the National Cancer Institute Thesaurus as an OWL-based ontology, play a critical role in acquiring, representing and processing information about human health. With increasing adoption and relevance, biomedical ontologies have also significantly increased in size. For example, the 11th revision of the International Classification of Diseases, which is currently under active development by the World Health Organization contains nearly 50,000 classes representing a vast variety of different diseases and causes of death. This evolution in terms of size was accompanied by an evolution in the way ontologies are engineered. Because no single individual has the expertise to develop such large-scale ontologies, ontology-engineering projects have evolved from small-scale efforts involving just a few domain experts to large-scale projects that require effective collaboration between dozens or even hundreds of experts, practitioners and other stakeholders. Understanding the way these different stakeholders collaborate will enable us to improve editing environments that support such collaborations. In this paper, we uncover how large ontology-engineering projects, such as the International Classification of Diseases in its 11th revision, unfold by analyzing usage logs of five different biomedical ontology-engineering projects of varying sizes and scopes using Markov chains. We discover intriguing interaction patterns (e.g., which properties users frequently change after specific given ones) that suggest that large collaborative ontology-engineering projects are governed by a few general principles that determine and drive development. From our analysis, we identify commonalities and differences between different projects that have implications for project managers, ontology editors, developers and contributors working on collaborative ontology

  14. Multilevel markov chain monte carlo method for high-contrast single-phase flow problems

    Efendiev, Yalchin R.

    2014-12-19

    In this paper we propose a general framework for the uncertainty quantification of quantities of interest for high-contrast single-phase flow problems. It is based on the generalized multiscale finite element method (GMsFEM) and multilevel Monte Carlo (MLMC) methods. The former provides a hierarchy of approximations of different resolution, whereas the latter gives an efficient way to estimate quantities of interest using samples on different levels. The number of basis functions in the online GMsFEM stage can be varied to determine the solution resolution and the computational cost, and to efficiently generate samples at different levels. In particular, it is cheap to generate samples on coarse grids but with low resolution, and it is expensive to generate samples on fine grids with high accuracy. By suitably choosing the number of samples at different levels, one can leverage the expensive computation in larger fine-grid spaces toward smaller coarse-grid spaces, while retaining the accuracy of the final Monte Carlo estimate. Further, we describe a multilevel Markov chain Monte Carlo method, which sequentially screens the proposal with different levels of approximations and reduces the number of evaluations required on fine grids, while combining the samples at different levels to arrive at an accurate estimate. The framework seamlessly integrates the multiscale features of the GMsFEM with the multilevel feature of the MLMC methods following the work in [26], and our numerical experiments illustrate its efficiency and accuracy in comparison with standard Monte Carlo estimates. © Global Science Press Limited 2015.

  15. Markov Chain Model-Based Optimal Cluster Heads Selection for Wireless Sensor Networks

    Ahmed, Gulnaz; Zou, Jianhua; Zhao, Xi; Sadiq Fareed, Mian Muhammad

    2017-01-01

    The longer network lifetime of Wireless Sensor Networks (WSNs) is a goal which is directly related to energy consumption. This energy consumption issue becomes more challenging when the energy load is not properly distributed in the sensing area. The hierarchal clustering architecture is the best choice for these kind of issues. In this paper, we introduce a novel clustering protocol called Markov chain model-based optimal cluster heads (MOCHs) selection for WSNs. In our proposed model, we introduce a simple strategy for the optimal number of cluster heads selection to overcome the problem of uneven energy distribution in the network. The attractiveness of our model is that the BS controls the number of cluster heads while the cluster heads control the cluster members in each cluster in such a restricted manner that a uniform and even load is ensured in each cluster. We perform an extensive range of simulation using five quality measures, namely: the lifetime of the network, stable and unstable region in the lifetime of the network, throughput of the network, the number of cluster heads in the network, and the transmission time of the network to analyze the proposed model. We compare MOCHs against Sleep-awake Energy Efficient Distributed (SEED) clustering, Artificial Bee Colony (ABC), Zone Based Routing (ZBR), and Centralized Energy Efficient Clustering (CEEC) using the above-discussed quality metrics and found that the lifetime of the proposed model is almost 1095, 2630, 3599, and 2045 rounds (time steps) greater than SEED, ABC, ZBR, and CEEC, respectively. The obtained results demonstrate that the MOCHs is better than SEED, ABC, ZBR, and CEEC in terms of energy efficiency and the network throughput. PMID:28241492

  16. Sparse Markov chain-based semi-supervised multi-instance multi-label method for protein function prediction.

    Han, Chao; Chen, Jian; Wu, Qingyao; Mu, Shuai; Min, Huaqing

    2015-10-01

    Automated assignment of protein function has received considerable attention in recent years for genome-wide study. With the rapid accumulation of genome sequencing data produced by high-throughput experimental techniques, the process of manually predicting functional properties of proteins has become increasingly cumbersome. Such large genomics data sets can only be annotated computationally. However, automated assignment of functions to unknown protein is challenging due to its inherent difficulty and complexity. Previous studies have revealed that solving problems involving complicated objects with multiple semantic meanings using the multi-instance multi-label (MIML) framework is effective. For the protein function prediction problems, each protein object in nature may associate with distinct structural units (instances) and multiple functional properties (class labels) where each unit is described by an instance and each functional property is considered as a class label. Thus, it is convenient and natural to tackle the protein function prediction problem by using the MIML framework. In this paper, we propose a sparse Markov chain-based semi-supervised MIML method, called Sparse-Markov. A sparse transductive probability graph is constructed to encode the affinity information of the data based on ensemble of Hausdorff distance metrics. Our goal is to exploit the affinity between protein objects in the sparse transductive probability graph to seek a sparse steady state probability of the Markov chain model to do protein function prediction, such that two proteins are given similar functional labels if they are close to each other in terms of an ensemble Hausdorff distance in the graph. Experimental results on seven real-world organism data sets covering three biological domains show that our proposed Sparse-Markov method is able to achieve better performance than four state-of-the-art MIML learning algorithms.

  17. Comparison of Bootstrapping and Markov Chain Monte Carlo for Copula Analysis of Hydrological Droughts

    Yang, P.; Ng, T. L.; Yang, W.

    2015-12-01

    Effective water resources management depends on the reliable estimation of the uncertainty of drought events. Confidence intervals (CIs) are commonly applied to quantify this uncertainty. A CI seeks to be at the minimal length necessary to cover the true value of the estimated variable with the desired probability. In drought analysis where two or more variables (e.g., duration and severity) are often used to describe a drought, copulas have been found suitable for representing the joint probability behavior of these variables. However, the comprehensive assessment of the parameter uncertainties of copulas of droughts has been largely ignored, and the few studies that have recognized this issue have not explicitly compared the various methods to produce the best CIs. Thus, the objective of this study to compare the CIs generated using two widely applied uncertainty estimation methods, bootstrapping and Markov Chain Monte Carlo (MCMC). To achieve this objective, (1) the marginal distributions lognormal, Gamma, and Generalized Extreme Value, and the copula functions Clayton, Frank, and Plackett are selected to construct joint probability functions of two drought related variables. (2) The resulting joint functions are then fitted to 200 sets of simulated realizations of drought events with known distribution and extreme parameters and (3) from there, using bootstrapping and MCMC, CIs of the parameters are generated and compared. The effect of an informative prior on the CIs generated by MCMC is also evaluated. CIs are produced for different sample sizes (50, 100, and 200) of the simulated drought events for fitting the joint probability functions. Preliminary results assuming lognormal marginal distributions and the Clayton copula function suggest that for cases with small or medium sample sizes (~50-100), MCMC to be superior method if an informative prior exists. Where an informative prior is unavailable, for small sample sizes (~50), both bootstrapping and MCMC

  18. Prediction on Human Resource Supply/Demand in Nuclear Industry Using Markov Chains Model and Job Coefficient

    Kwon, Hyuk; Min, Byung Joo; Lee, Eui Jin; You, Byung Hoon [Korea Atomic Energy Research Institute, Taejon (Korea, Republic of)

    2006-07-01

    According to the recent report by the OECD/NEA, there is a large imbalance between supply and demand of human resource in nuclear field. In the U.S., according to survey of Nuclear Engineering Department Heads Organization (NEDHO), 174 graduates in B.S or M.S degree were fed to nuclear industry in year 2004. Meanwhile, the total amount of demand in nuclear industry was about 642 engineers, which was approximately three times of the supply. In case of other developed western nations, the OECD/NEA report stated that the level of imbalance is similar to that of the U.S. However, nations having nuclear power development programs such as Korea, Japan and France seem to be in a different environment of supply and demand from that of the U.S. In this study, the difference of manpower status between the U.S and Korea has been investigated and the nuclear manpower required for the future in Korea is predicted. To investigate the factors making difference between the U.S. and NPP developing countries including Korea, a quantitative manpower planning model, Markov chains model, is applied. Since the Markov chains model has the strength of analyzing an inflow or push structure, the model fits the system governed by the inflow of manpower. A macroscopic status of manpower demand on nuclear industry is calculated up to 2015 using the Job coefficient (JC) and GDP, which are derived from the Survey for Roadmap of Electric Power Industry Manpower Planning. Furthermore, the total numbers of required manpower and supplied manpower up to 2030 were predicted by JC and Markov Chains model, respectively. Whereas the employee status of nuclear industries has been annually investigated by KAIF since 1995, the following data from the 10{sup th} survey and nuclear energy yearbooks from 1998 to 2005 are applied; (a) the status of the manpower demand of industry, (b) number of students entering, graduating and getting job in nuclear engineering.

  19. A Markov Chain Monte Carlo Algorithm for Infrasound Atmospheric Sounding: Application to the Humming Roadrunner experiment in New Mexico

    Lalande, Jean-Marie; Waxler, Roger; Velea, Doru

    2016-04-01

    As infrasonic waves propagate at long ranges through atmospheric ducts it has been suggested that observations of such waves can be used as a remote sensing techniques in order to update properties such as temperature and wind speed. In this study we investigate a new inverse approach based on Markov Chain Monte Carlo methods. This approach as the advantage of searching for the full Probability Density Function in the parameter space at a lower computational cost than extensive parameters search performed by the standard Monte Carlo approach. We apply this inverse methods to observations from the Humming Roadrunner experiment (New Mexico) and discuss implications for atmospheric updates, explosion characterization, localization and yield estimation.

  20. A new Markov Binomial distribution

    Leda D. Minkova; Omey, Edward

    2011-01-01

    In this paper, we introduce a two state homogeneous Markov chain and define a geometric distribution related to this Markov chain. We define also the negative binomial distribution similar to the classical case and call it NB related to interrupted Markov chain. The new binomial distribution is related to the interrupted Markov chain. Some characterization properties of the Geometric distributions are given. Recursion formulas and probability mass functions for the NB distribution and the new...

  1. Analysis of an on-line algorithm for solving large Markov chains

    Litvak, Nelly; Robert, Philippe

    2008-01-01

    Algorithms for ranking of web pages such as Google Page-Rank assign importance scores according to a stationary distribution of a Markov random walk on the web graph. Although in the classical search scheme the ranking scores are pre-computed off-line, several challenging problems in contemporary we

  2. Obesity status transitions across the elementary years: Use of Markov chain modeling

    Overweight and obesity status transition probabilities using first-order Markov transition models applied to elementary school children were assessed. Complete longitudinal data across eleven assessments were available from 1,494 elementary school children (from 7,599 students in 41 out of 45 school...

  3. Supply Chain as Complex Adaptive System and Its Modeling

    MingmingWang

    2004-01-01

    Supply chain is a complex, hierarchical, integrated, open and dynamic network.Every node in the network is an independent business unit that unites other organizations to develop its value, the competition and cooperation between these units are basic impetus of the development and evolution of the supply chain system. The characteristics of supply chain as a complex adaptive system and its modeling are discussed in this paper, and use an example demonstrating the feasibility of CAS modeling in supply chain management study.

  4. spMC: an R-package for 3D lithological reconstructions based on spatial Markov chains

    Sartore, Luca; Fabbri, Paolo; Gaetan, Carlo

    2016-09-01

    The paper presents the spatial Markov Chains (spMC) R-package and a case study of subsoil simulation/prediction located in a plain site of Northeastern Italy. spMC is a quite complete collection of advanced methods for data inspection, besides spMC implements Markov Chain models to estimate experimental transition probabilities of categorical lithological data. Furthermore, simulation methods based on most known prediction methods (as indicator Kriging and CoKriging) were implemented in spMC package. Moreover, other more advanced methods are available for simulations, e.g. path methods and Bayesian procedures, that exploit the maximum entropy. Since the spMC package was developed for intensive geostatistical computations, part of the code is implemented for parallel computations via the OpenMP constructs. A final analysis of this computational efficiency compares the simulation/prediction algorithms by using different numbers of CPU cores, and considering the example data set of the case study included in the package.

  5. A Markov chain Monte Carlo (MCMC) methodology with bootstrap percentile estimates for predicting presidential election results in Ghana.

    Nortey, Ezekiel N N; Ansah-Narh, Theophilus; Asah-Asante, Richard; Minkah, Richard

    2015-01-01

    Although, there exists numerous literature on the procedure for forecasting or predicting election results, in Ghana only opinion poll strategies have been used. To fill this gap, the paper develops Markov chain models for forecasting the 2016 presidential election results at the Regional, Zonal (i.e. Savannah, Coastal and Forest) and the National levels using past presidential election results of Ghana. The methodology develops a model for prediction of the 2016 presidential election results in Ghana using the Markov chains Monte Carlo (MCMC) methodology with bootstrap estimates. The results were that the ruling NDC may marginally win the 2016 Presidential Elections but would not obtain the more than 50 % votes to be declared an outright winner. This means that there is going to be a run-off election between the two giant political parties: the ruling NDC and the major opposition party, NPP. The prediction for the 2016 Presidential run-off election between the NDC and the NPP was rather in favour of the major opposition party, the NPP with a little over the 50 % votes obtained.

  6. Estimating Lower Bound and Upper Bound of a Markov chain over a noisy communication channel with Poisson distribution

    Vinay Mahajan

    2012-06-01

    Full Text Available Under the assumption that the encoders’ observations are conditionally independent Markov chains given an unobserved time-invariant random variable, results on the structure of optimal real-time encoding and decoding functions are obtained. The problem with noiseless channels and perfect memory at the receiver is then considered. A new methodology to find the structure of optimal real-time encoders is employed. A sufficient statistic with a time-invariant domain is found for this problem. This methodology exploits the presence of common information between the encoders and the receiver when communication is over noiseless channels. In this paper we estimate the lower bond, upper bond and define the encoder. In the previous design approach they follow Markov Chain approach to estimating the upper bound and define the encoder. In this dissertation we follow poison distribution to finding the lower bound and upper bound. Poisson can be viewed as an approximation to the binomial distribution. The approximation is good enough to be useful even when the sample size (N is only moderately large (say N > 50 and the probability (p is only relatively small (p < .2 The advantage of the Poisson distribution, of course, is that if N is large you need only know p to determine the approximate distribution of events. With the binomial distribution you also need to know N.

  7. Volume and Direction of the Atlantic Slave Trade, 1650-1870: Estimates by Markov Chain Carlo Analysis

    Patrick Manning

    2016-03-01

    Full Text Available This article presents methods and results in the application of the Markov Chain Monte Carlo (MCMC analysis to a problem in missing data. The data used here are from The Atlantic Slave Trade Database (TASTD, 2010 version, available online. Of the currently known 35,000 slaving voyages, original data on the size of the cargo of captives exist for some 25 percent of voyage embarkations in Africa and for about 50 percent of arrivals in the Americas. Previous efforts to estimate the missing data (and project the to- tal number of captives who made the transatlantic migration have proceeded through eclectic projections of maximum likelihood estimates of captives per voyage, without error margins. This paper creates new estimates of total mi- grant flow through two methods: one is a formally frequentist set of multiple methods, and the other is through Markov Chain Monte Carlo methodology. Comparison of the three methods, all based on the same raw data, show that the results of the two new methods are fairly close to one another and they yield total flows of migrant captives of more than 20 percent higher than the previous estimates. Quantitative results, presented in simplified graphs and tables within the text and in detailed spreadsheets available online, provide a new estimate of the volume of African embarkations and American arrivals in the transatlantic slave trade for the period from 1650 to 1870, by decade, for eleven African regions of embarkation and seven American and European regions of arrival.

  8. Averaging over fast variables in the fluid limit for Markov chains: application to the supermarket model with memory

    Luczak, M J

    2010-01-01

    We set out a general procedure which allows the approximation of certain Markov chains by the solutions of differential equations. The chains considered have some components which oscillate rapidly and randomly, while others are close to deterministic. The limiting dynamics are obtained by averaging the drift of the latter with respect to a local equilibrium distribution of the former. Some general estimates are proved under a uniform mixing condition on the fast variable which give explicit error probabilities for the fluid approximation. Mitzenmacher, Prabhakar and Shah \\cite{MPS} introduced a variant with memory of the `join the shortest queue' or `supermarket' model, and obtained a limit picture for the case of a stable system in which the number of queues and the total arrival rate are large. In this limit, the empirical distribution of queue sizes satisfies a differential equation, while the memory of the system oscillates rapidly and randomly. We illustrate our general fluid limit estimate in giving a ...

  9. Geochemical Characterization Using Geophysical Data and Markov Chain Monte Carlo Methods

    Chen, J.; Hubbard, S.; Rubin, Y.; Murray, C.; Roden, E.; Majer, E.

    2002-12-01

    if they were available from direct measurements or as variables otherwise. To estimate the geochemical parameters, we first assigned a prior model for each variable and a likelihood model for each type of data, which together define posterior probability distributions for each variable on the domain. Since the posterior probability distribution may involve hundreds of variables, we used a Markov Chain Monte Carlo (MCMC) method to explore each variable by generating and subsequently evaluating hundreds of realizations. Results from this case study showed that although geophysical attributes are not necessarily directly related to geochemical parameters, geophysical data could be very useful for providing accurate and high-resolution information about geochemical parameter distribution through their joint and indirect connections with hydrogeological properties such as lithofacies. This case study also demonstrated that MCMC methods were particularly useful for geochemical parameter estimation using geophysical data because they allow incorporation into the procedure of spatial correlation information, measurement errors, and cross correlations among different types of parameters.

  10. Theory of two-parameter Markov chain with an application in warranty study

    Calvache, Álvaro

    2012-01-01

    In this paper we present the classical results of Kolmogorov's backward and forward equations to the case of a two-parameter Markov process. These equations relates the infinitesimal transition matrix of the two-parameter Markov process. However, solving these equations is not possible and we require a numerical procedure. In this paper, we give an alternative method by use of double Laplace transform of the transition probability matrix and of the infinitesimal transition matrix of the process. An illustrative example is presented for the method proposed. In this example, we consider a two-parameter warranty model, in which a system can be any of these states: working, failure. We calculate the transition density matrix of these states and also the cost of the warranty for the proposed model.

  11. Reconstruction for distributed video coding: a Markov random field approach with context-adaptive smoothness prior

    Zhang, Yongsheng; Xiong, Hongkai; He, Zhihai; Yu, Songyu

    2010-07-01

    An important issue in Wyner-Ziv video coding is the reconstruction of Wyner-Ziv frames with decoded bit-planes. So far, there are two major approaches: the Maximum a Posteriori (MAP) reconstruction and the Minimum Mean Square Error (MMSE) reconstruction algorithms. However, these approaches do not exploit smoothness constraints in natural images. In this paper, we model a Wyner-Ziv frame by Markov random fields (MRFs), and produce reconstruction results by finding an MAP estimation of the MRF model. In the MRF model, the energy function consists of two terms: a data term, MSE distortion metric in this paper, measuring the statistical correlation between side-information and the source, and a smoothness term enforcing spatial coherence. In order to better describe the spatial constraints of images, we propose a context-adaptive smoothness term by analyzing the correspondence between the output of Slepian-Wolf decoding and successive frames available at decoders. The significance of the smoothness term varies in accordance with the spatial variation within different regions. To some extent, the proposed approach is an extension to the MAP and MMSE approaches by exploiting the intrinsic smoothness characteristic of natural images. Experimental results demonstrate a considerable performance gain compared with the MAP and MMSE approaches.

  12. Evaluation of air pollution control policies in Mexico City using finite Markov chain observation model

    Luis Hoyos; Pedro Lara; Elba Ortiz; Rafael López; Jesús González

    2009-01-01

    This paper proposes a Markov observation based model, where the transition matrix is formulated using air quality monitoring data for specific pollutant emissions, with the primary objective to analyze the corresponding stationary distributions and evaluate sceneries for the air quality impact of pollution control policies. The model is non predictive and could be applied to every source of pollutant emissions included in air monitoring data. Two cases of study are presented, ozone and sulfur...

  13. Markov chain-based analysis of a modified Cooper-Frieze model

    Jin-ying TONG; Zhen-ting HOU; Ding-hua SHI

    2009-01-01

    From the perspective of probability,the stability of a modified Cooper Frieze model is studied in the present paper.Based on the concept and technique of the first-passage probability in the Markov theory,we provide a rigorous proof for the existence of the steady-state degree distribution,and derive the explicit formula analytically.Moreover,we perform extensive numerical simulations of the model,including the degree distribution and the clustering.

  14. Transport-entropy inequalities and curvature in discrete-space Markov chains

    Eldan, Ronen; Lee, James R.; Lehec, Joseph

    2016-01-01

    We show that if the random walk on a graph has positive coarse Ricci curvature in the sense of Ollivier, then the stationary measure satisfies a W^1 transport-entropy inequality. Peres and Tetali have conjectured a stronger consequence, that a modified log-Sobolev inequality (MLSI) should hold, in analogy with the setting of Markov diffusions. We discuss how our entropy interpolation approach suggests a natural attack on the MLSI conjecture.

  15. Sequence-based Parameter Estimation for an Epidemiological Temporal Aftershock Forecasting Model using Markov Chain Monte Carlo Simulation

    Jalayer, Fatemeh; Ebrahimian, Hossein

    2014-05-01

    Introduction The first few days elapsed after the occurrence of a strong earthquake and in the presence of an ongoing aftershock sequence are quite critical for emergency decision-making purposes. Epidemic Type Aftershock Sequence (ETAS) models are used frequently for forecasting the spatio-temporal evolution of seismicity in the short-term (Ogata, 1988). The ETAS models are epidemic stochastic point process models in which every earthquake is a potential triggering event for subsequent earthquakes. The ETAS model parameters are usually calibrated a priori and based on a set of events that do not belong to the on-going seismic sequence (Marzocchi and Lombardi 2009). However, adaptive model parameter estimation, based on the events in the on-going sequence, may have several advantages such as, tuning the model to the specific sequence characteristics, and capturing possible variations in time of the model parameters. Simulation-based methods can be employed in order to provide a robust estimate for the spatio-temporal seismicity forecasts in a prescribed forecasting time interval (i.e., a day) within a post-main shock environment. This robust estimate takes into account the uncertainty in the model parameters expressed as the posterior joint probability distribution for the model parameters conditioned on the events that have already occurred (i.e., before the beginning of the forecasting interval) in the on-going seismic sequence. The Markov Chain Monte Carlo simulation scheme is used herein in order to sample directly from the posterior probability distribution for ETAS model parameters. Moreover, the sequence of events that is going to occur during the forecasting interval (and hence affecting the seismicity in an epidemic type model like ETAS) is also generated through a stochastic procedure. The procedure leads to two spatio-temporal outcomes: (1) the probability distribution for the forecasted number of events, and (2) the uncertainty in estimating the

  16. Application of Markov chain theory to ASTP natural environment launch criteria at Kennedy Space Center

    Graves, M. E.; Perlmutter, M.

    1974-01-01

    To aid the planning of the Apollo Soyuz Test Program (ASTP), certain natural environment statistical relationships are presented, based on Markov theory and empirical counts. The practical results are in terms of conditional probability of favorable and unfavorable launch conditions at Kennedy Space Center (KSC). They are based upon 15 years of recorded weather data which are analyzed under a set of natural environmental launch constraints. Three specific forecasting problems were treated: (1) the length of record of past weather which is useful to a prediction; (2) the effect of persistence in runs of favorable and unfavorable conditions; and (3) the forecasting of future weather in probabilistic terms.

  17. A Markov chain analysis of the effectiveness of drum-buffer-rope material flow management in job shop environment

    Masoud Rabbani

    2015-09-01

    Full Text Available The theory of constraints is an approach for production planning and control, which emphasizes on the constraints in the system to increase throughput. The theory of constraints is often referred to as Drum-Buffer-Rope developed originally by Goldratt. Drum-Buffer-Rope uses the drum or constraint to create a schedule based on the finite capacity of the first bottleneck. Because of complexity of the job shop environment, Drum-Buffer-Rope material flow management has very little attention to job shop environment. The objective of this paper is to apply the Drum-Buffer-Rope technique in the job shop environment using a Markov chain analysis to compare traditional method with Drum-Buffer-Rope. Four measurement parameters were considered and the result showed the advantage of Drum-Buffer-Rope approach compared with traditional one.

  18. Markov chain Monte Carlo searches for Galactic binaries in Mock LISA Data Challenge 1B data sets

    Trias, Miquel; Veitch, John

    2008-01-01

    We are developing a Bayesian approach based on Markov chain Monte Carlo techniques to search for and extract information about white dwarf binary systems with the Laser Interferometer Space Antenna (LISA). Here we present results obtained by applying an initial implementation of this method to some of the data sets released in Round 1B of the Mock LISA Data Challenges. For Challenges 1B.1.1a and 1b the signals were recovered with parameters lying within the 95.5% posterior probability interval and the correlation between the true and recovered waveform is in excess of 99%. Results were not submitted for Challenge 1B.1.1c due to some convergence problems of the algorithms, despite the signal was detected in a search over a 2 mHz band.

  19. Predicting Customer Churn and Retention Rates in Nigeria’s Mobile Telecommunication Industry Using Markov Chain Modelling

    Adebiyi Sulaimon Olanrewaju

    2015-12-01

    Full Text Available The telecommunication industry is one of the service industries that is most affected by the problem of subscribers’ churn. Although several techniques have been used to predict customer churn in developed countries, many of those studies used secondary data which are not readily available in Nigeria for researchers. This study investigates how Markov chains help in modelling and predicting the customer churn and retention rate in the Nigerian mobile telecommunication industry. The data generated through the survey were input in the Windows-based Quantitative System for Business (WinQSB for analysis. The results reveal that in the study area MTN has the highest retention rate (86.11%, followed by GLO (70.51%, Airtel (67%, and Etisalat (67.5%. This result has implications for telecom firms’ strategies for competitive advantage in particular and survival in general.

  20. Discrete probability models and methods probability on graphs and trees, Markov chains and random fields, entropy and coding

    Brémaud, Pierre

    2017-01-01

    The emphasis in this book is placed on general models (Markov chains, random fields, random graphs), universal methods (the probabilistic method, the coupling method, the Stein-Chen method, martingale methods, the method of types) and versatile tools (Chernoff's bound, Hoeffding's inequality, Holley's inequality) whose domain of application extends far beyond the present text. Although the examples treated in the book relate to the possible applications, in the communication and computing sciences, in operations research and in physics, this book is in the first instance concerned with theory. The level of the book is that of a beginning graduate course. It is self-contained, the prerequisites consisting merely of basic calculus (series) and basic linear algebra (matrices). The reader is not assumed to be trained in probability since the first chapters give in considerable detail the background necessary to understand the rest of the book. .

  1. Dark matter in disk galaxies I: a Markov Chain Monte Carlo method and application to DDO 154

    Hague, Peter R

    2013-01-01

    We present a new method to constrain the dark matter halo density profiles of disk galaxies. Our algorithm employs a Markov Chain Monte Carlo (MCMC) approach to explore the parameter space of a general family of dark matter profiles. We improve upon previous analyses by considering a wider range of halo profiles and by explicitly identifying cases in which the data are insufficient to break the degeneracies between the model parameters. We demonstrate the robustness of our algorithm using artificial data sets and show that reliable estimates of the halo density profile can be obtained from data of comparable quality to those currently available for low surface brightness (LSB) galaxies. We present our results in terms of physical quantities which are constrained by the data, and find that the logarithmic slope of the halo density profile at the radius of the innermost data point of a measured rotation curve can be strongly constrained in LSB galaxies. High surface brightness galaxies require additional inform...

  2. Application of stochastic automata networks for creation of continuous time Markov chain models of voltage gating of gap junction channels.

    Snipas, Mindaugas; Pranevicius, Henrikas; Pranevicius, Mindaugas; Pranevicius, Osvaldas; Paulauskas, Nerijus; Bukauskas, Feliksas F

    2015-01-01

    The primary goal of this work was to study advantages of numerical methods used for the creation of continuous time Markov chain models (CTMC) of voltage gating of gap junction (GJ) channels composed of connexin protein. This task was accomplished by describing gating of GJs using the formalism of the stochastic automata networks (SANs), which allowed for very efficient building and storing of infinitesimal generator of the CTMC that allowed to produce matrices of the models containing a distinct block structure. All of that allowed us to develop efficient numerical methods for a steady-state solution of CTMC models. This allowed us to accelerate CPU time, which is necessary to solve CTMC models, ~20 times.

  3. A hybrid Markov chain-von Mises density model for the drug-dosing interval and drug holiday distributions.

    Fellows, Kelly; Rodriguez-Cruz, Vivian; Covelli, Jenna; Droopad, Alyssa; Alexander, Sheril; Ramanathan, Murali

    2015-03-01

    Lack of adherence is a frequent cause of hospitalizations, but its effects on dosing patterns have not been extensively investigated. The purpose of this work was to critically evaluate a novel pharmacometric model for deriving the relationships of adherence to dosing patterns and the dosing interval distribution. The hybrid, stochastic model combines a Markov chain process with the von Mises distribution. The model was challenged with electronic medication monitoring data from 207 hypertension patients and against 5-year persistence data. The model estimates distributions of dosing runs, drug holidays, and dosing intervals. Drug holidays, which can vary between individuals with the same adherence, were characterized by the patient cooperativity index parameter. The drug holiday and dosing run distributions deviate markedly from normality. The dosing interval distribution exhibits complex patterns of multimodality and can be long-tailed. Dosing patterns are an important but under recognized covariate for explaining within-individual variance in drug concentrations.

  4. Optimized nested Markov chain Monte Carlo sampling: application to the liquid nitrogen Hugoniot using density functional theory

    Shaw, Milton Sam [Los Alamos National Laboratory; Coe, Joshua D [Los Alamos National Laboratory; Sewell, Thomas D [UNIV OF MISSOURI-COLUMBIA

    2009-01-01

    An optimized version of the Nested Markov Chain Monte Carlo sampling method is applied to the calculation of the Hugoniot for liquid nitrogen. The 'full' system of interest is calculated using density functional theory (DFT) with a 6-31 G* basis set for the configurational energies. The 'reference' system is given by a model potential fit to the anisotropic pair interaction of two nitrogen molecules from DFT calculations. The EOS is sampled in the isobaric-isothermal (NPT) ensemble with a trial move constructed from many Monte Carlo steps in the reference system. The trial move is then accepted with a probability chosen to give the full system distribution. The P's and T's of the reference and full systems are chosen separately to optimize the computational time required to produce the full system EOS. The method is numerically very efficient and predicts a Hugoniot in excellent agreement with experimental data.

  5. A Markov Chain Approach for Defining the Fundamental Efficiency Limits of Classical and Bifacial Multi-junction Tandem Solar Cells

    Alam, Muhammad A

    2016-01-01

    Bifacial tandem cells promise to reduce three fundamental losses (above-bandgap, below bandgap, and the uncollected light between panels) inherent in classical single junction PV systems. The successive filtering of light through the bandgap cascade, and requirement of current continuity make optimization of tandem cells difficult, accessible only to numerical solution through computer modeling. The challenge is even more complicated for bifacial design. In this paper, we use an elegantly simple Markov chain approach to show that the essential physics of optimization is intuitively obvious, and deeply insightful results can obtained analytically with a few lines of algebra. This powerful approach reproduces, as special cases, all the known results of traditional/bifacial tandem cells, and highlights the asymptotic efficiency gain of these technologies.

  6. Application of Stochastic Automata Networks for Creation of Continuous Time Markov Chain Models of Voltage Gating of Gap Junction Channels

    Mindaugas Snipas

    2015-01-01

    Full Text Available The primary goal of this work was to study advantages of numerical methods used for the creation of continuous time Markov chain models (CTMC of voltage gating of gap junction (GJ channels composed of connexin protein. This task was accomplished by describing gating of GJs using the formalism of the stochastic automata networks (SANs, which allowed for very efficient building and storing of infinitesimal generator of the CTMC that allowed to produce matrices of the models containing a distinct block structure. All of that allowed us to develop efficient numerical methods for a steady-state solution of CTMC models. This allowed us to accelerate CPU time, which is necessary to solve CTMC models, ∼20 times.

  7. INCOMPLETE SEMI-ITERATIVE METHODS FOR SOLVING SINGULAR LINEAR OPERATOR EQUATIONS IN BANACH SPACE WITH APPLICATIONS IN MARKOV CHAIN MODELING

    魏益民; 吴和兵

    2001-01-01

    We discuss the incomplete semi-iterative method (ISIM) for an approximate solution of a linear fixed point equations x= Tx+c with a bounded linear operator T acting on a complex Banach space X such that its resolvent has a pole of order k at the point 1. Sufficient conditions for the convergence of ISIM to a solution of x=Tx+c , where c belongs to the range space of (I-T)k, are established. We show that the ISIM has an attractive feature that it is usually convergent even when the spectral radius of the operator T is greater than 1 and Ind1T≥ 1. Applications in finite Markov chain is considered and illustrative examples are reported, showing the convergence rate of the ISIM is very high.

  8. Estimation of trace gas fluxes with objectively determined basis functions using reversible-jump Markov chain Monte Carlo

    Lunt, Mark F.; Rigby, Matt; Ganesan, Anita L.; Manning, Alistair J.

    2016-09-01

    Atmospheric trace gas inversions often attempt to attribute fluxes to a high-dimensional grid using observations. To make this problem computationally feasible, and to reduce the degree of under-determination, some form of dimension reduction is usually performed. Here, we present an objective method for reducing the spatial dimension of the parameter space in atmospheric trace gas inversions. In addition to solving for a set of unknowns that govern emissions of a trace gas, we set out a framework that considers the number of unknowns to itself be an unknown. We rely on the well-established reversible-jump Markov chain Monte Carlo algorithm to use the data to determine the dimension of the parameter space. This framework provides a single-step process that solves for both the resolution of the inversion grid, as well as the magnitude of fluxes from this grid. Therefore, the uncertainty that surrounds the choice of aggregation is accounted for in the posterior parameter distribution. The posterior distribution of this transdimensional Markov chain provides a naturally smoothed solution, formed from an ensemble of coarser partitions of the spatial domain. We describe the form of the reversible-jump algorithm and how it may be applied to trace gas inversions. We build the system into a hierarchical Bayesian framework in which other unknown factors, such as the magnitude of the model uncertainty, can also be explored. A pseudo-data example is used to show the usefulness of this approach when compared to a subjectively chosen partitioning of a spatial domain. An inversion using real data is also shown to illustrate the scales at which the data allow for methane emissions over north-west Europe to be resolved.

  9. Assessment of parameter uncertainty in hydrological model using a Markov-Chain-Monte-Carlo-based multilevel-factorial-analysis method

    Zhang, Junlong; Li, Yongping; Huang, Guohe; Chen, Xi; Bao, Anming

    2016-07-01

    Without a realistic assessment of parameter uncertainty, decision makers may encounter difficulties in accurately describing hydrologic processes and assessing relationships between model parameters and watershed characteristics. In this study, a Markov-Chain-Monte-Carlo-based multilevel-factorial-analysis (MCMC-MFA) method is developed, which can not only generate samples of parameters from a well constructed Markov chain and assess parameter uncertainties with straightforward Bayesian inference, but also investigate the individual and interactive effects of multiple parameters on model output through measuring the specific variations of hydrological responses. A case study is conducted for addressing parameter uncertainties in the Kaidu watershed of northwest China. Effects of multiple parameters and their interactions are quantitatively investigated using the MCMC-MFA with a three-level factorial experiment (totally 81 runs). A variance-based sensitivity analysis method is used to validate the results of parameters' effects. Results disclose that (i) soil conservation service runoff curve number for moisture condition II (CN2) and fraction of snow volume corresponding to 50% snow cover (SNO50COV) are the most significant factors to hydrological responses, implying that infiltration-excess overland flow and snow water equivalent represent important water input to the hydrological system of the Kaidu watershed; (ii) saturate hydraulic conductivity (SOL_K) and soil evaporation compensation factor (ESCO) have obvious effects on hydrological responses; this implies that the processes of percolation and evaporation would impact hydrological process in this watershed; (iii) the interactions of ESCO and SNO50COV as well as CN2 and SNO50COV have an obvious effect, implying that snow cover can impact the generation of runoff on land surface and the extraction of soil evaporative demand in lower soil layers. These findings can help enhance the hydrological model

  10. Computing the Stackelberg/Nash equilibria using the extraproximal method: Convergence analysis and implementation details for Markov chains games

    Trejo Kristal K.

    2015-06-01

    Full Text Available In this paper we present the extraproximal method for computing the Stackelberg/Nash equilibria in a class of ergodic controlled finite Markov chains games. We exemplify the original game formulation in terms of coupled nonlinear programming problems implementing the Lagrange principle. In addition, Tikhonov’s regularization method is employed to ensure the convergence of the cost-functions to a Stackelberg/Nash equilibrium point. Then, we transform the problem into a system of equations in the proximal format. We present a two-step iterated procedure for solving the extraproximal method: (a the first step (the extra-proximal step consists of a “prediction” which calculates the preliminary position approximation to the equilibrium point, and (b the second step is designed to find a “basic adjustment” of the previous prediction. The procedure is called the “extraproximal method” because of the use of an extrapolation. Each equation in this system is an optimization problem for which the necessary and efficient condition for a minimum is solved using a quadratic programming method. This solution approach provides a drastically quicker rate of convergence to the equilibrium point. We present the analysis of the convergence as well the rate of convergence of the method, which is one of the main results of this paper. Additionally, the extraproximal method is developed in terms of Markov chains for Stackelberg games. Our goal is to analyze completely a three-player Stackelberg game consisting of a leader and two followers. We provide all the details needed to implement the extraproximal method in an efficient and numerically stable way. For instance, a numerical technique is presented for computing the first step parameter (λ of the extraproximal method. The usefulness of the approach is successfully demonstrated by a numerical example related to a pricing oligopoly model for airlines companies.

  11. Fitting a distribution to censored contamination data using Markov Chain Monte Carlo methods and samples selected with unequal probabilities.

    Williams, Michael S; Ebel, Eric D

    2014-11-18

    The fitting of statistical distributions to chemical and microbial contamination data is a common application in risk assessment. These distributions are used to make inferences regarding even the most pedestrian of statistics, such as the population mean. The reason for the heavy reliance on a fitted distribution is the presence of left-, right-, and interval-censored observations in the data sets, with censored observations being the result of nondetects in an assay, the use of screening tests, and other practical limitations. Considerable effort has been expended to develop statistical distributions and fitting techniques for a wide variety of applications. Of the various fitting methods, Markov Chain Monte Carlo methods are common. An underlying assumption for many of the proposed Markov Chain Monte Carlo methods is that the data represent independent and identically distributed (iid) observations from an assumed distribution. This condition is satisfied when samples are collected using a simple random sampling design. Unfortunately, samples of food commodities are generally not collected in accordance with a strict probability design. Nevertheless, pseudosystematic sampling efforts (e.g., collection of a sample hourly or weekly) from a single location in the farm-to-table continuum are reasonable approximations of a simple random sample. The assumption that the data represent an iid sample from a single distribution is more difficult to defend if samples are collected at multiple locations in the farm-to-table continuum or risk-based sampling methods are employed to preferentially select samples that are more likely to be contaminated. This paper develops a weighted bootstrap estimation framework that is appropriate for fitting a distribution to microbiological samples that are collected with unequal probabilities of selection. An example based on microbial data, derived by the Most Probable Number technique, demonstrates the method and highlights the

  12. A Markov chain model for N-linked protein glycosylation – towards a low-parameter tool for model-driven glycoengineering

    Spahn, Philipp N.; Hansen, Anders Holmgaard; Hansen, Henning Gram

    2016-01-01

    present a novel low-parameter approach to describe glycosylation using flux-balance and Markov chain modeling. The model recapitulates the biological complexity of glycosylation, but does not require user-provided kinetic information. We use this method to predict and experimentally validate glycoprofiles...

  13. Inverse modeling of cloud-aerosol interactions -- Part 2: Sensitivity tests on liquid phase clouds using a Markov Chain Monte Carlo based simulation approach

    Partridge, D.G.; Vrugt, J.A.; Tunved, P.; Ekman, A.M.L.; Struthers, H.; Sooroshian, A.

    2012-01-01

    This paper presents a novel approach to investigate cloud-aerosol interactions by coupling a Markov chain Monte Carlo (MCMC) algorithm to an adiabatic cloud parcel model. Despite the number of numerical cloud-aerosol sensitivity studies previously conducted few have used statistical analysis tools t

  14. Recovery of Item Parameters in the Nominal Response Model: A Comparison of Marginal Maximum Likelihood Estimation and Markov Chain Monte Carlo Estimation.

    Wollack, James A.; Bolt, Daniel M.; Cohen, Allan S.; Lee, Young-Sun

    2002-01-01

    Compared the quality of item parameter estimates for marginal maximum likelihood (MML) and Markov Chain Monte Carlo (MCMC) with the nominal response model using simulation. The quality of item parameter recovery was nearly identical for MML and MCMC, and both methods tended to produce good estimates. (SLD)

  15. A Markov chain Monte Carlo Expectation Maximization Algorithm for Statistical Analysis of DNA Sequence Evolution with Neighbor-Dependent Substitution Rates

    Hobolth, Asger

    2008-01-01

    -dependent substitution models are analytically intractable and must be analyzed using either approximate or simulation-based methods. We describe statistical inference of neighbor-dependent models using a Markov chain Monte Carlo expectation maximization (MCMC-EM) algorithm. In the MCMC-EM algorithm, the high...

  16. MARKOV CHAIN MODEL FOR PROBABILITY OF DRY, WET DAYS AND STATISTICAL ANALISIS OF DAILY RAINFALL IN SOME CLIMATIC ZONE OF IRAN

    N. SHAHRAKI

    2013-03-01

    Full Text Available Water scarcity is a major problem in arid and semi-arid areas. The scarcity of water is further stressed by the growing demand due to increase in population growth in developing countries. Climate change and its outcomes on precipitation and water resources is the other problem in these areas. Several models are widely used for modeling daily precipitation occurrence. In this study, Markov Chain Model has been extensively used to study spell distribution. For this purpose, a day period was considered as the optimum length of time. Given the assumption that the Markov chain model is the right model for daily precipitation occurrence, the choice of Markov model order was examined on a daily basis for 4 synoptic weather stations with different climates in Iran (Gorgan, Khorram Abad, Zahedan, Tabrizduring 1978-2009. Based on probability rules, events possibility of sequential dry and wet days, these data were analyzed by stochastic process and Markov Chain method. Then probability matrix was calculated by maximum likelihood method. The possibility continuing2-5days of dry and wet days were calculated. The results showed that the probability maximum of consecutive dry period and climatic probability of dry days has occurred in Zahedan. The probability of consecutive dry period has fluctuated from 73.3 to 100 percent. Climatic probability of occurrence of dry days would change in the range of 70.96 to 100 percent with the average probability of about 90.45 percent.

  17. Transverse dispersion in ordered pillar arrays as a Markov chain: extension of the Galton-board model.

    Smirnov, Konstantin N; Shpigun, Oleg A

    2015-01-02

    An extension of the Galton-board model of the transverse solute dispersion in laminar flow through ordered arrays of non-porous cylindrical pillars was proposed. In contrast to the original model, which describes the dispersion process as a one-dimensional random walk with independent, equally probable steps, the extended model treats the process as a Markov chain, namely as a random walk with such correlated steps that the velocity-dependent probability to make a step in the same direction as the preceding step is smaller than the probability to reverse the direction of motion. The relationship between the average squared transverse displacement of the solute and the number of steps in the chain was used to find the expression for the velocity dependence of the transverse dispersion coefficient. The obtained equation differs from the one in the Galton-board model by the multiplier that accounts for the leveling-off of the experimental dependences at high reduced velocities. Although this modified Galton-board model cannot be directly applied to low velocities, a few additional assumptions lead to the expression that fits the whole range of the recent simulated dispersion data well.

  18. Weighted maximum posterior marginals for random fields using an ensemble of conditional densities from multiple Markov chain Monte Carlo simulations.

    Monaco, James Peter; Madabhushi, Anant

    2011-07-01

    The ability of classification systems to adjust their performance (sensitivity/specificity) is essential for tasks in which certain errors are more significant than others. For example, mislabeling cancerous lesions as benign is typically more detrimental than mislabeling benign lesions as cancerous. Unfortunately, methods for modifying the performance of Markov random field (MRF) based classifiers are noticeably absent from the literature, and thus most such systems restrict their performance to a single, static operating point (a paired sensitivity/specificity). To address this deficiency we present weighted maximum posterior marginals (WMPM) estimation, an extension of maximum posterior marginals (MPM) estimation. Whereas the MPM cost function penalizes each error equally, the WMPM cost function allows misclassifications associated with certain classes to be weighted more heavily than others. This creates a preference for specific classes, and consequently a means for adjusting classifier performance. Realizing WMPM estimation (like MPM estimation) requires estimates of the posterior marginal distributions. The most prevalent means for estimating these--proposed by Marroquin--utilizes a Markov chain Monte Carlo (MCMC) method. Though Marroquin's method (M-MCMC) yields estimates that are sufficiently accurate for MPM estimation, they are inadequate for WMPM. To more accurately estimate the posterior marginals we present an equally simple, but more effective extension of the MCMC method (E-MCMC). Assuming an identical number of iterations, E-MCMC as compared to M-MCMC yields estimates with higher fidelity, thereby 1) allowing a far greater number and diversity of operating points and 2) improving overall classifier performance. To illustrate the utility of WMPM and compare the efficacies of M-MCMC and E-MCMC, we integrate them into our MRF-based classification system for detecting cancerous glands in (whole-mount or quarter) histological sections of the prostate.

  19. Markov chain modelling analysis of HIV/AIDS progression: A race-based forecast in the United States

    S Lee

    2014-01-01

    Full Text Available HIV/AIDS has reached a pandemic level across the world with more than 33 million people who are living with HIV. In the United States, more than half a million people have been victims of AIDS. This study investigates the most vulnerable racial minority population (the African Americans in the United States and the second least affected (the Caucasians in order to predict the trends of the epidemic. A Markov chain analysis was used to model the progression of the disease among vulnerable people, infective people and AIDS cases for the two races separately, based on the 2009 Centers of Disease Control and Prevention HIV/AIDS Surveillance Report. Based on the Markov model, our study predicts that the number of African American people living with AIDS diagnosis and HIV infection and dead due to HIV/AIDS will be 662.2, 1225.3 and 62.9 in 2015 and 794.9, 1566.5 and 79.2 in 2030, respectively. The number of Caucasian people living with AIDS diagnosis and HIV infection and dead due to HIV/AIDS will be 96.4, 160 and 6.5 in 2015 and 118.6, 206.9 and 8.3 in 2030, respectively. The numbers of deaths due to HIV/AIDS are quite stable over the years in both the races. There is an increasing trend in the number of people living with HIV infection and AIDS diagnosis in Caucasians compared with African Americans. The absolute number of Caucasians living with AIDS diagnosis and HIV infection is quite smaller compared with African Americans. The results reveal discrepancy in HIV infection, AIDS diagnosis and deaths due to HIV/AIDS among the African Americans and the Caucasians races. There is a need for interventions focusing on HIV/AIDS prevention and management, optimum resource allocation and development of antiAIDS campaigns to reduce the infection rate.

  20. Markov Chain Modelling Analysis of HIV/AIDS Progression: A Race-based Forecast in the United States.

    Lee, S; Ko, J; Tan, Xi; Patel, Isha; Balkrishnan, R; Chang, J

    2014-03-01

    HIV/AIDS has reached a pandemic level across the world with more than 33 million people who are living with HIV. In the United States, more than half a million people have been victims of AIDS. This study investigates the most vulnerable racial minority population (the African Americans) in the United States and the second least affected (the Caucasians) in order to predict the trends of the epidemic. A Markov chain analysis was used to model the progression of the disease among vulnerable people, infective people and AIDS cases for the two races separately, based on the 2009 Centers of Disease Control and Prevention HIV/AIDS Surveillance Report. Based on the Markov model, our study predicts that the number of African American people living with AIDS diagnosis and HIV infection and dead due to HIV/AIDS will be 662.2, 1225.3 and 62.9 in 2015 and 794.9, 1566.5 and 79.2 in 2030, respectively. The number of Caucasian people living with AIDS diagnosis and HIV infection and dead due to HIV/AIDS will be 96.4, 160 and 6.5 in 2015 and 118.6, 206.9 and 8.3 in 2030, respectively. The numbers of deaths due to HIV/AIDS are quite stable over the years in both the races. There is an increasing trend in the number of people living with HIV infection and AIDS diagnosis in Caucasians compared with African Americans. The absolute number of Caucasians living with AIDS diagnosis and HIV infection is quite smaller compared with African Americans. The results reveal discrepancy in HIV infection, AIDS diagnosis and deaths due to HIV/AIDS among the African Americans and the Caucasians races. There is a need for interventions focusing on HIV/AIDS prevention and management, optimum resource allocation and development of antiAIDS campaigns to reduce the infection rate.