WorldWideScience

Sample records for adaptive markov chain

  1. An Adaptively Constructed Algebraic Multigrid Preconditioner for Irreducible Markov Chains

    OpenAIRE

    Brannick, James; Kahl, Karsten; Sokolovic, Sonja

    2014-01-01

    The computation of stationary distributions of Markov chains is an important task in the simulation of stochastic models. The linear systems arising in such applications involve non-symmetric M-matrices, making algebraic multigrid methods a natural choice for solving these systems. In this paper we investigate extensions and improvements of the bootstrap algebraic multigrid framework for solving these systems. This is achieved by reworking the bootstrap setup process to use singular vectors i...

  2. Adaptive continuous time Markov chain approximation model to general jump-diffusions

    OpenAIRE

    Mario Cerrato; Chia Chun Lo; Konstantinos Skindilias

    2011-01-01

    We propose a non-equidistant Q rate matrix formula and an adaptive numerical algorithm for a continuous time Markov chain to approximate jump-diffusions with affine or non-affine functional specifications. Our approach also accommodates state-dependent jump intensity and jump distribution, a flexibility that is very hard to achieve with other numerical methods. The Kolmogorov-Smirnov test shows that the proposed Markov chain transition density converges to the one given by the likelihood expa...

  3. Discrete Quantum Markov Chains

    CERN Document Server

    Faigle, Ulrich

    2010-01-01

    A framework for finite-dimensional quantum Markov chains on Hilbert spaces is introduced. Quantum Markov chains generalize both classical Markov chains with possibly hidden states and existing models of quantum walks on finite graphs. Quantum Markov chains are based on Markov operations that may be applied to quantum systems and include quantum measurements, for example. It is proved that quantum Markov chains are asymptotically stationary and hence possess ergodic and entropic properties. With a quantum Markov chain one may associate a quantum Markov process, which is a stochastic process in the classical sense. Generalized Markov chains allow a representation with respect to a generalized Markov source model with definite (but possibly hidden) states relative to which observables give rise to classical stochastic processes. It is demonstrated that this model allows for observables to violate Bell's inequality.

  4. Accelerating Markov chain Monte Carlo simulation by differential evolution with self-adaptive randomized subspace sampling

    Energy Technology Data Exchange (ETDEWEB)

    Vrugt, Jasper A [Los Alamos National Laboratory; Hyman, James M [Los Alamos National Laboratory; Robinson, Bruce A [Los Alamos National Laboratory; Higdon, Dave [Los Alamos National Laboratory; Ter Braak, Cajo J F [NETHERLANDS; Diks, Cees G H [UNIV OF AMSTERDAM

    2008-01-01

    Markov chain Monte Carlo (MCMC) methods have found widespread use in many fields of study to estimate the average properties of complex systems, and for posterior inference in a Bayesian framework. Existing theory and experiments prove convergence of well constructed MCMC schemes to the appropriate limiting distribution under a variety of different conditions. In practice, however this convergence is often observed to be disturbingly slow. This is frequently caused by an inappropriate selection of the proposal distribution used to generate trial moves in the Markov Chain. Here we show that significant improvements to the efficiency of MCMC simulation can be made by using a self-adaptive Differential Evolution learning strategy within a population-based evolutionary framework. This scheme, entitled DiffeRential Evolution Adaptive Metropolis or DREAM, runs multiple different chains simultaneously for global exploration, and automatically tunes the scale and orientation of the proposal distribution in randomized subspaces during the search. Ergodicity of the algorithm is proved, and various examples involving nonlinearity, high-dimensionality, and multimodality show that DREAM is generally superior to other adaptive MCMC sampling approaches. The DREAM scheme significantly enhances the applicability of MCMC simulation to complex, multi-modal search problems.

  5. Markov processes and controlled Markov chains

    CERN Document Server

    Filar, Jerzy; Chen, Anyue

    2002-01-01

    The general theory of stochastic processes and the more specialized theory of Markov processes evolved enormously in the second half of the last century. In parallel, the theory of controlled Markov chains (or Markov decision processes) was being pioneered by control engineers and operations researchers. Researchers in Markov processes and controlled Markov chains have been, for a long time, aware of the synergies between these two subject areas. However, this may be the first volume dedicated to highlighting these synergies and, almost certainly, it is the first volume that emphasizes the contributions of the vibrant and growing Chinese school of probability. The chapters that appear in this book reflect both the maturity and the vitality of modern day Markov processes and controlled Markov chains. They also will provide an opportunity to trace the connections that have emerged between the work done by members of the Chinese school of probability and the work done by the European, US, Central and South Ameri...

  6. Phasic Triplet Markov Chains.

    Science.gov (United States)

    El Yazid Boudaren, Mohamed; Monfrini, Emmanuel; Pieczynski, Wojciech; Aïssani, Amar

    2014-11-01

    Hidden Markov chains have been shown to be inadequate for data modeling under some complex conditions. In this work, we address the problem of statistical modeling of phenomena involving two heterogeneous system states. Such phenomena may arise in biology or communications, among other fields. Namely, we consider that a sequence of meaningful words is to be searched within a whole observation that also contains arbitrary one-by-one symbols. Moreover, a word may be interrupted at some site to be carried on later. Applying plain hidden Markov chains to such data, while ignoring their specificity, yields unsatisfactory results. The Phasic triplet Markov chain, proposed in this paper, overcomes this difficulty by means of an auxiliary underlying process in accordance with the triplet Markov chains theory. Related Bayesian restoration techniques and parameters estimation procedures according to the new model are then described. Finally, to assess the performance of the proposed model against the conventional hidden Markov chain model, experiments are conducted on synthetic and real data. PMID:26353069

  7. Variance bounding Markov chains

    OpenAIRE

    Roberts, Gareth O.; Jeffrey S. Rosenthal

    2008-01-01

    We introduce a new property of Markov chains, called variance bounding. We prove that, for reversible chains at least, variance bounding is weaker than, but closely related to, geometric ergodicity. Furthermore, variance bounding is equivalent to the existence of usual central limit theorems for all L2 functionals. Also, variance bounding (unlike geometric ergodicity) is preserved under the Peskun order. We close with some applications to Metropolis–Hastings algorithms.

  8. Musical Markov Chains

    Science.gov (United States)

    Volchenkov, Dima; Dawin, Jean René

    A system for using dice to compose music randomly is known as the musical dice game. The discrete time MIDI models of 804 pieces of classical music written by 29 composers have been encoded into the transition matrices and studied by Markov chains. Contrary to human languages, entropy dominates over redundancy, in the musical dice games based on the compositions of classical music. The maximum complexity is achieved on the blocks consisting of just a few notes (8 notes, for the musical dice games generated over Bach's compositions). First passage times to notes can be used to resolve tonality and feature a composer.

  9. Markov chains theory and applications

    CERN Document Server

    Sericola, Bruno

    2013-01-01

    Markov chains are a fundamental class of stochastic processes. They are widely used to solve problems in a large number of domains such as operational research, computer science, communication networks and manufacturing systems. The success of Markov chains is mainly due to their simplicity of use, the large number of available theoretical results and the quality of algorithms developed for the numerical evaluation of many metrics of interest.The author presents the theory of both discrete-time and continuous-time homogeneous Markov chains. He carefully examines the explosion phenomenon, the

  10. Hidden hybrid Markov/semi-Markov chains.

    OpenAIRE

    GUÉDON, YANN

    2005-01-01

    http://www.sciencedirect.com/science?ₒb=IssueURL&_tockey=%23TOC%235880%232005%23999509996%23596026%23FLA%23&ₐuth=y&view=c&ₐcct=C000056834&_version=1&_urlVersion=0&_userid=2292769&md5=87e7f8be94f92a8574da566c600ce631 International audience Models that combine Markovian states with implicit geometric state occupancy distributions and semi-Markovian states with explicit state occupancy distributions, are investigated. This type of model retains the flexibility of hidden semi-Markov chains ...

  11. Markov chains and mixing times

    CERN Document Server

    Levin, David A; Wilmer, Elizabeth L

    2009-01-01

    This book is an introduction to the modern approach to the theory of Markov chains. The main goal of this approach is to determine the rate of convergence of a Markov chain to the stationary distribution as a function of the size and geometry of the state space. The authors develop the key tools for estimating convergence times, including coupling, strong stationary times, and spectral methods. Whenever possible, probabilistic methods are emphasized. The book includes many examples and provides brief introductions to some central models of statistical mechanics. Also provided are accounts of r

  12. DREAM(D: an adaptive markov chain monte carlo simulation algorithm to solve discrete, noncontinuous, posterior parameter estimation problems

    Directory of Open Access Journals (Sweden)

    J. A. Vrugt

    2011-04-01

    Full Text Available Formal and informal Bayesian approaches are increasingly being used to treat forcing, model structural, parameter and calibration data uncertainty, and summarize hydrologic prediction uncertainty. This requires posterior sampling methods that approximate the (evolving posterior distribution. We recently introduced the DiffeRential Evolution Adaptive Metropolis (DREAM algorithm, an adaptive Markov Chain Monte Carlo (MCMC method that is especially designed to solve complex, high-dimensional and multimodal posterior probability density functions. The method runs multiple chains in parallel, and maintains detailed balance and ergodicity. Here, I present the latest algorithmic developments, and introduce a discrete sampling variant of DREAM that samples the parameter space at fixed points. The development of this new code, DREAM(D, has been inspired by the existing class of integer optimization problems, and emerging class of experimental design problems. Such non-continuous parameter estimation problems are of considerable theoretical and practical interest. The theory developed herein is applicable to DREAM(ZS (Vrugt et al., 2011 and MT-DREAM(ZS (Laloy and Vrugt, 2011 as well. Two case studies involving a sudoku puzzle and rainfall – runoff model calibration problem are used to illustrate DREAM(D.

  13. Beyond Markov Chains, Towards Adaptive Memristor Network-based Music Generation

    OpenAIRE

    Gale, Ella; Matthews, Oliver; Costello, Ben de Lacy; Adamatzky, Andrew

    2013-01-01

    We undertook a study of the use of a memristor network for music generation, making use of the memristor's memory to go beyond the Markov hypothesis. Seed transition matrices are created and populated using memristor equations, and which are shown to generate musical melodies and change in style over time as a result of feedback into the transition matrix. The spiking properties of simple memristor networks are demonstrated and discussed with reference to applications of music making. The lim...

  14. Consistency and Refinement for Interval Markov Chains

    DEFF Research Database (Denmark)

    Delahaye, Benoit; Larsen, Kim Guldstrand; Legay, Axel;

    2012-01-01

    Interval Markov Chains (IMC), or Markov Chains with probability intervals in the transition matrix, are the base of a classic specification theory for probabilistic systems [18]. The standard semantics of IMCs assigns to a specification the set of all Markov Chains that satisfy its interval const...

  15. Markov Chain Ontology Analysis (MCOA

    Directory of Open Access Journals (Sweden)

    Frost H

    2012-02-01

    Full Text Available Abstract Background Biomedical ontologies have become an increasingly critical lens through which researchers analyze the genomic, clinical and bibliographic data that fuels scientific research. Of particular relevance are methods, such as enrichment analysis, that quantify the importance of ontology classes relative to a collection of domain data. Current analytical techniques, however, remain limited in their ability to handle many important types of structural complexity encountered in real biological systems including class overlaps, continuously valued data, inter-instance relationships, non-hierarchical relationships between classes, semantic distance and sparse data. Results In this paper, we describe a methodology called Markov Chain Ontology Analysis (MCOA and illustrate its use through a MCOA-based enrichment analysis application based on a generative model of gene activation. MCOA models the classes in an ontology, the instances from an associated dataset and all directional inter-class, class-to-instance and inter-instance relationships as a single finite ergodic Markov chain. The adjusted transition probability matrix for this Markov chain enables the calculation of eigenvector values that quantify the importance of each ontology class relative to other classes and the associated data set members. On both controlled Gene Ontology (GO data sets created with Escherichia coli, Drosophila melanogaster and Homo sapiens annotations and real gene expression data extracted from the Gene Expression Omnibus (GEO, the MCOA enrichment analysis approach provides the best performance of comparable state-of-the-art methods. Conclusion A methodology based on Markov chain models and network analytic metrics can help detect the relevant signal within large, highly interdependent and noisy data sets and, for applications such as enrichment analysis, has been shown to generate superior performance on both real and simulated data relative to existing

  16. Spectral methods for quantum Markov chains

    International Nuclear Information System (INIS)

    The aim of this project is to contribute to our understanding of quantum time evolutions, whereby we focus on quantum Markov chains. The latter constitute a natural generalization of the ubiquitous concept of a classical Markov chain to describe evolutions of quantum mechanical systems. We contribute to the theory of such processes by introducing novel methods that allow us to relate the eigenvalue spectrum of the transition map to convergence as well as stability properties of the Markov chain.

  17. Stochastic seismic tomography by interacting Markov chains

    Science.gov (United States)

    Bottero, Alexis; Gesret, Alexandrine; Romary, Thomas; Noble, Mark; Maisons, Christophe

    2016-07-01

    Markov chain Monte Carlo sampling methods are widely used for non-linear Bayesian inversion where no analytical expression for the forward relation between data and model parameters is available. Contrary to the linear(ized) approaches they naturally allow to evaluate the uncertainties on the model found. Nevertheless their use is problematic in high dimensional model spaces especially when the computational cost of the forward problem is significant and/or the a posteriori distribution is multimodal. In this case the chain can stay stuck in one of the modes and hence not provide an exhaustive sampling of the distribution of interest. We present here a still relatively unknown algorithm that allows interaction between several Markov chains at different temperatures. These interactions (based on Importance Resampling) ensure a robust sampling of any posterior distribution and thus provide a way to efficiently tackle complex fully non linear inverse problems. The algorithm is easy to implement and is well adapted to run on parallel supercomputers. In this paper the algorithm is first introduced and applied to a synthetic multimodal distribution in order to demonstrate its robustness and efficiency compared to a Simulated Annealing method. It is then applied in the framework of first arrival traveltime seismic tomography on real data recorded in the context of hydraulic fracturing. To carry out this study a wavelet based adaptive model parametrization has been used. This allows to integrate the a priori information provided by sonic logs and to reduce optimally the dimension of the problem.

  18. Transition Probability Estimates for Reversible Markov Chains

    OpenAIRE

    Telcs, Andras

    2000-01-01

    This paper provides transition probability estimates of transient reversible Markov chains. The key condition of the result is the spatial symmetry and polynomial decay of the Green's function of the chain.

  19. Revisiting Causality in Markov Chains

    CERN Document Server

    Shojaee, Abbas

    2016-01-01

    Identifying causal relationships is a key premise of scientific research. The growth of observational data in different disciplines along with the availability of machine learning methods offers the possibility of using an empirical approach to identifying potential causal relationships, to deepen our understandings of causal behavior and to build theories accordingly. Conventional methods of causality inference from observational data require a considerable length of time series data to capture cause-effect relationship. We find that potential causal relationships can be inferred from the composition of one step transition rates to and from an event. Also known as Markov chain, one step transition rates are a commonly available resource in different scientific disciplines. Here we introduce a simple, effective and computationally efficient method that we termed 'Causality Inference using Composition of Transitions CICT' to reveal causal structure with high accuracy. We characterize the differences in causes,...

  20. Entropy Rate for Hidden Markov Chains with rare transitions

    OpenAIRE

    Peres, Yuval; Quas, Anthony

    2010-01-01

    We consider Hidden Markov Chains obtained by passing a Markov Chain with rare transitions through a noisy memoryless channel. We obtain asymptotic estimates for the entropy of the resulting Hidden Markov Chain as the transition rate is reduced to zero.

  1. Estimating hidden semi-Markov chains from discrete sequences.

    OpenAIRE

    Guédon, Yann

    2003-01-01

    International audience This article addresses the estimation of hidden semi-Markov chains from nonstationary discrete sequences. Hidden semi-Markov chains are particularly useful to model the succession of homogeneous zones or segments along sequences. A discrete hidden semi-Markov chain is composed of a nonobservable state process, which is a semi-Markov chain, and a discrete output process. Hidden semi-Markov chains generalize hidden Markov chains and enable the modeling of various durat...

  2. Markov chains models, algorithms and applications

    CERN Document Server

    Ching, Wai-Ki; Ng, Michael K; Siu, Tak-Kuen

    2013-01-01

    This new edition of Markov Chains: Models, Algorithms and Applications has been completely reformatted as a text, complete with end-of-chapter exercises, a new focus on management science, new applications of the models, and new examples with applications in financial risk management and modeling of financial data.This book consists of eight chapters.  Chapter 1 gives a brief introduction to the classical theory on both discrete and continuous time Markov chains. The relationship between Markov chains of finite states and matrix theory will also be highlighted. Some classical iterative methods

  3. Markov chains analytic and Monte Carlo computations

    CERN Document Server

    Graham, Carl

    2014-01-01

    Markov Chains: Analytic and Monte Carlo Computations introduces the main notions related to Markov chains and provides explanations on how to characterize, simulate, and recognize them. Starting with basic notions, this book leads progressively to advanced and recent topics in the field, allowing the reader to master the main aspects of the classical theory. This book also features: Numerous exercises with solutions as well as extended case studies.A detailed and rigorous presentation of Markov chains with discrete time and state space.An appendix presenting probabilistic notions that are nec

  4. Generalized crested products of Markov chains

    CERN Document Server

    D'Angeli, Daniele

    2010-01-01

    We define a finite Markov chain, called generalized crested product, which naturally appears as a generalization of the first crested product of Markov chains. A complete spectral analysis is developed and the $k$-step transition probability is given. It is important to remark that this Markov chain describes a more general version of the classical Ehrenfest diffusion model. As a particular case, one gets a generalization of the classical Insect Markov chain defined on the ultrametric space. Finally, an interpretation in terms of representation group theory is given, by showing the correspondence between the spectral decomposition of the generalized crested product and the Gelfand pairs associated with the generalized wreath product of permutation groups.

  5. Quantum Markov Chain Mixing and Dissipative Engineering

    DEFF Research Database (Denmark)

    Kastoryano, Michael James

    2012-01-01

    This thesis is the fruit of investigations on the extension of ideas of Markov chain mixing to the quantum setting, and its application to problems of dissipative engineering. A Markov chain describes a statistical process where the probability of future events depends only on the state of the sy....... Finally, we consider three independent tasks of dissipative engineering: dissipatively preparing a maximally entangled state of two atoms trapped in an optical cavity, dissipative preparation of graph states, and dissipative quantum computing construction....

  6. Markov chains and decision processes for engineers and managers

    CERN Document Server

    Sheskin, Theodore J

    2010-01-01

    Markov Chain Structure and ModelsHistorical NoteStates and TransitionsModel of the WeatherRandom WalksEstimating Transition ProbabilitiesMultiple-Step Transition ProbabilitiesState Probabilities after Multiple StepsClassification of StatesMarkov Chain StructureMarkov Chain ModelsProblemsReferencesRegular Markov ChainsSteady State ProbabilitiesFirst Passage to a Target StateProblemsReferencesReducible Markov ChainsCanonical Form of the Transition MatrixTh

  7. Cosmological Markov Chain Monte Carlo simulation with Cmbeasy

    CERN Document Server

    Müller, C M

    2004-01-01

    We introduce a Markov Chain Monte Carlo simulation and data analysis package for the cosmological computation package Cmbeasy. We have taken special care in implementing an adaptive step algorithm for the Markov Chain Monte Carlo in order to improve convergence. Data analysis routines are provided which allow to test models of the Universe against up-to-date measurements of the Cosmic Microwave Background, Supernovae Ia and Large Scale Structure. The observational data is provided with the software for convenient usage. The package is publicly available as part of the Cmbeasy software at www.cmbeasy.org.

  8. Entropy Computation in Partially Observed Markov Chains

    Science.gov (United States)

    Desbouvries, François

    2006-11-01

    Let X = {Xn}n∈N be a hidden process and Y = {Yn}n∈N be an observed process. We assume that (X,Y) is a (pairwise) Markov Chain (PMC). PMC are more general than Hidden Markov Chains (HMC) and yet enable the development of efficient parameter estimation and Bayesian restoration algorithms. In this paper we propose a fast (i.e., O(N)) algorithm for computing the entropy of {Xn}n=0N given an observation sequence {yn}n=0N.

  9. Performance Modeling of Communication Networks with Markov Chains

    CERN Document Server

    Mo, Jeonghoon

    2010-01-01

    This book is an introduction to Markov chain modeling with applications to communication networks. It begins with a general introduction to performance modeling in Chapter 1 where we introduce different performance models. We then introduce basic ideas of Markov chain modeling: Markov property, discrete time Markov chain (DTMe and continuous time Markov chain (CTMe. We also discuss how to find the steady state distributions from these Markov chains and how they can be used to compute the system performance metric. The solution methodologies include a balance equation technique, limiting probab

  10. Quantitative timed analysis of interactive Markov chains

    NARCIS (Netherlands)

    Guck, Dennis; Han, Tingting; Katoen, Joost-Pieter; Neuhausser, M.

    2012-01-01

    This paper presents new algorithms and accompanying tool support for analyzing interactive Markov chains (IMCs), a stochastic timed 1 1/2-player game in which delays are exponentially distributed. IMCs are compositional and act as semantic model for engineering formalisms such as AADL and dynamic fa

  11. Local stability in a transient Markov chain

    OpenAIRE

    Adan, Ivo; Foss, Sergey; Shneer, Seva; Weiss, Gideon

    2015-01-01

    We prove two lemmas with conditions that a system, which is described by a transient Markov chain, will display local stability. Examples of such systems include partly overloaded Jackson networks, partly overloaded polling systems, and overloaded multi-server queues with skill based service, under first come first served policy.

  12. Denumerable Markov decision chains: sensitive optimality criteria

    NARCIS (Netherlands)

    A. Hordijk (Arie); R. Dekker (Rommert)

    1991-01-01

    textabstractIn this paper we investigate denumerable state semi-Markov decision chains with small interest rates. We consider average and Blackwell optimality and allow for multiple closed sets and unbounded immediate rewards. Our analysis uses the existence of a Laurent series expansion for the tot

  13. Metric on state space of Markov chain

    OpenAIRE

    Rozinas, M. R.

    2010-01-01

    We consider finite irreducible Markov chains. It was shown that mean hitting time from one state to another satisfies the triangle inequality. Hence, sum of mean hitting time between couple of states in both directions is a metric on the space of states.

  14. Markov chains with quasitoeplitz transition matrix

    Directory of Open Access Journals (Sweden)

    Alexander M. Dukhovny

    1989-01-01

    Full Text Available This paper investigates a class of Markov chains which are frequently encountered in various applications (e.g. queueing systems, dams and inventories with feedback. Generating functions of transient and steady state probabilities are found by solving a special Riemann boundary value problem on the unit circle. A criterion of ergodicity is established.

  15. Markov Chains with Stochastically Stationary Transition Probabilities

    OpenAIRE

    Orey, Steven

    1991-01-01

    Markov chains on a countable state space are studied under the assumption that the transition probabilities $(P_n(x,y))$ constitute a stationary stochastic process. An introductory section exposing some basic results of Nawrotzki and Cogburn is followed by four sections of new results.

  16. Entropy rate of continuous-state hidden Markov chains

    OpenAIRE

    Han, G; Marcus, B

    2010-01-01

    We prove that under mild positivity assumptions, the entropy rate of a continuous-state hidden Markov chain, observed when passing a finite-state Markov chain through a discrete-time continuous-output channel, is analytic as a function of the transition probabilities of the underlying Markov chain. We further prove that the entropy rate of a continuous-state hidden Markov chain, observed when passing a mixing finite-type constrained Markov chain through a discrete-time Gaussian channel, is sm...

  17. MARKOV CHAIN PORTFOLIO LIQUIDITY OPTIMIZATION MODEL

    Directory of Open Access Journals (Sweden)

    Eder Oliveira Abensur

    2014-05-01

    Full Text Available The international financial crisis of September 2008 and May 2010 showed the importance of liquidity as an attribute to be considered in portfolio decisions. This study proposes an optimization model based on available public data, using Markov chain and Genetic Algorithms concepts as it considers the classic duality of risk versus return and incorporating liquidity costs. The work intends to propose a multi-criterion non-linear optimization model using liquidity based on a Markov chain. The non-linear model was tested using Genetic Algorithms with twenty five Brazilian stocks from 2007 to 2009. The results suggest that this is an innovative development methodology and useful for developing an efficient and realistic financial portfolio, as it considers many attributes such as risk, return and liquidity.

  18. Handbook of Markov chain Monte Carlo

    CERN Document Server

    Brooks, Steve

    2011-01-01

    ""Handbook of Markov Chain Monte Carlo"" brings together the major advances that have occurred in recent years while incorporating enough introductory material for new users of MCMC. Along with thorough coverage of the theoretical foundations and algorithmic and computational methodology, this comprehensive handbook includes substantial realistic case studies from a variety of disciplines. These case studies demonstrate the application of MCMC methods and serve as a series of templates for the construction, implementation, and choice of MCMC methodology.

  19. Bayesian Posterior Distributions Without Markov Chains

    OpenAIRE

    Cole, Stephen R.; Chu, Haitao; Greenland, Sander; Hamra, Ghassan; Richardson, David B.

    2012-01-01

    Bayesian posterior parameter distributions are often simulated using Markov chain Monte Carlo (MCMC) methods. However, MCMC methods are not always necessary and do not help the uninitiated understand Bayesian inference. As a bridge to understanding Bayesian inference, the authors illustrate a transparent rejection sampling method. In example 1, they illustrate rejection sampling using 36 cases and 198 controls from a case-control study (1976–1983) assessing the relation between residential ex...

  20. Markov Chain Monte Carlo and Irreversibility

    Science.gov (United States)

    Ottobre, Michela

    2016-06-01

    Markov Chain Monte Carlo (MCMC) methods are statistical methods designed to sample from a given measure π by constructing a Markov chain that has π as invariant measure and that converges to π. Most MCMC algorithms make use of chains that satisfy the detailed balance condition with respect to π; such chains are therefore reversible. On the other hand, recent work [18, 21, 28, 29] has stressed several advantages of using irreversible processes for sampling. Roughly speaking, irreversible diffusions converge to equilibrium faster (and lead to smaller asymptotic variance as well). In this paper we discuss some of the recent progress in the study of nonreversible MCMC methods. In particular: i) we explain some of the difficulties that arise in the analysis of nonreversible processes and we discuss some analytical methods to approach the study of continuous-time irreversible diffusions; ii) most of the rigorous results on irreversible diffusions are available for continuous-time processes; however, for computational purposes one needs to discretize such dynamics. It is well known that the resulting discretized chain will not, in general, retain all the good properties of the process that it is obtained from. In particular, if we want to preserve the invariance of the target measure, the chain might no longer be reversible. Therefore iii) we conclude by presenting an MCMC algorithm, the SOL-HMC algorithm [23], which results from a nonreversible discretization of a nonreversible dynamics.

  1. Adaptive Partially Hidden Markov Models

    DEFF Research Database (Denmark)

    Forchhammer, Søren Otto; Rasmussen, Tage

    1996-01-01

    Partially Hidden Markov Models (PHMM) have recently been introduced. The transition and emission probabilities are conditioned on the past. In this report, the PHMM is extended with a multiple token version. The different versions of the PHMM are applied to bi-level image coding.......Partially Hidden Markov Models (PHMM) have recently been introduced. The transition and emission probabilities are conditioned on the past. In this report, the PHMM is extended with a multiple token version. The different versions of the PHMM are applied to bi-level image coding....

  2. POISSON LIMIT THEOREM FOR COUNTABLE MARKOV CHAINS IN MARKOVIAN ENVIRONMENTS

    Institute of Scientific and Technical Information of China (English)

    方大凡; 王汉兴; 唐矛宁

    2003-01-01

    A countable Markov chain in a Markovian environment is considered. A Poisson limit theorem for the chain recurring to small cylindrical sets is mainly achieved. In order to prove this theorem, the entropy function h is introduced and the Shannon-McMillan-Breiman theorem for the Markov chain in a Markovian environment is shown. It' s well-known that a Markov process in a Markovian environment is generally not a standard Markov chain, so an example of Poisson approximation for a process which is not a Markov process is given. On the other hand, when the environmental process degenerates to a constant sequence, a Poisson limit theorem for countable Markov chains, which is the generalization of Pitskel's result for finite Markov chains is obtained.

  3. On the embedding problem for discrete-time Markov chains

    OpenAIRE

    Guerry, Marie-Anne

    2013-01-01

    When a discrete-time homogenous Markov chain is observed at time intervals that correspond to its time unit, then the transition probabilities of the chain can be estimated using known maximum likelihood estimators. In this paper we consider a situation when a Markov chain is observed on time intervals with length equal to twice the time unit of the Markov chain. The issue then arises of characterizing probability matrices whose square root(s) are also probability matrices. ...

  4. A Bootstrap Algebraic Multilevel method for Markov Chains

    CERN Document Server

    Bolten, M; Brannick, J; Frommer, A; Kahl, K; Livshits, I

    2010-01-01

    This work concerns the development of an Algebraic Multilevel method for computing stationary vectors of Markov chains. We present an efficient Bootstrap Algebraic Multilevel method for this task. In our proposed approach, we employ a multilevel eigensolver, with interpolation built using ideas based on compatible relaxation, algebraic distances, and least squares fitting of test vectors. Our adaptive variational strategy for computation of the state vector of a given Markov chain is then a combination of this multilevel eigensolver and associated multilevel preconditioned GMRES iterations. We show that the Bootstrap AMG eigensolver by itself can efficiently compute accurate approximations to the state vector. An additional benefit of the Bootstrap approach is that it yields an accurate interpolation operator for many other eigenmodes. This in turn allows for the use of the resulting AMG hierarchy to accelerate the MLE steps using standard multigrid correction steps. The proposed approach is applied to a rang...

  5. Growth and dissolution of macromolecular Markov chains

    CERN Document Server

    Gaspard, Pierre

    2016-01-01

    The kinetics and thermodynamics of free living copolymerization are studied for processes with rates depending on k monomeric units of the macromolecular chain behind the unit that is attached or detached. In this case, the sequence of monomeric units in the growing copolymer is a kth-order Markov chain. In the regime of steady growth, the statistical properties of the sequence are determined analytically in terms of the attachment and detachment rates. In this way, the mean growth velocity as well as the thermodynamic entropy production and the sequence disorder can be calculated systematically. These different properties are also investigated in the regime of depolymerization where the macromolecular chain is dissolved by the surrounding solution. In this regime, the entropy production is shown to satisfy Landauer's principle.

  6. Application of Markov Chains to Stock Trends

    Directory of Open Access Journals (Sweden)

    Kevin J. Doubleday

    2011-01-01

    Full Text Available Problem statement: Modeling of the Dow Jones Industrial Average is frequently attempted in order to determine trading strategies with maximum payoff. Changes in the DJIA are important since movements may affect both individuals and corporations profoundly. Previous work showed that modeling a market as a random walk was valid and that a market may be viewed as having the Markov property. Approach: The aim of this research was to determine the relationship between a diverse portfolio of stocks and the market as a whole. To that end, the DJIA was analyzed using a discrete time stochastic model, namely a Markov Chain. Two models were highlighted, where the DJIA was considered as being in a state of (1 gain or loss and (2 small, moderate, or large gain or loss. A portfolio of five stocks was then considered and two models of the portfolio much the same as those for the DJIA. These models were used to obtain transitional probabilities and steady state probabilities. Results: Our results indicated that the portfolio behaved similarly to the entire DJIA, both in the simple model and the partitioned model. Conclusion: When treated as a Markov process, the entire market was useful in gauging how a diverse portfolio of stocks might behave. Future work may include different classifications of states to refine the transition matrices.

  7. Markov Chain Analysis of Musical Dice Games

    Science.gov (United States)

    Volchenkov, D.; Dawin, J. R.

    2012-07-01

    A system for using dice to compose music randomly is known as the musical dice game. The discrete time MIDI models of 804 pieces of classical music written by 29 composers have been encoded into the transition matrices and studied by Markov chains. Contrary to human languages, entropy dominates over redundancy, in the musical dice games based on the compositions of classical music. The maximum complexity is achieved on the blocks consisting of just a few notes (8 notes, for the musical dice games generated over Bach's compositions). First passage times to notes can be used to resolve tonality and feature a composer.

  8. Analyticity of entropy rate of hidden Markov chains

    OpenAIRE

    Han, G; Marcus, B

    2006-01-01

    We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An example is given to estimate the radius of convergence for the entropy rate. We then show that the positivity assumptions can be relaxed, and examples are given for the relaxed conditions. We study a special class of hidden Markov chains in more detail: bin...

  9. Hitting time and inverse problems for Markov chains

    OpenAIRE

    de la Peña, Victor; Gzyl, Henryk; McDonald, Patrick

    2008-01-01

    Let Wn be a simple Markov chain on the integers. Suppose that Xn is a simple Markov chain on the integers whose transition probabilities coincide with those of Wn off a finite set. We prove that there is an M > 0 such that the Markov chain Wn and the joint distributions of the first hitting time and first hitting place of Xn started at the origin for the sets {-M, M} and {-(M + 1), (M + 1)} algorithmically determine the transition probabilities of Xn.

  10. Bounds on Lifting Continuous Markov Chains to Speed Up Mixing

    OpenAIRE

    Ramanan, Kavita; Smith, Aaron

    2016-01-01

    It is often possible to speed up the mixing of a Markov chain $\\{ X_{t} \\}_{t \\in \\mathbb{N}}$ on a state space $\\Omega$ by \\textit{lifting}, that is, running a more efficient Markov chain $\\{ \\hat{X}_{t} \\}_{t \\in \\mathbb{N}}$ on a larger state space $\\hat{\\Omega} \\supset \\Omega$ that projects to $\\{ X_{t} \\}_{t \\in \\mathbb{N}}$ in a certain sense. In [CLP99], Chen, Lov{\\'a}sz and Pak prove that for Markov chains on finite state spaces, the mixing time of any lift of a Markov chain is at lea...

  11. Remarks on a monotone Markov chain

    Directory of Open Access Journals (Sweden)

    P. Todorovic

    1987-01-01

    Full Text Available In applications, considerations on stochastic models often involve a Markov chain {ζn}0∞ with state space in R+, and a transition probability Q. For each x  R+ the support of Q(x,. is [0,x]. This implies that ζ0≥ζ1≥…. Under certain regularity assumptions on Q we show that Qn(x,Bu→1 as n→∞ for all u>0 and that 1−Qn(x,Bu≤[1−Q(x,Bu]n where Bu=[0,u. Set τ0=max{k;ζk=ζ0}, τn=max{k;ζk=ζτn−1+1} and write Xn=ζτn−1+1, Tn=τn−τn−1. We investigate some properties of the imbedded Markov chain {Xn}0∞ and of {Tn}0∞. We determine all the marginal distributions of {Tn}0∞ and show that it is asymptotically stationary and that it possesses a monotonicity property. We also prove that under some mild regularity assumptions on β(x=1−Q(x,Bx, ∑1n(Ti−a/bn→dZ∼N(0,1.

  12. Recursive smoothers for hidden discrete-time Markov chains

    Directory of Open Access Journals (Sweden)

    Lakhdar Aggoun

    2005-01-01

    Full Text Available We consider a discrete-time Markov chain observed through another Markov chain. The proposed model extends models discussed by Elliott et al. (1995. We propose improved recursive formulae to update smoothed estimates of processes related to the model. These recursive estimates are used to update the parameter of the model via the expectation maximization (EM algorithm.

  13. Markov chain approach to identifying Wiener systems

    Institute of Scientific and Technical Information of China (English)

    ZHAO WenXiao; CHEN HanFu

    2012-01-01

    Identification of the Wiener system composed of an infinite impulse response (IIR) linear subsystem followed by a static nonlinearity is considered.The recursive estimates for unknown coefficients of the linear subsystem and for the values of the nonlinear function at any fixed points are given by the stochastic approximation algorithms with expanding truncations (SAAWET).With the help of properties of the Markov chain connected with the linear subsystem,all estimates derived in the paper are proved to be strongly consistent.In comparison with the existing results on the topic,the method presented in the paper simplifies the convergence analysis and requires weaker conditions.A numerical example is given,and the simulation results are consistent with the theoretical analysis.

  14. NONLINEAR EXPECTATIONS AND NONLINEAR MARKOV CHAINS

    Institute of Scientific and Technical Information of China (English)

    PENG SHIGE

    2005-01-01

    This paper deals with nonlinear expectations. The author obtains a nonlinear generalization of the well-known Kolmogorov's consistent theorem and then use it to construct filtration-consistent nonlinear expectations via nonlinear Markov chains. Compared to the author's previous results, i.e., the theory of g-expectations introduced via BSDE on a probability space, the present framework is not based on a given probability measure. Many fully nonlinear and singular situations are covered. The induced topology is a natural generalization of Lp-norms and L∞-norm in linear situations.The author also obtains the existence and uniqueness result of BSDE under this new framework and develops a nonlinear type of von Neumann-Morgenstern representation theorem to utilities and present dynamic risk measures.

  15. A Markov Chain Model for Contagion

    Directory of Open Access Journals (Sweden)

    Angelos Dassios

    2014-11-01

    Full Text Available We introduce a bivariate Markov chain counting process with contagion for modelling the clustering arrival of loss claims with delayed settlement for an insurance company. It is a general continuous-time model framework that also has the potential to be applicable to modelling the clustering arrival of events, such as jumps, bankruptcies, crises and catastrophes in finance, insurance and economics with both internal contagion risk and external common risk. Key distributional properties, such as the moments and probability generating functions, for this process are derived. Some special cases with explicit results and numerical examples and the motivation for further actuarial applications are also discussed. The model can be considered a generalisation of the dynamic contagion process introduced by Dassios and Zhao (2011.

  16. Introduction to the numerical solutions of Markov chains

    CERN Document Server

    Stewart, Williams J

    1994-01-01

    A cornerstone of applied probability, Markov chains can be used to help model how plants grow, chemicals react, and atoms diffuse - and applications are increasingly being found in such areas as engineering, computer science, economics, and education. To apply the techniques to real problems, however, it is necessary to understand how Markov chains can be solved numerically. In this book, the first to offer a systematic and detailed treatment of the numerical solution of Markov chains, William Stewart provides scientists on many levels with the power to put this theory to use in the actual world, where it has applications in areas as diverse as engineering, economics, and education. His efforts make for essential reading in a rapidly growing field. Here, Stewart explores all aspects of numerically computing solutions of Markov chains, especially when the state is huge. He provides extensive background to both discrete-time and continuous-time Markov chains and examines many different numerical computing metho...

  17. Semi-Markov Chains and Hidden Semi-Markov Models toward Applications Their Use in Reliability and DNA Analysis

    CERN Document Server

    Barbu, Vlad

    2008-01-01

    Semi-Markov processes are much more general and better adapted to applications than the Markov ones because sojourn times in any state can be arbitrarily distributed, as opposed to the geometrically distributed sojourn time in the Markov case. This book concerns with the estimation of discrete-time semi-Markov and hidden semi-Markov processes

  18. Tight Markov chains and random compositions

    CERN Document Server

    Pittel, Boris

    2010-01-01

    For an ergodic Markov chain $\\{X(t)\\}$ on $\\Bbb N$, with a stationary distribution $\\pi$, let $T_n>0$ denote a hitting time for $[n]^c$, and let $X_n=X(T_n)$. Around 2005 Guy Louchard popularized a conjecture that, for $n\\to \\infty$, $T_n$ is almost Geometric($p$), $p=\\pi([n]^c)$, $X_n$ is almost stationarily distributed on $[n]^c$, and that $X_n$ and $T_n$ are almost independent, if $p(n):=\\sup_ip(i,[n]^c)\\to 0$ exponentially fast. For the chains with $p(n) \\to 0$ however slowly, and with $\\sup_{i,j}\\,\\|p(i,\\cdot)-p(j,\\cdot)\\|_{TV}<1$, we show that Louchard's conjecture is indeed true even for the hits of an arbitrary $S_n\\subset\\Bbb N$ with $\\pi(S_n)\\to 0$. More precisely, a sequence of $k$ consecutive hit locations paired with the time elapsed since a previous hit (for the first hit, since the starting moment) is approximated, within a total variation distance of order $k\\,\\sup_ip(i,S_n)$, by a $k$-long sequence of independent copies of $(\\ell_n,t_n)$, where $\\ell_n= \\text{Geometric}\\,(\\pi(S_n))$, $t_n$...

  19. Modeling Uncertainty of Directed Movement via Markov Chains

    Directory of Open Access Journals (Sweden)

    YIN Zhangcai

    2015-10-01

    Full Text Available Probabilistic time geography (PTG is suggested as an extension of (classical time geography, in order to present the uncertainty of an agent located at the accessible position by probability. This may provide a quantitative basis for most likely finding an agent at a location. In recent years, PTG based on normal distribution or Brown bridge has been proposed, its variance, however, is irrelevant with the agent's speed or divergent with the increase of the speed; so they are difficult to take into account application pertinence and stability. In this paper, a new method is proposed to model PTG based on Markov chain. Firstly, a bidirectional conditions Markov chain is modeled, the limit of which, when the moving speed is large enough, can be regarded as the Brown bridge, thus has the characteristics of digital stability. Then, the directed movement is mapped to Markov chains. The essential part is to build step length, the state space and transfer matrix of Markov chain according to the space and time position of directional movement, movement speed information, to make sure the Markov chain related to the movement speed. Finally, calculating continuously the probability distribution of the directed movement at any time by the Markov chains, it can be get the possibility of an agent located at the accessible position. Experimental results show that, the variance based on Markov chains not only is related to speed, but also is tending towards stability with increasing the agent's maximum speed.

  20. The Use of Markov Chains in Marketing Forecasting

    OpenAIRE

    Codruţa Dura

    2006-01-01

    The Markov chains model is frequently used to describe consumers’ behavior in relation to their loyalty towards a brand, a manufacturer, a product, o a chain of stores, etc. Most frequently, this model is applied in marketing for dynamic forecasts of the market quota against a background of intense rivalry between brands. In a Markov chain, the result of a trial depends on the result of the trial that directly precedes the former. If we associate the conditional probability pjk (which means t...

  1. Markov Chain-based Degree Distributions of Evolving Networks

    Institute of Scientific and Technical Information of China (English)

    Xiang Xing KONG; Zhen Ting HOU; Ding Hua SHI; Quan Rong CHEN; Qing Gui ZHAO

    2012-01-01

    In this paper,we study a class of stochastic processes,called evolving network Markov chains,in evolving networks. Our approach is to transform the degree distribution problem of an evolving network to a corresponding problem of evolving network Markov chains.We investigate the evolving network Markov chains,thereby obtaining some exact formulas as well as a precise criterion for determining whether the steady degree distribution of the evolving network is a power-law or not.With this new method,we finally obtain a rigorous,exact and unified solution of the steady degree distribution of the evolving network.

  2. Determining a Class of Markov Chains by Hitting Time

    Institute of Scientific and Technical Information of China (English)

    2001-01-01

    @@1 Introduction In many practical problems we often cannot observe the behavior of all states for a Markov chain (see [3-5]). A natural question is that from the observable data of a part of states, can one still obtain all statistical characteristics of the Markov chains. In this paper we give the positive answer for this question and prove the surprising result that the transition rate matrix of the birth-death chains with reflecting barriers and Markov chains on a star graph can be uniquely determined by the probability density functions (pdfs) of the sojourn times and the hitting times at a single special state. This result also suggest a new special type of statistics for Markov chains.

  3. The Laplace Functional and Moments for Markov Branching Chains in Random Environments

    Institute of Scientific and Technical Information of China (English)

    HU Di-he; ZHANG Shu-lin

    2005-01-01

    The concepts of random Markov matrix, Markov branching chain in random environment (MBCRE) and Laplace functional of Markov branching chain in random environment (LFMBCRE) are introduced. The properties of LFMBCRE and the explicit formulas of moments of MBCRE are given.

  4. On the Markov Chain Monte Carlo (MCMC) method

    Indian Academy of Sciences (India)

    Rajeeva L Karandikar

    2006-04-01

    Markov Chain Monte Carlo (MCMC) is a popular method used to generate samples from arbitrary distributions, which may be specified indirectly. In this article, we give an introduction to this method along with some examples.

  5. P Systems Computing the Period of Irreducible Markov Chains

    OpenAIRE

    Cardona Roca, Mónica; Colomer Cugat, M. Angeles; Riscos Núñez, Agustín; Rius Font, Miquel

    2009-01-01

    It is well known that any irreducible and aperiodic Markov chain has exactly one stationary distribution, and for any arbitrary initial distribution, the sequence of distributions at time n converges to the stationary distribution, that is, the Markov chain is approaching equilibrium as n→∞. In this paper, a characterization of the aperiodicity in existential terms of some state is given. At the same time, a P system with external output is associated with any irreducible Ma...

  6. Markov Chains as Tools for Jazz Improvisation Analysis

    OpenAIRE

    Franz, David Matthew

    1998-01-01

    This thesis describes an exploratory application of a statistical analysis and modeling technique (Markov chains) for the modeling of jazz improvisation with the intended subobjective of providing increased insight into an improviser's style and creativity through the postulation of quantitative measures of style and creativity based on the constructed Markovian analysis techniques. Using Visual Basic programming language, Markov chains of orders one to three are created using transcriptio...

  7. ON MARKOV CHAINS IN SPACE-TIME RANDOM ENVIRONMENTS

    Institute of Scientific and Technical Information of China (English)

    Hu Dihe; Hu Xiaoyu

    2009-01-01

    In Section 1, the authors establish the models of two kinds of Markov chains in space-time random environments (MCSTRE and MCSTRE(+)) with Abstract state space. In Section 2, the authors construct a MCSTRE and a MCSTRE(+) by an initial distribution Ф and a random Markov kernel (RMK) p(γ). In Section 3, the authors establish several equivalence theorems on MCSTRE and MCSTRE(+). Finally, the authors give two very important examples of MCMSTRE, the random walk in spce-time random environment and the Markov branching chain in space-time random environment.

  8. Comprehensive cosmographic analysis by Markov chain method

    International Nuclear Information System (INIS)

    We study the possibility of extracting model independent information about the dynamics of the Universe by using cosmography. We intend to explore it systematically, to learn about its limitations and its real possibilities. Here we are sticking to the series expansion approach on which cosmography is based. We apply it to different data sets: Supernovae type Ia (SNeIa), Hubble parameter extracted from differential galaxy ages, gamma ray bursts, and the baryon acoustic oscillations data. We go beyond past results in the literature extending the series expansion up to the fourth order in the scale factor, which implies the analysis of the deceleration q0, the jerk j0, and the snap s0. We use the Markov chain Monte Carlo method (MCMC) to analyze the data statistically. We also try to relate direct results from cosmography to dark energy (DE) dynamical models parametrized by the Chevallier-Polarski-Linder model, extracting clues about the matter content and the dark energy parameters. The main results are: (a) even if relying on a mathematical approximate assumption such as the scale factor series expansion in terms of time, cosmography can be extremely useful in assessing dynamical properties of the Universe; (b) the deceleration parameter clearly confirms the present acceleration phase; (c) the MCMC method can help giving narrower constraints in parameter estimation, in particular for higher order cosmographic parameters (the jerk and the snap), with respect to the literature; and (d) both the estimation of the jerk and the DE parameters reflect the possibility of a deviation from the ΛCDM cosmological model.

  9. Unsupervised Segmentation of Hidden Semi-Markov Non Stationary Chains

    Science.gov (United States)

    Lapuyade-Lahorgue, Jérôme; Pieczynski, Wojciech

    2006-11-01

    In the classical hidden Markov chain (HMC) model we have a hidden chain X, which is a Markov one and an observed chain Y. HMC are widely used; however, in some situations they have to be replaced by the more general "hidden semi-Markov chains" (HSMC) which are particular "triplet Markov chains" (TMC) T = (X, U, Y), where the auxiliary chain U models the semi-Markovianity of X. Otherwise, non stationary classical HMC can also be modeled by a triplet Markov stationary chain with, as a consequence, the possibility of parameters' estimation. The aim of this paper is to use simultaneously both properties. We consider a non stationary HSMC and model it as a TMC T = (X, U1, U2, Y), where U1 models the semi-Markovianity and U2 models the non stationarity. The TMC T being itself stationary, all parameters can be estimated by the general "Iterative Conditional Estimation" (ICE) method, which leads to unsupervised segmentation. We present some experiments showing the interest of the new model and related processing in image segmentation area.

  10. Infinitely dimensional control Markov branching chains in random environments

    Institute of Scientific and Technical Information of China (English)

    2006-01-01

    First of all we introduce the concepts of infinitely dimensional control Markov branching chains in random environments (β-MBCRE) and prove the existence of such chains, then we introduce the concepts of conditional generating functionals and random Markov transition functions of such chains and investigate their branching property. Base on these concepts we calculate the moments of the β-MBCRE and obtain the main results of this paper such as extinction probabilities, polarization and proliferation rate. Finally we discuss the classification ofβ-MBCRE according to the different standards.

  11. The Dynamics of Repeat Migration: A Markov Chain Analysis

    OpenAIRE

    Zimmermann, Klaus F.; Amelie F. Constant

    2003-01-01

    While the literature has established that there is substantial and highly selective return migration, the growing importance of repeat migration has been largely ignored. Using Markov chain analysis, this paper provides a modeling framework for repeated moves of migrants between the host and home countries. The Markov transition matrix between the states in two consecutive periods is parameterized and estimated using a logit specification and a large panel data with 14 waves. The analysis for...

  12. Invariance principle for additive functionals of Markov chains

    OpenAIRE

    Kartashov, Yuri N.; Kulik, Alexey M.

    2007-01-01

    We consider a sequence of additive functionals {\\phi_n}, set on a sequence of Markov chains {X_n} that weakly converges to a Markov process X. We give sufficient condition for such a sequence to converge in distribution, formulated in terms of the characteristics of the additive functionals, and related to the Dynkin's theorem on the convergence of W-functionals. As an application of the main theorem, the general sufficient condition for convergence of additive functionals in terms of transit...

  13. Intrusion detection based on system calls and homogeneous Markov chains

    Institute of Scientific and Technical Information of China (English)

    Tian Xinguang; Duan Miyi; Sun Chunlai; Li Wenfa

    2008-01-01

    A novel method for detecting anomalous program behavior is presented, which is applicable to hostbased intrusion detection systems that monitor system call activities. The method constructs a homogeneous Markov chain model to characterize the normal behavior of a privileged program, and associates the states of the Markov chain with the unique system calls in the training data. At the detection stage, the probabilities that the Markov chain model supports the system call sequences generated by the program are computed. A low probability indicates an anomalous sequence that may result from intrusive activities. Then a decision rule based on the number of anomalous sequences in a locality frame is adopted to classify the program's behavior. The method gives attention to both computational efficiency and detection accuracy, and is especially suitable for on-line detection. It has been applied to practical host-based intrusion detection systems.

  14. Limit Theorems for the Sample Entropy of Hidden Markov Chains

    CERN Document Server

    Han, Guangyue

    2011-01-01

    The Shannon-McMillan-Breiman theorem asserts that the sample entropy of a stationary and ergodic stochastic process converges to the entropy rate of the same process almost surely. In this paper, we focus our attention on the convergence behavior of the sample entropy of a hidden Markov chain. Under certain positivity assumption, we prove that a central limit theorem (CLT) with some Berry-Esseen bound for the sample entropy of a hidden Markov chain, and we use this CLT to establish a law of iterated logarithm (LIL) for the sample entropy.

  15. Influence of credit scoring on the dynamics of Markov chain

    Science.gov (United States)

    Galina, Timofeeva

    2015-11-01

    Markov processes are widely used to model the dynamics of a credit portfolio and forecast the portfolio risk and profitability. In the Markov chain model the loan portfolio is divided into several groups with different quality, which determined by presence of indebtedness and its terms. It is proposed that dynamics of portfolio shares is described by a multistage controlled system. The article outlines mathematical formalization of controls which reflect the actions of the bank's management in order to improve the loan portfolio quality. The most important control is the organization of approval procedure of loan applications. The credit scoring is studied as a control affecting to the dynamic system. Different formalizations of "good" and "bad" consumers are proposed in connection with the Markov chain model.

  16. Markov chain for estimating human mitochondrial DNA mutation pattern

    Science.gov (United States)

    Vantika, Sandy; Pasaribu, Udjianna S.

    2015-12-01

    The Markov chain was proposed to estimate the human mitochondrial DNA mutation pattern. One DNA sequence was taken randomly from 100 sequences in Genbank. The nucleotide transition matrix and mutation transition matrix were estimated from this sequence. We determined whether the states (mutation/normal) are recurrent or transient. The results showed that both of them are recurrent.

  17. Students' Progress throughout Examination Process as a Markov Chain

    Science.gov (United States)

    Hlavatý, Robert; Dömeová, Ludmila

    2014-01-01

    The paper is focused on students of Mathematical methods in economics at the Czech university of life sciences (CULS) in Prague. The idea is to create a model of students' progress throughout the whole course using the Markov chain approach. Each student has to go through various stages of the course requirements where his success depends on the…

  18. Ergodic degrees for continuous-time Markov chains

    Institute of Scientific and Technical Information of China (English)

    MAO Yonghua

    2004-01-01

    This paper studies the existence of the higher orders deviation matrices for continuous time Markov chains by the moments for the hitting times. An estimate of the polynomial convergence rates for the transition matrix to the stationary measure is obtained. Finally, the explicit formulas for birth-death processes are presented.

  19. Markov chains with quasitoeplitz transition matrix: first zero hitting

    Directory of Open Access Journals (Sweden)

    Alexander M. Dukhovny

    1989-01-01

    Full Text Available This paper continues the investigation of Markov Chains with a quasitoeplitz transition matrix. Generating functions of first zero hitting probabilities and mean times are found by the solution of special Riemann boundary value problems on the unit circle. Duality is discussed.

  20. Algebraic convergence for discrete-time ergodic Markov chains

    Institute of Scientific and Technical Information of China (English)

    MAO; Yonghua(毛永华)

    2003-01-01

    This paper studies the e-ergodicity for discrete-time recurrent Markov chains. It proves that thee-order deviation matrix exists and is finite if and only if the chain is (e + 2)-ergodic, and then the algebraicdecay rates of the n-step transition probability to the stationary distribution are obtained. The criteria fore-ergodicity are given in terms of existence of solution to an equation. The main results are illustrated by some examples.

  1. Markov Chain for Reuse Strategies of Product Families

    Institute of Scientific and Technical Information of China (English)

    LUO Jia; JIANG Lan

    2007-01-01

    A methodology is presented to plan reuse strategies of common modules in a product family by using the concepts of function degradation, reliability, function requirement, cost and life time. Markov chain model is employed to predict function degradation and reliability. A utility model is used to evaluate the preference between used modules and new modules. An example of cascading-requirment product family illustrates the main ideas of our work. The Markov models are used effectively to predict function degradation and reliability. Utility theory is helpful to evaluate the reuse options of common modules.

  2. Markov chain aggregation for agent-based models

    CERN Document Server

    Banisch, Sven

    2016-01-01

    This self-contained text develops a Markov chain approach that makes the rigorous analysis of a class of microscopic models that specify the dynamics of complex systems at the individual level possible. It presents a general framework of aggregation in agent-based and related computational models, one which makes use of lumpability and information theory in order to link the micro and macro levels of observation. The starting point is a microscopic Markov chain description of the dynamical process in complete correspondence with the dynamical behavior of the agent-based model (ABM), which is obtained by considering the set of all possible agent configurations as the state space of a huge Markov chain. An explicit formal representation of a resulting “micro-chain” including microscopic transition rates is derived for a class of models by using the random mapping representation of a Markov process. The type of probability distribution used to implement the stochastic part of the model, which defines the upd...

  3. Ensemble bayesian model averaging using markov chain Monte Carlo sampling

    Energy Technology Data Exchange (ETDEWEB)

    Vrugt, Jasper A [Los Alamos National Laboratory; Diks, Cees G H [NON LANL; Clark, Martyn P [NON LANL

    2008-01-01

    Bayesian model averaging (BMA) has recently been proposed as a statistical method to calibrate forecast ensembles from numerical weather models. Successful implementation of BMA however, requires accurate estimates of the weights and variances of the individual competing models in the ensemble. In their seminal paper (Raftery etal. Mon Weather Rev 133: 1155-1174, 2(05)) has recommended the Expectation-Maximization (EM) algorithm for BMA model training, even though global convergence of this algorithm cannot be guaranteed. In this paper, we compare the performance of the EM algorithm and the recently developed Differential Evolution Adaptive Metropolis (DREAM) Markov Chain Monte Carlo (MCMC) algorithm for estimating the BMA weights and variances. Simulation experiments using 48-hour ensemble data of surface temperature and multi-model stream-flow forecasts show that both methods produce similar results, and that their performance is unaffected by the length of the training data set. However, MCMC simulation with DREAM is capable of efficiently handling a wide variety of BMA predictive distributions, and provides useful information about the uncertainty associated with the estimated BMA weights and variances.

  4. Markov Chain Order estimation with Conditional Mutual Information

    CERN Document Server

    Papapetrou, Maria; 10.1016/j.physa.2012.12.017.

    2013-01-01

    We introduce the Conditional Mutual Information (CMI) for the estimation of the Markov chain order. For a Markov chain of $K$ symbols, we define CMI of order $m$, $I_c(m)$, as the mutual information of two variables in the chain being $m$ time steps apart, conditioning on the intermediate variables of the chain. We find approximate analytic significance limits based on the estimation bias of CMI and develop a randomization significance test of $I_c(m)$, where the randomized symbol sequences are formed by random permutation of the components of the original symbol sequence. The significance test is applied for increasing $m$ and the Markov chain order is estimated by the last order for which the null hypothesis is rejected. We present the appropriateness of CMI-testing on Monte Carlo simulations and compare it to the Akaike and Bayesian information criteria, the maximal fluctuation method (Peres-Shields estimator) and a likelihood ratio test for increasing orders using $\\phi$-divergence. The order criterion of...

  5. Asymptotics of Markov Kernels and the Tail Chain

    CERN Document Server

    Resnick, Sidney I

    2011-01-01

    An asymptotic model for extreme behavior of certain Markov chains is the "tail chain". Generally taking the form of a multiplicative random walk, it is useful in deriving extremal characteristics such as point process limits. We place this model in a more general context, formulated in terms of extreme value theory for transition kernels, and extend it by formalizing the distinction between extreme and non-extreme states. We make the link between the update function and transition kernel forms considered in previous work, and we show that the tail chain model leads to a multivariate regular variation property of the finite-dimensional distributions under assumptions on the marginal tails alone.

  6. An Approach of Diagnosis Based On The Hidden Markov Chains Model

    Directory of Open Access Journals (Sweden)

    Karim Bouamrane

    2008-07-01

    Full Text Available Diagnosis is a key element in industrial system maintenance process performance. A diagnosis tool is proposed allowing the maintenance operators capitalizing on the knowledge of their trade and subdividing it for better performance improvement and intervention effectiveness within the maintenance process service. The Tool is based on the Markov Chain Model and more precisely the Hidden Markov Chains (HMC which has the system failures determination advantage, taking into account the causal relations, stochastic context modeling of their dynamics and providing a relevant diagnosis help by their ability of dubious information use. Since the FMEA method is a well adapted artificial intelligence field, the modeling with Markov Chains is carried out with its assistance. Recently, a dynamic programming recursive algorithm, called 'Viterbi Algorithm', is being used in the Hidden Markov Chains field. This algorithm provides as input to the HMC a set of system observed effects and generates at exit the various causes having caused the loss from one or several system functions.

  7. Dynamics of market indices, Markov chains, and random walking problem

    CERN Document Server

    Krivoruchenko, M I

    2001-01-01

    Dynamics of the major USA market indices DJIA, S&P, Nasdaq, and NYSE is analyzed from the point of view of the random walking problem with two-step correlations of the market moves. The parameters characterizing the stochastic dynamics are determined empirically from the historical quotes for the daily, weekly, and monthly series. The results show existence of statistically significant correlations between the subsequent market moves. The weekly and monthly parameters are calculated in terms of the daily parameters, assuming that the Markov chains with two-step correlations give a complete description of the market stochastic dynamics. We show that the macro- and micro-parameters obey the renorm group equation. The comparison of the parameters determined from the renorm group equation with the historical values shows that the Markov chains approach gives reasonable predictions for the weekly quotes and underestimates the probability for continuation of the down trend in the monthly quotes. The return and ...

  8. Constructing 1/ωα noise from reversible Markov chains

    Science.gov (United States)

    Erland, Sveinung; Greenwood, Priscilla E.

    2007-09-01

    This paper gives sufficient conditions for the output of 1/ωα noise from reversible Markov chains on finite state spaces. We construct several examples exhibiting this behavior in a specified range of frequencies. We apply simple representations of the covariance function and the spectral density in terms of the eigendecomposition of the probability transition matrix. The results extend to hidden Markov chains. We generalize the results for aggregations of AR1-processes of C. W. J. Granger [J. Econometrics 14, 227 (1980)]. Given the eigenvalue function, there is a variety of ways to assign values to the states such that the 1/ωα condition is satisfied. We show that a random walk on a certain state space is complementary to the point process model of 1/ω noise of B. Kaulakys and T. Meskauskas [Phys. Rev. E 58, 7013 (1998)]. Passing to a continuous state space, we construct 1/ωα noise which also has a long memory.

  9. Bayesian Smoothing Algorithms in Partially Observed Markov Chains

    Science.gov (United States)

    Ait-el-Fquih, Boujemaa; Desbouvries, François

    2006-11-01

    Let x = {xn}n∈N be a hidden process, y = {yn}n∈N an observed process and r = {rn}n∈N some auxiliary process. We assume that t = {tn}n∈N with tn = (xn, rn, yn-1) is a (Triplet) Markov Chain (TMC). TMC are more general than Hidden Markov Chains (HMC) and yet enable the development of efficient restoration and parameter estimation algorithms. This paper is devoted to Bayesian smoothing algorithms for TMC. We first propose twelve algorithms for general TMC. In the Gaussian case, these smoothers reduce to a set of algorithms which include, among other solutions, extensions to TMC of classical Kalman-like smoothing algorithms (originally designed for HMC) such as the RTS algorithms, the Two-Filter algorithms or the Bryson and Frazier algorithm.

  10. On Dirichlet eigenvectors for neutral two-dimensional Markov chains

    CERN Document Server

    Champagnat, Nicolas; Miclo, Laurent

    2012-01-01

    We consider a general class of discrete, two-dimensional Markov chains modeling the dynamics of a population with two types, without mutation or immigration, and neutral in the sense that type has no influence on each individual's birth or death parameters. We prove that all the eigenvectors of the corresponding transition matrix or infinitesimal generator \\Pi\\ can be expressed as the product of "universal" polynomials of two variables, depending on each type's size but not on the specific transitions of the dynamics, and functions depending only on the total population size. These eigenvectors appear to be Dirichlet eigenvectors for \\Pi\\ on the complement of triangular subdomains, and as a consequence the corresponding eigenvalues are ordered in a specific way. As an application, we study the quasistationary behavior of finite, nearly neutral, two-dimensional Markov chains, absorbed in the sense that 0 is an absorbing state for each component of the process.

  11. On the Total Variation Distance of Semi-Markov Chains

    DEFF Research Database (Denmark)

    Bacci, Giorgio; Bacci, Giovanni; Larsen, Kim Guldstrand;

    2015-01-01

    Semi-Markov chains (SMCs) are continuous-time probabilistic transition systems where the residence time on states is governed by generic distributions on the positive real line. This paper shows the tight relation between the total variation distance on SMCs and their model checking problem over......) is NP-hard and its decidability is an open problem. Nevertheless, we propose an algorithm for approximating it with arbitrary precision....

  12. Maximum entropy estimation of transition probabilities of reversible Markov chains

    OpenAIRE

    Erik Van der Straeten

    2009-01-01

    In this paper, we develop a general theory for the estimation of the transition probabilities of reversible Markov chains using the maximum entropy principle. A broad range of physical models can be studied within this approach. We use one-dimensional classical spin systems to illustrate the theoretical ideas. The examples studied in this paper are: the Ising model, the Potts model and the Blume-Emery-Griffiths model.

  13. Maximum Entropy Estimation of Transition Probabilities of Reversible Markov Chains

    Directory of Open Access Journals (Sweden)

    Erik Van der Straeten

    2009-11-01

    Full Text Available In this paper, we develop a general theory for the estimation of the transition probabilities of reversible Markov chains using the maximum entropy principle. A broad range of physical models can be studied within this approach. We use one-dimensional classical spin systems to illustrate the theoretical ideas. The examples studied in this paper are: the Ising model, the Potts model and the Blume-Emery-Griffiths model.

  14. Computational Discrete Time Markov Chain with Correlated Transition Probabilities

    OpenAIRE

    Peerayuth Charnsethikul

    2006-01-01

    This study presents a computational procedure for analyzing statistics of steady state probabilities in a discrete time Markov chain with correlations among their transition probabilities. The proposed model simply uses the first order Taylor's series expansion and statistical expected value properties to obtain the resulting linear matrix equations system. Computationally, the bottleneck is O(n4) but can be improved by distributed and parallel processing. A preliminary computational experien...

  15. Efficient Texture Segmentation by Hierarchical Multiple Markov Chain Model

    OpenAIRE

    S.Allwin Devaraj; M.M. Vijay; S.Grout; S.Delfin Ruby; K.Kodeeswari

    2014-01-01

    A novel multiscale texture model and a related algorithm for the unsupervised segmentation of medical images to locate tumors are proposed in this project. Elementary textures are characterized by their spatial interactions with neighboring regions along selected directions. Such interactions are modeled, in turn, by means of a set of Markov chains, one for each direction, whose parameters are collected in a feature vector that synthetically describes the texture. Based on the fea...

  16. Fastest Mixing Markov Chain on Symmetric K-Partite Network

    CERN Document Server

    Jafarizadeh, Saber

    2010-01-01

    Solving fastest mixing Markov chain problem (i.e. finding transition probabilities on the edges to minimize the second largest eigenvalue modulus of the transition probability matrix) over networks with different topologies is one of the primary areas of research in the context of computer science and one of the well known networks in this issue is K-partite network. Here in this work we present analytical solution for the problem of fastest mixing Markov chain by means of stratification and semidefinite programming, for four particular types of K-partite networks, namely Symmetric K-PPDR, Semi Symmetric K-PPDR, Cycle K-PPDR and Semi Cycle K-PPDR networks. Our method in this paper is based on convexity of fastest mixing Markov chain problem, and inductive comparing of the characteristic polynomials initiated by slackness conditions in order to find the optimal transition probabilities. The presented results shows that a Symmetric K-PPDR network and its equivalent Semi Symmetric K-PPDR network have the same SL...

  17. Reversible Markov chain estimation using convex-concave programming

    CERN Document Server

    Trendelkamp-Schroer, Benjamin; Noe, Frank

    2016-01-01

    We present a convex-concave reformulation of the reversible Markov chain estimation problem and outline an efficient numerical scheme for the solution of the resulting problem based on a primal-dual interior point method for monotone variational inequalities. Extensions to situations in which information about the stationary vector is available can also be solved via the convex- concave reformulation. The method can be generalized and applied to the discrete transition matrix reweighting analysis method to perform inference from independent chains with specified couplings between the stationary probabilities. The proposed approach offers a significant speed-up compared to a fixed-point iteration for a number of relevant applications.

  18. Recurrence and invariant measure of Markov chains in double-infinite random environments

    Institute of Scientific and Technical Information of China (English)

    XING; Xiusan

    2001-01-01

    [1]Cogburn, R., Markov chains in random environments: The case of Markovian environments, Ann. Probab., 1980, 8(3): 908—916.[2]Cogburn, R., The ergodic theory of Markov chains in random environments, Z. W., 1984, 66(2): 109—128.[3]Orey, S., Markov chains with stochastically stationary transition probabilities, Ann. Probab., 1991, 19(3): 907—928.[4]Li Yingqiu, Some notes of Markov chains in Markov environments, Advances in Mathematics(in Chinese), 1999, 28(4): 358—360.

  19. Inferring animal densities from tracking data using Markov chains.

    Directory of Open Access Journals (Sweden)

    Hal Whitehead

    Full Text Available The distributions and relative densities of species are keys to ecology. Large amounts of tracking data are being collected on a wide variety of animal species using several methods, especially electronic tags that record location. These tracking data are effectively used for many purposes, but generally provide biased measures of distribution, because the starts of the tracks are not randomly distributed among the locations used by the animals. We introduce a simple Markov-chain method that produces unbiased measures of relative density from tracking data. The density estimates can be over a geographical grid, and/or relative to environmental measures. The method assumes that the tracked animals are a random subset of the population in respect to how they move through the habitat cells, and that the movements of the animals among the habitat cells form a time-homogenous Markov chain. We illustrate the method using simulated data as well as real data on the movements of sperm whales. The simulations illustrate the bias introduced when the initial tracking locations are not randomly distributed, as well as the lack of bias when the Markov method is used. We believe that this method will be important in giving unbiased estimates of density from the growing corpus of animal tracking data.

  20. Dynamic temperature selection for parallel-tempering in Markov chain Monte Carlo simulations

    CERN Document Server

    Vousden, Will; Mandel, Ilya

    2015-01-01

    Modern problems in astronomical Bayesian inference require efficient methods for sampling from complex, high-dimensional, often multi-modal probability distributions. Most popular methods, such as Markov chain Monte Carlo sampling, perform poorly on strongly multi-modal probability distributions, rarely jumping between modes or settling on just one mode without finding others. Parallel tempering addresses this problem by sampling simultaneously with separate Markov chains from tempered versions of the target distribution with reduced contrast levels. Gaps between modes can be traversed at higher temperatures, while individual modes can be efficiently explored at lower temperatures. In this paper, we investigate how one might choose the ladder of temperatures to achieve lower autocorrelation time for the sampler (and therefore more efficient sampling). In particular, we present a simple, easily-implemented algorithm for dynamically adapting the temperature configuration of a sampler while sampling in order to ...

  1. DREAM(D: an adaptive Markov Chain Monte Carlo simulation algorithm to solve discrete, noncontinuous, and combinatorial posterior parameter estimation problems

    Directory of Open Access Journals (Sweden)

    C. J. F. Ter Braak

    2011-12-01

    Full Text Available Formal and informal Bayesian approaches have found widespread implementation and use in environmental modeling to summarize parameter and predictive uncertainty. Successful implementation of these methods relies heavily on the availability of efficient sampling methods that approximate, as closely and consistently as possible the (evolving posterior target distribution. Much of this work has focused on continuous variables that can take on any value within their prior defined ranges. Here, we introduce theory and concepts of a discrete sampling method that resolves the parameter space at fixed points. This new code, entitled DREAM(D uses the recently developed DREAM algorithm (Vrugt et al., 2008, 2009a, b as its main building block but implements two novel proposal distributions to help solve discrete and combinatorial optimization problems. This novel MCMC sampler maintains detailed balance and ergodicity, and is especially designed to resolve the emerging class of optimal experimental design problems. Three different case studies involving a Sudoku puzzle, soil water retention curve, and rainfall – runoff model calibration problem are used to benchmark the performance of DREAM(D. The theory and concepts developed herein can be easily integrated into other (adaptive MCMC algorithms.

  2. Mixed Vehicle Flow At Signalized Intersection: Markov Chain Analysis

    Directory of Open Access Journals (Sweden)

    Gertsbakh Ilya B.

    2015-09-01

    Full Text Available We assume that a Poisson flow of vehicles arrives at isolated signalized intersection, and each vehicle, independently of others, represents a random number X of passenger car units (PCU’s. We analyze numerically the stationary distribution of the queue process {Zn}, where Zn is the number of PCU’s in a queue at the beginning of the n-th red phase, n → ∞. We approximate the number Yn of PCU’s arriving during one red-green cycle by a two-parameter Negative Binomial Distribution (NBD. The well-known fact is that {Zn} follow an infinite-state Markov chain. We approximate its stationary distribution using a finite-state Markov chain. We show numerically that there is a strong dependence of the mean queue length E[Zn] in equilibrium on the input distribution of Yn and, in particular, on the ”over dispersion” parameter γ= Var[Yn]/E[Yn]. For Poisson input, γ = 1. γ > 1 indicates presence of heavy-tailed input. In reality it means that a relatively large ”portion” of PCU’s, considerably exceeding the average, may arrive with high probability during one red-green cycle. Empirical formulas are presented for an accurate estimation of mean queue length as a function of load and g of the input flow. Using the Markov chain technique, we analyze the mean ”virtual” delay time for a car which always arrives at the beginning of the red phase.

  3. Renewal Theory for Markov Chains on the Real Line

    OpenAIRE

    Keener, Robert W.

    1982-01-01

    Standard renewal theory is concerned with expectations related to sums of positive i.i.d. variables, $S_n = \\sum^n_{i=1} Z_i$. We generalize this theory to the case where $\\{S_i\\}$ is a Markov chain on the real line with stationary transition probabilities satisfying a drift condition. The expectations we are concerned with satisfy generalized renewal equations, and in our main theorems, we show that these expectations are the unique solutions of the equations they satisfy.

  4. Imputing unknown competitor marketing activity with a Hidden Markov Chain

    OpenAIRE

    Haughton, Dominique; Hua, Guangying; Jin, Danny; Lin, John; Wei, Qizhi; Zhang, Changan

    2014-01-01

    We demonstrate on a case study with two competing products at a bank how one can use a Hidden Markov Chain (HMC) to estimate missing information on a competitor's marketing activity. The idea is that given time series with sales volumes for products A and B and marketing expenditures for product A, as well as suitable predictors of sales for products A and B, we can infer at each point in time whether it is likely or not that marketing activities took place for product B. The method is succes...

  5. Dynamic modeling of presence of occupants using inhomogeneous Markov chains

    DEFF Research Database (Denmark)

    Andersen, Philip Hvidthøft Delff; Iversen, Anne; Madsen, Henrik;

    2014-01-01

    Occupancy modeling is a necessary step towards reliable simulation of energy consumption in buildings. This paper outlines a method for fitting recordings of presence of occupants and simulation of single-person to multiple-persons office environments. The method includes modeling of dependence...... on inhomogeneous Markov chains with where the transition probabilities are estimated using generalized linear models with polynomials, B-splines, and a filter of passed observations as inputs. For treating the dispersion of the data series, a hierarchical model structure is used where one model is for low presence...

  6. Efficient Parallel Learning of Hidden Markov Chain Models on SMPs

    Science.gov (United States)

    Li, Lei; Fu, Bin; Faloutsos, Christos

    Quad-core cpus have been a common desktop configuration for today's office. The increasing number of processors on a single chip opens new opportunity for parallel computing. Our goal is to make use of the multi-core as well as multi-processor architectures to speed up large-scale data mining algorithms. In this paper, we present a general parallel learning framework, Cut-And-Stitch, for training hidden Markov chain models. Particularly, we propose two model-specific variants, CAS-LDS for learning linear dynamical systems (LDS) and CAS-HMM for learning hidden Markov models (HMM). Our main contribution is a novel method to handle the data dependencies due to the chain structure of hidden variables, so as to parallelize the EM-based parameter learning algorithm. We implement CAS-LDS and CAS-HMM using OpenMP on two supercomputers and a quad-core commercial desktop. The experimental results show that parallel algorithms using Cut-And-Stitch achieve comparable accuracy and almost linear speedups over the traditional serial version.

  7. On Markov Chains Induced by Partitioned Transition Probability Matrices

    Institute of Scientific and Technical Information of China (English)

    Thomas KAIJSER

    2011-01-01

    Let S be a denumerable state space and let P be a transition probability matrix on S. If a denumerable set M of nonnegative matrices is such that the sum of the matrices is equal to P, then we call M a partition of P. Let K denote the set of probability vectors on S. With every partition M of P we can associate a transition probability function PM on K defined in such a way that if p ∈ K and M ∈ M are such that ‖pM‖ > 0, then, with probability ‖pM‖, the vector p is transferred to the vector pM/‖pM‖. Here ‖· ‖ denotes the l1-norm. In this paper we investigate the convergence in distribution for Markov chains generated by transition probability functions induced by partitions of transition probability matrices. The main motivation for this investigation is the application of the convergence results obtained to filtering processes of partially observed Markov chains with denumerable state space.

  8. Optimized nested Markov chain Monte Carlo sampling: theory

    Energy Technology Data Exchange (ETDEWEB)

    Coe, Joshua D [Los Alamos National Laboratory; Shaw, M Sam [Los Alamos National Laboratory; Sewell, Thomas D [U. MISSOURI

    2009-01-01

    Metropolis Monte Carlo sampling of a reference potential is used to build a Markov chain in the isothermal-isobaric ensemble. At the endpoints of the chain, the energy is reevaluated at a different level of approximation (the 'full' energy) and a composite move encompassing all of the intervening steps is accepted on the basis of a modified Metropolis criterion. By manipulating the thermodynamic variables characterizing the reference system we maximize the average acceptance probability of composite moves, lengthening significantly the random walk made between consecutive evaluations of the full energy at a fixed acceptance probability. This provides maximally decorrelated samples of the full potential, thereby lowering the total number required to build ensemble averages of a given variance. The efficiency of the method is illustrated using model potentials appropriate to molecular fluids at high pressure. Implications for ab initio or density functional theory (DFT) treatment are discussed.

  9. Analysis of business demography using markov chains : an application to Belgian data

    OpenAIRE

    François Coppens; Fabienne Verduyn

    2009-01-01

    This paper applies the theory of finite Markov chains to analyse the demographic evolution of Belgian enterprises. While other methodologies concentrate on the entry and exit of firms, the Markov approach also analyses migrations between economic sectors. Besides helping to provide a fuller picture of the evolution of the population, Markov chains also enable forecasts of its future composition to be made, as well as the computation of average lifetimes of companies by branch of activity. The...

  10. Analysis of business demography using markov chains: an application to Belgian data

    OpenAIRE

    Coppens, François; Verduyn, Fabienne

    2009-01-01

    This paper applies the theory of finite Markov chains to analyse the demographic evolution of Belgian enterprises. While other methodologies concentrate on the entry and exit of firms, the Markov approach also analyses migrations between economic sectors. Besides helping to provide a fuller picture of the evolution of the population, Markov chains also enable forecasts of its future composition to be made, as well as the computation of average lifetimes of companies by branch of activity. The...

  11. On the relation between recurrence and ergodicity properties in denumerable Markov decision chains

    NARCIS (Netherlands)

    R. Dekker (Rommert); A. Hordijk (Arie); F.M. Spieksma

    1994-01-01

    textabstractThis paper studies two properties of the set of Markov chains induced by the deterministic policies in a Markov decision chain. These properties are called μ-uniform geometric ergodicity and μ-uniform geometric recurrence. μ-uniform ergodicity generalises a quasi-compactness condition. I

  12. Asymptotics of Entropy Rate in Special Families of Hidden Markov Chains

    OpenAIRE

    Han, G; Marcus, BH

    2008-01-01

    We derive an asymptotic formula for entropy rate of a hidden Markov chain under certain parameterizations. We also discuss applications of the asymptotic formula to the asymptotic behaviors of entropy rate of hidden Markov chains as outputs of certain channels, such as binary symmetric channel, binary erasure channel, and some special Gilbert-Elliot channel. © 2006 IEEE.

  13. Grey Markov chain and its application in drift prediction model of FOGs

    Institute of Scientific and Technical Information of China (English)

    Fan Chunling; Jin Zhihua; Tian Weifeng; Qian Feng

    2005-01-01

    A novel grey Markov chain predictive model is discussed to reduce drift influence on the output of fiber optical gyroscopes (FOGs) and to improve FOGs' measurement precision. The proposed method possesses advantages of grey model and Markov chain. It makes good use of dynamic modeling idea of the grey model to predict general trend of original data. Then according to the trend, states are divided so that it can overcome the disadvantage of high computational cost of state transition probability matrix in Markov chain. Moreover, the presented approach expands the applied scope of the grey model and makes it be fit for prediction of random data with bigger fluctuation. The numerical results of real drift data from a certain type FOG verify the effectiveness of the proposed grey Markov chain model powerfully. The Markov chain is also investigated to provide a comparison with the grey Markov chain model. It is shown that the hybrid grey Markov chain prediction model has higher modeling precision than Markov chain itself, which prove this proposed method is very applicable and effective.

  14. Reservoir Modeling Combining Geostatistics with Markov Chain Monte Carlo Inversion

    DEFF Research Database (Denmark)

    Zunino, Andrea; Lange, Katrine; Melnikova, Yulia;

    2014-01-01

    We present a study on the inversion of seismic reflection data generated from a synthetic reservoir model. Our aim is to invert directly for rock facies and porosity of the target reservoir zone. We solve this inverse problem using a Markov chain Monte Carlo (McMC) method to handle the nonlinear......, multi-step forward model (rock physics and seismology) and to provide realistic estimates of uncertainties. To generate realistic models which represent samples of the prior distribution, and to overcome the high computational demand, we reduce the search space utilizing an algorithm drawn from...... geostatistics. The geostatistical algorithm learns the multiple-point statistics from prototype models, then generates proposal models which are tested by a Metropolis sampler. The solution of the inverse problem is finally represented by a collection of reservoir models in terms of facies and porosity, which...

  15. HYDRA: a Java library for Markov Chain Monte Carlo

    Directory of Open Access Journals (Sweden)

    Gregory R. Warnes

    2002-03-01

    Full Text Available Hydra is an open-source, platform-neutral library for performing Markov Chain Monte Carlo. It implements the logic of standard MCMC samplers within a framework designed to be easy to use, extend, and integrate with other software tools. In this paper, we describe the problem that motivated our work, outline our goals for the Hydra pro ject, and describe the current features of the Hydra library. We then provide a step-by-step example of using Hydra to simulate from a mixture model drawn from cancer genetics, first using a variable-at-a-time Metropolis sampler and then a Normal Kernel Coupler. We conclude with a discussion of future directions for Hydra.

  16. Markov Chain Monte-Carlo Models of Starburst Clusters

    Science.gov (United States)

    Melnick, Jorge

    2015-01-01

    There are a number of stochastic effects that must be considered when comparing models to observations of starburst clusters: the IMF is never fully populated; the stars can never be strictly coeval; stars rotate and their photometric properties depend on orientation; a significant fraction of massive stars are in interacting binaries; and the extinction varies from star to star. The probability distributions of each of these effects are not a priori known, but must be extracted from the observations. Markov Chain Monte-Carlo methods appear to provide the best statistical approach. Here I present an example of stochastic age effects upon the upper mass limit of the IMF of the Arches cluster as derived from near-IR photometry.

  17. Analysis of Users Web Browsing Behavior Using Markov chain Model

    Directory of Open Access Journals (Sweden)

    Diwakar Shukla

    2011-03-01

    Full Text Available In present days of growing information technology, many browsers available for surfing and web mining. A user has option to use any of them at a time to mine out the desired website. Every browser has pre-defined level of popularity and reputation in the market. This paper considers the setup of only two browsers in a computer system and a user prefers to any one, if fails, switches to the other one .The behavior of user is modeled through Markov chain procedure and transition probabilities are calculated. The quitting to browsing is treated as a parameter of variation over the popularity. Graphical study is performed to explain the inter relationship between user behavior parameters and browser market popularity parameters. If rate of a company is lowest in terms of browser failure and lowest in terms of quitting probability then company enjoys better popularity and larger user proportion

  18. Logics and Models for Stochastic Analysis Beyond Markov Chains

    DEFF Research Database (Denmark)

    Zeng, Kebin

    form of discrete PH distributions as computational vehicle on measuring the performance of concurrent wireless sensor networks. Secondly, choosing stochastic process algebras as a widely accepted formalism, we study the compositionality of continuous PH distributions in order to support modelling......Within the last twenty years, logics and models for stochastic analysis of information systems have been widely studied in both theory and practice. The quantitative properties, such as performance and reliability, are evaluated over discrete–time and continuous–time Markov chains. This thesis...... kernels. The most appealing feature is that all the components before and after compositions are secured to have a minimal state space representation. To support quantitative verification, we also propose stochastic model checking algorithms to our problem....

  19. Markov Chain Monte Carlo Bayesian Learning for Neural Networks

    Science.gov (United States)

    Goodrich, Michael S.

    2011-01-01

    Conventional training methods for neural networks involve starting al a random location in the solution space of the network weights, navigating an error hyper surface to reach a minimum, and sometime stochastic based techniques (e.g., genetic algorithms) to avoid entrapment in a local minimum. It is further typically necessary to preprocess the data (e.g., normalization) to keep the training algorithm on course. Conversely, Bayesian based learning is an epistemological approach concerned with formally updating the plausibility of competing candidate hypotheses thereby obtaining a posterior distribution for the network weights conditioned on the available data and a prior distribution. In this paper, we developed a powerful methodology for estimating the full residual uncertainty in network weights and therefore network predictions by using a modified Jeffery's prior combined with a Metropolis Markov Chain Monte Carlo method.

  20. Bayesian inference and Markov chain Monte Carlo in imaging

    Science.gov (United States)

    Higdon, David M.; Bowsher, James E.

    1999-05-01

    Over the past 20 years, many problems in Bayesian inference that were previously intractable can now be fairly routinely dealt with using a computationally intensive technique for exploring the posterior distribution called Markov chain Monte Carlo (MCMC). Primarily because of insufficient computing capabilities, most MCMC applications have been limited to rather standard statistical models. However, with the computing power of modern workstations, a fully Bayesian approach with MCMC, is now possible for many imaging applications. Such an approach can be quite useful because it leads not only to `point' estimates of an underlying image or emission source, but it also gives a means for quantifying uncertainties regarding the image. This paper gives an overview of Bayesian image analysis and focuses on applications relevant to medical imaging. Particular focus is on prior image models and outlining MCMC methods for these models.

  1. Parameter estimation in deformable models using Markov chain Monte Carlo

    Science.gov (United States)

    Chalana, Vikram; Haynor, David R.; Sampson, Paul D.; Kim, Yongmin

    1997-04-01

    Deformable models have gained much popularity recently for many applications in medical imaging, such as image segmentation, image reconstruction, and image registration. Such models are very powerful because various kinds of information can be integrated together in an elegant statistical framework. Each such piece of information is typically associated with a user-defined parameter. The values of these parameters can have a significant effect on the results generated using these models. Despite the popularity of deformable models for various applications, not much attention has been paid to the estimation of these parameters. In this paper we describe systematic methods for the automatic estimation of these deformable model parameters. These methods are derived by posing the deformable models as a Bayesian inference problem. Our parameter estimation methods use Markov chain Monte Carlo methods for generating samples from highly complex probability distributions.

  2. On the multi-level solution algorithm for Markov chains

    Energy Technology Data Exchange (ETDEWEB)

    Horton, G. [Univ. of Erlangen, Nuernberg (Germany)

    1996-12-31

    We discuss the recently introduced multi-level algorithm for the steady-state solution of Markov chains. The method is based on the aggregation principle, which is well established in the literature. Recursive application of the aggregation yields a multi-level method which has been shown experimentally to give results significantly faster than the methods currently in use. The algorithm can be reformulated as an algebraic multigrid scheme of Galerkin-full approximation type. The uniqueness of the scheme stems from its solution-dependent prolongation operator which permits significant computational savings in the evaluation of certain terms. This paper describes the modeling of computer systems to derive information on performance, measured typically as job throughput or component utilization, and availability, defined as the proportion of time a system is able to perform a certain function in the presence of component failures and possibly also repairs.

  3. Recurrence and invariant measure of Markov chains in double-infinite random environments

    Institute of Scientific and Technical Information of China (English)

    2001-01-01

    The concepts of π-irreduciblity, recurrence and transience are introduced into the research field of Markov chains in random environments.That a π-irreducible chain must be either recurrent or transient is proved, a criterion is shown for recurrent Markov chains in double-infinite random environments, the existence of invariant measure of π-irreducible chains in double-infinite environments is discussed,and then Orey's open-questions are partially answered.

  4. Accelerating Monte Carlo Markov chains with proxy and error models

    Science.gov (United States)

    Josset, Laureline; Demyanov, Vasily; Elsheikh, Ahmed H.; Lunati, Ivan

    2015-12-01

    In groundwater modeling, Monte Carlo Markov Chain (MCMC) simulations are often used to calibrate aquifer parameters and propagate the uncertainty to the quantity of interest (e.g., pollutant concentration). However, this approach requires a large number of flow simulations and incurs high computational cost, which prevents a systematic evaluation of the uncertainty in the presence of complex physical processes. To avoid this computational bottleneck, we propose to use an approximate model (proxy) to predict the response of the exact model. Here, we use a proxy that entails a very simplified description of the physics with respect to the detailed physics described by the "exact" model. The error model accounts for the simplification of the physical process; and it is trained on a learning set of realizations, for which both the proxy and exact responses are computed. First, the key features of the set of curves are extracted using functional principal component analysis; then, a regression model is built to characterize the relationship between the curves. The performance of the proposed approach is evaluated on the Imperial College Fault model. We show that the joint use of the proxy and the error model to infer the model parameters in a two-stage MCMC set-up allows longer chains at a comparable computational cost. Unnecessary evaluations of the exact responses are avoided through a preliminary evaluation of the proposal made on the basis of the corrected proxy response. The error model trained on the learning set is crucial to provide a sufficiently accurate prediction of the exact response and guide the chains to the low misfit regions. The proposed methodology can be extended to multiple-chain algorithms or other Bayesian inference methods. Moreover, FPCA is not limited to the specific presented application and offers a general framework to build error models.

  5. Resilient model approximation for Markov jump time-delay systems via reduced model with hierarchical Markov chains

    Science.gov (United States)

    Zhu, Yanzheng; Zhang, Lixian; Sreeram, Victor; Shammakh, Wafa; Ahmad, Bashir

    2016-10-01

    In this paper, the resilient model approximation problem for a class of discrete-time Markov jump time-delay systems with input sector-bounded nonlinearities is investigated. A linearised reduced-order model is determined with mode changes subject to domination by a hierarchical Markov chain containing two different nonhomogeneous Markov chains. Hence, the reduced-order model obtained not only reflects the dependence of the original systems but also model external influence that is related to the mode changes of the original system. Sufficient conditions formulated in terms of bilinear matrix inequalities for the existence of such models are established, such that the resulting error system is stochastically stable and has a guaranteed l2-l∞ error performance. A linear matrix inequalities optimisation coupled with line search is exploited to solve for the corresponding reduced-order systems. The potential and effectiveness of the developed theoretical results are demonstrated via a numerical example.

  6. Some Limit Properties of Random Transition Probability for Second-Order Nonhomogeneous Markov Chains Indexed by a Tree

    OpenAIRE

    Zhiyan Shi; Weiguo Yang

    2009-01-01

    We study some limit properties of the harmonic mean of random transition probability for a second-order nonhomogeneous Markov chain and a nonhomogeneous Markov chain indexed by a tree. As corollary, we obtain the property of the harmonic mean of random transition probability for a nonhomogeneous Markov chain.

  7. Prediction of Synchrostate Transitions in EEG Signals Using Markov Chain Models

    CERN Document Server

    Jamal, Wasifa; Oprescu, Ioana-Anastasia; Maharatna, Koushik

    2014-01-01

    This paper proposes a stochastic model using the concept of Markov chains for the inter-state transitions of the millisecond order quasi-stable phase synchronized patterns or synchrostates, found in multi-channel Electroencephalogram (EEG) signals. First and second order transition probability matrices are estimated for Markov chain modelling from 100 trials of 128-channel EEG signals during two different face perception tasks. Prediction accuracies with such finite Markov chain models for synchrostate transition are also compared, under a data-partitioning based cross-validation scheme.

  8. Large Deviations for Empirical Measures of Not Necessarily Irreducible Countable Markov Chains with Arbitrary Initial Measures

    Institute of Scientific and Technical Information of China (English)

    Yi Wen JIANG; Li Ming WU

    2005-01-01

    All known results on large deviations of occupation measures of Markov processes are based on the assumption of (essential) irreducibility. In this paper we establish the weak* large deviation principle of occupation measures for any countable Markov chain with arbitrary initial measures. The new rate function that we obtain is not convex and depends on the initial measure, contrary to the (essentially) irreducible case.

  9. Real time Markov chains: Wind states in anemometric data

    CERN Document Server

    Sanchez, P A; Jaramillo, O A

    2015-01-01

    The description of wind phenomena is frequently based on data obtained from anemometers, which usually report the wind speed and direction only in a horizontal plane. Such measurements are commonly used either to develop wind generation farms or to forecast weather conditions in a geographical region. Beyond these standard applications, the information contained in the data may be richer than expected and may lead to a better understanding of the wind dynamics in a geographical area. In this work we propose a statistical analysis based on the wind velocity vectors, which we propose may be grouped in "wind states" associated to binormal distribution functions. We found that the velocity plane defined by the anemometric velocity data may be used as a phase space, where a finite number of states may be found and sorted using standard clustering methods. The main result is a discretization technique useful to model the wind with Markov chains. We applied such ideas in anemometric data for two different sites in M...

  10. Asteroid mass estimation using Markov-Chain Monte Carlo techniques

    Science.gov (United States)

    Siltala, Lauri; Granvik, Mikael

    2016-10-01

    Estimates for asteroid masses are based on their gravitational perturbations on the orbits of other objects such as Mars, spacecraft, or other asteroids and/or their satellites. In the case of asteroid-asteroid perturbations, this leads to a 13-dimensional inverse problem where the aim is to derive the mass of the perturbing asteroid and six orbital elements for both the perturbing asteroid and the test asteroid using astrometric observations. We have developed and implemented three different mass estimation algorithms utilizing asteroid-asteroid perturbations into the OpenOrb asteroid-orbit-computation software: the very rough 'marching' approximation, in which the asteroid orbits are fixed at a given epoch, reducing the problem to a one-dimensional estimation of the mass, an implementation of the Nelder-Mead simplex method, and most significantly, a Markov-Chain Monte Carlo (MCMC) approach. We will introduce each of these algorithms with particular focus on the MCMC algorithm, and present example results for both synthetic and real data. Our results agree with the published mass estimates, but suggest that the published uncertainties may be misleading as a consequence of using linearized mass-estimation methods. Finally, we discuss remaining challenges with the algorithms as well as future plans, particularly in connection with ESA's Gaia mission.

  11. Hidden Markov chain modeling for epileptic networks identification.

    Science.gov (United States)

    Le Cam, Steven; Louis-Dorr, Valérie; Maillard, Louis

    2013-01-01

    The partial epileptic seizures are often considered to be caused by a wrong balance between inhibitory and excitatory interneuron connections within a focal brain area. These abnormal balances are likely to result in loss of functional connectivities between remote brain structures, while functional connectivities within the incriminated zone are enhanced. The identification of the epileptic networks underlying these hypersynchronies are expected to contribute to a better understanding of the brain mechanisms responsible for the development of the seizures. In this objective, threshold strategies are commonly applied, based on synchrony measurements computed from recordings of the electrophysiologic brain activity. However, such methods are reported to be prone to errors and false alarms. In this paper, we propose a hidden Markov chain modeling of the synchrony states with the aim to develop a reliable machine learning methods for epileptic network inference. The method is applied on a real Stereo-EEG recording, demonstrating consistent results with the clinical evaluations and with the current knowledge on temporal lobe epilepsy. PMID:24110697

  12. ENSO informed Drought Forecasting Using Nonhomogeneous Hidden Markov Chain Model

    Science.gov (United States)

    Kwon, H.; Yoo, J.; Kim, T.

    2013-12-01

    The study aims at developing a new scheme to investigate the potential use of ENSO (El Niño/Southern Oscillation) for drought forecasting. In this regard, objective of this study is to extend a previously developed nonhomogeneous hidden Markov chain model (NHMM) to identify climate states associated with drought that can be potentially used to forecast drought conditions using climate information. As a target variable for forecasting, SPI(standardized precipitation index) is mainly utilized. This study collected monthly precipitation data over 56 stations that cover more than 30 years and K-means cluster analysis using drought properties was applied to partition regions into mutually exclusive clusters. In this study, six main clusters were distinguished through the regionalization procedure. For each cluster, the NHMM was applied to estimate the transition probability of hidden states as well as drought conditions informed by large scale climate indices (e.g. SOI, Nino1.2, Nino3, Nino3.4, MJO and PDO). The NHMM coupled with large scale climate information shows promise as a technique for forecasting drought scenarios. A more detailed explanation of large scale climate patterns associated with the identified hidden states will be provided with anomaly composites of SSTs and SLPs. Acknowledgement This research was supported by a grant(11CTIPC02) from Construction Technology Innovation Program (CTIP) funded by Ministry of Land, Transport and Maritime Affairs of Korean government.

  13. Threshold partitioning of sparse matrices and applications to Markov chains

    Energy Technology Data Exchange (ETDEWEB)

    Choi, Hwajeong; Szyld, D.B. [Temple Univ., Philadelphia, PA (United States)

    1996-12-31

    It is well known that the order of the variables and equations of a large, sparse linear system influences the performance of classical iterative methods. In particular if, after a symmetric permutation, the blocks in the diagonal have more nonzeros, classical block methods have a faster asymptotic rate of convergence. In this paper, different ordering and partitioning algorithms for sparse matrices are presented. They are modifications of PABLO. In the new algorithms, in addition to the location of the nonzeros, the values of the entries are taken into account. The matrix resulting after the symmetric permutation has dense blocks along the diagonal, and small entries in the off-diagonal blocks. Parameters can be easily adjusted to obtain, for example, denser blocks, or blocks with elements of larger magnitude. In particular, when the matrices represent Markov chains, the permuted matrices are well suited for block iterative methods that find the corresponding probability distribution. Applications to three types of methods are explored: (1) Classical block methods, such as Block Gauss Seidel. (2) Preconditioned GMRES, where a block diagonal preconditioner is used. (3) Iterative aggregation method (also called aggregation/disaggregation) where the partition obtained from the ordering algorithm with certain parameters is used as an aggregation scheme. In all three cases, experiments are presented which illustrate the performance of the methods with the new orderings. The complexity of the new algorithms is linear in the number of nonzeros and the order of the matrix, and thus adding little computational effort to the overall solution.

  14. Asymptotics of Entropy Rate in Special Families of Hidden Markov Chains

    CERN Document Server

    Han, Guangyue

    2008-01-01

    We derive an asymptotic formula for entropy rate of a hidden Markov chain around a "weak Black Hole". We also discuss applications of the asymptotic formula to the asymptotic behaviors of certain channels.

  15. Maps of sparse Markov chains efficiently reveal community structure in network flows with memory

    CERN Document Server

    Persson, Christian; Edler, Daniel; Rosvall, Martin

    2016-01-01

    To better understand the flows of ideas or information through social and biological systems, researchers develop maps that reveal important patterns in network flows. In practice, network flow models have implied memoryless first-order Markov chains, but recently researchers have introduced higher-order Markov chain models with memory to capture patterns in multi-step pathways. Higher-order models are particularly important for effectively revealing actual, overlapping community structure, but higher-order Markov chain models suffer from the curse of dimensionality: their vast parameter spaces require exponentially increasing data to avoid overfitting and therefore make mapping inefficient already for moderate-sized systems. To overcome this problem, we introduce an efficient cross-validated mapping approach based on network flows modeled by sparse Markov chains. To illustrate our approach, we present a map of citation flows in science with research fields that overlap in multidisciplinary journals. Compared...

  16. Technical manual for basic version of the Markov chain nest productivity model (MCnest)

    Science.gov (United States)

    The Markov Chain Nest Productivity Model (or MCnest) integrates existing toxicity information from three standardized avian toxicity tests with information on species life history and the timing of pesticide applications relative to the timing of avian breeding seasons to quantit...

  17. Characterizing the Aperiodicity of Irreducible Markov Chains by Using P Systems

    OpenAIRE

    Cardona, Mónica; Colomer, M. Angels; Pérez Jiménez, Mario de Jesús

    2009-01-01

    It is well known that any irreducible and aperiodic Markov chain has exactly one stationary distribution, and for any arbitrary initial distribution, the sequence of distributions at time n converges to the stationary distribution, that is, the Markov chain is approaching equilibrium as n ! 1. In this paper, a characterization of the aperiodicity in existential terms of some state is given. At the same time, a P system with external output is associated with any irreducible ...

  18. THE TRANSITION PROBABILITY MATRIX OF A MARKOV CHAIN MODEL IN AN ATM NETWORK

    Institute of Scientific and Technical Information of China (English)

    YUE Dequan; ZHANG Huachen; TU Fengsheng

    2003-01-01

    In this paper we consider a Markov chain model in an ATM network, which has been studied by Dag and Stavrakakis. On the basis of the iterative formulas obtained by Dag and Stavrakakis, we obtain the explicit analytical expression of the transition probability matrix. It is very simple to calculate the transition probabilities of the Markov chain by these expressions. In addition, we obtain some results about the structure of the transition probability matrix, which are helpful in numerical calculation and theoretical analysis.

  19. Markov Chain Computation for Homogeneous and Non-homogeneous Data: MARCH 1.1 Users Guide

    Directory of Open Access Journals (Sweden)

    Andre Berchtold

    2001-03-01

    Full Text Available MARCH is a free software for the computation of different types of Markovian models including homogeneous Markov Chains, Hidden Markov Models (HMMs and Double Chain Markov Models (DCMMs. The main characteristic of this software is the implementation of a powerful optimization method for HMMs and DCMMs combining a genetic algorithm with the standard Baum-Welch procedure. MARCH is distributed as a set of Matlab functions running under Matlab 5 or higher on any computing platform. A PC Windows version running independently from Matlab is also available.

  20. Dynamic temperature selection for parallel tempering in Markov chain Monte Carlo simulations

    Science.gov (United States)

    Vousden, W. D.; Farr, W. M.; Mandel, I.

    2016-01-01

    Modern problems in astronomical Bayesian inference require efficient methods for sampling from complex, high-dimensional, often multimodal probability distributions. Most popular methods, such as MCMC sampling, perform poorly on strongly multimodal probability distributions, rarely jumping between modes or settling on just one mode without finding others. Parallel tempering addresses this problem by sampling simultaneously with separate Markov chains from tempered versions of the target distribution with reduced contrast levels. Gaps between modes can be traversed at higher temperatures, while individual modes can be efficiently explored at lower temperatures. In this paper, we investigate how one might choose the ladder of temperatures to achieve more efficient sampling, as measured by the autocorrelation time of the sampler. In particular, we present a simple, easily implemented algorithm for dynamically adapting the temperature configuration of a sampler while sampling. This algorithm dynamically adjusts the temperature spacing to achieve a uniform rate of exchanges between chains at neighbouring temperatures. We compare the algorithm to conventional geometric temperature configurations on a number of test distributions and on an astrophysical inference problem, reporting efficiency gains by a factor of 1.2-2.5 over a well-chosen geometric temperature configuration and by a factor of 1.5-5 over a poorly chosen configuration. On all of these problems, a sampler using the dynamical adaptations to achieve uniform acceptance ratios between neighbouring chains outperforms one that does not.

  1. 3D+t brain MRI segmentation using robust 4D Hidden Markov Chain.

    Science.gov (United States)

    Lavigne, François; Collet, Christophe; Armspach, Jean-Paul

    2014-01-01

    In recent years many automatic methods have been developed to help physicians diagnose brain disorders, but the problem remains complex. In this paper we propose a method to segment brain structures on two 3D multi-modal MR images taken at different times (longitudinal acquisition). A bias field correction is performed with an adaptation of the Hidden Markov Chain (HMC) allowing us to take into account the temporal correlation in addition to spatial neighbourhood information. To improve the robustness of the segmentation of the principal brain structures and to detect Multiple Sclerosis Lesions as outliers the Trimmed Likelihood Estimator (TLE) is used during the process. The method is validated on 3D+t brain MR images. PMID:25571045

  2. Modeling and Computing of Stock Index Forecasting Based on Neural Network and Markov Chain

    Directory of Open Access Journals (Sweden)

    Yonghui Dai

    2014-01-01

    Full Text Available The stock index reflects the fluctuation of the stock market. For a long time, there have been a lot of researches on the forecast of stock index. However, the traditional method is limited to achieving an ideal precision in the dynamic market due to the influences of many factors such as the economic situation, policy changes, and emergency events. Therefore, the approach based on adaptive modeling and conditional probability transfer causes the new attention of researchers. This paper presents a new forecast method by the combination of improved back-propagation (BP neural network and Markov chain, as well as its modeling and computing technology. This method includes initial forecasting by improved BP neural network, division of Markov state region, computing of the state transition probability matrix, and the prediction adjustment. Results of the empirical study show that this method can achieve high accuracy in the stock index prediction, and it could provide a good reference for the investment in stock market.

  3. Markov chain order estimation with parametric significance tests of conditional mutual information

    CERN Document Server

    Papapetrou, Maria

    2015-01-01

    Besides the different approaches suggested in the literature, accurate estimation of the order of a Markov chain from a given symbol sequence is an open issue, especially when the order is moderately large. Here, parametric significance tests of conditional mutual information (CMI) of increasing order $m$, $I_c(m)$, on a symbol sequence are conducted for increasing orders $m$ in order to estimate the true order $L$ of the underlying Markov chain. CMI of order $m$ is the mutual information of two variables in the Markov chain being $m$ time steps apart, conditioning on the intermediate variables of the chain. The null distribution of CMI is approximated with a normal and gamma distribution deriving analytic expressions of their parameters, and a gamma distribution deriving its parameters from the mean and variance of the normal distribution. The accuracy of order estimation is assessed with the three parametric tests, and the parametric tests are compared to the randomization significance test and other known ...

  4. Weighted Markov Chains and Graphic State Nodes for Information Retrieval.

    Science.gov (United States)

    Benoit, G.

    2002-01-01

    Discusses users' search behavior and decision making in data mining and information retrieval. Describes iterative information seeking as a Markov process during which users advance through states of nodes; and explains how the information system records the decision as weights, allowing the incorporation of users' decisions into the Markov…

  5. MARKOV CHAIN-BASED ANALYSIS OF THE DEGREE DISTRIBUTION FOR A GROWING NETWORK

    Institute of Scientific and Technical Information of China (English)

    Hou Zhenting; Tong Jinying; Shi Dinghua

    2011-01-01

    In this article, we focus on discussing the degree distribution of the DMS model from the perspective of probability. On the basis of the concept and technique of first-passage probability in Markov theory, we provide a rigorous proof for existence of the steady-state degree distribution, mathematically re-deriving the exact formula of the distribution. The approach based on Markov chain theory is universal and performs well in a large class of growing networks.

  6. Enhancement of Markov chain model by integrating exponential smoothing: A case study on Muslims marriage and divorce

    Science.gov (United States)

    Jamaluddin, Fadhilah; Rahim, Rahela Abdul

    2015-12-01

    Markov Chain has been introduced since the 1913 for the purpose of studying the flow of data for a consecutive number of years of the data and also forecasting. The important feature in Markov Chain is obtaining the accurate Transition Probability Matrix (TPM). However to obtain the suitable TPM is hard especially in involving long-term modeling due to unavailability of data. This paper aims to enhance the classical Markov Chain by introducing Exponential Smoothing technique in developing the appropriate TPM.

  7. Reliability analysis and prediction of mixed mode load using Markov Chain Model

    Energy Technology Data Exchange (ETDEWEB)

    Nikabdullah, N. [Department of Mechanical and Materials Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia and Institute of Space Science (ANGKASA), Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia (Malaysia); Singh, S. S. K.; Alebrahim, R.; Azizi, M. A. [Department of Mechanical and Materials Engineering, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia (Malaysia); K, Elwaleed A. [Institute of Space Science (ANGKASA), Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia (Malaysia); Noorani, M. S. M. [School of Mathematical Sciences, Faculty of Science and Technology, Universiti Kebangsaan Malaysia (Malaysia)

    2014-06-19

    The aim of this paper is to present the reliability analysis and prediction of mixed mode loading by using a simple two state Markov Chain Model for an automotive crankshaft. The reliability analysis and prediction for any automotive component or structure is important for analyzing and measuring the failure to increase the design life, eliminate or reduce the likelihood of failures and safety risk. The mechanical failures of the crankshaft are due of high bending and torsion stress concentration from high cycle and low rotating bending and torsional stress. The Markov Chain was used to model the two states based on the probability of failure due to bending and torsion stress. In most investigations it revealed that bending stress is much serve than torsional stress, therefore the probability criteria for the bending state would be higher compared to the torsion state. A statistical comparison between the developed Markov Chain Model and field data was done to observe the percentage of error. The reliability analysis and prediction was derived and illustrated from the Markov Chain Model were shown in the Weibull probability and cumulative distribution function, hazard rate and reliability curve and the bathtub curve. It can be concluded that Markov Chain Model has the ability to generate near similar data with minimal percentage of error and for a practical application; the proposed model provides a good accuracy in determining the reliability for the crankshaft under mixed mode loading.

  8. Reliability analysis and prediction of mixed mode load using Markov Chain Model

    Science.gov (United States)

    Nikabdullah, N.; Singh, S. S. K.; Alebrahim, R.; Azizi, M. A.; K, Elwaleed A.; Noorani, M. S. M.

    2014-06-01

    The aim of this paper is to present the reliability analysis and prediction of mixed mode loading by using a simple two state Markov Chain Model for an automotive crankshaft. The reliability analysis and prediction for any automotive component or structure is important for analyzing and measuring the failure to increase the design life, eliminate or reduce the likelihood of failures and safety risk. The mechanical failures of the crankshaft are due of high bending and torsion stress concentration from high cycle and low rotating bending and torsional stress. The Markov Chain was used to model the two states based on the probability of failure due to bending and torsion stress. In most investigations it revealed that bending stress is much serve than torsional stress, therefore the probability criteria for the bending state would be higher compared to the torsion state. A statistical comparison between the developed Markov Chain Model and field data was done to observe the percentage of error. The reliability analysis and prediction was derived and illustrated from the Markov Chain Model were shown in the Weibull probability and cumulative distribution function, hazard rate and reliability curve and the bathtub curve. It can be concluded that Markov Chain Model has the ability to generate near similar data with minimal percentage of error and for a practical application; the proposed model provides a good accuracy in determining the reliability for the crankshaft under mixed mode loading.

  9. Reliability analysis and prediction of mixed mode load using Markov Chain Model

    International Nuclear Information System (INIS)

    The aim of this paper is to present the reliability analysis and prediction of mixed mode loading by using a simple two state Markov Chain Model for an automotive crankshaft. The reliability analysis and prediction for any automotive component or structure is important for analyzing and measuring the failure to increase the design life, eliminate or reduce the likelihood of failures and safety risk. The mechanical failures of the crankshaft are due of high bending and torsion stress concentration from high cycle and low rotating bending and torsional stress. The Markov Chain was used to model the two states based on the probability of failure due to bending and torsion stress. In most investigations it revealed that bending stress is much serve than torsional stress, therefore the probability criteria for the bending state would be higher compared to the torsion state. A statistical comparison between the developed Markov Chain Model and field data was done to observe the percentage of error. The reliability analysis and prediction was derived and illustrated from the Markov Chain Model were shown in the Weibull probability and cumulative distribution function, hazard rate and reliability curve and the bathtub curve. It can be concluded that Markov Chain Model has the ability to generate near similar data with minimal percentage of error and for a practical application; the proposed model provides a good accuracy in determining the reliability for the crankshaft under mixed mode loading

  10. Markov

    Directory of Open Access Journals (Sweden)

    Carlos Alejandro De Luna Ortega

    2006-01-01

    Full Text Available En este artículo se aborda el diseño de un reconocedor de voz, con el idioma español mexicano, del estado de Aguascalientes, de palabras aisladas, con dependencia del hablante y vocabulario pequeño, empleando Redes Neuronales Artificiales (ANN por sus siglas en inglés, Alineamiento Dinámico del Tiempo (DTW por sus siglas en inglés y Modelos Ocultos de Markov (HMM por sus siglas en inglés para la realización del algoritmo de reconocimiento.

  11. Markov Chain-Like Quantum Biological Modeling of Mutations, Aging, and Evolution

    Directory of Open Access Journals (Sweden)

    Ivan B. Djordjevic

    2015-08-01

    Full Text Available Recent evidence suggests that quantum mechanics is relevant in photosynthesis, magnetoreception, enzymatic catalytic reactions, olfactory reception, photoreception, genetics, electron-transfer in proteins, and evolution; to mention few. In our recent paper published in Life, we have derived the operator-sum representation of a biological channel based on codon basekets, and determined the quantum channel model suitable for study of the quantum biological channel capacity. However, this model is essentially memoryless and it is not able to properly model the propagation of mutation errors in time, the process of aging, and evolution of genetic information through generations. To solve for these problems, we propose novel quantum mechanical models to accurately describe the process of creation spontaneous, induced, and adaptive mutations and their propagation in time. Different biological channel models with memory, proposed in this paper, include: (i Markovian classical model, (ii Markovian-like quantum model, and (iii hybrid quantum-classical model. We then apply these models in a study of aging and evolution of quantum biological channel capacity through generations. We also discuss key differences of these models with respect to a multilevel symmetric channel-based Markovian model and a Kimura model-based Markovian process. These models are quite general and applicable to many open problems in biology, not only biological channel capacity, which is the main focus of the paper. We will show that the famous quantum Master equation approach, commonly used to describe different biological processes, is just the first-order approximation of the proposed quantum Markov chain-like model, when the observation interval tends to zero. One of the important implications of this model is that the aging phenotype becomes determined by different underlying transition probabilities in both programmed and random (damage Markov chain-like models of aging, which

  12. Random billiards with wall temperature and associated Markov chains

    OpenAIRE

    Cook, Scott; Feres, Renato

    2012-01-01

    By a random billiard we mean a billiard system in which the standard specular reflection rule is replaced with a Markov transition probabilities operator P that, at each collision of the billiard particle with the boundary of the billiard domain, gives the probability distribution of the post-collision velocity for a given pre-collision velocity. A random billiard with microstructure (RBM) is a random billiard for which P is derived from a choice of geometric/mechanical structure on the bound...

  13. Formal Reasoning About Finite-State Discrete-Time Markov Chains in HOL

    Institute of Scientific and Technical Information of China (English)

    Liya Liu; Osman Hasan; Sofiène Tahar

    2013-01-01

    Markov chains are extensively used in modeling different aspects of engineering and scientific systems,such as performance of algorithms and reliability of systems.Different techniques have been developed for analyzing Markovian models,for example,Markov Chain Monte Carlo based simulation,Markov Analyzer,and more recently probabilistic modelchecking.However,these techniques either do not guarantee accurate analysis or are not scalable.Higher-order-logic theorem proving is a formal method that has the ability to overcome the above mentioned limitations.However,it is not mature enough to handle all sorts of Markovian models.In this paper,we propose a formalization of Discrete-Time Markov Chain (DTMC) that facilitates formal reasoning about time-homogeneous finite-state discrete-time Markov chain.In particular,we provide a formal verification on some of its important properties,such as joint probabilities,Chapman-Kolmogorov equation,reversibility property,using higher-order logic.To demonstrate the usefulness of our work,we analyze two applications:a simplified binary communication channel and the Automatic Mail Quality Measurement protocol.

  14. Improvement of Markov Chain Model for Occurrence Degree Prediction of Myzus persicae (Sulzer) and Its Application

    Institute of Scientific and Technical Information of China (English)

    REN Guangwei; WANG Xiufang; WANG Xinwei; ZHOU Xiansheng; DONG Xiaowei

    2008-01-01

    For long-term prediction of occurrence degree of tobacco aphid Myzus persicae (Sulzer), Markov chain method was used to establish prediction model for occurrence degree of tobacco aphid. With 4 levels of occurrence degree, Markov chain model was established based on the data in 1987-2004. The results indicated that the accuracy for total prediction in 2005-2007 and the back prediction in 1987-2004 reached 88.89% and 85.12%, respectively. The method is simple and feasible for long-term prediction of occurrence degree of tobacco aphid.

  15. A note on asymptotic expansions for Markov chains using operator theory

    DEFF Research Database (Denmark)

    Jensen, J.L.

    1987-01-01

    We consider asymptotic expansions for sums Sn on the form Sn = fhook0(X0) + fhook(X1, X0) + ... + fhook(Xn, Xn-1), where Xi is a Markov chain. Under different ergodicity conditions on the Markov chain and certain conditional moment conditions on fhook(Xi, Xi-1), a simple representation of the cha...... characteristic function of Sn is obtained. The representation is in term of the maximal eigenvalue of the linear operator sending a function g(x) into the function x → E(g(Xi)exp[itfhook(Xi, x)]|Xi-1 = x). © 1987....

  16. An extension of a theorem of Kesten to topological Markov chains

    CERN Document Server

    Stadlbauer, Manuel

    2011-01-01

    The main results of this note extend a theorem of Kesten for symmetric random walks on discrete groups to group extensions of topological Markov chains. In contrast to the result in probability theory, there is a notable asymmetry in the assumptions on the base. That is, it turns out that, under very mild assumptions on the continuity and symmetry of the associated potential, amenability of the group implies that the Gurevic-pressures of the extension and the base coincide whereas the converse holds true if the potential is H\\"older continuous and the topological Markov chain has big images and preimages. Finally, an application to periodic hyperbolic manifolds is given.

  17. 2nd International Workshop on the Numerical Solution of Markov Chains

    CERN Document Server

    1995-01-01

    Computations with Markov Chains presents the edited and reviewed proceedings of the Second International Workshop on the Numerical Solution of Markov Chains, held January 16--18, 1995, in Raleigh, North Carolina. New developments of particular interest include recent work on stability and conditioning, Krylov subspace-based methods for transient solutions, quadratic convergent procedures for matrix geometric problems, further analysis of the GTH algorithm, the arrival of stochastic automata networks at the forefront of modelling stratagems, and more. An authoritative overview of the field for applied probabilists, numerical analysts and systems modelers, including computer scientists and engineers.

  18. THE CONSTRUCTION OF MULTITYPE CANONICAL MARKOV BRANCHING CHAINS IN RANDOM ENVIRONMENTS

    Institute of Scientific and Technical Information of China (English)

    2006-01-01

    The investigation for branching processes has a long history by their strong physics background, but only a few authors have investigated the branching processes in random environments. First of all, the author introduces the concepts of the multitype canonical Markov branching chain in random environment (CMBCRE) and multitype Markov branching chain in random environment (MBCRE) and proved that CMBCRE must be MBCRE, and any MBCRE must be equivalent to another CMBCRE in distribution. The main results of this article are the construction of CMBCRE and some of its probability properties.

  19. Exact test of Hardy-Weinberg equilibrium by Markov chain Monte Carlo.

    Science.gov (United States)

    Yuan, Ao; Bonney, George E

    2003-12-01

    The assumption of Hardy-Weinberg equilibrium (HWE) among alleles is of fundamental importance in genetic studies. There are numerous testing methods for it using genotype counts data. The exact test is used when the sample size is not large enough for asymptotic approximations. There are several numerical methods to carry out this test, such as complete enumeration, Monte Carlo and Markov chain Monte Carlo simulations. Complete enumeration is impractical in many applications, especially when the table counts are large. The Monte Carlo method is simple to use but still difficult when the table counts become large. The Markov chain Monte Carlo method, by sampling a sub-table each time, is suitable for this latter situation. Based on switches among a few (no more than four) cells, the existing Markov chain samplers are highly dependent and inefficient for large tables. Here we consider a new Markov chain sampling, in which a sub-table of user-specified size is updated at each iteration. The resulting chain is less dependent, and the sampling is flexible and efficient. The conventional test for HWE is based on a few test statistics, such as the likelihood and the chi-squared statistic. To expand the family of test statistics, we consider a class of divergence measures for the departure of HWE. Examples are given as illustrations.

  20. Integration of logistic regression, Markov chain and cellular automata models to simulate urban expansion

    NARCIS (Netherlands)

    Jokar Arsanjani, J.; Helbich, M.; Kainz, W.; Boloorani, A.

    2013-01-01

    This research analyses the suburban expansion in the metropolitan area of Tehran, Iran. A hybrid model consisting of logistic regression model, Markov chain (MC), and cellular automata (CA) was designed to improve the performance of the standard logistic regression model. Environmental and socio-eco

  1. First Passage Probability Estimation of Wind Turbines by Markov Chain Monte Carlo

    DEFF Research Database (Denmark)

    Sichani, Mahdi Teimouri; Nielsen, Søren R.K.

    2013-01-01

    Markov Chain Monte Carlo simulation has received considerable attention within the past decade as reportedly one of the most powerful techniques for the first passage probability estimation of dynamic systems. A very popular method in this direction capable of estimating probability of rare events...

  2. Behavior of an Almost Semicontinuous Poisson Process on a Markov Chain Upon Attainment of A Level

    CERN Document Server

    Karnaukh, Ievgen

    2011-01-01

    We consider the almost semi-continuous processes defined on a finite Markov chain. The representation of the moment generating functions for the absolute maximum after achievement positive level and for the recovery time are obtained. Modified processes with two-step rate of negative jumps are investigated.

  3. Automated compositional Markov chain generation for a plain-old telephone system

    NARCIS (Netherlands)

    Hermanns, H.; Katoen, J.P.

    2000-01-01

    Obtaining performance models, like Markov chains and queueing networks, for systems of significant complexity and magnitude is a difficult task that is usually tackled using human intelligence and experience. This holds in particular for performance models of a highly irregular nature. In this paper

  4. Avian life history profiles for use in the Markov chain nest productivity model (MCnest)

    Science.gov (United States)

    The Markov Chain nest productivity model, or MCnest, quantitatively estimates the effects of pesticides or other toxic chemicals on annual reproductive success of avian species (Bennett and Etterson 2013, Etterson and Bennett 2013). The Basic Version of MCnest was developed as a...

  5. On dynamic selection of households for direct marketing based on Markov chain models with memory

    NARCIS (Netherlands)

    Otter, Pieter W.

    2007-01-01

    A simple, dynamic selection procedure is proposed, based on conditional, expected profits using Markov chain models with memory. The method is easy to apply, only frequencies and mean values have to be calculated or estimated. The method is empirically illustrated using a data set from a charitable

  6. Average, sensitive and Blackwell-optimal policies in denumerable Markov decision chains with unbounded rewards

    NARCIS (Netherlands)

    R. Dekker (Rommert); A. Hordijk (Arie)

    1988-01-01

    textabstractIn this paper we consider a (discrete-time) Markov decision chain with a denumerabloe state space and compact action sets and we assume that for all states the rewards and transition probabilities depend continuously on the actions. The first objective of this paper is to develop an anal

  7. Treatment-based Markov chain models clarify mechanisms of invasion in an invaded grassland community.

    Science.gov (United States)

    Nelis, Lisa Castillo; Wootton, J Timothy

    2010-02-22

    What are the relative roles of mechanisms underlying plant responses in grassland communities invaded by both plants and mammals? What type of community can we expect in the future given current or novel conditions? We address these questions by comparing Markov chain community models among treatments from a field experiment on invasive species on Robinson Crusoe Island, Chile. Because of seed dispersal, grazing and disturbance, we predicted that the exotic European rabbit (Oryctolagus cuniculus) facilitates epizoochorous exotic plants (plants with seeds that stick to the skin an animal) at the expense of native plants. To test our hypothesis, we crossed rabbit exclosure treatments with disturbance treatments, and sampled the plant community in permanent plots over 3 years. We then estimated Markov chain model transition probabilities and found significant differences among treatments. As hypothesized, this modelling revealed that exotic plants survive better in disturbed areas, while natives prefer no rabbits or disturbance. Surprisingly, rabbits negatively affect epizoochorous plants. Markov chain dynamics indicate that an overall replacement of native plants by exotic plants is underway. Using a treatment-based approach to multi-species Markov chain models allowed us to examine the changes in the importance of mechanisms in response to experimental impacts on communities.

  8. Fitting optimum order of Markov chain models for daily rainfall occurrences in Peninsular Malaysia

    Science.gov (United States)

    Deni, Sayang Mohd; Jemain, Abdul Aziz; Ibrahim, Kamarulzaman

    2009-06-01

    The analysis of the daily rainfall occurrence behavior is becoming more important, particularly in water-related sectors. Many studies have identified a more comprehensive pattern of the daily rainfall behavior based on the Markov chain models. One of the aims in fitting the Markov chain models of various orders to the daily rainfall occurrence is to determine the optimum order. In this study, the optimum order of the Markov chain models for a 5-day sequence will be examined in each of the 18 rainfall stations in Peninsular Malaysia, which have been selected based on the availability of the data, using the Akaike’s (AIC) and Bayesian information criteria (BIC). The identification of the most appropriate order in describing the distribution of the wet (dry) spells for each of the rainfall stations is obtained using the Kolmogorov-Smirnov goodness-of-fit test. It is found that the optimum order varies according to the levels of threshold used (e.g., either 0.1 or 10.0 mm), the locations of the region and the types of monsoon seasons. At most stations, the Markov chain models of a higher order are found to be optimum for rainfall occurrence during the northeast monsoon season for both levels of threshold. However, it is generally found that regardless of the monsoon seasons, the first-order model is optimum for the northwestern and eastern regions of the peninsula when the level of thresholds of 10.0 mm is considered. The analysis indicates that the first order of the Markov chain model is found to be most appropriate for describing the distribution of wet spells, whereas the higher-order models are found to be adequate for the dry spells in most of the rainfall stations for both threshold levels and monsoon seasons.

  9. Descriptive and predictive evaluation of high resolution Markov chain precipitation models

    DEFF Research Database (Denmark)

    Sørup, Hjalte Jomo Danielsen; Madsen, Henrik; Arnbjerg-Nielsen, Karsten

    2012-01-01

    to reproduce the time series on event level. Extreme events with short (10 min), medium (60 min) and long (12 h) durations were investigated because of their importance in urban hydrology. Both the descriptive likelihood based statistics and the predictive Monte Carlo simulation based statistics are valuable....... The first‐order Markov model seems to capture most of the properties of precipitation, but inclusion of seasonal and diurnal variation improves the model. Including a second‐order Markov Chain component does improve the descriptive capabilities of the model, but is very expensive in its parameter use...

  10. Harnessing graphical structure in Markov chain Monte Carlo learning

    Energy Technology Data Exchange (ETDEWEB)

    Stolorz, P.E. [California Inst. of Technology, Pasadena, CA (United States); Chew P.C. [Univ. of Pennsylvania, Philadelphia, PA (United States)

    1996-12-31

    The Monte Carlo method is recognized as a useful tool in learning and probabilistic inference methods common to many datamining problems. Generalized Hidden Markov Models and Bayes nets are especially popular applications. However, the presence of multiple modes in many relevant integrands and summands often renders the method slow and cumbersome. Recent mean field alternatives designed to speed things up have been inspired by experience gleaned from physics. The current work adopts an approach very similar to this in spirit, but focusses instead upon dynamic programming notions as a basis for producing systematic Monte Carlo improvements. The idea is to approximate a given model by a dynamic programming-style decomposition, which then forms a scaffold upon which to build successively more accurate Monte Carlo approximations. Dynamic programming ideas alone fail to account for non-local structure, while standard Monte Carlo methods essentially ignore all structure. However, suitably-crafted hybrids can successfully exploit the strengths of each method, resulting in algorithms that combine speed with accuracy. The approach relies on the presence of significant {open_quotes}local{close_quotes} information in the problem at hand. This turns out to be a plausible assumption for many important applications. Example calculations are presented, and the overall strengths and weaknesses of the approach are discussed.

  11. From Brownian Dynamics to Markov Chain: An Ion Channel Example

    KAUST Repository

    Chen, Wan

    2014-02-27

    A discrete rate theory for multi-ion channels is presented, in which the continuous dynamics of ion diffusion is reduced to transitions between Markovian discrete states. In an open channel, the ion permeation process involves three types of events: an ion entering the channel, an ion escaping from the channel, or an ion hopping between different energy minima in the channel. The continuous dynamics leads to a hierarchy of Fokker-Planck equations, indexed by channel occupancy. From these the mean escape times and splitting probabilities (denoting from which side an ion has escaped) can be calculated. By equating these with the corresponding expressions from the Markov model, one can determine the Markovian transition rates. The theory is illustrated with a two-ion one-well channel. The stationary probability of states is compared with that from both Brownian dynamics simulation and the hierarchical Fokker-Planck equations. The conductivity of the channel is also studied, and the optimal geometry maximizing ion flux is computed. © 2014 Society for Industrial and Applied Mathematics.

  12. Generation of solar radiation values by using Markov chains; Generacion de valores de radiacion usando cadenas de Markov

    Energy Technology Data Exchange (ETDEWEB)

    Adaro, Jorge; Cesari, Daniela; Lema, Alba; Galimberti, Pablo [Universidad Nacional de Rio Cuarto (Argentina). Facultad de Ingenieria]. E-mail: aadaro@ing.unrc.edu.ar

    2000-07-01

    The objective of the present work is to adopt a methodology that allows to generate sequences of values of global solar radiation. It is carried out a preliminary study on the generation of radiation sequence a concept of Chains of Markov. For it is analyzed it the readiness of data and it is investigated about the possibility of using such a methodology calculating values of indexes of clarity previously. With data of available radiation and provided the National Meteorological Service for Rio Cuarto, the preliminary study is carried out the preliminary study looking for to validated the pattern to the effects of being able to transfer the use of the methodology in other regions. (author)

  13. Understanding for convergence monitoring for probabilistic risk assessment based on Markov Chain Monte Carlo Simulation

    Energy Technology Data Exchange (ETDEWEB)

    Kim, Joo Yeon; Jang, Han Ki; Jang, Sol Ah; Park, Tae Jin [Korean Association for Radiation Application, Seoul (Korea, Republic of)

    2014-04-15

    There is a question that the simulation actually leads to draws from its target distribution and the most basic one is whether such Markov chains can always be constructed and all chain values sampled from them. The problem to be solved is the determination of how large this iteration should be to achieve the target distribution. This problem can be answered as convergence monitoring. In this paper, two widely used methods, such as autocorrelation and potential scale reduction factor (PSRF) in MCMC are characterized. There is no general agreement on the subject of the convergence. Although it is generally agreed that running n parallel chains in practice is computationally inefficient and unnecessary, running multiple parallel chains is generally applied for the convergence monitoring due to easy implementation. The main debate is the number of parallel chains needed. If the convergence properties of the chain are well understood then clearly a single chain suffices. Therefore, autocorrelation using single chain and multiple parallel ones are tried and their results then compared with each other in this study. And, the following question is answered from the two convergence results: Have the Markov chain realizations for achieved the target distribution?.

  14. Fitting complex population models by combining particle filters with Markov chain Monte Carlo.

    Science.gov (United States)

    Knape, Jonas; de Valpine, Perry

    2012-02-01

    We show how a recent framework combining Markov chain Monte Carlo (MCMC) with particle filters (PFMCMC) may be used to estimate population state-space models. With the purpose of utilizing the strengths of each method, PFMCMC explores hidden states by particle filters, while process and observation parameters are estimated using an MCMC algorithm. PFMCMC is exemplified by analyzing time series data on a red kangaroo (Macropus rufus) population in New South Wales, Australia, using MCMC over model parameters based on an adaptive Metropolis-Hastings algorithm. We fit three population models to these data; a density-dependent logistic diffusion model with environmental variance, an unregulated stochastic exponential growth model, and a random-walk model. Bayes factors and posterior model probabilities show that there is little support for density dependence and that the random-walk model is the most parsimonious model. The particle filter Metropolis-Hastings algorithm is a brute-force method that may be used to fit a range of complex population models. Implementation is straightforward and less involved than standard MCMC for many models, and marginal densities for model selection can be obtained with little additional effort. The cost is mainly computational, resulting in long running times that may be improved by parallelizing the algorithm. PMID:22624307

  15. Forgetting of the initial condition for the filter in general state-space hidden Markov chain: a coupling approach

    OpenAIRE

    Douc, Randal; Moulines, Eric; Ritov, Ya'Acov

    2007-01-01

    21 We give simple conditions that ensure exponential forgetting of the initial conditions of the filter for general state-space hidden Markov chain. The proofs are based on the coupling argument applied to the posterior Markov kernels. These results are useful both for filtering hidden Markov models using approximation methods (e.g., particle filters) and for proving asymptotic properties of estimators. The results are general enough to cover models like the Gaussian state space model, wit...

  16. STRONG LAW OF LARGE NUMBERS AND SHANNON-MCMILLAN THEOREM FOR MARKOV CHAINS FIELD ON CAYLEY TREE

    Institute of Scientific and Technical Information of China (English)

    杨卫国; 刘文

    2001-01-01

    This paper studies the strong law of large numbers and the Shannom-McMillan theorem for Markov chains field on Cayley tree. The authors first prove the strong law of large number on the frequencies of states and orderd couples of states for Markov chains field on Cayley tree. Then they prove thc Shannon-McMillan theorem with a.e. convergence for Markov chains field on Cayley tree. In the proof, a new technique in the study the strong limit theorem in probability theory is applied.

  17. Modeling Urban Expansion in Bangkok Metropolitan Region Using Demographic–Economic Data through Cellular Automata-Markov Chain and Multi-Layer Perceptron-Markov Chain Models

    Directory of Open Access Journals (Sweden)

    Chudech Losiri

    2016-07-01

    Full Text Available Urban expansion is considered as one of the most important problems in several developing countries. Bangkok Metropolitan Region (BMR is the urbanized and agglomerated area of Bangkok Metropolis (BM and its vicinity, which confronts the expansion problem from the center of the city. Landsat images of 1988, 1993, 1998, 2003, 2008, and 2011 were used to detect the land use and land cover (LULC changes. The demographic and economic data together with corresponding maps were used to determine the driving factors for land conversions. This study applied Cellular Automata-Markov Chain (CA-MC and Multi-Layer Perceptron-Markov Chain (MLP-MC to model LULC and urban expansions. The performance of the CA-MC and MLP-MC yielded more than 90% overall accuracy to predict the LULC, especially the MLP-MC method. Further, the annual population and economic growth rates were considered to produce the land demand for the LULC in 2014 and 2035 using the statistical extrapolation and system dynamics (SD. It was evident that the simulated map in 2014 resulting from the SD yielded the highest accuracy. Therefore, this study applied the SD method to generate the land demand for simulating LULC in 2035. The outcome showed that urban occupied the land around a half of the BMR.

  18. An Expectation Maximization Algorithm to Model Failure Times by Continuous-Time Markov Chains

    Directory of Open Access Journals (Sweden)

    Qihong Duan

    2010-01-01

    Full Text Available In many applications, the failure rate function may present a bathtub shape curve. In this paper, an expectation maximization algorithm is proposed to construct a suitable continuous-time Markov chain which models the failure time data by the first time reaching the absorbing state. Assume that a system is described by methods of supplementary variables, the device of stage, and so on. Given a data set, the maximum likelihood estimators of the initial distribution and the infinitesimal transition rates of the Markov chain can be obtained by our novel algorithm. Suppose that there are m transient states in the system and that there are n failure time data. The devised algorithm only needs to compute the exponential of m×m upper triangular matrices for O(nm2 times in each iteration. Finally, the algorithm is applied to two real data sets, which indicates the practicality and efficiency of our algorithm.

  19. Risk Minimization for Insurance Products via F-Doubly Stochastic Markov Chains

    Directory of Open Access Journals (Sweden)

    Francesca Biagini

    2016-07-01

    Full Text Available We study risk-minimization for a large class of insurance contracts. Given that the individual progress in time of visiting an insurance policy’s states follows an F -doubly stochastic Markov chain, we describe different state-dependent types of insurance benefits. These cover single payments at maturity, annuity-type payments and payments at the time of a transition. Based on the intensity of the F -doubly stochastic Markov chain, we provide the Galtchouk-Kunita-Watanabe decomposition for a general insurance contract and specify risk-minimizing strategies in a Brownian financial market setting. The results are further illustrated explicitly within an affine structure for the intensity.

  20. A Stochastic Markov Chain Model to Describe Lung Cancer Growth and Metastasis

    OpenAIRE

    Newton, Paul K.; Jeremy Mason; Kelly Bethel; Bazhenova, Lyudmila A.; Jorge Nieva; Peter Kuhn

    2012-01-01

    A stochastic Markov chain model for metastatic progression is developed for primary lung cancer based on a network construction of metastatic sites with dynamics modeled as an ensemble of random walkers on the network. We calculate a transition matrix, with entries (transition probabilities) interpreted as random variables, and use it to construct a circular bi-directional network of primary and metastatic locations based on postmortem tissue analysis of 3827 autopsies on untreated patients d...

  1. Spreaders and Sponges define metastasis in lung cancer: A Markov chain Monte Carlo Mathematical Model

    OpenAIRE

    Newton, Paul K.; Mason, Jeremy; Bethel, Kelly; Bazhenova, Lyudmila; Nieva, Jorge; Norton, Larry; Kuhn, Peter

    2013-01-01

    The classic view of metastatic cancer progression is that it is a unidirectional process initiated at the primary tumor site, progressing to variably distant metastatic sites in a fairly predictable, though not perfectly understood, fashion. A Markov chain Monte Carlo mathematical approach can determine a pathway diagram that classifies metastatic tumors as ‘spreaders’ or ‘sponges’ and orders the timescales of progression from site to site. In light of recent experimental evidence highlightin...

  2. Analysis of a Lance missile platoon using a semi-Markov chain

    OpenAIRE

    Argo, Harry M.

    1989-01-01

    Approved for public release; distribution is unlimited This thesis develops a combat effectiveness model for the Lance missile system. The survivability and ability to accomplish the mission for a Lance missile launch platoon depends upon enemy capabilities, platoon configuration, missile reliability and many other tangible factors. The changing status of a launch platoon is modeled using a semi-Markov chain with transient and absorbing states. Expected number of missiles fi...

  3. Observational constraints on G-corrected holographic dark energy using a Markov chain Monte Carlo method

    OpenAIRE

    Alavirad, Hamzeh; Malekjani, Mohammad

    2013-01-01

    We constrain holographic dark energy (HDE) with time varying gravitational coupling constant in the framework of the modified Friedmann equations using cosmological data from type Ia supernovae, baryon acoustic oscillations, cosmic microwave background radiation and X-ray gas mass fraction. Applying a Markov Chain Monte Carlo (MCMC) simulation, we obtain the best fit values of the model and cosmological parameters within $1\\sigma$ confidence level (CL) in a flat universe as: $\\Omega_{\\rm b}h^...

  4. Batch means and spectral variance estimators in Markov chain Monte Carlo

    OpenAIRE

    Flegal, James M.; Jones, Galin L.

    2008-01-01

    Calculating a Monte Carlo standard error (MCSE) is an important step in the statistical analysis of the simulation output obtained from a Markov chain Monte Carlo experiment. An MCSE is usually based on an estimate of the variance of the asymptotic normal distribution. We consider spectral and batch means methods for estimating this variance. In particular, we establish conditions which guarantee that these estimators are strongly consistent as the simulation effort increases. In addition, fo...

  5. Inference in Kingman's Coalescent with Particle Markov Chain Monte Carlo Method

    OpenAIRE

    Chen, Yifei; Xie, Xiaohui

    2013-01-01

    We propose a new algorithm to do posterior sampling of Kingman's coalescent, based upon the Particle Markov Chain Monte Carlo methodology. Specifically, the algorithm is an instantiation of the Particle Gibbs Sampling method, which alternately samples coalescent times conditioned on coalescent tree structures, and tree structures conditioned on coalescent times via the conditional Sequential Monte Carlo procedure. We implement our algorithm as a C++ package, and demonstrate its utility via a ...

  6. Evaluating Stationary Distribution of the Binary GA Markov Chain in Special Cases

    OpenAIRE

    Mitavskiy, Boris S.; Cannings, Chris

    2008-01-01

    The evolutionary algorithm stochastic process is well-known to be Markovian. These have been under investigation in much of the theoretical evolutionary computing research. When mutation rate is positive, the Markov chain modeling an evolutionary algorithm is irreducible and, therefore, has a unique stationary distribution, yet, rather little is known about the stationary distribution. On the other hand, knowing the stationary distribution may provide some information abo...

  7. Approximate regenerative-block bootstrap for Markov chains: some simulation studies

    OpenAIRE

    Bertail, Patrice; Clémençon, Stéphan

    2007-01-01

    Abstract : In Bertail & Clémençon (2005a) a novel methodology for bootstrappinggeneral Harris Markov chains has been proposed, which crucially exploits their renewalproperties (when eventually extended via the Nummelin splitting technique) and has theoreticalproperties that surpass other existing methods within the Markovian framework(bmoving block bootstrap, sieve bootstrap etc...). This paper is devoted to discuss practicalissues related to the implementation of this specific resampling met...

  8. Explosion, implosion, and moments of passage times for continuous-time Markov chains: a semimartingale approach

    CERN Document Server

    Menshikov, Mikhail

    2012-01-01

    We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of occurrence of the phenomenon of explosion are also obtained. A new phenomenon of implosion is introduced and sharp conditions for its occurrence are proven. The general results are illustrated by treating models having a difficult behaviour even in discrete time.

  9. Laser-based detection and tracking moving objects using data-driven Markov chain Monte Carlo

    OpenAIRE

    Vu, Trung-Dung; Aycard, Olivier

    2009-01-01

    We present a method of simultaneous detection and tracking moving objects from a moving vehicle equipped with a single layer laser scanner. A model-based approach is introduced to interpret the laser measurement sequence by hypotheses of moving object trajectories over a sliding window of time. Knowledge of various aspects including object model, measurement model, motion model are integrated in one theoretically sound Bayesian framework. The data-driven Markov chain Monte Carlo (DDMCMC) tech...

  10. An Expectation Maximization Algorithm to Model Failure Times by Continuous-Time Markov Chains

    OpenAIRE

    Dengfu Zhao; Zhiping Chen; Qihong Duan

    2010-01-01

    In many applications, the failure rate function may present a bathtub shape curve. In this paper, an expectation maximization algorithm is proposed to construct a suitable continuous-time Markov chain which models the failure time data by the first time reaching the absorbing state. Assume that a system is described by methods of supplementary variables, the device of stage, and so on. Given a data set, the maximum likelihood estimators of the initial distribution and the infinitesimal transi...

  11. Convergence rates for reversible Markov chains without the assumption of nonnegative definite matrices

    Institute of Scientific and Technical Information of China (English)

    2010-01-01

    Explicit convergence rates in geometric and strong ergodicity for denumerable discrete time Markov chains with general reversible transition matrices are obtained in terms of the geometric moments or uniform moments of the hitting times to a fixed point.Another way by Lyapunov’s drift conditions is also used to derive these convergence rates.As a typical example,the discrete time birth-death process(random walk) is studied and the explicit criteria for geometric ergodicity are presented.

  12. Forecasting Income Distributions of Households in Poland on the Basis of Markov Chains

    OpenAIRE

    Czajkowski, Andrzej

    2009-01-01

    In order to forecast income distributions of population, we can make use of, among others, stochastic processes. These processes can be used to determine probabilities of transition of households from one income class to another. The paper attempts to present an application of homogenous Markov chains in the process of forecasting the income structure of six socio-economic groups of population in Poland for the years 2004, 2006 and 2008. Forecasts are based on results of individual household ...

  13. A MARKOV CHAIN ANALYSIS OF STRUCTURAL CHANGES IN THE TEXAS HIGH PLAINS COTTON GINNING INDUSTRY

    OpenAIRE

    Ethridge, Don E.; Roy, Sujit K.; Myers, David W.

    1985-01-01

    Markov chain analysis of changes in the number and size of cotton gin firms in West Texas was conducted assuming stationary and non-stationary transition probabilities. Projections of industry structure were made to 1999 with stationary probability assumptions and six sets of assumed conditions for labor and energy costs and technological change in the non-stationary transition model. Results indicate a continued decline in number of firms, but labor, energy, and technology conditions alter t...

  14. Experimental study of parallel iterative solutions of Markov chains with block partitions

    OpenAIRE

    Migallón Gomis, Violeta; Penadés, Jose; Szyld, Daniel B.

    1999-01-01

    Experiments are performed which demonstrate that parallel implementations of block stationary iterative methods can solve singular systems of equations in substantially less time than the sequential counteparts. Furthermore, these experiments illustrate the behavior of different partitions of matrices representing Markov chains, when parallel iterative methods are used for their solution. Several version of block iterative methods are tested. Spanish CICYT grant PB96-1054-CV02-1, National...

  15. Convergence measure and some parallel aspects of Markov-chain Monte Carlo algorithms

    Science.gov (United States)

    Malfait, Maurits J.; Roose, Dirk; Vandermeulen, Dirk

    1993-10-01

    We examine methods to assess the convergence of Markov chain Monte Carlo (MCMC) algorithms and to accelerate their execution via parallel computing. We propose a convergence measure based on the deviations between simultaneously running MCMC algorithms. We also examine the acceleration of MCMC algorithms when independent parallel sampler are used and report on some experiments with coupled samplers. As applications we use small Ising model simulations and a larger medical image processing algorithm.

  16. THE DECOMPOSITION OF STATE SPACE FOR MARKOV CHAIN IN RANDOM ENVIRONMENT

    Institute of Scientific and Technical Information of China (English)

    Hu Dihe

    2005-01-01

    This paper is a continuation of [8] and [9]. The author obtains the decomposition of state space χof an Markov chain in random environment by making use of the results in [8] and [9], gives three examples, random walk in random environment, renewal process in random environment and queue process in random environment, and obtains the decompositions of the state spaces of these three special examples.

  17. Analysis of Users’ Web Navigation Behavior using GRPA with Variable Length Markov Chains

    Directory of Open Access Journals (Sweden)

    Bindu Madhuri. Ch

    2011-03-01

    Full Text Available With the never-ending growth of Web services and Web-based information systems, the volumes of click stream and user data collected by Web-based organizations in their daily operations has reached enormous proportions. Analyzing such huge data can help to evaluate the effectiveness of promotional campaigns, optimize the functionality of Web-based applications, and provide more personalized content to visitors. In the previous work, we had proposed a method, Grey Relational Pattern Analysis using Markov chains, which involves to discovering the meaningful patterns and relationships from a large collection of data, often stored in Web and applications server access logs, proxy logs etc. Herein, we propose a novel approach to analyse the navigational behavior of User using GRPA with Variable-Length Markov Chains. A VLMC is a model extension that allows variable length history to be captured. GRPA with VariableLength Markov Chains, which reflects on sequential information in Web usage data effectively and efficiently, and it can be extended to allow integration with a Web user navigation behavior prediction model for better Web Usage mining Applications.

  18. Bayesian clustering of DNA sequences using Markov chains and a stochastic partition model.

    Science.gov (United States)

    Jääskinen, Väinö; Parkkinen, Ville; Cheng, Lu; Corander, Jukka

    2014-02-01

    In many biological applications it is necessary to cluster DNA sequences into groups that represent underlying organismal units, such as named species or genera. In metagenomics this grouping needs typically to be achieved on the basis of relatively short sequences which contain different types of errors, making the use of a statistical modeling approach desirable. Here we introduce a novel method for this purpose by developing a stochastic partition model that clusters Markov chains of a given order. The model is based on a Dirichlet process prior and we use conjugate priors for the Markov chain parameters which enables an analytical expression for comparing the marginal likelihoods of any two partitions. To find a good candidate for the posterior mode in the partition space, we use a hybrid computational approach which combines the EM-algorithm with a greedy search. This is demonstrated to be faster and yield highly accurate results compared to earlier suggested clustering methods for the metagenomics application. Our model is fairly generic and could also be used for clustering of other types of sequence data for which Markov chains provide a reasonable way to compress information, as illustrated by experiments on shotgun sequence type data from an Escherichia coli strain. PMID:24246289

  19. A Markov chain Monte Carlo method family in incomplete data analysis

    Directory of Open Access Journals (Sweden)

    Vasić Vladimir V.

    2003-01-01

    Full Text Available A Markov chain Monte Carlo method family is a collection of techniques for pseudorandom draws out of probability distribution function. In recent years, these techniques have been the subject of intensive interest of many statisticians. Roughly speaking, the essence of a Markov chain Monte Carlo method family is generating one or more values of a random variable Z, which is usually multidimensional. Let P(Z = f(Z denote a density function of a random variable Z, which we will refer to as a target distribution. Instead of sampling directly from the distribution f, we will generate [Z(1, Z(2..., Z(t,... ], in which each value is, in a way, dependant upon the previous value and where the stationary distribution will be a target distribution. For a sufficient value of t, Z(t will be approximately random sampling of the distribution f. A Markov chain Monte Carlo method family is useful when direct sampling is difficult, but when sampling of each value is not.

  20. State space orderings for Gauss-Seidel in Markov chains revisited

    Energy Technology Data Exchange (ETDEWEB)

    Dayar, T. [Bilkent Univ., Ankara (Turkey)

    1996-12-31

    Symmetric state space orderings of a Markov chain may be used to reduce the magnitude of the subdominant eigenvalue of the (Gauss-Seidel) iteration matrix. Orderings that maximize the elemental mass or the number of nonzero elements in the dominant term of the Gauss-Seidel splitting (that is, the term approximating the coefficient matrix) do not necessarily converge faster. An ordering of a Markov chain that satisfies Property-R is semi-convergent. On the other hand, there are semi-convergent symmetric state space orderings that do not satisfy Property-R. For a given ordering, a simple approach for checking Property-R is shown. An algorithm that orders the states of a Markov chain so as to increase the likelihood of satisfying Property-R is presented. The computational complexity of the ordering algorithm is less than that of a single Gauss-Seidel iteration (for sparse matrices). In doing all this, the aim is to gain an insight for faster converging orderings. Results from a variety of applications improve the confidence in the algorithm.

  1. Phase Transitions for Quantum XY-Model on the Cayley Tree of Order Three in Quantum Markov Chain Scheme

    International Nuclear Information System (INIS)

    In the present paper we study forward Quantum Markov Chains (QMC) defined on a Cayley tree. Using the tree structure of graphs, we give a construction of quantum Markov chains on a Cayley tree. By means of such constructions we prove the existence of a phase transition for the XY-model on a Cayley tree of order three in QMC scheme. By the phase transition we mean the existence of two distinct QMC for the given family of interaction operators {K}. (author)

  2. Markov chains or the game of structure and chance. From complex networks, to language evolution, to musical compositions

    Science.gov (United States)

    Blanchard, Ph.; Dawin, J. R.; Volchenkov, D.

    2010-06-01

    Markov chains provide us with a powerful tool for studying the structure of graphs and databases in details. We review the method of generalized inverses for Markov chains and apply it for the analysis of urban structures, evolution of languages, and musical compositions. We also discuss a generalization of Lévy flights over large complex networks and study the interplay between the nonlinearity of diffusion process and the topological structure of the network.

  3. Memory Functions of the Additive Markov chains: Applications to Complex Dynamic Systems

    CERN Document Server

    Melnyk, S S; Yampolskii, V A

    2004-01-01

    A new approach to describing correlation properties of complex dynamic systems with long-range memory based on a concept of additive Markov chains (Phys. Rev. E 68, 061107 (2003)) is developed. An equation connecting a memory function of the chain and its correlation function is presented. This equation allows reconstructing the memory function using the correlation function of the system. Thus, we have elaborated a novel method to generate a sequence with prescribed correlation function. Effectiveness and robustness of the proposed method is demonstrated by simple model examples. Memory functions of concrete coarse-grained literary texts are found and their universal power-law behavior at long distances is revealed.

  4. Analysis on the Spatial-Temporal Dynamics of Financial Agglomeration with Markov Chain Approach in China

    Directory of Open Access Journals (Sweden)

    Weimin Chen

    2014-01-01

    Full Text Available The standard approach to studying financial industrial agglomeration is to construct measures of the degree of agglomeration within financial industry. But such measures often fail to exploit the convergence or divergence of financial agglomeration. In this paper, we apply Markov chain approach to diagnose the convergence of financial agglomeration in China based on the location quotient coefficients across the provincial regions over 1993–2011. The estimation of Markov transition probability matrix offers more detailed insights into the mechanics of financial agglomeration evolution process in China during the research period. The results show that the spatial evolution of financial agglomeration changes faster in the period of 2003–2011 than that in the period of 1993–2002. Furthermore, there exists a very uneven financial development patterns, but there is regional convergence for financial agglomeration in China.

  5. Ancestry inference in complex admixtures via variable-length Markov chain linkage models.

    Science.gov (United States)

    Rodriguez, Jesse M; Bercovici, Sivan; Elmore, Megan; Batzoglou, Serafim

    2013-03-01

    Inferring the ancestral origin of chromosomal segments in admixed individuals is key for genetic applications, ranging from analyzing population demographics and history, to mapping disease genes. Previous methods addressed ancestry inference by using either weak models of linkage disequilibrium, or large models that make explicit use of ancestral haplotypes. In this paper we introduce ALLOY, an efficient method that incorporates generalized, but highly expressive, linkage disequilibrium models. ALLOY applies a factorial hidden Markov model to capture the parallel process producing the maternal and paternal admixed haplotypes, and models the background linkage disequilibrium in the ancestral populations via an inhomogeneous variable-length Markov chain. We test ALLOY in a broad range of scenarios ranging from recent to ancient admixtures with up to four ancestral populations. We show that ALLOY outperforms the previous state of the art, and is robust to uncertainties in model parameters. PMID:23421795

  6. Markov chain model helps predict pitting corrosion depth and rate in underground pipelines

    Energy Technology Data Exchange (ETDEWEB)

    Caleyo, F.; Velazquez, J.C.; Hallen, J. M. [ESIQIE, Instituto Politecnico Nacional, Mexico D. F. (Mexico); Esquivel-Amezcua, A. [PEMEX PEP Region Sur, Villahermosa, Tabasco (Mexico); Valor, A. [Universidad de la Habana, Vedado, La Habana (Cuba)

    2010-07-01

    Recent reports place pipeline corrosion costs in North America at seven billion dollars per year. Pitting corrosion causes the higher percentage of failures among other corrosion mechanisms. This has motivated multiple modelling studies to be focused on corrosion pitting of underground pipelines. In this study, a continuous-time, non-homogenous pure birth Markov chain serves to model external pitting corrosion in buried pipelines. The analytical solution of Kolmogorov's forward equations for this type of Markov process gives the transition probability function in a discrete space of pit depths. The transition probability function can be completely identified by making a correlation between the stochastic pit depth mean and the deterministic mean obtained experimentally. The model proposed in this study can be applied to pitting corrosion data from repeated in-line pipeline inspections. Case studies presented in this work show how pipeline inspection and maintenance planning can be improved by using the proposed Markovian model for pitting corrosion.

  7. Markov chain modeling of evolution of strains in reinforced concrete flexural beams

    Directory of Open Access Journals (Sweden)

    Anoop, M. B.

    2012-09-01

    Full Text Available From the analysis of experimentally observed variations in surface strains with loading in reinforced concrete beams, it is noted that there is a need to consider the evolution of strains (with loading as a stochastic process. Use of Markov Chains for modeling stochastic evolution of strains with loading in reinforced concrete flexural beams is studied in this paper. A simple, yet practically useful, bi-level homogeneous Gaussian Markov Chain (BLHGMC model is proposed for determining the state of strain in reinforced concrete beams. The BLHGMC model will be useful for predicting behavior/response of reinforced concrete beams leading to more rational design.A través del análisis de la evolución de la deformación superficial observada experimentalmente en vigas de hormigón armado al entrar en carga, se constata que dicho proceso debe considerarse estocástico. En este trabajo se estudia la utilización de cadenas de Markov para modelizar la evolución estocástica de la deformación de vigas flexotraccionadas. Se propone, para establecer el estado de deformación de estas, un modelo con distribución gaussiana tipo cadena de Markov homogénea de dos niveles (BLHGMC por sus siglas en inglés, cuyo empleo resulta sencillo y práctico. Se comprueba la utilidad del modelo BLHGMC para prever el comportamiento de estos elementos, lo que determina a su vez una mayor racionalidad a la hora de su cálculo y diseño

  8. Markov chain Monte Carlo based analysis of post-translationally modified VDAC gating kinetics.

    Science.gov (United States)

    Tewari, Shivendra G; Zhou, Yifan; Otto, Bradley J; Dash, Ranjan K; Kwok, Wai-Meng; Beard, Daniel A

    2014-01-01

    The voltage-dependent anion channel (VDAC) is the main conduit for permeation of solutes (including nucleotides and metabolites) of up to 5 kDa across the mitochondrial outer membrane (MOM). Recent studies suggest that VDAC activity is regulated via post-translational modifications (PTMs). Yet the nature and effect of these modifications is not understood. Herein, single channel currents of wild-type, nitrosated, and phosphorylated VDAC are analyzed using a generalized continuous-time Markov chain Monte Carlo (MCMC) method. This developed method describes three distinct conducting states (open, half-open, and closed) of VDAC activity. Lipid bilayer experiments are also performed to record single VDAC activity under un-phosphorylated and phosphorylated conditions, and are analyzed using the developed stochastic search method. Experimental data show significant alteration in VDAC gating kinetics and conductance as a result of PTMs. The effect of PTMs on VDAC kinetics is captured in the parameters associated with the identified Markov model. Stationary distributions of the Markov model suggest that nitrosation of VDAC not only decreased its conductance but also significantly locked VDAC in a closed state. On the other hand, stationary distributions of the model associated with un-phosphorylated and phosphorylated VDAC suggest a reversal in channel conformation from relatively closed state to an open state. Model analyses of the nitrosated data suggest that faster reaction of nitric oxide with Cys-127 thiol group might be responsible for the biphasic effect of nitric oxide on basal VDAC conductance. PMID:25628567

  9. Markov chain Monte Carlo based analysis of post-translationally modified VDAC1 gating kinetics

    Directory of Open Access Journals (Sweden)

    Shivendra eTewari

    2015-01-01

    Full Text Available The voltage-dependent anion channel (VDAC is the main conduit for permeation of solutes (including nucleotides and metabolites of up to 5 kDa across the mitochondrial outer membrane (MOM. Recent studies suggest that VDAC activity is regulated via post-translational modifications (PTMs. Yet the nature and effect of these modifications is not understood. Herein, single channel currents of wild-type, nitrosated and phosphorylated VDAC are analyzed using a generalized continuous-time Markov chain Monte Carlo (MCMC method. This developed method describes three distinct conducting states (open, half-open, and closed of VDAC1 activity. Lipid bilayer experiments are also performed to record single VDAC activity under un-phosphorylated and phosphorylated conditions, and are analyzed using the developed stochastic search method. Experimental data show significant alteration in VDAC gating kinetics and conductance as a result of PTMs. The effect of PTMs on VDAC kinetics is captured in the parameters associated with the identified Markov model. Stationary distributions of the Markov model suggests that nitrosation of VDAC not only decreased its conductance but also significantly locked VDAC in a closed state. On the other hand, stationary distributions of the model associated with un-phosphorylated and phosphorylated VDAC suggest a reversal in channel conformation from relatively closed state to an open state. Model analyses of the nitrosated data suggest that faster reaction of nitric oxide with Cys-127 thiol group might be responsible for the biphasic effect of nitric oxide on basal VDAC conductance.

  10. Inference of Kinetic Parameters of Delayed Stochastic Models of Gene Expression Using a Markov Chain Approximation

    Directory of Open Access Journals (Sweden)

    Mannerstrom Henrik

    2011-01-01

    Full Text Available We propose a Markov chain approximation of the delayed stochastic simulation algorithm to infer properties of the mechanisms in prokaryote transcription from the dynamics of RNA levels. We model transcription using the delayed stochastic modelling strategy and realistic parameter values for rate of transcription initiation and RNA degradation. From the model, we generate time series of RNA levels at the single molecule level, from which we use the method to infer the duration of the promoter open complex formation. This is found to be possible even when adding external Gaussian noise to the RNA levels.

  11. Exponential approximation for the nearly critical Galton-Watson process and occupation times of Markov chains

    CERN Document Server

    Peköz, E

    2010-01-01

    In this article we provide new applications for exponential approximation using the framework of Pek\\"oz and R\\"ollin (in press), which is based on Stein's method. We give error bounds for the nearly critical Galton-Watson process conditioned on non-extinction, and for the occupation times of Markov chains; for the latter, in particular, we give a new exponential approximation rate for the number of revisits to the origin for general two dimensional random walk, also known as the Erd\\H{o}s-Taylor theorem.

  12. Observational constraints on G-corrected holographic dark energy using a Markov chain Monte Carlo method

    Science.gov (United States)

    Alavirad, Hamzeh; Malekjani, Mohammad

    2014-02-01

    We constrain holographic dark energy (HDE) with time varying gravitational coupling constant in the framework of the modified Friedmann equations using cosmological data from type Ia supernovae, baryon acoustic oscillations, cosmic microwave background radiation and X-ray gas mass fraction. Applying a Markov Chain Monte Carlo (MCMC) simulation, we obtain the best fit values of the model and cosmological parameters within 1 σ confidence level (CL) in a flat universe as: , , and the HDE constant . Using the best fit values, the equation of state of the dark component at the present time w d0 at 1 σ CL can cross the phantom boundary w=-1.

  13. A Markov chain technique for determining the acquisition behavior of a digital tracking loop

    Science.gov (United States)

    Chadwick, H. D.

    1972-01-01

    An iterative procedure is presented for determining the acquisition behavior of discrete or digital implementations of a tracking loop. The technique is based on the theory of Markov chains and provides the cumulative probability of acquisition in the loop as a function of time in the presence of noise and a given set of initial condition probabilities. A digital second-order tracking loop to be used in the Viking command receiver for continuous tracking of the command subcarrier phase was analyzed using this technique, and the results agree closely with experimental data.

  14. Integration within the Felsenstein equation for improved Markov chain Monte Carlo methods in population genetics

    DEFF Research Database (Denmark)

    Hey, Jody; Nielsen, Rasmus

    2007-01-01

    In 1988, Felsenstein described a framework for assessing the likelihood of a genetic data set in which all of the possible genealogical histories of the data are considered, each in proportion to their probability. Although not analytically solvable, several approaches, including Markov chain Monte...... posterior density of the model parameters. For many purposes, this function can be treated as a likelihood, thereby permitting likelihood-based analyses, including likelihood ratio tests of nested models. Several examples, including an application to the divergence of chimpanzee subspecies, are provided....

  15. A comparison of strategies for Markov chain Monte Carlo computation in quantitative genetics

    DEFF Research Database (Denmark)

    Waagepetersen, Rasmus; Ibanez-Escriche, Noelia; Sorensen, Daniel

    2008-01-01

    in quantitative genetics is to obtain efficient updates of the high-dimensional vectors of genetic random effects and the associated covariance parameters. We discuss various strategies to approach this problem including reparameterization, Langevin-Hastings updates, and updates based on normal approximations......In quantitative genetics, Markov chain Monte Carlo (MCMC) methods are indispensable for statistical inference in non-standard models like generalized linear models with genetic random effects or models with genetically structured variance heterogeneity. A particular challenge for MCMC applications....... The methods are compared in applications to Bayesian inference for three data sets using a model with genetically structured variance heterogeneity...

  16. A new method based on Markov chains for deriving SB2 orbits directly from their spectra

    CERN Document Server

    Salomon, J -B; Guillout, P; Halbwachs, J -L; Arenou, F; Famaey, B; Lebreton, Y; Mazeh, T; Pourbaix, D; Tal-Or, L

    2012-01-01

    We present a new method to derive orbital elements of double-lined spectroscopic binaries (SB2). The aim is to have accurate orbital parameters of a selection of SB2 in order to prepare the exploitation of astrometric Gaia observations. Combined with our results, they should allow one to measure the mass of each star with a precision of better than 1%. The new method presented here consists of using the spectra at all epochs simultaneously to derive the orbital elements without templates. It is based on a Markov chain including a new method for disentangling the spectra.

  17. Sequential Tracking of a Hidden Markov Chain Using Point Process Observations

    CERN Document Server

    Bayraktar, Erhan

    2007-01-01

    We study finite horizon optimal switching problems for hidden Markov chain models under partially observable Poisson processes. The controller possesses a finite range of strategies and attempts to track the state of the unobserved state variable using Bayesian updates over the discrete observations. Such a model has applications in economic policy making, staffing under variable demand levels and generalized Poisson disorder problems. We show regularity of the value function and explicitly characterize an optimal strategy. We also provide an efficient numerical scheme and illustrate our results with several computational examples.

  18. A mixture representation of π with applications in Markov chain Monte Carlo and perfect sampling

    OpenAIRE

    Hobert, James P.; Robert, Christian P.

    2004-01-01

    Let X={Xn:n=0,1,2,…} be an irreducible, positive recurrent Markov chain with invariant probability measure π. We show that if X satisfies a one-step minorization condition, then π can be represented as an infinite mixture. The distributions in the mixture are associated with the hitting times on an accessible atom introduced via the splitting construction of Athreya and Ney [Trans. Amer. Math. Soc. 245 (1978) 493–501] and Nummelin [Z. Wahrsch. Verw. Gebiete 43 (1978) 309–318]. When the small ...

  19. Reliability measures for indexed semi-Markov chains applied to wind energy production

    CERN Document Server

    D'Amico, Guglielmo; Prattico, Flavio

    2013-01-01

    The computation of the dependability measures is a crucial point in the planning and development of a wind farm. In this paper we address the issue of energy production by wind turbine by using an indexed semi-Markov chain as a model of wind speed. We present the mathematical model, we describe the data and technical characteristics of a commercial wind turbine (Aircon HAWT-10kW). We show how to compute some of the main dependability measures such as reliability, availability and maintainability functions. We compare the results of the model with real energy production obtained from data available in the Lastem station (Italy) and sampled every 10 minutes.

  20. Markov chain Monte Carlo linkage analysis of a complex qualitative phenotype.

    Science.gov (United States)

    Hinrichs, A; Lin, J H; Reich, T; Bierut, L; Suarez, B K

    1999-01-01

    We tested a new computer program, LOKI, that implements a reversible jump Markov chain Monte Carlo (MCMC) technique for segregation and linkage analysis. Our objective was to determine whether this software, designed for use with continuously distributed phenotypes, has any efficacy when applied to the discrete disease states of the simulated data from the Mordor data from GAW Problem 1. Although we were able to identify the genomic location for two of the three quantitative trait loci by repeated application of the software, the MCMC sampler experienced significant mixing problems indicating that the method, as currently formulated in LOKI, was not suitable for the discrete phenotypes in this data set. PMID:10597502

  1. A neurocomputational model of stimulus-specific adaptation to oddball and Markov sequences.

    Directory of Open Access Journals (Sweden)

    Robert Mill

    2011-08-01

    Full Text Available Stimulus-specific adaptation (SSA occurs when the spike rate of a neuron decreases with repetitions of the same stimulus, but recovers when a different stimulus is presented. It has been suggested that SSA in single auditory neurons may provide information to change detection mechanisms evident at other scales (e.g., mismatch negativity in the event related potential, and participate in the control of attention and the formation of auditory streams. This article presents a spiking-neuron model that accounts for SSA in terms of the convergence of depressing synapses that convey feature-specific inputs. The model is anatomically plausible, comprising just a few homogeneously connected populations, and does not require organised feature maps. The model is calibrated to match the SSA measured in the cortex of the awake rat, as reported in one study. The effect of frequency separation, deviant probability, repetition rate and duration upon SSA are investigated. With the same parameter set, the model generates responses consistent with a wide range of published data obtained in other auditory regions using other stimulus configurations, such as block, sequential and random stimuli. A new stimulus paradigm is introduced, which generalises the oddball concept to Markov chains, allowing the experimenter to vary the tone probabilities and the rate of switching independently. The model predicts greater SSA for higher rates of switching. Finally, the issue of whether rarity or novelty elicits SSA is addressed by comparing the responses of the model to deviants in the context of a sequence of a single standard or many standards. The results support the view that synaptic adaptation alone can explain almost all aspects of SSA reported to date, including its purported novelty component, and that non-trivial networks of depressing synapses can intensify this novelty response.

  2. A neurocomputational model of stimulus-specific adaptation to oddball and Markov sequences.

    Science.gov (United States)

    Mill, Robert; Coath, Martin; Wennekers, Thomas; Denham, Susan L

    2011-08-01

    Stimulus-specific adaptation (SSA) occurs when the spike rate of a neuron decreases with repetitions of the same stimulus, but recovers when a different stimulus is presented. It has been suggested that SSA in single auditory neurons may provide information to change detection mechanisms evident at other scales (e.g., mismatch negativity in the event related potential), and participate in the control of attention and the formation of auditory streams. This article presents a spiking-neuron model that accounts for SSA in terms of the convergence of depressing synapses that convey feature-specific inputs. The model is anatomically plausible, comprising just a few homogeneously connected populations, and does not require organised feature maps. The model is calibrated to match the SSA measured in the cortex of the awake rat, as reported in one study. The effect of frequency separation, deviant probability, repetition rate and duration upon SSA are investigated. With the same parameter set, the model generates responses consistent with a wide range of published data obtained in other auditory regions using other stimulus configurations, such as block, sequential and random stimuli. A new stimulus paradigm is introduced, which generalises the oddball concept to Markov chains, allowing the experimenter to vary the tone probabilities and the rate of switching independently. The model predicts greater SSA for higher rates of switching. Finally, the issue of whether rarity or novelty elicits SSA is addressed by comparing the responses of the model to deviants in the context of a sequence of a single standard or many standards. The results support the view that synaptic adaptation alone can explain almost all aspects of SSA reported to date, including its purported novelty component, and that non-trivial networks of depressing synapses can intensify this novelty response. PMID:21876661

  3. A GM (1, 1 Markov Chain-Based Aeroengine Performance Degradation Forecast Approach Using Exhaust Gas Temperature

    Directory of Open Access Journals (Sweden)

    Ning-bo Zhao

    2014-01-01

    Full Text Available Performance degradation forecast technology for quantitatively assessing degradation states of aeroengine using exhaust gas temperature is an important technology in the aeroengine health management. In this paper, a GM (1, 1 Markov chain-based approach is introduced to forecast exhaust gas temperature by taking the advantages of GM (1, 1 model in time series and the advantages of Markov chain model in dealing with highly nonlinear and stochastic data caused by uncertain factors. In this approach, firstly, the GM (1, 1 model is used to forecast the trend by using limited data samples. Then, Markov chain model is integrated into GM (1, 1 model in order to enhance the forecast performance, which can solve the influence of random fluctuation data on forecasting accuracy and achieving an accurate estimate of the nonlinear forecast. As an example, the historical monitoring data of exhaust gas temperature from CFM56 aeroengine of China Southern is used to verify the forecast performance of the GM (1, 1 Markov chain model. The results show that the GM (1, 1 Markov chain model is able to forecast exhaust gas temperature accurately, which can effectively reflect the random fluctuation characteristics of exhaust gas temperature changes over time.

  4. A new grey forecasting model based on BP neural network and Markov chain

    Institute of Scientific and Technical Information of China (English)

    2007-01-01

    A new grey forecasting model based on BP neural network and Markov chain was proposed. In order to combine the grey forecasting model with neural network, an important theorem that the grey differential equation is equivalent to the time response model, was proved by analyzing the features of grey forecasting model(GM(1,1)). Based on this, the differential equation parameters were included in the network when the BP neural network was constructed, and the neural network was trained by extracting samples from grey system's known data. When BP network was converged, the whitened grey differential equation parameters were extracted and then the grey neural network forecasting model (GNNM(1,1)) was built. In order to reduce stochastic phenomenon in GNNM(1,1), the state transition probability between two states was defined and the Markov transition matrix was established by building the residual sequences between grey forecasting and actual value. Thus, the new grey forecasting model(MNNGM(1,1)) was proposed by combining Markov chain with GNNM(1,1). Based on the above discussion, three different approaches were put forward for forecasting China electricity demands. By comparing GM(1,1) and GNNM(1,1) with the proposed model, the results indicate that the absolute mean error of MNNGM(1,1) is about 0.4 times of GNNM(1,1) and 0.2 times of GM(1,1), and the mean square error of MNNGM(1,1) is about 0.25 times of GNNM(1,1) and 0.1 times of GM(1,1).

  5. A Scalable Multi-chain Markov Chain Monte Carlo Method for Inverting Subsurface Hydraulic and Geological Properties

    Science.gov (United States)

    Bao, J.; Ren, H.; Hou, Z.; Ray, J.; Swiler, L.; Huang, M.

    2015-12-01

    We developed a novel scalable multi-chain Markov chain Monte Carlo (MCMC) method for high-dimensional inverse problems. The method is scalable in terms of number of chains and processors, and is useful for Bayesian calibration of computationally expensive simulators typically used for scientific and engineering calculations. In this study, we demonstrate two applications of this method for hydraulic and geological inverse problems. The first one is monitoring soil moisture variations using tomographic ground penetrating radar (GPR) travel time data, where challenges exist in the inversion of GPR tomographic data for handling non-uniqueness and nonlinearity and high-dimensionality of unknowns. We integrated the multi-chain MCMC framework with the pilot point concept, a curved-ray GPR forward model, and a sequential Gaussian simulation (SGSIM) algorithm for estimating the dielectric permittivity at pilot point locations distributed within the tomogram, as well as its spatial correlation range, which are used to construct the whole field of dielectric permittivity using SGSIM. The second application is reservoir porosity and saturation estimation using the multi-chain MCMC approach to jointly invert marine seismic amplitude versus angle (AVA) and controlled-source electro-magnetic (CSEM) data for a layered reservoir model, where the unknowns to be estimated include the porosity and fluid saturation in each reservoir layer and the electrical conductivity of the overburden and bedrock. The computational efficiency, accuracy, and convergence behaviors of the inversion approach are systematically evaluated.

  6. Inferring Sequential Order of Somatic Mutations during Tumorgenesis based on Markov Chain Model.

    Science.gov (United States)

    Kang, Hao; Cho, Kwang-Hyun; Zhang, Xiaohua Douglas; Zeng, Tao; Chen, Luonan

    2015-01-01

    Tumors are developed and worsen with the accumulated mutations on DNA sequences during tumorigenesis. Identifying the temporal order of gene mutations in cancer initiation and development is a challenging topic. It not only provides a new insight into the study of tumorigenesis at the level of genome sequences but also is an effective tool for early diagnosis of tumors and preventive medicine. In this paper, we develop a novel method to accurately estimate the sequential order of gene mutations during tumorigenesis from genome sequencing data based on Markov chain model as TOMC (Temporal Order based on Markov Chain), and also provide a new criterion to further infer the order of samples or patients, which can characterize the severity or stage of the disease. We applied our method to the analysis of tumors based on several high-throughput datasets. Specifically, first, we revealed that tumor suppressor genes (TSG) tend to be mutated ahead of oncogenes, which are considered as important events for key functional loss and gain during tumorigenesis. Second, the comparisons of various methods demonstrated that our approach has clear advantages over the existing methods due to the consideration on the effect of mutation dependence among genes, such as co-mutation. Third and most important, our method is able to deduce the ordinal sequence of patients or samples to quantitatively characterize their severity of tumors. Therefore, our work provides a new way to quantitatively understand the development and progression of tumorigenesis based on high throughput sequencing data. PMID:26451822

  7. Spreaders and sponges define metastasis in lung cancer: a Markov chain Monte Carlo mathematical model.

    Science.gov (United States)

    Newton, Paul K; Mason, Jeremy; Bethel, Kelly; Bazhenova, Lyudmila; Nieva, Jorge; Norton, Larry; Kuhn, Peter

    2013-05-01

    The classic view of metastatic cancer progression is that it is a unidirectional process initiated at the primary tumor site, progressing to variably distant metastatic sites in a fairly predictable, although not perfectly understood, fashion. A Markov chain Monte Carlo mathematical approach can determine a pathway diagram that classifies metastatic tumors as "spreaders" or "sponges" and orders the timescales of progression from site to site. In light of recent experimental evidence highlighting the potential significance of self-seeding of primary tumors, we use a Markov chain Monte Carlo (MCMC) approach, based on large autopsy data sets, to quantify the stochastic, systemic, and often multidirectional aspects of cancer progression. We quantify three types of multidirectional mechanisms of progression: (i) self-seeding of the primary tumor, (ii) reseeding of the primary tumor from a metastatic site (primary reseeding), and (iii) reseeding of metastatic tumors (metastasis reseeding). The model shows that the combined characteristics of the primary and the first metastatic site to which it spreads largely determine the future pathways and timescales of systemic disease. PMID:23447576

  8. A Markov chain probability model to describe wet and dry patterns of weather at Colombo

    Science.gov (United States)

    Sonnadara, D. U. J.; Jayewardene, D. R.

    2015-01-01

    The hypothesis that the wet and dry patterns of daily precipitation observed in Colombo can be modeled by a first order Markov chain model was tested using daily rainfall data for a 60-year period (1941-2000). The probability of a day being wet or dry was defined with respect to the status of the previous day. Probabilities were assumed to be stationary within a given month. Except for isolated single events, the model is shown to describe the observed sequence of wet and dry spells satisfactorily depending on the season. The accuracy of modeling wet spells is high compared to dry spells. When the model-predicted mean length of wet spells for each month was compared with the estimated values from the data set, a reasonable agreement between the model prediction and estimation is seen (within ±0.1). In general, the data show a higher disagreement for the months having longer dry spells. The mean annual duration of wet spells is 2.6 days while the mean annual duration of dry spells is 3.8 days. It is shown that the model can be used to explore the return periods of long wet and dry spells. We conclude from the study that the Markov chain of order 1 is adequate to describe wet and dry patterns of weather in Colombo.

  9. MBMC: An Effective Markov Chain Approach for Binning Metagenomic Reads from Environmental Shotgun Sequencing Projects.

    Science.gov (United States)

    Wang, Ying; Hu, Haiyan; Li, Xiaoman

    2016-08-01

    Metagenomics is a next-generation omics field currently impacting postgenomic life sciences and medicine. Binning metagenomic reads is essential for the understanding of microbial function, compositions, and interactions in given environments. Despite the existence of dozens of computational methods for metagenomic read binning, it is still very challenging to bin reads. This is especially true for reads from unknown species, from species with similar abundance, and/or from low-abundance species in environmental samples. In this study, we developed a novel taxonomy-dependent and alignment-free approach called MBMC (Metagenomic Binning by Markov Chains). Different from all existing methods, MBMC bins reads by measuring the similarity of reads to the trained Markov chains for different taxa instead of directly comparing reads with known genomic sequences. By testing on more than 24 simulated and experimental datasets with species of similar abundance, species of low abundance, and/or unknown species, we report here that MBMC reliably grouped reads from different species into separate bins. Compared with four existing approaches, we demonstrated that the performance of MBMC was comparable with existing approaches when binning reads from sequenced species, and superior to existing approaches when binning reads from unknown species. MBMC is a pivotal tool for binning metagenomic reads in the current era of Big Data and postgenomic integrative biology. The MBMC software can be freely downloaded at http://hulab.ucf.edu/research/projects/metagenomics/MBMC.html . PMID:27447888

  10. Conditional probability Markov chain simulation based reliability analysis method for nonnormal variables

    Institute of Scientific and Technical Information of China (English)

    2010-01-01

    Based on fast Markov chain simulation for generating the samples distributed in failure region and saddlepoint approximation(SA) technique,an efficient reliability analysis method is presented to evaluate the small failure probability of non-linear limit state function(LSF) with non-normal variables.In the presented method,the failure probability of the non-linear LSF is transformed into a product of the failure probability of the introduced linear LSF and a feature ratio factor.The introduced linear LSF which approximately has the same maximum likelihood points as the non-linear LSF is constructed and its failure probability can be calculated by SA technique.The feature ratio factor,which can be evaluated on the basis of multiplicative rule of probability,exhibits the relation between the failure probability of the non-linear LSF and that of the linear LSF,and it can be fast computed by utilizing the Markov chain algorithm to directly simulate the samples distributed in the failure regions of the non-linear LSF and those of the linear LSF.Moreover,the expectation and variance of the failure probability estimate are derived.The results of several examples demonstrate that the presented method has wide applicability,can be easily implemented,and possesses high precision and high efficiency.

  11. Markov Chain Modelling of Reliability Analysis and Prediction under Mixed Mode Loading

    Institute of Scientific and Technical Information of China (English)

    SINGH Salvinder; ABDULLAH Shahrum; NIK MOHAMED Nik Abdullah; MOHD NOORANI Mohd Salmi

    2015-01-01

    The reliability assessment for an automobile crankshaft provides an important understanding in dealing with the design life of the component in order to eliminate or reduce the likelihood of failure and safety risks. The failures of the crankshafts are considered as a catastrophic failure that leads towards a severe failure of the engine block and its other connecting subcomponents. The reliability of an automotive crankshaft under mixed mode loading using the Markov Chain Model is studied. The Markov Chain is modelled by using a two-state condition to represent the bending and torsion loads that would occur on the crankshaft. The automotive crankshaft represents a good case study of a component under mixed mode loading due to the rotating bending and torsion stresses. An estimation of the Weibull shape parameter is used to obtain the probability density function, cumulative distribution function, hazard and reliability rate functions, the bathtub curve and the mean time to failure. The various properties of the shape parameter is used to model the failure characteristic through the bathtub curve is shown. Likewise, an understanding of the patterns posed by the hazard rate onto the component can be used to improve the design and increase the life cycle based on the reliability and dependability of the component. The proposed reliability assessment provides an accurate, efficient, fast and cost effective reliability analysis in contrast to costly and lengthy experimental techniques.

  12. Markov chain modelling of reliability analysis and prediction under mixed mode loading

    Science.gov (United States)

    Singh, Salvinder; Abdullah, Shahrum; Nik Mohamed, Nik Abdullah; Mohd Noorani, Mohd Salmi

    2015-03-01

    The reliability assessment for an automobile crankshaft provides an important understanding in dealing with the design life of the component in order to eliminate or reduce the likelihood of failure and safety risks. The failures of the crankshafts are considered as a catastrophic failure that leads towards a severe failure of the engine block and its other connecting subcomponents. The reliability of an automotive crankshaft under mixed mode loading using the Markov Chain Model is studied. The Markov Chain is modelled by using a two-state condition to represent the bending and torsion loads that would occur on the crankshaft. The automotive crankshaft represents a good case study of a component under mixed mode loading due to the rotating bending and torsion stresses. An estimation of the Weibull shape parameter is used to obtain the probability density function, cumulative distribution function, hazard and reliability rate functions, the bathtub curve and the mean time to failure. The various properties of the shape parameter is used to model the failure characteristic through the bathtub curve is shown. Likewise, an understanding of the patterns posed by the hazard rate onto the component can be used to improve the design and increase the life cycle based on the reliability and dependability of the component. The proposed reliability assessment provides an accurate, efficient, fast and cost effective reliability analysis in contrast to costly and lengthy experimental techniques.

  13. Variance-reduced simulation of lattice discrete-time Markov chains with applications in reaction networks

    Science.gov (United States)

    Maginnis, P. A.; West, M.; Dullerud, G. E.

    2016-10-01

    We propose an algorithm to accelerate Monte Carlo simulation for a broad class of stochastic processes. Specifically, the class of countable-state, discrete-time Markov chains driven by additive Poisson noise, or lattice discrete-time Markov chains. In particular, this class includes simulation of reaction networks via the tau-leaping algorithm. To produce the speedup, we simulate pairs of fair-draw trajectories that are negatively correlated. Thus, when averaged, these paths produce an unbiased Monte Carlo estimator that has reduced variance and, therefore, reduced error. Numerical results for three example systems included in this work demonstrate two to four orders of magnitude reduction of mean-square error. The numerical examples were chosen to illustrate different application areas and levels of system complexity. The areas are: gene expression (affine state-dependent rates), aerosol particle coagulation with emission and human immunodeficiency virus infection (both with nonlinear state-dependent rates). Our algorithm views the system dynamics as a "black-box", i.e., we only require control of pseudorandom number generator inputs. As a result, typical codes can be retrofitted with our algorithm using only minor changes. We prove several analytical results. Among these, we characterize the relationship of covariances between paths in the general nonlinear state-dependent intensity rates case, and we prove variance reduction of mean estimators in the special case of affine intensity rates.

  14. Testing and Evaluation for Web Usability Based on Extended Markov Chain Model

    Institute of Scientific and Technical Information of China (English)

    MAO Cheng-ying; LU Yan-sheng

    2004-01-01

    As the increasing popularity and complexity of Web applications and the emergence of their new characteristics, the testing and maintenance of large, complex Web applications are becoming more complex and difficult.Web applications generally contain lots of pages and are used by enormous users.Statistical testing is an effective way of ensuring their quality.Web usage can be accurately described by Markov chain which has been proved to be an ideal model for software statistical testing.The results of unit testing can be utilized in the latter stages, which is an important strategy for bottom-to-top integration testing, and the other improvement of extended Markov chain model (EMM) is to present the error type vector which is treated as a part of page node.This paper also proposes the algorithm for generating test cases of usage paths.Finally, optional usage reliability evaluation methods and an incremental usability regression testing model for testing and evaluation are presented.

  15. Brief Communication: Earthquake sequencing: analysis of time series constructed from the Markov chain model

    Science.gov (United States)

    Cavers, M. S.; Vasudevan, K.

    2015-10-01

    Directed graph representation of a Markov chain model to study global earthquake sequencing leads to a time series of state-to-state transition probabilities that includes the spatio-temporally linked recurrent events in the record-breaking sense. A state refers to a configuration comprised of zones with either the occurrence or non-occurrence of an earthquake in each zone in a pre-determined time interval. Since the time series is derived from non-linear and non-stationary earthquake sequencing, we use known analysis methods to glean new information. We apply decomposition procedures such as ensemble empirical mode decomposition (EEMD) to study the state-to-state fluctuations in each of the intrinsic mode functions. We subject the intrinsic mode functions, derived from the time series using the EEMD, to a detailed analysis to draw information content of the time series. Also, we investigate the influence of random noise on the data-driven state-to-state transition probabilities. We consider a second aspect of earthquake sequencing that is closely tied to its time-correlative behaviour. Here, we extend the Fano factor and Allan factor analysis to the time series of state-to-state transition frequencies of a Markov chain. Our results support not only the usefulness of the intrinsic mode functions in understanding the time series but also the presence of power-law behaviour exemplified by the Fano factor and the Allan factor.

  16. Short-term droughts forecast using Markov chain model in Victoria, Australia

    Science.gov (United States)

    Rahmat, Siti Nazahiyah; Jayasuriya, Niranjali; Bhuiyan, Muhammed A.

    2016-04-01

    A comprehensive risk management strategy for dealing with drought should include both short-term and long-term planning. The objective of this paper is to present an early warning method to forecast drought using the Standardised Precipitation Index (SPI) and a non-homogeneous Markov chain model. A model such as this is useful for short-term planning. The developed method has been used to forecast droughts at a number of meteorological monitoring stations that have been regionalised into six (6) homogenous clusters with similar drought characteristics based on SPI. The non-homogeneous Markov chain model was used to estimate drought probabilities and drought predictions up to 3 months ahead. The drought severity classes defined using the SPI were computed at a 12-month time scale. The drought probabilities and the predictions were computed for six clusters that depict similar drought characteristics in Victoria, Australia. Overall, the drought severity class predicted was quite similar for all the clusters, with the non-drought class probabilities ranging from 49 to 57 %. For all clusters, the near normal class had a probability of occurrence varying from 27 to 38 %. For the more moderate and severe classes, the probabilities ranged from 2 to 13 % and 3 to 1 %, respectively. The developed model predicted drought situations 1 month ahead reasonably well. However, 2 and 3 months ahead predictions should be used with caution until the models are developed further.

  17. Controlling influenza disease: Comparison between discrete time Markov chain and deterministic model

    Science.gov (United States)

    Novkaniza, F.; Ivana, Aldila, D.

    2016-04-01

    Mathematical model of respiratory diseases spread with Discrete Time Markov Chain (DTMC) and deterministic approach for constant total population size are analyzed and compared in this article. Intervention of medical treatment and use of medical mask included in to the model as a constant parameter to controlling influenza spreads. Equilibrium points and basic reproductive ratio as the endemic criteria and it level set depend on some variable are given analytically and numerically as a results from deterministic model analysis. Assuming total of human population is constant from deterministic model, number of infected people also analyzed with Discrete Time Markov Chain (DTMC) model. Since Δt → 0, we could assume that total number of infected people might change only from i to i + 1, i - 1, or i. Approximation probability of an outbreak with gambler's ruin problem will be presented. We find that no matter value of basic reproductive ℛ0, either its larger than one or smaller than one, number of infection will always tends to 0 for t → ∞. Some numerical simulation to compare between deterministic and DTMC approach is given to give a better interpretation and a better understanding about the models results.

  18. A new method for RGB to CIELAB color space transformation based on Markov chain Monte Carlo

    Science.gov (United States)

    Chen, Yajun; Liu, Ding; Liang, Junli

    2013-10-01

    During printing quality inspection, the inspection of color error is an important content. However, the RGB color space is device-dependent, usually RGB color captured from CCD camera must be transformed into CIELAB color space, which is perceptually uniform and device-independent. To cope with the problem, a Markov chain Monte Carlo (MCMC) based algorithms for the RGB to the CIELAB color space transformation is proposed in this paper. Firstly, the modeling color targets and testing color targets is established, respectively used in modeling and performance testing process. Secondly, we derive a Bayesian model for estimation the coefficients of a polynomial, which can be used to describe the relation between RGB and CIELAB color space. Thirdly, a Markov chain is set up base on Gibbs sampling algorithm (one of the MCMC algorithm) to estimate the coefficients of polynomial. Finally, the color difference of testing color targets is computed for evaluating the performance of the proposed method. The experimental results showed that the nonlinear polynomial regression based on MCMC algorithm is effective, whose performance is similar to the least square approach and can accurately model the RGB to the CIELAB color space conversion and guarantee the color error evaluation for printing quality inspection system.

  19. Bayesian Inference for LISA Pathfinder using Markov Chain Monte Carlo Methods

    CERN Document Server

    Ferraioli, Luigi; Plagnol, Eric

    2012-01-01

    We present a parameter estimation procedure based on a Bayesian framework by applying a Markov Chain Monte Carlo algorithm to the calibration of the dynamical parameters of a space based gravitational wave detector. The method is based on the Metropolis-Hastings algorithm and a two-stage annealing treatment in order to ensure an effective exploration of the parameter space at the beginning of the chain. We compare two versions of the algorithm with an application to a LISA Pathfinder data analysis problem. The two algorithms share the same heating strategy but with one moving in coordinate directions using proposals from a multivariate Gaussian distribution, while the other uses the natural logarithm of some parameters and proposes jumps in the eigen-space of the Fisher Information matrix. The algorithm proposing jumps in the eigen-space of the Fisher Information matrix demonstrates a higher acceptance rate and a slightly better convergence towards the equilibrium parameter distributions in the application to...

  20. Entropy and long-range memory in random symbolic additive Markov chains

    Science.gov (United States)

    Melnik, S. S.; Usatenko, O. V.

    2016-06-01

    The goal of this paper is to develop an estimate for the entropy of random symbolic sequences with elements belonging to a finite alphabet. As a plausible model, we use the high-order additive stationary ergodic Markov chain with long-range memory. Supposing that the correlations between random elements of the chain are weak, we express the conditional entropy of the sequence by means of the symbolic pair correlation function. We also examine an algorithm for estimating the conditional entropy of finite symbolic sequences. We show that the entropy contains two contributions, i.e., the correlation and the fluctuation. The obtained analytical results are used for numerical evaluation of the entropy of written English texts and DNA nucleotide sequences. The developed theory opens the way for constructing a more consistent and sophisticated approach to describe the systems with strong short-range and weak long-range memory.

  1. Enhanced modeling via network theory: Adaptive sampling of Markov state models

    OpenAIRE

    Bowman, Gregory R; Ensign, Daniel L.; Pande, Vijay S.

    2010-01-01

    Computer simulations can complement experiments by providing insight into molecular kinetics with atomic resolution. Unfortunately, even the most powerful supercomputers can only simulate small systems for short timescales, leaving modeling of most biologically relevant systems and timescales intractable. In this work, however, we show that molecular simulations driven by adaptive sampling of networks called Markov State Models (MSMs) can yield tremendous time and resource savings, allowing p...

  2. A Markov Chain-based quantitative study of angular distribution of photons through turbid slabs via isotropic light scattering

    Science.gov (United States)

    Li, Xuesong; Northrop, William F.

    2016-04-01

    This paper describes a quantitative approach to approximate multiple scattering through an isotropic turbid slab based on Markov Chain theorem. There is an increasing need to utilize multiple scattering for optical diagnostic purposes; however, existing methods are either inaccurate or computationally expensive. Here, we develop a novel Markov Chain approximation approach to solve multiple scattering angular distribution (AD) that can accurately calculate AD while significantly reducing computational cost compared to Monte Carlo simulation. We expect this work to stimulate ongoing multiple scattering research and deterministic reconstruction algorithm development with AD measurements.

  3. The Study of Reinforcement Learning for Traffic Self-Adaptive Control under Multiagent Markov Game Environment

    Directory of Open Access Journals (Sweden)

    Lun-Hui Xu

    2013-01-01

    Full Text Available Urban traffic self-adaptive control problem is dynamic and uncertain, so the states of traffic environment are hard to be observed. Efficient agent which controls a single intersection can be discovered automatically via multiagent reinforcement learning. However, in the majority of the previous works on this approach, each agent needed perfect observed information when interacting with the environment and learned individually with less efficient coordination. This study casts traffic self-adaptive control as a multiagent Markov game problem. The design employs traffic signal control agent (TSCA for each signalized intersection that coordinates with neighboring TSCAs. A mathematical model for TSCAs’ interaction is built based on nonzero-sum markov game which has been applied to let TSCAs learn how to cooperate. A multiagent Markov game reinforcement learning approach is constructed on the basis of single-agent Q-learning. This method lets each TSCA learn to update its Q-values under the joint actions and imperfect information. The convergence of the proposed algorithm is analyzed theoretically. The simulation results show that the proposed method is convergent and effective in realistic traffic self-adaptive control setting.

  4. Unsupervised SAR images change detection with hidden Markov chains on a sliding window

    Science.gov (United States)

    Bouyahia, Zied; Benyoussef, Lamia; Derrode, Stéphane

    2007-10-01

    This work deals with unsupervised change detection in bi-date Synthetic Aperture Radar (SAR) images. Whatever the indicator of change used, e.g. log-ratio or Kullback-Leibler divergence, we have observed poor quality change maps for some events when using the Hidden Markov Chain (HMC) model we focus on in this work. The main reason comes from the stationary assumption involved in this model - and in most Markovian models such as Hidden Markov Random Fields-, which can not be justified in most observed scenes: changed areas are not necessarily stationary in the image. Besides the few non stationary Markov models proposed in the literature, the aim of this paper is to describe a pragmatic solution to tackle stationarity by using a sliding window strategy. In this algorithm, the criterion image is scanned pixel by pixel, and a classical HMC model is applied only on neighboring pixels. By moving the window through the image, the process is able to produce a change map which can better exhibit non stationary changes than the classical HMC applied directly on the whole criterion image. Special care is devoted to the estimation of the number of classes in each window, which can vary from one (no change) to three (positive change, negative change and no change) by using the corrected Akaike Information Criterion (AICc) suited to small samples. The quality assessment of the proposed approach is achieved with speckle-simulated images in which simulated changes is introduced. The windowed strategy is also evaluated with a pair of RADARSAT images bracketing the Nyiragongo volcano eruption event in January 2002. The available ground truth confirms the effectiveness of the proposed approach compared to a classical HMC-based strategy.

  5. Reliability Measures of Second-Order Semi-Markov Chain Applied to Wind Energy Production

    Directory of Open Access Journals (Sweden)

    Guglielmo D'Amico

    2013-01-01

    Full Text Available We consider the problem of wind energy production by using a second-order semi-Markov chain in state and duration as a model of wind speed. The model used in this paper is based on our previous work where we have shown the ability of second-order semi-Markov process in reproducing statistical features of wind speed. Here we briefly present the mathematical model and describe the data and technical characteristics of a commercial wind turbine (Aircon HAWT-10 kW. We show how, by using our model, it is possible to compute some of the main dependability measures such as reliability, availability, and maintainability functions. We compare, by means of Monte Carlo simulations, the results of the model with real energy production obtained from data available in the Lastem station (Italy and sampled every 10 minutes. The computation of the dependability measures is a crucial point in the planning and development of a wind farm. Through our model, we show how the values of this quantity can be obtained both analytically and computationally.

  6. Retail Banking Loan Portfolio Equilibrium Mix : A Markov Chain Model Analysis

    Directory of Open Access Journals (Sweden)

    V. Thyagarajan

    2005-01-01

    Full Text Available The variance analysis of actual loan sanctions with the non-documented method of loan allocation of the selected retail bank, over a period of 24 months, revealed that there is a scope to improve their income earnings. Realizing its importance Markov Chain Market Share model was applied to inter temporal data of loan disbursements of the selected bank. By applying Estimate Transition Matrix, scope for probability of loan switching among its types was calculated to suggest the probable mix of loan portfolio. From the results it was suggested that the loan proportions among various types were as follows: Housing (32.0 %, Others (28.1 %, Business (20.0 % and Education (19.7 %. These proportions can be taken as guideline percentage within the government norms for the priority sector. Simulation studies were also done to calculate the expected income of interest using Markov proportions and compared with the actual interest earnings to prove the superiority of the model.

  7. Exit time tails from pairwise decorrelation in hidden Markov chains, with applications to dynamical percolation

    CERN Document Server

    Hammond, Alan; Pete, Gábor

    2011-01-01

    Consider a Markov process \\omega_t at equilibrium and some event C (a subset of the state-space of the process). A natural measure of correlations in the process is the pairwise correlation \\Pr[\\omega_0,\\omega_t \\in C] - \\Pr[\\omega_0 \\in C]^2. A second natural measure is the probability of the continual occurrence event \\{\\omega_s \\in C, \\forall s\\in [0,t]\\}. We show that for reversible Markov chains, and any event C, pairwise decorrelation of the event C implies a decay of the probability of the continual occurrence event \\{\\omega_s \\in C, \\forall s \\in [0,t]\\} as t\\to\\infty. We provide examples showing that our results are often sharp. Our main applications are to dynamical critical percolation. Let C be the left-right crossing event of a large box, and let us scale time so that the expected number of changes to C is order 1 in unit time. We show that the continual connection event has superpolynomial decay. Furthermore, on the infinite lattice without any time scaling, the first exceptional time with an in...

  8. Combined survival analysis of cardiac patients by a Cox PH model and a Markov chain.

    Science.gov (United States)

    Shauly, Michal; Rabinowitz, Gad; Gilutz, Harel; Parmet, Yisrael

    2011-10-01

    The control and treatment of dyslipidemia is a major public health challenge, particularly for patients with coronary heart diseases. In this paper we propose a framework for survival analysis of patients who had a major cardiac event, focusing on assessment of the effect of changing LDL-cholesterol level and statins consumption on survival. This framework includes a Cox PH model and a Markov chain, and combines their results into reinforced conclusions regarding the factors that affect survival time. We prospectively studied 2,277 cardiac patients, and the results show high congruence between the Markov model and the PH model; both evidence that diabetes, history of stroke, peripheral vascular disease and smoking significantly increase hazard rate and reduce survival time. On the other hand, statin consumption is correlated with a lower hazard rate and longer survival time in both models. The role of such a framework in understanding the therapeutic behavior of patients and implementing effective secondary and primary prevention of heart diseases is discussed here. PMID:21735134

  9. Markov chain modeling of precipitation time series: Modeling waiting times between tipping bucket rain gauge tips

    DEFF Research Database (Denmark)

    Sørup, Hjalte Jomo Danielsen; Madsen, Henrik; Arnbjerg-Nielsen, Karsten

    2011-01-01

    A very fine temporal and volumetric resolution precipitation time series is modeled using Markov models. Both 1st and 2nd order Markov models as well as seasonal and diurnal models are investigated and evaluated using likelihood based techniques. The 2nd order Markov model is found to be insignif......A very fine temporal and volumetric resolution precipitation time series is modeled using Markov models. Both 1st and 2nd order Markov models as well as seasonal and diurnal models are investigated and evaluated using likelihood based techniques. The 2nd order Markov model is found to be...

  10. Estimation of the transition matrix of a discrete-time Markov chain.

    Science.gov (United States)

    Craig, Bruce A; Sendi, Peter P

    2002-01-01

    Discrete-time Markov chains have been successfully used to investigate treatment programs and health care protocols for chronic diseases. In these situations, the transition matrix, which describes the natural progression of the disease, is often estimated from a cohort observed at common intervals. Estimation of the matrix, however, is often complicated by the complex relationship among transition probabilities. This paper summarizes methods to obtain the maximum likelihood estimate of the transition matrix when the cycle length of the model coincides with the observation interval, the cycle length does not coincide with the observation interval, and when the observation intervals are unequal in length. In addition, the bootstrap is discussed as a method to assess the uncertainty of the maximum likelihood estimate and to construct confidence intervals for functions of the transition matrix such as expected survival. PMID:11788980

  11. Observational constraints on G-corrected holographic dark energy using a Markov chain Monte Carlo method

    CERN Document Server

    Alavirad, Hamzeh

    2013-01-01

    We constrain holographic dark energy (HDE) with time varying gravitational coupling constant in the framework of the modified Friedmann equations using cosmological data from type Ia supernovae, baryon acoustic oscillations, cosmic microwave background radiation and X-ray gas mass fraction. Applying a Markov Chain Monte Carlo (MCMC) simulation, we obtain the best fit values of the model and cosmological parameters within $1\\sigma$ confidence level (CL) in a flat universe as: $\\Omega_{\\rm b}h^2=0.0222^{+0.0018}_{-0.0013}$, $\\Omega_{\\rm c}h^2 =0.1121^{+0.0110}_{-0.0079}$, $\\alpha_{\\rm G}\\equiv \\dot{G}/(HG) =0.1647^{+0.3547}_{-0.2971}$ and the HDE constant $c=0.9322^{+0.4569}_{-0.5447}$. Using the best fit values, the equation of state of the dark component at the present time $w_{\\rm d0}$ at $1\\sigma$ CL can cross the phantom boundary $w=-1$.

  12. Markov Chain Monte Carlo methods applied to measuring the fine structure constant from quasar spectroscopy

    Science.gov (United States)

    King, Julian; Mortlock, Daniel; Webb, John; Murphy, Michael

    2010-11-01

    Recent attempts to constrain cosmological variation in the fine structure constant, α, using quasar absorption lines have yielded two statistical samples which initially appear to be inconsistent. One of these samples was subsequently demonstrated to not pass consistency tests; it appears that the optimisation algorithm used to fit the model to the spectra failed. Nevertheless, the results of the other hinge on the robustness of the spectral fitting program VPFIT, which has been tested through simulation but not through direct exploration of the likelihood function. We present the application of Markov Chain Monte Carlo (MCMC) methods to this problem, and demonstrate that VPFIT produces similar values and uncertainties for Δα/α, the fractional change in the fine structure constant, as our MCMC algorithm, and thus that VPFIT is reliable.

  13. Markov Chain Monte Carlo methods applied to measuring the fine structure constant from quasar spectroscopy

    CERN Document Server

    King, Julian A; Webb, John K; Murphy, Michael T

    2009-01-01

    Recent attempts to constrain cosmological variation in the fine structure constant, alpha, using quasar absorption lines have yielded two statistical samples which initially appear to be inconsistent. One of these samples was subsequently demonstrated to not pass consistency tests; it appears that the optimisation algorithm used to fit the model to the spectra failed. Nevertheless, the results of the other hinge on the robustness of the spectral fitting program VPFIT, which has been tested through simulation but not through direct exploration of the likelihood function. We present the application of Markov Chain Monte Carlo (MCMC) methods to this problem, and demonstrate that VPFIT produces similar values and uncertainties for (Delta alpha)/(alpha), the fractional change in the fine structure constant, as our MCMC algorithm, and thus that VPFIT is reliable.

  14. Very short-term wind speed prediction: A new artificial neural network-Markov chain model

    Energy Technology Data Exchange (ETDEWEB)

    Pourmousavi Kani, S.A. [Electrical and Computer Engineering Department, 627 Cobleigh Hall, Montana State University, Bozeman, MT 59717 (United States); Ardehali, M.M. [Energy Research Center, Department of Electrical Engineering, Amirkabir University of Technology (Tehran Polytechnic), 424 Hafez Ave., Tehran 15914 (Iran, Islamic Republic of)

    2011-01-15

    As the objective of this study, artificial neural network (ANN) and Markov chain (MC) are used to develop a new ANN-MC model for forecasting wind speed in very short-term time scale. For prediction of very short-term wind speed in a few seconds in the future, data patterns for short-term (about an hour) and very short-term (about minutes or seconds) recorded prior to current time are considered. In this study, the short-term patterns in wind speed data are captured by ANN and the long-term patterns are considered utilizing MC approach and four neighborhood indices. The results are validated and the effectiveness of the new ANN-MC model is demonstrated. It is found that the prediction errors can be decreased, while the uncertainty of the predictions and calculation time are reduced. (author)

  15. Bayesian Lorentzian profile fitting using Markov-Chain Monte Carlo: An observer's approach

    CERN Document Server

    Gruberbauer, M; Weiss, W W

    2008-01-01

    Aims. Investigating stochastically driven pulsation puts strong requirements on the quality of (observed) pulsation frequency spectra, such as the accuracy of frequencies, amplitudes, and mode life times and -- important when fitting these parameters with models -- a realistic error estimate which can be quite different to the formal error. As has been shown by other authors, the method of fitting Lorentzian profiles to the power spectrum of time-resolved photometric or spectroscopic data via the Maximum Likelihood Estimation (MLE) procedure delivers good approximations for these quantities. We, however, intend to demonstrate that a conservative Bayesian approach allows to treat this problem in a more consistent way. Methods. We derive a conservative Bayesian treatment for the probability of Lorentzian profiles being present in a power spectrum and describe its implementation via evaluating the probability density distribution of parameters by using the Markov-Chain Monte Carlo (MCMC) technique. In addition, ...

  16. Study of behavior and determination of customer lifetime value(CLV) using Markov chain model

    International Nuclear Information System (INIS)

    Customer Lifetime Value or CLV is a restriction on interactive marketing to help a company in arranging financial for the marketing of new customer acquisition and customer retention. Additionally CLV can be able to segment customers for financial arrangements. Stochastic models for the fairly new CLV used a Markov chain. In this model customer retention probability and new customer acquisition probability play an important role. This model is originally introduced by Pfeifer and Carraway in 2000 [1]. They introduced several CLV models, one of them only involves customer and former customer. In this paper we expand the model by adding the assumption of the transition from former customer to customer. In the proposed model, the CLV value is higher than the CLV value obtained by Pfeifer and Caraway model. But our model still requires a longer convergence time

  17. Image Edge Detection Using Hidden Markov Chain Model Based on the Non-decimated Wavelet

    Directory of Open Access Journals (Sweden)

    Renqi Zhang

    2009-03-01

    Full Text Available Edge detection plays an important role in digital image processing. Based on the non-decimated wavelet which is shift invariant, in this paper, we develop a new edge detection technique using Hidden Markov Chain (HMC model. With this proposed model (NWHMC, each wavelet coefficient contains a hidden state, herein, we adopt Laplacian model and Gaussian model to represent the information of the state “big” and the state “small”. The model can be trained by EM algorithm, and then we employ Viterbi algorithm to reveal the hidden state of each coefficient according to MAP estimation. The detecting results of several images are provided to evaluate the algorithm. In addition, the algorithm can be applied to noisy images efficiently.

  18. Towards Analyzing Crossover Operators in Evolutionary Search via General Markov Chain Switching Theorem

    CERN Document Server

    Yu, Yang; Zhou, Zhi-Hua

    2011-01-01

    Evolutionary algorithms (EAs), simulating the evolution process of natural species, are used to solve optimization problems. Crossover (also called recombination), originated from simulating the chromosome exchange phenomena in zoogamy reproduction, is widely employed in EAs to generate offspring solutions, of which the effectiveness has been examined empirically in applications. However, due to the irregularity of crossover operators and the complicated interactions to mutation, crossover operators are hard to analyze and thus have few theoretical results. Therefore, analyzing crossover not only helps in understanding EAs, but also helps in developing novel techniques for analyzing sophisticated metaheuristic algorithms. In this paper, we derive the General Markov Chain Switching Theorem (GMCST) to facilitate theoretical studies of crossover-enabled EAs. The theorem allows us to analyze the running time of a sophisticated EA from an easy-to-analyze EA. Using this tool, we analyze EAs with several crossover o...

  19. Study of behavior and determination of customer lifetime value(CLV) using Markov chain model

    Energy Technology Data Exchange (ETDEWEB)

    Permana, Dony, E-mail: donypermana@students.itb.ac.id [Statistics Research Division, Faculty of Mathematics and Natural Science, Bandung Institute of Technology, Indonesia and Statistics Study Program, Faculty of Mathematics and Natural Sciences, Padang State University (Indonesia); Indratno, Sapto Wahyu; Pasaribu, Udjianna S. [Statistics Research Division, Faculty of Mathematics and Natural Science, Bandung Institute of Technology (Indonesia)

    2014-03-24

    Customer Lifetime Value or CLV is a restriction on interactive marketing to help a company in arranging financial for the marketing of new customer acquisition and customer retention. Additionally CLV can be able to segment customers for financial arrangements. Stochastic models for the fairly new CLV used a Markov chain. In this model customer retention probability and new customer acquisition probability play an important role. This model is originally introduced by Pfeifer and Carraway in 2000 [1]. They introduced several CLV models, one of them only involves customer and former customer. In this paper we expand the model by adding the assumption of the transition from former customer to customer. In the proposed model, the CLV value is higher than the CLV value obtained by Pfeifer and Caraway model. But our model still requires a longer convergence time.

  20. Analysis of aerial survey data on Florida manatee using Markov chain Monte Carlo.

    Science.gov (United States)

    Craig, B A; Newton, M A; Garrott, R A; Reynolds, J E; Wilcox, J R

    1997-06-01

    We assess population trends of the Atlantic coast population of Florida manatee, Trichechus manatus latirostris, by reanalyzing aerial survey data collected between 1982 and 1992. To do so, we develop an explicit biological model that accounts for the method by which the manatees are counted, the mammals' movement between surveys, and the behavior of the population total over time. Bayesian inference, enabled by Markov chain Monte Carlo, is used to combine the survey data with the biological model. We compute marginal posterior distributions for all model parameters and predictive distributions for future counts. Several conclusions, such as a decreasing population growth rate and low sighting probabilities, are consistent across different prior specifications. PMID:9192449

  1. Data Model Approach And Markov Chain Based Analysis Of Multi-Level Queue Scheduling

    Directory of Open Access Journals (Sweden)

    Diwakar Shukla

    2010-01-01

    Full Text Available There are many CPU scheduling algorithms inliterature like FIFO, Round Robin, Shortest-Job-First and so on.The Multilevel-Queue-Scheduling is superior to these due to itsbetter management of a variety of processes. In this paper, aMarkov chain model is used for a general setup of Multilevelqueue-scheduling and the scheduler is assumed to performrandom movement on queue over the quantum of time.Performance of scheduling is examined through a rowdependent data model. It is found that with increasing value of αand d, the chance of system going over the waiting state reduces.At some of the interesting combinations of α and d, it diminishesto zero, thereby, provides us some clue regarding better choice ofqueues over others for high priority jobs. It is found that ifqueue priorities are added in the scheduling intelligently thenbetter performance could be obtained. Data model helpschoosing appropriate preferences.

  2. Study of behavior and determination of customer lifetime value(CLV) using Markov chain model

    Science.gov (United States)

    Permana, Dony; Indratno, Sapto Wahyu; Pasaribu, Udjianna S.

    2014-03-01

    Customer Lifetime Value or CLV is a restriction on interactive marketing to help a company in arranging financial for the marketing of new customer acquisition and customer retention. Additionally CLV can be able to segment customers for financial arrangements. Stochastic models for the fairly new CLV used a Markov chain. In this model customer retention probability and new customer acquisition probability play an important role. This model is originally introduced by Pfeifer and Carraway in 2000 [1]. They introduced several CLV models, one of them only involves customer and former customer. In this paper we expand the model by adding the assumption of the transition from former customer to customer. In the proposed model, the CLV value is higher than the CLV value obtained by Pfeifer and Caraway model. But our model still requires a longer convergence time.

  3. From complex spatial dynamics to simple Markov chain models: do predators and prey leave footprints?

    DEFF Research Database (Denmark)

    Nachman, Gøsta Støger; Borregaard, Michael Krabbe

    2010-01-01

    patches with both prey and predators, with prey only, with predators only, and with neither species, along with the number of patches that change from one state to another in each time step. The average number of patches in the four states, as well as the average transition probabilities from one state to...... species) are reflected by different footprints The transition probabilities can be used to forecast the expected fate of a system given its current state. However, the transition probabilities in the modeled system depend on the number of patches in each state. We develop a model for the dependence of...... transition probabilities on state variables, and combine this information in a Markov chain transition matrix model. Finally, we use this extended model to predict the long-term dynamics of the system and to reveal its asymptotic steady state properties....

  4. A Markov Chain Model for the Analysis of Round-Robin Scheduling Scheme

    Directory of Open Access Journals (Sweden)

    D. Shukla

    2009-07-01

    Full Text Available In the literature of Round-Robin scheduling scheme, each job is processed, one after the another after giving a fix quantum. In case of First-come first-served, each process is executed, if the previously arrived processed is completed. Both these scheduling schemes are used in this paper as its special cases. A Markov chain model is used to compare several scheduling schemes of the class. An index measure is defined to compare the model based efficiency of different scheduling schemes. One scheduling scheme which is the mixture of FIFO and round robin is found efficient in terms of model based study. The system simulation procedure is used to derive the conclusion of the content.

  5. Markov Chain Monte Carlo (MCMC) methods for parameter estimation of a novel hybrid redundant robot

    International Nuclear Information System (INIS)

    This paper presents a statistical method for the calibration of a redundantly actuated hybrid serial-parallel robot IWR (Intersector Welding Robot). The robot under study will be used to carry out welding, machining, and remote handing for the assembly of vacuum vessel of International Thermonuclear Experimental Reactor (ITER). The robot has ten degrees of freedom (DOF), among which six DOF are contributed by the parallel mechanism and the rest are from the serial mechanism. In this paper, a kinematic error model which involves 54 unknown geometrical error parameters is developed for the proposed robot. Based on this error model, the mean values of the unknown parameters are statistically analyzed and estimated by means of Markov Chain Monte Carlo (MCMC) approach. The computer simulation is conducted by introducing random geometric errors and measurement poses which represent the corresponding real physical behaviors. The simulation results of the marginal posterior distributions of the estimated model parameters indicate that our method is reliable and robust.

  6. Unifying Markov Chain Approach for Disease and Rumor Spreading in Complex Networks

    CERN Document Server

    de Arruda, Guilherme Ferraz; Rodriiguez, Pablo Martin; Cozzo, Emanuele; Moreno, Yamir

    2016-01-01

    Spreading processes are ubiquitous in natural and artificial systems. They can be studied via a plethora of models, depending on the specific details of the phenomena under study. Disease contagion and rumor spreading are among the most important of these processes due to their practical relevance. However, despite the similarities between them, current models address both spreading dynamics separately. In this paper, we propose a general information spreading model that is based on discrete time Markov chains. The model includes all the transitions that are plausible for both a disease contagion process and rumor propagation. We show that our model not only covers the traditional spreading schemes, but that it also contains some features relevant in social dynamics, such as apathy, forgetting, and lost/recovering of interest. The model is evaluated analytically to obtain the spreading thresholds and the early time dynamical behavior for the contact and reactive processes in several scenarios. Comparison with...

  7. A Discrete Time Markov Chain Model for High Throughput Bidirectional Fano Decoders

    CERN Document Server

    Xu, Ran; Morris, Kevin; Kocak, Taskin

    2010-01-01

    The bidirectional Fano algorithm (BFA) can achieve at least two times decoding throughput compared to the conventional unidirectional Fano algorithm (UFA). In this paper, bidirectional Fano decoding is examined from the queuing theory perspective. A Discrete Time Markov Chain (DTMC) is employed to model the BFA decoder with a finite input buffer. The relationship between the input data rate, the input buffer size and the clock speed of the BFA decoder is established. The DTMC based modelling can be used in designing a high throughput parallel BFA decoding system. It is shown that there is a tradeoff between the number of BFA decoders and the input buffer size, and an optimal input buffer size can be chosen to minimize the hardware complexity for a target decoding throughput in designing a high throughput parallel BFA decoding system.

  8. Ideal-observer computation in medical imaging with use of Markov-chain Monte Carlo techniques

    Science.gov (United States)

    Kupinski, Matthew A.; Hoppin, John W.; Clarkson, Eric; Barrett, Harrison H.

    2003-03-01

    The ideal observer sets an upper limit on the performance of an observer on a detection or classification task. The performance of the ideal observer can be used to optimize hardware components of imaging systems and also to determine another observer's relative performance in comparison with the best possible observer. The ideal observer employs complete knowledge of the statistics of the imaging system, including the noise and object variability. Thus computing the ideal observer for images (large-dimensional vectors) is burdensome without severely restricting the randomness in the imaging system, e.g., assuming a flat object. We present a method for computing the ideal-observer test statistic and performance by using Markov-chain Monte Carlo techniques when we have a well-characterized imaging system, knowledge of the noise statistics, and a stochastic object model. We demonstrate the method by comparing three different parallel-hole collimator imaging systems in simulation.

  9. Soft Uncoupling of Markov Chains for Permeable Language Distinction: A New Algorithm

    CERN Document Server

    Nock, Richard; Nielsen, Frank; Henry, Claudia

    2008-01-01

    Without prior knowledge, distinguishing different languages may be a hard task, especially when their borders are permeable. We develop an extension of spectral clustering -- a powerful unsupervised classification toolbox -- that is shown to resolve accurately the task of soft language distinction. At the heart of our approach, we replace the usual hard membership assignment of spectral clustering by a soft, probabilistic assignment, which also presents the advantage to bypass a well-known complexity bottleneck of the method. Furthermore, our approach relies on a novel, convenient construction of a Markov chain out of a corpus. Extensive experiments with a readily available system clearly display the potential of the method, which brings a visually appealing soft distinction of languages that may define altogether a whole corpus.

  10. A stochastic Markov chain model to describe lung cancer growth and metastasis.

    Directory of Open Access Journals (Sweden)

    Paul K Newton

    Full Text Available A stochastic Markov chain model for metastatic progression is developed for primary lung cancer based on a network construction of metastatic sites with dynamics modeled as an ensemble of random walkers on the network. We calculate a transition matrix, with entries (transition probabilities interpreted as random variables, and use it to construct a circular bi-directional network of primary and metastatic locations based on postmortem tissue analysis of 3827 autopsies on untreated patients documenting all primary tumor locations and metastatic sites from this population. The resulting 50 potential metastatic sites are connected by directed edges with distributed weightings, where the site connections and weightings are obtained by calculating the entries of an ensemble of transition matrices so that the steady-state distribution obtained from the long-time limit of the Markov chain dynamical system corresponds to the ensemble metastatic distribution obtained from the autopsy data set. We condition our search for a transition matrix on an initial distribution of metastatic tumors obtained from the data set. Through an iterative numerical search procedure, we adjust the entries of a sequence of approximations until a transition matrix with the correct steady-state is found (up to a numerical threshold. Since this constrained linear optimization problem is underdetermined, we characterize the statistical variance of the ensemble of transition matrices calculated using the means and variances of their singular value distributions as a diagnostic tool. We interpret the ensemble averaged transition probabilities as (approximately normally distributed random variables. The model allows us to simulate and quantify disease progression pathways and timescales of progression from the lung position to other sites and we highlight several key findings based on the model.

  11. A stochastic Markov chain model to describe lung cancer growth and metastasis.

    Science.gov (United States)

    Newton, Paul K; Mason, Jeremy; Bethel, Kelly; Bazhenova, Lyudmila A; Nieva, Jorge; Kuhn, Peter

    2012-01-01

    A stochastic Markov chain model for metastatic progression is developed for primary lung cancer based on a network construction of metastatic sites with dynamics modeled as an ensemble of random walkers on the network. We calculate a transition matrix, with entries (transition probabilities) interpreted as random variables, and use it to construct a circular bi-directional network of primary and metastatic locations based on postmortem tissue analysis of 3827 autopsies on untreated patients documenting all primary tumor locations and metastatic sites from this population. The resulting 50 potential metastatic sites are connected by directed edges with distributed weightings, where the site connections and weightings are obtained by calculating the entries of an ensemble of transition matrices so that the steady-state distribution obtained from the long-time limit of the Markov chain dynamical system corresponds to the ensemble metastatic distribution obtained from the autopsy data set. We condition our search for a transition matrix on an initial distribution of metastatic tumors obtained from the data set. Through an iterative numerical search procedure, we adjust the entries of a sequence of approximations until a transition matrix with the correct steady-state is found (up to a numerical threshold). Since this constrained linear optimization problem is underdetermined, we characterize the statistical variance of the ensemble of transition matrices calculated using the means and variances of their singular value distributions as a diagnostic tool. We interpret the ensemble averaged transition probabilities as (approximately) normally distributed random variables. The model allows us to simulate and quantify disease progression pathways and timescales of progression from the lung position to other sites and we highlight several key findings based on the model. PMID:22558094

  12. Effects of tour boats on dolphin activity examined with sensitivity analysis of Markov chains.

    Science.gov (United States)

    Dans, Silvana Laura; Degrati, Mariana; Pedraza, Susana Noemí; Crespo, Enrique Alberto

    2012-08-01

    In Patagonia, Argentina, watching dolphins, especially dusky dolphins (Lagenorhynchus obscurus), is a new tourist activity. Feeding time decreases and time to return to feeding after feeding is abandoned and time it takes a group of dolphins to feed increase in the presence of boats. Such effects on feeding behavior may exert energetic costs on dolphins and thus reduce an individual's survival and reproductive capacity or maybe associated with shifts in distribution. We sought to predict which behavioral changes modify the activity pattern of dolphins the most. We modeled behavioral sequences of dusky dolphins with Markov chains. We calculated transition probabilities from one activity to another and arranged them in a stochastic matrix model. The proportion of time dolphins dedicated to a given activity (activity budget) and the time it took a dolphin to resume that activity after it had been abandoned (recurrence time) were calculated. We used a sensitivity analysis of Markov chains to calculate the sensitivity of the time budget and the activity-resumption time to changes in behavioral transition probabilities. Feeding-time budget was most sensitive to changes in the probability of dolphins switching from traveling to feeding behavior and of maintaining feeding behavior. Thus, an increase in these probabilities would be associated with the largest reduction in the time dedicated to feeding. A reduction in the probability of changing from traveling to feeding would also be associated with the largest increases in the time it takes dolphins to resume feeding. To approach dolphins when they are traveling would not affect behavior less because presence of the boat may keep dolphins from returning to feeding. Our results may help operators of dolphin-watching vessels minimize negative effects on dolphins. PMID:22624561

  13. A Markov Chain Model for Changes in Users' Assessment of Search Results.

    Science.gov (United States)

    Zhitomirsky-Geffet, Maayan; Bar-Ilan, Judit; Levene, Mark

    2016-01-01

    Previous research shows that users tend to change their assessment of search results over time. This is a first study that investigates the factors and reasons for these changes, and describes a stochastic model of user behaviour that may explain these changes. In particular, we hypothesise that most of the changes are local, i.e. between results with similar or close relevance to the query, and thus belong to the same"coarse" relevance category. According to the theory of coarse beliefs and categorical thinking, humans tend to divide the range of values under consideration into coarse categories, and are thus able to distinguish only between cross-category values but not within them. To test this hypothesis we conducted five experiments with about 120 subjects divided into 3 groups. Each student in every group was asked to rank and assign relevance scores to the same set of search results over two or three rounds, with a period of three to nine weeks between each round. The subjects of the last three-round experiment were then exposed to the differences in their judgements and were asked to explain them. We make use of a Markov chain model to measure change in users' judgments between the different rounds. The Markov chain demonstrates that the changes converge, and that a majority of the changes are local to a neighbouring relevance category. We found that most of the subjects were satisfied with their changes, and did not perceive them as mistakes but rather as a legitimate phenomenon, since they believe that time has influenced their relevance assessment. Both our quantitative analysis and user comments support the hypothesis of the existence of coarse relevance categories resulting from categorical thinking in the context of user evaluation of search results.

  14. A Markov Chain Model for Changes in Users’ Assessment of Search Results

    Science.gov (United States)

    Zhitomirsky-Geffet, Maayan; Bar-Ilan, Judit; Levene, Mark

    2016-01-01

    Previous research shows that users tend to change their assessment of search results over time. This is a first study that investigates the factors and reasons for these changes, and describes a stochastic model of user behaviour that may explain these changes. In particular, we hypothesise that most of the changes are local, i.e. between results with similar or close relevance to the query, and thus belong to the same”coarse” relevance category. According to the theory of coarse beliefs and categorical thinking, humans tend to divide the range of values under consideration into coarse categories, and are thus able to distinguish only between cross-category values but not within them. To test this hypothesis we conducted five experiments with about 120 subjects divided into 3 groups. Each student in every group was asked to rank and assign relevance scores to the same set of search results over two or three rounds, with a period of three to nine weeks between each round. The subjects of the last three-round experiment were then exposed to the differences in their judgements and were asked to explain them. We make use of a Markov chain model to measure change in users’ judgments between the different rounds. The Markov chain demonstrates that the changes converge, and that a majority of the changes are local to a neighbouring relevance category. We found that most of the subjects were satisfied with their changes, and did not perceive them as mistakes but rather as a legitimate phenomenon, since they believe that time has influenced their relevance assessment. Both our quantitative analysis and user comments support the hypothesis of the existence of coarse relevance categories resulting from categorical thinking in the context of user evaluation of search results. PMID:27171426

  15. A Markov Chain Model for Changes in Users' Assessment of Search Results.

    Directory of Open Access Journals (Sweden)

    Maayan Zhitomirsky-Geffet

    Full Text Available Previous research shows that users tend to change their assessment of search results over time. This is a first study that investigates the factors and reasons for these changes, and describes a stochastic model of user behaviour that may explain these changes. In particular, we hypothesise that most of the changes are local, i.e. between results with similar or close relevance to the query, and thus belong to the same"coarse" relevance category. According to the theory of coarse beliefs and categorical thinking, humans tend to divide the range of values under consideration into coarse categories, and are thus able to distinguish only between cross-category values but not within them. To test this hypothesis we conducted five experiments with about 120 subjects divided into 3 groups. Each student in every group was asked to rank and assign relevance scores to the same set of search results over two or three rounds, with a period of three to nine weeks between each round. The subjects of the last three-round experiment were then exposed to the differences in their judgements and were asked to explain them. We make use of a Markov chain model to measure change in users' judgments between the different rounds. The Markov chain demonstrates that the changes converge, and that a majority of the changes are local to a neighbouring relevance category. We found that most of the subjects were satisfied with their changes, and did not perceive them as mistakes but rather as a legitimate phenomenon, since they believe that time has influenced their relevance assessment. Both our quantitative analysis and user comments support the hypothesis of the existence of coarse relevance categories resulting from categorical thinking in the context of user evaluation of search results.

  16. Fuzzy hidden Markov chains segmentation for volume determination and quantitation in PET

    Science.gov (United States)

    Hatt, M.; Lamare, F.; Boussion, N.; Turzo, A.; Collet, C.; Salzenstein, F.; Roux, C.; Jarritt, P.; Carson, K.; Cheze-LeRest, C.; Visvikis, D.

    2007-07-01

    Accurate volume of interest (VOI) estimation in PET is crucial in different oncology applications such as response to therapy evaluation and radiotherapy treatment planning. The objective of our study was to evaluate the performance of the proposed algorithm for automatic lesion volume delineation; namely the fuzzy hidden Markov chains (FHMC), with that of current state of the art in clinical practice threshold based techniques. As the classical hidden Markov chain (HMC) algorithm, FHMC takes into account noise, voxel intensity and spatial correlation, in order to classify a voxel as background or functional VOI. However the novelty of the fuzzy model consists of the inclusion of an estimation of imprecision, which should subsequently lead to a better modelling of the 'fuzzy' nature of the object of interest boundaries in emission tomography data. The performance of the algorithms has been assessed on both simulated and acquired datasets of the IEC phantom, covering a large range of spherical lesion sizes (from 10 to 37 mm), contrast ratios (4:1 and 8:1) and image noise levels. Both lesion activity recovery and VOI determination tasks were assessed in reconstructed images using two different voxel sizes (8 mm3 and 64 mm3). In order to account for both the functional volume location and its size, the concept of % classification errors was introduced in the evaluation of volume segmentation using the simulated datasets. Results reveal that FHMC performs substantially better than the threshold based methodology for functional volume determination or activity concentration recovery considering a contrast ratio of 4:1 and lesion sizes of <28 mm. Furthermore differences between classification and volume estimation errors evaluated were smaller for the segmented volumes provided by the FHMC algorithm. Finally, the performance of the automatic algorithms was less susceptible to image noise levels in comparison to the threshold based techniques. The analysis of both

  17. MCMC-ODPR: Primer design optimization using Markov Chain Monte Carlo sampling

    Directory of Open Access Journals (Sweden)

    Kitchen James L

    2012-11-01

    Full Text Available Abstract Background Next generation sequencing technologies often require numerous primer designs that require good target coverage that can be financially costly. We aimed to develop a system that would implement primer reuse to design degenerate primers that could be designed around SNPs, thus find the fewest necessary primers and the lowest cost whilst maintaining an acceptable coverage and provide a cost effective solution. We have implemented Metropolis-Hastings Markov Chain Monte Carlo for optimizing primer reuse. We call it the Markov Chain Monte Carlo Optimized Degenerate Primer Reuse (MCMC-ODPR algorithm. Results After repeating the program 1020 times to assess the variance, an average of 17.14% fewer primers were found to be necessary using MCMC-ODPR for an equivalent coverage without implementing primer reuse. The algorithm was able to reuse primers up to five times. We compared MCMC-ODPR with single sequence primer design programs Primer3 and Primer-BLAST and achieved a lower primer cost per amplicon base covered of 0.21 and 0.19 and 0.18 primer nucleotides on three separate gene sequences, respectively. With multiple sequences, MCMC-ODPR achieved a lower cost per base covered of 0.19 than programs BatchPrimer3 and PAMPS, which achieved 0.25 and 0.64 primer nucleotides, respectively. Conclusions MCMC-ODPR is a useful tool for designing primers at various melting temperatures at good target coverage. By combining degeneracy with optimal primer reuse the user may increase coverage of sequences amplified by the designed primers at significantly lower costs. Our analyses showed that overall MCMC-ODPR outperformed the other primer-design programs in our study in terms of cost per covered base.

  18. Strong law of large numbers for Markov chains indexed by an infinite tree with uniformly bounded degree

    Institute of Scientific and Technical Information of China (English)

    2008-01-01

    In this paper,we study the strong law of large numbers and Shannon-McMillan (S-M) theorem for Markov chains indexed by an infinite tree with uniformly bounded degree.The results generalize the analogous results on a homogeneous tree.

  19. SEMI-BLIND CHANNEL ESTIMATION OF MULTIPLE-INPUT/MULTIPLE-OUTPUT SYSTEMS BASED ON MARKOV CHAIN MONTE CARLO METHODS

    Institute of Scientific and Technical Information of China (English)

    Jiang Wei; Xiang Haige

    2004-01-01

    This paper addresses the issues of channel estimation in a Multiple-Input/Multiple-Output (MIMO) system. Markov Chain Monte Carlo (MCMC) method is employed to jointly estimate the Channel State Information (CSI) and the transmitted signals. The deduced algorithms can work well under circumstances of low Signal-to-Noise Ratio (SNR). Simulation results are presented to demonstrate their effectiveness.

  20. Markov chains with hybrid repeating rows - upper-Hessenberg, quasi-Toeplitz structure of the block transition probability matrix

    OpenAIRE

    Dudin, Alexander; Kim, Chesoong; Klimenok, Valentina

    2008-01-01

    In this paper we consider discrete-time multidimensional Markov chains having a block transition probability matrix which is the sum of a matrix with repeating block rows and a matrix of upper-Hessenberg, quasi-Toeplitz structure. We derive sufficient conditions for the existence of the stationary distribution, and outline two algorithms for calculating the stationary distribution.

  1. Multi-Physics Markov Chain Monte Carlo Methods for Subsurface Flows

    Science.gov (United States)

    Rigelo, J.; Ginting, V.; Rahunanthan, A.; Pereira, F.

    2014-12-01

    For CO2 sequestration in deep saline aquifers, contaminant transport in subsurface, and oil or gas recovery, we often need to forecast flow patterns. Subsurface characterization is a critical and challenging step in flow forecasting. To characterize subsurface properties we establish a statistical description of the subsurface properties that are conditioned to existing dynamic and static data. A Markov Chain Monte Carlo (MCMC) algorithm is used in a Bayesian statistical description to reconstruct the spatial distribution of rock permeability and porosity. The MCMC algorithm requires repeatedly solving a set of nonlinear partial differential equations describing displacement of fluids in porous media for different values of permeability and porosity. The time needed for the generation of a reliable MCMC chain using the algorithm can be too long to be practical for flow forecasting. In this work we develop fast and effective computational methods for generating MCMC chains in the Bayesian framework for the subsurface characterization. Our strategy consists of constructing a family of computationally inexpensive preconditioners based on simpler physics as well as on surrogate models such that the number of fine-grid simulations is drastically reduced in the generated MCMC chains. In particular, we introduce a huff-puff technique as screening step in a three-stage multi-physics MCMC algorithm to reduce the number of expensive final stage simulations. The huff-puff technique in the algorithm enables a better characterization of subsurface near wells. We assess the quality of the proposed multi-physics MCMC methods by considering Monte Carlo simulations for forecasting oil production in an oil reservoir.

  2. Efficient variants of the minimal diffusion formulation of Markov chain ensembles.

    Science.gov (United States)

    Güler, Marifi

    2016-02-01

    This study is concerned with ensembles of continuous-time Markov chains evolving independently under a common transition rate matrix in some finite state space. In this context, our prior work [Phys. Rev. E 91, 062116 (2015)] has formulated an approximation scheme, called the minimal diffusion formulation, to deduce how the number of chains in a prescribed relevant state evolves in time. The formulation consists of two specifically coupled Ornstein-Uhlenbeck processes in a stochastic differential equation representation; it is minimal in the sense that its structure does not change with the state space size or the transition matrix density, and it requires no matrix square-root operations. In the present study, we first calculate the autocorrelation function of the relevant state density in the minimal diffusion formulation, which is fundamental to the identification of the ensemble dynamics. The obtained autocorrelation function is then employed to develop two diffusion formulations that reduce the structural complexity of the minimal diffusion formulation without significant loss of accuracy in the dynamics. One of these variant formulations includes one less noise term than the minimal diffusion formulation and still satisfies the above-mentioned autocorrelation function in its dynamics. The second variant is in the form of a one-dimensional Langevin equation, therefore it is the simplest possible diffusion formulation one can obtain for the problem, yet its autocorrelation function is first-order accurate in time gap. Numerical simulations supporting the theoretical analysis are delivered. PMID:26986304

  3. Comparison of methods for calculating conditional expectations of sufficient statistics for continuous time Markov chains

    Directory of Open Access Journals (Sweden)

    Tataru Paula

    2011-12-01

    Full Text Available Abstract Background Continuous time Markov chains (CTMCs is a widely used model for describing the evolution of DNA sequences on the nucleotide, amino acid or codon level. The sufficient statistics for CTMCs are the time spent in a state and the number of changes between any two states. In applications past evolutionary events (exact times and types of changes are unaccessible and the past must be inferred from DNA sequence data observed in the present. Results We describe and implement three algorithms for computing linear combinations of expected values of the sufficient statistics, conditioned on the end-points of the chain, and compare their performance with respect to accuracy and running time. The first algorithm is based on an eigenvalue decomposition of the rate matrix (EVD, the second on uniformization (UNI, and the third on integrals of matrix exponentials (EXPM. The implementation in R of the algorithms is available at http://www.birc.au.dk/~paula/. Conclusions We use two different models to analyze the accuracy and eight experiments to investigate the speed of the three algorithms. We find that they have similar accuracy and that EXPM is the slowest method. Furthermore we find that UNI is usually faster than EVD.

  4. Using Markov chains to predict the natural progression of diabetic retinopathy

    Institute of Scientific and Technical Information of China (English)

    Priyanka; Srikanth

    2015-01-01

    AIM: To study the natural progression of diabetic retinopathy in patients with type 2 diabetes.METHODS: This was an observational study of 153 cases with type 2 diabetes from 2010 to 2013. The state of patient was noted at end of each year and transition matrices were developed to model movement between years. Patients who progressed to severe non-proliferative diabetic retinopathy(NPDR) were treated.Markov Chains and Chi-square test were used for statistical analysis.RESULTS: We modelled the transition of 153 patients from NPDR to blindness on an annual basis. At the end of year 3, we compared results from the Markov model versus actual data. The results from Chi-square test confirmed that there was statistically no significant difference(P =0.70) which provided assurance that the model was robust to estimate mean sojourn times. The key finding was that a patient entering the system in mild NPDR state is expected to stay in that state for 5y followed by 1.07 y in moderate NPDR, be in the severe NPDR state for 1.33 y before moving into PDR for roughly8 y. It is therefore expected that such a patient entering the model in a state of mild NPDR will enter blindness after 15.29 y.CONCLUSION: Patients stay for long time periods in mild NPDR before transitioning into moderate NPDR.However, they move rapidly from moderate NPDR to proliferative diabetic retinopathy(PDR) and stay in that state for long periods before transitioning into blindness.

  5. Evaluation of screening for nasopharyngeal carcinoma: trial design using Markov chain models.

    Science.gov (United States)

    Chen, H H; Prevost, T C; Duffy, S W

    1999-04-01

    In this paper, we develop a Markov chain model to estimate parameters pertaining to the natural history of nasopharyngeal carcinoma (NPC). The model is of progression from no disease to Epstein-Barr virus (EBV) infection, preclinical screen-detectable tumour and clinical tumour. We derive tentative estimates of the parameters of the model, based on limited published data, to assess the efficacy of serum screening in conjunction with clinical assessment (indirect mirror examination for NPC), for example the average duration of the preclinical screen-detectable phase is estimated as 3.1 years. We further apply these parameters to a hypothetical screening trial in the Hong Kong population to assess the efficacy of serum screening with clinical assessment by different combinations of screening regime. Results suggest: (1) there is no substantial difference between 3-yearly and 6-yearly serum screening; and (2) within the same serum screening regime annual and 3-yearly clinical assessment can prevent 33% and 28% of deaths from NPC respectively. Prediction of deaths and surrogate end points can be used to estimate the required sample size and duration for designing a randomized trial of screening for NPC. Based on these findings and power projections, we suggest a design for a randomized trial in a high incidence area such as Hong Kong. PMID:10206310

  6. Solving inverse problem for Markov chain model of customer lifetime value using flower pollination algorithm

    Science.gov (United States)

    Al-Ma'shumah, Fathimah; Permana, Dony; Sidarto, Kuntjoro Adji

    2015-12-01

    Customer Lifetime Value is an important and useful concept in marketing. One of its benefits is to help a company for budgeting marketing expenditure for customer acquisition and customer retention. Many mathematical models have been introduced to calculate CLV considering the customer retention/migration classification scheme. A fairly new class of these models which will be described in this paper uses Markov Chain Models (MCM). This class of models has the major advantage for its flexibility to be modified to several different cases/classification schemes. In this model, the probabilities of customer retention and acquisition play an important role. From Pfeifer and Carraway, 2000, the final formula of CLV obtained from MCM usually contains nonlinear form of the transition probability matrix. This nonlinearity makes the inverse problem of CLV difficult to solve. This paper aims to solve this inverse problem, yielding the approximate transition probabilities for the customers, by applying metaheuristic optimization algorithm developed by Yang, 2013, Flower Pollination Algorithm. The major interpretation of obtaining the transition probabilities are to set goals for marketing teams in keeping the relative frequencies of customer acquisition and customer retention.

  7. DIM SUM: demography and individual migration simulated using a Markov chain.

    Science.gov (United States)

    Brown, Jeremy M; Savidge, Kevin; McTavish, Emily Jane B

    2011-03-01

    An increasing number of studies seek to infer demographic history, often jointly with genetic relationships. Despite numerous analytical methods for such data, few simulations have investigated the methods' power and robustness, especially when underlying assumptions have been violated. DIM SUM (Demography and Individual Migration Simulated Using a Markov chain) is a stand-alone Java program for the simulation of population demography and individual migration while recording ancestor-descendant relationships. It does not employ coalescent assumptions or discrete population boundaries. It is extremely flexible, allowing the user to specify border positions, reactions of organisms to borders, local and global carrying capacities, individual dispersal kernels, rates of reproduction and strategies for sampling individuals. Spatial variables may be specified using image files (e.g., as exported from gis software) and may vary through time. In combination with software for genetic marker simulation, DIM SUM will be useful for testing phylogeographic (e.g., nested clade phylogeographic analysis, coalescent-based tests and continuous-landscape frameworks) and landscape-genetic methods, specifically regarding violations of coalescent assumptions. It can also be used to explore the qualitative features of proposed demographic scenarios (e.g. regarding biological invasions) and as a pedagogical tool. DIM SUM (with user's manual) can be downloaded from http://code.google.com/p/bio-dimsum. PMID:21429144

  8. Dynamical Models for NGC 6503 using a Markov Chain Monte Carlo Technique

    CERN Document Server

    Puglielli, David; Courteau, Stéphane

    2010-01-01

    We use Bayesian statistics and Markov chain Monte Carlo (MCMC) techniques to construct dynamical models for the spiral galaxy NGC 6503. The constraints include surface brightness profiles which display a Freeman Type II structure; HI and ionized gas rotation curves; the stellar rotation, which is nearly coincident with the ionized gas curve; and the line of sight stellar dispersion, with a sigma-drop at the centre. The galaxy models consist of a Sersic bulge, an exponential disc with an optional inner truncation and a cosmologically motivated dark halo. The Bayesian/MCMC technique yields the joint posterior probability distribution function for the input parameters. We examine several interpretations of the data: the Type II surface brightness profile may be due to dust extinction, to an inner truncated disc or to a ring of bright stars; and we test separate fits to the gas and stellar rotation curves to determine if the gas traces the gravitational potential. We test each of these scenarios for bar stability...

  9. A Markov chain model for image ranking system in social networks

    Science.gov (United States)

    Zin, Thi Thi; Tin, Pyke; Toriu, Takashi; Hama, Hiromitsu

    2014-03-01

    In today world, different kinds of networks such as social, technological, business and etc. exist. All of the networks are similar in terms of distributions, continuously growing and expanding in large scale. Among them, many social networks such as Facebook, Twitter, Flickr and many others provides a powerful abstraction of the structure and dynamics of diverse kinds of inter personal connection and interaction. Generally, the social network contents are created and consumed by the influences of all different social navigation paths that lead to the contents. Therefore, identifying important and user relevant refined structures such as visual information or communities become major factors in modern decision making world. Moreover, the traditional method of information ranking systems cannot be successful due to their lack of taking into account the properties of navigation paths driven by social connections. In this paper, we propose a novel image ranking system in social networks by using the social data relational graphs from social media platform jointly with visual data to improve the relevance between returned images and user intentions (i.e., social relevance). Specifically, we propose a Markov chain based Social-Visual Ranking algorithm by taking social relevance into account. By using some extensive experiments, we demonstrated the significant and effectiveness of the proposed social-visual ranking method.

  10. Phase Transitions for Quantum Markov Chains Associated with Ising Type Models on a Cayley Tree

    Science.gov (United States)

    Mukhamedov, Farrukh; Barhoumi, Abdessatar; Souissi, Abdessatar

    2016-05-01

    The main aim of the present paper is to prove the existence of a phase transition in quantum Markov chain (QMC) scheme for the Ising type models on a Cayley tree. Note that this kind of models do not have one-dimensional analogous, i.e. the considered model persists only on trees. In this paper, we provide a more general construction of forward QMC. In that construction, a QMC is defined as a weak limit of finite volume states with boundary conditions, i.e. QMC depends on the boundary conditions. Our main result states the existence of a phase transition for the Ising model with competing interactions on a Cayley tree of order two. By the phase transition we mean the existence of two distinct QMC which are not quasi-equivalent and their supports do not overlap. We also study some algebraic property of the disordered phase of the model, which is a new phenomena even in a classical setting.

  11. Performance evaluation of corrosion-affected reinforced concrete bridge girders using Markov chains with fuzzy states

    Indian Academy of Sciences (India)

    M B ANOOP; K BALAJI RAO

    2016-08-01

    A methodology for performance evaluation of reinforced concrete bridge girders in corrosive environments is proposed. The methodology uses the concept of performability and considers both serviceability- and ultimate-limit states. The serviceability limit states are defined based on the degree of cracking (characterized by crack width) in the girder due to chloride induced corrosion of reinforcement, and the ultimate limit states are defined based on the flexural load carrying capacity of the girder (characterized in terms of rating factor using the load and resistance factor rating method). The condition of the bridge girder is specified by the assignment of a condition state from a set of predefined condition states. Generally, the classification of condition states is linguistic, while the condition states are considered to be mutually exclusive and collectivelyexhaustive. In the present study, the condition states of the bridge girder are also represented by fuzzy sets to consider the ambiguities arising due to the linguistic classification of condition states. A non-homogeneous Markov chain (MC) model is used for modeling the condition state evolution of the bridge girder with time. The usefulness of the proposed methodology is demonstrated through a case study of a severely distressed beam of the Rocky Point Viaduct. The results obtained using the proposed approach are compared with those obtained using conventional MC model. It is noted that the use of MC with fuzzy states leads to conservative decision making for the problem considered in the case study.

  12. Markov chain models of coupled calcium channels: Kronecker representations and iterative solution methods

    International Nuclear Information System (INIS)

    Mathematical models of calcium release sites derived from Markov chain models of intracellular calcium channels exhibit collective gating reminiscent of the experimentally observed phenomenon of stochastic calcium excitability (i.e., calcium puffs and sparks). Calcium release site models are stochastic automata networks that involve many functional transitions, that is, the transition probabilities of each channel depend on the local calcium concentration and thus the state of the other channels. We present a Kronecker-structured representation for calcium release site models and perform benchmark stationary distribution calculations using both exact and approximate iterative numerical solution techniques that leverage this structure. When it is possible to obtain an exact solution, response measures such as the number of channels in a particular state converge more quickly using the iterative numerical methods than occupation measures calculated via Monte Carlo simulation. In particular, multi-level methods provide excellent convergence with modest additional memory requirements for the Kronecker representation of calcium release site models. When an exact solution is not feasible, iterative approximate methods based on the power method may be used, with performance similar to Monte Carlo estimates. This suggests approximate methods with multi-level iterative engines as a promising avenue of future research for large-scale calcium release site models

  13. Use of Markov Chains to Design an Agent Bidding Strategy for Continuous Double Auctions

    CERN Document Server

    Birmingham, W P; Park, S; 10.1613/jair.1466

    2011-01-01

    As computational agents are developed for increasingly complicated e-commerce applications, the complexity of the decisions they face demands advances in artificial intelligence techniques. For example, an agent representing a seller in an auction should try to maximize the seller?s profit by reasoning about a variety of possibly uncertain pieces of information, such as the maximum prices various buyers might be willing to pay, the possible prices being offered by competing sellers, the rules by which the auction operates, the dynamic arrival and matching of offers to buy and sell, and so on. A naive application of multiagent reasoning techniques would require the seller?s agent to explicitly model all of the other agents through an extended time horizon, rendering the problem intractable for many realistically-sized problems. We have instead devised a new strategy that an agent can use to determine its bid price based on a more tractable Markov chain model of the auction process. We have experimentally ident...

  14. On stochastic error and computational efficiency of the Markov Chain Monte Carlo method

    KAUST Repository

    Li, Jun

    2014-01-01

    In Markov Chain Monte Carlo (MCMC) simulations, thermal equilibria quantities are estimated by ensemble average over a sample set containing a large number of correlated samples. These samples are selected in accordance with the probability distribution function, known from the partition function of equilibrium state. As the stochastic error of the simulation results is significant, it is desirable to understand the variance of the estimation by ensemble average, which depends on the sample size (i.e., the total number of samples in the set) and the sampling interval (i.e., cycle number between two consecutive samples). Although large sample sizes reduce the variance, they increase the computational cost of the simulation. For a given CPU time, the sample size can be reduced greatly by increasing the sampling interval, while having the corresponding increase in variance be negligible if the original sampling interval is very small. In this work, we report a few general rules that relate the variance with the sample size and the sampling interval. These results are observed and confirmed numerically. These variance rules are derived for theMCMCmethod but are also valid for the correlated samples obtained using other Monte Carlo methods. The main contribution of this work includes the theoretical proof of these numerical observations and the set of assumptions that lead to them. © 2014 Global-Science Press.

  15. Study on the Calculation Models of Bus Delay at Bays Using Queueing Theory and Markov Chain

    Directory of Open Access Journals (Sweden)

    Feng Sun

    2015-01-01

    Full Text Available Traffic congestion at bus bays has decreased the service efficiency of public transit seriously in China, so it is crucial to systematically study its theory and methods. However, the existing studies lack theoretical model on computing efficiency. Therefore, the calculation models of bus delay at bays are studied. Firstly, the process that buses are delayed at bays is analyzed, and it was found that the delay can be divided into entering delay and exiting delay. Secondly, the queueing models of bus bays are formed, and the equilibrium distribution functions are proposed by applying the embedded Markov chain to the traditional model of queuing theory in the steady state; then the calculation models of entering delay are derived at bays. Thirdly, the exiting delay is studied by using the queueing theory and the gap acceptance theory. Finally, the proposed models are validated using field-measured data, and then the influencing factors are discussed. With these models the delay is easily assessed knowing the characteristics of the dwell time distribution and traffic volume at the curb lane in different locations and different periods. It can provide basis for the efficiency evaluation of bus bays.

  16. Improving Markov Chain Monte Carlo algorithms in LISA Pathfinder Data Analysis

    Science.gov (United States)

    Karnesis, N.; Nofrarias, M.; Sopuerta, C. F.; Lobo, A.

    2012-06-01

    The LISA Pathfinder mission (LPF) aims to test key technologies for the future LISA mission. The LISA Technology Package (LTP) on-board LPF will consist of an exhaustive suite of experiments and its outcome will be crucial for the future detection of gravitational waves. In order to achieve maximum sensitivity, we need to have an understanding of every instrument on-board and parametrize the properties of the underlying noise models. The Data Analysis team has developed algorithms for parameter estimation of the system. A very promising one implemented for LISA Pathfinder data analysis is the Markov Chain Monte Carlo. A series of experiments are going to take place during flight operations and each experiment is going to provide us with essential information for the next in the sequence. Therefore, it is a priority to optimize and improve our tools available for data analysis during the mission. Using a Bayesian framework analysis allows us to apply prior knowledge for each experiment, which means that we can efficiently use our prior estimates for the parameters, making the method more accurate and significantly faster. This, together with other algorithm improvements, will lead us to our main goal, which is no other than creating a robust and reliable tool for parameter estimation during the LPF mission.

  17. Modelling heterogeneity of Ljubljana polje aquifer using Markov chain and geostatistics

    Directory of Open Access Journals (Sweden)

    Mitja Janža

    2009-12-01

    Full Text Available Heterogeneity of the aquifers is one of the key factors that control transport processes in groundwater. It is defined by the spatial distribution of hydofacies-sedimentsformed in characteristic depositional environments and have typical hydrogeological properties. Due to the (in time and space changing sedimetological conditions, is the distribution of hydrofacies in nature often complex and difficulttodefine.The difficulty of this procedure most of ten limits reliability and consequently applicability of numerical transport models. For the purposes of improvement of reliability of hydrological modelling in the area of alluvial deposits of Ljubljana polje a hydrogeological model was constructed. It is based on the borehole logs, supplemented with geological conceptual information and geostatistical methods, combined with Markov chain models. The model consists of four units - hydrofacies with different volume tricportions (Gravel 45 %, Silt and clay with gravel 36 %, Silt and clay 5 % and Conglomerate 14 %. The used approach enables development of a set of equally probable realisations of spatial distribution of hydrofacies that are conditioned to the borehole data and represent geologically plausible image of the heterogeneity of the aquifer.

  18. Mapping absorption processes onto a Markov chain, conserving the mean first passage time

    Science.gov (United States)

    Biswas, Katja

    2013-04-01

    The dynamics of a multidimensional system is projected onto a discrete state master equation using the transition rates W(k → k‧ t, t + dt) between a set of states {k} represented by the regions {ζk} in phase or discrete state space. Depending on the dynamics Γi(t) of the original process and the choice of ζk, the discretized process can be Markovian or non-Markovian. For absorption processes, it is shown that irrespective of these properties of the projection, a master equation with time-independent transition rates \\bar{W}(k\\rightarrow k^{\\prime }) can be obtained, which conserves the total occupation time of the partitions of the phase or discrete state space of the original process. An expression for the transition probabilities \\bar{p}(k^{\\prime }|k) is derived based on either time-discrete measurements {ti} with variable time stepping Δ(i + 1)i = ti + 1 - ti or the theoretical knowledge at continuous times t. This allows computational methods of absorbing Markov chains to be used to obtain the mean first passage time (MFPT) of the system. To illustrate this approach, the procedure is applied to obtain the MFPT for the overdamped Brownian motion of particles subject to a system with dichotomous noise and the escape from an entropic barrier. The high accuracy of the simulation results confirms with the theory.

  19. Study on the Calculation Models of Bus Delay at Bays Using Queueing Theory and Markov Chain

    Science.gov (United States)

    Sun, Li; Sun, Shao-wei; Wang, Dian-hai

    2015-01-01

    Traffic congestion at bus bays has decreased the service efficiency of public transit seriously in China, so it is crucial to systematically study its theory and methods. However, the existing studies lack theoretical model on computing efficiency. Therefore, the calculation models of bus delay at bays are studied. Firstly, the process that buses are delayed at bays is analyzed, and it was found that the delay can be divided into entering delay and exiting delay. Secondly, the queueing models of bus bays are formed, and the equilibrium distribution functions are proposed by applying the embedded Markov chain to the traditional model of queuing theory in the steady state; then the calculation models of entering delay are derived at bays. Thirdly, the exiting delay is studied by using the queueing theory and the gap acceptance theory. Finally, the proposed models are validated using field-measured data, and then the influencing factors are discussed. With these models the delay is easily assessed knowing the characteristics of the dwell time distribution and traffic volume at the curb lane in different locations and different periods. It can provide basis for the efficiency evaluation of bus bays. PMID:25759720

  20. Two-state Markov-chain Poisson nature of individual cellphone call statistics

    Science.gov (United States)

    Jiang, Zhi-Qiang; Xie, Wen-Jie; Li, Ming-Xia; Zhou, Wei-Xing; Sornette, Didier

    2016-07-01

    Unfolding the burst patterns in human activities and social interactions is a very important issue especially for understanding the spreading of disease and information and the formation of groups and organizations. Here, we conduct an in-depth study of the temporal patterns of cellphone conversation activities of 73 339 anonymous cellphone users, whose inter-call durations are Weibull distributed. We find that the individual call events exhibit a pattern of bursts, that high activity periods are alternated with low activity periods. In both periods, the number of calls are exponentially distributed for individuals, but power-law distributed for the population. Together with the exponential distributions of inter-call durations within bursts and of the intervals between consecutive bursts, we demonstrate that the individual call activities are driven by two independent Poisson processes, which can be combined within a minimal model in terms of a two-state first-order Markov chain, giving significant fits for nearly half of the individuals. By measuring directly the distributions of call rates across the population, which exhibit power-law tails, we purport the existence of power-law distributions, via the ‘superposition of distributions’ mechanism. Our findings shed light on the origins of bursty patterns in other human activities.

  1. Predicting protein subcellular locations using hierarchical ensemble of Bayesian classifiers based on Markov chains

    Directory of Open Access Journals (Sweden)

    Eils Roland

    2006-06-01

    Full Text Available Abstract Background The subcellular location of a protein is closely related to its function. It would be worthwhile to develop a method to predict the subcellular location for a given protein when only the amino acid sequence of the protein is known. Although many efforts have been made to predict subcellular location from sequence information only, there is the need for further research to improve the accuracy of prediction. Results A novel method called HensBC is introduced to predict protein subcellular location. HensBC is a recursive algorithm which constructs a hierarchical ensemble of classifiers. The classifiers used are Bayesian classifiers based on Markov chain models. We tested our method on six various datasets; among them are Gram-negative bacteria dataset, data for discriminating outer membrane proteins and apoptosis proteins dataset. We observed that our method can predict the subcellular location with high accuracy. Another advantage of the proposed method is that it can improve the accuracy of the prediction of some classes with few sequences in training and is therefore useful for datasets with imbalanced distribution of classes. Conclusion This study introduces an algorithm which uses only the primary sequence of a protein to predict its subcellular location. The proposed recursive scheme represents an interesting methodology for learning and combining classifiers. The method is computationally efficient and competitive with the previously reported approaches in terms of prediction accuracies as empirical results indicate. The code for the software is available upon request.

  2. Markov chain formalism for generalized radiative transfer in a plane-parallel medium, accounting for polarization

    Science.gov (United States)

    Xu, Feng; Davis, Anthony B.; Diner, David J.

    2016-11-01

    A Markov chain formalism is developed for computing the transport of polarized radiation according to Generalized Radiative Transfer (GRT) theory, which was developed recently to account for unresolved random fluctuations of scattering particle density and can also be applied to unresolved spectral variability of gaseous absorption as an improvement over the standard correlated-k method. Using Gamma distribution to describe the probability density function of the extinction or absorption coefficient, a shape parameter a that quantifies the variability is introduced, defined as the mean extinction or absorption coefficient squared divided by its variance. It controls the decay rate of a power-law transmission that replaces the usual exponential Beer-Lambert-Bouguer law. Exponential transmission, hence classic RT, is recovered when a→∞. The new approach is verified to high accuracy against numerical benchmark results obtained with a custom Monte Carlo method. For a<∞, angular reciprocity is violated to a degree that increases with the spatial variability, as observed for finite portions of real-world cloudy scenes. While the degree of linear polarization in liquid water cloudbows, supernumerary bows, and glories is affected by spatial heterogeneity, the positions in scattering angle of these features are relatively unchanged. As a result, a single-scattering model based on the assumption of subpixel homogeneity can still be used to derive droplet size distributions from polarimetric measurements of extended stratocumulus clouds.

  3. FINITE MARKOV CHAINS IN THE MODEL REPRESENTATION OF THE HUMAN OPERATOR ACTIVITY IN QUASI-FUNCTIONAL ENVIRONMENT

    Directory of Open Access Journals (Sweden)

    M. V. Serzhantova

    2016-05-01

    Full Text Available Subject of Research. We analyze the problems of finite Markov chains apparatus application for simulating a human operator activity in the quasi-static functional environment. It is shown that the functional environment stochastic nature is generated by a factor of interval character of human operator properties. Method. The problem is solved in the class of regular (recurrent finite Markov chains with three states of the human operator: with a favorable, median and unfavorable combination of the values of mathematical model parameters of the human operator in a quasi-static functional environment. The finite Markov chain is designed taking into account the factors of human operator tiredness and interval character of parameters of the model representation of his properties. The device is based on the usage of mathematical approximation of the standard curve of the human operator activity performance during work shift. The standard curve of the human operator activity performance is based on the extensive research experience of functional activity of the human operator with the help of photos of the day, his action timing and ergonomic generalizations. Main Results. The apparatus of regular finite Markov chains gave the possibility to evaluate correctly the human operator activity performance in a quasi-static functional environment with the use of the main information component of these chains as a vector of final probabilities. In addition, we managed to build an algorithmic basis for estimating the stationary time (time study for transit of human operator from arbitrary initial functional state into a state corresponding to a vector of final probabilities for a used chain after it reaches the final state based on the analysis of the eigenvalues spectrum of the matrix of transition probabilities for a regular (recurrent finite Markov chain. Practical Relevance. Obtained theoretical results are confirmed by illustrative examples, which

  4. Identifying the role of typhoons as drought busters in South Korea based on hidden Markov chain models

    Science.gov (United States)

    Yoo, Jiyoung; Kwon, Hyun-Han; So, Byung-Jin; Rajagopalan, Balaji; Kim, Tae-Woong

    2015-04-01

    This study proposed a hidden Markov chain model-based drought analysis (HMM-DA) tool to understand the beginning and ending of meteorological drought and to further characterize typhoon-induced drought busters (TDB) by exploring spatiotemporal drought patterns in South Korea. It was found that typhoons have played a dominant role in ending drought events (EDE) during the typhoon season (July-September) over the last four decades (1974-2013). The percentage of EDEs terminated by TDBs was about 43-90% mainly along coastal regions in South Korea. Furthermore, the TDBs, mainly during summer, have a positive role in managing extreme droughts during the subsequent autumn and spring seasons. The HMM-DA models the temporal dependencies between drought states using Markov chain, consequently capturing the dependencies between droughts and typhoons well, thus, enabling a better performance in modeling spatiotemporal drought attributes compared to traditional methods.

  5. MODEL FOR OPTIMAL BLOCK REPLACEMENT DECISION OF AIR CONDITIONERS USING FIRST ORDER MARKOV CHAINS WITH & WITHOUT CONSIDERING INFLATION

    Directory of Open Access Journals (Sweden)

    Y HARI PRASADA REDDY

    2012-05-01

    Full Text Available In this paper, a mathematical model has been developed for group replacement of a block of Air Conditioners using discrete-time First Order Markov Chains. To make the model more realistic, threeintermediate states viz., Minor Repair State, Semi-Major Repair State and Major Repair States have been introduced between Functioning State & Complete Failure States of the system. The Transition Probabilities for future periods for First Order Markov Chain (FOMC are estimated by Spectral Decomposition Method. Using these probabilities, the number of systems in each state and accordingly the corresponding average maintenance cost is computed. The forecasted inflation for Air Conditioners and the real value of money using Fisherman’s relation are employed to study and develop the real time mathematical model for block replacement decision making.

  6. An open Markov chain scheme model for a credit consumption portfolio fed by ARIMA and SARMA processes

    Science.gov (United States)

    Esquível, Manuel L.; Fernandes, José Moniz; Guerreiro, Gracinda R.

    2016-06-01

    We introduce a schematic formalism for the time evolution of a random population entering some set of classes and such that each member of the population evolves among these classes according to a scheme based on a Markov chain model. We consider that the flow of incoming members is modeled by a time series and we detail the time series structure of the elements in each of the classes. We present a practical application to data from a credit portfolio of a Cape Verdian bank; after modeling the entering population in two different ways - namely as an ARIMA process and as a deterministic sigmoid type trend plus a SARMA process for the residues - we simulate the behavior of the population and compare the results. We get that the second method is more accurate in describing the behavior of the populations when compared to the observed values in a direct simulation of the Markov chain.

  7. On The Transition Probabilities for the Fuzzy States of a Fuzzy Markov Chain

    Directory of Open Access Journals (Sweden)

    J.Earnest Lazarus Piriyakumar

    2015-12-01

    Full Text Available In this paper the theory of fuzzy logic is mixed with the theory of Markov systems and the abstraction of a Markov system with fuzzy states introduced. The notions such as fuzzy transient, fuzzy recurrent etc., were introduced. The results based on these notions are introduced.

  8. A Markov chain Monte Carlo (MCMC) methodology with bootstrap percentile estimates for predicting presidential election results in Ghana

    OpenAIRE

    Nortey, Ezekiel N. N.; Ansah-Narh, Theophilus; Asah-Asante, Richard; Minkah, Richard

    2015-01-01

    Although, there exists numerous literature on the procedure for forecasting or predicting election results, in Ghana only opinion poll strategies have been used. To fill this gap, the paper develops Markov chain models for forecasting the 2016 presidential election results at the Regional, Zonal (i.e. Savannah, Coastal and Forest) and the National levels using past presidential election results of Ghana. The methodology develops a model for prediction of the 2016 presidential election results...

  9. Clarification of basic factorization identity is for the almost semi-continuous latticed Poisson processes on the Markov chain

    Directory of Open Access Journals (Sweden)

    Gerich M. S.

    2012-12-01

    Full Text Available Let ${xi(t, x(t}$ be a homogeneous semi-continuous lattice Poisson process on the Markov chain.The jumps of one sign are geometrically distributed, and jumps of the opposite sign are arbitrary latticed distribution. For a suchprocesses the relations for the components of two-sided matrix factorization are established.This relations define the moment genereting functions for extremumf of the process and their complements.

  10. A Birthday Paradox for Markov chains with an optimal bound for collision in the Pollard Rho algorithm for discrete logarithm

    OpenAIRE

    Kim, Jeong Han; Montenegro, Ravi; Peres, Yuval; Tetali, Prasad

    2010-01-01

    We show a Birthday Paradox for self-intersections of Markov chains with uniform stationary distribution. As an application, we analyze Pollard's Rho algorithm for finding the discrete logarithm in a cyclic group $G$ and find that if the partition in the algorithm is given by a random oracle, then with high probability a collision occurs in $\\Theta(\\sqrt{|G|})$ steps. Moreover, for the parallelized distinguished points algorithm on $J$ processors we find that $\\Theta(\\sqrt{|G|}/J)$ steps suffi...

  11. Test of The Weak Form Efficient Market Hypothesis for The Istanbul Stock Exchange By Markov Chains Methodology

    OpenAIRE

    KILIÇ, Öğr. Gör. Dr. Süleyman Bilgin

    2013-01-01

    In this study Markov chain methodology is used to test whether or not the daily returns of the Istanbul Stock Exchange ISE 100 index follows a martingale random walk process If the Weak Form Efficient Market Hypothesis EMH holds in any stock market stocks prices or returns follow a random walk process The random walk theory asserts that price movements will not follow any patterns or trends and that past price movements cannot be used to predict future price movements hence technic...

  12. Development of a Compound Distribution Markov Chain Model for Stochastic Generation of Rainfall with Long Term Persistence

    Science.gov (United States)

    Kamal Chowdhury, AFM; Lockart, Natalie; Willgoose, Garry; Kuczera, George

    2015-04-01

    One of the overriding issues in the rainfall simulation is the underestimation of observed rainfall variability in longer timescales (e.g. monthly, annual and multi-year), which usually results into under-estimation of reservoir reliability in urban water planning. This study has developed a Compound Distribution Markov Chain (CDMC) model for stochastic generation of daily rainfall. We used two parameters of Markov Chain process (transition probabilities of wet-to-wet and dry-to-dry days) for simulating rainfall occurrence and two parameters of gamma distribution (calculated from mean and standard deviation of wet-day rainfall) for simulating wet-day rainfall amounts. While two models with deterministic parameters underestimated long term variability, our investigation found that the long term variability of rainfall in the model is predominantly governed by the long term variability of gamma parameters, rather than the variability of Markov Chain parameters. Therefore, in the third approach, we developed the CDMC model with deterministic parameters of Markov Chain process, but stochastic parameters of gamma distribution by sampling the mean and standard deviation of wet-day rainfall from their log-normal and bivariate-normal distribution. We have found that the CDMC is able to replicate both short term and long term rainfall variability, when we calibrated the model at two sites in east coast of Australia using three types of daily rainfall data - (1) dynamically downscaled, 10 km resolution gridded data produced by NSW/ACT Regional Climate Modelling project, (2) 5 km resolution gridded data by Australian Water Availability Project and (3) point scale raingauge stations data by Bureau of Meteorology, Australia. We also examined the spatial variability of parameters and their link with local orography at our field site. The suitability of the model in runoff generation and urban reservoir-water simulation will be discussed.

  13. Additive N-step Markov chains as prototype model of symbolic stochastic dynamical systems with long-range correlations

    Energy Technology Data Exchange (ETDEWEB)

    Mayzelis, Z.A. [Department of Physics, Kharkov National University, 4 Svoboda Sq., Kharkov 61077 (Ukraine); Apostolov, S.S. [Department of Physics, Kharkov National University, 4 Svoboda Sq., Kharkov 61077 (Ukraine); Melnyk, S.S. [A. Ya. Usikov Institute for Radiophysics and Electronics, Ukrainian Academy of Science, 12 Proskura Street, 61085 Kharkov (Ukraine); Usatenko, O.V. [A. Ya. Usikov Institute for Radiophysics and Electronics, Ukrainian Academy of Science, 12 Proskura Street, 61085 Kharkov (Ukraine)]. E-mail: usatenko@ire.kharkov.ua; Yampol' skii, V.A. [A. Ya. Usikov Institute for Radiophysics and Electronics, Ukrainian Academy of Science, 12 Proskura Street, 61085 Kharkov (Ukraine)

    2007-10-15

    A theory of symbolic dynamic systems with long-range correlations based on the consideration of the binary N-step Markov chains developed earlier in Phys Rev Lett 2003;90:110601 is generalized to the biased case (non-equal numbers of zeros and unities in the chain). In the model, the conditional probability that the ith symbol in the chain equals zero (or unity) is a linear function of the number of unities (zeros) among the preceding N symbols. The correlation and distribution functions as well as the variance of number of symbols in the words of arbitrary length L are obtained analytically and verified by numerical simulations. A self-similarity of the studied stochastic process is revealed and the similarity group transformation of the chain parameters is presented. The diffusion Fokker-Planck equation governing the distribution function of the L-words is explored. If the persistent correlations are not extremely strong, the distribution function is shown to be the Gaussian with the variance being nonlinearly dependent on L. An equation connecting the memory and correlation function of the additive Markov chain is presented. This equation allows reconstructing a memory function using a correlation function of the system. Effectiveness and robustness of the proposed method is demonstrated by simple model examples. Memory functions of concrete coarse-grained literary texts are found and their universal power-law behavior at long distances is revealed.

  14. A Markov Chain Monte Carlo Approach to Estimate AIDS after HIV Infection.

    Directory of Open Access Journals (Sweden)

    Ofosuhene O Apenteng

    Full Text Available The spread of human immunodeficiency virus (HIV infection and the resulting acquired immune deficiency syndrome (AIDS is a major health concern in many parts of the world, and mathematical models are commonly applied to understand the spread of the HIV epidemic. To understand the spread of HIV and AIDS cases and their parameters in a given population, it is necessary to develop a theoretical framework that takes into account realistic factors. The current study used this framework to assess the interaction between individuals who developed AIDS after HIV infection and individuals who did not develop AIDS after HIV infection (pre-AIDS. We first investigated how probabilistic parameters affect the model in terms of the HIV and AIDS population over a period of time. We observed that there is a critical threshold parameter, R0, which determines the behavior of the model. If R0 ≤ 1, there is a unique disease-free equilibrium; if R0 1, the disease-free equilibrium is unstable. We also show how a Markov chain Monte Carlo (MCMC approach could be used as a supplement to forecast the numbers of reported HIV and AIDS cases. An approach using a Monte Carlo analysis is illustrated to understand the impact of model-based predictions in light of uncertain parameters on the spread of HIV. Finally, to examine this framework and demonstrate how it works, a case study was performed of reported HIV and AIDS cases from an annual data set in Malaysia, and then we compared how these approaches complement each other. We conclude that HIV disease in Malaysia shows epidemic behavior, especially in the context of understanding and predicting emerging cases of HIV and AIDS.

  15. A Markov Chain Monte Carlo Approach to Estimate AIDS after HIV Infection.

    Science.gov (United States)

    Apenteng, Ofosuhene O; Ismail, Noor Azina

    2015-01-01

    The spread of human immunodeficiency virus (HIV) infection and the resulting acquired immune deficiency syndrome (AIDS) is a major health concern in many parts of the world, and mathematical models are commonly applied to understand the spread of the HIV epidemic. To understand the spread of HIV and AIDS cases and their parameters in a given population, it is necessary to develop a theoretical framework that takes into account realistic factors. The current study used this framework to assess the interaction between individuals who developed AIDS after HIV infection and individuals who did not develop AIDS after HIV infection (pre-AIDS). We first investigated how probabilistic parameters affect the model in terms of the HIV and AIDS population over a period of time. We observed that there is a critical threshold parameter, R0, which determines the behavior of the model. If R0 ≤ 1, there is a unique disease-free equilibrium; if R0 1, the disease-free equilibrium is unstable. We also show how a Markov chain Monte Carlo (MCMC) approach could be used as a supplement to forecast the numbers of reported HIV and AIDS cases. An approach using a Monte Carlo analysis is illustrated to understand the impact of model-based predictions in light of uncertain parameters on the spread of HIV. Finally, to examine this framework and demonstrate how it works, a case study was performed of reported HIV and AIDS cases from an annual data set in Malaysia, and then we compared how these approaches complement each other. We conclude that HIV disease in Malaysia shows epidemic behavior, especially in the context of understanding and predicting emerging cases of HIV and AIDS. PMID:26147199

  16. An informational transition in conditioned Markov chains: Applied to genetics and evolution.

    Science.gov (United States)

    Zhao, Lei; Lascoux, Martin; Waxman, David

    2016-08-01

    In this work we assume that we have some knowledge about the state of a population at two known times, when the dynamics is governed by a Markov chain such as a Wright-Fisher model. Such knowledge could be obtained, for example, from observations made on ancient and contemporary DNA, or during laboratory experiments involving long term evolution. A natural assumption is that the behaviour of the population, between observations, is related to (or constrained by) what was actually observed. The present work shows that this assumption has limited validity. When the time interval between observations is larger than a characteristic value, which is a property of the population under consideration, there is a range of intermediate times where the behaviour of the population has reduced or no dependence on what was observed and an equilibrium-like distribution applies. Thus, for example, if the frequency of an allele is observed at two different times, then for a large enough time interval between observations, the population has reduced or no dependence on the two observed frequencies for a range of intermediate times. Given observations of a population at two times, we provide a general theoretical analysis of the behaviour of the population at all intermediate times, and determine an expression for the characteristic time interval, beyond which the observations do not constrain the population's behaviour over a range of intermediate times. The findings of this work relate to what can be meaningfully inferred about a population at intermediate times, given knowledge of terminal states. PMID:27105672

  17. Multilevel markov chain monte carlo method for high-contrast single-phase flow problems

    KAUST Repository

    Efendiev, Yalchin R.

    2014-12-19

    In this paper we propose a general framework for the uncertainty quantification of quantities of interest for high-contrast single-phase flow problems. It is based on the generalized multiscale finite element method (GMsFEM) and multilevel Monte Carlo (MLMC) methods. The former provides a hierarchy of approximations of different resolution, whereas the latter gives an efficient way to estimate quantities of interest using samples on different levels. The number of basis functions in the online GMsFEM stage can be varied to determine the solution resolution and the computational cost, and to efficiently generate samples at different levels. In particular, it is cheap to generate samples on coarse grids but with low resolution, and it is expensive to generate samples on fine grids with high accuracy. By suitably choosing the number of samples at different levels, one can leverage the expensive computation in larger fine-grid spaces toward smaller coarse-grid spaces, while retaining the accuracy of the final Monte Carlo estimate. Further, we describe a multilevel Markov chain Monte Carlo method, which sequentially screens the proposal with different levels of approximations and reduces the number of evaluations required on fine grids, while combining the samples at different levels to arrive at an accurate estimate. The framework seamlessly integrates the multiscale features of the GMsFEM with the multilevel feature of the MLMC methods following the work in [26], and our numerical experiments illustrate its efficiency and accuracy in comparison with standard Monte Carlo estimates. © Global Science Press Limited 2015.

  18. Input estimation for drug discovery using optimal control and Markov chain Monte Carlo approaches.

    Science.gov (United States)

    Trägårdh, Magnus; Chappell, Michael J; Ahnmark, Andrea; Lindén, Daniel; Evans, Neil D; Gennemark, Peter

    2016-04-01

    Input estimation is employed in cases where it is desirable to recover the form of an input function which cannot be directly observed and for which there is no model for the generating process. In pharmacokinetic and pharmacodynamic modelling, input estimation in linear systems (deconvolution) is well established, while the nonlinear case is largely unexplored. In this paper, a rigorous definition of the input-estimation problem is given, and the choices involved in terms of modelling assumptions and estimation algorithms are discussed. In particular, the paper covers Maximum a Posteriori estimates using techniques from optimal control theory, and full Bayesian estimation using Markov Chain Monte Carlo (MCMC) approaches. These techniques are implemented using the optimisation software CasADi, and applied to two example problems: one where the oral absorption rate and bioavailability of the drug eflornithine are estimated using pharmacokinetic data from rats, and one where energy intake is estimated from body-mass measurements of mice exposed to monoclonal antibodies targeting the fibroblast growth factor receptor (FGFR) 1c. The results from the analysis are used to highlight the strengths and weaknesses of the methods used when applied to sparsely sampled data. The presented methods for optimal control are fast and robust, and can be recommended for use in drug discovery. The MCMC-based methods can have long running times and require more expertise from the user. The rigorous definition together with the illustrative examples and suggestions for software serve as a highly promising starting point for application of input-estimation methods to problems in drug discovery. PMID:26932466

  19. Large Deviations for Stationary Probabilities of a Family of Continuous Time Markov Chains via Aubry-Mather Theory

    Science.gov (United States)

    Lopes, Artur O.; Neumann, Adriana

    2015-05-01

    In the present paper, we consider a family of continuous time symmetric random walks indexed by , . For each the matching random walk take values in the finite set of states ; notice that is a subset of , where is the unitary circle. The infinitesimal generator of such chain is denoted by . The stationary probability for such process converges to the uniform distribution on the circle, when . Here we want to study other natural measures, obtained via a limit on , that are concentrated on some points of . We will disturb this process by a potential and study for each the perturbed stationary measures of this new process when . We disturb the system considering a fixed potential and we will denote by the restriction of to . Then, we define a non-stochastic semigroup generated by the matrix , where is the infinifesimal generator of . From the continuous time Perron's Theorem one can normalized such semigroup, and, then we get another stochastic semigroup which generates a continuous time Markov Chain taking values on . This new chain is called the continuous time Gibbs state associated to the potential , see (Lopes et al. in J Stat Phys 152:894-933, 2013). The stationary probability vector for such Markov Chain is denoted by . We assume that the maximum of is attained in a unique point of , and from this will follow that . Thus, here, our main goal is to analyze the large deviation principle for the family , when . The deviation function , which is defined on , will be obtained from a procedure based on fixed points of the Lax-Oleinik operator and Aubry-Mather theory. In order to obtain the associated Lax-Oleinik operator we use the Varadhan's Lemma for the process . For a careful analysis of the problem we present full details of the proof of the Large Deviation Principle, in the Skorohod space, for such family of Markov Chains, when . Finally, we compute the entropy of the invariant probabilities on the Skorohod space associated to the Markov Chains we analyze.

  20. A Bayesian approach to the study of white dwarf binaries in LISA data: The application of a reversible jump Markov chain Monte Carlo method

    CERN Document Server

    Stroeer, Alexander

    2009-01-01

    The Laser Interferometer Space Antenna (LISA) defines new demands on data analysis efforts in its all-sky gravitational wave survey, recording simultaneously thousands of galactic compact object binary foreground sources and tens to hundreds of background sources like binary black hole mergers and extreme mass ratio inspirals. We approach this problem with an adaptive and fully automatic Reversible Jump Markov Chain Monte Carlo sampler, able to sample from the joint posterior density function (as established by Bayes theorem) for a given mixture of signals "out of the box'', handling the total number of signals as an additional unknown parameter beside the unknown parameters of each individual source and the noise floor. We show in examples from the LISA Mock Data Challenge implementing the full response of LISA in its TDI description that this sampler is able to extract monochromatic Double White Dwarf signals out of colored instrumental noise and additional foreground and background noise successfully in a ...

  1. Finding starting points for Markov chain Monte Carlo analysis of genetic data from large and complex pedigrees.

    Science.gov (United States)

    Luo, Yuqun; Lin, Shili

    2003-07-01

    Genetic data from founder populations are advantageous for studies of complex traits that are often plagued by the problem of genetic heterogeneity. However, the desire to analyze large and complex pedigrees that often arise from such populations, coupled with the need to handle many linked and highly polymorphic loci simultaneously, poses challenges to current standard approaches. A viable alternative to solving such problems is via Markov chain Monte Carlo (MCMC) procedures, where a Markov chain, defined on the state space of a latent variable (e.g., genotypic configuration or inheritance vector), is constructed. However, finding starting points for the Markov chains is a difficult problem when the pedigree is not single-locus peelable; methods proposed in the literature have not yielded completely satisfactory solutions. We propose a generalization of the heated Gibbs sampler with relaxed penetrances (HGRP) of Lin et al., ([1993] IMA J. Math. Appl. Med. Biol. 10:1-17) to search for starting points. HGRP guarantees that a starting point will be found if there is no error in the data, but the chain usually needs to be run for a long time if the pedigree is extremely large and complex. By introducing a forcing step, the current algorithm substantially reduces the state space, and hence effectively speeds up the process of finding a starting point. Our algorithm also has a built-in preprocessing procedure for Mendelian error detection. The algorithm has been applied to both simulated and real data on two large and complex Hutterite pedigrees under many settings, and good results are obtained. The algorithm has been implemented in a user-friendly package called START. PMID:12813723

  2. Comparison of Bootstrapping and Markov Chain Monte Carlo for Copula Analysis of Hydrological Droughts

    Science.gov (United States)

    Yang, P.; Ng, T. L.; Yang, W.

    2015-12-01

    Effective water resources management depends on the reliable estimation of the uncertainty of drought events. Confidence intervals (CIs) are commonly applied to quantify this uncertainty. A CI seeks to be at the minimal length necessary to cover the true value of the estimated variable with the desired probability. In drought analysis where two or more variables (e.g., duration and severity) are often used to describe a drought, copulas have been found suitable for representing the joint probability behavior of these variables. However, the comprehensive assessment of the parameter uncertainties of copulas of droughts has been largely ignored, and the few studies that have recognized this issue have not explicitly compared the various methods to produce the best CIs. Thus, the objective of this study to compare the CIs generated using two widely applied uncertainty estimation methods, bootstrapping and Markov Chain Monte Carlo (MCMC). To achieve this objective, (1) the marginal distributions lognormal, Gamma, and Generalized Extreme Value, and the copula functions Clayton, Frank, and Plackett are selected to construct joint probability functions of two drought related variables. (2) The resulting joint functions are then fitted to 200 sets of simulated realizations of drought events with known distribution and extreme parameters and (3) from there, using bootstrapping and MCMC, CIs of the parameters are generated and compared. The effect of an informative prior on the CIs generated by MCMC is also evaluated. CIs are produced for different sample sizes (50, 100, and 200) of the simulated drought events for fitting the joint probability functions. Preliminary results assuming lognormal marginal distributions and the Clayton copula function suggest that for cases with small or medium sample sizes (~50-100), MCMC to be superior method if an informative prior exists. Where an informative prior is unavailable, for small sample sizes (~50), both bootstrapping and MCMC

  3. Stochastic Monte-Carlo Markov Chain Inversions on Models Regionalized Using Receiver Functions

    Science.gov (United States)

    Larmat, C. S.; Maceira, M.; Kato, Y.; Bodin, T.; Calo, M.; Romanowicz, B. A.; Chai, C.; Ammon, C. J.

    2014-12-01

    There is currently a strong interest in stochastic approaches to seismic modeling - versus deterministic methods such as gradient methods - due to the ability of these methods to better deal with highly non-linear problems. Another advantage of stochastic methods is that they allow the estimation of the a posteriori probability distribution of the derived parameters, meaning the envisioned Bayesian inversion of Tarantola allowing the quantification of the solution error. The cost to pay of stochastic methods is that they require testing thousands of variations of each unknown parameter and their associated weights to ensure reliable probabilistic inferences. Even with the best High-Performance Computing resources available, 3D stochastic full waveform modeling at the regional scale still remains out-of-reach. We are exploring regionalization as one way to reduce the dimension of the parameter space, allowing the identification of areas in the models that can be treated as one block in a subsequent stochastic inversion. Regionalization is classically performed through the identification of tectonic or structural elements. Lekic & Romanowicz (2011) proposed a new approach with a cluster analysis of the tomographic velocity models instead. Here we present the results of a clustering analysis on the P-wave receiver-functions used in the subsequent inversion. Different clustering algorithms and quality of clustering are tested for different datasets of North America and China. Preliminary results with the kmean clustering algorithm show that an interpolated receiver function wavefield (Chai et al., GRL, in review) improve the agreement with the geological and tectonic regions of North America compared to the traditional approach of stacked receiver functions. After regionalization, 1D profile for each region is stochastically inferred using a parallelized code based on Monte-Carlo Markov Chains (MCMC), and modeling surfacewave-dispersion and receiver

  4. Unsupervised polarimetric synthetic aperture radar image classification based on sketch map and adaptive Markov random field

    Science.gov (United States)

    Shi, Junfei; Li, Lingling; Liu, Fang; Jiao, Licheng; Liu, Hongying; Yang, Shuyuan; Liu, Lu; Hao, Hongxia

    2016-04-01

    Markov random field (MRF) model is an effective tool for polarimetric synthetic aperture radar (PolSAR) image classification. However, due to the lack of suitable contextual information in conventional MRF methods, there is usually a contradiction between edge preservation and region homogeneity in the classification result. To preserve edge details and obtain homogeneous regions simultaneously, an adaptive MRF framework is proposed based on a polarimetric sketch map. The polarimetric sketch map can provide the edge positions and edge directions in detail, which can guide the selection of neighborhood structures. Specifically, the polarimetric sketch map is extracted to partition a PolSAR image into structural and nonstructural parts, and then adaptive neighborhoods are learned for two parts. For structural areas, geometric weighted neighborhood structures are constructed to preserve image details. For nonstructural areas, the maximum homogeneous regions are obtained to improve the region homogeneity. Experiments are taken on both the simulated and real PolSAR data, and the experimental results illustrate that the proposed method can obtain better performance on both region homogeneity and edge preservation than the state-of-the-art methods.

  5. Markov Chain Modelling Analysis of HIV/AIDS Progression: A Race-based Forecast in the United States

    OpenAIRE

    Lee, S; Ko, J.; Xi Tan; Isha Patel; Balkrishnan, R.; J Chang

    2014-01-01

    HIV/AIDS has reached a pandemic level across the world with more than 33 million people who are living with HIV. In the United States, more than half a million people have been victims of AIDS. This study investigates the most vulnerable racial minority population (the African Americans) in the United States and the second least affected (the Caucasians) in order to predict the trends of the epidemic. A Markov chain analysis was used to model the progression of the disease among vulnerable pe...

  6. A Markov Chain Monte Carlo Algorithm for Infrasound Atmospheric Sounding: Application to the Humming Roadrunner experiment in New Mexico

    Science.gov (United States)

    Lalande, Jean-Marie; Waxler, Roger; Velea, Doru

    2016-04-01

    As infrasonic waves propagate at long ranges through atmospheric ducts it has been suggested that observations of such waves can be used as a remote sensing techniques in order to update properties such as temperature and wind speed. In this study we investigate a new inverse approach based on Markov Chain Monte Carlo methods. This approach as the advantage of searching for the full Probability Density Function in the parameter space at a lower computational cost than extensive parameters search performed by the standard Monte Carlo approach. We apply this inverse methods to observations from the Humming Roadrunner experiment (New Mexico) and discuss implications for atmospheric updates, explosion characterization, localization and yield estimation.

  7. Bayesian Markov chain Monte Carlo Inversion of Time-Lapse Geophysical Data To Characterize the Vadose Zone

    DEFF Research Database (Denmark)

    Scholer, Marie; Irving, James; Zibar, Majken Caroline Looms;

    Geophysical methods have the potential to provide valuable information on hydrological properties in the unsaturated zone. In particular, time-lapse geophysical data, when coupled with a hydrological model and inverted stochastically, may allow for the effective estimation of subsurface hydraulic...... parameters and their corresponding uncertainties. In this study, we use a Bayesian Markov-chain-Monte-Carlo (MCMC) inversion approach to investigate how much information regarding vadose zone hydraulic properties can be retrieved from time-lapse crosshole GPR data collected at the Arrenaes field site...

  8. A new Markov Binomial distribution.

    OpenAIRE

    Omey, Edward; Minkova, Leda D.

    2011-01-01

    In this paper, we introduce a two state homogeneous Markov chain and define a geometric distribution related to this Markov chain. We define also the negative binomial distribution similar to the classical case and call it NB related to interrupted Markov chain. The new binomial distribution is related to the interrupted Markov chain. Some characterization properties of the Geometric distributions are given. Recursion formulas and probability mass functions for the NB distribution and the new...

  9. Predicting the Trend of Land Use Changes Using Artificial Neural Network and Markov Chain Model (Case Study: Kermanshah City

    Directory of Open Access Journals (Sweden)

    Behzad Saeedi Razavi

    2014-04-01

    Full Text Available Nowadays, cities are expanding and developing with a rapid growth, so that the urban development process is currently one of the most important issues facing researchers in urban issues. In addition to the growth of the cities, how land use changes in macro level is also considered. Studying the changes and degradation of the resources in the past few years, as well as feasibility study and predicting these changes in the future years may play a significant role in planning and optimal use of resources and harnessing the non-normative changes in the future. There are diverse approaches for modeling the land use and cover changes among which may point to the Markov chain model. In this study, the changes in land use and land cover in Kermanshah City, Iran during 19 years has been studied using multi-temporal Landsat satellite images in 1987, 2000 and 2006, side information and Markov Chain Model. Results shows the decreasing trend in range land, forest, garden and green space area and in the other hand, an increased in residential land, agriculture and water suggesting the general trend of degradation in the study area through the growth in the residential land and agriculture rather than other land uses. Finally, the state of land use classes of next 19 years (2025 was anticipated using Markov Model. Results obtained from changes prediction matrix based on the maps of years 1987 and 2006 it is likely that 82% of residential land, 58.51% of agriculture, 34.47% of water, 8.94% of green space, 30.78% of gardens, 23.93% of waste land and 16.76% of range lands will remain unchanged from 2006 to 2025, among which residential lands and green space have the most and the least sustainability, respectively.

  10. Estimating Lower Bound and Upper Bound of a Markov chain over a noisy communication channel with Poisson distribution

    Directory of Open Access Journals (Sweden)

    Vinay Mahajan

    2012-06-01

    Full Text Available Under the assumption that the encoders’ observations are conditionally independent Markov chains given an unobserved time-invariant random variable, results on the structure of optimal real-time encoding and decoding functions are obtained. The problem with noiseless channels and perfect memory at the receiver is then considered. A new methodology to find the structure of optimal real-time encoders is employed. A sufficient statistic with a time-invariant domain is found for this problem. This methodology exploits the presence of common information between the encoders and the receiver when communication is over noiseless channels. In this paper we estimate the lower bond, upper bond and define the encoder. In the previous design approach they follow Markov Chain approach to estimating the upper bound and define the encoder. In this dissertation we follow poison distribution to finding the lower bound and upper bound. Poisson can be viewed as an approximation to the binomial distribution. The approximation is good enough to be useful even when the sample size (N is only moderately large (say N > 50 and the probability (p is only relatively small (p < .2 The advantage of the Poisson distribution, of course, is that if N is large you need only know p to determine the approximate distribution of events. With the binomial distribution you also need to know N.

  11. spMC: an R-package for 3D lithological reconstructions based on spatial Markov chains

    Science.gov (United States)

    Sartore, Luca; Fabbri, Paolo; Gaetan, Carlo

    2016-09-01

    The paper presents the spatial Markov Chains (spMC) R-package and a case study of subsoil simulation/prediction located in a plain site of Northeastern Italy. spMC is a quite complete collection of advanced methods for data inspection, besides spMC implements Markov Chain models to estimate experimental transition probabilities of categorical lithological data. Furthermore, simulation methods based on most known prediction methods (as indicator Kriging and CoKriging) were implemented in spMC package. Moreover, other more advanced methods are available for simulations, e.g. path methods and Bayesian procedures, that exploit the maximum entropy. Since the spMC package was developed for intensive geostatistical computations, part of the code is implemented for parallel computations via the OpenMP constructs. A final analysis of this computational efficiency compares the simulation/prediction algorithms by using different numbers of CPU cores, and considering the example data set of the case study included in the package.

  12. Reconstruction for distributed video coding: a Markov random field approach with context-adaptive smoothness prior

    Science.gov (United States)

    Zhang, Yongsheng; Xiong, Hongkai; He, Zhihai; Yu, Songyu

    2010-07-01

    An important issue in Wyner-Ziv video coding is the reconstruction of Wyner-Ziv frames with decoded bit-planes. So far, there are two major approaches: the Maximum a Posteriori (MAP) reconstruction and the Minimum Mean Square Error (MMSE) reconstruction algorithms. However, these approaches do not exploit smoothness constraints in natural images. In this paper, we model a Wyner-Ziv frame by Markov random fields (MRFs), and produce reconstruction results by finding an MAP estimation of the MRF model. In the MRF model, the energy function consists of two terms: a data term, MSE distortion metric in this paper, measuring the statistical correlation between side-information and the source, and a smoothness term enforcing spatial coherence. In order to better describe the spatial constraints of images, we propose a context-adaptive smoothness term by analyzing the correspondence between the output of Slepian-Wolf decoding and successive frames available at decoders. The significance of the smoothness term varies in accordance with the spatial variation within different regions. To some extent, the proposed approach is an extension to the MAP and MMSE approaches by exploiting the intrinsic smoothness characteristic of natural images. Experimental results demonstrate a considerable performance gain compared with the MAP and MMSE approaches.

  13. Averaging over fast variables in the fluid limit for Markov chains: application to the supermarket model with memory

    CERN Document Server

    Luczak, M J

    2010-01-01

    We set out a general procedure which allows the approximation of certain Markov chains by the solutions of differential equations. The chains considered have some components which oscillate rapidly and randomly, while others are close to deterministic. The limiting dynamics are obtained by averaging the drift of the latter with respect to a local equilibrium distribution of the former. Some general estimates are proved under a uniform mixing condition on the fast variable which give explicit error probabilities for the fluid approximation. Mitzenmacher, Prabhakar and Shah \\cite{MPS} introduced a variant with memory of the `join the shortest queue' or `supermarket' model, and obtained a limit picture for the case of a stable system in which the number of queues and the total arrival rate are large. In this limit, the empirical distribution of queue sizes satisfies a differential equation, while the memory of the system oscillates rapidly and randomly. We illustrate our general fluid limit estimate in giving a ...

  14. Theory of two-parameter Markov chain with an application in warranty study

    CERN Document Server

    Calvache, Álvaro

    2012-01-01

    In this paper we present the classical results of Kolmogorov's backward and forward equations to the case of a two-parameter Markov process. These equations relates the infinitesimal transition matrix of the two-parameter Markov process. However, solving these equations is not possible and we require a numerical procedure. In this paper, we give an alternative method by use of double Laplace transform of the transition probability matrix and of the infinitesimal transition matrix of the process. An illustrative example is presented for the method proposed. In this example, we consider a two-parameter warranty model, in which a system can be any of these states: working, failure. We calculate the transition density matrix of these states and also the cost of the warranty for the proposed model.

  15. Modelling human control behaviour with a Markov-chain switched bank of control laws

    OpenAIRE

    Murray-Smith, R.

    1998-01-01

    A probabilistic model of human control behaviour is described. It assumes that human behaviour can be represented by switching among a number of relatively simple behaviours. The model structure is closely related to the Hidden Markov Models (HMMs) commonly used for speech recognition. An HMM with context-dependent transition functions switching between linear control laws is identified from experimental data. The applicability of the approach is demonstrated in a pitch control task for a sim...

  16. State Dependence and Wage Dynamics: A Heterogeneous Markov Chain Model for Wage Mobility in Austria

    OpenAIRE

    Weber, Andrea

    2002-01-01

    Abstract: The behaviour of individual movements in the wage distribution over time can be described by a Markov process. To investigate wage mobility in terms of transitions between quintiles in the wage distribution we apply a fixed effects panel estimation method suggested by Honorè and Kyriazidou (2000). This method of mobility measurement is robust to data contamination like all methods that treat fractiles. Moreover it allows for the inclusion of exogenous variables that change over time...

  17. A complete solution to Blackwell's unique ergodicity problem for hidden Markov chains

    CERN Document Server

    Chigansky, Pavel

    2009-01-01

    We develop necessary and sufficient conditions for uniqueness of the invariant measure of the filtering process associated to an ergodic hidden Markov model in a finite or countable state space. These results provide a complete solution to a problem posed by Blackwell (1957), and subsume earlier partial results due to Kaijser, Kochman and Reeds. The proofs of our main results are based on the stability theory of nonlinear filters.

  18. Supply Chain as Complex Adaptive System and Its Modeling

    Institute of Scientific and Technical Information of China (English)

    MingmingWang

    2004-01-01

    Supply chain is a complex, hierarchical, integrated, open and dynamic network.Every node in the network is an independent business unit that unites other organizations to develop its value, the competition and cooperation between these units are basic impetus of the development and evolution of the supply chain system. The characteristics of supply chain as a complex adaptive system and its modeling are discussed in this paper, and use an example demonstrating the feasibility of CAS modeling in supply chain management study.

  19. Additive N-Step Markov Chains as Prototype Model of Symbolic Stochastic Dynamical Systems with Long-Range Correlations

    CERN Document Server

    Mayzelis, Z A; Usatenko, O V; Yampolskii, V A

    2006-01-01

    A theory of symbolic dynamic systems with long-range correlations based on the consideration of the binary N-step Markov chains developed earlier in Phys. Rev. Lett. 90, 110601 (2003) is generalized to the biased case (non equal numbers of zeros and unities in the chain). In the model, the conditional probability that the i-th symbol in the chain equals zero (or unity) is a linear function of the number of unities (zeros) among the preceding N symbols. The correlation and distribution functions as well as the variance of number of symbols in the words of arbitrary length L are obtained analytically and verified by numerical simulations. A self-similarity of the studied stochastic process is revealed and the similarity group transformation of the chain parameters is presented. The diffusion Fokker-Planck equation governing the distribution function of the L-words is explored. If the persistent correlations are not extremely strong, the distribution function is shown to be the Gaussian with the variance being n...

  20. A hybrid Markov chain-von Mises density model for the drug-dosing interval and drug holiday distributions.

    Science.gov (United States)

    Fellows, Kelly; Rodriguez-Cruz, Vivian; Covelli, Jenna; Droopad, Alyssa; Alexander, Sheril; Ramanathan, Murali

    2015-03-01

    Lack of adherence is a frequent cause of hospitalizations, but its effects on dosing patterns have not been extensively investigated. The purpose of this work was to critically evaluate a novel pharmacometric model for deriving the relationships of adherence to dosing patterns and the dosing interval distribution. The hybrid, stochastic model combines a Markov chain process with the von Mises distribution. The model was challenged with electronic medication monitoring data from 207 hypertension patients and against 5-year persistence data. The model estimates distributions of dosing runs, drug holidays, and dosing intervals. Drug holidays, which can vary between individuals with the same adherence, were characterized by the patient cooperativity index parameter. The drug holiday and dosing run distributions deviate markedly from normality. The dosing interval distribution exhibits complex patterns of multimodality and can be long-tailed. Dosing patterns are an important but under recognized covariate for explaining within-individual variance in drug concentrations. PMID:25609224

  1. Dark matter in disk galaxies I: a Markov Chain Monte Carlo method and application to DDO 154

    CERN Document Server

    Hague, Peter R

    2013-01-01

    We present a new method to constrain the dark matter halo density profiles of disk galaxies. Our algorithm employs a Markov Chain Monte Carlo (MCMC) approach to explore the parameter space of a general family of dark matter profiles. We improve upon previous analyses by considering a wider range of halo profiles and by explicitly identifying cases in which the data are insufficient to break the degeneracies between the model parameters. We demonstrate the robustness of our algorithm using artificial data sets and show that reliable estimates of the halo density profile can be obtained from data of comparable quality to those currently available for low surface brightness (LSB) galaxies. We present our results in terms of physical quantities which are constrained by the data, and find that the logarithmic slope of the halo density profile at the radius of the innermost data point of a measured rotation curve can be strongly constrained in LSB galaxies. High surface brightness galaxies require additional inform...

  2. A Markov Chain Approach for Defining the Fundamental Efficiency Limits of Classical and Bifacial Multi-junction Tandem Solar Cells

    CERN Document Server

    Alam, Muhammad A

    2016-01-01

    Bifacial tandem cells promise to reduce three fundamental losses (above-bandgap, below bandgap, and the uncollected light between panels) inherent in classical single junction PV systems. The successive filtering of light through the bandgap cascade, and requirement of current continuity make optimization of tandem cells difficult, accessible only to numerical solution through computer modeling. The challenge is even more complicated for bifacial design. In this paper, we use an elegantly simple Markov chain approach to show that the essential physics of optimization is intuitively obvious, and deeply insightful results can obtained analytically with a few lines of algebra. This powerful approach reproduces, as special cases, all the known results of traditional/bifacial tandem cells, and highlights the asymptotic efficiency gain of these technologies.

  3. Predicting Customer Churn and Retention Rates in Nigeria’s Mobile Telecommunication Industry Using Markov Chain Modelling

    Directory of Open Access Journals (Sweden)

    Adebiyi Sulaimon Olanrewaju

    2015-12-01

    Full Text Available The telecommunication industry is one of the service industries that is most affected by the problem of subscribers’ churn. Although several techniques have been used to predict customer churn in developed countries, many of those studies used secondary data which are not readily available in Nigeria for researchers. This study investigates how Markov chains help in modelling and predicting the customer churn and retention rate in the Nigerian mobile telecommunication industry. The data generated through the survey were input in the Windows-based Quantitative System for Business (WinQSB for analysis. The results reveal that in the study area MTN has the highest retention rate (86.11%, followed by GLO (70.51%, Airtel (67%, and Etisalat (67.5%. This result has implications for telecom firms’ strategies for competitive advantage in particular and survival in general.

  4. Stochastic differential equations and comparison of financial models with levy process using Markov chain Monte Carlo (MCMC simulation

    Directory of Open Access Journals (Sweden)

    Kianoush Fathi Vajargah

    2015-01-01

    Full Text Available An available method of modeling and predicting the economic time series is the use of stochastic differential equations, which are often determined as jump-diffusion stochastic differential equations in financial markets and underlier economic dynamics. Besides the diffusion term that is a geometric Brownian model with Wiener random process, these equations contain a jump term that follows Poisson process and depends on the type of market. This study presented two different models based on a certain class of jump-diffusion stochastic differential equations with random fluctuations: Black- Scholes model and Merton model (1976, including jump-diffusion (JD model, which were compared, and their parameters and hidden variables were evaluated using Markov chain Monte Carlo (MCMC method.

  5. Optimized nested Markov chain Monte Carlo sampling: application to the liquid nitrogen Hugoniot using density functional theory

    Energy Technology Data Exchange (ETDEWEB)

    Shaw, Milton Sam [Los Alamos National Laboratory; Coe, Joshua D [Los Alamos National Laboratory; Sewell, Thomas D [UNIV OF MISSOURI-COLUMBIA

    2009-01-01

    An optimized version of the Nested Markov Chain Monte Carlo sampling method is applied to the calculation of the Hugoniot for liquid nitrogen. The 'full' system of interest is calculated using density functional theory (DFT) with a 6-31 G* basis set for the configurational energies. The 'reference' system is given by a model potential fit to the anisotropic pair interaction of two nitrogen molecules from DFT calculations. The EOS is sampled in the isobaric-isothermal (NPT) ensemble with a trial move constructed from many Monte Carlo steps in the reference system. The trial move is then accepted with a probability chosen to give the full system distribution. The P's and T's of the reference and full systems are chosen separately to optimize the computational time required to produce the full system EOS. The method is numerically very efficient and predicts a Hugoniot in excellent agreement with experimental data.

  6. Application of Stochastic Automata Networks for Creation of Continuous Time Markov Chain Models of Voltage Gating of Gap Junction Channels

    Directory of Open Access Journals (Sweden)

    Mindaugas Snipas

    2015-01-01

    Full Text Available The primary goal of this work was to study advantages of numerical methods used for the creation of continuous time Markov chain models (CTMC of voltage gating of gap junction (GJ channels composed of connexin protein. This task was accomplished by describing gating of GJs using the formalism of the stochastic automata networks (SANs, which allowed for very efficient building and storing of infinitesimal generator of the CTMC that allowed to produce matrices of the models containing a distinct block structure. All of that allowed us to develop efficient numerical methods for a steady-state solution of CTMC models. This allowed us to accelerate CPU time, which is necessary to solve CTMC models, ∼20 times.

  7. Simulation from endpoint-conditioned, continuous-time Markov chains on a finite state space, with applications to molecular evolution

    DEFF Research Database (Denmark)

    Hobolth, Asger; Stone, Eric

    2009-01-01

    Analyses of serially-sampled data often begin with the assumption that the observations represent discrete samples from a latent continuous-time stochastic process. The continuous-time Markov chain (CTMC) is one such generative model whose popularity extends to a variety of disciplines ranging from...... computational finance to human genetics and genomics. A common theme among these diverse applications is the need to simulate sample paths of a CTMC conditional on realized data that is discretely observed. Here we present a general solution to this sampling problem when the CTMC is defined on a discrete...... approaches: (1) modified rejection sampling, (2) direct sampling, and (3) uniformization. We then give analytical results for the complexity and efficiency of each method in terms of the instantaneous transition rate matrix Q of the CTMC, its beginning and ending states, and the length of sampling time T...

  8. Reliability measures of second order semi-Markov chain in state and duration with application to wind energy production

    CERN Document Server

    D'Amico, Guglielmo; Prattico, Flavio

    2012-01-01

    In this paper we consider the problem of wind energy production using a second order semi-Markov chain in state and duration as a model of wind speed. We present the mathematical model, we describe the data and technical characteristics of a commercial wind turbine (Aircon HAWT-10kW). We show how to compute some of the main dependability measures such as reliability, availability and maintainability functions. We compare the results of the model with real energy production obtained from data available in the Lastem station (Italy) and sampled every 10 minutes. The computation of the dependability measures is a crucial point in the planning and development of a wind farm.

  9. An optimal energy management system for islanded Microgrids based on multi-period artificial bee colony combined with Markov Chain

    DEFF Research Database (Denmark)

    Marzband, Mousa; Azarinejadian, Fatemeh; Savaghebi, Mehdi;

    2016-01-01

    of a consumer characteristic in the DR is becoming a very important issue in these systems. The customer information as the registration and participation information of the DR is used to provide additional indexes for evaluating the customer response, such as consumer's information based on the offer priority......The optimal operation programming of electrical systems through the minimization of the production cost and the market clearing price, as well as the better utilization of renewable energy resources, has attracted the attention of many researchers. To reach this aim, energy management systems (EMSs...... neural network combined with a Markov chain (ANN-MC) approach is used to predict nondispatchable power generation and load demand considering uncertainties. Furthermore, other capabilities such as extendibility, reliability, and flexibility are examined about the proposed approach....

  10. Fitting a distribution to censored contamination data using Markov Chain Monte Carlo methods and samples selected with unequal probabilities.

    Science.gov (United States)

    Williams, Michael S; Ebel, Eric D

    2014-11-18

    The fitting of statistical distributions to chemical and microbial contamination data is a common application in risk assessment. These distributions are used to make inferences regarding even the most pedestrian of statistics, such as the population mean. The reason for the heavy reliance on a fitted distribution is the presence of left-, right-, and interval-censored observations in the data sets, with censored observations being the result of nondetects in an assay, the use of screening tests, and other practical limitations. Considerable effort has been expended to develop statistical distributions and fitting techniques for a wide variety of applications. Of the various fitting methods, Markov Chain Monte Carlo methods are common. An underlying assumption for many of the proposed Markov Chain Monte Carlo methods is that the data represent independent and identically distributed (iid) observations from an assumed distribution. This condition is satisfied when samples are collected using a simple random sampling design. Unfortunately, samples of food commodities are generally not collected in accordance with a strict probability design. Nevertheless, pseudosystematic sampling efforts (e.g., collection of a sample hourly or weekly) from a single location in the farm-to-table continuum are reasonable approximations of a simple random sample. The assumption that the data represent an iid sample from a single distribution is more difficult to defend if samples are collected at multiple locations in the farm-to-table continuum or risk-based sampling methods are employed to preferentially select samples that are more likely to be contaminated. This paper develops a weighted bootstrap estimation framework that is appropriate for fitting a distribution to microbiological samples that are collected with unequal probabilities of selection. An example based on microbial data, derived by the Most Probable Number technique, demonstrates the method and highlights the

  11. Estimation of trace gas fluxes with objectively determined basis functions using reversible-jump Markov chain Monte Carlo

    Science.gov (United States)

    Lunt, Mark F.; Rigby, Matt; Ganesan, Anita L.; Manning, Alistair J.

    2016-09-01

    Atmospheric trace gas inversions often attempt to attribute fluxes to a high-dimensional grid using observations. To make this problem computationally feasible, and to reduce the degree of under-determination, some form of dimension reduction is usually performed. Here, we present an objective method for reducing the spatial dimension of the parameter space in atmospheric trace gas inversions. In addition to solving for a set of unknowns that govern emissions of a trace gas, we set out a framework that considers the number of unknowns to itself be an unknown. We rely on the well-established reversible-jump Markov chain Monte Carlo algorithm to use the data to determine the dimension of the parameter space. This framework provides a single-step process that solves for both the resolution of the inversion grid, as well as the magnitude of fluxes from this grid. Therefore, the uncertainty that surrounds the choice of aggregation is accounted for in the posterior parameter distribution. The posterior distribution of this transdimensional Markov chain provides a naturally smoothed solution, formed from an ensemble of coarser partitions of the spatial domain. We describe the form of the reversible-jump algorithm and how it may be applied to trace gas inversions. We build the system into a hierarchical Bayesian framework in which other unknown factors, such as the magnitude of the model uncertainty, can also be explored. A pseudo-data example is used to show the usefulness of this approach when compared to a subjectively chosen partitioning of a spatial domain. An inversion using real data is also shown to illustrate the scales at which the data allow for methane emissions over north-west Europe to be resolved.

  12. Computing the Stackelberg/Nash equilibria using the extraproximal method: Convergence analysis and implementation details for Markov chains games

    Directory of Open Access Journals (Sweden)

    Trejo Kristal K.

    2015-06-01

    Full Text Available In this paper we present the extraproximal method for computing the Stackelberg/Nash equilibria in a class of ergodic controlled finite Markov chains games. We exemplify the original game formulation in terms of coupled nonlinear programming problems implementing the Lagrange principle. In addition, Tikhonov’s regularization method is employed to ensure the convergence of the cost-functions to a Stackelberg/Nash equilibrium point. Then, we transform the problem into a system of equations in the proximal format. We present a two-step iterated procedure for solving the extraproximal method: (a the first step (the extra-proximal step consists of a “prediction” which calculates the preliminary position approximation to the equilibrium point, and (b the second step is designed to find a “basic adjustment” of the previous prediction. The procedure is called the “extraproximal method” because of the use of an extrapolation. Each equation in this system is an optimization problem for which the necessary and efficient condition for a minimum is solved using a quadratic programming method. This solution approach provides a drastically quicker rate of convergence to the equilibrium point. We present the analysis of the convergence as well the rate of convergence of the method, which is one of the main results of this paper. Additionally, the extraproximal method is developed in terms of Markov chains for Stackelberg games. Our goal is to analyze completely a three-player Stackelberg game consisting of a leader and two followers. We provide all the details needed to implement the extraproximal method in an efficient and numerically stable way. For instance, a numerical technique is presented for computing the first step parameter (λ of the extraproximal method. The usefulness of the approach is successfully demonstrated by a numerical example related to a pricing oligopoly model for airlines companies.

  13. Assessment of parameter uncertainty in hydrological model using a Markov-Chain-Monte-Carlo-based multilevel-factorial-analysis method

    Science.gov (United States)

    Zhang, Junlong; Li, Yongping; Huang, Guohe; Chen, Xi; Bao, Anming

    2016-07-01

    Without a realistic assessment of parameter uncertainty, decision makers may encounter difficulties in accurately describing hydrologic processes and assessing relationships between model parameters and watershed characteristics. In this study, a Markov-Chain-Monte-Carlo-based multilevel-factorial-analysis (MCMC-MFA) method is developed, which can not only generate samples of parameters from a well constructed Markov chain and assess parameter uncertainties with straightforward Bayesian inference, but also investigate the individual and interactive effects of multiple parameters on model output through measuring the specific variations of hydrological responses. A case study is conducted for addressing parameter uncertainties in the Kaidu watershed of northwest China. Effects of multiple parameters and their interactions are quantitatively investigated using the MCMC-MFA with a three-level factorial experiment (totally 81 runs). A variance-based sensitivity analysis method is used to validate the results of parameters' effects. Results disclose that (i) soil conservation service runoff curve number for moisture condition II (CN2) and fraction of snow volume corresponding to 50% snow cover (SNO50COV) are the most significant factors to hydrological responses, implying that infiltration-excess overland flow and snow water equivalent represent important water input to the hydrological system of the Kaidu watershed; (ii) saturate hydraulic conductivity (SOL_K) and soil evaporation compensation factor (ESCO) have obvious effects on hydrological responses; this implies that the processes of percolation and evaporation would impact hydrological process in this watershed; (iii) the interactions of ESCO and SNO50COV as well as CN2 and SNO50COV have an obvious effect, implying that snow cover can impact the generation of runoff on land surface and the extraction of soil evaporative demand in lower soil layers. These findings can help enhance the hydrological model

  14. Application of Markov chain model to daily maximum temperature for thermal comfort in Malaysia

    International Nuclear Information System (INIS)

    The Markov chain’s first order principle has been widely used to model various meteorological fields, for prediction purposes. In this study, a 14-year (2000-2013) data of daily maximum temperatures in Bayan Lepas were used. Earlier studies showed that the outdoor thermal comfort range based on physiologically equivalent temperature (PET) index in Malaysia is less than 34°C, thus the data obtained were classified into two state: normal state (within thermal comfort range) and hot state (above thermal comfort range). The long-run results show the probability of daily temperature exceed TCR will be only 2.2%. On the other hand, the probability daily temperature within TCR will be 97.8%

  15. Optimizing the Loads of multi-player online game Servers using Markov Chains

    DEFF Research Database (Denmark)

    Saeed, Aamir; Olsen, Rasmus Løvenstein; Pedersen, Jens Myrup

    2015-01-01

    Multiplayer online games are on the rise with millions of registered player and hundreds of thousands concurrent players. Current state of the art servers achieve scalability by splitting the game world into linked mini worlds that can be hosted on separate servers. One of the problems is that......, players are flocking to one area, resulting in an overloaded server with decreasing Quality of Service(QoS) for the players. A number of approaches were developed to address these issues, by balancing loads between the over-loaded and under-loaded servers. This paper investigates a new dimension...... that need to be considered when developing load balancing algorithm, that is the reliability of the information that is shared. Simulation results show that Markov based prediction of load information performed better from the normal load status information sharing....

  16. Model Markov untuk Pengambilan Keputusan Medis

    OpenAIRE

    Zada, T. Muhammad Shah

    2016-01-01

    Markov model method is a method that has been widely known in stochastic models. In this research, one of Markov models, which is Markov chain, is used for medical decision making, especially hypertension in Indonesian people. Data in this research are total population, hypertension patients, and death rate of Indonesian people. Markov chain analysis is used to inform the probability of hypertension. The result of Markov chain analysis shows that: probability of healthy people who stay health...

  17. MARKOV CHAIN MODEL FOR PROBABILITY OF DRY, WET DAYS AND STATISTICAL ANALISIS OF DAILY RAINFALL IN SOME CLIMATIC ZONE OF IRAN

    Directory of Open Access Journals (Sweden)

    N. SHAHRAKI

    2013-03-01

    Full Text Available Water scarcity is a major problem in arid and semi-arid areas. The scarcity of water is further stressed by the growing demand due to increase in population growth in developing countries. Climate change and its outcomes on precipitation and water resources is the other problem in these areas. Several models are widely used for modeling daily precipitation occurrence. In this study, Markov Chain Model has been extensively used to study spell distribution. For this purpose, a day period was considered as the optimum length of time. Given the assumption that the Markov chain model is the right model for daily precipitation occurrence, the choice of Markov model order was examined on a daily basis for 4 synoptic weather stations with different climates in Iran (Gorgan, Khorram Abad, Zahedan, Tabrizduring 1978-2009. Based on probability rules, events possibility of sequential dry and wet days, these data were analyzed by stochastic process and Markov Chain method. Then probability matrix was calculated by maximum likelihood method. The possibility continuing2-5days of dry and wet days were calculated. The results showed that the probability maximum of consecutive dry period and climatic probability of dry days has occurred in Zahedan. The probability of consecutive dry period has fluctuated from 73.3 to 100 percent. Climatic probability of occurrence of dry days would change in the range of 70.96 to 100 percent with the average probability of about 90.45 percent.

  18. Markov Chain Modelling Analysis of HIV/AIDS Progression: A Race-based Forecast in the United States.

    Science.gov (United States)

    Lee, S; Ko, J; Tan, Xi; Patel, Isha; Balkrishnan, R; Chang, J

    2014-03-01

    HIV/AIDS has reached a pandemic level across the world with more than 33 million people who are living with HIV. In the United States, more than half a million people have been victims of AIDS. This study investigates the most vulnerable racial minority population (the African Americans) in the United States and the second least affected (the Caucasians) in order to predict the trends of the epidemic. A Markov chain analysis was used to model the progression of the disease among vulnerable people, infective people and AIDS cases for the two races separately, based on the 2009 Centers of Disease Control and Prevention HIV/AIDS Surveillance Report. Based on the Markov model, our study predicts that the number of African American people living with AIDS diagnosis and HIV infection and dead due to HIV/AIDS will be 662.2, 1225.3 and 62.9 in 2015 and 794.9, 1566.5 and 79.2 in 2030, respectively. The number of Caucasian people living with AIDS diagnosis and HIV infection and dead due to HIV/AIDS will be 96.4, 160 and 6.5 in 2015 and 118.6, 206.9 and 8.3 in 2030, respectively. The numbers of deaths due to HIV/AIDS are quite stable over the years in both the races. There is an increasing trend in the number of people living with HIV infection and AIDS diagnosis in Caucasians compared with African Americans. The absolute number of Caucasians living with AIDS diagnosis and HIV infection is quite smaller compared with African Americans. The results reveal discrepancy in HIV infection, AIDS diagnosis and deaths due to HIV/AIDS among the African Americans and the Caucasians races. There is a need for interventions focusing on HIV/AIDS prevention and management, optimum resource allocation and development of antiAIDS campaigns to reduce the infection rate. PMID:24843183

  19. Markov chain modelling analysis of HIV/AIDS progression: A race-based forecast in the United States

    Directory of Open Access Journals (Sweden)

    S Lee

    2014-01-01

    Full Text Available HIV/AIDS has reached a pandemic level across the world with more than 33 million people who are living with HIV. In the United States, more than half a million people have been victims of AIDS. This study investigates the most vulnerable racial minority population (the African Americans in the United States and the second least affected (the Caucasians in order to predict the trends of the epidemic. A Markov chain analysis was used to model the progression of the disease among vulnerable people, infective people and AIDS cases for the two races separately, based on the 2009 Centers of Disease Control and Prevention HIV/AIDS Surveillance Report. Based on the Markov model, our study predicts that the number of African American people living with AIDS diagnosis and HIV infection and dead due to HIV/AIDS will be 662.2, 1225.3 and 62.9 in 2015 and 794.9, 1566.5 and 79.2 in 2030, respectively. The number of Caucasian people living with AIDS diagnosis and HIV infection and dead due to HIV/AIDS will be 96.4, 160 and 6.5 in 2015 and 118.6, 206.9 and 8.3 in 2030, respectively. The numbers of deaths due to HIV/AIDS are quite stable over the years in both the races. There is an increasing trend in the number of people living with HIV infection and AIDS diagnosis in Caucasians compared with African Americans. The absolute number of Caucasians living with AIDS diagnosis and HIV infection is quite smaller compared with African Americans. The results reveal discrepancy in HIV infection, AIDS diagnosis and deaths due to HIV/AIDS among the African Americans and the Caucasians races. There is a need for interventions focusing on HIV/AIDS prevention and management, optimum resource allocation and development of antiAIDS campaigns to reduce the infection rate.

  20. Weighted maximum posterior marginals for random fields using an ensemble of conditional densities from multiple Markov chain Monte Carlo simulations.

    Science.gov (United States)

    Monaco, James Peter; Madabhushi, Anant

    2011-07-01

    The ability of classification systems to adjust their performance (sensitivity/specificity) is essential for tasks in which certain errors are more significant than others. For example, mislabeling cancerous lesions as benign is typically more detrimental than mislabeling benign lesions as cancerous. Unfortunately, methods for modifying the performance of Markov random field (MRF) based classifiers are noticeably absent from the literature, and thus most such systems restrict their performance to a single, static operating point (a paired sensitivity/specificity). To address this deficiency we present weighted maximum posterior marginals (WMPM) estimation, an extension of maximum posterior marginals (MPM) estimation. Whereas the MPM cost function penalizes each error equally, the WMPM cost function allows misclassifications associated with certain classes to be weighted more heavily than others. This creates a preference for specific classes, and consequently a means for adjusting classifier performance. Realizing WMPM estimation (like MPM estimation) requires estimates of the posterior marginal distributions. The most prevalent means for estimating these--proposed by Marroquin--utilizes a Markov chain Monte Carlo (MCMC) method. Though Marroquin's method (M-MCMC) yields estimates that are sufficiently accurate for MPM estimation, they are inadequate for WMPM. To more accurately estimate the posterior marginals we present an equally simple, but more effective extension of the MCMC method (E-MCMC). Assuming an identical number of iterations, E-MCMC as compared to M-MCMC yields estimates with higher fidelity, thereby 1) allowing a far greater number and diversity of operating points and 2) improving overall classifier performance. To illustrate the utility of WMPM and compare the efficacies of M-MCMC and E-MCMC, we integrate them into our MRF-based classification system for detecting cancerous glands in (whole-mount or quarter) histological sections of the prostate

  1. The condition of a finite Markov chain and perturbation bounds for the limiting probabilities

    Science.gov (United States)

    Meyer, C. D., Jr.

    1979-01-01

    The inequalities bounding the relative error the norm of w- w squiggly/the norm of w are exhibited by a very simple function of E and A. Let T denote the transition matrix of an ergodic chain, C, and let A = I - T. Let E be a perturbation matrix such that T squiggly = T - E is also the transition matrix of an ergodic chain, C squiggly. Let w and w squiggly denote the limiting probability (row) vectors for C and C squiggly. The inequality is the best one possible. This bound can be significant in the numerical determination of the limiting probabilities for an ergodic chain. In addition to presenting a sharp bound for the norm of w-w squiggly/the norm of w an explicit expression for w squiggly will be derived in which w squiggly is given as a function of E, A, w and some other related terms.

  2. Modelagem da gestão de estoques de peças de reposição através de cadeias de Markov A model for spare parts stock management using Markov chains

    Directory of Open Access Journals (Sweden)

    Antonio Vinicius Pimpão Gomes

    2008-04-01

    Full Text Available Nessa pesquisa é apresentada uma abordagem para gestão de estoques de peças de reposição com base em cadeias de Markov. É feita uma comparação com a simulação convencional, a fim de validar esta abordagem, bem como é apresentada uma heurística para determinação dos parâmetros da política (S, s de gestão de estoques, dado um conjunto de itens de custo (falta, excesso e ressuprimento e de demanda com distribuição Poisson. A análise dos gráficos desses itens de custo em função dos parâmetros da política (S, s fornece os trade-offs básicos para a formulação da heurística.In this study, we propose a model for a management stock system of spare parts using Markov chains. We compare this method with a conventional simulation showing that both methods are equivalent. In addition, we propose heuristics to find the system parameters based on the properties of Markov Chains and graphics related to the costs implied in the stock management of spare parts.

  3. Distribusi Markov-Binomial Negatif

    OpenAIRE

    Widyasari, Rina

    2015-01-01

    The way to find a new distribution of random variables is defining the distribution which associated with Markov chain. In this research, researcher defines all the random variables identically independent distributed negative binomial distribution and form a Markov chain. Suppose that Xn is a sequence of Bernoulli trials that if 1 occurs means ”success” and 0 occurs means ”failure”. Nb(s) defined as random variables sth success in n trials. Each trial form a Markov chain, in n...

  4. Analysis and forecast of employees’ mobility on the labor market in Romania using Markov chains

    OpenAIRE

    Mariana Balan; Carmen Uzlau; Corina Maria Ene

    2013-01-01

    The mobility of labor, defined as responsiveness and adaptation of persons or groups of persons on the challenges of the social and economic environment is therefore a social phenomenon depending on time and space. A high mobility increases opportunities for workers to find a job and employers to find persons with an adequate level of skills, thus boosting employment and economic growth. In recent years, in Romania there has been an accentuation of existing gaps, compared with the European Un...

  5. An analysis of the efficacy of serial screening for familial nasopharyngeal carcinoma based on Markov chain models.

    Science.gov (United States)

    Choi, Cheuk Wai; Lee, Michael C H; Ng, Wai Tong; Law, Lai Yau; Yau, Tsz Kok; Lee, Anne W M

    2011-03-01

    Treatment of nasopharyngeal carcinoma (NPC) can be improved by early detection of the disease as treatment outcome worsens with disease's progression. This can be achieved with a mass screening program using Epstein Barr virus (EBV) serology and nasopharyngoscopy. The efficacy of any screening strategy should be evaluated before putting it into practice. Such evaluation is ideally performed with simulation as time and cost often preclude the evaluation by randomized trial. This study simulated and compared the outcomes of 4 screening strategies over a period of 12 years: (A) Annual screening, (B) biennial screening, (C) triennial screening, and (D) triennial screening for participants tested EBV negative and annual screening once the participants are tested EBV positive. Progression of the disease was divided into 4 phases and calculated by applying Markov chain model. Parameters of the transition matrix and probabilities were estimated using data from previous screening results of 1,072 family members of NPC patients. The early detection rates with strategies A, B, C and D are 88, 79, 71 and 87% respectively. The 5-year overall survival with screening is 10-12% higher than that without and is the highest with strategies A and D. Strategy D, however, requires only 64% screening tests compared with strategy A and has almost identical resultant disease stage distribution to strategy A. We concluded that strategy D offered the highest efficacy for NPC screening of family members of NPC patients among the four strategies studied. PMID:21052850

  6. Computing short-interval transition matrices of a discrete-time Markov chain from partially observed data.

    Science.gov (United States)

    Charitos, Theodore; de Waal, Peter R; van der Gaag, Linda C

    2008-03-15

    Markov chains constitute a common way of modelling the progression of a chronic disease through various severity states. For these models, a transition matrix with the probabilities of moving from one state to another for a specific time interval is usually estimated from cohort data. Quite often, however, the cohort is observed at specific times with intervals that may be greater than the interval of interest. The transition matrix computed then needs to be decomposed in order to estimate the desired interval transition matrix suited to the model. Although simple to implement, this method of matrix decomposition can yet result in an invalid short-interval transition matrix with negative or complex entries. In this paper, we present a method for computing short-interval transition matrices that is based on regularization techniques. Our method operates separately on each row of the invalid short-interval transition matrix aiming to minimize an appropriate distance measure. We test our method on various matrix structures and sizes, and evaluate its performance on a real-life transition model for HIV-infected individuals. PMID:17579926

  7. Study on mapping Quantitative Trait Loci for animal complex binary traits using Bayesian-Markov chain Monte Carlo approach

    Institute of Scientific and Technical Information of China (English)

    LIU; Jianfeng; ZHANG; Yuan; ZHANG; Qin; WANG; Lixian; ZHANG; Jigang

    2006-01-01

    It is a challenging issue to map Quantitative Trait Loci (QTL) underlying complex discrete traits, which usually show discontinuous distribution and less information, using conventional statistical methods. Bayesian-Markov chain Monte Carlo (Bayesian-MCMC) approach is the key procedure in mapping QTL for complex binary traits, which provides a complete posterior distribution for QTL parameters using all prior information. As a consequence, Bayesian estimates of all interested variables can be obtained straightforwardly basing on their posterior samples simulated by the MCMC algorithm. In our study, utilities of Bayesian-MCMC are demonstrated using simulated several animal outbred full-sib families with different family structures for a complex binary trait underlied by both a QTL and polygene. Under the Identity-by-Descent-Based variance component random model, three samplers basing on MCMC, including Gibbs sampling, Metropolis algorithm and reversible jump MCMC, were implemented to generate the joint posterior distribution of all unknowns so that the QTL parameters were obtained by Bayesian statistical inferring. The results showed that Bayesian-MCMC approach could work well and robust under different family structures and QTL effects. As family size increases and the number of family decreases, the accuracy of the parameter estimates will be improved. When the true QTL has a small effect, using outbred population experiment design with large family size is the optimal mapping strategy.

  8. Bayesian parameter inference by Markov chain Monte Carlo with hybrid fitness measures: theory and test in apoptosis signal transduction network.

    Science.gov (United States)

    Murakami, Yohei; Takada, Shoji

    2013-01-01

    When model parameters in systems biology are not available from experiments, they need to be inferred so that the resulting simulation reproduces the experimentally known phenomena. For the purpose, Bayesian statistics with Markov chain Monte Carlo (MCMC) is a useful method. Conventional MCMC needs likelihood to evaluate a posterior distribution of acceptable parameters, while the approximate Bayesian computation (ABC) MCMC evaluates posterior distribution with use of qualitative fitness measure. However, none of these algorithms can deal with mixture of quantitative, i.e., likelihood, and qualitative fitness measures simultaneously. Here, to deal with this mixture, we formulated Bayesian formula for hybrid fitness measures (HFM). Then we implemented it to MCMC (MCMC-HFM). We tested MCMC-HFM first for a kinetic toy model with a positive feedback. Inferring kinetic parameters mainly related to the positive feedback, we found that MCMC-HFM reliably infer them using both qualitative and quantitative fitness measures. Then, we applied the MCMC-HFM to an apoptosis signal transduction network previously proposed. For kinetic parameters related to implicit positive feedbacks, which are important for bistability and irreversibility of the output, the MCMC-HFM reliably inferred these kinetic parameters. In particular, some kinetic parameters that have experimental estimates were inferred without using these data and the results were consistent with experiments. Moreover, for some parameters, the mixed use of quantitative and qualitative fitness measures narrowed down the acceptable range of parameters.

  9. Bayesian uncertainty quantification for flows in heterogeneous porous media using reversible jump Markov chain Monte Carlo methods

    KAUST Repository

    Mondal, A.

    2010-03-01

    In this paper, we study the uncertainty quantification in inverse problems for flows in heterogeneous porous media. Reversible jump Markov chain Monte Carlo algorithms (MCMC) are used for hierarchical modeling of channelized permeability fields. Within each channel, the permeability is assumed to have a lognormal distribution. Uncertainty quantification in history matching is carried out hierarchically by constructing geologic facies boundaries as well as permeability fields within each facies using dynamic data such as production data. The search with Metropolis-Hastings algorithm results in very low acceptance rate, and consequently, the computations are CPU demanding. To speed-up the computations, we use a two-stage MCMC that utilizes upscaled models to screen the proposals. In our numerical results, we assume that the channels intersect the wells and the intersection locations are known. Our results show that the proposed algorithms are capable of capturing the channel boundaries and describe the permeability variations within the channels using dynamic production history at the wells. © 2009 Elsevier Ltd. All rights reserved.

  10. Holographic dark energy in a universe with spatial curvature and massive neutrinos: a full Markov Chain Monte Carlo exploration

    International Nuclear Information System (INIS)

    In this paper, we report the results of constraining the holographic dark energy model with spatial curvature and massive neutrinos, based on a Markov Chain Monte Carlo global fit technique. The cosmic observational data include the full WMAP 7-yr temperature and polarization data, the type Ia supernova data from Union2.1 sample, the baryon acoustic oscillation data from SDSS DR7 and WiggleZ Dark Energy Survey, and the latest measurements of H0 from HST. To deal with the perturbations of dark energy, we adopt the parameterized post-Friedmann method. We find that, for the simplest holographic dark energy model without spatial curvature and massive neutrinos, the phenomenological parameter c k0 are still in order of 10−2; for the model with massive neutrinos but without spatial curvature, the 2σ upper bound of the total mass of neutrinos is Σmν ν by more than 2 times. In addition, we demonstrate that, making use of the full WMAP data can give better constraints on the holographic dark energy model, compared with the case using the WMAP ''distance priors''

  11. A trans-dimensional Bayesian Markov chain Monte Carlo algorithm for model assessment using frequency-domain electromagnetic data

    Science.gov (United States)

    Minsley, B.J.

    2011-01-01

    A meaningful interpretation of geophysical measurements requires an assessment of the space of models that are consistent with the data, rather than just a single, 'best' model which does not convey information about parameter uncertainty. For this purpose, a trans-dimensional Bayesian Markov chain Monte Carlo (MCMC) algorithm is developed for assessing frequency-domain electromagnetic (FDEM) data acquired from airborne or ground-based systems. By sampling the distribution of models that are consistent with measured data and any prior knowledge, valuable inferences can be made about parameter values such as the likely depth to an interface, the distribution of possible resistivity values as a function of depth and non-unique relationships between parameters. The trans-dimensional aspect of the algorithm allows the number of layers to be a free parameter that is controlled by the data, where models with fewer layers are inherently favoured, which provides a natural measure of parsimony and a significant degree of flexibility in parametrization. The MCMC algorithm is used with synthetic examples to illustrate how the distribution of acceptable models is affected by the choice of prior information, the system geometry and configuration and the uncertainty in the measured system elevation. An airborne FDEM data set that was acquired for the purpose of hydrogeological characterization is also studied. The results compare favourably with traditional least-squares analysis, borehole resistivity and lithology logs from the site, and also provide new information about parameter uncertainty necessary for model assessment. ?? 2011. Geophysical Journal International ?? 2011 RAS.

  12. Markov Chain Monte Carlo inversion of temperature and salinity structure of an internal solitary wave packet from marine seismic data

    Science.gov (United States)

    Tang, Qunshu; Hobbs, Richard; Zheng, Chan; Biescas, Berta; Caiado, Camila

    2016-06-01

    Marine seismic reflection technique is used to observe the strong ocean dynamic process of nonlinear internal solitary waves (ISWs or solitons) in the near-surface water. Analysis of ISWs is problematical because of their transient nature and limitations of classical physical oceanography methods. This work explores a Markov Chain Monte Carlo (MCMC) approach to recover the temperature and salinity of ISW field using the seismic reflectivity data and in situ hydrographic data. The MCMC approach is designed to directly sample the posterior probability distributions of temperature and salinity which are the solutions of the system under investigation. The principle improvement is the capability of incorporating uncertainties in observations and prior models which then provide quantified uncertainties in the output model parameters. We tested the MCMC approach on two acoustic reflectivity data sets one synthesized from a CTD cast and the other derived from multichannel seismic reflections. This method finds the solutions faithfully within the significantly narrowed confidence intervals from the provided priors. Combined with a low frequency initial model interpreted from seismic horizons of ISWs, the MCMC method is used to compute the finescale temperature, salinity, acoustic velocity, and density of ISW field. The statistically derived results are equivalent to the conventional linearized inversion method. However, the former provides us the quantified uncertainties of the temperature and salinity along the whole section whilst the latter does not. These results are the first time ISWs have been mapped with sufficient detail for further analysis of their dynamic properties.

  13. Neutrino masses and cosmological parameters from a Euclid-like survey: Markov Chain Monte Carlo forecasts including theoretical errors

    CERN Document Server

    Audren, Benjamin; Bird, Simeon; Haehnelt, Martin G; Viel, Matteo

    2013-01-01

    We present forecasts for the accuracy of determining the parameters of a minimal cosmological model and the total neutrino mass based on combined mock data for a future Euclid-like galaxy survey and Planck. We consider two different galaxy surveys: a spectroscopic redshift survey and a cosmic shear survey. We make use of the Monte Carlo Markov Chains (MCMC) technique and assume two sets of theoretical errors. The first error is meant to account for uncertainties in the modelling of the effect of neutrinos on the non-linear galaxy power spectrum and we assume this error to be fully correlated in Fourier space. The second error is meant to parametrize the overall residual uncertainties in modelling the non-linear galaxy power spectrum at small scales, and is conservatively assumed to be uncorrelated and to increase with the ratio of a given scale to the scale of non-linearity. It hence increases with wavenumber and decreases with redshift. With these two assumptions for the errors and assuming further conservat...

  14. Detection and prediction of land cover changes using Markov chain model in semi-arid rangeland in western Iran.

    Science.gov (United States)

    Fathizad, Hassan; Rostami, Noredin; Faramarzi, Marzban

    2015-10-01

    The study of changes and destruction rate in the previous years as well as the possibility of prediction of these changes in the following years has a key role in optimal planning, controlling, and restricting non-normative changes in the future. This research was approached to detecting land use/cover changes (1985-2007) and to forecast the changes in the future (2021) use of multitemporal satellite imagery in semi-arid area in western Iran. A supervised classification of multilayer perceptron (MLP) was applied for detecting land use changes. The study area was classified into five classes, those of forest, rangeland, agriculture, residential, and barren lands. The change detection analysis indicated a decreasing trend in forest cover by 30.42%, while other land uses were increased during 1985 to 2007. The land use changes were predicted using Markov chain model for 2021. The model was calibrated by comparing the simulated map with the real detected classes of land cover in 2007. Then, for further model processing, an acceptable accuracy at 83% was achieved between them. Finally, land use changes were predicted by using transition matrix derived from calibrated approach. The findings of this study demonstrate a rapid change in land use/cover for the coming years. Transforming the forest into other land uses especially rangeland and cropland is the main land cover changes in the future. Therefore, the planning of protection and restoration of forest cover should be an essential program for decision-makers in the study area. PMID:26373304

  15. The properties of tests for spatial effects in discrete Markov chain models of regional income distribution dynamics

    Science.gov (United States)

    Rey, Sergio J.; Kang, Wei; Wolf, Levi

    2016-10-01

    Discrete Markov chain models (DMCs) have been widely applied to the study of regional income distribution dynamics and convergence. This popularity reflects the rich body of DMC theory on the one hand and the ability of this framework to provide insights on the internal and external properties of regional income distribution dynamics on the other. In this paper we examine the properties of tests for spatial effects in DMC models of regional distribution dynamics. We do so through a series of Monte Carlo simulations designed to examine the size, power and robustness of tests for spatial heterogeneity and spatial dependence in transitional dynamics. This requires that we specify a data generating process for not only the null, but also alternatives when spatial heterogeneity or spatial dependence is present in the transitional dynamics. We are not aware of any work which has examined these types of data generating processes in the spatial distribution dynamics literature. Results indicate that tests for spatial heterogeneity and spatial dependence display good power for the presence of spatial effects. However, tests for spatial heterogeneity are not robust to the presence of strong spatial dependence, while tests for spatial dependence are sensitive to the spatial configuration of heterogeneity. When the spatial configuration can be considered random, dependence tests are robust to the dynamic spatial heterogeneity, but not so to the process mean heterogeneity when the difference in process means is large relative to the variance of the time series.

  16. Markov Chain Monte Carlo Random Effects Modeling in Magnetic Resonance Image Processing Using the BRugs Interface to WinBUGS

    Directory of Open Access Journals (Sweden)

    David G. Gadian

    2011-10-01

    Full Text Available A common feature of many magnetic resonance image (MRI data processing methods is the voxel-by-voxel (a voxel is a volume element manner in which the processing is performed. In general, however, MRI data are expected to exhibit some level of spatial correlation, rendering an independent-voxels treatment inefficient in its use of the data. Bayesian random effect models are expected to be more efficient owing to their information-borrowing behaviour. To illustrate the Bayesian random effects approach, this paper outlines a Markov chain Monte Carlo (MCMC analysis of a perfusion MRI dataset, implemented in R using the BRugs package. BRugs provides an interface to WinBUGS and its GeoBUGS add-on. WinBUGS is a widely used programme for performing MCMC analyses, with a focus on Bayesian random effect models. A simultaneous modeling of both voxels (restricted to a region of interest and multiple subjects is demonstrated. Despite the low signal-to-noise ratio in the magnetic resonance signal intensity data, useful model signal intensity profiles are obtained. The merits of random effects modeling are discussed in comparison with the alternative approaches based on region-of-interest averaging and repeated independent voxels analysis. This paper focuses on perfusion MRI for the purpose of illustration, the main proposition being that random effects modeling is expected to be beneficial in many other MRI applications in which the signal-to-noise ratio is a limiting factor.

  17. Assimilation of Satellite Soil Moisture observation with the Particle Filter-Markov Chain Monte Carlo and Geostatistical Modeling

    Science.gov (United States)

    Moradkhani, Hamid; Yan, Hongxiang

    2016-04-01

    Soil moisture simulation and prediction are increasingly used to characterize agricultural droughts but the process suffers from data scarcity and quality. The satellite soil moisture observations could be used to improve model predictions with data assimilation. Remote sensing products, however, are typically discontinuous in spatial-temporal coverages; while simulated soil moisture products are potentially biased due to the errors in forcing data, parameters, and deficiencies of model physics. This study attempts to provide a detailed analysis of the joint and separate assimilation of streamflow and Advanced Scatterometer (ASCAT) surface soil moisture into a fully distributed hydrologic model, with the use of recently developed particle filter-Markov chain Monte Carlo (PF-MCMC) method. A geostatistical model is introduced to overcome the satellite soil moisture discontinuity issue where satellite data does not cover the whole study region or is significantly biased, and the dominant land cover is dense vegetation. The results indicate that joint assimilation of soil moisture and streamflow has minimal effect in improving the streamflow prediction, however, the surface soil moisture field is significantly improved. The combination of DA and geostatistical approach can further improve the surface soil moisture prediction.

  18. Application of the Markov Chain Monte Carlo method for snow water equivalent retrieval based on passive microwave measurements

    Science.gov (United States)

    Pan, J.; Durand, M. T.; Vanderjagt, B. J.

    2015-12-01

    Markov Chain Monte Carlo (MCMC) method is a retrieval algorithm based on Bayes' rule, which starts from an initial state of snow/soil parameters, and updates it to a series of new states by comparing the posterior probability of simulated snow microwave signals before and after each time of random walk. It is a realization of the Bayes' rule, which gives an approximation to the probability of the snow/soil parameters in condition of the measured microwave TB signals at different bands. Although this method could solve all snow parameters including depth, density, snow grain size and temperature at the same time, it still needs prior information of these parameters for posterior probability calculation. How the priors will influence the SWE retrieval is a big concern. Therefore, in this paper at first, a sensitivity test will be carried out to study how accurate the snow emission models and how explicit the snow priors need to be to maintain the SWE error within certain amount. The synthetic TB simulated from the measured snow properties plus a 2-K observation error will be used for this purpose. It aims to provide a guidance on the MCMC application under different circumstances. Later, the method will be used for the snowpits at different sites, including Sodankyla, Finland, Churchill, Canada and Colorado, USA, using the measured TB from ground-based radiometers at different bands. Based on the previous work, the error in these practical cases will be studied, and the error sources will be separated and quantified.

  19. Markov Chain Method for Radiative Transfer Modeling: A Case Study of Aerosol/Surface Retrieval using AirMSPI Measurements

    Science.gov (United States)

    Xu, F.; Diner, D. J.; Davis, A. B.; Latyshev, S.; Garay, M. J.; Kalashnikova, O.; Ge, C.; Wang, J.

    2013-12-01

    A vector Markov chain (MarCh) radiative transfer (RT) code developed at JPL that includes forward modeling of radiance and polarization fields and linearization (analytical estimation of Jacobians) was incorporated into an aerosol and surface retrieval package for a plane-parallel atmosphere/surface system. The RT computation by MarCh is based on matrix operations. To improve the code's computational efficiency, the forward model is currently undergoing acceleration through the exploration of different strategies for matrix operation and inversion, including numerical optimization, multi-threading/multi-processing techniques on a CPU. Implementation on a graphics processing unit (GPU) is also planned. Following a benchmarking study of the forward model, the performance of MarCh in aerosol and surface retrieval is being tested. With an optimized algorithm, we started from aerosol optical depth and surface retrieval using imagery acquired by Airborne Multiangle SpectroPolarimetric Imager (AirMSPI) over Fresno, CA. Aerosol properties including concentration and size distribution of different species provided by the Weather Research and Forecasting (WRF)-Chem model were used to constrain the retrieval and reduce the parameter space. The assumptions of spectral invariance in the angular shape of surface bidirectional reflectance factors (BRFs) and the magnitude of polarized surface BRFs were tested. The aerosol and surface properties are then relaxed in a stepwise way to refine the aerosol retrieval results and enable comparison with independent retrievals obtained from a collocated AErosol RObotic NETwork (AERONET) station.

  20. Modeling the effects and uncertainties of contaminated sediment remediation scenarios in a Norwegian fjord by Markov chain Monte Carlo simulation.

    Science.gov (United States)

    Saloranta, Tuomo M; Armitage, James M; Haario, Heikki; Naes, Kristoffer; Cousins, Ian T; Barton, David N

    2008-01-01

    Multimedia environmental fate models are useful tools to investigate the long-term impacts of remediation measures designed to alleviate potential ecological and human health concerns in contaminated areas. Estimating and communicating the uncertainties associated with the model simulations is a critical task for demonstrating the transparency and reliability of the results. The Extended Fourier Amplitude Sensitivity Test(Extended FAST) method for sensitivity analysis and Bayesian Markov chain Monte Carlo (MCMC) method for uncertainty analysis and model calibration have several advantages over methods typically applied for multimedia environmental fate models. Most importantly, the simulation results and their uncertainties can be anchored to the available observations and their uncertainties. We apply these techniques for simulating the historical fate of polychlorinated dibenzo-p-dioxins and dibenzofurans (PCDD/Fs) in the Grenland fjords, Norway, and for predicting the effects of different contaminated sediment remediation (capping) scenarios on the future levels of PCDD/Fs in cod and crab therein. The remediation scenario simulations show that a significant remediation effect can first be seen when significant portions of the contaminated sediment areas are cleaned up, and that increase in capping area leads to both earlier achievement of good fjord status and narrower uncertainty in the predicted timing for this. PMID:18350897

  1. Kullback-Leibler Divergence-Based Differential Evolution Markov Chain Filter for Global Localization of Mobile Robots

    Directory of Open Access Journals (Sweden)

    Fernando Martín

    2015-09-01

    Full Text Available One of the most important skills desired for a mobile robot is the ability to obtain its own location even in challenging environments. The information provided by the sensing system is used here to solve the global localization problem. In our previous work, we designed different algorithms founded on evolutionary strategies in order to solve the aforementioned task. The latest developments are presented in this paper. The engine of the localization module is a combination of the Markov chain Monte Carlo sampling technique and the Differential Evolution method, which results in a particle filter based on the minimization of a fitness function. The robot’s pose is estimated from a set of possible locations weighted by a cost value. The measurements of the perceptive sensors are used together with the predicted ones in a known map to define a cost function to optimize. Although most localization methods rely on quadratic fitness functions, the sensed information is processed asymmetrically in this filter. The Kullback-Leibler divergence is the basis of a cost function that makes it possible to deal with different types of occlusions. The algorithm performance has been checked in a real map. The results are excellent in environments with dynamic and unmodeled obstacles, a fact that causes occlusions in the sensing area.

  2. Kullback-Leibler Divergence-Based Differential Evolution Markov Chain Filter for Global Localization of Mobile Robots.

    Science.gov (United States)

    Martín, Fernando; Moreno, Luis; Garrido, Santiago; Blanco, Dolores

    2015-09-16

    One of the most important skills desired for a mobile robot is the ability to obtain its own location even in challenging environments. The information provided by the sensing system is used here to solve the global localization problem. In our previous work, we designed different algorithms founded on evolutionary strategies in order to solve the aforementioned task. The latest developments are presented in this paper. The engine of the localization module is a combination of the Markov chain Monte Carlo sampling technique and the Differential Evolution method, which results in a particle filter based on the minimization of a fitness function. The robot's pose is estimated from a set of possible locations weighted by a cost value. The measurements of the perceptive sensors are used together with the predicted ones in a known map to define a cost function to optimize. Although most localization methods rely on quadratic fitness functions, the sensed information is processed asymmetrically in this filter. The Kullback-Leibler divergence is the basis of a cost function that makes it possible to deal with different types of occlusions. The algorithm performance has been checked in a real map. The results are excellent in environments with dynamic and unmodeled obstacles, a fact that causes occlusions in the sensing area.

  3. Efficient Markov Chain Monte Carlo Implementation of Bayesian Analysis of Additive and Dominance Genetic Variances in Noninbred Pedigrees

    Science.gov (United States)

    Waldmann, Patrik; Hallander, Jon; Hoti, Fabian; Sillanpää, Mikko J.

    2008-01-01

    Accurate and fast computation of quantitative genetic variance parameters is of great importance in both natural and breeding populations. For experimental designs with complex relationship structures it can be important to include both additive and dominance variance components in the statistical model. In this study, we introduce a Bayesian Gibbs sampling approach for estimation of additive and dominance genetic variances in the traditional infinitesimal model. The method can handle general pedigrees without inbreeding. To optimize between computational time and good mixing of the Markov chain Monte Carlo (MCMC) chains, we used a hybrid Gibbs sampler that combines a single site and a blocked Gibbs sampler. The speed of the hybrid sampler and the mixing of the single-site sampler were further improved by the use of pretransformed variables. Two traits (height and trunk diameter) from a previously published diallel progeny test of Scots pine (Pinus sylvestris L.) and two large simulated data sets with different levels of dominance variance were analyzed. We also performed Bayesian model comparison on the basis of the posterior predictive loss approach. Results showed that models with both additive and dominance components had the best fit for both height and diameter and for the simulated data with high dominance. For the simulated data with low dominance, we needed an informative prior to avoid the dominance variance component becoming overestimated. The narrow-sense heritability estimates in the Scots pine data were lower compared to the earlier results, which is not surprising because the level of dominance variance was rather high, especially for diameter. In general, the hybrid sampler was considerably faster than the blocked sampler and displayed better mixing properties than the single-site sampler. PMID:18558655

  4. Efficient Markov chain Monte Carlo implementation of Bayesian analysis of additive and dominance genetic variances in noninbred pedigrees.

    Science.gov (United States)

    Waldmann, Patrik; Hallander, Jon; Hoti, Fabian; Sillanpää, Mikko J

    2008-06-01

    Accurate and fast computation of quantitative genetic variance parameters is of great importance in both natural and breeding populations. For experimental designs with complex relationship structures it can be important to include both additive and dominance variance components in the statistical model. In this study, we introduce a Bayesian Gibbs sampling approach for estimation of additive and dominance genetic variances in the traditional infinitesimal model. The method can handle general pedigrees without inbreeding. To optimize between computational time and good mixing of the Markov chain Monte Carlo (MCMC) chains, we used a hybrid Gibbs sampler that combines a single site and a blocked Gibbs sampler. The speed of the hybrid sampler and the mixing of the single-site sampler were further improved by the use of pretransformed variables. Two traits (height and trunk diameter) from a previously published diallel progeny test of Scots pine (Pinus sylvestris L.) and two large simulated data sets with different levels of dominance variance were analyzed. We also performed Bayesian model comparison on the basis of the posterior predictive loss approach. Results showed that models with both additive and dominance components had the best fit for both height and diameter and for the simulated data with high dominance. For the simulated data with low dominance, we needed an informative prior to avoid the dominance variance component becoming overestimated. The narrow-sense heritability estimates in the Scots pine data were lower compared to the earlier results, which is not surprising because the level of dominance variance was rather high, especially for diameter. In general, the hybrid sampler was considerably faster than the blocked sampler and displayed better mixing properties than the single-site sampler. PMID:18558655

  5. Application of Markov Chain Monte Carlo Method to Mantle Melting: An Example from REE Abundances in Abyssal Peridotites

    Science.gov (United States)

    LIU, B.; Liang, Y.

    2015-12-01

    Markov chain Monte Carlo (MCMC) simulation is a powerful statistical method in solving inverse problems that arise from a wide range of applications, such as nuclear physics, computational biology, financial engineering, among others. In Earth sciences applications of MCMC are primarily in the field of geophysics [1]. The purpose of this study is to introduce MCMC to geochemical inverse problems related to trace element fractionation during concurrent melting, melt transport and melt-rock reaction in the mantle. MCMC method has several advantages over linearized least squares methods in inverting trace element patterns in basalts and mantle rocks. First, MCMC can handle equations that have no explicit analytical solutions which are required by linearized least squares methods for gradient calculation. Second, MCMC converges to global minimum while linearized least squares methods may be stuck at a local minimum or converge slowly due to nonlinearity. Furthermore, MCMC can provide insight into uncertainties of model parameters with non-normal trade-off. We use MCMC to invert for extent of melting, amount of trapped melt, and extent of chemical disequilibrium between the melt and residual solid from REE data in abyssal peridotites from Central Indian Ridge and Mid-Atlantic Ridge. In the first step, we conduct forward calculation of REE evolution with melting models in a reasonable model space. We then build up a chain of melting models according to Metropolis-Hastings algorithm to represent the probability of specific model. We show that chemical disequilibrium is likely to play an important role in fractionating LREE in residual peridotites. In the future, MCMC will be applied to more realistic but also more complicated melting models in which partition coefficients, diffusion coefficients, as well as melting and melt suction rates vary as functions of temperature, pressure and mineral compositions. [1]. Sambridge & Mosegarrd [2002] Rev. Geophys.

  6. Significance testing of clinical data using virus dynamics models with a Markov chain Monte Carlo method: application to emergence of lamivudine-resistant hepatitis B virus.

    OpenAIRE

    Burroughs, N. J.; Pillay, D.; Mutimer, D

    1999-01-01

    Bayesian analysis using a virus dynamics model is demonstrated to facilitate hypothesis testing of patterns in clinical time-series. Our Markov chain Monte Carlo implementation demonstrates that the viraemia time-series observed in two sets of hepatitis B patients on antiviral (lamivudine) therapy, chronic carriers and liver transplant patients, are significantly different, overcoming clinical trial design differences that question the validity of non-parametric tests. We show that lamivudine...

  7. SU-E-J-115: Using Markov Chain Modeling to Elucidate Patterns in Breast Cancer Metastasis Over Time and Space

    Energy Technology Data Exchange (ETDEWEB)

    Comen, E; Mason, J; Kuhn, P [The Scripps Research Institute, La Jolla, CA (United States); Nieva, J [Billings Clinic, Billings, Montana (United States); Newton, P [University of Southern California, Los Angeles, CA (United States); Norton, L; Venkatappa, N; Jochelson, M [Memorial Sloan-Kettering Cancer Center, NY, NY (United States)

    2014-06-01

    Purpose: Traditionally, breast cancer metastasis is described as a process wherein cancer cells spread from the breast to multiple organ systems via hematogenous and lymphatic routes. Mapping organ specific patterns of cancer spread over time is essential to understanding metastatic progression. In order to better predict sites of metastases, here we demonstrate modeling of the patterned migration of metastasis. Methods: We reviewed the clinical history of 453 breast cancer patients from Memorial Sloan Kettering Cancer Center who were non-metastatic at diagnosis but developed metastasis over time. We used the variables of organ site of metastases as well as time to create a Markov chain model of metastasis. We illustrate the probabilities of metastasis occurring at a given anatomic site together with the probability of spread to additional sites. Results: Based on the clinical histories of 453 breast cancer patients who developed metastasis, we have learned (i) how to create the Markov transition matrix governing the probabilities of cancer progression from site to site; (ii) how to create a systemic network diagram governing disease progression modeled as a random walk on a directed graph; (iii) how to classify metastatic sites as ‘sponges’ that tend to only receive cancer cells or ‘spreaders’ that receive and release them; (iv) how to model the time-scales of disease progression as a Weibull probability distribution function; (v) how to perform Monte Carlo simulations of disease progression; and (vi) how to interpret disease progression as an entropy-increasing stochastic process. Conclusion: Based on our modeling, metastatic spread may follow predictable pathways. Mapping metastasis not simply by organ site, but by function as either a ‘spreader’ or ‘sponge’ fundamentally reframes our understanding of metastatic processes. This model serves as a novel platform from which we may integrate the evolving genomic landscape that drives cancer

  8. SU-E-J-115: Using Markov Chain Modeling to Elucidate Patterns in Breast Cancer Metastasis Over Time and Space

    International Nuclear Information System (INIS)

    Purpose: Traditionally, breast cancer metastasis is described as a process wherein cancer cells spread from the breast to multiple organ systems via hematogenous and lymphatic routes. Mapping organ specific patterns of cancer spread over time is essential to understanding metastatic progression. In order to better predict sites of metastases, here we demonstrate modeling of the patterned migration of metastasis. Methods: We reviewed the clinical history of 453 breast cancer patients from Memorial Sloan Kettering Cancer Center who were non-metastatic at diagnosis but developed metastasis over time. We used the variables of organ site of metastases as well as time to create a Markov chain model of metastasis. We illustrate the probabilities of metastasis occurring at a given anatomic site together with the probability of spread to additional sites. Results: Based on the clinical histories of 453 breast cancer patients who developed metastasis, we have learned (i) how to create the Markov transition matrix governing the probabilities of cancer progression from site to site; (ii) how to create a systemic network diagram governing disease progression modeled as a random walk on a directed graph; (iii) how to classify metastatic sites as ‘sponges’ that tend to only receive cancer cells or ‘spreaders’ that receive and release them; (iv) how to model the time-scales of disease progression as a Weibull probability distribution function; (v) how to perform Monte Carlo simulations of disease progression; and (vi) how to interpret disease progression as an entropy-increasing stochastic process. Conclusion: Based on our modeling, metastatic spread may follow predictable pathways. Mapping metastasis not simply by organ site, but by function as either a ‘spreader’ or ‘sponge’ fundamentally reframes our understanding of metastatic processes. This model serves as a novel platform from which we may integrate the evolving genomic landscape that drives cancer

  9. Sensitivity of hidden Markov models

    OpenAIRE

    Mitrophanov, Alexander Yu.; Lomsadze, Alexandre; Borodovsky, Mark

    2005-01-01

    We derive a tight perturbation bound for hidden Markov models. Using this bound, we show that, in many cases, the distribution of a hidden Markov model is considerably more sensitive to perturbations in the emission probabilities than to perturbations in the transition probability matrix and the initial distribution of the underlying Markov chain. Our approach can also be used to assess the sensitivity of other stochastic models, such as mixture processes and semi-Markov ...

  10. Analysis and forecast of employees’ mobility on the labor market in Romania using Markov chains

    Directory of Open Access Journals (Sweden)

    Mariana Balan

    2013-06-01

    Full Text Available The mobility of labor, defined as responsiveness and adaptation of persons or groups of persons on the challenges of the social and economic environment is therefore a social phenomenon depending on time and space. A high mobility increases opportunities for workers to find a job and employers to find persons with an adequate level of skills, thus boosting employment and economic growth. In recent years, in Romania there has been an accentuation of existing gaps, compared with the European Union countries, as regards the occupational structure of employment. In this context, the paper proposes an analysis of the evolution of labor mobility in the main sectors of the Romanian economy. Also, it was pursued the Markovian modeling of employees’ mobility on the labor market and its forecast in Romania, under the impact of rapid and profound social and economic changes, and the correlation between them as well, with a view to make forecasts of the Romanian economy evolution in the short term.

  11. Estimation of markov chain transition probabilities and rates from fully and partially observed data: uncertainty propagation, evidence synthesis, and model calibration.

    Science.gov (United States)

    Welton, Nicky J; Ades, A E

    2005-01-01

    Markov transition models are frequently used to model disease progression. The authors show how the solution to Kolmogorov's forward equations can be exploited to map between transition rates and probabilities from probability data in multistate models. They provide a uniform, Bayesian treatment of estimation and propagation of uncertainty of transition rates and probabilities when 1) observations are available on all transitions and exact time at risk in each state (fully observed data) and 2) observations are on initial state and final state after a fixed interval of time but not on the sequence of transitions (partially observed data). The authors show how underlying transition rates can be recovered from partially observed data using Markov chain Monte Carlo methods in WinBUGS, and they suggest diagnostics to investigate inconsistencies between evidence from different starting states. An illustrative example for a 3-state model is given, which shows how the methods extend to more complex Markov models using the software WBDiff to compute solutions. Finally, the authors illustrate how to statistically combine data from multiple sources, including partially observed data at several follow-up times and also how to calibrate a Markov model to be consistent with data from one specific study. PMID:16282214

  12. 关于树指标隐Markov链及其等价定义∗%Equivalent Definitions of T-indexed Hidden Markov Chains

    Institute of Scientific and Technical Information of China (English)

    王豹; 杨卫国

    2015-01-01

    本文参照直线上隐Markov模型的概念,给出有限树指标隐Markov链的定义。在该定义中,树指标隐Markov链由两个树指标随机过程组成,其中第一个树指标随机过程是树指标Markov链,是不能被直接观测到的隐藏链;第二个树指标随机过程是可被观测的且关于第一个树指标随机过程条件独立,对于树上的任意一个顶点,第二个随机过程此处的取值只信赖于隐藏链中此处的取值。最后,我们给出了树指标隐Markov链的三个等价定义。%In this paper, we give the definition of tree indexed hidden Markov chain with finite state space based on the concept of hidden Markov model. In our definition, tree indexed hidden Markov chain consists of two tree indexed random processes. The underlying process is a tree indexed Markov chain and can not be observed, and the second process is conditional independent of the former. For the arbitrary vertex in tree, the second process only dependents on the underlying process. Finally, we propose three equivalent definitions.

  13. Estimation in autoregressive models with Markov regime

    OpenAIRE

    Ríos, Ricardo; Rodríguez, Luis

    2005-01-01

    In this paper we derive the consistency of the penalized likelihood method for the number state of the hidden Markov chain in autoregressive models with Markov regimen. Using a SAEM type algorithm to estimate the models parameters. We test the null hypothesis of hidden Markov Model against an autoregressive process with Markov regime.

  14. Meta-heuristic ant colony optimization technique to forecast the amount of summer monsoon rainfall: skill comparison with Markov chain model

    Science.gov (United States)

    Chaudhuri, Sutapa; Goswami, Sayantika; Das, Debanjana; Middey, Anirban

    2014-05-01

    Forecasting summer monsoon rainfall with precision becomes crucial for the farmers to plan for harvesting in a country like India where the national economy is mostly based on regional agriculture. The forecast of monsoon rainfall based on artificial neural network is a well-researched problem. In the present study, the meta-heuristic ant colony optimization (ACO) technique is implemented to forecast the amount of summer monsoon rainfall for the next day over Kolkata (22.6°N, 88.4°E), India. The ACO technique belongs to swarm intelligence and simulates the decision-making processes of ant colony similar to other adaptive learning techniques. ACO technique takes inspiration from the foraging behaviour of some ant species. The ants deposit pheromone on the ground in order to mark a favourable path that should be followed by other members of the colony. A range of rainfall amount replicating the pheromone concentration is evaluated during the summer monsoon season. The maximum amount of rainfall during summer monsoon season (June—September) is observed to be within the range of 7.5-35 mm during the period from 1998 to 2007, which is in the range 4 category set by the India Meteorological Department (IMD). The result reveals that the accuracy in forecasting the amount of rainfall for the next day during the summer monsoon season using ACO technique is 95 % where as the forecast accuracy is 83 % with Markov chain model (MCM). The forecast through ACO and MCM are compared with other existing models and validated with IMD observations from 2008 to 2012.

  15. An application of Embedded Markov chain for soil sequences: Case study in North Western part of Algeria

    Directory of Open Access Journals (Sweden)

    Lotfi Mustapha Kazi-Tani

    2016-07-01

    Full Text Available Embedded Markov chain (EMC has long history in geological domains, particularly to define the most representative sequences from statigraphic logs. In other words, what is viewed as a meaningless and disordered stratigraphic layer stack can be reorganized in a meaningful sequence by using EMC. This method was transposed in this paper to obtain soil sequences from data retrieved from soil map made by authors, covering a part of the region of Traras (N.W. of Algeria and containing 13 major soil types. Each major soil type occupies at least one polygon in the map and allow to establish soil adjacencies, which have been tabulated in a matrix regardless to the direction. Three EMC methods have been tested, Walker, Harper and Türk using Strati-signal software and to erect soil relationship diagrams (SRD representing the most significant links between soils. Significant test is the main difference between the above mentioned three EMC methods. It has been shown that Harper method is quite insensitive to small number of transitions. Besides, all three methods agreed for one soil sequence made by four soils: lithics leptosols- cambisols chormics- cambisols calcarics- fluvisols representing theoretical catena the most representative to the study area. This soil sequence is relevant to the study region and even to the whole Mediterranean region, and is commanded by the topography and the Mediterranean bioclimate. Walker SRD is the most realistic but the most difficult to interpret because of the high number of soil links, Harper SRD gives interesting results. Although the results didn’t bring something new to the soil interpretation and soil pedogensis but EMC applied to a finer scale may highlights other hidden relationships between soils.

  16. Phase-coexistence simulations of fluid mixtures by the Markov Chain Monte Carlo method using single-particle models

    KAUST Repository

    Li, Jun

    2013-09-01

    We present a single-particle Lennard-Jones (L-J) model for CO2 and N2. Simplified L-J models for other small polyatomic molecules can be obtained following the methodology described herein. The phase-coexistence diagrams of single-component systems computed using the proposed single-particle models for CO2 and N2 agree well with experimental data over a wide range of temperatures. These diagrams are computed using the Markov Chain Monte Carlo method based on the Gibbs-NVT ensemble. This good agreement validates the proposed simplified models. That is, with properly selected parameters, the single-particle models have similar accuracy in predicting gas-phase properties as more complex, state-of-the-art molecular models. To further test these single-particle models, three binary mixtures of CH4, CO2 and N2 are studied using a Gibbs-NPT ensemble. These results are compared against experimental data over a wide range of pressures. The single-particle model has similar accuracy in the gas phase as traditional models although its deviation in the liquid phase is greater. Since the single-particle model reduces the particle number and avoids the time-consuming Ewald summation used to evaluate Coulomb interactions, the proposed model improves the computational efficiency significantly, particularly in the case of high liquid density where the acceptance rate of the particle-swap trial move increases. We compare, at constant temperature and pressure, the Gibbs-NPT and Gibbs-NVT ensembles to analyze their performance differences and results consistency. As theoretically predicted, the agreement between the simulations implies that Gibbs-NVT can be used to validate Gibbs-NPT predictions when experimental data is not available. © 2013 Elsevier Inc.

  17. Changes in mangrove species assemblages and future prediction of the Bangladesh Sundarbans using Markov chain model and cellular automata.

    Science.gov (United States)

    Mukhopadhyay, Anirban; Mondal, Parimal; Barik, Jyotiskona; Chowdhury, S M; Ghosh, Tuhin; Hazra, Sugata

    2015-06-01

    The composition and assemblage of mangroves in the Bangladesh Sundarbans are changing systematically in response to several environmental factors. In order to understand the impact of the changing environmental conditions on the mangrove forest, species composition maps for the years 1985, 1995 and 2005 were studied. In the present study, 1985 and 1995 species zonation maps were considered as base data and the cellular automata-Markov chain model was run to predict the species zonation for the year 2005. The model output was validated against the actual dataset for 2005 and calibrated. Finally, using the model, mangrove species zonation maps for the years 2025, 2055 and 2105 have been prepared. The model was run with the assumption that the continuation of the current tempo and mode of drivers of environmental factors (temperature, rainfall, salinity change) of the last two decades will remain the same in the next few decades. Present findings show that the area distribution of the following species assemblages like Goran (Ceriops), Sundari (Heritiera), Passur (Xylocarpus), and Baen (Avicennia) would decrease in the descending order, whereas the area distribution of Gewa (Excoecaria), Keora (Sonneratia) and Kankra (Bruguiera) dominated assemblages would increase. The spatial distribution of projected mangrove species assemblages shows that more salt tolerant species will dominate in the future; which may be used as a proxy to predict the increase of salinity and its spatial variation in Sundarbans. Considering the present rate of loss of forest land, 17% of the total mangrove cover is predicted to be lost by the year 2105 with a significant loss of fresh water loving mangroves and related ecosystem services. This paper describes a unique approach to assess future changes in species composition and future forest zonation in mangroves under the 'business as usual' scenario of climate change. PMID:25719448

  18. Epistasis Test in Meta-Analysis: A Multi-Parameter Markov Chain Monte Carlo Model for Consistency of Evidence.

    Science.gov (United States)

    Lin, Chin; Chu, Chi-Ming; Su, Sui-Lung

    2016-01-01

    Conventional genome-wide association studies (GWAS) have been proven to be a successful strategy for identifying genetic variants associated with complex human traits. However, there is still a large heritability gap between GWAS and transitional family studies. The "missing heritability" has been suggested to be due to lack of studies focused on epistasis, also called gene-gene interactions, because individual trials have often had insufficient sample size. Meta-analysis is a common method for increasing statistical power. However, sufficient detailed information is difficult to obtain. A previous study employed a meta-regression-based method to detect epistasis, but it faced the challenge of inconsistent estimates. Here, we describe a Markov chain Monte Carlo-based method, called "Epistasis Test in Meta-Analysis" (ETMA), which uses genotype summary data to obtain consistent estimates of epistasis effects in meta-analysis. We defined a series of conditions to generate simulation data and tested the power and type I error rates in ETMA, individual data analysis and conventional meta-regression-based method. ETMA not only successfully facilitated consistency of evidence but also yielded acceptable type I error and higher power than conventional meta-regression. We applied ETMA to three real meta-analysis data sets. We found significant gene-gene interactions in the renin-angiotensin system and the polycyclic aromatic hydrocarbon metabolism pathway, with strong supporting evidence. In addition, glutathione S-transferase (GST) mu 1 and theta 1 were confirmed to exert independent effects on cancer. We concluded that the application of ETMA to real meta-analysis data was successful. Finally, we developed an R package, etma, for the detection of epistasis in meta-analysis [etma is available via the Comprehensive R Archive Network (CRAN) at https://cran.r-project.org/web/packages/etma/index.html].

  19. Epistasis Test in Meta-Analysis: A Multi-Parameter Markov Chain Monte Carlo Model for Consistency of Evidence

    Science.gov (United States)

    Lin, Chin; Chu, Chi-Ming; Su, Sui-Lung

    2016-01-01

    Conventional genome-wide association studies (GWAS) have been proven to be a successful strategy for identifying genetic variants associated with complex human traits. However, there is still a large heritability gap between GWAS and transitional family studies. The “missing heritability” has been suggested to be due to lack of studies focused on epistasis, also called gene–gene interactions, because individual trials have often had insufficient sample size. Meta-analysis is a common method for increasing statistical power. However, sufficient detailed information is difficult to obtain. A previous study employed a meta-regression-based method to detect epistasis, but it faced the challenge of inconsistent estimates. Here, we describe a Markov chain Monte Carlo-based method, called “Epistasis Test in Meta-Analysis” (ETMA), which uses genotype summary data to obtain consistent estimates of epistasis effects in meta-analysis. We defined a series of conditions to generate simulation data and tested the power and type I error rates in ETMA, individual data analysis and conventional meta-regression-based method. ETMA not only successfully facilitated consistency of evidence but also yielded acceptable type I error and higher power than conventional meta-regression. We applied ETMA to three real meta-analysis data sets. We found significant gene–gene interactions in the renin–angiotensin system and the polycyclic aromatic hydrocarbon metabolism pathway, with strong supporting evidence. In addition, glutathione S-transferase (GST) mu 1 and theta 1 were confirmed to exert independent effects on cancer. We concluded that the application of ETMA to real meta-analysis data was successful. Finally, we developed an R package, etma, for the detection of epistasis in meta-analysis [etma is available via the Comprehensive R Archive Network (CRAN) at https://cran.r-project.org/web/packages/etma/index.html]. PMID:27045371

  20. Real-Time Landmine Detection with Ground-Penetrating Radar Using Discriminative and Adaptive Hidden Markov Models

    Directory of Open Access Journals (Sweden)

    Paul Gader

    2005-07-01

    Full Text Available We propose a real-time software system for landmine detection using ground-penetrating radar (GPR. The system includes an efficient and adaptive preprocessing component; a hidden Markov model- (HMM- based detector; a corrective training component; and an incremental update of the background model. The preprocessing is based on frequency-domain processing and performs ground-level alignment and background removal. The HMM detector is an improvement of a previously proposed system (baseline. It includes additional pre- and postprocessing steps to improve the time efficiency and enable real-time application. The corrective training component is used to adjust the initial model parameters to minimize the number of misclassification sequences. This component could be used offline, or online through feedback to adapt an initial model to specific sites and environments. The background update component adjusts the parameters of the background model to adapt it to each lane during testing. The proposed software system is applied to data acquired from three outdoor test sites at different geographic locations, using a state-of-the-art array GPR prototype. The first collection was used as training, and the other two (contain data from more than 1200 m2 of simulated dirt and gravel roads for testing. Our results indicate that, on average, the corrective training can improve the performance by about 10% for each site. For individual lanes, the performance gain can reach 50%.

  1. Markov Chains Used to Determine the Model of Stock Value and Compared with Other New Models of Stock Value (P/E Model and Ohlson Model

    Directory of Open Access Journals (Sweden)

    Abbasali Pouraghajan

    2012-10-01

    Full Text Available The aim of this study a comparison between the three models for the valuation of stocks in Tehran Stock Exchange. These three names PE, Olson or residual income and a Markov chain (Markov are. Researchers in their study were to calculate the valuation of shares in the first two terms and then calculate the value of the enamel Markov chain to achieve a comparative mode. Result of research shows that almost in all cases, there is no significant difference between explanatory power of these models in determining shares value and investments in Tehran exchange market can for assessment of shares uses from these 3 models, but in most cases residual income assessment model by considering less standard error of regression can say, partly is better model in determining the company's value which maybe the main reason be have high explanatory power of two dependent profit variable overall, and book value of share holder's salary by using the overall accounting relation in comparison with two other models.

  2. Subdivision of the MDR superfamily of medium-chain dehydrogenases/reductases through iterative hidden Markov model refinement

    Directory of Open Access Journals (Sweden)

    Persson Bengt

    2010-10-01

    Full Text Available Abstract Background The Medium-chain Dehydrogenases/Reductases (MDR form a protein superfamily whose size and complexity defeats traditional means of subclassification; it currently has over 15000 members in the databases, the pairwise sequence identity is typically around 25%, there are members from all kingdoms of life, the chain-lengths vary as does the oligomericity, and the members are partaking in a multitude of biological processes. There are profile hidden Markov models (HMMs available for detecting MDR superfamily members, but none for determining which MDR family each protein belongs to. The current torrential influx of new sequence data enables elucidation of more and more protein families, and at an increasingly fine granularity. However, gathering good quality training data usually requires manual attention by experts and has therefore been the rate limiting step for expanding the number of available models. Results We have developed an automated algorithm for HMM refinement that produces stable and reliable models for protein families. This algorithm uses relationships found in data to generate confident seed sets. Using this algorithm we have produced HMMs for 86 distinct MDR families and 34 of their subfamilies which can be used in automated annotation of new sequences. We find that MDR forms with 2 Zn2+ ions in general are dehydrogenases, while MDR forms with no Zn2+ in general are reductases. Furthermore, in Bacteria MDRs without Zn2+ are more frequent than those with Zn2+, while the opposite is true for eukaryotic MDRs, indicating that Zn2+ has been recruited into the MDR superfamily after the initial life kingdom separations. We have also developed a web site http://mdr-enzymes.org that provides textual and numeric search against various characterised MDR family properties, as well as sequence scan functions for reliable classification of novel MDR sequences. Conclusions Our method of refinement can be readily applied to

  3. A Comparison of Bayesian Monte Carlo Markov Chain and Maximum Likelihood Estimation Methods for the Statistical Analysis of Geodetic Time Series

    Science.gov (United States)

    Olivares, G.; Teferle, F. N.

    2013-12-01

    Geodetic time series provide information which helps to constrain theoretical models of geophysical processes. It is well established that such time series, for example from GPS, superconducting gravity or mean sea level (MSL), contain time-correlated noise which is usually assumed to be a combination of a long-term stochastic process (characterized by a power-law spectrum) and random noise. Therefore, when fitting a model to geodetic time series it is essential to also estimate the stochastic parameters beside the deterministic ones. Often the stochastic parameters include the power amplitudes of both time-correlated and random noise, as well as, the spectral index of the power-law process. To date, the most widely used method for obtaining these parameter estimates is based on maximum likelihood estimation (MLE). We present an integration method, the Bayesian Monte Carlo Markov Chain (MCMC) method, which, by using Markov chains, provides a sample of the posteriori distribution of all parameters and, thereby, using Monte Carlo integration, all parameters and their uncertainties are estimated simultaneously. This algorithm automatically optimizes the Markov chain step size and estimates the convergence state by spectral analysis of the chain. We assess the MCMC method through comparison with MLE, using the recently released GPS position time series from JPL and apply it also to the MSL time series from the Revised Local Reference data base of the PSMSL. Although the parameter estimates for both methods are fairly equivalent, they suggest that the MCMC method has some advantages over MLE, for example, without further computations it provides the spectral index uncertainty, is computationally stable and detects multimodality.

  4. Neutrino masses and cosmological parameters from a Euclid-like survey: Markov Chain Monte Carlo forecasts including theoretical errors

    Energy Technology Data Exchange (ETDEWEB)

    Audren, Benjamin; Lesgourgues, Julien [Institut de Théorie des Phénomènes Physiques, École PolytechniqueFédérale de Lausanne, CH-1015, Lausanne (Switzerland); Bird, Simeon [Institute for Advanced Study, 1 Einstein Drive, Princeton, NJ, 08540 (United States); Haehnelt, Martin G. [Kavli Institute for Cosmology and Institute of Astronomy, Madingley Road, Cambridge, CB3 0HA (United Kingdom); Viel, Matteo, E-mail: benjamin.audren@epfl.ch, E-mail: julien.lesgourgues@cern.ch, E-mail: spb@ias.edu, E-mail: haehnelt@ast.cam.ac.uk, E-mail: viel@oats.inaf.it [INAF/Osservatorio Astronomico di Trieste, Via Tiepolo 11, 34143, Trieste (Italy)

    2013-01-01

    We present forecasts for the accuracy of determining the parameters of a minimal cosmological model and the total neutrino mass based on combined mock data for a future Euclid-like galaxy survey and Planck. We consider two different galaxy surveys: a spectroscopic redshift survey and a cosmic shear survey. We make use of the Monte Carlo Markov Chains (MCMC) technique and assume two sets of theoretical errors. The first error is meant to account for uncertainties in the modelling of the effect of neutrinos on the non-linear galaxy power spectrum and we assume this error to be fully correlated in Fourier space. The second error is meant to parametrize the overall residual uncertainties in modelling the non-linear galaxy power spectrum at small scales, and is conservatively assumed to be uncorrelated and to increase with the ratio of a given scale to the scale of non-linearity. It hence increases with wavenumber and decreases with redshift. With these two assumptions for the errors and assuming further conservatively that the uncorrelated error rises above 2% at k = 0.4 h/Mpc and z = 0.5, we find that a future Euclid-like cosmic shear/galaxy survey achieves a 1-σ error on M{sub ν} close to 32 meV/25 meV, sufficient for detecting the total neutrino mass with good significance. If the residual uncorrelated errors indeed rises rapidly towards smaller scales in the non-linear regime as we have assumed here then the data on non-linear scales does not increase the sensitivity to the total neutrino mass. Assuming instead a ten times smaller theoretical error with the same scale dependence, the error on the total neutrino mass decreases moderately from σ(M{sub ν}) = 18 meV to 14 meV when mildly non-linear scales with 0.1 h/Mpc < k < 0.6 h/Mpc are included in the analysis of the galaxy survey data.

  5. Behavioral Analysis of Visitors to a Medical Institution’s Website Using Markov Chain Monte Carlo Methods

    Science.gov (United States)

    Tani, Yuji

    2016-01-01

    Background Consistent with the “attention, interest, desire, memory, action” (AIDMA) model of consumer behavior, patients collect information about available medical institutions using the Internet to select information for their particular needs. Studies of consumer behavior may be found in areas other than medical institution websites. Such research uses Web access logs for visitor search behavior. At this time, research applying the patient searching behavior model to medical institution website visitors is lacking. Objective We have developed a hospital website search behavior model using a Bayesian approach to clarify the behavior of medical institution website visitors and determine the probability of their visits, classified by search keyword. Methods We used the website data access log of a clinic of internal medicine and gastroenterology in the Sapporo suburbs, collecting data from January 1 through June 31, 2011. The contents of the 6 website pages included the following: home, news, content introduction for medical examinations, mammography screening, holiday person-on-duty information, and other. The search keywords we identified as best expressing website visitor needs were listed as the top 4 headings from the access log: clinic name, clinic name + regional name, clinic name + medical examination, and mammography screening. Using the search keywords as the explaining variable, we built a binomial probit model that allows inspection of the contents of each purpose variable. Using this model, we determined a beta value and generated a posterior distribution. We performed the simulation using Markov Chain Monte Carlo methods with a noninformation prior distribution for this model and determined the visit probability classified by keyword for each category. Results In the case of the keyword “clinic name,” the visit probability to the website, repeated visit to the website, and contents page for medical examination was positive. In the case of the

  6. Three-dimensional modeling of chromatin structure from interaction frequency data using Markov chain Monte Carlo sampling

    Directory of Open Access Journals (Sweden)

    Dostie Josée

    2011-10-01

    Full Text Available Abstract Background Long-range interactions between regulatory DNA elements such as enhancers, insulators and promoters play an important role in regulating transcription. As chromatin contacts have been found throughout the human genome and in different cell types, spatial transcriptional control is now viewed as a general mechanism of gene expression regulation. Chromosome Conformation Capture Carbon Copy (5C and its variant Hi-C are techniques used to measure the interaction frequency (IF between specific regions of the genome. Our goal is to use the IF data generated by these experiments to computationally model and analyze three-dimensional chromatin organization. Results We formulate a probabilistic model linking 5C/Hi-C data to physical distances and describe a Markov chain Monte Carlo (MCMC approach called MCMC5C to generate a representative sample from the posterior distribution over structures from IF data. Structures produced from parallel MCMC runs on the same dataset demonstrate that our MCMC method mixes quickly and is able to sample from the posterior distribution of structures and find subclasses of structures. Structural properties (base looping, condensation, and local density were defined and their distribution measured across the ensembles of structures generated. We applied these methods to a biological model of human myelomonocyte cellular differentiation and identified distinct chromatin conformation signatures (CCSs corresponding to each of the cellular states. We also demonstrate the ability of our method to run on Hi-C data and produce a model of human chromosome 14 at 1Mb resolution that is consistent with previously observed structural properties as measured by 3D-FISH. Conclusions We believe that tools like MCMC5C are essential for the reliable analysis of data from the 3C-derived techniques such as 5C and Hi-C. By integrating complex, high-dimensional and noisy datasets into an easy to interpret ensemble of three

  7. DOUBLE-MARKOV RISK MODEL

    Institute of Scientific and Technical Information of China (English)

    Xiaoyun MO; Jieming ZHOU; Hui OU; Xiangqun YANG

    2013-01-01

    Given a new Double-Markov risk model DM =(μ,Q,v,H; Y,Z) and Double-Markov risk process U ={U(t),t ≥ 0}.The ruin or survival problem is addressed.Equations which the survival probability satisfied and the formulas of calculating survival probability are obtained.Recursion formulas of calculating the survival probability and analytic expression of recursion items are obtained.The conclusions are expressed by Q matrix for a Markov chain and transition probabilities for another Markov Chain.

  8. Evaluation by Markov chain models of a non-randomised breast cancer screening programme in women aged under 50 years in Sweden

    OpenAIRE

    Chen, H.H; Thurfjell, E.; Duffy, S W; Tabar, L

    1998-01-01

    STUDY OBJECTIVE: To apply Markov chain models that have previously been used on data in randomised trials of breast cancer screening to data from an uncontrolled service screening programme; to compare results with those from a randomised trial. DESIGN: A service screening programme in Uppsala county, Sweden, inviting 25,660 women aged 39-49 to mammographic screening every 20 months, and the Swedish Two-County Trial inviting 19,844 women aged 40-49 to two yearly screening, compared with...

  9. A neurocomputational model of stimulus-specific adaptation to oddball and Markov sequences.

    OpenAIRE

    Robert Mill; Martin Coath; Thomas Wennekers; Denham, Susan L.

    2011-01-01

    Stimulus-specific adaptation (SSA) occurs when the spike rate of a neuron decreases with repetitions of the same stimulus, but recovers when a different stimulus is presented. It has been suggested that SSA in single auditory neurons may provide information to change detection mechanisms evident at other scales (e.g., mismatch negativity in the event related potential), and participate in the control of attention and the formation of auditory streams. This article presents a spiking-neuron mo...

  10. The Study of Reinforcement Learning for Traffic Self-Adaptive Control under Multiagent Markov Game Environment

    OpenAIRE

    Lun-Hui Xu; Xin-Hai Xia; Qiang Luo

    2013-01-01

    Urban traffic self-adaptive control problem is dynamic and uncertain, so the states of traffic environment are hard to be observed. Efficient agent which controls a single intersection can be discovered automatically via multiagent reinforcement learning. However, in the majority of the previous works on this approach, each agent needed perfect observed information when interacting with the environment and learned individually with less efficient coordination. This study casts traffic self-ad...

  11. Adaptive double chain quantum genetic algorithm for constrained optimization problems

    Institute of Scientific and Technical Information of China (English)

    Kong Haipeng; Li Ni; Shen Yuzhong

    2015-01-01

    Optimization problems are often highly constrained and evolutionary algorithms (EAs) are effective methods to tackle this kind of problems. To further improve search efficiency and con-vergence rate of EAs, this paper presents an adaptive double chain quantum genetic algorithm (ADCQGA) for solving constrained optimization problems. ADCQGA makes use of double-individuals to represent solutions that are classified as feasible and infeasible solutions. Fitness (or evaluation) functions are defined for both types of solutions. Based on the fitness function, three types of step evolution (SE) are defined and utilized for judging evolutionary individuals. An adaptive rotation is proposed and used to facilitate updating individuals in different solutions. To further improve the search capability and convergence rate, ADCQGA utilizes an adaptive evolution process (AEP), adaptive mutation and replacement techniques. ADCQGA was first tested on a widely used benchmark function to illustrate the relationship between initial parameter values and the convergence rate/search capability. Then the proposed ADCQGA is successfully applied to solve other twelve benchmark functions and five well-known constrained engineering design problems. Multi-aircraft cooperative target allocation problem is a typical constrained optimization problem and requires efficient methods to tackle. Finally, ADCQGA is successfully applied to solving the target allocation problem.

  12. Insight into earthquake sequencing: analysis and interpretation of time-series constructed from the directed graph of the Markov chain model

    Science.gov (United States)

    Cavers, M. S.; Vasudevan, K.

    2015-02-01

    Directed graph representation of a Markov chain model to study global earthquake sequencing leads to a time-series of state-to-state transition probabilities that includes the spatio-temporally linked recurrent events in the record-breaking sense. A state refers to a configuration comprised of zones with either the occurrence or non-occurrence of an earthquake in each zone in a pre-determined time interval. Since the time-series is derived from non-linear and non-stationary earthquake sequencing, we use known analysis methods to glean new information. We apply decomposition procedures such as ensemble empirical mode decomposition (EEMD) to study the state-to-state fluctuations in each of the intrinsic mode functions. We subject the intrinsic mode functions, the orthogonal basis set derived from the time-series using the EEMD, to a detailed analysis to draw information-content of the time-series. Also, we investigate the influence of random-noise on the data-driven state-to-state transition probabilities. We consider a second aspect of earthquake sequencing that is closely tied to its time-correlative behavior. Here, we extend the Fano factor and Allan factor analysis to the time-series of state-to state transition frequencies of a Markov chain. Our results support not only the usefulness the intrinsic mode functions in understanding the time-series but also the presence of power-law behaviour exemplified by the Fano factor and the Allan factor.

  13. Temporal relation between the ADC and DC potential responses to transient focal ischemia in the rat: a Markov chain Monte Carlo simulation analysis.

    Science.gov (United States)

    King, Martin D; Crowder, Martin J; Hand, David J; Harris, Neil G; Williams, Stephen R; Obrenovitch, Tihomir P; Gadian, David G

    2003-06-01

    Markov chain Monte Carlo simulation was used in a reanalysis of the longitudinal data obtained by Harris et al. (J Cereb Blood Flow Metab 20:28-36) in a study of the direct current (DC) potential and apparent diffusion coefficient (ADC) responses to focal ischemia. The main purpose was to provide a formal analysis of the temporal relationship between the ADC and DC responses, to explore the possible involvement of a common latent (driving) process. A Bayesian nonlinear hierarchical random coefficients model was adopted. DC and ADC transition parameter posterior probability distributions were generated using three parallel Markov chains created using the Metropolis algorithm. Particular attention was paid to the within-subject differences between the DC and ADC time course characteristics. The results show that the DC response is biphasic, whereas the ADC exhibits monophasic behavior, and that the two DC components are each distinguishable from the ADC response in their time dependencies. The DC and ADC changes are not, therefore, driven by a common latent process. This work demonstrates a general analytical approach to the multivariate, longitudinal data-processing problem that commonly arises in stroke and other biomedical research. PMID:12796716

  14. Cambios en el patrón de desarrollo floral de Neoregelia 'Flandria' (Bromeliaceae analizados mediante Cadenas de Markov Floral Developmental Pattern Changes on Neoregelia‘Flandria’(BROMELIACEAE Analysed by Markov Chain methodologyADRIANA

    Directory of Open Access Journals (Sweden)

    Chaparo de Barrera Ángela

    2003-06-01

    Full Text Available Neoregelia 'Flandria'y N. 'Van Durme'son cultivares ornamentales de bromelias. Sussemillas no son viables y la propagación in vitrocausa alteraciones en el variegado delfollaje, por tanto la brotación vegetativa se constituye en la única opción de propa-gación estable. Este artículo hace referencia a los patrones de desarrollo floral (p de72 bromelias y su relación con la brotación vegetativa por efecto de 3 tratamientos:ANA 190 ppm (T1, Ethrel: ANA + ETHREL (T2 y ETHREL, (T3, y divididos poredad: E1 y E2. El resultado de la aplicación de estos tratamientos fue que se obtu-vieron plantas más elongadas y 6 nuevos patrones. Mediante la metodología de cade-nas de Markov se estimaron la evolución de cada patrón hacia los demás a nivel deprobabilidad (si/no y el porcentaje de esta para cada caso.Neoregelia 'Flandria'and N. 'Van Durme'are ornamental cultivars of Bromelia.Propagation by seeds is not viable and prunes constitutes the only way to propagateavoiding alterations. In this article the developmental floral pattern of 72 Bromeliastreated with ANA 190 ppm (T1, Ethrel: ANA + ETHREL,(T2 y ETHREL, (T3 anddivided into two age groups: E1 y E2 are shown. The treatments studied generated moreelongated plants and six new patterns. Using the Markov chain methodology theprobability to evolve to any pattern and the percentage of each were studied.

  15. Calibration of environmental radionuclide transfer models using a Bayesian approach with Markov chain Monte Carlo simulations and model comparisons - Calibration of radionuclides transfer models in the environment using a Bayesian approach with Markov chain Monte Carlo simulation and comparison of models

    Energy Technology Data Exchange (ETDEWEB)

    Nicoulaud-Gouin, V.; Giacalone, M.; Gonze, M.A. [Institut de Radioprotection et de Surete Nucleaire-PRP-ENV/SERIS/LM2E (France); Martin-Garin, A.; Garcia-Sanchez, L. [IRSN-PRP-ENV/SERIS/L2BT (France)

    2014-07-01

    , distinguishes instantaneous (K{sub d}1) and first-order kinetics of sorption and desorption processes (λ{sub fix}, λ{sub rem}), each having potentially a limited sorption capacity. A Soil-Plant Deposition Model describing the weeds contamination in {sup 137}Cs, {sup 134}Cs and {sup 131}I, with in situ measures in the Fukushima prefecture (Gonze et al. submitted to this conference). This model considers two foliage pools and a root pool, and describes foliar biomass growth with a Verhulst model. One prerequisite for calibration is model identifiability. Here, we showed that there are not unique parameter values corresponding to a data set. However, sharp distributions were found when several data sets were involved. One numerical difficulty of Markov Chains is to check convergence. It was here examined with Raftery and Lewis diagnostic, Gelman and Rubin plots, and simulation trails. Failing to converge may indicate that the model is not adapted to the observations. The Bayes factor was used to decide between competing models, which applies even if they are not nested. For most data series, EK model was preferable to the nested K{sub d} approach. An Empirical Dynamical Model -consisting of two exponential functions- was compared to the Soil-Plant Deposition Model, by distinguishing site-specific parameters and invariant parameters between stations, in order to study the goodness-of-fit of the Soil-Plant Deposition Model. (authors)

  16. Performability analysis of the second order semi-Markov chains in state and duration for wind speed modeling

    CERN Document Server

    D'Amico, Guglielmo; Prattico, Flavio

    2012-01-01

    In this paper second order semi-Markov reward models are presented and equations for the higher order moments of the reward process are presented for the first time and applied to wind energy production. A real application is executed by considering a database, freely available from the web, in which are included wind speed data taken from L.S.I. - Lastem station (Italy) and sampled every 10 minutes. We compute the expected total energy produced by using the blade Aircon HAWT - 10 kW.

  17. Distributed Spectrum Detection Based on Sequential Markov Chain%基于时序马尔可夫链的分布式频谱检测

    Institute of Scientific and Technical Information of China (English)

    罗银辉; 华漫

    2011-01-01

    针对无线传感网络中的合作谱检测问题,提出一种基于时序马尔可夫链的分布式频谱检测算法.假定单节点对频谱的感知是一个马尔可夫过程,本地序列检测采用序列概率比测试进行频谱探测,得到本地序列检测值.各个感知节点将检测结果发送到数据融合中心,根据设定门限确定最终检测结果.通过Matlab仿真验证了该算法的时序检测性能.%Aiming at the cooperation spectrum detection in Wireless Sensor Network(WSN), this paper presents a distributed spectrum detection based on sequential Markov chain. It assumes the detection of the single node is a Markov process, and the result value of local sequence detection is obtained by the Sequential Probability Ratio Test(SPRT). After the measurements result of each node's own is sent to the data fusion center, the final results can be determined in line with the threshold. Matlab simulation validates the sequential detection performance.

  18. Analysis and Optimization of Power Supply Structure Based on Markov Chain and Error Optimization for Renewable Energy from the Perspective of Sustainability

    Directory of Open Access Journals (Sweden)

    Xiaomin Xu

    2016-07-01

    Full Text Available With the rapid development of renewable energy, power supply structure is changing. However, thermal power is still dominant. With the background in low carbon economy, reasonable adjustment and optimization of the power supply structure is the trend of future development in the power industry. It is also a reliable guarantee of a fast, healthy and stable development of national economy. In this paper, the sustainable development of renewable energy sources is analyzed from the perspective of power supply. Through the research on the development of power supply structure, we find that regional power supply structure development mode conforms to dynamic characteristics and there must exist a Markov chain in the final equilibrium state. Combined with the characteristics of no aftereffect and small samples, this paper applies a Markov model to the power supply structure prediction. The optimization model is established to ensure that the model can fit the historical data as much as possible. Taking actual data of a certain area of Ningxia Province as an example, the models proposed in this paper are applied to the practice and results verify the validity and robustness of the model, which can provide decision basis for enterprise managers.

  19. Markov Random Field Surface Reconstruction

    DEFF Research Database (Denmark)

    Paulsen, Rasmus Reinhold; Bærentzen, Jakob Andreas; Larsen, Rasmus

    2010-01-01

    A method for implicit surface reconstruction is proposed. The novelty in this paper is the adaption of Markov Random Field regularization of a distance field. The Markov Random Field formulation allows us to integrate both knowledge about the type of surface we wish to reconstruct (the prior...

  20. Inventory Optimization Decision in Dual Channel Supply Chain Based On Markov Chain%基于马尔科夫链的双渠道供应链库存优化决策

    Institute of Scientific and Technical Information of China (English)

    范丹丹; 徐琪

    2014-01-01

    针对目前双渠道顾客需求不稳定带来的库存控制风险,利用顾客需求转移定性描述顾客需求不稳定,提出基于马尔科夫链的双渠道供应链库存优化模型。考虑双渠道不合作和合作两种情况,应用马尔科夫链方法,分别建立双渠道库存均衡状态概率模型和库存合作转运价格契约模型。通过Matlab算例仿真,分析渠道间顾客转移率对双渠道供应链最优库存策略的影响,得出在两个渠道顾客需求转移时,双渠道供应链合作下的库存策略要优于不合作下的库存策略。%With the inventory control risk caused by customer demand instability in the dual channel sup -ply chain considered , based on Markov chain , an inventory optimization model in dual channel supply chain is put forward by using customer demand shift to describe customer demand instability .For dual channel cooperation and non-cooperation cases , Markov theory is used to build a model of steady-state probability of inventory and the inventory cooperation transfer price contract , respectively .Then, through Matbab numerical simulation , the influence of customer shift rate between channels on optimal inventory strategy in a dual channel supply chain is analyzed .It is found that inventory strategy with dual channel supply chain cooperation is better than that without cooperation when there is a customer demand shift in the two channels .

  1. (1)H NMR z-spectra of acetate methyl in stretched hydrogels: quantum-mechanical description and Markov chain Monte Carlo relaxation-parameter estimation.

    Science.gov (United States)

    Shishmarev, Dmitry; Chapman, Bogdan E; Naumann, Christoph; Mamone, Salvatore; Kuchel, Philip W

    2015-01-01

    The (1)H NMR signal of the methyl group of sodium acetate is shown to be a triplet in the anisotropic environment of stretched gelatin gel. The multiplet structure of the signal is due to the intra-methyl residual dipolar couplings. The relaxation properties of the spin system were probed by recording steady-state irradiation envelopes ('z-spectra'). A quantum-mechanical model based on irreducible spherical tensors formed by the three magnetically equivalent spins of the methyl group was used to simulate and fit experimental z-spectra. The multiple parameter values of the relaxation model were estimated by using a Bayesian-based Markov chain Monte Carlo algorithm. PMID:25486634

  2. Reconstruction of Exposure to m-Xylene from Human Biomonitoring Data Using PBPK Modelling, Bayesian Inference, and Markov Chain Monte Carlo Simulation

    Directory of Open Access Journals (Sweden)

    Kevin McNally

    2012-01-01

    Full Text Available There are numerous biomonitoring programs, both recent and ongoing, to evaluate environmental exposure of humans to chemicals. Due to the lack of exposure and kinetic data, the correlation of biomarker levels with exposure concentrations leads to difficulty in utilizing biomonitoring data for biological guidance values. Exposure reconstruction or reverse dosimetry is the retrospective interpretation of external exposure consistent with biomonitoring data. We investigated the integration of physiologically based pharmacokinetic modelling, global sensitivity analysis, Bayesian inference, and Markov chain Monte Carlo simulation to obtain a population estimate of inhalation exposure to m-xylene. We used exhaled breath and venous blood m-xylene and urinary 3-methylhippuric acid measurements from a controlled human volunteer study in order to evaluate the ability of our computational framework to predict known inhalation exposures. We also investigated the importance of model structure and dimensionality with respect to its ability to reconstruct exposure.

  3. Deciphering the Glacial-Interglacial Landscape History in Greenland Based on Markov Chain Monte Carlo Inversion of Existing 10Be-26Al Data

    DEFF Research Database (Denmark)

    Strunk, Astrid; Knudsen, Mads Faurschou; Larsen, Nicolaj Krog;

    investigate the landscape history in eastern and western Greenland by applying a novel Markov Chain Monte Carlo (MCMC) inversion approach to the existing 10Be-26Al data from these regions. The new MCMC approach allows us to constrain the most likely landscape history based on comparisons between simulated...... the landscape history in previously glaciated terrains may be difficult, however, due to unknown erosion rates and the presence of inherited nuclides. The potential use of cosmogenic nuclides in landscapes with a complex history of exposure and erosion is therefore often quite limited. In this study, we...... and measured cosmogenic nuclide concentrations. It is a fundamental assumption of the model approach that the exposure history at the site/location can be divided into two distinct regimes: i) interglacial periods characterized by zero shielding due to overlying ice and a uniform interglacial erosion rate...

  4. Multi-Decadal Mangrove Forest Change Detection and Prediction in Honduras, Central America, with Landsat Imagery and a Markov Chain Model

    Directory of Open Access Journals (Sweden)

    Chi-Farn Chen

    2013-11-01

    Full Text Available Mangrove forests play an important role in providing ecological and socioeconomic services for human society. Coastal development, which converts mangrove forests to other land uses, has often ignored the services that mangrove may provide, leading to irreversible environmental degradation. Monitoring the spatiotemporal distribution of mangrove forests is thus critical for natural resources management of mangrove ecosystems. This study investigates spatiotemporal changes in Honduran mangrove forests using Landsat imagery during the periods 1985–1996, 1996–2002, and 2002–2013. The future trend of mangrove forest changes was projected by a Markov chain model to support decision-making for coastal management. The remote sensing data were processed through three main steps: (1 data pre-processing to correct geometric errors between the Landsat imageries and to perform reflectance normalization; (2 image classification with the unsupervised Otsu’s method and change detection; and (3 mangrove change projection using a Markov chain model. Validation of the unsupervised Otsu’s method was made by comparing the classification results with the ground reference data in 2002, which yielded satisfactory agreement with an overall accuracy of 91.1% and Kappa coefficient of 0.82. When examining mangrove changes from 1985 to 2013, approximately 11.9% of the mangrove forests were transformed to other land uses, especially shrimp farming, while little effort (3.9% was applied for mangrove rehabilitation during this 28-year period. Changes in the extent of mangrove forests were further projected until 2020, indicating that the area of mangrove forests could be continuously reduced by 1,200 ha from 2013 (approximately 36,700 ha to 2020 (approximately 35,500 ha. Institutional interventions should be taken for sustainable management of mangrove ecosystems in this coastal region.

  5. Insight into earthquake sequencing: analysis and interpretation of time-series constructed from the directed graph of the Markov chain model

    Directory of Open Access Journals (Sweden)

    M. S. Cavers

    2015-02-01

    Full Text Available Directed graph representation of a Markov chain model to study global earthquake sequencing leads to a time-series of state-to-state transition probabilities that includes the spatio-temporally linked recurrent events in the record-breaking sense. A state refers to a configuration comprised of zones with either the occurrence or non-occurrence of an earthquake in each zone in a pre-determined time interval. Since the time-series is derived from non-linear and non-stationary earthquake sequencing, we use known analysis methods to glean new information. We apply decomposition procedures such as ensemble empirical mode decomposition (EEMD to study the state-to-state fluctuations in each of the intrinsic mode functions. We subject the intrinsic mode functions, the orthogonal basis set derived from the time-series using the EEMD, to a detailed analysis to draw information-content of the time-series. Also, we investigate the influence of random-noise on the data-driven state-to-state transition probabilities. We consider a second aspect of earthquake sequencing that is closely tied to its time-correlative behavior. Here, we extend the Fano factor and Allan factor analysis to the time-series of state-to state transition frequencies of a Markov chain. Our results support not only the usefulness the intrinsic mode functions in understanding the time-series but also the presence of power-law behaviour exemplified by the Fano factor and the Allan factor.

  6. Twelve years of succession on sandy substrates in a post-mining landscape: a Markov chain analysis.

    Science.gov (United States)

    Baasch, Annett; Tischew, Sabine; Bruelheide, Helge

    2010-06-01

    Knowledge of succession rates and pathways is crucial for devising restoration strategies for highly disturbed ecosystems such as surface-mined land. As these processes have often only been described in qualitative terms, we used Markov models to quantify transitions between successional stages. However, Markov models are often considered not attractive for some reasons, such as model assumptions (e.g., stationarity in space and time, or the high expenditure of time required to estimate successional transitions in the field). Here we present a solution for converting multivariate ecological time series into transition matrices and demonstrate the applicability of this approach for a data set that resulted from monitoring the succession of sandy dry grassland in a post-mining landscape. We analyzed five transition matrices, four one-step matrices referring to specific periods of transition (1995-1998, 1998-2001, 2001-2004, 2004-2007), and one matrix for the whole study period (stationary model, 1995-2007). Finally, the stationary model was enhanced to a partly time-variable model. Applying the stationary and the time-variable models, we started a prediction well outside our calibration period, beginning with 100% bare soil in 1974 as the known start of the succession, and generated the coverage of 12 predefined vegetation types in three-year intervals. Transitions among vegetation types changed significantly in space and over time. While the probability of colonization was almost constant over time, the replacement rate tended to increase, indicating that the speed of succession accelerated with time or fluctuations became stronger. The predictions of both models agreed surprisingly well with the vegetation data observed more than two decades later. This shows that our dry grassland succession in a post-mining landscape can be adequately described by comparably simple types of Markov models, although some model assumptions have not been fulfilled and within

  7. 图上的随机动力系统及其诱导的马尔可夫链%Markov Chain Induced by Random Dynamical System on Graph

    Institute of Scientific and Technical Information of China (English)

    郑洁; 刘朝阳

    2008-01-01

    In this paper,we define a model of random dynamical systems(RDS)on graphs and prove that they are actually homogeneous discrete-time Markov chains.Moreover,a necessary and sufficient condition is obtained for that two state vectors can communicate with each other in a random dynamical system(RDS).

  8. 灰色加权马氏链组合在航空装备事故预测中的应用%Grey Weighted Markov Chains Combination Method and Application in Aviation Equipment Accident Prediction

    Institute of Scientific and Technical Information of China (English)

    甘旭升; 李华平; 高建国

    2014-01-01

    为提高传统灰色马氏链组合模型的航空装备事故预测水平,引入加权马氏链,提出了一种改进的灰色马氏链组合预测方法。该方法先通过建立灰色模型,提取事故序列的趋势信息,然后再利用灰色残余信息构建加权马氏链模型,合理发挥各步长马氏链的作用,以期准确刻画随机波动规律。为验证其有效性,在美国空军A级飞行事故万时率实际数据基础上,建立了灰色加权马氏链组合预测模型,结果表明,模型对2000年~2002年预测的相对误差平均控制在4.59%以内,远高于灰色叠加马氏链模型,所建的模型能够比较客观地反映航空安全的未来发展现状。%To improve the prediction level of traditional Grey Markov chains model in aviation accident,an improved Grey Markov chains combination prediction method based on Weighted Markov chains is proposed. In the method,Grey model is firstly established to extract the trend information from accident data series,and then weighted Markov chains model for the residual information of Grey model is constructed to reasonably play the role of various step-size Markov chains for describing the stochastic fluctuation law. To prove its validity,on the basis of a class flight accident 10,000-hour-rate actual data of USAF,Grey weight Markov chains combination model is established. The result shows that the average relative error of prediction of model during 2000 to 2002 is 4.59%,well above that of Grey stacked Markov chains model. The built model can objectively reflect the future development tendency of aviation safety.

  9. Web User Clustering and Personalized Recommendation Based on Mixtures of Hidden Markov Chain Models%基于混合隐Markov链浏览模型的WEB用户聚类与个性化推荐

    Institute of Scientific and Technical Information of China (English)

    林文龙; 刘业政; 朱庆生; 奚冬芹

    2009-01-01

    针对传统的Markov链模型不能有效的表征长串访问序列所蕴含的丰富的用户行为特征(用户类别特征、访问兴趣迁移特征)的缺点,提出混合隐Markov链浏览模型.混合隐Markov链模型使用多个不同的模型来区分不同类别用户的浏览特征,并为每个类别的用户设置了能跟踪捕捉其访问兴趣变化的类隐Markov链模型,能更好地对WWW长串访问序列的复杂特征进行建模,在真实WWW站点访问日志数据上的用户聚类实验与个性化推荐实验的结果表明,混合隐Markov链模型与传统的Markov链模型相比,具有更理想的聚类性能和推荐性能.%Since the Markov Chain Model can not denote the abundant users' behavioral characteristics(such as: characteristics of users' type, characteristics of users' interests transfer ) of a long access sequence effectively, the Mixtures of Hidden Markov Chain Models is proposed. Mixtures of Hidden Markov Chain Models use different models to distinguish the browsing categories of users from different types, and set a Hidden Markov Chain Models (can track and catch the changes of users' interests) for each users' type. Mixtures of Hidden Markov Chain Models can model the complex characteristics of the WWW long access sequences better. The results of users clustering experiment and personalized recommendation experiment with a real WWW web access log data show that Mixtures of Hidden Markov Chain Models have more perfect clustering and recommendation performance than Markov Chain Model.

  10. Semi-Markov Graph Dynamics

    CERN Document Server

    Raberto, Marco; Scalas, Enrico

    2011-01-01

    In this paper, we outline a model of graph (or network) dynamics based on two ingredients. The first ingredient is a Markov chain on the space of possible graphs. The second ingredient is a semi-Markov counting process of renewal type. The model consists in subordinating the Markov chain to the semi-Markov counting process. In simple words, this means that the chain transitions occur at random time instants called epochs. The model is quite rich and its possible connections with algebraic geometry are briefly discussed. Moreover, for the sake of simplicity, we focus on the space of undirected graphs with a fixed number of nodes. However, in an example, we present an interbank market model where it is meaningful to use directed graphs or even weighted graphs.

  11. Pair and triplet approximation of a spatial lattice population model with multiscale dispersal using Markov chains for estimating spatial autocorrelation.

    Science.gov (United States)

    Hiebeler, David E; Millett, Nicholas E

    2011-06-21

    We investigate a spatial lattice model of a population employing dispersal to nearest and second-nearest neighbors, as well as long-distance dispersal across the landscape. The model is studied via stochastic spatial simulations, ordinary pair approximation, and triplet approximation. The latter method, which uses the probabilities of state configurations of contiguous blocks of three sites as its state variables, is demonstrated to be greatly superior to pair approximations for estimating spatial correlation information at various scales. Correlations between pairs of sites separated by arbitrary distances are estimated by constructing spatial Markov processes using the information from both approximations. These correlations demonstrate why pair approximation misses basic qualitative features of the model, such as decreasing population density as a large proportion of offspring are dropped on second-nearest neighbors, and why triplet approximation is able to include them. Analytical and numerical results show that, excluding long-distance dispersal, the initial growth rate of an invading population is maximized and the equilibrium population density is also roughly maximized when the population spreads its offspring evenly over nearest and second-nearest neighboring sites.

  12. Climate change risks and adaptation options across Australian seafood supply chains – A preliminary assessment

    Directory of Open Access Journals (Sweden)

    A. Fleming

    2014-01-01

    Full Text Available Climate change is already impacting the biology of the oceans and some dependent industries are in turn responding to these impacts. The development of response options for users of marine resources, such as fishers, is important in guiding adaptation efforts. However, harvesting fish is only the first step in a supply chain that delivers seafood to consumers. Impacts higher up the chain have seldom been considered in fisheries-climate research yet an understanding of these impacts and how climate risks and adaptation information are interpreted and used by stakeholders across the chain is vital for developing viable and sustainable adaptation options. We examined stakeholder perceptions of points where climate change impacts and adaptations currently occur, or may occur in the future, across the supply chains of several Australian fisheries (southern rock lobster, tropical rock lobster, prawn and aquaculture sectors (oyster, aquaculture prawn. We found that climate change impacts are well understood at the harvest stage and there is evidence of potential impacts and disruption to supply chains. Yet, there currently is no strong driver for change higher up the chain. Holistic adaptation planning along the supply chain, underpinned by targeted information and policy for the catch, processing and distribution, and marketing phases is needed. This effort is needed now, as some adaptation options have long lead times, and a delay in adaptation planning may limit future options. Given potential lead times and associated uncertainty, a risk-based approach is recommended with regard to adaptation planning for Australia’s seafood sector.

  13. Masquerade detection based on shell commands and Markov chain models%基于shell命令和Markov链模型的用户伪装攻击检测

    Institute of Scientific and Technical Information of China (English)

    肖喜; 田新广; 翟起滨; 叶润国

    2011-01-01

    提出一种新的基于shell命令的用户伪装攻击检测方法.该方法在训练阶段充分考虑了用户行为的多变性和伪装攻击的特点,采用平稳的齐次Markov链对合法用户的正常行为进行建模,根据shell命令的出现频率进行阶梯式数据归并来划分状态,同现有的Markov链方法相比大幅度减少了状态个数和转移概率矩阵的存储量,提高了泛化能力.针对检测实时性需求和shell命令操作的短时相关性,采用了基于频率优先的状态匹配方法,并通过对状态短序列的出现概率进行加窗平滑滤噪处理来计算判决值,能够有效减少系统计算开销,降低误报率.实验表明,该方法具有很高的检测准确率和较强的可操作性,特别适用于在线检测.%A novel method for masquerade attack detection based on shell commands was proposed. At the training stage,the variability of users' behavior and the feature of masquerade attack were thoroughly considered, and stationary homogeneous Markov chains were employed to profile the normal users' behavior. The shell commands were gradationally merged into multiple sets according to their frequencies and then states were constructed accordingly, which significantly reduced the number of states and the memory of the transition probability matrix and improved the generalization of the detection system, compared with existing Markov chain methods. Considering the real-time detection demand and the short-time relevance of shell commands, the states were matched with a high-frequency-first scheme at the detection stage, and the decision measure was computed by smoothing the probabilities of short state sequences. This decreased computational complexity and the false-alarm rate. Experimental results indicate that our method can achieve high detection accuracy and practicability, and is especially applicable for on-line detection.

  14. Application and Evaluation of a Snowmelt Runoff Model in the Tamor River Basin, Eastern Himalaya Using a Markov Chain Monte Carlo (MCMC) Data Assimilation Approach

    Science.gov (United States)

    Panday, Prajjwal K.; Williams, Christopher A.; Frey, Karen E.; Brown, Molly E.

    2013-01-01

    Previous studies have drawn attention to substantial hydrological changes taking place in mountainous watersheds where hydrology is dominated by cryospheric processes. Modelling is an important tool for understanding these changes but is particularly challenging in mountainous terrain owing to scarcity of ground observations and uncertainty of model parameters across space and time. This study utilizes a Markov Chain Monte Carlo data assimilation approach to examine and evaluate the performance of a conceptual, degree-day snowmelt runoff model applied in the Tamor River basin in the eastern Nepalese Himalaya. The snowmelt runoff model is calibrated using daily streamflow from 2002 to 2006 with fairly high accuracy (average Nash-Sutcliffe metric approx. 0.84, annual volume bias <3%). The Markov Chain Monte Carlo approach constrains the parameters to which the model is most sensitive (e.g. lapse rate and recession coefficient) and maximizes model fit and performance. Model simulated streamflow using an interpolated precipitation data set decreases the fractional contribution from rainfall compared with simulations using observed station precipitation. The average snowmelt contribution to total runoff in the Tamor River basin for the 2002-2006 period is estimated to be 29.7+/-2.9% (which includes 4.2+/-0.9% from snowfall that promptly melts), whereas 70.3+/-2.6% is attributed to contributions from rainfall. On average, the elevation zone in the 4000-5500m range contributes the most to basin runoff, averaging 56.9+/-3.6% of all snowmelt input and 28.9+/-1.1% of all rainfall input to runoff. Model simulated streamflow using an interpolated precipitation data set decreases the fractional contribution from rainfall versus snowmelt compared with simulations using observed station precipitation. Model experiments indicate that the hydrograph itself does not constrain estimates of snowmelt versus rainfall contributions to total outflow but that this derives from the degree

  15. Reliability Measurement for Software with Variable Length Markov Control Transfer Chain%VLMC控制流软件可靠性度量方法

    Institute of Scientific and Technical Information of China (English)

    赵瑞曼; 潘冠华; 方建勇

    2014-01-01

    The classical software reliability model of Cheung based on architecture analysis is imperfect when applied to soft-ware which has non-uniqueness terminal node because of ideal assumptions. In order to handle this problem, an improved model is established through two improvement as follows:first, probability matrix of control transfer from start node to other inner nodes in a given software is computed independently of control transfer within inner nodes on the assumptions that the start node is inaccessible to control transfer from any inner nodes;and then, probability matrixes of control transfer within in-ner nodes are reconfigured by weaken the difference between inner nodes and ending nodes. In order to measure the reliability of software with VLMC control transfer flow, the variable length Markov control sequence chain is converted to simple Markov chain by node expansion and conditional transfer probability redistribution. By using deductive inference, a formal proof for the new model is given in this essay. In the end, effectiveness and simplicity of the proposed method is verified by a software example.%针对现有软件可靠性模型普遍不适用于实际软件的问题,分析了软件内部模块间控制转移机理,通过分离入口模块,提取一阶控制转移概率矩阵;弱化内部模块与出口模块间差异,重构二阶以上转移概率矩阵,在改进Cheung模型的基础上建立一个更符合软件实际的可靠性度量模型。针对变阶依赖( VLMC)控制转移导致的可靠性度量难问题,通过对导致复杂依赖的多入多出模块进行节点扩展,将VLMC控制流转化为Markov链,利用所建立的软件可靠性模型对VLMC控制流软件进行可靠性度量。研究利用演绎推理对所建立模型进行了正确性形式化证明。最后给出了方法的实例验证。

  16. Characterization of environmental quality of forest fragments changes in Jundiaí-Mirim river basin-Brazil using the Markov Chain model

    Science.gov (United States)

    Hasimoto Fengler, Felipe; Leite de Moraes, Jener Fernando; Irio Ribeiro, Admilson; Peche Filho, Afonso; Araujo de Medeiros, Gerson; Baldin Damame, Desirée; Márcia Longo, Regina

    2015-04-01

    In Brazil is common practice the concurrency of large urban centers water catchment in distant sites. There's no policy to preserve strategic springs in the urban territory. Thus, rural areas, located in the surrounds of municipals, usually provide water and others environment services to the population that reside on cities. The Jundiaí-Mirim river basin, located in the most urbanized state in Brazil, São Paulo, composes an interesting example of this situation. It is located in a rural area near large urban centers, with large industrial parks, near the capital of state. As result of expansion of the cities on its surrounds their lands have had a historic of monetary valorization, making its territories attractive to the housing market. Consequently, the region has an intense process of urbanization that resulted in an increasing environmental disturbance in the areas of natural vegetation. In the other hand, the watershed is the principal water supplier of Jundiaí city, and houses forest remaining of an important Biome in Brazil, the Atlantic Rain Forest. Given the need to preserve its water production capacity and the forest remnants there, this study modeled the environmental quality of forest fragments through indicators of disturbance and evaluated the changes that occur between 1972 and 2013 using the Markov Chain model. The environment quality was determined by nine indicators of environmental disturbance (distance of urban areas, roads, edge land use, size, distance of others forest fragments, land capacity of use, watershed forest cover, number of forest fragments in the watersheds, shape of the forest fragment), obtained by techniques of Geoprocessing, and integrated by Multicriteria Analysis. The Markov Chain model showed a constant tendency of deteriorating in natural vegetation environmental quality, attributed to the intense process of occupation of the river basin. The results showed a historical trend of transformation in forest fragments with

  17. Accelerating Monte Carlo molecular simulations by reweighting and reconstructing Markov chains: Extrapolation of canonical ensemble averages and second derivatives to different temperature and density conditions

    KAUST Repository

    Kadoura, Ahmad Salim

    2014-08-01

    Accurate determination of thermodynamic properties of petroleum reservoir fluids is of great interest to many applications, especially in petroleum engineering and chemical engineering. Molecular simulation has many appealing features, especially its requirement of fewer tuned parameters but yet better predicting capability; however it is well known that molecular simulation is very CPU expensive, as compared to equation of state approaches. We have recently introduced an efficient thermodynamically consistent technique to regenerate rapidly Monte Carlo Markov Chains (MCMCs) at different thermodynamic conditions from the existing data points that have been pre-computed with expensive classical simulation. This technique can speed up the simulation more than a million times, making the regenerated molecular simulation almost as fast as equation of state approaches. In this paper, this technique is first briefly reviewed and then numerically investigated in its capability of predicting ensemble averages of primary quantities at different neighboring thermodynamic conditions to the original simulated MCMCs. Moreover, this extrapolation technique is extended to predict second derivative properties (e.g. heat capacity and fluid compressibility). The method works by reweighting and reconstructing generated MCMCs in canonical ensemble for Lennard-Jones particles. In this paper, system\\'s potential energy, pressure, isochoric heat capacity and isothermal compressibility along isochors, isotherms and paths of changing temperature and density from the original simulated points were extrapolated. Finally, an optimized set of Lennard-Jones parameters (ε, σ) for single site models were proposed for methane, nitrogen and carbon monoxide. © 2014 Elsevier Inc.

  18. Accelerating Monte Carlo molecular simulations by reweighting and reconstructing Markov chains: Extrapolation of canonical ensemble averages and second derivatives to different temperature and density conditions

    Energy Technology Data Exchange (ETDEWEB)

    Kadoura, Ahmad; Sun, Shuyu, E-mail: shuyu.sun@kaust.edu.sa; Salama, Amgad

    2014-08-01

    Accurate determination of thermodynamic properties of petroleum reservoir fluids is of great interest to many applications, especially in petroleum engineering and chemical engineering. Molecular simulation has many appealing features, especially its requirement of fewer tuned parameters but yet better predicting capability; however it is well known that molecular simulation is very CPU expensive, as compared to equation of state approaches. We have recently introduced an efficient thermodynamically consistent technique to regenerate rapidly Monte Carlo Markov Chains (MCMCs) at different thermodynamic conditions from the existing data points that have been pre-computed with expensive classical simulation. This technique can speed up the simulation more than a million times, making the regenerated molecular simulation almost as fast as equation of state approaches. In this paper, this technique is first briefly reviewed and then numerically investigated in its capability of predicting ensemble averages of primary quantities at different neighboring thermodynamic conditions to the original simulated MCMCs. Moreover, this extrapolation technique is extended to predict second derivative properties (e.g. heat capacity and fluid compressibility). The method works by reweighting and reconstructing generated MCMCs in canonical ensemble for Lennard-Jones particles. In this paper, system's potential energy, pressure, isochoric heat capacity and isothermal compressibility along isochors, isotherms and paths of changing temperature and density from the original simulated points were extrapolated. Finally, an optimized set of Lennard-Jones parameters (ε, σ) for single site models were proposed for methane, nitrogen and carbon monoxide.

  19. Study of Disease Progression and Relevant Risk Factors in Diabetic Foot Patients Using a Multistate Continuous-Time Markov Chain Model.

    Science.gov (United States)

    Begun, Alexander; Morbach, Stephan; Rümenapf, Gerhard; Icks, Andrea

    2016-01-01

    The diabetic foot is a lifelong disease. The longer patients with diabetes and foot ulcers are observed, the higher the likelihood that they will develop comorbidities that adversely influence ulcer recurrence, amputation and survival (for example peripheral arterial disease, renal failure or ischaemic heart disease). The purpose of our study was to quantify person and limb-related disease progression and the time-dependent influence of any associated factors (present at baseline or appearing during observation) based on which effective prevention and/or treatment strategies could be developed. Using a nine-state continuous-time Markov chain model with time-dependent risk factors, all living patients were divided into eight groups based on their ulceration (previous or current) and previous amputation (none, minor or major) status. State nine is an absorbing state (death). If all transitions are fully observable, this model can be decomposed into eight submodels, which can be analyzed using the methods of survival analysis for competing risks. The dependencies of the risk factors (covariates) were included in the submodels using Cox-like regression. The transition intensities and relative risks for covariates were calculated from long-term data of patients with diabetic foot ulcers collected in a single specialized center in North-Rhine Westphalia (Germany). The detected estimates were in line with previously published, but scarce, data. Together with the interesting new results obtained, this indicates that the proposed model may be useful for studying disease progression in larger samples of patients with diabetic foot ulcers.

  20. New Product Development in an Emerging Economy: Analysing the Role of Supplier Involvement Practices by Using Bayesian Markov Chain Monte Carlo Technique

    Directory of Open Access Journals (Sweden)

    Kanagi Kanapathy

    2014-01-01

    Full Text Available The research question is whether the positive relationship found between supplier involvement practices and new product development performances in developed economies also holds in emerging economies. The role of supplier involvement practices in new product development performance is yet to be substantially investigated in the emerging economies (other than China. This premise was examined by distributing a survey instrument (Jayaram’s (2008 published survey instrument that has been utilised in developed economies to Malaysian manufacturing companies. To gauge the relationship between the supplier involvement practices and new product development (NPD project performance of 146 companies, structural equation modelling was adopted. Our findings prove that supplier involvement practices have a significant positive impact on NPD project performance in an emerging economy with respect to quality objectives, design objectives, cost objectives, and “time-to-market” objectives. Further analysis using the Bayesian Markov Chain Monte Carlo algorithm, yielding a more credible and feasible differentiation, confirmed these results (even in the case of an emerging economy and indicated that these practices have a 28% impact on variance of NPD project performance. This considerable effect implies that supplier involvement is a must have, although further research is needed to identify the contingencies for its practices.

  1. Probabilistic Forecasting of Drought Events Using Markov Chain- and Bayesian Network-Based Models: A Case Study of an Andean Regulated River Basin

    Directory of Open Access Journals (Sweden)

    Alex Avilés

    2016-01-01

    Full Text Available The scarcity of water resources in mountain areas can distort normal water application patterns with among other effects, a negative impact on water supply and river ecosystems. Knowing the probability of droughts might help to optimize a priori the planning and management of the water resources in general and of the Andean watersheds in particular. This study compares Markov chain- (MC and Bayesian network- (BN based models in drought forecasting using a recently developed drought index with respect to their capability to characterize different drought severity states. The copula functions were used to solve the BNs and the ranked probability skill score (RPSS to evaluate the performance of the models. Monthly rainfall and streamflow data of the Chulco River basin, located in Southern Ecuador, were used to assess the performance of both approaches. Global evaluation results revealed that the MC-based models predict better wet and dry periods, and BN-based models generate slightly more accurately forecasts of the most severe droughts. However, evaluation of monthly results reveals that, for each month of the hydrological year, either the MC- or BN-based model provides better forecasts. The presented approach could be of assistance to water managers to ensure that timely decision-making on drought response is undertaken.

  2. Trend analysis of public opinion based on Markov chain%基于马尔可夫链的舆情热度趋势分析

    Institute of Scientific and Technical Information of China (English)

    刘勘; 李晶; 刘萍

    2011-01-01

    Public opinion often has a hidden trend behind its characteristics shown online.A Markov chain based public opin ion analysis method is proposed.In this method, indicator data of public opinion are collected to get the value of time series for the trend.Then the trend changes and the state spaces are analyzed, and the state transition matrix for the final trend is built.Experiments demonstrate that this method can effectively predict the trend of focus public opinion, and be of signifi cance in guiding and controlling the public opinion.%针对舆情热度的波动范围较大,并会呈现某种趋势的特点,提出了一种基于马尔可夫链的舆情热度趋势分析模型.该模型采集相关热点舆情的指标数据,得到热度的时间序列值;分析热度的趋势变化,划分状态空间,构建状态转移矩阵,预测热度的趋势变化区间.实验表明,该方法能有效地预测热点舆情的走势,进而辅助对舆情的引导和控制.

  3. Aeolus: A Markov--Chain Monte Carlo code for mapping ultracool atmospheres. An application on Jupiter and brown dwarf HST light curves

    CERN Document Server

    Karalidi, Theodora; Schneider, Glenn; Hanson, Jake R; Pasachoff, Jay M

    2015-01-01

    Deducing the cloud cover and its temporal evolution from the observed planetary spectra and phase curves can give us major insight into the atmospheric dynamics. In this paper, we present Aeolus, a Markov-Chain Monte Carlo code that maps the structure of brown dwarf and other ultracool atmospheres. We validated Aeolus on a set of unique Jupiter Hubble Space Telescope (HST) light curves. Aeolus accurately retrieves the properties of the major features of the jovian atmosphere such as the Great Red Spot and a major 5um hot spot. Aeolus is the first mapping code validated on actual observations of a giant planet over a full rotational period. For this study, we applied Aeolus to J and H-bands HST light curves of 2MASSJ21392676+0220226 and 2MASSJ0136565+093347. Aeolus retrieves three spots at the top-of-the-atmosphere (per observational wavelength) of these two brown dwarfs, with a surface coverage of 21+-3% and 20.3+-1.5% respectively. The Jupiter HST light curves will be publicly available via ADS/VIZIR.

  4. Assessment of myocardial metabolic rate of glucose by means of Bayesian ICA and Markov Chain Monte Carlo methods in small animal PET imaging

    Science.gov (United States)

    Berradja, Khadidja; Boughanmi, Nabil

    2016-09-01

    In dynamic cardiac PET FDG studies the assessment of myocardial metabolic rate of glucose (MMRG) requires the knowledge of the blood input function (IF). IF can be obtained by manual or automatic blood sampling and cross calibrated with PET. These procedures are cumbersome, invasive and generate uncertainties. The IF is contaminated by spillover of radioactivity from the adjacent myocardium and this could cause important error in the estimated MMRG. In this study, we show that the IF can be extracted from the images in a rat heart study with 18F-fluorodeoxyglucose (18F-FDG) by means of Independent Component Analysis (ICA) based on Bayesian theory and Markov Chain Monte Carlo (MCMC) sampling method (BICA). Images of the heart from rats were acquired with the Sherbrooke small animal PET scanner. A region of interest (ROI) was drawn around the rat image and decomposed into blood and tissue using BICA. The Statistical study showed that there is a significant difference (p < 0.05) between MMRG obtained with IF extracted by BICA with respect to IF extracted from measured images corrupted with spillover.

  5. A comparison between Gauss-Newton and Markov chain Monte Carlo basedmethods for inverting spectral induced polarization data for Cole-Coleparameters

    Energy Technology Data Exchange (ETDEWEB)

    Chen, Jinsong; Kemna, Andreas; Hubbard, Susan S.

    2008-05-15

    We develop a Bayesian model to invert spectral induced polarization (SIP) data for Cole-Cole parameters using Markov chain Monte Carlo (MCMC) sampling methods. We compare the performance of the MCMC based stochastic method with an iterative Gauss-Newton based deterministic method for Cole-Cole parameter estimation through inversion of synthetic and laboratory SIP data. The Gauss-Newton based method can provide an optimal solution for given objective functions under constraints, but the obtained optimal solution generally depends on the choice of initial values and the estimated uncertainty information is often inaccurate or insufficient. In contrast, the MCMC based inversion method provides extensive global information on unknown parameters, such as the marginal probability distribution functions, from which we can obtain better estimates and tighter uncertainty bounds of the parameters than with the deterministic method. Additionally, the results obtained with the MCMC method are independent of the choice of initial values. Because the MCMC based method does not explicitly offer single optimal solution for given objective functions, the deterministic and stochastic methods can complement each other. For example, the stochastic method can first be used to obtain the means of the unknown parameters by starting from an arbitrary set of initial values and the deterministic method can then be initiated using the means as starting values to obtain the optimal estimates of the Cole-Cole parameters.

  6. ELUCID - Exploring the Local Universe with reConstructed Initial Density field I: Hamiltonian Markov Chain Monte Carlo Method with Particle Mesh Dynamics

    CERN Document Server

    Wang, Huiyuan; Yang, Xiaohu; Jing, Y P; Lin, W P

    2014-01-01

    Simulating the evolution of the local universe is important for studying galaxies and the intergalactic medium in a way free of cosmic variance. Here we present a method to reconstruct the initial linear density field from an input non-linear density field, employing the Hamiltonian Markov Chain Monte Carlo (HMC) algorithm combined with Particle Mesh (PM) dynamics. The HMC+PM method is applied to cosmological simulations, and the reconstructed linear density fields are then evolved to the present day with N-body simulations. The constrained simulations so obtained accurately reproduce both the amplitudes and phases of the input simulations at various $z$. Using a PM model with a grid cell size of 0.75 Mpc/h and 40 time-steps in the HMC can recover more than half of the phase information down to a scale k~0.85 h/Mpc at high z and to k~3.4 h/Mpc at z=0, which represents an improvement of a factor of two to three in spatial scale over similar reconstruction models in the literature, and indicates that our model ...

  7. Analysis of transtheoretical model of health behavioral changes in a nutrition intervention study--a continuous time Markov chain model with Bayesian approach.

    Science.gov (United States)

    Ma, Junsheng; Chan, Wenyaw; Tsai, Chu-Lin; Xiong, Momiao; Tilley, Barbara C

    2015-11-30

    Continuous time Markov chain (CTMC) models are often used to study the progression of chronic diseases in medical research but rarely applied to studies of the process of behavioral change. In studies of interventions to modify behaviors, a widely used psychosocial model is based on the transtheoretical model that often has more than three states (representing stages of change) and conceptually permits all possible instantaneous transitions. Very little attention is given to the study of the relationships between a CTMC model and associated covariates under the framework of transtheoretical model. We developed a Bayesian approach to evaluate the covariate effects on a CTMC model through a log-linear regression link. A simulation study of this approach showed that model parameters were accurately and precisely estimated. We analyzed an existing data set on stages of change in dietary intake from the Next Step Trial using the proposed method and the generalized multinomial logit model. We found that the generalized multinomial logit model was not suitable for these data because it ignores the unbalanced data structure and temporal correlation between successive measurements. Our analysis not only confirms that the nutrition intervention was effective but also provides information on how the intervention affected the transitions among the stages of change. We found that, compared with the control group, subjects in the intervention group, on average, spent substantively less time in the precontemplation stage and were more/less likely to move from an unhealthy/healthy state to a healthy/unhealthy state. PMID:26123093

  8. Markov chains-cellular automata modeling and multicriteria analysis of land cover change in the Lower Nhecolândia subregion of the Brazilian Pantanal wetland

    Science.gov (United States)

    Bacani, Vitor Matheus; Sakamoto, Arnaldo Yoso; Quénol, Hervé; Vannier, Clémence; Corgne, Samuel

    2016-01-01

    The dynamics of land use/land cover change in the Lower Nhecolândia wetland are marked by deforestation for pasture expansion, resulting in a real threat to the ecological stability. The aim of our work was to analyze the spatial distribution of land cover changes in the Lower Nhecolândia from 1985 to 2013 and to predict changes in trends for 2040. The mapping of land cover changes was developed using Landsat satellite images of 1985, 1999, 2007, and 2013, based on geographic object-based image analysis approach. This study uses integrated Markov chains and cellular automata modeling and multicriteria evaluation techniques to produce transition probability maps and describe the trajectory analysis methodology to construct a continuity of spatial and temporal changes for the wetland. The results of the multitemporal change detection classification show that, from 1985 to 2013, the forest woodland decreased by 6.89% and the grassland class increased by 18.29%. On the other hand, all water bodies showed a reducing trend, while the bare soil class increased compared to 1985, but did not present a regular trend of increase or decrease. From the present day, the trend for the future is a reduction of almost 6.4% by 2040. We found that deforestation actions will be concentrated in the areas with the highest concentration of saline lakes, constituting a serious threat to the natural functioning of this environmental system.

  9. Zipf exponent of trajectory distribution in the hidden Markov model

    Science.gov (United States)

    Bochkarev, V. V.; Lerner, E. Yu

    2014-03-01

    This paper is the first step of generalization of the previously obtained full classification of the asymptotic behavior of the probability for Markov chain trajectories for the case of hidden Markov models. The main goal is to study the power (Zipf) and nonpower asymptotics of the frequency list of trajectories of hidden Markov frequencys and to obtain explicit formulae for the exponent of the power asymptotics. We consider several simple classes of hidden Markov models. We prove that the asymptotics for a hidden Markov model and for the corresponding Markov chain can be essentially different.

  10. Implementation of a Markov Chain Monte Carlo method to inorganic aerosol modeling of observations from the MCMA-2003 campaign ? Part II: Model application to the CENICA, Pedregal and Santa Ana sites

    OpenAIRE

    San Martini, F. M.; E. J. Dunlea; R. Volkamer; Onasch, T. B.; J. T. Jayne; Canagaratna, M. R.; Worsnop, D. R.; C. E. Kolb; J. H. Shorter; S. C. Herndon; M. S. Zahniser; D. Salcedo; Dzepina, K.; Jimenez, J. L; Ortega, J. M.

    2006-01-01

    A Markov Chain Monte Carlo model for integrating the observations of inorganic species with a thermodynamic equilibrium model was presented in Part I of this series. Using observations taken at three ground sites, i.e. a residential, industrial and rural site, during the MCMA-2003 campaign in Mexico City, the model is used to analyze the inorganic particle and ammonia data and to predict gas phase concentrations of nitric and hydrochloric acid. In general, the model is able to accurately pred...

  11. Implementation of a Markov Chain Monte Carlo method to inorganic aerosol modeling of observations from the MCMA-2003 campaign – Part II: Model application to the CENICA, Pedregal and Santa Ana sites

    OpenAIRE

    San Martini, F. M.; Dunlea, E. J.; R. Volkamer; Onasch, T. B.; Jayne, J. T.; Canagaratna, M. R.; Worsnop, D. R.; Kolb, C. E.; Shorter, J. H.; Herndon, S. C.; Zahniser, M. S.; D. Salcedo; Dzepina, K.; Jimenez, J. L.; Ortega, J. M.

    2006-01-01

    A Markov Chain Monte Carlo model for integrating the observations of inorganic species with a thermodynamic equilibrium model was presented in Part I of this series. Using observations taken at three ground sites, i.e. a residential, industrial and rural site, during the MCMA-2003 campaign in Mexico City, the model is used to analyze the inorganic particle and ammonia data and to predict gas phase concentrations of nitric and hydrochloric acid. In general, the mode...

  12. Bayesian Gibbs Markov chain: MRF-based Stochastic Joint Inversion of Hydrological and Geophysical Datasets for Improved Characterization of Aquifer Heterogeneities.

    Science.gov (United States)

    Oware, E. K.

    2015-12-01

    Modeling aquifer heterogeneities (AH) is a complex, multidimensional problem that mostly requires stochastic imaging strategies for tractability. While the traditional Bayesian Markov chain Monte Carlo (McMC) provides a powerful framework to model AH, the generic McMC is computationally prohibitive and, thus, unappealing for large-scale problems. An innovative variant of the McMC scheme that imposes priori spatial statistical constraints on model parameter updates, for improved characterization in a computationally efficient manner is proposed. The proposed algorithm (PA) is based on Markov random field (MRF) modeling, which is an image processing technique that infers the global behavior of a random field from its local properties, making the MRF approach well suited for imaging AH. MRF-based modeling leverages the equivalence of Gibbs (or Boltzmann) distribution (GD) and MRF to identify the local properties of an MRF in terms of the easily quantifiable Gibbs energy. The PA employs the two-step approach to model the lithological structure of the aquifer and the hydraulic properties within the identified lithologies simultaneously. It performs local Gibbs energy minimizations along a random path, which requires parameters of the GD (spatial statistics) to be specified. A PA that implicitly infers site-specific GD parameters within a Bayesian framework is also presented. The PA is illustrated with a synthetic binary facies aquifer with a lognormal heterogeneity simulated within each facies. GD parameters of 2.6, 1.2, -0.4, and -0.2 were estimated for the horizontal, vertical, NESW, and NWSE directions, respectively. Most of the high hydraulic conductivity zones (facies 2) were fairly resolved (see results below) with facies identification accuracy rate of 81%, 89%, and 90% for the inversions conditioned on concentration (R1), resistivity (R2), and joint (R3), respectively. The incorporation of the conditioning datasets improved on the root mean square error (RMSE

  13. Convergence in distribution for filtering processes associated to Hidden Markov Models with densities

    OpenAIRE

    Kaijser, Thomas

    2013-01-01

    A Hidden Markov Model generates two basic stochastic processes, a Markov chain, which is hidden, and an observation sequence. The filtering process of a Hidden Markov Model is, roughly speaking, the sequence of conditional distributions of the hidden Markov chain that is obtained as new observations are received. It is well-known, that the filtering process itself, is also a Markov chain. A classical, theoretical problem is to find conditions which implies that the distributions of the filter...

  14. A Markov-binomial distribution

    OpenAIRE

    Santos, J.; S. Van Gulck; OMEY, E.

    2007-01-01

    Let ${X_{i},igeq 1}$ denote a sequence of $left{ 0,1 ight} $%-variables and suppose that the sequence forms a {sc Markov} Chain. In the paperwe study the number of successes $S_{n}=X_{1}+X_{2}+cdots+X_{n}$ and we studythe number of experiments $Y(r)$ up to the $r$-$th$ success. In the i.i.d.case $S_{n}$ has a binomial distribution and $Y(r)$ has a negative binomialdistribution and the asymptotic behaviour is well known. In the more general{sc Markov} chain case, we prove a central limit theor...

  15. 基于优化Markov链使用模型的可靠性测试技术∗%Software Reliability Testing Method Based on Optimizated Markov Chain Usage Model

    Institute of Scientific and Technical Information of China (English)

    吴亮; 方建勇

    2015-01-01

    Markov链使用模型是一种应用较为广泛的软件可靠性测试模型,但是如何生成使用模型中的转移概率仍然是一个亟待解决的问题。传统Markov链模型中的转移概率往往依赖用户、开发者及领域专家给出一个具体的数值,是非常不合理的。因此,提出了一种使用三参数区间数表示转移概率的方法,对Markov链模型进行优化并给出可靠性测试用例的生成方法。实验表明,使用三参数区间数构造出的软件使用模型要优于传统Markov链,并可以生成较为符合实际情况的可靠性测试用例,从而有助于提高可靠性评估的准确性。%Markov chain usage model is a widely used model in software reliability testing, and how to assignment the migra⁃ting probability between states of Markov chain usage model is still a problem to be solved. In traditional Markov chain mod⁃el, the probability is often given as a fixed value by software users, developers and domain experts, and this approach is un⁃reasonable. In order to deal with this problem, a migrating probability assignment mothod based on three⁃parameter interval number is proposed to build the Markov chain usage model and the method for generating test cases is also given. As the tes⁃ting result shows, the software usage model based on the three⁃parameter interval number is better than the traditional Markov chain model and it is helpful to improve the accuracy of the software reliability assess.

  16. Finite Markov processes and their applications

    CERN Document Server

    Iosifescu, Marius

    2007-01-01

    A self-contained treatment of finite Markov chains and processes, this text covers both theory and applications. Author Marius Iosifescu, vice president of the Romanian Academy and director of its Center for Mathematical Statistics, begins with a review of relevant aspects of probability theory and linear algebra. Experienced readers may start with the second chapter, a treatment of fundamental concepts of homogeneous finite Markov chain theory that offers examples of applicable models.The text advances to studies of two basic types of homogeneous finite Markov chains: absorbing and ergodic ch

  17. Study of Disease Progression and Relevant Risk Factors in Diabetic Foot Patients Using a Multistate Continuous-Time Markov Chain Model.

    Directory of Open Access Journals (Sweden)

    Alexander Begun

    Full Text Available The diabetic foot is a lifelong disease. The longer patients with diabetes and foot ulcers are observed, the higher the likelihood that they will develop comorbidities that adversely influence ulcer recurrence, amputation and survival (for example peripheral arterial disease, renal failure or ischaemic heart disease. The purpose of our study was to quantify person and limb-related disease progression and the time-dependent influence of any associated factors (present at baseline or appearing during observation based on which effective prevention and/or treatment strategies could be developed. Using a nine-state continuous-time Markov chain model with time-dependent risk factors, all living patients were divided into eight groups based on their ulceration (previous or current and previous amputation (none, minor or major status. State nine is an absorbing state (death. If all transitions are fully observable, this model can be decomposed into eight submodels, which can be analyzed using the methods of survival analysis for competing risks. The dependencies of the risk factors (covariates were included in the submodels using Cox-like regression. The transition intensities and relative risks for covariates were calculated from long-term data of patients with diabetic foot ulcers collected in a single specialized center in North-Rhine Westphalia (Germany. The detected estimates were in line with previously published, but scarce, data. Together with the interesting new results obtained, this indicates that the proposed model may be useful for studying disease progression in larger samples of patients with diabetic foot ulcers.

  18. Study of Disease Progression and Relevant Risk Factors in Diabetic Foot Patients Using a Multistate Continuous-Time Markov Chain Model.

    Science.gov (United States)

    Begun, Alexander; Morbach, Stephan; Rümenapf, Gerhard; Icks, Andrea

    2016-01-01

    The diabetic foot is a lifelong disease. The longer patients with diabetes and foot ulcers are observed, the higher the likelihood that they will develop comorbidities that adversely influence ulcer recurrence, amputation and survival (for example peripheral arterial disease, renal failure or ischaemic heart disease). The purpose of our study was to quantify person and limb-related disease progression and the time-dependent influence of any associated factors (present at baseline or appearing during observation) based on which effective prevention and/or treatment strategies could be developed. Using a nine-state continuous-time Markov chain model with time-dependent risk factors, all living patients were divided into eight groups based on their ulceration (previous or current) and previous amputation (none, minor or major) status. State nine is an absorbing state (death). If all transitions are fully observable, this model can be decomposed into eight submodels, which can be analyzed using the methods of survival analysis for competing risks. The dependencies of the risk factors (covariates) were included in the submodels using Cox-like regression. The transition intensities and relative risks for covariates were calculated from long-term data of patients with diabetic foot ulcers collected in a single specialized center in North-Rhine Westphalia (Germany). The detected estimates were in line with previously published, but scarce, data. Together with the interesting new results obtained, this indicates that the proposed model may be useful for studying disease progression in larger samples of patients with diabetic foot ulcers. PMID:26814723

  19. 基于Markov链的FSM容软错误设计%FSM Soft Fault-Tolerant Design Based on Markov Chain Model

    Institute of Scientific and Technical Information of China (English)

    曹源; 梁华国; 黄正峰; 陈秀美; 吴珍妮

    2011-01-01

    随着深亚微米工艺的广泛应用,持续攀升的软错误率对宇航设备的电路可靠性造成了极大影响.针对现有冗余防护技术面积开销大的不足,将慨率统计的思想运用到有限状态机(FSM)的软容错设计上,提出了选择部分状态冗余的容错方法.在Markov链模型的基础上,计算出各状态概率,从而为状态冗余提供依据.与未经选择状态冗余的电路相比,以平均增加14.9%的面积开销为代价,这种电路可屏蔽87.6%的时序逻辑单翻转(SEU).%Since deep submicron technology is widely used in microprocessors , increasing soft error rate greatly affects the reliability of controllers. In this paper, the idea of statistical probability is applied to soft fault-tolerant design for finite state machine ( FSM). Based on a Markov chain model, a redundancy state selection method is proposed. Compared with the original circuit. this design can mask 87.6% single event upsel ( SEV) at the cast of additional 14.9% area overhead in average.

  20. Characterization of effective saturated hydraulic conductivity in an agricultural field using Karhunen-Loève expansion with the Markov chain Monte Carlo technique

    Science.gov (United States)

    Das, N. N.; Mohanty, B. P.; Efendiev, Y.

    2010-06-01

    Process-based soil hydrologic models require input of saturated hydraulic conductivity (Ksat). However, model users often have limited access to measured data and thus use published or estimated values for many site-specific hydrologic and environmental applications. We proposed an algorithm that uses the Karhunen-Loève expansion (KLE) in conjunction with the Markov chain Monte Carlo (MCMC) technique, which employs measured soil moisture values to characterize the saturated hydraulic conductivity of an agricultural field at a 30 m resolution. The study domain is situated in the Walnut Creek watershed, Iowa, with soybean crop (in 2005) and well-defined top (atmospheric) and bottom (groundwater) boundary conditions. The KLE algorithm parameterizes and generates Ksat fields with random correlation lengths that are used in the SWMS_3D model for predicting the soil moisture dynamics for two different scenarios: (1) the van Genuchten soil hydraulic parameters (except Ksat) are constant and are based on the soil type of the grid block within the domain, and (2) Ksat is correlated with the van Genuchten parameter α as Ksat ∝ α2. The predicted soil moisture fields for both the scenarios are evaluated with the measured soil moisture in the MCMC algorithm for acceptance (or rejection) of the Ksat fields. The accepted Ksat fields are evaluated against the laboratory-measured Ksat at specific locations as well as with a large Ksat data set measured in situ in a nearby field with similar hydrologic conditions, and the comparisons show reasonably good agreement. The KLE-MCMC algorithm was further tested in the same study domain for another year (2002) having different vegetation (corn) and local forcings. The algorithm shows potential to characterize the effective saturated hydraulic conductivity fields at 30 m resolution using inexpensive and more regularly measured soil moisture measurements. Further studies are required to incorporate variability in different

  1. Generation and simplification of software Markov chain usage model%软件Markov链使用模型生成与化简技术

    Institute of Scientific and Technical Information of China (English)

    冯俊池; 于磊; 刘洋

    2015-01-01

    为解决软件可靠性测试中 Markov链使用模型的状态空间爆炸问题,研究基于 UML 模型的使用模型生成与化简技术。基于 UML模型中的顺序图获取软件与外部环境之间的消息交互,通过分析激励与响应消息来获取状态生成软件Markov链使用模型,准确描述软件的使用情况。针对状态空间爆炸问题,提出冗余状态和等价状态的定义,设计使用模型化简算法,针对化简算法给出相关理论证明。实验结果表明了该方法的有效性。%To solve the state space explosion problem of Markov chain usage model in the software reliability testing,the tech-nology to generate and simplify usage model based on UML model was studied.Based on the sequence diagram of UML model, the messages between software and environment were derived.Based on the stimulus and response messages,the states of usage model were derived.The usage model described the usage of software accurately.After analyzing the state space explosion prob-lem,the concepts of equivalent states and redundant states were defined.An algorithm to simplify the state space was proposed. Related theoretical proof was given.Finally,the effectiveness of the proposed method was verified by experiments.

  2. Basic Lithofacies-Succession Model for the Wunishan Cyclothems:Their Markov Chain Analysis and Regularly Vertical Stacking Patternsin the Third-Order Sequences

    Institute of Scientific and Technical Information of China (English)

    2001-01-01

    The Mesoproterozoic Wumishan Formation in the Jixian section ofTianjin is a succession of 3300-m-thick carbonate strata formed in a period of about 100 Ma (1310 ± 20 Ma-1207 ± 10 Ma). In this succession of strata, the carbonate metre-scale cyclic sequences belonging to peritidal type with an approximately symmetrical lithofacies succession are best developed. The wide development of 1:4 stacking patterns shows that these metre-scale cyclic se quences are genetically related to the short-eccentricity cycles, which are called the Wumishan cyclothems that could truly represent sedimentary cycles. Generally, massive and thick-bedded calcareous dolomites and dolomitic limestones of stromatolite biostromes and thrombolite bioherms constitute the central part of the Wumishan cyclothems. The lower and upper parts consist of tidal flat dolostones, sandy-muddy dolostone and the top part is composed of lagoonal facies dolo mitic shales with a paleosol cap. Therefore, an approximately symmetrical lithofacies-succession is formed. Many features such as the clear deepening and shoaling vectors of cyclothems, and all kinds of marks of fresh-water diagenesis indicate that the Wumishan cyclothems are the product of autocyclic sedimentation governed by allocyclic high-frequency sea level changes. The results of a Markov chain analysis reaffirm the existence of the lithofacies-succession model of the Wumishan cyclothems. The boundaries of the Wumishan cyclothems are marked by the instantaneous exposed punctuated surface, which leads to the discrepancy between the cyclothems and the parasequences of the sequence stratigraphy ter minology system. It is difficult to form a judgment that the time span of the Milankovitch cycles in the Precambrian is certainly equal to that of the Phanerozoic, but the regularly vertical stacking patterns of the seventh-order rhythms, sixth order cyclothems and fifth-order parasequence sets still indicate their consistency with the duration of the

  3. Building Simple Hidden Markov Models. Classroom Notes

    Science.gov (United States)

    Ching, Wai-Ki; Ng, Michael K.

    2004-01-01

    Hidden Markov models (HMMs) are widely used in bioinformatics, speech recognition and many other areas. This note presents HMMs via the framework of classical Markov chain models. A simple example is given to illustrate the model. An estimation method for the transition probabilities of the hidden states is also discussed.

  4. Probabilistic Reachability for Parametric Markov Models

    DEFF Research Database (Denmark)

    Hahn, Ernst Moritz; Hermanns, Holger; Zhang, Lijun

    2011-01-01

    Given a parametric Markov model, we consider the problem of computing the rational function expressing the probability of reaching a given set of states. To attack this principal problem, Daws has suggested to first convert the Markov chain into a finite automaton, from which a regular expression...

  5. Characteristic Number and Taboo Probability of Markov Chains in Random Environments%随机环境中马氏链的特征数与禁止概率

    Institute of Scientific and Technical Information of China (English)

    崔静

    2012-01-01

    On the basic of the general theory of Markov Chains,relations between characteristic number and taboo probability of Markov chains in random enviroments are discussed and the decomposition of characteristic number of strong recurrence is given.The property of distribrution moments is obtained by means of the general decomposition formula of Taboo probability,which has enriched the correspanding result of Markov Chairs in randon envirionments described in literatures.%利用马氏链的一般理论讨论了随机环境中马氏链的特征数和禁止概率的关系,给出了强常返状态特征数的分解公式,利用禁止概率的一般分解公式研究了分布矩的性质,丰富了现有文献中的随机环境中马氏链的相关结果。

  6. Adaptive designs for sequential experiments

    Institute of Scientific and Technical Information of China (English)

    林正炎; 张立新

    2003-01-01

    Various adaptive designs have been proposed and applied to clinical trials, bioassay, psychophysics, etc.Adaptive designs are also useful in high cost engineering trials.More and more people have been paying attention to these design methods. This paper introduces several broad families of designs, such as the play-the-winner rule, randomized play-the-winner rule and its generalization to the multi-arm case, doubly biased coin adaptive design, Markov chain model.

  7. Adaptation of ECOSYS for Nordic food chain modelling

    International Nuclear Information System (INIS)

    The ECOSYS model is the basic food chain model in the current European state-of-the-art systems for decision-aiding in connection with major nuclear accidents. To apply this model optimally in the decision process in a Nordic country contaminated due to an accident, it is necessary to change a number of the default parameters in ECOSYS to reflect Nordic, national or regional conditions. A number of potentially problematic parameters are identified and examined in this context. The importance of obtaining adequate parameters for, e.g., human consumption habits, animal feeding regimes, crop development dynamics and soil-plant transfer processes are discussed. Also the generic importance of considering particle size in connection with deposition modelling is demonstrated. (au)

  8. FEP Analysis and Markov Chains

    NARCIS (Netherlands)

    Nepveu, M.; Yavuz, F.; David, P.

    2009-01-01

    Uncertainties related with underground CO2 storage play a vital role in risk assessment with respect to Carbon storage and capture projects (CCS). The main purpose of risk assessment is to determine a qualitative and quantitative picture of hazardous processes or events. One makes a comprehensive in

  9. Revisiting Causality in Markov Chains

    OpenAIRE

    Shojaee, Abbas

    2016-01-01

    Identifying causal relationships is a key premise of scientific research. The growth of observational data in different disciplines along with the availability of machine learning methods offers the possibility of using an empirical approach to identifying potential causal relationships, to deepen our understandings of causal behavior and to build theories accordingly. Conventional methods of causality inference from observational data require a considerable length of time series data to capt...

  10. Maximizing entropy over Markov processes

    DEFF Research Database (Denmark)

    Biondi, Fabrizio; Legay, Axel; Nielsen, Bo Friis;

    2014-01-01

    The channel capacity of a deterministic system with confidential data is an upper bound on the amount of bits of data an attacker can learn from the system. We encode all possible attacks to a system using a probabilistic specification, an Interval Markov Chain. Then the channel capacity computat...

  11. Grey-Markov Model for Road Accidents Forecasting

    Institute of Scientific and Technical Information of China (English)

    李相勇; 严余松; 蒋葛夫

    2003-01-01

    In order to improve the forecasting precision of road accidents, by introducing Markov chains forecasting method, a grey-Markov model for forecasting road accidents is established based on grey forecasting method. The model combines the advantages of both grey forecasting method and Markov chains forecasting method, overcomes the influence of random fluctuation data on forecasting precision and widens the application scope of the grey forecasting. An application example is conducted to evaluate the grey-Markov model, which shows that the precision of the grey-Markov model is better than that of grey model in forecasting road accidents.

  12. Testing the Markov condition in ion channel recordings

    CERN Document Server

    Timmer, J

    1997-01-01

    A statistical test is presented to decide whether data are adequately described by probabilistic functions of finite state Markov chains (''hidden Markov models'') as applied in the analysis of ion channel data. Particularly, the test can be used to decide whether a system obeys the Markov condition. Simulation studies are performed in order to investigate the sensitivity of the proposed test against violations of the model assumptions. The test can be applied analogously to Markov models.

  13. The effect of reproductive performance on the dairy cattle herd value assessed by integrating a daily dynamic programming model with a daily Markov chain model.

    Science.gov (United States)

    Kalantari, A S; Cabrera, V E

    2012-10-01

    The objective of this study was to determine the effect of reproductive performance on dairy cattle herd value. Herd value was defined as the herd's average retention payoff (RPO). Individual cow RPO is the expected profit from keeping the cow compared with immediate replacement. First, a daily dynamic programming model was developed to calculate the RPO of all cow states in a herd. Second, a daily Markov chain model was applied to estimate the herd demographics. Finally, the herd value was calculated by aggregating the RPO of all cows in the herd. Cow states were described by 5 milk yield classes (76, 88, 100, 112, and 124% with respect to the average), 9 lactations, 750 d in milk, and 282 d in pregnancy. Five different reproductive programs were studied (RP1 to RP5). Reproductive program 1 used 100% timed artificial insemination (TAI; 42% conception rate for first TAI and 30% for second and later services) and the other programs combined TAI with estrus detection. The proportion of cows receiving artificial insemination after estrus detection ranged from 30 to 80%, and conception rate ranged from 25 to 35%. These 5 reproductive programs were categorized according to their 21-d pregnancy rate (21-d PR), which is an indication of the rate that eligible cows become pregnant every 21 d. The 21-d PR was 17% for RP1, 14% for RP2, 16% for RP3, 18% for RP4, and 20% for RP5. Results showed a positive relationship between 21-d PR and herd value. The most extreme herd value difference between 2 reproductive programs was $77/cow per yr for average milk yield (RP5 - RP2), $13/cow per yr for lowest milk yield (RP5 - RP1), and $160/cow per yr for highest milk yield (RP5 - RP2). Reproductive programs were ranked based on their calculated herd value. With the exception of the best reproductive program (RP5), all other programs showed some level of ranking change according to milk yield. The most dramatic ranking change was observed in RP1, which moved from being the worst ranked

  14. Programming of adaptive repair process chains using repair features and function blocks

    Directory of Open Access Journals (Sweden)

    Gunter Spöcker

    2016-01-01

    Full Text Available The current trends of product customization and repair of high value parts with individual defects demand automation and a high degree of flexibility of the involved manufacturing process chains. To determine the corresponding requirements this paper gives an overview of manufacturing process chains by distinguishing between horizontal and vertical process chains. The established way of modeling and programming processes with CAx systems and existing approaches is shown. Furthermore, the different types of possible adaptions of a manufacturing process chain are shown and considered as a cascaded control loop. Following this it is discussed which key requirements of repair process chains are unresolved by existing approaches. To overcome the deficits this paper introduces repair features which comprise the idea of geometric features and defines analytical auxiliary geometries based on the measurement input data. This meets challenges normally caused by working directly on reconstructed geometries in the form of triangulated surfaces which are prone to artifacts. Embedded into function blocks, this allows the use of traditional approaches for manufacturing process chains to be applied to adaptive repair process chains.

  15. EVALUATION OF HIDDEN MARKOV MODEL BASED ADAPTIVE PROVISIONING OF OPTICAL BURST SWITCHING NETWORKS AMENABLE FOR UPGRADATION TO GREEN FLEXIGRID NETWORKS

    Directory of Open Access Journals (Sweden)

    P. G.V. Ramesh

    2014-01-01

    Full Text Available Catering to the evolving bandwidth-on-demand applications requires flexible provisioning architectures that ensure fairness to both high end and low end users of internet. Dynamic classification of network paths into long and short, based on traffic dependent connection holding times and a wavelength allocation from different subsets for these categories is a necessity for implementing energy saving hybrid switching, loss recovery and flexi grid bit rate variable schemes. Therefore, this study evaluates such a scheme of dynamic classification based on HMM predicted connection-holding times with tightly integrated adaptive burst sizing and segregated wavelength allocation for long and short categories. Simulation study of a 28 node OBS network shows that this coupled scheme reduces delays and results in throughput improvement of 67% for long and 12% for short traffic over schemes that employ adaptive burst sizing based on number of active traffic flows and independent wavelength assignment schemes.

  16. A Markov chain Monte Carlo Expectation Maximization Algorithm for Statistical Analysis of DNA Sequence Evolution with Neighbor-Dependent Substitution Rates

    DEFF Research Database (Denmark)

    Hobolth, Asger

    2008-01-01

    The evolution of DNA sequences can be described by discrete state continuous time Markov processes on a phylogenetic tree. We consider neighbor-dependent evolutionary models where the instantaneous rate of substitution at a site depends on the states of the neighboring sites. Neighbor...

  17. THE ADOPTION/ADAPTATION OF THE 'SUPPLY CHAIN' CONCEPT IN ROMANIAN

    Directory of Open Access Journals (Sweden)

    Butilca Delia - Alexandra

    2011-12-01

    Full Text Available There are several terms used in Romanian for the English term supply chain. We have counted eleven. It has been adopted in several ways and it is perceived (adapted in several ways in Romania. There are several causes which have determined this fact: the English terminology inconsistence regarding supply chain has been translated into other languages, professionals roots determine the usage (logistics professionals use it for logistics, strategic logistics use it in general etc., the lack of Romanian discussions regarding the new professional terminology at general level and supply chain and logistics in particular. We expose the different ways by which supply chain is reflected in Romanian, we discuss the causes which have generated this total confusion and then we suggest a proper term for supply chain in Romanian, considering the real meaning of the term and the strategies adopted by other countries regarding supply chain adoption. In conclusion, we propose a general adoption and adaptation for the term in Romanian.

  18. Learning Markov models for stationary system behaviors

    DEFF Research Database (Denmark)

    Chen, Yingke; Mao, Hua; Jaeger, Manfred;

    2012-01-01

    to a single long observation sequence, and in these situations existing automatic learning methods cannot be applied. In this paper, we adapt algorithms for learning variable order Markov chains from a single observation sequence of a target system, so that stationary system properties can be verified using......Establishing an accurate model for formal verification of an existing hardware or software system is often a manual process that is both time consuming and resource demanding. In order to ease the model construction phase, methods have recently been proposed for automatically learning accurate...... the learned model. Experiments demonstrate that system properties (formulated as stationary probabilities of LTL formulas) can be reliably identified using the learned model....

  19. On Determining the Order of Markov Dependence of an Observed Process Governed by a Hidden Markov Model

    Directory of Open Access Journals (Sweden)

    R.J. Boys

    2002-01-01

    Full Text Available This paper describes a Bayesian approach to determining the order of a finite state Markov chain whose transition probabilities are themselves governed by a homogeneous finite state Markov chain. It extends previous work on homogeneous Markov chains to more general and applicable hidden Markov models. The method we describe uses a Markov chain Monte Carlo algorithm to obtain samples from the (posterior distribution for both the order of Markov dependence in the observed sequence and the other governing model parameters. These samples allow coherent inferences to be made straightforwardly in contrast to those which use information criteria. The methods are illustrated by their application to both simulated and real data sets.

  20. Markov链利率风险模型下破产概率的近似计算%Approximation for Ruin Probability in Risk Model with Markov Chain Interest Rate

    Institute of Scientific and Technical Information of China (English)

    宗志迅; 李志民; 郭红财

    2012-01-01

    基于带Markov链利率的离散时问风险模型和Markov链将来利率与过去利率的独立性,假设个体净风险是重尾分布的,利用全概率公式和递推方法,得到该风险模型下有限时间破产概率的近似表达式,并在个体净风险是Pareto分布时,利用Matlab软件数值模拟近似值.%Based on a discrete time risk model with Markov chain interest and a Markov chain reflecting the indepen- dence of future interest rates and the past rate, the individual net risk is heavy tailed that is presumed, the approxi- mate expressions of the finite time ruin probability in this risk model by the full probability formula and the recurrence method are obtained. When the distribution of individual net risk belongs to Pareto, numerical approximation by using the software MATLAB are presented.

  1. A Markov Chain Monte Carlo technique to sample transport and source parameters of Galactic cosmic rays: II. Results for the diffusion model combining B/C and radioactive nuclei

    CERN Document Server

    Putze, A; Maurin, D

    2010-01-01

    On-going measurements of the cosmic radiation (nuclear, electronic, and gamma-ray) are shedding new light on cosmic-ray physics. A comprehensive picture of these data relies on an accurate determination of the transport and source parameters of propagation models. A Markov Chain Monte Carlo is used to obtain these parameters in a diffusion model. From the measurement of the B/C ratio and radioactive cosmic-ray clocks, we calculate their probability density functions, with a special emphasis on the halo size L of the Galaxy and the local underdense bubble of size r_h. The analysis relies on the USINE code for propagation and on a Markov Chain Monte Carlo technique (Putze et al. 2009, paper I of this series) for the parameter determination. As found in previous studies, the B/C best-fit model favours diffusion/convection/reacceleration (Model III) over diffusion/reacceleration (Model II). A combined fit on B/C and the isotopic ratios (10Be/9Be, 26Al/27Al, 36Cl/Cl) leads to L ~ 8 kpc and r_h ~ 120 pc for the bes...

  2. Adapting chain referral methods to sample new migrants: Possibilities and limitations

    Directory of Open Access Journals (Sweden)

    Lucinda Platt

    2015-09-01

    Full Text Available Background: Demographic research on migration requires representative samples of migrant populations. Yet recent immigrants, who are particularly informative about current migrant flows, are difficult to capture even in specialist surveys. Respondent-driven sampling (RDS, a chain referral sampling and analysis technique, potentially offers the opportunity to achieve population-level inference of recently arrived migrant populations. Objective: We evaluate the attempt to use RDS to sample two groups of migrants, from Pakistan and Poland, who had arrived in the UK within the previous 18 months, and we present an alternative approach adapted to recent migrants. Methods: We discuss how connectedness, privacy, clustering, and motivation are expected to differ among recently arrived migrants, compared to typical applications of RDS. We develop a researcher-led chain referral approach, and compare success in recruitment and indicators of representativeness to standard RDS recruitment. Results: Our researcher-led approach led to higher rates of chain-referral, and enabled us to reach population members with smaller network sizes. The researcher-led approach resulted in similar recruiter-recruit transition probabilities to traditional RDS across many demographic and social characteristics. However, we did not succeed in building up long referral chains, largely due to the lack of connectedness of our target populations and some reluctance to refer. There were some differences between the two migrant groups, with less mobile and less hidden Pakistani men producing longer referral chains. Conclusions: Chain referral is difficult to implement for sampling newly arrived migrants. However, our researcher-led adaptation shows promise for less hidden and more stable recent immigrant populations. Contribution: The paper offers an evaluation of RDS for surveying recent immigrants and an adaptation that may be effective under certain conditions.

  3. Context Tree Estimation in Variable Length Hidden Markov Models

    CERN Document Server

    Dumont, Thierry

    2011-01-01

    We address the issue of context tree estimation in variable length hidden Markov models. We propose an estimator of the context tree of the hidden Markov process which needs no prior upper bound on the depth of the context tree. We prove that the estimator is strongly consistent. This uses information-theoretic mixture inequalities in the spirit of Finesso and Lorenzo(Consistent estimation of the order for Markov and hidden Markov chains(1990)) and E.Gassiat and S.Boucheron (Optimal error exponents in hidden Markov model order estimation(2003)). We propose an algorithm to efficiently compute the estimator and provide simulation studies to support our result.

  4. Detecting Structural Breaks using Hidden Markov Models

    DEFF Research Database (Denmark)

    Ntantamis, Christos

    Testing for structural breaks and identifying their location is essential for econometric modeling. In this paper, a Hidden Markov Model (HMM) approach is used in order to perform these tasks. Breaks are defined as the data points where the underlying Markov Chain switches from one state to anoth...... in the monetary policy of United States, the dierent functional form being variants of the Taylor (1993) rule.......Testing for structural breaks and identifying their location is essential for econometric modeling. In this paper, a Hidden Markov Model (HMM) approach is used in order to perform these tasks. Breaks are defined as the data points where the underlying Markov Chain switches from one state to another....... The estimation of the HMM is conducted using a variant of the Iterative Conditional Expectation-Generalized Mixture (ICE-GEMI) algorithm proposed by Delignon et al. (1997), that permits analysis of the conditional distributions of economic data and allows for different functional forms across regimes...

  5. Hidden Markov models estimation and control

    CERN Document Server

    Elliott, Robert J; Moore, John B

    1995-01-01

    As more applications are found, interest in Hidden Markov Models continues to grow. Following comments and feedback from colleagues, students and other working with Hidden Markov Models the corrected 3rd printing of this volume contains clarifications, improvements and some new material, including results on smoothing for linear Gaussian dynamics. In Chapter 2 the derivation of the basic filters related to the Markov chain are each presented explicitly, rather than as special cases of one general filter. Furthermore, equations for smoothed estimates are given. The dynamics for the Kalman filte

  6. USO DE CADENAS DE MARKOV PARA LA PREDICCIÓN DE LA DINÁMICA DEL COMPORTAMIENTO DE PACIENTES EN UNA UNIDAD DE CUIDADO INTENSIVO CARDIOLÓGICA USE OF MARKOV CHAINS IN PREDICTION OF THE DYNAMICS OF PATIENT BEHAVIOR IN A CARDIOLOGIC INTENSIVE CARE UNIT

    Directory of Open Access Journals (Sweden)

    Víctor Albornoz

    2006-08-01

    Full Text Available En este trabajo presentamos un modelo probabilístico que contribuye al estudio de la dinámica en el comportamiento y permanencia de pacientes en una unidad de cuidados intensivos cardiológica. El modelo utilizado corresponde a una Cadena de Markov en tiempo discreto, que mediante la definición de determinados niveles de gravedad de un paciente (estados y la obtención de las correspondientes probabilidades de transición entre un nivel de gravedad y otro, permite predecir los tiempos de permanencia. Los diferentes estados empleados se basan en la construcción de un nuevo score creado para este propósito. Se muestran los detalles de la metodología adoptada y los principales resultados alcanzados en la aplicación del modelo empleado.In this paper we present a probabilistic model that contributes to the study of dynamics in the behavior and permanence of patients in a cardiovascular intensive care unit. The model corresponds to a discrete Markov Chain, that allows to predict the time that a patient remains in the system through the time, by means of certain severity of illness states and the corresponding transition probabilities between those states. The different states are based on the construction of a new score created for this study. We summarize the details of the adopted methodology and the main results reached in the application of the model.

  7. CHARACTERIZATION OF MARKOV-BERNOULLI GEOMETRIC DISTRIBUTION RELATED TO RANDOM SUMS

    OpenAIRE

    Mohamed Gharib; Ramadan, Mahmoud M.; Khaled A.H. Al-Ajmi

    2014-01-01

    The Markov-Bernoulli geometric distribution is obtained when a generalization, as a Markov process, of the independent Bernoulli sequence of random variables is introduced by considering the success probability changes with respect to the Markov chain. The resulting model is called the Markov- Bernoulli model and it has a wide variety of application fields. In this study, some characterizations are given concerning the Markov-Bernoulli geometric distribution as the distribution of the summati...

  8. The Mitochondrial Respiratory Chain Is Required for Organismal Adaptation to Hypoxia

    Directory of Open Access Journals (Sweden)

    Robert B. Hamanaka

    2016-04-01

    Full Text Available Hypoxia-inducible factors (HIFs are crucial for cellular and organismal adaptation to hypoxia. The mitochondrial respiratory chain is the largest consumer of oxygen in most mammalian cells; however, it is unknown whether the respiratory chain is necessary for in vivo activation of HIFs and organismal adaptation to hypoxia. HIF-1 activation in the epidermis has been shown to be a key regulator of the organismal response to hypoxic conditions, including renal production of erythropoietin (Epo. Therefore, we conditionally deleted expression of TFAM in mouse epidermal keratinocytes. TFAM is required for maintenance of the mitochondrial genome, and TFAM-null cells are respiratory deficient. TFAM loss in epidermal keratinocytes reduced epidermal levels of HIF-1α protein and diminished the hypoxic induction of HIF-dependent transcription in epidermis. Furthermore, epidermal TFAM deficiency impaired hypoxic induction of renal Epo expression. Our results demonstrate that the mitochondrial respiratory chain is essential for in vivo HIF activation and organismal adaptation to hypoxia.

  9. Supply Chain Management Or Adaptive Business Network? – Coordination Versus Collaboration

    Directory of Open Access Journals (Sweden)

    Ilies Liviu

    2008-04-01

    Full Text Available You are not the only company capable to create value for your clients. For making valuable products, for performing high standard services, one company is no longer enough in this new informational era, in this new needs era. Better business performing means now to satisfy more complex needs quicker than ever. Could one company face this challenge alone? Is there another way? Collaboration may be the answer. Maybe the most well-known term when it comes to collaboration between companies is Supply Chain. Why Supply Chain, why Supply Chain Management (SCM? This article presents the answers to these questions, but also presents different approaches regarding SCM: the logistics approach regarding supply chain management, the strategic approach, the new entrepreneurial approach, supply chain as a win-win game. New paradigms regarding collaboration appeared regarding business collaboration: Adaptive Business Network (ABN. Do these new concepts imply the dead of SCM? Or are them only a new wave in SCM terminology and business orientation? Our conclusion is that these new approach is a normal change in business: businesses are made by people and people don’t like to be conducted (managed. The old-fashioned SCM was based on a coordinator versus several obedients relation. It is absolutely normal to dream at freedom and to be not very efficient while you have to play after somebody else rules. ABN is in the other side of human relations and also business relations – it insists on partnering. Everybody is a part of a chain which has as its main goal customer satisfaction, has the right to make proposals, to negotiate, and to be a winner. Of course, ABN appearance does not involve SCM disappearance, but change in how some chains are managed, in how some chains function. We shall see for the future if an organization with several brains is more successful.

  10. 基于Markov链的嵌入式系统硬件可靠性研究%Hardware reliability study of embedded system based on Markov chain

    Institute of Scientific and Technical Information of China (English)

    郭荣佐; 黄君; 王霖

    2012-01-01

    The products of embedded system often have hardware failure while in use, which affects the safety and reliability of the system. In this paper, its reliability was researched from the embedded system hardware level. At first, it defined the goal of the embedded system hardware, and introduced the theory of Markov process. And then, a single IP hardcore and embedded system hardware Markov model was set up. At last, the reliability of the embedded controller hardware automatic block was calculated and analyzed through the established model. The results show that the Markov model can accurately describe the embedded system hardware state change, and can calculate and analyze their reliability. Therefore, the model has certain practical value.%嵌入式系统产品在使用过程中经常出现硬件故障,从而影响系统的安全可靠性.从嵌入式系统硬件层面研究其可靠性.首先定义嵌入式系统硬件目标,简单介绍了Markov过程理论;建立了单个IP硬核和嵌入式系统硬件的Markov模型;应用所建立的模型,对嵌入式站间自动闭塞控制器硬件进行了可靠度计算和分析.实验结果表明,该Markov模型能够准确描述嵌入式系统硬件的状态变迁,并能计算和分析其可靠度,具有一定的实用价值.

  11. The One-Dimensional Distribution and Construction of Semi-Markov Processes%半马氏过程的一维分布及构造

    Institute of Scientific and Technical Information of China (English)

    唐荣; 郭先平; 刘再明

    2008-01-01

    In this paper, we obtain the transition probability of jump chain of semi-Markov process, the distribution of sojourn time and one-dimensional distribution of semi-Markov process.Furthermore, the semi-Markov process X(t, ω) is constructed from the semi-Markov matrix and it is proved that two definitions of semi-Markov process are equivalent.

  12. Nonuniform Markov Geometric Measures

    OpenAIRE

    Neunhäuserer, J.

    2015-01-01

    We generalize results of Fan and Zhang [6] on absolute continuity and singularity of the golden Markov geometric series to nonuniform stochastic series given by arbitrary Markov process. In addition we describe an application of these results in fractal geometry.

  13. Simulation and Research on Frame Slotted ALOHA Anti-collision Algorithm Based on Markov Chain Model%基于马尔科夫链的帧时隙ALOHA防碰撞算法仿真与研究

    Institute of Scientific and Technical Information of China (English)

    马耀庭; 张新龙

    2014-01-01

    为了研究帧时隙 ALOHA防碰撞算法性能,应用马尔科夫链模型对该算法标签识别过程进行数学分析,得到成功识别出的标签数量的状态转移概率矩阵。用蒙特卡罗统计方法模拟这一过程,对马尔科夫链模型求解,得到了标签数量、时隙数和成功识别率之间的关系曲线。%In order to study the performance of the frame slotted ALOHA anti-collision algorithm,by use of the Markov Model,a mathematical analysis was done to the tag identification process of this algorithm and a state transition probability matrix is thus obtained for the successful identification of the number of tags.By using the Monte-Carlo method to simulate this process,the solution of the Markov Chain Model was worked out and the relationship curve between the number of tags, the number of slots and the rate of successful recognition was obtained.

  14. Based on Markov Chain in the IT project schedule control and management%基于马尔科夫链在IT项目进度中的控制管理

    Institute of Scientific and Technical Information of China (English)

    龚雯; 胡家福

    2013-01-01

    In the IT project development process,IT project management is increasingly concerned by people, so many IT companies are active in management of the project into the progress of development activities, on the development of effective management.This paper uses Markov chain models and methods for IT project network diagrams and schedule management to predict,in the import and use of IT projects Markov model en-ables managers to more easily manage the project and make it better forecasting manpower savings and cost.%在IT项目的开发过程中,IT项目管理日益受到人们的关注,各个IT企业都在积极将项目管理引入到开发活动的进度当中,对开发实行有效的管理。本文运用马尔可夫链模型和网络图方法进行IT项目进度管理并进行预测,在IT项目中导入并使用马尔可夫模型使管理者们更加容易对项目进行管理以及预测使之更好的节约人力以及成本。

  15. Forest cover change prediction using hybrid methodology of geoinformatics and Markov chain model: A case study on sub-Himalayan town Gangtok, India

    Indian Academy of Sciences (India)

    Anirban Mukhopadhyay; Arun Mondal; Sandip Mukherjee; Dipam Khatua; Subhajit Ghosh; Debasish Mitra; Tuhin Ghosh

    2014-08-01

    In the Himalayan states of India, with increasing population and activities, large areas of forested land are being converted into other land-use features. There is a definite cause and effect relationship between changing practice for development and changes in land use. So, an estimation of land use dynamics and a futuristic trend pattern is essential. A combination of geospatial and statistical techniques were applied to assess the present and future land use/land cover scenario of Gangtok, the subHimalayan capital of Sikkim. Multi-temporal satellite imageries of the Landsat series were used to map the changes in land use of Gangtok from 1990 to 2010. Only three major land use classes (built-up area and bare land, step cultivated area, and forest) were considered as the most dynamic land use practices of Gangtok. The conventional supervised classification, and spectral indices-based thresholding using NDVI (Normalized Difference Vegetation Index) and SAVI (Soil Adjusted Vegetation Index) were applied along with the accuracy assessments. Markov modelling was applied for prediction of land use/land cover change and was validated. SAVI provides the most accurate estimate, i.e., the difference between predicted and actual data is minimal. Finally, a combination of Markov modelling and SAVI was used to predict the probable land-use scenario in Gangtok in 2020 AD, which indicted that more forest areas will be converted for step cultivation by the year 2020.

  16. 非结构化P2P网络基于马尔科夫链的搜索算法研究%Discrete Markov Chain Model for Resource Searching of Peer-to-Peer Network

    Institute of Scientific and Technical Information of China (English)

    刘璇; 于双元

    2015-01-01

    Existing resources searching algorithm of unstructured P2P network has not considered the interests of node combining with the load of the whole network.The paper not only considers the node search resources query based on interest,but also considers the load information of each node.Thus,this paper puts forward the resources searching algorithm,BoMC,which is based on Markov Chain model.The BoMC algorithm makes full use of Markov Chain to discrete transition probability matrix for unstructured P2P network though random sampling.However,transi-tion probability is based on forwarding factor of the node that contains of the information of interests and load.On the basis of the characteristics of the stationary distribution of Markov Chain,the algorithm could make the entire network become the convergence condition.And in the process of query,the proposed algorithm would update transition proba-bility dynamically.In PeerSim simulation environment,this paper realizes resources searching algorithm BoMC and discuss the traditional P2P resources searching algorithm with the proposed algorithm.%现有的非结构化 P2P 资源搜索算法并没有将兴趣与负载结合进行考虑,本文不仅考虑节点搜索资源时基于兴趣的查询转发,也综合考虑了各个节点的负载信息.基于此,本文设计提出了基于Markov Chain 模型的资源搜索改进算法BoMC.BoMC算法利用马尔科夫模型为非结构化P2P网络节点通过随机采样建立状态转移概率矩阵.而转移概率是基于节点的转发因子,其中包含有节点兴趣及负载的综合信息.我们知道,基于马尔科夫链平稳分布的特性可以使整个网络在查询过程中趋于收敛状态,进而达到节点的负载均衡.根据网络负载分布情况,该算法考虑到动态更新转移概率.在PeerSim的仿真环境下,实现BoMC算法并将其与传统的P2P资源搜索算法作比较.

  17. Narrowing the gap between climate science and adaptation action: The role of boundary chains

    Directory of Open Access Journals (Sweden)

    Christine J. Kirchhoff

    2015-01-01

    Full Text Available Boundary organizations play a critical role at the interface between science and decision making. They create, protect and sustain an interactive space for co-production of science and decision-making while simultaneously bridging the two domains. In this special issue we advance the concept of boundary chains, whereby two or more boundary organizations link together synergistically to influence one another and to leverage each other’s resources and strengths to achieve shared goals. In this process both the level of complementary and embeddedness between these organizations is critical for achieving these goals. Through a series of case studies focusing primarily but not exclusively on climate information use in the United States, we aim to advance scholarship in the field by examining innovation among boundary organizations and testing the boundary chain concept. In doing so, we focus on boundary chains both as a theoretical construct to re-think the structure, function, and adaptability of boundary organizations and as a practical strategy to further increase the usability of climate knowledge for adaptation action across a wider range of users.

  18. On multitarget pairwise-Markov models

    Science.gov (United States)

    Mahler, Ronald

    2015-05-01

    Single- and multi-target tracking are both typically based on strong independence assumptions regarding both the target states and sensor measurements. In particular, both are theoretically based on the hidden Markov chain (HMC) model. That is, the target process is a Markov chain that is observed by an independent observation process. Since HMC assumptions are invalid in many practical applications, the pairwise Markov chain (PMC) model has been proposed as a way to weaken those assumptions. In this paper it is shown that the PMC model can be directly generalized to multitarget problems. Since the resulting tracking filters are computationally intractable, the paper investigates generalizations of the cardinalized probability hypothesis density (CPHD) filter to applications with PMC models.

  19. Theory of Markov processes

    CERN Document Server

    Dynkin, E B

    1960-01-01

    Theory of Markov Processes provides information pertinent to the logical foundations of the theory of Markov random processes. This book discusses the properties of the trajectories of Markov processes and their infinitesimal operators.Organized into six chapters, this book begins with an overview of the necessary concepts and theorems from measure theory. This text then provides a general definition of Markov process and investigates the operations that make possible an inspection of the class of Markov processes corresponding to a given transition function. Other chapters consider the more c

  20. A Self-Adaptive Image Steganography Algorithm Based on Cover-Coding and Markov Model%基于隐写编码和Markov模型的自适应图像隐写算法

    Institute of Scientific and Technical Information of China (English)

    张湛; 刘光杰; 戴跃伟; 王执铨

    2012-01-01

    如何构造大容量、低失真和高统计安全的隐写算法一直是隐写研究的难点和热点.提出一种兼顾感知失真和二阶统计安全的自适应图像隐写算法设计思路.算法将载体各部分的平滑度引入隐写编码的生成过程,自适应地利用一簇隐写编码在载体各部分的合理运用降低载密图像失真度;在隐秘信息嵌入方式上利用基于Markov链模型的动态补偿方法提高载密图像统计安全性;算法对载体最低有效位和次最低有效位进行嵌入以保证嵌入容量.实验表明算法在相同嵌入量下相较双层随机LSB匹配算法以及仅使用一种隐写编码的算法,失真度更低且载体统计分布的改变更小,而在失真度和统计分布改变相近时嵌入容量更大.%It is a difficulty and hotspot how to desigh steganography algorithms with large-capacity, low-distortion and high statistical security. A self-adaptive image steganography algorithm which takes account of the perceptual distortion and second-order statistical security is proposed. It introduces the smoothness of the various parts of the cover-object to the encoding generation process of cover codes, and reduces the distortion by the reasonable use of a cluster of cover codes in each part of cover-object. In the embedding aspect, in order to improve the statistic security, the algorithm uses a dynamic compensate method based on the image Markov chain model, and it embeds secret information into the least two significant bit (LTSB) planes in order to ensure the capacity. Experiment results show the proposed algorithm has lower distortion and smaller changes of cover statistical distribution than the stochastic LTSB match steganography algorithm and the algorithm which only uses one cover code under the same embedded payload. And the proposed algorithm has larger payloads than one cover code embedding when the distortion and statistical distribution changes are close.