#### Sample records for underlying markov chain

1. Markov chains

CERN Document Server

Revuz, D

1984-01-01

This is the revised and augmented edition of a now classic book which is an introduction to sub-Markovian kernels on general measurable spaces and their associated homogeneous Markov chains. The first part, an expository text on the foundations of the subject, is intended for post-graduate students. A study of potential theory, the basic classification of chains according to their asymptotic behaviour and the celebrated Chacon-Ornstein theorem are examined in detail. The second part of the book is at a more advanced level and includes a treatment of random walks on general locally compact abelian groups. Further chapters develop renewal theory, an introduction to Martin boundary and the study of chains recurrent in the Harris sense. Finally, the last chapter deals with the construction of chains starting from a kernel satisfying some kind of maximum principle.

2. Markov Chain Monte Carlo

. Keywords. Gibbs sampling, Markov Chain. Monte Carlo, Bayesian inference, stationary distribution, conver- gence, image restoration. Arnab Chakraborty. We describe the mathematics behind the Markov. Chain Monte Carlo method of ...

3. Error Bounds for Augmented Truncations of Discrete-Time Block-Monotone Markov Chains under Geometric Drift Conditions

OpenAIRE

Masuyama, Hiroyuki

2014-01-01

In this paper we study the augmented truncation of discrete-time block-monotone Markov chains under geometric drift conditions. We first present a bound for the total variation distance between the stationary distributions of an original Markov chain and its augmented truncation. We also obtain such error bounds for more general cases, where an original Markov chain itself is not necessarily block monotone but is blockwise dominated by a block-monotone Markov chain. Finally,...

4. Fields From Markov Chains

DEFF Research Database (Denmark)

Justesen, Jørn

2005-01-01

A simple construction of two-dimensional (2-D) fields is presented. Rows and columns are outcomes of the same Markov chain. The entropy can be calculated explicitly.......A simple construction of two-dimensional (2-D) fields is presented. Rows and columns are outcomes of the same Markov chain. The entropy can be calculated explicitly....

5. Regeneration and general Markov chains

Directory of Open Access Journals (Sweden)

1994-01-01

Full Text Available Ergodicity, continuity, finite approximations and rare visits of general Markov chains are investigated. The obtained results permit further quantitative analysis of characteristics, such as, rates of convergence, continuity (measured as a distance between perturbed and non-perturbed characteristics, deviations between Markov chains, accuracy of approximations and bounds on the distribution function of the first visit time to a chosen subset, etc. The underlying techniques use the embedding of the general Markov chain into a wide sense regenerative process with the help of splitting construction.

6. Markov processes and controlled Markov chains

CERN Document Server

Filar, Jerzy; Chen, Anyue

2002-01-01

The general theory of stochastic processes and the more specialized theory of Markov processes evolved enormously in the second half of the last century. In parallel, the theory of controlled Markov chains (or Markov decision processes) was being pioneered by control engineers and operations researchers. Researchers in Markov processes and controlled Markov chains have been, for a long time, aware of the synergies between these two subject areas. However, this may be the first volume dedicated to highlighting these synergies and, almost certainly, it is the first volume that emphasizes the contributions of the vibrant and growing Chinese school of probability. The chapters that appear in this book reflect both the maturity and the vitality of modern day Markov processes and controlled Markov chains. They also will provide an opportunity to trace the connections that have emerged between the work done by members of the Chinese school of probability and the work done by the European, US, Central and South Ameri...

7. Markov Chain Monte Carlo

Home; Journals; Resonance – Journal of Science Education; Volume 7; Issue 3. Markov Chain Monte Carlo - Examples. Arnab Chakraborty. General Article Volume 7 Issue 3 March 2002 pp 25-34. Fulltext. Click here to view fulltext PDF. Permanent link: https://www.ias.ac.in/article/fulltext/reso/007/03/0025-0034. Keywords.

8. Error bounds for augmented truncations of discrete-time block-monotone Markov chains under subgeometric drift conditions

OpenAIRE

Masuyama, Hiroyuki

2015-01-01

This paper studies the last-column-block-augmented northwest-corner truncation (LC-block-augmented truncation, for short) of discrete-time block-monotone Markov chains under subgeometric drift conditions. The main result of this paper is to present an upper bound for the total variation distance between the stationary probability vectors of a block-monotone Markov chain and its LC-block-augmented truncation. The main result is extended to Markov chains that themselves may not be block monoton...

9. Process Algebra and Markov Chains

NARCIS (Netherlands)

Brinksma, Hendrik; Hermanns, H.; Brinksma, Hendrik; Hermanns, H.; Katoen, Joost P.

This paper surveys and relates the basic concepts of process algebra and the modelling of continuous time Markov chains. It provides basic introductions to both fields, where we also study the Markov chains from an algebraic perspective, viz. that of Markov chain algebra. We then proceed to study

10. Approximate quantum Markov chains

CERN Document Server

Sutter, David

2018-01-01

This book is an introduction to quantum Markov chains and explains how this concept is connected to the question of how well a lost quantum mechanical system can be recovered from a correlated subsystem. To achieve this goal, we strengthen the data-processing inequality such that it reveals a statement about the reconstruction of lost information. The main difficulty in order to understand the behavior of quantum Markov chains arises from the fact that quantum mechanical operators do not commute in general. As a result we start by explaining two techniques of how to deal with non-commuting matrices: the spectral pinching method and complex interpolation theory. Once the reader is familiar with these techniques a novel inequality is presented that extends the celebrated Golden-Thompson inequality to arbitrarily many matrices. This inequality is the key ingredient in understanding approximate quantum Markov chains and it answers a question from matrix analysis that was open since 1973, i.e., if Lieb's triple ma...

11. Musical Markov Chains

Science.gov (United States)

Volchenkov, Dima; Dawin, Jean René

A system for using dice to compose music randomly is known as the musical dice game. The discrete time MIDI models of 804 pieces of classical music written by 29 composers have been encoded into the transition matrices and studied by Markov chains. Contrary to human languages, entropy dominates over redundancy, in the musical dice games based on the compositions of classical music. The maximum complexity is achieved on the blocks consisting of just a few notes (8 notes, for the musical dice games generated over Bach's compositions). First passage times to notes can be used to resolve tonality and feature a composer.

12. Markov Chain Monte Carlo Methods

Keywords. Markov chain; state space; stationary transition probability; stationary distribution; irreducibility; aperiodicity; stationarity; M-H algorithm; proposal distribution; acceptance probability; image processing; Gibbs sampler.

13. On Markov Chains and Filtrations

OpenAIRE

Spreij, Peter

1997-01-01

In this paper we rederive some well known results for continuous time Markov processes that live on a finite state space.Martingale techniques are used throughout the paper. Special attention is paid to the construction of a continuous timeMarkov process, when we start from a discrete time Markov chain. The Markov property here holds with respect tofiltrations that need not be minimal.

14. Flux through a Markov chain

International Nuclear Information System (INIS)

Floriani, Elena; Lima, Ricardo; Ourrad, Ouerdia; Spinelli, Lionel

2016-01-01

Highlights: • The flux through a Markov chain of a conserved quantity (mass) is studied. • Mass is supplied by an external source and ends in the absorbing states of the chain. • Meaningful for modeling open systems whose dynamics has a Markov property. • The analytical expression of mass distribution is given for a constant source. • The expression of mass distribution is given for periodic or random sources. - Abstract: In this paper we study the flux through a finite Markov chain of a quantity, that we will call mass, which moves through the states of the chain according to the Markov transition probabilities. Mass is supplied by an external source and accumulates in the absorbing states of the chain. We believe that studying how this conserved quantity evolves through the transient (non-absorbing) states of the chain could be useful for the modelization of open systems whose dynamics has a Markov property.

15. Markov Chain Monte Carlo Methods

time Technical Consultant to. Systat Software Asia-Pacific. (P) Ltd., in Bangalore, where the technical work for the development of the statistical software Systat takes place. His research interests have been in statistical pattern recognition and biostatistics. Keywords. Markov chain, Monte Carlo sampling, Markov chain Monte.

16. YMCA: Why Markov Chain Algebra?

NARCIS (Netherlands)

Bravetti, Mario; Hermanns, H.; Katoen, Joost P.; Aceto, L.; Gordon, A.

2006-01-01

Markov chains are widely used to determine system performance and reliability characteristics. The vast majority of applications considers continuous-time Markov chains (CTMCs). This note motivates how concurrency theory can be extended (as opposed to twisted) to CTMCs. We provide the core

17. Markov Chain Monte Carlo Methods

Markov Chain Monte Carlo Methods. 2. The Markov Chain Case. K B Athreya, Mohan Delampady and T Krishnan. K B Athreya is a Professor at. Cornell University. His research interests include mathematical analysis, probability theory and its application and statistics. He enjoys writing for Resonance. His spare time is ...

18. Markov chains theory and applications

CERN Document Server

Sericola, Bruno

2013-01-01

Markov chains are a fundamental class of stochastic processes. They are widely used to solve problems in a large number of domains such as operational research, computer science, communication networks and manufacturing systems. The success of Markov chains is mainly due to their simplicity of use, the large number of available theoretical results and the quality of algorithms developed for the numerical evaluation of many metrics of interest.The author presents the theory of both discrete-time and continuous-time homogeneous Markov chains. He carefully examines the explosion phenomenon, the

19. Quadratic Variation by Markov Chains

DEFF Research Database (Denmark)

Hansen, Peter Reinhard; Horel, Guillaume

We introduce a novel estimator of the quadratic variation that is based on the the- ory of Markov chains. The estimator is motivated by some general results concerning filtering contaminated semimartingales. Specifically, we show that filtering can in prin- ciple remove the effects of market...... microstructure noise in a general framework where little is assumed about the noise. For the practical implementation, we adopt the dis- crete Markov chain model that is well suited for the analysis of financial high-frequency prices. The Markov chain framework facilitates simple expressions and elegant analyti...

20. Markov Chain Monte Carlo Methods

ter of the 20th century, due to rapid developments in computing technology ... early part of this development saw a host of Monte ... These iterative. Monte Carlo procedures typically generate a random se- quence with the Markov property such that the Markov chain is ergodic with a limiting distribution coinciding with the ...

1. Bibliometric Application of Markov Chains.

Science.gov (United States)

Pao, Miranda Lee; McCreery, Laurie

1986-01-01

A rudimentary description of Markov Chains is presented in order to introduce its use to describe and to predict authors' movements among subareas of the discipline of ethnomusicology. Other possible applications are suggested. (Author)

2. Markov chains and mixing times

CERN Document Server

Levin, David A; Wilmer, Elizabeth L

2009-01-01

This book is an introduction to the modern approach to the theory of Markov chains. The main goal of this approach is to determine the rate of convergence of a Markov chain to the stationary distribution as a function of the size and geometry of the state space. The authors develop the key tools for estimating convergence times, including coupling, strong stationary times, and spectral methods. Whenever possible, probabilistic methods are emphasized. The book includes many examples and provides brief introductions to some central models of statistical mechanics. Also provided are accounts of r

3. Markov Chain Monte Carlo Methods

GENERAL ! ARTICLE. Markov Chain Monte Carlo Methods. 3. Statistical Concepts. K B Athreya, Mohan Delampady and T Krishnan. K B Athreya is a Professor at. Cornell University. His research interests include mathematical analysis, probability theory and its application and statistics. He enjoys writing for Resonance.

4. Markov Chain Monte Carlo Methods

2. The Markov Chain Case. K B Athreya, Mohan Delampady and T Krishnan. K B Athreya is a Professor at. Cornell University. His research interests include mathematical analysis, probability theory and its application and statistics. He enjoys writing for Resonance. His spare time is spent listening to Indian classical music.

5. Markov Chain Monte Carlo Methods

Systat Software Asia-Pacific. (P) Ltd., in Bangalore, where the technical work for the development of the ... Markov chain structure) with applications to integration including integration in a Bayesian context. In Pa.rt 2, ... The applications of MCMC to Bayesian infer- ence will have to wait for the concluding pa,rt of this series.

6. Markov chains and mixing times

CERN Document Server

Levin, David A

2017-01-01

Markov Chains and Mixing Times is a magical book, managing to be both friendly and deep. It gently introduces probabilistic techniques so that an outsider can follow. At the same time, it is the first book covering the geometric theory of Markov chains and has much that will be new to experts. It is certainly THE book that I will use to teach from. I recommend it to all comers, an amazing achievement. -Persi Diaconis, Mary V. Sunseri Professor of Statistics and Mathematics, Stanford University Mixing times are an active research topic within many fields from statistical physics to the theory of algorithms, as well as having intrinsic interest within mathematical probability and exploiting discrete analogs of important geometry concepts. The first edition became an instant classic, being accessible to advanced undergraduates and yet bringing readers close to current research frontiers. This second edition adds chapters on monotone chains, the exclusion process and hitting time parameters. Having both exercises...

7. Consistency and Refinement for Interval Markov Chains

DEFF Research Database (Denmark)

Delahaye, Benoit; Larsen, Kim Guldstrand; Legay, Axel

2012-01-01

Interval Markov Chains (IMC), or Markov Chains with probability intervals in the transition matrix, are the base of a classic specification theory for probabilistic systems [18]. The standard semantics of IMCs assigns to a specification the set of all Markov Chains that satisfy its interval...

8. Markov Chain Ontology Analysis (MCOA

Directory of Open Access Journals (Sweden)

Frost H

2012-02-01

Full Text Available Abstract Background Biomedical ontologies have become an increasingly critical lens through which researchers analyze the genomic, clinical and bibliographic data that fuels scientific research. Of particular relevance are methods, such as enrichment analysis, that quantify the importance of ontology classes relative to a collection of domain data. Current analytical techniques, however, remain limited in their ability to handle many important types of structural complexity encountered in real biological systems including class overlaps, continuously valued data, inter-instance relationships, non-hierarchical relationships between classes, semantic distance and sparse data. Results In this paper, we describe a methodology called Markov Chain Ontology Analysis (MCOA and illustrate its use through a MCOA-based enrichment analysis application based on a generative model of gene activation. MCOA models the classes in an ontology, the instances from an associated dataset and all directional inter-class, class-to-instance and inter-instance relationships as a single finite ergodic Markov chain. The adjusted transition probability matrix for this Markov chain enables the calculation of eigenvector values that quantify the importance of each ontology class relative to other classes and the associated data set members. On both controlled Gene Ontology (GO data sets created with Escherichia coli, Drosophila melanogaster and Homo sapiens annotations and real gene expression data extracted from the Gene Expression Omnibus (GEO, the MCOA enrichment analysis approach provides the best performance of comparable state-of-the-art methods. Conclusion A methodology based on Markov chain models and network analytic metrics can help detect the relevant signal within large, highly interdependent and noisy data sets and, for applications such as enrichment analysis, has been shown to generate superior performance on both real and simulated data relative to existing

9. Markov Chain Ontology Analysis (MCOA).

Science.gov (United States)

Frost, H Robert; McCray, Alexa T

2012-02-03

Biomedical ontologies have become an increasingly critical lens through which researchers analyze the genomic, clinical and bibliographic data that fuels scientific research. Of particular relevance are methods, such as enrichment analysis, that quantify the importance of ontology classes relative to a collection of domain data. Current analytical techniques, however, remain limited in their ability to handle many important types of structural complexity encountered in real biological systems including class overlaps, continuously valued data, inter-instance relationships, non-hierarchical relationships between classes, semantic distance and sparse data. In this paper, we describe a methodology called Markov Chain Ontology Analysis (MCOA) and illustrate its use through a MCOA-based enrichment analysis application based on a generative model of gene activation. MCOA models the classes in an ontology, the instances from an associated dataset and all directional inter-class, class-to-instance and inter-instance relationships as a single finite ergodic Markov chain. The adjusted transition probability matrix for this Markov chain enables the calculation of eigenvector values that quantify the importance of each ontology class relative to other classes and the associated data set members. On both controlled Gene Ontology (GO) data sets created with Escherichia coli, Drosophila melanogaster and Homo sapiens annotations and real gene expression data extracted from the Gene Expression Omnibus (GEO), the MCOA enrichment analysis approach provides the best performance of comparable state-of-the-art methods. A methodology based on Markov chain models and network analytic metrics can help detect the relevant signal within large, highly interdependent and noisy data sets and, for applications such as enrichment analysis, has been shown to generate superior performance on both real and simulated data relative to existing state-of-the-art approaches.

10. Spectral methods for quantum Markov chains

International Nuclear Information System (INIS)

Szehr, Oleg

2014-01-01

The aim of this project is to contribute to our understanding of quantum time evolutions, whereby we focus on quantum Markov chains. The latter constitute a natural generalization of the ubiquitous concept of a classical Markov chain to describe evolutions of quantum mechanical systems. We contribute to the theory of such processes by introducing novel methods that allow us to relate the eigenvalue spectrum of the transition map to convergence as well as stability properties of the Markov chain.

11. An interlacing theorem for reversible Markov chains

International Nuclear Information System (INIS)

Grone, Robert; Salamon, Peter; Hoffmann, Karl Heinz

2008-01-01

Reversible Markov chains are an indispensable tool in the modeling of a vast class of physical, chemical, biological and statistical problems. Examples include the master equation descriptions of relaxing physical systems, stochastic optimization algorithms such as simulated annealing, chemical dynamics of protein folding and Markov chain Monte Carlo statistical estimation. Very often the large size of the state spaces requires the coarse graining or lumping of microstates into fewer mesoscopic states, and a question of utmost importance for the validity of the physical model is how the eigenvalues of the corresponding stochastic matrix change under this operation. In this paper we prove an interlacing theorem which gives explicit bounds on the eigenvalues of the lumped stochastic matrix. (fast track communication)

12. A scaling analysis of a cat and mouse Markov chain

NARCIS (Netherlands)

Litvak, Nelli; Robert, Philippe

2012-01-01

If ($C_n$) a Markov chain on a discrete state space $S$, a Markov chain ($C_n, M_n$) on the product space $S \\times S$, the cat and mouse Markov chain, is constructed. The first coordinate of this Markov chain behaves like the original Markov chain and the second component changes only when both

13. Markov Chains on Orbits of Permutation Groups

OpenAIRE

Niepert, Mathias

2014-01-01

We present a novel approach to detecting and utilizing symmetries in probabilistic graphical models with two main contributions. First, we present a scalable approach to computing generating sets of permutation groups representing the symmetries of graphical models. Second, we introduce orbital Markov chains, a novel family of Markov chains leveraging model symmetries to reduce mixing times. We establish an insightful connection between model symmetries and rapid mixing of orbital Markov chai...

14. Quantum Markov Chain Mixing and Dissipative Engineering

DEFF Research Database (Denmark)

Kastoryano, Michael James

2012-01-01

(stationary states). The aim of Markov chain mixing is to obtain (upper and/or lower) bounds on the number of steps it takes for the Markov chain to reach a stationary state. The natural quantum extensions of these notions are density matrices and quantum channels. We set out to develop a general mathematical......This thesis is the fruit of investigations on the extension of ideas of Markov chain mixing to the quantum setting, and its application to problems of dissipative engineering. A Markov chain describes a statistical process where the probability of future events depends only on the state...... framework for studying quantum Markov chain mixing. We introduce two new distance measures into the quantum setting; the quantum $\\chi^2$-divergence and Hilbert's projective metric. With the help of these distance measures, we are able to derive some basic bounds on the the mixing times of quantum channels...

15. Markov chain Monte Carlo simulation for Bayesian Hidden Markov Models

Science.gov (United States)

Chan, Lay Guat; Ibrahim, Adriana Irawati Nur Binti

2016-10-01

A hidden Markov model (HMM) is a mixture model which has a Markov chain with finite states as its mixing distribution. HMMs have been applied to a variety of fields, such as speech and face recognitions. The main purpose of this study is to investigate the Bayesian approach to HMMs. Using this approach, we can simulate from the parameters' posterior distribution using some Markov chain Monte Carlo (MCMC) sampling methods. HMMs seem to be useful, but there are some limitations. Therefore, by using the Mixture of Dirichlet processes Hidden Markov Model (MDPHMM) based on Yau et. al (2011), we hope to overcome these limitations. We shall conduct a simulation study using MCMC methods to investigate the performance of this model.

16. Honest Importance Sampling with Multiple Markov Chains.

Science.gov (United States)

Tan, Aixin; Doss, Hani; Hobert, James P

2015-01-01

Importance sampling is a classical Monte Carlo technique in which a random sample from one probability density, π 1 , is used to estimate an expectation with respect to another, π . The importance sampling estimator is strongly consistent and, as long as two simple moment conditions are satisfied, it obeys a central limit theorem (CLT). Moreover, there is a simple consistent estimator for the asymptotic variance in the CLT, which makes for routine computation of standard errors. Importance sampling can also be used in the Markov chain Monte Carlo (MCMC) context. Indeed, if the random sample from π 1 is replaced by a Harris ergodic Markov chain with invariant density π 1 , then the resulting estimator remains strongly consistent. There is a price to be paid however, as the computation of standard errors becomes more complicated. First, the two simple moment conditions that guarantee a CLT in the iid case are not enough in the MCMC context. Second, even when a CLT does hold, the asymptotic variance has a complex form and is difficult to estimate consistently. In this paper, we explain how to use regenerative simulation to overcome these problems. Actually, we consider a more general set up, where we assume that Markov chain samples from several probability densities, π 1 , …, π k , are available. We construct multiple-chain importance sampling estimators for which we obtain a CLT based on regeneration. We show that if the Markov chains converge to their respective target distributions at a geometric rate, then under moment conditions similar to those required in the iid case, the MCMC-based importance sampling estimator obeys a CLT. Furthermore, because the CLT is based on a regenerative process, there is a simple consistent estimator of the asymptotic variance. We illustrate the method with two applications in Bayesian sensitivity analysis. The first concerns one-way random effects models under different priors. The second involves Bayesian variable

17. Markov chains analytic and Monte Carlo computations

CERN Document Server

Graham, Carl

2014-01-01

Markov Chains: Analytic and Monte Carlo Computations introduces the main notions related to Markov chains and provides explanations on how to characterize, simulate, and recognize them. Starting with basic notions, this book leads progressively to advanced and recent topics in the field, allowing the reader to master the main aspects of the classical theory. This book also features: Numerous exercises with solutions as well as extended case studies.A detailed and rigorous presentation of Markov chains with discrete time and state space.An appendix presenting probabilistic notions that are nec

18. Markov chains models, algorithms and applications

CERN Document Server

Ching, Wai-Ki; Ng, Michael K; Siu, Tak-Kuen

2013-01-01

This new edition of Markov Chains: Models, Algorithms and Applications has been completely reformatted as a text, complete with end-of-chapter exercises, a new focus on management science, new applications of the models, and new examples with applications in financial risk management and modeling of financial data.This book consists of eight chapters.  Chapter 1 gives a brief introduction to the classical theory on both discrete and continuous time Markov chains. The relationship between Markov chains of finite states and matrix theory will also be highlighted. Some classical iterative methods

19. Adaptive Markov Chain Monte Carlo

KAUST Repository

2016-08-08

A substantial interpretation of electromagnetic induction (EMI) measurements requires quantifying optimal model parameters and uncertainty of a nonlinear inverse problem. For this purpose, an adaptive Bayesian Markov chain Monte Carlo (MCMC) algorithm is used to assess multi-orientation and multi-offset EMI measurements in an agriculture field with non-saline and saline soil. In the MCMC simulations, posterior distribution was computed using Bayes rule. The electromagnetic forward model based on the full solution of Maxwell\\'s equations was used to simulate the apparent electrical conductivity measured with the configurations of EMI instrument, the CMD mini-Explorer. The model parameters and uncertainty for the three-layered earth model are investigated by using synthetic data. Our results show that in the scenario of non-saline soil, the parameters of layer thickness are not well estimated as compared to layers electrical conductivity because layer thicknesses in the model exhibits a low sensitivity to the EMI measurements, and is hence difficult to resolve. Application of the proposed MCMC based inversion to the field measurements in a drip irrigation system demonstrate that the parameters of the model can be well estimated for the saline soil as compared to the non-saline soil, and provide useful insight about parameter uncertainty for the assessment of the model outputs.

20. Bayesian tomography by interacting Markov chains

Science.gov (United States)

Romary, T.

2017-12-01

In seismic tomography, we seek to determine the velocity of the undergound from noisy first arrival travel time observations. In most situations, this is an ill posed inverse problem that admits several unperfect solutions. Given an a priori distribution over the parameters of the velocity model, the Bayesian formulation allows to state this problem as a probabilistic one, with a solution under the form of a posterior distribution. The posterior distribution is generally high dimensional and may exhibit multimodality. Moreover, as it is known only up to a constant, the only sensible way to addressthis problem is to try to generate simulations from the posterior. The natural tools to perform these simulations are Monte Carlo Markov chains (MCMC). Classical implementations of MCMC algorithms generally suffer from slow mixing: the generated states are slow to enter the stationary regime, that is to fit the observations, and when one mode of the posterior is eventually identified, it may become difficult to visit others. Using a varying temperature parameter relaxing the constraint on the data may help to enter the stationary regime. Besides, the sequential nature of MCMC makes them ill fitted toparallel implementation. Running a large number of chains in parallel may be suboptimal as the information gathered by each chain is not mutualized. Parallel tempering (PT) can be seen as a first attempt to make parallel chains at different temperatures communicate but only exchange information between current states. In this talk, I will show that PT actually belongs to a general class of interacting Markov chains algorithm. I will also show that this class enables to design interacting schemes that can take advantage of the whole history of the chain, by authorizing exchanges toward already visited states. The algorithms will be illustrated with toy examples and an application to first arrival traveltime tomography.

1. A scaling analysis of a cat and mouse Markov chain

NARCIS (Netherlands)

Litvak, Nelli; Robert, Philippe

Motivated by an original on-line page-ranking algorithm, starting from an arbitrary Markov chain $(C_n)$ on a discrete state space ${\\cal S}$, a Markov chain $(C_n,M_n)$ on the product space ${\\cal S}^2$, the cat and mouse Markov chain, is constructed. The first coordinate of this Markov chain

2. Markov chains and decision processes for engineers and managers

CERN Document Server

Sheskin, Theodore J

2010-01-01

Markov Chain Structure and ModelsHistorical NoteStates and TransitionsModel of the WeatherRandom WalksEstimating Transition ProbabilitiesMultiple-Step Transition ProbabilitiesState Probabilities after Multiple StepsClassification of StatesMarkov Chain StructureMarkov Chain ModelsProblemsReferencesRegular Markov ChainsSteady State ProbabilitiesFirst Passage to a Target StateProblemsReferencesReducible Markov ChainsCanonical Form of the Transition MatrixTh

3. Approximating Markov Chains: What and why

International Nuclear Information System (INIS)

Pincus, S.

1996-01-01

Much of the current study of dynamical systems is focused on geometry (e.g., chaos and bifurcations) and ergodic theory. Yet dynamical systems were originally motivated by an attempt to open-quote open-quote solve,close-quote close-quote or at least understand, a discrete-time analogue of differential equations. As such, numerical, analytical solution techniques for dynamical systems would seem desirable. We discuss an approach that provides such techniques, the approximation of dynamical systems by suitable finite state Markov Chains. Steady state distributions for these Markov Chains, a straightforward calculation, will converge to the true dynamical system steady state distribution, with appropriate limit theorems indicated. Thus (i) approximation by a computable, linear map holds the promise of vastly faster steady state solutions for nonlinear, multidimensional differential equations; (ii) the solution procedure is unaffected by the presence or absence of a probability density function for the attractor, entirely skirting singularity, fractal/multifractal, and renormalization considerations. The theoretical machinery underpinning this development also implies that under very general conditions, steady state measures are weakly continuous with control parameter evolution. This means that even though a system may change periodicity, or become chaotic in its limiting behavior, such statistical parameters as the mean, standard deviation, and tail probabilities change continuously, not abruptly with system evolution. copyright 1996 American Institute of Physics

4. Efficient Markov Chain Monte Carlo Sampling for Hierarchical Hidden Markov Models

OpenAIRE

Turek, Daniel; de Valpine, Perry; Paciorek, Christopher J.

2016-01-01

Traditional Markov chain Monte Carlo (MCMC) sampling of hidden Markov models (HMMs) involves latent states underlying an imperfect observation process, and generates posterior samples for top-level parameters concurrently with nuisance latent variables. When potentially many HMMs are embedded within a hierarchical model, this can result in prohibitively long MCMC runtimes. We study combinations of existing methods, which are shown to vastly improve computational efficiency for these hierarchi...

5. Markov chains for testing redundant software

Science.gov (United States)

White, Allan L.; Sjogren, Jon A.

1988-01-01

A preliminary design for a validation experiment has been developed that addresses several problems unique to assuring the extremely high quality of multiple-version programs in process-control software. The procedure uses Markov chains to model the error states of the multiple version programs. The programs are observed during simulated process-control testing, and estimates are obtained for the transition probabilities between the states of the Markov chain. The experimental Markov chain model is then expanded into a reliability model that takes into account the inertia of the system being controlled. The reliability of the multiple version software is computed from this reliability model at a given confidence level using confidence intervals obtained for the transition probabilities during the experiment. An example demonstrating the method is provided.

6. Multi-rate Poisson tree processes for single-locus species delimitation under maximum likelihood and Markov chain Monte Carlo.

Science.gov (United States)

Kapli, P; Lutteropp, S; Zhang, J; Kobert, K; Pavlidis, P; Stamatakis, A; Flouri, T

2017-06-01

In recent years, molecular species delimitation has become a routine approach for quantifying and classifying biodiversity. Barcoding methods are of particular importance in large-scale surveys as they promote fast species discovery and biodiversity estimates. Among those, distance-based methods are the most common choice as they scale well with large datasets; however, they are sensitive to similarity threshold parameters and they ignore evolutionary relationships. The recently introduced "Poisson Tree Processes" (PTP) method is a phylogeny-aware approach that does not rely on such thresholds. Yet, two weaknesses of PTP impact its accuracy and practicality when applied to large datasets; it does not account for divergent intraspecific variation and is slow for a large number of sequences. We introduce the multi-rate PTP (mPTP), an improved method that alleviates the theoretical and technical shortcomings of PTP. It incorporates different levels of intraspecific genetic diversity deriving from differences in either the evolutionary history or sampling of each species. Results on empirical data suggest that mPTP is superior to PTP and popular distance-based methods as it, consistently yields more accurate delimitations with respect to the taxonomy (i.e., identifies more taxonomic species, infers species numbers closer to the taxonomy). Moreover, mPTP does not require any similarity threshold as input. The novel dynamic programming algorithm attains a speedup of at least five orders of magnitude compared to PTP, allowing it to delimit species in large (meta-) barcoding data. In addition, Markov Chain Monte Carlo sampling provides a comprehensive evaluation of the inferred delimitation in just a few seconds for millions of steps, independently of tree size. mPTP is implemented in C and is available for download at http://github.com/Pas-Kapli/mptp under the GNU Affero 3 license. A web-service is available at http://mptp.h-its.org . : paschalia.kapli@h-its.org or

7. Differential Evolution Markov Chain with snooker updater and fewer chains

NARCIS (Netherlands)

Braak, ter C.J.F.; Vrugt, J.A.

2008-01-01

Differential Evolution Markov Chain (DE-MC) is an adaptive MCMC algorithm, in which multiple chains are run in parallel. Standard DE-MC requires at least N=2d chains to be run in parallel, where d is the dimensionality of the posterior. This paper extends DE-MC with a snooker updater and shows by

8. On the Markov Chain Monte Carlo (MCMC) method

In this article, we give an introduction to Monte Carlo techniques with special emphasis on. Markov Chain Monte Carlo (MCMC). Since the latter needs Markov chains with state space that is R or Rd and most text books on Markov chains do not discuss such chains, we have included a short appendix that gives basic ...

9. Bisimulation and Simulation Relations for Markov Chains

NARCIS (Netherlands)

Baier, Christel; Hermanns, H.; Katoen, Joost P.; Wolf, Verena; Aceto, L.; Gordon, A.

2006-01-01

Formal notions of bisimulation and simulation relation play a central role for any kind of process algebra. This short paper sketches the main concepts for bisimulation and simulation relations for probabilistic systems, modelled by discrete- or continuous-time Markov chains.

10. Model Checking Structured Infinite Markov Chains

NARCIS (Netherlands)

Remke, Anne Katharina Ingrid

2008-01-01

In the past probabilistic model checking hast mostly been restricted to finite state models. This thesis explores the possibilities of model checking with continuous stochastic logic (CSL) on infinite-state Markov chains. We present an in-depth treatment of model checking algorithms for two special

11. A Martingale Decomposition of Discrete Markov Chains

DEFF Research Database (Denmark)

Hansen, Peter Reinhard

We consider a multivariate time series whose increments are given from a homogeneous Markov chain. We show that the martingale component of this process can be extracted by a filtering method and establish the corresponding martingale decomposition in closed-form. This representation is useful...

12. Revisiting Weak Simulation for Substochastic Markov Chains

DEFF Research Database (Denmark)

Jansen, David N.; Song, Lei; Zhang, Lijun

2013-01-01

of the logic PCTL\\x, and its completeness was conjectured. We revisit this result and show that soundness does not hold in general, but only for Markov chains without divergence. It is refuted for some systems with substochastic distributions. Moreover, we provide a counterexample to completeness...

13. Assessing significance in a Markov chain without mixing.

Science.gov (United States)

Chikina, Maria; Frieze, Alan; Pegden, Wesley

2017-03-14

We present a statistical test to detect that a presented state of a reversible Markov chain was not chosen from a stationary distribution. In particular, given a value function for the states of the Markov chain, we would like to show rigorously that the presented state is an outlier with respect to the values, by establishing a [Formula: see text] value under the null hypothesis that it was chosen from a stationary distribution of the chain. A simple heuristic used in practice is to sample ranks of states from long random trajectories on the Markov chain and compare these with the rank of the presented state; if the presented state is a [Formula: see text] outlier compared with the sampled ranks (its rank is in the bottom [Formula: see text] of sampled ranks), then this observation should correspond to a [Formula: see text] value of [Formula: see text] This significance is not rigorous, however, without good bounds on the mixing time of the Markov chain. Our test is the following: Given the presented state in the Markov chain, take a random walk from the presented state for any number of steps. We prove that observing that the presented state is an [Formula: see text]-outlier on the walk is significant at [Formula: see text] under the null hypothesis that the state was chosen from a stationary distribution. We assume nothing about the Markov chain beyond reversibility and show that significance at [Formula: see text] is best possible in general. We illustrate the use of our test with a potential application to the rigorous detection of gerrymandering in Congressional districting.

14. Performance evaluation:= (process algebra + model checking) x Markov chains

NARCIS (Netherlands)

Hermanns, H.; Larsen, K.G.; Nielsen, Mogens; Katoen, Joost P.

2001-01-01

Markov chains are widely used in practice to determine system performance and reliability characteristics. The vast majority of applications considers continuous-time Markov chains (CTMCs). This tutorial paper shows how successful model specification and analysis techniques from concurrency theory

15. Continuously monitored barrier options under Markov processes

NARCIS (Netherlands)

Mijatović, A.; Pistorius, M.

2011-01-01

In this paper, we present an algorithm for pricing barrier options in one-dimensional Markov models. The approach rests on the construction of an approximating continuous-time Markov chain that closely follows the dynamics of the given Markov model. We illustrate the method by implementing it for a

16. The How and Why of Interactive Markov Chains

NARCIS (Netherlands)

Hermanns, H.; Katoen, Joost P.; de Boer, F.S; Bonsangue, S.H.; Leuschel, M

2010-01-01

This paper reviews the model of interactive Markov chains (IMCs, for short), an extension of labelled transition systems with exponentially delayed transitions. We show that IMCs are closed under parallel composition and hiding, and show how IMCs can be compositionally aggregated prior to analysis

17. MARKOV CHAIN PORTFOLIO LIQUIDITY OPTIMIZATION MODEL

Directory of Open Access Journals (Sweden)

Eder Oliveira Abensur

2014-05-01

Full Text Available The international financial crisis of September 2008 and May 2010 showed the importance of liquidity as an attribute to be considered in portfolio decisions. This study proposes an optimization model based on available public data, using Markov chain and Genetic Algorithms concepts as it considers the classic duality of risk versus return and incorporating liquidity costs. The work intends to propose a multi-criterion non-linear optimization model using liquidity based on a Markov chain. The non-linear model was tested using Genetic Algorithms with twenty five Brazilian stocks from 2007 to 2009. The results suggest that this is an innovative development methodology and useful for developing an efficient and realistic financial portfolio, as it considers many attributes such as risk, return and liquidity.

18. Handbook of Markov chain Monte Carlo

CERN Document Server

Brooks, Steve

2011-01-01

""Handbook of Markov Chain Monte Carlo"" brings together the major advances that have occurred in recent years while incorporating enough introductory material for new users of MCMC. Along with thorough coverage of the theoretical foundations and algorithmic and computational methodology, this comprehensive handbook includes substantial realistic case studies from a variety of disciplines. These case studies demonstrate the application of MCMC methods and serve as a series of templates for the construction, implementation, and choice of MCMC methodology.

19. Numerical methods in Markov chain modeling

Science.gov (United States)

Philippe, Bernard; Saad, Youcef; Stewart, William J.

1989-01-01

Several methods for computing stationary probability distributions of Markov chains are described and compared. The main linear algebra problem consists of computing an eigenvector of a sparse, usually nonsymmetric, matrix associated with a known eigenvalue. It can also be cast as a problem of solving a homogeneous singular linear system. Several methods based on combinations of Krylov subspace techniques are presented. The performance of these methods on some realistic problems are compared.

20. Second Order Optimality in Markov Decision Chains

Czech Academy of Sciences Publication Activity Database

2017-01-01

Roč. 53, č. 6 (2017), s. 1086-1099 ISSN 0023-5954 R&D Projects: GA ČR GA15-10331S Institutional support: RVO:67985556 Keywords : Markov decision chains * second order optimality * optimalilty conditions for transient, discounted and average models * policy and value iterations Subject RIV: BB - Applied Statistics, Operational Research OBOR OECD: Statistics and probability Impact factor: 0.379, year: 2016 http://library.utia.cas.cz/separaty/2017/E/sladky-0485146.pdf

1. Bayesian Posterior Distributions Without Markov Chains

OpenAIRE

Cole, Stephen R.; Chu, Haitao; Greenland, Sander; Hamra, Ghassan; Richardson, David B.

2012-01-01

Bayesian posterior parameter distributions are often simulated using Markov chain Monte Carlo (MCMC) methods. However, MCMC methods are not always necessary and do not help the uninitiated understand Bayesian inference. As a bridge to understanding Bayesian inference, the authors illustrate a transparent rejection sampling method. In example 1, they illustrate rejection sampling using 36 cases and 198 controls from a case-control study (1976–1983) assessing the relation between residential ex...

2. Temperature scaling method for Markov chains.

Science.gov (United States)

Crosby, Lonnie D; Windus, Theresa L

2009-01-22

The use of ab initio potentials in Monte Carlo simulations aimed at investigating the nucleation kinetics of water clusters is complicated by the computational expense of the potential energy determinations. Furthermore, the common desire to investigate the temperature dependence of kinetic properties leads to an urgent need to reduce the expense of performing simulations at many different temperatures. A method is detailed that allows a Markov chain (obtained via Monte Carlo) at one temperature to be scaled to other temperatures of interest without the need to perform additional large simulations. This Markov chain temperature-scaling (TeS) can be generally applied to simulations geared for numerous applications. This paper shows the quality of results which can be obtained by TeS and the possible quantities which may be extracted from scaled Markov chains. Results are obtained for a 1-D analytical potential for which the exact solutions are known. Also, this method is applied to water clusters consisting of between 2 and 5 monomers, using Dynamical Nucleation Theory to determine the evaporation rate constant for monomer loss. Although ab initio potentials are not utilized in this paper, the benefit of this method is made apparent by using the Dang-Chang polarizable classical potential for water to obtain statistical properties at various temperatures.

3. Dynamic system evolution and markov chain approximation

Directory of Open Access Journals (Sweden)

Roderick V. Nicholas Melnik

1998-01-01

Full Text Available In this paper computational aspects of the mathematical modelling of dynamic system evolution have been considered as a problem in information theory. The construction of mathematical models is treated as a decision making process with limited available information.The solution of the problem is associated with a computational model based on heuristics of a Markov Chain in a discrete space–time of events. A stable approximation of the chain has been derived and the limiting cases are discussed. An intrinsic interconnection of constructive, sequential, and evolutionary approaches in related optimization problems provides new challenges for future work.

4. Markov Chain Analysis of Musical Dice Games

Science.gov (United States)

Volchenkov, D.; Dawin, J. R.

2012-07-01

A system for using dice to compose music randomly is known as the musical dice game. The discrete time MIDI models of 804 pieces of classical music written by 29 composers have been encoded into the transition matrices and studied by Markov chains. Contrary to human languages, entropy dominates over redundancy, in the musical dice games based on the compositions of classical music. The maximum complexity is achieved on the blocks consisting of just a few notes (8 notes, for the musical dice games generated over Bach's compositions). First passage times to notes can be used to resolve tonality and feature a composer.

5. Pseudo-extended Markov chain Monte Carlo

OpenAIRE

Nemeth, Christopher; Lindsten, Fredrik; Filippone, Maurizio; Hensman, James

2017-01-01

Sampling from the posterior distribution using Markov chain Monte Carlo (MCMC) methods can require an exhaustive number of iterations to fully explore the correct posterior. This is often the case when the posterior of interest is multi-modal, as the MCMC sampler can become trapped in a local mode for a large number of iterations. In this paper, we introduce the pseudo-extended MCMC method as an approach for improving the mixing of the MCMC sampler in complex posterior distributions. The pseu...

6. Uncertainty of the Soil–Water Characteristic Curve and Its Effects on Slope Seepage and Stability Analysis under Conditions of Rainfall Using the Markov Chain Monte Carlo Method

Directory of Open Access Journals (Sweden)

Weiping Liu

2017-10-01

Full Text Available It is important to determine the soil–water characteristic curve (SWCC for analyzing slope seepage and stability under the conditions of rainfall. However, SWCCs exhibit high uncertainty because of complex influencing factors, which has not been previously considered in slope seepage and stability analysis under conditions of rainfall. This study aimed to evaluate the uncertainty of the SWCC and its effects on the seepage and stability analysis of an unsaturated soil slope under conditions of rainfall. The SWCC model parameters were treated as random variables. An uncertainty evaluation of the parameters was conducted based on the Bayesian approach and the Markov chain Monte Carlo (MCMC method. Observed data from granite residual soil were used to test the uncertainty of the SWCC. Then, different confidence intervals for the model parameters of the SWCC were constructed. The slope seepage and stability analysis under conditions of rainfall with the SWCC of different confidence intervals was investigated using finite element software (SEEP/W and SLOPE/W. The results demonstrated that SWCC uncertainty had significant effects on slope seepage and stability. In general, the larger the percentile value, the greater the reduction of negative pore-water pressure in the soil layer and the lower the safety factor of the slope. Uncertainties in the model parameters of the SWCC can lead to obvious errors in predicted pore-water pressure profiles and the estimated safety factor of the slope under conditions of rainfall.

7. A Markov Chain Model for Contagion

Directory of Open Access Journals (Sweden)

Angelos Dassios

2014-11-01

Full Text Available We introduce a bivariate Markov chain counting process with contagion for modelling the clustering arrival of loss claims with delayed settlement for an insurance company. It is a general continuous-time model framework that also has the potential to be applicable to modelling the clustering arrival of events, such as jumps, bankruptcies, crises and catastrophes in finance, insurance and economics with both internal contagion risk and external common risk. Key distributional properties, such as the moments and probability generating functions, for this process are derived. Some special cases with explicit results and numerical examples and the motivation for further actuarial applications are also discussed. The model can be considered a generalisation of the dynamic contagion process introduced by Dassios and Zhao (2011.

8. Irreversible Local Markov Chains with Rapid Convergence towards Equilibrium

Science.gov (United States)

Kapfer, Sebastian C.; Krauth, Werner

2017-12-01

We study the continuous one-dimensional hard-sphere model and present irreversible local Markov chains that mix on faster time scales than the reversible heat bath or Metropolis algorithms. The mixing time scales appear to fall into two distinct universality classes, both faster than for reversible local Markov chains. The event-chain algorithm, the infinitesimal limit of one of these Markov chains, belongs to the class presenting the fastest decay. For the lattice-gas limit of the hard-sphere model, reversible local Markov chains correspond to the symmetric simple exclusion process (SEP) with periodic boundary conditions. The two universality classes for irreversible Markov chains are realized by the totally asymmetric SEP (TASEP), and by a faster variant (lifted TASEP) that we propose here. We discuss how our irreversible hard-sphere Markov chains generalize to arbitrary repulsive pair interactions and carry over to higher dimensions through the concept of lifted Markov chains and the recently introduced factorized Metropolis acceptance rule.

9. Modeling Uncertainty of Directed Movement via Markov Chains

Directory of Open Access Journals (Sweden)

YIN Zhangcai

2015-10-01

Full Text Available Probabilistic time geography (PTG is suggested as an extension of (classical time geography, in order to present the uncertainty of an agent located at the accessible position by probability. This may provide a quantitative basis for most likely finding an agent at a location. In recent years, PTG based on normal distribution or Brown bridge has been proposed, its variance, however, is irrelevant with the agent's speed or divergent with the increase of the speed; so they are difficult to take into account application pertinence and stability. In this paper, a new method is proposed to model PTG based on Markov chain. Firstly, a bidirectional conditions Markov chain is modeled, the limit of which, when the moving speed is large enough, can be regarded as the Brown bridge, thus has the characteristics of digital stability. Then, the directed movement is mapped to Markov chains. The essential part is to build step length, the state space and transfer matrix of Markov chain according to the space and time position of directional movement, movement speed information, to make sure the Markov chain related to the movement speed. Finally, calculating continuously the probability distribution of the directed movement at any time by the Markov chains, it can be get the possibility of an agent located at the accessible position. Experimental results show that, the variance based on Markov chains not only is related to speed, but also is tending towards stability with increasing the agent's maximum speed.

10. Bayesian posterior distributions without Markov chains.

Science.gov (United States)

Cole, Stephen R; Chu, Haitao; Greenland, Sander; Hamra, Ghassan; Richardson, David B

2012-03-01

Bayesian posterior parameter distributions are often simulated using Markov chain Monte Carlo (MCMC) methods. However, MCMC methods are not always necessary and do not help the uninitiated understand Bayesian inference. As a bridge to understanding Bayesian inference, the authors illustrate a transparent rejection sampling method. In example 1, they illustrate rejection sampling using 36 cases and 198 controls from a case-control study (1976-1983) assessing the relation between residential exposure to magnetic fields and the development of childhood cancer. Results from rejection sampling (odds ratio (OR) = 1.69, 95% posterior interval (PI): 0.57, 5.00) were similar to MCMC results (OR = 1.69, 95% PI: 0.58, 4.95) and approximations from data-augmentation priors (OR = 1.74, 95% PI: 0.60, 5.06). In example 2, the authors apply rejection sampling to a cohort study of 315 human immunodeficiency virus seroconverters (1984-1998) to assess the relation between viral load after infection and 5-year incidence of acquired immunodeficiency syndrome, adjusting for (continuous) age at seroconversion and race. In this more complex example, rejection sampling required a notably longer run time than MCMC sampling but remained feasible and again yielded similar results. The transparency of the proposed approach comes at a price of being less broadly applicable than MCMC.

11. Bayesian Posterior Distributions Without Markov Chains

Science.gov (United States)

Cole, Stephen R.; Chu, Haitao; Greenland, Sander; Hamra, Ghassan; Richardson, David B.

2012-01-01

Bayesian posterior parameter distributions are often simulated using Markov chain Monte Carlo (MCMC) methods. However, MCMC methods are not always necessary and do not help the uninitiated understand Bayesian inference. As a bridge to understanding Bayesian inference, the authors illustrate a transparent rejection sampling method. In example 1, they illustrate rejection sampling using 36 cases and 198 controls from a case-control study (1976–1983) assessing the relation between residential exposure to magnetic fields and the development of childhood cancer. Results from rejection sampling (odds ratio (OR) = 1.69, 95% posterior interval (PI): 0.57, 5.00) were similar to MCMC results (OR = 1.69, 95% PI: 0.58, 4.95) and approximations from data-augmentation priors (OR = 1.74, 95% PI: 0.60, 5.06). In example 2, the authors apply rejection sampling to a cohort study of 315 human immunodeficiency virus seroconverters (1984–1998) to assess the relation between viral load after infection and 5-year incidence of acquired immunodeficiency syndrome, adjusting for (continuous) age at seroconversion and race. In this more complex example, rejection sampling required a notably longer run time than MCMC sampling but remained feasible and again yielded similar results. The transparency of the proposed approach comes at a price of being less broadly applicable than MCMC. PMID:22306565

12. Differential evolution Markov chain with snooker updater and fewer chains

Energy Technology Data Exchange (ETDEWEB)

Vrugt, Jasper A [Los Alamos National Laboratory; Ter Braak, Cajo J F [NON LANL

2008-01-01

Differential Evolution Markov Chain (DE-MC) is an adaptive MCMC algorithm, in which multiple chains are run in parallel. Standard DE-MC requires at least N=2d chains to be run in parallel, where d is the dimensionality of the posterior. This paper extends DE-MC with a snooker updater and shows by simulation and real examples that DE-MC can work for d up to 50--100 with fewer parallel chains (e.g. N=3) by exploiting information from their past by generating jumps from differences of pairs of past states. This approach extends the practical applicability of DE-MC and is shown to be about 5--26 times more efficient than the optimal Normal random walk Metropolis sampler for the 97.5% point of a variable from a 25--50 dimensional Student T{sub 3} distribution. In a nonlinear mixed effects model example the approach outperformed a block-updater geared to the specific features of the model.

13. On the Markov Chain Monte Carlo (MCMC) method

Abstract. Markov Chain Monte Carlo (MCMC) is a popular method used to generate samples from arbitrary distributions, which may be speciﬁed indirectly. In this article, we give an introduction to this method along with some examples.

14. Prognostics for Steam Generator Tube Rupture using Markov Chain model

International Nuclear Information System (INIS)

Kim, Gibeom; Heo, Gyunyoung; Kim, Hyeonmin

2016-01-01

This paper will describe the prognostics method for evaluating and forecasting the ageing effect and demonstrate the procedure of prognostics for the Steam Generator Tube Rupture (SGTR) accident. Authors will propose the data-driven method so called MCMC (Markov Chain Monte Carlo) which is preferred to the physical-model method in terms of flexibility and availability. Degradation data is represented as growth of burst probability over time. Markov chain model is performed based on transition probability of state. And the state must be discrete variable. Therefore, burst probability that is continuous variable have to be changed into discrete variable to apply Markov chain model to the degradation data. The Markov chain model which is one of prognostics methods was described and the pilot demonstration for a SGTR accident was performed as a case study. The Markov chain model is strong since it is possible to be performed without physical models as long as enough data are available. However, in the case of the discrete Markov chain used in this study, there must be loss of information while the given data is discretized and assigned to the finite number of states. In this process, original information might not be reflected on prediction sufficiently. This should be noted as the limitation of discrete models. Now we will be studying on other prognostics methods such as GPM (General Path Model) which is also data-driven method as well as the particle filer which belongs to physical-model method and conducting comparison analysis

15. Switching Markov chains for a holistic modeling of SIS unavailability

International Nuclear Information System (INIS)

Mechri, Walid; Simon, Christophe; BenOthman, Kamel

2015-01-01

This paper proposes a holistic approach to model the Safety Instrumented Systems (SIS). The model is based on Switching Markov Chain and integrates several parameters like Common Cause Failure, Imperfect Proof testing, partial proof testing, etc. The basic concepts of Switching Markov Chain applied to reliability analysis are introduced and a model to compute the unavailability for a case study is presented. The proposed Switching Markov Chain allows us to assess the effect of each parameter on the SIS performance. The proposed method ensures the relevance of the results. - Highlights: • A holistic approach to model the unavailability safety systems using Switching Markov chains. • The model integrates several parameters like probability of failure due to the test, the probability of not detecting a failure in a test. • The basic concepts of the Switching Markov Chains are introduced and applied to compute the unavailability for safety systems. • The proposed Switching Markov Chain allows assessing the effect of each parameter on the chemical reactor performance

16. Automated generation of partial Markov chain from high level descriptions

International Nuclear Information System (INIS)

Brameret, P.-A.; Rauzy, A.; Roussel, J.-M.

2015-01-01

We propose an algorithm to generate partial Markov chains from high level implicit descriptions, namely AltaRica models. This algorithm relies on two components. First, a variation on Dijkstra's algorithm to compute shortest paths in a graph. Second, the definition of a notion of distance to select which states must be kept and which can be safely discarded. The proposed method solves two problems at once. First, it avoids a manual construction of Markov chains, which is both tedious and error prone. Second, up the price of acceptable approximations, it makes it possible to push back dramatically the exponential blow-up of the size of the resulting chains. We report experimental results that show the efficiency of the proposed approach. - Highlights: • We generate Markov chains from a higher level safety modeling language (AltaRica). • We use a variation on Dijkstra's algorithm to generate partial Markov chains. • Hence we solve two problems: the first problem is the tedious manual construction of Markov chains. • The second problem is the blow-up of the size of the chains, at the cost of decent approximations. • The experimental results highlight the efficiency of the method

17. A comparison of time-homogeneous Markov chain and Markov process multi-state models.

Science.gov (United States)

Wan, Lijie; Lou, Wenjie; Abner, Erin; Kryscio, Richard J

2016-01-01

Time-homogeneous Markov models are widely used tools for analyzing longitudinal data about the progression of a chronic disease over time. There are advantages to modeling the true disease progression as a discrete time stationary Markov chain. However, one limitation of this method is its inability to handle uneven follow-up assessments or skipped visits. A continuous time version of a homogeneous Markov process multi-state model could be an alternative approach. In this article, we conduct comparisons of these two methods for unevenly spaced observations. Simulations compare the performance of the two methods and two applications illustrate the results.

18. Medical imaging feasibility in body fluids using Markov chains

Science.gov (United States)

2017-02-01

A relatively wide field-of-view and high resolution imaging is necessary for navigating the scope within the body, inspecting tissue, diagnosing disease, and guiding surgical interventions. As the large number of modes available in the multimode fibers (MMF) provides higher resolution, MMFs could replace the millimeters-thick bundles of fibers and lenses currently used in endoscopes. However, attributes of body fluids and obscurants such as blood, impose perennial limitations on resolution and reliability of optical imaging inside human body. To design and evaluate optimum imaging techniques that operate under realistic body fluids conditions, a good understanding of the channel (medium) behavior is necessary. In most prior works, Monte-Carlo Ray Tracing (MCRT) algorithm has been used to analyze the channel behavior. This task is quite numerically intensive. The focus of this paper is on investigating the possibility of simplifying this task by a direct extraction of state transition matrices associated with standard Markov modeling from the MCRT computer simulations programs. We show that by tracing a photon's trajectory in the body fluids via a Markov chain model, the angular distribution can be calculated by simple matrix multiplications. We also demonstrate that the new approach produces result that are close to those obtained by MCRT and other known methods. Furthermore, considering the fact that angular, spatial, and temporal distributions of energy are inter-related, mixing time of Monte- Carlo Markov Chain (MCMC) for different types of liquid concentrations is calculated based on Eigen-analysis of the state transition matrix and possibility of imaging in scattering media are investigated. To this end, we have started to characterize the body fluids that reduce the resolution of imaging [1].

19. Martingales and Markov chains solved exercises and elements of theory

CERN Document Server

Baldi, Paolo; Priouret, Pierre

2002-01-01

CONDITIONAL EXPECTATIONSIntroductionDefinition and First PropertiesConditional Expectations and Conditional LawsExercisesSolutionsSTOCHASTIC PROCESSESGeneral FactsStopping TimesExercisesSolutionsMARTINGALESFirst DefinitionsFirst PropertiesThe Stopping TheoremMaximal InequalitiesSquare Integral MartingalesConvergence TheoremsRegular MartingalesExercisesProblemsSolutionsMARKOV CHAINSTransition Matrices, Markov ChainsConstruction and ExistenceComputations on the Canonical ChainPotential OperatorsPassage ProblemsRecurrence, TransienceRecurrent Irreducible ChainsPeriodicityExercisesProblemsSolution

20. Markov Chain Models for the Stochastic Modeling of Pitting Corrosion

OpenAIRE

Valor, A.; Caleyo, F.; Alfonso, L.; Velázquez, J. C.; Hallen, J. M.

2013-01-01

The stochastic nature of pitting corrosion of metallic structures has been widely recognized. It is assumed that this kind of deterioration retains no memory of the past, so only the current state of the damage influences its future development. This characteristic allows pitting corrosion to be categorized as a Markov process. In this paper, two different models of pitting corrosion, developed using Markov chains, are presented. Firstly, a continuous-time, nonhomogeneous linear growth (pure ...

1. Markov chains and semi-Markov models in time-to-event analysis

Science.gov (United States)

Abner, Erin L.; Charnigo, Richard J.; Kryscio, Richard J.

2014-01-01

A variety of statistical methods are available to investigators for analysis of time-to-event data, often referred to as survival analysis. Kaplan-Meier estimation and Cox proportional hazards regression are commonly employed tools but are not appropriate for all studies, particularly in the presence of competing risks and when multiple or recurrent outcomes are of interest. Markov chain models can accommodate censored data, competing risks (informative censoring), multiple outcomes, recurrent outcomes, frailty, and non-constant survival probabilities. Markov chain models, though often overlooked by investigators in time-to-event analysis, have long been used in clinical studies and have widespread application in other fields. PMID:24818062

2. Stochastic Dynamics through Hierarchically Embedded Markov Chains

Science.gov (United States)

Vasconcelos, Vítor V.; Santos, Fernando P.; Santos, Francisco C.; Pacheco, Jorge M.

2017-02-01

Studying dynamical phenomena in finite populations often involves Markov processes of significant mathematical and/or computational complexity, which rapidly becomes prohibitive with increasing population size or an increasing number of individual configuration states. Here, we develop a framework that allows us to define a hierarchy of approximations to the stationary distribution of general systems that can be described as discrete Markov processes with time invariant transition probabilities and (possibly) a large number of states. This results in an efficient method for studying social and biological communities in the presence of stochastic effects—such as mutations in evolutionary dynamics and a random exploration of choices in social systems—including situations where the dynamics encompasses the existence of stable polymorphic configurations, thus overcoming the limitations of existing methods. The present formalism is shown to be general in scope, widely applicable, and of relevance to a variety of interdisciplinary problems.

3. Markov Chains For Testing Redundant Software

Science.gov (United States)

White, Allan L.; Sjogren, Jon A.

1990-01-01

Preliminary design developed for validation experiment that addresses problems unique to assuring extremely high quality of multiple-version programs in process-control software. Approach takes into account inertia of controlled system in sense it takes more than one failure of control program to cause controlled system to fail. Verification procedure consists of two steps: experimentation (numerical simulation) and computation, with Markov model for each step.

4. Markov chain solution of photon multiple scattering through turbid slabs.

Science.gov (United States)

Lin, Ying; Northrop, William F; Li, Xuesong

2016-11-14

This work introduces a Markov Chain solution to model photon multiple scattering through turbid slabs via anisotropic scattering process, i.e., Mie scattering. Results show that the proposed Markov Chain model agree with commonly used Monte Carlo simulation for various mediums such as medium with non-uniform phase functions and absorbing medium. The proposed Markov Chain solution method successfully converts the complex multiple scattering problem with practical phase functions into a matrix form and solves transmitted/reflected photon angular distributions by matrix multiplications. Such characteristics would potentially allow practical inversions by matrix manipulation or stochastic algorithms where widely applied stochastic methods such as Monte Carlo simulations usually fail, and thus enable practical diagnostics reconstructions such as medical diagnosis, spray analysis, and atmosphere sciences.

5. On a Markov chain roulette-type game

International Nuclear Information System (INIS)

El-Shehawey, M A; El-Shreef, Gh A

2009-01-01

A Markov chain on non-negative integers which arises in a roulette-type game is discussed. The transition probabilities are p 01 = ρ, p Nj = δ Nj , p i,i+W = q, p i,i-1 = p = 1 - q, 1 ≤ W < N, 0 ≤ ρ ≤ 1, N - W < j ≤ N and i = 1, 2, ..., N - W. Using formulae for the determinant of a partitioned matrix, a closed form expression for the solution of the Markov chain roulette-type game is deduced. The present analysis is supported by two mathematical models from tumor growth and war with bargaining

6. First hitting probabilities for semi markov chains and estimation

DEFF Research Database (Denmark)

2017-01-01

. In the latter case, a nonparametric estimator for the first hitting probability is proposed and the asymptotic properties of strong consistency and asymptotic normality are proven. Finally, a numerical application on a five-state system is presented to illustrate the performance of this estimator.......We first consider a stochastic system described by an absorbing semi-Markov chain with finite state space and we introduce the absorption probability to a class of recurrent states. Afterwards, we study the first hitting probability to a subset of states for an irreducible semi-Markov chain...

7. Integration by Parts and Martingale Representation for a Markov Chain

Directory of Open Access Journals (Sweden)

Tak Kuen Siu

2014-01-01

Full Text Available Integration-by-parts formulas for functions of fundamental jump processes relating to a continuous-time, finite-state Markov chain are derived using Bismut's change of measures approach to Malliavin calculus. New expressions for the integrands in stochastic integrals corresponding to representations of martingales for the fundamental jump processes are derived using the integration-by-parts formulas. These results are then applied to hedge contingent claims in a Markov chain financial market, which provides a practical motivation for the developments of the integration-by-parts formulas and the martingale representations.

8. Classification of customer lifetime value models using Markov chain

Science.gov (United States)

Permana, Dony; Pasaribu, Udjianna S.; Indratno, Sapto W.; Suprayogi

2017-10-01

A firm’s potential reward in future time from a customer can be determined by customer lifetime value (CLV). There are some mathematic methods to calculate it. One method is using Markov chain stochastic model. Here, a customer is assumed through some states. Transition inter the states follow Markovian properties. If we are given some states for a customer and the relationships inter states, then we can make some Markov models to describe the properties of the customer. As Markov models, CLV is defined as a vector contains CLV for a customer in the first state. In this paper we make a classification of Markov Models to calculate CLV. Start from two states of customer model, we make develop in many states models. The development a model is based on weaknesses in previous model. Some last models can be expected to describe how real characters of customers in a firm.

9. Influence of credit scoring on the dynamics of Markov chain

Science.gov (United States)

Galina, Timofeeva

2015-11-01

Markov processes are widely used to model the dynamics of a credit portfolio and forecast the portfolio risk and profitability. In the Markov chain model the loan portfolio is divided into several groups with different quality, which determined by presence of indebtedness and its terms. It is proposed that dynamics of portfolio shares is described by a multistage controlled system. The article outlines mathematical formalization of controls which reflect the actions of the bank's management in order to improve the loan portfolio quality. The most important control is the organization of approval procedure of loan applications. The credit scoring is studied as a control affecting to the dynamic system. Different formalizations of "good" and "bad" consumers are proposed in connection with the Markov chain model.

10. Markov chain model for demersal fish catch analysis in Indonesia

Science.gov (United States)

Firdaniza; Gusriani, N.

2018-03-01

As an archipelagic country, Indonesia has considerable potential fishery resources. One of the fish resources that has high economic value is demersal fish. Demersal fish is a fish with a habitat in the muddy seabed. Demersal fish scattered throughout the Indonesian seas. Demersal fish production in each Indonesia’s Fisheries Management Area (FMA) varies each year. In this paper we have discussed the Markov chain model for demersal fish yield analysis throughout all Indonesia’s Fisheries Management Area. Data of demersal fish catch in every FMA in 2005-2014 was obtained from Directorate of Capture Fisheries. From this data a transition probability matrix is determined by the number of transitions from the catch that lie below the median or above the median. The Markov chain model of demersal fish catch data was an ergodic Markov chain model, so that the limiting probability of the Markov chain model can be determined. The predictive value of demersal fishing yields was obtained by calculating the combination of limiting probability with average catch results below the median and above the median. The results showed that for 2018 and long-term demersal fishing results in most of FMA were below the median value.

11. Markov Chain: A Predictive Model for Manpower Planning ...

African Journals Online (AJOL)

numerous previous studies have applied Markov chain models in describing title or level promotions .... secondary data related to workforce movement selected ... leave. Babu and Rao, (2013) carried out studies on two graded manpower model with bulk recruitment in both grades. They assumed that the organization is.

12. Markov Chain Monte Carlo Methods-Simple Monte Carlo

Home; Journals; Resonance – Journal of Science Education; Volume 8; Issue 4. Markov Chain Monte Carlo ... New York 14853, USA. Indian Statistical Institute 8th Mile, Mysore Road Bangalore 560 059, India. Systat Software Asia-Pacific (PI Ltd., Floor 5, 'C' Tower Golden Enclave, Airport Road Bangalore 560017, India.

13. Markov chain for estimating human mitochondrial DNA mutation pattern

Science.gov (United States)

Vantika, Sandy; Pasaribu, Udjianna S.

2015-12-01

The Markov chain was proposed to estimate the human mitochondrial DNA mutation pattern. One DNA sequence was taken randomly from 100 sequences in Genbank. The nucleotide transition matrix and mutation transition matrix were estimated from this sequence. We determined whether the states (mutation/normal) are recurrent or transient. The results showed that both of them are recurrent.

14. finite markov chain model in lithofacies analysis: an example from ...

African Journals Online (AJOL)

The Markov Chain Stochastic Process has been used both to analyze the vertical lithofacies of the Bida. Sandstone (Campanian – Maastrichtian) in Bida area .... a particular lithofacies state overlies another. Fig. 2 a: Lithofacies F1 – F6 in outcrop section of the Bida Sandstone at the Bida Cemetery behind the Government.

15. Students' Progress throughout Examination Process as a Markov Chain

Science.gov (United States)

Hlavatý, Robert; Dömeová, Ludmila

2014-01-01

The paper is focused on students of Mathematical methods in economics at the Czech university of life sciences (CULS) in Prague. The idea is to create a model of students' progress throughout the whole course using the Markov chain approach. Each student has to go through various stages of the course requirements where his success depends on the…

16. A Parallel Solver for Large-Scale Markov Chains

Czech Academy of Sciences Publication Activity Database

Benzi, M.; Tůma, Miroslav

2002-01-01

Roč. 41, - (2002), s. 135-153 ISSN 0168-9274 R&D Projects: GA AV ČR IAA2030801; GA ČR GA101/00/1035 Keywords : parallel preconditioning * iterative methods * discrete Markov chains * generalized inverses * singular matrices * graph partitioning * AINV * Bi-CGSTAB Subject RIV: BA - General Mathematics Impact factor: 0.504, year: 2002

17. The deviation matrix of a continuous-time Markov chain

NARCIS (Netherlands)

Coolen-Schrijner, Pauline; van Doorn, Erik A.

2002-01-01

he deviation matrix of an ergodic, continuous-time Markov chain with transition probability matrix $P(.)$ and ergodic matrix $\\Pi$ is the matrix $D \\equiv \\int_0^{\\infty} (P(t)-\\Pi)dt$. We give conditions for $D$ to exist and discuss properties and a representation of $D$. The deviation matrix of a

18. The deviation matrix of a continuous-time Markov chain

NARCIS (Netherlands)

Coolen-Schrijner, P.; van Doorn, E.A.

2001-01-01

The deviation matrix of an ergodic, continuous-time Markov chain with transition probability matrix $P(.)$ and ergodic matrix $\\Pi$ is the matrix $D \\equiv \\int_0^{\\infty} (P(t)-\\Pi)dt$. We give conditions for $D$ to exist and discuss properties and a representation of $D$. The deviation matrix of a

19. A theoretical Markov chain model for evaluating correctional ...

African Journals Online (AJOL)

In this paper a stochastic method is applied in the study of the long time effect of confinement in a correctional institution on the behaviour of a person with criminal tendencies. The approach used is Markov chain, which uses past history to predict the state of a system in the future. A model is developed for comparing the ...

20. Markov chain: a predictive model for manpower planning | Ezugwu ...

African Journals Online (AJOL)

The use of Mathematical models for manpower planning has increased in recent times for better manpower planning quantitatively. In respect of organizational management, numerous previous studies have applied Markov chain models in describing title or level promotions, demotions, recruitments, withdrawals, ...

1. When are two Markov chains the same? | Cowen | Quaestiones ...

African Journals Online (AJOL)

Given two one-sided Markov chains, the authors illustrate a procedure for ascertaining whether they are essentially the same. Precisely, they show how one can determine whether they are block-isomorphic. An application to hydrology is investigated with an example. Quaestiones Mathematicae 23(2000), 507–513 ...

2. Model checking conditional CSL for continuous-time Markov chains

DEFF Research Database (Denmark)

Gao, Yang; Xu, Ming; Zhan, Naijun

2013-01-01

In this paper, we consider the model-checking problem of continuous-time Markov chains (CTMCs) with respect to conditional logic. To the end, we extend Continuous Stochastic Logic introduced in Aziz et al. (2000) [1] to Conditional Continuous Stochastic Logic (CCSL) by introducing a conditional p...

3. Reliability analysis and prediction of mixed mode load using Markov Chain Model

International Nuclear Information System (INIS)

Nikabdullah, N.; Singh, S. S. K.; Alebrahim, R.; Azizi, M. A.; K, Elwaleed A.; Noorani, M. S. M.

2014-01-01

The aim of this paper is to present the reliability analysis and prediction of mixed mode loading by using a simple two state Markov Chain Model for an automotive crankshaft. The reliability analysis and prediction for any automotive component or structure is important for analyzing and measuring the failure to increase the design life, eliminate or reduce the likelihood of failures and safety risk. The mechanical failures of the crankshaft are due of high bending and torsion stress concentration from high cycle and low rotating bending and torsional stress. The Markov Chain was used to model the two states based on the probability of failure due to bending and torsion stress. In most investigations it revealed that bending stress is much serve than torsional stress, therefore the probability criteria for the bending state would be higher compared to the torsion state. A statistical comparison between the developed Markov Chain Model and field data was done to observe the percentage of error. The reliability analysis and prediction was derived and illustrated from the Markov Chain Model were shown in the Weibull probability and cumulative distribution function, hazard rate and reliability curve and the bathtub curve. It can be concluded that Markov Chain Model has the ability to generate near similar data with minimal percentage of error and for a practical application; the proposed model provides a good accuracy in determining the reliability for the crankshaft under mixed mode loading

4. Markov chain analysis of single spin flip Ising simulations

International Nuclear Information System (INIS)

Hennecke, M.

1997-01-01

The Markov processes defined by random and loop-based schemes for single spin flip attempts in Monte Carlo simulations of the 2D Ising model are investigated, by explicitly constructing their transition matrices. Their analysis reveals that loops over all lattice sites using a Metropolis-type single spin flip probability often do not define ergodic Markov chains, and have distorted dynamical properties even if they are ergodic. The transition matrices also enable a comparison of the dynamics of random versus loop spin selection and Glauber versus Metropolis probabilities

5. Hierarchical Multiple Markov Chain Model for Unsupervised Texture Segmentation

Czech Academy of Sciences Publication Activity Database

Scarpa, G.; Gaetano, R.; Haindl, Michal; Zerubia, J.

2009-01-01

Roč. 18, č. 8 (2009), s. 1830-1843 ISSN 1057-7149 R&D Projects: GA ČR GA102/08/0593 EU Projects: European Commission(XE) 507752 - MUSCLE Institutional research plan: CEZ:AV0Z10750506 Keywords : Classification * texture analysis * segmentation * hierarchical image models * Markov process Subject RIV: BD - Theory of Information Impact factor: 2.848, year: 2009 http://library.utia.cas.cz/separaty/2009/RO/haindl-hierarchical multiple markov chain model for unsupervised texture segmentation.pdf

6. Markov chain aggregation for agent-based models

CERN Document Server

Banisch, Sven

2016-01-01

This self-contained text develops a Markov chain approach that makes the rigorous analysis of a class of microscopic models that specify the dynamics of complex systems at the individual level possible. It presents a general framework of aggregation in agent-based and related computational models, one which makes use of lumpability and information theory in order to link the micro and macro levels of observation. The starting point is a microscopic Markov chain description of the dynamical process in complete correspondence with the dynamical behavior of the agent-based model (ABM), which is obtained by considering the set of all possible agent configurations as the state space of a huge Markov chain. An explicit formal representation of a resulting “micro-chain” including microscopic transition rates is derived for a class of models by using the random mapping representation of a Markov process. The type of probability distribution used to implement the stochastic part of the model, which defines the upd...

7. Markov chain modelling of pitting corrosion in underground pipelines

International Nuclear Information System (INIS)

Caleyo, F.; Velazquez, J.C.; Valor, A.; Hallen, J.M.

2009-01-01

A continuous-time, non-homogenous linear growth (pure birth) Markov process has been used to model external pitting corrosion in underground pipelines. The closed form solution of Kolmogorov's forward equations for this type of Markov process is used to describe the transition probability function in a discrete pit depth space. The identification of the transition probability function can be achieved by correlating the stochastic pit depth mean with the deterministic mean obtained experimentally. Monte-Carlo simulations previously reported have been used to predict the time evolution of the mean value of the pit depth distribution for different soil textural classes. The simulated distributions have been used to create an empirical Markov chain-based stochastic model for predicting the evolution of pitting corrosion depth and rate distributions from the observed properties of the soil. The proposed model has also been applied to pitting corrosion data from pipeline repeated in-line inspections and laboratory immersion experiments.

8. Markov Chain Models for the Stochastic Modeling of Pitting Corrosion

Directory of Open Access Journals (Sweden)

A. Valor

2013-01-01

Full Text Available The stochastic nature of pitting corrosion of metallic structures has been widely recognized. It is assumed that this kind of deterioration retains no memory of the past, so only the current state of the damage influences its future development. This characteristic allows pitting corrosion to be categorized as a Markov process. In this paper, two different models of pitting corrosion, developed using Markov chains, are presented. Firstly, a continuous-time, nonhomogeneous linear growth (pure birth Markov process is used to model external pitting corrosion in underground pipelines. A closed-form solution of the system of Kolmogorov's forward equations is used to describe the transition probability function in a discrete pit depth space. The transition probability function is identified by correlating the stochastic pit depth mean with the empirical deterministic mean. In the second model, the distribution of maximum pit depths in a pitting experiment is successfully modeled after the combination of two stochastic processes: pit initiation and pit growth. Pit generation is modeled as a nonhomogeneous Poisson process, in which induction time is simulated as the realization of a Weibull process. Pit growth is simulated using a nonhomogeneous Markov process. An analytical solution of Kolmogorov's system of equations is also found for the transition probabilities from the first Markov state. Extreme value statistics is employed to find the distribution of maximum pit depths.

9. Markov chains and semi-Markov models in time-to-event analysis

OpenAIRE

Abner, Erin L.; Charnigo, Richard J.; Kryscio, Richard J.

2013-01-01

A variety of statistical methods are available to investigators for analysis of time-to-event data, often referred to as survival analysis. Kaplan-Meier estimation and Cox proportional hazards regression are commonly employed tools but are not appropriate for all studies, particularly in the presence of competing risks and when multiple or recurrent outcomes are of interest. Markov chain models can accommodate censored data, competing risks (informative censoring), multiple outcomes, recurren...

10. Geometric allocation approaches in Markov chain Monte Carlo

International Nuclear Information System (INIS)

Todo, S; Suwa, H

2013-01-01

The Markov chain Monte Carlo method is a versatile tool in statistical physics to evaluate multi-dimensional integrals numerically. For the method to work effectively, we must consider the following key issues: the choice of ensemble, the selection of candidate states, the optimization of transition kernel, algorithm for choosing a configuration according to the transition probabilities. We show that the unconventional approaches based on the geometric allocation of probabilities or weights can improve the dynamics and scaling of the Monte Carlo simulation in several aspects. Particularly, the approach using the irreversible kernel can reduce or sometimes completely eliminate the rejection of trial move in the Markov chain. We also discuss how the space-time interchange technique together with Walker's method of aliases can reduce the computational time especially for the case where the number of candidates is large, such as models with long-range interactions

11. Reservoir Modeling Combining Geostatistics with Markov Chain Monte Carlo Inversion

DEFF Research Database (Denmark)

Zunino, Andrea; Lange, Katrine; Melnikova, Yulia

2014-01-01

We present a study on the inversion of seismic reflection data generated from a synthetic reservoir model. Our aim is to invert directly for rock facies and porosity of the target reservoir zone. We solve this inverse problem using a Markov chain Monte Carlo (McMC) method to handle the nonlinear,...... constitute samples of the posterior distribution.......We present a study on the inversion of seismic reflection data generated from a synthetic reservoir model. Our aim is to invert directly for rock facies and porosity of the target reservoir zone. We solve this inverse problem using a Markov chain Monte Carlo (McMC) method to handle the nonlinear......, multi-step forward model (rock physics and seismology) and to provide realistic estimates of uncertainties. To generate realistic models which represent samples of the prior distribution, and to overcome the high computational demand, we reduce the search space utilizing an algorithm drawn from...

12. Maximum Entropy Estimation of Transition Probabilities of Reversible Markov Chains

Directory of Open Access Journals (Sweden)

Erik Van der Straeten

2009-11-01

Full Text Available In this paper, we develop a general theory for the estimation of the transition probabilities of reversible Markov chains using the maximum entropy principle. A broad range of physical models can be studied within this approach. We use one-dimensional classical spin systems to illustrate the theoretical ideas. The examples studied in this paper are: the Ising model, the Potts model and the Blume-Emery-Griffiths model.

13. Parallel algorithms for simulating continuous time Markov chains

Science.gov (United States)

Nicol, David M.; Heidelberger, Philip

1992-01-01

We have previously shown that the mathematical technique of uniformization can serve as the basis of synchronization for the parallel simulation of continuous-time Markov chains. This paper reviews the basic method and compares five different methods based on uniformization, evaluating their strengths and weaknesses as a function of problem characteristics. The methods vary in their use of optimism, logical aggregation, communication management, and adaptivity. Performance evaluation is conducted on the Intel Touchstone Delta multiprocessor, using up to 256 processors.

14. A simple introduction to Markov Chain Monte-Carlo sampling.

Science.gov (United States)

van Ravenzwaaij, Don; Cassey, Pete; Brown, Scott D

2016-03-11

Markov Chain Monte-Carlo (MCMC) is an increasingly popular method for obtaining information about distributions, especially for estimating posterior distributions in Bayesian inference. This article provides a very basic introduction to MCMC sampling. It describes what MCMC is, and what it can be used for, with simple illustrative examples. Highlighted are some of the benefits and limitations of MCMC sampling, as well as different approaches to circumventing the limitations most likely to trouble cognitive scientists.

15. Finding metastabilities in reversible Markov chains based on incomplete sampling

Directory of Open Access Journals (Sweden)

Fackeldey Konstantin

2017-01-01

Full Text Available In order to fully characterize the state-transition behaviour of finite Markov chains one needs to provide the corresponding transition matrix P. In many applications such as molecular simulation and drug design, the entries of the transition matrix P are estimated by generating realizations of the Markov chain and determining the one-step conditional probability Pij for a transition from one state i to state j. This sampling can be computational very demanding. Therefore, it is a good idea to reduce the sampling effort. The main purpose of this paper is to design a sampling strategy, which provides a partial sampling of only a subset of the rows of such a matrix P. Our proposed approach fits very well to stochastic processes stemming from simulation of molecular systems or random walks on graphs and it is different from the matrix completion approaches which try to approximate the transition matrix by using a low-rank-assumption. It will be shown how Markov chains can be analyzed on the basis of a partial sampling. More precisely. First, we will estimate the stationary distribution from a partially given matrix P. Second, we will estimate the infinitesimal generator Q of P on the basis of this stationary distribution. Third, from the generator we will compute the leading invariant subspace, which should be identical to the leading invariant subspace of P. Forth, we will apply Robust Perron Cluster Analysis (PCCA+ in order to identify metastabilities using this subspace.

16. Vehicle Driving Risk Prediction Based on Markov Chain Model

Directory of Open Access Journals (Sweden)

Xiaoxia Xiong

2018-01-01

Full Text Available A driving risk status prediction algorithm based on Markov chain is presented. Driving risk states are classified using clustering techniques based on feature variables describing the instantaneous risk levels within time windows, where instantaneous risk levels are determined in time-to-collision and time-headway two-dimension plane. Multinomial Logistic models with recursive feature variable estimation method are developed to improve the traditional state transition probability estimation, which also takes into account the comprehensive effects of driving behavior, traffic, and road environment factors on the evolution of driving risk status. The “100-car” natural driving data from Virginia Tech is employed for the training and validation of the prediction model. The results show that, under the 5% false positive rate, the prediction algorithm could have high prediction accuracy rate for future medium-to-high driving risks and could meet the timeliness requirement of collision avoidance warning. The algorithm could contribute to timely warning or auxiliary correction to drivers in the approaching-danger state.

17. Markov chain Monte Carlo methods: an introductory example

Science.gov (United States)

Klauenberg, Katy; Elster, Clemens

2016-02-01

When the Guide to the Expression of Uncertainty in Measurement (GUM) and methods from its supplements are not applicable, the Bayesian approach may be a valid and welcome alternative. Evaluating the posterior distribution, estimates or uncertainties involved in Bayesian inferences often requires numerical methods to avoid high-dimensional integrations. Markov chain Monte Carlo (MCMC) sampling is such a method—powerful, flexible and widely applied. Here, a concise introduction is given, illustrated by a simple, typical example from metrology. The Metropolis-Hastings algorithm is the most basic and yet flexible MCMC method. Its underlying concepts are explained and the algorithm is given step by step. The few lines of software code required for its implementation invite interested readers to get started. Diagnostics to evaluate the performance and common algorithmic choices are illustrated to calibrate the Metropolis-Hastings algorithm for efficiency. Routine application of MCMC algorithms may be hindered currently by the difficulty to assess the convergence of MCMC output and thus to assure the validity of results. An example points to the importance of convergence and initiates discussion about advantages as well as areas of research. Available software tools are mentioned throughout.

18. Algebraic decay in self-similar Markov chains

International Nuclear Information System (INIS)

Hanson, J.D.; Cary, J.R.; Meiss, J.D.

1984-10-01

A continuous time Markov chain is used to model motion in the neighborhood of a critical noble invariant circle in an area-preserving map. States in the infinite chain represent successive rational approximants to the frequency of the invariant circle. The nonlinear integral equation for the first passage time distribution is solved exactly. The asymptotic distribution is a power law times a function periodic in the logarithm of the time. For parameters relevant to Hamiltonian systems the decay proceeds as t -4 05

19. Transportation and concentration inequalities for bifurcating Markov chains

DEFF Research Database (Denmark)

Penda, S. Valère Bitseki; Escobar-Bach, Mikael; Guillin, Arnaud

2017-01-01

concentration inequalities.We also study deviation inequalities for the empirical means under relaxed assumptions on the Wasserstein contraction for the Markov kernels. Applications to bifurcating nonlinear autoregressive processes are considered for point-wise estimates of the non-linear autoregressive...

20. Robust Dynamics and Control of a Partially Observed Markov Chain

International Nuclear Information System (INIS)

Elliott, R. J.; Malcolm, W. P.; Moore, J. P.

2007-01-01

In a seminal paper, Martin Clark (Communications Systems and Random Process Theory, Darlington, 1977, pp. 721-734, 1978) showed how the filtered dynamics giving the optimal estimate of a Markov chain observed in Gaussian noise can be expressed using an ordinary differential equation. These results offer substantial benefits in filtering and in control, often simplifying the analysis and an in some settings providing numerical benefits, see, for example Malcolm et al. (J. Appl. Math. Stoch. Anal., 2007, to appear).Clark's method uses a gauge transformation and, in effect, solves the Wonham-Zakai equation using variation of constants. In this article, we consider the optimal control of a partially observed Markov chain. This problem is discussed in Elliott et al. (Hidden Markov Models Estimation and Control, Applications of Mathematics Series, vol. 29, 1995). The innovation in our results is that the robust dynamics of Clark are used to compute forward in time dynamics for a simplified adjoint process. A stochastic minimum principle is established

1. Treatment-based Markov chain models clarify mechanisms of invasion in an invaded grassland community.

Science.gov (United States)

Nelis, Lisa Castillo; Wootton, J Timothy

2010-02-22

What are the relative roles of mechanisms underlying plant responses in grassland communities invaded by both plants and mammals? What type of community can we expect in the future given current or novel conditions? We address these questions by comparing Markov chain community models among treatments from a field experiment on invasive species on Robinson Crusoe Island, Chile. Because of seed dispersal, grazing and disturbance, we predicted that the exotic European rabbit (Oryctolagus cuniculus) facilitates epizoochorous exotic plants (plants with seeds that stick to the skin an animal) at the expense of native plants. To test our hypothesis, we crossed rabbit exclosure treatments with disturbance treatments, and sampled the plant community in permanent plots over 3 years. We then estimated Markov chain model transition probabilities and found significant differences among treatments. As hypothesized, this modelling revealed that exotic plants survive better in disturbed areas, while natives prefer no rabbits or disturbance. Surprisingly, rabbits negatively affect epizoochorous plants. Markov chain dynamics indicate that an overall replacement of native plants by exotic plants is underway. Using a treatment-based approach to multi-species Markov chain models allowed us to examine the changes in the importance of mechanisms in response to experimental impacts on communities.

2. Mixed Vehicle Flow At Signalized Intersection: Markov Chain Analysis

Directory of Open Access Journals (Sweden)

Gertsbakh Ilya B.

2015-09-01

Full Text Available We assume that a Poisson flow of vehicles arrives at isolated signalized intersection, and each vehicle, independently of others, represents a random number X of passenger car units (PCU’s. We analyze numerically the stationary distribution of the queue process {Zn}, where Zn is the number of PCU’s in a queue at the beginning of the n-th red phase, n → ∞. We approximate the number Yn of PCU’s arriving during one red-green cycle by a two-parameter Negative Binomial Distribution (NBD. The well-known fact is that {Zn} follow an infinite-state Markov chain. We approximate its stationary distribution using a finite-state Markov chain. We show numerically that there is a strong dependence of the mean queue length E[Zn] in equilibrium on the input distribution of Yn and, in particular, on the ”over dispersion” parameter γ= Var[Yn]/E[Yn]. For Poisson input, γ = 1. γ > 1 indicates presence of heavy-tailed input. In reality it means that a relatively large ”portion” of PCU’s, considerably exceeding the average, may arrive with high probability during one red-green cycle. Empirical formulas are presented for an accurate estimation of mean queue length as a function of load and g of the input flow. Using the Markov chain technique, we analyze the mean ”virtual” delay time for a car which always arrives at the beginning of the red phase.

3. LISA data analysis using Markov chain Monte Carlo methods

International Nuclear Information System (INIS)

Cornish, Neil J.; Crowder, Jeff

2005-01-01

The Laser Interferometer Space Antenna (LISA) is expected to simultaneously detect many thousands of low-frequency gravitational wave signals. This presents a data analysis challenge that is very different to the one encountered in ground based gravitational wave astronomy. LISA data analysis requires the identification of individual signals from a data stream containing an unknown number of overlapping signals. Because of the signal overlaps, a global fit to all the signals has to be performed in order to avoid biasing the solution. However, performing such a global fit requires the exploration of an enormous parameter space with a dimension upwards of 50 000. Markov Chain Monte Carlo (MCMC) methods offer a very promising solution to the LISA data analysis problem. MCMC algorithms are able to efficiently explore large parameter spaces, simultaneously providing parameter estimates, error analysis, and even model selection. Here we present the first application of MCMC methods to simulated LISA data and demonstrate the great potential of the MCMC approach. Our implementation uses a generalized F-statistic to evaluate the likelihoods, and simulated annealing to speed convergence of the Markov chains. As a final step we supercool the chains to extract maximum likelihood estimates, and estimates of the Bayes factors for competing models. We find that the MCMC approach is able to correctly identify the number of signals present, extract the source parameters, and return error estimates consistent with Fisher information matrix predictions

4. A Markov chain representation of the multiple testing problem.

Science.gov (United States)

Cabras, Stefano

2018-02-01

The problem of multiple hypothesis testing can be represented as a Markov process where a new alternative hypothesis is accepted in accordance with its relative evidence to the currently accepted one. This virtual and not formally observed process provides the most probable set of non null hypotheses given the data; it plays the same role as Markov Chain Monte Carlo in approximating a posterior distribution. To apply this representation and obtain the posterior probabilities over all alternative hypotheses, it is enough to have, for each test, barely defined Bayes Factors, e.g. Bayes Factors obtained up to an unknown constant. Such Bayes Factors may either arise from using default and improper priors or from calibrating p-values with respect to their corresponding Bayes Factor lower bound. Both sources of evidence are used to form a Markov transition kernel on the space of hypotheses. The approach leads to easy interpretable results and involves very simple formulas suitable to analyze large datasets as those arising from gene expression data (microarray or RNA-seq experiments).

5. Dynamic modeling of presence of occupants using inhomogeneous Markov chains

DEFF Research Database (Denmark)

Andersen, Philip Hvidthøft Delff; Iversen, Anne; Madsen, Henrik

2014-01-01

on inhomogeneous Markov chains with where the transition probabilities are estimated using generalized linear models with polynomials, B-splines, and a filter of passed observations as inputs. For treating the dispersion of the data series, a hierarchical model structure is used where one model is for low presence......Occupancy modeling is a necessary step towards reliable simulation of energy consumption in buildings. This paper outlines a method for fitting recordings of presence of occupants and simulation of single-person to multiple-persons office environments. The method includes modeling of dependence...

6. Converging from Branching to Linear Metrics on Markov Chains

DEFF Research Database (Denmark)

Bacci, Giorgio; Bacci, Giovanni; Larsen, Kim Guldstrand

2015-01-01

We study the strong and strutter trace distances on Markov chains (MCs). Our interest in these metrics is motivated by their relation to the probabilistic LTL-model checking problem: we prove that they correspond to the maximal differences in the probability of satisfying the same LTL and LTL......-헑 ( LTL without next operator) formulas, respectively. The threshold problem for these distances (whether their value exceeds a given threshold) is NP-hard and not known to be decidable. Nevertheless, we provide an approximation schema where each lower and upper-approximant is computable in polynomial...

7. Converging from branching to linear metrics on Markov chains

DEFF Research Database (Denmark)

Bacci, Giorgio; Bacci, Giovanni; Larsen, Kim G.

2017-01-01

We study two well known linear-time metrics on Markov chains (MCs), namely, the strong and strutter trace distances. Our interest in these metrics is motivated by their relation to the probabilistic LTL-model checking problem: we prove that they correspond to the maximal differences...... in the probability of satisfying the same LTL and LTL-X (LTL without next operator) formulas, respectively. The threshold problem for these distances (whether their value exceeds a given threshold) is NP-hard and not known to be decidable. Nevertheless, we provide an approximation schema where each lower and upper...

8. On the Metric-Based Approximate Minimization of Markov Chains

DEFF Research Database (Denmark)

Bacci, Giovanni; Bacci, Giorgio; Larsen, Kim Guldstrand

2017-01-01

We address the behavioral metric-based approximate minimization problem of Markov Chains (MCs), i.e., given a finite MC and a positive integer k, we are interested in finding a k-state MC of minimal distance to the original. By considering as metric the bisimilarity distance of Desharnais at al......., we show that optimal approximations always exist; show that the problem can be solved as a bilinear program; and prove that its threshold problem is in PSPACE and NP-hard. Finally, we present an approach inspired by expectation maximization techniques that provides suboptimal solutions. Experiments...

9. Uncovering and testing the fuzzy clusters based on lumped Markov chain in complex network.

Science.gov (United States)

Jing, Fan; Jianbin, Xie; Jinlong, Wang; Jinshuai, Qu

2013-01-01

Identifying clusters, namely groups of nodes with comparatively strong internal connectivity, is a fundamental task for deeply understanding the structure and function of a network. By means of a lumped Markov chain model of a random walker, we propose two novel ways of inferring the lumped markov transition matrix. Furthermore, some useful results are proposed based on the analysis of the properties of the lumped Markov process. To find the best partition of complex networks, a novel framework including two algorithms for network partition based on the optimal lumped Markovian dynamics is derived to solve this problem. The algorithms are constructed to minimize the objective function under this framework. It is demonstrated by the simulation experiments that our algorithms can efficiently determine the probabilities with which a node belongs to different clusters during the learning process and naturally supports the fuzzy partition. Moreover, they are successfully applied to real-world network, including the social interactions between members of a karate club.

10. Markov Chain Modelling for Short-Term NDVI Time Series Forecasting

Directory of Open Access Journals (Sweden)

Stepčenko Artūrs

2016-12-01

Full Text Available In this paper, the NDVI time series forecasting model has been developed based on the use of discrete time, continuous state Markov chain of suitable order. The normalised difference vegetation index (NDVI is an indicator that describes the amount of chlorophyll (the green mass and shows the relative density and health of vegetation; therefore, it is an important variable for vegetation forecasting. A Markov chain is a stochastic process that consists of a state space. This stochastic process undergoes transitions from one state to another in the state space with some probabilities. A Markov chain forecast model is flexible in accommodating various forecast assumptions and structures. The present paper discusses the considerations and techniques in building a Markov chain forecast model at each step. Continuous state Markov chain model is analytically described. Finally, the application of the proposed Markov chain model is illustrated with reference to a set of NDVI time series data.

11. A Markov Chain Estimator of Multivariate Volatility from High Frequency Data

DEFF Research Database (Denmark)

Hansen, Peter Reinhard; Horel, Guillaume; Lunde, Asger

We introduce a multivariate estimator of financial volatility that is based on the theory of Markov chains. The Markov chain framework takes advantage of the discreteness of high-frequency returns. We study the finite sample properties of the estimation in a simulation study and apply it to highf......We introduce a multivariate estimator of financial volatility that is based on the theory of Markov chains. The Markov chain framework takes advantage of the discreteness of high-frequency returns. We study the finite sample properties of the estimation in a simulation study and apply...

12. Markov chain aggregation and its applications to combinatorial reaction networks.

Science.gov (United States)

Ganguly, Arnab; Petrov, Tatjana; Koeppl, Heinz

2014-09-01

We consider a continuous-time Markov chain (CTMC) whose state space is partitioned into aggregates, and each aggregate is assigned a probability measure. A sufficient condition for defining a CTMC over the aggregates is presented as a variant of weak lumpability, which also characterizes that the measure over the original process can be recovered from that of the aggregated one. We show how the applicability of de-aggregation depends on the initial distribution. The application section is devoted to illustrate how the developed theory aids in reducing CTMC models of biochemical systems particularly in connection to protein-protein interactions. We assume that the model is written by a biologist in form of site-graph-rewrite rules. Site-graph-rewrite rules compactly express that, often, only a local context of a protein (instead of a full molecular species) needs to be in a certain configuration in order to trigger a reaction event. This observation leads to suitable aggregate Markov chains with smaller state spaces, thereby providing sufficient reduction in computational complexity. This is further exemplified in two case studies: simple unbounded polymerization and early EGFR/insulin crosstalk.

13. Simulation of daily rainfall through markov chain modeling

International Nuclear Information System (INIS)

2015-01-01

Being an agricultural country, the inhabitants of dry land in cultivated areas mainly rely on the daily rainfall for watering their fields. A stochastic model based on first order Markov Chain was developed to simulate daily rainfall data for Multan, D. I. Khan, Nawabshah, Chilas and Barkhan for the period 1981-2010. Transitional probability matrices of first order Markov Chain was utilized to generate the daily rainfall occurrence while gamma distribution was used to generate the daily rainfall amount. In order to achieve the parametric values of mentioned cities, method of moments is used to estimate the shape and scale parameters which lead to synthetic sequence generation as per gamma distribution. In this study, unconditional and conditional probabilities of wet and dry days in sum with means and standard deviations are considered as the essential parameters for the simulated stochastic generation of daily rainfalls. It has been found that the computerized synthetic rainfall series concurred pretty well with the actual observed rainfall series. (author)

14. Uniform ergodicities and perturbation bounds of Markov chains on ordered Banach spaces

Science.gov (United States)

Erkursun Özcan, Nazife; Mukhamedov, Farrukh

2017-03-01

In this paper, we consider uniformly mean ergodic and uniformly asymptotical stable Markov operators on ordered Banach spaces. In terms of the ergodicity coefficient, we show the equivalence of uniform and weak mean ergodicities of Markov operators. This result allowed us to establish a category theorem for uniformly mean ergodic Markov operators. Furthermore, using properties of the ergodicity coefficient, we develop the perturbation theory for uniformly asymptotical stable Markov chains in the abstract scheme.

15. Portfolio allocation under the vendor managed inventory: A Markov ...

African Journals Online (AJOL)

Portfolio allocation under the vendor managed inventory: A Markov decision process. ... Journal of Applied Sciences and Environmental Management ... a review of Markov decision processes and investigates its suitability for solutions to portfolio allocation problems under vendor managed inventory in an uncertain market ...

16. HYDRA: a Java library for Markov Chain Monte Carlo

Directory of Open Access Journals (Sweden)

Gregory R. Warnes

2002-03-01

Full Text Available Hydra is an open-source, platform-neutral library for performing Markov Chain Monte Carlo. It implements the logic of standard MCMC samplers within a framework designed to be easy to use, extend, and integrate with other software tools. In this paper, we describe the problem that motivated our work, outline our goals for the Hydra pro ject, and describe the current features of the Hydra library. We then provide a step-by-step example of using Hydra to simulate from a mixture model drawn from cancer genetics, first using a variable-at-a-time Metropolis sampler and then a Normal Kernel Coupler. We conclude with a discussion of future directions for Hydra.

17. Reservoir Modeling Combining Geostatistics with Markov Chain Monte Carlo Inversion

DEFF Research Database (Denmark)

Zunino, Andrea; Lange, Katrine; Melnikova, Yulia

2014-01-01

We present a study on the inversion of seismic reflection data generated from a synthetic reservoir model. Our aim is to invert directly for rock facies and porosity of the target reservoir zone. We solve this inverse problem using a Markov chain Monte Carlo (McMC) method to handle the nonlinear......, multi-step forward model (rock physics and seismology) and to provide realistic estimates of uncertainties. To generate realistic models which represent samples of the prior distribution, and to overcome the high computational demand, we reduce the search space utilizing an algorithm drawn from...... geostatistics. The geostatistical algorithm learns the multiple-point statistics from prototype models, then generates proposal models which are tested by a Metropolis sampler. The solution of the inverse problem is finally represented by a collection of reservoir models in terms of facies and porosity, which...

18. Markov Chain Monte Carlo Bayesian Learning for Neural Networks

Science.gov (United States)

Goodrich, Michael S.

2011-01-01

Conventional training methods for neural networks involve starting al a random location in the solution space of the network weights, navigating an error hyper surface to reach a minimum, and sometime stochastic based techniques (e.g., genetic algorithms) to avoid entrapment in a local minimum. It is further typically necessary to preprocess the data (e.g., normalization) to keep the training algorithm on course. Conversely, Bayesian based learning is an epistemological approach concerned with formally updating the plausibility of competing candidate hypotheses thereby obtaining a posterior distribution for the network weights conditioned on the available data and a prior distribution. In this paper, we developed a powerful methodology for estimating the full residual uncertainty in network weights and therefore network predictions by using a modified Jeffery's prior combined with a Metropolis Markov Chain Monte Carlo method.

19. Projection methods for the numerical solution of Markov chain models

Science.gov (United States)

1989-01-01

Projection methods for computing stationary probability distributions for Markov chain models are presented. A general projection method is a method which seeks an approximation from a subspace of small dimension to the original problem. Thus, the original matrix problem of size N is approximated by one of dimension m, typically much smaller than N. A particularly successful class of methods based on this principle is that of Krylov subspace methods which utilize subspaces of the form span(v,av,...,A(exp m-1)v). These methods are effective in solving linear systems and eigenvalue problems (Lanczos, Arnoldi,...) as well as nonlinear equations. They can be combined with more traditional iterative methods such as successive overrelaxation, symmetric successive overrelaxation, or with incomplete factorization methods to enhance convergence.

20. Uncovering mental representations with Markov chain Monte Carlo.

Science.gov (United States)

Sanborn, Adam N; Griffiths, Thomas L; Shiffrin, Richard M

2010-03-01

A key challenge for cognitive psychology is the investigation of mental representations, such as object categories, subjective probabilities, choice utilities, and memory traces. In many cases, these representations can be expressed as a non-negative function defined over a set of objects. We present a behavioral method for estimating these functions. Our approach uses people as components of a Markov chain Monte Carlo (MCMC) algorithm, a sophisticated sampling method originally developed in statistical physics. Experiments 1 and 2 verified the MCMC method by training participants on various category structures and then recovering those structures. Experiment 3 demonstrated that the MCMC method can be used estimate the structures of the real-world animal shape categories of giraffes, horses, dogs, and cats. Experiment 4 combined the MCMC method with multidimensional scaling to demonstrate how different accounts of the structure of categories, such as prototype and exemplar models, can be tested, producing samples from the categories of apples, oranges, and grapes.

1. On the multi-level solution algorithm for Markov chains

Energy Technology Data Exchange (ETDEWEB)

Horton, G. [Univ. of Erlangen, Nuernberg (Germany)

1996-12-31

We discuss the recently introduced multi-level algorithm for the steady-state solution of Markov chains. The method is based on the aggregation principle, which is well established in the literature. Recursive application of the aggregation yields a multi-level method which has been shown experimentally to give results significantly faster than the methods currently in use. The algorithm can be reformulated as an algebraic multigrid scheme of Galerkin-full approximation type. The uniqueness of the scheme stems from its solution-dependent prolongation operator which permits significant computational savings in the evaluation of certain terms. This paper describes the modeling of computer systems to derive information on performance, measured typically as job throughput or component utilization, and availability, defined as the proportion of time a system is able to perform a certain function in the presence of component failures and possibly also repairs.

2. The spectral method and the central limit theorem for general Markov chains

Science.gov (United States)

Nagaev, S. V.

2017-12-01

We consider Markov chains with an arbitrary phase space and develop a modification of the spectral method that enables us to prove the central limit theorem (CLT) for non-uniformly ergodic Markov chains. The conditions imposed on the transition function are more general than those by Athreya-Ney and Nummelin. Our proof of the CLT is purely analytical.

3. Descriptive and predictive evaluation of high resolution Markov chain precipitation models

DEFF Research Database (Denmark)

Sørup, Hjalte Jomo Danielsen; Madsen, Henrik; Arnbjerg-Nielsen, Karsten

2012-01-01

. Continuous modelling of the Markov process proved attractive because of a marked decrease in the number of parameters. Inclusion of seasonality into the continuous Markov chain model proved difficult. Monte Carlo simulations with the models show that it is very difficult for all the model formulations......A time series of tipping bucket recordings of very high temporal and volumetric resolution precipitation is modelled using Markov chain models. Both first and second‐order Markov models as well as seasonal and diurnal models are investigated and evaluated using likelihood based techniques....... The first‐order Markov model seems to capture most of the properties of precipitation, but inclusion of seasonal and diurnal variation improves the model. Including a second‐order Markov Chain component does improve the descriptive capabilities of the model, but is very expensive in its parameter use...

4. A Graph-Algorithmic Approach for the Study of Metastability in Markov Chains

Science.gov (United States)

Gan, Tingyue; Cameron, Maria

2017-06-01

Large continuous-time Markov chains with exponentially small transition rates arise in modeling complex systems in physics, chemistry, and biology. We propose a constructive graph-algorithmic approach to determine the sequence of critical timescales at which the qualitative behavior of a given Markov chain changes, and give an effective description of the dynamics on each of them. This approach is valid for both time-reversible and time-irreversible Markov processes, with or without symmetry. Central to this approach are two graph algorithms, Algorithm 1 and Algorithm 2, for obtaining the sequences of the critical timescales and the hierarchies of Typical Transition Graphs or T-graphs indicating the most likely transitions in the system without and with symmetry, respectively. The sequence of critical timescales includes the subsequence of the reciprocals of the real parts of eigenvalues. Under a certain assumption, we prove sharp asymptotic estimates for eigenvalues (including pre-factors) and show how one can extract them from the output of Algorithm 1. We discuss the relationship between Algorithms 1 and 2 and explain how one needs to interpret the output of Algorithm 1 if it is applied in the case with symmetry instead of Algorithm 2. Finally, we analyze an example motivated by R. D. Astumian's model of the dynamics of kinesin, a molecular motor, by means of Algorithm 2.

5. Tokunaga self-similarity for symmetric homogeneous Markov chains

Science.gov (United States)

Kovchegov, Y.; Zaliapin, I.

2010-12-01

Hierarchical branching organization is ubiquitous in nature. It is readily seen in river basins, drainage networks, bronchial passages, botanical trees, and snowflakes, to mention but a few. Empirical evidence suggests that one can describe many natural hierarchies by so-called Tokunaga self-similar trees (SSTs) [Shreve, 1969; Tokunaga, 1978; Ossadnik, 1992; Peckham, 1995; Newman et al., 1997; Pelletier and Turcotte, 2000]; Tokunaga SST have been proven to describe the Galton-Watson critical branching [Burd et al., 2000] and a general particle coagulation process [Gabrielov et al., 1999]. Tokunaga SSTs form a special two-parametric class of SSTs that preserves its statistical properties under the operation of pruning, i.e., cutting the leaves. It has been conjectured (Webb and Zaliapin, 2009; Zaliapin et al. 2009) that Tokunaga self-similarity is a characteristic property of the inverse aggregation (coagulation) process. This study provides further evidence in support of this hypothesis by focusing on trees that describe the topological structure of level sets of a time series, so-called level-set trees (LST). We prove that the LST for a symmetric homogeneous Markov chain (HMC) is a Tokunaga SST with the same parameters as the famous Shreve tree and critical Galton-Watson tree. We show, furthermore, that the Tokunaga property holds for any transformation F[X(G(t))] of a symmetric HMC X(t), where F and G are monotone increasing functions, and as a result - for the regular Brownian motion. At the same time, the Tokunaga property does not hold in general in asymmetric HMCs, a Brownian motion with a drift, ARMA, and some other conventional models. We discuss the relation of our results to the Tokunaga self-similarity of the nearest-neighbor trees for random point sets. References: 1. Gabrielov, A., W.I. Newman, D.L. Turcotte (1999) An exactly soluble hierarchical clustering model: inverse cascades, self-similarity, and scaling. Phys. Rev. E, 1999, 60, 5293-5300. 2

6. Ensemble bayesian model averaging using markov chain Monte Carlo sampling

Energy Technology Data Exchange (ETDEWEB)

Vrugt, Jasper A [Los Alamos National Laboratory; Diks, Cees G H [NON LANL; Clark, Martyn P [NON LANL

2008-01-01

Bayesian model averaging (BMA) has recently been proposed as a statistical method to calibrate forecast ensembles from numerical weather models. Successful implementation of BMA however, requires accurate estimates of the weights and variances of the individual competing models in the ensemble. In their seminal paper (Raftery etal. Mon Weather Rev 133: 1155-1174, 2(05)) has recommended the Expectation-Maximization (EM) algorithm for BMA model training, even though global convergence of this algorithm cannot be guaranteed. In this paper, we compare the performance of the EM algorithm and the recently developed Differential Evolution Adaptive Metropolis (DREAM) Markov Chain Monte Carlo (MCMC) algorithm for estimating the BMA weights and variances. Simulation experiments using 48-hour ensemble data of surface temperature and multi-model stream-flow forecasts show that both methods produce similar results, and that their performance is unaffected by the length of the training data set. However, MCMC simulation with DREAM is capable of efficiently handling a wide variety of BMA predictive distributions, and provides useful information about the uncertainty associated with the estimated BMA weights and variances.

7. Markov chain Monte Carlo sampling based terahertz holography image denoising.

Science.gov (United States)

Chen, Guanghao; Li, Qi

2015-05-10

Terahertz digital holography has attracted much attention in recent years. This technology combines the strong transmittance of terahertz and the unique features of digital holography. Nonetheless, the low clearness of the images captured has hampered the popularization of this imaging technique. In this paper, we perform a digital image denoising technique on our multiframe superposed images. The noise suppression model is concluded as Bayesian least squares estimation and is solved with Markov chain Monte Carlo (MCMC) sampling. In this algorithm, a weighted mean filter with a Gaussian kernel is first applied to the noisy image, and then by nonlinear contrast transform, the contrast of the image is restored to the former level. By randomly walking on the preprocessed image, the MCMC-based filter keeps collecting samples, assigning them weights by similarity assessment, and constructs multiple sample sequences. Finally, these sequences are used to estimate the value of each pixel. Our algorithm shares some good qualities with nonlocal means filtering and the algorithm based on conditional sampling proposed by Wong et al. [Opt. Express18, 8338 (2010)10.1364/OE.18.008338OPEXFF1094-4087], such as good uniformity, and, moreover, reveals better performance in structure preservation, as shown in numerical comparison using the structural similarity index measurement and the peak signal-to-noise ratio.

8. Seriation in paleontological data using markov chain Monte Carlo methods.

Directory of Open Access Journals (Sweden)

Kai Puolamäki

2006-02-01

Full Text Available Given a collection of fossil sites with data about the taxa that occur in each site, the task in biochronology is to find good estimates for the ages or ordering of sites. We describe a full probabilistic model for fossil data. The parameters of the model are natural: the ordering of the sites, the origination and extinction times for each taxon, and the probabilities of different types of errors. We show that the posterior distributions of these parameters can be estimated reliably by using Markov chain Monte Carlo techniques. The posterior distributions of the model parameters can be used to answer many different questions about the data, including seriation (finding the best ordering of the sites and outlier detection. We demonstrate the usefulness of the model and estimation method on synthetic data and on real data on large late Cenozoic mammals. As an example, for the sites with large number of occurrences of common genera, our methods give orderings, whose correlation with geochronologic ages is 0.95.

9. Bayesian adaptive Markov chain Monte Carlo estimation of genetic parameters.

Science.gov (United States)

Mathew, B; Bauer, A M; Koistinen, P; Reetz, T C; Léon, J; Sillanpää, M J

2012-10-01

Accurate and fast estimation of genetic parameters that underlie quantitative traits using mixed linear models with additive and dominance effects is of great importance in both natural and breeding populations. Here, we propose a new fast adaptive Markov chain Monte Carlo (MCMC) sampling algorithm for the estimation of genetic parameters in the linear mixed model with several random effects. In the learning phase of our algorithm, we use the hybrid Gibbs sampler to learn the covariance structure of the variance components. In the second phase of the algorithm, we use this covariance structure to formulate an effective proposal distribution for a Metropolis-Hastings algorithm, which uses a likelihood function in which the random effects have been integrated out. Compared with the hybrid Gibbs sampler, the new algorithm had better mixing properties and was approximately twice as fast to run. Our new algorithm was able to detect different modes in the posterior distribution. In addition, the posterior mode estimates from the adaptive MCMC method were close to the REML (residual maximum likelihood) estimates. Moreover, our exponential prior for inverse variance components was vague and enabled the estimated mode of the posterior variance to be practically zero, which was in agreement with the support from the likelihood (in the case of no dominance). The method performance is illustrated using simulated data sets with replicates and field data in barley.

10. Threshold partitioning of sparse matrices and applications to Markov chains

Energy Technology Data Exchange (ETDEWEB)

Choi, Hwajeong; Szyld, D.B. [Temple Univ., Philadelphia, PA (United States)

1996-12-31

It is well known that the order of the variables and equations of a large, sparse linear system influences the performance of classical iterative methods. In particular if, after a symmetric permutation, the blocks in the diagonal have more nonzeros, classical block methods have a faster asymptotic rate of convergence. In this paper, different ordering and partitioning algorithms for sparse matrices are presented. They are modifications of PABLO. In the new algorithms, in addition to the location of the nonzeros, the values of the entries are taken into account. The matrix resulting after the symmetric permutation has dense blocks along the diagonal, and small entries in the off-diagonal blocks. Parameters can be easily adjusted to obtain, for example, denser blocks, or blocks with elements of larger magnitude. In particular, when the matrices represent Markov chains, the permuted matrices are well suited for block iterative methods that find the corresponding probability distribution. Applications to three types of methods are explored: (1) Classical block methods, such as Block Gauss Seidel. (2) Preconditioned GMRES, where a block diagonal preconditioner is used. (3) Iterative aggregation method (also called aggregation/disaggregation) where the partition obtained from the ordering algorithm with certain parameters is used as an aggregation scheme. In all three cases, experiments are presented which illustrate the performance of the methods with the new orderings. The complexity of the new algorithms is linear in the number of nonzeros and the order of the matrix, and thus adding little computational effort to the overall solution.

11. Markov chain Monte Carlo with the Integrated Nested Laplace Approximation

KAUST Repository

Gómez-Rubio, Virgilio

2017-10-06

The Integrated Nested Laplace Approximation (INLA) has established itself as a widely used method for approximate inference on Bayesian hierarchical models which can be represented as a latent Gaussian model (LGM). INLA is based on producing an accurate approximation to the posterior marginal distributions of the parameters in the model and some other quantities of interest by using repeated approximations to intermediate distributions and integrals that appear in the computation of the posterior marginals. INLA focuses on models whose latent effects are a Gaussian Markov random field. For this reason, we have explored alternative ways of expanding the number of possible models that can be fitted using the INLA methodology. In this paper, we present a novel approach that combines INLA and Markov chain Monte Carlo (MCMC). The aim is to consider a wider range of models that can be fitted with INLA only when some of the parameters of the model have been fixed. We show how new values of these parameters can be drawn from their posterior by using conditional models fitted with INLA and standard MCMC algorithms, such as Metropolis–Hastings. Hence, this will extend the use of INLA to fit models that can be expressed as a conditional LGM. Also, this new approach can be used to build simpler MCMC samplers for complex models as it allows sampling only on a limited number of parameters in the model. We will demonstrate how our approach can extend the class of models that could benefit from INLA, and how the R-INLA package will ease its implementation. We will go through simple examples of this new approach before we discuss more advanced applications with datasets taken from the relevant literature. In particular, INLA within MCMC will be used to fit models with Laplace priors in a Bayesian Lasso model, imputation of missing covariates in linear models, fitting spatial econometrics models with complex nonlinear terms in the linear predictor and classification of data with

12. Technical manual for basic version of the Markov chain nest productivity model (MCnest)

Science.gov (United States)

The Markov Chain Nest Productivity Model (or MCnest) integrates existing toxicity information from three standardized avian toxicity tests with information on species life history and the timing of pesticide applications relative to the timing of avian breeding seasons to quantit...

13. Accelerating Markov chain Monte Carlo simulation by differential evolution with self-adaptive randomized subspace sampling

Energy Technology Data Exchange (ETDEWEB)

Vrugt, Jasper A [Los Alamos National Laboratory; Hyman, James M [Los Alamos National Laboratory; Robinson, Bruce A [Los Alamos National Laboratory; Higdon, Dave [Los Alamos National Laboratory; Ter Braak, Cajo J F [NETHERLANDS; Diks, Cees G H [UNIV OF AMSTERDAM

2008-01-01

Markov chain Monte Carlo (MCMC) methods have found widespread use in many fields of study to estimate the average properties of complex systems, and for posterior inference in a Bayesian framework. Existing theory and experiments prove convergence of well constructed MCMC schemes to the appropriate limiting distribution under a variety of different conditions. In practice, however this convergence is often observed to be disturbingly slow. This is frequently caused by an inappropriate selection of the proposal distribution used to generate trial moves in the Markov Chain. Here we show that significant improvements to the efficiency of MCMC simulation can be made by using a self-adaptive Differential Evolution learning strategy within a population-based evolutionary framework. This scheme, entitled DiffeRential Evolution Adaptive Metropolis or DREAM, runs multiple different chains simultaneously for global exploration, and automatically tunes the scale and orientation of the proposal distribution in randomized subspaces during the search. Ergodicity of the algorithm is proved, and various examples involving nonlinearity, high-dimensionality, and multimodality show that DREAM is generally superior to other adaptive MCMC sampling approaches. The DREAM scheme significantly enhances the applicability of MCMC simulation to complex, multi-modal search problems.

14. Robust filtering and prediction for systems with embedded finite-state Markov-Chain dynamics

International Nuclear Information System (INIS)

Pate, E.B.

1986-01-01

This research developed new methodologies for the design of robust near-optimal filters/predictors for a class of system models that exhibit embedded finite-state Markov-chain dynamics. These methodologies are developed through the concepts and methods of stochastic model building (including time-series analysis), game theory, decision theory, and filtering/prediction for linear dynamic systems. The methodology is based on the relationship between the robustness of a class of time-series models and quantization which is applied to the time series as part of the model identification process. This relationship is exploited by utilizing the concept of an equivalence, through invariance of spectra, between the class of Markov-chain models and the class of autoregressive moving average (ARMA) models. This spectral equivalence permits a straightforward implementation of the desirable robust properties of the Markov-chain approximation in a class of models which may be applied in linear-recursive form in a linear Kalman filter/predictor structure. The linear filter/predictor structure is shown to provide asymptotically optimal estimates of states which represent one or more integrations of the Markov-chain state. The development of a new saddle-point theorem for a game based on the Markov-chain model structure gives rise to a technique for determining a worst case Markov-chain process, upon which a robust filter/predictor design if based

15. Markov-chain model of classified atomistic transition states for discrete kinetic Monte Carlo simulations.

Science.gov (United States)

Numazawa, Satoshi; Smith, Roger

2011-10-01

Classical harmonic transition state theory is considered and applied in discrete lattice cells with hierarchical transition levels. The scheme is then used to determine transitions that can be applied in a lattice-based kinetic Monte Carlo (KMC) atomistic simulation model. The model results in an effective reduction of KMC simulation steps by utilizing a classification scheme of transition levels for thermally activated atomistic diffusion processes. Thermally activated atomistic movements are considered as local transition events constrained in potential energy wells over certain local time periods. These processes are represented by Markov chains of multidimensional Boolean valued functions in three-dimensional lattice space. The events inhibited by the barriers under a certain level are regarded as thermal fluctuations of the canonical ensemble and accepted freely. Consequently, the fluctuating system evolution process is implemented as a Markov chain of equivalence class objects. It is shown that the process can be characterized by the acceptance of metastable local transitions. The method is applied to a problem of Au and Ag cluster growth on a rippled surface. The simulation predicts the existence of a morphology-dependent transition time limit from a local metastable to stable state for subsequent cluster growth by accretion. Excellent agreement with observed experimental results is obtained.

16. On the optimization of free resources using non-homogeneous Markov chain software rejuvenation model

International Nuclear Information System (INIS)

Koutras, Vasilis P.; Platis, Agapios N.; Gravvanis, George A.

2007-01-01

Software rejuvenation is an important way to counteract the phenomenon of software aging and system failures. It is a preventive and proactive technique, which consists of periodically restarting an application at a clean internal state. Starting an application generally means that an amount of memory is captured and closing an application engenders the release of an amount of memory. In general, when an application is initiated an amount of memory is captured and when terminated an amount of memory is released. In this paper a model describing the amount of free memory on a system is presented. The modelling is formulated under a continuous time Markov chain framework. Additionally the cost of performing rejuvenation is also taken into consideration, a cost function for the model is produced and a rejuvenation policy is proposed. The contribution of this paper consists of using a cyclic non-homogeneous Markov chain in order to study the overall behaviour of the system capturing time dependence of the rejuvenation rates and deriving an optimal rejuvenation policy. Finally, a case study is presented in order to illustrate the results of the cost analysis

17. A semi-Markov chain approach to modeling respiratory patterns prior to extubation in preterm infants.

Science.gov (United States)

Onu, Charles C; Kanbar, Lara J; Shalish, Wissam; Brown, Karen A; Sant'Anna, Guilherme M; Kearney, Robert E; Precup, Doina

2017-07-01

After birth, extremely preterm infants often require specialized respiratory management in the form of invasive mechanical ventilation (IMV). Protracted IMV is associated with detrimental outcomes and morbidities. Premature extubation, on the other hand, would necessitate reintubation which is risky, technically challenging and could further lead to lung injury or disease. We present an approach to modeling respiratory patterns of infants who succeeded extubation and those who required reintubation which relies on Markov models. We compare the use of traditional Markov chains to semi-Markov models which emphasize cross-pattern transitions and timing information, and to multi-chain Markov models which can concisely represent non-stationarity in respiratory behavior over time. The models we developed expose specific, unique similarities as well as vital differences between the two populations.

18. Analysis of drought areas in northern Algeria using Markov chains

memoryless': loosely speaking, a process satisfies the Markov property if one can make predictions for the future of the process based solely on its present state just as well as one could know the process's full history. (Gabriel and Neuman 1962; ...

19. Application of Markov chain and entropy analysis to lithologic ...

A statistical approach by a modified Markov process model and entropy function is used to prove that the early Permian Barakar Formation of the Bellampalli coalfield developed distinct cyclicities during deposition. From results, the transition path of lithological states typical for the Bellampalli basin is as: coarse to ...

20. Quasi-stationary distributions for reducible absorbing Markov chains in discrete time

NARCIS (Netherlands)

van Doorn, Erik A.; Pollett, P.K.

2009-01-01

We consider discrete-time Markov chains with one coffin state and a finite set $S$ of transient states, and are interested in the limiting behaviour of such a chain as time $n \\to \\infty,$ conditional on survival up to $n$. It is known that, when $S$ is irreducible, the limiting conditional

1. Asymptotic period of an aperiodic Markov chain and the strong ratio limit property

NARCIS (Netherlands)

van Doorn, Erik A.

We introduce the concept of asymptotic period for an irreducible and aperiodic discrete-time Markov chain on a countable state space. If the chain is transient its asymptotic period may be larger than one. We present some sufficient conditions and, in the more restricted setting of birth-death

2. Portfolio Allocation under the Vendor Managed Inventory: A Markov ...

African Journals Online (AJOL)

also studied. The main objective of the study is to apply Markov decision process to portfolio allocation problem under vendor managed inventory environment in order to obtain the expected reward for each decision and the optimal policy that maps an action to a given state. Inventory management is very important in most.

3. Enhancement of Markov chain model by integrating exponential smoothing: A case study on Muslims marriage and divorce

Science.gov (United States)

2015-12-01

Markov Chain has been introduced since the 1913 for the purpose of studying the flow of data for a consecutive number of years of the data and also forecasting. The important feature in Markov Chain is obtaining the accurate Transition Probability Matrix (TPM). However to obtain the suitable TPM is hard especially in involving long-term modeling due to unavailability of data. This paper aims to enhance the classical Markov Chain by introducing Exponential Smoothing technique in developing the appropriate TPM.

4. A Probabilistic Short-Term Water Demand Forecasting Model Based on the Markov Chain

Directory of Open Access Journals (Sweden)

Francesca Gagliardi

2017-07-01

Full Text Available This paper proposes a short-term water demand forecasting method based on the use of the Markov chain. This method provides estimates of future demands by calculating probabilities that the future demand value will fall within pre-assigned intervals covering the expected total variability. More specifically, two models based on homogeneous and non-homogeneous Markov chains were developed and presented. These models, together with two benchmark models (based on artificial neural network and naïve methods, were applied to three real-life case studies for the purpose of forecasting the respective water demands from 1 to 24 h ahead. The results obtained show that the model based on a homogeneous Markov chain provides more accurate short-term forecasts than the one based on a non-homogeneous Markov chain, which is in line with the artificial neural network model. Both Markov chain models enable probabilistic information regarding the stochastic demand forecast to be easily obtained.

5. The Impact of Short-Sale Constraints on Asset Allocation Strategies via the Backward Markov Chain Approximation Method

OpenAIRE

Carl Chiarella; Chih-Ying Hsiao

2005-01-01

This paper considers an asset allocation strategy over a finite period under investment uncertainty and short-sale constraints as a continuous time stochastic control problem. Investment uncertainty is characterised by a stochastic interest rate and inflation risk. If there are no short-sale constraints, the optimal asset allocation strategy can be solved analytically. We consider several kinds of short-sale constraints and employ the backward Markov chain approximation method to explore the ...

6. The cluster index of regularly varying sequences with applications to limit theory for functions of multivariate Markov chains

DEFF Research Database (Denmark)

Mikosch, Thomas Valentin; Wintenberger, Olivier

2014-01-01

We introduce the cluster index of a multivariate stationary sequence and characterize the index in terms of the spectral tail process. This index plays a major role in limit theory for partial sums of sequences. We illustrate the use of the cluster index by characterizing infinite variance stable...... limit distributions and precise large deviation results for sums of multivariate functions acting on a stationary Markov chain under a drift condition....

7. Forecasting the Electricity Demand and Market Shares in Retail Electricity Market Based on System Dynamics and Markov Chain

OpenAIRE

Qingyou Yan; Chao Qin; Mingjian Nie; Le Yang

2018-01-01

Due to the deregulation of retail electricity market, consumers can choose retail electric suppliers freely, and market entities are facing fierce competition because of the increasing number of new entrants. Under these circumstances, forecasting the changes in all market entities, when market share stabilized, is important for suppliers making marketing decisions. In this paper, a market share forecasting model was established based on Markov chain, and a system dynamics model was construct...

8. Adjoint sensitivity analysis of dynamic reliability models based on Markov chains - I: Theory

International Nuclear Information System (INIS)

Cacuci, D. G.; Cacuci, D. G.; Ionescu-Bujor, M.

2008-01-01

The development of the adjoint sensitivity analysis procedure (ASAP) for generic dynamic reliability models based on Markov chains is presented, together with applications of this procedure to the analysis of several systems of increasing complexity. The general theory is presented in Part I of this work and is accompanied by a paradigm application to the dynamic reliability analysis of a simple binary component, namely a pump functioning on an 'up/down' cycle until it fails irreparably. This paradigm example admits a closed form analytical solution, which permits a clear illustration of the main characteristics of the ASAP for Markov chains. In particular, it is shown that the ASAP for Markov chains presents outstanding computational advantages over other procedures currently in use for sensitivity and uncertainty analysis of the dynamic reliability of large-scale systems. This conclusion is further underscored by the large-scale applications presented in Part II. (authors)

9. Strong diffusion formulation of Markov chain ensembles and its optimal weaker reductions

Science.gov (United States)

Güler, Marifi

2017-10-01

Two self-contained diffusion formulations, in the form of coupled stochastic differential equations, are developed for the temporal evolution of state densities over an ensemble of Markov chains evolving independently under a common transition rate matrix. Our first formulation derives from Kurtz's strong approximation theorem of density-dependent Markov jump processes [Stoch. Process. Their Appl. 6, 223 (1978), 10.1016/0304-4149(78)90020-0] and, therefore, strongly converges with an error bound of the order of lnN /N for ensemble size N . The second formulation eliminates some fluctuation variables, and correspondingly some noise terms, within the governing equations of the strong formulation, with the objective of achieving a simpler analytic formulation and a faster computation algorithm when the transition rates are constant or slowly varying. There, the reduction of the structural complexity is optimal in the sense that the elimination of any given set of variables takes place with the lowest attainable increase in the error bound. The resultant formulations are supported by numerical simulations.

10. Markov Chain Monte Carlo (MCMC) methods for parameter estimation of a novel hybrid redundant robot

International Nuclear Information System (INIS)

Wang Yongbo; Wu Huapeng; Handroos, Heikki

2011-01-01

This paper presents a statistical method for the calibration of a redundantly actuated hybrid serial-parallel robot IWR (Intersector Welding Robot). The robot under study will be used to carry out welding, machining, and remote handing for the assembly of vacuum vessel of International Thermonuclear Experimental Reactor (ITER). The robot has ten degrees of freedom (DOF), among which six DOF are contributed by the parallel mechanism and the rest are from the serial mechanism. In this paper, a kinematic error model which involves 54 unknown geometrical error parameters is developed for the proposed robot. Based on this error model, the mean values of the unknown parameters are statistically analyzed and estimated by means of Markov Chain Monte Carlo (MCMC) approach. The computer simulation is conducted by introducing random geometric errors and measurement poses which represent the corresponding real physical behaviors. The simulation results of the marginal posterior distributions of the estimated model parameters indicate that our method is reliable and robust.

11. Markov Chain Monte Carlo (MCMC) methods for parameter estimation of a novel hybrid redundant robot

Energy Technology Data Exchange (ETDEWEB)

Wang Yongbo, E-mail: yongbo.wang@hotmail.com [Laboratory of Intelligent Machine, Lappeenranta University of Technology, FIN-53851 Lappeenranta (Finland); Wu Huapeng; Handroos, Heikki [Laboratory of Intelligent Machine, Lappeenranta University of Technology, FIN-53851 Lappeenranta (Finland)

2011-10-15

This paper presents a statistical method for the calibration of a redundantly actuated hybrid serial-parallel robot IWR (Intersector Welding Robot). The robot under study will be used to carry out welding, machining, and remote handing for the assembly of vacuum vessel of International Thermonuclear Experimental Reactor (ITER). The robot has ten degrees of freedom (DOF), among which six DOF are contributed by the parallel mechanism and the rest are from the serial mechanism. In this paper, a kinematic error model which involves 54 unknown geometrical error parameters is developed for the proposed robot. Based on this error model, the mean values of the unknown parameters are statistically analyzed and estimated by means of Markov Chain Monte Carlo (MCMC) approach. The computer simulation is conducted by introducing random geometric errors and measurement poses which represent the corresponding real physical behaviors. The simulation results of the marginal posterior distributions of the estimated model parameters indicate that our method is reliable and robust.

12. Transition probabilities of health states for workers in Malaysia using a Markov chain model

Science.gov (United States)

Samsuddin, Shamshimah; Ismail, Noriszura

2017-04-01

The aim of our study is to estimate the transition probabilities of health states for workers in Malaysia who contribute to the Employment Injury Scheme under the Social Security Organization Malaysia using the Markov chain model. Our study uses four states of health (active, temporary disability, permanent disability and death) based on the data collected from the longitudinal studies of workers in Malaysia for 5 years. The transition probabilities vary by health state, age and gender. The results show that men employees are more likely to have higher transition probabilities to any health state compared to women employees. The transition probabilities can be used to predict the future health of workers in terms of a function of current age, gender and health state.

13. From complex spatial dynamics to simple Markov chain models: do predators and prey leave footprints?

DEFF Research Database (Denmark)

Nachman, Gøsta Støger; Borregaard, Michael Krabbe

2010-01-01

to another, are then depicted in a state transition diagram, constituting the "footprints" of the underlying population dynamics. We investigate to what extent changes in the population processes modeled in the complex simulation (i.e. the predator's functional response and the dispersal rates of both......In this paper we present a concept for using presence-absence data to recover information on the population dynamics of predator-prey systems. We use a highly complex and spatially explicit simulation model of a predator-prey mite system to generate simple presence-absence data: the number...... of transition probabilities on state variables, and combine this information in a Markov chain transition matrix model. Finally, we use this extended model to predict the long-term dynamics of the system and to reveal its asymptotic steady state properties....

14. A MATLAB Package for Markov Chain Monte Carlo with a Multi-Unidimensional IRT Model

Directory of Open Access Journals (Sweden)

Yanyan Sheng

2008-11-01

Full Text Available Unidimensional item response theory (IRT models are useful when each item is designed to measure some facet of a unified latent trait. In practical applications, items are not necessarily measuring the same underlying trait, and hence the more general multi-unidimensional model should be considered. This paper provides the requisite information and description of software that implements the Gibbs sampler for such models with two item parameters and a normal ogive form. The software developed is written in the MATLAB package IRTmu2no. The package is flexible enough to allow a user the choice to simulate binary response data with multiple dimensions, set the number of total or burn-in iterations, specify starting values or prior distributions for model parameters, check convergence of the Markov chain, as well as obtain Bayesian fit statistics. Illustrative examples are provided to demonstrate and validate the use of the software package.

15. 2nd International Workshop on the Numerical Solution of Markov Chains

CERN Document Server

1995-01-01

Computations with Markov Chains presents the edited and reviewed proceedings of the Second International Workshop on the Numerical Solution of Markov Chains, held January 16--18, 1995, in Raleigh, North Carolina. New developments of particular interest include recent work on stability and conditioning, Krylov subspace-based methods for transient solutions, quadratic convergent procedures for matrix geometric problems, further analysis of the GTH algorithm, the arrival of stochastic automata networks at the forefront of modelling stratagems, and more. An authoritative overview of the field for applied probabilists, numerical analysts and systems modelers, including computer scientists and engineers.

16. A Cost-Effective Smoothed Multigrid with Modified Neighborhood-Based Aggregation for Markov Chains

Directory of Open Access Journals (Sweden)

Zhao-Li Shen

2015-01-01

Full Text Available Smoothed aggregation multigrid method is considered for computing stationary distributions of Markov chains. A judgement which determines whether to implement the whole aggregation procedure is proposed. Through this strategy, a large amount of time in the aggregation procedure is saved without affecting the convergence behavior. Besides this, we explain the shortage and irrationality of the Neighborhood-Based aggregation which is commonly used in multigrid methods. Then a modified version is presented to remedy and improve it. Numerical experiments on some typical Markov chain problems are reported to illustrate the performance of these methods.

17. PERAMALAN INDEKS HARGA SAHAM GABUNGAN (IHSG DENGAN METODE FUZZY TIME SERIES MARKOV CHAIN

Directory of Open Access Journals (Sweden)

Y Aristyani

2016-04-01

Full Text Available Tujuan penelitian ini adalah untuk mengetahui akurasi metode Fuzzy Time Series Markov Chain pada peramalan IHSG dan membuat aplikasi untuk peramalan IHSG menggunakan software MATLAB. Dalam penelitian  ini, data bersumber dari yahoo finance. Data historis diambil dari data Composite Indeks (IHSG periode Januari 2010 sampai dengan Februari 2014. Dengan mengubah data time series IHSG ke dalam fuzzy logic group untuk menentukan matriks probabilitas transisi, maka hasil peramalan dapat diperoleh. Tahap awal pembuatan aplikasi yaitu perancangan sistem. Aplikasi untuk peramalan IHSG dirancang dengan menggunakan GUI pada MATLAB dengan melakukan coding yang sesuai agar aplikasi bisa berjalan. Setelah dilakukan pengujian sistem diperoleh hasil MSE untuk metode Fuzzy Time Series Markov Chain sebesar 9827.1292 dan MSE untuk  metode Fuzzy Time Series S&C sebesar 15769.7036. Karena  memperoleh nilai MSE yang lebih kecil maka metode Fuzzy Time Series Markov Chain lebih akurat dan memiliki kinerja yang lebih baik untuk peramalan. Aplikasi yang dibuat memiliki persentase akurasi peramalan dengan metode Fuzzy Time Series Markov Chain sebesar 98,03458% dan persentase akurasi peramalan dengan metode Fuzzy Time Series S&C sebesar 97,38003%.The purpose of  this research were to determine the accuracy of the Markov Chain Fuzzy Time Series method on JCI forecasting and make an application for JCI forecasting using MATLAB software. In this research, the data sourced from Yahoo Finance. Historical data is taken from Data Composite Index (JCI in the period of January 2010 to February 2014. By transfering time series data into fuzzy logic groups to determine the transition probability matrix, then the forecasting results can be obtained. The initial phase to making the application  is system design. Application for JCI forecasting designed using GUI on MATLAB with appropriate coding in order to run the application. After testing the system then obtained MSE results

18. Bounding spectral gaps of Markov chains: a novel exact multi-decomposition technique

International Nuclear Information System (INIS)

Destainville, N

2003-01-01

We propose an exact technique to calculate lower bounds of spectral gaps of discrete time reversible Markov chains on finite state sets. Spectral gaps are a common tool for evaluating convergence rates of Markov chains. As an illustration, we successfully use this technique to evaluate the 'absorption time' of the 'Backgammon model', a paradigmatic model for glassy dynamics. We also discuss the application of this technique to the 'contingency table problem', a notoriously difficult problem from probability theory. The interest of this technique is that it connects spectral gaps, which are quantities related to dynamics, with static quantities, calculated at equilibrium

19. Strong, Weak and Branching Bisimulation for Transition Systems and Markov Reward Chains: A Unifying Matrix Approach

Directory of Open Access Journals (Sweden)

Nikola Trčka

2009-12-01

Full Text Available We first study labeled transition systems with explicit successful termination. We establish the notions of strong, weak, and branching bisimulation in terms of boolean matrix theory, introducing thus a novel and powerful algebraic apparatus. Next we consider Markov reward chains which are standardly presented in real matrix theory. By interpreting the obtained matrix conditions for bisimulations in this setting, we automatically obtain the definitions of strong, weak, and branching bisimulation for Markov reward chains. The obtained strong and weak bisimulations are shown to coincide with some existing notions, while the obtained branching bisimulation is new, but its usefulness is questionable.

20. Markov Chain Model with Catastrophe to Determine Mean Time to Default of Credit Risky Assets

Science.gov (United States)

Dharmaraja, Selvamuthu; Pasricha, Puneet; Tardelli, Paola

2017-11-01

This article deals with the problem of probabilistic prediction of the time distance to default for a firm. To model the credit risk, the dynamics of an asset is described as a function of a homogeneous discrete time Markov chain subject to a catastrophe, the default. The behaviour of the Markov chain is investigated and the mean time to the default is expressed in a closed form. The methodology to estimate the parameters is given. Numerical results are provided to illustrate the applicability of the proposed model on real data and their analysis is discussed.

1. The dynamics of stochastic attrition viewed as an absorption time on a terminating Markov chain.

Science.gov (United States)

DeRemigio, Hilary; Smith, Gregory D

2005-08-01

Localized Ca(2+) elevations known as Ca(2+) puffs and sparks are cellular signals that arise from the cooperative activity of clusters of inositol 1,4,5-trisphosphate receptors and ryanodine receptors clustered at Ca(2+) release sites on the surface of the endoplasmic reticulum or sarcoplasmic reticulum. When Markov chain models of these intracellular Ca(2+)-regulated Ca(2+) channels are coupled via a mathematical representation of Ca(2+) microdomain, simulated Ca(2+) release sites may exhibit the phenomenon of "stochastic Ca(2+) excitability" where the inositol 1,4,5-trisphosphate receptors (IP(3)Rs) or ryanodine receptors (RyRs) open and close in a concerted fashion. Interestingly, under some conditions simulated puffs and sparks can be observed even when the single-channel model used does not include slow Ca(2+) inactivation or, indeed, any long-lived closed/refractory state [V. Nguyen, R. Mathias, G. Smith, Stochastic automata network descriptor for Markov chain models of instantaneously-coupled intracellular Ca(2+) channels, Bull. Math. Biol. 67 (2005) 393-432]. In this case, termination of the localized Ca(2+) elevation occurs when all of the intracellular channels at a release site simultaneously close through a process referred to as stochastic attrition [M. Stern, Theory of excitation-contraction coupling in cardiac muscle, Biophys. J. 63 (1992) 497-517]. In this paper, we investigate the statistical properties of stochastic attrition viewed as an absorption time on a terminating Markov chain that represents a Ca(2+) release site composed of N two-state channels that are activated by Ca(2+). Assuming that the local [Ca(2+)] experienced by a channel depends only on the number of open channels at the Ca(2+) release site (i.e., instantaneous mean-field coupling [ibid.], we derive the probability distribution function for the time until stochastic attrition occurs and present an analytical formula for the expectation of this random variable. We explore how the

2. DOOB-MARTIN COMPACTIFICATION OF A MARKOV CHAIN FOR GROWING RANDOM WORDS SEQUENTIALLY.

Science.gov (United States)

Choi, Hye Soo; Evans, Steven N

2017-07-01

We consider a Markov chain that iteratively generates a sequence of random finite words in such a way that the n th word is uniformly distributed over the set of words of length 2 n in which n letters are a and n letters are b: at each step an a and a b are shuffled in uniformly at random among the letters of the current word. We obtain a concrete characterization of the Doob-Martin boundary of this Markov chain and thereby delineate all the ways in which the Markov chain can be conditioned to behave at large times. Writing N(u) for the number of letters a (equivalently, b ) in the finite word u , we show that a sequence ( u n ) n ∈ℕ of finite words converges to a point in the boundary if, for an arbitrary word ν, there is convergence as n tends to infinity of the probability that the selection of N ( ν ) letters a and N ( ν ) letters b uniformly at random from u n and maintaining their relative order results in ν . We exhibit a bijective correspondence between the points in the boundary and ergodic random total orders on the set { a 1 , b 1 , a 2 , b 2 , …} that have distributions which are separately invariant under finite permutations of the indices of the a 's and those of the b 's. We establish a further bijective correspondence between the set of such random total orders and the set of pairs ( μ , ν ) of diffuse probability measures on [0,1] such that ½( μ + ν ) is Lebesgue measure: the restriction of the random total order to { a 1 , b 1 ,…, a n , b n } is obtained by taking X 1 ,…, X n (resp. Y 1 ,… , Y n ) i.i.d. with common distribution μ (resp. ν ), letting ( Z 1 ,…, Z 2n ) be { X 1 , Y 1 ,…, X n , Y n } in increasing order, and declaring that the k th smallest element in the restricted total order is a i (resp. b j ) if Z k = X i (resp. Z k = Y j ).

3. Markov Chain-Like Quantum Biological Modeling of Mutations, Aging, and Evolution

Directory of Open Access Journals (Sweden)

Ivan B. Djordjevic

2015-08-01

Full Text Available Recent evidence suggests that quantum mechanics is relevant in photosynthesis, magnetoreception, enzymatic catalytic reactions, olfactory reception, photoreception, genetics, electron-transfer in proteins, and evolution; to mention few. In our recent paper published in Life, we have derived the operator-sum representation of a biological channel based on codon basekets, and determined the quantum channel model suitable for study of the quantum biological channel capacity. However, this model is essentially memoryless and it is not able to properly model the propagation of mutation errors in time, the process of aging, and evolution of genetic information through generations. To solve for these problems, we propose novel quantum mechanical models to accurately describe the process of creation spontaneous, induced, and adaptive mutations and their propagation in time. Different biological channel models with memory, proposed in this paper, include: (i Markovian classical model, (ii Markovian-like quantum model, and (iii hybrid quantum-classical model. We then apply these models in a study of aging and evolution of quantum biological channel capacity through generations. We also discuss key differences of these models with respect to a multilevel symmetric channel-based Markovian model and a Kimura model-based Markovian process. These models are quite general and applicable to many open problems in biology, not only biological channel capacity, which is the main focus of the paper. We will show that the famous quantum Master equation approach, commonly used to describe different biological processes, is just the first-order approximation of the proposed quantum Markov chain-like model, when the observation interval tends to zero. One of the important implications of this model is that the aging phenotype becomes determined by different underlying transition probabilities in both programmed and random (damage Markov chain-like models of aging, which

4. Refinement of River Basin Topography and Bathymetry Using Markov Chain Monte Carlo sampling

Science.gov (United States)

Askarizadeh Bardsiri, A.; Vrugt, J. A.; Schubert, J. E.; Sanders, B. F.

2011-12-01

Hydrodynamic models have found widespread application and use to predict (spatially distributed) water level depths within a river system and its adjacent flood plains. The accuracy of the flood extent, depth and velocity predictions of such models depends strongly on the availability of reliable topographic data that properly characterizes the features of the terrain under investigation. Commonly used topographic data measured with remote sensing or lidar equipment are subject to measurement error that can severely corrupt the predictive skill of hydrodynamic models, but efforts to fit hydrodynamic models to data have rarely considered this effect and have mainly focused on resistance parameters and forcing uncertainty. Here, we introduce an alternative hydrodynamic inverse modeling framework that explicitly considers the role of topographic errors. Multipliers are assigned to the most important topographic features of the terrain, and these scalars are estimated simultaneously with surface resistance parameters and friction coefficients using available calibration data. Our methodology is illustrated using the ParBreZo hydrodynamic flood simulation algorithm and data from the Platte river basin in Nebraska, USA. Markov Chain Monte Carlo (MCMC) simulation with the Generalized Likelihood function of Schoups and Vrugt (2010) was used to summarize posterior parameter and model prediction distributions. Our poster will discuss the historical developments in hydrodynamic modeling that led to current perspectives, and advocates a paradigm shift in favor of a better treatment of topographic errors and forcing data uncertainty. We will show ParBreZo modeling results for this alternative blueprint. Keywords: Hydrodynamics, Topography, River Bathymetry, Platte river, ParBrezo Model, Markov Chain Monte Carlo simulation, Generalized Likelihood Function.

5. On dynamic selection of households for direct marketing based on Markov chain models with memory

NARCIS (Netherlands)

Otter, Pieter W.

A simple, dynamic selection procedure is proposed, based on conditional, expected profits using Markov chain models with memory. The method is easy to apply, only frequencies and mean values have to be calculated or estimated. The method is empirically illustrated using a data set from a charitable

6. Particle Markov Chain Monte Carlo Techniques of Unobserved Component Time Series Models Using Ox

DEFF Research Database (Denmark)

This paper details Particle Markov chain Monte Carlo techniques for analysis of unobserved component time series models using several economic data sets. PMCMC combines the particle filter with the Metropolis-Hastings algorithm. Overall PMCMC provides a very compelling, computationally fast...

7. On estimates of the rate of convergence In the global limit theorems for Homogeneous markov chains

OpenAIRE

Gharib, M. [محمد غريب محمود

1997-01-01

In this paper some estimates are obtained for the remainder term in the limit theorems for the weighted sum of random variables forming a homogeneous Markov chain with arbitrary set of possible states . The achieved results make it possible to estimate the rate of convergence in these theorems in the metric of the space Lp, 1

8. Markov Chain model for the stochastic behaviors of wind-direction data

International Nuclear Information System (INIS)

Masseran, Nurulkamal

2015-01-01

Highlights: • I develop a Markov chain model to describe about the stochastic and probabilistic behaviors of wind direction data. • I describe some of the theoretical arguments regarding the Markov chain model in term of wind direction data. • I suggest a limiting probabilities approach to determine a dominant directions of wind blow. - Abstract: Analyzing the behaviors of wind direction can complement knowledge concerning wind speed and help researchers draw conclusions regarding wind energy potential. Knowledge of the wind’s direction enables the wind turbine to be positioned in such a way as to maximize the total amount of captured energy and optimize the wind farm’s performance. In this paper, first-order and higher-order Markov chain models are proposed to describe the probabilistic behaviors of wind-direction data. A case study is conducted using data from Mersing, Malaysia. The wind-direction data are classified according to an eight-state Markov chain based on natural geographical directions. The model’s parameters are estimated using the maximum likelihood method and the linear programming formulation. Several theoretical arguments regarding the model are also discussed. Finally, limiting probabilities are used to determine a long-run proportion of the wind directions generated. The results explain the dominant direction for Mersing’s wind in terms of probability metrics

9. Modelling Faculty Replacement Strategies Using a Time-Dependent Finite Markov-Chain Process.

Science.gov (United States)

Hackett, E. Raymond; Magg, Alexander A.; Carrigan, Sarah D.

1999-01-01

Describes the use of a time-dependent Markov-chain model to develop faculty-replacement strategies within a college at a research university. The study suggests that a stochastic modelling approach can provide valuable insight when planning for personnel needs in the immediate (five-to-ten year) future. (MSE)

10. Counseling as a Stochastic Process: Fitting a Markov Chain Model to Initial Counseling Interviews

Science.gov (United States)

Lichtenberg, James W.; Hummel, Thomas J.

1976-01-01

The goodness of fit of a first-order Markov chain model to six counseling interviews was assessed by using chi-square tests of homogeneity and simulating sampling distributions of selected process characteristics against which the same characteristics in the actual interviews were compared. The model fit four of the interviews. Presented at AERA,…

11. Avian life history profiles for use in the Markov chain nest productivity model (MCnest)

Science.gov (United States)

The Markov Chain nest productivity model, or MCnest, quantitatively estimates the effects of pesticides or other toxic chemicals on annual reproductive success of avian species (Bennett and Etterson 2013, Etterson and Bennett 2013). The Basic Version of MCnest was developed as a...

12. Automated compositional Markov chain generation for a plain-old telephone system

NARCIS (Netherlands)

Hermanns, H.; Katoen, Joost P.

Obtaining performance models, like Markov chains and queueing networks, for systems of significant complexity and magnitude is a difficult task that is usually tackled using human intelligence and experience. This holds in particular for performance models of a highly irregular nature. In this paper

13. Tokunaga and Horton self-similarity for level set trees of Markov chains

International Nuclear Information System (INIS)

Zaliapin, Ilia; Kovchegov, Yevgeniy

2012-01-01

Highlights: ► Self-similar properties of the level set trees for Markov chains are studied. ► Tokunaga and Horton self-similarity are established for symmetric Markov chains and regular Brownian motion. ► Strong, distributional self-similarity is established for symmetric Markov chains with exponential jumps. ► It is conjectured that fractional Brownian motions are Tokunaga self-similar. - Abstract: The Horton and Tokunaga branching laws provide a convenient framework for studying self-similarity in random trees. The Horton self-similarity is a weaker property that addresses the principal branching in a tree; it is a counterpart of the power-law size distribution for elements of a branching system. The stronger Tokunaga self-similarity addresses so-called side branching. The Horton and Tokunaga self-similarity have been empirically established in numerous observed and modeled systems, and proven for two paradigmatic models: the critical Galton–Watson branching process with finite progeny and the finite-tree representation of a regular Brownian excursion. This study establishes the Tokunaga and Horton self-similarity for a tree representation of a finite symmetric homogeneous Markov chain. We also extend the concept of Horton and Tokunaga self-similarity to infinite trees and establish self-similarity for an infinite-tree representation of a regular Brownian motion. We conjecture that fractional Brownian motions are also Tokunaga and Horton self-similar, with self-similarity parameters depending on the Hurst exponent.

14. Learning Bayesian network classifiers for credit scoring using Markov Chain Monte Carlo search

NARCIS (Netherlands)

Baesens, B.; Egmont-Petersen, M.; Castelo, R.; Vanthienen, J.

2001-01-01

In this paper, we will evaluate the power and usefulness of Bayesian network classifiers for credit scoring. Various types of Bayesian network classifiers will be evaluated and contrasted including unrestricted Bayesian network classifiers learnt using Markov Chain Monte Carlo (MCMC) search.

15. Confronting uncertainty in model-based geostatistics using Markov Chain Monte Carlo simulation

NARCIS (Netherlands)

Minasny, B.; Vrugt, J.A.; McBratney, A.B.

2011-01-01

This paper demonstrates for the first time the use of Markov Chain Monte Carlo (MCMC) simulation for parameter inference in model-based soil geostatistics. We implemented the recently developed DiffeRential Evolution Adaptive Metropolis (DREAM) algorithm to jointly summarize the posterior

16. Accelerating Markov chain Monte Carlo simulation by differential evolution with self-adaptive randomized subspace sampling

NARCIS (Netherlands)

Vrugt, J.A.; Braak, ter C.J.F.; Diks, C.G.H.; Robinson, B.A.; Hyman, J.M.; Higdon, D.

2009-01-01

Markov chain Monte Carlo (MCMC) methods have found widespread use in many fields of study to estimate the average properties of complex systems, and for posterior inference in a Bayesian framework. Existing theory and experiments prove convergence of well-constructed MCMC schemes to the appropriate

17. Accelerating Markov chain Monte Carlo simulation by differential evolution with self-adaptive randomized subspace sampling

NARCIS (Netherlands)

Vrugt, J.A.; Braak, C.J.F.; Diks, C.G.H.; Robinson, B.A.; Hyman, J.M.; Higdon, D.

2009-01-01

Markov chain Monte Carlo (MCMC) methods have found widespread use in many fields of study to estimate the average properties of complex systems, and for posterior inference in a Bayesian framework. Existing theory and experiments prove convergence of well constructed MCMC schemes to the appropriate

18. Treatment of input uncertainty in hydrologic modeling: Doing hydrology backward with Markov chain Monte Carlo simulation

NARCIS (Netherlands)

Vrugt, J.A.; Braak, ter C.J.F.; Clark, M.P.; Hyman, J.M.; Robinson, B.A.

2008-01-01

There is increasing consensus in the hydrologic literature that an appropriate framework for streamflow forecasting and simulation should include explicit recognition of forcing and parameter and model structural error. This paper presents a novel Markov chain Monte Carlo (MCMC) sampler, entitled

19. Integration within the Felsenstein equation for improved Markov chain Monte Carlo methods in population genetics

DEFF Research Database (Denmark)

Hey, Jody; Nielsen, Rasmus

2007-01-01

Carlo methods, have been developed to find approximate solutions. Here, we describe an approach in which Markov chain Monte Carlo simulations are used to integrate over the space of genealogies, whereas other parameters are integrated out analytically. The result is an approximation to the full joint...

20. Some strong limit theorems for nonhomogeneous Markov chains indexed by controlled trees

Directory of Open Access Journals (Sweden)

Weicai Peng

2016-02-01

Full Text Available Abstract In this paper, a kind of infinite, local finite tree T, named a controlled tree, is introduced. Some strong limit properties, such as the strong law of large numbers and the asymptotic equipartition property, for nonhomogeneous Markov chains indexed by T, are established. The outcomes are the generalizations of some well-known results.

1. Derivation of a Markov state model of the dynamics of a protein-like chain immersed in an implicit solvent.

Science.gov (United States)

Schofield, Jeremy; Bayat, Hanif

2014-09-07

A Markov state model of the dynamics of a protein-like chain immersed in an implicit hard sphere solvent is derived from first principles for a system of monomers that interact via discontinuous potentials designed to account for local structure and bonding in a coarse-grained sense. The model is based on the assumption that the implicit solvent interacts on a fast time scale with the monomers of the chain compared to the time scale for structural rearrangements of the chain and provides sufficient friction so that the motion of monomers is governed by the Smoluchowski equation. A microscopic theory for the dynamics of the system is developed that reduces to a Markovian model of the kinetics under well-defined conditions. Microscopic expressions for the rate constants that appear in the Markov state model are analyzed and expressed in terms of a temperature-dependent linear combination of escape rates that themselves are independent of temperature. Excellent agreement is demonstrated between the theoretical predictions of the escape rates and those obtained through simulation of a stochastic model of the dynamics of bond formation. Finally, the Markov model is studied by analyzing the eigenvalues and eigenvectors of the matrix of transition rates, and the equilibration process for a simple helix-forming system from an ensemble of initially extended configurations to mainly folded configurations is investigated as a function of temperature for a number of different chain lengths. For short chains, the relaxation is primarily single-exponential and becomes independent of temperature in the low-temperature regime. The profile is more complicated for longer chains, where multi-exponential relaxation behavior is seen at intermediate temperatures followed by a low temperature regime in which the folding becomes rapid and single exponential. It is demonstrated that the behavior of the equilibration profile as the temperature is lowered can be understood in terms of the

2. The Fracture Mechanical Markov Chain Fatigue Model Compared with Empirical Data

DEFF Research Database (Denmark)

Gansted, L.; Brincker, Rune; Hansen, Lars Pilegaard

The applicability of the FMF-model (Fracture Mechanical Markov Chain Fatigue Model) introduced in Gansted, L., R. Brincker and L. Pilegaard Hansen (1991) is tested by simulations and compared with empirical data. Two sets of data have been used, the Virkler data (aluminium alloy) and data...... established at the Laboratory of Structural Engineering at Aalborg University, the AUC-data, (mild steel). The model, which is based on the assumption, that the crack propagation process can be described by a discrete Space Markov theory, is applicable to constant as well as random loading. It is shown...

3. From Brownian Dynamics to Markov Chain: An Ion Channel Example

KAUST Repository

Chen, Wan

2014-02-27

A discrete rate theory for multi-ion channels is presented, in which the continuous dynamics of ion diffusion is reduced to transitions between Markovian discrete states. In an open channel, the ion permeation process involves three types of events: an ion entering the channel, an ion escaping from the channel, or an ion hopping between different energy minima in the channel. The continuous dynamics leads to a hierarchy of Fokker-Planck equations, indexed by channel occupancy. From these the mean escape times and splitting probabilities (denoting from which side an ion has escaped) can be calculated. By equating these with the corresponding expressions from the Markov model, one can determine the Markovian transition rates. The theory is illustrated with a two-ion one-well channel. The stationary probability of states is compared with that from both Brownian dynamics simulation and the hierarchical Fokker-Planck equations. The conductivity of the channel is also studied, and the optimal geometry maximizing ion flux is computed. © 2014 Society for Industrial and Applied Mathematics.

4. First and second order semi-Markov chains for wind speed modeling

Science.gov (United States)

Prattico, F.; Petroni, F.; D'Amico, G.

2012-04-01

The increasing interest in renewable energy leads scientific research to find a better way to recover most of the available energy. Particularly, the maximum energy recoverable from wind is equal to 59.3% of that available (Betz law) at a specific pitch angle and when the ratio between the wind speed in output and in input is equal to 1/3. The pitch angle is the angle formed between the airfoil of the blade of the wind turbine and the wind direction. Old turbine and a lot of that actually marketed, in fact, have always the same invariant geometry of the airfoil. This causes that wind turbines will work with an efficiency that is lower than 59.3%. New generation wind turbines, instead, have a system to variate the pitch angle by rotating the blades. This system able the wind turbines to recover, at different wind speed, always the maximum energy, working in Betz limit at different speed ratios. A powerful system control of the pitch angle allows the wind turbine to recover better the energy in transient regime. A good stochastic model for wind speed is then needed to help both the optimization of turbine design and to assist the system control to predict the value of the wind speed to positioning the blades quickly and correctly. The possibility to have synthetic data of wind speed is a powerful instrument to assist designer to verify the structures of the wind turbines or to estimate the energy recoverable from a specific site. To generate synthetic data, Markov chains of first or higher order are often used [1,2,3]. In particular in [3] is presented a comparison between a first-order Markov chain and a second-order Markov chain. A similar work, but only for the first-order Markov chain, is conduced by [2], presenting the probability transition matrix and comparing the energy spectral density and autocorrelation of real and synthetic wind speed data. A tentative to modeling and to join speed and direction of wind is presented in [1], by using two models, first

5. Understanding for convergence monitoring for probabilistic risk assessment based on Markov Chain Monte Carlo Simulation

International Nuclear Information System (INIS)

Kim, Joo Yeon; Jang, Han Ki; Jang, Sol Ah; Park, Tae Jin

2014-01-01

There is a question that the simulation actually leads to draws from its target distribution and the most basic one is whether such Markov chains can always be constructed and all chain values sampled from them. The problem to be solved is the determination of how large this iteration should be to achieve the target distribution. This problem can be answered as convergence monitoring. In this paper, two widely used methods, such as autocorrelation and potential scale reduction factor (PSRF) in MCMC are characterized. There is no general agreement on the subject of the convergence. Although it is generally agreed that running n parallel chains in practice is computationally inefficient and unnecessary, running multiple parallel chains is generally applied for the convergence monitoring due to easy implementation. The main debate is the number of parallel chains needed. If the convergence properties of the chain are well understood then clearly a single chain suffices. Therefore, autocorrelation using single chain and multiple parallel ones are tried and their results then compared with each other in this study. And, the following question is answered from the two convergence results: Have the Markov chain realizations for achieved the target distribution?

6. Detecting memory and structure in human navigation patterns using Markov chain models of varying order.

Directory of Open Access Journals (Sweden)

Philipp Singer

Full Text Available One of the most frequently used models for understanding human navigation on the Web is the Markov chain model, where Web pages are represented as states and hyperlinks as probabilities of navigating from one page to another. Predominantly, human navigation on the Web has been thought to satisfy the memoryless Markov property stating that the next page a user visits only depends on her current page and not on previously visited ones. This idea has found its way in numerous applications such as Google's PageRank algorithm and others. Recently, new studies suggested that human navigation may better be modeled using higher order Markov chain models, i.e., the next page depends on a longer history of past clicks. Yet, this finding is preliminary and does not account for the higher complexity of higher order Markov chain models which is why the memoryless model is still widely used. In this work we thoroughly present a diverse array of advanced inference methods for determining the appropriate Markov chain order. We highlight strengths and weaknesses of each method and apply them for investigating memory and structure of human navigation on the Web. Our experiments reveal that the complexity of higher order models grows faster than their utility, and thus we confirm that the memoryless model represents a quite practical model for human navigation on a page level. However, when we expand our analysis to a topical level, where we abstract away from specific page transitions to transitions between topics, we find that the memoryless assumption is violated and specific regularities can be observed. We report results from experiments with two types of navigational datasets (goal-oriented vs. free form and observe interesting structural differences that make a strong argument for more contextual studies of human navigation in future work.

7. MODELING OF FUTURE LAND COVER LAND USE CHANGE IN NORTH CAROLINA USING MARKOV CHAIN AND CELLULAR AUTOMATA MODEL

OpenAIRE

2014-01-01

State wide variant topographic features in North Carolina attract the hydro-climatologist. There is none modeling study found that predict future Land Cover Land Use (LCLU) change for whole North Carolina. In this study, satellite-derived land cover maps of year 1992, 2001 and 2006 of North Carolina were integrated within the framework of the Markov-Cellular Automata (Markov-CA) model which combines the Markov chain and Cellular Automata (CA) techniques. A Multi-Criteria Evaluation (MCE) was ...

8. Psychotherapy as Stochastic Process: Fitting a Markov Chain Model to Interviews of Ellis and Rogers. University of Minnesota Office of Student Affairs Research Bulletin, Vol. 15, No. 18.

Science.gov (United States)

Lichtenberg, James W.; Hummel, Thomas J.

This investigation tested the hypothesis that the probabilistic structure underlying psychotherapy interviews is Markovian. The "goodness of fit" of a first-order Markov chain model to actual therapy interviews was assessed using a x squared test of homogeneity, and by generating by Monte Carlo methods empirical sampling distributions of…

9. Modeling Urban Expansion in Bangkok Metropolitan Region Using Demographic–Economic Data through Cellular Automata-Markov Chain and Multi-Layer Perceptron-Markov Chain Models

Directory of Open Access Journals (Sweden)

Chudech Losiri

2016-07-01

Full Text Available Urban expansion is considered as one of the most important problems in several developing countries. Bangkok Metropolitan Region (BMR is the urbanized and agglomerated area of Bangkok Metropolis (BM and its vicinity, which confronts the expansion problem from the center of the city. Landsat images of 1988, 1993, 1998, 2003, 2008, and 2011 were used to detect the land use and land cover (LULC changes. The demographic and economic data together with corresponding maps were used to determine the driving factors for land conversions. This study applied Cellular Automata-Markov Chain (CA-MC and Multi-Layer Perceptron-Markov Chain (MLP-MC to model LULC and urban expansions. The performance of the CA-MC and MLP-MC yielded more than 90% overall accuracy to predict the LULC, especially the MLP-MC method. Further, the annual population and economic growth rates were considered to produce the land demand for the LULC in 2014 and 2035 using the statistical extrapolation and system dynamics (SD. It was evident that the simulated map in 2014 resulting from the SD yielded the highest accuracy. Therefore, this study applied the SD method to generate the land demand for simulating LULC in 2035. The outcome showed that urban occupied the land around a half of the BMR.

10. Combine Grey System Theory and Markov Chain to Forecast Groundwater Level

Science.gov (United States)

Huang, C.; Huang, S.; Wen, J.; Lee, J.; Tseng, C.

2010-12-01

In this study, the grey relational analysis was used to infer the infiltration time during rainfall that infiltrated into the water table, then the infiltration time was employed to improve the GM(1,2) to predict the changes in groundwater level. Finally a random process (Markov chain) was used to amend the systematic errors of GM(1,2). For this reason, this study has two purposes: (1) to use the grey relational analysis to estimate the infiltration time to improve GM(1,2), then to explore the possibility of predicting the groundwater level, and (2) to explore the feasibility of a random process (Markov chain) to amend the systematic errors of GM(1,2). In this study, four groundwater observation wells (BH05, BH06, BH08 and BH10) were set up. The observation data was used to analysis the infiltration time, to built the GM(1,2), to built the Markov chain model, and finally to combine the grey system with the Markov chain. After calculations were made, the infiltration time Tbi for the wells was obtained (BH05:infiltration time Tb5=40 hours BH06:infiltration time Tb6=40hours BH08:infiltration time Tb8=39hours BH10: infiltration time Tb10=44hours). According to the results, the infiltration time for each well was not the same. This is because the field was a non-homogeneous and non-isotropic site; however the grey relational grade still had the similar trend. During the model building process, the average value of the grey parameters C1 and C2 was 0.9996 and 0.0561, respectively. After testing and verification, the average values obtained of MAE and RMSE was 5mm and 8 mm, respectively. The verification accuracy was fairly good. Even though the forecast time was raised from one hour to 24 hours, the verification accuracy was still acceptable. During the Markov chain process, after making the grey prediction, the disturbance value was analyzed from one to 24 hours. It was detected that if the time exceeded 12 hours, the maximum and minimum disturbance values would

11. MC3: Multi-core Markov-chain Monte Carlo code

Science.gov (United States)

Cubillos, Patricio; Harrington, Joseph; Lust, Nate; Foster, AJ; Stemm, Madison; Loredo, Tom; Stevenson, Kevin; Campo, Chris; Hardin, Matt; Hardy, Ryan

2016-10-01

MC3 (Multi-core Markov-chain Monte Carlo) is a Bayesian statistics tool that can be executed from the shell prompt or interactively through the Python interpreter with single- or multiple-CPU parallel computing. It offers Markov-chain Monte Carlo (MCMC) posterior-distribution sampling for several algorithms, Levenberg-Marquardt least-squares optimization, and uniform non-informative, Jeffreys non-informative, or Gaussian-informative priors. MC3 can share the same value among multiple parameters and fix the value of parameters to constant values, and offers Gelman-Rubin convergence testing and correlated-noise estimation with time-averaging or wavelet-based likelihood estimation methods.

12. Summary statistics for end-point conditioned continuous-time Markov chains

DEFF Research Database (Denmark)

Hobolth, Asger; Jensen, Jens Ledet

two states and the distribution of the total number of jumps) for discretely observed continuous time Markov chains. Three alternative methods for calculating properties of summary statistics are described and the pros and cons of the methods are discussed. The methods are based on (i) an eigenvalue...... decomposition of the rate matrix, (ii) the uniformization method, and (iii) integrals of matrix exponentials. In particular we develop a framework that allows for analyses of rather general summary statistics using the uniformization method.......Continuous-time Markov chains are a widely used modelling tool. Applications include DNA sequence evolution, ion channel gating behavior and mathematical finance. We consider the problem of calculating properties of summary statistics (e.g. mean time spent in a state, mean number of jumps between...

13. Network Security Risk Assessment System Based on Attack Graph and Markov Chain

Science.gov (United States)

Sun, Fuxiong; Pi, Juntao; Lv, Jin; Cao, Tian

2017-10-01

Network security risk assessment technology can be found in advance of the network problems and related vulnerabilities, it has become an important means to solve the problem of network security. Based on attack graph and Markov chain, this paper provides a Network Security Risk Assessment Model (NSRAM). Based on the network infiltration tests, NSRAM generates the attack graph by the breadth traversal algorithm. Combines with the international standard CVSS, the attack probability of atomic nodes are counted, and then the attack transition probabilities of ones are calculated by Markov chain. NSRAM selects the optimal attack path after comprehensive measurement to assessment network security risk. The simulation results show that NSRAM can reflect the actual situation of network security objectively.

14. A Revisit of Block Power Methods for Finite State Markov Chain Applications

OpenAIRE

Ji, Hao; Weinberg, Seth H.; Li, Yaohang

2016-01-01

In this paper, we revisit the generalized block power methods for approximating the eigenvector associated with $\\lambda_1 = 1$ of a Markov chain transition matrix. Our analysis of the block power method shows that when $s$ linearly independent probability vectors are used as the initial block, the convergence of the block power method to the stationary distribution depends on the magnitude of the $(s+1)$th dominant eigenvalue $\\lambda_{s+1}$ of $P$ instead of that of $\\lambda_2$ in the power...

15. A new fuzzy Monte Carlo method for solving SLAE with ergodic fuzzy Markov chains

Directory of Open Access Journals (Sweden)

Maryam Gharehdaghi

2015-05-01

Full Text Available In this paper we introduce a new fuzzy Monte Carlo method for solving system of linear algebraic equations (SLAE over the possibility theory and max-min algebra. To solve the SLAE, we first define a fuzzy estimator and prove that this is an unbiased estimator of the solution. To prove unbiasedness, we apply the ergodic fuzzy Markov chains. This new approach works even for cases with coefficients matrix with a norm greater than one.

16. A Markov Chain Model for evaluating the effectiveness of randomized surveillance procedures

Energy Technology Data Exchange (ETDEWEB)

Edmunds, T.A.

1994-01-01

A Markov Chain Model has been developed to evaluate the effectiveness of randomized surveillance procedures. The model is applicable for surveillance systems that monitor a collection of assets by randomly selecting and inspecting the assets. The model provides an estimate of the detection probability as a function of the amount of time that an adversary would require to steal or sabotage the asset. An interactive computer code has been written to perform the necessary computations.

17. System reliability assessment via sensitivity analysis in the Markov chain scheme

International Nuclear Information System (INIS)

Gandini, A.

1988-01-01

Methods for reliability sensitivity analysis in the Markov chain scheme are presented, together with a new formulation which makes use of Generalized Perturbation Theory (GPT) methods. As well known, sensitivity methods are fundamental in system risk analysis, since they allow to identify important components, so to assist the analyst in finding weaknesses in design and operation and in suggesting optimal modifications for system upgrade. The relationship between the GPT sensitivity expression and the Birnbaum importance is also given [fr

18. Stationary population flow of a semi-open Markov Chain. | Yakasai ...

African Journals Online (AJOL)

In this paper we study the state vector of a semi-open Markov chain of a stochastic population flow. We consider stationary inflow of new members into the system and derive the limiting value of the state vector X(n) = (X1(n),....,(Xm(n) ∈Z+ as n→ ∞ , when the system's capacity, is known. Journal of the Nigerian Association ...

19. Recursive estimation of high-order Markov chains: Approximation by finite mixtures

Czech Academy of Sciences Publication Activity Database

Kárný, Miroslav

2016-01-01

Roč. 326, č. 1 (2016), s. 188-201 ISSN 0020-0255 R&D Projects: GA ČR GA13-13502S Institutional support: RVO:67985556 Keywords : Markov chain * Approximate parameter estimation * Bayesian recursive estimation * Adaptive systems * Kullback–Leibler divergence * Forgetting Subject RIV: BC - Control Systems Theory Impact factor: 4.832, year: 2016 http://library.utia.cas.cz/separaty/2015/AS/karny-0447119.pdf

20. Fuzzy hidden Markov chains segmentation for volume determination and quantitation in PET

Science.gov (United States)

Hatt, Mathieu; Lamare, Frédéric; Boussion, Nicolas; Roux, Christian; Turzo, Alexandre; Cheze-Lerest, Catherine; Jarritt, Peter; Carson, Kathryn; Salzenstein, Fabien; Collet, Christophe; Visvikis, Dimitris

2007-01-01

Accurate volume of interest (VOI) estimation in PET is crucial in different oncology applications such as response to therapy evaluation and radiotherapy treatment planning. The objective of our study was to evaluate the performance of the proposed algorithm for automatic lesion volume delineation; namely the Fuzzy Hidden Markov Chains (FHMC), with that of current state of the art in clinical practice threshold based techniques. As the classical Hidden Markov Chain (HMC) algorithm, FHMC takes into account noise, voxel’s intensity and spatial correlation, in order to classify a voxel as background or functional VOI. However the novelty of the fuzzy model consists of the inclusion of an estimation of imprecision, which should subsequently lead to a better modelling of the “fuzzy” nature of the object on interest boundaries in emission tomography data. The performance of the algorithms has been assessed on both simulated and acquired datasets of the IEC phantom, covering a large range of spherical lesion sizes (from 10 to 37mm), contrast ratios (4:1 and 8:1) and image noise levels. Both lesion activity recovery and VOI determination tasks were assessed in reconstructed images using two different voxel sizes (8mm3 and 64mm3). In order to account for both the functional volume location and its size, the concept of % classification errors was introduced in the evaluation of volume segmentation using the simulated datasets. Results reveal that FHMC performs substantially better than the threshold based methodology for functional volume determination or activity concentration recovery considering a contrast ratio of 4:1 and lesion sizes of <28mm. Furthermore differences between classification and volume estimation errors evaluated were smaller for the segmented volumes provided by the FHMC algorithm. Finally, the performance of the automatic algorithms was less susceptible to image noise levels in comparison to the threshold based techniques. The analysis of both

1. A new Markov-chain-related statistical approach for modelling synthetic wind power time series

International Nuclear Information System (INIS)

Pesch, T; Hake, J F; Schröders, S; Allelein, H J

2015-01-01

The integration of rising shares of volatile wind power in the generation mix is a major challenge for the future energy system. To address the uncertainties involved in wind power generation, models analysing and simulating the stochastic nature of this energy source are becoming increasingly important. One statistical approach that has been frequently used in the literature is the Markov chain approach. Recently, the method was identified as being of limited use for generating wind time series with time steps shorter than 15–40 min as it is not capable of reproducing the autocorrelation characteristics accurately. This paper presents a new Markov-chain-related statistical approach that is capable of solving this problem by introducing a variable second lag. Furthermore, additional features are presented that allow for the further adjustment of the generated synthetic time series. The influences of the model parameter settings are examined by meaningful parameter variations. The suitability of the approach is demonstrated by an application analysis with the example of the wind feed-in in Germany. It shows that—in contrast to conventional Markov chain approaches—the generated synthetic time series do not systematically underestimate the required storage capacity to balance wind power fluctuation. (paper)

2. Probabilistic approach of water residence time and connectivity using Markov chains with application to tidal embayments

Science.gov (United States)

Bacher, C.; Filgueira, R.; Guyondet, T.

2016-01-01

Markov chain analysis was recently proposed to assess the time scales and preferential pathways into biological or physical networks by computing residence time, first passage time, rates of transfer between nodes and number of passages in a node. We propose to adapt an algorithm already published for simple systems to physical systems described with a high resolution hydrodynamic model. The method is applied to bays and estuaries on the Eastern Coast of Canada for their interest in shellfish aquaculture. Current velocities have been computed by using a 2 dimensional grid of elements and circulation patterns were summarized by averaging Eulerian flows between adjacent elements. Flows and volumes allow computing probabilities of transition between elements and to assess the average time needed by virtual particles to move from one element to another, the rate of transfer between two elements, and the average residence time of each system. We also combined transfer rates and times to assess the main pathways of virtual particles released in farmed areas and the potential influence of farmed areas on other areas. We suggest that Markov chain is complementary to other sets of ecological indicators proposed to analyse the interactions between farmed areas - e.g., depletion index, carrying capacity assessment. Markov chain has several advantages with respect to the estimation of connectivity between pair of sites. It makes possible to estimate transfer rates and times at once in a very quick and efficient way, without the need to perform long term simulations of particle or tracer concentration.

3. State space orderings for Gauss-Seidel in Markov chains revisited

Energy Technology Data Exchange (ETDEWEB)

Dayar, T. [Bilkent Univ., Ankara (Turkey)

1996-12-31

Symmetric state space orderings of a Markov chain may be used to reduce the magnitude of the subdominant eigenvalue of the (Gauss-Seidel) iteration matrix. Orderings that maximize the elemental mass or the number of nonzero elements in the dominant term of the Gauss-Seidel splitting (that is, the term approximating the coefficient matrix) do not necessarily converge faster. An ordering of a Markov chain that satisfies Property-R is semi-convergent. On the other hand, there are semi-convergent symmetric state space orderings that do not satisfy Property-R. For a given ordering, a simple approach for checking Property-R is shown. An algorithm that orders the states of a Markov chain so as to increase the likelihood of satisfying Property-R is presented. The computational complexity of the ordering algorithm is less than that of a single Gauss-Seidel iteration (for sparse matrices). In doing all this, the aim is to gain an insight for faster converging orderings. Results from a variety of applications improve the confidence in the algorithm.

4. Analysis of drought areas in northern Algeria using Markov chains

Introduction. Global warming and climate change refer to an in- crease in average global temperatures, which causes a change in the amount and pattern of precipita- tion and consequently facilitates the installation of the drought. The increase in the GHGs (greenhouse gases) is an underlying cause of the global warming.

5. Markov chain-based mass estimation method for loose part monitoring system and its performance

Directory of Open Access Journals (Sweden)

Sung-Hwan Shin

2017-10-01

Full Text Available A loose part monitoring system is used to identify unexpected loose parts in a nuclear reactor vessel or steam generator. It is still necessary for the mass estimation of loose parts, one function of a loose part monitoring system, to develop a new method due to the high estimation error of conventional methods such as Hertz's impact theory and the frequency ratio method. The purpose of this study is to propose a mass estimation method using a Markov decision process and compare its performance with a method using an artificial neural network model proposed in a previous study. First, how to extract feature vectors using discrete cosine transform was explained. Second, Markov chains were designed with codebooks obtained from the feature vector. A 1/8-scaled mockup of the reactor vessel for OPR1000 was employed, and all used signals were obtained by impacting its surface with several solid spherical masses. Next, the performance of mass estimation by the proposed Markov model was compared with that of the artificial neural network model. Finally, it was investigated that the proposed Markov model had matching error below 20% in mass estimation. That was a similar performance to the method using an artificial neural network model and considerably improved in comparison with the conventional methods.

6. Phase Transitions for Quantum XY-Model on the Cayley Tree of Order Three in Quantum Markov Chain Scheme

International Nuclear Information System (INIS)

Mukhamedov, Farrukh; Saburov, Mansoor

2010-06-01

In the present paper we study forward Quantum Markov Chains (QMC) defined on a Cayley tree. Using the tree structure of graphs, we give a construction of quantum Markov chains on a Cayley tree. By means of such constructions we prove the existence of a phase transition for the XY-model on a Cayley tree of order three in QMC scheme. By the phase transition we mean the existence of two distinct QMC for the given family of interaction operators {K }. (author)

7. Markov chains or the game of structure and chance. From complex networks, to language evolution, to musical compositions

Science.gov (United States)

Blanchard, Ph.; Dawin, J. R.; Volchenkov, D.

2010-06-01

Markov chains provide us with a powerful tool for studying the structure of graphs and databases in details. We review the method of generalized inverses for Markov chains and apply it for the analysis of urban structures, evolution of languages, and musical compositions. We also discuss a generalization of Lévy flights over large complex networks and study the interplay between the nonlinearity of diffusion process and the topological structure of the network.

8. Inverse Modeling Using Markov Chain Monte Carlo Aided by Adaptive Stochastic Collocation Method with Transformation

Science.gov (United States)

Zhang, D.; Liao, Q.

2016-12-01

The Bayesian inference provides a convenient framework to solve statistical inverse problems. In this method, the parameters to be identified are treated as random variables. The prior knowledge, the system nonlinearity, and the measurement errors can be directly incorporated in the posterior probability density function (PDF) of the parameters. The Markov chain Monte Carlo (MCMC) method is a powerful tool to generate samples from the posterior PDF. However, since the MCMC usually requires thousands or even millions of forward simulations, it can be a computationally intensive endeavor, particularly when faced with large-scale flow and transport models. To address this issue, we construct a surrogate system for the model responses in the form of polynomials by the stochastic collocation method. In addition, we employ interpolation based on the nested sparse grids and takes into account the different importance of the parameters, under the condition of high random dimensions in the stochastic space. Furthermore, in case of low regularity such as discontinuous or unsmooth relation between the input parameters and the output responses, we introduce an additional transform process to improve the accuracy of the surrogate model. Once we build the surrogate system, we may evaluate the likelihood with very little computational cost. We analyzed the convergence rate of the forward solution and the surrogate posterior by Kullback-Leibler divergence, which quantifies the difference between probability distributions. The fast convergence of the forward solution implies fast convergence of the surrogate posterior to the true posterior. We also tested the proposed algorithm on water-flooding two-phase flow reservoir examples. The posterior PDF calculated from a very long chain with direct forward simulation is assumed to be accurate. The posterior PDF calculated using the surrogate model is in reasonable agreement with the reference, revealing a great improvement in terms of

9. Markov processes

CERN Document Server

Kirkwood, James R

2015-01-01

Review of ProbabilityShort HistoryReview of Basic Probability DefinitionsSome Common Probability DistributionsProperties of a Probability DistributionProperties of the Expected ValueExpected Value of a Random Variable with Common DistributionsGenerating FunctionsMoment Generating FunctionsExercisesDiscrete-Time, Finite-State Markov ChainsIntroductionNotationTransition MatricesDirected Graphs: Examples of Markov ChainsRandom Walk with Reflecting BoundariesGamblerâ€™s RuinEhrenfest ModelCentral Problem of Markov ChainsCondition to Ensure a Unique Equilibrium StateFinding the Equilibrium StateTransient and Recurrent StatesIndicator FunctionsPerron-Frobenius TheoremAbsorbing Markov ChainsMean First Passage TimeMean Recurrence Time and the Equilibrium StateFundamental Matrix for Regular Markov ChainsDividing a Markov Chain into Equivalence ClassesPeriodic Markov ChainsReducible Markov ChainsSummaryExercisesDiscrete-Time, Infinite-State Markov ChainsRenewal ProcessesDelayed Renewal ProcessesEquilibrium State f...

10. Lie Markov models.

Science.gov (United States)

Sumner, J G; Fernández-Sánchez, J; Jarvis, P D

2012-04-07

Recent work has discussed the importance of multiplicative closure for the Markov models used in phylogenetics. For continuous-time Markov chains, a sufficient condition for multiplicative closure of a model class is ensured by demanding that the set of rate-matrices belonging to the model class form a Lie algebra. It is the case that some well-known Markov models do form Lie algebras and we refer to such models as "Lie Markov models". However it is also the case that some other well-known Markov models unequivocally do not form Lie algebras (GTR being the most conspicuous example). In this paper, we will discuss how to generate Lie Markov models by demanding that the models have certain symmetries under nucleotide permutations. We show that the Lie Markov models include, and hence provide a unifying concept for, "group-based" and "equivariant" models. For each of two and four character states, the full list of Lie Markov models with maximal symmetry is presented and shown to include interesting examples that are neither group-based nor equivariant. We also argue that our scheme is pleasing in the context of applied phylogenetics, as, for a given symmetry of nucleotide substitution, it provides a natural hierarchy of models with increasing number of parameters. We also note that our methods are applicable to any application of continuous-time Markov chains beyond the initial motivations we take from phylogenetics. Crown Copyright Â© 2011. Published by Elsevier Ltd. All rights reserved.

11. Analysis on the Spatial-Temporal Dynamics of Financial Agglomeration with Markov Chain Approach in China

Directory of Open Access Journals (Sweden)

Weimin Chen

2014-01-01

Full Text Available The standard approach to studying financial industrial agglomeration is to construct measures of the degree of agglomeration within financial industry. But such measures often fail to exploit the convergence or divergence of financial agglomeration. In this paper, we apply Markov chain approach to diagnose the convergence of financial agglomeration in China based on the location quotient coefficients across the provincial regions over 1993–2011. The estimation of Markov transition probability matrix offers more detailed insights into the mechanics of financial agglomeration evolution process in China during the research period. The results show that the spatial evolution of financial agglomeration changes faster in the period of 2003–2011 than that in the period of 1993–2002. Furthermore, there exists a very uneven financial development patterns, but there is regional convergence for financial agglomeration in China.

12. Limit theorems for Markov chains and stochastic properties of dynamical systems by quasi-compactness

CERN Document Server

Hervé, Loïc

2001-01-01

This book shows how techniques from the perturbation theory of operators, applied to a quasi-compact positive kernel, may be used to obtain limit theorems for Markov chains or to describe stochastic properties of dynamical systems. A general framework for this method is given and then applied to treat several specific cases. An essential element of this work is the description of the peripheral spectra of a quasi-compact Markov kernel and of its Fourier-Laplace perturbations. This is first done in the ergodic but non-mixing case. This work is extended by the second author to the non-ergodic case. The only prerequisites for this book are a knowledge of the basic techniques of probability theory and of notions of elementary functional analysis.

13. Reliability measures for indexed semi-Markov chains applied to wind energy production

International Nuclear Information System (INIS)

D'Amico, Guglielmo; Petroni, Filippo; Prattico, Flavio

2015-01-01

The computation of the dependability measures is a crucial point in many engineering problems as well as in the planning and development of a wind farm. In this paper we address the issue of energy production by wind turbines by using an indexed semi-Markov chain as a model of wind speed. We present the mathematical model, the data and technical characteristics of a commercial wind turbine (Aircon HAWT-10kW). We show how to compute some of the main dependability measures such as reliability, availability and maintainability functions. We compare the results of the model with real energy production obtained from data available in the Lastem station (Italy) and sampled every 10 min. - Highlights: • Semi-Markov models. • Time series generation of wind speed. • Computation of availability, reliability and maintainability.

14. Markov chain model helps predict pitting corrosion depth and rate in underground pipelines

Energy Technology Data Exchange (ETDEWEB)

Caleyo, F.; Velazquez, J.C.; Hallen, J. M. [ESIQIE, Instituto Politecnico Nacional, Mexico D. F. (Mexico); Esquivel-Amezcua, A. [PEMEX PEP Region Sur, Villahermosa, Tabasco (Mexico); Valor, A. [Universidad de la Habana, Vedado, La Habana (Cuba)

2010-07-01

Recent reports place pipeline corrosion costs in North America at seven billion dollars per year. Pitting corrosion causes the higher percentage of failures among other corrosion mechanisms. This has motivated multiple modelling studies to be focused on corrosion pitting of underground pipelines. In this study, a continuous-time, non-homogenous pure birth Markov chain serves to model external pitting corrosion in buried pipelines. The analytical solution of Kolmogorov's forward equations for this type of Markov process gives the transition probability function in a discrete space of pit depths. The transition probability function can be completely identified by making a correlation between the stochastic pit depth mean and the deterministic mean obtained experimentally. The model proposed in this study can be applied to pitting corrosion data from repeated in-line pipeline inspections. Case studies presented in this work show how pipeline inspection and maintenance planning can be improved by using the proposed Markovian model for pitting corrosion.

15. Derivation of a Markov state model of the dynamics of a protein-like chain immersed in an implicit solvent

International Nuclear Information System (INIS)

Schofield, Jeremy; Bayat, Hanif

2014-01-01

A Markov state model of the dynamics of a protein-like chain immersed in an implicit hard sphere solvent is derived from first principles for a system of monomers that interact via discontinuous potentials designed to account for local structure and bonding in a coarse-grained sense. The model is based on the assumption that the implicit solvent interacts on a fast time scale with the monomers of the chain compared to the time scale for structural rearrangements of the chain and provides sufficient friction so that the motion of monomers is governed by the Smoluchowski equation. A microscopic theory for the dynamics of the system is developed that reduces to a Markovian model of the kinetics under well-defined conditions. Microscopic expressions for the rate constants that appear in the Markov state model are analyzed and expressed in terms of a temperature-dependent linear combination of escape rates that themselves are independent of temperature. Excellent agreement is demonstrated between the theoretical predictions of the escape rates and those obtained through simulation of a stochastic model of the dynamics of bond formation. Finally, the Markov model is studied by analyzing the eigenvalues and eigenvectors of the matrix of transition rates, and the equilibration process for a simple helix-forming system from an ensemble of initially extended configurations to mainly folded configurations is investigated as a function of temperature for a number of different chain lengths. For short chains, the relaxation is primarily single-exponential and becomes independent of temperature in the low-temperature regime. The profile is more complicated for longer chains, where multi-exponential relaxation behavior is seen at intermediate temperatures followed by a low temperature regime in which the folding becomes rapid and single exponential. It is demonstrated that the behavior of the equilibration profile as the temperature is lowered can be understood in terms of the

16. Markov chain modeling of daily rainfall in Lay Gaint Woreda, South Gonder Zone, Ethiopia

Directory of Open Access Journals (Sweden)

Birhan Getachew

2018-01-01

Full Text Available Information on seasonal Kiremet and seasonal Belg rainfall amount is important in the rain fed agriculture of Ethiopia since more than 85% of the population is dependent on agriculture particularly on rain fed farming practices. The distribution pattern of rainfall rather than the total amount of rainfall within the entire period of time is more important for studying the pattern of rainfall occurrence. A two-state Markov chain was used to describe the characteristics of rainfall occurrences in this woreda. The states, as considered were; dry (d and rainy (r. The overall chance of rain and the fitted curve tells us that the chance of getting rain in the main rainy season is about twice as compared to the small rainy season. The first order Markov chain model indicates that the probability of getting rain in the small rainy season is significantly dependent on whether the earlier date was dry or wet. While the second order Marko chain indicates that the main rainy season the dependence of the probability of rain on the previous two dates’ conditions is less as compared with the small rainy season. Rainfall amounts are very variable and are usually modeled by a gamma distribution. Therefore, the pattern of rainfall is somewhat unimodial having only one extreme value in August.  Onset, cessation and length of growing season of rainfall for the main rainy season show medium variation compared to the small rainy season.

17. Floral Developmental Pattern Changes on Neoregelia‘Flandria’(BROMELIACEAE Analysed by Markov Chain methodologyADRIANA

Directory of Open Access Journals (Sweden)

2003-01-01

Full Text Available Neoregelia 'Flandria'and N. 'Van Durme'are ornamental cultivars of Bromelia.Propagation by seeds is not viable and prunes constitutes the only way to propagateavoiding alterations. In this article the developmental floral pattern of 72 Bromeliastreated with ANA 190 ppm (T1, Ethrel: ANA + ETHREL,(T2 y ETHREL, (T3 anddivided into two age groups: E1 y E2 are shown. The treatments studied generated moreelongated plants and six new patterns. Using the Markov chain methodology theprobability to evolve to any pattern and the percentage of each were studied.

18. Metis: A Pure Metropolis Markov Chain Monte Carlo Bayesian Inference Library

Energy Technology Data Exchange (ETDEWEB)

Bates, Cameron Russell [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Mckigney, Edward Allen [Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

2018-01-09

The use of Bayesian inference in data analysis has become the standard for large scienti c experiments [1, 2]. The Monte Carlo Codes Group(XCP-3) at Los Alamos has developed a simple set of algorithms currently implemented in C++ and Python to easily perform at-prior Markov Chain Monte Carlo Bayesian inference with pure Metropolis sampling. These implementations are designed to be user friendly and extensible for customization based on speci c application requirements. This document describes the algorithmic choices made and presents two use cases.

19. Analysis of aerial survey data on Florida manatee using Markov chain Monte Carlo.

Science.gov (United States)

Craig, B A; Newton, M A; Garrott, R A; Reynolds, J E; Wilcox, J R

1997-06-01

We assess population trends of the Atlantic coast population of Florida manatee, Trichechus manatus latirostris, by reanalyzing aerial survey data collected between 1982 and 1992. To do so, we develop an explicit biological model that accounts for the method by which the manatees are counted, the mammals' movement between surveys, and the behavior of the population total over time. Bayesian inference, enabled by Markov chain Monte Carlo, is used to combine the survey data with the biological model. We compute marginal posterior distributions for all model parameters and predictive distributions for future counts. Several conclusions, such as a decreasing population growth rate and low sighting probabilities, are consistent across different prior specifications.

20. A brief history of the introduction of generalized ensembles to Markov chain Monte Carlo simulations

Science.gov (United States)

Berg, Bernd A.

2017-03-01

The most efficient weights for Markov chain Monte Carlo calculations of physical observables are not necessarily those of the canonical ensemble. Generalized ensembles, which do not exist in nature but can be simulated on computers, lead often to a much faster convergence. In particular, they have been used for simulations of first order phase transitions and for simulations of complex systems in which conflicting constraints lead to a rugged free energy landscape. Starting off with the Metropolis algorithm and Hastings' extension, I present a minireview which focuses on the explosive use of generalized ensembles in the early 1990s. Illustrations are given, which range from spin models to peptides.

1. Markov chain Monte Carlo methods for state-space models with point process observations.

Science.gov (United States)

Yuan, Ke; Girolami, Mark; Niranjan, Mahesan

2012-06-01

This letter considers how a number of modern Markov chain Monte Carlo (MCMC) methods can be applied for parameter estimation and inference in state-space models with point process observations. We quantified the efficiencies of these MCMC methods on synthetic data, and our results suggest that the Reimannian manifold Hamiltonian Monte Carlo method offers the best performance. We further compared such a method with a previously tested variational Bayes method on two experimental data sets. Results indicate similar performance on the large data sets and superior performance on small ones. The work offers an extensive suite of MCMC algorithms evaluated on an important class of models for physiological signal analysis.

2. A comparison of strategies for Markov chain Monte Carlo computation in quantitative genetics

DEFF Research Database (Denmark)

Waagepetersen, Rasmus; Ibanez-Escriche, Noelia; Sorensen, Daniel

2008-01-01

In quantitative genetics, Markov chain Monte Carlo (MCMC) methods are indispensable for statistical inference in non-standard models like generalized linear models with genetic random effects or models with genetically structured variance heterogeneity. A particular challenge for MCMC applications...... in quantitative genetics is to obtain efficient updates of the high-dimensional vectors of genetic random effects and the associated covariance parameters. We discuss various strategies to approach this problem including reparameterization, Langevin-Hastings updates, and updates based on normal approximations....... The methods are compared in applications to Bayesian inference for three data sets using a model with genetically structured variance heterogeneity...

3. An Auxiliary Variable Method for Markov Chain Monte Carlo Algorithms in High Dimension

Directory of Open Access Journals (Sweden)

Yosra Marnissi

2018-02-01

Full Text Available In this paper, we are interested in Bayesian inverse problems where either the data fidelity term or the prior distribution is Gaussian or driven from a hierarchical Gaussian model. Generally, Markov chain Monte Carlo (MCMC algorithms allow us to generate sets of samples that are employed to infer some relevant parameters of the underlying distributions. However, when the parameter space is high-dimensional, the performance of stochastic sampling algorithms is very sensitive to existing dependencies between parameters. In particular, this problem arises when one aims to sample from a high-dimensional Gaussian distribution whose covariance matrix does not present a simple structure. Another challenge is the design of Metropolis–Hastings proposals that make use of information about the local geometry of the target density in order to speed up the convergence and improve mixing properties in the parameter space, while not being too computationally expensive. These two contexts are mainly related to the presence of two heterogeneous sources of dependencies stemming either from the prior or the likelihood in the sense that the related covariance matrices cannot be diagonalized in the same basis. In this work, we address these two issues. Our contribution consists of adding auxiliary variables to the model in order to dissociate the two sources of dependencies. In the new augmented space, only one source of correlation remains directly related to the target parameters, the other sources of correlations being captured by the auxiliary variables. Experiments are conducted on two practical image restoration problems—namely the recovery of multichannel blurred images embedded in Gaussian noise and the recovery of signal corrupted by a mixed Gaussian noise. Experimental results indicate that adding the proposed auxiliary variables makes the sampling problem simpler since the new conditional distribution no longer contains highly heterogeneous

4. Discovering beaten paths in collaborative ontology-engineering projects using Markov chains.

Science.gov (United States)

Walk, Simon; Singer, Philipp; Strohmaier, Markus; Tudorache, Tania; Musen, Mark A; Noy, Natalya F

2014-10-01

Biomedical taxonomies, thesauri and ontologies in the form of the International Classification of Diseases as a taxonomy or the National Cancer Institute Thesaurus as an OWL-based ontology, play a critical role in acquiring, representing and processing information about human health. With increasing adoption and relevance, biomedical ontologies have also significantly increased in size. For example, the 11th revision of the International Classification of Diseases, which is currently under active development by the World Health Organization contains nearly 50,000 classes representing a vast variety of different diseases and causes of death. This evolution in terms of size was accompanied by an evolution in the way ontologies are engineered. Because no single individual has the expertise to develop such large-scale ontologies, ontology-engineering projects have evolved from small-scale efforts involving just a few domain experts to large-scale projects that require effective collaboration between dozens or even hundreds of experts, practitioners and other stakeholders. Understanding the way these different stakeholders collaborate will enable us to improve editing environments that support such collaborations. In this paper, we uncover how large ontology-engineering projects, such as the International Classification of Diseases in its 11th revision, unfold by analyzing usage logs of five different biomedical ontology-engineering projects of varying sizes and scopes using Markov chains. We discover intriguing interaction patterns (e.g., which properties users frequently change after specific given ones) that suggest that large collaborative ontology-engineering projects are governed by a few general principles that determine and drive development. From our analysis, we identify commonalities and differences between different projects that have implications for project managers, ontology editors, developers and contributors working on collaborative ontology

5. Single-Server Queueing System with Markov-Modulated Arrivals and Service Times

OpenAIRE

Dimitrov, Mitko

2011-01-01

Key words: Markov-modulated queues, waiting time, heavy traffic. Markov-modulated queueing systems are those in which the input process or service mechanism is influenced by an underlying Markov chain. Several models for such systems have been investigated. In this paper we present heavy traffic analysis of single queueing system with Poisson arrival process whose arrival rate is a function of the state of Markov chain and service times depend on the state of the same Markov chain at the e...

6. Optimal Linear Responses for Markov Chains and Stochastically Perturbed Dynamical Systems

Science.gov (United States)

Antown, Fadi; Dragičević, Davor; Froyland, Gary

2018-03-01

The linear response of a dynamical system refers to changes to properties of the system when small external perturbations are applied. We consider the little-studied question of selecting an optimal perturbation so as to (i) maximise the linear response of the equilibrium distribution of the system, (ii) maximise the linear response of the expectation of a specified observable, and (iii) maximise the linear response of the rate of convergence of the system to the equilibrium distribution. We also consider the inhomogeneous, sequential, or time-dependent situation where the governing dynamics is not stationary and one wishes to select a sequence of small perturbations so as to maximise the overall linear response at some terminal time. We develop the theory for finite-state Markov chains, provide explicit solutions for some illustrative examples, and numerically apply our theory to stochastically perturbed dynamical systems, where the Markov chain is replaced by a matrix representation of an approximate annealed transfer operator for the random dynamical system.

7. Fracture overprinting history using Markov chain analysis: Windsor-Kennetcook subbasin, Maritimes Basin, Canada

Science.gov (United States)

Snyder, Morgan E.; Waldron, John W. F.

2018-03-01

The deformation history of the Upper Paleozoic Maritimes Basin, Atlantic Canada, can be partially unraveled by examining fractures (joints, veins, and faults) that are well exposed on the shorelines of the macrotidal Bay of Fundy, in subsurface core, and on image logs. Data were collected from coastal outcrops and well core across the Windsor-Kennetcook subbasin, a subbasin in the Maritimes Basin, using the circular scan-line and vertical scan-line methods in outcrop, and FMI Image log analysis of core. We use cross-cutting and abutting relationships between fractures to understand relative timing of fracturing, followed by a statistical test (Markov chain analysis) to separate groups of fractures. This analysis, previously used in sedimentology, was modified to statistically test the randomness of fracture timing relationships. The results of the Markov chain analysis suggest that fracture initiation can be attributed to movement along the Minas Fault Zone, an E-W fault system that bounds the Windsor-Kennetcook subbasin to the north. Four sets of fractures are related to dextral strike slip along the Minas Fault Zone in the late Paleozoic, and four sets are related to sinistral reactivation of the same boundary in the Mesozoic.

8. A joint logistic regression and covariate-adjusted continuous-time Markov chain model.

Science.gov (United States)

Rubin, Maria Laura; Chan, Wenyaw; Yamal, Jose-Miguel; Robertson, Claudia Sue

2017-12-10

The use of longitudinal measurements to predict a categorical outcome is an increasingly common goal in research studies. Joint models are commonly used to describe two or more models simultaneously by considering the correlated nature of their outcomes and the random error present in the longitudinal measurements. However, there is limited research on joint models with longitudinal predictors and categorical cross-sectional outcomes. Perhaps the most challenging task is how to model the longitudinal predictor process such that it represents the true biological mechanism that dictates the association with the categorical response. We propose a joint logistic regression and Markov chain model to describe a binary cross-sectional response, where the unobserved transition rates of a two-state continuous-time Markov chain are included as covariates. We use the method of maximum likelihood to estimate the parameters of our model. In a simulation study, coverage probabilities of about 95%, standard deviations close to standard errors, and low biases for the parameter values show that our estimation method is adequate. We apply the proposed joint model to a dataset of patients with traumatic brain injury to describe and predict a 6-month outcome based on physiological data collected post-injury and admission characteristics. Our analysis indicates that the information provided by physiological changes over time may help improve prediction of long-term functional status of these severely ill subjects. Copyright © 2017 John Wiley & Sons, Ltd. Copyright © 2017 John Wiley & Sons, Ltd.

9. The use of Markov chains in forecasting wind speed: Matlab source code and applied case study

Directory of Open Access Journals (Sweden)

Ionuţ Alexandru Petre

2017-01-01

Full Text Available The ability to predict the wind speed has an important role for renewable energy industry which relies on wind speed forecasts in order to calculate the power a wind farm can produce in an area. There are several well-known methods to predict wind speed, but in this paper we focus on short-term wind forecasting using Markov chains. Often gaps can be found in the time series of the wind speed measurements and repeating the measurements is usually not a valid option. In this study it is shown that using Markov chains these gaps from the time series can be filled (they can be generated in an efficient way, but only when the missing data is for a short period of time. Also, the developed Matlab programms that are used in the case study, are included in the paper beeing presented and commented by the authors. In the case study data from a wind farm in Italy is used. The available data are as average wind speed at an interval of 10 minutes in the time period 11/23/2005 - 4/27/2006.

10. Predicting Urban Medical Services Demand in China: An Improved Grey Markov Chain Model by Taylor Approximation

Directory of Open Access Journals (Sweden)

Jinli Duan

2017-08-01

Full Text Available The sharp increase of the aging population has raised the pressure on the current limited medical resources in China. To better allocate resources, a more accurate prediction on medical service demand is very urgently needed. This study aims to improve the prediction on medical services demand in China. To achieve this aim, the study combines Taylor Approximation into the Grey Markov Chain model, and develops a new model named Taylor-Markov Chain GM (1,1 (T-MCGM (1,1. The new model has been tested by adopting the historical data, which includes the medical service on treatment of diabetes, heart disease, and cerebrovascular disease from 1997 to 2015 in China. The model provides a predication on medical service demand of these three types of disease up to 2022. The results reveal an enormous growth of urban medical service demand in the future. The findings provide practical implications for the Health Administrative Department to allocate medical resources, and help hospitals to manage investments on medical facilities.

11. [Analysis and modelling of safety culture in a Mexican hospital by Markov chains].

Science.gov (United States)

Velázquez-Martínez, J D; Cruz-Suárez, H; Santos-Reyes, J

2016-01-01

The objective of this study was to analyse and model the safety culture with Markov chains, as well as predicting and/or prioritizing over time the evolutionary behaviour of the safety culture of the health's staff in one Mexican hospital. The Markov chain theory has been employed in the analysis, and the input data has been obtained from a previous study based on the Safety Attitude Questionnaire (CAS-MX-II), by considering the following 6 dimensions: safety climate, teamwork, job satisfaction, recognition of stress, perception of management, and work environment. The results highlighted the predictions and/or prioritisation of the approximate time for the possible integration into the evolutionary behaviour of the safety culture as regards the "slightly agree" (Likert scale) for: safety climate (in 12 years; 24.13%); teamwork (8 years; 34.61%); job satisfaction (11 years; 52.41%); recognition of the level of stress (8 years; 19.35%); and perception of the direction (22 years; 27.87%). The work environment dimension was unable to determine the behaviour of staff information, i.e. no information cultural roots were obtained. In general, it has been shown that there are weaknesses in the safety culture of the hospital, which is an opportunity to suggest changes to the mandatory policies in order to strengthen it. Copyright © 2016 SECA. Publicado por Elsevier España, S.L.U. All rights reserved.

12. Atmospheric Dispersion Unknown Source Parameters Determination Using AERMOD and Bayesian Inference Along Markov Chain Monte Carlo

International Nuclear Information System (INIS)

Haghighattalab, A.; Zolfaghari, A. R.; Minouchehr, A. H.; Kiya, H. A.

2012-01-01

Occurrence of hazardous accident in nuclear power plants and industrial units usually lead to release of radioactive materials and pollutants in environment. These materials and pollutants can be transported to a far downstream by the wind flow. In this paper, we implemented an atmospheric dispersion code to solve the inverse problem. Having received and detected the pollutants in one region, we may estimate the rate and location of the unknown source. For the modeling, one needs a model with ability of atmospheric dispersion calculation. Furthermore, it is required to implement a mathematical approach to infer the source location and the related rates. In this paper the AERMOD software and Bayesian inference along the Markov Chain Monte Carlo have been applied. Implementing, Bayesian approach and Markov Chain Monte Carlo for the aforementioned subject is not a new approach, but the AERMOD model coupled with the said methods is a new and well known regulatory software, and enhances the reliability of outcomes. To evaluate the method, an example is considered by defining pollutants concentration in a specific region and then obtaining the source location and intensity by a direct calculation. The result of the calculation estimates the average source location at a distance of 7km with an accuracy of 5m which is good enough to support the ability of the proposed algorithm.

13. Short-term droughts forecast using Markov chain model in Victoria, Australia

Science.gov (United States)

Rahmat, Siti Nazahiyah; Jayasuriya, Niranjali; Bhuiyan, Muhammed A.

2017-07-01

A comprehensive risk management strategy for dealing with drought should include both short-term and long-term planning. The objective of this paper is to present an early warning method to forecast drought using the Standardised Precipitation Index (SPI) and a non-homogeneous Markov chain model. A model such as this is useful for short-term planning. The developed method has been used to forecast droughts at a number of meteorological monitoring stations that have been regionalised into six (6) homogenous clusters with similar drought characteristics based on SPI. The non-homogeneous Markov chain model was used to estimate drought probabilities and drought predictions up to 3 months ahead. The drought severity classes defined using the SPI were computed at a 12-month time scale. The drought probabilities and the predictions were computed for six clusters that depict similar drought characteristics in Victoria, Australia. Overall, the drought severity class predicted was quite similar for all the clusters, with the non-drought class probabilities ranging from 49 to 57 %. For all clusters, the near normal class had a probability of occurrence varying from 27 to 38 %. For the more moderate and severe classes, the probabilities ranged from 2 to 13 % and 3 to 1 %, respectively. The developed model predicted drought situations 1 month ahead reasonably well. However, 2 and 3 months ahead predictions should be used with caution until the models are developed further.

14. An efficient interpolation technique for jump proposals in reversible-jump Markov chain Monte Carlo calculations

Science.gov (United States)

Farr, W. M.; Mandel, I.; Stevens, D.

2015-06-01

Selection among alternative theoretical models given an observed dataset is an important challenge in many areas of physics and astronomy. Reversible-jump Markov chain Monte Carlo (RJMCMC) is an extremely powerful technique for performing Bayesian model selection, but it suffers from a fundamental difficulty and it requires jumps between model parameter spaces, but cannot efficiently explore both parameter spaces at once. Thus, a naive jump between parameter spaces is unlikely to be accepted in the Markov chain Monte Carlo (MCMC) algorithm and convergence is correspondingly slow. Here, we demonstrate an interpolation technique that uses samples from single-model MCMCs to propose intermodel jumps from an approximation to the single-model posterior of the target parameter space. The interpolation technique, based on a kD-tree data structure, is adaptive and efficient in modest dimensionality. We show that our technique leads to improved convergence over naive jumps in an RJMCMC, and compare it to other proposals in the literature to improve the convergence of RJMCMCs. We also demonstrate the use of the same interpolation technique as a way to construct efficient "global" proposal distributions for single-model MCMCs without prior knowledge of the structure of the posterior distribution, and discuss improvements that permit the method to be used in higher dimensional spaces efficiently.

15. Predicting Urban Medical Services Demand in China: An Improved Grey Markov Chain Model by Taylor Approximation.

Science.gov (United States)

Duan, Jinli; Jiao, Feng; Zhang, Qishan; Lin, Zhibin

2017-08-06

The sharp increase of the aging population has raised the pressure on the current limited medical resources in China. To better allocate resources, a more accurate prediction on medical service demand is very urgently needed. This study aims to improve the prediction on medical services demand in China. To achieve this aim, the study combines Taylor Approximation into the Grey Markov Chain model, and develops a new model named Taylor-Markov Chain GM (1,1) (T-MCGM (1,1)). The new model has been tested by adopting the historical data, which includes the medical service on treatment of diabetes, heart disease, and cerebrovascular disease from 1997 to 2015 in China. The model provides a predication on medical service demand of these three types of disease up to 2022. The results reveal an enormous growth of urban medical service demand in the future. The findings provide practical implications for the Health Administrative Department to allocate medical resources, and help hospitals to manage investments on medical facilities.

16. Studying stellar binary systems with the Laser Interferometer Space Antenna using delayed rejection Markov chain Monte Carlo methods

Energy Technology Data Exchange (ETDEWEB)

Trias, Miquel [Departament de Fisica, Universitat de les Illes Balears, Cra. Valldemossa Km. 7.5, E-07122 Palma de Mallorca (Spain); Vecchio, Alberto; Veitch, John, E-mail: miquel.trias@uib.e, E-mail: av@star.sr.bham.ac.u, E-mail: jveitch@star.sr.bham.ac.u [School of Physics and Astronomy, University of Birmingham, Edgbaston, Birmingham B15 2TT (United Kingdom)

2009-10-21

Bayesian analysis of Laser Interferometer Space Antenna (LISA) data sets based on Markov chain Monte Carlo methods has been shown to be a challenging problem, in part due to the complicated structure of the likelihood function consisting of several isolated local maxima that dramatically reduces the efficiency of the sampling techniques. Here we introduce a new fully Markovian algorithm, a delayed rejection Metropolis-Hastings Markov chain Monte Carlo method, to efficiently explore these kind of structures and we demonstrate its performance on selected LISA data sets containing a known number of stellar-mass binary signals embedded in Gaussian stationary noise.

17. Adjoint sensitivity analysis procedure of Markov chains with applications on reliability of IFMIF accelerator-system facilities

International Nuclear Information System (INIS)

Balan, I.

2005-05-01

This work presents the implementation of the Adjoint Sensitivity Analysis Procedure (ASAP) for the Continuous Time, Discrete Space Markov chains (CTMC), as an alternative to the other computational expensive methods. In order to develop this procedure as an end product in reliability studies, the reliability of the physical systems is analyzed using a coupled Fault-Tree - Markov chain technique, i.e. the abstraction of the physical system is performed using as the high level interface the Fault-Tree and afterwards this one is automatically converted into a Markov chain. The resulting differential equations based on the Markov chain model are solved in order to evaluate the system reliability. Further sensitivity analyses using ASAP applied to CTMC equations are performed to study the influence of uncertainties in input data to the reliability measures and to get the confidence in the final reliability results. The methods to generate the Markov chain and the ASAP for the Markov chain equations have been implemented into the new computer code system QUEFT/MARKOMAGS/MCADJSEN for reliability and sensitivity analysis of physical systems. The validation of this code system has been carried out by using simple problems for which analytical solutions can be obtained. Typical sensitivity results show that the numerical solution using ASAP is robust, stable and accurate. The method and the code system developed during this work can be used further as an efficient and flexible tool to evaluate the sensitivities of reliability measures for any physical system analyzed using the Markov chain. Reliability and sensitivity analyses using these methods have been performed during this work for the IFMIF Accelerator System Facilities. The reliability studies using Markov chain have been concentrated around the availability of the main subsystems of this complex physical system for a typical mission time. The sensitivity studies for two typical responses using ASAP have been

18. Markov Chain-Based Stochastic Modeling of Chloride Ion Transport in Concrete Bridges

Directory of Open Access Journals (Sweden)

Yan Zhang

2018-03-01

Full Text Available Over the last decade, there has been an increasing interest in models for the evaluation and prediction of the condition of bridges in Canada due to their large number in an advanced state of deterioration. The models are used to develop optimal maintenance and replacement strategies to extend service life and optimally allocate financial and technical resources. The main process of deterioration of concrete bridges in Canada is corrosion of the reinforcing steel due to the widespread use of de-icing salts. In this article, numerical models of the diffusion process and chemical reactions of chloride ions in concrete are used to estimate the time to initiation of corrosion and for the progression of corrosion. The analyses are performed for a range of typical concrete properties, exposure and climatic conditions. The results from these simulations are used to develop parametric surrogate Markov chain models of increasing states of deterioration. The surrogate models are more efficient than physical models for the portfolio analysis of a large number of structures. The procedure provides an alternative to Markov models derived from condition ratings when historical inspection data is limited.

19. Using Markov Chains to predict the natural progression of diabetic retinopathy.

Science.gov (United States)

Srikanth, Priyanka

2015-01-01

To study the natural progression of diabetic retinopathy in patients with type 2 diabetes. This was an observational study of 153 cases with type 2 diabetes from 2010 to 2013. The state of patient was noted at end of each year and transition matrices were developed to model movement between years. Patients who progressed to severe non-proliferative diabetic retinopathy (NPDR) were treated. Markov Chains and Chi-square test were used for statistical analysis. We modelled the transition of 153 patients from NPDR to blindness on an annual basis. At the end of year 3, we compared results from the Markov model versus actual data. The results from Chi-square test confirmed that there was statistically no significant difference (P=0.70) which provided assurance that the model was robust to estimate mean sojourn times. The key finding was that a patient entering the system in mild NPDR state is expected to stay in that state for 5y followed by 1.07y in moderate NPDR, be in the severe NPDR state for 1.33y before moving into PDR for roughly 8y. It is therefore expected that such a patient entering the model in a state of mild NPDR will enter blindness after 15.29y. Patients stay for long time periods in mild NPDR before transitioning into moderate NPDR. However, they move rapidly from moderate NPDR to proliferative diabetic retinopathy (PDR) and stay in that state for long periods before transitioning into blindness.

20. Modeling and Computing of Stock Index Forecasting Based on Neural Network and Markov Chain

Science.gov (United States)

Dai, Yonghui; Han, Dongmei; Dai, Weihui

2014-01-01

The stock index reflects the fluctuation of the stock market. For a long time, there have been a lot of researches on the forecast of stock index. However, the traditional method is limited to achieving an ideal precision in the dynamic market due to the influences of many factors such as the economic situation, policy changes, and emergency events. Therefore, the approach based on adaptive modeling and conditional probability transfer causes the new attention of researchers. This paper presents a new forecast method by the combination of improved back-propagation (BP) neural network and Markov chain, as well as its modeling and computing technology. This method includes initial forecasting by improved BP neural network, division of Markov state region, computing of the state transition probability matrix, and the prediction adjustment. Results of the empirical study show that this method can achieve high accuracy in the stock index prediction, and it could provide a good reference for the investment in stock market. PMID:24782659

1. Markov Chain-Based Acute Effect Estimation of Air Pollution on Elder Asthma Hospitalization

Directory of Open Access Journals (Sweden)

Li Luo

2017-01-01

Full Text Available Background. Asthma caused substantial economic and health care burden and is susceptible to air pollution. Particularly, when it comes to elder asthma patient (older than 65, the phenomenon is more significant. The aim of this study is to investigate the Markov-based acute effects of air pollution on elder asthma hospitalizations, in forms of transition probabilities. Methods. A retrospective, population-based study design was used to assess temporal patterns in hospitalizations for asthma in a region of Sichuan province, China. Approximately 12 million residents were covered during this period. Relative risk analysis and Markov chain model were employed on daily hospitalization state estimation. Results. Among PM2.5, PM10, NO2, and SO2, only SO2 was significant. When air pollution is severe, the transition probability from a low-admission state (previous day to high-admission state (next day is 35.46%, while it is 20.08% when air pollution is mild. In particular, for female-cold subgroup, the counterparts are 30.06% and 0.01%, respectively. Conclusions. SO2 was a significant risk factor for elder asthma hospitalization. When air pollution worsened, the transition probabilities from each state to high admission states increase dramatically. This phenomenon appeared more evidently, especially in female-cold subgroup (which is in cold season for female admissions. Based on our work, admission amount forecast, asthma intervention, and corresponding healthcare allocation can be done.

2. Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems.

Science.gov (United States)

Ballnus, Benjamin; Hug, Sabine; Hatz, Kathrin; Görlitz, Linus; Hasenauer, Jan; Theis, Fabian J

2017-06-24

In quantitative biology, mathematical models are used to describe and analyze biological processes. The parameters of these models are usually unknown and need to be estimated from experimental data using statistical methods. In particular, Markov chain Monte Carlo (MCMC) methods have become increasingly popular as they allow for a rigorous analysis of parameter and prediction uncertainties without the need for assuming parameter identifiability or removing non-identifiable parameters. A broad spectrum of MCMC algorithms have been proposed, including single- and multi-chain approaches. However, selecting and tuning sampling algorithms suited for a given problem remains challenging and a comprehensive comparison of different methods is so far not available. We present the results of a thorough benchmarking of state-of-the-art single- and multi-chain sampling methods, including Adaptive Metropolis, Delayed Rejection Adaptive Metropolis, Metropolis adjusted Langevin algorithm, Parallel Tempering and Parallel Hierarchical Sampling. Different initialization and adaptation schemes are considered. To ensure a comprehensive and fair comparison, we consider problems with a range of features such as bifurcations, periodical orbits, multistability of steady-state solutions and chaotic regimes. These problem properties give rise to various posterior distributions including uni- and multi-modal distributions and non-normally distributed mode tails. For an objective comparison, we developed a pipeline for the semi-automatic comparison of sampling results. The comparison of MCMC algorithms, initialization and adaptation schemes revealed that overall multi-chain algorithms perform better than single-chain algorithms. In some cases this performance can be further increased by using a preceding multi-start local optimization scheme. These results can inform the selection of sampling methods and the benchmark collection can serve for the evaluation of new algorithms. Furthermore, our

3. First Passage Probability Estimation of Wind Turbines by Markov Chain Monte Carlo

DEFF Research Database (Denmark)

Sichani, Mahdi Teimouri; Nielsen, Søren R.K.

2013-01-01

Markov Chain Monte Carlo simulation has received considerable attention within the past decade as reportedly one of the most powerful techniques for the first passage probability estimation of dynamic systems. A very popular method in this direction capable of estimating probability of rare events...... of the method by modifying the conditional sampler. In this paper, applicability of the original SS is compared to the recently introduced modifications of the method on a wind turbine model. The model incorporates a PID pitch controller which aims at keeping the rotational speed of the wind turbine rotor equal...... to its nominal value. Finally Monte Carlo simulations are performed which allow assessment of the accuracy of the first passage probability estimation by the SS methods....

4. Data Model Approach And Markov Chain Based Analysis Of Multi-Level Queue Scheduling

Directory of Open Access Journals (Sweden)

Diwakar Shukla

2010-01-01

Full Text Available There are many CPU scheduling algorithms inliterature like FIFO, Round Robin, Shortest-Job-First and so on.The Multilevel-Queue-Scheduling is superior to these due to itsbetter management of a variety of processes. In this paper, aMarkov chain model is used for a general setup of Multilevelqueue-scheduling and the scheduler is assumed to performrandom movement on queue over the quantum of time.Performance of scheduling is examined through a rowdependent data model. It is found that with increasing value of αand d, the chance of system going over the waiting state reduces.At some of the interesting combinations of α and d, it diminishesto zero, thereby, provides us some clue regarding better choice ofqueues over others for high priority jobs. It is found that ifqueue priorities are added in the scheduling intelligently thenbetter performance could be obtained. Data model helpschoosing appropriate preferences.

5. Information Entropy Production of Maximum Entropy Markov Chains from Spike Trains

Directory of Open Access Journals (Sweden)

Rodrigo Cofré

2018-01-01

Full Text Available The spiking activity of neuronal networks follows laws that are not time-reversal symmetric; the notion of pre-synaptic and post-synaptic neurons, stimulus correlations and noise correlations have a clear time order. Therefore, a biologically realistic statistical model for the spiking activity should be able to capture some degree of time irreversibility. We use the thermodynamic formalism to build a framework in the context maximum entropy models to quantify the degree of time irreversibility, providing an explicit formula for the information entropy production of the inferred maximum entropy Markov chain. We provide examples to illustrate our results and discuss the importance of time irreversibility for modeling the spike train statistics.

6. Study of behavior and determination of customer lifetime value(CLV) using Markov chain model

Science.gov (United States)

Permana, Dony; Indratno, Sapto Wahyu; Pasaribu, Udjianna S.

2014-03-01

Customer Lifetime Value or CLV is a restriction on interactive marketing to help a company in arranging financial for the marketing of new customer acquisition and customer retention. Additionally CLV can be able to segment customers for financial arrangements. Stochastic models for the fairly new CLV used a Markov chain. In this model customer retention probability and new customer acquisition probability play an important role. This model is originally introduced by Pfeifer and Carraway in 2000 [1]. They introduced several CLV models, one of them only involves customer and former customer. In this paper we expand the model by adding the assumption of the transition from former customer to customer. In the proposed model, the CLV value is higher than the CLV value obtained by Pfeifer and Caraway model. But our model still requires a longer convergence time.

7. A methodology for stochastic analysis of share prices as Markov chains with finite states.

Science.gov (United States)

Mettle, Felix Okoe; Quaye, Enoch Nii Boi; Laryea, Ravenhill Adjetey

2014-01-01

Price volatilities make stock investments risky, leaving investors in critical position when uncertain decision is made. To improve investor evaluation confidence on exchange markets, while not using time series methodology, we specify equity price change as a stochastic process assumed to possess Markov dependency with respective state transition probabilities matrices following the identified state pace (i.e. decrease, stable or increase). We established that identified states communicate, and that the chains are aperiodic and ergodic thus possessing limiting distributions. We developed a methodology for determining expected mean return time for stock price increases and also establish criteria for improving investment decision based on highest transition probabilities, lowest mean return time and highest limiting distributions. We further developed an R algorithm for running the methodology introduced. The established methodology is applied to selected equities from Ghana Stock Exchange weekly trading data.

8. Markov Chain Monte Carlo Exploration of Minimal Supergravity with Implications for Dark Matter

International Nuclear Information System (INIS)

Baltz, E

2004-01-01

We explore the full parameter space of Minimal Supergravity (mSUGRA), allowing all four continuous parameters (the scalar mass m 0 , the gaugino mass m 1/2 , the trilinear coupling A 0 , and the ratio of Higgs vacuum expectation values tan β) to vary freely. We apply current accelerator constraints on sparticle and Higgs masses, and on the b → sγ branching ratio, and discuss the impact of the constraints on g μ -2. To study dark matter, we apply the WMAP constraint on the cold dark matter density. We develop Markov Chain Monte Carlo (MCMC) techniques to explore the parameter regions consistent with WMAP, finding them to be considerably superior to previously used methods for exploring supersymmetric parameter spaces. Finally, we study the reach of current and future direct detection experiments in light of the WMAP constraint

9. ProMT: effective human promoter prediction using Markov chain model based on DNA structural properties.

Science.gov (United States)

Xiong, Dapeng; Liu, Rongjie; Xiao, Fen; Gao, Xieping

2014-12-01

The core promoters play significant and extensive roles for the initiation and regulation of DNA transcription. The identification of core promoters is one of the most challenging problems yet. Due to the diverse nature of core promoters, the results obtained through existing computational approaches are not satisfactory. None of them considered the potential influence on performance of predictive approach resulted by the interference between neighboring TSSs in TSS clusters. In this paper, we sufficiently considered this main factor and proposed an approach to locate potential TSS clusters according to the correlation of regional profiles of DNA and TSS clusters. On this basis, we further presented a novel computational approach (ProMT) for promoter prediction using Markov chain model and predictive TSS clusters based on structural properties of DNA. Extensive experiments demonstrated that ProMT can significantly improve the predictive performance. Therefore, considering interference between neighboring TSSs is essential for a wider range of promoter prediction.

10. Uncertainty assessment of integrated distributed hydrological models using GLUE with Markov chain Monte Carlo sampling

DEFF Research Database (Denmark)

Blasone, Roberta-Serena; Madsen, Henrik; Rosbjerg, Dan

2008-01-01

uncertainty estimation (GLUE) procedure based on Markov chain Monte Carlo sampling is applied in order to improve the performance of the methodology in estimating parameters and posterior output distributions. The description of the spatial variations of the hydrological processes is accounted for by defining......-distributed responses are, however, still quite unexplored. Especially for complex models, rigorous parameterization, reduction of the parameter space and use of efficient and effective algorithms are essential to facilitate the calibration process and make it more robust. Moreover, for these models multi...... the identifiability of the parameters and results in satisfactory multi-variable simulations and uncertainty estimates. However, the parameter uncertainty alone cannot explain the total uncertainty at all the sites, due to limitations in the distributed data included in the model calibration. The study also indicates...

11. Markov chain analysis of weekly rainfall data in determining drought-proneness

Directory of Open Access Journals (Sweden)

Pabitra Banik

2002-01-01

Full Text Available Markov chain models have been used to evaluate probabilities of getting a sequence of wet and dry weeks during South-West monsoon period over the districts Purulia in West Bengal and Giridih in Bihar state and dry farming tract in the state of Maharashtra of India. An index based on the parameters of this model has been suggested to indicate the extend of drought-proneness of a region. This study will be useful to agricultural planners and irrigation engineers to identifying the areas where agricultural development should be focused as a long term drought mitigation strategy. Also this study will contribute toward a better understanding of the climatology of drought in a major drought-prone region of the world.

12. A toolbox for safety instrumented system evaluation based on improved continuous-time Markov chain

Science.gov (United States)

Wardana, Awang N. I.; Kurniady, Rahman; Pambudi, Galih; Purnama, Jaka; Suryopratomo, Kutut

2017-08-01

Safety instrumented system (SIS) is designed to restore a plant into a safe condition when pre-hazardous event is occur. It has a vital role especially in process industries. A SIS shall be meet with safety requirement specifications. To confirm it, SIS shall be evaluated. Typically, the evaluation is calculated by hand. This paper presents a toolbox for SIS evaluation. It is developed based on improved continuous-time Markov chain. The toolbox supports to detailed approach of evaluation. This paper also illustrates an industrial application of the toolbox to evaluate arch burner safety system of primary reformer. The results of the case study demonstrates that the toolbox can be used to evaluate industrial SIS in detail and to plan the maintenance strategy.

13. Modelling maximum river flow by using Bayesian Markov Chain Monte Carlo

Science.gov (United States)

Cheong, R. Y.; Gabda, D.

2017-09-01

Analysis of flood trends is vital since flooding threatens human living in terms of financial, environment and security. The data of annual maximum river flows in Sabah were fitted into generalized extreme value (GEV) distribution. Maximum likelihood estimator (MLE) raised naturally when working with GEV distribution. However, previous researches showed that MLE provide unstable results especially in small sample size. In this study, we used different Bayesian Markov Chain Monte Carlo (MCMC) based on Metropolis-Hastings algorithm to estimate GEV parameters. Bayesian MCMC method is a statistical inference which studies the parameter estimation by using posterior distribution based on Bayes’ theorem. Metropolis-Hastings algorithm is used to overcome the high dimensional state space faced in Monte Carlo method. This approach also considers more uncertainty in parameter estimation which then presents a better prediction on maximum river flow in Sabah.

14. Markov Chain Monte Carlo Exploration of Minimal Supergravity with Implications for Dark Matter

Energy Technology Data Exchange (ETDEWEB)

Baltz, E

2004-07-19

We explore the full parameter space of Minimal Supergravity (mSUGRA), allowing all four continuous parameters (the scalar mass m{sub 0}, the gaugino mass m{sub 1/2}, the trilinear coupling A{sub 0}, and the ratio of Higgs vacuum expectation values tan {beta}) to vary freely. We apply current accelerator constraints on sparticle and Higgs masses, and on the b {yields} s{gamma} branching ratio, and discuss the impact of the constraints on g{sub {mu}}-2. To study dark matter, we apply the WMAP constraint on the cold dark matter density. We develop Markov Chain Monte Carlo (MCMC) techniques to explore the parameter regions consistent with WMAP, finding them to be considerably superior to previously used methods for exploring supersymmetric parameter spaces. Finally, we study the reach of current and future direct detection experiments in light of the WMAP constraint.

15. A multiple shock model for common cause failures using discrete Markov chain

International Nuclear Information System (INIS)

Chung, Dae Wook; Kang, Chang Soon

1992-01-01

The most widely used models in common cause analysis are (single) shock models such as the BFR, and the MFR. But, single shock model can not treat the individual common cause separately and has some irrational assumptions. Multiple shock model for common cause failures is developed using Markov chain theory. This model treats each common cause shock as separately and sequently occuring event to implicate the change in failure probability distribution due to each common cause shock. The final failure probability distribution is evaluated and compared with that from the BFR model. The results show that multiple shock model which minimizes the assumptions in the BFR model is more realistic and conservative than the BFR model. The further work for application is the estimations of parameters such as common cause shock rate and component failure probability given a shock,p, through the data analysis

16. IMPLEMENTASI METODE MARKOV CHAIN MONTE CARLO DALAM PENENTUAN HARGA KONTRAK BERJANGKA KOMODITAS

Directory of Open Access Journals (Sweden)

PUTU AMANDA SETIAWANI

2015-06-01

Full Text Available The aim of the research is to implement Markov Chain Monte Carlo (MCMC simulation method to price the futures contract of cocoa commodities. The result shows that MCMC is more flexible than Standard Monte Carlo (SMC simulation method because MCMC method uses hit-and-run sampler algorithm to generate proposal movements that are subsequently accepted or rejected with a probability that depends on the distribution of the target that we want to be achieved. This research shows that MCMC method is suitable to be used to simulate the model of cocoa commodity price movement. The result of this research is a simulation of future contract prices for the next three months and future contract prices that must be paid at the time the contract expires. Pricing future contract by using MCMC method will produce the cheaper contract price if it compares to Standard Monte Carlo simulation.

17. A Markov Chain Based Demand Prediction Model for Stations in Bike Sharing Systems

Directory of Open Access Journals (Sweden)

Yajun Zhou

2018-01-01

Full Text Available Accurate transfer demand prediction at bike stations is the key to develop balancing solutions to address the overutilization or underutilization problem often occurring in bike sharing system. At the same time, station transfer demand prediction is helpful to bike station layout and optimization of the number of public bikes within the station. Traditional traffic demand prediction methods, such as gravity model, cannot be easily adapted to the problem of forecasting bike station transfer demand due to the difficulty in defining impedance and distinct characteristics of bike stations (Xu et al. 2013. Therefore, this paper proposes a prediction method based on Markov chain model. The proposed model is evaluated based on field data collected from Zhongshan City bike sharing system. The daily production and attraction of stations are forecasted. The experimental results show that the model of this paper performs higher forecasting accuracy and better generalization ability.

18. A stochastic Markov chain model to describe lung cancer growth and metastasis.

Science.gov (United States)

Newton, Paul K; Mason, Jeremy; Bethel, Kelly; Bazhenova, Lyudmila A; Nieva, Jorge; Kuhn, Peter

2012-01-01

A stochastic Markov chain model for metastatic progression is developed for primary lung cancer based on a network construction of metastatic sites with dynamics modeled as an ensemble of random walkers on the network. We calculate a transition matrix, with entries (transition probabilities) interpreted as random variables, and use it to construct a circular bi-directional network of primary and metastatic locations based on postmortem tissue analysis of 3827 autopsies on untreated patients documenting all primary tumor locations and metastatic sites from this population. The resulting 50 potential metastatic sites are connected by directed edges with distributed weightings, where the site connections and weightings are obtained by calculating the entries of an ensemble of transition matrices so that the steady-state distribution obtained from the long-time limit of the Markov chain dynamical system corresponds to the ensemble metastatic distribution obtained from the autopsy data set. We condition our search for a transition matrix on an initial distribution of metastatic tumors obtained from the data set. Through an iterative numerical search procedure, we adjust the entries of a sequence of approximations until a transition matrix with the correct steady-state is found (up to a numerical threshold). Since this constrained linear optimization problem is underdetermined, we characterize the statistical variance of the ensemble of transition matrices calculated using the means and variances of their singular value distributions as a diagnostic tool. We interpret the ensemble averaged transition probabilities as (approximately) normally distributed random variables. The model allows us to simulate and quantify disease progression pathways and timescales of progression from the lung position to other sites and we highlight several key findings based on the model.

19. Multi-site Stochastic Simulation of Daily Streamflow with Markov Chain and KNN Algorithm

Science.gov (United States)

Mathai, J.; Mujumdar, P.

2017-12-01

A key focus of this study is to develop a method which is physically consistent with the hydrologic processes that can capture short-term characteristics of daily hydrograph as well as the correlation of streamflow in temporal and spatial domains. In complex water resource systems, flow fluctuations at small time intervals require that discretisation be done at small time scales such as daily scales. Also, simultaneous generation of synthetic flows at different sites in the same basin are required. We propose a method to equip water managers with a streamflow generator within a stochastic streamflow simulation framework. The motivation for the proposed method is to generate sequences that extend beyond the variability represented in the historical record of streamflow time series. The method has two steps: In step 1, daily flow is generated independently at each station by a two-state Markov chain, with rising limb increments randomly sampled from a Gamma distribution and the falling limb modelled as exponential recession and in step 2, the streamflow generated in step 1 is input to a nonparametric K-nearest neighbor (KNN) time series bootstrap resampler. The KNN model, being data driven, does not require assumptions on the dependence structure of the time series. A major limitation of KNN based streamflow generators is that they do not produce new values, but merely reshuffle the historical data to generate realistic streamflow sequences. However, daily flow generated using the Markov chain approach is capable of generating a rich variety of streamflow sequences. Furthermore, the rising and falling limbs of daily hydrograph represent different physical processes, and hence they need to be modelled individually. Thus, our method combines the strengths of the two approaches. We show the utility of the method and improvement over the traditional KNN by simulating daily streamflow sequences at 7 locations in the Godavari River basin in India.

20. Effects of tour boats on dolphin activity examined with sensitivity analysis of Markov chains.

Science.gov (United States)

Dans, Silvana Laura; Degrati, Mariana; Pedraza, Susana Noemí; Crespo, Enrique Alberto

2012-08-01

In Patagonia, Argentina, watching dolphins, especially dusky dolphins (Lagenorhynchus obscurus), is a new tourist activity. Feeding time decreases and time to return to feeding after feeding is abandoned and time it takes a group of dolphins to feed increase in the presence of boats. Such effects on feeding behavior may exert energetic costs on dolphins and thus reduce an individual's survival and reproductive capacity or maybe associated with shifts in distribution. We sought to predict which behavioral changes modify the activity pattern of dolphins the most. We modeled behavioral sequences of dusky dolphins with Markov chains. We calculated transition probabilities from one activity to another and arranged them in a stochastic matrix model. The proportion of time dolphins dedicated to a given activity (activity budget) and the time it took a dolphin to resume that activity after it had been abandoned (recurrence time) were calculated. We used a sensitivity analysis of Markov chains to calculate the sensitivity of the time budget and the activity-resumption time to changes in behavioral transition probabilities. Feeding-time budget was most sensitive to changes in the probability of dolphins switching from traveling to feeding behavior and of maintaining feeding behavior. Thus, an increase in these probabilities would be associated with the largest reduction in the time dedicated to feeding. A reduction in the probability of changing from traveling to feeding would also be associated with the largest increases in the time it takes dolphins to resume feeding. To approach dolphins when they are traveling would not affect behavior less because presence of the boat may keep dolphins from returning to feeding. Our results may help operators of dolphin-watching vessels minimize negative effects on dolphins. ©2012 Society for Conservation Biology.

1. MCMC-ODPR: primer design optimization using Markov Chain Monte Carlo sampling.

Science.gov (United States)

Kitchen, James L; Moore, Jonathan D; Palmer, Sarah A; Allaby, Robin G

2012-11-05

Next generation sequencing technologies often require numerous primer designs that require good target coverage that can be financially costly. We aimed to develop a system that would implement primer reuse to design degenerate primers that could be designed around SNPs, thus find the fewest necessary primers and the lowest cost whilst maintaining an acceptable coverage and provide a cost effective solution. We have implemented Metropolis-Hastings Markov Chain Monte Carlo for optimizing primer reuse. We call it the Markov Chain Monte Carlo Optimized Degenerate Primer Reuse (MCMC-ODPR) algorithm. After repeating the program 1020 times to assess the variance, an average of 17.14% fewer primers were found to be necessary using MCMC-ODPR for an equivalent coverage without implementing primer reuse. The algorithm was able to reuse primers up to five times. We compared MCMC-ODPR with single sequence primer design programs Primer3 and Primer-BLAST and achieved a lower primer cost per amplicon base covered of 0.21 and 0.19 and 0.18 primer nucleotides on three separate gene sequences, respectively. With multiple sequences, MCMC-ODPR achieved a lower cost per base covered of 0.19 than programs BatchPrimer3 and PAMPS, which achieved 0.25 and 0.64 primer nucleotides, respectively. MCMC-ODPR is a useful tool for designing primers at various melting temperatures at good target coverage. By combining degeneracy with optimal primer reuse the user may increase coverage of sequences amplified by the designed primers at significantly lower costs. Our analyses showed that overall MCMC-ODPR outperformed the other primer-design programs in our study in terms of cost per covered base.

2. MCMC-ODPR: Primer design optimization using Markov Chain Monte Carlo sampling

Directory of Open Access Journals (Sweden)

Kitchen James L

2012-11-01

Full Text Available Abstract Background Next generation sequencing technologies often require numerous primer designs that require good target coverage that can be financially costly. We aimed to develop a system that would implement primer reuse to design degenerate primers that could be designed around SNPs, thus find the fewest necessary primers and the lowest cost whilst maintaining an acceptable coverage and provide a cost effective solution. We have implemented Metropolis-Hastings Markov Chain Monte Carlo for optimizing primer reuse. We call it the Markov Chain Monte Carlo Optimized Degenerate Primer Reuse (MCMC-ODPR algorithm. Results After repeating the program 1020 times to assess the variance, an average of 17.14% fewer primers were found to be necessary using MCMC-ODPR for an equivalent coverage without implementing primer reuse. The algorithm was able to reuse primers up to five times. We compared MCMC-ODPR with single sequence primer design programs Primer3 and Primer-BLAST and achieved a lower primer cost per amplicon base covered of 0.21 and 0.19 and 0.18 primer nucleotides on three separate gene sequences, respectively. With multiple sequences, MCMC-ODPR achieved a lower cost per base covered of 0.19 than programs BatchPrimer3 and PAMPS, which achieved 0.25 and 0.64 primer nucleotides, respectively. Conclusions MCMC-ODPR is a useful tool for designing primers at various melting temperatures at good target coverage. By combining degeneracy with optimal primer reuse the user may increase coverage of sequences amplified by the designed primers at significantly lower costs. Our analyses showed that overall MCMC-ODPR outperformed the other primer-design programs in our study in terms of cost per covered base.

3. A stochastic Markov chain model to describe lung cancer growth and metastasis.

Directory of Open Access Journals (Sweden)

Paul K Newton

Full Text Available A stochastic Markov chain model for metastatic progression is developed for primary lung cancer based on a network construction of metastatic sites with dynamics modeled as an ensemble of random walkers on the network. We calculate a transition matrix, with entries (transition probabilities interpreted as random variables, and use it to construct a circular bi-directional network of primary and metastatic locations based on postmortem tissue analysis of 3827 autopsies on untreated patients documenting all primary tumor locations and metastatic sites from this population. The resulting 50 potential metastatic sites are connected by directed edges with distributed weightings, where the site connections and weightings are obtained by calculating the entries of an ensemble of transition matrices so that the steady-state distribution obtained from the long-time limit of the Markov chain dynamical system corresponds to the ensemble metastatic distribution obtained from the autopsy data set. We condition our search for a transition matrix on an initial distribution of metastatic tumors obtained from the data set. Through an iterative numerical search procedure, we adjust the entries of a sequence of approximations until a transition matrix with the correct steady-state is found (up to a numerical threshold. Since this constrained linear optimization problem is underdetermined, we characterize the statistical variance of the ensemble of transition matrices calculated using the means and variances of their singular value distributions as a diagnostic tool. We interpret the ensemble averaged transition probabilities as (approximately normally distributed random variables. The model allows us to simulate and quantify disease progression pathways and timescales of progression from the lung position to other sites and we highlight several key findings based on the model.

4. A reversible-jump Markov chain Monte Carlo algorithm for 1D inversion of magnetotelluric data

Science.gov (United States)

Mandolesi, Eric; Ogaya, Xenia; Campanyà, Joan; Piana Agostinetti, Nicola

2018-04-01

This paper presents a new computer code developed to solve the 1D magnetotelluric (MT) inverse problem using a Bayesian trans-dimensional Markov chain Monte Carlo algorithm. MT data are sensitive to the depth-distribution of rock electric conductivity (or its reciprocal, resistivity). The solution provided is a probability distribution - the so-called posterior probability distribution (PPD) for the conductivity at depth, together with the PPD of the interface depths. The PPD is sampled via a reversible-jump Markov Chain Monte Carlo (rjMcMC) algorithm, using a modified Metropolis-Hastings (MH) rule to accept or discard candidate models along the chains. As the optimal parameterization for the inversion process is generally unknown a trans-dimensional approach is used to allow the dataset itself to indicate the most probable number of parameters needed to sample the PPD. The algorithm is tested against two simulated datasets and a set of MT data acquired in the Clare Basin (County Clare, Ireland). For the simulated datasets the correct number of conductive layers at depth and the associated electrical conductivity values is retrieved, together with reasonable estimates of the uncertainties on the investigated parameters. Results from the inversion of field measurements are compared with results obtained using a deterministic method and with well-log data from a nearby borehole. The PPD is in good agreement with the well-log data, showing as a main structure a high conductive layer associated with the Clare Shale formation. In this study, we demonstrate that our new code go beyond algorithms developend using a linear inversion scheme, as it can be used: (1) to by-pass the subjective choices in the 1D parameterizations, i.e. the number of horizontal layers in the 1D parameterization, and (2) to estimate realistic uncertainties on the retrieved parameters. The algorithm is implemented using a simple MPI approach, where independent chains run on isolated CPU, to take

5. Comparison of methods for calculating conditional expectations of sufficient statistics for continuous time Markov chains

Directory of Open Access Journals (Sweden)

Tataru Paula

2011-12-01

Full Text Available Abstract Background Continuous time Markov chains (CTMCs is a widely used model for describing the evolution of DNA sequences on the nucleotide, amino acid or codon level. The sufficient statistics for CTMCs are the time spent in a state and the number of changes between any two states. In applications past evolutionary events (exact times and types of changes are unaccessible and the past must be inferred from DNA sequence data observed in the present. Results We describe and implement three algorithms for computing linear combinations of expected values of the sufficient statistics, conditioned on the end-points of the chain, and compare their performance with respect to accuracy and running time. The first algorithm is based on an eigenvalue decomposition of the rate matrix (EVD, the second on uniformization (UNI, and the third on integrals of matrix exponentials (EXPM. The implementation in R of the algorithms is available at http://www.birc.au.dk/~paula/. Conclusions We use two different models to analyze the accuracy and eight experiments to investigate the speed of the three algorithms. We find that they have similar accuracy and that EXPM is the slowest method. Furthermore we find that UNI is usually faster than EVD.

6. Using Markov Chains to predict the natural progression of diabetic retinopathy

Directory of Open Access Journals (Sweden)

Priyanka Srikanth

2015-02-01

Full Text Available AIM: To study the natural progression of diabetic retinopathy in patients with type 2 diabetes. METHODS: This was an observational study of 153 cases with type 2 diabetes from 2010 to 2013. The state of patient was noted at end of each year and transition matrices were developed to model movement between years. Patients who progressed to severe non-proliferative diabetic retinopathy (NPDR were treated. Markov Chains and Chi-square test were used for statistical analysis. RESULTS: We modelled the transition of 153 patients from NPDR to blindness on an annual basis. At the end of year 3, we compared results from the Markov model versus actual data. The results from Chi-square test confirmed that there was statistically no significant difference (P=0.70 which provided assurance that the model was robust to estimate mean sojourn times. The key finding was that a patient entering the system in mild NPDR state is expected to stay in that state for 5y followed by 1.07y in moderate NPDR, be in the severe NPDR state for 1.33y before moving into PDR for roughly 8y. It is therefore expected that such a patient entering the model in a state of mild NPDR will enter blindness after 15.29y. CONCLUSION: Patients stay for long time periods in mild NPDR before transitioning into moderate NPDR. However, they move rapidly from moderate NPDR to proliferative diabetic retinopathy (PDR and stay in that state for long periods before transitioning into blindness.

7. A high-fidelity weather time series generator using the Markov Chain process on a piecewise level

Science.gov (United States)

Hersvik, K.; Endrerud, O.-E. V.

2017-12-01

A method is developed for generating a set of unique weather time-series based on an existing weather series. The method allows statistically valid weather variations to take place within repeated simulations of offshore operations. The numerous generated time series need to share the same statistical qualities as the original time series. Statistical qualities here refer mainly to the distribution of weather windows available for work, including durations and frequencies of such weather windows, and seasonal characteristics. The method is based on the Markov chain process. The core new development lies in how the Markov Process is used, specifically by joining small pieces of random length time series together rather than joining individual weather states, each from a single time step, which is a common solution found in the literature. This new Markov model shows favorable characteristics with respect to the requirements set forth and all aspects of the validation performed.

8. Prediction of Annual Rainfall Pattern Using Hidden Markov Model ...

African Journals Online (AJOL)

the stochastic processes is an underlying Markov chain, the other stochastic process is an observable stochastic ... Keywords: Markov model, Hidden Markov model, Transition probability, Observation probability, Crop. Production, Annual Rainfall .... with highest value of the forward probability at time. T+1 is taken as ...

9. A Novel Grey Prediction Model Combining Markov Chain with Functional-Link Net and Its Application to Foreign Tourist Forecasting

Directory of Open Access Journals (Sweden)

Yi-Chung Hu

2017-10-01

Full Text Available Grey prediction models for time series have been widely applied to demand forecasting because only limited data are required for them to build a time series model without any statistical assumptions. Previous studies have demonstrated that the combination of grey prediction with neural networks helps grey prediction perform better. Some methods have been presented to improve the prediction accuracy of the popular GM(1,1 model by using the Markov chain to estimate the residual needed to modify a predicted value. Compared to the previous Grey-Markov models, this study contributes to apply the functional-link net to estimate the degree to which a predicted value obtained from the GM(1,1 model can be adjusted. Furthermore, the troublesome number of states and their bounds that are not easily specified in Markov chain have been determined by a genetic algorithm. To verify prediction performance, the proposed grey prediction model was applied to an important grey system problem—foreign tourist forecasting. Experimental results show that the proposed model provides satisfactory results compared to the other Grey-Markov models considered.

10. Exact Markov chain and approximate diffusion solution for haploid genetic drift with one-way mutation.

Science.gov (United States)

Hössjer, Ola; Tyvand, Peder A; Miloh, Touvia

2016-02-01

The classical Kimura solution of the diffusion equation is investigated for a haploid random mating (Wright-Fisher) model, with one-way mutations and initial-value specified by the founder population. The validity of the transient diffusion solution is checked by exact Markov chain computations, using a Jordan decomposition of the transition matrix. The conclusion is that the one-way diffusion model mostly works well, although the rate of convergence depends on the initial allele frequency and the mutation rate. The diffusion approximation is poor for mutation rates so low that the non-fixation boundary is regular. When this happens we perturb the diffusion solution around the non-fixation boundary and obtain a more accurate approximation that takes quasi-fixation of the mutant allele into account. The main application is to quantify how fast a specific genetic variant of the infinite alleles model is lost. We also discuss extensions of the quasi-fixation approach to other models with small mutation rates. Copyright © 2015 Elsevier Inc. All rights reserved.

11. Markov Chain Monte Carlo Simulation to Assess Uncertainty in Models of Naturally Deformed Rock

Science.gov (United States)

Davis, J. R.; Titus, S.; Giorgis, S. D.; Horsman, E. M.

2015-12-01

Field studies in tectonics and structural geology involve many kinds of data, such as foliation-lineation pairs, folded and boudinaged veins, deformed clasts, and lattice preferred orientations. Each data type can inform a model of deformation, for example by excluding certain geometries or constraining model parameters. In past work we have demonstrated how to systematically integrate a wide variety of data types into the computation of best-fit deformations. However, because even the simplest deformation models tend to be highly non-linear in their parameters, evaluating the uncertainty in the best fit has been difficult. In this presentation we describe an approach to rigorously assessing the uncertainty in models of naturally deformed rock. Rather than finding a single vector of parameter values that fits the data best, we use Bayesian Markov chain Monte Carlo methods to generate a large set of vectors of varying fitness. Taken together, these vectors approximate the probability distribution of the parameters given the data. From this distribution, various auxiliary statistical quantities and conclusions can be derived. Further, the relative probability of differing models can be quantified. We apply this approach to two example data sets, from the Gem Lake shear zone and western Idaho shear zone. Our findings address shear zone geometry, magnitude of deformation, strength of field fabric, and relative viscosity of clasts. We compare our model predictions to those of earlier studies.

12. The behavior of Metropolis-coupled Markov chains when sampling rugged phylogenetic distributions.

Science.gov (United States)

Brown, Jeremy M; Thomson, Robert C

2018-02-15

Bayesian phylogenetic inference involves sampling from posterior distributions of trees, which sometimes exhibit local optima, or peaks, separated by regions of low posterior density. Markov chain Monte Carlo (MCMC) algorithms are the most widely used numerical method for generating samples from these posterior distributions, but they are susceptible to entrapment on individual optima in rugged distributions when they are unable to easily cross through or jump across regions of low posterior density. Ruggedness of posterior distributions can result from a variety of factors, including unmodeled variation in evolutionary processes and unrecognized variation in the true topology across sites or genes. Ruggedness can also become exaggerated when constraints are placed on topologies that require the presence or absence of particular bipartitions (often referred to as positive or negative constraints, respectively). These types of constraints are frequently employed when conducting tests of topological hypotheses (Bergsten et al. 2013; Brown and Thomson 2017). Negative constraints can lead to particularly rugged distributions when the data strongly support a forbidden clade, because monophyly of the clade can be disrupted by inserting outgroup taxa in many different ways. However, topological moves between the alternative disruptions are very difficult, because they require swaps between the inserted outgroup taxa while the data constrain taxa from the forbidden clade to remain close together on the tree. While this precise form of ruggedness is particular to negative constraints, trees with high posterior density can be separated by similarly complicated topological rearrangements, even in the absence of constraints.

13. Study on the Calculation Models of Bus Delay at Bays Using Queueing Theory and Markov Chain

Directory of Open Access Journals (Sweden)

Feng Sun

2015-01-01

Full Text Available Traffic congestion at bus bays has decreased the service efficiency of public transit seriously in China, so it is crucial to systematically study its theory and methods. However, the existing studies lack theoretical model on computing efficiency. Therefore, the calculation models of bus delay at bays are studied. Firstly, the process that buses are delayed at bays is analyzed, and it was found that the delay can be divided into entering delay and exiting delay. Secondly, the queueing models of bus bays are formed, and the equilibrium distribution functions are proposed by applying the embedded Markov chain to the traditional model of queuing theory in the steady state; then the calculation models of entering delay are derived at bays. Thirdly, the exiting delay is studied by using the queueing theory and the gap acceptance theory. Finally, the proposed models are validated using field-measured data, and then the influencing factors are discussed. With these models the delay is easily assessed knowing the characteristics of the dwell time distribution and traffic volume at the curb lane in different locations and different periods. It can provide basis for the efficiency evaluation of bus bays.

14. Mathematical modeling, analysis and Markov Chain Monte Carlo simulation of Ebola epidemics

Science.gov (United States)

Tulu, Thomas Wetere; Tian, Boping; Wu, Zunyou

Ebola virus infection is a severe infectious disease with the highest case fatality rate which become the global public health treat now. What makes the disease the worst of all is no specific effective treatment available, its dynamics is not much researched and understood. In this article a new mathematical model incorporating both vaccination and quarantine to study the dynamics of Ebola epidemic has been developed and comprehensively analyzed. The existence as well as uniqueness of the solution to the model is also verified and the basic reproduction number is calculated. Besides, stability conditions are also checked and finally simulation is done using both Euler method and one of the top ten most influential algorithm known as Markov Chain Monte Carlo (MCMC) method. Different rates of vaccination to predict the effect of vaccination on the infected individual over time and that of quarantine are discussed. The results show that quarantine and vaccination are very effective ways to control Ebola epidemic. From our study it was also seen that there is less possibility of an individual for getting Ebola virus for the second time if they survived his/her first infection. Last but not least real data has been fitted to the model, showing that it can used to predict the dynamic of Ebola epidemic.

15. Markov chain-based promoter structure modeling for tissue-specific expression pattern prediction.

Science.gov (United States)

Vandenbon, Alexis; Miyamoto, Yuki; Takimoto, Noriko; Kusakabe, Takehiro; Nakai, Kenta

2008-02-29

Transcriptional regulation is the first level of regulation of gene expression and is therefore a major topic in computational biology. Genes with similar expression patterns can be assumed to be co-regulated at the transcriptional level by promoter sequences with a similar structure. Current approaches for modeling shared regulatory features tend to focus mainly on clustering of cis-regulatory sites. Here we introduce a Markov chain-based promoter structure model that uses both shared motifs and shared features from an input set of promoter sequences to predict candidate genes with similar expression. The model uses positional preference, order, and orientation of motifs. The trained model is used to score a genomic set of promoter sequences: high-scoring promoters are assumed to have a structure similar to the input sequences and are thus expected to drive similar expression patterns. We applied our model on two datasets in Caenorhabditis elegans and in Ciona intestinalis. Both computational and experimental verifications indicate that this model is capable of predicting candidate promoters driving similar expression patterns as the input-regulatory sequences. This model can be useful for finding promising candidate genes for wet-lab experiments and for increasing our understanding of transcriptional regulation.

16. Hydrologic data assimilation using particle Markov chain Monte Carlo simulation: Theory, concepts and applications

Science.gov (United States)

Vrugt, Jasper A.; ter Braak, Cajo J. F.; Diks, Cees G. H.; Schoups, Gerrit

2013-01-01

During the past decades much progress has been made in the development of computer based methods for parameter and predictive uncertainty estimation of hydrologic models. The goal of this paper is twofold. As part of this special anniversary issue we first shortly review the most important historical developments in hydrologic model calibration and uncertainty analysis that has led to current perspectives. Then, we introduce theory, concepts and simulation results of a novel data assimilation scheme for joint inference of model parameters and state variables. This Particle-DREAM method combines the strengths of sequential Monte Carlo sampling and Markov chain Monte Carlo simulation and is especially designed for treatment of forcing, parameter, model structural and calibration data error. Two different variants of Particle-DREAM are presented to satisfy assumptions regarding the temporal behavior of the model parameters. Simulation results using a 40-dimensional atmospheric “toy” model, the Lorenz attractor and a rainfall-runoff model show that Particle-DREAM, P-DREAM(VP) and P-DREAM(IP) require far fewer particles than current state-of-the-art filters to closely track the evolving target distribution of interest, and provide important insights into the information content of discharge data and non-stationarity of model parameters. Our development follows formal Bayes, yet Particle-DREAM and its variants readily accommodate hydrologic signatures, informal likelihood functions or other (in)sufficient statistics if those better represent the salient features of the calibration data and simulation model used.

17. Mapping absorption processes onto a Markov chain, conserving the mean first passage time

International Nuclear Information System (INIS)

Biswas, Katja

2013-01-01

The dynamics of a multidimensional system is projected onto a discrete state master equation using the transition rates W(k → k′; t, t + dt) between a set of states {k} represented by the regions {ζ k } in phase or discrete state space. Depending on the dynamics Γ i (t) of the original process and the choice of ζ k , the discretized process can be Markovian or non-Markovian. For absorption processes, it is shown that irrespective of these properties of the projection, a master equation with time-independent transition rates W-bar (k→k ' ) can be obtained, which conserves the total occupation time of the partitions of the phase or discrete state space of the original process. An expression for the transition probabilities p-bar (k ' |k) is derived based on either time-discrete measurements {t i } with variable time stepping Δ (i+1)i = t i+1 − t i or the theoretical knowledge at continuous times t. This allows computational methods of absorbing Markov chains to be used to obtain the mean first passage time (MFPT) of the system. To illustrate this approach, the procedure is applied to obtain the MFPT for the overdamped Brownian motion of particles subject to a system with dichotomous noise and the escape from an entropic barrier. The high accuracy of the simulation results confirms with the theory. (paper)

18. Estimating stepwise debromination pathways of polybrominated diphenyl ethers with an analogue Markov Chain Monte Carlo algorithm.

Science.gov (United States)

Zou, Yonghong; Christensen, Erik R; Zheng, Wei; Wei, Hua; Li, An

2014-11-01

A stochastic process was developed to simulate the stepwise debromination pathways for polybrominated diphenyl ethers (PBDEs). The stochastic process uses an analogue Markov Chain Monte Carlo (AMCMC) algorithm to generate PBDE debromination profiles. The acceptance or rejection of the randomly drawn stepwise debromination reactions was determined by a maximum likelihood function. The experimental observations at certain time points were used as target profiles; therefore, the stochastic processes are capable of presenting the effects of reaction conditions on the selection of debromination pathways. The application of the model is illustrated by adopting the experimental results of decabromodiphenyl ether (BDE209) in hexane exposed to sunlight. Inferences that were not obvious from experimental data were suggested by model simulations. For example, BDE206 has much higher accumulation at the first 30 min of sunlight exposure. By contrast, model simulation suggests that, BDE206 and BDE207 had comparable yields from BDE209. The reason for the higher BDE206 level is that BDE207 has the highest depletion in producing octa products. Compared to a previous version of the stochastic model based on stochastic reaction sequences (SRS), the AMCMC approach was determined to be more efficient and robust. Due to the feature of only requiring experimental observations as input, the AMCMC model is expected to be applicable to a wide range of PBDE debromination processes, e.g. microbial, photolytic, or joint effects in natural environments. Copyright © 2014 Elsevier Ltd. All rights reserved.

19. Enhancing hydrologic data assimilation by evolutionary Particle Filter and Markov Chain Monte Carlo

Science.gov (United States)

2018-01-01

Particle Filters (PFs) have received increasing attention by researchers from different disciplines including the hydro-geosciences, as an effective tool to improve model predictions in nonlinear and non-Gaussian dynamical systems. The implication of dual state and parameter estimation using the PFs in hydrology has evolved since 2005 from the PF-SIR (sampling importance resampling) to PF-MCMC (Markov Chain Monte Carlo), and now to the most effective and robust framework through evolutionary PF approach based on Genetic Algorithm (GA) and MCMC, the so-called EPFM. In this framework, the prior distribution undergoes an evolutionary process based on the designed mutation and crossover operators of GA. The merit of this approach is that the particles move to an appropriate position by using the GA optimization and then the number of effective particles is increased by means of MCMC, whereby the particle degeneracy is avoided and the particle diversity is improved. In this study, the usefulness and effectiveness of the proposed EPFM is investigated by applying the technique on a conceptual and highly nonlinear hydrologic model over four river basins located in different climate and geographical regions of the United States. Both synthetic and real case studies demonstrate that the EPFM improves both the state and parameter estimation more effectively and reliably as compared with the PF-MCMC.

20. Synchrotron imaging and Markov Chain Monte Carlo reveal tooth mineralization patterns.

Directory of Open Access Journals (Sweden)

Daniel R Green

Full Text Available The progressive character of tooth formation records aspects of mammalian life history, diet, seasonal behavior and climate. Tooth mineralization occurs in two stages: secretion and maturation, which overlap to some degree. Despite decades of study, the spatial and temporal pattern of elemental incorporation during enamel mineralization remains poorly characterized. Here we use synchrotron X-ray microtomography and Markov Chain Monte Carlo sampling to estimate mineralization patterns from an ontogenetic series of sheep molars (n = 45 M1s, 18 M2s. We adopt a Bayesian approach that posits a general pattern of maturation estimated from individual- and population-level mineral density variation over time. This approach converts static images of mineral density into a dynamic model of mineralization, and demonstrates that enamel secretion and maturation waves advance at nonlinear rates with distinct geometries. While enamel secretion is ordered, maturation geometry varies within a population and appears to be driven by diffusive processes. Our model yields concrete expectations for the integration of physiological and environmental signals, which is of particular significance for paleoseasonality research. This study also provides an avenue for characterizing mineralization patterns in other taxa. Our synchrotron imaging data and model are available for application to multiple disciplines, including health, material science, and paleontological research.

1. A Markov chain model for image ranking system in social networks

Science.gov (United States)

Zin, Thi Thi; Tin, Pyke; Toriu, Takashi; Hama, Hiromitsu

2014-03-01

In today world, different kinds of networks such as social, technological, business and etc. exist. All of the networks are similar in terms of distributions, continuously growing and expanding in large scale. Among them, many social networks such as Facebook, Twitter, Flickr and many others provides a powerful abstraction of the structure and dynamics of diverse kinds of inter personal connection and interaction. Generally, the social network contents are created and consumed by the influences of all different social navigation paths that lead to the contents. Therefore, identifying important and user relevant refined structures such as visual information or communities become major factors in modern decision making world. Moreover, the traditional method of information ranking systems cannot be successful due to their lack of taking into account the properties of navigation paths driven by social connections. In this paper, we propose a novel image ranking system in social networks by using the social data relational graphs from social media platform jointly with visual data to improve the relevance between returned images and user intentions (i.e., social relevance). Specifically, we propose a Markov chain based Social-Visual Ranking algorithm by taking social relevance into account. By using some extensive experiments, we demonstrated the significant and effectiveness of the proposed social-visual ranking method.

2. Solving inverse problem for Markov chain model of customer lifetime value using flower pollination algorithm

Science.gov (United States)

Al-Ma'shumah, Fathimah; Permana, Dony; Sidarto, Kuntjoro Adji

2015-12-01

Customer Lifetime Value is an important and useful concept in marketing. One of its benefits is to help a company for budgeting marketing expenditure for customer acquisition and customer retention. Many mathematical models have been introduced to calculate CLV considering the customer retention/migration classification scheme. A fairly new class of these models which will be described in this paper uses Markov Chain Models (MCM). This class of models has the major advantage for its flexibility to be modified to several different cases/classification schemes. In this model, the probabilities of customer retention and acquisition play an important role. From Pfeifer and Carraway, 2000, the final formula of CLV obtained from MCM usually contains nonlinear form of the transition probability matrix. This nonlinearity makes the inverse problem of CLV difficult to solve. This paper aims to solve this inverse problem, yielding the approximate transition probabilities for the customers, by applying metaheuristic optimization algorithm developed by Yang, 2013, Flower Pollination Algorithm. The major interpretation of obtaining the transition probabilities are to set goals for marketing teams in keeping the relative frequencies of customer acquisition and customer retention.

3. Bayesian calibration of terrestrial ecosystem models: a study of advanced Markov chain Monte Carlo methods

Science.gov (United States)

Lu, Dan; Ricciuto, Daniel; Walker, Anthony; Safta, Cosmin; Munger, William

2017-09-01

Calibration of terrestrial ecosystem models is important but challenging. Bayesian inference implemented by Markov chain Monte Carlo (MCMC) sampling provides a comprehensive framework to estimate model parameters and associated uncertainties using their posterior distributions. The effectiveness and efficiency of the method strongly depend on the MCMC algorithm used. In this work, a differential evolution adaptive Metropolis (DREAM) algorithm is used to estimate posterior distributions of 21 parameters for the data assimilation linked ecosystem carbon (DALEC) model using 14 years of daily net ecosystem exchange data collected at the Harvard Forest Environmental Measurement Site eddy-flux tower. The calibration of DREAM results in a better model fit and predictive performance compared to the popular adaptive Metropolis (AM) scheme. Moreover, DREAM indicates that two parameters controlling autumn phenology have multiple modes in their posterior distributions while AM only identifies one mode. The application suggests that DREAM is very suitable to calibrate complex terrestrial ecosystem models, where the uncertain parameter size is usually large and existence of local optima is always a concern. In addition, this effort justifies the assumptions of the error model used in Bayesian calibration according to the residual analysis. The result indicates that a heteroscedastic, correlated, Gaussian error model is appropriate for the problem, and the consequent constructed likelihood function can alleviate the underestimation of parameter uncertainty that is usually caused by using uncorrelated error models.

4. Prediction of small molecule binding property of protein domains with Bayesian classifiers based on Markov chains.

Science.gov (United States)

Bulashevska, Alla; Stein, Martin; Jackson, David; Eils, Roland

2009-12-01

Accurate computational methods that can help to predict biological function of a protein from its sequence are of great interest to research biologists and pharmaceutical companies. One approach to assume the function of proteins is to predict the interactions between proteins and other molecules. In this work, we propose a machine learning method that uses a primary sequence of a domain to predict its propensity for interaction with small molecules. By curating the Pfam database with respect to the small molecule binding ability of its component domains, we have constructed a dataset of small molecule binding and non-binding domains. This dataset was then used as training set to learn a Bayesian classifier, which should distinguish members of each class. The domain sequences of both classes are modelled with Markov chains. In a Jack-knife test, our classification procedure achieved the predictive accuracies of 77.2% and 66.7% for binding and non-binding classes respectively. We demonstrate the applicability of our classifier by using it to identify previously unknown small molecule binding domains. Our predictions are available as supplementary material and can provide very useful information to drug discovery specialists. Given the ubiquitous and essential role small molecules play in biological processes, our method is important for identifying pharmaceutically relevant components of complete proteomes. The software is available from the author upon request.

5. Bayesian calibration of terrestrial ecosystem models: a study of advanced Markov chain Monte Carlo methods

Directory of Open Access Journals (Sweden)

D. Lu

2017-09-01

Full Text Available Calibration of terrestrial ecosystem models is important but challenging. Bayesian inference implemented by Markov chain Monte Carlo (MCMC sampling provides a comprehensive framework to estimate model parameters and associated uncertainties using their posterior distributions. The effectiveness and efficiency of the method strongly depend on the MCMC algorithm used. In this work, a differential evolution adaptive Metropolis (DREAM algorithm is used to estimate posterior distributions of 21 parameters for the data assimilation linked ecosystem carbon (DALEC model using 14 years of daily net ecosystem exchange data collected at the Harvard Forest Environmental Measurement Site eddy-flux tower. The calibration of DREAM results in a better model fit and predictive performance compared to the popular adaptive Metropolis (AM scheme. Moreover, DREAM indicates that two parameters controlling autumn phenology have multiple modes in their posterior distributions while AM only identifies one mode. The application suggests that DREAM is very suitable to calibrate complex terrestrial ecosystem models, where the uncertain parameter size is usually large and existence of local optima is always a concern. In addition, this effort justifies the assumptions of the error model used in Bayesian calibration according to the residual analysis. The result indicates that a heteroscedastic, correlated, Gaussian error model is appropriate for the problem, and the consequent constructed likelihood function can alleviate the underestimation of parameter uncertainty that is usually caused by using uncorrelated error models.

6. Bayesian parameter estimation in dynamic population model via particle Markov chain Monte Carlo

Directory of Open Access Journals (Sweden)

Meng Gao

2012-12-01

Full Text Available In nature, population dynamics are subject to multiple sources of stochasticity. State-space models (SSMs provide an ideal framework for incorporating both environmental noises and measurement errors into dynamic population models. In this paper, we present a recently developed method, Particle Markov Chain Monte Carlo (Particle MCMC, for parameter estimation in nonlinear SSMs. We use one effective algorithm of Particle MCMC, Particle Gibbs sampling algorithm, to estimate the parameters of a state-space model of population dynamics. The posterior distributions of parameters are derived given the conjugate prior distribution. Numerical simulations showed that the model parameters can be accurately estimated, no matter the deterministic model is stable, periodic or chaotic. Moreover, we fit the model to 16 representative time series from Global Population Dynamics Database (GPDD. It is verified that the results of parameter and state estimation using Particle Gibbs sampling algorithm are satisfactory for a majority of time series. For other time series, the quality of parameter estimation can also be improved, if prior knowledge is constrained. In conclusion, Particle Gibbs sampling algorithm provides a new Bayesian parameter inference method for studying population dynamics.

7. On stochastic error and computational efficiency of the Markov Chain Monte Carlo method

KAUST Repository

Li, Jun

2014-01-01

In Markov Chain Monte Carlo (MCMC) simulations, thermal equilibria quantities are estimated by ensemble average over a sample set containing a large number of correlated samples. These samples are selected in accordance with the probability distribution function, known from the partition function of equilibrium state. As the stochastic error of the simulation results is significant, it is desirable to understand the variance of the estimation by ensemble average, which depends on the sample size (i.e., the total number of samples in the set) and the sampling interval (i.e., cycle number between two consecutive samples). Although large sample sizes reduce the variance, they increase the computational cost of the simulation. For a given CPU time, the sample size can be reduced greatly by increasing the sampling interval, while having the corresponding increase in variance be negligible if the original sampling interval is very small. In this work, we report a few general rules that relate the variance with the sample size and the sampling interval. These results are observed and confirmed numerically. These variance rules are derived for theMCMCmethod but are also valid for the correlated samples obtained using other Monte Carlo methods. The main contribution of this work includes the theoretical proof of these numerical observations and the set of assumptions that lead to them. © 2014 Global-Science Press.

8. On the Study of Transience and Recurrence of the Markov Chain Defined by Directed Weighted Circuits Associated with a Random Walk in Fixed Environment

Directory of Open Access Journals (Sweden)

Chrysoula Ganatsiou

2013-01-01

Full Text Available By using the cycle representation theory of Markov processes, we investigate proper criterions regarding transience and recurrence of the corresponding Markov chain represented uniquely by directed cycles (especially by directed circuits and weights of a random walk with jumps in a fixed environment.

9. A Birthday Paradox for Markov chains with an optimal bound for collision in the Pollard Rho algorithm for discrete logarithm

OpenAIRE

Kim, Jeong Han; Montenegro, Ravi; Peres, Yuval; Tetali, Prasad

2010-01-01

We show a Birthday Paradox for self-intersections of Markov chains with uniform stationary distribution. As an application, we analyze Pollard's Rho algorithm for finding the discrete logarithm in a cyclic group $G$ and find that if the partition in the algorithm is given by a random oracle, then with high probability a collision occurs in $\\Theta(\\sqrt{|G|})$ steps. Moreover, for the parallelized distinguished points algorithm on $J$ processors we find that $\\Theta(\\sqrt{|G|}/J)$ steps suffi...

10. Mathematical model of the loan portfolio dynamics in the form of Markov chain considering the process of new customers attraction

Science.gov (United States)

Bozhalkina, Yana

2017-12-01

Mathematical model of the loan portfolio structure change in the form of Markov chain is explored. This model considers in one scheme both the process of customers attraction, their selection based on the credit score, and loans repayment. The model describes the structure and volume of the loan portfolio dynamics, which allows to make medium-term forecasts of profitability and risk. Within the model corrective actions of bank management in order to increase lending volumes or to reduce the risk are formalized.

11. Markov decision processes: a tool for sequential decision making under uncertainty.

Science.gov (United States)

Alagoz, Oguzhan; Hsu, Heather; Schaefer, Andrew J; Roberts, Mark S

2010-01-01

We provide a tutorial on the construction and evaluation of Markov decision processes (MDPs), which are powerful analytical tools used for sequential decision making under uncertainty that have been widely used in many industrial and manufacturing applications but are underutilized in medical decision making (MDM). We demonstrate the use of an MDP to solve a sequential clinical treatment problem under uncertainty. Markov decision processes generalize standard Markov models in that a decision process is embedded in the model and multiple decisions are made over time. Furthermore, they have significant advantages over standard decision analysis. We compare MDPs to standard Markov-based simulation models by solving the problem of the optimal timing of living-donor liver transplantation using both methods. Both models result in the same optimal transplantation policy and the same total life expectancies for the same patient and living donor. The computation time for solving the MDP model is significantly smaller than that for solving the Markov model. We briefly describe the growing literature of MDPs applied to medical decisions.

12. Markov Chain Monte Carlo inversion of temperature and salinity structure of an internal solitary wave packet from marine seismic data

Science.gov (United States)

Tang, Qunshu; Hobbs, Richard; Zheng, Chan; Biescas, Berta; Caiado, Camila

2016-06-01

Marine seismic reflection technique is used to observe the strong ocean dynamic process of nonlinear internal solitary waves (ISWs or solitons) in the near-surface water. Analysis of ISWs is problematical because of their transient nature and limitations of classical physical oceanography methods. This work explores a Markov Chain Monte Carlo (MCMC) approach to recover the temperature and salinity of ISW field using the seismic reflectivity data and in situ hydrographic data. The MCMC approach is designed to directly sample the posterior probability distributions of temperature and salinity which are the solutions of the system under investigation. The principle improvement is the capability of incorporating uncertainties in observations and prior models which then provide quantified uncertainties in the output model parameters. We tested the MCMC approach on two acoustic reflectivity data sets one synthesized from a CTD cast and the other derived from multichannel seismic reflections. This method finds the solutions faithfully within the significantly narrowed confidence intervals from the provided priors. Combined with a low frequency initial model interpreted from seismic horizons of ISWs, the MCMC method is used to compute the finescale temperature, salinity, acoustic velocity, and density of ISW field. The statistically derived results are equivalent to the conventional linearized inversion method. However, the former provides us the quantified uncertainties of the temperature and salinity along the whole section whilst the latter does not. These results are the first time ISWs have been mapped with sufficient detail for further analysis of their dynamic properties.

13. Numerical estimation of structure constants in the three-dimensional Ising conformal field theory through Markov chain uv sampler

Science.gov (United States)

Herdeiro, Victor

2017-09-01

Herdeiro and Doyon [Phys. Rev. E 94, 043322 (2016), 10.1103/PhysRevE.94.043322] introduced a numerical recipe, dubbed uv sampler, offering precise estimations of the conformal field theory (CFT) data of the planar two-dimensional (2D) critical Ising model. It made use of scale invariance emerging at the critical point in order to sample finite sublattice marginals of the infinite plane Gibbs measure of the model by producing holographic boundary distributions. The main ingredient of the Markov chain Monte Carlo sampler is the invariance under dilation. This paper presents a generalization to higher dimensions with the critical 3D Ising model. This leads to numerical estimations of a subset of the CFT data—scaling weights and structure constants—through fitting of measured correlation functions. The results are shown to agree with the recent most precise estimations from numerical bootstrap methods [Kos, Poland, Simmons-Duffin, and Vichi, J. High Energy Phys. 08 (2016) 036, 10.1007/JHEP08(2016)036].

14. Mission reliability of semi-Markov systems under generalized operational time requirements

International Nuclear Information System (INIS)

Wu, Xiaoyue; Hillston, Jane

2015-01-01

Mission reliability of a system depends on specific criteria for mission success. To evaluate the mission reliability of some mission systems that do not need to work normally for the whole mission time, two types of mission reliability for such systems are studied. The first type corresponds to the mission requirement that the system must remain operational continuously for a minimum time within the given mission time interval, while the second corresponds to the mission requirement that the total operational time of the system within the mission time window must be greater than a given value. Based on Markov renewal properties, matrix integral equations are derived for semi-Markov systems. Numerical algorithms and a simulation procedure are provided for both types of mission reliability. Two examples are used for illustration purposes. One is a one-unit repairable Markov system, and the other is a cold standby semi-Markov system consisting of two components. By the proposed approaches, the mission reliability of systems with time redundancy can be more precisely estimated to avoid possible unnecessary redundancy of system resources. - Highlights: • Two types of mission reliability under generalized requirements are defined. • Equations for both types of reliability are derived for semi-Markov systems. • Numerical methods are given for solving both types of reliability. • Simulation procedure is given for estimating both types of reliability. • Verification of the numerical methods is given by the results of simulation

15. Bayesian inversion of seismic and electromagnetic data for marine gas reservoir characterization using multi-chain Markov chain Monte Carlo sampling

Science.gov (United States)

Ren, Huiying; Ray, Jaideep; Hou, Zhangshuan; Huang, Maoyi; Bao, Jie; Swiler, Laura

2017-12-01

In this study we developed an efficient Bayesian inversion framework for interpreting marine seismic Amplitude Versus Angle and Controlled-Source Electromagnetic data for marine reservoir characterization. The framework uses a multi-chain Markov-chain Monte Carlo sampler, which is a hybrid of DiffeRential Evolution Adaptive Metropolis and Adaptive Metropolis samplers. The inversion framework is tested by estimating reservoir-fluid saturations and porosity based on marine seismic and Controlled-Source Electromagnetic data. The multi-chain Markov-chain Monte Carlo is scalable in terms of the number of chains, and is useful for computationally demanding Bayesian model calibration in scientific and engineering problems. As a demonstration, the approach is used to efficiently and accurately estimate the porosity and saturations in a representative layered synthetic reservoir. The results indicate that the seismic Amplitude Versus Angle and Controlled-Source Electromagnetic joint inversion provides better estimation of reservoir saturations than the seismic Amplitude Versus Angle only inversion, especially for the parameters in deep layers. The performance of the inversion approach for various levels of noise in observational data was evaluated - reasonable estimates can be obtained with noise levels up to 25%. Sampling efficiency due to the use of multiple chains was also checked and was found to have almost linear scalability.

16. Additive N-step Markov chains as prototype model of symbolic stochastic dynamical systems with long-range correlations

International Nuclear Information System (INIS)

Mayzelis, Z.A.; Apostolov, S.S.; Melnyk, S.S.; Usatenko, O.V.; Yampol'skii, V.A.

2007-01-01

A theory of symbolic dynamic systems with long-range correlations based on the consideration of the binary N-step Markov chains developed earlier in Phys Rev Lett 2003;90:110601 is generalized to the biased case (non-equal numbers of zeros and unities in the chain). In the model, the conditional probability that the ith symbol in the chain equals zero (or unity) is a linear function of the number of unities (zeros) among the preceding N symbols. The correlation and distribution functions as well as the variance of number of symbols in the words of arbitrary length L are obtained analytically and verified by numerical simulations. A self-similarity of the studied stochastic process is revealed and the similarity group transformation of the chain parameters is presented. The diffusion Fokker-Planck equation governing the distribution function of the L-words is explored. If the persistent correlations are not extremely strong, the distribution function is shown to be the Gaussian with the variance being nonlinearly dependent on L. An equation connecting the memory and correlation function of the additive Markov chain is presented. This equation allows reconstructing a memory function using a correlation function of the system. Effectiveness and robustness of the proposed method is demonstrated by simple model examples. Memory functions of concrete coarse-grained literary texts are found and their universal power-law behavior at long distances is revealed

17. Additive N-step Markov chains as prototype model of symbolic stochastic dynamical systems with long-range correlations

Energy Technology Data Exchange (ETDEWEB)

Mayzelis, Z.A. [Department of Physics, Kharkov National University, 4 Svoboda Sq., Kharkov 61077 (Ukraine); Apostolov, S.S. [Department of Physics, Kharkov National University, 4 Svoboda Sq., Kharkov 61077 (Ukraine); Melnyk, S.S. [A. Ya. Usikov Institute for Radiophysics and Electronics, Ukrainian Academy of Science, 12 Proskura Street, 61085 Kharkov (Ukraine); Usatenko, O.V. [A. Ya. Usikov Institute for Radiophysics and Electronics, Ukrainian Academy of Science, 12 Proskura Street, 61085 Kharkov (Ukraine)]. E-mail: usatenko@ire.kharkov.ua; Yampol' skii, V.A. [A. Ya. Usikov Institute for Radiophysics and Electronics, Ukrainian Academy of Science, 12 Proskura Street, 61085 Kharkov (Ukraine)

2007-10-15

A theory of symbolic dynamic systems with long-range correlations based on the consideration of the binary N-step Markov chains developed earlier in Phys Rev Lett 2003;90:110601 is generalized to the biased case (non-equal numbers of zeros and unities in the chain). In the model, the conditional probability that the ith symbol in the chain equals zero (or unity) is a linear function of the number of unities (zeros) among the preceding N symbols. The correlation and distribution functions as well as the variance of number of symbols in the words of arbitrary length L are obtained analytically and verified by numerical simulations. A self-similarity of the studied stochastic process is revealed and the similarity group transformation of the chain parameters is presented. The diffusion Fokker-Planck equation governing the distribution function of the L-words is explored. If the persistent correlations are not extremely strong, the distribution function is shown to be the Gaussian with the variance being nonlinearly dependent on L. An equation connecting the memory and correlation function of the additive Markov chain is presented. This equation allows reconstructing a memory function using a correlation function of the system. Effectiveness and robustness of the proposed method is demonstrated by simple model examples. Memory functions of concrete coarse-grained literary texts are found and their universal power-law behavior at long distances is revealed.

18. Bayesian prediction of future ice sheet volume using local approximation Markov chain Monte Carlo methods

Science.gov (United States)

Davis, A. D.; Heimbach, P.; Marzouk, Y.

2017-12-01

We develop a Bayesian inverse modeling framework for predicting future ice sheet volume with associated formal uncertainty estimates. Marine ice sheets are drained by fast-flowing ice streams, which we simulate using a flowline model. Flowline models depend on geometric parameters (e.g., basal topography), parameterized physical processes (e.g., calving laws and basal sliding), and climate parameters (e.g., surface mass balance), most of which are unknown or uncertain. Given observations of ice surface velocity and thickness, we define a Bayesian posterior distribution over static parameters, such as basal topography. We also define a parameterized distribution over variable parameters, such as future surface mass balance, which we assume are not informed by the data. Hyperparameters are used to represent climate change scenarios, and sampling their distributions mimics internal variation. For example, a warming climate corresponds to increasing mean surface mass balance but an individual sample may have periods of increasing or decreasing surface mass balance. We characterize the predictive distribution of ice volume by evaluating the flowline model given samples from the posterior distribution and the distribution over variable parameters. Finally, we determine the effect of climate change on future ice sheet volume by investigating how changing the hyperparameters affects the predictive distribution. We use state-of-the-art Bayesian computation to address computational feasibility. Characterizing the posterior distribution (using Markov chain Monte Carlo), sampling the full range of variable parameters and evaluating the predictive model is prohibitively expensive. Furthermore, the required resolution of the inferred basal topography may be very high, which is often challenging for sampling methods. Instead, we leverage regularity in the predictive distribution to build a computationally cheaper surrogate over the low dimensional quantity of interest (future ice

19. Multilevel markov chain monte carlo method for high-contrast single-phase flow problems

KAUST Repository

Efendiev, Yalchin R.

2014-12-19

In this paper we propose a general framework for the uncertainty quantification of quantities of interest for high-contrast single-phase flow problems. It is based on the generalized multiscale finite element method (GMsFEM) and multilevel Monte Carlo (MLMC) methods. The former provides a hierarchy of approximations of different resolution, whereas the latter gives an efficient way to estimate quantities of interest using samples on different levels. The number of basis functions in the online GMsFEM stage can be varied to determine the solution resolution and the computational cost, and to efficiently generate samples at different levels. In particular, it is cheap to generate samples on coarse grids but with low resolution, and it is expensive to generate samples on fine grids with high accuracy. By suitably choosing the number of samples at different levels, one can leverage the expensive computation in larger fine-grid spaces toward smaller coarse-grid spaces, while retaining the accuracy of the final Monte Carlo estimate. Further, we describe a multilevel Markov chain Monte Carlo method, which sequentially screens the proposal with different levels of approximations and reduces the number of evaluations required on fine grids, while combining the samples at different levels to arrive at an accurate estimate. The framework seamlessly integrates the multiscale features of the GMsFEM with the multilevel feature of the MLMC methods following the work in [26], and our numerical experiments illustrate its efficiency and accuracy in comparison with standard Monte Carlo estimates. © Global Science Press Limited 2015.

20. Combination of Markov chain and optimal control solved by Pontryagin’s Minimum Principle for a fuel cell/supercapacitor vehicle

International Nuclear Information System (INIS)

Hemi, Hanane; Ghouili, Jamel; Cheriti, Ahmed

2015-01-01

Highlights: • A combination of Markov chain and an optimal control solved by Pontryagin’s Minimum Principle is presented. • This strategy is applied to hybrid electric vehicle dynamic model. • The hydrogen consumption is analyzed for two different vehicle mass and drive cycle. • The supercapacitor and fuel cell behavior is analyzed at high or sudden required power. - Abstract: In this article, a real time optimal control strategy based on Pontryagin’s Minimum Principle (PMP) combined with the Markov chain approach is used for a fuel cell/supercapacitor electrical vehicle. In real time, at high power and at high speed, two phenomena are observed. The first is obtained at higher required power, and the second is observed at sudden power demand. To avoid these situations, the Markov chain model is proposed to predict the future power demand during a driving cycle. The optimal control problem is formulated as an equivalent consumption minimization strategy (ECMS), that has to be solved by using the Pontryagin’s Minimum Principle. A Markov chain model is added as a separate block for a prediction of required power. This approach and the whole system are modeled and implemented using the MATLAB/Simulink. The model without Markov chain block and the model is with it are compared. The results presented demonstrate the importance of a Markov chain block added to a model

1. Discrete time Markov chains (DTMC) susceptible infected susceptible (SIS) epidemic model with two pathogens in two patches

Science.gov (United States)

Lismawati, Eka; Respatiwulan; Widyaningsih, Purnami

2017-06-01

The SIS epidemic model describes the pattern of disease spread with characteristics that recovered individuals can be infected more than once. The number of susceptible and infected individuals every time follows the discrete time Markov process. It can be represented by the discrete time Markov chains (DTMC) SIS. The DTMC SIS epidemic model can be developed for two pathogens in two patches. The aims of this paper are to reconstruct and to apply the DTMC SIS epidemic model with two pathogens in two patches. The model was presented as transition probabilities. The application of the model obtain that the number of susceptible individuals decreases while the number of infected individuals increases for each pathogen in each patch.

2. Prediction on Human Resource Supply/Demand in Nuclear Industry Using Markov Chains Model and Job Coefficient

International Nuclear Information System (INIS)

Kwon, Hyuk; Min, Byung Joo; Lee, Eui Jin; You, Byung Hoon

2006-01-01

According to the recent report by the OECD/NEA, there is a large imbalance between supply and demand of human resource in nuclear field. In the U.S., according to survey of Nuclear Engineering Department Heads Organization (NEDHO), 174 graduates in B.S or M.S degree were fed to nuclear industry in year 2004. Meanwhile, the total amount of demand in nuclear industry was about 642 engineers, which was approximately three times of the supply. In case of other developed western nations, the OECD/NEA report stated that the level of imbalance is similar to that of the U.S. However, nations having nuclear power development programs such as Korea, Japan and France seem to be in a different environment of supply and demand from that of the U.S. In this study, the difference of manpower status between the U.S and Korea has been investigated and the nuclear manpower required for the future in Korea is predicted. To investigate the factors making difference between the U.S. and NPP developing countries including Korea, a quantitative manpower planning model, Markov chains model, is applied. Since the Markov chains model has the strength of analyzing an inflow or push structure, the model fits the system governed by the inflow of manpower. A macroscopic status of manpower demand on nuclear industry is calculated up to 2015 using the Job coefficient (JC) and GDP, which are derived from the Survey for Roadmap of Electric Power Industry Manpower Planning. Furthermore, the total numbers of required manpower and supplied manpower up to 2030 were predicted by JC and Markov Chains model, respectively. Whereas the employee status of nuclear industries has been annually investigated by KAIF since 1995, the following data from the 10 th survey and nuclear energy yearbooks from 1998 to 2005 are applied; (a) the status of the manpower demand of industry, (b) number of students entering, graduating and getting job in nuclear engineering

3. Kajian Perubahan Penggunaan Lahan Berbasis Citra Satelit Penginderaan Jauh Resolusi Menengah dengan Metode Multi Layer Perceptrondan Markov Chain

Directory of Open Access Journals (Sweden)

Diana Wisnu Wardani

2016-10-01

Full Text Available Perkembangan Kota Yogyakarta yang sangat pesat mempengaruhi perubahan penggunaan lahan di daerah urban fringe salah satunya adalah Kabupaten Bantul. Perubahan penggunaan lahan yang tidak terkendali akan menimbulkan masalah sosial ekonomi dan lingkungan.Penelitian mengenai kajian perubahan penggunaan lahan berbasis citra satelit penginderaan jauh resolusi menengah dengan metode Multi Layer Perceptron dan Markov Chain di sebagian Kabupaten Bantul ini bertujuan: (amengkaji kemampuan citra satelit penginderaan jauh resolusi menengah Landsat untuk ekstraksi informasi penggunaan lahan tahun 2002, 2009 dan 2013, (b mengkaji perubahan penggunaan lahan secara spasial dari tahun 2002 sampai dengan tahun 2009 serta (c menyusun pemodelan perubahan penggunaan lahan dengan metode Multi Layer Perceptron dan Markov Chain pada tahun 2013 dengan mempertimbangkan faktor-faktor yang mempengaruhi perubahan penggunaan lahan berdasar penggunaan lahan 2002-2009.Faktor-faktor yang mempengaruhi perubahan (variabel perubahan penggunaan lahan meliputi aksesbilitas (kepadatan jalan, jarak terhadap jalan, jarak terhadap sungai dan kesesuaian lahan (kemiringan lereng. Citra penginderaan jauh multitemporal dalam penelitian ini menggunakan citra Landsat 5 TM  tahun 2002, Landsat 7 ETM+ tahun 2009 dan Landsat 8 OLI  tahun 2013. Penelitian ini menggunakan klasifikasi multispektral dengan metode maximum likelihood. Klasifikasi multispektral menghasilkan peta penutup lahan (2002, 2009, 2013 yang selanjutnya diturunkan menjadi peta penggunaan lahan. Hasil perubahan penggunaan lahan 2002-2009 selanjutnya dikaji dan diintegrasikan dengan variabel perubahan sebagai input dalam regresi non linear dengan Multi Layer Perceptron. Besar probabilitas perubahan ditentukan dengan metode Markov Chain. Hasil penelitian menunjukkan perubahan penggunaan lahan dari lahan pertanian menjadi permukiman pada periode tahun 2002-2009 seluas 2.766,78ha. Perubahan terluas terjadi di Kecamatan Banguntapan

4. Simulation from endpoint-conditioned, continuous-time Markov chains on a finite state space, with applications to molecular evolution

DEFF Research Database (Denmark)

Hobolth, Asger; Stone, Eric

2009-01-01

Analyses of serially-sampled data often begin with the assumption that the observations represent discrete samples from a latent continuous-time stochastic process. The continuous-time Markov chain (CTMC) is one such generative model whose popularity extends to a variety of disciplines ranging from...... computational finance to human genetics and genomics. A common theme among these diverse applications is the need to simulate sample paths of a CTMC conditional on realized data that is discretely observed. Here we present a general solution to this sampling problem when the CTMC is defined on a discrete...

5. A Markov chain description of the stepwise mutation model: local and global behaviour of the allele process.

Science.gov (United States)

Caliebe, Amke; Jochens, Arne; Krawczak, Michael; Rösler, Uwe

2010-09-21

The stepwise mutation model (SMM) is a simple, widely used model to describe the evolutionary behaviour of microsatellites. We apply a Markov chain description of the SMM and derive the marginal and joint properties of this process. In addition to the standard SMM, we also consider the normalised allele process. In contrast to the standard process, the normalised process converges to a stationary distribution. We show that the marginal stationary distribution is unimodal. The standard and normalised processes capture the global and the local behaviour of the SMM, respectively. Copyright (c) 2010 Elsevier Ltd. All rights reserved.

6. A vendor managed inventory model using continuous approximations for route length estimates and Markov chain modeling for cost estimates

DEFF Research Database (Denmark)

Larsen, Christian; Turkensteen, Marcel

2014-01-01

We consider a vendor that supplies a set of geographically dispersed retailers and that can monitor the inventory levels at the retailers. Such an arrangement is often called Vendor Managed Inventory (VMI). The dispatch of a vehicle is made to a fixed group of retailers. Normally, the inventory...... own approximation for one-dimensional ones. We choose to use a Markov chain approach to minimize transport and inventory model simultaneously. When the routes through the retailers are not fixed, such an approach would require a large number of states if we keep track of all the inventory positions...

7. Parallel GPGPU Evaluation of Small Angle X-ray Scattering Profiles in a Markov Chain Monte Carlo Framework

DEFF Research Database (Denmark)

Antonov, Lubomir Dimitrov; Andreetta, Christian; Hamelryck, Thomas Wim

2013-01-01

directly determines the complexity of the systems that can be explored. We present an efficient implementation of the forward model for SAXS with full hardware utilization of Graphics Processor Units (GPUs). The proposed algorithm is orders of magnitude faster than an efficient CPU implementation....... To infer protein structure from SAXS data, it is necessary to calculate the expected experimental observations given a protein structure, by making use of a so-called forward model. This calculation needs to be performed many times during a conformational search. Therefore, computational efficiency......, and implements a caching procedure employed in the partial forward model evaluations within a Markov chain Monte Carlo framework....

8. Changes in mangrove species assemblages and future prediction of the Bangladesh Sundarbans using Markov chain model and cellular automata.

Science.gov (United States)

Mukhopadhyay, Anirban; Mondal, Parimal; Barik, Jyotiskona; Chowdhury, S M; Ghosh, Tuhin; Hazra, Sugata

2015-06-01

The composition and assemblage of mangroves in the Bangladesh Sundarbans are changing systematically in response to several environmental factors. In order to understand the impact of the changing environmental conditions on the mangrove forest, species composition maps for the years 1985, 1995 and 2005 were studied. In the present study, 1985 and 1995 species zonation maps were considered as base data and the cellular automata-Markov chain model was run to predict the species zonation for the year 2005. The model output was validated against the actual dataset for 2005 and calibrated. Finally, using the model, mangrove species zonation maps for the years 2025, 2055 and 2105 have been prepared. The model was run with the assumption that the continuation of the current tempo and mode of drivers of environmental factors (temperature, rainfall, salinity change) of the last two decades will remain the same in the next few decades. Present findings show that the area distribution of the following species assemblages like Goran (Ceriops), Sundari (Heritiera), Passur (Xylocarpus), and Baen (Avicennia) would decrease in the descending order, whereas the area distribution of Gewa (Excoecaria), Keora (Sonneratia) and Kankra (Bruguiera) dominated assemblages would increase. The spatial distribution of projected mangrove species assemblages shows that more salt tolerant species will dominate in the future; which may be used as a proxy to predict the increase of salinity and its spatial variation in Sundarbans. Considering the present rate of loss of forest land, 17% of the total mangrove cover is predicted to be lost by the year 2105 with a significant loss of fresh water loving mangroves and related ecosystem services. This paper describes a unique approach to assess future changes in species composition and future forest zonation in mangroves under the 'business as usual' scenario of climate change.

9. Analysis of an on-line algorithm for solving large Markov chains

NARCIS (Netherlands)

Litvak, Nelli; Robert, Philippe

2008-01-01

Algorithms for ranking of web pages such as Google Page-Rank assign importance scores according to a stationary distribution of a Markov random walk on the web graph. Although in the classical search scheme the ranking scores are pre-computed off-line, several challenging problems in contemporary

10. A Markov chain approach to modelling charge exchange processes of an ion beam in monotonically increasing or decreasing potentials

International Nuclear Information System (INIS)

Shrier, O; Khachan, J; Bosi, S

2006-01-01

A Markov chain method is presented as an alternative approach to Monte Carlo simulations of charge exchange collisions by an energetic hydrogen ion beam with a cold background hydrogen gas. This method was used to determine the average energy of the resulting energetic neutrals along the path of the beam. A comparison with Monte Carlo modelling showed a good agreement but with the advantage that it required much less computing time and produced no numerical noise. In particular, the Markov chain method works well for monotonically increasing or decreasing electrostatic potentials. Finally, a good agreement is obtained with experimental results from Doppler shift spectroscopy on energetic beams from a hollow cathode discharge. In particular, the average energy of ions that undergo charge exchange reaches a plateau that can be well below the full energy that might be expected from the applied voltage bias, depending on the background gas pressure. For example, pressures of ∼20 mTorr limit the ion energy to ∼20% of the applied voltage

11. A Markov chain Monte Carlo (MCMC) methodology with bootstrap percentile estimates for predicting presidential election results in Ghana.

Science.gov (United States)

Nortey, Ezekiel N N; Ansah-Narh, Theophilus; Asah-Asante, Richard; Minkah, Richard

2015-01-01

Although, there exists numerous literature on the procedure for forecasting or predicting election results, in Ghana only opinion poll strategies have been used. To fill this gap, the paper develops Markov chain models for forecasting the 2016 presidential election results at the Regional, Zonal (i.e. Savannah, Coastal and Forest) and the National levels using past presidential election results of Ghana. The methodology develops a model for prediction of the 2016 presidential election results in Ghana using the Markov chains Monte Carlo (MCMC) methodology with bootstrap estimates. The results were that the ruling NDC may marginally win the 2016 Presidential Elections but would not obtain the more than 50 % votes to be declared an outright winner. This means that there is going to be a run-off election between the two giant political parties: the ruling NDC and the major opposition party, NPP. The prediction for the 2016 Presidential run-off election between the NDC and the NPP was rather in favour of the major opposition party, the NPP with a little over the 50 % votes obtained.

12. Time-dependent earthquake hazard evaluation in seismogenic systems using mixed Markov Chains: An application to the Japan area

Science.gov (United States)

Herrera, C.; Nava, F. A.; Lomnitz, C.

2006-08-01

A previous work introduced a new method for seismic hazard evaluation in a system (a geographic area with distinct, but related seismogenic regions) based on modeling the transition probabilities of states (patterns of presence or absence of seismicity, with magnitude greater or equal to a threshold magnitude Mr, in the regions of the system, during a time interval Δt) as a Markov chain. Application of this direct method to the Japan area gave very good results. Given that the most important limitation of the direct method is the relative scarcity of large magnitude events, we decided to explore the possibility that seismicity with magnitude M ≥ Mmr contains information about the future occurrence of earthquakes with M ≥ Mmr > Mmr. This mixed Markov chain method estimates the probabilities of occurrence of a system state for M ≥ MMr on the basis of the observed state for M ≥ Mmr in the previous Δt. Application of the mixed method to the area of Japan gives better hazard estimations than the direct method; in particular for large earthquakes. As part of this study, the problem of performance evaluation of hazard estimation methods is addressed, leading to the use of grading functions.

13. Adjoint sensitivity analysis of dynamic reliability models based on Markov chains - II: Application to IFMIF reliability assessment

International Nuclear Information System (INIS)

Cacuci, D. G.; Cacuci, D. G.; Balan, I.; Ionescu-Bujor, M.

2008-01-01

In Part II of this work, the adjoint sensitivity analysis procedure developed in Part I is applied to perform sensitivity analysis of several dynamic reliability models of systems of increasing complexity, culminating with the consideration of the International Fusion Materials Irradiation Facility (IFMIF) accelerator system. Section II presents the main steps of a procedure for the automated generation of Markov chains for reliability analysis, including the abstraction of the physical system, construction of the Markov chain, and the generation and solution of the ensuing set of differential equations; all of these steps have been implemented in a stand-alone computer code system called QUEFT/MARKOMAG-S/MCADJSEN. This code system has been applied to sensitivity analysis of dynamic reliability measures for a paradigm '2-out-of-3' system comprising five components and also to a comprehensive dynamic reliability analysis of the IFMIF accelerator system facilities for the average availability and, respectively, the system's availability at the final mission time. The QUEFT/MARKOMAG-S/MCADJSEN has been used to efficiently compute sensitivities to 186 failure and repair rates characterizing components and subsystems of the first-level fault tree of the IFMIF accelerator system. (authors)

14. Markov Chains for Investigating and Predicting Migration: A Case from Southwestern China

Science.gov (United States)

Qin, Bo; Wang, Yiyu; Xu, Haoming

2018-03-01

In order to accurately predict the population’s happiness, this paper conducted two demographic surveys on a new district of a city in western China, and carried out a dynamic analysis using related mathematical methods. This paper argues that the migration of migrants in the city will change the pattern of spatial distribution of human resources in the city and thus affect the social and economic development in all districts. The migration status of the population will change randomly with the passage of time, so it can be predicted and analyzed through the Markov process. The Markov process provides the local government and decision-making bureau a valid basis for the dynamic analysis of the mobility of migrants in the city as well as the ways for promoting happiness of local people’s lives.

15. The Study of Reinforcement Learning for Traffic Self-Adaptive Control under Multiagent Markov Game Environment

Directory of Open Access Journals (Sweden)

Lun-Hui Xu

2013-01-01

Full Text Available Urban traffic self-adaptive control problem is dynamic and uncertain, so the states of traffic environment are hard to be observed. Efficient agent which controls a single intersection can be discovered automatically via multiagent reinforcement learning. However, in the majority of the previous works on this approach, each agent needed perfect observed information when interacting with the environment and learned individually with less efficient coordination. This study casts traffic self-adaptive control as a multiagent Markov game problem. The design employs traffic signal control agent (TSCA for each signalized intersection that coordinates with neighboring TSCAs. A mathematical model for TSCAs’ interaction is built based on nonzero-sum markov game which has been applied to let TSCAs learn how to cooperate. A multiagent Markov game reinforcement learning approach is constructed on the basis of single-agent Q-learning. This method lets each TSCA learn to update its Q-values under the joint actions and imperfect information. The convergence of the proposed algorithm is analyzed theoretically. The simulation results show that the proposed method is convergent and effective in realistic traffic self-adaptive control setting.

16. A Markov Chain Monte Carlo Inversion Approach For Inverting InSAR Data With Application To Subsurface CO2 Injection

Science.gov (United States)

Ramirez, A. L.; Foxall, W.

2011-12-01

Surface displacements caused by reservoir pressure perturbations resulting from CO2 injection can often be measured by geodetic methods such as InSAR, tilt and GPS. We have developed a Markov Chain Monte Carlo (MCMC) approach to invert surface displacements measured by InSAR to map the pressure distribution associated with CO2 injection at the In Salah Krechba field, Algeria. The MCMC inversion entails sampling the solution space by proposing a series of trial 3D pressure-plume models. In the case of In Salah, the range of allowable models is constrained by prior information provided by well and geophysical data for the reservoir and possible fluid pathways in the overburden, and injection pressures and volumes. Each trial pressure distribution source is run through a (mathematical) forward model to calculate a set of synthetic surface deformation data. The likelihood that a particular proposal represents the true source is determined from the fit of the calculated data to the InSAR measurements, and those having higher likelihoods are passed to the posterior distribution. This procedure is repeated over typically ~104 - 105 trials until the posterior distribution converges to a stable solution. The solution to each stochastic inversion is in the form of Bayesian posterior probability density function (pdf) over the range of the alternative models that are consistent with the measured data and prior information. Therefore, the solution provides not only the highest likelihood model but also a realistic estimate of the solution uncertainty. Our InSalah work considered three flow model alternatives: 1) The first model assumed that the CO2 saturation and fluid pressure changes were confined to the reservoir; 2) the second model allowed the perturbations to occur also in a damage zone inferred in the lower caprock from 3D seismic surveys; and 3) the third model allowed fluid pressure changes anywhere within the reservoir and overburden. Alternative (2) yielded optimal

17. Bayesian inversion of seismic and electromagnetic data for marine gas reservoir characterization using multi-chain Markov chain Monte Carlo sampling

Energy Technology Data Exchange (ETDEWEB)

Ren, Huiying; Ray, Jaideep; Hou, Zhangshuan; Huang, Maoyi; Bao, Jie; Swiler, Laura

2017-12-01

In this study we developed an efficient Bayesian inversion framework for interpreting marine seismic amplitude versus angle (AVA) and controlled source electromagnetic (CSEM) data for marine reservoir characterization. The framework uses a multi-chain Markov-chain Monte Carlo (MCMC) sampler, which is a hybrid of DiffeRential Evolution Adaptive Metropolis (DREAM) and Adaptive Metropolis (AM) samplers. The inversion framework is tested by estimating reservoir-fluid saturations and porosity based on marine seismic and CSEM data. The multi-chain MCMC is scalable in terms of the number of chains, and is useful for computationally demanding Bayesian model calibration in scientific and engineering problems. As a demonstration, the approach is used to efficiently and accurately estimate the porosity and saturations in a representative layered synthetic reservoir. The results indicate that the seismic AVA and CSEM joint inversion provides better estimation of reservoir saturations than the seismic AVA-only inversion, especially for the parameters in deep layers. The performance of the inversion approach for various levels of noise in observational data was evaluated – reasonable estimates can be obtained with noise levels up to 25%. Sampling efficiency due to the use of multiple chains was also checked and was found to have almost linear scalability.

18. Optimizing the Loads of multi-player online game Servers using Markov Chains

DEFF Research Database (Denmark)

Saeed, Aamir; Olsen, Rasmus Løvenstein; Pedersen, Jens Myrup

2015-01-01

Multiplayer online games are on the rise with millions of registered player and hundreds of thousands concurrent players. Current state of the art servers achieve scalability by splitting the game world into linked mini worlds that can be hosted on separate servers. One of the problems is that...... that need to be considered when developing load balancing algorithm, that is the reliability of the information that is shared. Simulation results show that Markov based prediction of load information performed better from the normal load status information sharing....

19. Three-dimensional modeling of chromatin structure from interaction frequency data using Markov chain Monte Carlo sampling

Directory of Open Access Journals (Sweden)

Dostie Josée

2011-10-01

Full Text Available Abstract Background Long-range interactions between regulatory DNA elements such as enhancers, insulators and promoters play an important role in regulating transcription. As chromatin contacts have been found throughout the human genome and in different cell types, spatial transcriptional control is now viewed as a general mechanism of gene expression regulation. Chromosome Conformation Capture Carbon Copy (5C and its variant Hi-C are techniques used to measure the interaction frequency (IF between specific regions of the genome. Our goal is to use the IF data generated by these experiments to computationally model and analyze three-dimensional chromatin organization. Results We formulate a probabilistic model linking 5C/Hi-C data to physical distances and describe a Markov chain Monte Carlo (MCMC approach called MCMC5C to generate a representative sample from the posterior distribution over structures from IF data. Structures produced from parallel MCMC runs on the same dataset demonstrate that our MCMC method mixes quickly and is able to sample from the posterior distribution of structures and find subclasses of structures. Structural properties (base looping, condensation, and local density were defined and their distribution measured across the ensembles of structures generated. We applied these methods to a biological model of human myelomonocyte cellular differentiation and identified distinct chromatin conformation signatures (CCSs corresponding to each of the cellular states. We also demonstrate the ability of our method to run on Hi-C data and produce a model of human chromosome 14 at 1Mb resolution that is consistent with previously observed structural properties as measured by 3D-FISH. Conclusions We believe that tools like MCMC5C are essential for the reliable analysis of data from the 3C-derived techniques such as 5C and Hi-C. By integrating complex, high-dimensional and noisy datasets into an easy to interpret ensemble of three

20. Three-dimensional modeling of chromatin structure from interaction frequency data using Markov chain Monte Carlo sampling.

Science.gov (United States)

Rousseau, Mathieu; Fraser, James; Ferraiuolo, Maria A; Dostie, Josée; Blanchette, Mathieu

2011-10-25

Long-range interactions between regulatory DNA elements such as enhancers, insulators and promoters play an important role in regulating transcription. As chromatin contacts have been found throughout the human genome and in different cell types, spatial transcriptional control is now viewed as a general mechanism of gene expression regulation. Chromosome Conformation Capture Carbon Copy (5C) and its variant Hi-C are techniques used to measure the interaction frequency (IF) between specific regions of the genome. Our goal is to use the IF data generated by these experiments to computationally model and analyze three-dimensional chromatin organization. We formulate a probabilistic model linking 5C/Hi-C data to physical distances and describe a Markov chain Monte Carlo (MCMC) approach called MCMC5C to generate a representative sample from the posterior distribution over structures from IF data. Structures produced from parallel MCMC runs on the same dataset demonstrate that our MCMC method mixes quickly and is able to sample from the posterior distribution of structures and find subclasses of structures. Structural properties (base looping, condensation, and local density) were defined and their distribution measured across the ensembles of structures generated. We applied these methods to a biological model of human myelomonocyte cellular differentiation and identified distinct chromatin conformation signatures (CCSs) corresponding to each of the cellular states. We also demonstrate the ability of our method to run on Hi-C data and produce a model of human chromosome 14 at 1Mb resolution that is consistent with previously observed structural properties as measured by 3D-FISH. We believe that tools like MCMC5C are essential for the reliable analysis of data from the 3C-derived techniques such as 5C and Hi-C. By integrating complex, high-dimensional and noisy datasets into an easy to interpret ensemble of three-dimensional conformations, MCMC5C allows researchers to

1. Discrete probability models and methods probability on graphs and trees, Markov chains and random fields, entropy and coding

CERN Document Server

Brémaud, Pierre

2017-01-01

The emphasis in this book is placed on general models (Markov chains, random fields, random graphs), universal methods (the probabilistic method, the coupling method, the Stein-Chen method, martingale methods, the method of types) and versatile tools (Chernoff's bound, Hoeffding's inequality, Holley's inequality) whose domain of application extends far beyond the present text. Although the examples treated in the book relate to the possible applications, in the communication and computing sciences, in operations research and in physics, this book is in the first instance concerned with theory. The level of the book is that of a beginning graduate course. It is self-contained, the prerequisites consisting merely of basic calculus (series) and basic linear algebra (matrices). The reader is not assumed to be trained in probability since the first chapters give in considerable detail the background necessary to understand the rest of the book. .

2. Unfolding neutron spectrum with Markov Chain Monte Carlo at MIT research Reactor with He-3 Neutral Current Detectors

Science.gov (United States)

Leder, A.; Anderson, A. J.; Billard, J.; Figueroa-Feliciano, E.; Formaggio, J. A.; Hasselkus, C.; Newman, E.; Palladino, K.; Phuthi, M.; Winslow, L.; Zhang, L.

2018-02-01

The Ricochet experiment seeks to measure Coherent (neutral-current) Elastic Neutrino-Nucleus Scattering (CEνNS) using dark-matter-style detectors with sub-keV thresholds placed near a neutrino source, such as the MIT (research) Reactor (MITR), which operates at 5.5 MW generating approximately 2.2 × 1018 ν/second in its core. Currently, Ricochet is characterizing the backgrounds at MITR, the main component of which comes in the form of neutrons emitted from the core simultaneous with the neutrino signal. To characterize this background, we wrapped Bonner cylinders around a 32He thermal neutron detector, whose data was then unfolded via a Markov Chain Monte Carlo (MCMC) to produce a neutron energy spectrum across several orders of magnitude. We discuss the resulting spectrum and its implications for deploying Ricochet at the MITR site as well as the feasibility of reducing this background level via the addition of polyethylene shielding around the detector setup.

3. Application of Stochastic Automata Networks for Creation of Continuous Time Markov Chain Models of Voltage Gating of Gap Junction Channels

Directory of Open Access Journals (Sweden)

Mindaugas Snipas

2015-01-01

Full Text Available The primary goal of this work was to study advantages of numerical methods used for the creation of continuous time Markov chain models (CTMC of voltage gating of gap junction (GJ channels composed of connexin protein. This task was accomplished by describing gating of GJs using the formalism of the stochastic automata networks (SANs, which allowed for very efficient building and storing of infinitesimal generator of the CTMC that allowed to produce matrices of the models containing a distinct block structure. All of that allowed us to develop efficient numerical methods for a steady-state solution of CTMC models. This allowed us to accelerate CPU time, which is necessary to solve CTMC models, ∼20 times.

4. Computing the Stackelberg/Nash equilibria using the extraproximal method: Convergence analysis and implementation details for Markov chains games

Directory of Open Access Journals (Sweden)

Trejo Kristal K.

2015-06-01

Full Text Available In this paper we present the extraproximal method for computing the Stackelberg/Nash equilibria in a class of ergodic controlled finite Markov chains games. We exemplify the original game formulation in terms of coupled nonlinear programming problems implementing the Lagrange principle. In addition, Tikhonov’s regularization method is employed to ensure the convergence of the cost-functions to a Stackelberg/Nash equilibrium point. Then, we transform the problem into a system of equations in the proximal format. We present a two-step iterated procedure for solving the extraproximal method: (a the first step (the extra-proximal step consists of a “prediction” which calculates the preliminary position approximation to the equilibrium point, and (b the second step is designed to find a “basic adjustment” of the previous prediction. The procedure is called the “extraproximal method” because of the use of an extrapolation. Each equation in this system is an optimization problem for which the necessary and efficient condition for a minimum is solved using a quadratic programming method. This solution approach provides a drastically quicker rate of convergence to the equilibrium point. We present the analysis of the convergence as well the rate of convergence of the method, which is one of the main results of this paper. Additionally, the extraproximal method is developed in terms of Markov chains for Stackelberg games. Our goal is to analyze completely a three-player Stackelberg game consisting of a leader and two followers. We provide all the details needed to implement the extraproximal method in an efficient and numerically stable way. For instance, a numerical technique is presented for computing the first step parameter (λ of the extraproximal method. The usefulness of the approach is successfully demonstrated by a numerical example related to a pricing oligopoly model for airlines companies.

5. Structure of polymer chains under confinement

cluded volume interactions (so-called regime of “semi-dilute cigars”). For confined charged polymers, a peak is observed whose intensity increases with molecular weight and the asymptotic 1/q scattering region is extended compared to the bulk. We infer that the chains are sufficiently extended, under the influence of ...

6. Transitions of tethered chain molecules under tension.

Science.gov (United States)

Luettmer-Strathmann, Jutta; Binder, Kurt

2014-09-21

An applied tension force changes the equilibrium conformations of a polymer chain tethered to a planar substrate and thus affects the adsorption transition as well as the coil-globule and crystallization transitions. Conversely, solvent quality and surface attraction are reflected in equilibrium force-extension curves that can be measured in experiments. To investigate these effects theoretically, we study tethered chains under tension with Wang-Landau simulations of a bond-fluctuation lattice model. Applying our model to pulling experiments on biological molecules we obtain a good description of experimental data in the intermediate force range, where universal features dominate and finite size effects are small. For tethered chains in poor solvent, we observe the predicted two-phase coexistence at transitions from the globule to stretched conformations and also discover direct transitions from crystalline to stretched conformations. A phase portrait for finite chains constructed by evaluating the density of states for a broad range of solvent conditions and tensions shows how increasing tension leads to a disappearance of the globular phase. For chains in good solvents tethered to hard and attractive surfaces we find the predicted scaling with the chain length in the low-force regime and show that our results are well described by an analytical, independent-bond approximation for the bond-fluctuation model for the highest tensions. Finally, for a hard or slightly attractive surface the stretching of a tethered chain is a conformational change that does not correspond to a phase transition. However, when the surface attraction is sufficient to adsorb a chain it will undergo a desorption transition at a critical value of the applied force. Our results for force-induced desorption show the transition to be discontinuous with partially desorbed conformations in the coexistence region.

7. Inverse modeling of cloud-aerosol interactions — Part 2: Sensitivity tests on liquid phase clouds using a Markov Chain Monte Carlo based simulation approach

NARCIS (Netherlands)

Partridge, D.G.; Vrugt, J.A.; Tunved, P.; Ekman, A.M.L.; Struthers, H.; Sorooshian, A.

2012-01-01

This paper presents a novel approach to investigate cloud-aerosol interactions by coupling a Markov Chain Monte Carlo (MCMC) algorithm to a pseudo-adiabatic cloud parcel model. Despite the number of numerical cloud-aerosol sensitivity studies previously conducted few have used statistical analysis

8. Supply chain network design under uncertainty

DEFF Research Database (Denmark)

Govindan, Kannan; Fattahi, Mohammad; Keyvanshokooh, Esmaeil

2017-01-01

Supply chain network design (SCND) is one of the most crucial planning problems in supply chain management (SCM). Nowadays, design decisions should be viable enough to function well under complex and uncertain business environments for many years or decades. Therefore, it is essential to make...... programming, risk-averse stochastic programming, robust optimization, and fuzzy mathematical programming are explored in terms of mathematical modeling and solution approaches. Finally, the drawbacks and missing aspects of the related literature are highlighted and a list of potential issues for future...

9. An estimator of the survival function based on the semi-Markov model under dependent censorship.

Science.gov (United States)

Lee, Seung-Yeoun; Tsai, Wei-Yann

2005-06-01

Lee and Wolfe (Biometrics vol. 54 pp. 1176-1178, 1998) proposed the two-stage sampling design for testing the assumption of independent censoring, which involves further follow-up of a subset of lost-to-follow-up censored subjects. They also proposed an adjusted estimator for the survivor function for a proportional hazards model under the dependent censoring model. In this paper, a new estimator for the survivor function is proposed for the semi-Markov model under the dependent censorship on the basis of the two-stage sampling data. The consistency and the asymptotic distribution of the proposed estimator are derived. The estimation procedure is illustrated with an example of lung cancer clinical trial and simulation results are reported of the mean squared errors of estimators under a proportional hazards and two different nonproportional hazards models.

10. DREAM(D): an adaptive markov chain monte carlo simulation algorithm to solve discrete, noncontinuous, posterior parameter estimation problems

Science.gov (United States)

Vrugt, J. A.

2011-04-01

Formal and informal Bayesian approaches are increasingly being used to treat forcing, model structural, parameter and calibration data uncertainty, and summarize hydrologic prediction uncertainty. This requires posterior sampling methods that approximate the (evolving) posterior distribution. We recently introduced the DiffeRential Evolution Adaptive Metropolis (DREAM) algorithm, an adaptive Markov Chain Monte Carlo (MCMC) method that is especially designed to solve complex, high-dimensional and multimodal posterior probability density functions. The method runs multiple chains in parallel, and maintains detailed balance and ergodicity. Here, I present the latest algorithmic developments, and introduce a discrete sampling variant of DREAM that samples the parameter space at fixed points. The development of this new code, DREAM(D), has been inspired by the existing class of integer optimization problems, and emerging class of experimental design problems. Such non-continuous parameter estimation problems are of considerable theoretical and practical interest. The theory developed herein is applicable to DREAM(ZS) (Vrugt et al., 2011) and MT-DREAM(ZS) (Laloy and Vrugt, 2011) as well. Two case studies involving a sudoku puzzle and rainfall - runoff model calibration problem are used to illustrate DREAM(D).

11. MARKOV CHAIN MODEL FOR PROBABILITY OF DRY, WET DAYS AND STATISTICAL ANALISIS OF DAILY RAINFALL IN SOME CLIMATIC ZONE OF IRAN

Directory of Open Access Journals (Sweden)

N. SHAHRAKI

2013-03-01

Full Text Available Water scarcity is a major problem in arid and semi-arid areas. The scarcity of water is further stressed by the growing demand due to increase in population growth in developing countries. Climate change and its outcomes on precipitation and water resources is the other problem in these areas. Several models are widely used for modeling daily precipitation occurrence. In this study, Markov Chain Model has been extensively used to study spell distribution. For this purpose, a day period was considered as the optimum length of time. Given the assumption that the Markov chain model is the right model for daily precipitation occurrence, the choice of Markov model order was examined on a daily basis for 4 synoptic weather stations with different climates in Iran (Gorgan, Khorram Abad, Zahedan, Tabrizduring 1978-2009. Based on probability rules, events possibility of sequential dry and wet days, these data were analyzed by stochastic process and Markov Chain method. Then probability matrix was calculated by maximum likelihood method. The possibility continuing2-5days of dry and wet days were calculated. The results showed that the probability maximum of consecutive dry period and climatic probability of dry days has occurred in Zahedan. The probability of consecutive dry period has fluctuated from 73.3 to 100 percent. Climatic probability of occurrence of dry days would change in the range of 70.96 to 100 percent with the average probability of about 90.45 percent.

12. Topology of polymer chains under nanoscale confinement.

Science.gov (United States)

Satarifard, Vahid; Heidari, Maziar; Mashaghi, Samaneh; Tans, Sander J; Ejtehadi, Mohammad Reza; Mashaghi, Alireza

2017-08-24

Spatial confinement limits the conformational space accessible to biomolecules but the implications for bimolecular topology are not yet known. Folded linear biopolymers can be seen as molecular circuits formed by intramolecular contacts. The pairwise arrangement of intra-chain contacts can be categorized as parallel, series or cross, and has been identified as a topological property. Using molecular dynamics simulations, we determine the contact order distributions and topological circuits of short semi-flexible linear and ring polymer chains with a persistence length of l p under a spherical confinement of radius R c . At low values of l p /R c , the entropy of the linear chain leads to the formation of independent contacts along the chain and accordingly, increases the fraction of series topology with respect to other topologies. However, at high l p /R c , the fraction of cross and parallel topologies are enhanced in the chain topological circuits with cross becoming predominant. At an intermediate confining regime, we identify a critical value of l p /R c , at which all topological states have equal probability. Confinement thus equalizes the probability of more complex cross and parallel topologies to the level of the more simple, non-cooperative series topology. Moreover, our topology analysis reveals distinct behaviours for ring- and linear polymers under weak confinement; however, we find no difference between ring- and linear polymers under strong confinement. Under weak confinement, ring polymers adopt parallel and series topologies with equal likelihood, while linear polymers show a higher tendency for series arrangement. The radial distribution analysis of the topology reveals a non-uniform effect of confinement on the topology of polymer chains, thereby imposing more pronounced effects on the core region than on the confinement surface. Additionally, our results reveal that over a wide range of confining radii, loops arranged in parallel and cross

13. Comparison of methods for calculating conditional expectations of sufficient statistics for continuous time Markov chains

DEFF Research Database (Denmark)

Tataru, Paula Cristina; Hobolth, Asger

2011-01-01

on the end-points of the chain, and compare their performance with respect to accuracy and running time. The first algorithm is based on an eigenvalue decomposition of the rate matrix (EVD), the second on uniformization (UNI), and the third on integrals of matrix exponentials (EXPM). The implementation in R...

14. A Markov chain Monte Carlo Expectation Maximization Algorithm for Statistical Analysis of DNA Sequence Evolution with Neighbor-Dependent Substitution Rates

DEFF Research Database (Denmark)

Hobolth, Asger

2008-01-01

-dimensional integrals required in the EM algorithm are estimated using MCMC sampling. The MCMC sampler requires simulation of sample paths from a continuous time Markov process, conditional on the beginning and ending states and the paths of the neighboring sites. An exact path sampling algorithm is developed......The evolution of DNA sequences can be described by discrete state continuous time Markov processes on a phylogenetic tree. We consider neighbor-dependent evolutionary models where the instantaneous rate of substitution at a site depends on the states of the neighboring sites. Neighbor......-dependent substitution models are analytically intractable and must be analyzed using either approximate or simulation-based methods. We describe statistical inference of neighbor-dependent models using a Markov chain Monte Carlo expectation maximization (MCMC-EM) algorithm. In the MCMC-EM algorithm, the high...

15. Modelagem da gestão de estoques de peças de reposição através de cadeias de Markov A model for spare parts stock management using Markov chains

Directory of Open Access Journals (Sweden)

Antonio Vinicius Pimpão Gomes

2008-04-01

Full Text Available Nessa pesquisa é apresentada uma abordagem para gestão de estoques de peças de reposição com base em cadeias de Markov. É feita uma comparação com a simulação convencional, a fim de validar esta abordagem, bem como é apresentada uma heurística para determinação dos parâmetros da política (S, s de gestão de estoques, dado um conjunto de itens de custo (falta, excesso e ressuprimento e de demanda com distribuição Poisson. A análise dos gráficos desses itens de custo em função dos parâmetros da política (S, s fornece os trade-offs básicos para a formulação da heurística.In this study, we propose a model for a management stock system of spare parts using Markov chains. We compare this method with a conventional simulation showing that both methods are equivalent. In addition, we propose heuristics to find the system parameters based on the properties of Markov Chains and graphics related to the costs implied in the stock management of spare parts.

16. Structure of polymer chains under confinement

Single chain form factor was observed both for bulk and confined chains using the condition of zero average contrast. Our measurements on neutral polymer chains are in agreement with the theoretical predictions established by Daoud and de Gennes for chains confined in a cylindrical pore when the chains are entangled ...

17. Inferring soil salinity in a drip irrigation system from multi-configuration EMI measurements using adaptive Markov chain Monte Carlo

Directory of Open Access Journals (Sweden)

2017-10-01

Full Text Available A substantial interpretation of electromagnetic induction (EMI measurements requires quantifying optimal model parameters and uncertainty of a nonlinear inverse problem. For this purpose, an adaptive Bayesian Markov chain Monte Carlo (MCMC algorithm is used to assess multi-orientation and multi-offset EMI measurements in an agriculture field with non-saline and saline soil. In MCMC the posterior distribution is computed using Bayes' rule. The electromagnetic forward model based on the full solution of Maxwell's equations was used to simulate the apparent electrical conductivity measured with the configurations of EMI instrument, the CMD Mini-Explorer. Uncertainty in the parameters for the three-layered earth model are investigated by using synthetic data. Our results show that in the scenario of non-saline soil, the parameters of layer thickness as compared to layers electrical conductivity are not very informative and are therefore difficult to resolve. Application of the proposed MCMC-based inversion to field measurements in a drip irrigation system demonstrates that the parameters of the model can be well estimated for the saline soil as compared to the non-saline soil, and provides useful insight about parameter uncertainty for the assessment of the model outputs.

18. Markov chain analysis of the rainfall patterns of five geographical locations in the south eastern coast of Ghana

Directory of Open Access Journals (Sweden)

Meshach Tettey

2017-08-01

Full Text Available Abstract This study develops an objective rainfall pattern assessment through Markov chain analysis using daily rainfall data from 1980 to 2010, a period of 30 years, for five cities or towns along the south eastern coastal belt of Ghana; Cape Coast, Accra, Akuse, Akatsi and Keta. Transition matrices were computed for each town and each month using the conditional probability of rain or no rain on a particular day given that it rained or did not rain on the previous day. The steady state transition matrices and the steady state probability vectors were also computed for each town and each month. It was found that, the rainy or dry season pattern observed using the monthly steady state rainfall vectors tended to reflect the monthly rainfall time series trajectory. Overall, the probability of rain on any day was low to average: Keta 0.227, Akuse 0.382, Accra 0.467, Cape Coast, 0.50 and Akatsi 0.50. In particular, for Accra, the rainy season was observed to be in the months of May to June and September to October. We also determined that the probability of rainfall generally tended to increase from east to west along the south eastern coast of Ghana.

19. Stochastic dietary restriction using a Markov-chain feeding protocol elicits complex, life history response in medflies

Science.gov (United States)

Carey, James R.; Liedo, Pablo; Müller, Hans-Georg; Wang, Jane-Ling; Zhang, Ying; Harshman, Lawrence

2008-01-01

Summary Lifespan in individually housed medflies (virgins of both sexes) and daily reproduction for females were studied following one of 12 dietary restriction (DR) treatments in which the availability of high-quality food (yeast–sugar mixture) for each fly was based on a Markov chain feeding scheme – a stochastic dietary regime which specifies that the future dietary state depends only on the present dietary state and not on the path by which the present state was achieved. The stochastic treatments consisted of a combination of one of four values of a ‘discovery’ parameter and one of three values of a ‘persistence’ parameter. The results supported the hypotheses that: (i) longevity is extended in most medfly cohorts subject to stochastic DR; and (ii) longevity is more affected by the patch discovery than the patch persistence parameter. One of the main conclusions of the study is that, in combination with the results of earlier dietary restriction studies on the medfly, the results reinforce the concept that the details of the dietary restriction protocols have a profound impact on the sign and magnitude of the longevity extension relative to ad libitum cohorts and that a deeper understanding of the effect of food restriction on longevity is not possible without an understanding of its effect on reproduction. PMID:15659211

20. Application of spatial Markov chains to the analysis of the temporal-spatial evolution of soil erosion.

Science.gov (United States)

Liu, Ruimin; Men, Cong; Wang, Xiujuan; Xu, Fei; Yu, Wenwen

Soil and water conservation in the Three Gorges Reservoir Area of China is important, and soil erosion is a significant issue. In the present study, spatial Markov chains were applied to explore the impacts of the regional context on soil erosion in the Xiangxi River watershed, and Thematic Mapper remote sensing data from 1999 and 2007 were employed. The results indicated that the observed changes in soil erosion were closely related to the soil erosion levels of the surrounding areas. When neighboring regions were not considered, the probability that moderate erosion transformed into slight and severe erosion was 0.8330 and 0.0049, respectively. However, when neighboring regions that displayed intensive erosion were considered, the probabilities were 0.2454 and 0.7513, respectively. Moreover, the different levels of soil erosion in neighboring regions played different roles in soil erosion. If the erosion levels in the neighboring region were lower, the probability of a high erosion class transferring to a lower level was relatively high. In contrast, if erosion levels in the neighboring region were higher, the probability was lower. The results of the present study provide important information for the planning and implementation of soil conservation measures in the study area.

1. Bayesian uncertainty quantification for flows in heterogeneous porous media using reversible jump Markov chain Monte Carlo methods

KAUST Repository

Mondal, A.

2010-03-01

In this paper, we study the uncertainty quantification in inverse problems for flows in heterogeneous porous media. Reversible jump Markov chain Monte Carlo algorithms (MCMC) are used for hierarchical modeling of channelized permeability fields. Within each channel, the permeability is assumed to have a lognormal distribution. Uncertainty quantification in history matching is carried out hierarchically by constructing geologic facies boundaries as well as permeability fields within each facies using dynamic data such as production data. The search with Metropolis-Hastings algorithm results in very low acceptance rate, and consequently, the computations are CPU demanding. To speed-up the computations, we use a two-stage MCMC that utilizes upscaled models to screen the proposals. In our numerical results, we assume that the channels intersect the wells and the intersection locations are known. Our results show that the proposed algorithms are capable of capturing the channel boundaries and describe the permeability variations within the channels using dynamic production history at the wells. © 2009 Elsevier Ltd. All rights reserved.

2. SIMULATION FROM ENDPOINT-CONDITIONED, CONTINUOUS-TIME MARKOV CHAINS ON A FINITE STATE SPACE, WITH APPLICATIONS TO MOLECULAR EVOLUTION.

Science.gov (United States)

Hobolth, Asger; Stone, Eric A

2009-09-01

Analyses of serially-sampled data often begin with the assumption that the observations represent discrete samples from a latent continuous-time stochastic process. The continuous-time Markov chain (CTMC) is one such generative model whose popularity extends to a variety of disciplines ranging from computational finance to human genetics and genomics. A common theme among these diverse applications is the need to simulate sample paths of a CTMC conditional on realized data that is discretely observed. Here we present a general solution to this sampling problem when the CTMC is defined on a discrete and finite state space. Specifically, we consider the generation of sample paths, including intermediate states and times of transition, from a CTMC whose beginning and ending states are known across a time interval of length T. We first unify the literature through a discussion of the three predominant approaches: (1) modified rejection sampling, (2) direct sampling, and (3) uniformization. We then give analytical results for the complexity and efficiency of each method in terms of the instantaneous transition rate matrix Q of the CTMC, its beginning and ending states, and the length of sampling time T. In doing so, we show that no method dominates the others across all model specifications, and we give explicit proof of which method prevails for any given Q, T, and endpoints. Finally, we introduce and compare three applications of CTMCs to demonstrate the pitfalls of choosing an inefficient sampler.

3. Markov chain beam randomization: a study of the impact of PLANCK beam measurement errors on cosmological parameter estimation

Science.gov (United States)

Rocha, G.; Pagano, L.; Górski, K. M.; Huffenberger, K. M.; Lawrence, C. R.; Lange, A. E.

2010-04-01

We introduce a new method to propagate uncertainties in the beam shapes used to measure the cosmic microwave background to cosmological parameters determined from those measurements. The method, called markov chain beam randomization (MCBR), randomly samples from a set of templates or functions that describe the beam uncertainties. The method is much faster than direct numerical integration over systematic “nuisance” parameters, and is not restricted to simple, idealized cases as is analytic marginalization. It does not assume the data are normally distributed, and does not require Gaussian priors on the specific systematic uncertainties. We show that MCBR properly accounts for and provides the marginalized errors of the parameters. The method can be generalized and used to propagate any systematic uncertainties for which a set of templates is available. We apply the method to the Planck satellite, and consider future experiments. Beam measurement errors should have a small effect on cosmological parameters as long as the beam fitting is performed after removal of 1/f noise.

4. A trans-dimensional Bayesian Markov chain Monte Carlo algorithm for model assessment using frequency-domain electromagnetic data

Science.gov (United States)

Minsley, Burke J.

2011-01-01

A meaningful interpretation of geophysical measurements requires an assessment of the space of models that are consistent with the data, rather than just a single, ‘best’ model which does not convey information about parameter uncertainty. For this purpose, a trans-dimensional Bayesian Markov chain Monte Carlo (MCMC) algorithm is developed for assessing frequencydomain electromagnetic (FDEM) data acquired from airborne or ground-based systems. By sampling the distribution of models that are consistent with measured data and any prior knowledge, valuable inferences can be made about parameter values such as the likely depth to an interface, the distribution of possible resistivity values as a function of depth and non-unique relationships between parameters. The trans-dimensional aspect of the algorithm allows the number of layers to be a free parameter that is controlled by the data, where models with fewer layers are inherently favoured, which provides a natural measure of parsimony and a signiﬁcant degree of ﬂexibility in parametrization. The MCMC algorithm is used with synthetic examples to illustrate how the distribution of acceptable models is affected by the choice of prior information, the system geometry and conﬁguration and the uncertainty in the measured system elevation. An airborne FDEM data set that was acquired for the purpose of hydrogeological characterization is also studied. The results compare favorably with traditional least-squares analysis, borehole resistivity and lithology logs from the site, and also provide new information about parameter uncertainty necessary for model assessment.

5. Computation of identity by descent probabilities conditional on DNA markers via a Monte Carlo Markov Chain method

Directory of Open Access Journals (Sweden)

Pérez-Enciso Miguel

2000-09-01

Full Text Available Abstract The accurate estimation of the probability of identity by descent (IBD at loci or genome positions of interest is paramount to the genetic study of quantitative and disease resistance traits. We present a Monte Carlo Markov Chain method to compute IBD probabilities between individuals conditional on DNA markers and on pedigree information. The IBDs can be obtained in a completely general pedigree at any genome position of interest, and all marker and pedigree information available is used. The method can be split into two steps at each iteration. First, phases are sampled using current genotypic configurations of relatives and second, crossover events are simulated conditional on phases. Internal track is kept of all founder origins and crossovers such that the IBD probabilities averaged over replicates are rapidly obtained. We illustrate the method with some examples. First, we show that all pedigree information should be used to obtain line origin probabilities in F2 crosses. Second, the distribution of genetic relationships between half and full sibs is analysed in both simulated data and in real data from an F2 cross in pigs.

6. A Hybrid Secure Scheme for Wireless Sensor Networks against Timing Attacks Using Continuous-Time Markov Chain and Queueing Model.

Science.gov (United States)

Meng, Tianhui; Li, Xiaofan; Zhang, Sha; Zhao, Yubin

2016-09-28

Wireless sensor networks (WSNs) have recently gained popularity for a wide spectrum of applications. Monitoring tasks can be performed in various environments. This may be beneficial in many scenarios, but it certainly exhibits new challenges in terms of security due to increased data transmission over the wireless channel with potentially unknown threats. Among possible security issues are timing attacks, which are not prevented by traditional cryptographic security. Moreover, the limited energy and memory resources prohibit the use of complex security mechanisms in such systems. Therefore, balancing between security and the associated energy consumption becomes a crucial challenge. This paper proposes a secure scheme for WSNs while maintaining the requirement of the security-performance tradeoff. In order to proceed to a quantitative treatment of this problem, a hybrid continuous-time Markov chain (CTMC) and queueing model are put forward, and the tradeoff analysis of the security and performance attributes is carried out. By extending and transforming this model, the mean time to security attributes failure is evaluated. Through tradeoff analysis, we show that our scheme can enhance the security of WSNs, and the optimal rekeying rate of the performance and security tradeoff can be obtained.

7. Kullback-Leibler Divergence-Based Differential Evolution Markov Chain Filter for Global Localization of Mobile Robots.

Science.gov (United States)

Martín, Fernando; Moreno, Luis; Garrido, Santiago; Blanco, Dolores

2015-09-16

One of the most important skills desired for a mobile robot is the ability to obtain its own location even in challenging environments. The information provided by the sensing system is used here to solve the global localization problem. In our previous work, we designed different algorithms founded on evolutionary strategies in order to solve the aforementioned task. The latest developments are presented in this paper. The engine of the localization module is a combination of the Markov chain Monte Carlo sampling technique and the Differential Evolution method, which results in a particle filter based on the minimization of a fitness function. The robot's pose is estimated from a set of possible locations weighted by a cost value. The measurements of the perceptive sensors are used together with the predicted ones in a known map to define a cost function to optimize. Although most localization methods rely on quadratic fitness functions, the sensed information is processed asymmetrically in this filter. The Kullback-Leibler divergence is the basis of a cost function that makes it possible to deal with different types of occlusions. The algorithm performance has been checked in a real map. The results are excellent in environments with dynamic and unmodeled obstacles, a fact that causes occlusions in the sensing area.

8. Kullback-Leibler Divergence-Based Differential Evolution Markov Chain Filter for Global Localization of Mobile Robots

Directory of Open Access Journals (Sweden)

Fernando Martín

2015-09-01

Full Text Available One of the most important skills desired for a mobile robot is the ability to obtain its own location even in challenging environments. The information provided by the sensing system is used here to solve the global localization problem. In our previous work, we designed different algorithms founded on evolutionary strategies in order to solve the aforementioned task. The latest developments are presented in this paper. The engine of the localization module is a combination of the Markov chain Monte Carlo sampling technique and the Differential Evolution method, which results in a particle filter based on the minimization of a fitness function. The robot’s pose is estimated from a set of possible locations weighted by a cost value. The measurements of the perceptive sensors are used together with the predicted ones in a known map to define a cost function to optimize. Although most localization methods rely on quadratic fitness functions, the sensed information is processed asymmetrically in this filter. The Kullback-Leibler divergence is the basis of a cost function that makes it possible to deal with different types of occlusions. The algorithm performance has been checked in a real map. The results are excellent in environments with dynamic and unmodeled obstacles, a fact that causes occlusions in the sensing area.

9. Inferring soil salinity in a drip irrigation system from multi-configuration EMI measurements using adaptive Markov chain Monte Carlo

KAUST Repository

2017-10-26

A substantial interpretation of electromagnetic induction (EMI) measurements requires quantifying optimal model parameters and uncertainty of a nonlinear inverse problem. For this purpose, an adaptive Bayesian Markov chain Monte Carlo (MCMC) algorithm is used to assess multi-orientation and multi-offset EMI measurements in an agriculture field with non-saline and saline soil. In MCMC the posterior distribution is computed using Bayes\\' rule. The electromagnetic forward model based on the full solution of Maxwell\\'s equations was used to simulate the apparent electrical conductivity measured with the configurations of EMI instrument, the CMD Mini-Explorer. Uncertainty in the parameters for the three-layered earth model are investigated by using synthetic data. Our results show that in the scenario of non-saline soil, the parameters of layer thickness as compared to layers electrical conductivity are not very informative and are therefore difficult to resolve. Application of the proposed MCMC-based inversion to field measurements in a drip irrigation system demonstrates that the parameters of the model can be well estimated for the saline soil as compared to the non-saline soil, and provides useful insight about parameter uncertainty for the assessment of the model outputs.

10. Efficient Markov chain Monte Carlo implementation of Bayesian analysis of additive and dominance genetic variances in noninbred pedigrees.

Science.gov (United States)

Waldmann, Patrik; Hallander, Jon; Hoti, Fabian; Sillanpää, Mikko J

2008-06-01

Accurate and fast computation of quantitative genetic variance parameters is of great importance in both natural and breeding populations. For experimental designs with complex relationship structures it can be important to include both additive and dominance variance components in the statistical model. In this study, we introduce a Bayesian Gibbs sampling approach for estimation of additive and dominance genetic variances in the traditional infinitesimal model. The method can handle general pedigrees without inbreeding. To optimize between computational time and good mixing of the Markov chain Monte Carlo (MCMC) chains, we used a hybrid Gibbs sampler that combines a single site and a blocked Gibbs sampler. The speed of the hybrid sampler and the mixing of the single-site sampler were further improved by the use of pretransformed variables. Two traits (height and trunk diameter) from a previously published diallel progeny test of Scots pine (Pinus sylvestris L.) and two large simulated data sets with different levels of dominance variance were analyzed. We also performed Bayesian model comparison on the basis of the posterior predictive loss approach. Results showed that models with both additive and dominance components had the best fit for both height and diameter and for the simulated data with high dominance. For the simulated data with low dominance, we needed an informative prior to avoid the dominance variance component becoming overestimated. The narrow-sense heritability estimates in the Scots pine data were lower compared to the earlier results, which is not surprising because the level of dominance variance was rather high, especially for diameter. In general, the hybrid sampler was considerably faster than the blocked sampler and displayed better mixing properties than the single-site sampler.

11. Bayesian Markov-Chain-Monte-Carlo inversion of time-lapse crosshole GPR data to characterize the vadose zone at the Arrenaes Site, Denmark

DEFF Research Database (Denmark)

Scholer, Marie; Irving, James; Zibar, Majken Caroline Looms

2012-01-01

We examined to what extent time-lapse crosshole ground-penetrating radar traveltimes, measured during a forced infiltration experiment at the Arreneas field site in Denmark, could help to quantify vadose zone hydraulic properties and their corresponding uncertainties using a Bayesian Markov......-chain-Monte-Carlo inversion approach with different priors. The ground-penetrating radar (GPR) geophysical method has the potential to provide valuable information on the hydraulic properties of the vadose zone because of its strong sensitivity to soil water content. In particular, recent evidence has suggested...... in Denmark, could help to quantify VGM parameters and their uncertainties in a layered medium, as well as the corresponding soil hydraulic properties. We used a Bayesian Markov-chain-Monte-Carlo inversion approach. We first explored the advantages and limitations of this approach with regard to a realistic...

12. On Determining the Order of Markov Dependence of an Observed Process Governed by a Hidden Markov Model

OpenAIRE

R.J. Boys; D.A. Henderson

2002-01-01

This paper describes a Bayesian approach to determining the order of a finite state Markov chain whose transition probabilities are themselves governed by a homogeneous finite state Markov chain. It extends previous work on homogeneous Markov chains to more general and applicable hidden Markov models. The method we describe uses a Markov chain Monte Carlo algorithm to obtain samples from the (posterior) distribution for both the order of Markov dependence in the observed sequence and the othe...

13. Effects of stochastic interest rates in decision making under risk: A Markov decision process model for forest management

Science.gov (United States)

Mo Zhou; Joseph Buongiorno

2011-01-01

Most economic studies of forest decision making under risk assume a fixed interest rate. This paper investigated some implications of this stochastic nature of interest rates. Markov decision process (MDP) models, used previously to integrate stochastic stand growth and prices, can be extended to include variable interest rates as well. This method was applied to...

14. SU-E-J-115: Using Markov Chain Modeling to Elucidate Patterns in Breast Cancer Metastasis Over Time and Space

International Nuclear Information System (INIS)

Comen, E; Mason, J; Kuhn, P; Nieva, J; Newton, P; Norton, L; Venkatappa, N; Jochelson, M

2014-01-01

15. SU-E-J-115: Using Markov Chain Modeling to Elucidate Patterns in Breast Cancer Metastasis Over Time and Space

Energy Technology Data Exchange (ETDEWEB)

Comen, E; Mason, J; Kuhn, P [The Scripps Research Institute, La Jolla, CA (United States); Nieva, J [Billings Clinic, Billings, Montana (United States); Newton, P [University of Southern California, Los Angeles, CA (United States); Norton, L; Venkatappa, N; Jochelson, M [Memorial Sloan-Kettering Cancer Center, NY, NY (United States)

2014-06-01

16. A Markov chain model for N-linked protein glycosylation – towards a low-parameter tool for model-driven glycoengineering

DEFF Research Database (Denmark)

Spahn, Philipp N.; Hansen, Anders Holmgaard; Hansen, Henning Gram

2016-01-01

Glycosylation is a critical quality attribute of most recombinant biotherapeutics. Consequently, drug development requires careful control of glycoforms to meet bioactivity and biosafety requirements. However, glycoengineering can be extraordinarily difficult given the complex reaction networks...... present a novel low-parameter approach to describe glycosylation using flux-balance and Markov chain modeling. The model recapitulates the biological complexity of glycosylation, but does not require user-provided kinetic information. We use this method to predict and experimentally validate glycoprofiles...

17. Specification test for Markov models with measurement errors.

Science.gov (United States)

Kim, Seonjin; Zhao, Zhibiao

2014-09-01

Most existing works on specification testing assume that we have direct observations from the model of interest. We study specification testing for Markov models based on contaminated observations. The evolving model dynamics of the unobservable Markov chain is implicitly coded into the conditional distribution of the observed process. To test whether the underlying Markov chain follows a parametric model, we propose measuring the deviation between nonparametric and parametric estimates of conditional regression functions of the observed process. Specifically, we construct a nonparametric simultaneous confidence band for conditional regression functions and check whether the parametric estimate is contained within the band.

18. Phase-coexistence simulations of fluid mixtures by the Markov Chain Monte Carlo method using single-particle models

KAUST Repository

Li, Jun

2013-09-01

We present a single-particle Lennard-Jones (L-J) model for CO2 and N2. Simplified L-J models for other small polyatomic molecules can be obtained following the methodology described herein. The phase-coexistence diagrams of single-component systems computed using the proposed single-particle models for CO2 and N2 agree well with experimental data over a wide range of temperatures. These diagrams are computed using the Markov Chain Monte Carlo method based on the Gibbs-NVT ensemble. This good agreement validates the proposed simplified models. That is, with properly selected parameters, the single-particle models have similar accuracy in predicting gas-phase properties as more complex, state-of-the-art molecular models. To further test these single-particle models, three binary mixtures of CH4, CO2 and N2 are studied using a Gibbs-NPT ensemble. These results are compared against experimental data over a wide range of pressures. The single-particle model has similar accuracy in the gas phase as traditional models although its deviation in the liquid phase is greater. Since the single-particle model reduces the particle number and avoids the time-consuming Ewald summation used to evaluate Coulomb interactions, the proposed model improves the computational efficiency significantly, particularly in the case of high liquid density where the acceptance rate of the particle-swap trial move increases. We compare, at constant temperature and pressure, the Gibbs-NPT and Gibbs-NVT ensembles to analyze their performance differences and results consistency. As theoretically predicted, the agreement between the simulations implies that Gibbs-NVT can be used to validate Gibbs-NPT predictions when experimental data is not available. © 2013 Elsevier Inc.

19. Modeling kinetics of a large-scale fed-batch CHO cell culture by Markov chain Monte Carlo method.

Science.gov (United States)

Xing, Zizhuo; Bishop, Nikki; Leister, Kirk; Li, Zheng Jian

2010-01-01

Markov chain Monte Carlo (MCMC) method was applied to model kinetics of a fed-batch Chinese hamster ovary cell culture process in 5,000-L bioreactors. The kinetic model consists of six differential equations, which describe dynamics of viable cell density and concentrations of glucose, glutamine, ammonia, lactate, and the antibody fusion protein B1 (B1). The kinetic model has 18 parameters, six of which were calculated from the cell culture data, whereas the other 12 were estimated from a training data set that comprised of seven cell culture runs using a MCMC method. The model was confirmed in two validation data sets that represented a perturbation of the cell culture condition. The agreement between the predicted and measured values of both validation data sets may indicate high reliability of the model estimates. The kinetic model uniquely incorporated the ammonia removal and the exponential function of B1 protein concentration. The model indicated that ammonia and lactate play critical roles in cell growth and that low concentrations of glucose (0.17 mM) and glutamine (0.09 mM) in the cell culture medium may help reduce ammonia and lactate production. The model demonstrated that 83% of the glucose consumed was used for cell maintenance during the late phase of the cell cultures, whereas the maintenance coefficient for glutamine was negligible. Finally, the kinetic model suggests that it is critical for B1 production to sustain a high number of viable cells. The MCMC methodology may be a useful tool for modeling kinetics of a fed-batch mammalian cell culture process.

20. Estimation of soil salinity by using Markov Chain Monte Carlo simulation for multi-configuration electromagnetic induction measurements

Science.gov (United States)

Jadoon, K. Z.; Altaf, M. U.; McCabe, M. F.; Hoteit, I.; Moghadas, D.

2014-12-01

In arid and semi-arid regions, soil salinity has a major impact on agro-ecosystems, agricultural productivity, environment and sustainability. High levels of soil salinity adversely affect plant growth and productivity, soil and water quality, and may eventually result in soil erosion and land degradation. Being essentially a hazard, it's important to monitor and map soil salinity at an early stage to effectively use soil resources and maintain soil salinity level below the salt tolerance of crops. In this respect, low frequency electromagnetic induction (EMI) systems can be used as a noninvasive method to map the distribution of soil salinity at the field scale and at a high spatial resolution. In this contribution, an EMI system (the CMD Mini-Explorer) is used to estimate soil salinity using a Bayesian approach implemented via a Markov chain Monte Carlo (MCMC) sampling for inversion of multi-configuration EMI measurements. In-situ and EMI measurements were conducted across a farm where Acacia trees are irrigated with brackish water using a drip irrigation system. The electromagnetic forward model is based on the full solution of Maxwell's equation, and the subsurface is considered as a three-layer problem. In total, five parameters (electrical conductivity of three layers and thickness of top two layers) were inverted and modeled electrical conductivities were converted into the universal standard of soil salinity measurement (i.e. using the method of electrical conductivity of a saturated soil paste extract). Simulation results demonstrate that the proposed scheme successfully recovers soil salinity and reduces the uncertainties in the prior estimate. Analysis of the resulting posterior distribution of parameters indicates that electrical conductivity of the top two layers and the thickness of the first layer are well constrained by the EMI measurements. The proposed approach allows for quantitative mapping and monitoring of the spatial electrical conductivity

1. Corruption of accuracy and efficiency of Markov chain Monte Carlo simulation by inaccurate numerical implementation of conceptual hydrologic models

Science.gov (United States)

Schoups, G.; Vrugt, J. A.; Fenicia, F.; van de Giesen, N. C.

2010-10-01

Conceptual rainfall-runoff models have traditionally been applied without paying much attention to numerical errors induced by temporal integration of water balance dynamics. Reliance on first-order, explicit, fixed-step integration methods leads to computationally cheap simulation models that are easy to implement. Computational speed is especially desirable for estimating parameter and predictive uncertainty using Markov chain Monte Carlo (MCMC) methods. Confirming earlier work of Kavetski et al. (2003), we show here that the computational speed of first-order, explicit, fixed-step integration methods comes at a cost: for a case study with a spatially lumped conceptual rainfall-runoff model, it introduces artificial bimodality in the marginal posterior parameter distributions, which is not present in numerically accurate implementations of the same model. The resulting effects on MCMC simulation include (1) inconsistent estimates of posterior parameter and predictive distributions, (2) poor performance and slow convergence of the MCMC algorithm, and (3) unreliable convergence diagnosis using the Gelman-Rubin statistic. We studied several alternative numerical implementations to remedy these problems, including various adaptive-step finite difference schemes and an operator splitting method. Our results show that adaptive-step, second-order methods, based on either explicit finite differencing or operator splitting with analytical integration, provide the best alternative for accurate and efficient MCMC simulation. Fixed-step or adaptive-step implicit methods may also be used for increased accuracy, but they cannot match the efficiency of adaptive-step explicit finite differencing or operator splitting. Of the latter two, explicit finite differencing is more generally applicable and is preferred if the individual hydrologic flux laws cannot be integrated analytically, as the splitting method then loses its advantage.

2. Improving Speaker Identification Performance Under the Shouted Talking Condition Using the Second-Order Hidden Markov Models

Directory of Open Access Journals (Sweden)

Shahin Ismail

2005-01-01

Full Text Available Speaker identification systems perform well under the neutral talking condition; however, they suffer sharp degradation under the shouted talking condition. In this paper, the second-order hidden Markov models (HMM2s have been used to improve the recognition performance of isolated-word text-dependent speaker identification systems under the shouted talking condition. Our results show that HMM2s significantly improve the speaker identification performance compared to the first-order hidden Markov models (HMM1s. The average speaker identification performance under the shouted talking condition based on HMM1s is . On the other hand, the average speaker identification performance based on HMM2s is .

3. Estimation in autoregressive models with Markov regime

OpenAIRE

Ríos, Ricardo; Rodríguez, Luis

2005-01-01

In this paper we derive the consistency of the penalized likelihood method for the number state of the hidden Markov chain in autoregressive models with Markov regimen. Using a SAEM type algorithm to estimate the models parameters. We test the null hypothesis of hidden Markov Model against an autoregressive process with Markov regime.

4. Stencil method: a Markov model for transport in porous media

Science.gov (United States)

Delgoshaie, A. H.; Tchelepi, H.; Jenny, P.

2016-12-01

In porous media the transport of fluid is dominated by flow-field heterogeneity resulting from the underlying transmissibility field. Since the transmissibility is highly uncertain, many realizations of a geological model are used to describe the statistics of the transport phenomena in a Monte Carlo framework. One possible way to avoid the high computational cost of physics-based Monte Carlo simulations is to model the velocity field as a Markov process and use Markov Chain Monte Carlo. In previous works multiple Markov models for discrete velocity processes have been proposed. These models can be divided into two general classes of Markov models in time and Markov models in space. Both of these choices have been shown to be effective to some extent. However some studies have suggested that the Markov property cannot be confirmed for a temporal Markov process; Therefore there is not a consensus about the validity and value of Markov models in time. Moreover, previous spacial Markov models have only been used for modeling transport on structured networks and can not be readily applied to model transport in unstructured networks. In this work we propose a novel approach for constructing a Markov model in time (stencil method) for a discrete velocity process. The results form the stencil method are compared to previously proposed spacial Markov models for structured networks. The stencil method is also applied to unstructured networks and can successfully describe the dispersion of particles in this setting. Our conclusion is that both temporal Markov models and spacial Markov models for discrete velocity processes can be valid for a range of model parameters. Moreover, we show that the stencil model can be more efficient in many practical settings and is suited to model dispersion both on structured and unstructured networks.

5. Markov Chain Monte Carlo

degrees before you begin simulation; further, it will be practicable only if the number of required chits is small. There are many situations where one of these is not true. In [1], we have seen how randomization is used to start a game of cricket. Another game where randomization is much more important is any game of cards.

6. Markov Chain Monte Carlo

coloured counters (one for each player) and a die. (Fig- ure 1). The 100 ... case of a 4 by 4 board and a single player (Figure 2). Note that I have .... probability approaches 1 in the limit as t tends to infinity was obvious even without all this mathematics, since it is a common experience that all games of Ludo eventually end ...

7. Estimation with Right-Censored Observations Under A Semi-Markov Model.

Science.gov (United States)

Zhao, Lihui; Hu, X Joan

2013-06-01

The semi-Markov process often provides a better framework than the classical Markov process for the analysis of events with multiple states. The purpose of this paper is twofold. First, we show that in the presence of right censoring, when the right end-point of the support of the censoring time is strictly less than the right end-point of the support of the semi-Markov kernel, the transition probability of the semi-Markov process is nonidentifiable, and the estimators proposed in the literature are inconsistent in general. We derive the set of all attainable values for the transition probability based on the censored data, and we propose a nonparametric inference procedure for the transition probability using this set. Second, the conventional approach to constructing confidence bands is not applicable for the semi-Markov kernel and the sojourn time distribution. We propose new perturbation resampling methods to construct these confidence bands. Different weights and transformations are explored in the construction. We use simulation to examine our proposals and illustrate them with hospitalization data from a recent cancer survivor study.

8. The Profit Distribution of Supply Chain under E-Commerce

Directory of Open Access Journals (Sweden)

Jiang-Hua Zhang

2014-01-01

Full Text Available With the development of e-commerce, its influence on supply chain and supply chain management is becoming increasingly significant too. In this paper, the literature on the supply chain profit is reviewed first, and then a two-level and four-party supply chain which consists of a supplier, an e-commerce platform, third-party logistics, and demander is taken into consideration. The profit function of supply chain under e-commerce is formulated by taking the price of product and the maximum supply amount under certain investment as decision-making variables and taking the expected value of random variables of price as the setting sales quantity. Finally, the existence of maximum profit in the supply chain is proved in the model, and the coordination of supply chain under e-commerce environment can be achieved by setting coordination parameters when the relevant cost parameters of supply chain members satisfy certain conditions.

9. Ensuring effective supply chain management under uncertainty

Directory of Open Access Journals (Sweden)

Lutsenko Iryna Sergiivna

2016-09-01

Full Text Available Identified the main sources of uncertainty in supply chains and tools to mitigate them. The necessity of functional, spatial and temporal integration and linkage of decision-making at different management levels. Determined that the optimization of information flow can occur due to the “shrink” in time, volume and direction, this process should be preceded by a thorough analysis and rethinking of the business processes of a complex system of supply chains.

10. EVALUATION OF THE FREQUENCY OF DIAGNOSTICS OF COMPONENTS AND ASSEMBLIES FOR TRANSPORT AND TECHNOLOGICAL MACHINES ON THE BASIS OF HIDDEN MARKOV CHAINS

Directory of Open Access Journals (Sweden)

Andrey Borisovich Nikolaev

2017-05-01

Full Text Available In this article a statistical analysis of supply volumes of spare parts, components and accessories was carried out, with some persistent patterns and laws of distribution of failures of major components revealed. There are suggested evaluation models of components and assemblies reliability for the formation of order management procedures of spare parts, components and accessories for the maintenance and repair of transport and technological machines. For the purpose of identification of components operational condition there is proposed a model of hidden Markov chain which allows to classify the condition by indirect evidence, based on the collected statistics.

11. Analysis of transtheoretical model of health behavioral changes in a nutrition intervention study--a continuous time Markov chain model with Bayesian approach.

Science.gov (United States)

Ma, Junsheng; Chan, Wenyaw; Tsai, Chu-Lin; Xiong, Momiao; Tilley, Barbara C

2015-11-30

Continuous time Markov chain (CTMC) models are often used to study the progression of chronic diseases in medical research but rarely applied to studies of the process of behavioral change. In studies of interventions to modify behaviors, a widely used psychosocial model is based on the transtheoretical model that often has more than three states (representing stages of change) and conceptually permits all possible instantaneous transitions. Very little attention is given to the study of the relationships between a CTMC model and associated covariates under the framework of transtheoretical model. We developed a Bayesian approach to evaluate the covariate effects on a CTMC model through a log-linear regression link. A simulation study of this approach showed that model parameters were accurately and precisely estimated. We analyzed an existing data set on stages of change in dietary intake from the Next Step Trial using the proposed method and the generalized multinomial logit model. We found that the generalized multinomial logit model was not suitable for these data because it ignores the unbalanced data structure and temporal correlation between successive measurements. Our analysis not only confirms that the nutrition intervention was effective but also provides information on how the intervention affected the transitions among the stages of change. We found that, compared with the control group, subjects in the intervention group, on average, spent substantively less time in the precontemplation stage and were more/less likely to move from an unhealthy/healthy state to a healthy/unhealthy state. Copyright © 2015 John Wiley & Sons, Ltd.

12. Semi-Markov Arnason-Schwarz models.

Science.gov (United States)

King, Ruth; Langrock, Roland

2016-06-01

We consider multi-state capture-recapture-recovery data where observed individuals are recorded in a set of possible discrete states. Traditionally, the Arnason-Schwarz model has been fitted to such data where the state process is modeled as a first-order Markov chain, though second-order models have also been proposed and fitted to data. However, low-order Markov models may not accurately represent the underlying biology. For example, specifying a (time-independent) first-order Markov process involves the assumption that the dwell time in each state (i.e., the duration of a stay in a given state) has a geometric distribution, and hence that the modal dwell time is one. Specifying time-dependent or higher-order processes provides additional flexibility, but at the expense of a potentially significant number of additional model parameters. We extend the Arnason-Schwarz model by specifying a semi-Markov model for the state process, where the dwell-time distribution is specified more generally, using, for example, a shifted Poisson or negative binomial distribution. A state expansion technique is applied in order to represent the resulting semi-Markov Arnason-Schwarz model in terms of a simpler and computationally tractable hidden Markov model. Semi-Markov Arnason-Schwarz models come with only a very modest increase in the number of parameters, yet permit a significantly more flexible state process. Model selection can be performed using standard procedures, and in particular via the use of information criteria. The semi-Markov approach allows for important biological inference to be drawn on the underlying state process, for example, on the times spent in the different states. The feasibility of the approach is demonstrated in a simulation study, before being applied to real data corresponding to house finches where the states correspond to the presence or absence of conjunctivitis. © 2015, The International Biometric Society.

13. Detecting Structural Breaks using Hidden Markov Models

DEFF Research Database (Denmark)

Ntantamis, Christos

Testing for structural breaks and identifying their location is essential for econometric modeling. In this paper, a Hidden Markov Model (HMM) approach is used in order to perform these tasks. Breaks are defined as the data points where the underlying Markov Chain switches from one state to another....... The estimation of the HMM is conducted using a variant of the Iterative Conditional Expectation-Generalized Mixture (ICE-GEMI) algorithm proposed by Delignon et al. (1997), that permits analysis of the conditional distributions of economic data and allows for different functional forms across regimes...

14. Study of the applicability of Markov chain Monte Carlo methods to the statistical separation of electron sources via the impact parameter for ALICE

Energy Technology Data Exchange (ETDEWEB)

Wittner, Manuel [Physikalisches Institut, Universitaet Heidelberg, Heidelberg (Germany); Collaboration: ALICE-Collaboration

2015-07-01

One particularly interesting measurement detected by the ALICE set-up at the LHC are electrons from charm and beauty hadron decays. Heavy quarks originate from initial hard scattering processes and thus experience the whole history of a heavy ion collision. Therefore, they are valuable probes to study the mechanisms of energy loss and hadronization in the hot and dense state of matter, that is expected to be formed in a heavy-ion collision at LHC. One important task is the distinction of the different electron sources, for which a method was developed. Hereby, the impact parameter distribution of the measurement data is compared with impact parameter distributions for the individual sources, which are created through Monte Carlo simulations. Afterwards, a maximum likelihood fit is applied. However, creating a posterior distribution of the likelihood according to Bayes' theorem and sampling it with Markov Chain Monte Carlo algorithms provides several advantages, e.g. a mathematically correct estimation of the uncertainties or the usage of prior knowledge. Hence for the first time in this particular problem, a Markov Chain Monte Carlo algorithm, namely the Metropolis algorithm, was implemented and investigated for its applicability in heavy flavor physics. First studies indicate its great usefulness in this field of physics.

15. Monitoring and Modeling of Spatiotemporal Urban Expansion and Land-Use/Land-Cover Change Using Integrated Markov Chain Cellular Automata Model

Directory of Open Access Journals (Sweden)

Bhagawat Rimal

2017-09-01

Full Text Available Spatial–temporal analysis of land-use/land-cover (LULC change as well as the monitoring and modeling of urban expansion are essential for the planning and management of urban environments. Such environments reflect the economic conditions and quality of life of the individual country. Urbanization is generally influenced by national laws, plans and policies and by power, politics and poor governance in many less-developed countries. Remote sensing tools play a vital role in monitoring LULC change and measuring the rate of urbanization at both the local and global levels. The current study evaluated the LULC changes and urban expansion of Jhapa district of Nepal. The spatial–temporal dynamics of LULC were identified using six time-series atmospherically-corrected surface reflectance Landsat images from 1989 to 2016. A hybrid cellular automata Markov chain (CA–Markov model was used to simulate future urbanization by 2026 and 2036. The analysis shows that the urban area has increased markedly and is expected to continue to grow rapidly in the future, whereas the area for agriculture has decreased. Meanwhile, forest and shrub areas have remained almost constant. Seasonal rainfall and flooding routinely cause predictable transformation of sand, water bodies and cultivated land from one type to another. The results suggest that the use of Landsat time-series archive images and the CA–Markov model are the best options for long-term spatiotemporal analysis and achieving an acceptable level of prediction accuracy. Furthermore, understanding the relationship between the spatiotemporal dynamics of urbanization and LULC change and simulating future landscape change is essential, as they are closely interlinked. These scientific findings of past, present and future land-cover scenarios of the study area will assist planners/decision-makers to formulate sustainable urban development and environmental protection plans and will remain a scientific asset

16. Thermodynamically accurate modeling of the catalytic cycle of photosynthetic oxygen evolution: a mathematical solution to asymmetric Markov chains.

Science.gov (United States)

Vinyard, David J; Zachary, Chase E; Ananyev, Gennady; Dismukes, G Charles

2013-07-01

Forty-three years ago, Kok and coworkers introduced a phenomenological model describing period-four oscillations in O2 flash yields during photosynthetic water oxidation (WOC), which had been first reported by Joliot and coworkers. The original two-parameter Kok model was subsequently extended in its level of complexity to better simulate diverse data sets, including intact cells and isolated PSII-WOCs, but at the expense of introducing physically unrealistic assumptions necessary to enable numerical solutions. To date, analytical solutions have been found only for symmetric Kok models (inefficiencies are equally probable for all intermediates, called "S-states"). However, it is widely accepted that S-state reaction steps are not identical and some are not reversible (by thermodynamic restraints) thereby causing asymmetric cycles. We have developed a mathematically more rigorous foundation that eliminates unphysical assumptions known to be in conflict with experiments and adopts a new experimental constraint on solutions. This new algorithm termed STEAMM for S-state Transition Eigenvalues of Asymmetric Markov Models enables solutions to models having fewer adjustable parameters and uses automated fitting to experimental data sets, yielding higher accuracy and precision than the classic Kok or extended Kok models. This new tool provides a general mathematical framework for analyzing damped oscillations arising from any cycle period using any appropriate Markov model, regardless of symmetry. We illustrate applications of STEAMM that better describe the intrinsic inefficiencies for photon-to-charge conversion within PSII-WOCs that are responsible for damped period-four and period-two oscillations of flash O2 yields across diverse species, while using simpler Markov models free from unrealistic assumptions. Copyright © 2013 Elsevier B.V. All rights reserved.

17. A Markov random walk under constraint for discovering overlapping communities in complex networks

International Nuclear Information System (INIS)

Jin, Di; Yang, Bo; Liu, Dayou; He, Dongxiao; Liu, Jie; Baquero, Carlos

2011-01-01

The detection of overlapping communities in complex networks has motivated recent research in relevant fields. Aiming to address this problem, we propose a Markov-dynamics-based algorithm, called UEOC, which means 'unfold and extract overlapping communities'. In UEOC, when identifying each natural community that overlaps, a Markov random walk method combined with a constraint strategy, which is based on the corresponding annealed network (degree conserving random network), is performed to unfold the community. Then, a cutoff criterion with the aid of a local community function, called conductance, which can be thought of as the ratio between the number of edges inside the community and those leaving it, is presented to extract this emerged community from the entire network. The UEOC algorithm depends on only one parameter whose value can be easily set, and it requires no prior knowledge of the hidden community structures. The proposed UEOC has been evaluated both on synthetic benchmarks and on some real-world networks, and has been compared with a set of competing algorithms. The experimental result has shown that UEOC is highly effective and efficient for discovering overlapping communities

18. Comparison of the kinetics of different Markov models for ligand binding under varying conditions

Science.gov (United States)

Martini, Johannes W. R.; Habeck, Michael

2015-03-01

We recently derived a Markov model for macromolecular ligand binding dynamics from few physical assumptions and showed that its stationary distribution is the grand canonical ensemble [J. W. R. Martini, M. Habeck, and M. Schlather, J. Math. Chem. 52, 665 (2014)]. The transition probabilities of the proposed Markov process define a particular Glauber dynamics and have some similarity to the Metropolis-Hastings algorithm. Here, we illustrate that this model is the stochastic analog of (pseudo) rate equations and the corresponding system of differential equations. Moreover, it can be viewed as a limiting case of general stochastic simulations of chemical kinetics. Thus, the model links stochastic and deterministic approaches as well as kinetics and equilibrium described by the grand canonical ensemble. We demonstrate that the family of transition matrices of our model, parameterized by temperature and ligand activity, generates ligand binding kinetics that respond to changes in these parameters in a qualitatively similar way as experimentally observed kinetics. In contrast, neither the Metropolis-Hastings algorithm nor the Glauber heat bath reflects changes in the external conditions correctly. Both converge rapidly to the stationary distribution, which is advantageous when the major interest is in the equilibrium state, but fail to describe the kinetics of ligand binding realistically. To simulate cellular processes that involve the reversible stochastic binding of multiple factors, our pseudo rate equation model should therefore be preferred to the Metropolis-Hastings algorithm and the Glauber heat bath, if the stationary distribution is not of only interest.

19. Comparison of the kinetics of different Markov models for ligand binding under varying conditions

International Nuclear Information System (INIS)

Martini, Johannes W. R.; Habeck, Michael

2015-01-01

We recently derived a Markov model for macromolecular ligand binding dynamics from few physical assumptions and showed that its stationary distribution is the grand canonical ensemble [J. W. R. Martini, M. Habeck, and M. Schlather, J. Math. Chem. 52, 665 (2014)]. The transition probabilities of the proposed Markov process define a particular Glauber dynamics and have some similarity to the Metropolis-Hastings algorithm. Here, we illustrate that this model is the stochastic analog of (pseudo) rate equations and the corresponding system of differential equations. Moreover, it can be viewed as a limiting case of general stochastic simulations of chemical kinetics. Thus, the model links stochastic and deterministic approaches as well as kinetics and equilibrium described by the grand canonical ensemble. We demonstrate that the family of transition matrices of our model, parameterized by temperature and ligand activity, generates ligand binding kinetics that respond to changes in these parameters in a qualitatively similar way as experimentally observed kinetics. In contrast, neither the Metropolis-Hastings algorithm nor the Glauber heat bath reflects changes in the external conditions correctly. Both converge rapidly to the stationary distribution, which is advantageous when the major interest is in the equilibrium state, but fail to describe the kinetics of ligand binding realistically. To simulate cellular processes that involve the reversible stochastic binding of multiple factors, our pseudo rate equation model should therefore be preferred to the Metropolis-Hastings algorithm and the Glauber heat bath, if the stationary distribution is not of only interest

20. Supply chain management under fuzziness recent developments and techniques

CERN Document Server

Öztayşi, Başar

2014-01-01

Supply Chain Management Under Fuzziness presents recently developed fuzzy models and techniques for supply chain management. These include: fuzzy PROMETHEE, fuzzy AHP, fuzzy ANP, fuzzy VIKOR, fuzzy DEMATEL, fuzzy clustering, fuzzy linear programming, and fuzzy inference systems. The book covers both practical applications and new developments concerning these methods. This book offers an excellent resource for researchers and practitioners in supply chain management and logistics, and will provide them with new suggestions and directions for future research. Moreover, it will support graduate students in their university courses, such as specialized courses on supply chains and logistics, as well as related courses in the fields of industrial engineering, engineering management and business administration.

1. Analysis and Optimization of Power Supply Structure Based on Markov Chain and Error Optimization for Renewable Energy from the Perspective of Sustainability

Directory of Open Access Journals (Sweden)

Xiaomin Xu

2016-07-01

Full Text Available With the rapid development of renewable energy, power supply structure is changing. However, thermal power is still dominant. With the background in low carbon economy, reasonable adjustment and optimization of the power supply structure is the trend of future development in the power industry. It is also a reliable guarantee of a fast, healthy and stable development of national economy. In this paper, the sustainable development of renewable energy sources is analyzed from the perspective of power supply. Through the research on the development of power supply structure, we find that regional power supply structure development mode conforms to dynamic characteristics and there must exist a Markov chain in the final equilibrium state. Combined with the characteristics of no aftereffect and small samples, this paper applies a Markov model to the power supply structure prediction. The optimization model is established to ensure that the model can fit the historical data as much as possible. Taking actual data of a certain area of Ningxia Province as an example, the models proposed in this paper are applied to the practice and results verify the validity and robustness of the model, which can provide decision basis for enterprise managers.

2. Coordinating a Two-Echelon Supply Chain under Carbon Tax

Directory of Open Access Journals (Sweden)

Wei Yu

2017-12-01

Full Text Available In this paper, we study the impact of carbon tax on carbon emission and retail price in a two-echelon supply chain consisting of a manufacturer and a retailer. Specifically, by adopting two types of contracts, i.e., the modified wholesale price (MW and the modified cost-sharing contract (MS, supply chain coordination is achieved, which promotes the supply chain efficiency. Our study shows that: (1 with the increase of carbon tax, both the optimal emission reduction level and the optimal retail price increase, and then keep unchanged; (2 neither MW nor MS benefits the manufacturer after the supply chain coordination; and (3 to effectively coordinate the supply chain, we propose an innovative supply chain contract that integrates the firms’ optimal decisions under MW or MS with a two part tariff contract (TPT and a fixed fee the retailer can pay to ensure a win–win solution.

3. Deciphering the Glacial-Interglacial Landscape History in Greenland Based on Markov Chain Monte Carlo Inversion of Existing 10Be-26Al Data

DEFF Research Database (Denmark)

Strunk, Astrid; Knudsen, Mads Faurschou; Larsen, Nicolaj Krog

, and ii) glacial periods characterized by 100 % shielding and a uniform glacial erosion rate. We incorporate the exposure/burial history in the model framework by applying a threshold value to the global marine benthic d18O record and include the threshold value as a free model parameter, hereby taking...... investigate the landscape history in eastern and western Greenland by applying a novel Markov Chain Monte Carlo (MCMC) inversion approach to the existing 10Be-26Al data from these regions. The new MCMC approach allows us to constrain the most likely landscape history based on comparisons between simulated...... and measured cosmogenic nuclide concentrations. It is a fundamental assumption of the model approach that the exposure history at the site/location can be divided into two distinct regimes: i) interglacial periods characterized by zero shielding due to overlying ice and a uniform interglacial erosion rate...

4. Time-domain induced polarization - an analysis of Cole-Cole parameter resolution and correlation using Markov Chain Monte Carlo inversion

DEFF Research Database (Denmark)

Madsen, Line Meldgaard; Fiandaca, Gianluca; Auken, Esben

2017-01-01

The application of time-domain induced polarization (TDIP) is increasing with advances in acquisition techniques, data processing and spectral inversion schemes. An inversion of TDIP data for the spectral Cole-Cole parameters is a non-linear problem, but by applying a 1-D Markov Chain Monte Carlo...... increase and become non-linear. It is further investigated how waveform and parameter values influence the resolution of the Cole-Cole parameters. A limiting factor is the value of the frequency exponent, C. As C decreases, the resolution of all the Cole-Cole parameters decreases and the results become...... increasingly non-linear. While the values of the time constant, tau, must be in the acquisition range to resolve the parameters well, the choice between a 50 per cent and a 100 per cent duty cycle for the current injection does not have an influence on the parameter resolution. The limits of resolution...

5. Open Markov Processes and Reaction Networks

Science.gov (United States)

Swistock Pollard, Blake Stephen

2017-01-01

We begin by defining the concept of "open" Markov processes, which are continuous-time Markov chains where probability can flow in and out through certain "boundary" states. We study open Markov processes which in the absence of such boundary flows admit equilibrium states satisfying detailed balance, meaning that the net flow…

6. Multi-Decadal Mangrove Forest Change Detection and Prediction in Honduras, Central America, with Landsat Imagery and a Markov Chain Model

Directory of Open Access Journals (Sweden)

Chi-Farn Chen

2013-11-01

Full Text Available Mangrove forests play an important role in providing ecological and socioeconomic services for human society. Coastal development, which converts mangrove forests to other land uses, has often ignored the services that mangrove may provide, leading to irreversible environmental degradation. Monitoring the spatiotemporal distribution of mangrove forests is thus critical for natural resources management of mangrove ecosystems. This study investigates spatiotemporal changes in Honduran mangrove forests using Landsat imagery during the periods 1985–1996, 1996–2002, and 2002–2013. The future trend of mangrove forest changes was projected by a Markov chain model to support decision-making for coastal management. The remote sensing data were processed through three main steps: (1 data pre-processing to correct geometric errors between the Landsat imageries and to perform reflectance normalization; (2 image classification with the unsupervised Otsu’s method and change detection; and (3 mangrove change projection using a Markov chain model. Validation of the unsupervised Otsu’s method was made by comparing the classification results with the ground reference data in 2002, which yielded satisfactory agreement with an overall accuracy of 91.1% and Kappa coefficient of 0.82. When examining mangrove changes from 1985 to 2013, approximately 11.9% of the mangrove forests were transformed to other land uses, especially shrimp farming, while little effort (3.9% was applied for mangrove rehabilitation during this 28-year period. Changes in the extent of mangrove forests were further projected until 2020, indicating that the area of mangrove forests could be continuously reduced by 1,200 ha from 2013 (approximately 36,700 ha to 2020 (approximately 35,500 ha. Institutional interventions should be taken for sustainable management of mangrove ecosystems in this coastal region.

7. Twelve years of succession on sandy substrates in a post-mining landscape: a Markov chain analysis.

Science.gov (United States)

Baasch, Annett; Tischew, Sabine; Bruelheide, Helge

2010-06-01

Knowledge of succession rates and pathways is crucial for devising restoration strategies for highly disturbed ecosystems such as surface-mined land. As these processes have often only been described in qualitative terms, we used Markov models to quantify transitions between successional stages. However, Markov models are often considered not attractive for some reasons, such as model assumptions (e.g., stationarity in space and time, or the high expenditure of time required to estimate successional transitions in the field). Here we present a solution for converting multivariate ecological time series into transition matrices and demonstrate the applicability of this approach for a data set that resulted from monitoring the succession of sandy dry grassland in a post-mining landscape. We analyzed five transition matrices, four one-step matrices referring to specific periods of transition (1995-1998, 1998-2001, 2001-2004, 2004-2007), and one matrix for the whole study period (stationary model, 1995-2007). Finally, the stationary model was enhanced to a partly time-variable model. Applying the stationary and the time-variable models, we started a prediction well outside our calibration period, beginning with 100% bare soil in 1974 as the known start of the succession, and generated the coverage of 12 predefined vegetation types in three-year intervals. Transitions among vegetation types changed significantly in space and over time. While the probability of colonization was almost constant over time, the replacement rate tended to increase, indicating that the speed of succession accelerated with time or fluctuations became stronger. The predictions of both models agreed surprisingly well with the vegetation data observed more than two decades later. This shows that our dry grassland succession in a post-mining landscape can be adequately described by comparably simple types of Markov models, although some model assumptions have not been fulfilled and within

8. Sieve estimation in a Markov illness-death process under dual censoring.

Science.gov (United States)

Boruvka, Audrey; Cook, Richard J

2016-04-01

Semiparametric methods are well established for the analysis of a progressive Markov illness-death process observed up to a noninformative right censoring time. However, often the intermediate and terminal events are censored in different ways, leading to a dual censoring scheme. In such settings, unbiased estimation of the cumulative transition intensity functions cannot be achieved without some degree of smoothing. To overcome this problem, we develop a sieve maximum likelihood approach for inference on the hazard ratio. A simulation study shows that the sieve estimator offers improved finite-sample performance over common imputation-based alternatives and is robust to some forms of dependent censoring. The proposed method is illustrated using data from cancer trials. © The Author 2015. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

9. Sustainable and Resilient Supply Chain Network Design under Disruption Risks

Directory of Open Access Journals (Sweden)

2014-09-01

Full Text Available Sustainable supply chain network design is a rich area for academic research that is still in its infancy and has potential to affect supply chain performance. Increasing regulations for carbon and waste management are forcing firms to consider their supply chains from ecological and social objectives, but in reality, however, facilities and the links connecting them are disrupted from time to time, due to poor weather, natural or manmade disasters or a combination of any other factors. Supply chain systems drop their sustainability objectives while coping with these unexpected disruptions. Hence, the new challenges for supply chain managers are to design an efficient and effective supply chain network that will be resilient enough to bounce back from any disruption and that also should have sufficient vigilance to offer same sustainability under a disruption state. This paper focuses on ecological sustainability, because an environmental focus in a supply chain system is more important and also links with other pillars of sustainability, as the products need to be produced, packed and transported in an ethical way, which should not harm social balance and the environment. Owing to importance of the considered issue, this paper attempts to introduce a network optimization model for a sustainable and resilient supply chain network by incorporating (1 sustainability via carbon emissions and embodied carbon footprints and (2 resilience by incorporating location-specific risks. The proposed goal programming (GP model optimizes the total cost, while considering the resilience and sustainability of the supply chain network.

10. Supply Chain Bilateral Coordination with Option Contracts under Inflation Scenarios

Directory of Open Access Journals (Sweden)

Nana Wan

2015-01-01

Full Text Available There exist obvious changes in price and demand during the inflationary period, both of which are regarded as the key factors leading to supply chain uncertainty. In this paper, we focus our discussion on price increase and demand contraction caused by inflation, integrate the effect of inflation and option contracts within the model framework, and analyze how to use option contracts to achieve supply chain coordination under inflation scenarios. We consider a one-period two-stage supply chain consisting of one supplier and one retailer and explore the effect of inflation on the optimal ordering and production decisions under three different types of contracts: wholesale price contracts, option contracts, and portfolio contracts. Moreover, we explore the impact of option contracts on the supply chain through using wholesale price contracts model as the benchmark. We find that the retailer prefers adopting portfolio contracts, but the supplier prefers providing option contracts under inflation scenarios. Ultimately, option contracts will be implemented owing to the supplier’s market dominant position. In addition, we discuss the supply chain bilateral coordination mechanism with option contracts from the perspectives of two members and derive that option contracts can coordinate the supply chain and achieve Pareto improvement under inflation scenarios.

11. A Markov chain analysis of the movements of juvenile salmonids, including sockeye salmon, in the forebay of McNary Dam, Washington and Oregon, 2006-09

Science.gov (United States)

Adams, Noah S.; Hatton, Tyson W.

2012-01-01

Passage and survival data were collected at McNary Dam between 2006 and 2009. These data have provided critical information for resource managers to implement structural and operational changes designed to improve the survival of juvenile salmonids as they migrate past the dam. Much of the valuable information collected at McNary Dam was in the form of three-dimensional (hereafter referred to as 3-D) tracks of fish movements in the forebay. These data depicted the behavior of multiple species (in three dimensions) during different diel periods, spill conditions, powerhouse operations, and testing of the surface bypass structures (temporary spillway weirs; TSWs). One of the challenges in reporting 3-D results is presenting the information in a manner that allows interested parties to summarize the behavior of many fish over many different conditions across multiple years. To accomplish this, we used a Markov chain analysis to characterize fish movement patterns in the forebay of McNary Dam. The Markov chain analysis allowed us to numerically summarize the behavior of fish in the forebay. This report is the second report published in 2012 that uses this analytical method. The first report included only fish released as part of the annual studies conducted at McNary Dam. This second report includes sockeye salmon that were released as part of studies conducted by the Chelan and Grant County Public Utility Districts at mid-Columbia River dams. The studies conducted in the mid-Columbia used the same transmitters as were used for McNary Dam studies, but transmitter pulse width was different between studies. Additionally, no passive integrated transponder tags were implanted in sockeye salmon. Differences in transmitter pulse width resulted in lower detection probabilities for sockeye salmon at McNary Dam. The absence of passive integrated transponder tags prevented us from determining if fish passed the powerhouse through the juvenile bypass system (JBS) or turbines. To

12. Optimal capacity and buffer size estimation under Generalized Markov Fluids Models and QoS parameters

International Nuclear Information System (INIS)

Bavio, José; Marrón, Beatriz

2014-01-01

Quality of service (QoS) for internet traffic management requires good traffic models and good estimation of sharing network resource. A link of a network processes all traffic and it is designed with certain capacity C and buffer size B. A Generalized Markov Fluid model (GMFM), introduced by Marrón (2011), is assumed for the sources because describes in a versatile way the traffic, allows estimation based on traffic traces, and also consistent effective bandwidth estimation can be done. QoS, interpreted as buffer overflow probability, can be estimated for GMFM through the effective bandwidth estimation and solving the optimization problem presented in Courcoubetis (2002), the so call inf-sup formulas. In this work we implement a code to solve the inf-sup problem and other optimization related with it, that allow us to do traffic engineering in links of data networks to calculate both, minimum capacity required when QoS and buffer size are given or minimum buffer size required when QoS and capacity are given

13. Semi-Markov graph dynamics.

Directory of Open Access Journals (Sweden)

Marco Raberto

Full Text Available In this paper, we outline a model of graph (or network dynamics based on two ingredients. The first ingredient is a Markov chain on the space of possible graphs. The second ingredient is a semi-Markov counting process of renewal type. The model consists in subordinating the Markov chain to the semi-Markov counting process. In simple words, this means that the chain transitions occur at random time instants called epochs. The model is quite rich and its possible connections with algebraic geometry are briefly discussed. Moreover, for the sake of simplicity, we focus on the space of undirected graphs with a fixed number of nodes. However, in an example, we present an interbank market model where it is meaningful to use directed graphs or even weighted graphs.

14. A more efficient approach to parallel-tempered Markov-chain MonteÂ Carlo for the highly structured posteriors of gravitational-wave signals

Science.gov (United States)

Farr, Benjamin; Kalogera, Vicky; Luijten, Erik

2014-07-01

We introduce a new Markov-chain Monte Carlo (MCMC) approach designed for the efficient sampling of highly correlated and multimodal posteriors. Parallel tempering, though effective, is a costly technique for sampling such posteriors. Our approach minimizes the use of parallel tempering, only applying it for a short time to build a proposal distribution that is based upon estimation of the kernel density and tuned to the target posterior. This proposal makes subsequent use of parallel tempering unnecessary, allowing all chains to be cooled to sample the target distribution. Gains in efficiency are found to increase with increasing posterior complexity, ranging from tens of percent in the simplest cases to over a factor of 10 for the more complex cases. Our approach is particularly useful in the context of parameter estimation of gravitational-wave signals measured by ground-based detectors, which is currently done through Bayesian inference with MCMC, one of the leading sampling methods. Posteriors for these signals are typically multimodal with strong nonlinear correlations, making sampling difficult. As we enter the advanced-detector era, improved sensitivities and wider bandwidths will drastically increase the computational cost of analyses, demanding more efficient search algorithms to meet these challenges.

15. Characterization of environmental quality of forest fragments changes in Jundiaí-Mirim river basin-Brazil using the Markov Chain model

Science.gov (United States)

Hasimoto Fengler, Felipe; Leite de Moraes, Jener Fernando; Irio Ribeiro, Admilson; Peche Filho, Afonso; Araujo de Medeiros, Gerson; Baldin Damame, Desirée; Márcia Longo, Regina

2015-04-01

In Brazil is common practice the concurrency of large urban centers water catchment in distant sites. There's no policy to preserve strategic springs in the urban territory. Thus, rural areas, located in the surrounds of municipals, usually provide water and others environment services to the population that reside on cities. The Jundiaí-Mirim river basin, located in the most urbanized state in Brazil, São Paulo, composes an interesting example of this situation. It is located in a rural area near large urban centers, with large industrial parks, near the capital of state. As result of expansion of the cities on its surrounds their lands have had a historic of monetary valorization, making its territories attractive to the housing market. Consequently, the region has an intense process of urbanization that resulted in an increasing environmental disturbance in the areas of natural vegetation. In the other hand, the watershed is the principal water supplier of Jundiaí city, and houses forest remaining of an important Biome in Brazil, the Atlantic Rain Forest. Given the need to preserve its water production capacity and the forest remnants there, this study modeled the environmental quality of forest fragments through indicators of disturbance and evaluated the changes that occur between 1972 and 2013 using the Markov Chain model. The environment quality was determined by nine indicators of environmental disturbance (distance of urban areas, roads, edge land use, size, distance of others forest fragments, land capacity of use, watershed forest cover, number of forest fragments in the watersheds, shape of the forest fragment), obtained by techniques of Geoprocessing, and integrated by Multicriteria Analysis. The Markov Chain model showed a constant tendency of deteriorating in natural vegetation environmental quality, attributed to the intense process of occupation of the river basin. The results showed a historical trend of transformation in forest fragments with

16. Application and Evaluation of a Snowmelt Runoff Model in the Tamor River Basin, Eastern Himalaya Using a Markov Chain Monte Carlo (MCMC) Data Assimilation Approach

Science.gov (United States)

Panday, Prajjwal K.; Williams, Christopher A.; Frey, Karen E.; Brown, Molly E.

2013-01-01

Previous studies have drawn attention to substantial hydrological changes taking place in mountainous watersheds where hydrology is dominated by cryospheric processes. Modelling is an important tool for understanding these changes but is particularly challenging in mountainous terrain owing to scarcity of ground observations and uncertainty of model parameters across space and time. This study utilizes a Markov Chain Monte Carlo data assimilation approach to examine and evaluate the performance of a conceptual, degree-day snowmelt runoff model applied in the Tamor River basin in the eastern Nepalese Himalaya. The snowmelt runoff model is calibrated using daily streamflow from 2002 to 2006 with fairly high accuracy (average Nash-Sutcliffe metric approx. 0.84, annual volume bias <3%). The Markov Chain Monte Carlo approach constrains the parameters to which the model is most sensitive (e.g. lapse rate and recession coefficient) and maximizes model fit and performance. Model simulated streamflow using an interpolated precipitation data set decreases the fractional contribution from rainfall compared with simulations using observed station precipitation. The average snowmelt contribution to total runoff in the Tamor River basin for the 2002-2006 period is estimated to be 29.7+/-2.9% (which includes 4.2+/-0.9% from snowfall that promptly melts), whereas 70.3+/-2.6% is attributed to contributions from rainfall. On average, the elevation zone in the 4000-5500m range contributes the most to basin runoff, averaging 56.9+/-3.6% of all snowmelt input and 28.9+/-1.1% of all rainfall input to runoff. Model simulated streamflow using an interpolated precipitation data set decreases the fractional contribution from rainfall versus snowmelt compared with simulations using observed station precipitation. Model experiments indicate that the hydrograph itself does not constrain estimates of snowmelt versus rainfall contributions to total outflow but that this derives from the degree

17. Decentralization Policies for Supply Chain Investments under Asymmetric Information

DEFF Research Database (Denmark)

Agrell, Per Joakim; Bogetoft, Peter

2017-01-01

Supply chains need specific investments for improved performance in terms of lead-time, cost, and quality. We study the contractual choice of a coordinator to either centralize or delegate the investment decision in a three-stage chain. The analysis derives closed-form results for the economic...... performance of three decentralized contracting schemes under asymmetric information on investment cost, as well as the optimal full revelation results. The results show that the observed practice of tier-1 delegated investments leads to relatively poor performance because of underinvestment. We illustrate...

18. AEOLUS: A MARKOV CHAIN MONTE CARLO CODE FOR MAPPING ULTRACOOL ATMOSPHERES. AN APPLICATION ON JUPITER AND BROWN DWARF HST LIGHT CURVES

Energy Technology Data Exchange (ETDEWEB)

Karalidi, Theodora; Apai, Dániel; Schneider, Glenn; Hanson, Jake R. [Steward Observatory, Department of Astronomy, University of Arizona, 933 N. Cherry Avenue, Tucson, AZ 85721 (United States); Pasachoff, Jay M., E-mail: tkaralidi@email.arizona.edu [Hopkins Observatory, Williams College, 33 Lab Campus Drive, Williamstown, MA 01267 (United States)

2015-11-20

Deducing the cloud cover and its temporal evolution from the observed planetary spectra and phase curves can give us major insight into the atmospheric dynamics. In this paper, we present Aeolus, a Markov chain Monte Carlo code that maps the structure of brown dwarf and other ultracool atmospheres. We validated Aeolus on a set of unique Jupiter Hubble Space Telescope (HST) light curves. Aeolus accurately retrieves the properties of the major features of the Jovian atmosphere, such as the Great Red Spot and a major 5 μm hot spot. Aeolus is the first mapping code validated on actual observations of a giant planet over a full rotational period. For this study, we applied Aeolus to J- and H-band HST light curves of 2MASS J21392676+0220226 and 2MASS J0136565+093347. Aeolus retrieves three spots at the top of the atmosphere (per observational wavelength) of these two brown dwarfs, with a surface coverage of 21% ± 3% and 20.3% ± 1.5%, respectively. The Jupiter HST light curves will be publicly available via ADS/VIZIR.

19. Characterizing the Trade Space Between Capability and Complexity in Next Generation Cloud and Precipitation Observing Systems Using Markov Chain Monte Carlos Techniques

Science.gov (United States)

Xu, Z.; Mace, G. G.; Posselt, D. J.

2017-12-01

As we begin to contemplate the next generation atmospheric observing systems, it will be critically important that we are able to make informed decisions regarding the trade space between scientific capability and the need to keep complexity and cost within definable limits. To explore this trade space as it pertains to understanding key cloud and precipitation processes, we are developing a Markov Chain Monte Carlo (MCMC) algorithm suite that allows us to arbitrarily define the specifications of candidate observing systems and then explore how the uncertainties in key retrieved geophysical parameters respond to that observing system. MCMC algorithms produce a more complete posterior solution space, and allow for an objective examination of information contained in measurements. In our initial implementation, MCMC experiments are performed to retrieve vertical profiles of cloud and precipitation properties from a spectrum of active and passive measurements collected by aircraft during the ACE Radiation Definition Experiments (RADEX). Focusing on shallow cumulus clouds observed during the Integrated Precipitation and Hydrology EXperiment (IPHEX), observing systems in this study we consider W and Ka-band radar reflectivity, path-integrated attenuation at those frequencies, 31 and 94 GHz brightness temperatures as well as visible and near-infrared reflectance. By varying the sensitivity and uncertainty of these measurements, we quantify the capacity of various combinations of observations to characterize the physical properties of clouds and precipitation.

20. New Product Development in an Emerging Economy: Analysing the Role of Supplier Involvement Practices by Using Bayesian Markov Chain Monte Carlo Technique

Directory of Open Access Journals (Sweden)

Kanagi Kanapathy

2014-01-01

Full Text Available The research question is whether the positive relationship found between supplier involvement practices and new product development performances in developed economies also holds in emerging economies. The role of supplier involvement practices in new product development performance is yet to be substantially investigated in the emerging economies (other than China. This premise was examined by distributing a survey instrument (Jayaram’s (2008 published survey instrument that has been utilised in developed economies to Malaysian manufacturing companies. To gauge the relationship between the supplier involvement practices and new product development (NPD project performance of 146 companies, structural equation modelling was adopted. Our findings prove that supplier involvement practices have a significant positive impact on NPD project performance in an emerging economy with respect to quality objectives, design objectives, cost objectives, and “time-to-market” objectives. Further analysis using the Bayesian Markov Chain Monte Carlo algorithm, yielding a more credible and feasible differentiation, confirmed these results (even in the case of an emerging economy and indicated that these practices have a 28% impact on variance of NPD project performance. This considerable effect implies that supplier involvement is a must have, although further research is needed to identify the contingencies for its practices.

1. Probabilistic Forecasting of Drought Events Using Markov Chain- and Bayesian Network-Based Models: A Case Study of an Andean Regulated River Basin

Directory of Open Access Journals (Sweden)

Alex Avilés

2016-01-01

Full Text Available The scarcity of water resources in mountain areas can distort normal water application patterns with among other effects, a negative impact on water supply and river ecosystems. Knowing the probability of droughts might help to optimize a priori the planning and management of the water resources in general and of the Andean watersheds in particular. This study compares Markov chain- (MC and Bayesian network- (BN based models in drought forecasting using a recently developed drought index with respect to their capability to characterize different drought severity states. The copula functions were used to solve the BNs and the ranked probability skill score (RPSS to evaluate the performance of the models. Monthly rainfall and streamflow data of the Chulco River basin, located in Southern Ecuador, were used to assess the performance of both approaches. Global evaluation results revealed that the MC-based models predict better wet and dry periods, and BN-based models generate slightly more accurately forecasts of the most severe droughts. However, evaluation of monthly results reveals that, for each month of the hydrological year, either the MC- or BN-based model provides better forecasts. The presented approach could be of assistance to water managers to ensure that timely decision-making on drought response is undertaken.

2. Combined state and parameter identification of nonlinear structural dynamical systems based on Rao-Blackwellization and Markov chain Monte Carlo simulations

Science.gov (United States)

Abhinav, S.; Manohar, C. S.

2018-03-01

The problem of combined state and parameter estimation in nonlinear state space models, based on Bayesian filtering methods, is considered. A novel approach, which combines Rao-Blackwellized particle filters for state estimation with Markov chain Monte Carlo (MCMC) simulations for parameter identification, is proposed. In order to ensure successful performance of the MCMC samplers, in situations involving large amount of dynamic measurement data and (or) low measurement noise, the study employs a modified measurement model combined with an importance sampling based correction. The parameters of the process noise covariance matrix are also included as quantities to be identified. The study employs the Rao-Blackwellization step at two stages: one, associated with the state estimation problem in the particle filtering step, and, secondly, in the evaluation of the ratio of likelihoods in the MCMC run. The satisfactory performance of the proposed method is illustrated on three dynamical systems: (a) a computational model of a nonlinear beam-moving oscillator system, (b) a laboratory scale beam traversed by a loaded trolley, and (c) an earthquake shake table study on a bending-torsion coupled nonlinear frame subjected to uniaxial support motion.

3. Assessment of myocardial metabolic rate of glucose by means of Bayesian ICA and Markov Chain Monte Carlo methods in small animal PET imaging

Science.gov (United States)

2016-09-01

In dynamic cardiac PET FDG studies the assessment of myocardial metabolic rate of glucose (MMRG) requires the knowledge of the blood input function (IF). IF can be obtained by manual or automatic blood sampling and cross calibrated with PET. These procedures are cumbersome, invasive and generate uncertainties. The IF is contaminated by spillover of radioactivity from the adjacent myocardium and this could cause important error in the estimated MMRG. In this study, we show that the IF can be extracted from the images in a rat heart study with 18F-fluorodeoxyglucose (18F-FDG) by means of Independent Component Analysis (ICA) based on Bayesian theory and Markov Chain Monte Carlo (MCMC) sampling method (BICA). Images of the heart from rats were acquired with the Sherbrooke small animal PET scanner. A region of interest (ROI) was drawn around the rat image and decomposed into blood and tissue using BICA. The Statistical study showed that there is a significant difference (p corrupted with spillover.

4. Nonlinear calibration transfer based on hierarchical Bayesian models and Lagrange Multipliers: Error bounds of estimates via Monte Carlo - Markov Chain sampling.

Science.gov (United States)

Seichter, Felicia; Vogt, Josef; Radermacher, Peter; Mizaikoff, Boris

2017-01-25

The calibration of analytical systems is time-consuming and the effort for daily calibration routines should therefore be minimized, while maintaining the analytical accuracy and precision. The 'calibration transfer' approach proposes to combine calibration data already recorded with actual calibrations measurements. However, this strategy was developed for the multivariate, linear analysis of spectroscopic data, and thus, cannot be applied to sensors with a single response channel and/or a non-linear relationship between signal and desired analytical concentration. To fill this gap for a non-linear calibration equation, we assume that the coefficients for the equation, collected over several calibration runs, are normally distributed. Considering that coefficients of an actual calibration are a sample of this distribution, only a few standards are needed for a complete calibration data set. The resulting calibration transfer approach is demonstrated for a fluorescence oxygen sensor and implemented as a hierarchical Bayesian model, combined with a Lagrange Multipliers technique and Monte-Carlo Markov-Chain sampling. The latter provides realistic estimates for coefficients and prediction together with accurate error bounds by simulating known measurement errors and system fluctuations. Performance criteria for validation and optimal selection of a reduced set of calibration samples were developed and lead to a setup which maintains the analytical performance of a full calibration. Strategies for a rapid determination of problems occurring in a daily calibration routine, are proposed, thereby opening the possibility of correcting the problem just in time. Copyright © 2016 Elsevier B.V. All rights reserved.

5. Aeolus: A Markov Chain Monte Carlo Code for Mapping Ultracool Atmospheres. An Application on Jupiter and Brown Dwarf HST Light Curves

Science.gov (United States)

Karalidi, Theodora; Apai, Dániel; Schneider, Glenn; Hanson, Jake R.; Pasachoff, Jay M.

2015-11-01

Deducing the cloud cover and its temporal evolution from the observed planetary spectra and phase curves can give us major insight into the atmospheric dynamics. In this paper, we present Aeolus, a Markov chain Monte Carlo code that maps the structure of brown dwarf and other ultracool atmospheres. We validated Aeolus on a set of unique Jupiter Hubble Space Telescope (HST) light curves. Aeolus accurately retrieves the properties of the major features of the Jovian atmosphere, such as the Great Red Spot and a major 5 μm hot spot. Aeolus is the first mapping code validated on actual observations of a giant planet over a full rotational period. For this study, we applied Aeolus to J- and H-band HST light curves of 2MASS J21392676+0220226 and 2MASS J0136565+093347. Aeolus retrieves three spots at the top of the atmosphere (per observational wavelength) of these two brown dwarfs, with a surface coverage of 21% ± 3% and 20.3% ± 1.5%, respectively. The Jupiter HST light curves will be publicly available via ADS/VIZIR.

6. Markov chains-cellular automata modeling and multicriteria analysis of land cover change in the Lower Nhecolândia subregion of the Brazilian Pantanal wetland

Science.gov (United States)

Bacani, Vitor Matheus; Sakamoto, Arnaldo Yoso; Quénol, Hervé; Vannier, Clémence; Corgne, Samuel

2016-01-01

The dynamics of land use/land cover change in the Lower Nhecolândia wetland are marked by deforestation for pasture expansion, resulting in a real threat to the ecological stability. The aim of our work was to analyze the spatial distribution of land cover changes in the Lower Nhecolândia from 1985 to 2013 and to predict changes in trends for 2040. The mapping of land cover changes was developed using Landsat satellite images of 1985, 1999, 2007, and 2013, based on geographic object-based image analysis approach. This study uses integrated Markov chains and cellular automata modeling and multicriteria evaluation techniques to produce transition probability maps and describe the trajectory analysis methodology to construct a continuity of spatial and temporal changes for the wetland. The results of the multitemporal change detection classification show that, from 1985 to 2013, the forest woodland decreased by 6.89% and the grassland class increased by 18.29%. On the other hand, all water bodies showed a reducing trend, while the bare soil class increased compared to 1985, but did not present a regular trend of increase or decrease. From the present day, the trend for the future is a reduction of almost 6.4% by 2040. We found that deforestation actions will be concentrated in the areas with the highest concentration of saline lakes, constituting a serious threat to the natural functioning of this environmental system.

7. Accelerating Monte Carlo molecular simulations by reweighting and reconstructing Markov chains: Extrapolation of canonical ensemble averages and second derivatives to different temperature and density conditions

KAUST Repository

2014-08-01

Accurate determination of thermodynamic properties of petroleum reservoir fluids is of great interest to many applications, especially in petroleum engineering and chemical engineering. Molecular simulation has many appealing features, especially its requirement of fewer tuned parameters but yet better predicting capability; however it is well known that molecular simulation is very CPU expensive, as compared to equation of state approaches. We have recently introduced an efficient thermodynamically consistent technique to regenerate rapidly Monte Carlo Markov Chains (MCMCs) at different thermodynamic conditions from the existing data points that have been pre-computed with expensive classical simulation. This technique can speed up the simulation more than a million times, making the regenerated molecular simulation almost as fast as equation of state approaches. In this paper, this technique is first briefly reviewed and then numerically investigated in its capability of predicting ensemble averages of primary quantities at different neighboring thermodynamic conditions to the original simulated MCMCs. Moreover, this extrapolation technique is extended to predict second derivative properties (e.g. heat capacity and fluid compressibility). The method works by reweighting and reconstructing generated MCMCs in canonical ensemble for Lennard-Jones particles. In this paper, system\\'s potential energy, pressure, isochoric heat capacity and isothermal compressibility along isochors, isotherms and paths of changing temperature and density from the original simulated points were extrapolated. Finally, an optimized set of Lennard-Jones parameters (ε, σ) for single site models were proposed for methane, nitrogen and carbon monoxide. © 2014 Elsevier Inc.

8. Development of Shale Gas Supply Chain Network under Market Uncertainties

Directory of Open Access Journals (Sweden)

Jorge Chebeir

2017-02-01

Full Text Available The increasing demand of energy has turned the shale gas and shale oil into one of the most promising sources of energy in the United States. In this article, a model is proposed to address the long-term planning problem of the shale gas supply chain under uncertain conditions. A two-stage stochastic programming model is proposed to describe and optimize the shale gas supply chain network. Inherent uncertainty in final products’ prices, such as natural gas and natural gas liquids (NGL, is treated through the utilization of a scenario-based method. A binomial option pricing model is utilized to approximate the stochastic process through the generation of scenario trees. The aim of the proposed model is to generate an appropriate and realistic supply chain network configuration as well as scheduling of different operations throughout the planning horizon of a shale gas development project.

9. [The contamination under polymerase chain reaction studies: problems and solutions].

Science.gov (United States)

Titov, V N; Ameliushkina, V A; Rozhkova, T A

2015-01-01

The study was carried out to determine risk factors of false positive and false negative results under polymerase chain reaction-analysis of clinical material. The samples with high viral load can be the source of false positive results. The contamination with nucleic acids can occur at any section of polymerase chain reaction analysis. The study data permitted to establish that the most sensitive stage is isolation and purification of nucleic acids especially under manual mode of operation. The detection of positive signal in most samples of one setting indicates total contamination. The cases when only several samples are polluted are special challenge. The presence of sample with high concentration of viral nucleic acid and several samples with low concentration in one setting means necessity of repeated analysis beginning with stage of isolation of nucleic acid. The analysis of curves of accumulation of products of amplification, their forms and positioning on chart is the obligatory stage of polymerase chain reaction study in real time regimen. These actions permit to exclude the readouts of false negative testing results to departments. The study conclusions are equipotent for polymerase chain reaction testing of any nucleic acid targets.

10. Convergence in distribution for filtering processes associated to Hidden Markov Models with densities

OpenAIRE

Kaijser, Thomas

2013-01-01

A Hidden Markov Model generates two basic stochastic processes, a Markov chain, which is hidden, and an observation sequence. The filtering process of a Hidden Markov Model is, roughly speaking, the sequence of conditional distributions of the hidden Markov chain that is obtained as new observations are received. It is well-known, that the filtering process itself, is also a Markov chain. A classical, theoretical problem is to find conditions which implies that the distributions of the filter...

11. Meta-heuristic ant colony optimization technique to forecast the amount of summer monsoon rainfall: skill comparison with Markov chain model

Science.gov (United States)

Chaudhuri, Sutapa; Goswami, Sayantika; Das, Debanjana; Middey, Anirban

2014-05-01

Forecasting summer monsoon rainfall with precision becomes crucial for the farmers to plan for harvesting in a country like India where the national economy is mostly based on regional agriculture. The forecast of monsoon rainfall based on artificial neural network is a well-researched problem. In the present study, the meta-heuristic ant colony optimization (ACO) technique is implemented to forecast the amount of summer monsoon rainfall for the next day over Kolkata (22.6°N, 88.4°E), India. The ACO technique belongs to swarm intelligence and simulates the decision-making processes of ant colony similar to other adaptive learning techniques. ACO technique takes inspiration from the foraging behaviour of some ant species. The ants deposit pheromone on the ground in order to mark a favourable path that should be followed by other members of the colony. A range of rainfall amount replicating the pheromone concentration is evaluated during the summer monsoon season. The maximum amount of rainfall during summer monsoon season (June—September) is observed to be within the range of 7.5-35 mm during the period from 1998 to 2007, which is in the range 4 category set by the India Meteorological Department (IMD). The result reveals that the accuracy in forecasting the amount of rainfall for the next day during the summer monsoon season using ACO technique is 95 % where as the forecast accuracy is 83 % with Markov chain model (MCM). The forecast through ACO and MCM are compared with other existing models and validated with IMD observations from 2008 to 2012.

12. Time-domain induced polarization - an analysis of Cole-Cole parameter resolution and correlation using Markov Chain Monte Carlo inversion

Science.gov (United States)

Madsen, Line Meldgaard; Fiandaca, Gianluca; Auken, Esben; Christiansen, Anders Vest

2017-12-01

The application of time-domain induced polarization (TDIP) is increasing with advances in acquisition techniques, data processing and spectral inversion schemes. An inversion of TDIP data for the spectral Cole-Cole parameters is a non-linear problem, but by applying a 1-D Markov Chain Monte Carlo (MCMC) inversion algorithm, a full non-linear uncertainty analysis of the parameters and the parameter correlations can be accessed. This is essential to understand to what degree the spectral Cole-Cole parameters can be resolved from TDIP data. MCMC inversions of synthetic TDIP data, which show bell-shaped probability distributions with a single maximum, show that the Cole-Cole parameters can be resolved from TDIP data if an acquisition range above two decades in time is applied. Linear correlations between the Cole-Cole parameters are observed and by decreasing the acquisitions ranges, the correlations increase and become non-linear. It is further investigated how waveform and parameter values influence the resolution of the Cole-Cole parameters. A limiting factor is the value of the frequency exponent, C. As C decreases, the resolution of all the Cole-Cole parameters decreases and the results become increasingly non-linear. While the values of the time constant, τ, must be in the acquisition range to resolve the parameters well, the choice between a 50 per cent and a 100 per cent duty cycle for the current injection does not have an influence on the parameter resolution. The limits of resolution and linearity are also studied in a comparison between the MCMC and a linearized gradient-based inversion approach. The two methods are consistent for resolved models, but the linearized approach tends to underestimate the uncertainties for poorly resolved parameters due to the corresponding non-linear features. Finally, an MCMC inversion of 1-D field data verifies that spectral Cole-Cole parameters can also be resolved from TD field measurements.

13. Fast Markov chain Monte Carlo sampling for sparse Bayesian inference in high-dimensional inverse problems using L1-type priors

International Nuclear Information System (INIS)

Lucka, Felix

2012-01-01

Sparsity has become a key concept for solving of high-dimensional inverse problems using variational regularization techniques. Recently, using similar sparsity-constraints in the Bayesian framework for inverse problems by encoding them in the prior distribution has attracted attention. Important questions about the relation between regularization theory and Bayesian inference still need to be addressed when using sparsity promoting inversion. A practical obstacle for these examinations is the lack of fast posterior sampling algorithms for sparse, high-dimensional Bayesian inversion. Accessing the full range of Bayesian inference methods requires being able to draw samples from the posterior probability distribution in a fast and efficient way. This is usually done using Markov chain Monte Carlo (MCMC) sampling algorithms. In this paper, we develop and examine a new implementation of a single component Gibbs MCMC sampler for sparse priors relying on L1-norms. We demonstrate that the efficiency of our Gibbs sampler increases when the level of sparsity or the dimension of the unknowns is increased. This property is contrary to the properties of the most commonly applied Metropolis–Hastings (MH) sampling schemes. We demonstrate that the efficiency of MH schemes for L1-type priors dramatically decreases when the level of sparsity or the dimension of the unknowns is increased. Practically, Bayesian inversion for L1-type priors using MH samplers is not feasible at all. As this is commonly believed to be an intrinsic feature of MCMC sampling, the performance of our Gibbs sampler also challenges common beliefs about the applicability of sample based Bayesian inference. (paper)

14. Nation-Scale Adoption of Shorter Breast Radiation Therapy Schedules Can Increase Survival in Resource Constrained Economies: Results From a Markov Chain Analysis

International Nuclear Information System (INIS)

Khan, Atif J.; Rafique, Raza; Zafar, Waleed; Shah, Chirag; Haffty, Bruce G.; Vicini, Frank; Jamshed, Arif; Zhao, Yao

2017-01-01

15. Nation-Scale Adoption of Shorter Breast Radiation Therapy Schedules Can Increase Survival in Resource Constrained Economies: Results From a Markov Chain Analysis.

Science.gov (United States)

Khan, Atif J; Rafique, Raza; Zafar, Waleed; Shah, Chirag; Haffty, Bruce G; Vicini, Frank; Jamshed, Arif; Zhao, Yao

2017-02-01

16. Nation-Scale Adoption of Shorter Breast Radiation Therapy Schedules Can Increase Survival in Resource Constrained Economies: Results From a Markov Chain Analysis

Energy Technology Data Exchange (ETDEWEB)

Khan, Atif J., E-mail: atif.j.khan@rutgers.edu [Department of Radiation Oncology, Robert Wood Johnson Medical School/Cancer Institute of New Jersey, New Brunswick, New Jersey (United States); Rafique, Raza [Suleman Dawood School of Business, Lahore University of Management Sciences, Lahore (Pakistan); Zafar, Waleed [Shaukat Khanum Memorial Cancer Hospital and Research Centre, Lahore (Pakistan); Shah, Chirag [Department of Radiation Oncology, Cleveland Clinic, Cleveland, Ohio (United States); Haffty, Bruce G. [Department of Radiation Oncology, Robert Wood Johnson Medical School/Cancer Institute of New Jersey, New Brunswick, New Jersey (United States); Vicini, Frank [Michigan HealthCare Professionals, Farmington Hills, Michigan (United States); Jamshed, Arif [Shaukat Khanum Memorial Cancer Hospital and Research Centre, Lahore (Pakistan); Zhao, Yao [Rutgers University School of Business, Newark, New Jersey (United States)

2017-02-01

17. Application of integrated Bayesian modeling and Markov chain Monte Carlo methods to the conservation of a harvested species

Directory of Open Access Journals (Sweden)

Fonnesbeck, C. J.

2004-06-01

Full Text Available When endeavoring to make informed decisions, conservation biologists must frequently contend with disparate sources of data and competing hypotheses about the likely impacts of proposed decisions on the resource status. Frequently, statistical analyses, modeling (e.g., for population projection and optimization or simulation are conducted as separate exercises. For example, a population model might be constructed, whose parameters are then estimated from data (e.g., ringing studies, population surveys. This model might then be used to predict future population states, from current population estimates, under a particular management regime. Finally, the parameterized model might also be used to evaluate alternative candidate management decisions, via simulation, optimization, or both. This approach, while effective, does not take full advantage of the integration of data and model components for prediction and updating; we propose a hierarchical Bayesian context for this integration. In the case of American black ducks (Anas rubripes, managers are simultaneously faced with trying to extract a sustainable harvest from the species, while maintaining individual stocks above acceptable thresholds. The problem is complicated by spatial heterogeneity in the growth rates and carrying capacity of black ducks stocks, movement between stocks, regional differences in the intensity of harvest pressure, and heterogeneity in the degree of competition from a close congener, mallards (Anas platyrynchos among stocks. We have constructed a population life cycle model that takes these components into account and simultaneously performs parameter estimation and population prediction in a Bayesian framework. Ringing data are used to develop posterior predictive distributions for harvest mortality rates, given as input decisions about harvest regulations. Population surveys of black ducks and mallards are used to obtain stock-specific estimates of population size for

18. Markov reward processes

Science.gov (United States)

Smith, R. M.

1991-01-01

Numerous applications in the area of computer system analysis can be effectively studied with Markov reward models. These models describe the behavior of the system with a continuous-time Markov chain, where a reward rate is associated with each state. In a reliability/availability model, upstates may have reward rate 1 and down states may have reward rate zero associated with them. In a queueing model, the number of jobs of certain type in a given state may be the reward rate attached to that state. In a combined model of performance and reliability, the reward rate of a state may be the computational capacity, or a related performance measure. Expected steady-state reward rate and expected instantaneous reward rate are clearly useful measures of the Markov reward model. More generally, the distribution of accumulated reward or time-averaged reward over a finite time interval may be determined from the solution of the Markov reward model. This information is of great practical significance in situations where the workload can be well characterized (deterministically, or by continuous functions e.g., distributions). The design process in the development of a computer system is an expensive and long term endeavor. For aerospace applications the reliability of the computer system is essential, as is the ability to complete critical workloads in a well defined real time interval. Consequently, effective modeling of such systems must take into account both performance and reliability. This fact motivates our use of Markov reward models to aid in the development and evaluation of fault tolerant computer systems.

19. Using satellite observations to improve model estimates of CO2 and CH4 flux: a Metropolis Hastings Markov Chain Monte Carlo approach

Science.gov (United States)

MacBean, Natasha; Disney, Mathias; Lewis, Philip; Ineson, Phil

2010-05-01

profile as a whole. We present results from an Observing System Simulation Experiment (OSSE) designed to investigate the impact of management and climate change on peatland carbon fluxes, as well as how observations from satellites may be able to constrain modeled carbon fluxes. We use an adapted version of the Carnegie-Ames-Stanford Approach (CASA) model (Potter et al., 1993) that includes a representation of methane dynamics (Potter, 1997). The model formulation is further modified to allow for assimilation of satellite observations of surface soil moisture and land surface temperature. The observations are used to update model estimates using a Metropolis Hastings Markov Chain Monte Carlo (MCMC) approach. We examine the effect of temporal frequency and precision of satellite observations with a view to establishing how, and at what level, such observations would make a significant improvement in model uncertainty. We compare this with the system characteristics of existing and future satellites. We believe this is the first attempt to assimilate surface soil moisture and land surface temperature into an ecosystem model that includes a full representation of CH4 flux. Bubier, J., and T. Moore (1994), An ecological perspective on methane emissions from northern wetlands, TREE, 9, 460-464. Charman, D. (2002), Peatlands and Environmental Change, JohnWiley and Sons, Ltd, England. Gorham, E. (1991), Northern peatlands: Role in the carbon cycle and probable responses to climatic warming, Ecological Applications, 1, 182-195. Lai, D. (2009), Methane dynamics in northern peatlands: A review, Pedosphere, 19, 409-421. Le Mer, J., and P. Roger (2001), Production, oxidation, emission and consumption of methane by soils: A review, European Journal of Soil Biology, 37, 25-50. Limpens, J., F. Berendse, J. Canadell, C. Freeman, J. Holden, N. Roulet, H. Rydin, and Potter, C. (1997), An ecosystem simulation model for methane production and emission from wetlands, Global Biogeochemical

20. Entropy: The Markov Ordering Approach

Directory of Open Access Journals (Sweden)

Alexander N. Gorban

2010-05-01

Full Text Available The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant “additivity” properties: (i existence of a monotonic transformation which makes the functional additive with respect to the joining of independent systems and (ii existence of a monotonic transformation which makes the functional additive with respect to the partitioning of the space of states. All Lyapunov functionals for Markov chains which have properties (i and (ii are derived. We describe the most general ordering of the distribution space, with respect to which all continuous-time Markov processes are monotonic (the Markov order. The solution differs significantly from the ordering given by the inequality of entropy growth. For inference, this approach results in a convex compact set of conditionally “most random” distributions.

1. Finite Markov processes and their applications

CERN Document Server

Iosifescu, Marius

2007-01-01

A self-contained treatment of finite Markov chains and processes, this text covers both theory and applications. Author Marius Iosifescu, vice president of the Romanian Academy and director of its Center for Mathematical Statistics, begins with a review of relevant aspects of probability theory and linear algebra. Experienced readers may start with the second chapter, a treatment of fundamental concepts of homogeneous finite Markov chain theory that offers examples of applicable models.The text advances to studies of two basic types of homogeneous finite Markov chains: absorbing and ergodic ch

2. Lie Markov models with purine/pyrimidine symmetry.

Science.gov (United States)

Fernández-Sánchez, Jesús; Sumner, Jeremy G; Jarvis, Peter D; Woodhams, Michael D

2015-03-01

Continuous-time Markov chains are a standard tool in phylogenetic inference. If homogeneity is assumed, the chain is formulated by specifying time-independent rates of substitutions between states in the chain. In applications, there are usually extra constraints on the rates, depending on the situation. If a model is formulated in this way, it is possible to generalise it and allow for an inhomogeneous process, with time-dependent rates satisfying the same constraints. It is then useful to require that, under some time restrictions, there exists a homogeneous average of this inhomogeneous process within the same model. This leads to the definition of "Lie Markov models" which, as we will show, are precisely the class of models where such an average exists. These models form Lie algebras and hence concepts from Lie group theory are central to their derivation. In this paper, we concentrate on applications to phylogenetics and nucleotide evolution, and derive the complete hierarchy of Lie Markov models that respect the grouping of nucleotides into purines and pyrimidines-that is, models with purine/pyrimidine symmetry. We also discuss how to handle the subtleties of applying Lie group methods, most naturally defined over the complex field, to the stochastic case of a Markov process, where parameter values are restricted to be real and positive. In particular, we explore the geometric embedding of the cone of stochastic rate matrices within the ambient space of the associated complex Lie algebra.

3. Markov-modulated and feedback fluid queues

NARCIS (Netherlands)

Scheinhardt, Willem R.W.

1998-01-01

In the last twenty years the field of Markov-modulated fluid queues has received considerable attention. In these models a fluid reservoir receives and/or releases fluid at rates which depend on the actual state of a background Markov chain. In the first chapter of this thesis we give a short

4. Markov Chain Monte Carlo Methods

... Dirichlet prior; Metropolis-Hastings algorithm; rejection sampling; Gibbs sampler; proposal density; Rao-Blackwellisation; binomial; multinomial; Gamma; uniform. ... School of ORIE Rhodes Hall Cornell University, Ithaca New York 14853, USA; Indian Statistical Institute 8th Mile, Mysore Road Bangalore 560059, India.

5. Markov Chain Monte Carlo Methods

K B Athreya1 Mohan Delampady2 T Krishnan3. School of ORIE Rhodes Hall Cornell University, Ithaca New York 14853, USA. Indian Statistical Institute 8th Mile, Mysore Rood Bangalore 560 059, India. Systat Software Asia-Pacific Ltd. Floor 5, 'C' Tower Golden Enclave, Airport Rood Bangalore 560 017, India.

6. Markov Chain Monte Carlo Methods

listening to Indian classical music. Mohan Delampady is at the. Indian Statistical Institute,. Bangalore. His research interests include robustness, nonparametric inference and computing in Bayesian statistics. T Krishnan is now a full- time Technical Consultant t(J. Systat Software Asia-Pacific. Ltd., in Bangalore, where the.

7. Markov Chain Monte Carlo Methods

Author Affiliations. K B Athreya1 Mohan Delampady2 T Krishnan3. School of ORIE Rhodes Hall Cornell University, Ithaca New York 14853, USA; Indian Statistical Institute 8th Mile, Mysore Road Bangalore 560059, India. Systat Software Asia-Pacific Ltd. Floor 5, 'C' Tower Golden Enclave, Airport Road Bangalore 560 017, ...

8. Markov Chain Monte Carlo Methods

Author Affiliations. K B Athreya1 Mohan Delampady2 T Krishnan3. School of ORIE Rhodes Hall Cornell University, Ithaca New York 14853, USA; Indian Statistical Institute 8th Mile, Mysore Road Bangalore 560 059, India. Systat Software Asia-Pacific Ltd. Floor 5, 'C' Tower Golden Enclave, Airport Road Bangalore 560 017, ...

9. Markov Chain Monte Carlo Methods

Author Affiliations. K B Athreya1 Mohan Delampady2 T Krishnan3. School of ORIE Rhodes Hall Cornell University, Ithaca New York 14853, USA. Indian Statistical Institute 8th Mile, Mysore Rood Bangalore 560 059, India. Systat Software Asia-Pacific Ltd. Floor 5, 'C' Tower Golden Enclave, Airport Rood Bangalore 560 017, ...

10. The role of uncertainty in supply chains under dynamic modeling

Directory of Open Access Journals (Sweden)

M. Fera

2017-01-01

Full Text Available The uncertainty in the supply chains (SCs for manufacturing and services firms is going to be, over the coming decades, more important for the companies that are called to compete in a new globalized economy. Risky situations for manufacturing are considered in trying to individuate the optimal positioning of the order penetration point (OPP. It aims at defining the best level of information of the client’s order going back through the several supply chain (SC phases, i.e. engineering, procurement, production and distribution. This work aims at defining a system dynamics model to assess competitiveness coming from the positioning of the order in different SC locations. A Taguchi analysis has been implemented to create a decision map for identifying possible strategic decisions under different scenarios and with alternatives for order location in the SC levels. Centralized and decentralized strategies for SC integration are discussed. In the model proposed, the location of OPP is influenced by the demand variation, production time, stock-outs and stock amount. Results of this research are as follows: (i customer-oriented strategies are preferable under high volatility of demand, (ii production-focused strategies are suggested when the probability of stock-outs is high, (iii no specific location is preferable if a centralized control architecture is implemented, (iv centralization requires cooperation among partners to achieve the SC optimum point, (v the producer must not prefer the OPP location at the Retailer level when the general strategy is focused on a decentralized approach.

11. The effect of reproductive performance on the dairy cattle herd value assessed by integrating a daily dynamic programming model with a daily Markov chain model.

Science.gov (United States)

Kalantari, A S; Cabrera, V E

2012-10-01

The objective of this study was to determine the effect of reproductive performance on dairy cattle herd value. Herd value was defined as the herd's average retention payoff (RPO). Individual cow RPO is the expected profit from keeping the cow compared with immediate replacement. First, a daily dynamic programming model was developed to calculate the RPO of all cow states in a herd. Second, a daily Markov chain model was applied to estimate the herd demographics. Finally, the herd value was calculated by aggregating the RPO of all cows in the herd. Cow states were described by 5 milk yield classes (76, 88, 100, 112, and 124% with respect to the average), 9 lactations, 750 d in milk, and 282 d in pregnancy. Five different reproductive programs were studied (RP1 to RP5). Reproductive program 1 used 100% timed artificial insemination (TAI; 42% conception rate for first TAI and 30% for second and later services) and the other programs combined TAI with estrus detection. The proportion of cows receiving artificial insemination after estrus detection ranged from 30 to 80%, and conception rate ranged from 25 to 35%. These 5 reproductive programs were categorized according to their 21-d pregnancy rate (21-d PR), which is an indication of the rate that eligible cows become pregnant every 21 d. The 21-d PR was 17% for RP1, 14% for RP2, 16% for RP3, 18% for RP4, and 20% for RP5. Results showed a positive relationship between 21-d PR and herd value. The most extreme herd value difference between 2 reproductive programs was $77/cow per yr for average milk yield (RP5 - RP2),$13/cow per yr for lowest milk yield (RP5 - RP1), and $160/cow per yr for highest milk yield (RP5 - RP2). Reproductive programs were ranked based on their calculated herd value. With the exception of the best reproductive program (RP5), all other programs showed some level of ranking change according to milk yield. The most dramatic ranking change was observed in RP1, which moved from being the worst ranked 12. A daily herd Markov-chain model to study the reproductive and economic impact of reproductive programs combining timed artificial insemination and estrus detection. Science.gov (United States) Giordano, J O; Kalantari, A S; Fricke, P M; Wiltbank, M C; Cabrera, V E 2012-09-01 Our objective was to compare the economic and reproductive performance of programs combining timed artificial insemination (TAI) and different levels of AI after estrus detection (ED) using a daily Markov-chain model. A dairy herd was modeled with every cow following daily probabilistic events of aging, replacement, mortality, pregnancy, pregnancy loss, and calving. The probability of pregnancy depended on the combination of probability of insemination and conception rate (CR). All nonpregnant cows had a probability of pregnancy between the end of the voluntary waiting period and days in milk cutoff for AI. After the cutoff, cows were labeled as do not breed and replaced when milk production was below a minimum milk threshold. A similar model was created to represent a replacement heifer herd to simulate and adjust the supply and demand of replacements. The net value (NV) of a program was the sum of milk income over feed cost, replacement and mortality cost, income from newborns, and reproductive costs. The model was used to compare the NV of 19 programs. One program used 100% TAI (42% CR for first TAI and 30% for second-and-later services), whereas the other programs combined TAI with ED. The proportion of cows receiving AI after ED for the combined programs ranged from 30 to 80%, with levels of CR of 25, 30, and 35%. As the proportion of cows receiving AI after ED increased, the CR of cows receiving TAI decreased. The combined programs with CR of 35% for cows receiving AI after ED had the greatest NV and reproductive performance at all levels of ED. The program using 100% TAI had greater NV and better reproductive performance than all programs with 25% CR after ED inseminations, whereas it had very similar performance to combined programs with up to 60% of cows receiving AI after ED and 30% CR. The factor with the greatest relative contribution to the differences among programs was income over feed cost, followed by replacement and reproductive costs. Adjusting 13. Supply Chain Collaboration under Uncertainty in the Albanian Beer Market Directory of Open Access Journals (Sweden) Denisa MAMILLO 2015-03-01 Full Text Available Today supply chain uncertainty is higher due to the global crisis, the fast changing technology and the increasing vulnerability of supply chains. Companies use different strategies to reduce uncertainty, like building agile supply chains, increasing resilience, postponement, etc. All these strategies require strong supply chain collaboration. Although research interest in supply chain collaboration is growing, no research has been done in Albania. This paper is one of the first to investigate supply chain management practices and the extent of supply chain collaboration in the Albanian beer industry. The aim of this research is twofold: first, to investigate how supply chain uncertainty influences the extent of collaboration with the supply chain members, and second, to analyze how organizational culture facilitates the collaboration process. Semi-structured interviews were conducted with the managers of the main beer companies. A guide questionnaire was prepared. It consisted of open and rate-scale questions about supply chain collaboration, supply chain uncertainty, supply chain management practices and organizational culture. The research will show that a high level of supply chain uncertainty does not always lead to a high degree of collaboration with the supply chain members. Organizational culture is the key driver of a successful collaboration. Not all types of culture can facilitate collaboration but only the ones with an external orientation. 14. Algorithmic analysis of the maximum level length in general-block two-dimensional Markov processes Directory of Open Access Journals (Sweden) 2006-01-01 Full Text Available Two-dimensional continuous-time Markov chains (CTMCs are useful tools for studying stochastic models such as queueing, inventory, and production systems. Of particular interest in this paper is the distribution of the maximal level visited in a busy period because this descriptor provides an excellent measure of the system congestion. We present an algorithmic analysis for the computation of its distribution which is valid for Markov chains with general-block structure. For a multiserver batch arrival queue with retrials and negative arrivals, we exploit the underlying internal block structure and present numerical examples that reveal some interesting facts of the system. 15. Equivalence of chain conformations in the surface region of a polymer melt and a single Gaussian chain under critical conditions. Science.gov (United States) Skvortsov, A M; Leermakers, F A M; Fleer, G J 2013-08-07 In the melt polymer conformations are nearly ideal according to Flory's ideality hypothesis. Silberberg generalized this statement for chains in the interfacial region. We check the Silberberg argument by analyzing the conformations of a probe chain end-grafted at a solid surface in a sea of floating free chains of concentration φ by the self-consistent field (SCF) method. Apart from the grafting, probe chain and floating chains are identical. Most of the results were obtained for a standard SCF model with freely jointed chains on a six-choice lattice, where immediate step reversals are allowed. A few data were generated for a five-choice lattice, where such step reversals are forbidden. These coarse-grained models describe the equilibrium properties of flexible atactic polymer chains at the scale of the segment length. The concentration was varied over the whole range from φ = 0 (single grafted chain) to φ = 1 (probe chain in the melt). The number of contacts with the surface, average height of the free end and its dispersion, average loop and train length, tail size distribution, end-point and overall segment distributions were calculated for a grafted probe chain as a function of φ, for several chain lengths and substrate∕polymer interactions, which were varied from strong repulsion to strong adsorption. The computations show that the conformations of the probe chain in the melt do not depend on substrate∕polymer interactions and are very similar to the conformations of a single end-grafted chain under critical conditions, and can thus be described analytically. When the substrate∕polymer interaction is fixed at the value corresponding to critical conditions, all equilibrium properties of a probe chain are independent of φ, over the whole range from a dilute solution to the melt. We believe that the conformations of all flexible chains in the surface region of the melt are close to those of an appropriate single chain in critical conditions, provided 16. Reviving Markov processes and applications International Nuclear Information System (INIS) Cai, H. 1988-01-01 In this dissertation we study a procedure which restarts a Markov process when the process is killed by some arbitrary multiplicative functional. The regenerative nature of this revival procedure is characterized through a Markov renewal equation. An interesting duality between the revival procedure and the classical killing operation is found. Under the condition that the multiplicative functional possesses an intensity, the generators of the revival process can be written down explicitly. An intimate connection is also found between the perturbation of the sample path of a Markov process and the perturbation of a generator (in Kato's sense). The applications of the theory include the study of the processes like piecewise-deterministic Markov process, virtual waiting time process and the first entrance decomposition (taboo probability) 17. Recursive Markov Process OpenAIRE Hidaka, Shohei 2015-01-01 A Markov process, which is constructed recursively, arises in stochastic games with Markov strategies. In this study, we defined a special class of random processes called the recursive Markov process, which has infinitely many states but can be expressed in a closed form. We derive the characteristic equation which the marginal stationary distribution of an arbitrary recursive Markov process needs to satisfy. 18. Stability of llama heavy chain antibody fragments under extreme conditions NARCIS (Netherlands) Dolk, E. 2004-01-01 Camelids have next to their normal antibodies, a unique subset of antibodies lacking light chains. The resulting single binding domain, VHH, of these heavy chain antibodies consequently have unique properties. A high stability is one of these properties, which was investigated in this thesis. The 19. Comparing Cost Of New Supply Chain Designs Under Uncertainty DEFF Research Database (Denmark) Wæhrens, Brian Vejrum; Kristensen, Jesper; Asmussen, Jesper Normann 2016-01-01 Accounting, Operational Modelling and SCM inform decision making for new SCDs. Through four embedded cases, a gap is found between the practice of a global OEM and literature. Results shows complications when assessing SCDs due to limited understanding of the internal activity costs, supply chain dynamics......Companies pursuing improvements to their global supply chain (SC) are challenged with unravelling the true cost of operating with their supply chain design (SCD). This challenge is further intensified as SCs are faced with increasing uncertainty. To rectify this it is investigated how Cost...... and how uncertainties affects the SC cost performance.... 20. What Happens When the Supply Chain Breaks? Implications for the Army Supply Chain Under Attack National Research Council Canada - National Science Library DeBusk, Steven 2003-01-01 .... In a system of tightly linked supply chains consisting of consumers, retailers, suppliers, and manufacturers, a sudden change in their ability to communicate data or distribute product can have a... 1. USO DE CADENAS DE MARKOV PARA LA PREDICCIÓN DE LA DINÁMICA DEL COMPORTAMIENTO DE PACIENTES EN UNA UNIDAD DE CUIDADO INTENSIVO CARDIOLÓGICA USE OF MARKOV CHAINS IN PREDICTION OF THE DYNAMICS OF PATIENT BEHAVIOR IN A CARDIOLOGIC INTENSIVE CARE UNIT Directory of Open Access Journals (Sweden) Víctor Albornoz 2006-08-01 Full Text Available En este trabajo presentamos un modelo probabilístico que contribuye al estudio de la dinámica en el comportamiento y permanencia de pacientes en una unidad de cuidados intensivos cardiológica. El modelo utilizado corresponde a una Cadena de Markov en tiempo discreto, que mediante la definición de determinados niveles de gravedad de un paciente (estados y la obtención de las correspondientes probabilidades de transición entre un nivel de gravedad y otro, permite predecir los tiempos de permanencia. Los diferentes estados empleados se basan en la construcción de un nuevo score creado para este propósito. Se muestran los detalles de la metodología adoptada y los principales resultados alcanzados en la aplicación del modelo empleado.In this paper we present a probabilistic model that contributes to the study of dynamics in the behavior and permanence of patients in a cardiovascular intensive care unit. The model corresponds to a discrete Markov Chain, that allows to predict the time that a patient remains in the system through the time, by means of certain severity of illness states and the corresponding transition probabilities between those states. The different states are based on the construction of a new score created for this study. We summarize the details of the adopted methodology and the main results reached in the application of the model. 2. A robust optimization model for agile and build-to-order supply chain planning under uncertainties DEFF Research Database (Denmark) Lalmazloumian, Morteza; Wong, Kuan Yew; Govindan, Kannan 2016-01-01 Supply chain planning as one of the most important processes within the supply chain management concept, has a great impact on firms' success or failure. This paper considers a supply chain planning problem of an agile manufacturing company operating in a build-to-order environment under various... 3. First passage of time-reversible spectrally negative Markov additive processes NARCIS (Netherlands) Ivanovs, J.; Mandjes, M. 2010-01-01 We study the first passage process of a spectrally negative Markov additive process (MAP). The focus is on the background Markov chain at the times of the first passage. This process is a Markov chain itself with a transition rate matrix Λ. Assuming time reversibility, we show that all the 4. Financial Applications of Bivariate Markov Processes OpenAIRE Ortobelli Lozza, Sergio; Angelelli, Enrico; Bianchi, Annamaria 2011-01-01 This paper describes a methodology to approximate a bivariate Markov process by means of a proper Markov chain and presents possible financial applications in portfolio theory, option pricing and risk management. In particular, we first show how to model the joint distribution between market stochastic bounds and future wealth and propose an application to large-scale portfolio problems. Secondly, we examine an application to VaR estimation. Finally, we propose a methodology... 5. The value of supply chain coordination under moral hazard: A case study of the consumer product supply chain Science.gov (United States) Lee, Yumi; Song, Sang Hwa 2018-01-01 In this paper, we examine a real-world case related to the consumer product supply chain to analyze the value of supply chain coordination under the condition of moral hazard. Because of the characteristics of a buyback contract scheme employed in the supply chain, the supplier company’s sales department encourages retailers to order more inventory to meet their sales target, whereas retailers pay less attention to their inventory level and leftovers at the end of the season. This condition induces moral hazard problems in the operation of the supply chain, as suppliers suffer from huge returns of leftover inventory. This, in turn, is related to the obsolescence of returned inventory, even with penalty terms in the contract for the return of any leftovers. In this study, we show under the current buyback-based supply chain operation, the inventory levels of both the supplier and retailers exceed customer demand and develop vendor-managed inventory (VMI) system with base stock policy to remove any mismatch of supply and demand. A comparison of both systems shows that through the proper coordination of supply chain operations, both suppliers and retailers can gain additional benefits while providing proper services to end customers. PMID:29547625 6. The value of supply chain coordination under moral hazard: A case study of the consumer product supply chain. Science.gov (United States) Lee, Yumi; Song, Sang Hwa; Cheong, Taesu 2018-01-01 In this paper, we examine a real-world case related to the consumer product supply chain to analyze the value of supply chain coordination under the condition of moral hazard. Because of the characteristics of a buyback contract scheme employed in the supply chain, the supplier company's sales department encourages retailers to order more inventory to meet their sales target, whereas retailers pay less attention to their inventory level and leftovers at the end of the season. This condition induces moral hazard problems in the operation of the supply chain, as suppliers suffer from huge returns of leftover inventory. This, in turn, is related to the obsolescence of returned inventory, even with penalty terms in the contract for the return of any leftovers. In this study, we show under the current buyback-based supply chain operation, the inventory levels of both the supplier and retailers exceed customer demand and develop vendor-managed inventory (VMI) system with base stock policy to remove any mismatch of supply and demand. A comparison of both systems shows that through the proper coordination of supply chain operations, both suppliers and retailers can gain additional benefits while providing proper services to end customers. 7. Supply chain value creation methodology under BSC approach Science.gov (United States) Golrizgashti, Seyedehfatemeh 2014-06-01 The objective of this paper is proposing a developed balanced scorecard approach to measure supply chain performance with the aim of creating more value in manufacturing and business operations. The most important metrics have been selected based on experts' opinion acquired by in-depth interviews focused on creating more value for stakeholders. Using factor analysis method, a survey research has been used to categorize selected metrics into balanced scorecard perspectives. The result identifies the intensity of correlation between perspectives and cause-and-effect chains among them using statistical method based on a real case study in home appliance manufacturing industries. 8. Time-homogeneous Markov process for HIV/AIDS progression under a combination treatment therapy: cohort study, South Africa. Science.gov (United States) Shoko, Claris; Chikobvu, Delson 2018-01-18 As HIV enters the human body, its main target is the CD4 cell which it turns into a factory that produces millions of other HIV particles. These HIV particles target new CD4 cells resulting in the progression of HIV infection to AIDS. A continuous depletion of CD4 cells results in opportunistic infections, for example tuberculosis (TB). The purpose of this study is to model and describe the progression of HIV/AIDS disease in an individual on antiretroviral therapy (ART) follow up using a continuous time homogeneous Markov process. A cohort of 319 HIV infected patients on ART follow up at a Wellness Clinic in Bela Bela, South Africa is used in this study. Though Markov models based on CD4 cell counts is a common approach in HIV/AIDS modelling, this paper is unique clinically in that tuberculosis (TB) co-infection is included as a covariate. The method partitions the HIV infection period into five CD4-cell count intervals followed by the end points; death, and withdrawal from study. The effectiveness of treatment is analysed by comparing the forward transitions with the backward transitions. The effects of reaction to treatment, TB co-infection, gender and age on the transition rates are also examined. The developed models give very good fit to the data. The results show that the strongest predictor of transition from a state of CD4 cell count greater than 750 to a state of CD4 between 500 and 750 is a negative reaction to drug therapy. Development of TB during the course of treatment is the greatest predictor of transitions to states of lower CD4 cell count. Transitions from good states to bad states are higher on male patients than their female counterparts. Patients in the cohort spend a greater proportion of their total follow-up time in higher CD4 states. From some of these findings we conclude that there is need to monitor adverse reaction to drugs more frequently, screen HIV/AIDS patients for any signs and symptoms of TB and check for factors that may explain 9. Special issue: Supply chain and logistics decisions under environmental aspect Directory of Open Access Journals (Sweden) Neslihan Demirel 2017-05-01 Full Text Available This editor’s note has a twofold objective: (1 to present a brief summary about the environmental issues in supply chain and logistics decisions (2 to present general information about the papers published in the special issue. 10. Robust environmental closed-loop supply chain design under uncertainty International Nuclear Information System (INIS) MA, Ruimin; YAO, Lifei; JIN, Maozhu; REN, Peiyu; LV, Zhihan 2016-01-01 With the fast developments in product remanufacturing to improve economic and environmental performance, an environmental closed-loop supply (ECLSC) chain is important for enterprises' competitiveness. In this paper, a robust ECLSC network is investigated which includes multiple plants, collection centers, demand zones, and products, and consists of both forward and reverse supply chains. First, a robust multi-objective mixed integer nonlinear programming model is proposed to deal with ECLSC considering two conflicting objectives simultaneously, as well as the uncertain nature of the supply chain. Cost parameters of the supply chain and demand fluctuations are subject to uncertainty. The first objective function aims to minimize the economical cost and the second objective function is to minimize the environmental influence. Then, the proposed model is solved as a single-objective mixed integer programming model applying the LP-metrics method. Finally, numerical example has been presented to test the model. The results indicate that the proposed model is applicable in practice. 11. Life cycle cost optimization of biofuel supply chains under uncertainties based on interval linear programming DEFF Research Database (Denmark) Ren, Jingzheng; Dong, Liang; Sun, Lu 2015-01-01 The aim of this work was to develop a model for optimizing the life cycle cost of biofuel supply chain under uncertainties. Multiple agriculture zones, multiple transportation modes for the transport of grain and biofuel, multiple biofuel plants, and multiple market centers were considered...... model, and the results showed that the proposed model is feasible for designing biofuel supply chain under uncertainties... 12. Designing a Supply Chain Network under the Risk of Disruptions Directory of Open Access Journals (Sweden) Armin Jabbarzadeh 2012-01-01 Full Text Available This paper studies a supply chain design problem with the risk of disruptions at facilities. At any point of time, the facilities are subject to various types of disruptions caused by natural disasters, man-made defections, and equipment breakdowns. We formulate the problem as a mixed-integer nonlinear program which maximizes the total profit for the whole system. The model simultaneously determines the number and location of facilities, the subset of customers to serve, the assignment of customers to facilities, and the cycle-order quantities at facilities. In order to obtain near-optimal solutions with reasonable computational requirements for large problem instances, two solution methods based on Lagrangian relaxation and genetic algorithm are developed. The effectiveness of the proposed solution approaches is shown using numerical experiments. The computational results, in addition, demonstrate that the benefits of considering disruptions in the supply chain design model can be significant. 13. Ferrofluid aggregation in chains under the influence of a magnetic field International Nuclear Information System (INIS) Ivanov, Alexey O.; Kantorovich, Sofia S.; Mendelev, Valentin S.; Pyanzina, Elena S. 2006-01-01 The paper is devoted to the basic problem of chain aggregate formation in magnetic fluids under the influence of an external magnetic field. Chain distribution in dynamic equilibrium is obtained on the basis of free energy minimization method under the condition when the interparticle dipole-dipole interaction between the nearest neighboring ferroparticles in each chain is taken into account. The modified mean field approach is used for considering the dipole-dipole interaction between all particles in a ferrofluid. The model describes well the molecular dynamics simulations of magnetostatic properties for monodisperse ferrofluids containing chain aggregates 14. A Markov decision model for optimising economic production lot size ... African Journals Online (AJOL) Adopting such a Markov decision process approach, the states of a Markov chain represent possible states of demand. The decision of whether or not to produce additional inventory units is made using dynamic programming. This approach demonstrates the existence of an optimal state-dependent EPL size, and produces ... 15. two-level inventory optimization under probability event chain African Journals Online (AJOL) . ... Journal of Modeling, Design and Management of Engineering Systems ... It is shown that optimization and coordination of the safety factor lead to inventory cost savings at two sites, especially under large lead time variability and stock ... 16. Non-markovian limits of additive functionals of Markov processes OpenAIRE Jara, Milton; Komorowski, Tomasz 2009-01-01 In this paper we consider an additive functional of an observable$V(x)$of a Markov jump process. We assume that the law of the expected jump time$t(x)$under the invariant probability measure$\\pi$of the skeleton chain belongs to the domain of attraction of a subordinator. Then, the scaled limit of the functional is a Mittag-Leffler proces, provided that$\\Psi(x):=V(x)t(x)$is square integrable w.r.t.$\\pi$. When the law of$\\Psi(x)\$ belongs to a domain of attraction of a stable law the r...

17. The research on optimization of auto supply chain network robust model under macroeconomic fluctuations

International Nuclear Information System (INIS)

Guo, Chunxiang; Liu, Xiaoli; Jin, Maozhu; Lv, Zhihan

2016-01-01

Considering the uncertainty of the macroeconomic environment, the robust optimization method is studied for constructing and designing the automotive supply chain network, and based on the definition of robust solution a robust optimization model is built for integrated supply chain network design that consists of supplier selection problem and facility location–distribution problem. The tabu search algorithm is proposed for supply chain node configuration, analyzing the influence of the level of uncertainty on robust results, and by comparing the performance of supply chain network design through the stochastic programming model and robustness optimize model, on this basis, determining the rational layout of supply chain network under macroeconomic fluctuations. At last the contrastive test result validates that the performance of tabu search algorithm is outstanding on convergence and computational time. Meanwhile it is indicated that the robust optimization model can reduce investment risks effectively when it is applied to supply chain network design.

18. Context Tree Estimation in Variable Length Hidden Markov Models

OpenAIRE

Dumont, Thierry

2011-01-01

We address the issue of context tree estimation in variable length hidden Markov models. We propose an estimator of the context tree of the hidden Markov process which needs no prior upper bound on the depth of the context tree. We prove that the estimator is strongly consistent. This uses information-theoretic mixture inequalities in the spirit of Finesso and Lorenzo(Consistent estimation of the order for Markov and hidden Markov chains(1990)) and E.Gassiat and S.Boucheron (Optimal error exp...

19. Semi-Markov processes

CERN Document Server

Grabski

2014-01-01

Semi-Markov Processes: Applications in System Reliability and Maintenance is a modern view of discrete state space and continuous time semi-Markov processes and their applications in reliability and maintenance. The book explains how to construct semi-Markov models and discusses the different reliability parameters and characteristics that can be obtained from those models. The book is a useful resource for mathematicians, engineering practitioners, and PhD and MSc students who want to understand the basic concepts and results of semi-Markov process theory. Clearly defines the properties and

20. Learning Markov models for stationary system behaviors

DEFF Research Database (Denmark)

Chen, Yingke; Mao, Hua; Jaeger, Manfred

2012-01-01

to a single long observation sequence, and in these situations existing automatic learning methods cannot be applied. In this paper, we adapt algorithms for learning variable order Markov chains from a single observation sequence of a target system, so that stationary system properties can be verified using...

1. Pruning Boltzmann networks and hidden Markov models

DEFF Research Database (Denmark)

Pedersen, Morten With; Stork, D.

1996-01-01

Boltzmann chains and hidden Markov models (HMMs), we argue that our method can be applied to HMMs as well. We illustrate pruning on Boltzmann zippers, which are equivalent to two HMMs with cross-connection links. We verify that our second-order approximation preserves the rank ordering of weight saliencies...

2. Maximizing entropy over Markov processes

DEFF Research Database (Denmark)

Biondi, Fabrizio; Legay, Axel; Nielsen, Bo Friis

2014-01-01

computation reduces to finding a model of a specification with highest entropy. Entropy maximization for probabilistic process specifications has not been studied before, even though it is well known in Bayesian inference for discrete distributions. We give a characterization of global entropy of a process...... as a reward function, a polynomial algorithm to verify the existence of a system maximizing entropy among those respecting a specification, a procedure for the maximization of reward functions over Interval Markov Chains and its application to synthesize an implementation maximizing entropy. We show how...

3. Maximizing Entropy over Markov Processes

DEFF Research Database (Denmark)

Biondi, Fabrizio; Legay, Axel; Nielsen, Bo Friis

2013-01-01

computation reduces to finding a model of a specification with highest entropy. Entropy maximization for probabilistic process specifications has not been studied before, even though it is well known in Bayesian inference for discrete distributions. We give a characterization of global entropy of a process...... as a reward function, a polynomial algorithm to verify the existence of an system maximizing entropy among those respecting a specification, a procedure for the maximization of reward functions over Interval Markov Chains and its application to synthesize an implementation maximizing entropy. We show how...

4. Impact of Asymmetric Carbon Information on Supply Chain Decisions under Low-Carbon Policies

Directory of Open Access Journals (Sweden)

Lei Yang

2016-01-01

Full Text Available Through the establishment of the leading manufacturer Stackelberg game model under asymmetric carbon information, this paper investigates the misreporting behaviors of the supply chain members and their influences on supply chain performance. Based on “Benchmarking” allocation mechanism, three policies are considered: carbon emission trading, carbon tax, and a new policy which combined carbon quota and carbon tax mechanism. The results show that, in the three models, the leader in the supply chain, even if he has advantages of carbon information, will not lie about his information. That is because the manufacturer’s misreporting behavior has no effect on supply chain members’ performance. But the retailer will lie about the information when he has carbon information advantage. The high-carbon-emission retailers under the carbon trading policy, all the retailers under the carbon tax policy, and the high-carbon-emission retailers under combined quotas and tax policy would like to understate their carbon emissions. Coordination of revenue sharing contract is studied in supply chain to induce the retailer to declare his real carbon information. Optimal contractual parameters are deduced in the three models, under which the profit of the supply chain can be maximized.

5. Studi Performansi Pada Peta Kendali Variabel Dengan Pendekatan Rantai Markov

OpenAIRE

2005-01-01

Many products which are non-conforming with specification from company decision require the development and improvement of control chart to investigation and problematic analyzing. One of tool that used to get information of improvement action is control chart variable with Markov Chain analysis to increase of performance plywood product quality.To acquire information about variable of control chart with Markov Chain analysis can be watched on ARL value, data plotted pattern and calculating o...

6. Optimizing pricing and ordering strategies in a three-level supply chain under return policy

Science.gov (United States)

2018-03-01

This paper develops an economic production quantity model in a three-echelon supply chain composing of a supplier, a manufacturer and a wholesaler under two scenarios. As the first scenario, we consider a return contract between the outside supplier and the supplier and also between the manufacturer and the wholesaler, but in the second one, the return policy between the manufacturer and the wholesaler is not applied. Here, it is assumed that shortage is permitted and demand is price-sensitive. The principal goal of the research is to maximize the total profit of the chain by optimizing the order quantity of the supplier and the selling prices of the manufacturer and the wholesaler. Nash-equilibrium approach is considered between the chain members. In the end, a numerical example is presented to clarify the applicability of the introduced model and compare the profit of the chain under two scenarios.

7. Life Cycle Energy and CO2 Emission Optimization for Biofuel Supply Chain Planning under Uncertainties

DEFF Research Database (Denmark)

Ren, Jingzheng; An, Da; Liang, Hanwei

2016-01-01

The purpose of this paper is to develop a model for the decision-makers/stakeholders to design biofuel supply chain under uncertainties. Life cycle energy and CO2 emission of biofuel supply chain are employed as the objective functions, multiple feedstocks, multiple transportation modes, multiple...... in this study. A bi-objective interval mix integer programming model has been developed for biofuel supply chain design under uncertainties, and the bio-objective interval programming method has been developed to solve this model. An illustrative case of a multiple-feedstock-bioethanol system has been studied...... by the proposed method, and the results show that the proposed model can help decision-makers/stakeholders plan and design the biofuel supply chain by proposing feasible solutions to them....

8. Markov bridges, bisection and variance reduction

DEFF Research Database (Denmark)

Asmussen, Søren; Hobolth, Asger

Time-continuous Markov jump processes is a popular modelling tool in disciplines ranging from computational finance and operations research to human genetics and genomics. The data is often sampled at discrete points in time, and it can be useful to simulate sample paths between the datapoints....... In this paper we firstly consider the problem of generating sample paths from a continuous-time Markov chain conditioned on the endpoints using a new algorithm based on the idea of bisection. Secondly we study the potential of the bisection algorithm for variance reduction. In particular, examples are presented...

9. LAND USE/LAND COVER CHANGES IN SEMI-ARID MOUNTAIN LANDSCAPE IN SOUTHERN INDIA: A GEOINFORMATICS BASED MARKOV CHAIN APPROACH

Directory of Open Access Journals (Sweden)

S. A. Rahaman

2017-05-01

Full Text Available Nowadays land use/ land cover in mountain landscape is in critical condition; it leads to high risky and uncertain environments. These areas are facing multiple stresses including degradation of land resources; vagaries of climate and depletion of water resources continuously affect land use practices and livelihoods. To understand the Land use/Land cover (Lu/Lc changes in a semi-arid mountain landscape, Kallar watershed of Bhavani basin, in southern India has been chosen. Most of the hilly part in the study area covers with forest, plantation, orchards and vegetables and which are highly affected by severe soil erosion, landslide, frequent rainfall failures and associated drought. The foothill regions are mainly utilized for agriculture practices; due to water scarcity and meagre income, the productive agriculture lands are converted into settlement plots and wasteland. Hence, land use/land cover change deduction; a stochastic processed based method is indispensable for future prediction. For identification of land use/land cover, and vegetation changes, Landsat TM, ETM (1995, 2005 and IRS P6- LISS IV (2015 images were used. Through CAMarkov chain analysis, Lu/Lc changes in past three decades (1995, 2005, and 2015 were identified and projected for (2020 and 2025; Normalized Difference Vegetation Index (NDVI were used to find the vegetation changes. The result shows that, maximum changes occur in the plantation and slight changes found in forest cover in the hilly terrain. In foothill areas, agriculture lands were decreased while wastelands and settlement plots were increased. The outcome of the results helps to farmer and policy makers to draw optimal lands use planning and better management strategies for sustainable development of natural resources.

10. Fast-NPS-A Markov Chain Monte Carlo-based analysis tool to obtain structural information from single-molecule FRET measurements

Science.gov (United States)

Eilert, Tobias; Beckers, Maximilian; Drechsler, Florian; Michaelis, Jens

2017-10-01

The analysis tool and software package Fast-NPS can be used to analyse smFRET data to obtain quantitative structural information about macromolecules in their natural environment. In the algorithm a Bayesian model gives rise to a multivariate probability distribution describing the uncertainty of the structure determination. Since Fast-NPS aims to be an easy-to-use general-purpose analysis tool for a large variety of smFRET networks, we established an MCMC based sampling engine that approximates the target distribution and requires no parameter specification by the user at all. For an efficient local exploration we automatically adapt the multivariate proposal kernel according to the shape of the target distribution. In order to handle multimodality, the sampler is equipped with a parallel tempering scheme that is fully adaptive with respect to temperature spacing and number of chains. Since the molecular surrounding of a dye molecule affects its spatial mobility and thus the smFRET efficiency, we introduce dye models which can be selected for every dye molecule individually. These models allow the user to represent the smFRET network in great detail leading to an increased localisation precision. Finally, a tool to validate the chosen model combination is provided. Programme Files doi:http://dx.doi.org/10.17632/7ztzj63r68.1 Licencing provisions: Apache-2.0 Programming language: GUI in MATLAB (The MathWorks) and the core sampling engine in C++ Nature of problem: Sampling of highly diverse multivariate probability distributions in order to solve for macromolecular structures from smFRET data. Solution method: MCMC algorithm with fully adaptive proposal kernel and parallel tempering scheme.

11. Land Use/land Cover Changes in Semi-Arid Mountain Landscape in Southern India: a Geoinformatics Based Markov Chain Approach

Science.gov (United States)

Rahaman, S. A.; Aruchamy, S.; Balasubramani, K.; Jegankumar, R.

2017-05-01

Nowadays land use/ land cover in mountain landscape is in critical condition; it leads to high risky and uncertain environments. These areas are facing multiple stresses including degradation of land resources; vagaries of climate and depletion of water resources continuously affect land use practices and livelihoods. To understand the Land use/Land cover (Lu/Lc) changes in a semi-arid mountain landscape, Kallar watershed of Bhavani basin, in southern India has been chosen. Most of the hilly part in the study area covers with forest, plantation, orchards and vegetables and which are highly affected by severe soil erosion, landslide, frequent rainfall failures and associated drought. The foothill regions are mainly utilized for agriculture practices; due to water scarcity and meagre income, the productive agriculture lands are converted into settlement plots and wasteland. Hence, land use/land cover change deduction; a stochastic processed based method is indispensable for future prediction. For identification of land use/land cover, and vegetation changes, Landsat TM, ETM (1995, 2005) and IRS P6- LISS IV (2015) images were used. Through CAMarkov chain analysis, Lu/Lc changes in past three decades (1995, 2005, and 2015) were identified and projected for (2020 and 2025); Normalized Difference Vegetation Index (NDVI) were used to find the vegetation changes. The result shows that, maximum changes occur in the plantation and slight changes found in forest cover in the hilly terrain. In foothill areas, agriculture lands were decreased while wastelands and settlement plots were increased. The outcome of the results helps to farmer and policy makers to draw optimal lands use planning and better management strategies for sustainable development of natural resources.

12. Flory-type theories of polymer chains under different external stimuli

Science.gov (United States)

Budkov, Yu A.; Kiselev, M. G.

2018-01-01

In this Review, we present a critical analysis of various applications of the Flory-type theories to a theoretical description of the conformational behavior of single polymer chains in dilute polymer solutions under a few external stimuli. Different theoretical models of flexible polymer chains in the supercritical fluid are discussed and analysed. Different points of view on the conformational behavior of the polymer chain near the liquid–gas transition critical point of the solvent are presented. A theoretical description of the co-solvent-induced coil-globule transitions within the implicit-solvent-explicit-co-solvent models is discussed. Several explicit-solvent-explicit-co-solvent theoretical models of the coil-to-globule-to-coil transition of the polymer chain in a mixture of good solvents (co-nonsolvency) are analysed and compared with each other. Finally, a new theoretical model of the conformational behavior of the dielectric polymer chain under the external constant electric field in the dilute polymer solution with an explicit account for the many-body dipole correlations is discussed. The polymer chain collapse induced by many-body dipole correlations of monomers in the context of statistical thermodynamics of dielectric polymers is analysed.

13. Consistent Estimation of Partition Markov Models

Directory of Open Access Journals (Sweden)

Jesús E. García

2017-04-01

Full Text Available The Partition Markov Model characterizes the process by a partition L of the state space, where the elements in each part of L share the same transition probability to an arbitrary element in the alphabet. This model aims to answer the following questions: what is the minimal number of parameters needed to specify a Markov chain and how to estimate these parameters. In order to answer these questions, we build a consistent strategy for model selection which consist of: giving a size n realization of the process, finding a model within the Partition Markov class, with a minimal number of parts to represent the process law. From the strategy, we derive a measure that establishes a metric in the state space. In addition, we show that if the law of the process is Markovian, then, eventually, when n goes to infinity, L will be retrieved. We show an application to model internet navigation patterns.

14. Coordinating a Supply Chain with a Loss-Averse Retailer under Yield and Demand Uncertainties

Directory of Open Access Journals (Sweden)

Weiwei Luo

2016-01-01

Full Text Available This paper investigates the channel coordination of a supply chain (SC consisting of a loss-averse retailer and a risk-neutral supplier under yield and demand uncertainties. Three existing contracts are analyzed. Our results demonstrate that the buyback (BB and quantity flexibility (QF contracts can not only coordinate the supply chain but also lead to Pareto improvement for each player, while the wholesale price (WP contract fails to coordinate the chain due to the effects of double marginalization and risk preference. For comparison, a chain with a risk-neutral retailer is also analyzed. Furthermore, numerical examples are provided to demonstrate the effectiveness of the coordination contracts, and the impacts of loss aversion and random yield on the decision-making behaviors and system performance are then discussed.

15. Orbits for 18 Visual Binaries and Two Double-line Spectroscopic Binaries Observed with HRCAM on the CTIO SOAR 4 m Telescope, Using a New Bayesian Orbit Code Based on Markov Chain Monte Carlo

Science.gov (United States)

Mendez, Rene A.; Claveria, Ruben M.; Orchard, Marcos E.; Silva, Jorge F.

2017-11-01

We present orbital elements and mass sums for 18 visual binary stars of spectral types B to K (five of which are new orbits) with periods ranging from 20 to more than 500 yr. For two double-line spectroscopic binaries with no previous orbits, the individual component masses, using combined astrometric and radial velocity data, have a formal uncertainty of ˜ 0.1 {M}⊙ . Adopting published photometry and trigonometric parallaxes, plus our own measurements, we place these objects on an H-R diagram and discuss their evolutionary status. These objects are part of a survey to characterize the binary population of stars in the Southern Hemisphere using the SOAR 4 m telescope+HRCAM at CTIO. Orbital elements are computed using a newly developed Markov chain Monte Carlo (MCMC) algorithm that delivers maximum-likelihood estimates of the parameters, as well as posterior probability density functions that allow us to evaluate the uncertainty of our derived parameters in a robust way. For spectroscopic binaries, using our approach, it is possible to derive a self-consistent parallax for the system from the combined astrometric and radial velocity data (“orbital parallax”), which compares well with the trigonometric parallaxes. We also present a mathematical formalism that allows a dimensionality reduction of the feature space from seven to three search parameters (or from 10 to seven dimensions—including parallax—in the case of spectroscopic binaries with astrometric data), which makes it possible to explore a smaller number of parameters in each case, improving the computational efficiency of our MCMC code. Based on observations obtained at the Southern Astrophysical Research (SOAR) telescope, which is a joint project of the Ministério da Ciência, Tecnologia, e Inovação (MCTI) da República Federativa do Brasil, the U.S. National Optical Astronomy Observatory (NOAO), the University of North Carolina at Chapel Hill (UNC), and Michigan State University (MSU).

16. Optimal Financing Order Decisions of a Supply Chain under the Retailer's Delayed Payment

Directory of Open Access Journals (Sweden)

Honglin Yang

2014-01-01

Full Text Available In real supply chain, a capital-constrained retailer has two typical payment choices: the up-front payment to receive a high discount price or the delayed payment to reduce capital pressure. We compare with the efficiency of optimal decisions of different participants, that is, supplier, retailer, and bank, under both types of payments based on a game equilibrium analysis. It shows that under the equilibrium, the delayed payment leads to a greater optimal order quantity from the retailer compared to the up-front payment and, thus, improves the whole benefit of the supply chain. The numerical simulation for the random demand following a uniform distribution further verifies our findings. This study provides novel evidence that a dominant supplier who actively offers trade credit helps enhance the whole efficiency of a supply chain.

17. Evaluation of Bayesian estimation of a hidden continuous-time Markov chain model with application to threshold violation in water-quality indicators

Science.gov (United States)

Deviney, Frank A.; Rice, Karen; Brown, Donald E.

2012-01-01

Natural resource managers require information concerning  the frequency, duration, and long-term probability of occurrence of water-quality indicator (WQI) violations of defined thresholds. The timing of these threshold crossings often is hidden from the observer, who is restricted to relatively infrequent observations. Here, a model for the hidden process is linked with a model for the observations, and the parameters describing duration, return period, and long-term probability of occurrence are estimated using Bayesian methods. A simulation experiment is performed to evaluate the approach under scenarios based on the equivalent of a total monitoring period of 5-30 years and an observation frequency of 1-50 observations per year. Given constant threshold crossing rate, accuracy and precision of parameter estimates increased with longer total monitoring period and more-frequent observations. Given fixed monitoring period and observation frequency, accuracy and precision of parameter estimates increased with longer times between threshold crossings. For most cases where the long-term probability of being in violation is greater than 0.10, it was determined that at least 600 observations are needed to achieve precise estimates.  An application of the approach is presented using 22 years of quasi-weekly observations of acid-neutralizing capacity from Deep Run, a stream in Shenandoah National Park, Virginia. The time series also was sub-sampled to simulate monthly and semi-monthly sampling protocols. Estimates of the long-term probability of violation were unbiased despite sampling frequency; however, the expected duration and return period were over-estimated using the sub-sampled time series with respect to the full quasi-weekly time series.

18. Resonance Energy Transfer-Based Molecular Switch Designed Using a Systematic Design Process Based on Monte Carlo Methods and Markov Chains

Science.gov (United States)

Rallapalli, Arjun

A RET network consists of a network of photo-active molecules called chromophores that can participate in inter-molecular energy transfer called resonance energy transfer (RET). RET networks are used in a variety of applications including cryptographic devices, storage systems, light harvesting complexes, biological sensors, and molecular rulers. In this dissertation, we focus on creating a RET device called closed-diffusive exciton valve (C-DEV) in which the input to output transfer function is controlled by an external energy source, similar to a semiconductor transistor like the MOSFET. Due to their biocompatibility, molecular devices like the C-DEVs can be used to introduce computing power in biological, organic, and aqueous environments such as living cells. Furthermore, the underlying physics in RET devices are stochastic in nature, making them suitable for stochastic computing in which true random distribution generation is critical. In order to determine a valid configuration of chromophores for the C-DEV, we developed a systematic process based on user-guided design space pruning techniques and built-in simulation tools. We show that our C-DEV is 15x better than C-DEVs designed using ad hoc methods that rely on limited data from prior experiments. We also show ways in which the C-DEV can be improved further and how different varieties of C-DEVs can be combined to form more complex logic circuits. Moreover, the systematic design process can be used to search for valid chromophore network configurations for a variety of RET applications. We also describe a feasibility study for a technique used to control the orientation of chromophores attached to DNA. Being able to control the orientation can expand the design space for RET networks because it provides another parameter to tune their collective behavior. While results showed limited control over orientation, the analysis required the development of a mathematical model that can be used to determine the

19. Sourcing and pricing strategies for two retailers in a decentralized supply chain system under supply disruption

Directory of Open Access Journals (Sweden)

M.A. Azarmehr

2012-01-01

Full Text Available This paper presents the decentralized supply chain with two suppliers and two competing retailers. It also investigates the sourcing and pricing strategies of two retailers in a decentralized supply chain system under a supply disruption environment. These retailers face their individual stochastic demand markets; however, they compete with each other through a two-stage price and service operation. The interactive dynamics among retailers is characterized, including the existence and uniqueness of the Nash Equilibrium in service and price games demonstrated.

20. Effect Analysis of Service Supply Chain with Dynamic Game under the Condition of Sensitive Demand

Directory of Open Access Journals (Sweden)

Guanglan Zhou

2015-01-01

Full Text Available Under the real circumstances of service supply chain, there is one demand appearing as the sensitive feature, to face the increasing uncertainty. It could be elaborated upon the decision variables such as price, quantity, and efforts. The member behaviors are operated and coordinated in the process of multiperiod dynamic game. Based on the multiperiod dynamic game theory, the service demand and price, quantity of goods, and efforts of members in the secondary service supply chain are considered. The paper discusses the reputation effect and ratchet effect in the multiperiod dynamic game service supply chain. Additionally, the paper describes this problem, builds a programming model based on the multiperiod dynamic game, and deduces the optimal solution. Furthermore, the paper analyzes the impact of reputation effect and ratchet effect on the agent’s revenue. Through the simulation, it is found that the agency efforts are a combination result of reputation effect and ratchet effect in the process of multiperiod dynamic game. Through the long-term dynamic game, the short-term moral risk in service supply chain can be restrained so that the result under the asymmetric information is the same as that under the complete information.