k– fading channels: a finite state Markov modelling approach
Indian Academy of Sciences (India)
C Priyanka
2018-02-07
Feb 7, 2018 ... 1 Department of Electronics and Communication Engineering, SRM University, ... probability; level crossing rate; state-transition probability; state-time duration. 1. ... statistics of the mobile radio signals are described by dif-.
Yan, Huaicheng; Zhang, Hao; Yang, Fuwen; Zhan, Xisheng; Peng, Chen
2017-08-18
This paper is concerned with the guaranteed cost control problem for a class of Markov jump discrete-time neural networks (NNs) with event-triggered mechanism, asynchronous jumping, and fading channels. The Markov jump NNs are introduced to be close to reality, where the modes of the NNs and guaranteed cost controller are determined by two mutually independent Markov chains. The asynchronous phenomenon is considered, which increases the difficulty of designing required mode-dependent controller. The event-triggered mechanism is designed by comparing the relative measurement error with the last triggered state at the process of data transmission, which is used to eliminate dispensable transmission and reduce the networked energy consumption. In addition, the signal fading is considered for the effect of signal reflection and shadow in wireless networks, which is modeled by the novel Rice fading models. Some novel sufficient conditions are obtained to guarantee that the closed-loop system reaches a specified cost value under the designed jumping state feedback control law in terms of linear matrix inequalities. Finally, some simulation results are provided to illustrate the effectiveness of the proposed method.
Model Checking Infinite-State Markov Chains
Remke, Anne Katharina Ingrid; Haverkort, Boudewijn R.H.M.; Cloth, L.
2004-01-01
In this paper algorithms for model checking CSL (continuous stochastic logic) against infinite-state continuous-time Markov chains of so-called quasi birth-death type are developed. In doing so we extend the applicability of CSL model checking beyond the recently proposed case for finite-state
Markov state models of protein misfolding
Sirur, Anshul; De Sancho, David; Best, Robert B.
2016-02-01
Markov state models (MSMs) are an extremely useful tool for understanding the conformational dynamics of macromolecules and for analyzing MD simulations in a quantitative fashion. They have been extensively used for peptide and protein folding, for small molecule binding, and for the study of native ensemble dynamics. Here, we adapt the MSM methodology to gain insight into the dynamics of misfolded states. To overcome possible flaws in root-mean-square deviation (RMSD)-based metrics, we introduce a novel discretization approach, based on coarse-grained contact maps. In addition, we extend the MSM methodology to include "sink" states in order to account for the irreversibility (on simulation time scales) of processes like protein misfolding. We apply this method to analyze the mechanism of misfolding of tandem repeats of titin domains, and how it is influenced by confinement in a chaperonin-like cavity.
NonMarkov Ito Processes with 1- state memory
McCauley, Joseph L.
2010-08-01
A Markov process, by definition, cannot depend on any previous state other than the last observed state. An Ito process implies the Fokker-Planck and Kolmogorov backward time partial differential eqns. for transition densities, which in turn imply the Chapman-Kolmogorov eqn., but without requiring the Markov condition. We present a class of Ito process superficially resembling Markov processes, but with 1-state memory. In finance, such processes would obey the efficient market hypothesis up through the level of pair correlations. These stochastic processes have been mislabeled in recent literature as 'nonlinear Markov processes'. Inspired by Doob and Feller, who pointed out that the ChapmanKolmogorov eqn. is not restricted to Markov processes, we exhibit a Gaussian Ito transition density with 1-state memory in the drift coefficient that satisfies both of Kolmogorov's partial differential eqns. and also the Chapman-Kolmogorov eqn. In addition, we show that three of the examples from McKean's seminal 1966 paper are also nonMarkov Ito processes. Last, we show that the transition density of the generalized Black-Scholes type partial differential eqn. describes a martingale, and satisfies the ChapmanKolmogorov eqn. This leads to the shortest-known proof that the Green function of the Black-Scholes eqn. with variable diffusion coefficient provides the so-called martingale measure of option pricing.
Embedding a State Space Model Into a Markov Decision Process
DEFF Research Database (Denmark)
Nielsen, Lars Relund; Jørgensen, Erik; Højsgaard, Søren
2011-01-01
In agriculture Markov decision processes (MDPs) with finite state and action space are often used to model sequential decision making over time. For instance, states in the process represent possible levels of traits of the animal and transition probabilities are based on biological models...
International Nuclear Information System (INIS)
Usenko, Vladyslav C; Filip, Radim; Heim, Bettina; Peuntinger, Christian; Wittmann, Christoffer; Marquardt, Christoph; Leuchs, Gerd
2012-01-01
Entanglement properties of Gaussian states of light as well as the security of continuous variable quantum key distribution with Gaussian states in free-space fading channels are studied. These qualities are shown to be sensitive to the statistical properties of the transmittance distribution in the cases when entanglement is strong or when channel excess noise is present. Fading, i.e. transmission fluctuations, caused by beam wandering due to atmospheric turbulence, is a frequent challenge in free-space communication. We introduce a method of fading discrimination and subsequent post-selection of the corresponding sub-states and show that it can improve the entanglement resource and restore the security of the key distribution over a realistic fading link. Furthermore, the optimal post-selection strategy in combination with an optimized entangled resource is shown to drastically increase the protocol's robustness to excess noise, which is confirmed for experimentally measured fading channel characteristics. The stability of the result against finite data ensemble size and imperfect channel estimation is also addressed. (paper)
Entanglement revival can occur only when the system-environment state is not a Markov state
Sargolzahi, Iman
2018-06-01
Markov states have been defined for tripartite quantum systems. In this paper, we generalize the definition of the Markov states to arbitrary multipartite case and find the general structure of an important subset of them, which we will call strong Markov states. In addition, we focus on an important property of the Markov states: If the initial state of the whole system-environment is a Markov state, then each localized dynamics of the whole system-environment reduces to a localized subdynamics of the system. This provides us a necessary condition for entanglement revival in an open quantum system: Entanglement revival can occur only when the system-environment state is not a Markov state. To illustrate (a part of) our results, we consider the case that the environment is modeled as classical. In this case, though the correlation between the system and the environment remains classical during the evolution, the change of the state of the system-environment, from its initial Markov state to a state which is not a Markov one, leads to the entanglement revival in the system. This shows that the non-Markovianity of a state is not equivalent to the existence of non-classical correlation in it, in general.
On the capacity of Rician fading channels with full channel state information at low SNR
Rezki, Zouheir
2012-06-01
The capacity of flat Rayleigh fading channels with full channel state information (CSI) at the transmitter and at the receiver at asymptotically low SNR has been recently shown to scale essentially as SNR log (1/SNR). In this paper, we investigate the Rician fading channel capacity with full CSI, and show that the capacity of this channel scales essentially as 1/1+K SNR log (1 /SNR), where K is the Rician factor. This characterization includes perfect CSI at both the transmitter and the receiver or noisy CSI at the transmitter and perfect CSI at the receiver. We also show that one-bit CSI at the transmitter is enough to achieve this asymptotic capacity using an On-Off power control scheme. Our framework may be seen as a generalization of previous works as it captures the Rayleigh fading channel as a special case by letting K goes to zero. © 2012 IEEE.
On the low SNR capacity of MIMO fading channels with imperfect channel state information
Benkhelifa, Fatma
2014-06-01
The capacity of multiple-input multiple-output (MIMO) Rayleigh fading channels with full knowledge of channel state information (CSI) at both the transmitter and the receiver (CSI-TR) has been shown recently to scale at low signal-to-noise ratio (SNR) essentially as SNR log(1/SNR), independently of the number of transmit and receive antennas. In this paper, we investigate the ergodic capacity of MIMO Rayleigh fading channel with estimated channel state information at the transmitter (CSI-T) and possibly imperfect channel state information at the receiver (CSI-R). Our framework can be seen as a generalization of previous works as it can capture the perfect CSI-TR as a special case when the estimation error variance goes to zero. In this paper, we mainly focus on the low SNR regime, and we show that the capacity scales as (1-α) SNR log(1/SNR), where α is the estimation error variance. This characterization shows the loss of performance due to error estimation over the perfect channel state information at both the transmitter and the receiver. As a by-product of our new analysis, we show that our framework can be also extended to characterize the capacity of MIMO Rician fading channels at low SNR with possibly imperfect CSI-T and CSI-R. © 1972-2012 IEEE.
On the low SNR capacity of MIMO fading channels with imperfect channel state information
Benkhelifa, Fatma
2014-05-01
The capacity of Multiple Input Multiple Output (MIMO) Rayleigh fading channels with full knowledge of channel state information (CSI) at both the transmitter and the receiver (CSI-TR) has been shown recently to scale at low Signal-to-Noise Ratio (SNR) essentially as SNR log(1=SNR), independently of the number of transmit and receive antennas. In this paper, we investigate the ergodic capacity of MIMO Rayleigh fading channel with estimated channel state information at the transmitter (CSI-T) and possibly imperfect channel state information at the receiver (CSI-R). Our framework can be seen as a generalization of previous works as it can capture the perfect CSI-TR as a special case when the estimation error variance goes to zero. In our work, we mainly focus on the low SNR regime and we show that the capacity scales as (1-α) SNR log(1=SNR), where α is the estimation error variance. This characterization shows the loss of performance due to error estimation over the perfect channel state information at both the transmitter and the receiver. As a by-product of our new analysis, we show that our framework can also be extended to characterize the capacity of MIMO Rician fading channels at low SNR with possibly imperfect CSI-T and CSI-R. © 2014 IFIP.
On the capacity of cognitive radio under limited channel state information over fading channels
Rezki, Zouheir
2011-06-01
A spectrum-sharing communication system where the secondary user is aware of the instantaneous channel state information (CSI) of the secondary link, but knows only the statistics and an estimated version of the secondary transmitter-primary receiver (ST-PR) link, is investigated. The optimum power profile and the ergodic capacity of the secondary link are derived for general fading channels (with continuous probability density function) under average and peak transmit-power constraints and with respect to two different interference constraints: an interference outage constraint and a signal-to-interference (SI) outage constraint. When applied to Rayleigh fading channels, our results show, for instance, that the interference constraint is harmful at high-power regime, whereas at low-power regime, it has a marginal impact and no-interference performance may be achieved. © 2011 IEEE.
Rezki, Zouheir
2013-06-01
We study the throughput capacity region of the Gaussian multiaccess (MAC) fading channel with perfect channel state information (CSI) at the receiver (CSI-R) and at the transmitters (CSI-T), at low power regime. We show that it has a multidimensional rectangle structure and thus is simply characterized by single user capacity points. More specifically, we show that at low power regime, the boundary surface of the capacity region shrinks to a single point corresponding to the sum rate maximizer and that the coordinates of this point coincide with single user capacity bounds. Inspired from this result, we propose an on-off scheme, compute its achievable rate, and provide a necessary condition on the fading channels under which this scheme achieves single user capacity bounds of the MAC channel at asymptotically low power regime. We argue that this necessary condition characterizes a class of fading that encompasses all known wireless channels, where the capacity region of the MAC channel has a simple expression in terms of users\\' average power constraints only. © 2013 IEEE.
Extending Markov Automata with State and Action Rewards
Guck, Dennis; Timmer, Mark; Blom, Stefan; Bertrand, N.; Bortolussi, L.
This presentation introduces the Markov Reward Automaton (MRA), an extension of the Markov automaton that allows the modelling of systems incorporating rewards in addition to nondeterminism, discrete probabilistic choice and continuous stochastic timing. Our models support both rewards that are
On the capacity of nakagami-m fading Channels with full channel state information at low SNR
Rezki, Zouheir
2012-06-01
The capacity of flat Rayleigh fading channels with full channel state information (CSI) at the transmitter and at the receiver at asymptotically low SNR has been recently shown to scale essentially as SNR log(1/SNR)}. In this paper, we investigate the Nakagami-m fading channel capacity with full CSI, and show that the capacity of this channel scales essentially as m/ Omega SNR log(1/SNR), where m is the Nakagami-m fading parameter and where Ω is the channel mean-square. We also show that one-bit CSI at the transmitter is enough to achieve this asymptotic capacity using an On-Off power control scheme. Our framework may be seen as a generalization of previous works as it captures the Rayleigh fading channel as a special case by taking m=1. © 2012 IEEE.
Algorithms for a parallel implementation of Hidden Markov Models with a small state space
DEFF Research Database (Denmark)
Nielsen, Jesper; Sand, Andreas
2011-01-01
Two of the most important algorithms for Hidden Markov Models are the forward and the Viterbi algorithms. We show how formulating these using linear algebra naturally lends itself to parallelization. Although the obtained algorithms are slow for Hidden Markov Models with large state spaces...
Using multi-state markov models to identify credit card risk
Directory of Open Access Journals (Sweden)
Daniel Evangelista Régis
2016-06-01
Full Text Available Abstract The main interest of this work is to analyze the application of multi-state Markov models to evaluate credit card risk by investigating the characteristics of different state transitions in client-institution relationships over time, thereby generating score models for various purposes. We also used logistic regression models to compare the results with those obtained using multi-state Markov models. The models were applied to an actual database of a Brazilian financial institution. In this application, multi-state Markov models performed better than logistic regression models in predicting default risk, and logistic regression models performed better in predicting cancellation risk.
Rezki, Zouheir
2014-01-01
We study the throughput capacity region of the Gaussian multi-access (MAC) fading channel with perfect channel state information (CSI) at the receiver and at the transmitters, at low power regime. We show that it has a multidimensional rectangle structure and thus is simply characterized by single user capacity points.More specifically, we show that at low power regime, the boundary surface of the capacity region shrinks to a single point corresponding to the sum rate maximizer and that the coordinates of this point coincide with single user capacity bounds. Inspired from this result, we propose an on-off scheme, compute its achievable rate, and show that this scheme achieves single user capacity bounds of the MAC channel for a wide class of fading channels at asymptotically low power regime. We argue that this class of fading encompasses all known wireless channels for which the capacity region of the MAC channel has even a simpler expression in terms of users\\' average power constraints only. Using the duality of Gaussian MAC and broadcast channels (BC), we deduce a simple characterization of the BC capacity region at low power regime and show that for a class of fading channels (including Rayleigh fading), time-sharing is asymptotically optimal. © 2014 IEEE.
Girsanov reweighting for path ensembles and Markov state models
Donati, L.; Hartmann, C.; Keller, B. G.
2017-06-01
The sensitivity of molecular dynamics on changes in the potential energy function plays an important role in understanding the dynamics and function of complex molecules. We present a method to obtain path ensemble averages of a perturbed dynamics from a set of paths generated by a reference dynamics. It is based on the concept of path probability measure and the Girsanov theorem, a result from stochastic analysis to estimate a change of measure of a path ensemble. Since Markov state models (MSMs) of the molecular dynamics can be formulated as a combined phase-space and path ensemble average, the method can be extended to reweight MSMs by combining it with a reweighting of the Boltzmann distribution. We demonstrate how to efficiently implement the Girsanov reweighting in a molecular dynamics simulation program by calculating parts of the reweighting factor "on the fly" during the simulation, and we benchmark the method on test systems ranging from a two-dimensional diffusion process and an artificial many-body system to alanine dipeptide and valine dipeptide in implicit and explicit water. The method can be used to study the sensitivity of molecular dynamics on external perturbations as well as to reweight trajectories generated by enhanced sampling schemes to the original dynamics.
Benkhelifa, Fatma
2013-04-01
In this letter, we study the ergodic capacity of a maximum ratio combining (MRC) Rician fading channel with full channel state information (CSI) at the transmitter and at the receiver. We focus on the low Signal-to-Noise Ratio (SNR) regime and we show that the capacity scales as L ΩK+L SNRx log(1SNR), where Ω is the expected channel gain per branch, K is the Rician fading factor, and L is the number of diversity branches. We show that one-bit CSI feedback at the transmitter is enough to achieve this capacity using an on-off power control scheme. Our framework can be seen as a generalization of recently established results regarding the fading-channels capacity characterization in the low-SNR regime. © 2012 IEEE.
Benkhelifa, Fatma; Rezki, Zouheir; Alouini, Mohamed-Slim
2013-01-01
In this letter, we study the ergodic capacity of a maximum ratio combining (MRC) Rician fading channel with full channel state information (CSI) at the transmitter and at the receiver. We focus on the low Signal-to-Noise Ratio (SNR) regime and we show that the capacity scales as L ΩK+L SNRx log(1SNR), where Ω is the expected channel gain per branch, K is the Rician fading factor, and L is the number of diversity branches. We show that one-bit CSI feedback at the transmitter is enough to achieve this capacity using an on-off power control scheme. Our framework can be seen as a generalization of recently established results regarding the fading-channels capacity characterization in the low-SNR regime. © 2012 IEEE.
Sheng, Li; Wang, Zidong; Tian, Engang; Alsaadi, Fuad E
2016-12-01
This paper deals with the H ∞ state estimation problem for a class of discrete-time neural networks with stochastic delays subject to state- and disturbance-dependent noises (also called (x,v)-dependent noises) and fading channels. The time-varying stochastic delay takes values on certain intervals with known probability distributions. The system measurement is transmitted through fading channels described by the Rice fading model. The aim of the addressed problem is to design a state estimator such that the estimation performance is guaranteed in the mean-square sense against admissible stochastic time-delays, stochastic noises as well as stochastic fading signals. By employing the stochastic analysis approach combined with the Kronecker product, several delay-distribution-dependent conditions are derived to ensure that the error dynamics of the neuron states is stochastically stable with prescribed H ∞ performance. Finally, a numerical example is provided to illustrate the effectiveness of the obtained results. Copyright © 2016 Elsevier Ltd. All rights reserved.
Kirkwood, James R
2015-01-01
Review of ProbabilityShort HistoryReview of Basic Probability DefinitionsSome Common Probability DistributionsProperties of a Probability DistributionProperties of the Expected ValueExpected Value of a Random Variable with Common DistributionsGenerating FunctionsMoment Generating FunctionsExercisesDiscrete-Time, Finite-State Markov ChainsIntroductionNotationTransition MatricesDirected Graphs: Examples of Markov ChainsRandom Walk with Reflecting BoundariesGamblerâ€™s RuinEhrenfest ModelCentral Problem of Markov ChainsCondition to Ensure a Unique Equilibrium StateFinding the Equilibrium StateTransient and Recurrent StatesIndicator FunctionsPerron-Frobenius TheoremAbsorbing Markov ChainsMean First Passage TimeMean Recurrence Time and the Equilibrium StateFundamental Matrix for Regular Markov ChainsDividing a Markov Chain into Equivalence ClassesPeriodic Markov ChainsReducible Markov ChainsSummaryExercisesDiscrete-Time, Infinite-State Markov ChainsRenewal ProcessesDelayed Renewal ProcessesEquilibrium State f...
Wang, Wei; Cao, Siqin; Zhu, Lizhe; Huang, Xuhui
2017-01-01
bioengineering applications and rational drug design. Constructing Markov State Models (MSMs) based on large-scale molecular dynamics simulations has emerged as a powerful approach to model functional conformational changes of the biomolecular system
Optimal Number of States in Hidden Markov Models and its ...
African Journals Online (AJOL)
In this paper, Hidden Markov Model is applied to model human movements as to facilitate an automatic detection of the same. A number of activities were simulated with the help of two persons. The four movements considered are walking, sitting down-getting up, fall while walking and fall while standing. The data is ...
Markov Chains and Markov Processes
Ogunbayo, Segun
2016-01-01
Markov chain, which was named after Andrew Markov is a mathematical system that transfers a state to another state. Many real world systems contain uncertainty. This study helps us to understand the basic idea of a Markov chain and how is been useful in our daily lives. For some times there had been suspense on distinct predictions and future existences. Also in different games there had been different expectations or results involved. That is the reason why we need Markov chains to predict o...
Robust filtering and prediction for systems with embedded finite-state Markov-Chain dynamics
International Nuclear Information System (INIS)
Pate, E.B.
1986-01-01
This research developed new methodologies for the design of robust near-optimal filters/predictors for a class of system models that exhibit embedded finite-state Markov-chain dynamics. These methodologies are developed through the concepts and methods of stochastic model building (including time-series analysis), game theory, decision theory, and filtering/prediction for linear dynamic systems. The methodology is based on the relationship between the robustness of a class of time-series models and quantization which is applied to the time series as part of the model identification process. This relationship is exploited by utilizing the concept of an equivalence, through invariance of spectra, between the class of Markov-chain models and the class of autoregressive moving average (ARMA) models. This spectral equivalence permits a straightforward implementation of the desirable robust properties of the Markov-chain approximation in a class of models which may be applied in linear-recursive form in a linear Kalman filter/predictor structure. The linear filter/predictor structure is shown to provide asymptotically optimal estimates of states which represent one or more integrations of the Markov-chain state. The development of a new saddle-point theorem for a game based on the Markov-chain model structure gives rise to a technique for determining a worst case Markov-chain process, upon which a robust filter/predictor design if based
Sio, Betsy Menson
2009-01-01
A sky fading from blue to white to red at the horizon, and water darkening from light to midnight blue. Strong diagonals slashing through the image, drawing a viewer's eyes deeper into the picture, and delicate trees poised to convey a sense of beauty. These are the fascinating strengths of the ukiyo-e woodblock prints of Japanese artist Ando…
RESEARCH ABSORBING STATES OF THE SYSTEM USING MARKOV CHAINS AND FUNDAMENTAL MATRIX
Directory of Open Access Journals (Sweden)
Тетяна Мефодіївна ОЛЕХ
2016-02-01
Full Text Available The article discusses the use Markov chains to research models that reflect the essential properties of systems, including methods of measuring the parameters of projects and assess their effectiveness. In the study carried out by its decomposition system for certain discrete state and create a diagram of transitions between these states. Specificity displays various objects Markov homogeneous chains with discrete states and discrete time determined by the method of calculation of transition probabilities. A model of success criteria for absorbing state system that is universal for all projects. A breakdown of passages to the matrix submatrices. The variation elements under matrix Q n with growth linked to the definition of important quantitative characteristics of absorbing circuits: 1 the probability of achieving the status of absorbing any given; 2 the mean number of steps needed to achieve the absorbing state; 3 the mean time that the system spends in each state to hit irreversible system in absorbing state. Built fundamental matrix that allowed calculating the different characteristics of the system. Considered fundamental matrix for supposedly modeled absorbing Markov chain, which gives the forecast for the behavior of the system in the future regardless of the absolute value of the time elapsed from the starting point. This property illustrates the fundamental matrix Markov process that characterizes it as a process without aftereffect.
Chen, Pei; Liu, Rui; Li, Yongjun; Chen, Luonan
2016-07-15
Identifying the critical state or pre-transition state just before the occurrence of a phase transition is a challenging task, because the state of the system may show little apparent change before this critical transition during the gradual parameter variations. Such dynamics of phase transition is generally composed of three stages, i.e. before-transition state, pre-transition state and after-transition state, which can be considered as three different Markov processes. By exploring the rich dynamical information provided by high-throughput data, we present a novel computational method, i.e. hidden Markov model (HMM) based approach, to detect the switching point of the two Markov processes from the before-transition state (a stationary Markov process) to the pre-transition state (a time-varying Markov process), thereby identifying the pre-transition state or early-warning signals of the phase transition. To validate the effectiveness, we apply this method to detect the signals of the imminent phase transitions of complex systems based on the simulated datasets, and further identify the pre-transition states as well as their critical modules for three real datasets, i.e. the acute lung injury triggered by phosgene inhalation, MCF-7 human breast cancer caused by heregulin and HCV-induced dysplasia and hepatocellular carcinoma. Both functional and pathway enrichment analyses validate the computational results. The source code and some supporting files are available at https://github.com/rabbitpei/HMM_based-method lnchen@sibs.ac.cn or liyj@scut.edu.cn Supplementary data are available at Bioinformatics online. © The Author 2016. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.
438 Optimal Number of States in Hidden Markov Models and its ...
African Journals Online (AJOL)
In this paper, Hidden Markov Model is applied to model human movements as to .... emit either discrete information or a continuous data derived from a Probability .... For each hidden state in the test set, the probability = ... by applying the Kullback-Leibler distance (Juang & Rabiner, 1985) which ..... One Size Does Not Fit.
Estimation and asymptotic theory for transition probabilities in Markov Renewal Multi–state models
Spitoni, C.; Verduijn, M.; Putter, H.
2012-01-01
In this paper we discuss estimation of transition probabilities for semi–Markov multi–state models. Non–parametric and semi–parametric estimators of the transition probabilities for a large class of models (forward going models) are proposed. Large sample theory is derived using the functional
CSL Model Checking Algorithms for Infinite-state Structured Markov chains
Remke, Anne Katharina Ingrid; Haverkort, Boudewijn R.H.M.; Raskin, J.-F.; Thiagarajan, P.S.
2007-01-01
Jackson queueing networks (JQNs) are a very general class of queueing networks that find their application in a variety of settings. The state space of the continuous-time Markov chain (CTMC) that underlies such a JQN, is highly structured, however, of infinite size in as many dimensions as there
The Markov process admits a consistent steady-state thermodynamic formalism
Peng, Liangrong; Zhu, Yi; Hong, Liu
2018-01-01
The search for a unified formulation for describing various non-equilibrium processes is a central task of modern non-equilibrium thermodynamics. In this paper, a novel steady-state thermodynamic formalism was established for general Markov processes described by the Chapman-Kolmogorov equation. Furthermore, corresponding formalisms of steady-state thermodynamics for the master equation and Fokker-Planck equation could be rigorously derived in mathematics. To be concrete, we proved that (1) in the limit of continuous time, the steady-state thermodynamic formalism for the Chapman-Kolmogorov equation fully agrees with that for the master equation; (2) a similar one-to-one correspondence could be established rigorously between the master equation and Fokker-Planck equation in the limit of large system size; (3) when a Markov process is restrained to one-step jump, the steady-state thermodynamic formalism for the Fokker-Planck equation with discrete state variables also goes to that for master equations, as the discretization step gets smaller and smaller. Our analysis indicated that general Markov processes admit a unified and self-consistent non-equilibrium steady-state thermodynamic formalism, regardless of underlying detailed models.
A descriptive model of resting-state networks using Markov chains.
Xie, H; Pal, R; Mitra, S
2016-08-01
Resting-state functional connectivity (RSFC) studies considering pairwise linear correlations have attracted great interests while the underlying functional network structure still remains poorly understood. To further our understanding of RSFC, this paper presents an analysis of the resting-state networks (RSNs) based on the steady-state distributions and provides a novel angle to investigate the RSFC of multiple functional nodes. This paper evaluates the consistency of two networks based on the Hellinger distance between the steady-state distributions of the inferred Markov chain models. The results show that generated steady-state distributions of default mode network have higher consistency across subjects than random nodes from various RSNs.
Directory of Open Access Journals (Sweden)
Mokaedi V. Lekgari
2014-01-01
Full Text Available We investigate random-time state-dependent Foster-Lyapunov analysis on subgeometric rate ergodicity of continuous-time Markov chains (CTMCs. We are mainly concerned with making use of the available results on deterministic state-dependent drift conditions for CTMCs and on random-time state-dependent drift conditions for discrete-time Markov chains and transferring them to CTMCs.
State space orderings for Gauss-Seidel in Markov chains revisited
Energy Technology Data Exchange (ETDEWEB)
Dayar, T. [Bilkent Univ., Ankara (Turkey)
1996-12-31
Symmetric state space orderings of a Markov chain may be used to reduce the magnitude of the subdominant eigenvalue of the (Gauss-Seidel) iteration matrix. Orderings that maximize the elemental mass or the number of nonzero elements in the dominant term of the Gauss-Seidel splitting (that is, the term approximating the coefficient matrix) do not necessarily converge faster. An ordering of a Markov chain that satisfies Property-R is semi-convergent. On the other hand, there are semi-convergent symmetric state space orderings that do not satisfy Property-R. For a given ordering, a simple approach for checking Property-R is shown. An algorithm that orders the states of a Markov chain so as to increase the likelihood of satisfying Property-R is presented. The computational complexity of the ordering algorithm is less than that of a single Gauss-Seidel iteration (for sparse matrices). In doing all this, the aim is to gain an insight for faster converging orderings. Results from a variety of applications improve the confidence in the algorithm.
Rezki, Zouheir
2013-07-01
We study the throughput capacity region of the Gaussian multi-access (MAC) fading channel with perfect channel state information (CSI) at the receiver and at the transmitters (CSI-TR), at low power regime. We show that it has a multidimensional rectangle structure and thus is simply characterized by single user capacity points. More specifically, we show that at low power regime, the boundary surface of the capacity region shrinks to a single point corresponding to the sum-rate maximizer and that the coordinates of this point coincide with single user capacity bounds. Using the duality of Gaussian MAC and broadcast channels (BC), we provide a simple characterization of the BC capacity region at low power regime. © 2013 IEEE.
Rate Reduction for State-labelled Markov Chains with Upper Time-bounded CSL Requirements
Directory of Open Access Journals (Sweden)
Bharath Siva Kumar Tati
2016-07-01
Full Text Available This paper presents algorithms for identifying and reducing a dedicated set of controllable transition rates of a state-labelled continuous-time Markov chain model. The purpose of the reduction is to make states to satisfy a given requirement, specified as a CSL upper time-bounded Until formula. We distinguish two different cases, depending on the type of probability bound. A natural partitioning of the state space allows us to develop possible solutions, leading to simple algorithms for both cases.
Multi-state Markov model for disability: A case of Malaysia Social Security (SOCSO)
Samsuddin, Shamshimah; Ismail, Noriszura
2016-06-01
Studies of SOCSO's contributor outcomes like disability are usually restricted to a single outcome. In this respect, the study has focused on the approach of multi-state Markov model for estimating the transition probabilities among SOCSO's contributor in Malaysia between states: work, temporary disability, permanent disability and death at yearly intervals on age, gender, year and disability category; ignoring duration and past disability experience which is not consider of how or when someone arrived in that category. These outcomes represent different states which depend on health status among the workers.
Prediction and generation of binary Markov processes: Can a finite-state fox catch a Markov mouse?
Ruebeck, Joshua B.; James, Ryan G.; Mahoney, John R.; Crutchfield, James P.
2018-01-01
Understanding the generative mechanism of a natural system is a vital component of the scientific method. Here, we investigate one of the fundamental steps toward this goal by presenting the minimal generator of an arbitrary binary Markov process. This is a class of processes whose predictive model is well known. Surprisingly, the generative model requires three distinct topologies for different regions of parameter space. We show that a previously proposed generator for a particular set of binary Markov processes is, in fact, not minimal. Our results shed the first quantitative light on the relative (minimal) costs of prediction and generation. We find, for instance, that the difference between prediction and generation is maximized when the process is approximately independently, identically distributed.
Abdulla, Parosh Aziz; Henda, Noomene Ben; Mayr, Richard
2007-01-01
We consider qualitative and quantitative verification problems for infinite-state Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which F can no longer be reached. While all finite Markov chains are trivially decisive (for every set F), this also holds for many classes of infinite Markov chains. Infinite Markov chains which contain a finite attractor are decisive w.r.t. every set F. In part...
The combinational structure of non-homogeneous Markov chains with countable states
Directory of Open Access Journals (Sweden)
A. Mukherjea
1983-01-01
Full Text Available Let P(s,t denote a non-homogeneous continuous parameter Markov chain with countable state space E and parameter space [a,b], −∞0}. It is shown in this paper that R(s,t is reflexive, transitive, and independent of (s,t, s
A Multistep Extending Truncation Method towards Model Construction of Infinite-State Markov Chains
Directory of Open Access Journals (Sweden)
Kemin Wang
2014-01-01
Full Text Available The model checking of Infinite-State Continuous Time Markov Chains will inevitably encounter the state explosion problem when constructing the CTMCs model; our method is to get a truncated model of the infinite one; to get a sufficient truncated model to meet the model checking of Continuous Stochastic Logic based system properties, we propose a multistep extending advanced truncation method towards model construction of CTMCs and implement it in the INFAMY model checker; the experiment results show that our method is effective.
Mixture estimation with state-space components and Markov model of switching
Czech Academy of Sciences Publication Activity Database
Nagy, Ivan; Suzdaleva, Evgenia
2013-01-01
Roč. 37, č. 24 (2013), s. 9970-9984 ISSN 0307-904X R&D Projects: GA TA ČR TA01030123 Institutional support: RVO:67985556 Keywords : probabilistic dynamic mixtures, * probability density function * state-space models * recursive mixture estimation * Bayesian dynamic decision making under uncertainty * Kerridge inaccuracy Subject RIV: BC - Control Systems Theory Impact factor: 2.158, year: 2013 http://library.utia.cas.cz/separaty/2013/AS/nagy-mixture estimation with state-space components and markov model of switching.pdf
First Passage Moments of Finite-State Semi-Markov Processes
Energy Technology Data Exchange (ETDEWEB)
Warr, Richard [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Cordeiro, James [Air Force Research Lab. (AFRL), Wright-Patterson AFB, OH (United States)
2014-03-31
In this paper, we discuss the computation of first-passage moments of a regular time-homogeneous semi-Markov process (SMP) with a finite state space to certain of its states that possess the property of universal accessibility (UA). A UA state is one which is accessible from any other state of the SMP, but which may or may not connect back to one or more other states. An important characteristic of UA is that it is the state-level version of the oft-invoked process-level property of irreducibility. We adapt existing results for irreducible SMPs to the derivation of an analytical matrix expression for the first passage moments to a single UA state of the SMP. In addition, consistent point estimators for these first passage moments, together with relevant R code, are provided.
Sinitskiy, Anton V.; Pande, Vijay S.
2018-01-01
Markov state models (MSMs) have been widely used to analyze computer simulations of various biomolecular systems. They can capture conformational transitions much slower than an average or maximal length of a single molecular dynamics (MD) trajectory from the set of trajectories used to build the MSM. A rule of thumb claiming that the slowest implicit time scale captured by an MSM should be comparable by the order of magnitude to the aggregate duration of all MD trajectories used to build this MSM has been known in the field. However, this rule has never been formally proved. In this work, we present analytical results for the slowest time scale in several types of MSMs, supporting the above rule. We conclude that the slowest implicit time scale equals the product of the aggregate sampling and four factors that quantify: (1) how much statistics on the conformational transitions corresponding to the longest implicit time scale is available, (2) how good the sampling of the destination Markov state is, (3) the gain in statistics from using a sliding window for counting transitions between Markov states, and (4) a bias in the estimate of the implicit time scale arising from finite sampling of the conformational transitions. We demonstrate that in many practically important cases all these four factors are on the order of unity, and we analyze possible scenarios that could lead to their significant deviation from unity. Overall, we provide for the first time analytical results on the slowest time scales captured by MSMs. These results can guide further practical applications of MSMs to biomolecular dynamics and allow for higher computational efficiency of simulations.
Utility evaluations for Markov states of lung cancer for PET-based disease management
International Nuclear Information System (INIS)
Papatheofanis, F.J.
2000-01-01
Utilities for the health outcomes states (Markov states) of non-small cell lung carcinoma (NSCLCL) should be measured to evaluate management options for patients because patients are key participants in the process of care, and their assessment of diagnostic and therapeutic value in the options presented to them ultimately impacts their net health outcomes. This investigation sought to measure utilities for stage-dependent outcomes states of NSCLC. Persons (n=23) with suspected NSCLC based on physical findings and computed tomography completed a short utilities survey. Utility valuations were obtained according to severity of morbidity and varied considerably. Respondents rated these health states according to accuracy measures for 18 flurodeoxyglucose ( 18 FDG) positron emission tomography (PET) imaging and mediastinoscopy. The results demonstrate that stage-dependent morbidity is an important consideration for patients with NSCLC and should be included in any decision analysis regarding the evaluation or treatment of NSCLC. Respondents valued the quality of information obtained from non-invasive mediastinoscopy comparably. The utilities obtained from this investigation are useful in clinical decision-making based on Markov processes because they provide an initial estimation of utility assessment for 18 FDG-based diagnostic evaluation of lung cancer. Consequently, these utilities will be useful in future decision analyses that require patient preference in the assignment of the evaluation of decision options (branches)
Transition probabilities of health states for workers in Malaysia using a Markov chain model
Samsuddin, Shamshimah; Ismail, Noriszura
2017-04-01
The aim of our study is to estimate the transition probabilities of health states for workers in Malaysia who contribute to the Employment Injury Scheme under the Social Security Organization Malaysia using the Markov chain model. Our study uses four states of health (active, temporary disability, permanent disability and death) based on the data collected from the longitudinal studies of workers in Malaysia for 5 years. The transition probabilities vary by health state, age and gender. The results show that men employees are more likely to have higher transition probabilities to any health state compared to women employees. The transition probabilities can be used to predict the future health of workers in terms of a function of current age, gender and health state.
Optimal State Estimation for Discrete-Time Markov Jump Systems with Missing Observations
Directory of Open Access Journals (Sweden)
Qing Sun
2014-01-01
Full Text Available This paper is concerned with the optimal linear estimation for a class of direct-time Markov jump systems with missing observations. An observer-based approach of fault detection and isolation (FDI is investigated as a detection mechanic of fault case. For systems with known information, a conditional prediction of observations is applied and fault observations are replaced and isolated; then, an FDI linear minimum mean square error estimation (LMMSE can be developed by comprehensive utilizing of the correct information offered by systems. A recursive equation of filtering based on the geometric arguments can be obtained. Meanwhile, a stability of the state estimator will be guaranteed under appropriate assumption.
Markov state modeling and dynamical coarse-graining via discrete relaxation path sampling.
Fačkovec, B; Vanden-Eijnden, E; Wales, D J
2015-07-28
A method is derived to coarse-grain the dynamics of complex molecular systems to a Markov jump process (MJP) describing how the system jumps between cells that fully partition its state space. The main inputs are relaxation times for each pair of cells, which are shown to be robust with respect to positioning of the cell boundaries. These relaxation times can be calculated via molecular dynamics simulations performed in each cell separately and are used in an efficient estimator for the rate matrix of the MJP. The method is illustrated through applications to Sinai billiards and a cluster of Lennard-Jones discs.
Markov stochasticity coordinates
International Nuclear Information System (INIS)
Eliazar, Iddo
2017-01-01
Markov dynamics constitute one of the most fundamental models of random motion between the states of a system of interest. Markov dynamics have diverse applications in many fields of science and engineering, and are particularly applicable in the context of random motion in networks. In this paper we present a two-dimensional gauging method of the randomness of Markov dynamics. The method–termed Markov Stochasticity Coordinates–is established, discussed, and exemplified. Also, the method is tweaked to quantify the stochasticity of the first-passage-times of Markov dynamics, and the socioeconomic equality and mobility in human societies.
Markov stochasticity coordinates
Energy Technology Data Exchange (ETDEWEB)
Eliazar, Iddo, E-mail: iddo.eliazar@intel.com
2017-01-15
Markov dynamics constitute one of the most fundamental models of random motion between the states of a system of interest. Markov dynamics have diverse applications in many fields of science and engineering, and are particularly applicable in the context of random motion in networks. In this paper we present a two-dimensional gauging method of the randomness of Markov dynamics. The method–termed Markov Stochasticity Coordinates–is established, discussed, and exemplified. Also, the method is tweaked to quantify the stochasticity of the first-passage-times of Markov dynamics, and the socioeconomic equality and mobility in human societies.
Benkhelifa, Fatma
2013-09-01
The effective capacity have been introduced by Wu and Neji as a link-layer model supporting statistical delay QoS requirements. In this paper, we propose to study the effective capacity of a Nakagami-m fading channel with full channel state information (CSI) at both the transmitter and at the receiver. We focus on the low Signal-to-Noise Ratio (SNR) regime. We show that the effective capacity for any arbitrary but finite statistically delay Quality of Service (QoS) exponent θ, scales essentially as S NRlog(1/SNR) exactly as the ergodic capacity, independently of any QoS constraint. We also characterize the minimum energy required for reliable communication, and the wideband slope to show that our results are in agreement with results established recently by Gursoy et al. We also propose an on-off power control scheme that achieves the capacity asymptotically using only one bit CSI feedback at the transmitter. Finally, some numerical results are presented to show the accuracy of our asymptotic results. © 2013 IEEE.
A methodology for stochastic analysis of share prices as Markov chains with finite states.
Mettle, Felix Okoe; Quaye, Enoch Nii Boi; Laryea, Ravenhill Adjetey
2014-01-01
Price volatilities make stock investments risky, leaving investors in critical position when uncertain decision is made. To improve investor evaluation confidence on exchange markets, while not using time series methodology, we specify equity price change as a stochastic process assumed to possess Markov dependency with respective state transition probabilities matrices following the identified state pace (i.e. decrease, stable or increase). We established that identified states communicate, and that the chains are aperiodic and ergodic thus possessing limiting distributions. We developed a methodology for determining expected mean return time for stock price increases and also establish criteria for improving investment decision based on highest transition probabilities, lowest mean return time and highest limiting distributions. We further developed an R algorithm for running the methodology introduced. The established methodology is applied to selected equities from Ghana Stock Exchange weekly trading data.
On the Capacity of the Dirty Paper Channel with Fast Fading and Discrete Channel States
Rini, Stefano; Shitz, Shlomo Shamai
2016-01-01
The "writing dirty paper" capacity result crucially dependents on the perfect channel knowledge at the transmitter as the presence of even a small uncertainty in the channel realization gravely hampers the ability of the transmitter to pre-code its transmission against the channel state. This is particularly disappointing as it implies that interference pre-coding in practical systems is effective only when the channel estimates at the users have very high precision, a condition which is gene...
Capturing the state transitions of seizure-like events using Hidden Markov models.
Guirgis, Mirna; Serletis, Demitre; Carlen, Peter L; Bardakjian, Berj L
2011-01-01
The purpose of this study was to investigate the number of states present in the progression of a seizure-like event (SLE). Of particular interest is to determine if there are more than two clearly defined states, as this would suggest that there is a distinct state preceding an SLE. Whole-intact hippocampus from C57/BL mice was used to model epileptiform activity induced by the perfusion of a low Mg(2+)/high K(+) solution while extracellular field potentials were recorded from CA3 pyramidal neurons. Hidden Markov models (HMM) were used to model the state transitions of the recorded SLEs by incorporating various features of the Hilbert transform into the training algorithm; specifically, 2- and 3-state HMMs were explored. Although the 2-state model was able to distinguish between SLE and nonSLE behavior, it provided no improvements compared to visual inspection alone. However, the 3-state model was able to capture two distinct nonSLE states that visual inspection failed to discriminate. Moreover, by developing an HMM based system a priori knowledge of the state transitions was not required making this an ideal platform for seizure prediction algorithms.
Dawid, H.; Keoula, M.Y.; Kort, Peter
2017-01-01
This paper presents a numerical method for the characterization of Markov-perfect equilibria of symmetric differential games exhibiting coexisting stable steady states. The method relying on the calculation of ‘local value functions’ through collocation in overlapping parts of the state space, is
Simplification of reversible Markov chains by removal of states with low equilibrium occupancy.
Ullah, Ghanim; Bruno, William J; Pearson, John E
2012-10-21
We present a practical method for simplifying Markov chains on a potentially large state space when detailed balance holds. A simple and transparent technique is introduced to remove states with low equilibrium occupancy. The resulting system has fewer parameters. The resulting effective rates between the remaining nodes give dynamics identical to the original system's except on very fast timescales. This procedure amounts to using separation of timescales to neglect small capacitance nodes in a network of resistors and capacitors. We illustrate the technique by simplifying various reaction networks, including transforming an acyclic four-node network to a three-node cyclic network. For a reaction step in which a ligand binds, the law of mass action implies a forward rate proportional to ligand concentration. The effective rates in the simplified network are found to be rational functions of ligand concentration. Copyright © 2012 Elsevier Ltd. All rights reserved.
A versatile infinite-state Markov reward model to study bottlenecks in 2-hop ad hoc networks
Remke, Anne Katharina Ingrid; Haverkort, Boudewijn R.H.M.; Cloth, L.
2006-01-01
In a 2-hop IEEE 801.11-based wireless LAN, the distributed coordination function (DCF) tends to equally share the available capacity among the contending stations. Recently alternative capacity sharing strategies have been made possible. We propose a versatile infinite-state Markov reward model to
Sweeting, M J; Farewell, V T; De Angelis, D
2010-05-20
In many chronic diseases it is important to understand the rate at which patients progress from infection through a series of defined disease states to a clinical outcome, e.g. cirrhosis in hepatitis C virus (HCV)-infected individuals or AIDS in HIV-infected individuals. Typically data are obtained from longitudinal studies, which often are observational in nature, and where disease state is observed only at selected examinations throughout follow-up. Transition times between disease states are therefore interval censored. Multi-state Markov models are commonly used to analyze such data, but rely on the assumption that the examination times are non-informative, and hence the examination process is ignorable in a likelihood-based analysis. In this paper we develop a Markov model that relaxes this assumption through the premise that the examination process is ignorable only after conditioning on a more regularly observed auxiliary variable. This situation arises in a study of HCV disease progression, where liver biopsies (the examinations) are sparse, irregular, and potentially informative with respect to the transition times. We use additional information on liver function tests (LFTs), commonly collected throughout follow-up, to inform current disease state and to assume an ignorable examination process. The model developed has a similar structure to a hidden Markov model and accommodates both the series of LFT measurements and the partially latent series of disease states. We show through simulation how this model compares with the commonly used ignorable Markov model, and a Markov model that assumes the examination process is non-ignorable. Copyright 2010 John Wiley & Sons, Ltd.
International Nuclear Information System (INIS)
Lim, KaiChin; Bastawrous, Hany Ayad; Duong, Van-Huan; See, Khay Wai; Zhang, Peng; Dou, Shi Xue
2016-01-01
Highlights: • Real-time battery model parameters and SoC estimation with novel method is proposed. • Cascading filtering stages are used for parameters identification and SoC estimation. • Optimized fading Kalman filter is implemented for SoC estimation. • Accurate SoC estimation is validated in UDDS load profile experiment. • This approach is suitable for BMS in EV applications due to its simplicity. - Abstract: A novel online estimation technique for estimating the state of charge (SoC) of a lithium iron phosphate (LiFePO_4) battery has been developed. Based on a simplified model, the open circuit voltage (OCV) of the battery is estimated through two cascaded linear filtering stages. A recursive least squares filter is employed in the first stage to dynamically estimate the battery model parameters in real-time, and then, a fading Kalman filter (FKF) is used to estimate the OCV from these parameters. FKF can avoid the possibility of large estimation errors, which may occur with a conventional Kalman filter, due to its capability to compensate any modeling error through a fading factor. By optimizing the value of the fading factor in the set of recursion equations of FKF with genetic algorithms, the errors in estimating the battery’s SoC in urban dynamometer driving schedules-based experiments and real vehicle driving cycle experiments were below 3% compared to more than 9% in the case of using an ordinary Kalman filter. The proposed method with its simplified model provides the simplicity and feasibility required for real-time application with highly accurate SoC estimation.
Confluence reduction for Markov automata
Timmer, Mark; Katoen, Joost P.; van de Pol, Jaco; Stoelinga, Mariëlle Ida Antoinette
2016-01-01
Markov automata are a novel formalism for specifying systems exhibiting nondeterminism, probabilistic choices and Markovian rates. As expected, the state space explosion threatens the analysability of these models. We therefore introduce confluence reduction for Markov automata, a powerful reduction
Zeng, Xiaojun; Zhang, Liyun; Xiao, Xiuchan; Jiang, Yuanyuan; Guo, Yanzhi; Yu, Xinyan; Pu, Xuemei; Li, Menglong
2016-04-05
Thrombin-binding aptamer (TBA) with the sequence 5'GGTTGGTGTGGTTGG3' could fold into G-quadruplex, which correlates with functionally important genomic regionsis. However, unfolding mechanism involved in the structural stability of G-quadruplex has not been satisfactorily elucidated on experiments so far. Herein, we studied the unfolding pathway of TBA by a combination of molecular dynamics simulation (MD) and Markov State Model (MSM). Our results revealed that the unfolding of TBA is not a simple two-state process but proceeds along multiple pathways with multistate intermediates. One high flux confirms some observations from NMR experiment. Another high flux exhibits a different and simpler unfolding pathway with less intermediates. Two important intermediate states were identified. One is similar to the G-triplex reported in the folding of G-quadruplex, but lack of H-bonding between guanines in the upper plane. More importantly, another intermediate state acting as a connector to link the folding region and the unfolding one, was the first time identified, which exhibits higher population and stability than the G-triplex-like intermediate. These results will provide valuable information for extending our understanding the folding landscape of G-quadruplex formation.
Grabski
2014-01-01
Semi-Markov Processes: Applications in System Reliability and Maintenance is a modern view of discrete state space and continuous time semi-Markov processes and their applications in reliability and maintenance. The book explains how to construct semi-Markov models and discusses the different reliability parameters and characteristics that can be obtained from those models. The book is a useful resource for mathematicians, engineering practitioners, and PhD and MSc students who want to understand the basic concepts and results of semi-Markov process theory. Clearly defines the properties and
Radford, Isolde H; Fersht, Alan R; Settanni, Giovanni
2011-06-09
Atomistic molecular dynamics simulations of the TZ1 beta-hairpin peptide have been carried out using an implicit model for the solvent. The trajectories have been analyzed using a Markov state model defined on the projections along two significant observables and a kinetic network approach. The Markov state model allowed for an unbiased identification of the metastable states of the system, and provided the basis for commitment probability calculations performed on the kinetic network. The kinetic network analysis served to extract the main transition state for folding of the peptide and to validate the results from the Markov state analysis. The combination of the two techniques allowed for a consistent and concise characterization of the dynamics of the peptide. The slowest relaxation process identified is the exchange between variably folded and denatured species, and the second slowest process is the exchange between two different subsets of the denatured state which could not be otherwise identified by simple inspection of the projected trajectory. The third slowest process is the exchange between a fully native and a partially folded intermediate state characterized by a native turn with a proximal backbone H-bond, and frayed side-chain packing and termini. The transition state for the main folding reaction is similar to the intermediate state, although a more native like side-chain packing is observed.
A Markov State-based Quantitative Kinetic Model of Sodium Release from the Dopamine Transporter
Razavi, Asghar M.; Khelashvili, George; Weinstein, Harel
2017-01-01
The dopamine transporter (DAT) belongs to the neurotransmitter:sodium symporter (NSS) family of membrane proteins that are responsible for reuptake of neurotransmitters from the synaptic cleft to terminate a neuronal signal and enable subsequent neurotransmitter release from the presynaptic neuron. The release of one sodium ion from the crystallographically determined sodium binding site Na2 had been identified as an initial step in the transport cycle which prepares the transporter for substrate translocation by stabilizing an inward-open conformation. We have constructed Markov State Models (MSMs) from extensive molecular dynamics simulations of human DAT (hDAT) to explore the mechanism of this sodium release. Our results quantify the release process triggered by hydration of the Na2 site that occurs concomitantly with a conformational transition from an outward-facing to an inward-facing state of the transporter. The kinetics of the release process are computed from the MSM, and transition path theory is used to identify the most probable sodium release pathways. An intermediate state is discovered on the sodium release pathway, and the results reveal the importance of various modes of interaction of the N-terminus of hDAT in controlling the pathways of release.
Analysis of the trajectory surface hopping method from the Markov state model perspective
International Nuclear Information System (INIS)
Akimov, Alexey V.; Wang, Linjun; Prezhdo, Oleg V.; Trivedi, Dhara
2015-01-01
We analyze the applicability of the seminal fewest switches surface hopping (FSSH) method of Tully to modeling quantum transitions between electronic states that are not coupled directly, in the processes such as Auger recombination. We address the known deficiency of the method to describe such transitions by introducing an alternative definition for the surface hopping probabilities, as derived from the Markov state model perspective. We show that the resulting transition probabilities simplify to the quantum state populations derived from the time-dependent Schrödinger equation, reducing to the rapidly switching surface hopping approach of Tully and Preston. The resulting surface hopping scheme is simple and appeals to the fundamentals of quantum mechanics. The computational approach is similar to the FSSH method of Tully, yet it leads to a notably different performance. We demonstrate that the method is particularly accurate when applied to superexchange modeling. We further show improved accuracy of the method, when applied to one of the standard test problems. Finally, we adapt the derived scheme to atomistic simulation, combine it with the time-domain density functional theory, and show that it provides the Auger energy transfer timescales which are in good agreement with experiment, significantly improving upon other considered techniques. (author)
Jia, Chen
2017-09-01
Here we develop an effective approach to simplify two-time-scale Markov chains with infinite state spaces by removal of states with fast leaving rates, which improves the simplification method of finite Markov chains. We introduce the concept of fast transition paths and show that the effective transitions of the reduced chain can be represented as the superposition of the direct transitions and the indirect transitions via all the fast transition paths. Furthermore, we apply our simplification approach to the standard Markov model of single-cell stochastic gene expression and provide a mathematical theory of random gene expression bursts. We give the precise mathematical conditions for the bursting kinetics of both mRNAs and proteins. It turns out that random bursts exactly correspond to the fast transition paths of the Markov model. This helps us gain a better understanding of the physics behind the bursting kinetics as an emergent behavior from the fundamental multiscale biochemical reaction kinetics of stochastic gene expression.
Wang, Wei
2017-10-06
The function of complex biomolecular machines relies heavily on their conformational changes. Investigating these functional conformational changes is therefore essential for understanding the corresponding biological processes and promoting bioengineering applications and rational drug design. Constructing Markov State Models (MSMs) based on large-scale molecular dynamics simulations has emerged as a powerful approach to model functional conformational changes of the biomolecular system with sufficient resolution in both time and space. However, the rapid development of theory and algorithms for constructing MSMs has made it difficult for nonexperts to understand and apply the MSM framework, necessitating a comprehensive guidance toward its theory and practical usage. In this study, we introduce the MSM theory of conformational dynamics based on the projection operator scheme. We further propose a general protocol of constructing MSM to investigate functional conformational changes, which integrates the state-of-the-art techniques for building and optimizing initial pathways, performing adaptive sampling and constructing MSMs. We anticipate this protocol to be widely applied and useful in guiding nonexperts to study the functional conformational changes of large biomolecular systems via the MSM framework. We also discuss the current limitations of MSMs and some alternative methods to alleviate them.
A novel seizure detection algorithm informed by hidden Markov model event states
Baldassano, Steven; Wulsin, Drausin; Ung, Hoameng; Blevins, Tyler; Brown, Mesha-Gay; Fox, Emily; Litt, Brian
2016-06-01
Objective. Recently the FDA approved the first responsive, closed-loop intracranial device to treat epilepsy. Because these devices must respond within seconds of seizure onset and not miss events, they are tuned to have high sensitivity, leading to frequent false positive stimulations and decreased battery life. In this work, we propose a more robust seizure detection model. Approach. We use a Bayesian nonparametric Markov switching process to parse intracranial EEG (iEEG) data into distinct dynamic event states. Each event state is then modeled as a multidimensional Gaussian distribution to allow for predictive state assignment. By detecting event states highly specific for seizure onset zones, the method can identify precise regions of iEEG data associated with the transition to seizure activity, reducing false positive detections associated with interictal bursts. The seizure detection algorithm was translated to a real-time application and validated in a small pilot study using 391 days of continuous iEEG data from two dogs with naturally occurring, multifocal epilepsy. A feature-based seizure detector modeled after the NeuroPace RNS System was developed as a control. Main results. Our novel seizure detection method demonstrated an improvement in false negative rate (0/55 seizures missed versus 2/55 seizures missed) as well as a significantly reduced false positive rate (0.0012 h versus 0.058 h-1). All seizures were detected an average of 12.1 ± 6.9 s before the onset of unequivocal epileptic activity (unequivocal epileptic onset (UEO)). Significance. This algorithm represents a computationally inexpensive, individualized, real-time detection method suitable for implantable antiepileptic devices that may considerably reduce false positive rate relative to current industry standards.
Adaptive hidden Markov model with anomaly States for price manipulation detection.
Cao, Yi; Li, Yuhua; Coleman, Sonya; Belatreche, Ammar; McGinnity, Thomas Martin
2015-02-01
Price manipulation refers to the activities of those traders who use carefully designed trading behaviors to manually push up or down the underlying equity prices for making profits. With increasing volumes and frequency of trading, price manipulation can be extremely damaging to the proper functioning and integrity of capital markets. The existing literature focuses on either empirical studies of market abuse cases or analysis of particular manipulation types based on certain assumptions. Effective approaches for analyzing and detecting price manipulation in real time are yet to be developed. This paper proposes a novel approach, called adaptive hidden Markov model with anomaly states (AHMMAS) for modeling and detecting price manipulation activities. Together with wavelet transformations and gradients as the feature extraction methods, the AHMMAS model caters to price manipulation detection and basic manipulation type recognition. The evaluation experiments conducted on seven stock tick data from NASDAQ and the London Stock Exchange and 10 simulated stock prices by stochastic differential equation show that the proposed AHMMAS model can effectively detect price manipulation patterns and outperforms the selected benchmark models.
Two-state Markov-chain Poisson nature of individual cellphone call statistics
Jiang, Zhi-Qiang; Xie, Wen-Jie; Li, Ming-Xia; Zhou, Wei-Xing; Sornette, Didier
2016-07-01
Unfolding the burst patterns in human activities and social interactions is a very important issue especially for understanding the spreading of disease and information and the formation of groups and organizations. Here, we conduct an in-depth study of the temporal patterns of cellphone conversation activities of 73 339 anonymous cellphone users, whose inter-call durations are Weibull distributed. We find that the individual call events exhibit a pattern of bursts, that high activity periods are alternated with low activity periods. In both periods, the number of calls are exponentially distributed for individuals, but power-law distributed for the population. Together with the exponential distributions of inter-call durations within bursts and of the intervals between consecutive bursts, we demonstrate that the individual call activities are driven by two independent Poisson processes, which can be combined within a minimal model in terms of a two-state first-order Markov chain, giving significant fits for nearly half of the individuals. By measuring directly the distributions of call rates across the population, which exhibit power-law tails, we purport the existence of power-law distributions, via the ‘superposition of distributions’ mechanism. Our findings shed light on the origins of bursty patterns in other human activities.
2016-03-24
current USD) 179 179 164 1990 0 Improved Water Source (% population with access ) 178 175 175 1960 1 Life Expectancy (years) 182 182 1990 2 Military...Microsoft Office visual basic ( VBA ) based consolidated 42 database tool was developed to compile the 39 separate identifying-information and data...conflict remains in conflictP A brief discussion of the accessibility of states within a Markov model is warranted before we proceed further
Improving Markov Chain Models for Road Profiles Simulation via Definition of States
2012-04-01
wavelet transform in pavement profile analysis," Vehicle System Dynamics: International Journal of Vehicle Mechanics and Mobility, vol. 47, no. 4...34Estimating Markov Transition Probabilities from Micro -Unit Data," Journal of the Royal Statistical Society. Series C (Applied Statistics), pp. 355-371
Multi-state reliability for pump group in system based on UGF and semi-Markov process
International Nuclear Information System (INIS)
Shang Yanlong; Cai Qi; Zhao Xinwen; Chen Ling
2012-01-01
In this paper, multi-state reliability value of pump group in nuclear power system is obtained by the combination method of the universal generating function (UGF) and Semi-Markov process. UGF arithmetic model of multi-state system reliability is studied, and the performance state probability expression of multi-state component is derived using semi-Markov theory. A quantificational model is defined to express the performance rate of the system and component. Different availability results by multi-state and binary state analysis method are compared under the condition whether the performance rate can satisfy the demanded value, and the mean value of system instantaneous output performance is also obtained. It shows that this combination method is an effective and feasible one which can quantify the effect of the partial failure on the system reliability, and the result of multi-state system reliability by this method deduces the modesty of the reliability value obtained by binary reliability analysis method. (authors)
Markov Chain Monte Carlo Methods
Indian Academy of Sciences (India)
Keywords. Markov chain; state space; stationary transition probability; stationary distribution; irreducibility; aperiodicity; stationarity; M-H algorithm; proposal distribution; acceptance probability; image processing; Gibbs sampler.
Song, Jian; Gao, Fang; Cui, Raymond Z; Shuang, Feng; Liang, Wanzhen; Huang, Xuhui; Zhuang, Wei
2012-10-25
Vibrationally resolved fluorescence spectra of the β-hairpin trpzip2 peptide at two temperatures as well as during a T-jump unfolding process are simulated on the basis of a combination of Markov state models and quantum chemistry schemes. The broad asymmetric spectral line shape feature is reproduced by considering the exciton-phonon couplings. The temperature dependent red shift observed in the experiment has been attributed to the state population changes of specific chromophores. Through further theoretical study, it is found that both the environment's electric field and the chromophores' geometry distortions are responsible for tryptophan fluorescence shift.
On Capacity of the Writing onto Fast Fading Dirt Channel
Rini, Stefano; Shamai, Shlomo
2016-01-01
The "Writing onto Fast Fading Dirt" (WFFD) channel is investigated to study the effects of partial channel knowledge on the capacity of the "writing on dirty paper" channel. The WFFD channel is the Gel'fand-Pinsker channel in which the output is obtained as the sum of the input, white Gaussian noise and a fading-times-state term. The fading-times-state term is equal to the element-wise product of the channel state sequence, known only at the transmitter, and a fast fading process, known only ...
Wei, Shaoceng; Kryscio, Richard J
2016-12-01
Continuous-time multi-state stochastic processes are useful for modeling the flow of subjects from intact cognition to dementia with mild cognitive impairment and global impairment as intervening transient cognitive states and death as a competing risk. Each subject's cognition is assessed periodically resulting in interval censoring for the cognitive states while death without dementia is not interval censored. Since back transitions among the transient states are possible, Markov chains are often applied to this type of panel data. In this manuscript, we apply a semi-Markov process in which we assume that the waiting times are Weibull distributed except for transitions from the baseline state, which are exponentially distributed and in which we assume no additional changes in cognition occur between two assessments. We implement a quasi-Monte Carlo (QMC) method to calculate the higher order integration needed for likelihood estimation. We apply our model to a real dataset, the Nun Study, a cohort of 461 participants. © The Author(s) 2014.
Schofield, Jeremy; Bayat, Hanif
2014-09-07
A Markov state model of the dynamics of a protein-like chain immersed in an implicit hard sphere solvent is derived from first principles for a system of monomers that interact via discontinuous potentials designed to account for local structure and bonding in a coarse-grained sense. The model is based on the assumption that the implicit solvent interacts on a fast time scale with the monomers of the chain compared to the time scale for structural rearrangements of the chain and provides sufficient friction so that the motion of monomers is governed by the Smoluchowski equation. A microscopic theory for the dynamics of the system is developed that reduces to a Markovian model of the kinetics under well-defined conditions. Microscopic expressions for the rate constants that appear in the Markov state model are analyzed and expressed in terms of a temperature-dependent linear combination of escape rates that themselves are independent of temperature. Excellent agreement is demonstrated between the theoretical predictions of the escape rates and those obtained through simulation of a stochastic model of the dynamics of bond formation. Finally, the Markov model is studied by analyzing the eigenvalues and eigenvectors of the matrix of transition rates, and the equilibration process for a simple helix-forming system from an ensemble of initially extended configurations to mainly folded configurations is investigated as a function of temperature for a number of different chain lengths. For short chains, the relaxation is primarily single-exponential and becomes independent of temperature in the low-temperature regime. The profile is more complicated for longer chains, where multi-exponential relaxation behavior is seen at intermediate temperatures followed by a low temperature regime in which the folding becomes rapid and single exponential. It is demonstrated that the behavior of the equilibration profile as the temperature is lowered can be understood in terms of the
A scaling analysis of a cat and mouse Markov chain
Litvak, Nelli; Robert, Philippe
2012-01-01
If ($C_n$) a Markov chain on a discrete state space $S$, a Markov chain ($C_n, M_n$) on the product space $S \\times S$, the cat and mouse Markov chain, is constructed. The first coordinate of this Markov chain behaves like the original Markov chain and the second component changes only when both
Barbu, Vlad
2008-01-01
Semi-Markov processes are much more general and better adapted to applications than the Markov ones because sojourn times in any state can be arbitrarily distributed, as opposed to the geometrically distributed sojourn time in the Markov case. This book concerns with the estimation of discrete-time semi-Markov and hidden semi-Markov processes
A Hybrid of Deep Network and Hidden Markov Model for MCI Identification with Resting-State fMRI.
Suk, Heung-Il; Lee, Seong-Whan; Shen, Dinggang
2015-10-01
In this paper, we propose a novel method for modelling functional dynamics in resting-state fMRI (rs-fMRI) for Mild Cognitive Impairment (MCI) identification. Specifically, we devise a hybrid architecture by combining Deep Auto-Encoder (DAE) and Hidden Markov Model (HMM). The roles of DAE and HMM are, respectively, to discover hierarchical non-linear relations among features, by which we transform the original features into a lower dimension space, and to model dynamic characteristics inherent in rs-fMRI, i.e. , internal state changes. By building a generative model with HMMs for each class individually, we estimate the data likelihood of a test subject as MCI or normal healthy control, based on which we identify the clinical label. In our experiments, we achieved the maximal accuracy of 81.08% with the proposed method, outperforming state-of-the-art methods in the literature.
International Nuclear Information System (INIS)
Floriani, Elena; Lima, Ricardo; Ourrad, Ouerdia; Spinelli, Lionel
2016-01-01
Highlights: • The flux through a Markov chain of a conserved quantity (mass) is studied. • Mass is supplied by an external source and ends in the absorbing states of the chain. • Meaningful for modeling open systems whose dynamics has a Markov property. • The analytical expression of mass distribution is given for a constant source. • The expression of mass distribution is given for periodic or random sources. - Abstract: In this paper we study the flux through a finite Markov chain of a quantity, that we will call mass, which moves through the states of the chain according to the Markov transition probabilities. Mass is supplied by an external source and accumulates in the absorbing states of the chain. We believe that studying how this conserved quantity evolves through the transient (non-absorbing) states of the chain could be useful for the modelization of open systems whose dynamics has a Markov property.
Prinz, Jan-Hendrik; Chodera, John D; Pande, Vijay S; Swope, William C; Smith, Jeremy C; Noé, Frank
2011-06-28
Parallel tempering (PT) molecular dynamics simulations have been extensively investigated as a means of efficient sampling of the configurations of biomolecular systems. Recent work has demonstrated how the short physical trajectories generated in PT simulations of biomolecules can be used to construct the Markov models describing biomolecular dynamics at each simulated temperature. While this approach describes the temperature-dependent kinetics, it does not make optimal use of all available PT data, instead estimating the rates at a given temperature using only data from that temperature. This can be problematic, as some relevant transitions or states may not be sufficiently sampled at the temperature of interest, but might be readily sampled at nearby temperatures. Further, the comparison of temperature-dependent properties can suffer from the false assumption that data collected from different temperatures are uncorrelated. We propose here a strategy in which, by a simple modification of the PT protocol, the harvested trajectories can be reweighted, permitting data from all temperatures to contribute to the estimated kinetic model. The method reduces the statistical uncertainty in the kinetic model relative to the single temperature approach and provides estimates of transition probabilities even for transitions not observed at the temperature of interest. Further, the method allows the kinetics to be estimated at temperatures other than those at which simulations were run. We illustrate this method by applying it to the generation of a Markov model of the conformational dynamics of the solvated terminally blocked alanine peptide.
Saddlepoint expansions for sums of Markov dependent variables on a continuous state space
DEFF Research Database (Denmark)
Jensen, J.L.
1991-01-01
Based on the conjugate kernel studied in Iscoe et al. (1985) we derive saddlepoint expansions for either the density or distribution function of a sum f(X1)+...+f(Xn), where the Xi's constitute a Markov chain. The chain is assumed to satisfy a strong recurrence condition which makes the results...... here very similar to the classical results for i.i.d. variables. In particular we establish also conditions under which the expansions hold uniformly over the range of the saddlepoint. Expansions are also derived for sums of the form f(X1, X0)+f(X2, X1)+...+f(Xn, Xn-1) although the uniformity result...
Energy Technology Data Exchange (ETDEWEB)
Schofield, Jeremy, E-mail: jmschofi@chem.utoronto.ca; Bayat, Hanif, E-mail: hbayat@chem.utoronto.ca [Chemical Physics Theory Group, Department of Chemistry, University of Toronto, Toronto, Ontario M5S 3H6 (Canada)
2014-09-07
A Markov state model of the dynamics of a protein-like chain immersed in an implicit hard sphere solvent is derived from first principles for a system of monomers that interact via discontinuous potentials designed to account for local structure and bonding in a coarse-grained sense. The model is based on the assumption that the implicit solvent interacts on a fast time scale with the monomers of the chain compared to the time scale for structural rearrangements of the chain and provides sufficient friction so that the motion of monomers is governed by the Smoluchowski equation. A microscopic theory for the dynamics of the system is developed that reduces to a Markovian model of the kinetics under well-defined conditions. Microscopic expressions for the rate constants that appear in the Markov state model are analyzed and expressed in terms of a temperature-dependent linear combination of escape rates that themselves are independent of temperature. Excellent agreement is demonstrated between the theoretical predictions of the escape rates and those obtained through simulation of a stochastic model of the dynamics of bond formation. Finally, the Markov model is studied by analyzing the eigenvalues and eigenvectors of the matrix of transition rates, and the equilibration process for a simple helix-forming system from an ensemble of initially extended configurations to mainly folded configurations is investigated as a function of temperature for a number of different chain lengths. For short chains, the relaxation is primarily single-exponential and becomes independent of temperature in the low-temperature regime. The profile is more complicated for longer chains, where multi-exponential relaxation behavior is seen at intermediate temperatures followed by a low temperature regime in which the folding becomes rapid and single exponential. It is demonstrated that the behavior of the equilibration profile as the temperature is lowered can be understood in terms of the
International Nuclear Information System (INIS)
Schofield, Jeremy; Bayat, Hanif
2014-01-01
A Markov state model of the dynamics of a protein-like chain immersed in an implicit hard sphere solvent is derived from first principles for a system of monomers that interact via discontinuous potentials designed to account for local structure and bonding in a coarse-grained sense. The model is based on the assumption that the implicit solvent interacts on a fast time scale with the monomers of the chain compared to the time scale for structural rearrangements of the chain and provides sufficient friction so that the motion of monomers is governed by the Smoluchowski equation. A microscopic theory for the dynamics of the system is developed that reduces to a Markovian model of the kinetics under well-defined conditions. Microscopic expressions for the rate constants that appear in the Markov state model are analyzed and expressed in terms of a temperature-dependent linear combination of escape rates that themselves are independent of temperature. Excellent agreement is demonstrated between the theoretical predictions of the escape rates and those obtained through simulation of a stochastic model of the dynamics of bond formation. Finally, the Markov model is studied by analyzing the eigenvalues and eigenvectors of the matrix of transition rates, and the equilibration process for a simple helix-forming system from an ensemble of initially extended configurations to mainly folded configurations is investigated as a function of temperature for a number of different chain lengths. For short chains, the relaxation is primarily single-exponential and becomes independent of temperature in the low-temperature regime. The profile is more complicated for longer chains, where multi-exponential relaxation behavior is seen at intermediate temperatures followed by a low temperature regime in which the folding becomes rapid and single exponential. It is demonstrated that the behavior of the equilibration profile as the temperature is lowered can be understood in terms of the
El Yazid Boudaren, Mohamed; Monfrini, Emmanuel; Pieczynski, Wojciech; Aïssani, Amar
2014-11-01
Hidden Markov chains have been shown to be inadequate for data modeling under some complex conditions. In this work, we address the problem of statistical modeling of phenomena involving two heterogeneous system states. Such phenomena may arise in biology or communications, among other fields. Namely, we consider that a sequence of meaningful words is to be searched within a whole observation that also contains arbitrary one-by-one symbols. Moreover, a word may be interrupted at some site to be carried on later. Applying plain hidden Markov chains to such data, while ignoring their specificity, yields unsatisfactory results. The Phasic triplet Markov chain, proposed in this paper, overcomes this difficulty by means of an auxiliary underlying process in accordance with the triplet Markov chains theory. Related Bayesian restoration techniques and parameters estimation procedures according to the new model are then described. Finally, to assess the performance of the proposed model against the conventional hidden Markov chain model, experiments are conducted on synthetic and real data.
Modular Hamiltonians on the null plane and the Markov property of the vacuum state
Casini, Horacio; Testé, Eduardo; Torroba, Gonzalo
2017-09-01
We compute the modular Hamiltonians of regions having the future horizon lying on a null plane. For a CFT this is equivalent to regions with a boundary of arbitrary shape lying on the null cone. These Hamiltonians have a local expression on the horizon formed by integrals of the stress tensor. We prove this result in two different ways, and show that the modular Hamiltonians of these regions form an infinite dimensional Lie algebra. The corresponding group of unitary transformations moves the fields on the null surface locally along the null generators with arbitrary null line dependent velocities, but act non-locally outside the null plane. We regain this result in greater generality using more abstract tools on the algebraic quantum field theory. Finally, we show that modular Hamiltonians on the null surface satisfy a Markov property that leads to the saturation of the strong sub-additive inequality for the entropies and to the strong super-additivity of the relative entropy.
Dynamic of foreign direct investment in the states of Mexico: An analysis of Markov's spatial chains
Directory of Open Access Journals (Sweden)
Víctor Hugo Torres Preciado
2017-01-01
Full Text Available El objetivo de esta investigación consiste en analizar la evolución de la distribución espacial y temporal de la inversión extranjera directa (IED en las entidades federativas de México. La literatura que aborda el análisis de la IED en México es abundante y diversa; sin embargo, se argumenta que el análisis de la distribución espacio-temporal de la IED condicionada a la interacción espacial en México, aún está ausente. En este sentido, mediante la aplicación del enfoque de cadenas de Markov espaciales propuesto por Rey (2001, se encuentra que la divergencia regional en la captación de IED es un proceso que parece afianzarse cuando se analizan diferentes cortes en el tiempo. En particular, durante el periodo entre 2006 y 2013 el proceso de divergencia hacia estratos de mayor captación estaría impulsado por las entidades federativas que interactúan con entidades contiguas ubicadas en estratos de captación de IED menores.
Solan, Eilon; Vieille, Nicolas
2015-01-01
We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the transition matrix. We define a new closeness relation between transition matrices, and use graph-theoretic techniques, in contrast with the matrix analysis techniques previously used.
Partially Hidden Markov Models
DEFF Research Database (Denmark)
Forchhammer, Søren Otto; Rissanen, Jorma
1996-01-01
Partially Hidden Markov Models (PHMM) are introduced. They differ from the ordinary HMM's in that both the transition probabilities of the hidden states and the output probabilities are conditioned on past observations. As an illustration they are applied to black and white image compression where...
Benoit, Julia S; Chan, Wenyaw; Luo, Sheng; Yeh, Hung-Wen; Doody, Rachelle
2016-04-30
Understanding the dynamic disease process is vital in early detection, diagnosis, and measuring progression. Continuous-time Markov chain (CTMC) methods have been used to estimate state-change intensities but challenges arise when stages are potentially misclassified. We present an analytical likelihood approach where the hidden state is modeled as a three-state CTMC model allowing for some observed states to be possibly misclassified. Covariate effects of the hidden process and misclassification probabilities of the hidden state are estimated without information from a 'gold standard' as comparison. Parameter estimates are obtained using a modified expectation-maximization (EM) algorithm, and identifiability of CTMC estimation is addressed. Simulation studies and an application studying Alzheimer's disease caregiver stress-levels are presented. The method was highly sensitive to detecting true misclassification and did not falsely identify error in the absence of misclassification. In conclusion, we have developed a robust longitudinal method for analyzing categorical outcome data when classification of disease severity stage is uncertain and the purpose is to study the process' transition behavior without a gold standard. Copyright © 2016 John Wiley & Sons, Ltd.
Matsunaga, Y.; Sugita, Y.
2018-06-01
A data-driven modeling scheme is proposed for conformational dynamics of biomolecules based on molecular dynamics (MD) simulations and experimental measurements. In this scheme, an initial Markov State Model (MSM) is constructed from MD simulation trajectories, and then, the MSM parameters are refined using experimental measurements through machine learning techniques. The second step can reduce the bias of MD simulation results due to inaccurate force-field parameters. Either time-series trajectories or ensemble-averaged data are available as a training data set in the scheme. Using a coarse-grained model of a dye-labeled polyproline-20, we compare the performance of machine learning estimations from the two types of training data sets. Machine learning from time-series data could provide the equilibrium populations of conformational states as well as their transition probabilities. It estimates hidden conformational states in more robust ways compared to that from ensemble-averaged data although there are limitations in estimating the transition probabilities between minor states. We discuss how to use the machine learning scheme for various experimental measurements including single-molecule time-series trajectories.
Hobolth, Asger; Stone, Eric A
2009-09-01
Analyses of serially-sampled data often begin with the assumption that the observations represent discrete samples from a latent continuous-time stochastic process. The continuous-time Markov chain (CTMC) is one such generative model whose popularity extends to a variety of disciplines ranging from computational finance to human genetics and genomics. A common theme among these diverse applications is the need to simulate sample paths of a CTMC conditional on realized data that is discretely observed. Here we present a general solution to this sampling problem when the CTMC is defined on a discrete and finite state space. Specifically, we consider the generation of sample paths, including intermediate states and times of transition, from a CTMC whose beginning and ending states are known across a time interval of length T. We first unify the literature through a discussion of the three predominant approaches: (1) modified rejection sampling, (2) direct sampling, and (3) uniformization. We then give analytical results for the complexity and efficiency of each method in terms of the instantaneous transition rate matrix Q of the CTMC, its beginning and ending states, and the length of sampling time T. In doing so, we show that no method dominates the others across all model specifications, and we give explicit proof of which method prevails for any given Q, T, and endpoints. Finally, we introduce and compare three applications of CTMCs to demonstrate the pitfalls of choosing an inefficient sampler.
International Nuclear Information System (INIS)
Pourkarim Guilani, Pedram; Sharifi, Mani; Niaki, S.T.A.; Zaretalab, Arash
2014-01-01
In multi-state systems (MSS) reliability problems, it is assumed that the components of each subsystem have different performance rates with certain probabilities. This leads into extensive computational efforts involved in using the commonly employed universal generation function (UGF) and the recursive algorithm to obtain reliability of systems consisting of a large number of components. This research deals with evaluating non-repairable three-state systems reliability and proposes a novel method based on a Markov process for which an appropriate state definition is provided. It is shown that solving the derived differential equations significantly reduces the computational time compared to the UGF and the recursive algorithm. - Highlights: • Reliability evaluation of a non-repairable three-state systems is aimed. • A novel method based on a Markov process is proposed. • An appropriate state definition is provided. • Computational time is significantly less compared to the ones in the UGF and the recursive methods
Optimal Training for Time-Selective Wireless Fading Channels Using Cutoff Rate
Directory of Open Access Journals (Sweden)
Tong Lang
2006-01-01
Full Text Available We consider the optimal allocation of resources—power and bandwidth—between training and data transmissions for single-user time-selective Rayleigh flat-fading channels under the cutoff rate criterion. The transmitter exploits statistical channel state information (CSI in the form of the channel Doppler spectrum to embed pilot symbols into the transmission stream. At the receiver, instantaneous, though imperfect, CSI is acquired through minimum mean-square estimation of the channel based on some set of pilot observations. We compute the ergodic cutoff rate for this scenario. Assuming estimator-based interleaving and -PSK inputs, we study two special cases in-depth. First, we derive the optimal resource allocation for the Gauss-Markov correlation model. Next, we validate and refine these insights by studying resource allocation for the Jakes model.
Online scenario labeling using a hidden Markov model for assessment of nuclear plant state
International Nuclear Information System (INIS)
Zamalieva, Daniya; Yilmaz, Alper; Aldemir, Tunc
2013-01-01
By taking into account both aleatory and epistemic uncertainties within the same probabilistic framework, dynamic event trees (DETs) provide more comprehensive and systematic coverage of possible scenarios following an initiating event compared to conventional event trees. When DET generation algorithms are applied to complex realistic systems, extremely large amounts of data can be produced due to both the large number of scenarios generated following a single initiating event and the large number of data channels that represent these scenarios. In addition, the computational time required for the simulation of each scenario can be very large (e.g. about 24 h of serial run simulation time for a 4 h station blackout scenario). Since scenarios leading to system failure are more of interest, a method is proposed for online labeling of scenarios as failure or non-failure. The algorithm first trains a Hidden Markov Model, which represents the behavior of non-failure scenarios, using a training set from previous simulations. Then, the maximum likelihoods of sample failure and non-failure scenarios fitting this model are computed. These values are used to determine the timestamp at which the labeling of a certain scenario should be performed. Finally, during the succeeding timestamps, the likelihood of each scenario fitting the learned model is computed, and a dynamic thresholding based on the previously calculated likelihood values is applied. The scenarios whose likelihood is higher than the threshold are labeled as non-failure. The proposed algorithm can further delay the non-failure scenarios or discontinue them in order to redirect the computational resources toward the failure scenarios, and reduce computational time and complexity. Experiments using RELAP5/3D model of a fast reactor utilizing an Reactor Vessel Auxiliary Cooling System (RVACS) passive decay heat removal system and dynamic analysis of a station blackout (SBO) event show that the proposed method is
Confluence reduction for Markov automata
Timmer, Mark; van de Pol, Jan Cornelis; Stoelinga, Mariëlle Ida Antoinette
Markov automata are a novel formalism for specifying systems exhibiting nondeterminism, probabilistic choices and Markovian rates. Recently, the process algebra MAPA was introduced to efficiently model such systems. As always, the state space explosion threatens the analysability of the models
Confluence Reduction for Markov Automata
Timmer, Mark; van de Pol, Jan Cornelis; Stoelinga, Mariëlle Ida Antoinette; Braberman, Victor; Fribourg, Laurent
Markov automata are a novel formalism for specifying systems exhibiting nondeterminism, probabilistic choices and Markovian rates. Recently, the process algebra MAPA was introduced to efficiently model such systems. As always, the state space explosion threatens the analysability of the models
Phillips, Joe Scutt; Patterson, Toby A; Leroy, Bruno; Pilling, Graham M; Nicol, Simon J
2015-07-01
Analysis of complex time-series data from ecological system study requires quantitative tools for objective description and classification. These tools must take into account largely ignored problems of bias in manual classification, autocorrelation, and noise. Here we describe a method using existing estimation techniques for multivariate-normal hidden Markov models (HMMs) to develop such a classification. We use high-resolution behavioral data from bio-loggers attached to free-roaming pelagic tuna as an example. Observed patterns are assumed to be generated by an unseen Markov process that switches between several multivariate-normal distributions. Our approach is assessed in two parts. The first uses simulation experiments, from which the ability of the HMM to estimate known parameter values is examined using artificial time series of data consistent with hypotheses about pelagic predator foraging ecology. The second is the application to time series of continuous vertical movement data from yellowfin and bigeye tuna taken from tuna tagging experiments. These data were compressed into summary metrics capturing the variation of patterns in diving behavior and formed into a multivariate time series used to estimate a HMM. Each observation was associated with covariate information incorporating the effect of day and night on behavioral switching. Known parameter values were well recovered by the HMMs in our simulation experiments, resulting in mean correct classification rates of 90-97%, although some variance-covariance parameters were estimated less accurately. HMMs with two distinct behavioral states were selected for every time series of real tuna data, predicting a shallow warm state, which was similar across all individuals, and a deep colder state, which was more variable. Marked diurnal behavioral switching was predicted, consistent with many previous empirical studies on tuna. HMMs provide easily interpretable models for the objective classification of
Directory of Open Access Journals (Sweden)
Fang Ming
2017-01-01
Full Text Available As a main direction of Human Resource Management, employee turnover can provide decision support for managers. In this paper, we aim at predicting the turnover amount of employee on condition of different variable values. The properties of employee and job position are formulated as two variables, where the value of variable varies according to the the state of properties. Additionally, state-transition model is applied to describing employee’s job-state as well as the turnover type. Subsequently, we proposed a semi-Markov model to calculate the conditional turnover amount of employee. Then, we provide a dataset of employee records to illustrate how these models work in reality. Finally, it is proven that the proposed method in this paper is with great significance for managers to develop recruitment plans, promote rules, and retire regulations
A scaling analysis of a cat and mouse Markov chain
Litvak, Nelli; Robert, Philippe
Motivated by an original on-line page-ranking algorithm, starting from an arbitrary Markov chain $(C_n)$ on a discrete state space ${\\cal S}$, a Markov chain $(C_n,M_n)$ on the product space ${\\cal S}^2$, the cat and mouse Markov chain, is constructed. The first coordinate of this Markov chain
Energy Technology Data Exchange (ETDEWEB)
Frank, T D [Center for the Ecological Study of Perception and Action, Department of Psychology, University of Connecticut, 406 Babbidge Road, Storrs, CT 06269 (United States)
2008-07-18
We discuss nonlinear Markov processes defined on discrete time points and discrete state spaces using Markov chains. In this context, special attention is paid to the distinction between linear and nonlinear Markov processes. We illustrate that the Chapman-Kolmogorov equation holds for nonlinear Markov processes by a winner-takes-all model for social conformity. (fast track communication)
International Nuclear Information System (INIS)
Frank, T D
2008-01-01
We discuss nonlinear Markov processes defined on discrete time points and discrete state spaces using Markov chains. In this context, special attention is paid to the distinction between linear and nonlinear Markov processes. We illustrate that the Chapman-Kolmogorov equation holds for nonlinear Markov processes by a winner-takes-all model for social conformity. (fast track communication)
Uniqueness of Gibbs states and global Markov property for Euclidean fields
International Nuclear Information System (INIS)
Albeverio, S.; Hoeegh-Krohn, R.
1981-01-01
The authors briefly discuss the proof of the uniqueness of solutions of the DLR equations (uniqueness of Gibbs states) in the class of regular generalized random fields (in the sense of having second moments bounded by those of some Euclidean field), for the Euclidean fields with trigonometric interaction. (Auth.)
Generalized Markov branching models
Li, Junping
2005-01-01
In this thesis, we first considered a modified Markov branching process incorporating both state-independent immigration and resurrection. After establishing the criteria for regularity and uniqueness, explicit expressions for the extinction probability and mean extinction time are presented. The criteria for recurrence and ergodicity are also established. In addition, an explicit expression for the equilibrium distribution is presented.\\ud \\ud We then moved on to investigate the basic proper...
International Nuclear Information System (INIS)
Liang Jinling; Lam, James; Wang Zidong
2009-01-01
This Letter is concerned with the robust state estimation problem for uncertain time-delay Markovian jumping genetic regulatory networks (GRNs) with SUM logic, where the uncertainties enter into both the network parameters and the mode transition rate. The nonlinear functions describing the feedback regulation are assumed to satisfy the sector-like conditions. The main purpose of the problem addressed is to design a linear estimator to approximate the true concentrations of the mRNA and protein through available measurement outputs. By resorting to the Lyapunov functional method and some stochastic analysis tools, it is shown that if a set of linear matrix inequalities (LMIs) is feasible, the desired state estimator, that can ensure the estimation error dynamics to be globally robustly asymptotically stable in the mean square, exists. The obtained LMI conditions are dependent on both the lower and the upper bounds of the delays. An illustrative example is presented to demonstrate the feasibility of the proposed estimation schemes.
A Hidden Markov Movement Model for rapidly identifying behavioral states from animal tracks
DEFF Research Database (Denmark)
Whoriskey, Kim; Auger-Méthé, Marie; Albertsen, Christoffer Moesgaard
2017-01-01
by fitting it to real tracks from a grey seal, lake trout, and blue shark, as well as to simulated data. 4. The HMMM is a fast and reliable tool for making meaningful inference from animal movement data that is ideally suited for ecologists who want to use the popular DCRWS implementation for highly accurate......1. Electronic telemetry is frequently used to document animal movement through time. Methods that can identify underlying behaviors driving specific movement patterns can help us understand how and why animals use available space, thereby aiding conservation and management efforts. For aquatic...... animal tracking data with significant measurement error, a Bayesian state-space model called the first-Difference Correlated Random Walk with Switching (DCRWS) has often been used for this purpose. However, for aquatic animals, highly accurate tracking data of animal movement are now becoming more common...
Markov processes and controlled Markov chains
Filar, Jerzy; Chen, Anyue
2002-01-01
The general theory of stochastic processes and the more specialized theory of Markov processes evolved enormously in the second half of the last century. In parallel, the theory of controlled Markov chains (or Markov decision processes) was being pioneered by control engineers and operations researchers. Researchers in Markov processes and controlled Markov chains have been, for a long time, aware of the synergies between these two subject areas. However, this may be the first volume dedicated to highlighting these synergies and, almost certainly, it is the first volume that emphasizes the contributions of the vibrant and growing Chinese school of probability. The chapters that appear in this book reflect both the maturity and the vitality of modern day Markov processes and controlled Markov chains. They also will provide an opportunity to trace the connections that have emerged between the work done by members of the Chinese school of probability and the work done by the European, US, Central and South Ameri...
Directory of Open Access Journals (Sweden)
Becka M Warfield
Full Text Available RNA aptamers are oligonucleotides that bind with high specificity and affinity to target ligands. In the absence of bound ligand, secondary structures of RNA aptamers are generally stable, but single-stranded and loop regions, including ligand binding sites, lack defined structures and exist as ensembles of conformations. For example, the well-characterized theophylline-binding aptamer forms a highly stable binding site when bound to theophylline, but the binding site is unstable and disordered when theophylline is absent. Experimental methods have not revealed at atomic resolution the conformations that the theophylline aptamer explores in its unbound state. Consequently, in the present study we applied 21 microseconds of molecular dynamics simulations to structurally characterize the ensemble of conformations that the aptamer adopts in the absence of theophylline. Moreover, we apply Markov state modeling to predict the kinetics of transitions between unbound conformational states. Our simulation results agree with experimental observations that the theophylline binding site is found in many distinct binding-incompetent states and show that these states lack a binding pocket that can accommodate theophylline. The binding-incompetent states interconvert with binding-competent states through structural rearrangement of the binding site on the nanosecond to microsecond timescale. Moreover, we have simulated the complete theophylline binding pathway. Our binding simulations supplement prior experimental observations of slow theophylline binding kinetics by showing that the binding site must undergo a large conformational rearrangement after the aptamer and theophylline form an initial complex, most notably, a major rearrangement of the C27 base from a buried to solvent-exposed orientation. Theophylline appears to bind by a combination of conformational selection and induced fit mechanisms. Finally, our modeling indicates that when Mg2+ ions are
Prediction of Annual Rainfall Pattern Using Hidden Markov Model ...
African Journals Online (AJOL)
ADOWIE PERE
Hidden Markov model is very influential in stochastic world because of its ... the earth from the clouds. The usual ... Rainfall modelling and ... Markov Models have become popular tools ... environment sciences, University of Jos, plateau state,.
Ciampi, Antonio; Dyachenko, Alina; Cole, Martin; McCusker, Jane
2011-12-01
The study of mental disorders in the elderly presents substantial challenges due to population heterogeneity, coexistence of different mental disorders, and diagnostic uncertainty. While reliable tools have been developed to collect relevant data, new approaches to study design and analysis are needed. We focus on a new analytic approach. Our framework is based on latent class analysis and hidden Markov chains. From repeated measurements of a multivariate disease index, we extract the notion of underlying state of a patient at a time point. The course of the disorder is then a sequence of transitions among states. States and transitions are not observable; however, the probability of being in a state at a time point, and the transition probabilities from one state to another over time can be estimated. Data from 444 patients with and without diagnosis of delirium and dementia were available from a previous study. The Delirium Index was measured at diagnosis, and at 2 and 6 months from diagnosis. Four latent classes were identified: fairly healthy, moderately ill, clearly sick, and very sick. Dementia and delirium could not be separated on the basis of these data alone. Indeed, as the probability of delirium increased, so did the probability of decline of mental functions. Eight most probable courses were identified, including good and poor stable courses, and courses exhibiting various patterns of improvement. Latent class analysis and hidden Markov chains offer a promising tool for studying mental disorders in the elderly. Its use may show its full potential as new data become available.
B. HEITZ
2005-01-01
This paper seeks to apply the general framework of Markov-switching models to inflation in France and in the USA. We propose a model where inflation can, alternatively, follow two regimes: the first one, where inflation is stationary, is interpreted as a situation where there exists a credible inflation target, even if it is not explicit; the second one where inflation is integrated. Moreover, observing that the two oil shocks were followed by accelerating inflation periods, we allow dependen...
Energy Technology Data Exchange (ETDEWEB)
Kaflowski, Grzegorz; Kizilcay, Mustafa [Siegen Univ. (Germany). Lehrstuhl fuer Elektrische Energieversorgung
2010-02-15
In long-term planning of modernization measures in electric power supplies, forecasting of the future behaviour of the equipment is of great importance for its optimum utilization. Based on limited historical information on the technical condition of a collective of equipment, its future changes of condition can be modelled using Markov chains, which may support an assessment of the time of modernization or replacement. (orig.)
DEFF Research Database (Denmark)
Kohlenbach, Ulrich Wilhelm
2002-01-01
We show that the so-called weak Markov's principle (WMP) which states that every pseudo-positive real number is positive is underivable in E-HA + AC. Since allows one to formalize (atl eastl arge parts of) Bishop's constructive mathematics, this makes it unlikely that WMP can be proved within...... the framework of Bishop-style mathematics (which has been open for about 20 years). The underivability even holds if the ine.ective schema of full comprehension (in all types) for negated formulas (in particular for -free formulas) is added, which allows one to derive the law of excluded middle...
Performance Modeling of Communication Networks with Markov Chains
Mo, Jeonghoon
2010-01-01
This book is an introduction to Markov chain modeling with applications to communication networks. It begins with a general introduction to performance modeling in Chapter 1 where we introduce different performance models. We then introduce basic ideas of Markov chain modeling: Markov property, discrete time Markov chain (DTMe and continuous time Markov chain (CTMe. We also discuss how to find the steady state distributions from these Markov chains and how they can be used to compute the system performance metric. The solution methodologies include a balance equation technique, limiting probab
Quantum Markov Chain Mixing and Dissipative Engineering
DEFF Research Database (Denmark)
Kastoryano, Michael James
2012-01-01
This thesis is the fruit of investigations on the extension of ideas of Markov chain mixing to the quantum setting, and its application to problems of dissipative engineering. A Markov chain describes a statistical process where the probability of future events depends only on the state...... of the system at the present point in time, but not on the history of events. Very many important processes in nature are of this type, therefore a good understanding of their behaviour has turned out to be very fruitful for science. Markov chains always have a non-empty set of limiting distributions...... (stationary states). The aim of Markov chain mixing is to obtain (upper and/or lower) bounds on the number of steps it takes for the Markov chain to reach a stationary state. The natural quantum extensions of these notions are density matrices and quantum channels. We set out to develop a general mathematical...
Probabilistic Reachability for Parametric Markov Models
DEFF Research Database (Denmark)
Hahn, Ernst Moritz; Hermanns, Holger; Zhang, Lijun
2011-01-01
Given a parametric Markov model, we consider the problem of computing the rational function expressing the probability of reaching a given set of states. To attack this principal problem, Daws has suggested to first convert the Markov chain into a finite automaton, from which a regular expression...
Markov-modulated and feedback fluid queues
Scheinhardt, Willem R.W.
1998-01-01
In the last twenty years the field of Markov-modulated fluid queues has received considerable attention. In these models a fluid reservoir receives and/or releases fluid at rates which depend on the actual state of a background Markov chain. In the first chapter of this thesis we give a short
Zhou, Shuangyan; Wang, Qianqian; Wang, Yuwei; Yao, Xiaojun; Han, Wei; Liu, Huanxiang
2017-05-10
The structural transition of prion proteins from a native α-helix (PrP C ) to a misfolded β-sheet-rich conformation (PrP Sc ) is believed to be the main cause of a number of prion diseases in humans and animals. Understanding the molecular basis of misfolding and aggregation of prion proteins will be valuable for unveiling the etiology of prion diseases. However, due to the limitation of conventional experimental techniques and the heterogeneous property of oligomers, little is known about the molecular architecture of misfolded PrP Sc and the mechanism of structural transition from PrP C to PrP Sc . The prion fragment 127-147 (PrP127-147) has been reported to be a critical region for PrP Sc formation in Gerstmann-Straussler-Scheinker (GSS) syndrome and thus has been used as a model for the study of prion aggregation. In the present study, we employ molecular dynamics (MD) simulation techniques to study the conformational change of this fragment that could be relevant to the PrP C -PrP Sc transition. Employing extensive replica exchange molecular dynamics (REMD) and conventional MD simulations, we sample a huge number of conformations of PrP127-147. Using the Markov state model (MSM), we identify the metastable conformational states of this fragment and the kinetic network of transitions between the states. The resulting MSM reveals that disordered random-coiled conformations are the dominant structures. A key metastable folded state with typical extended β-sheet structures is identified with Pro137 being located in a turn region, consistent with a previous experimental report. Conformational analysis reveals that intrapeptide hydrophobic interaction and two key residue interactions, including Arg136-His140 and Pro137-His140, contribute a lot to the formation of ordered extended β-sheet states. However, network pathway analysis from the most populated disordered state indicates that the formation of extended β-sheet states is quite slow (at the millisecond
Wireless Transmission of Big Data: A Transmission Time Analysis over Fading Channel
Wang, Wen-Jing; Yang, Hong-Chuan; Alouini, Mohamed-Slim
2018-01-01
In this paper, we investigate the transmission time of a large amount of data over fading wireless channel with adaptive modulation and coding (AMC). Unlike traditional transmission systems, where the transmission time of a fixed amount of data is typically regarded as a constant, the transmission time with AMC becomes a random variable, as the transmission rate varies with the fading channel condition. To facilitate the design and optimization of wireless transmission schemes for big data applications, we present an analytical framework to determine statistical characterizations for the transmission time of big data with AMC. In particular, we derive the exact statistics of transmission time over block fading channels. The probability mass function (PMF) and cumulative distribution function (CDF) of transmission time are obtained for both slow and fast fading scenarios. We further extend our analysis to Markov channel, where transmission time becomes the sum of a sequence of exponentially distributed time slots. Analytical expression for the probability density function (PDF) of transmission time is derived for both fast fading and slow fading scenarios. These analytical results are essential to the optimal design and performance analysis of future wireless transmission systems for big data applications.
Wireless Transmission of Big Data: A Transmission Time Analysis over Fading Channel
Wang, Wen-Jing
2018-04-10
In this paper, we investigate the transmission time of a large amount of data over fading wireless channel with adaptive modulation and coding (AMC). Unlike traditional transmission systems, where the transmission time of a fixed amount of data is typically regarded as a constant, the transmission time with AMC becomes a random variable, as the transmission rate varies with the fading channel condition. To facilitate the design and optimization of wireless transmission schemes for big data applications, we present an analytical framework to determine statistical characterizations for the transmission time of big data with AMC. In particular, we derive the exact statistics of transmission time over block fading channels. The probability mass function (PMF) and cumulative distribution function (CDF) of transmission time are obtained for both slow and fast fading scenarios. We further extend our analysis to Markov channel, where transmission time becomes the sum of a sequence of exponentially distributed time slots. Analytical expression for the probability density function (PDF) of transmission time is derived for both fast fading and slow fading scenarios. These analytical results are essential to the optimal design and performance analysis of future wireless transmission systems for big data applications.
Markov chains and mixing times
Levin, David A; Wilmer, Elizabeth L
2009-01-01
This book is an introduction to the modern approach to the theory of Markov chains. The main goal of this approach is to determine the rate of convergence of a Markov chain to the stationary distribution as a function of the size and geometry of the state space. The authors develop the key tools for estimating convergence times, including coupling, strong stationary times, and spectral methods. Whenever possible, probabilistic methods are emphasized. The book includes many examples and provides brief introductions to some central models of statistical mechanics. Also provided are accounts of r
Directory of Open Access Journals (Sweden)
Jaewook Lee
2015-06-01
Full Text Available This paper presents an efficient method for estimating capacity-fade uncertainty in lithium-ion batteries (LIBs in order to integrate them into the battery-management system (BMS of electric vehicles, which requires simple and inexpensive computation for successful application. The study uses the pseudo-two-dimensional (P2D electrochemical model, which simulates the battery state by solving a system of coupled nonlinear partial differential equations (PDEs. The model parameters that are responsible for electrode degradation are identified and estimated, based on battery data obtained from the charge cycles. The Bayesian approach, with parameters estimated by probability distributions, is employed to account for uncertainties arising in the model and battery data. The Markov Chain Monte Carlo (MCMC technique is used to draw samples from the distributions. The complex computations that solve a PDE system for each sample are avoided by employing a polynomial-based metamodel. As a result, the computational cost is reduced from 5.5 h to a few seconds, enabling the integration of the method into the vehicle BMS. Using this approach, the conservative bound of capacity fade can be determined for the vehicle in service, which represents the safety margin reflecting the uncertainty.
Transition Effect Matrices and Quantum Markov Chains
Gudder, Stan
2009-06-01
A transition effect matrix (TEM) is a quantum generalization of a classical stochastic matrix. By employing a TEM we obtain a quantum generalization of a classical Markov chain. We first discuss state and operator dynamics for a quantum Markov chain. We then consider various types of TEMs and vector states. In particular, we study invariant, equilibrium and singular vector states and investigate projective, bistochastic, invertible and unitary TEMs.
Observations of anomalous fading in maiolica
International Nuclear Information System (INIS)
Bowman, S.G.E.
1988-01-01
In the course of an authenticity study on Italian maiolica (tin-glazed earthenware of the Renaissance period), storage at elevated temperature was used to accelerate anomalous fading. Substantial levels of fading were observed in about half of the samples, and in these cases the variation of fading with glow curve temperature accounted for the lack of an equivalent dose plateau. Some evidence was found for a difference in the fading between alpha and beta induced thermoluminescence (TL). More importantly, some samples with unstable natural TL were found: the implications of this for dating and the circumvention of fading are discussed. (author)
Semi-Markov Arnason-Schwarz models.
King, Ruth; Langrock, Roland
2016-06-01
We consider multi-state capture-recapture-recovery data where observed individuals are recorded in a set of possible discrete states. Traditionally, the Arnason-Schwarz model has been fitted to such data where the state process is modeled as a first-order Markov chain, though second-order models have also been proposed and fitted to data. However, low-order Markov models may not accurately represent the underlying biology. For example, specifying a (time-independent) first-order Markov process involves the assumption that the dwell time in each state (i.e., the duration of a stay in a given state) has a geometric distribution, and hence that the modal dwell time is one. Specifying time-dependent or higher-order processes provides additional flexibility, but at the expense of a potentially significant number of additional model parameters. We extend the Arnason-Schwarz model by specifying a semi-Markov model for the state process, where the dwell-time distribution is specified more generally, using, for example, a shifted Poisson or negative binomial distribution. A state expansion technique is applied in order to represent the resulting semi-Markov Arnason-Schwarz model in terms of a simpler and computationally tractable hidden Markov model. Semi-Markov Arnason-Schwarz models come with only a very modest increase in the number of parameters, yet permit a significantly more flexible state process. Model selection can be performed using standard procedures, and in particular via the use of information criteria. The semi-Markov approach allows for important biological inference to be drawn on the underlying state process, for example, on the times spent in the different states. The feasibility of the approach is demonstrated in a simulation study, before being applied to real data corresponding to house finches where the states correspond to the presence or absence of conjunctivitis. © 2015, The International Biometric Society.
Confluence reduction for Markov automata (extended version)
Timmer, Mark; van de Pol, Jan Cornelis; Stoelinga, Mariëlle Ida Antoinette
Markov automata are a novel formalism for specifying systems exhibiting nondeterminism, probabilistic choices and Markovian rates. Recently, the process algebra MAPA was introduced to efficiently model such systems. As always, the state space explosion threatens the analysability of the models
A Novel Simulation Model for Nonstationary Rice Fading Channels
Directory of Open Access Journals (Sweden)
Kaili Jiang
2018-01-01
Full Text Available In this paper, we propose a new simulator for nonstationary Rice fading channels under nonisotropic scattering scenarios, as well as the improved computation method of simulation parameters. The new simulator can also be applied on generating Rayleigh fading channels by adjusting parameters. The proposed simulator takes into account the smooth transition of fading phases between the adjacent channel states. The time-variant statistical properties of the proposed simulator, that is, the probability density functions (PDFs of envelope and phase, autocorrelation function (ACF, and Doppler power spectrum density (DPSD, are also analyzed and derived. Simulation results have demonstrated that our proposed simulator provides good approximation on the statistical properties with the corresponding theoretical ones, which indicates its usefulness for the performance evaluation and validation of the wireless communication systems under nonstationary and nonisotropic scenarios.
Subharmonic projections for a quantum Markov semigroup
International Nuclear Information System (INIS)
Fagnola, Franco; Rebolledo, Rolando
2002-01-01
This article introduces a concept of subharmonic projections for a quantum Markov semigroup, in view of characterizing the support projection of a stationary state in terms of the semigroup generator. These results, together with those of our previous article [J. Math. Phys. 42, 1296 (2001)], lead to a method for proving the existence of faithful stationary states. This is often crucial in the analysis of ergodic properties of quantum Markov semigroups. The method is illustrated by applications to physical models
Jeong, Hyundoo; Qian, Xiaoning; Yoon, Byung-Jun
2016-10-06
Comparative analysis of protein-protein interaction (PPI) networks provides an effective means of detecting conserved functional network modules across different species. Such modules typically consist of orthologous proteins with conserved interactions, which can be exploited to computationally predict the modules through network comparison. In this work, we propose a novel probabilistic framework for comparing PPI networks and effectively predicting the correspondence between proteins, represented as network nodes, that belong to conserved functional modules across the given PPI networks. The basic idea is to estimate the steady-state network flow between nodes that belong to different PPI networks based on a Markov random walk model. The random walker is designed to make random moves to adjacent nodes within a PPI network as well as cross-network moves between potential orthologous nodes with high sequence similarity. Based on this Markov random walk model, we estimate the steady-state network flow - or the long-term relative frequency of the transitions that the random walker makes - between nodes in different PPI networks, which can be used as a probabilistic score measuring their potential correspondence. Subsequently, the estimated scores can be used for detecting orthologous proteins in conserved functional modules through network alignment. Through evaluations based on multiple real PPI networks, we demonstrate that the proposed scheme leads to improved alignment results that are biologically more meaningful at reduced computational cost, outperforming the current state-of-the-art algorithms. The source code and datasets can be downloaded from http://www.ece.tamu.edu/~bjyoon/CUFID .
Generated dynamics of Markov and quantum processes
Janßen, Martin
2016-01-01
This book presents Markov and quantum processes as two sides of a coin called generated stochastic processes. It deals with quantum processes as reversible stochastic processes generated by one-step unitary operators, while Markov processes are irreversible stochastic processes generated by one-step stochastic operators. The characteristic feature of quantum processes are oscillations, interference, lots of stationary states in bounded systems and possible asymptotic stationary scattering states in open systems, while the characteristic feature of Markov processes are relaxations to a single stationary state. Quantum processes apply to systems where all variables, that control reversibility, are taken as relevant variables, while Markov processes emerge when some of those variables cannot be followed and are thus irrelevant for the dynamic description. Their absence renders the dynamic irreversible. A further aim is to demonstrate that almost any subdiscipline of theoretical physics can conceptually be put in...
A Novel Method for Decoding Any High-Order Hidden Markov Model
Directory of Open Access Journals (Sweden)
Fei Ye
2014-01-01
Full Text Available This paper proposes a novel method for decoding any high-order hidden Markov model. First, the high-order hidden Markov model is transformed into an equivalent first-order hidden Markov model by Hadar’s transformation. Next, the optimal state sequence of the equivalent first-order hidden Markov model is recognized by the existing Viterbi algorithm of the first-order hidden Markov model. Finally, the optimal state sequence of the high-order hidden Markov model is inferred from the optimal state sequence of the equivalent first-order hidden Markov model. This method provides a unified algorithm framework for decoding hidden Markov models including the first-order hidden Markov model and any high-order hidden Markov model.
Tveito, Aslak; Lines, Glenn T; Edwards, Andrew G; McCulloch, Andrew
2016-07-01
Markov models are ubiquitously used to represent the function of single ion channels. However, solving the inverse problem to construct a Markov model of single channel dynamics from bilayer or patch-clamp recordings remains challenging, particularly for channels involving complex gating processes. Methods for solving the inverse problem are generally based on data from voltage clamp measurements. Here, we describe an alternative approach to this problem based on measurements of voltage traces. The voltage traces define probability density functions of the functional states of an ion channel. These probability density functions can also be computed by solving a deterministic system of partial differential equations. The inversion is based on tuning the rates of the Markov models used in the deterministic system of partial differential equations such that the solution mimics the properties of the probability density function gathered from (pseudo) experimental data as well as possible. The optimization is done by defining a cost function to measure the difference between the deterministic solution and the solution based on experimental data. By evoking the properties of this function, it is possible to infer whether the rates of the Markov model are identifiable by our method. We present applications to Markov model well-known from the literature. Copyright © 2016 The Authors. Published by Elsevier Inc. All rights reserved.
Markov chains models, algorithms and applications
Ching, Wai-Ki; Ng, Michael K; Siu, Tak-Kuen
2013-01-01
This new edition of Markov Chains: Models, Algorithms and Applications has been completely reformatted as a text, complete with end-of-chapter exercises, a new focus on management science, new applications of the models, and new examples with applications in financial risk management and modeling of financial data.This book consists of eight chapters. Chapter 1 gives a brief introduction to the classical theory on both discrete and continuous time Markov chains. The relationship between Markov chains of finite states and matrix theory will also be highlighted. Some classical iterative methods
Markov chains analytic and Monte Carlo computations
Graham, Carl
2014-01-01
Markov Chains: Analytic and Monte Carlo Computations introduces the main notions related to Markov chains and provides explanations on how to characterize, simulate, and recognize them. Starting with basic notions, this book leads progressively to advanced and recent topics in the field, allowing the reader to master the main aspects of the classical theory. This book also features: Numerous exercises with solutions as well as extended case studies.A detailed and rigorous presentation of Markov chains with discrete time and state space.An appendix presenting probabilistic notions that are nec
Fade Mitigation Techniques at Ka-Band
Dissanayake, Asoka (Editor)
1996-01-01
Rain fading is the dominant propagation impairment affecting Ka-band satellite links and rain fade mitigation is a key element in the design of Ka-band satellite networks. Some of the common fade mitigation techniques include: power control, diversity, adaptive coding, and resource sharing. The Advanced Communications Technology Satellite (ACTS) provides an excellent opportunity to develop and test Ka-band rain impairment amelioration techniques. Up-link power control and diversity are discussed in this paper.
DEFF Research Database (Denmark)
Justesen, Jørn
2005-01-01
A simple construction of two-dimensional (2-D) fields is presented. Rows and columns are outcomes of the same Markov chain. The entropy can be calculated explicitly.......A simple construction of two-dimensional (2-D) fields is presented. Rows and columns are outcomes of the same Markov chain. The entropy can be calculated explicitly....
Jiang, Hanlun
2015-07-16
Argonaute (Ago) proteins and microRNAs (miRNAs) are central components in RNA interference, which is a key cellular mechanism for sequence-specific gene silencing. Despite intensive studies, molecular mechanisms of how Ago recognizes miRNA remain largely elusive. In this study, we propose a two-step mechanism for this molecular recognition: selective binding followed by structural re-arrangement. Our model is based on the results of a combination of Markov State Models (MSMs), large-scale protein-RNA docking, and molecular dynamics (MD) simulations. Using MSMs, we identify an open state of apo human Ago-2 in fast equilibrium with partially open and closed states. Conformations in this open state are distinguished by their largely exposed binding grooves that can geometrically accommodate miRNA as indicated in our protein-RNA docking studies. miRNA may then selectively bind to these open conformations. Upon the initial binding, the complex may perform further structural re-arrangement as shown in our MD simulations and eventually reach the stable binary complex structure. Our results provide novel insights in Ago-miRNA recognition mechanisms and our methodology holds great potential to be widely applied in the studies of other important molecular recognition systems.
Abhinav, S.; Manohar, C. S.
2018-03-01
The problem of combined state and parameter estimation in nonlinear state space models, based on Bayesian filtering methods, is considered. A novel approach, which combines Rao-Blackwellized particle filters for state estimation with Markov chain Monte Carlo (MCMC) simulations for parameter identification, is proposed. In order to ensure successful performance of the MCMC samplers, in situations involving large amount of dynamic measurement data and (or) low measurement noise, the study employs a modified measurement model combined with an importance sampling based correction. The parameters of the process noise covariance matrix are also included as quantities to be identified. The study employs the Rao-Blackwellization step at two stages: one, associated with the state estimation problem in the particle filtering step, and, secondly, in the evaluation of the ratio of likelihoods in the MCMC run. The satisfactory performance of the proposed method is illustrated on three dynamical systems: (a) a computational model of a nonlinear beam-moving oscillator system, (b) a laboratory scale beam traversed by a loaded trolley, and (c) an earthquake shake table study on a bending-torsion coupled nonlinear frame subjected to uniaxial support motion.
Jiang, Hanlun; Sheong, Fu Kit; Zhu, Lizhe; Gao, Xin; Bernauer, Julie; Huang, Xuhui
2015-01-01
Argonaute (Ago) proteins and microRNAs (miRNAs) are central components in RNA interference, which is a key cellular mechanism for sequence-specific gene silencing. Despite intensive studies, molecular mechanisms of how Ago recognizes miRNA remain largely elusive. In this study, we propose a two-step mechanism for this molecular recognition: selective binding followed by structural re-arrangement. Our model is based on the results of a combination of Markov State Models (MSMs), large-scale protein-RNA docking, and molecular dynamics (MD) simulations. Using MSMs, we identify an open state of apo human Ago-2 in fast equilibrium with partially open and closed states. Conformations in this open state are distinguished by their largely exposed binding grooves that can geometrically accommodate miRNA as indicated in our protein-RNA docking studies. miRNA may then selectively bind to these open conformations. Upon the initial binding, the complex may perform further structural re-arrangement as shown in our MD simulations and eventually reach the stable binary complex structure. Our results provide novel insights in Ago-miRNA recognition mechanisms and our methodology holds great potential to be widely applied in the studies of other important molecular recognition systems.
Markov process of muscle motors
International Nuclear Information System (INIS)
Kondratiev, Yu; Pechersky, E; Pirogov, S
2008-01-01
We study a Markov random process describing muscle molecular motor behaviour. Every motor is either bound up with a thin filament or unbound. In the bound state the motor creates a force proportional to its displacement from the neutral position. In both states the motor spends an exponential time depending on the state. The thin filament moves at a velocity proportional to the average of all displacements of all motors. We assume that the time which a motor stays in the bound state does not depend on its displacement. Then one can find an exact solution of a nonlinear equation appearing in the limit of an infinite number of motors
Hsiu Chen, Chen; Wen, Fur-Hsing; Hou, Ming-Mo; Hsieh, Chia-Hsun; Chou, Wen-Chi; Chen, Jen-Shi; Chang, Wen-Cheng; Tang, Siew Tzuh
2017-09-01
Developing accurate prognostic awareness, a cornerstone of preference-based end-of-life (EOL) care decision-making, is a dynamic process involving more prognostic-awareness states than knowing or not knowing. Understanding the transition probabilities and time spent in each prognostic-awareness state can help clinicians identify trigger points for facilitating transitions toward accurate prognostic awareness. We examined transition probabilities in distinct prognostic-awareness states between consecutive time points in 247 cancer patients' last 6 months and estimated the time spent in each state. Prognostic awareness was categorized into four states: (a) unknown and not wanting to know, state 1; (b) unknown but wanting to know, state 2; (c) inaccurate awareness, state 3; and (d) accurate awareness, state 4. Transitional probabilities were examined by multistate Markov modeling. Initially, 59.5% of patients had accurate prognostic awareness, whereas the probabilities of being in states 1-3 were 8.1%, 17.4%, and 15.0%, respectively. Patients' prognostic awareness generally remained unchanged (probabilities of remaining in the same state: 45.5%-92.9%). If prognostic awareness changed, it tended to shift toward higher prognostic-awareness states (probabilities of shifting to state 4 were 23.2%-36.6% for patients initially in states 1-3, followed by probabilities of shifting to state 3 for those in states 1 and 2 [9.8%-10.1%]). Patients were estimated to spend 1.29, 0.42, 0.68, and 3.61 months in states 1-4, respectively, in their last 6 months. Terminally ill cancer patients' prognostic awareness generally remained unchanged, with a tendency to become more aware of their prognosis. Health care professionals should facilitate patients' transitions toward accurate prognostic awareness in a timely manner to promote preference-based EOL decisions. Terminally ill Taiwanese cancer patients' prognostic awareness generally remained stable, with a tendency toward developing
Bai, Qifeng; Pérez-Sánchez, Horacio; Zhang, Yang; Shao, Yonghua; Shi, Danfeng; Liu, Huanxiang; Yao, Xiaojun
2014-08-14
The reported crystal structures of β2 adrenergic receptor (β2AR) reveal that the open and closed states of the water channel are correlated with the inactive and active conformations of β2AR. However, more details about the process by which the water channel states are affected by the active to inactive conformational change of β2AR remain illusive. In this work, molecular dynamics simulations are performed to study the dynamical inactive and active conformational change of β2AR induced by inverse agonist ICI 118,551. Markov state model analysis and free energy calculation are employed to explore the open and close states of the water channel. The simulation results show that inverse agonist ICI 118,551 can induce water channel opening during the conformational transition of β2AR. Markov state model (MSM) analysis proves that the energy contour can be divided into seven states. States S1, S2 and S5, which represent the active conformation of β2AR, show that the water channel is in the closed state, while states S4 and S6, which correspond to the intermediate state conformation of β2AR, indicate the water channel opens gradually. State S7, which represents the inactive structure of β2AR, corresponds to the full open state of the water channel. The opening mechanism of the water channel is involved in the ligand-induced conformational change of β2AR. These results can provide useful information for understanding the opening mechanism of the water channel and will be useful for the rational design of potent inverse agonists of β2AR.
First hitting probabilities for semi markov chains and estimation
DEFF Research Database (Denmark)
Georgiadis, Stylianos
2017-01-01
We first consider a stochastic system described by an absorbing semi-Markov chain with finite state space and we introduce the absorption probability to a class of recurrent states. Afterwards, we study the first hitting probability to a subset of states for an irreducible semi-Markov chain...
A Markov decision model for optimising economic production lot size ...
African Journals Online (AJOL)
Adopting such a Markov decision process approach, the states of a Markov chain represent possible states of demand. The decision of whether or not to produce additional inventory units is made using dynamic programming. This approach demonstrates the existence of an optimal state-dependent EPL size, and produces ...
Deteksi Fraud Menggunakan Metode Model Markov Tersembunyi Pada Proses Bisnis
Directory of Open Access Journals (Sweden)
Andrean Hutama Koosasi
2017-03-01
Full Text Available Model Markov Tersembunyi merupakan sebuah metode statistik berdasarkan Model Markov sederhana yang memodelkan sistem serta membaginya dalam 2 (dua state, state tersembunyi dan state observasi. Dalam pengerjaan tugas akhir ini, penulis mengusulkan penggunaan metode Model Markov Tersembunyi untuk menemukan fraud didalam sebuah pelaksanaan proses bisnis. Dengan penggunaan metode Model Markov Tersembunyi ini, maka pengamatan terhadap elemen penyusun sebuah kasus/kejadian, yakni beberapa aktivitas, akan diperoleh sebuah nilai peluang, yang sekaligus memberikan prediksi terhadap kasus/kejadian tersebut, sebuah fraud atau tidak. Hasil ekpserimen ini menunjukkan bahwa metode yang diusulkan mampu memberikan prediksi akhir dengan evaluasi TPR sebesar 87,5% dan TNR sebesar 99,4%.
On Outage Performance of Spectrum-Sharing Communication over M-Block Fading
Alabbasi, AbdulRahman; Rezki, Zouheir; Shihada, Basem
2015-01-01
In this paper, we consider a cognitive radio system in which a block-fading channel is assumed. Each transmission frame consists of M blocks and each block undergoes a different channel gain. Instantaneous channel state information about
Markov Chain Ontology Analysis (MCOA).
Frost, H Robert; McCray, Alexa T
2012-02-03
Biomedical ontologies have become an increasingly critical lens through which researchers analyze the genomic, clinical and bibliographic data that fuels scientific research. Of particular relevance are methods, such as enrichment analysis, that quantify the importance of ontology classes relative to a collection of domain data. Current analytical techniques, however, remain limited in their ability to handle many important types of structural complexity encountered in real biological systems including class overlaps, continuously valued data, inter-instance relationships, non-hierarchical relationships between classes, semantic distance and sparse data. In this paper, we describe a methodology called Markov Chain Ontology Analysis (MCOA) and illustrate its use through a MCOA-based enrichment analysis application based on a generative model of gene activation. MCOA models the classes in an ontology, the instances from an associated dataset and all directional inter-class, class-to-instance and inter-instance relationships as a single finite ergodic Markov chain. The adjusted transition probability matrix for this Markov chain enables the calculation of eigenvector values that quantify the importance of each ontology class relative to other classes and the associated data set members. On both controlled Gene Ontology (GO) data sets created with Escherichia coli, Drosophila melanogaster and Homo sapiens annotations and real gene expression data extracted from the Gene Expression Omnibus (GEO), the MCOA enrichment analysis approach provides the best performance of comparable state-of-the-art methods. A methodology based on Markov chain models and network analytic metrics can help detect the relevant signal within large, highly interdependent and noisy data sets and, for applications such as enrichment analysis, has been shown to generate superior performance on both real and simulated data relative to existing state-of-the-art approaches.
Mdala, Ibrahimu; Olsen, Ingar; Haffajee, Anne D; Socransky, Sigmund S; Thoresen, Magne; de Blasio, Birgitte Freiesleben
2014-09-01
To understand degeneration of healthy sites and identify factors associated with disease progression in patients with chronic periodontitis. Data on healthy sites from 163 American and Swedish subjects were analysed using two-three-state (health, gingivitis, chronic periodontitis) Markov models based on bleeding on probing (BOP), and either clinical attachment level (CAL) + BOP or pocket depth (PD) + BOP. In 2 years, 10% (CAL + BOP) and 3% (PD + BOP) of healthy sites developed chronic periodontitis. On average, healthy sites remained healthy for 32 months before transiting in both models. Most transitions (87-97%) from health were to the gingivitis state. The expected duration of the gingivitis lesion was 4-5 months and sites recovered with a high probability (96-98%). Disease severity as measured by number of sites with CAL/PD > 4 mm at baseline and smoking, were associated with fast progression from health to chronic periodontitis within 6 months as were gingival redness in the PD + BOP model only. With age, the rate of disease progression to gingivitis decreased. Transition probabilities for gingivitis and chronic periodontitis were higher with CAL + BOP than with PD + BOP. Smoking and disease severity were significant predictors for fast progression. © 2014 The Authors. Journal of Clinical Periodontology Published by John Wiley & Sons Ltd.
Mdala, Ibrahimu; Olsen, Ingar; Haffajee, Anne D; Socransky, Sigmund S; Thoresen, Magne; de Blasio, Birgitte Freiesleben
2014-01-01
Aim To understand degeneration of healthy sites and identify factors associated with disease progression in patients with chronic periodontitis. Material and Methods Data on healthy sites from 163 American and Swedish subjects were analysed using two-three-state (health, gingivitis, chronic periodontitis) Markov models based on bleeding on probing (BOP), and either clinical attachment level (CAL) + BOP or pocket depth (PD) + BOP. Results In 2 years, 10% (CAL + BOP) and 3% (PD + BOP) of healthy sites developed chronic periodontitis. On average, healthy sites remained healthy for 32 months before transiting in both models. Most transitions (87–97%) from health were to the gingivitis state. The expected duration of the gingivitis lesion was 4–5 months and sites recovered with a high probability (96–98%). Disease severity as measured by number of sites with CAL/PD > 4 mm at baseline and smoking, were associated with fast progression from health to chronic periodontitis within 6 months as were gingival redness in the PD + BOP model only. With age, the rate of disease progression to gingivitis decreased. Conclusion Transition probabilities for gingivitis and chronic periodontitis were higher with CAL + BOP than with PD + BOP. Smoking and disease severity were significant predictors for fast progression. PMID:24888705
janssen, Anja; Segers, Johan
2013-01-01
The extremes of a univariate Markov chain with regularly varying stationary marginal distribution and asymptotically linear behavior are known to exhibit a multiplicative random walk structure called the tail chain. In this paper we extend this fact to Markov chains with multivariate regularly varying marginal distributions in Rd. We analyze both the forward and the backward tail process and show that they mutually determine each other through a kind of adjoint relation. In ...
Figueiredo, Danilo Zucolli; Costa, Oswaldo Luiz do Valle
2017-10-01
This paper deals with the H2 optimal control problem of discrete-time Markov jump linear systems (MJLS) considering the case in which the Markov chain takes values in a general Borel space ?. It is assumed that the controller has access only to an output variable and to the jump parameter. The goal, in this case, is to design a dynamic Markov jump controller such that the H2-norm of the closed-loop system is minimised. It is shown that the H2-norm can be written as the sum of two H2-norms, such that one of them does not depend on the control, and the other one is obtained from the optimal filter for an infinite-horizon filtering problem. This result can be seen as a separation principle for MJLS with Markov chain in a Borel space ? considering the infinite time horizon case.
Markov processes characterization and convergence
Ethier, Stewart N
2009-01-01
The Wiley-Interscience Paperback Series consists of selected books that have been made more accessible to consumers in an effort to increase global appeal and general circulation. With these new unabridged softcover volumes, Wiley hopes to extend the lives of these works by making them available to future generations of statisticians, mathematicians, and scientists."[A]nyone who works with Markov processes whose state space is uncountably infinite will need this most impressive book as a guide and reference."-American Scientist"There is no question but that space should immediately be reserved for [this] book on the library shelf. Those who aspire to mastery of the contents should also reserve a large number of long winter evenings."-Zentralblatt f?r Mathematik und ihre Grenzgebiete/Mathematics Abstracts"Ethier and Kurtz have produced an excellent treatment of the modern theory of Markov processes that [is] useful both as a reference work and as a graduate textbook."-Journal of Statistical PhysicsMarkov Proce...
When does fading enhance perceptual category learning?
Pashler, Harold; Mozer, Michael C
2013-07-01
Training that uses exaggerated versions of a stimulus discrimination (fading) has sometimes been found to enhance category learning, mostly in studies involving animals and impaired populations. However, little is known about whether and when fading facilitates learning for typical individuals. This issue was explored in 7 experiments. In Experiments 1 and 2, observers discriminated stimuli based on a single sensory continuum (time duration and line length, respectively). Adaptive fading dramatically improved performance in training (unsurprisingly) but did not enhance learning as assessed in a final test. The same was true for nonadaptive linear fading (Experiment 3). However, when variation in length (predicting category membership) was embedded among other (category-irrelevant) variation, fading dramatically enhanced not only performance in training but also learning as assessed in a final test (Experiments 4 and 5). Fading also helped learners to acquire a color saturation discrimination amid category-irrelevant variation in hue and brightness, although this learning proved transitory after feedback was withdrawn (Experiment 7). Theoretical implications are discussed, and we argue that fading should have practical utility in naturalistic category learning tasks, which involve extremely high dimensional stimuli and many irrelevant dimensions. PsycINFO Database Record (c) 2013 APA, all rights reserved.
Noise can speed convergence in Markov chains.
Franzke, Brandon; Kosko, Bart
2011-10-01
A new theorem shows that noise can speed convergence to equilibrium in discrete finite-state Markov chains. The noise applies to the state density and helps the Markov chain explore improbable regions of the state space. The theorem ensures that a stochastic-resonance noise benefit exists for states that obey a vector-norm inequality. Such noise leads to faster convergence because the noise reduces the norm components. A corollary shows that a noise benefit still occurs if the system states obey an alternate norm inequality. This leads to a noise-benefit algorithm that requires knowledge of the steady state. An alternative blind algorithm uses only past state information to achieve a weaker noise benefit. Simulations illustrate the predicted noise benefits in three well-known Markov models. The first model is a two-parameter Ehrenfest diffusion model that shows how noise benefits can occur in the class of birth-death processes. The second model is a Wright-Fisher model of genotype drift in population genetics. The third model is a chemical reaction network of zeolite crystallization. A fourth simulation shows a convergence rate increase of 64% for states that satisfy the theorem and an increase of 53% for states that satisfy the corollary. A final simulation shows that even suboptimal noise can speed convergence if the noise applies over successive time cycles. Noise benefits tend to be sharpest in Markov models that do not converge quickly and that do not have strong absorbing states.
Chen, Gong; Kong, Xian; Lu, Diannan; Wu, Jianzhong; Liu, Zheng
2017-05-10
Molecular dynamics (MD) simulations, in combination with the Markov-state model (MSM), were applied to probe CO 2 diffusion from an aqueous solution into the active site of human carbonic anhydrase II (hCA-II), an enzyme useful for enhanced CO 2 capture and utilization. The diffusion process in the hydrophobic pocket of hCA-II was illustrated in terms of a two-dimensional free-energy landscape. We found that CO 2 diffusion in hCA-II is a rate-limiting step in the CO 2 diffusion-binding-reaction process. The equilibrium distribution of CO 2 shows its preferential accumulation within a hydrophobic domain in the protein core region. An analysis of the committors and reactive fluxes indicates that the main pathway for CO 2 diffusion into the active site of hCA-II is through a binding pocket where residue Gln 136 contributes to the maximal flux. The simulation results offer a new perspective on the CO 2 hydration kinetics and useful insights toward the development of novel biochemical processes for more efficient CO 2 sequestration and utilization.
Paul, Fabian; Noé, Frank; Weikl, Thomas R
2018-03-27
Unstructured proteins and peptides typically fold during binding to ligand proteins. A challenging problem is to identify the mechanism and kinetics of these binding-induced folding processes in experiments and atomistic simulations. In this Article, we present a detailed picture for the folding of the inhibitor peptide PMI into a helix during binding to the oncoprotein fragment 25-109 Mdm2 obtained from atomistic, explicit-water simulations and Markov state modeling. We find that binding-induced folding of PMI is highly parallel and can occur along a multitude of pathways. Some pathways are induced-fit-like with binding occurring prior to PMI helix formation, while other pathways are conformational-selection-like with binding after helix formation. On the majority of pathways, however, binding is intricately coupled to folding, without clear temporal ordering. A central feature of these pathways is PMI motion on the Mdm2 surface, along the binding groove of Mdm2 or over the rim of this groove. The native binding groove of Mdm2 thus appears as an asymmetric funnel for PMI binding. Overall, binding-induced folding of PMI does not fit into the classical picture of induced fit or conformational selection that implies a clear temporal ordering of binding and folding events. We argue that this holds in general for binding-induced folding processes because binding and folding events in these processes likely occur on similar time scales and do exhibit the time-scale separation required for temporal ordering.
Global solutions to the equation of thermoelasticity with fading memory
Okada, Mari; Kawashima, Shuichi
2017-07-01
We consider the initial-history value problem for the one-dimensional equation of thermoelasticity with fading memory. It is proved that if the data are smooth and small, then a unique smooth solution exists globally in time and converges to the constant equilibrium state as time goes to infinity. Our proof is based on a technical energy method which makes use of the strict convexity of the entropy function and the properties of strongly positive definite kernels.
Model Checking Structured Infinite Markov Chains
Remke, Anne Katharina Ingrid
2008-01-01
In the past probabilistic model checking hast mostly been restricted to finite state models. This thesis explores the possibilities of model checking with continuous stochastic logic (CSL) on infinite-state Markov chains. We present an in-depth treatment of model checking algorithms for two special
Fading and shadowing in wireless systems
Shankar, P Mohana
2017-01-01
This book offers a comprehensive overview of fading and shadowing in wireless channels. A number of statistical models including simple, hybrid, compound and cascaded ones are presented along with a detailed discussion of diversity techniques employed to mitigate the effects of fading and shadowing. The effects of co-channel interference before and after the implementation of diversity are also analyzed. To facilitate easy understanding of the models and the analysis, the background on probability and random variables is presented with relevant derivations of densities of the sums, products, ratios as well as order statistics of random variables. The book also provides material on digital modems of interest in wireless systems. The updated edition expands the background materials on probability by offering sections on Laplace and Mellin transforms, parameter estimation, statistical testing and receiver operating characteristics. Newer models for fading, shadowing and shadowed fading are included along with th...
Capacity of Fading Channels in the Low Power Regime
Benkhelifa, Fatma
2013-01-01
The low power regime has attracted various researchers in the information theory and communication communities to understand the performance limits of wireless systems. Indeed, the energy consumption is becoming one of the major limiting factors in wireless systems. As such, energy-efficient wireless systems are of major importance to the next generation wireless systems designers. The capacity is a metric that measures the performance limit of a wireless system. The study of the ergodic capacity of some fading channels in the low power regime is the main subject of this thesis. In our study, we consider that the receiver has always a full knowledge of the channel state information. However, we assume that the transmitter has possibly imperfect knowledge of the channel state information, i.e. he knows either perfectly the channel or only an estimated version of the channel. Both radio frequency and free space optical communication channel models are considered. The main contribution of this work is the explicit characterization of how the capacity scales as function of the signal-to-noise ratio in the low power regime. This allows us to characterize the gain due to the perfect knowledge compared to no knowledge of the channel state information at the transmitter. In particular, we show that the gain increases logarithmically for radio frequency communication. However, the gain increases as log2(Pavg) or log4(Pavg) for free-space optical communication, where Pavg is the average power constraint imposed to the input. Furthermore, we characterize the capacity of cascaded fading channels and we applied the result to Rayleigh-product fading channel and to a free-space optical link over gamma-gamma atmospheric turbulence in the presence of pointing errors. Finally, we study the capacity of Nakagami-m fading channel under quality of service constraints, namely the effective capacity. We have shown that the effective capacity converges to Shannon capacity in the very low
Fading and interference mitigation in wireless communications
Panic, Stefan; Anastasov, Jelena; Spalevic, Petar
2013-01-01
The rapid advancement of various wireless communication system services has created the need to analyze the possibility of their performance improvement. Introducing the basic principles of digital communications performance analysis and its mathematical formalization, Fading and Interference Mitigation in Wireless Communications will help you stay up to date with recent developments in the performance analysis of space diversity reception over fading channels in the presence of cochannel interference.The book presents a unified method for computing the performance of digital communication sys
Consistent Estimation of Partition Markov Models
Directory of Open Access Journals (Sweden)
Jesús E. García
2017-04-01
Full Text Available The Partition Markov Model characterizes the process by a partition L of the state space, where the elements in each part of L share the same transition probability to an arbitrary element in the alphabet. This model aims to answer the following questions: what is the minimal number of parameters needed to specify a Markov chain and how to estimate these parameters. In order to answer these questions, we build a consistent strategy for model selection which consist of: giving a size n realization of the process, finding a model within the Partition Markov class, with a minimal number of parts to represent the process law. From the strategy, we derive a measure that establishes a metric in the state space. In addition, we show that if the law of the process is Markovian, then, eventually, when n goes to infinity, L will be retrieved. We show an application to model internet navigation patterns.
Hartfiel, Darald J
1998-01-01
In this study extending classical Markov chain theory to handle fluctuating transition matrices, the author develops a theory of Markov set-chains and provides numerous examples showing how that theory can be applied. Chapters are concluded with a discussion of related research. Readers who can benefit from this monograph are those interested in, or involved with, systems whose data is imprecise or that fluctuate with time. A background equivalent to a course in linear algebra and one in probability theory should be sufficient.
Model Checking Markov Reward Models with Impulse Rewards
Cloth, Lucia; Katoen, Joost-Pieter; Khattri, Maneesh; Pulungan, Reza; Bondavalli, Andrea; Haverkort, Boudewijn; Tang, Dong
This paper considers model checking of Markov reward models (MRMs), continuous-time Markov chains with state rewards as well as impulse rewards. The reward extension of the logic CSL (Continuous Stochastic Logic) is interpreted over such MRMs, and two numerical algorithms are provided to check the
ANALYTIC WORD RECOGNITION WITHOUT SEGMENTATION BASED ON MARKOV RANDOM FIELDS
Coisy, C.; Belaid, A.
2004-01-01
In this paper, a method for analytic handwritten word recognition based on causal Markov random fields is described. The words models are HMMs where each state corresponds to a letter; each letter is modelled by a NSHPHMM (Markov field). Global models are build dynamically, and used for recognition
Continuous-variable quantum key distribution in uniform fast-fading channels
Papanastasiou, Panagiotis; Weedbrook, Christian; Pirandola, Stefano
2018-03-01
We investigate the performance of several continuous-variable quantum key distribution protocols in the presence of uniform fading channels. These are lossy channels whose transmissivity changes according to a uniform probability distribution. We assume the worst-case scenario where an eavesdropper induces a fast-fading process, where she chooses the instantaneous transmissivity while the remote parties may only detect the mean statistical effect. We analyze coherent-state protocols in various configurations, including the one-way switching protocol in reverse reconciliation, the measurement-device-independent protocol in the symmetric configuration, and its extension to a three-party network. We show that, regardless of the advantage given to the eavesdropper (control of the fading), these protocols can still achieve high rates under realistic attacks, within reasonable values for the variance of the probability distribution associated with the fading process.
Process Algebra and Markov Chains
Brinksma, Hendrik; Hermanns, H.; Brinksma, Hendrik; Hermanns, H.; Katoen, Joost P.
This paper surveys and relates the basic concepts of process algebra and the modelling of continuous time Markov chains. It provides basic introductions to both fields, where we also study the Markov chains from an algebraic perspective, viz. that of Markov chain algebra. We then proceed to study
Process algebra and Markov chains
Brinksma, E.; Hermanns, H.; Brinksma, E.; Hermanns, H.; Katoen, J.P.
2001-01-01
This paper surveys and relates the basic concepts of process algebra and the modelling of continuous time Markov chains. It provides basic introductions to both fields, where we also study the Markov chains from an algebraic perspective, viz. that of Markov chain algebra. We then proceed to study
Indian Academy of Sciences (India)
be obtained as a limiting value of a sample path of a suitable ... makes a mathematical model of chance and deals with the problem by .... Is the Markov chain aperiodic? It is! Here is how you can see it. Suppose that after you do the cut, you hold the top half in your right hand, and the bottom half in your left. Then there.
Composable Markov Building Blocks
Evers, S.; Fokkinga, M.M.; Apers, Peter M.G.; Prade, H.; Subrahmanian, V.S.
2007-01-01
In situations where disjunct parts of the same process are described by their own first-order Markov models and only one model applies at a time (activity in one model coincides with non-activity in the other models), these models can be joined together into one. Under certain conditions, nearly all
Composable Markov Building Blocks
Evers, S.; Fokkinga, M.M.; Apers, Peter M.G.
2007-01-01
In situations where disjunct parts of the same process are described by their own first-order Markov models, these models can be joined together under the constraint that there can only be one activity at a time, i.e. the activities of one model coincide with non-activity in the other models. Under
Indian Academy of Sciences (India)
Home; Journals; Resonance – Journal of Science Education; Volume 7; Issue 3. Markov Chain Monte Carlo - Examples. Arnab Chakraborty. General Article Volume 7 Issue 3 March 2002 pp 25-34. Fulltext. Click here to view fulltext PDF. Permanent link: https://www.ias.ac.in/article/fulltext/reso/007/03/0025-0034. Keywords.
Classification of customer lifetime value models using Markov chain
Permana, Dony; Pasaribu, Udjianna S.; Indratno, Sapto W.; Suprayogi
2017-10-01
A firm’s potential reward in future time from a customer can be determined by customer lifetime value (CLV). There are some mathematic methods to calculate it. One method is using Markov chain stochastic model. Here, a customer is assumed through some states. Transition inter the states follow Markovian properties. If we are given some states for a customer and the relationships inter states, then we can make some Markov models to describe the properties of the customer. As Markov models, CLV is defined as a vector contains CLV for a customer in the first state. In this paper we make a classification of Markov Models to calculate CLV. Start from two states of customer model, we make develop in many states models. The development a model is based on weaknesses in previous model. Some last models can be expected to describe how real characters of customers in a firm.
Detecting Structural Breaks using Hidden Markov Models
DEFF Research Database (Denmark)
Ntantamis, Christos
Testing for structural breaks and identifying their location is essential for econometric modeling. In this paper, a Hidden Markov Model (HMM) approach is used in order to perform these tasks. Breaks are defined as the data points where the underlying Markov Chain switches from one state to another....... The estimation of the HMM is conducted using a variant of the Iterative Conditional Expectation-Generalized Mixture (ICE-GEMI) algorithm proposed by Delignon et al. (1997), that permits analysis of the conditional distributions of economic data and allows for different functional forms across regimes...
Coding with partially hidden Markov models
DEFF Research Database (Denmark)
Forchhammer, Søren; Rissanen, J.
1995-01-01
Partially hidden Markov models (PHMM) are introduced. They are a variation of the hidden Markov models (HMM) combining the power of explicit conditioning on past observations and the power of using hidden states. (P)HMM may be combined with arithmetic coding for lossless data compression. A general...... 2-part coding scheme for given model order but unknown parameters based on PHMM is presented. A forward-backward reestimation of parameters with a redefined backward variable is given for these models and used for estimating the unknown parameters. Proof of convergence of this reestimation is given....... The PHMM structure and the conditions of the convergence proof allows for application of the PHMM to image coding. Relations between the PHMM and hidden Markov models (HMM) are treated. Results of coding bi-level images with the PHMM coding scheme is given. The results indicate that the PHMM can adapt...
Markov and mixed models with applications
DEFF Research Database (Denmark)
Mortensen, Stig Bousgaard
This thesis deals with mathematical and statistical models with focus on applications in pharmacokinetic and pharmacodynamic (PK/PD) modelling. These models are today an important aspect of the drug development in the pharmaceutical industry and continued research in statistical methodology within...... or uncontrollable factors in an individual. Modelling using SDEs also provides new tools for estimation of unknown inputs to a system and is illustrated with an application to estimation of insulin secretion rates in diabetic patients. Models for the eect of a drug is a broader area since drugs may affect...... for non-parametric estimation of Markov processes are proposed to give a detailed description of the sleep process during the night. Statistically the Markov models considered for sleep states are closely related to the PK models based on SDEs as both models share the Markov property. When the models...
Observation of auroral fading before breakup
International Nuclear Information System (INIS)
Pellinen, R.J.; Heikkila, W.J.
1978-02-01
We have obtained detailed observations of the onset of auroral breakup using a variety of instruments with time resolution of some tens of seconds. Rapid sequences of all-sky photographs, and fast meridian scans by photometers, show that breakup is usually preceded by moderate brightening, followed by fading of the auroral brightness lasting one or two minutes, before the actual breakup itself. At the time of the fading there is a brief darkening of the poleward sky. Often the breakup is preceded by one or more rapid intensifications, each one preceded by local fading. Pseudo-breakups may also occur without the development of a major event. A bonafide breakup may begin on the fading arc, on an adjacent arc, or in an entirely new region nearby. This optical activity is closely correlated with the development of auroral radar echoes, suggesting that variations in the ionospheric and magnetospheric electric and magnetic fields are responsible for the observed auroral variations. Data from the IMS magnetometer network provide some indication of a correlated response by the local auroral and ionospheric current, although this could be partly due to changes in conductivity. Riometer recordings show a slow decrease in ionsperic radio wave absorption over a period of about ten minutes prior to breakup, with the largest decrease essentially to quiet-time values in the region of auroral fading and subsequent breakup. The implications of these observations regarding the trigger mechanism for the expansion phase of a magnetospheric substorm are discussed. (author)
Discounted Markov games : generalized policy iteration method
Wal, van der J.
1978-01-01
In this paper, we consider two-person zero-sum discounted Markov games with finite state and action spaces. We show that the Newton-Raphson or policy iteration method as presented by Pollats-chek and Avi-Itzhak does not necessarily converge, contradicting a proof of Rao, Chandrasekaran, and Nair.
Optimal dividend distribution under Markov regime switching
Jiang, Z.; Pistorius, M.
2012-01-01
We investigate the problem of optimal dividend distribution for a company in the presence of regime shifts. We consider a company whose cumulative net revenues evolve as a Brownian motion with positive drift that is modulated by a finite state Markov chain, and model the discount rate as a
Markov chains with quasitoeplitz transition matrix
Directory of Open Access Journals (Sweden)
Alexander M. Dukhovny
1989-01-01
Full Text Available This paper investigates a class of Markov chains which are frequently encountered in various applications (e.g. queueing systems, dams and inventories with feedback. Generating functions of transient and steady state probabilities are found by solving a special Riemann boundary value problem on the unit circle. A criterion of ergodicity is established.
Model Checking Markov Chains: Techniques and Tools
Zapreev, I.S.
2008-01-01
This dissertation deals with four important aspects of model checking Markov chains: the development of efficient model-checking tools, the improvement of model-checking algorithms, the efficiency of the state-space reduction techniques, and the development of simulation-based model-checking
Continuity Properties of Distances for Markov Processes
DEFF Research Database (Denmark)
Jaeger, Manfred; Mao, Hua; Larsen, Kim Guldstrand
2014-01-01
In this paper we investigate distance functions on finite state Markov processes that measure the behavioural similarity of non-bisimilar processes. We consider both probabilistic bisimilarity metrics, and trace-based distances derived from standard Lp and Kullback-Leibler distances. Two desirable...
International Nuclear Information System (INIS)
Rasheedy, M.S.; El-Sherif, M.A.; Hefni, M.A.
2008-01-01
Thermoluminescence solid-state detector is widely used to determine the dose in personnel and environmental monitoring for radiation protection purposes, for instance in the field of nuclear power production, medicine and research. However, thermal fading is a limiting factor for a long-term application, especially where temperature is changing significantly during the accumulation period. This paper studied the influence of temperature and duration of storage after irradiation on the thermal fading of the TL signal. Also, this paper discussed the dependence of the thermal fading on the trap parameters of TL glow peak. The most important parameters, which were considered here include the order of kinetics b, the depth of the trap level E (eV) and the frequency factor S (s-1). The dependence of the thermal fading on thermal stability parameters, namely trap depths and frequency factors for the glow peaks is discussed. The variation of the thermal fading as a function of the order of kinetics is demonstrated. In addition, this paper discussed the dependence of the thermal fading on the absorbed dose in case of first-, second- and general-order kinetics. The above-mentioned studies were arranged considering the models of first-, second- and general-order of kinetics
Quantum tomography, phase-space observables and generalized Markov kernels
International Nuclear Information System (INIS)
Pellonpaeae, Juha-Pekka
2009-01-01
We construct a generalized Markov kernel which transforms the observable associated with the homodyne tomography into a covariant phase-space observable with a regular kernel state. Illustrative examples are given in the cases of a 'Schroedinger cat' kernel state and the Cahill-Glauber s-parametrized distributions. Also we consider an example of a kernel state when the generalized Markov kernel cannot be constructed.
Markov and semi-Markov switching linear mixed models used to identify forest tree growth components.
Chaubert-Pereira, Florence; Guédon, Yann; Lavergne, Christian; Trottier, Catherine
2010-09-01
Tree growth is assumed to be mainly the result of three components: (i) an endogenous component assumed to be structured as a succession of roughly stationary phases separated by marked change points that are asynchronous among individuals, (ii) a time-varying environmental component assumed to take the form of synchronous fluctuations among individuals, and (iii) an individual component corresponding mainly to the local environment of each tree. To identify and characterize these three components, we propose to use semi-Markov switching linear mixed models, i.e., models that combine linear mixed models in a semi-Markovian manner. The underlying semi-Markov chain represents the succession of growth phases and their lengths (endogenous component) whereas the linear mixed models attached to each state of the underlying semi-Markov chain represent-in the corresponding growth phase-both the influence of time-varying climatic covariates (environmental component) as fixed effects, and interindividual heterogeneity (individual component) as random effects. In this article, we address the estimation of Markov and semi-Markov switching linear mixed models in a general framework. We propose a Monte Carlo expectation-maximization like algorithm whose iterations decompose into three steps: (i) sampling of state sequences given random effects, (ii) prediction of random effects given state sequences, and (iii) maximization. The proposed statistical modeling approach is illustrated by the analysis of successive annual shoots along Corsican pine trunks influenced by climatic covariates. © 2009, The International Biometric Society.
Ragain, Stephen; Ugander, Johan
2016-01-01
As datasets capturing human choices grow in richness and scale---particularly in online domains---there is an increasing need for choice models that escape traditional choice-theoretic axioms such as regularity, stochastic transitivity, and Luce's choice axiom. In this work we introduce the Pairwise Choice Markov Chain (PCMC) model of discrete choice, an inferentially tractable model that does not assume any of the above axioms while still satisfying the foundational axiom of uniform expansio...
Distinguishing Hidden Markov Chains
Kiefer, Stefan; Sistla, A. Prasad
2015-01-01
Hidden Markov Chains (HMCs) are commonly used mathematical models of probabilistic systems. They are employed in various fields such as speech recognition, signal processing, and biological sequence analysis. We consider the problem of distinguishing two given HMCs based on an observation sequence that one of the HMCs generates. More precisely, given two HMCs and an observation sequence, a distinguishing algorithm is expected to identify the HMC that generates the observation sequence. Two HM...
Fannes, Mark; Wouters, Jeroen
2012-01-01
We study a quantum process that can be considered as a quantum analogue for the classical Markov process. We specifically construct a version of these processes for free Fermions. For such free Fermionic processes we calculate the entropy density. This can be done either directly using Szeg\\"o's theorem for asymptotic densities of functions of Toeplitz matrices, or through an extension of said theorem to rates of functions, which we present in this article.
Pemodelan Markov Switching Autoregressive
Ariyani, Fiqria Devi; Warsito, Budi; Yasin, Hasbi
2014-01-01
Transition from depreciation to appreciation of exchange rate is one of regime switching that ignored by classic time series model, such as ARIMA, ARCH, or GARCH. Therefore, economic variables are modeled by Markov Switching Autoregressive (MSAR) which consider the regime switching. MLE is not applicable to parameters estimation because regime is an unobservable variable. So that filtering and smoothing process are applied to see the regime probabilities of observation. Using this model, tran...
Introduction to the numerical solutions of Markov chains
Stewart, Williams J
1994-01-01
A cornerstone of applied probability, Markov chains can be used to help model how plants grow, chemicals react, and atoms diffuse - and applications are increasingly being found in such areas as engineering, computer science, economics, and education. To apply the techniques to real problems, however, it is necessary to understand how Markov chains can be solved numerically. In this book, the first to offer a systematic and detailed treatment of the numerical solution of Markov chains, William Stewart provides scientists on many levels with the power to put this theory to use in the actual world, where it has applications in areas as diverse as engineering, economics, and education. His efforts make for essential reading in a rapidly growing field. Here, Stewart explores all aspects of numerically computing solutions of Markov chains, especially when the state is huge. He provides extensive background to both discrete-time and continuous-time Markov chains and examines many different numerical computing metho...
Approximate quantum Markov chains
Sutter, David
2018-01-01
This book is an introduction to quantum Markov chains and explains how this concept is connected to the question of how well a lost quantum mechanical system can be recovered from a correlated subsystem. To achieve this goal, we strengthen the data-processing inequality such that it reveals a statement about the reconstruction of lost information. The main difficulty in order to understand the behavior of quantum Markov chains arises from the fact that quantum mechanical operators do not commute in general. As a result we start by explaining two techniques of how to deal with non-commuting matrices: the spectral pinching method and complex interpolation theory. Once the reader is familiar with these techniques a novel inequality is presented that extends the celebrated Golden-Thompson inequality to arbitrarily many matrices. This inequality is the key ingredient in understanding approximate quantum Markov chains and it answers a question from matrix analysis that was open since 1973, i.e., if Lieb's triple ma...
A relation between non-Markov and Markov processes
International Nuclear Information System (INIS)
Hara, H.
1980-01-01
With the aid of a transformation technique, it is shown that some memory effects in the non-Markov processes can be eliminated. In other words, some non-Markov processes are rewritten in a form obtained by the random walk process; the Markov process. To this end, two model processes which have some memory or correlation in the random walk process are introduced. An explanation of the memory in the processes is given. (orig.)
Modeling cellular networks in fading environments with dominant specular components
AlAmmouri, Ahmad
2016-07-26
Stochastic geometry (SG) has been widely accepted as a fundamental tool for modeling and analyzing cellular networks. However, the fading models used with SG analysis are mainly confined to the simplistic Rayleigh fading, which is extended to the Nakagami-m fading in some special cases. However, neither the Rayleigh nor the Nakagami-m accounts for dominant specular components (DSCs) which may appear in realistic fading channels. In this paper, we present a tractable model for cellular networks with generalized two-ray (GTR) fading channel. The GTR fading explicitly accounts for two DSCs in addition to the diffuse components and offers high flexibility to capture diverse fading channels that appear in realistic outdoor/indoor wireless communication scenarios. It also encompasses the famous Rayleigh and Rician fading as special cases. To this end, the prominent effect of DSCs is highlighted in terms of average spectral efficiency. © 2016 IEEE.
Wireless Energy Harvesting Using Signals from Multiple Fading Channels
Chen, Yunfei; Zhao, Nan; Alouini, Mohamed-Slim
2017-01-01
fading or Gamma-shadowed Rician fading. The received signals are then harvested by using either a single harvester for simultaneous transmissions or multiple harvesters for transmissions at different frequencies, antennas or time slots. Both linear
Mallak, Saed
1996-01-01
Ankara : Department of Mathematics and Institute of Engineering and Sciences of Bilkent University, 1996. Thesis (Master's) -- Bilkent University, 1996. Includes bibliographical references leaves leaf 29 In thi.s work, we studierl the Ergodicilv of Non-Stationary .Markov chains. We gave several e.xainples with different cases. We proved that given a sec[uence of Markov chains such that the limit of this sec|uence is an Ergodic Markov chain, then the limit of the combination ...
International Nuclear Information System (INIS)
Hirschmann, H.
1983-06-01
The consequences of the basic assumptions of the semi-Markov process as defined from a homogeneous renewal process with a stationary Markov condition are reviewed. The notion of the semi-Markov process is generalized by its redefinition from a nonstationary Markov renewal process. For both the nongeneralized and the generalized case a representation of the first order conditional state probabilities is derived in terms of the transition probabilities of the Markov renewal process. Some useful calculation rules (regeneration rules) are derived for the conditional state probabilities of the semi-Markov process. Compared to the semi-Markov process in its usual definition the generalized process allows the analysis of a larger class of systems. For instance systems with arbitrarily distributed lifetimes of their components can be described. There is also a chance to describe systems which are modified during time by forces or manipulations from outside. (Auth.)
Ning, G.; Shum, P.
2007-06-01
We derive the expressions for the power fading including first-order polarization mode dispersion (PMD), chromatic dispersion, chirp parameter as well as polarization-dependent chromatic dispersion (PCD), which is dependent on the angle of precession of output state of polarization around the PMD vector. From the expression for radio frequency (RF) signals power fading, we get the average power fading for chromatic dispersion, chirp parameter, first-order PMD and PCD for both double sideband (DSB) modulation and single sideband (SSB) modulation. We also demonstrate a fast PMD and chromatic dispersion monitoring technology with reduced polarization-dependent gain. The measured results agree well with theoretical analysis.
Hidden Markov models applied to a subsequence of the Xylella fastidiosa genome
Directory of Open Access Journals (Sweden)
Silva Cibele Q. da
2003-01-01
Full Text Available Dependencies in DNA sequences are frequently modeled using Markov models. However, Markov chains cannot account for heterogeneity that may be present in different regions of the same DNA sequence. Hidden Markov models are more realistic than Markov models since they allow for the identification of heterogeneous regions of a DNA sequence. In this study we present an application of hidden Markov models to a subsequence of the Xylella fastidiosa DNA data. We found that a three-state model provides a good description for the data considered.
Volchenkov, Dima; Dawin, Jean René
A system for using dice to compose music randomly is known as the musical dice game. The discrete time MIDI models of 804 pieces of classical music written by 29 composers have been encoded into the transition matrices and studied by Markov chains. Contrary to human languages, entropy dominates over redundancy, in the musical dice games based on the compositions of classical music. The maximum complexity is achieved on the blocks consisting of just a few notes (8 notes, for the musical dice games generated over Bach's compositions). First passage times to notes can be used to resolve tonality and feature a composer.
Quantum Enhanced Inference in Markov Logic Networks.
Wittek, Peter; Gogolin, Christian
2017-04-19
Markov logic networks (MLNs) reconcile two opposing schools in machine learning and artificial intelligence: causal networks, which account for uncertainty extremely well, and first-order logic, which allows for formal deduction. An MLN is essentially a first-order logic template to generate Markov networks. Inference in MLNs is probabilistic and it is often performed by approximate methods such as Markov chain Monte Carlo (MCMC) Gibbs sampling. An MLN has many regular, symmetric structures that can be exploited at both first-order level and in the generated Markov network. We analyze the graph structures that are produced by various lifting methods and investigate the extent to which quantum protocols can be used to speed up Gibbs sampling with state preparation and measurement schemes. We review different such approaches, discuss their advantages, theoretical limitations, and their appeal to implementations. We find that a straightforward application of a recent result yields exponential speedup compared to classical heuristics in approximate probabilistic inference, thereby demonstrating another example where advanced quantum resources can potentially prove useful in machine learning.
Renewal characterization of Markov modulated Poisson processes
Directory of Open Access Journals (Sweden)
Marcel F. Neuts
1989-01-01
Full Text Available A Markov Modulated Poisson Process (MMPP M(t defined on a Markov chain J(t is a pure jump process where jumps of M(t occur according to a Poisson process with intensity λi whenever the Markov chain J(t is in state i. M(t is called strongly renewal (SR if M(t is a renewal process for an arbitrary initial probability vector of J(t with full support on P={i:λi>0}. M(t is called weakly renewal (WR if there exists an initial probability vector of J(t such that the resulting MMPP is a renewal process. The purpose of this paper is to develop general characterization theorems for the class SR and some sufficiency theorems for the class WR in terms of the first passage times of the bivariate Markov chain [J(t,M(t]. Relevance to the lumpability of J(t is also studied.
Quantum Enhanced Inference in Markov Logic Networks
Wittek, Peter; Gogolin, Christian
2017-04-01
Markov logic networks (MLNs) reconcile two opposing schools in machine learning and artificial intelligence: causal networks, which account for uncertainty extremely well, and first-order logic, which allows for formal deduction. An MLN is essentially a first-order logic template to generate Markov networks. Inference in MLNs is probabilistic and it is often performed by approximate methods such as Markov chain Monte Carlo (MCMC) Gibbs sampling. An MLN has many regular, symmetric structures that can be exploited at both first-order level and in the generated Markov network. We analyze the graph structures that are produced by various lifting methods and investigate the extent to which quantum protocols can be used to speed up Gibbs sampling with state preparation and measurement schemes. We review different such approaches, discuss their advantages, theoretical limitations, and their appeal to implementations. We find that a straightforward application of a recent result yields exponential speedup compared to classical heuristics in approximate probabilistic inference, thereby demonstrating another example where advanced quantum resources can potentially prove useful in machine learning.
Sampling rare fluctuations of discrete-time Markov chains
Whitelam, Stephen
2018-03-01
We describe a simple method that can be used to sample the rare fluctuations of discrete-time Markov chains. We focus on the case of Markov chains with well-defined steady-state measures, and derive expressions for the large-deviation rate functions (and upper bounds on such functions) for dynamical quantities extensive in the length of the Markov chain. We illustrate the method using a series of simple examples, and use it to study the fluctuations of a lattice-based model of active matter that can undergo motility-induced phase separation.
Markov Chain Modelling for Short-Term NDVI Time Series Forecasting
Directory of Open Access Journals (Sweden)
Stepčenko Artūrs
2016-12-01
Full Text Available In this paper, the NDVI time series forecasting model has been developed based on the use of discrete time, continuous state Markov chain of suitable order. The normalised difference vegetation index (NDVI is an indicator that describes the amount of chlorophyll (the green mass and shows the relative density and health of vegetation; therefore, it is an important variable for vegetation forecasting. A Markov chain is a stochastic process that consists of a state space. This stochastic process undergoes transitions from one state to another in the state space with some probabilities. A Markov chain forecast model is flexible in accommodating various forecast assumptions and structures. The present paper discusses the considerations and techniques in building a Markov chain forecast model at each step. Continuous state Markov chain model is analytically described. Finally, the application of the proposed Markov chain model is illustrated with reference to a set of NDVI time series data.
International Nuclear Information System (INIS)
Tao, Laifa; Cheng, Yujie; Lu, Chen; Su, Yuzhuan; Chong, Jin; Jin, Haizu; Lin, Yongshou; Noktehdan, Azadeh
2017-01-01
Highlights: •The model is linked to known physicochemical degradation processes and material properties. •Aging dynamics of various battery formulations can be understood by the proposed model. •Large number of experiments will be reduced to accelerate the battery design process. •This approach can describe batteries under various operating conditions. •The proposed model is simple and easily implemented. -- Abstract: A five-state nonhomogeneous Markov chain model, which is an effective and promising way to accelerate the Li-ion battery design process by investigating the capacity fading dynamics of different formulations during the battery design phase, is reported. The parameters of this model are linked to known physicochemical degradation dynamics and material properties. Herein, the states and behaviors of the active materials in Li-ion batteries are modelled. To verify the efficiency of the proposed model, a dataset from approximately 3 years of cycling capacity fading experiments of various formulations using several different materials provided by Contemporary Amperex Technology Limited (CATL), as well as a NASA dataset, are employed. The capabilities of the proposed model for different amounts (50%, 70%, and 90%) of available experimental capacity data are tested and analyzed to assist with the final design determination for manufacturers. The average relative errors of life cycling prediction acquired from these tests are less than 2.4%, 0.8%, and 0.3%, even when only 50%, 70%, and 90% of the data, respectively, is available for different anode materials, electrolyte materials, and individual batteries. Furthermore, the variance is 0.518% when only 50% of the data are available; i.e., one can save at least 50% of the total experimental time and cost with an accuracy greater than 97% in the design phase, which demonstrates an effective and promising way to accelerate the Li-ion battery design process. The qualitative and quantitative analyses
Nonlinear Markov processes: Deterministic case
International Nuclear Information System (INIS)
Frank, T.D.
2008-01-01
Deterministic Markov processes that exhibit nonlinear transition mechanisms for probability densities are studied. In this context, the following issues are addressed: Markov property, conditional probability densities, propagation of probability densities, multistability in terms of multiple stationary distributions, stability analysis of stationary distributions, and basin of attraction of stationary distribution
Monocular depth effects on perceptual fading.
Hsu, Li-Chuan; Kramer, Peter; Yeh, Su-Ling
2010-08-06
After prolonged viewing, a static target among moving non-targets is perceived to repeatedly disappear and reappear. An uncrossed stereoscopic disparity of the target facilitates this Motion-Induced Blindness (MIB). Here we test whether monocular depth cues can affect MIB too, and whether they can also affect perceptual fading in static displays. Experiment 1 reveals an effect of interposition: more MIB when the target appears partially covered by, than when it appears to cover, its surroundings. Experiment 2 shows that the effect is indeed due to interposition and not to the target's contours. Experiment 3 induces depth with the watercolor illusion and replicates Experiment 1. Experiments 4 and 5 replicate Experiments 1 and 3 without the use of motion. Since almost any stimulus contains a monocular depth cue, we conclude that perceived depth affects perceptual fading in almost any stimulus, whether dynamic or static. Copyright 2010 Elsevier Ltd. All rights reserved.
Fading and Shadowing in Wireless Systems
Shankar, P Mohana
2012-01-01
In recent decades, growth in the field of wireless communications has led to an exponential rise in the number of journals catering to the research community. Still unmet, however, is the need to fully and comprehensively understand the effects of channel degradation brought on by the statistical fluctuations in the channel. These fluctuations mainly manifest as variations in signal power observed in the channel generally modeled using a variety of probability distributions, both in straight forms as well as in compound forms. While the former might explain some of the effects, it is the latter, namely, the compound models, which incorporate both short term and long term power fluctuations in the channel, explain the much more complex nature of the signals in these channels. Fading and Shadowing in Wireless Systems offers a pedagogical approach to the topic, with insight into the modeling and analysis of fading and shadowing. Beginning with statistical background and digital communications, the book is formul...
Balancing Long Lifetime and Satisfying Fairness in WBAN Using a Constrained Markov Decision Process
Directory of Open Access Journals (Sweden)
Yingqi Yin
2015-01-01
Full Text Available As an important part of the Internet of Things (IOT and the special case of device-to-device (D2D communication, wireless body area network (WBAN gradually becomes the focus of attention. Since WBAN is a body-centered network, the energy of sensor nodes is strictly restrained since they are supplied by battery with limited power. In each data collection, only one sensor node is scheduled to transmit its measurements directly to the access point (AP through the fading channel. We formulate the problem of dynamically choosing which sensor should communicate with the AP to maximize network lifetime under the constraint of fairness as a constrained markov decision process (CMDP. The optimal lifetime and optimal policy are obtained by Bellman equation in dynamic programming. The proposed algorithm defines the limiting performance in WBAN lifetime under different degrees of fairness constraints. Due to the defect of large implementation overhead in acquiring global channel state information (CSI, we put forward a distributed scheduling algorithm that adopts local CSI, which saves the network overhead and simplifies the algorithm. It was demonstrated via simulation that this scheduling algorithm can allocate time slot reasonably under different channel conditions to balance the performances of network lifetime and fairness.
An interlacing theorem for reversible Markov chains
International Nuclear Information System (INIS)
Grone, Robert; Salamon, Peter; Hoffmann, Karl Heinz
2008-01-01
Reversible Markov chains are an indispensable tool in the modeling of a vast class of physical, chemical, biological and statistical problems. Examples include the master equation descriptions of relaxing physical systems, stochastic optimization algorithms such as simulated annealing, chemical dynamics of protein folding and Markov chain Monte Carlo statistical estimation. Very often the large size of the state spaces requires the coarse graining or lumping of microstates into fewer mesoscopic states, and a question of utmost importance for the validity of the physical model is how the eigenvalues of the corresponding stochastic matrix change under this operation. In this paper we prove an interlacing theorem which gives explicit bounds on the eigenvalues of the lumped stochastic matrix. (fast track communication)
An interlacing theorem for reversible Markov chains
Energy Technology Data Exchange (ETDEWEB)
Grone, Robert; Salamon, Peter [Department of Mathematics and Statistics, San Diego State University, San Diego, CA 92182-7720 (United States); Hoffmann, Karl Heinz [Institut fuer Physik, Technische Universitaet Chemnitz, D-09107 Chemnitz (Germany)
2008-05-30
Reversible Markov chains are an indispensable tool in the modeling of a vast class of physical, chemical, biological and statistical problems. Examples include the master equation descriptions of relaxing physical systems, stochastic optimization algorithms such as simulated annealing, chemical dynamics of protein folding and Markov chain Monte Carlo statistical estimation. Very often the large size of the state spaces requires the coarse graining or lumping of microstates into fewer mesoscopic states, and a question of utmost importance for the validity of the physical model is how the eigenvalues of the corresponding stochastic matrix change under this operation. In this paper we prove an interlacing theorem which gives explicit bounds on the eigenvalues of the lumped stochastic matrix. (fast track communication)
Stochastic Dynamics through Hierarchically Embedded Markov Chains.
Vasconcelos, Vítor V; Santos, Fernando P; Santos, Francisco C; Pacheco, Jorge M
2017-02-03
Studying dynamical phenomena in finite populations often involves Markov processes of significant mathematical and/or computational complexity, which rapidly becomes prohibitive with increasing population size or an increasing number of individual configuration states. Here, we develop a framework that allows us to define a hierarchy of approximations to the stationary distribution of general systems that can be described as discrete Markov processes with time invariant transition probabilities and (possibly) a large number of states. This results in an efficient method for studying social and biological communities in the presence of stochastic effects-such as mutations in evolutionary dynamics and a random exploration of choices in social systems-including situations where the dynamics encompasses the existence of stable polymorphic configurations, thus overcoming the limitations of existing methods. The present formalism is shown to be general in scope, widely applicable, and of relevance to a variety of interdisciplinary problems.
Modeling Uncertainty of Directed Movement via Markov Chains
Directory of Open Access Journals (Sweden)
YIN Zhangcai
2015-10-01
Full Text Available Probabilistic time geography (PTG is suggested as an extension of (classical time geography, in order to present the uncertainty of an agent located at the accessible position by probability. This may provide a quantitative basis for most likely finding an agent at a location. In recent years, PTG based on normal distribution or Brown bridge has been proposed, its variance, however, is irrelevant with the agent's speed or divergent with the increase of the speed; so they are difficult to take into account application pertinence and stability. In this paper, a new method is proposed to model PTG based on Markov chain. Firstly, a bidirectional conditions Markov chain is modeled, the limit of which, when the moving speed is large enough, can be regarded as the Brown bridge, thus has the characteristics of digital stability. Then, the directed movement is mapped to Markov chains. The essential part is to build step length, the state space and transfer matrix of Markov chain according to the space and time position of directional movement, movement speed information, to make sure the Markov chain related to the movement speed. Finally, calculating continuously the probability distribution of the directed movement at any time by the Markov chains, it can be get the possibility of an agent located at the accessible position. Experimental results show that, the variance based on Markov chains not only is related to speed, but also is tending towards stability with increasing the agent's maximum speed.
Modelling of cyclical stratigraphy using Markov chains
Energy Technology Data Exchange (ETDEWEB)
Kulatilake, P.H.S.W.
1987-07-01
State-of-the-art on modelling of cyclical stratigraphy using first-order Markov chains is reviewed. Shortcomings of the presently available procedures are identified. A procedure which eliminates all the identified shortcomings is presented. Required statistical tests to perform this modelling are given in detail. An example (the Oficina formation in eastern Venezuela) is given to illustrate the presented procedure. 12 refs., 3 tabs. 1 fig.
Markov Chain Models for the Stochastic Modeling of Pitting Corrosion
Valor, A.; Caleyo, F.; Alfonso, L.; Velázquez, J. C.; Hallen, J. M.
2013-01-01
The stochastic nature of pitting corrosion of metallic structures has been widely recognized. It is assumed that this kind of deterioration retains no memory of the past, so only the current state of the damage influences its future development. This characteristic allows pitting corrosion to be categorized as a Markov process. In this paper, two different models of pitting corrosion, developed using Markov chains, are presented. Firstly, a continuous-time, nonhomogeneous linear growth (pure ...
Multiple-Symbol Decision-Feedback Space-Time Differential Decoding in Fading Channels
Directory of Open Access Journals (Sweden)
Wang Xiaodong
2002-01-01
Full Text Available Space-time differential coding (STDC is an effective technique for exploiting transmitter diversity while it does not require the channel state information at the receiver. However, like conventional differential modulation schemes, it exhibits an error floor in fading channels. In this paper, we develop an STDC decoding technique based on multiple-symbol detection and decision-feedback, which makes use of the second-order statistic of the fading processes and has a very low computational complexity. This decoding method can significantly lower the error floor of the conventional STDC decoding algorithm, especially in fast fading channels. The application of the proposed multiple-symbol decision-feedback STDC decoding technique in orthogonal frequency-division multiplexing (OFDM system is also discussed.
Space-Time Trellis Coded 8PSK Schemes for Rapid Rayleigh Fading Channels
Directory of Open Access Journals (Sweden)
Salam A. Zummo
2002-05-01
Full Text Available This paper presents the design of 8PSK space-time (ST trellis codes suitable for rapid fading channels. The proposed codes utilize the design criteria of ST codes over rapid fading channels. Two different approaches have been used. The first approach maximizes the symbol-wise Hamming distance (HD between signals leaving from or entering to the same encoderÃ¢Â€Â²s state. In the second approach, set partitioning based on maximizing the sum of squared Euclidean distances (SSED between the ST signals is performed; then, the branch-wise HD is maximized. The proposed codes were simulated over independent and correlated Rayleigh fading channels. Coding gains up to 4 dB have been observed over other ST trellis codes of the same complexity.
Outage performance of reactive cooperation in Nakagami-m fading channels
Benjillali, Mustapha
2010-06-01
In this paper, we investigate the outage performance of Decode-and-Forward with reactive relaying in dual-hop cooperetive Nakagaml-m fading links. The destination, based on the umque knowledge of local second hop channel state information, selects the best relay to increase the chances of cooperation when the direct link is also available. After deriving the exact distribution of the variables of interest, the outage probability of the system - with and without the direct link - is obtained in closed-form, and the ε-outage capacity is derived in the particular c.se wh.ere the channel model is reduced to a Rayleigh fading. Simulation results confirm the accuracy of our analysis for a large selection of system and fading parameters.
Outage performance of Decode-and-Forward partial selection in Nakagami-m fading channels
Benjillali, Mustapha
2010-01-01
In this paper, we investigate the outage performance of Decode-and-Forward with partial selection relaying in dualhop cooperative Nakagami-m fading links. The source, based on the unique knowledge of local first hop channel state information, selects the best relay to increase the chances of successful decoding and hence the possibility of cooperation when the direct link is also available. After deriving the exact distribution of the sum of two gamma variates with the same shape parameter, the outage probability of the system-with and without the direct link-is obtained in closed-form. We also derive the ε-outage capacity in different particular cases, and the obtained results- when the channel model is reduced to a Rayleigh fading-are either new or correspond to those previously obtained in other works. Simulation results confirm the accuracy of our analysis for a large selection of system and fading parameters. © 2009 IEEE.
Sebastian, Tunny; Jeyaseelan, Visalakshi; Jeyaseelan, Lakshmanan; Anandan, Shalini; George, Sebastian; Bangdiwala, Shrikant I
2018-01-01
Hidden Markov models are stochastic models in which the observations are assumed to follow a mixture distribution, but the parameters of the components are governed by a Markov chain which is unobservable. The issues related to the estimation of Poisson-hidden Markov models in which the observations are coming from mixture of Poisson distributions and the parameters of the component Poisson distributions are governed by an m-state Markov chain with an unknown transition probability matrix are explained here. These methods were applied to the data on Vibrio cholerae counts reported every month for 11-year span at Christian Medical College, Vellore, India. Using Viterbi algorithm, the best estimate of the state sequence was obtained and hence the transition probability matrix. The mean passage time between the states were estimated. The 95% confidence interval for the mean passage time was estimated via Monte Carlo simulation. The three hidden states of the estimated Markov chain are labelled as 'Low', 'Moderate' and 'High' with the mean counts of 1.4, 6.6 and 20.2 and the estimated average duration of stay of 3, 3 and 4 months, respectively. Environmental risk factors were studied using Markov ordinal logistic regression analysis. No significant association was found between disease severity levels and climate components.
Prognostics for Steam Generator Tube Rupture using Markov Chain model
International Nuclear Information System (INIS)
Kim, Gibeom; Heo, Gyunyoung; Kim, Hyeonmin
2016-01-01
This paper will describe the prognostics method for evaluating and forecasting the ageing effect and demonstrate the procedure of prognostics for the Steam Generator Tube Rupture (SGTR) accident. Authors will propose the data-driven method so called MCMC (Markov Chain Monte Carlo) which is preferred to the physical-model method in terms of flexibility and availability. Degradation data is represented as growth of burst probability over time. Markov chain model is performed based on transition probability of state. And the state must be discrete variable. Therefore, burst probability that is continuous variable have to be changed into discrete variable to apply Markov chain model to the degradation data. The Markov chain model which is one of prognostics methods was described and the pilot demonstration for a SGTR accident was performed as a case study. The Markov chain model is strong since it is possible to be performed without physical models as long as enough data are available. However, in the case of the discrete Markov chain used in this study, there must be loss of information while the given data is discretized and assigned to the finite number of states. In this process, original information might not be reflected on prediction sufficiently. This should be noted as the limitation of discrete models. Now we will be studying on other prognostics methods such as GPM (General Path Model) which is also data-driven method as well as the particle filer which belongs to physical-model method and conducting comparison analysis
Probability distributions for Markov chain based quantum walks
Balu, Radhakrishnan; Liu, Chaobin; Venegas-Andraca, Salvador E.
2018-01-01
We analyze the probability distributions of the quantum walks induced from Markov chains by Szegedy (2004). The first part of this paper is devoted to the quantum walks induced from finite state Markov chains. It is shown that the probability distribution on the states of the underlying Markov chain is always convergent in the Cesaro sense. In particular, we deduce that the limiting distribution is uniform if the transition matrix is symmetric. In the case of a non-symmetric Markov chain, we exemplify that the limiting distribution of the quantum walk is not necessarily identical with the stationary distribution of the underlying irreducible Markov chain. The Szegedy scheme can be extended to infinite state Markov chains (random walks). In the second part, we formulate the quantum walk induced from a lazy random walk on the line. We then obtain the weak limit of the quantum walk. It is noted that the current quantum walk appears to spread faster than its counterpart-quantum walk on the line driven by the Grover coin discussed in literature. The paper closes with an outlook on possible future directions.
Outage performance of reactive cooperation in Nakagami-m fading channels
Benjillali, Mustapha; Alouini, Mohamed-Slim
2010-01-01
In this paper, we investigate the outage performance of Decode-and-Forward with reactive relaying in dual-hop cooperetive Nakagaml-m fading links. The destination, based on the umque knowledge of local second hop channel state information, selects
Regeneration and general Markov chains
Directory of Open Access Journals (Sweden)
Vladimir V. Kalashnikov
1994-01-01
Full Text Available Ergodicity, continuity, finite approximations and rare visits of general Markov chains are investigated. The obtained results permit further quantitative analysis of characteristics, such as, rates of convergence, continuity (measured as a distance between perturbed and non-perturbed characteristics, deviations between Markov chains, accuracy of approximations and bounds on the distribution function of the first visit time to a chosen subset, etc. The underlying techniques use the embedding of the general Markov chain into a wide sense regenerative process with the help of splitting construction.
Markov chains theory and applications
Sericola, Bruno
2013-01-01
Markov chains are a fundamental class of stochastic processes. They are widely used to solve problems in a large number of domains such as operational research, computer science, communication networks and manufacturing systems. The success of Markov chains is mainly due to their simplicity of use, the large number of available theoretical results and the quality of algorithms developed for the numerical evaluation of many metrics of interest.The author presents the theory of both discrete-time and continuous-time homogeneous Markov chains. He carefully examines the explosion phenomenon, the
Quadratic Variation by Markov Chains
DEFF Research Database (Denmark)
Hansen, Peter Reinhard; Horel, Guillaume
We introduce a novel estimator of the quadratic variation that is based on the the- ory of Markov chains. The estimator is motivated by some general results concerning filtering contaminated semimartingales. Specifically, we show that filtering can in prin- ciple remove the effects of market...... microstructure noise in a general framework where little is assumed about the noise. For the practical implementation, we adopt the dis- crete Markov chain model that is well suited for the analysis of financial high-frequency prices. The Markov chain framework facilitates simple expressions and elegant analyti...
Modeling nonhomogeneous Markov processes via time transformation.
Hubbard, R A; Inoue, L Y T; Fann, J R
2008-09-01
Longitudinal studies are a powerful tool for characterizing the course of chronic disease. These studies are usually carried out with subjects observed at periodic visits giving rise to panel data. Under this observation scheme the exact times of disease state transitions and sequence of disease states visited are unknown and Markov process models are often used to describe disease progression. Most applications of Markov process models rely on the assumption of time homogeneity, that is, that the transition rates are constant over time. This assumption is not satisfied when transition rates depend on time from the process origin. However, limited statistical tools are available for dealing with nonhomogeneity. We propose models in which the time scale of a nonhomogeneous Markov process is transformed to an operational time scale on which the process is homogeneous. We develop a method for jointly estimating the time transformation and the transition intensity matrix for the time transformed homogeneous process. We assess maximum likelihood estimation using the Fisher scoring algorithm via simulation studies and compare performance of our method to homogeneous and piecewise homogeneous models. We apply our methodology to a study of delirium progression in a cohort of stem cell transplantation recipients and show that our method identifies temporal trends in delirium incidence and recovery.
Constructing Dynamic Event Trees from Markov Models
International Nuclear Information System (INIS)
Paolo Bucci; Jason Kirschenbaum; Tunc Aldemir; Curtis Smith; Ted Wood
2006-01-01
In the probabilistic risk assessment (PRA) of process plants, Markov models can be used to model accurately the complex dynamic interactions between plant physical process variables (e.g., temperature, pressure, etc.) and the instrumentation and control system that monitors and manages the process. One limitation of this approach that has prevented its use in nuclear power plant PRAs is the difficulty of integrating the results of a Markov analysis into an existing PRA. In this paper, we explore a new approach to the generation of failure scenarios and their compilation into dynamic event trees from a Markov model of the system. These event trees can be integrated into an existing PRA using software tools such as SAPHIRE. To implement our approach, we first construct a discrete-time Markov chain modeling the system of interest by: (a) partitioning the process variable state space into magnitude intervals (cells), (b) using analytical equations or a system simulator to determine the transition probabilities between the cells through the cell-to-cell mapping technique, and, (c) using given failure/repair data for all the components of interest. The Markov transition matrix thus generated can be thought of as a process model describing the stochastic dynamic behavior of the finite-state system. We can therefore search the state space starting from a set of initial states to explore all possible paths to failure (scenarios) with associated probabilities. We can also construct event trees of arbitrary depth by tracing paths from a chosen initiating event and recording the following events while keeping track of the probabilities associated with each branch in the tree. As an example of our approach, we use the simple level control system often used as benchmark in the literature with one process variable (liquid level in a tank), and three control units: a drain unit and two supply units. Each unit includes a separate level sensor to observe the liquid level in the tank
Markov Chain Monte Carlo Methods
Indian Academy of Sciences (India)
Systat Software Asia-Pacific. Ltd., in Bangalore, where the technical work for the development of the statistical software Systat takes ... In Part 4, we discuss some applications of the Markov ... one can construct the joint probability distribution of.
Reviving Markov processes and applications
International Nuclear Information System (INIS)
Cai, H.
1988-01-01
In this dissertation we study a procedure which restarts a Markov process when the process is killed by some arbitrary multiplicative functional. The regenerative nature of this revival procedure is characterized through a Markov renewal equation. An interesting duality between the revival procedure and the classical killing operation is found. Under the condition that the multiplicative functional possesses an intensity, the generators of the revival process can be written down explicitly. An intimate connection is also found between the perturbation of the sample path of a Markov process and the perturbation of a generator (in Kato's sense). The applications of the theory include the study of the processes like piecewise-deterministic Markov process, virtual waiting time process and the first entrance decomposition (taboo probability)
Power plant reliability calculation with Markov chain models
International Nuclear Information System (INIS)
Senegacnik, A.; Tuma, M.
1998-01-01
In the paper power plant operation is modelled using continuous time Markov chains with discrete state space. The model is used to compute the power plant reliability and the importance and influence of individual states, as well as the transition probabilities between states. For comparison the model is fitted to data for coal and nuclear power plants recorded over several years. (orig.) [de
Computing continuous-time Markov chains as transformers of unbounded observables
DEFF Research Database (Denmark)
Danos, Vincent; Heindel, Tobias; Garnier, Ilias
2017-01-01
The paper studies continuous-time Markov chains (CTMCs) as transformers of real-valued functions on their state space, considered as generalised predicates and called observables. Markov chains are assumed to take values in a countable state space S; observables f: S → ℝ may be unbounded...
Tornadoes and related damage costs: statistical modeling with a semi-Markov approach
Corini, Chiara; D'Amico, Guglielmo; Petroni, Filippo; Prattico, Flavio; Manca, Raimondo
2015-01-01
We propose a statistical approach to tornadoes modeling for predicting and simulating occurrences of tornadoes and accumulated cost distributions over a time interval. This is achieved by modeling the tornadoes intensity, measured with the Fujita scale, as a stochastic process. Since the Fujita scale divides tornadoes intensity into six states, it is possible to model the tornadoes intensity by using Markov and semi-Markov models. We demonstrate that the semi-Markov approach is able to reprod...
Rate estimation in partially observed Markov jump processes with measurement errors
Amrein, Michael; Kuensch, Hans R.
2010-01-01
We present a simulation methodology for Bayesian estimation of rate parameters in Markov jump processes arising for example in stochastic kinetic models. To handle the problem of missing components and measurement errors in observed data, we embed the Markov jump process into the framework of a general state space model. We do not use diffusion approximations. Markov chain Monte Carlo and particle filter type algorithms are introduced, which allow sampling from the posterior distribution of t...
Decoding LDPC Convolutional Codes on Markov Channels
Directory of Open Access Journals (Sweden)
Kashyap Manohar
2008-01-01
Full Text Available Abstract This paper describes a pipelined iterative technique for joint decoding and channel state estimation of LDPC convolutional codes over Markov channels. Example designs are presented for the Gilbert-Elliott discrete channel model. We also compare the performance and complexity of our algorithm against joint decoding and state estimation of conventional LDPC block codes. Complexity analysis reveals that our pipelined algorithm reduces the number of operations per time step compared to LDPC block codes, at the expense of increased memory and latency. This tradeoff is favorable for low-power applications.
Decoding LDPC Convolutional Codes on Markov Channels
Directory of Open Access Journals (Sweden)
Chris Winstead
2008-04-01
Full Text Available This paper describes a pipelined iterative technique for joint decoding and channel state estimation of LDPC convolutional codes over Markov channels. Example designs are presented for the Gilbert-Elliott discrete channel model. We also compare the performance and complexity of our algorithm against joint decoding and state estimation of conventional LDPC block codes. Complexity analysis reveals that our pipelined algorithm reduces the number of operations per time step compared to LDPC block codes, at the expense of increased memory and latency. This tradeoff is favorable for low-power applications.
Asymptotic analysis of downlink MISO systems over Rician fading channels
Falconet, Hugo
2016-06-24
In this work, we focus on the ergodic sum rate in the downlink of a single-cell large-scale multi-user MIMO system in which the base station employs N antennas to communicate with K single-antenna user equipments. A regularized zero-forcing (RZF) scheme is used for precoding under the assumption that each link forms a spatially correlated MIMO Rician fading channel. The analysis is conducted assuming N and K grow large with a non trivial ratio and perfect channel state information is available at the base station. Recent results from random matrix theory and large system analysis are used to compute an asymptotic expression of the signal-to-interference-plus-noise ratio as a function of the system parameters, the spatial correlation matrix and the Rician factor. Numerical results are used to evaluate the performance gap in the finite system regime under different operating conditions. © 2016 IEEE.
Asymptotic analysis of downlink MISO systems over Rician fading channels
Falconet, Hugo; Sanguinetti, Luca; Kammoun, Abla; Debbah, Merouane
2016-01-01
In this work, we focus on the ergodic sum rate in the downlink of a single-cell large-scale multi-user MIMO system in which the base station employs N antennas to communicate with K single-antenna user equipments. A regularized zero-forcing (RZF) scheme is used for precoding under the assumption that each link forms a spatially correlated MIMO Rician fading channel. The analysis is conducted assuming N and K grow large with a non trivial ratio and perfect channel state information is available at the base station. Recent results from random matrix theory and large system analysis are used to compute an asymptotic expression of the signal-to-interference-plus-noise ratio as a function of the system parameters, the spatial correlation matrix and the Rician factor. Numerical results are used to evaluate the performance gap in the finite system regime under different operating conditions. © 2016 IEEE.
An Application of Graph Theory in Markov Chains Reliability Analysis
Directory of Open Access Journals (Sweden)
Pavel Skalny
2014-01-01
Full Text Available The paper presents reliability analysis which was realized for an industrial company. The aim of the paper is to present the usage of discrete time Markov chains and the flow in network approach. Discrete Markov chains a well-known method of stochastic modelling describes the issue. The method is suitable for many systems occurring in practice where we can easily distinguish various amount of states. Markov chains are used to describe transitions between the states of the process. The industrial process is described as a graph network. The maximal flow in the network corresponds to the production. The Ford-Fulkerson algorithm is used to quantify the production for each state. The combination of both methods are utilized to quantify the expected value of the amount of manufactured products for the given time period.
English, Thomas
2005-01-01
A standard tool of reliability analysis used at NASA-JSC is the event tree. An event tree is simply a probability tree, with the probabilities determining the next step through the tree specified at each node. The nodal probabilities are determined by a reliability study of the physical system at work for a particular node. The reliability study performed at a node is typically referred to as a fault tree analysis, with the potential of a fault tree existing.for each node on the event tree. When examining an event tree it is obvious why the event tree/fault tree approach has been adopted. Typical event trees are quite complex in nature, and the event tree/fault tree approach provides a systematic and organized approach to reliability analysis. The purpose of this study was two fold. Firstly, we wanted to explore the possibility that a semi-Markov process can create dependencies between sojourn times (the times it takes to transition from one state to the next) that can decrease the uncertainty when estimating time to failures. Using a generalized semi-Markov model, we studied a four element reliability model and were able to demonstrate such sojourn time dependencies. Secondly, we wanted to study the use of semi-Markov processes to introduce a time variable into the event tree diagrams that are commonly developed in PRA (Probabilistic Risk Assessment) analyses. Event tree end states which change with time are more representative of failure scenarios than are the usual static probability-derived end states.
Approximating Markov Chains: What and why
International Nuclear Information System (INIS)
Pincus, S.
1996-01-01
Much of the current study of dynamical systems is focused on geometry (e.g., chaos and bifurcations) and ergodic theory. Yet dynamical systems were originally motivated by an attempt to open-quote open-quote solve,close-quote close-quote or at least understand, a discrete-time analogue of differential equations. As such, numerical, analytical solution techniques for dynamical systems would seem desirable. We discuss an approach that provides such techniques, the approximation of dynamical systems by suitable finite state Markov Chains. Steady state distributions for these Markov Chains, a straightforward calculation, will converge to the true dynamical system steady state distribution, with appropriate limit theorems indicated. Thus (i) approximation by a computable, linear map holds the promise of vastly faster steady state solutions for nonlinear, multidimensional differential equations; (ii) the solution procedure is unaffected by the presence or absence of a probability density function for the attractor, entirely skirting singularity, fractal/multifractal, and renormalization considerations. The theoretical machinery underpinning this development also implies that under very general conditions, steady state measures are weakly continuous with control parameter evolution. This means that even though a system may change periodicity, or become chaotic in its limiting behavior, such statistical parameters as the mean, standard deviation, and tail probabilities change continuously, not abruptly with system evolution. copyright 1996 American Institute of Physics
Prediction of inspection intervals using the Markov analysis
International Nuclear Information System (INIS)
Rea, R.; Arellano, J.
2005-01-01
To solve the unmanageable number of states of Markov of systems that have a great number of components, it is intends a modification to the method of Markov, denominated Markov truncated analysis, in which is assumed that it is worthless the dependence among faults of components. With it the number of states is increased in a lineal way (not exponential) with the number of components of the system, simplifying the analysis vastly. As example, the proposed method was applied to the system HPCS of the CLV considering its 18 main components. It thinks about that each component can take three states: operational, with hidden fault and with revealed fault. Additionally, it takes into account the configuration of the system HPCS by means of a block diagram of dependability to estimate their unavailability at level system. The results of the model here proposed are compared with other methods and approaches used to simplify the Markov analysis. It also intends the modification of the intervals of inspection of three components of the system HPCS. This finishes with base in the developed Markov model and in the maximum time allowed by the code ASME (NUREG-1482) to inspect components of systems that are in reservation in nuclear power plants. (Author)
Modeling cellular networks in fading environments with dominant specular components
Alammouri, Ahmad; Elsawy, Hesham; Salem, Ahmed Sultan; Di Renzo, Marco; Alouini, Mohamed-Slim
2016-01-01
to the Nakagami-m fading in some special cases. However, neither the Rayleigh nor the Nakagami-m accounts for dominant specular components (DSCs) which may appear in realistic fading channels. In this paper, we present a tractable model for cellular networks
Computer models for fading channels with applications to digital transmission
Loo, Chun; Secord, Norman
1991-11-01
The authors describe computer models for Rayleigh, Rician, log-normal, and land-mobile-satellite fading channels. All computer models for the fading channels are based on the manipulation of a white Gaussian random process. This process is approximated by a sum of sinusoids with random phase angle. These models compare very well with analytical models in terms of their probability distribution of envelope and phase of the fading signal. For the land mobile satellite fading channel, results of level crossing rate and average fade duration are given. These results show that the computer models can provide a good coarse estimate of the time statistic of the faded signal. Also, for the land-mobile-satellite fading channel, the results show that a 3-pole Butterworth shaping filter should be used with the model. An example of the application of the land-mobile-satellite fading-channel model to predict the performance of a differential phase-shift keying signal is described.
Flash-induced fading: Dependence on colour and shape similarity
Vergeer, M.L.T.; Lier, R.J. van
2005-01-01
We investigated the effects of perceptual grouping by colour and shape similarity on flash-induced perceptual fading. This flash-induced fading effect (Kanai et al, 2003 Journal of Cognitive Neuroscience 15 664 - 672) is considered as a time-locked variant of the Troxler effect. In the original
On Channel Estimation for OFDM/TDM Using MMSE-FDE in a Fast Fading Channel
Directory of Open Access Journals (Sweden)
Gacanin Haris
2009-01-01
Full Text Available Abstract MMSE-FDE can improve the transmission performance of OFDM combined with time division multiplexing (OFDM/TDM, but knowledge of the channel state information and the noise variance is required to compute the MMSE weight. In this paper, a performance evaluation of OFDM/TDM using MMSE-FDE with pilot-assisted channel estimation over a fast fading channel is presented. To improve the tracking ability against fast fading a robust pilot-assisted channel estimation is presented that uses time-domain filtering on a slot-by-slot basis and frequency-domain interpolation. We derive the mean square error (MSE of the channel estimator and then discuss a tradeoff between improving the tracking ability against fading and the noise reduction. The achievable bit error rate (BER performance is evaluated by computer simulation and compared with conventional OFDM. It is shown that the OFDM/TDM using MMSE-FDE achieves a lower BER and a better tracking ability against fast fading in comparison with conventional OFDM.
Autonomous detection of ISO fade point with color laser printers
Yan, Ni; Maggard, Eric; Fothergill, Roberta; Jessome, Renee J.; Allebach, Jan P.
2015-01-01
Image quality assessment is a very important field in image processing. Human observation is slow and subjective, it also requires strict environment setup for the psychological test 1. Thus developing algorithms to match desired human experiments is always in need. Many studies have focused on detecting the fading phenomenon after the materials are printed, that is to monitor the persistence of the color ink 2-4. However, fading is also a common artifact produced by printing systems when the cartridges run low. We want to develop an automatic system to monitor cartridge life and report fading defects when they appear. In this paper, we first describe a psychological experiment that studies the human perspective on printed fading pages. Then we propose an algorithm based on Color Space Projection and K-means clustering to predict the visibility of fading defects. At last, we integrate the psychological experiment result with our algorithm to give a machine learning tool that monitors cartridge life.
Multivariate η-μ fading distribution with arbitrary correlation model
Ghareeb, Ibrahim; Atiani, Amani
2018-03-01
An extensive analysis for the multivariate ? distribution with arbitrary correlation is presented, where novel analytical expressions for the multivariate probability density function, cumulative distribution function and moment generating function (MGF) of arbitrarily correlated and not necessarily identically distributed ? power random variables are derived. Also, this paper provides exact-form expression for the MGF of the instantaneous signal-to-noise ratio at the combiner output in a diversity reception system with maximal-ratio combining and post-detection equal-gain combining operating in slow frequency nonselective arbitrarily correlated not necessarily identically distributed ?-fading channels. The average bit error probability of differentially detected quadrature phase shift keying signals with post-detection diversity reception system over arbitrarily correlated and not necessarily identical fading parameters ?-fading channels is determined by using the MGF-based approach. The effect of fading correlation between diversity branches, fading severity parameters and diversity level is studied.
Secure Path Selection under Random Fading
Directory of Open Access Journals (Sweden)
Furqan Jameel
2017-05-01
Full Text Available Application-oriented Wireless Sensor Networks (WSNs promises to be one of the most useful technologies of this century. However, secure communication between nodes in WSNs is still an unresolved issue. In this context, we propose two protocols (i.e. Optimal Secure Path (OSP and Sub-optimal Secure Path (SSP to minimize the outage probability of secrecy capacity in the presence of multiple eavesdroppers. We consider dissimilar fading at the main and wiretap link and provide detailed evaluation of the impact of Nakagami-m and Rician-K factors on the secrecy performance of WSNs. Extensive simulations are performed to validate our findings. Although the optimal scheme ensures more security, yet the sub-optimal scheme proves to be a more practical approach to secure wireless links.
Detecting Faults By Use Of Hidden Markov Models
Smyth, Padhraic J.
1995-01-01
Frequency of false alarms reduced. Faults in complicated dynamic system (e.g., antenna-aiming system, telecommunication network, or human heart) detected automatically by method of automated, continuous monitoring. Obtains time-series data by sampling multiple sensor outputs at discrete intervals of t and processes data via algorithm determining whether system in normal or faulty state. Algorithm implements, among other things, hidden first-order temporal Markov model of states of system. Mathematical model of dynamics of system not needed. Present method is "prior" method mentioned in "Improved Hidden-Markov-Model Method of Detecting Faults" (NPO-18982).
Energy Technology Data Exchange (ETDEWEB)
Rea, R.; Arellano, J. [IIE, Calle Reforma 113, Col. Palmira, Cuernavaca, Morelos (Mexico)]. e-mail: rrea@iie.org.mx
2005-07-01
To solve the unmanageable number of states of Markov of systems that have a great number of components, it is intends a modification to the method of Markov, denominated Markov truncated analysis, in which is assumed that it is worthless the dependence among faults of components. With it the number of states is increased in a lineal way (not exponential) with the number of components of the system, simplifying the analysis vastly. As example, the proposed method was applied to the system HPCS of the CLV considering its 18 main components. It thinks about that each component can take three states: operational, with hidden fault and with revealed fault. Additionally, it takes into account the configuration of the system HPCS by means of a block diagram of dependability to estimate their unavailability at level system. The results of the model here proposed are compared with other methods and approaches used to simplify the Markov analysis. It also intends the modification of the intervals of inspection of three components of the system HPCS. This finishes with base in the developed Markov model and in the maximum time allowed by the code ASME (NUREG-1482) to inspect components of systems that are in reservation in nuclear power plants. (Author)
Maximizing Entropy over Markov Processes
DEFF Research Database (Denmark)
Biondi, Fabrizio; Legay, Axel; Nielsen, Bo Friis
2013-01-01
The channel capacity of a deterministic system with confidential data is an upper bound on the amount of bits of data an attacker can learn from the system. We encode all possible attacks to a system using a probabilistic specification, an Interval Markov Chain. Then the channel capacity...... as a reward function, a polynomial algorithm to verify the existence of an system maximizing entropy among those respecting a specification, a procedure for the maximization of reward functions over Interval Markov Chains and its application to synthesize an implementation maximizing entropy. We show how...... to use Interval Markov Chains to model abstractions of deterministic systems with confidential data, and use the above results to compute their channel capacity. These results are a foundation for ongoing work on computing channel capacity for abstractions of programs derived from code....
Maximizing entropy over Markov processes
DEFF Research Database (Denmark)
Biondi, Fabrizio; Legay, Axel; Nielsen, Bo Friis
2014-01-01
The channel capacity of a deterministic system with confidential data is an upper bound on the amount of bits of data an attacker can learn from the system. We encode all possible attacks to a system using a probabilistic specification, an Interval Markov Chain. Then the channel capacity...... as a reward function, a polynomial algorithm to verify the existence of a system maximizing entropy among those respecting a specification, a procedure for the maximization of reward functions over Interval Markov Chains and its application to synthesize an implementation maximizing entropy. We show how...... to use Interval Markov Chains to model abstractions of deterministic systems with confidential data, and use the above results to compute their channel capacity. These results are a foundation for ongoing work on computing channel capacity for abstractions of programs derived from code. © 2014 Elsevier...
Markov Networks in Evolutionary Computation
Shakya, Siddhartha
2012-01-01
Markov networks and other probabilistic graphical modes have recently received an upsurge in attention from Evolutionary computation community, particularly in the area of Estimation of distribution algorithms (EDAs). EDAs have arisen as one of the most successful experiences in the application of machine learning methods in optimization, mainly due to their efficiency to solve complex real-world optimization problems and their suitability for theoretical analysis. This book focuses on the different steps involved in the conception, implementation and application of EDAs that use Markov networks, and undirected models in general. It can serve as a general introduction to EDAs but covers also an important current void in the study of these algorithms by explaining the specificities and benefits of modeling optimization problems by means of undirected probabilistic models. All major developments to date in the progressive introduction of Markov networks based EDAs are reviewed in the book. Hot current researc...
Markov Chain Models for the Stochastic Modeling of Pitting Corrosion
Directory of Open Access Journals (Sweden)
A. Valor
2013-01-01
Full Text Available The stochastic nature of pitting corrosion of metallic structures has been widely recognized. It is assumed that this kind of deterioration retains no memory of the past, so only the current state of the damage influences its future development. This characteristic allows pitting corrosion to be categorized as a Markov process. In this paper, two different models of pitting corrosion, developed using Markov chains, are presented. Firstly, a continuous-time, nonhomogeneous linear growth (pure birth Markov process is used to model external pitting corrosion in underground pipelines. A closed-form solution of the system of Kolmogorov's forward equations is used to describe the transition probability function in a discrete pit depth space. The transition probability function is identified by correlating the stochastic pit depth mean with the empirical deterministic mean. In the second model, the distribution of maximum pit depths in a pitting experiment is successfully modeled after the combination of two stochastic processes: pit initiation and pit growth. Pit generation is modeled as a nonhomogeneous Poisson process, in which induction time is simulated as the realization of a Weibull process. Pit growth is simulated using a nonhomogeneous Markov process. An analytical solution of Kolmogorov's system of equations is also found for the transition probabilities from the first Markov state. Extreme value statistics is employed to find the distribution of maximum pit depths.
Markov Models for Handwriting Recognition
Plotz, Thomas
2011-01-01
Since their first inception, automatic reading systems have evolved substantially, yet the recognition of handwriting remains an open research problem due to its substantial variation in appearance. With the introduction of Markovian models to the field, a promising modeling and recognition paradigm was established for automatic handwriting recognition. However, no standard procedures for building Markov model-based recognizers have yet been established. This text provides a comprehensive overview of the application of Markov models in the field of handwriting recognition, covering both hidden
Fast-slow asymptotics for a Markov chain model of fast sodium current
Starý, Tomáš; Biktashev, Vadim N.
2017-09-01
We explore the feasibility of using fast-slow asymptotics to eliminate the computational stiffness of discrete-state, continuous-time deterministic Markov chain models of ionic channels underlying cardiac excitability. We focus on a Markov chain model of fast sodium current, and investigate its asymptotic behaviour with respect to small parameters identified in different ways.
Quantum Markov processes and applications in many-body systems
International Nuclear Information System (INIS)
Temme, P. K.
2010-01-01
This thesis is concerned with the investigation of quantum as well as classical Markov processes and their application in the field of strongly correlated many-body systems. A Markov process is a special kind of stochastic process, which is determined by an evolution that is independent of its history and only depends on the current state of the system. The application of Markov processes has a long history in the field of statistical mechanics and classical many-body theory. Not only are Markov processes used to describe the dynamics of stochastic systems, but they predominantly also serve as a practical method that allows for the computation of fundamental properties of complex many-body systems by means of probabilistic algorithms. The aim of this thesis is to investigate the properties of quantum Markov processes, i.e. Markov processes taking place in a quantum mechanical state space, and to gain a better insight into complex many-body systems by means thereof. Moreover, we formulate a novel quantum algorithm which allows for the computation of the thermal and ground states of quantum many-body systems. After a brief introduction to quantum Markov processes we turn to an investigation of their convergence properties. We find bounds on the convergence rate of the quantum process by generalizing geometric bounds found for classical processes. We generalize a distance measure that serves as the basis for our investigations, the chi-square divergence, to non-commuting probability spaces. This divergence allows for a convenient generalization of the detailed balance condition to quantum processes. We then devise the quantum algorithm that can be seen as the natural generalization of the ubiquitous Metropolis algorithm to simulate quantum many-body Hamiltonians. By this we intend to provide further evidence, that a quantum computer can serve as a fully-fledged quantum simulator, which is not only capable of describing the dynamical evolution of quantum systems, but
Markov transitions and the propagation of chaos
International Nuclear Information System (INIS)
Gottlieb, A.
1998-01-01
The propagation of chaos is a central concept of kinetic theory that serves to relate the equations of Boltzmann and Vlasov to the dynamics of many-particle systems. Propagation of chaos means that molecular chaos, i.e., the stochastic independence of two random particles in a many-particle system, persists in time, as the number of particles tends to infinity. We establish a necessary and sufficient condition for a family of general n-particle Markov processes to propagate chaos. This condition is expressed in terms of the Markov transition functions associated to the n-particle processes, and it amounts to saying that chaos of random initial states propagates if it propagates for pure initial states. Our proof of this result relies on the weak convergence approach to the study of chaos due to Sztitman and Tanaka. We assume that the space in which the particles live is homomorphic to a complete and separable metric space so that we may invoke Prohorov's theorem in our proof. We also show that, if the particles can be in only finitely many states, then molecular chaos implies that the specific entropies in the n-particle distributions converge to the entropy of the limiting single-particle distribution
Consistency and refinement for Interval Markov Chains
DEFF Research Database (Denmark)
Delahaye, Benoit; Larsen, Kim Guldstrand; Legay, Axel
2012-01-01
Interval Markov Chains (IMC), or Markov Chains with probability intervals in the transition matrix, are the base of a classic specification theory for probabilistic systems [18]. The standard semantics of IMCs assigns to a specification the set of all Markov Chains that satisfy its interval...
Katoen, Joost P.; Maneesh Khattri, M.; Zapreev, I.S.; Zapreev, I.S.
2005-01-01
This short tool paper introduces MRMC, a model checker for discrete-time and continuous-time Markov reward models. It supports reward extensions of PCTL and CSL, and allows for the automated verification of properties concerning long-run and instantaneous rewards as well as cumulative rewards. In
Adaptive Partially Hidden Markov Models
DEFF Research Database (Denmark)
Forchhammer, Søren Otto; Rasmussen, Tage
1996-01-01
Partially Hidden Markov Models (PHMM) have recently been introduced. The transition and emission probabilities are conditioned on the past. In this report, the PHMM is extended with a multiple token version. The different versions of the PHMM are applied to bi-level image coding....
Markov Decision Processes in Practice
Boucherie, Richardus J.; van Dijk, N.M.
2017-01-01
It is over 30 years ago since D.J. White started his series of surveys on practical applications of Markov decision processes (MDP), over 20 years after the phenomenal book by Martin Puterman on the theory of MDP, and over 10 years since Eugene A. Feinberg and Adam Shwartz published their Handbook
Fade detector for the FODA-TDMA access scheme
Celandroni, Nedo; Ferro, Erina; Marzoli, Antonio
1989-05-01
The First in first out Ordered Demand Assignment-Time Division Multiple Access (FODA-TDMA) satellite access scheme designed for simultaneous transmissions of real time data, like packetized voice and slow-scan images (stream traffic) and data coming from standard EDP applications, such as bulk data tansfer, interactive computer access, mailing, data base enquiry and updating (datagram traffic) is described. When deep fades are experienced due to rain attenuation, the system is able to counter the fade. Techniques to detect the fade are presented.
Directory of Open Access Journals (Sweden)
Bao Wang
2014-01-01
Full Text Available We study the strong law of large numbers for the frequencies of occurrence of states and ordered couples of states for countable Markov chains indexed by an infinite tree with uniformly bounded degree, which extends the corresponding results of countable Markov chains indexed by a Cayley tree and generalizes the relative results of finite Markov chains indexed by a uniformly bounded tree.
Automated generation of partial Markov chain from high level descriptions
International Nuclear Information System (INIS)
Brameret, P.-A.; Rauzy, A.; Roussel, J.-M.
2015-01-01
We propose an algorithm to generate partial Markov chains from high level implicit descriptions, namely AltaRica models. This algorithm relies on two components. First, a variation on Dijkstra's algorithm to compute shortest paths in a graph. Second, the definition of a notion of distance to select which states must be kept and which can be safely discarded. The proposed method solves two problems at once. First, it avoids a manual construction of Markov chains, which is both tedious and error prone. Second, up the price of acceptable approximations, it makes it possible to push back dramatically the exponential blow-up of the size of the resulting chains. We report experimental results that show the efficiency of the proposed approach. - Highlights: • We generate Markov chains from a higher level safety modeling language (AltaRica). • We use a variation on Dijkstra's algorithm to generate partial Markov chains. • Hence we solve two problems: the first problem is the tedious manual construction of Markov chains. • The second problem is the blow-up of the size of the chains, at the cost of decent approximations. • The experimental results highlight the efficiency of the method
The capacity of the cascaded fading channel in the low power regime
Benkhelifa, Fatma
2014-04-01
In this paper, we present a simple way to compute the ergodic capacity of cascaded channels with perfect channel state information at both the transmitter and the receiver. We apply our generic results to the Rayleigh-double fading channel, and to the free-space optical channel in the presence of pointing errors and we express their low signal-to-noise ratio capacities. We mainly focus on the low signal-to-noise ratio range.
Application of Hidden Markov Models in Biomolecular Simulations.
Shukla, Saurabh; Shamsi, Zahra; Moffett, Alexander S; Selvam, Balaji; Shukla, Diwakar
2017-01-01
Hidden Markov models (HMMs) provide a framework to analyze large trajectories of biomolecular simulation datasets. HMMs decompose the conformational space of a biological molecule into finite number of states that interconvert among each other with certain rates. HMMs simplify long timescale trajectories for human comprehension, and allow comparison of simulations with experimental data. In this chapter, we provide an overview of building HMMs for analyzing bimolecular simulation datasets. We demonstrate the procedure for building a Hidden Markov model for Met-enkephalin peptide simulation dataset and compare the timescales of the process.
The spectral method and ergodic theorems for general Markov chains
International Nuclear Information System (INIS)
Nagaev, S V
2015-01-01
We study the ergodic properties of Markov chains with an arbitrary state space and prove a geometric ergodic theorem. The method of the proof is new: it may be described as an operator method. Our main result is an ergodic theorem for Harris-Markov chains in the case when the return time to some fixed set has finite expectation. Our conditions for the transition function are more general than those used by Athreya-Ney and Nummelin. Unlike them, we impose restrictions not on the original transition function but on the transition function of an embedded Markov chain constructed from the return times to the fixed set mentioned above. The proof uses the spectral theory of linear operators on a Banach space
A Markov Model for Commen-Cause Failures
DEFF Research Database (Denmark)
Platz, Ole
1984-01-01
A continuous time four-state Markov chain is shown to cover several of the models that have been used for describing dependencies between failures of components in redundant systems. Among these are the models derived by Marshall and Olkin and by Freund and models for one-out-of-three and two...
On the Metric-based Approximate Minimization of Markov Chains
DEFF Research Database (Denmark)
Bacci, Giovanni; Bacci, Giorgio; Larsen, Kim Guldstrand
2018-01-01
In this paper we address the approximate minimization problem of Markov Chains (MCs) from a behavioral metric-based perspective. Specifically, given a finite MC and a positive integer k, we are looking for an MC with at most k states having minimal distance to the original. The metric considered...
On the Total Variation Distance of Semi-Markov Chains
DEFF Research Database (Denmark)
Bacci, Giorgio; Bacci, Giovanni; Larsen, Kim Guldstrand
2015-01-01
Semi-Markov chains (SMCs) are continuous-time probabilistic transition systems where the residence time on states is governed by generic distributions on the positive real line. This paper shows the tight relation between the total variation distance on SMCs and their model checking problem over...
On the Metric-Based Approximate Minimization of Markov Chains
DEFF Research Database (Denmark)
Bacci, Giovanni; Bacci, Giorgio; Larsen, Kim Guldstrand
2017-01-01
We address the behavioral metric-based approximate minimization problem of Markov Chains (MCs), i.e., given a finite MC and a positive integer k, we are interested in finding a k-state MC of minimal distance to the original. By considering as metric the bisimilarity distance of Desharnais at al...
A theoretical Markov chain model for evaluating correctional ...
African Journals Online (AJOL)
In this paper a stochastic method is applied in the study of the long time effect of confinement in a correctional institution on the behaviour of a person with criminal tendencies. The approach used is Markov chain, which uses past history to predict the state of a system in the future. A model is developed for comparing the ...
A Markov deterioration model for predicting recurrent maintenance ...
African Journals Online (AJOL)
The parameters of the Markov chain model for predicting the condition of the road at a design · period for· the flexible pavement failures of wheel track rutting, cracks and pot holes were developed for the Niger State· road network . in Nigeria. Twelve sampled candidate roads were each subjected to standard inventory, traffic ...
OPRA capacity bounds for selection diversity over generalized fading channels
Hanif, Muhammad Fainan; Yang, Hongchuan; Alouini, Mohamed-Slim
2014-01-01
, lower and upper bounds on OPRA capacity for selection diversity scheme are presented. These bounds hold for variety of fading channels including log-normal and generalized Gamma distributed models and have very simple analytic expressions for easy
Efficient incremental relaying for packet transmission over fading channels
Fareed, Muhammad Mehboob; Alouini, Mohamed-Slim; Yang, Hongchuan
2014-01-01
In this paper, we propose a novel relaying scheme for packet transmission over fading channels, which improves the spectral efficiency of cooperative diversity systems by utilizing limited feedback from the destination. Our scheme capitalizes
An Automated Fading Procedure to Alter Sexual Responsiveness in Pedophiles
Laws, D. R.; Pawlowski, A. V.
1975-01-01
An automated stimulus fading procedure was used to strengthen sexual responsiveness to adult stimuli in two pedophiles. The degree of responsiveness was indicated by changes in the penile response. Implications for future research are discussed. (Author)
An overview of turbo decoding on fading channels
ATILGAN, Doğan
2009-01-01
A review of turbo coding and decoding has been presented in the literature [1]. In that paper, turbo coding and decoding on AWGN (Additive White Gaussian Noise) channels has been elaborated. In wireless communications, a phenomennon called multipath fading is frequently encountered. Therefore, investigation of efficient techniques to tackle with the destructive effects of fading is essential. Turbo coding has been proven as an efficient channel coding technique for AWGN channels. Some of the ...
A reason of fast and deep fading of centimeter wave
International Nuclear Information System (INIS)
Danzan, D.; Damdinsuren, E.; Hiamjav, J.; Chuluunbaatar, Ch.; Battulga, S.
1992-01-01
First discovered experimentally exactly correlation between of appearance and of disappearance of optical mirage and fast and deep fading of horizontal polarization of centimeter wave. Proved the interference of the straight and reflected rays from the thin layer of air in mirage a reason of this fading. The physical parameters data of the layer of mirage: change of dielectric permeability and n/ h gradient of refraction index of air in this layer are been showed
Analysis of multipath channel fading techniques in wireless communication systems
Mahender, Kommabatla; Kumar, Tipparti Anil; Ramesh, K. S.
2018-04-01
Multipath fading occurs in any environment where there is multipath propagation and there is some movement of elements within the radio communications system. This may include the radio transmitter or receiver position, or in the elements that give rise to the reflections. The multipath fading can often be relatively deep, i.e. the signals fade completely away, whereas at other times the fading may not cause the signal to fall below a useable strength. Multipath fading may also cause distortion to the radio signal. As the various paths that can be taken by the signals vary in length, the signal transmitted at a particular instance will arrive at the receiver over a spread of times. This can cause problems with phase distortion and inter symbol interference when data transmissions are made. As a result, it may be necessary to incorporate features within the radio communications system that enables the effects of these problems to be minimized. This paper analyses the effects of various types of multipath fading in wireless transmission system.
A new simple model for composite fading channels: Second order statistics and channel capacity
Yilmaz, Ferkan
2010-09-01
In this paper, we introduce the most general composite fading distribution to model the envelope and the power of the received signal in such fading channels as millimeter wave (60 GHz or above) fading channels and free-space optical channels, which we term extended generalized-K (EGK) composite fading distribution. We obtain the second-order statistics of the received signal envelope characterized by the EGK composite fading distribution. Expressions for probability density function, cumulative distribution function, level crossing rate and average fade duration, moments, amount of fading and average capacity are derived. Numerical and computer simulation examples validate the accuracy of the presented mathematical analysis. © 2010 IEEE.
On the outage capacity of the block fading channel at low-power regime
Rezki, Zouheir
2014-06-01
Outage performance of the M-block fading with additive white Gaussian noise (BF-AWGN) is investigated at low-power regime. We consider delay-constrained constant-rate communications with perfect channel state information (CSI) at both the transmitter and the receiver (CSI-TR), under a short-term power constraint. We show that selection diversity that allocates all the power to the strongest block is asymptotically optimal. Then, we provide a simple characterization of the outage probability in the regime of interest. We quantify the reward due to CSI-TR over the constant-rate constant-power scheme and show that this reward increases with the delay constraint. For instance, for Rayleigh fading, we find that a power gain up to 4.3 dB is achievable. © 2014 IEEE.
Delay-limited capacity of fading multiple access and broadcast channels in the low power regime
Rezki, Zouheir
2015-09-11
We study delay-limited (also called zero-outage) capacity region of the fading multi-access channel (MAC) with Gaussian noise and perfect channel state information (CSI) at the receiver and at the transmitters (CSI-TR), in the low-power regime. We show that for fading channels where the MAC capacity region is strictly positive, it has a multidimensional rectangle structure and thus is simply characterized by single user capacity points. More specifically, we show that at low power, the boundary surface of the capacity region shrinks to a single point corresponding to the sum-rate maximizer and that the coordinates of this point coincide with single user capacity bounds. Using the duality of the Gaussian MAC and broadcast channels (BC), we show that time-sharing (or time division multiple access (TDMA)) is asymptotically optimal. © 2015 IEEE.
On Outage Performance of Spectrum-Sharing Communication over M-Block Fading
Alabbasi, AbdulRahman
2015-12-06
In this paper, we consider a cognitive radio system in which a block-fading channel is assumed. Each transmission frame consists of M blocks and each block undergoes a different channel gain. Instantaneous channel state information about the interference links remains unknown to the primary and secondary users. We minimize the secondary user\\'s targeted outage probability over the block-fading channels. To protect the primary user, a statistical constraint on its targeted outage probability is enforced. The secondary user\\'s targeted outage region and the corresponding optimal power are derived. We also propose two sub-optimal power strategies and derive compact expressions for the corresponding outage probabilities. These probabilities are shown to be asymptotic lower and upper bounds on the outage probability. Utilizing these bounds, we derive the exact diversity order of the secondary user outage probability. Selected numerical results are presented to characterize the system\\'s behavior.
Capacity of spectrum sharing Cognitive Radio systems over Nakagami fading channels at low SNR
Sboui, Lokman
2013-06-01
In this paper, we study the ergodic capacity of Cognitive Radio (CR) spectrum sharing systems at low power regime. We focus on Nakagami fading channels. We formally define the low power regime and present closed form expressions of the capacity in the low power regime under various types of interference and/or power constraints, depending on the available channel state information (CSI) of the cross link (CL) between the secondary user transmitter and the primary user receiver. We explicitly characterize two regimes where either the interference constraint or the power constraint dictates the optimal power profile. Our framework also highlights the effects of different fading parameters on the secondary link ergodic capacity. Interestingly, we show that the low power regime analysis provides a specific insight on the capacity behavior of CR that has not been reported by previous studies. © 2013 IEEE.
Exact goodness-of-fit tests for Markov chains.
Besag, J; Mondal, D
2013-06-01
Goodness-of-fit tests are useful in assessing whether a statistical model is consistent with available data. However, the usual χ² asymptotics often fail, either because of the paucity of the data or because a nonstandard test statistic is of interest. In this article, we describe exact goodness-of-fit tests for first- and higher order Markov chains, with particular attention given to time-reversible ones. The tests are obtained by conditioning on the sufficient statistics for the transition probabilities and are implemented by simple Monte Carlo sampling or by Markov chain Monte Carlo. They apply both to single and to multiple sequences and allow a free choice of test statistic. Three examples are given. The first concerns multiple sequences of dry and wet January days for the years 1948-1983 at Snoqualmie Falls, Washington State, and suggests that standard analysis may be misleading. The second one is for a four-state DNA sequence and lends support to the original conclusion that a second-order Markov chain provides an adequate fit to the data. The last one is six-state atomistic data arising in molecular conformational dynamics simulation of solvated alanine dipeptide and points to strong evidence against a first-order reversible Markov chain at 6 picosecond time steps. © 2013, The International Biometric Society.
Asymptotic evolution of quantum Markov chains
Energy Technology Data Exchange (ETDEWEB)
Novotny, Jaroslav [FNSPE, CTU in Prague, 115 19 Praha 1 - Stare Mesto (Czech Republic); Alber, Gernot [Institut fuer Angewandte Physik, Technische Universitaet Darmstadt, D-64289 Darmstadt (Germany)
2012-07-01
The iterated quantum operations, so called quantum Markov chains, play an important role in various branches of physics. They constitute basis for many discrete models capable to explore fundamental physical problems, such as the approach to thermal equilibrium, or the asymptotic dynamics of macroscopic physical systems far from thermal equilibrium. On the other hand, in the more applied area of quantum technology they also describe general characteristic properties of quantum networks or they can describe different quantum protocols in the presence of decoherence. A particularly, an interesting aspect of these quantum Markov chains is their asymptotic dynamics and its characteristic features. We demonstrate there is always a vector subspace (typically low-dimensional) of so-called attractors on which the resulting superoperator governing the iterative time evolution of quantum states can be diagonalized and in which the asymptotic quantum dynamics takes place. As the main result interesting algebraic relations are presented for this set of attractors which allow to specify their dual basis and to determine them in a convenient way. Based on this general theory we show some generalizations concerning the theory of fixed points or asymptotic evolution of random quantum operations.
Markov chains and mixing times
Levin, David A
2017-01-01
Markov Chains and Mixing Times is a magical book, managing to be both friendly and deep. It gently introduces probabilistic techniques so that an outsider can follow. At the same time, it is the first book covering the geometric theory of Markov chains and has much that will be new to experts. It is certainly THE book that I will use to teach from. I recommend it to all comers, an amazing achievement. -Persi Diaconis, Mary V. Sunseri Professor of Statistics and Mathematics, Stanford University Mixing times are an active research topic within many fields from statistical physics to the theory of algorithms, as well as having intrinsic interest within mathematical probability and exploiting discrete analogs of important geometry concepts. The first edition became an instant classic, being accessible to advanced undergraduates and yet bringing readers close to current research frontiers. This second edition adds chapters on monotone chains, the exclusion process and hitting time parameters. Having both exercises...
Bayesian tomography by interacting Markov chains
Romary, T.
2017-12-01
In seismic tomography, we seek to determine the velocity of the undergound from noisy first arrival travel time observations. In most situations, this is an ill posed inverse problem that admits several unperfect solutions. Given an a priori distribution over the parameters of the velocity model, the Bayesian formulation allows to state this problem as a probabilistic one, with a solution under the form of a posterior distribution. The posterior distribution is generally high dimensional and may exhibit multimodality. Moreover, as it is known only up to a constant, the only sensible way to addressthis problem is to try to generate simulations from the posterior. The natural tools to perform these simulations are Monte Carlo Markov chains (MCMC). Classical implementations of MCMC algorithms generally suffer from slow mixing: the generated states are slow to enter the stationary regime, that is to fit the observations, and when one mode of the posterior is eventually identified, it may become difficult to visit others. Using a varying temperature parameter relaxing the constraint on the data may help to enter the stationary regime. Besides, the sequential nature of MCMC makes them ill fitted toparallel implementation. Running a large number of chains in parallel may be suboptimal as the information gathered by each chain is not mutualized. Parallel tempering (PT) can be seen as a first attempt to make parallel chains at different temperatures communicate but only exchange information between current states. In this talk, I will show that PT actually belongs to a general class of interacting Markov chains algorithm. I will also show that this class enables to design interacting schemes that can take advantage of the whole history of the chain, by authorizing exchanges toward already visited states. The algorithms will be illustrated with toy examples and an application to first arrival traveltime tomography.
Automatic creation of Markov models for reliability assessment of safety instrumented systems
International Nuclear Information System (INIS)
Guo Haitao; Yang Xianhui
2008-01-01
After the release of new international functional safety standards like IEC 61508, people care more for the safety and availability of safety instrumented systems. Markov analysis is a powerful and flexible technique to assess the reliability measurements of safety instrumented systems, but it is fallible and time-consuming to create Markov models manually. This paper presents a new technique to automatically create Markov models for reliability assessment of safety instrumented systems. Many safety related factors, such as failure modes, self-diagnostic, restorations, common cause and voting, are included in Markov models. A framework is generated first based on voting, failure modes and self-diagnostic. Then, repairs and common-cause failures are incorporated into the framework to build a complete Markov model. Eventual simplification of Markov models can be done by state merging. Examples given in this paper show how explosively the size of Markov model increases as the system becomes a little more complicated as well as the advancement of automatic creation of Markov models
Mohammad Sayemuzzaman; Manoj K. Jha
2014-01-01
State wide variant topographic features in North Carolina attract the hydro-climatologist. There is none modeling study found that predict future Land Cover Land Use (LCLU) change for whole North Carolina. In this study, satellite-derived land cover maps of year 1992, 2001 and 2006 of North Carolina were integrated within the framework of the Markov-Cellular Automata (Markov-CA) model which combines the Markov chain and Cellular Automata (CA) techniques. A Multi-Criteria Evaluation (MCE) was ...
Achievable rate of spectrum sharing cognitive radio systems over fading channels at low-power regime
Sboui, Lokman
2014-11-01
We study the achievable rate of cognitive radio (CR) spectrum sharing systems at the low-power regime for general fading channels and then for Nakagami fading. We formally define the low-power regime and present the corresponding closed-form expressions of the achievable rate lower bound under various types of interference and/or power constraints, depending on the available channel state information of the cross link (CL) between the secondary-user transmitter and the primary-user receiver. We explicitly characterize two regimes where either the interference constraint or the power constraint dictates the optimal power profile. Our framework also highlights the effects of different fading parameters on the secondary link (SL) ergodic achievable rate. We also study more realistic scenarios when there is either 1-bit quantized channel feedback from the CL alone or 2-bit feedback from both the CL and the SL and propose simple power control schemes and show that these schemes achieve the previously achieved rate at the low-power regime. Interestingly, we show that the low-power regime analysis provides a specific insight into the maximum achievable rate behavior of CR that has not been reported by previous studies.
The computation of stationary distributions of Markov chains through perturbations
Directory of Open Access Journals (Sweden)
Jeffery J. Hunter
1991-01-01
Full Text Available An algorithmic procedure for the determination of the stationary distribution of a finite, m-state, irreducible Markov chain, that does not require the use of methods for solving systems of linear equations, is presented. The technique is based upon a succession of m, rank one, perturbations of the trivial doubly stochastic matrix whose known steady state vector is updated at each stage to yield the required stationary probability vector.
MARKOV CHAIN MODELING OF PERFORMANCE DEGRADATION OF PHOTOVOLTAIC SYSTEM
E. Suresh Kumar; Asis Sarkar; Dhiren kumar Behera
2012-01-01
Modern probability theory studies chance processes for which theknowledge of previous outcomes influence predictions for future experiments. In principle, when a sequence of chance experiments, all of the past outcomes could influence the predictions for the next experiment. In Markov chain type of chance, the outcome of a given experiment can affect the outcome of the next experiment. The system state changes with time and the state X and time t are two random variables. Each of these variab...
Verification of Open Interactive Markov Chains
Brazdil, Tomas; Hermanns, Holger; Krcal, Jan; Kretinsky, Jan; Rehak, Vojtech
2012-01-01
Interactive Markov chains (IMC) are compositional behavioral models extending both labeled transition systems and continuous-time Markov chains. IMC pair modeling convenience - owed to compositionality properties - with effective verification algorithms and tools - owed to Markov properties. Thus far however, IMC verification did not consider compositionality properties, but considered closed systems. This paper discusses the evaluation of IMC in an open and thus compositional interpretation....
Spectral methods for quantum Markov chains
Energy Technology Data Exchange (ETDEWEB)
Szehr, Oleg
2014-05-08
The aim of this project is to contribute to our understanding of quantum time evolutions, whereby we focus on quantum Markov chains. The latter constitute a natural generalization of the ubiquitous concept of a classical Markov chain to describe evolutions of quantum mechanical systems. We contribute to the theory of such processes by introducing novel methods that allow us to relate the eigenvalue spectrum of the transition map to convergence as well as stability properties of the Markov chain.
Spectral methods for quantum Markov chains
International Nuclear Information System (INIS)
Szehr, Oleg
2014-01-01
The aim of this project is to contribute to our understanding of quantum time evolutions, whereby we focus on quantum Markov chains. The latter constitute a natural generalization of the ubiquitous concept of a classical Markov chain to describe evolutions of quantum mechanical systems. We contribute to the theory of such processes by introducing novel methods that allow us to relate the eigenvalue spectrum of the transition map to convergence as well as stability properties of the Markov chain.
Criterion of Semi-Markov Dependent Risk Model
Institute of Scientific and Technical Information of China (English)
Xiao Yun MO; Xiang Qun YANG
2014-01-01
A rigorous definition of semi-Markov dependent risk model is given. This model is a generalization of the Markov dependent risk model. A criterion and necessary conditions of semi-Markov dependent risk model are obtained. The results clarify relations between elements among semi-Markov dependent risk model more clear and are applicable for Markov dependent risk model.
Earth-Space Links and Fade-Duration Statistics
Davarian, Faramaz
1996-01-01
In recent years, fade-duration statistics have been the subject of several experimental investigations. A good knowledge of the fade-duration distribution is important for the assessment of a satellite communication system's channel dynamics: What is a typical link outage duration? How often do link outages exceeding a given duration occur? Unfortunately there is yet no model that can universally answer the above questions. The available field measurements mainly come from temperate climatic zones and only from a few sites. Furthermore, the available statistics are also limited in the choice of frequency and path elevation angle. Yet, much can be learned from the available information. For example, we now know that the fade-duration distribution is approximately lognormal. Under certain conditions, we can even determine the median and other percentiles of the distribution. This paper reviews the available data obtained by several experimenters in different parts of the world. Areas of emphasis are mobile and fixed satellite links. Fades in mobile links are due to roadside-tree shadowing, whereas fades in fixed links are due to rain attenuation.
Rain Fade Compensation for Ka-Band Communications Satellites
Mitchell, W. Carl; Nguyen, Lan; Dissanayake, Asoka; Markey, Brian; Le, Anh
1997-01-01
This report provides a review and evaluation of rain fade measurement and compensation techniques for Ka-band satellite systems. This report includes a description of and cost estimates for performing three rain fade measurement and compensation experiments. The first experiment deals with rain fade measurement techniques while the second one covers the rain fade compensation techniques. The third experiment addresses a feedback flow control technique for the ABR service (for ATM-based traffic). The following conclusions were observed in this report; a sufficient system signal margin should be allocated for all carriers in a network, that is a fixed clear-sky margin should be typically in the range of 4-5 dB and should be more like 15 dB in the up link for moderate and heavy rain zones; to obtain a higher system margin it is desirable to combine the uplink power control technique with the technique that implements the source information rate and FEC code rate changes resulting in a 4-5 dB increase in the dynamic part of the system margin. The experiments would assess the feasibility of the fade measurements and compensation techniques, and ABR feedback control technique.
Multivariate Markov chain modeling for stock markets
Maskawa, Jun-ichi
2003-06-01
We study a multivariate Markov chain model as a stochastic model of the price changes of portfolios in the framework of the mean field approximation. The time series of price changes are coded into the sequences of up and down spins according to their signs. We start with the discussion for small portfolios consisting of two stock issues. The generalization of our model to arbitrary size of portfolio is constructed by a recurrence relation. The resultant form of the joint probability of the stationary state coincides with Gibbs measure assigned to each configuration of spin glass model. Through the analysis of actual portfolios, it has been shown that the synchronization of the direction of the price changes is well described by the model.
Hidden Markov Model Application to Transfer The Trader Online Forex Brokers
Directory of Open Access Journals (Sweden)
Farida Suharleni
2012-05-01
Full Text Available Hidden Markov Model is elaboration of Markov chain, which is applicable to cases that can’t directly observe. In this research, Hidden Markov Model is used to know trader’s transition to broker forex online. In Hidden Markov Model, observed state is observable part and hidden state is hidden part. Hidden Markov Model allows modeling system that contains interrelated observed state and hidden state. As observed state in trader’s transition to broker forex online is category 1, category 2, category 3, category 4, category 5 by condition of every broker forex online, whereas as hidden state is broker forex online Marketiva, Masterforex, Instaforex, FBS and Others. First step on application of Hidden Markov Model in this research is making construction model by making a probability of transition matrix (A from every broker forex online. Next step is making a probability of observation matrix (B by making conditional probability of five categories, that is category 1, category 2, category 3, category 4, category 5 by condition of every broker forex online and also need to determine an initial state probability (π from every broker forex online. The last step is using Viterbi algorithm to find hidden state sequences that is broker forex online sequences which is the most possible based on model and observed state that is the five categories. Application of Hidden Markov Model is done by making program with Viterbi algorithm using Delphi 7.0 software with observed state based on simulation data. Example: By the number of observation T = 5 and observed state sequences O = (2,4,3,5,1 is found hidden state sequences which the most possible with observed state O as following : where X1 = FBS, X2 = Masterforex, X3 = Marketiva, X4 = Others, and X5 = Instaforex.
Berlow, Noah; Pal, Ranadip
2011-01-01
Genetic Regulatory Networks (GRNs) are frequently modeled as Markov Chains providing the transition probabilities of moving from one state of the network to another. The inverse problem of inference of the Markov Chain from noisy and limited experimental data is an ill posed problem and often generates multiple model possibilities instead of a unique one. In this article, we address the issue of intervention in a genetic regulatory network represented by a family of Markov Chains. The purpose of intervention is to alter the steady state probability distribution of the GRN as the steady states are considered to be representative of the phenotypes. We consider robust stationary control policies with best expected behavior. The extreme computational complexity involved in search of robust stationary control policies is mitigated by using a sequential approach to control policy generation and utilizing computationally efficient techniques for updating the stationary probability distribution of a Markov chain following a rank one perturbation.
Ye, Jing; Dang, Yaoguo; Li, Bingjun
2018-01-01
Grey-Markov forecasting model is a combination of grey prediction model and Markov chain which show obvious optimization effects for data sequences with characteristics of non-stationary and volatility. However, the state division process in traditional Grey-Markov forecasting model is mostly based on subjective real numbers that immediately affects the accuracy of forecasting values. To seek the solution, this paper introduces the central-point triangular whitenization weight function in state division to calculate possibilities of research values in each state which reflect preference degrees in different states in an objective way. On the other hand, background value optimization is applied in the traditional grey model to generate better fitting data. By this means, the improved Grey-Markov forecasting model is built. Finally, taking the grain production in Henan Province as an example, it verifies this model's validity by comparing with GM(1,1) based on background value optimization and the traditional Grey-Markov forecasting model.
Directory of Open Access Journals (Sweden)
Biçer Cenker
2016-01-01
Full Text Available In this paper, the stability of the adaptive fading extended Kalman filter with the matrix forgetting factor when applied to the state estimation problem with noise terms in the non–linear discrete–time stochastic systems has been analysed. The analysis is conducted in a similar manner to the standard extended Kalman filter’s stability analysis based on stochastic framework. The theoretical results show that under certain conditions on the initial estimation error and the noise terms, the estimation error remains bounded and the state estimation is stable.
Markov Decision Process Measurement Model.
LaMar, Michelle M
2018-03-01
Within-task actions can provide additional information on student competencies but are challenging to model. This paper explores the potential of using a cognitive model for decision making, the Markov decision process, to provide a mapping between within-task actions and latent traits of interest. Psychometric properties of the model are explored, and simulation studies report on parameter recovery within the context of a simple strategy game. The model is then applied to empirical data from an educational game. Estimates from the model are found to correlate more strongly with posttest results than a partial-credit IRT model based on outcome data alone.
Directory of Open Access Journals (Sweden)
Jean B. Lasserre
2000-01-01
Full Text Available We consider the class of Markov kernels for which the weak or strong Feller property fails to hold at some discontinuity set. We provide a simple necessary and sufficient condition for existence of an invariant probability measure as well as a Foster-Lyapunov sufficient condition. We also characterize a subclass, the quasi (weak or strong Feller kernels, for which the sequences of expected occupation measures share the same asymptotic properties as for (weak or strong Feller kernels. In particular, it is shown that the sequences of expected occupation measures of strong and quasi strong-Feller kernels with an invariant probability measure converge setwise to an invariant measure.
Adiabatic condition and the quantum hitting time of Markov chains
International Nuclear Information System (INIS)
Krovi, Hari; Ozols, Maris; Roland, Jeremie
2010-01-01
We present an adiabatic quantum algorithm for the abstract problem of searching marked vertices in a graph, or spatial search. Given a random walk (or Markov chain) P on a graph with a set of unknown marked vertices, one can define a related absorbing walk P ' where outgoing transitions from marked vertices are replaced by self-loops. We build a Hamiltonian H(s) from the interpolated Markov chain P(s)=(1-s)P+sP ' and use it in an adiabatic quantum algorithm to drive an initial superposition over all vertices to a superposition over marked vertices. The adiabatic condition implies that, for any reversible Markov chain and any set of marked vertices, the running time of the adiabatic algorithm is given by the square root of the classical hitting time. This algorithm therefore demonstrates a novel connection between the adiabatic condition and the classical notion of hitting time of a random walk. It also significantly extends the scope of previous quantum algorithms for this problem, which could only obtain a full quadratic speedup for state-transitive reversible Markov chains with a unique marked vertex.
Markov chain aggregation for agent-based models
Banisch, Sven
2016-01-01
This self-contained text develops a Markov chain approach that makes the rigorous analysis of a class of microscopic models that specify the dynamics of complex systems at the individual level possible. It presents a general framework of aggregation in agent-based and related computational models, one which makes use of lumpability and information theory in order to link the micro and macro levels of observation. The starting point is a microscopic Markov chain description of the dynamical process in complete correspondence with the dynamical behavior of the agent-based model (ABM), which is obtained by considering the set of all possible agent configurations as the state space of a huge Markov chain. An explicit formal representation of a resulting “micro-chain” including microscopic transition rates is derived for a class of models by using the random mapping representation of a Markov process. The type of probability distribution used to implement the stochastic part of the model, which defines the upd...
Monte Carlo methods for the reliability analysis of Markov systems
International Nuclear Information System (INIS)
Buslik, A.J.
1985-01-01
This paper presents Monte Carlo methods for the reliability analysis of Markov systems. Markov models are useful in treating dependencies between components. The present paper shows how the adjoint Monte Carlo method for the continuous time Markov process can be derived from the method for the discrete-time Markov process by a limiting process. The straightforward extensions to the treatment of mean unavailability (over a time interval) are given. System unavailabilities can also be estimated; this is done by making the system failed states absorbing, and not permitting repair from them. A forward Monte Carlo method is presented in which the weighting functions are related to the adjoint function. In particular, if the exact adjoint function is known then weighting factors can be constructed such that the exact answer can be obtained with a single Monte Carlo trial. Of course, if the exact adjoint function is known, there is no need to perform the Monte Carlo calculation. However, the formulation is useful since it gives insight into choices of the weight factors which will reduce the variance of the estimator
Secret Sharing over Fast-Fading MIMO Wiretap Channels
Directory of Open Access Journals (Sweden)
Bloch Matthieu
2009-01-01
Full Text Available Secret sharing over the fast-fading MIMO wiretap channel is considered. A source and a destination try to share secret information over a fast-fading MIMO channel in the presence of an eavesdropper who also makes channel observations that are different from but correlated to those made by the destination. An interactive, authenticated public channel with unlimited capacity is available to the source and destination for the secret sharing process. This situation is a special case of the "channel model with wiretapper" considered by Ahlswede and Csiszár. An extension of their result to continuous channel alphabets is employed to evaluate the key capacity of the fast-fading MIMO wiretap channel. The effects of spatial dimensionality provided by the use of multiple antennas at the source, destination, and eavesdropper are then investigated.
Optimum Combining for Rapidly Fading Channels in Ad Hoc Networks
Directory of Open Access Journals (Sweden)
Sonia Furman
2003-10-01
Full Text Available Research and technology in wireless communication systems such as radar and cellular networks have successfully implemented alternative design approaches that utilize antenna array techniques such as optimum combining, to mitigate the degradation effects of multipath in rapid fading channels. In ad hoc networks, these methods have not yet been exploited primarily due to the complexity inherent in the network's architecture. With the high demand for improved signal link quality, devices configured with omnidirectional antennas can no longer meet the growing need for link quality and spectrum efficiency. This study takes an empirical approach to determine an optimum combining antenna array based on 3 variants of interelement spacing. For rapid fading channels, the simulation results show that the performance in the network of devices retrofitted with our antenna arrays consistently exceeded those with an omnidirectional antenna. Further, with the optimum combiner, the performance increased by over 60% compared to that of an omnidirectional antenna in a rapid fading channel.
Channel allocation and rate adaptation for relayed transmission over correlated fading channels
Hwang, Kyusung
2009-09-01
We consider, in this paper, channel allocation and rate adaptation scheme for relayed transmission over correlated fading channels via cross-layer design. Specifically, jointly considering the data link layer buffer occupancy and channel quality at both the source and relay nodes, we develop an optimal channel allocation and rate adaptation policy for a dual-hop relayed transmission. As such the overall transmit power for the relayed system is minimized while a target packet dropping rate (PDR) due to buffer over flows is guaranteed. In order to find such an optimal policy, the channel allocation and rate adaptation transmission framework is formulated as a constraint Markov decision process (CMDP). The PDR performance of the optimal policy is compared with that of two conventional suboptimal schemes, namely the channel quality based and the buffer occupancy based channel allocation schemes. Numerical results show that for a given power budget, the optimal scheme requires significantly less power than the conventional schemes in order to maintain a target PDR. ©2009 IEEE.
Yahampath, Pradeepa
2017-12-01
Consider communicating a correlated Gaussian source over a Rayleigh fading channel with no knowledge of the channel signal-to-noise ratio (CSNR) at the transmitter. In this case, a digital system cannot be optimal for a range of CSNRs. Analog transmission however is optimal at all CSNRs, if the source and channel are memoryless and bandwidth matched. This paper presents new hybrid digital-analog (HDA) systems for sources with memory and channels with bandwidth expansion, which outperform both digital-only and analog-only systems over a wide range of CSNRs. The digital part is either a predictive quantizer or a transform code, used to achieve a coding gain. Analog part uses linear encoding to transmit the quantization error which improves the performance under CSNR variations. The hybrid encoder is optimized to achieve the minimum AMMSE (average minimum mean square error) over the CSNR distribution. To this end, analytical expressions are derived for the AMMSE of asymptotically optimal systems. It is shown that the outage CSNR of the channel code and the analog-digital power allocation must be jointly optimized to achieve the minimum AMMSE. In the case of HDA predictive quantization, a simple algorithm is presented to solve the optimization problem. Experimental results are presented for both Gauss-Markov sources and speech signals.
Cooperative Spectrum Sensing over Non-Identical Nakagami Fading Channels
Rao, Anlei
2012-09-08
Previous works in cooperative spectrum sensing assumed that the channels for sensing and reporting are independent identical distributed (i.i.d). A more practical and appropriate assumption, however, should be that the sensing channels and reporting channels are independent but not necessarily identically distributed (i.n.i.d). In this paper, we derive the false-alarm probability and the detection probability of cooperative spectrum sensing with energy fusion over i.n.i.d Nakagami fading channels. Selected numerical results show that cooperative spectrum sensing still gives considerably better performance results even over i.n.i.d fading channels.
Fast Faraday fading of long range satellite signals.
Heron, M. L.
1972-01-01
20 MHz radio signals have been received during the day from satellite Beacon-B when it was below the optical horizon by using a bank of narrow filters to improve the signal to noise ratio. The Faraday fading rate becomes constant, under these conditions, at a level determined by the plasma frequency just below the F-layer peak. Variations in the Faraday fading rate reveal fluctuations in the electron density near the peak, while the rate of attaining the constant level depends on the shape of the electron density profile.
Timed Comparisons of Semi-Markov Processes
DEFF Research Database (Denmark)
Pedersen, Mathias Ruggaard; Larsen, Kim Guldstrand; Bacci, Giorgio
2018-01-01
-Markov processes, and investigate the question of how to compare two semi-Markov processes with respect to their time-dependent behaviour. To this end, we introduce the relation of being “faster than” between processes and study its algorithmic complexity. Through a connection to probabilistic automata we obtain...
Inhomogeneous Markov point processes by transformation
DEFF Research Database (Denmark)
Jensen, Eva B. Vedel; Nielsen, Linda Stougaard
2000-01-01
We construct parametrized models for point processes, allowing for both inhomogeneity and interaction. The inhomogeneity is obtained by applying parametrized transformations to homogeneous Markov point processes. An interesting model class, which can be constructed by this transformation approach......, is that of exponential inhomogeneous Markov point processes. Statistical inference For such processes is discussed in some detail....
Applying Markov Chains for NDVI Time Series Forecasting of Latvian Regions
Directory of Open Access Journals (Sweden)
Stepchenko Arthur
2015-12-01
Full Text Available Time series of earth observation based estimates of vegetation inform about variations in vegetation at the scale of Latvia. A vegetation index is an indicator that describes the amount of chlorophyll (the green mass and shows the relative density and health of vegetation. NDVI index is an important variable for vegetation forecasting and management of various problems, such as climate change monitoring, energy usage monitoring, managing the consumption of natural resources, agricultural productivity monitoring, drought monitoring and forest fire detection. In this paper, we make a one-step-ahead prediction of 7-daily time series of NDVI index using Markov chains. The choice of a Markov chain is due to the fact that a Markov chain is a sequence of random variables where each variable is located in some state. And a Markov chain contains probabilities of moving from one state to other.
Classification Using Markov Blanket for Feature Selection
DEFF Research Database (Denmark)
Zeng, Yifeng; Luo, Jian
2009-01-01
Selecting relevant features is in demand when a large data set is of interest in a classification task. It produces a tractable number of features that are sufficient and possibly improve the classification performance. This paper studies a statistical method of Markov blanket induction algorithm...... for filtering features and then applies a classifier using the Markov blanket predictors. The Markov blanket contains a minimal subset of relevant features that yields optimal classification performance. We experimentally demonstrate the improved performance of several classifiers using a Markov blanket...... induction as a feature selection method. In addition, we point out an important assumption behind the Markov blanket induction algorithm and show its effect on the classification performance....
Pemodelan Markov Switching Dengan Time-varying Transition Probability
Savitri, Anggita Puri; Warsito, Budi; Rahmawati, Rita
2016-01-01
Exchange rate or currency is an economic variable which reflects country's state of economy. It fluctuates over time because of its ability to switch the condition or regime caused by economic and political factors. The changes in the exchange rate are depreciation and appreciation. Therefore, it could be modeled using Markov Switching with Time-Varying Transition Probability which observe the conditional changes and use information variable. From this model, time-varying transition probabili...
ON THE ISSUE OF "MEMORY" MARKOV MODEL OF DAMAGE ACCUMULATION
Directory of Open Access Journals (Sweden)
A. I. Lantuh-Lyaschenko
2010-04-01
Full Text Available This paper presents the application of a probabilistic approach for the modeling of service life of highway bridge elements. The focus of this paper is on the Markov stochastic deterioration models. These models can be used as effective tool for technical state assessments and prediction of residual resource of a structure. For the bridge maintenance purpose these models can give quantitative criteria of a reliability level, risk and prediction algorithms of the residual resource.
Learning classifier systems with memory condition to solve non-Markov problems
Zang, Zhaoxiang; Li, Dehua; Wang, Junying
2012-01-01
In the family of Learning Classifier Systems, the classifier system XCS has been successfully used for many applications. However, the standard XCS has no memory mechanism and can only learn optimal policy in Markov environments, where the optimal action is determined solely by the state of current sensory input. In practice, most environments are partially observable environments on agent's sensation, which are also known as non-Markov environments. Within these environments, XCS either fail...
International Nuclear Information System (INIS)
Xiao Gang; Li Zhizhong
2004-01-01
Based on integral equaiton describing the life-history of Markov system, six types of estimators of the current unavailability of Markov system with dependent repair are propounded. Combining with the biased sampling of state transition time of system, six types of Monte Carlo for estimating the current unavailability are given. Two numerical examples are given to deal with the variances and efficiencies of the six types of Monte Carlo methods. (authors)
The Bacterial Sequential Markov Coalescent.
De Maio, Nicola; Wilson, Daniel J
2017-05-01
Bacteria can exchange and acquire new genetic material from other organisms directly and via the environment. This process, known as bacterial recombination, has a strong impact on the evolution of bacteria, for example, leading to the spread of antibiotic resistance across clades and species, and to the avoidance of clonal interference. Recombination hinders phylogenetic and transmission inference because it creates patterns of substitutions (homoplasies) inconsistent with the hypothesis of a single evolutionary tree. Bacterial recombination is typically modeled as statistically akin to gene conversion in eukaryotes, i.e. , using the coalescent with gene conversion (CGC). However, this model can be very computationally demanding as it needs to account for the correlations of evolutionary histories of even distant loci. So, with the increasing popularity of whole genome sequencing, the need has emerged for a faster approach to model and simulate bacterial genome evolution. We present a new model that approximates the coalescent with gene conversion: the bacterial sequential Markov coalescent (BSMC). Our approach is based on a similar idea to the sequential Markov coalescent (SMC)-an approximation of the coalescent with crossover recombination. However, bacterial recombination poses hurdles to a sequential Markov approximation, as it leads to strong correlations and linkage disequilibrium across very distant sites in the genome. Our BSMC overcomes these difficulties, and shows a considerable reduction in computational demand compared to the exact CGC, and very similar patterns in simulated data. We implemented our BSMC model within new simulation software FastSimBac. In addition to the decreased computational demand compared to previous bacterial genome evolution simulators, FastSimBac provides more general options for evolutionary scenarios, allowing population structure with migration, speciation, population size changes, and recombination hotspots. FastSimBac is
Ansari, Imran Shafique
2010-01-01
The introduction of new schemes that are based on the communication among nodes has motivated the use of composite fading models due to the fact that the nodes experience different multipath fading and shadowing statistics, which subsequently
Wireless Energy Harvesting Using Signals from Multiple Fading Channels
Chen, Yunfei
2017-08-01
In this paper, we study the average, the probability density function and the cumulative distribution function of the harvested power. In the study, the signals are transmitted from multiple sources. The channels are assumed to be either Rician fading or Gamma-shadowed Rician fading. The received signals are then harvested by using either a single harvester for simultaneous transmissions or multiple harvesters for transmissions at different frequencies, antennas or time slots. Both linear and nonlinear models for the energy harvester at the receiver are examined. Numerical results are presented to show that, when a large amount of harvested power is required, a single harvester or the linear range of a practical nonlinear harvester are more efficient, to avoid power outage. Further, the power transfer strategy can be optimized for fixed total power. Specifically, for Rayleigh fading, the optimal strategy is to put the total power at the source with the best channel condition and switch off all other sources, while for general Rician fading, the optimum magnitudes and phases of the transmitting waveforms depend on the channel parameters.
Further investigations on 'non-fading' in K-Feldspar
DEFF Research Database (Denmark)
Jain, Mayank; Buylaert, Jan-Pieter; Thomsen, Kristina Jørkov
2015-01-01
, suggesting that the higher the stimulation temperature in post IR-IRSL methods, the greater the ability to access distant electron hole pairs. We further find that the delayed off-time signal in time-resolved IRSL has immense potential for sampling non-fading signal and should be explored further...
Physical Layer Design in Wireless Sensor Networks for Fading Mitigation
Directory of Open Access Journals (Sweden)
Nuo Chen
2013-09-01
Full Text Available This paper presents the theoretical analysis, simulation results and suggests design in digital technology of a physical layer for wireless sensor networks. The proposed design is able to mitigate fading inside communication channel. To mitigate fading the chip interleaving technique is proposed. For the proposed theoretical model of physical layer, a rigorous mathematical analysis is conducted, where all signals are presented and processed in discrete time domain form which is suitable for further direct processing necessary for devices design in digital technology. Three different channels are used to investigate characteristics of the physical layer: additive white Gaussian noise channel (AWGN, AWG noise and flat fading channel and AWG noise and flat fading channel with interleaver and deinterleaver blocks in the receiver and transmitter respectively. Firstly, the mathematical model of communication system representing physical layer is developed based on the discrete time domain signal representation and processing. In the existing theory, these signals and their processing are represented in continuous time form, which is not suitable for direct implementation in digital technology. Secondly, the expressions for the probability of chip, symbol and bit error are derived. Thirdly, the communication system simulators are developed in MATLAB. The simulation results confirmed theoretical findings.
Finding metastabilities in reversible Markov chains based on incomplete sampling
Directory of Open Access Journals (Sweden)
Fackeldey Konstantin
2017-01-01
Full Text Available In order to fully characterize the state-transition behaviour of finite Markov chains one needs to provide the corresponding transition matrix P. In many applications such as molecular simulation and drug design, the entries of the transition matrix P are estimated by generating realizations of the Markov chain and determining the one-step conditional probability Pij for a transition from one state i to state j. This sampling can be computational very demanding. Therefore, it is a good idea to reduce the sampling effort. The main purpose of this paper is to design a sampling strategy, which provides a partial sampling of only a subset of the rows of such a matrix P. Our proposed approach fits very well to stochastic processes stemming from simulation of molecular systems or random walks on graphs and it is different from the matrix completion approaches which try to approximate the transition matrix by using a low-rank-assumption. It will be shown how Markov chains can be analyzed on the basis of a partial sampling. More precisely. First, we will estimate the stationary distribution from a partially given matrix P. Second, we will estimate the infinitesimal generator Q of P on the basis of this stationary distribution. Third, from the generator we will compute the leading invariant subspace, which should be identical to the leading invariant subspace of P. Forth, we will apply Robust Perron Cluster Analysis (PCCA+ in order to identify metastabilities using this subspace.
Schmidt games and Markov partitions
International Nuclear Information System (INIS)
Tseng, Jimmy
2009-01-01
Let T be a C 2 -expanding self-map of a compact, connected, C ∞ , Riemannian manifold M. We correct a minor gap in the proof of a theorem from the literature: the set of points whose forward orbits are nondense has full Hausdorff dimension. Our correction allows us to strengthen the theorem. Combining the correction with Schmidt games, we generalize the theorem in dimension one: given a point x 0 in M, the set of points whose forward orbit closures miss x 0 is a winning set. Finally, our key lemma, the no matching lemma, may be of independent interest in the theory of symbolic dynamics or the theory of Markov partitions
Cao, Qi; Buskens, Erik; Feenstra, Talitha; Jaarsma, Tiny; Hillege, Hans; Postmus, Douwe
2016-01-01
Continuous-time state transition models may end up having large unwieldy structures when trying to represent all relevant stages of clinical disease processes by means of a standard Markov model. In such situations, a more parsimonious, and therefore easier-to-grasp, model of a patient's disease progression can often be obtained by assuming that the future state transitions do not depend only on the present state (Markov assumption) but also on the past through time since entry in the present state. Despite that these so-called semi-Markov models are still relatively straightforward to specify and implement, they are not yet routinely applied in health economic evaluation to assess the cost-effectiveness of alternative interventions. To facilitate a better understanding of this type of model among applied health economic analysts, the first part of this article provides a detailed discussion of what the semi-Markov model entails and how such models can be specified in an intuitive way by adopting an approach called vertical modeling. In the second part of the article, we use this approach to construct a semi-Markov model for assessing the long-term cost-effectiveness of 3 disease management programs for heart failure. Compared with a standard Markov model with the same disease states, our proposed semi-Markov model fitted the observed data much better. When subsequently extrapolating beyond the clinical trial period, these relatively large differences in goodness-of-fit translated into almost a doubling in mean total cost and a 60-d decrease in mean survival time when using the Markov model instead of the semi-Markov model. For the disease process considered in our case study, the semi-Markov model thus provided a sensible balance between model parsimoniousness and computational complexity. © The Author(s) 2015.
Maximally reliable Markov chains under energy constraints.
Escola, Sean; Eisele, Michael; Miller, Kenneth; Paninski, Liam
2009-07-01
Signal-to-noise ratios in physical systems can be significantly degraded if the outputs of the systems are highly variable. Biological processes for which highly stereotyped signal generations are necessary features appear to have reduced their signal variabilities by employing multiple processing steps. To better understand why this multistep cascade structure might be desirable, we prove that the reliability of a signal generated by a multistate system with no memory (i.e., a Markov chain) is maximal if and only if the system topology is such that the process steps irreversibly through each state, with transition rates chosen such that an equal fraction of the total signal is generated in each state. Furthermore, our result indicates that by increasing the number of states, it is possible to arbitrarily increase the reliability of the system. In a physical system, however, an energy cost is associated with maintaining irreversible transitions, and this cost increases with the number of such transitions (i.e., the number of states). Thus, an infinite-length chain, which would be perfectly reliable, is infeasible. To model the effects of energy demands on the maximally reliable solution, we numerically optimize the topology under two distinct energy functions that penalize either irreversible transitions or incommunicability between states, respectively. In both cases, the solutions are essentially irreversible linear chains, but with upper bounds on the number of states set by the amount of available energy. We therefore conclude that a physical system for which signal reliability is important should employ a linear architecture, with the number of states (and thus the reliability) determined by the intrinsic energy constraints of the system.
Process Modeling for Energy Usage in “Smart House” System with a Help of Markov Discrete Chain
Directory of Open Access Journals (Sweden)
Victor Kravets
2016-05-01
Full Text Available Method for evaluating economic efficiency of technical systems using discrete Markov chains modelling illustrated by the system of “Smart house”, consisting, for example, of the three independently functioning elements. Dynamic model of a random power consumption process in the form of a symmetrical state graph of heterogeneous discrete Markov chain is built. The corresponding mathematical model of a random Markov process of power consumption in the “smart house” system in recurrent matrix form is being developed. Technique of statistical determination of probability of random transition elements of the system and the corresponding to the transition probability matrix of the discrete inhomogeneous Markov chain are developed. Statistically determined random transitions of system elements power consumption and the corresponding distribution laws are introduced. The matrix of transition prices, expectations for the possible states of a system price transition and, eventually, the cost of Markov process of power consumption throughout the day.
[Compared Markov with fractal models by using single-channel experimental and simulation data].
Lan, Tonghan; Wu, Hongxiu; Lin, Jiarui
2006-10-01
The gating mechanical kinetical of ion channels has been modeled as a Markov process. In these models it is assumed that the channel protein has a small number of discrete conformational states and kinetic rate constants connecting these states are constant, the transition rate constants among the states is independent both of time and of the previous channel activity. It is assumed in Liebovitch's fractal model that the channel exists in an infinite number of energy states, consequently, transitions from one conductance state to another would be governed by a continuum of rate constants. In this paper, a statistical comparison is presented of Markov and fractal models of ion channel gating, the analysis is based on single-channel data from ion channel voltage-dependence K+ single channel of neuron cell and simulation data from three-states Markov model.
A new simple model for composite fading channels: Second order statistics and channel capacity
Yilmaz, Ferkan; Alouini, Mohamed-Slim
2010-01-01
In this paper, we introduce the most general composite fading distribution to model the envelope and the power of the received signal in such fading channels as millimeter wave (60 GHz or above) fading channels and free-space optical channels, which
Directory of Open Access Journals (Sweden)
Amit Grover
2014-08-01
Full Text Available The reliable services along with high throughput can be achieved by using wireless communication systems. These systems also provides a wide coverage because of their features, no doubt MIMO Communication System [1] is one among them. Features provided by these systems ensure the improved system coverage and increased data transmission rate by considering multiple numbers of transmitter and receiver antennas. In this article, the concept of equalization has been considered and finally the performance of the MIMO Systems in Rician flat fading [5] channel is compared with the Rayleigh flat fading channel. It has also been observed that the performance of these Systems in Rician Flat Fading Channel is the best as compare to the Rayleigh Flat Fading Channel [10]. It has been concluded that the successive interference methods provide better performance as compare to others, but their complexity is high. Simulation results shows that ML provides the better performance in comparison to other equalizers but Sphere decoder provides the best performance.
Assessing type I error and power of multistate Markov models for panel data-A simulation study.
Cassarly, Christy; Martin, Renee' H; Chimowitz, Marc; Peña, Edsel A; Ramakrishnan, Viswanathan; Palesch, Yuko Y
2017-01-01
Ordinal outcomes collected at multiple follow-up visits are common in clinical trials. Sometimes, one visit is chosen for the primary analysis and the scale is dichotomized amounting to loss of information. Multistate Markov models describe how a process moves between states over time. Here, simulation studies are performed to investigate the type I error and power characteristics of multistate Markov models for panel data with limited non-adjacent state transitions. The results suggest that the multistate Markov models preserve the type I error and adequate power is achieved with modest sample sizes for panel data with limited non-adjacent state transitions.
Analysis and design of Markov jump systems with complex transition probabilities
Zhang, Lixian; Shi, Peng; Zhu, Yanzheng
2016-01-01
The book addresses the control issues such as stability analysis, control synthesis and filter design of Markov jump systems with the above three types of TPs, and thus is mainly divided into three parts. Part I studies the Markov jump systems with partially unknown TPs. Different methodologies with different conservatism for the basic stability and stabilization problems are developed and compared. Then the problems of state estimation, the control of systems with time-varying delays, the case involved with both partially unknown TPs and uncertain TPs in a composite way are also tackled. Part II deals with the Markov jump systems with piecewise homogeneous TPs. Methodologies that can effectively handle control problems in the scenario are developed, including the one coping with the asynchronous switching phenomenon between the currently activated system mode and the controller/filter to be designed. Part III focuses on the Markov jump systems with memory TPs. The concept of σ-mean square stability is propo...
International Nuclear Information System (INIS)
Kupchishin, A.A.; Kupchishin, A.I.; Shmygaleva, T.A.
2002-01-01
Within framework of the cascade-probabilistic (CP) method the radiation and physical processes are studied, theirs relation with Markov's processes are found. The conclusion that CP-function for electrons, protons, alpha-particles and ions are describing by unhomogeneous Markov's chain is drawn. The algorithms are developed, the CP-functions calculations for charged particles, concentration of radiation defects in solids at ion irradiation are carried out as well. Tables for CPF different parameters and radiation defects concentration at charged particle interaction with solids are given. The book consists of the introduction and two chapters: (1) Cascade probabilistic function and the Markov's processes; (2) Radiation defects formation in solids as a part of the Markov's processes. The book is intended for specialists on the radiation defects mathematical stimulation, solid state physics, elementary particles physics and applied mathematics
Finite Markov processes and their applications
Iosifescu, Marius
2007-01-01
A self-contained treatment of finite Markov chains and processes, this text covers both theory and applications. Author Marius Iosifescu, vice president of the Romanian Academy and director of its Center for Mathematical Statistics, begins with a review of relevant aspects of probability theory and linear algebra. Experienced readers may start with the second chapter, a treatment of fundamental concepts of homogeneous finite Markov chain theory that offers examples of applicable models.The text advances to studies of two basic types of homogeneous finite Markov chains: absorbing and ergodic ch
Identification of Optimal Policies in Markov Decision Processes
Czech Academy of Sciences Publication Activity Database
Sladký, Karel
46 2010, č. 3 (2010), s. 558-570 ISSN 0023-5954. [International Conference on Mathematical Methods in Economy and Industry. České Budějovice, 15.06.2009-18.06.2009] R&D Projects: GA ČR(CZ) GA402/08/0107; GA ČR GA402/07/1113 Institutional research plan: CEZ:AV0Z10750506 Keywords : finite state Markov decision processes * discounted and average costs * elimination of suboptimal policies Subject RIV: BB - Applied Statistics, Operational Research Impact factor: 0.461, year: 2010 http://library.utia.cas.cz/separaty/2010/E/sladky-identification of optimal policies in markov decision processes.pdf
Geometric allocation approaches in Markov chain Monte Carlo
International Nuclear Information System (INIS)
Todo, S; Suwa, H
2013-01-01
The Markov chain Monte Carlo method is a versatile tool in statistical physics to evaluate multi-dimensional integrals numerically. For the method to work effectively, we must consider the following key issues: the choice of ensemble, the selection of candidate states, the optimization of transition kernel, algorithm for choosing a configuration according to the transition probabilities. We show that the unconventional approaches based on the geometric allocation of probabilities or weights can improve the dynamics and scaling of the Monte Carlo simulation in several aspects. Particularly, the approach using the irreversible kernel can reduce or sometimes completely eliminate the rejection of trial move in the Markov chain. We also discuss how the space-time interchange technique together with Walker's method of aliases can reduce the computational time especially for the case where the number of candidates is large, such as models with long-range interactions
Extracting Markov Models of Peptide Conformational Dynamics from Simulation Data.
Schultheis, Verena; Hirschberger, Thomas; Carstens, Heiko; Tavan, Paul
2005-07-01
A high-dimensional time series obtained by simulating a complex and stochastic dynamical system (like a peptide in solution) may code an underlying multiple-state Markov process. We present a computational approach to most plausibly identify and reconstruct this process from the simulated trajectory. Using a mixture of normal distributions we first construct a maximum likelihood estimate of the point density associated with this time series and thus obtain a density-oriented partition of the data space. This discretization allows us to estimate the transfer operator as a matrix of moderate dimension at sufficient statistics. A nonlinear dynamics involving that matrix and, alternatively, a deterministic coarse-graining procedure are employed to construct respective hierarchies of Markov models, from which the model most plausibly mapping the generating stochastic process is selected by consideration of certain observables. Within both procedures the data are classified in terms of prototypical points, the conformations, marking the various Markov states. As a typical example, the approach is applied to analyze the conformational dynamics of a tripeptide in solution. The corresponding high-dimensional time series has been obtained from an extended molecular dynamics simulation.
Quantitative risk stratification in Markov chains with limiting conditional distributions.
Chan, David C; Pollett, Philip K; Weinstein, Milton C
2009-01-01
Many clinical decisions require patient risk stratification. The authors introduce the concept of limiting conditional distributions, which describe the equilibrium proportion of surviving patients occupying each disease state in a Markov chain with death. Such distributions can quantitatively describe risk stratification. The authors first establish conditions for the existence of a positive limiting conditional distribution in a general Markov chain and describe a framework for risk stratification using the limiting conditional distribution. They then apply their framework to a clinical example of a treatment indicated for high-risk patients, first to infer the risk of patients selected for treatment in clinical trials and then to predict the outcomes of expanding treatment to other populations of risk. For the general chain, a positive limiting conditional distribution exists only if patients in the earliest state have the lowest combined risk of progression or death. The authors show that in their general framework, outcomes and population risk are interchangeable. For the clinical example, they estimate that previous clinical trials have selected the upper quintile of patient risk for this treatment, but they also show that expanded treatment would weakly dominate this degree of targeted treatment, and universal treatment may be cost-effective. Limiting conditional distributions exist in most Markov models of progressive diseases and are well suited to represent risk stratification quantitatively. This framework can characterize patient risk in clinical trials and predict outcomes for other populations of risk.
Hidden Markov models: the best models for forager movements?
Joo, Rocio; Bertrand, Sophie; Tam, Jorge; Fablet, Ronan
2013-01-01
One major challenge in the emerging field of movement ecology is the inference of behavioural modes from movement patterns. This has been mainly addressed through Hidden Markov models (HMMs). We propose here to evaluate two sets of alternative and state-of-the-art modelling approaches. First, we consider hidden semi-Markov models (HSMMs). They may better represent the behavioural dynamics of foragers since they explicitly model the duration of the behavioural modes. Second, we consider discriminative models which state the inference of behavioural modes as a classification issue, and may take better advantage of multivariate and non linear combinations of movement pattern descriptors. For this work, we use a dataset of >200 trips from human foragers, Peruvian fishermen targeting anchovy. Their movements were recorded through a Vessel Monitoring System (∼1 record per hour), while their behavioural modes (fishing, searching and cruising) were reported by on-board observers. We compare the efficiency of hidden Markov, hidden semi-Markov, and three discriminative models (random forests, artificial neural networks and support vector machines) for inferring the fishermen behavioural modes, using a cross-validation procedure. HSMMs show the highest accuracy (80%), significantly outperforming HMMs and discriminative models. Simulations show that data with higher temporal resolution, HSMMs reach nearly 100% of accuracy. Our results demonstrate to what extent the sequential nature of movement is critical for accurately inferring behavioural modes from a trajectory and we strongly recommend the use of HSMMs for such purpose. In addition, this work opens perspectives on the use of hybrid HSMM-discriminative models, where a discriminative setting for the observation process of HSMMs could greatly improve inference performance.
Hidden Markov models: the best models for forager movements?
Directory of Open Access Journals (Sweden)
Rocio Joo
Full Text Available One major challenge in the emerging field of movement ecology is the inference of behavioural modes from movement patterns. This has been mainly addressed through Hidden Markov models (HMMs. We propose here to evaluate two sets of alternative and state-of-the-art modelling approaches. First, we consider hidden semi-Markov models (HSMMs. They may better represent the behavioural dynamics of foragers since they explicitly model the duration of the behavioural modes. Second, we consider discriminative models which state the inference of behavioural modes as a classification issue, and may take better advantage of multivariate and non linear combinations of movement pattern descriptors. For this work, we use a dataset of >200 trips from human foragers, Peruvian fishermen targeting anchovy. Their movements were recorded through a Vessel Monitoring System (∼1 record per hour, while their behavioural modes (fishing, searching and cruising were reported by on-board observers. We compare the efficiency of hidden Markov, hidden semi-Markov, and three discriminative models (random forests, artificial neural networks and support vector machines for inferring the fishermen behavioural modes, using a cross-validation procedure. HSMMs show the highest accuracy (80%, significantly outperforming HMMs and discriminative models. Simulations show that data with higher temporal resolution, HSMMs reach nearly 100% of accuracy. Our results demonstrate to what extent the sequential nature of movement is critical for accurately inferring behavioural modes from a trajectory and we strongly recommend the use of HSMMs for such purpose. In addition, this work opens perspectives on the use of hybrid HSMM-discriminative models, where a discriminative setting for the observation process of HSMMs could greatly improve inference performance.
Using hidden Markov models to align multiple sequences.
Mount, David W
2009-07-01
A hidden Markov model (HMM) is a probabilistic model of a multiple sequence alignment (msa) of proteins. In the model, each column of symbols in the alignment is represented by a frequency distribution of the symbols (called a "state"), and insertions and deletions are represented by other states. One moves through the model along a particular path from state to state in a Markov chain (i.e., random choice of next move), trying to match a given sequence. The next matching symbol is chosen from each state, recording its probability (frequency) and also the probability of going to that state from a previous one (the transition probability). State and transition probabilities are multiplied to obtain a probability of the given sequence. The hidden nature of the HMM is due to the lack of information about the value of a specific state, which is instead represented by a probability distribution over all possible values. This article discusses the advantages and disadvantages of HMMs in msa and presents algorithms for calculating an HMM and the conditions for producing the best HMM.
Observation uncertainty in reversible Markov chains.
Metzner, Philipp; Weber, Marcus; Schütte, Christof
2010-09-01
In many applications one is interested in finding a simplified model which captures the essential dynamical behavior of a real life process. If the essential dynamics can be assumed to be (approximately) memoryless then a reasonable choice for a model is a Markov model whose parameters are estimated by means of Bayesian inference from an observed time series. We propose an efficient Monte Carlo Markov chain framework to assess the uncertainty of the Markov model and related observables. The derived Gibbs sampler allows for sampling distributions of transition matrices subject to reversibility and/or sparsity constraints. The performance of the suggested sampling scheme is demonstrated and discussed for a variety of model examples. The uncertainty analysis of functions of the Markov model under investigation is discussed in application to the identification of conformations of the trialanine molecule via Robust Perron Cluster Analysis (PCCA+) .
Secure Multiple-Antenna Block-Fading Wiretap Channels with Limited CSI Feedback
Hyadi, Amal
2017-07-18
In this paper, we investigate the ergodic secrecy capacity of a block-fading wiretap channel with limited channel knowledge at the transmitter. We consider that the legitimate receiver, the eavesdropper and the transmitter are equipped with multiple antennas and that the receiving nodes are aware of their respective channel matrices. The transmitter, on the other hand, is only provided by a B-bit feedback of the main channel state information. The feedback bits are sent by the legitimate receiver, at the beginning of each fading block, over an error-free public link with limited capacity. The statistics of the main and the eavesdropper channel state information are known at all nodes. Assuming an average transmit power constraint, we establish upper and lower bounds on the ergodic secrecy capacity. Then, we present a framework to design the optimal codebooks for feedback and transmission. In addition, we show that the proposed lower and upper bounds coincide asymptotically as the capacity of the feedback link becomes large, i.e.
Multi-Sensor Optimal Data Fusion Based on the Adaptive Fading Unscented Kalman Filter.
Gao, Bingbing; Hu, Gaoge; Gao, Shesheng; Zhong, Yongmin; Gu, Chengfan
2018-02-06
This paper presents a new optimal data fusion methodology based on the adaptive fading unscented Kalman filter for multi-sensor nonlinear stochastic systems. This methodology has a two-level fusion structure: at the bottom level, an adaptive fading unscented Kalman filter based on the Mahalanobis distance is developed and serves as local filters to improve the adaptability and robustness of local state estimations against process-modeling error; at the top level, an unscented transformation-based multi-sensor optimal data fusion for the case of N local filters is established according to the principle of linear minimum variance to calculate globally optimal state estimation by fusion of local estimations. The proposed methodology effectively refrains from the influence of process-modeling error on the fusion solution, leading to improved adaptability and robustness of data fusion for multi-sensor nonlinear stochastic systems. It also achieves globally optimal fusion results based on the principle of linear minimum variance. Simulation and experimental results demonstrate the efficacy of the proposed methodology for INS/GNSS/CNS (inertial navigation system/global navigation satellite system/celestial navigation system) integrated navigation.
Secure Multiple-Antenna Block-Fading Wiretap Channels with Limited CSI Feedback
Hyadi, Amal; Rezki, Zouheir; Alouini, Mohamed-Slim
2017-01-01
In this paper, we investigate the ergodic secrecy capacity of a block-fading wiretap channel with limited channel knowledge at the transmitter. We consider that the legitimate receiver, the eavesdropper and the transmitter are equipped with multiple antennas and that the receiving nodes are aware of their respective channel matrices. The transmitter, on the other hand, is only provided by a B-bit feedback of the main channel state information. The feedback bits are sent by the legitimate receiver, at the beginning of each fading block, over an error-free public link with limited capacity. The statistics of the main and the eavesdropper channel state information are known at all nodes. Assuming an average transmit power constraint, we establish upper and lower bounds on the ergodic secrecy capacity. Then, we present a framework to design the optimal codebooks for feedback and transmission. In addition, we show that the proposed lower and upper bounds coincide asymptotically as the capacity of the feedback link becomes large, i.e.
A Bayesian model for binary Markov chains
Directory of Open Access Journals (Sweden)
Belkheir Essebbar
2004-02-01
Full Text Available This note is concerned with Bayesian estimation of the transition probabilities of a binary Markov chain observed from heterogeneous individuals. The model is founded on the Jeffreys' prior which allows for transition probabilities to be correlated. The Bayesian estimator is approximated by means of Monte Carlo Markov chain (MCMC techniques. The performance of the Bayesian estimates is illustrated by analyzing a small simulated data set.
Bayesian analysis of Markov point processes
DEFF Research Database (Denmark)
Berthelsen, Kasper Klitgaard; Møller, Jesper
2006-01-01
Recently Møller, Pettitt, Berthelsen and Reeves introduced a new MCMC methodology for drawing samples from a posterior distribution when the likelihood function is only specified up to a normalising constant. We illustrate the method in the setting of Bayesian inference for Markov point processes...... a partially ordered Markov point process as the auxiliary variable. As the method requires simulation from the "unknown" likelihood, perfect simulation algorithms for spatial point processes become useful....
Radio resource allocation over fading channels under statistical delay constraints
Le-Ngoc, Tho
2017-01-01
This SpringerBrief presents radio resource allocation schemes for buffer-aided communications systems over fading channels under statistical delay constraints in terms of upper-bounded average delay or delay-outage probability. This Brief starts by considering a source-destination communications link with data arriving at the source transmission buffer. The first scenario, the joint optimal data admission control and power allocation problem for throughput maximization is considered, where the source is assumed to have a maximum power and an average delay constraints. The second scenario, optimal power allocation problems for energy harvesting (EH) communications systems under average delay or delay-outage constraints are explored, where the EH source harvests random amounts of energy from renewable energy sources, and stores the harvested energy in a battery during data transmission. Online resource allocation algorithms are developed when the statistical knowledge of the random channel fading, data arrivals...
FADES: A tool for automated fault analysis of complex systems
International Nuclear Information System (INIS)
Wood, C.
1990-01-01
FADES is an Expert System for performing fault analyses on complex connected systems. By using a graphical editor to draw components and link them together the FADES system allows the analyst to describe a given system. The knowledge base created is used to qualitatively simulate the system behaviour. By inducing all possible component failures in the system and determining their effects, a set of facts is built up. These facts are then used to create Fault Trees, or FMEA tables. The facts may also be used for explanation effects and to generate diagnostic rules allowing system instrumentation to be optimised. The prototype system has been built and tested and is preently undergoing testing by users. All comments from these trials will be used to tailor the system to the requirements of the user so that the end product performs the exact task required
Markov Processes in Image Processing
Petrov, E. P.; Kharina, N. L.
2018-05-01
Digital images are used as an information carrier in different sciences and technologies. The aspiration to increase the number of bits in the image pixels for the purpose of obtaining more information is observed. In the paper, some methods of compression and contour detection on the basis of two-dimensional Markov chain are offered. Increasing the number of bits on the image pixels will allow one to allocate fine object details more precisely, but it significantly complicates image processing. The methods of image processing do not concede by the efficiency to well-known analogues, but surpass them in processing speed. An image is separated into binary images, and processing is carried out in parallel with each without an increase in speed, when increasing the number of bits on the image pixels. One more advantage of methods is the low consumption of energy resources. Only logical procedures are used and there are no computing operations. The methods can be useful in processing images of any class and assignment in processing systems with a limited time and energy resources.
Adaptive Markov Chain Monte Carlo
Jadoon, Khan
2016-08-08
A substantial interpretation of electromagnetic induction (EMI) measurements requires quantifying optimal model parameters and uncertainty of a nonlinear inverse problem. For this purpose, an adaptive Bayesian Markov chain Monte Carlo (MCMC) algorithm is used to assess multi-orientation and multi-offset EMI measurements in an agriculture field with non-saline and saline soil. In the MCMC simulations, posterior distribution was computed using Bayes rule. The electromagnetic forward model based on the full solution of Maxwell\\'s equations was used to simulate the apparent electrical conductivity measured with the configurations of EMI instrument, the CMD mini-Explorer. The model parameters and uncertainty for the three-layered earth model are investigated by using synthetic data. Our results show that in the scenario of non-saline soil, the parameters of layer thickness are not well estimated as compared to layers electrical conductivity because layer thicknesses in the model exhibits a low sensitivity to the EMI measurements, and is hence difficult to resolve. Application of the proposed MCMC based inversion to the field measurements in a drip irrigation system demonstrate that the parameters of the model can be well estimated for the saline soil as compared to the non-saline soil, and provide useful insight about parameter uncertainty for the assessment of the model outputs.
Fitting Hidden Markov Models to Psychological Data
Directory of Open Access Journals (Sweden)
Ingmar Visser
2002-01-01
Full Text Available Markov models have been used extensively in psychology of learning. Applications of hidden Markov models are rare however. This is partially due to the fact that comprehensive statistics for model selection and model assessment are lacking in the psychological literature. We present model selection and model assessment statistics that are particularly useful in applying hidden Markov models in psychology. These statistics are presented and evaluated by simulation studies for a toy example. We compare AIC, BIC and related criteria and introduce a prediction error measure for assessing goodness-of-fit. In a simulation study, two methods of fitting equality constraints are compared. In two illustrative examples with experimental data we apply selection criteria, fit models with constraints and assess goodness-of-fit. First, data from a concept identification task is analyzed. Hidden Markov models provide a flexible approach to analyzing such data when compared to other modeling methods. Second, a novel application of hidden Markov models in implicit learning is presented. Hidden Markov models are used in this context to quantify knowledge that subjects express in an implicit learning task. This method of analyzing implicit learning data provides a comprehensive approach for addressing important theoretical issues in the field.
Use of Markov chains for forecasting labor requirements in black coal mines
Energy Technology Data Exchange (ETDEWEB)
Penar, L.; Przybyla, H.
1987-01-01
Increasing mining depth, deterioration of mining conditions and technology development are causes of changes in labor requirements. In mines with stable coal output these changes in most cases are of a qualitative character, in mines with an increasing or decreasing coal output they are of a quantitative character. Methods for forecasting personnel needs, in particular professional requirements, are discussed. Quantitative and qualitative changes are accurately described by heterogenous Markov chains. A structure consisting of interdependent variables is the subject of a forecast. Changes that occur within the structure of time units is the subject of investigations. For a homogenous Markov chain probabilities of a transition from the i-state to the j-state are determined (the probabilities being time independent). For a heterogenous Markov chain probabilities of a transition from the i-state to the j-state are non-conditioned. The method was developed for the ODRA 1325 computers. 8 refs.
Troxler Fading, Eye Movements, and Retinal Ganglion Cell Properties
Directory of Open Access Journals (Sweden)
Romain Bachy
2014-12-01
Full Text Available We present four movies demonstrating the effect of flicker and blur on the magnitude and speed of adaptation for foveal and peripheral vision along the three color axes that isolate retinal ganglion cells projecting to magno, parvo, and konio layers of the LGN. The demonstrations support the eye movement hypothesis for Troxler fading for brightness and color, and demonstrate the effects of flicker and blur on adaptation of each class of retinal ganglion cells.
''adding'' algorithm for the Markov chain formalism for radiation transfer
International Nuclear Information System (INIS)
Esposito, L.W.
1979-01-01
The Markov chain radiative transfer method of Esposito and House has been shown to be both efficient and accurate for calculation of the diffuse reflection from a homogeneous scattering planetary atmosphere. The use of a new algorithm similar to the ''adding'' formula of Hansen and Travis extends the application of this formalism to an arbitrarily deep atmosphere. The basic idea for this algorithm is to consider a preceding calculation as a single state of a new Markov chain. Successive application of this procedure makes calculation possible for any optical depth without increasing the size of the linear system used. The time required for the algorithm is comparable to that for a doubling calculation for a homogeneous atmosphere, but for a non-homogeneous atmosphere the new method is considerably faster than the standard ''adding'' routine. As with he standard ''adding'' method, the information on the internal radiation field is lost during the calculation. This method retains the advantage of the earlier Markov chain method that the time required is relatively insensitive to the number of illumination angles or observation angles for which the diffuse reflection is calculated. A technical write-up giving fuller details of the algorithm and a sample code are available from the author
Radiation degradation of molasses pigment. The fading color and product
International Nuclear Information System (INIS)
Sawai, Teruko; Sekiguchi, Masayuki; Tanabe, Hiroko; Sawai, Takeshi
1993-01-01
Water demand in Tokyo has increased rapidly. Because of the scarcity of water supply sources within the city, Tokyo is dependent on the water from other prefectures. Recycling of municipal effluent is an effective means of coping with water shortage in Tokyo. We have studied the radiation treatment of waste water for recycling. In this paper, the radiation decomposition methods for fading color of molasses pigment in the effluent from the sewage treatment plant and in the food industry wastwater were reported. The refractory organic substances (molasses pigment) in samples were degraded by gamma irradiation. The COD values decreased and the dark brown color faded with increasing dosage. The high molecular weight components of molasses pigment were degraded to lower molecular weight substances and were decomposed finally to carbon dioxide. The organic acids, such as formic acid, acetic acid, oxalic acid, citric acid and succinic acid were measured as intermediates of radiolytic decomposition. When we added hydrogen peroxide in samples to the gamma irradiation process, the dark brown color of molasses pigment faded with greater efficiency. (author)
Analysis of Fade Dynamic at Ku-Band in Malaysia
Directory of Open Access Journals (Sweden)
Siat Ling Jong
2014-01-01
Full Text Available This work investigates fade dynamics of satellite communication systems in equatorial heavy rain region based on a one year of Ku-band propagation measurement campaign carried out in Universiti Teknologi Malaysia (UTM, Johor, Malaysia. First order statistics of rain attenuation are deduced and the results are found to be in good agreement with those obtained from other beacon measurements gathered within the same area (Kuala Lumpur. Moreover, the fade duration and slope statistics of the satellite signal variations are also carefully derived and subsequently compared with the ITU-R recommendation model. Such information is useful for the system operator and radio communication engineer for the design of appropriate fade mitigation techniques as well as the quality of service that could be offered to the user (according to the time interval for a typical day. Further evaluation on the performances of several ITU-R models in the heavy rain region are needed based on the measurement database available of this climatic region.
Secrecy Capacity Analysis over α−μ Fading Channels
Lei, Hongjiang
2017-02-15
In this work, we study the secrecy capacity of the classic Wyner’s model over the α − μ fading channels, where α and μ specify the nonlinearity and clustering of fading channels, respectively. The average secrecy capacity (ASC) is derived in closed-form by using the extended generalized bivariate Fox’s Hfunction (EGBFHF). Moreover, the asymptotic analysis of ASC in high signal-to-noise ratio (SNR) regime is conducted. The asymptotic results unveil that the ASC follows the scaling law of Θ(ln p), where p stands for the ratio between the average powers of main channels and eavesdropping channels. Moreover, the ASC can be enhanced by increasing the transmit SNR, while there exists a ceiling of ASC as the SNRs at both sides are improved simultaneously. The accuracy of the analytical results is validated by Monte-Carlo simulations. The numerical results show that rigorous fading channels are beneficial to the secrecy performance, that is, serious nonlinearity (small α) and sparse clustering (small μ) will lead to the improvement of ASC.
Secrecy Capacity Analysis over α−μ Fading Channels
Lei, Hongjiang; Ansari, Imran Shafique; Pan, Gaofeng; Alomair, Basel; Alouini, Mohamed-Slim
2017-01-01
In this work, we study the secrecy capacity of the classic Wyner’s model over the α − μ fading channels, where α and μ specify the nonlinearity and clustering of fading channels, respectively. The average secrecy capacity (ASC) is derived in closed-form by using the extended generalized bivariate Fox’s Hfunction (EGBFHF). Moreover, the asymptotic analysis of ASC in high signal-to-noise ratio (SNR) regime is conducted. The asymptotic results unveil that the ASC follows the scaling law of Θ(ln p), where p stands for the ratio between the average powers of main channels and eavesdropping channels. Moreover, the ASC can be enhanced by increasing the transmit SNR, while there exists a ceiling of ASC as the SNRs at both sides are improved simultaneously. The accuracy of the analytical results is validated by Monte-Carlo simulations. The numerical results show that rigorous fading channels are beneficial to the secrecy performance, that is, serious nonlinearity (small α) and sparse clustering (small μ) will lead to the improvement of ASC.
DISK-RELATED BURSTS AND FADES IN YOUNG STARS
International Nuclear Information System (INIS)
Findeisen, Krzysztof; Hillenbrand, Lynne; Levitan, David; Sesar, Branimir; Ofek, Eran; Laher, Russ; Surace, Jason
2013-01-01
We present first results from a new, multiyear, time domain survey of young stars in the North America Nebula complex using the Palomar Transient Factory. Our survey is providing an unprecedented view of aperiodic variability in young stars on timescales of days to years. The analyzed sample covers R PTF ≈ 13.5-18 and spans a range of mid-infrared color, with larger-amplitude optical variables (exceeding 0.4 mag root mean squared) more likely to have mid-infrared evidence for circumstellar material. This paper characterizes infrared excess stars with distinct bursts above or fades below a baseline of lower-level variability, identifying 41 examples. The light curves exhibit a remarkable diversity of amplitudes, timescales, and morphologies, with a continuum of behaviors that cannot be classified into distinct groups. Among the bursters, we identify three particularly promising sources that may represent theoretically predicted short-timescale accretion instabilities. Finally, we find that fading behavior is approximately twice as common as bursting behavior on timescales of days to years, although the bursting and fading duty cycle for individual objects often varies from year to year.
ANALYSIS OF MARKOV NETWORK WITH INCOMES, POSITIVE AND NEGATIVE MESSAGES
Directory of Open Access Journals (Sweden)
V. V. Naumenko
2014-01-01
Full Text Available Markov queuing network with income in transient regime is considered. It has positive and negative messages, which can be used in forecasting income of information and telecommunication systems and networks affected by viruses. Investigations are carried out in the cases when incomes from transitions between network states are deterministic functions dependent on states, or they are random variables with given mean values. In the last case it is assumed that all network systems operate in a high load mode. An example is given.
Variance reduction techniques in the simulation of Markov processes
International Nuclear Information System (INIS)
Lessi, O.
1987-01-01
We study a functional r of the stationary distribution of a homogeneous Markov chain. It is often difficult or impossible to perform the analytical calculation of r and so it is reasonable to estimate r by a simulation process. A consistent estimator r(n) of r is obtained with respect to a chain with a countable state space. Suitably modifying the estimator r(n) of r one obtains a new consistent estimator which has a smaller variance than r(n). The same is obtained in the case of finite state space
Hidden Markov models for the activity profile of terrorist groups
Raghavan, Vasanthan; Galstyan, Aram; Tartakovsky, Alexander G.
2012-01-01
The main focus of this work is on developing models for the activity profile of a terrorist group, detecting sudden spurts and downfalls in this profile, and, in general, tracking it over a period of time. Toward this goal, a $d$-state hidden Markov model (HMM) that captures the latent states underlying the dynamics of the group and thus its activity profile is developed. The simplest setting of $d=2$ corresponds to the case where the dynamics are coarsely quantized as Active and Inactive, re...
Markov source model for printed music decoding
Kopec, Gary E.; Chou, Philip A.; Maltz, David A.
1995-03-01
This paper describes a Markov source model for a simple subset of printed music notation. The model is based on the Adobe Sonata music symbol set and a message language of our own design. Chord imaging is the most complex part of the model. Much of the complexity follows from a rule of music typography that requires the noteheads for adjacent pitches to be placed on opposite sides of the chord stem. This rule leads to a proliferation of cases for other typographic details such as dot placement. We describe the language of message strings accepted by the model and discuss some of the imaging issues associated with various aspects of the message language. We also point out some aspects of music notation that appear problematic for a finite-state representation. Development of the model was greatly facilitated by the duality between image synthesis and image decoding. Although our ultimate objective was a music image model for use in decoding, most of the development proceeded by using the evolving model for image synthesis, since it is computationally far less costly to image a message than to decode an image.
Tornadoes and related damage costs: statistical modelling with a semi-Markov approach
Directory of Open Access Journals (Sweden)
Guglielmo D’Amico
2016-09-01
Full Text Available We propose a statistical approach to modelling for predicting and simulating occurrences of tornadoes and accumulated cost distributions over a time interval. This is achieved by modelling the tornado intensity, measured with the Fujita scale, as a stochastic process. Since the Fujita scale divides tornado intensity into six states, it is possible to model the tornado intensity by using Markov and semi-Markov models. We demonstrate that the semi-Markov approach is able to reproduce the duration effect that is detected in tornado occurrence. The superiority of the semi-Markov model as compared to the Markov chain model is also affirmed by means of a statistical test of hypothesis. As an application, we compute the expected value and the variance of the costs generated by the tornadoes over a given time interval in a given area. The paper contributes to the literature by demonstrating that semi-Markov models represent an effective tool for physical analysis of tornadoes as well as for the estimation of the economic damages to human things.
On the representability of complete genomes by multiple competing finite-context (Markov models.
Directory of Open Access Journals (Sweden)
Armando J Pinho
Full Text Available A finite-context (Markov model of order k yields the probability distribution of the next symbol in a sequence of symbols, given the recent past up to depth k. Markov modeling has long been applied to DNA sequences, for example to find gene-coding regions. With the first studies came the discovery that DNA sequences are non-stationary: distinct regions require distinct model orders. Since then, Markov and hidden Markov models have been extensively used to describe the gene structure of prokaryotes and eukaryotes. However, to our knowledge, a comprehensive study about the potential of Markov models to describe complete genomes is still lacking. We address this gap in this paper. Our approach relies on (i multiple competing Markov models of different orders (ii careful programming techniques that allow orders as large as sixteen (iii adequate inverted repeat handling (iv probability estimates suited to the wide range of context depths used. To measure how well a model fits the data at a particular position in the sequence we use the negative logarithm of the probability estimate at that position. The measure yields information profiles of the sequence, which are of independent interest. The average over the entire sequence, which amounts to the average number of bits per base needed to describe the sequence, is used as a global performance measure. Our main conclusion is that, from the probabilistic or information theoretic point of view and according to this performance measure, multiple competing Markov models explain entire genomes almost as well or even better than state-of-the-art DNA compression methods, such as XM, which rely on very different statistical models. This is surprising, because Markov models are local (short-range, contrasting with the statistical models underlying other methods, where the extensive data repetitions in DNA sequences is explored, and therefore have a non-local character.
Burkatovskaya, Yuliya Borisovna; Kabanova, T.; Khaustov, Pavel Aleksandrovich
2016-01-01
CUSUM algorithm for controlling chain state switching in the Markov modulated Poissonprocess was investigated via simulation. Recommendations concerning the parameter choice were givensubject to characteristics of the process. Procedure of the process parameter estimation was described.
Reliability analysis and prediction of mixed mode load using Markov Chain Model
International Nuclear Information System (INIS)
Nikabdullah, N.; Singh, S. S. K.; Alebrahim, R.; Azizi, M. A.; K, Elwaleed A.; Noorani, M. S. M.
2014-01-01
The aim of this paper is to present the reliability analysis and prediction of mixed mode loading by using a simple two state Markov Chain Model for an automotive crankshaft. The reliability analysis and prediction for any automotive component or structure is important for analyzing and measuring the failure to increase the design life, eliminate or reduce the likelihood of failures and safety risk. The mechanical failures of the crankshaft are due of high bending and torsion stress concentration from high cycle and low rotating bending and torsional stress. The Markov Chain was used to model the two states based on the probability of failure due to bending and torsion stress. In most investigations it revealed that bending stress is much serve than torsional stress, therefore the probability criteria for the bending state would be higher compared to the torsion state. A statistical comparison between the developed Markov Chain Model and field data was done to observe the percentage of error. The reliability analysis and prediction was derived and illustrated from the Markov Chain Model were shown in the Weibull probability and cumulative distribution function, hazard rate and reliability curve and the bathtub curve. It can be concluded that Markov Chain Model has the ability to generate near similar data with minimal percentage of error and for a practical application; the proposed model provides a good accuracy in determining the reliability for the crankshaft under mixed mode loading
Soury, Hamza
2013-07-01
This paper considers the average symbol error probability of square Quadrature Amplitude Modulation (QAM) coherent signaling over flat fading channels subject to additive generalized Gaussian noise. More specifically, a generic closedform expression in terms of the Fox H function and the bivariate Fox H function is offered for the extended generalized-K fading case. Simplifications for some special fading distributions such as generalized-K fading, Nakagami-m fading, and Rayleigh fading and special additive noise distributions such as Gaussian and Laplacian noise are then presented. Finally, the mathematical formalism is illustrated by some numerical examples verified by computer based simulations for a variety of fading and additive noise parameters.
Fading AGN Candidates: AGN Histories and Outflow Signatures
Energy Technology Data Exchange (ETDEWEB)
Keel, William C.; Maksym, W. Peter [Department of Physics and Astronomy, University of Alabama, Box 870324, Tuscaloosa, AL 35487 (United States); Lintott, Chris J. [Astrophysics, Oxford University and Adler Planetarium, 1300 S. Lakeshore Drive, Chicago, IL 60605 (United States); Bennert, Vardha N.; Scott, Bryan; Showley, Charles; Flatland, Kelsi [Physics Department, California Polytechnic State University, San Luis Obispo, CA 93407 (United States); Chojnowski, S. Drew [Department of Astronomy, New Mexico State University, P.O. Box 30001, MSC 4500, Las Cruces, NM 88003-8001 (United States); Moiseev, Alexei; Smirnova, Aleksandrina [Special Astrophysical Observatory, Russian Academy of Sciences, Nizhny Arkhyz, 369167 (Russian Federation); Schawinski, Kevin; Sartori, Lia F. [Institute for Astronomy, ETH Zürich, Wolfgang-Pauli-Straße 27, CH-8093 Zurich (Switzerland); Urry, C. Megan [Department of Physics, Yale University, P.O. Box 208120, New Haven, CT 06520-8120 (United States); Pancoast, Anna [Center for Astrophysics, 60 Garden St., Cambridge, MA 02138 (United States); Schirmer, Mischa, E-mail: wkeel@ua.edu [Gemini Observatory, La Serena (Chile)
2017-02-01
We consider the energy budgets and radiative history of eight fading active galactic nuclei (AGNs), identified from an energy shortfall between the requirements to ionize very extended (radius > 10 kpc) ionized clouds and the luminosity of the nucleus as we view it directly. All show evidence of significant fading on timescales of ≈50,000 yr. We explore the use of minimum ionizing luminosity Q {sub ion} derived from photoionization balance in the brightest pixels in H α at each projected radius. Tests using presumably constant Palomar–Green QSOs, and one of our targets with detailed photoionization modeling, suggest that we can derive useful histories of individual AGNs, with the caveat that the minimum ionizing luminosity is always an underestimate and subject to uncertainties about fine structure in the ionized material. These consistency tests suggest that the degree of underestimation from the upper envelope of reconstructed Q {sub ion} values is roughly constant for a given object and therefore does not prevent such derivation. The AGNs in our sample show a range of behaviors, with rapid drops and standstills; the common feature is a rapid drop in the last ≈2×10{sup 4} yr before the direct view of the nucleus. The e -folding timescales for ionizing luminosity are mostly in the thousands of years, with a few episodes as short as 400 yr. In the limit of largely obscured AGNs, we find additional evidence for fading from the shortfall between even the lower limits from recombination balance and the maximum luminosities derived from far-infrared fluxes. We compare these long-term light curves, and the occurrence of these fading objects among all optically identified AGNs, to simulations of AGN accretion; the strongest variations over these timespans are seen in models with strong and local (parsec-scale) feedback. We present Gemini integral-field optical spectroscopy, which shows a very limited role for outflows in these ionized structures. While rings and
Fading AGN Candidates: AGN Histories and Outflow Signatures
International Nuclear Information System (INIS)
Keel, William C.; Maksym, W. Peter; Lintott, Chris J.; Bennert, Vardha N.; Scott, Bryan; Showley, Charles; Flatland, Kelsi; Chojnowski, S. Drew; Moiseev, Alexei; Smirnova, Aleksandrina; Schawinski, Kevin; Sartori, Lia F.; Urry, C. Megan; Pancoast, Anna; Schirmer, Mischa
2017-01-01
We consider the energy budgets and radiative history of eight fading active galactic nuclei (AGNs), identified from an energy shortfall between the requirements to ionize very extended (radius > 10 kpc) ionized clouds and the luminosity of the nucleus as we view it directly. All show evidence of significant fading on timescales of ≈50,000 yr. We explore the use of minimum ionizing luminosity Q ion derived from photoionization balance in the brightest pixels in H α at each projected radius. Tests using presumably constant Palomar–Green QSOs, and one of our targets with detailed photoionization modeling, suggest that we can derive useful histories of individual AGNs, with the caveat that the minimum ionizing luminosity is always an underestimate and subject to uncertainties about fine structure in the ionized material. These consistency tests suggest that the degree of underestimation from the upper envelope of reconstructed Q ion values is roughly constant for a given object and therefore does not prevent such derivation. The AGNs in our sample show a range of behaviors, with rapid drops and standstills; the common feature is a rapid drop in the last ≈2×10 4 yr before the direct view of the nucleus. The e -folding timescales for ionizing luminosity are mostly in the thousands of years, with a few episodes as short as 400 yr. In the limit of largely obscured AGNs, we find additional evidence for fading from the shortfall between even the lower limits from recombination balance and the maximum luminosities derived from far-infrared fluxes. We compare these long-term light curves, and the occurrence of these fading objects among all optically identified AGNs, to simulations of AGN accretion; the strongest variations over these timespans are seen in models with strong and local (parsec-scale) feedback. We present Gemini integral-field optical spectroscopy, which shows a very limited role for outflows in these ionized structures. While rings and loops of emission
Exact Outage Probability of Dual-Hop CSI-Assisted AF Relaying Over Nakagami-m Fading Channels
Xia, Minghua
2012-10-01
In this correspondence, considering dual-hop channel state information (CSI)-assisted amplify-and-forward (AF) relaying over Nakagami- m fading channels, the cumulative distribution function (CDF) of the end-to-end signal-to-noise ratio (SNR) is derived. In particular, when the fading shape factors m1 and m2 at consecutive hops take non-integer values, the bivariate H-function and G -function are exploited to obtain an exact analytical expression for the CDF. The obtained CDF is then applied to evaluate the outage performance of the system under study. The analytical results of outage probability coincide exactly with Monte-Carlo simulation results and outperform the previously reported upper bounds in the low and medium SNR regions.
Assessing type I error and power of multistate Markov models for panel data-A simulation study
Cassarly, Christy; Martin, Renee’ H.; Chimowitz, Marc; Peña, Edsel A.; Ramakrishnan, Viswanathan; Palesch, Yuko Y.
2016-01-01
Ordinal outcomes collected at multiple follow-up visits are common in clinical trials. Sometimes, one visit is chosen for the primary analysis and the scale is dichotomized amounting to loss of information. Multistate Markov models describe how a process moves between states over time. Here, simulation studies are performed to investigate the type I error and power characteristics of multistate Markov models for panel data with limited non-adjacent state transitions. The results suggest that ...
Relay selection in cooperative communication systems over continuous time-varying fading channel
Directory of Open Access Journals (Sweden)
Ke Geng
2017-02-01
Full Text Available In this paper, we study relay selection under outdated channel state information (CSI in a decode-and-forward (DF cooperative system. Unlike previous researches on cooperative communication under outdated CSI, we consider that the channel varies continuously over time, i.e., the channel not only changes between relay selection and data transmission but also changes during data transmission. Thus the level of accuracy of the CSI used in relay selection degrades with data transmission. We first evaluate the packet error rate (PER of the cooperative system under continuous time-varying fading channel, and find that the PER performance deteriorates more seriously under continuous time-varying fading channel than when the channel is assumed to be constant during data transmission. Then, we propose a repeated relay selection (RRS strategy to improve the PER performance, in which the forwarded data is divided into multiple segments and relay is reselected before the transmission of each segment based on the updated CSI. Finally, we propose a combined relay selection (CRS strategy which takes advantage of three different relay selection strategies to further mitigate the impact of outdated CSI.
Maximizing Expected Achievable Rates for Block-Fading Buffer-Aided Relay Channels
Shaqfeh, Mohammad
2016-05-25
In this paper, the long-term average achievable rate over block-fading buffer-aided relay channels is maximized using a hybrid scheme that combines three essential transmission strategies, which are decode-and-forward, compress-and-forward, and direct transmission. The proposed hybrid scheme is dynamically adapted based on the channel state information. The integration and optimization of these three strategies provide a more generic and fundamental solution and give better achievable rates than the known schemes in the literature. Despite the large number of optimization variables, the proposed hybrid scheme can be optimized using simple closed-form formulas that are easy to apply in practical relay systems. This includes adjusting the transmission rate and compression when compress-and-forward is the selected strategy based on the channel conditions. Furthermore, in this paper, the hybrid scheme is applied to three different models of the Gaussian block-fading buffer-aided relay channels, depending on whether the relay is half or full duplex and whether the source and the relay have orthogonal or non-orthogonal channel access. Several numerical examples are provided to demonstrate the achievable rate results and compare them to the upper bounds of the ergodic capacity for each one of the three channel models under consideration.
On the Secrecy Capacity Region of the Block-Fading BCC with Limited CSI Feedback
Hyadi, Amal
2017-02-07
In this work, we examine the secrecy capacity region of the block-fading broadcast channel with confidential messages (BCC) when the transmitter has limited knowledge of the channel. In particular, we consider a two-user communication system where the transmitter has one common message to be transmitted to both users and one confidential message intended to only one of them. The confidential message has to be kept secret from the other user to whom the information is not intended. The transmitter is not aware of the channel state information (CSI) of neither channel and is only provided by limited CSI feedback sent at the beginning of each fading block. Assuming an error-free feedback link, we characterize the secrecy capacity region of this channel and show that even with a 1-bit CSI feedback, a positive secrecy rate can still be achieved. Then, we look at the case where the feedback link is not error- free and is rather a binary erasure channel (BEC). In the latter case, we provide an achievable secrecy rate region and show that as long as the erasure event is not a probability 1 event, the transmitter can still transmit the confidential information with a positive secrecy rate.
Maximizing Expected Achievable Rates for Block-Fading Buffer-Aided Relay Channels
Shaqfeh, Mohammad; Zafar, Ammar; Alnuweiri, Hussein; Alouini, Mohamed-Slim
2016-01-01
In this paper, the long-term average achievable rate over block-fading buffer-aided relay channels is maximized using a hybrid scheme that combines three essential transmission strategies, which are decode-and-forward, compress-and-forward, and direct transmission. The proposed hybrid scheme is dynamically adapted based on the channel state information. The integration and optimization of these three strategies provide a more generic and fundamental solution and give better achievable rates than the known schemes in the literature. Despite the large number of optimization variables, the proposed hybrid scheme can be optimized using simple closed-form formulas that are easy to apply in practical relay systems. This includes adjusting the transmission rate and compression when compress-and-forward is the selected strategy based on the channel conditions. Furthermore, in this paper, the hybrid scheme is applied to three different models of the Gaussian block-fading buffer-aided relay channels, depending on whether the relay is half or full duplex and whether the source and the relay have orthogonal or non-orthogonal channel access. Several numerical examples are provided to demonstrate the achievable rate results and compare them to the upper bounds of the ergodic capacity for each one of the three channel models under consideration.
A hidden Markov model approach to neuron firing patterns.
Camproux, A C; Saunier, F; Chouvet, G; Thalabard, J C; Thomas, G
1996-11-01
Analysis and characterization of neuronal discharge patterns are of interest to neurophysiologists and neuropharmacologists. In this paper we present a hidden Markov model approach to modeling single neuron electrical activity. Basically the model assumes that each interspike interval corresponds to one of several possible states of the neuron. Fitting the model to experimental series of interspike intervals by maximum likelihood allows estimation of the number of possible underlying neuron states, the probability density functions of interspike intervals corresponding to each state, and the transition probabilities between states. We present an application to the analysis of recordings of a locus coeruleus neuron under three pharmacological conditions. The model distinguishes two states during halothane anesthesia and during recovery from halothane anesthesia, and four states after administration of clonidine. The transition probabilities yield additional insights into the mechanisms of neuron firing.
Road maintenance optimization through a discrete-time semi-Markov decision process
International Nuclear Information System (INIS)
Zhang Xueqing; Gao Hui
2012-01-01
Optimization models are necessary for efficient and cost-effective maintenance of a road network. In this regard, road deterioration is commonly modeled as a discrete-time Markov process such that an optimal maintenance policy can be obtained based on the Markov decision process, or as a renewal process such that an optimal maintenance policy can be obtained based on the renewal theory. However, the discrete-time Markov process cannot capture the real time at which the state transits while the renewal process considers only one state and one maintenance action. In this paper, road deterioration is modeled as a semi-Markov process in which the state transition has the Markov property and the holding time in each state is assumed to follow a discrete Weibull distribution. Based on this semi-Markov process, linear programming models are formulated for both infinite and finite planning horizons in order to derive optimal maintenance policies to minimize the life-cycle cost of a road network. A hypothetical road network is used to illustrate the application of the proposed optimization models. The results indicate that these linear programming models are practical for the maintenance of a road network having a large number of road segments and that they are convenient to incorporate various constraints on the decision process, for example, performance requirements and available budgets. Although the optimal maintenance policies obtained for the road network are randomized stationary policies, the extent of this randomness in decision making is limited. The maintenance actions are deterministic for most states and the randomness in selecting actions occurs only for a few states.
failure analysis of a uav flight control system using markov analysis
African Journals Online (AJOL)
Failure analysis of a flight control system proposed for Air Force Institute of Technology (AFIT) Unmanned Aerial Vehicle (UAV) was studied using Markov Analysis (MA). It was perceived that understanding of the number of failure states and the probability of being in those state are of paramount importance in order to ...
Quasi-stationary distributions for reducible absorbing Markov chains in discrete time
van Doorn, Erik A.; Pollett, P.K.
2009-01-01
We consider discrete-time Markov chains with one coffin state and a finite set $S$ of transient states, and are interested in the limiting behaviour of such a chain as time $n \\to \\infty,$ conditional on survival up to $n$. It is known that, when $S$ is irreducible, the limiting conditional
Zipf exponent of trajectory distribution in the hidden Markov model
Bochkarev, V. V.; Lerner, E. Yu
2014-03-01
This paper is the first step of generalization of the previously obtained full classification of the asymptotic behavior of the probability for Markov chain trajectories for the case of hidden Markov models. The main goal is to study the power (Zipf) and nonpower asymptotics of the frequency list of trajectories of hidden Markov frequencys and to obtain explicit formulae for the exponent of the power asymptotics. We consider several simple classes of hidden Markov models. We prove that the asymptotics for a hidden Markov model and for the corresponding Markov chain can be essentially different.
Zipf exponent of trajectory distribution in the hidden Markov model
International Nuclear Information System (INIS)
Bochkarev, V V; Lerner, E Yu
2014-01-01
This paper is the first step of generalization of the previously obtained full classification of the asymptotic behavior of the probability for Markov chain trajectories for the case of hidden Markov models. The main goal is to study the power (Zipf) and nonpower asymptotics of the frequency list of trajectories of hidden Markov frequencys and to obtain explicit formulae for the exponent of the power asymptotics. We consider several simple classes of hidden Markov models. We prove that the asymptotics for a hidden Markov model and for the corresponding Markov chain can be essentially different
Zhao, Zhibiao
2011-06-01
We address the nonparametric model validation problem for hidden Markov models with partially observable variables and hidden states. We achieve this goal by constructing a nonparametric simultaneous confidence envelope for transition density function of the observable variables and checking whether the parametric density estimate is contained within such an envelope. Our specification test procedure is motivated by a functional connection between the transition density of the observable variables and the Markov transition kernel of the hidden states. Our approach is applicable for continuous time diffusion models, stochastic volatility models, nonlinear time series models, and models with market microstructure noise.
Un calcul de Viterbi pour un Modèle de Markov Caché Contraint
DEFF Research Database (Denmark)
Petit, Matthieu; Christiansen, Henning
2009-01-01
A hidden Markov model (HMM) is a statistical model in which the system being modeled is assumed to be a Markov process with hidden states. This model has been widely used in speech recognition and biological sequence analysis. Viterbi algorithm has been proposed to compute the most probable value....... Several constraint techniques are used to reduce the search of the most probable value of hidden states of a constrained HMM. An implementation based on PRISM, a logic programming language for statistical modeling, is presented....
Discrete-time semi-Markov modeling of human papillomavirus persistence
Mitchell, C. E.; Hudgens, M. G.; King, C. C.; Cu-Uvin, S.; Lo, Y.; Rompalo, A.; Sobel, J.; Smith, J. S.
2011-01-01
Multi-state modeling is often employed to describe the progression of a disease process. In epidemiological studies of certain diseases, the disease state is typically only observed at periodic clinical visits, producing incomplete longitudinal data. In this paper we consider fitting semi-Markov models to estimate the persistence of human papillomavirus (HPV) type-specific infection in studies where the status of HPV type(s) is assessed periodically. Simulation study results are presented indicating the semi-Markov estimator is more accurate than an estimator currently used in the HPV literature. The methods are illustrated using data from the HIV Epidemiology Research Study (HERS). PMID:21538985
Generating intrinsically disordered protein conformational ensembles from a Markov chain
Cukier, Robert I.
2018-03-01
Intrinsically disordered proteins (IDPs) sample a diverse conformational space. They are important to signaling and regulatory pathways in cells. An entropy penalty must be payed when an IDP becomes ordered upon interaction with another protein or a ligand. Thus, the degree of conformational disorder of an IDP is of interest. We create a dichotomic Markov model that can explore entropic features of an IDP. The Markov condition introduces local (neighbor residues in a protein sequence) rotamer dependences that arise from van der Waals and other chemical constraints. A protein sequence of length N is characterized by its (information) entropy and mutual information, MIMC, the latter providing a measure of the dependence among the random variables describing the rotamer probabilities of the residues that comprise the sequence. For a Markov chain, the MIMC is proportional to the pair mutual information MI which depends on the singlet and pair probabilities of neighbor residue rotamer sampling. All 2N sequence states are generated, along with their probabilities, and contrasted with the probabilities under the assumption of independent residues. An efficient method to generate realizations of the chain is also provided. The chain entropy, MIMC, and state probabilities provide the ingredients to distinguish different scenarios using the terminologies: MoRF (molecular recognition feature), not-MoRF, and not-IDP. A MoRF corresponds to large entropy and large MIMC (strong dependence among the residues' rotamer sampling), a not-MoRF corresponds to large entropy but small MIMC, and not-IDP corresponds to low entropy irrespective of the MIMC. We show that MorFs are most appropriate as descriptors of IDPs. They provide a reasonable number of high-population states that reflect the dependences between neighbor residues, thus classifying them as IDPs, yet without very large entropy that might lead to a too high entropy penalty.
Markov dynamic models for long-timescale protein motion.
Chiang, Tsung-Han
2010-06-01
Molecular dynamics (MD) simulation is a well-established method for studying protein motion at the atomic scale. However, it is computationally intensive and generates massive amounts of data. One way of addressing the dual challenges of computation efficiency and data analysis is to construct simplified models of long-timescale protein motion from MD simulation data. In this direction, we propose to use Markov models with hidden states, in which the Markovian states represent potentially overlapping probabilistic distributions over protein conformations. We also propose a principled criterion for evaluating the quality of a model by its ability to predict long-timescale protein motions. Our method was tested on 2D synthetic energy landscapes and two extensively studied peptides, alanine dipeptide and the villin headpiece subdomain (HP-35 NleNle). One interesting finding is that although a widely accepted model of alanine dipeptide contains six states, a simpler model with only three states is equally good for predicting long-timescale motions. We also used the constructed Markov models to estimate important kinetic and dynamic quantities for protein folding, in particular, mean first-passage time. The results are consistent with available experimental measurements.
The application of Markov decision process in restaurant delivery robot
Wang, Yong; Hu, Zhen; Wang, Ying
2017-05-01
As the restaurant delivery robot is often in a dynamic and complex environment, including the chairs inadvertently moved to the channel and customers coming and going. The traditional path planning algorithm is not very ideal. To solve this problem, this paper proposes the Markov dynamic state immediate reward (MDR) path planning algorithm according to the traditional Markov decision process. First of all, it uses MDR to plan a global path, then navigates along this path. When the sensor detects there is no obstructions in front state, increase its immediate state reward value; when the sensor detects there is an obstacle in front, plan a global path that can avoid obstacle with the current position as the new starting point and reduce its state immediate reward value. This continues until the target is reached. When the robot learns for a period of time, it can avoid those places where obstacles are often present when planning the path. By analyzing the simulation experiment, the algorithm has achieved good results in the global path planning under the dynamic environment.
Markov dynamic models for long-timescale protein motion.
Chiang, Tsung-Han; Hsu, David; Latombe, Jean-Claude
2010-01-01
Molecular dynamics (MD) simulation is a well-established method for studying protein motion at the atomic scale. However, it is computationally intensive and generates massive amounts of data. One way of addressing the dual challenges of computation efficiency and data analysis is to construct simplified models of long-timescale protein motion from MD simulation data. In this direction, we propose to use Markov models with hidden states, in which the Markovian states represent potentially overlapping probabilistic distributions over protein conformations. We also propose a principled criterion for evaluating the quality of a model by its ability to predict long-timescale protein motions. Our method was tested on 2D synthetic energy landscapes and two extensively studied peptides, alanine dipeptide and the villin headpiece subdomain (HP-35 NleNle). One interesting finding is that although a widely accepted model of alanine dipeptide contains six states, a simpler model with only three states is equally good for predicting long-timescale motions. We also used the constructed Markov models to estimate important kinetic and dynamic quantities for protein folding, in particular, mean first-passage time. The results are consistent with available experimental measurements.
Directory of Open Access Journals (Sweden)
David Overbye
2005-06-01
Full Text Available In this paper we examine the impact of channel fading on the bit error rate of a DS-CDMA communication system. The system employs detectors that incorporate neural networks effecting methods of independent component analysis (ICA, subspace estimation of channel noise, and Hopfield type neural networks. The Rayleigh fading channel model is used. When employed in a Rayleigh fading environment, the ICA neural network detectors that give superior performance in a flat fading channel did not retain this superior performance. We then present a new method of compensating for channel fading based on the incorporation of priors in the ICA neural network learning algorithms. When the ICA neural network detectors were compensated using the incorporation of priors, they give significantly better performance than the traditional detectors and the uncompensated ICA detectors. Keywords: CDMA, Multi-user Detection, Rayleigh Fading, Multipath Detection, Independent Component Analysis, Prior Probability Hebbian Learning, Natural Gradient
Lei, Hongjiang; Zhang, Huan; Ansari, Imran Shafique; Ren, Zhi; Pan, Gaofeng; Qaraqe, Khalid A.; Alouini, Mohamed-Slim
2017-01-01
In this paper, the secrecy outage performance of an underlay cognitive decode-and-forward relay network over independent but not necessarily identical distributed (i.n.i.d) Nakagami-m fading channels is investigated, in which the secondary user transmitter communicates with the secondary destination via relays, and an eavesdropper attempts to overhear the information. Based on whether the channel state information (CSI) of the wiretap links is available or not, we analyze the secrecy outage performance with optimal relay selection (ORS) and suboptimal relay selection (SRS) schemes, and multiple relays combining scheme (MRC) scheme is considered for comparison purpose. The exact and asymptotic closed-form expressions for the secrecy outage probability with three different relay selection schemes are derived and verified by Monte-Carlo simulations. The numerical results illustrate that ORS scheme always outperforms SRS and MRC schemes, and SRS scheme is better than MRC scheme in the lower fading parameters scenario. Furthermore, through asymptotic analysis, we find that these three different schemes achieve the same secrecy diversity order, which is determined by the number of the relays, and the fading parameters of the links among the relays and the destination.
Lei, Hongjiang
2017-10-02
In this paper, the secrecy outage performance of an underlay cognitive decode-and-forward relay network over independent but not necessarily identical distributed (i.n.i.d) Nakagami-m fading channels is investigated, in which the secondary user transmitter communicates with the secondary destination via relays, and an eavesdropper attempts to overhear the information. Based on whether the channel state information (CSI) of the wiretap links is available or not, we analyze the secrecy outage performance with optimal relay selection (ORS) and suboptimal relay selection (SRS) schemes, and multiple relays combining scheme (MRC) scheme is considered for comparison purpose. The exact and asymptotic closed-form expressions for the secrecy outage probability with three different relay selection schemes are derived and verified by Monte-Carlo simulations. The numerical results illustrate that ORS scheme always outperforms SRS and MRC schemes, and SRS scheme is better than MRC scheme in the lower fading parameters scenario. Furthermore, through asymptotic analysis, we find that these three different schemes achieve the same secrecy diversity order, which is determined by the number of the relays, and the fading parameters of the links among the relays and the destination.
Algebraic decay in self-similar Markov chains
International Nuclear Information System (INIS)
Hanson, J.D.; Cary, J.R.; Meiss, J.D.
1985-01-01
A continuous-time Markov chain is used to model motion in the neighborhood of a critical invariant circle for a Hamiltonian map. States in the infinite chain represent successive rational approximants to the frequency of the invariant circle. For the case of a noble frequency, the chain is self-similar and the nonlinear integral equation for the first passage time distribution is solved exactly. The asymptotic distribution is a power law times a function periodic in the logarithm of the time. For parameters relevant to the critical noble circle, the decay proceeds as t/sup -4.05/
Absolute continuity of the distribution of some Markov geometric series
Institute of Scientific and Technical Information of China (English)
Ai-hua; FAN; Ji-hong; ZHANG
2007-01-01
Let (∈n)≥0 be the Markov chain of two states with respect to the probability measure of the maximal entropy on the subshift space ∑A defined by Fibonacci incident matrix A.We consider the measure μλ of the probability distribution of the random series ∑∞n=0 εnλn (0 ＜λ＜ 1).It is proved that μλ is singular if λ∈ (0,√5-1/2) and that μλ is absolutely continuous for almost all λ∈ (√5-1/2,0.739).
On Construction of Quantum Markov Chains on Cayley trees
International Nuclear Information System (INIS)
Accardi, Luigi; Mukhamedov, Farrukh; Souissi, Abdessatar
2016-01-01
The main aim of the present paper is to provide a new construction of quantum Markov chain (QMC) on arbitrary order Cayley tree. In that construction, a QMC is defined as a weak limit of finite volume states with boundary conditions, i.e. QMC depends on the boundary conditions. Note that this construction reminds statistical mechanics models with competing interactions on trees. If one considers one dimensional tree, then the provided construction reduces to well-known one, which was studied by the first author. Our construction will allow to investigate phase transition problem in a quantum setting. (paper)
Algebraic decay in self-similar Markov chains
International Nuclear Information System (INIS)
Hanson, J.D.; Cary, J.R.; Meiss, J.D.
1984-10-01
A continuous time Markov chain is used to model motion in the neighborhood of a critical noble invariant circle in an area-preserving map. States in the infinite chain represent successive rational approximants to the frequency of the invariant circle. The nonlinear integral equation for the first passage time distribution is solved exactly. The asymptotic distribution is a power law times a function periodic in the logarithm of the time. For parameters relevant to Hamiltonian systems the decay proceeds as t -4 05
APPLICATION OF HIDDEN MARKOV CHAINS IN QUALITY CONTROL
Directory of Open Access Journals (Sweden)
Hanife DEMIRALP
2013-01-01
Full Text Available The ever growing technological innovations and sophistication in industrial processes require adequate checks on quality. Thus, there is an increasing demand for simple and efficient quality control methods. In this regard the control charts stand out in simplicity and efficiency. In this paper, we propose a method of controlling quality based on the theory of hidden Markov chains. Based on samples drawn at different times from the production process, the method obtains the state of the process probabilistically. The main advantage of the method is that it requires no assumption on the normality of the process output.
Hidden-Markov-Model Analysis Of Telemanipulator Data
Hannaford, Blake; Lee, Paul
1991-01-01
Mathematical model and procedure based on hidden-Markov-model concept undergoing development for use in analysis and prediction of outputs of force and torque sensors of telerobotic manipulators. In model, overall task broken down into subgoals, and transition probabilities encode ease with which operator completes each subgoal. Process portion of model encodes task-sequence/subgoal structure, and probability-density functions for forces and torques associated with each state of manipulation encode sensor signals that one expects to observe at subgoal. Parameters of model constructed from engineering knowledge of task.
International Nuclear Information System (INIS)
Ben-Shachar, B.; German, U.; Weiser, G.
1983-03-01
The fading of LiF and CaF 2 :Dy was investigated and the results were compared to the literature. The effect of thermal annealing was studied in order to reduce the fading in both phosphors and to minimize the effects of the environment on CaF 2 :Dy. Minimizing the fading and knowing its time dependence make possible the exact personal and environmental dosimetry. (Author)
Projected metastable Markov processes and their estimation with observable operator models
International Nuclear Information System (INIS)
Wu, Hao; Prinz, Jan-Hendrik; Noé, Frank
2015-01-01
The determination of kinetics of high-dimensional dynamical systems, such as macromolecules, polymers, or spin systems, is a difficult and generally unsolved problem — both in simulation, where the optimal reaction coordinate(s) are generally unknown and are difficult to compute, and in experimental measurements, where only specific coordinates are observable. Markov models, or Markov state models, are widely used but suffer from the fact that the dynamics on a coarsely discretized state spaced are no longer Markovian, even if the dynamics in the full phase space are. The recently proposed projected Markov models (PMMs) are a formulation that provides a description of the kinetics on a low-dimensional projection without making the Markovianity assumption. However, as yet no general way of estimating PMMs from data has been available. Here, we show that the observed dynamics of a PMM can be exactly described by an observable operator model (OOM) and derive a PMM estimator based on the OOM learning
Long memory of financial time series and hidden Markov models with time-varying parameters
DEFF Research Database (Denmark)
Nystrup, Peter; Madsen, Henrik; Lindström, Erik
Hidden Markov models are often used to capture stylized facts of daily returns and to infer the hidden state of financial markets. Previous studies have found that the estimated models change over time, but the implications of the time-varying behavior for the ability to reproduce the stylized...... facts have not been thoroughly examined. This paper presents an adaptive estimation approach that allows for the parameters of the estimated models to be time-varying. It is shown that a two-state Gaussian hidden Markov model with time-varying parameters is able to reproduce the long memory of squared...... daily returns that was previously believed to be the most difficult fact to reproduce with a hidden Markov model. Capturing the time-varying behavior of the parameters also leads to improved one-step predictions....
Peng, Zhihang; Bao, Changjun; Zhao, Yang; Yi, Honggang; Xia, Letian; Yu, Hao; Shen, Hongbing; Chen, Feng
2010-01-01
This paper first applies the sequential cluster method to set up the classification standard of infectious disease incidence state based on the fact that there are many uncertainty characteristics in the incidence course. Then the paper presents a weighted Markov chain, a method which is used to predict the future incidence state. This method assumes the standardized self-coefficients as weights based on the special characteristics of infectious disease incidence being a dependent stochastic variable. It also analyzes the characteristics of infectious diseases incidence via the Markov chain Monte Carlo method to make the long-term benefit of decision optimal. Our method is successfully validated using existing incidents data of infectious diseases in Jiangsu Province. In summation, this paper proposes ways to improve the accuracy of the weighted Markov chain, specifically in the field of infection epidemiology. PMID:23554632
Peng, Zhihang; Bao, Changjun; Zhao, Yang; Yi, Honggang; Xia, Letian; Yu, Hao; Shen, Hongbing; Chen, Feng
2010-05-01
This paper first applies the sequential cluster method to set up the classification standard of infectious disease incidence state based on the fact that there are many uncertainty characteristics in the incidence course. Then the paper presents a weighted Markov chain, a method which is used to predict the future incidence state. This method assumes the standardized self-coefficients as weights based on the special characteristics of infectious disease incidence being a dependent stochastic variable. It also analyzes the characteristics of infectious diseases incidence via the Markov chain Monte Carlo method to make the long-term benefit of decision optimal. Our method is successfully validated using existing incidents data of infectious diseases in Jiangsu Province. In summation, this paper proposes ways to improve the accuracy of the weighted Markov chain, specifically in the field of infection epidemiology.
Stochastic modeling of pitting corrosion in underground pipelines using Markov chains
Energy Technology Data Exchange (ETDEWEB)
Velazquez, J.C.; Caleyo, F.; Hallen, J.M.; Araujo, J.E. [Instituto Politecnico Nacional (IPN), Mexico D.F. (Mexico). Escuela Superior de Ingenieria Quimica e Industrias Extractivas (ESIQIE); Valor, A. [Universidad de La Habana, La Habana (Cuba)
2009-07-01
A non-homogenous, linear growth (pure birth) Markov process, with discrete states in continuous time, has been used to model external pitting corrosion in underground pipelines. The transition probability function for the pit depth is obtained from the analytical solution of the forward Kolmogorov equations for this process. The parameters of the transition probability function between depth states can be identified from the observed time evolution of the mean of the pit depth distribution. Monte Carlo simulations were used to predict the time evolution of the mean value of the pit depth distribution in soils with different physicochemical characteristics. The simulated distributions have been used to create an empirical Markov-chain-based stochastic model for predicting the evolution of pitting corrosion from the observed properties of the soil in contact with the pipeline. Real- life case studies, involving simulated and measured pit depth distributions are presented to illustrate the application of the proposed Markov chains model. (author)
OPRA capacity bounds for selection diversity over generalized fading channels
Hanif, Muhammad Fainan
2014-05-01
Channel side information at the transmitter can increase the average capacity by enabling optimal power and rate adaptation. The resulting optimal power and rate adaptation (OPRA) capacity rarely has a closed-form analytic expression. In this paper, lower and upper bounds on OPRA capacity for selection diversity scheme are presented. These bounds hold for variety of fading channels including log-normal and generalized Gamma distributed models and have very simple analytic expressions for easy evaluation even for kth best path selection. Some selected numerical results show that the newly proposed bounds closely approximate the actual OPRA capacity. © 2014 IEEE.
Fade statistics of M-turbulent optical links
DEFF Research Database (Denmark)
Jurado-Navas, Antonio; Maria Garrido-Balsells, Jose; Castillo-Vazquez, Miguel
2017-01-01
A new and generalized statistical model, called Malaga or simply M distribution, has been derived recently to characterize the irradiance fluctuations of an unbounded optical wavefront propagating through a turbulent medium under all irradiance fluctuation conditions. The aforementioned model...... extends and unifies in a simple analytical closed-form expression most of the proposed statistical models for free-space optical (FSO) communications widely employed until now in the scientific literature. Based on that M model, we have studied some important features associated to its fade statistics...
Markov decision processes in artificial intelligence
Sigaud, Olivier
2013-01-01
Markov Decision Processes (MDPs) are a mathematical framework for modeling sequential decision problems under uncertainty as well as Reinforcement Learning problems. Written by experts in the field, this book provides a global view of current research using MDPs in Artificial Intelligence. It starts with an introductory presentation of the fundamental aspects of MDPs (planning in MDPs, Reinforcement Learning, Partially Observable MDPs, Markov games and the use of non-classical criteria). Then it presents more advanced research trends in the domain and gives some concrete examples using illustr
Markov bridges, bisection and variance reduction
DEFF Research Database (Denmark)
Asmussen, Søren; Hobolth, Asger
. In this paper we firstly consider the problem of generating sample paths from a continuous-time Markov chain conditioned on the endpoints using a new algorithm based on the idea of bisection. Secondly we study the potential of the bisection algorithm for variance reduction. In particular, examples are presented......Time-continuous Markov jump processes is a popular modelling tool in disciplines ranging from computational finance and operations research to human genetics and genomics. The data is often sampled at discrete points in time, and it can be useful to simulate sample paths between the datapoints...
Inhomogeneous Markov Models for Describing Driving Patterns
DEFF Research Database (Denmark)
Iversen, Emil Banning; Møller, Jan K.; Morales, Juan Miguel
2017-01-01
. Specifically, an inhomogeneous Markov model that captures the diurnal variation in the use of a vehicle is presented. The model is defined by the time-varying probabilities of starting and ending a trip, and is justified due to the uncertainty associated with the use of the vehicle. The model is fitted to data...... collected from the actual utilization of a vehicle. Inhomogeneous Markov models imply a large number of parameters. The number of parameters in the proposed model is reduced using B-splines....
Inhomogeneous Markov Models for Describing Driving Patterns
DEFF Research Database (Denmark)
Iversen, Jan Emil Banning; Møller, Jan Kloppenborg; Morales González, Juan Miguel
. Specically, an inhomogeneous Markov model that captures the diurnal variation in the use of a vehicle is presented. The model is dened by the time-varying probabilities of starting and ending a trip and is justied due to the uncertainty associated with the use of the vehicle. The model is tted to data...... collected from the actual utilization of a vehicle. Inhomogeneous Markov models imply a large number of parameters. The number of parameters in the proposed model is reduced using B-splines....
Predicting Protein Secondary Structure with Markov Models
DEFF Research Database (Denmark)
Fischer, Paul; Larsen, Simon; Thomsen, Claus
2004-01-01
we are considering here, is to predict the secondary structure from the primary one. To this end we train a Markov model on training data and then use it to classify parts of unknown protein sequences as sheets, helices or coils. We show how to exploit the directional information contained...... in the Markov model for this task. Classifications that are purely based on statistical models might not always be biologically meaningful. We present combinatorial methods to incorporate biological background knowledge to enhance the prediction performance....
Markov processes an introduction for physical scientists
Gillespie, Daniel T
1991-01-01
Markov process theory is basically an extension of ordinary calculus to accommodate functions whos time evolutions are not entirely deterministic. It is a subject that is becoming increasingly important for many fields of science. This book develops the single-variable theory of both continuous and jump Markov processes in a way that should appeal especially to physicists and chemists at the senior and graduate level.Key Features* A self-contained, prgamatic exposition of the needed elements of random variable theory* Logically integrated derviations of the Chapman-Kolmogorov e
National Research Council Canada - National Science Library
Ong, Choon
1998-01-01
The performance analysis of a differential phase shift keyed (DPSK) communications system, operating in a Rayleigh fading environment, employing convolutional coding and diversity processing is presented...
Performance Analysis of Rayleigh Fading and Cochannel Interference in Wireless Communication
National Research Council Canada - National Science Library
Gao, Chunjun
2000-01-01
...) performance of adaptive arrays for wireless communications over fading channels in the presence of cochannel interference, particularly the case when the number of interference sources exceeds...
An MGF-based capacity analysis of equal gain combining over fading channels
Yilmaz, Ferkan
2010-09-01
Exact average capacity results for L-branch coherent equal-gain combining (EGC) in correlated and uncorrelated fading channels are not known. This paper develops a novel framework (approach) for capacity analysis of L-branch EGC in generalized fading channels. In addition, Gamma shadowed generalized Nakagami-m fading model is proposed in order to statistically model the fading environments in high frequencies such as 60 GHz and above. Some simulations are carried out and then the obtained results are accentuated by means of numerical analysis based on the proposed framework. Numerical and simulation results, performed to verify the correctness of the proposed framework, are in perfect agreement. ©2010 IEEE.
An MGF-based capacity analysis of equal gain combining over fading channels
Yilmaz, Ferkan; Alouini, Mohamed-Slim
2010-01-01
Exact average capacity results for L-branch coherent equal-gain combining (EGC) in correlated and uncorrelated fading channels are not known. This paper develops a novel framework (approach) for capacity analysis of L-branch EGC in generalized fading channels. In addition, Gamma shadowed generalized Nakagami-m fading model is proposed in order to statistically model the fading environments in high frequencies such as 60 GHz and above. Some simulations are carried out and then the obtained results are accentuated by means of numerical analysis based on the proposed framework. Numerical and simulation results, performed to verify the correctness of the proposed framework, are in perfect agreement. ©2010 IEEE.
van Rosmalen, Joost; Toy, Mehlika; O'Mahony, James F
2013-08-01
Markov models are a simple and powerful tool for analyzing the health and economic effects of health care interventions. These models are usually evaluated in discrete time using cohort analysis. The use of discrete time assumes that changes in health states occur only at the end of a cycle period. Discrete-time Markov models only approximate the process of disease progression, as clinical events typically occur in continuous time. The approximation can yield biased cost-effectiveness estimates for Markov models with long cycle periods and if no half-cycle correction is made. The purpose of this article is to present an overview of methods for evaluating Markov models in continuous time. These methods use mathematical results from stochastic process theory and control theory. The methods are illustrated using an applied example on the cost-effectiveness of antiviral therapy for chronic hepatitis B. The main result is a mathematical solution for the expected time spent in each state in a continuous-time Markov model. It is shown how this solution can account for age-dependent transition rates and discounting of costs and health effects, and how the concept of tunnel states can be used to account for transition rates that depend on the time spent in a state. The applied example shows that the continuous-time model yields more accurate results than the discrete-time model but does not require much computation time and is easily implemented. In conclusion, continuous-time Markov models are a feasible alternative to cohort analysis and can offer several theoretical and practical advantages.
Mixed Vehicle Flow At Signalized Intersection: Markov Chain Analysis
Directory of Open Access Journals (Sweden)
Gertsbakh Ilya B.
2015-09-01
Full Text Available We assume that a Poisson flow of vehicles arrives at isolated signalized intersection, and each vehicle, independently of others, represents a random number X of passenger car units (PCU’s. We analyze numerically the stationary distribution of the queue process {Zn}, where Zn is the number of PCU’s in a queue at the beginning of the n-th red phase, n → ∞. We approximate the number Yn of PCU’s arriving during one red-green cycle by a two-parameter Negative Binomial Distribution (NBD. The well-known fact is that {Zn} follow an infinite-state Markov chain. We approximate its stationary distribution using a finite-state Markov chain. We show numerically that there is a strong dependence of the mean queue length E[Zn] in equilibrium on the input distribution of Yn and, in particular, on the ”over dispersion” parameter γ= Var[Yn]/E[Yn]. For Poisson input, γ = 1. γ > 1 indicates presence of heavy-tailed input. In reality it means that a relatively large ”portion” of PCU’s, considerably exceeding the average, may arrive with high probability during one red-green cycle. Empirical formulas are presented for an accurate estimation of mean queue length as a function of load and g of the input flow. Using the Markov chain technique, we analyze the mean ”virtual” delay time for a car which always arrives at the beginning of the red phase.
Hidden Markov latent variable models with multivariate longitudinal data.
Song, Xinyuan; Xia, Yemao; Zhu, Hongtu
2017-03-01
Cocaine addiction is chronic and persistent, and has become a major social and health problem in many countries. Existing studies have shown that cocaine addicts often undergo episodic periods of addiction to, moderate dependence on, or swearing off cocaine. Given its reversible feature, cocaine use can be formulated as a stochastic process that transits from one state to another, while the impacts of various factors, such as treatment received and individuals' psychological problems on cocaine use, may vary across states. This article develops a hidden Markov latent variable model to study multivariate longitudinal data concerning cocaine use from a California Civil Addict Program. The proposed model generalizes conventional latent variable models to allow bidirectional transition between cocaine-addiction states and conventional hidden Markov models to allow latent variables and their dynamic interrelationship. We develop a maximum-likelihood approach, along with a Monte Carlo expectation conditional maximization (MCECM) algorithm, to conduct parameter estimation. The asymptotic properties of the parameter estimates and statistics for testing the heterogeneity of model parameters are investigated. The finite sample performance of the proposed methodology is demonstrated by simulation studies. The application to cocaine use study provides insights into the prevention of cocaine use. © 2016, The International Biometric Society.
Markov analysis of different standby computer based systems
International Nuclear Information System (INIS)
Srinivas, G.; Guptan, Rajee; Mohan, Nalini; Ghadge, S.G.; Bajaj, S.S.
2006-01-01
As against the conventional triplicated systems of hardware and the generation of control signals for the actuator elements by means of redundant hardwired median circuits, employed in the early Indian PHWR's, a new approach of generating control signals based on software by a redundant system of computers is introduced in the advanced/current generation of Indian PHWR's. Reliability is increased by fault diagnostics and automatic switch over of all the loads to one computer in case of total failure of the other computer. Independent processing by a redundant CPU in each system enables inter-comparison to quickly identify system failure, in addition to the other self-diagnostic features provided. Combinatorial models such as reliability block diagrams and fault trees are frequently used to predict the reliability, maintainability and safety of complex systems. Unfortunately, these methods cannot accurately model dynamic system behavior; Because of its unique ability to handle dynamic cases, Markov analysis can be a powerful tool in the reliability maintainability and safety (RMS) analyses of dynamic systems. A Markov model breaks the system configuration into a number of states. Each of these states is connected to all other states by transition rates. It then utilizes transition matrices to evaluate the reliability and safety of the systems, either through matrix manipulation or other analytical solution methods, such as Laplace transforms. Thus, Markov analysis is a powerful reliability, maintainability and safety analysis tool. It allows the analyst to model complex, dynamic, highly distributed, fault tolerant systems that would otherwise be very difficult to model using classical techniques like the Fault tree method. The Dual Processor Hot Standby Process Control System (DPHS-PCS) and the Computerized Channel Temperature Monitoring System (CCTM) are typical examples of hot standby systems in the Indian PHWR's. While such systems currently in use in Indian PHWR
Efficient incremental relaying for packet transmission over fading channels
Fareed, Muhammad Mehboob
2014-07-01
In this paper, we propose a novel relaying scheme for packet transmission over fading channels, which improves the spectral efficiency of cooperative diversity systems by utilizing limited feedback from the destination. Our scheme capitalizes on the fact that relaying is only required when direct transmission suffers deep fading. We calculate the packet error rate for the proposed efficient incremental relaying (EIR) scheme with both amplify and forward and decode and forward relaying. We compare the performance of the EIR scheme with the threshold-based incremental relaying (TIR) scheme. It is shown that the efficiency of the TIR scheme is better for lower values of the threshold. However, the efficiency of the TIR scheme for higher values of threshold is outperformed by the EIR. In addition, three new threshold-based adaptive EIR are devised to further improve the efficiency of the EIR scheme. We calculate the packet error rate and the efficiency of these new schemes to provide the analytical insight. © 2014 IEEE.
The fading of irradiated blue-colored pearls
International Nuclear Information System (INIS)
Okamoto, Shinichi
1982-01-01
The fading of irradiated and natural blue-colored pearls was investigated in this experiment. Thirty natural blue-colored pearls and sixty irradiated blue-colored pearls were used. Some of them were placed at a light position of RT. Another pearls were placed at a dark position of 50 0 C. The irradiated pearls placed at a light position of RT didn't show remarkable fading in their color in 294 days. But the natural blue-colored pearls showed a little recovery from 4% to 8% in reflection factors in 223 days at RT. The irradiated pearls placed at a dark position of 50 0 C showed the recovery from 9% to 14% in 264 days independently of irradiation times. The natural blue-colored pearls also showed the bleaching from 5% to 10% in reflection factor in 86 days at 50 0 C. Both irradiated and natural blue-colored pearls hardly showed their remarkable changes in their chromaticities independently of temperatures. (author)
Color transplant for reverse ageing of faded artworks
Del Mastio, A.; Piva, A.; Barni, M.; Cappellini, V.; Stefanini, L.
2008-02-01
Nowadays, photographs are one of the most used media for communication. Images are used for the representation of documents, Cultural goods, and so on: they are used to pass on a wedge of historical memory of the society. Since its origin, the photographic technique has got several improvements; nevertheless, photos are liable to several damages, both concerning the physical support and concerning the colors and figures which are depicted in it: for example, think about scratches or rips happened to a photo, or think about the fading or red (or yellow) toning concerning the colors of a photo. In this paper, we propose a novel method which is able to assess the original beauty of digital reproductions of aged photos, as well as digital reproductions of faded goods. The method is based on the comparison of the degraded image with a not-degraded one showing similar contents; thus, the colors of the not-degraded image can be transplanted in the degraded one. The key idea is a dualism between the analytical mechanics and the color theory: for each of the degraded and not-degraded images we compute first a scatter plot of the x and y normalized coordinates of their colors; these scatter diagrams can be regarded as a system of point masses, thus provided of inertia axes and an inertia ellipsoid. Moving the scatter diagram of the degraded image over the one belonging to the not-degraded image, the colors of the degraded image can be restored.
Tackling capacity fading in vanadium flow batteries with amphoteric membranes
Oldenburg, Fabio J.; Schmidt, Thomas J.; Gubler, Lorenz
2017-11-01
Capacity fading and poor electrolyte utilization caused by electrolyte imbalance effects are major drawbacks for the commercialization of vanadium flow batteries (VFB). The influence of membrane type (cationic, anionic, amphoteric) on these effects is studied by determining the excess and net flux of each vanadium ion in an operating VFB assembled with a cation exchange membrane (CEM), Nafion® NR212, an anion exchange membrane (AEM), Fumatech FAP-450, and an amphoteric ion exchange membrane (AIEM) synthesized in-house. It is shown that the net vanadium flux, accompanied by water transport, is directed towards the positive side for the CEM and towards the negative side for the AEM. The content of cation and anion exchange groups in the AIEM is adjusted via radiation grafting to balance the vanadium flux between the two electrolyte sides. With the AIEM the net vanadium flux is significantly reduced and capacity fading due to electrolyte imbalances can be largely eliminated. The membrane's influence on electrolyte imbalance effects is characterized and quantified in one single charge-discharge cycle by analyzing the content of the four different vanadium species in the two electrolytes. The experimental data recorded herewith conclusively explains the electrolyte composition after 80 cycles.
Zhao, Ning-bo; Yang, Jia-long; Li, Shu-ying; Sun, Yue-wu
2014-01-01
Performance degradation forecast technology for quantitatively assessing degradation states of aeroengine using exhaust gas temperature is an important technology in the aeroengine health management. In this paper, a GM (1, 1) Markov chain-based approach is introduced to forecast exhaust gas temperature by taking the advantages of GM (1, 1) model in time series and the advantages of Markov chain model in dealing with highly nonlinear and stochastic data caused by uncertain factors. In this ap...
Sadhana | Indian Academy of Sciences
Indian Academy of Sciences (India)
Finite state Markov channel (FSMC) is the automatic choice for accurate modelling of slow fading channels with memory. FSMC model for a κ–μ fading channel is investigated in this paper. Small-scale variations of the fading signal under Line-Of-Sight conditions are represented by κ–μ fading distributions. Here, FSMC is ...
An information-guided channel-hopping scheme for block-fading channels with estimation errors
Yang, Yuli
2010-12-01
Information-guided channel-hopping technique employing multiple transmit antennas was previously proposed for supporting high data rate transmission over fading channels. This scheme achieves higher data rates than some mature schemes, such as the well-known cyclic transmit antenna selection and space-time block coding, by exploiting the independence character of multiple channels, which effectively results in having an additional information transmitting channel. Moreover, maximum likelihood decoding may be performed by simply decoupling the signals conveyed by the different mapping methods. In this paper, we investigate the achievable spectral efficiency of this scheme in the case of having channel estimation errors, with optimum pilot overhead for minimum meansquare error channel estimation, when transmitting over blockfading channels. Our numerical results further substantiate the robustness of the presented scheme, even with imperfect channel state information. ©2010 IEEE.
Perturbation theory for Markov chains via Wasserstein distance
Rudolf, Daniel; Schweizer, Nikolaus
2017-01-01
Perturbation theory for Markov chains addresses the question of how small differences in the transition probabilities of Markov chains are reflected in differences between their distributions. We prove powerful and flexible bounds on the distance of the nth step distributions of two Markov chains
Soury, Hamza
2012-06-01
This letter considers the average bit error probability of binary coherent signaling over flat fading channels subject to additive generalized Gaussian noise. More specifically, a generic closed form expression in terms of the Fox\\'s H function is offered for the extended generalized-K fading case. Simplifications for some special fading distributions such as generalized-K fading and Nakagami-m fading and special additive noise distributions such as Gaussian and Laplacian noise are then presented. Finally, the mathematical formalism is illustrated by some numerical examples verified by computer based simulations for a variety of fading and additive noise parameters. © 2012 IEEE.
Markov Random Fields on Triangle Meshes
DEFF Research Database (Denmark)
Andersen, Vedrana; Aanæs, Henrik; Bærentzen, Jakob Andreas
2010-01-01
In this paper we propose a novel anisotropic smoothing scheme based on Markov Random Fields (MRF). Our scheme is formulated as two coupled processes. A vertex process is used to smooth the mesh by displacing the vertices according to a MRF smoothness prior, while an independent edge process label...
A Martingale Decomposition of Discrete Markov Chains
DEFF Research Database (Denmark)
Hansen, Peter Reinhard
We consider a multivariate time series whose increments are given from a homogeneous Markov chain. We show that the martingale component of this process can be extracted by a filtering method and establish the corresponding martingale decomposition in closed-form. This representation is useful fo...
Evaluation of Usability Utilizing Markov Models
Penedo, Janaina Rodrigues; Diniz, Morganna; Ferreira, Simone Bacellar Leal; Silveira, Denis S.; Capra, Eliane
2012-01-01
Purpose: The purpose of this paper is to analyze the usability of a remote learning system in its initial development phase, using a quantitative usability evaluation method through Markov models. Design/methodology/approach: The paper opted for an exploratory study. The data of interest of the research correspond to the possible accesses of users…
Bayesian analysis for reversible Markov chains
Diaconis, P.; Rolles, S.W.W.
2006-01-01
We introduce a natural conjugate prior for the transition matrix of a reversible Markov chain. This allows estimation and testing. The prior arises from random walk with reinforcement in the same way the Dirichlet prior arises from Pólya’s urn. We give closed form normalizing constants, a simple
Bisimulation and Simulation Relations for Markov Chains
Baier, Christel; Hermanns, H.; Katoen, Joost P.; Wolf, Verena; Aceto, L.; Gordon, A.
2006-01-01
Formal notions of bisimulation and simulation relation play a central role for any kind of process algebra. This short paper sketches the main concepts for bisimulation and simulation relations for probabilistic systems, modelled by discrete- or continuous-time Markov chains.
Hidden Markov Models for Human Genes
DEFF Research Database (Denmark)
Baldi, Pierre; Brunak, Søren; Chauvin, Yves
1997-01-01
We analyse the sequential structure of human genomic DNA by hidden Markov models. We apply models of widely different design: conventional left-right constructs and models with a built-in periodic architecture. The models are trained on segments of DNA sequences extracted such that they cover com...
Markov Trends in Macroeconomic Time Series
R. Paap (Richard)
1997-01-01
textabstractMany macroeconomic time series are characterised by long periods of positive growth, expansion periods, and short periods of negative growth, recessions. A popular model to describe this phenomenon is the Markov trend, which is a stochastic segmented trend where the slope depends on the
Revisiting Weak Simulation for Substochastic Markov Chains
DEFF Research Database (Denmark)
Jansen, David N.; Song, Lei; Zhang, Lijun
2013-01-01
of the logic PCTL\\x, and its completeness was conjectured. We revisit this result and show that soundness does not hold in general, but only for Markov chains without divergence. It is refuted for some systems with substochastic distributions. Moreover, we provide a counterexample to completeness...
Fracture Mechanical Markov Chain Crack Growth Model
DEFF Research Database (Denmark)
Gansted, L.; Brincker, Rune; Hansen, Lars Pilegaard
1991-01-01
propagation process can be described by a discrete space Markov theory. The model is applicable to deterministic as well as to random loading. Once the model parameters for a given material have been determined, the results can be used for any structure as soon as the geometrical function is known....
Multi-dimensional quasitoeplitz Markov chains
Directory of Open Access Journals (Sweden)
Alexander N. Dudin
1999-01-01
Full Text Available This paper deals with multi-dimensional quasitoeplitz Markov chains. We establish a sufficient equilibrium condition and derive a functional matrix equation for the corresponding vector-generating function, whose solution is given algorithmically. The results are demonstrated in the form of examples and applications in queues with BMAP-input, which operate in synchronous random environment.
Markov Chain Estimation of Avian Seasonal Fecundity
To explore the consequences of modeling decisions on inference about avian seasonal fecundity we generalize previous Markov chain (MC) models of avian nest success to formulate two different MC models of avian seasonal fecundity that represent two different ways to model renestin...
Nonlinearly perturbed semi-Markov processes
Silvestrov, Dmitrii
2017-01-01
The book presents new methods of asymptotic analysis for nonlinearly perturbed semi-Markov processes with a finite phase space. These methods are based on special time-space screening procedures for sequential phase space reduction of semi-Markov processes combined with the systematical use of operational calculus for Laurent asymptotic expansions. Effective recurrent algorithms are composed for getting asymptotic expansions, without and with explicit upper bounds for remainders, for power moments of hitting times, stationary and conditional quasi-stationary distributions for nonlinearly perturbed semi-Markov processes. These results are illustrated by asymptotic expansions for birth-death-type semi-Markov processes, which play an important role in various applications. The book will be a useful contribution to the continuing intensive studies in the area. It is an essential reference for theoretical and applied researchers in the field of stochastic processes and their applications that will cont...
Markov chain of distances between parked cars
International Nuclear Information System (INIS)
Seba, Petr
2008-01-01
We describe the distribution of distances between parked cars as a solution of certain Markov processes and show that its solution is obtained with the help of a distributional fixed point equation. Under certain conditions the process is solved explicitly. The resulting probability density is compared with the actual parking data measured in the city. (fast track communication)
Hidden Markov models for labeled sequences
DEFF Research Database (Denmark)
Krogh, Anders Stærmose
1994-01-01
A hidden Markov model for labeled observations, called a class HMM, is introduced and a maximum likelihood method is developed for estimating the parameters of the model. Instead of training it to model the statistics of the training sequences it is trained to optimize recognition. It resembles MMI...
Efficient Modelling and Generation of Markov Automata
Koutny, M.; Timmer, Mark; Ulidowski, I.; Katoen, Joost P.; van de Pol, Jan Cornelis; Stoelinga, Mariëlle Ida Antoinette
This paper introduces a framework for the efficient modelling and generation of Markov automata. It consists of (1) the data-rich process-algebraic language MAPA, allowing concise modelling of systems with nondeterminism, probability and Markovian timing; (2) a restricted form of the language, the
A Metrized Duality Theorem for Markov Processes
DEFF Research Database (Denmark)
Kozen, Dexter; Mardare, Radu Iulian; Panangaden, Prakash
2014-01-01
We extend our previous duality theorem for Markov processes by equipping the processes with a pseudometric and the algebras with a notion of metric diameter. We are able to show that the isomorphisms of our previous duality theorem become isometries in this quantitative setting. This opens the wa...
Estimation with Right-Censored Observations Under A Semi-Markov Model.
Zhao, Lihui; Hu, X Joan
2013-06-01
The semi-Markov process often provides a better framework than the classical Markov process for the analysis of events with multiple states. The purpose of this paper is twofold. First, we show that in the presence of right censoring, when the right end-point of the support of the censoring time is strictly less than the right end-point of the support of the semi-Markov kernel, the transition probability of the semi-Markov process is nonidentifiable, and the estimators proposed in the literature are inconsistent in general. We derive the set of all attainable values for the transition probability based on the censored data, and we propose a nonparametric inference procedure for the transition probability using this set. Second, the conventional approach to constructing confidence bands is not applicable for the semi-Markov kernel and the sojourn time distribution. We propose new perturbation resampling methods to construct these confidence bands. Different weights and transformations are explored in the construction. We use simulation to examine our proposals and illustrate them with hospitalization data from a recent cancer survivor study.
International Nuclear Information System (INIS)
Son, Kwang Seop; Kim, Dong Hoon; Kim, Chang Hwoi; Kang, Hyun Gook
2016-01-01
The Markov analysis is a technique for modeling system state transitions and calculating the probability of reaching various system states. While it is a proper tool for modeling complex system designs involving timing, sequencing, repair, redundancy, and fault tolerance, as the complexity or size of the system increases, so does the number of states of interest, leading to difficulty in constructing and solving the Markov model. This paper introduces a systematic approach of Markov modeling to analyze the dependability of a complex fault-tolerant system. This method is based on the decomposition of the system into independent subsystem sets, and the system-level failure rate and the unavailability rate for the decomposed subsystems. A Markov model for the target system is easily constructed using the system-level failure and unavailability rates for the subsystems, which can be treated separately. This approach can decrease the number of states to consider simultaneously in the target system by building Markov models of the independent subsystems stage by stage, and results in an exact solution for the Markov model of the whole target system. To apply this method we construct a Markov model for the reactor protection system found in nuclear power plants, a system configured with four identical channels and various fault-tolerant architectures. The results show that the proposed method in this study treats the complex architecture of the system in an efficient manner using the merits of the Markov model, such as a time dependent analysis and a sequential process analysis. - Highlights: • Systematic approach of Markov modeling for system dependability analysis is proposed based on the independent subsystem set, its failure rate and unavailability rate. • As an application example, we construct the Markov model for the digital reactor protection system configured with four identical and independent channels, and various fault-tolerant architectures. • The
Markov chain aggregation and its applications to combinatorial reaction networks.
Ganguly, Arnab; Petrov, Tatjana; Koeppl, Heinz
2014-09-01
We consider a continuous-time Markov chain (CTMC) whose state space is partitioned into aggregates, and each aggregate is assigned a probability measure. A sufficient condition for defining a CTMC over the aggregates is presented as a variant of weak lumpability, which also characterizes that the measure over the original process can be recovered from that of the aggregated one. We show how the applicability of de-aggregation depends on the initial distribution. The application section is devoted to illustrate how the developed theory aids in reducing CTMC models of biochemical systems particularly in connection to protein-protein interactions. We assume that the model is written by a biologist in form of site-graph-rewrite rules. Site-graph-rewrite rules compactly express that, often, only a local context of a protein (instead of a full molecular species) needs to be in a certain configuration in order to trigger a reaction event. This observation leads to suitable aggregate Markov chains with smaller state spaces, thereby providing sufficient reduction in computational complexity. This is further exemplified in two case studies: simple unbounded polymerization and early EGFR/insulin crosstalk.
Bounding spectral gaps of Markov chains: a novel exact multi-decomposition technique
International Nuclear Information System (INIS)
Destainville, N
2003-01-01
We propose an exact technique to calculate lower bounds of spectral gaps of discrete time reversible Markov chains on finite state sets. Spectral gaps are a common tool for evaluating convergence rates of Markov chains. As an illustration, we successfully use this technique to evaluate the 'absorption time' of the 'Backgammon model', a paradigmatic model for glassy dynamics. We also discuss the application of this technique to the 'contingency table problem', a notoriously difficult problem from probability theory. The interest of this technique is that it connects spectral gaps, which are quantities related to dynamics, with static quantities, calculated at equilibrium
Long Memory of Financial Time Series and Hidden Markov Models with Time-Varying Parameters
DEFF Research Database (Denmark)
Nystrup, Peter; Madsen, Henrik; Lindström, Erik
2016-01-01
Hidden Markov models are often used to model daily returns and to infer the hidden state of financial markets. Previous studies have found that the estimated models change over time, but the implications of the time-varying behavior have not been thoroughly examined. This paper presents an adaptive...... to reproduce with a hidden Markov model. Capturing the time-varying behavior of the parameters also leads to improved one-step density forecasts. Finally, it is shown that the forecasting performance of the estimated models can be further improved using local smoothing to forecast the parameter variations....
Bounding spectral gaps of Markov chains: a novel exact multi-decomposition technique
Energy Technology Data Exchange (ETDEWEB)
Destainville, N [Laboratoire de Physique Theorique - IRSAMC, CNRS/Universite Paul Sabatier, 118, route de Narbonne, 31062 Toulouse Cedex 04 (France)
2003-04-04
We propose an exact technique to calculate lower bounds of spectral gaps of discrete time reversible Markov chains on finite state sets. Spectral gaps are a common tool for evaluating convergence rates of Markov chains. As an illustration, we successfully use this technique to evaluate the 'absorption time' of the 'Backgammon model', a paradigmatic model for glassy dynamics. We also discuss the application of this technique to the 'contingency table problem', a notoriously difficult problem from probability theory. The interest of this technique is that it connects spectral gaps, which are quantities related to dynamics, with static quantities, calculated at equilibrium.
Directory of Open Access Journals (Sweden)
Mansoor Ahmed Siddiqui
2017-06-01
Full Text Available This research work is aimed at optimizing the availability of a framework comprising of two units linked together in series configuration utilizing Markov Model and Monte Carlo (MC Simulation techniques. In this article, effort has been made to develop a maintenance model that incorporates three distinct states for each unit, while taking into account their different levels of deterioration. Calculations are carried out using the proposed model for two distinct cases of corrective repair, namely perfect and imperfect repairs, with as well as without opportunistic maintenance. Initially, results are accomplished using an analytical technique i.e., Markov Model. Validation of the results achieved is later carried out with the help of MC Simulation. In addition, MC Simulation based codes also work well for the frameworks that follow non-exponential failure and repair rates, and thus overcome the limitations offered by the Markov Model.
Swallowing sound detection using hidden markov modeling of recurrence plot features
International Nuclear Information System (INIS)
Aboofazeli, Mohammad; Moussavi, Zahra
2009-01-01
Automated detection of swallowing sounds in swallowing and breath sound recordings is of importance for monitoring purposes in which the recording durations are long. This paper presents a novel method for swallowing sound detection using hidden Markov modeling of recurrence plot features. Tracheal sound recordings of 15 healthy and nine dysphagic subjects were studied. The multidimensional state space trajectory of each signal was reconstructed using the Taken method of delays. The sequences of three recurrence plot features of the reconstructed trajectories (which have shown discriminating capability between swallowing and breath sounds) were modeled by three hidden Markov models. The Viterbi algorithm was used for swallowing sound detection. The results were validated manually by inspection of the simultaneously recorded airflow signal and spectrogram of the sounds, and also by auditory means. The experimental results suggested that the performance of the proposed method using hidden Markov modeling of recurrence plot features was superior to the previous swallowing sound detection methods.
Canonical Structure and Orthogonality of Forces and Currents in Irreversible Markov Chains
Kaiser, Marcus; Jack, Robert L.; Zimmer, Johannes
2018-03-01
We discuss a canonical structure that provides a unifying description of dynamical large deviations for irreversible finite state Markov chains (continuous time), Onsager theory, and Macroscopic Fluctuation Theory (MFT). For Markov chains, this theory involves a non-linear relation between probability currents and their conjugate forces. Within this framework, we show how the forces can be split into two components, which are orthogonal to each other, in a generalised sense. This splitting allows a decomposition of the pathwise rate function into three terms, which have physical interpretations in terms of dissipation and convergence to equilibrium. Similar decompositions hold for rate functions at level 2 and level 2.5. These results clarify how bounds on entropy production and fluctuation theorems emerge from the underlying dynamical rules. We discuss how these results for Markov chains are related to similar structures within MFT, which describes hydrodynamic limits of such microscopic models.
Swallowing sound detection using hidden markov modeling of recurrence plot features
Energy Technology Data Exchange (ETDEWEB)
Aboofazeli, Mohammad [Faculty of Engineering, Department of Electrical and Computer Engineering, University of Manitoba, Winnipeg, Manitoba, R3T 5V6 (Canada)], E-mail: umaboofa@cc.umanitoba.ca; Moussavi, Zahra [Faculty of Engineering, Department of Electrical and Computer Engineering, University of Manitoba, Winnipeg, Manitoba, R3T 5V6 (Canada)], E-mail: mousavi@ee.umanitoba.ca
2009-01-30
Automated detection of swallowing sounds in swallowing and breath sound recordings is of importance for monitoring purposes in which the recording durations are long. This paper presents a novel method for swallowing sound detection using hidden Markov modeling of recurrence plot features. Tracheal sound recordings of 15 healthy and nine dysphagic subjects were studied. The multidimensional state space trajectory of each signal was reconstructed using the Taken method of delays. The sequences of three recurrence plot features of the reconstructed trajectories (which have shown discriminating capability between swallowing and breath sounds) were modeled by three hidden Markov models. The Viterbi algorithm was used for swallowing sound detection. The results were validated manually by inspection of the simultaneously recorded airflow signal and spectrogram of the sounds, and also by auditory means. The experimental results suggested that the performance of the proposed method using hidden Markov modeling of recurrence plot features was superior to the previous swallowing sound detection methods.
A Markov random field approach for microstructure synthesis
International Nuclear Information System (INIS)
Kumar, A; Nguyen, L; DeGraef, M; Sundararaghavan, V
2016-01-01
We test the notion that many microstructures have an underlying stationary probability distribution. The stationary probability distribution is ubiquitous: we know that different windows taken from a polycrystalline microstructure are generally ‘statistically similar’. To enable computation of such a probability distribution, microstructures are represented in the form of undirected probabilistic graphs called Markov Random Fields (MRFs). In the model, pixels take up integer or vector states and interact with multiple neighbors over a window. Using this lattice structure, algorithms are developed to sample the conditional probability density for the state of each pixel given the known states of its neighboring pixels. The sampling is performed using reference experimental images. 2D microstructures are artificially synthesized using the sampled probabilities. Statistical features such as grain size distribution and autocorrelation functions closely match with those of the experimental images. The mechanical properties of the synthesized microstructures were computed using the finite element method and were also found to match the experimental values. (paper)
Energy Technology Data Exchange (ETDEWEB)
Borg, Matthew K., E-mail: matthew.borg@strath.ac.uk [Department of Mechanical and Aerospace Engineering, University of Strathclyde, Glasgow G1 1XJ (United Kingdom); Lockerby, Duncan A., E-mail: duncan.lockerby@warwick.ac.uk [School of Engineering, University of Warwick, Coventry CV4 7AL (United Kingdom); Reese, Jason M., E-mail: jason.reese@ed.ac.uk [School of Engineering, University of Edinburgh, Edinburgh EH9 3JL (United Kingdom)
2014-02-21
The emergence of new applications of molecular dynamics (MD) simulation calls for the development of mass-statting procedures that insert or delete particles on-the-fly. In this paper we present a new mass-stat which we term FADE, because it gradually “fades-in” (inserts) or “fades-out” (deletes) molecules over a short relaxation period within a MD simulation. FADE applies a time-weighted relaxation to the intermolecular pair forces between the inserting/deleting molecule and any neighbouring molecules. The weighting function we propose in this paper is a piece-wise polynomial that can be described entirely by two parameters: the relaxation time scale and the order of the polynomial. FADE inherently conserves overall system momentum independent of the form of the weighting function. We demonstrate various simulations of insertions of atomic argon, polyatomic TIP4P water, polymer strands, and C{sub 60} Buckminsterfullerene molecules. We propose FADE parameters and a maximum density variation per insertion-instance that restricts spurious potential energy changes entering the system within desired tolerances. We also demonstrate in this paper that FADE compares very well to an existing insertion algorithm called USHER, in terms of accuracy, insertion rate (in dense fluids), and computational efficiency. The USHER algorithm is applicable to monatomic and water molecules only, but we demonstrate that FADE can be generally applied to various forms and sizes of molecules, such as polymeric molecules of long aspect ratio, and spherical carbon fullerenes with hollow interiors.
International Nuclear Information System (INIS)
Borg, Matthew K.; Lockerby, Duncan A.; Reese, Jason M.
2014-01-01
The emergence of new applications of molecular dynamics (MD) simulation calls for the development of mass-statting procedures that insert or delete particles on-the-fly. In this paper we present a new mass-stat which we term FADE, because it gradually “fades-in” (inserts) or “fades-out” (deletes) molecules over a short relaxation period within a MD simulation. FADE applies a time-weighted relaxation to the intermolecular pair forces between the inserting/deleting molecule and any neighbouring molecules. The weighting function we propose in this paper is a piece-wise polynomial that can be described entirely by two parameters: the relaxation time scale and the order of the polynomial. FADE inherently conserves overall system momentum independent of the form of the weighting function. We demonstrate various simulations of insertions of atomic argon, polyatomic TIP4P water, polymer strands, and C 60 Buckminsterfullerene molecules. We propose FADE parameters and a maximum density variation per insertion-instance that restricts spurious potential energy changes entering the system within desired tolerances. We also demonstrate in this paper that FADE compares very well to an existing insertion algorithm called USHER, in terms of accuracy, insertion rate (in dense fluids), and computational efficiency. The USHER algorithm is applicable to monatomic and water molecules only, but we demonstrate that FADE can be generally applied to various forms and sizes of molecules, such as polymeric molecules of long aspect ratio, and spherical carbon fullerenes with hollow interiors
Directory of Open Access Journals (Sweden)
Srinivas Nallagonda
2016-04-01
Full Text Available In this paper we evaluate the performance of cooperative spectrum sensing (CSS where each cognitive radio (CR employs an improved energy detector (IED with multiple antennas and uses selection combining (SC for detecting the primary user (PU in noisy and faded sensing (S channels. We derive an expression for the probability of false alarm and expressions for probability of missed detection in non-faded (AWGN and Rayleigh faded sensing environments in terms of cumulative distribution function (CDF. Each CR transmits its decision about PU via noisy and faded reporting (R channel to fusion center (FC. In this paper we assume that S-channels are noisy and Rayleigh faded while several cases of fading are considered for R-channels such as: (i Hoyt (or Nakagami-q, (ii Rayleigh, (iii Rician (or Nakagami-n, and (iv Weibull. A Binary Symmetric channel (BSC with a fixed error probability (r in the R-channel is also considered. The impact of fading in R-channel, S-channel and several network parameters such as IED parameter, normalized detection threshold, number of CRs, and number of antennas on missed detection and total error probability is assessed. The effects of Hoyt, Rician, and Weibull fading parameters on overall performance of IED-CSS are also highlighted.
ter Vrugte, Judith; de Jong, Anthonius J.M.; Vandercruysse, Sylke; Wouters, Pieter; van Oostendorp, Herre; Elen, Jan
This study addresses the added value of faded worked examples in a computer game-based learning environment. The faded worked examples were introduced to encourage active selection and processing of domain content in the game. The content of the game was proportional reasoning and participants were
Dependence of (anomalous) fading of infra-red stimulated luminescence on trap occupancy in feldspars
DEFF Research Database (Denmark)
Morthekai, P.; Jain, Mayank; Gach, Grzegorz
2013-01-01
Dose dependency of anomalous fading of infra-red stimulated luminescence (IRSL) from feldspars has been investigated using radiations of different ionisation qualities. The rate of fading of the IRSL signal after proton irradiation (9.4–30%/decade) is on an average almost twice compared...
Nonparametric Estimation of Interval Reliability for Discrete-Time Semi-Markov Systems
DEFF Research Database (Denmark)
Georgiadis, Stylianos; Limnios, Nikolaos
2016-01-01
In this article, we consider a repairable discrete-time semi-Markov system with finite state space. The measure of the interval reliability is given as the probability of the system being operational over a given finite-length time interval. A nonparametric estimator is proposed for the interval...
Strategy Complexity of Finite-Horizon Markov Decision Processes and Simple Stochastic Games
DEFF Research Database (Denmark)
Ibsen-Jensen, Rasmus; Chatterjee, Krishnendu
2012-01-01
Markov decision processes (MDPs) and simple stochastic games (SSGs) provide a rich mathematical framework to study many important problems related to probabilistic systems. MDPs and SSGs with finite-horizon objectives, where the goal is to maximize the probability to reach a target state in a given...
Activity recognition using semi-Markov models on real world smart home datasets
van Kasteren, T.L.M.; Englebienne, G.; Kröse, B.J.A.
2010-01-01
Accurately recognizing human activities from sensor data recorded in a smart home setting is a challenging task. Typically, probabilistic models such as the hidden Markov model (HMM) or conditional random fields (CRF) are used to map the observed sensor data onto the hidden activity states. A
Hidden Semi Markov Models for Multiple Observation Sequences: The mhsmm Package for R
DEFF Research Database (Denmark)
O'Connell, Jarad Michael; Højsgaard, Søren
2011-01-01
models only allow a geometrically distributed sojourn time in a given state, while hidden semi-Markov models extend this by allowing an arbitrary sojourn distribution. We demonstrate the software with simulation examples and an application involving the modelling of the ovarian cycle of dairy cows...
M.J.C. Nuijten (Mark); D.L. Andress (Dennis); S.E. Marx (Steven); R. Sterz (Raimund)
2009-01-01
textabstractObjective: The objective of this study was to determine the cost effectiveness of paricalcitol versus calcitriol for the treatment of secondary hyperparathyroidism in patients with chronic kidney disease in the United States setting. Methods: A Markov process model was developed
Directory of Open Access Journals (Sweden)
Madeiro Francisco
2010-01-01
Full Text Available Abstract This paper presents an alternative method for determining exact expressions for the bit error probability (BEP of modulation schemes subject to Nakagami- fading. In this method, the Nakagami- fading channel is seen as an additive noise channel whose noise is modeled as the ratio between Gaussian and Nakagami- random variables. The method consists of using the cumulative density function of the resulting noise to obtain closed-form expressions for the BEP of modulation schemes subject to Nakagami- fading. In particular, the proposed method is used to obtain closed-form expressions for the BEP of -ary quadrature amplitude modulation ( -QAM, -ary pulse amplitude modulation ( -PAM, and rectangular quadrature amplitude modulation ( -QAM under Nakagami- fading. The main contribution of this paper is to show that this alternative method can be used to reduce the computational complexity for detecting signals in the presence of fading.
Large deviations for Markov chains in the positive quadrant
Energy Technology Data Exchange (ETDEWEB)
Borovkov, A A; Mogul' skii, A A [S.L. Sobolev Institute for Mathematics, Siberian Branch of the Russian Academy of Sciences, Novosibirsk (Russian Federation)
2001-10-31
The paper deals with so-called N-partially space-homogeneous time-homogeneous Markov chains X(y,n), n=0,1,2,..., X(y,0)=y, in the positive quadrant. These Markov chains are characterized by the following property of the transition probabilities P(y,A)=P(X(y,1) element of A): for some N{>=}0 the measure P(y,dx) depends only on x{sub 2}, y{sub 2}, and x{sub 1}-y{sub 1} in the domain x{sub 1}>N, y{sub 1}>N, and only on x{sub 1}, y{sub 1}, and x{sub 2}-y{sub 2} in the domain x{sub 2}>N, y{sub 2}>N. For such chains the asymptotic behaviour is found for a fixed set B as s{yields}{infinity}, |x|{yields}{infinity}, and n{yields}{infinity}. Some other conditions on the growth of parameters are also considered, for example, |x-y|{yields}{infinity}, |y|{yields}{infinity}. A study is made of the structure of the most probable trajectories, which give the main contribution to this asymptotics, and a number of other results pertaining to the topic are established. Similar results are obtained for the narrower class of 0-partially homogeneous ergodic chains under less restrictive moment conditions on the transition probabilities P(y,dx). Moreover, exact asymptotic expressions for the probabilities P(X(0,n) element of x+B) are found for 0-partially homogeneous ergodic chains under some additional conditions. The interest in partially homogeneous Markov chains in positive octants is due to the mathematical aspects (new and interesting problems arise in the framework of general large deviation theory) as well as applied issues, for such chains prove to be quite accurate mathematical models for numerous basic types of queueing and communication networks such as the widely known Jackson networks, polling systems, or communication networks associated with the ALOHA algorithm. There is a vast literature dealing with the analysis of these objects. The present paper is an attempt to find the extent to which an asymptotic analysis is possible for Markov chains of this type in their general
Polarimetry and photometry of M87: is the jet fading
Energy Technology Data Exchange (ETDEWEB)
Warren-Smith, R F; King, D J; Scarrott, S M [Durham Univ. (UK). Dept. of Physics
1984-09-15
Optical linear polarization mapping and photometry of M87 is presented in B and R wavebands. The results indicate significant polarization in the galactic nucleus. Polarization in the jet in B is consistent with other recent maps obtained for this area in blue light and no significant variation with wavelength between B and R wavebands is found. The spectral index of the jet radiation is measured to be S=-1.65+-0.2 over the wavelength range observed. Comparison of the integrated B magnitude of the jet with previous independent measurements over the period 1934-80 suggests that the jet is variable and has been fading more or less uniformly by about 0.8 mag per decade between 1964 and 1980.
Asymptotic analysis of multicell massive MIMO over Rician fading channels
Sanguinetti, Luca; Kammoun, Abla; Debbah, Merouane
2017-01-01
This work considers the downlink of a multicell massive MIMO system in which L base stations (BSs) of N antennas each communicate with K single-antenna user equipments randomly positioned in the coverage area. Within this setting, we are interested in evaluating the sum rate of the system when MRT and RZF are employed under the assumption that each intracell link forms a MIMO Rician uncorrelated fading channel. The analysis is conducted assuming that N and K grow large with a non-trivial ratio N/K under the assumption that the data transmission in each cell is affected by channel estimation errors, pilot contamination, and an arbitrary large scale attenuation. Numerical results are used to validate the asymptotic analysis in the finite system regime and to evaluate the network performance under different settings. The asymptotic results are also instrumental to get insights into the interplay among system parameters.
Asymptotic analysis of multicell massive MIMO over Rician fading channels
Sanguinetti, Luca
2017-06-20
This work considers the downlink of a multicell massive MIMO system in which L base stations (BSs) of N antennas each communicate with K single-antenna user equipments randomly positioned in the coverage area. Within this setting, we are interested in evaluating the sum rate of the system when MRT and RZF are employed under the assumption that each intracell link forms a MIMO Rician uncorrelated fading channel. The analysis is conducted assuming that N and K grow large with a non-trivial ratio N/K under the assumption that the data transmission in each cell is affected by channel estimation errors, pilot contamination, and an arbitrary large scale attenuation. Numerical results are used to validate the asymptotic analysis in the finite system regime and to evaluate the network performance under different settings. The asymptotic results are also instrumental to get insights into the interplay among system parameters.
Developing a numerical simulation for fading in feldspar
International Nuclear Information System (INIS)
Larsen, A.; Greilich, S.; Jain, M.; Murray, A.S.
2009-01-01
Most models describing anomalous fading in feldspars are based on analytical solutions. As an alternative approach we present an entirely numerical model based on statistical sampling that simulates stepwise the charge creation/trapping and recombination in a given phosphor. We assume the number density of electrons and holes at any time to be equal, although the model is not bound to that assumption. The model is flexible enough to be used on any combination of geological and laboratory timescales and for any defined configuration of defects. Using this approach we observed reorganization of nearest-neighbor distances with time. The best agreement with experimental data is achieved if we assume the crystal to consist of small subvolumes (nanocrystals) only within which charge carriers are allowed to recombine.
Atropine and ODQ antagonize tetanic fade induced by L-arginine in cats
Directory of Open Access Journals (Sweden)
J.M. Cruciol-Souza
1999-10-01
Full Text Available Although it has been demonstrated that nitric oxide (NO released from sodium nitrite induces tetanic fade in the cat neuromuscular preparations, the effect of L-arginine on tetanic fade and its origin induced by NO have not been studied in these preparations. Furthermore, atropine reduces tetanic fade induced by several cholinergic and anticholinergic drugs in these preparations, whose mechanism is suggested to be mediated by the interaction of acetylcholine with inhibitory presynaptic muscarinic receptors. The present study was conducted in cats to determine the effects of L-arginine alone or after pretreatment with atropine or 1H-[1,2,4]oxadiazole [4,3-a]quinoxalin-1-one (ODQ on neuromuscular preparations indirectly stimulated at high frequency. Drugs were injected into the middle genicular artery. L-arginine (2 mg/kg and S-nitroso-N-acetylpenicillamine (SNAP; 16 µg/kg induced tetanic fade. The Nw-nitro-L-arginine (L-NOARG; 2 mg/kg alone did not produce any effect, but reduced the tetanic fade induced by L-arginine. D-arginine (2 mg/kg did not induce changes in tetanic fade. The tetanic fade induced by L-arginine or SNAP was reduced by previous injection of atropine (1.0 µg/kg or ODQ (15 µg/kg. ODQ alone did not change tetanic fade. The data suggest that the NO-synthase-GC pathway participates in the L-arginine-induced tetanic fade in cat neuromuscular preparations. The tetanic fade induced by L-arginine probably depends on the action of NO at the presynaptic level. NO may stimulate guanylate cyclase increasing acetylcholine release and thereby stimulating presynaptic muscarinic receptors.
Schmandt, Nicolaus T; Galán, Roberto F
2012-09-14
Markov chains provide realistic models of numerous stochastic processes in nature. We demonstrate that in any Markov chain, the change in occupation number in state A is correlated to the change in occupation number in state B if and only if A and B are directly connected. This implies that if we are only interested in state A, fluctuations in B may be replaced with their mean if state B is not directly connected to A, which shortens computing time considerably. We show the accuracy and efficacy of our approximation theoretically and in simulations of stochastic ion-channel gating in neurons.
Belief Bisimulation for Hidden Markov Models Logical Characterisation and Decision Algorithm
DEFF Research Database (Denmark)
Jansen, David N.; Nielson, Flemming; Zhang, Lijun
2012-01-01
This paper establishes connections between logical equivalences and bisimulation relations for hidden Markov models (HMM). Both standard and belief state bisimulations are considered. We also present decision algorithms for the bisimilarities. For standard bisimilarity, an extension of the usual...... partition refinement algorithm is enough. Belief bisimilarity, being a relation on the continuous space of belief states, cannot be described directly. Instead, we show how to generate a linear equation system in time cubic in the number of states....
The algebra of the general Markov model on phylogenetic trees and networks.
Sumner, J G; Holland, B R; Jarvis, P D
2012-04-01
It is known that the Kimura 3ST model of sequence evolution on phylogenetic trees can be extended quite naturally to arbitrary split systems. However, this extension relies heavily on mathematical peculiarities of the associated Hadamard transformation, and providing an analogous augmentation of the general Markov model has thus far been elusive. In this paper, we rectify this shortcoming by showing how to extend the general Markov model on trees to include incompatible edges; and even further to more general network models. This is achieved by exploring the algebra of the generators of the continuous-time Markov chain together with the “splitting” operator that generates the branching process on phylogenetic trees. For simplicity, we proceed by discussing the two state case and then show that our results are easily extended to more states with little complication. Intriguingly, upon restriction of the two state general Markov model to the parameter space of the binary symmetric model, our extension is indistinguishable from the Hadamard approach only on trees; as soon as any incompatible splits are introduced the two approaches give rise to differing probability distributions with disparate structure. Through exploration of a simple example, we give an argument that our extension to more general networks has desirable properties that the previous approaches do not share. In particular, our construction allows for convergent evolution of previously divergent lineages; a property that is of significant interest for biological applications.
Markov switching of the electricity supply curve and power prices dynamics
Mari, Carlo; Cananà, Lucianna
2012-02-01
Regime-switching models seem to well capture the main features of power prices behavior in deregulated markets. In a recent paper, we have proposed an equilibrium methodology to derive electricity prices dynamics from the interplay between supply and demand in a stochastic environment. In particular, assuming that the supply function is described by a power law where the exponent is a two-state strictly positive Markov process, we derived a regime switching dynamics of power prices in which regime switches are induced by transitions between Markov states. In this paper, we provide a dynamical model to describe the random behavior of power prices where the only non-Brownian component of the motion is endogenously introduced by Markov transitions in the exponent of the electricity supply curve. In this context, the stochastic process driving the switching mechanism becomes observable, and we will show that the non-Brownian component of the dynamics induced by transitions from Markov states is responsible for jumps and spikes of very high magnitude. The empirical analysis performed on three Australian markets confirms that the proposed approach seems quite flexible and capable of incorporating the main features of power prices time-series, thus reproducing the first four moments of log-returns empirical distributions in a satisfactory way.
Entropy, complexity, and Markov diagrams for random walk cancer models.
Newton, Paul K; Mason, Jeremy; Hurt, Brian; Bethel, Kelly; Bazhenova, Lyudmila; Nieva, Jorge; Kuhn, Peter
2014-12-19
The notion of entropy is used to compare the complexity associated with 12 common cancers based on metastatic tumor distribution autopsy data. We characterize power-law distributions, entropy, and Kullback-Liebler divergence associated with each primary cancer as compared with data for all cancer types aggregated. We then correlate entropy values with other measures of complexity associated with Markov chain dynamical systems models of progression. The Markov transition matrix associated with each cancer is associated with a directed graph model where nodes are anatomical locations where a metastatic tumor could develop, and edge weightings are transition probabilities of progression from site to site. The steady-state distribution corresponds to the autopsy data distribution. Entropy correlates well with the overall complexity of the reduced directed graph structure for each cancer and with a measure of systemic interconnectedness of the graph, called graph conductance. The models suggest that grouping cancers according to their entropy values, with skin, breast, kidney, and lung cancers being prototypical high entropy cancers, stomach, uterine, pancreatic and ovarian being mid-level entropy cancers, and colorectal, cervical, bladder, and prostate cancers being prototypical low entropy cancers, provides a potentially useful framework for viewing metastatic cancer in terms of predictability, complexity, and metastatic potential.
Saliency Detection via Absorbing Markov Chain With Learnt Transition Probability.
Lihe Zhang; Jianwu Ai; Bowen Jiang; Huchuan Lu; Xiukui Li
2018-02-01
In this paper, we propose a bottom-up saliency model based on absorbing Markov chain (AMC). First, a sparsely connected graph is constructed to capture the local context information of each node. All image boundary nodes and other nodes are, respectively, treated as the absorbing nodes and transient nodes in the absorbing Markov chain. Then, the expected number of times from each transient node to all other transient nodes can be used to represent the saliency value of this node. The absorbed time depends on the weights on the path and their spatial coordinates, which are completely encoded in the transition probability matrix. Considering the importance of this matrix, we adopt different hierarchies of deep features extracted from fully convolutional networks and learn a transition probability matrix, which is called learnt transition probability matrix. Although the performance is significantly promoted, salient objects are not uniformly highlighted very well. To solve this problem, an angular embedding technique is investigated to refine the saliency results. Based on pairwise local orderings, which are produced by the saliency maps of AMC and boundary maps, we rearrange the global orderings (saliency value) of all nodes. Extensive experiments demonstrate that the proposed algorithm outperforms the state-of-the-art methods on six publicly available benchmark data sets.
Nuclear security assessment with Markov model approach
International Nuclear Information System (INIS)
Suzuki, Mitsutoshi; Terao, Norichika
2013-01-01
Nuclear security risk assessment with the Markov model based on random event is performed to explore evaluation methodology for physical protection in nuclear facilities. Because the security incidences are initiated by malicious and intentional acts, expert judgment and Bayes updating are used to estimate scenario and initiation likelihood, and it is assumed that the Markov model derived from stochastic process can be applied to incidence sequence. Both an unauthorized intrusion as Design Based Threat (DBT) and a stand-off attack as beyond-DBT are assumed to hypothetical facilities, and performance of physical protection and mitigation and minimization of consequence are investigated to develop the assessment methodology in a semi-quantitative manner. It is shown that cooperation between facility operator and security authority is important to respond to the beyond-DBT incidence. (author)
MARKOV CHAIN PORTFOLIO LIQUIDITY OPTIMIZATION MODEL
Directory of Open Access Journals (Sweden)
Eder Oliveira Abensur
2014-05-01
Full Text Available The international financial crisis of September 2008 and May 2010 showed the importance of liquidity as an attribute to be considered in portfolio decisions. This study proposes an optimization model based on available public data, using Markov chain and Genetic Algorithms concepts as it considers the classic duality of risk versus return and incorporating liquidity costs. The work intends to propose a multi-criterion non-linear optimization model using liquidity based on a Markov chain. The non-linear model was tested using Genetic Algorithms with twenty five Brazilian stocks from 2007 to 2009. The results suggest that this is an innovative development methodology and useful for developing an efficient and realistic financial portfolio, as it considers many attributes such as risk, return and liquidity.
Exact solution of the hidden Markov processes
Saakian, David B.
2017-11-01
We write a master equation for the distributions related to hidden Markov processes (HMPs) and solve it using a functional equation. Thus the solution of HMPs is mapped exactly to the solution of the functional equation. For a general case the latter can be solved only numerically. We derive an exact expression for the entropy of HMPs. Our expression for the entropy is an alternative to the ones given before by the solution of integral equations. The exact solution is possible because actually the model can be considered as a generalized random walk on a one-dimensional strip. While we give the solution for the two second-order matrices, our solution can be easily generalized for the L values of the Markov process and M values of observables: We should be able to solve a system of L functional equations in the space of dimension M -1 .
A Bayesian method for construction of Markov models to describe dynamics on various time-scales.
Rains, Emily K; Andersen, Hans C
2010-10-14
The dynamics of many biological processes of interest, such as the folding of a protein, are slow and complicated enough that a single molecular dynamics simulation trajectory of the entire process is difficult to obtain in any reasonable amount of time. Moreover, one such simulation may not be sufficient to develop an understanding of the mechanism of the process, and multiple simulations may be necessary. One approach to circumvent this computational barrier is the use of Markov state models. These models are useful because they can be constructed using data from a large number of shorter simulations instead of a single long simulation. This paper presents a new Bayesian method for the construction of Markov models from simulation data. A Markov model is specified by (τ,P,T), where τ is the mesoscopic time step, P is a partition of configuration space into mesostates, and T is an N(P)×N(P) transition rate matrix for transitions between the mesostates in one mesoscopic time step, where N(P) is the number of mesostates in P. The method presented here is different from previous Bayesian methods in several ways. (1) The method uses Bayesian analysis to determine the partition as well as the transition probabilities. (2) The method allows the construction of a Markov model for any chosen mesoscopic time-scale τ. (3) It constructs Markov models for which the diagonal elements of T are all equal to or greater than 0.5. Such a model will be called a "consistent mesoscopic Markov model" (CMMM). Such models have important advantages for providing an understanding of the dynamics on a mesoscopic time-scale. The Bayesian method uses simulation data to find a posterior probability distribution for (P,T) for any chosen τ. This distribution can be regarded as the Bayesian probability that the kinetics observed in the atomistic simulation data on the mesoscopic time-scale τ was generated by the CMMM specified by (P,T). An optimization algorithm is used to find the most
Hidden Markov modelling of movement data from fish
DEFF Research Database (Denmark)
Pedersen, Martin Wæver
Movement data from marine animals tagged with electronic tags are becoming increasingly diverse and plentiful. This trend entails a need for statistical methods that are able to filter the observations to extract the ecologically relevant content. This dissertation focuses on the development...... the behaviour of the animal. With the extended model can migratory and resident movement behaviour be related to geographical regions. For population inference multiple individual state-space analyses can be interconnected using mixed effects modelling. This framework provides parameter estimates...... approximated. This furthermore enables accurate probability densities of location to be computed. Finally, the performance of the HMM approach in analysing nonlinear state space models is compared with two alternatives: the AD Model Builder framework and BUGS, which relies on Markov chain Monte Carlo...
Accelerated decomposition techniques for large discounted Markov decision processes
Larach, Abdelhadi; Chafik, S.; Daoui, C.
2017-12-01
Many hierarchical techniques to solve large Markov decision processes (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorithm, which is a variant of Tarjan's algorithm that simultaneously finds the SCCs and their belonging levels. Second, a new definition of the restricted MDPs is presented to ameliorate some hierarchical solutions in discounted MDPs using value iteration (VI) algorithm based on a list of state-action successors. Finally, a robotic motion-planning example and the experiment results are presented to illustrate the benefit of the proposed decomposition algorithms.
Nonequilibrium thermodynamic potentials for continuous-time Markov chains.
Verley, Gatien
2016-01-01
We connect the rare fluctuations of an equilibrium (EQ) process and the typical fluctuations of a nonequilibrium (NE) stationary process. In the framework of large deviation theory, this observation allows us to introduce NE thermodynamic potentials. For continuous-time Markov chains, we identify the relevant pairs of conjugated variables and propose two NE ensembles: one with fixed dynamics and fluctuating time-averaged variables, and another with fixed time-averaged variables, but a fluctuating dynamics. Accordingly, we show that NE processes are equivalent to conditioned EQ processes ensuring that NE potentials are Legendre dual. We find a variational principle satisfied by the NE potentials that reach their maximum in the NE stationary state and whose first derivatives produce the NE equations of state and second derivatives produce the NE Maxwell relations generalizing the Onsager reciprocity relations.