WorldWideScience

Sample records for nonparametric covariance matrix

  1. A Nonparametric Analogy of Analysis of Covariance

    Science.gov (United States)

    Burnett, Thomas D.; Barr, Donald R.

    1977-01-01

    A nonparametric test of the hypothesis of no treatment effect is suggested for a situation where measures of the severity of the condition treated can be obtained and ranked both pre- and post-treatment. The test allows the pre-treatment rank to be used as a concomitant variable. (Author/JKS)

  2. Effect on Prediction when Modeling Covariates in Bayesian Nonparametric Models.

    Science.gov (United States)

    Cruz-Marcelo, Alejandro; Rosner, Gary L; Müller, Peter; Stewart, Clinton F

    2013-04-01

    In biomedical research, it is often of interest to characterize biologic processes giving rise to observations and to make predictions of future observations. Bayesian nonparametric methods provide a means for carrying out Bayesian inference making as few assumptions about restrictive parametric models as possible. There are several proposals in the literature for extending Bayesian nonparametric models to include dependence on covariates. Limited attention, however, has been directed to the following two aspects. In this article, we examine the effect on fitting and predictive performance of incorporating covariates in a class of Bayesian nonparametric models by one of two primary ways: either in the weights or in the locations of a discrete random probability measure. We show that different strategies for incorporating continuous covariates in Bayesian nonparametric models can result in big differences when used for prediction, even though they lead to otherwise similar posterior inferences. When one needs the predictive density, as in optimal design, and this density is a mixture, it is better to make the weights depend on the covariates. We demonstrate these points via a simulated data example and in an application in which one wants to determine the optimal dose of an anticancer drug used in pediatric oncology.

  3. Bayes linear covariance matrix adjustment

    CERN Document Server

    Wilkinson, Darren J

    1995-01-01

    In this thesis, a Bayes linear methodology for the adjustment of covariance matrices is presented and discussed. A geometric framework for quantifying uncertainties about covariance matrices is set up, and an inner-product for spaces of random matrices is motivated and constructed. The inner-product on this space captures aspects of our beliefs about the relationship between covariance matrices of interest to us, providing a structure rich enough for us to adjust beliefs about unknown matrices in the light of data such as sample covariance matrices, exploiting second-order exchangeability and related specifications to obtain representations allowing analysis. Adjustment is associated with orthogonal projection, and illustrated with examples of adjustments for some common problems. The problem of adjusting the covariance matrices underlying exchangeable random vectors is tackled and discussed. Learning about the covariance matrices associated with multivariate time series dynamic linear models is shown to be a...

  4. An Investigation into the Dimensionality of TOEFL Using Conditional Covariance-Based Nonparametric Approach

    Science.gov (United States)

    Jang, Eunice Eunhee; Roussos, Louis

    2007-01-01

    This article reports two studies to illustrate methodologies for conducting a conditional covariance-based nonparametric dimensionality assessment using data from two forms of the Test of English as a Foreign Language (TOEFL). Study 1 illustrates how to assess overall dimensionality of the TOEFL including all three subtests. Study 2 is aimed at…

  5. Hierarchical matrix approximation of large covariance matrices

    KAUST Repository

    Litvinenko, Alexander

    2015-01-07

    We approximate large non-structured covariance matrices in the H-matrix format with a log-linear computational cost and storage O(n log n). We compute inverse, Cholesky decomposition and determinant in H-format. As an example we consider the class of Matern covariance functions, which are very popular in spatial statistics, geostatistics, machine learning and image analysis. Applications are: kriging and optimal design

  6. Hierarchical matrix approximation of large covariance matrices

    KAUST Repository

    Litvinenko, Alexander

    2015-01-05

    We approximate large non-structured covariance matrices in the H-matrix format with a log-linear computational cost and storage O(nlogn). We compute inverse, Cholesky decomposition and determinant in H-format. As an example we consider the class of Matern covariance functions, which are very popular in spatial statistics, geostatistics, machine learning and image analysis. Applications are: kriging and op- timal design.

  7. Hierarchical matrix approximation of large covariance matrices

    KAUST Repository

    Litvinenko, Alexander

    2015-11-30

    We approximate large non-structured Matérn covariance matrices of size n×n in the H-matrix format with a log-linear computational cost and storage O(kn log n), where rank k ≪ n is a small integer. Applications are: spatial statistics, machine learning and image analysis, kriging and optimal design.

  8. Bayesian Nonparametric Regression Analysis of Data with Random Effects Covariates from Longitudinal Measurements

    KAUST Repository

    Ryu, Duchwan

    2010-09-28

    We consider nonparametric regression analysis in a generalized linear model (GLM) framework for data with covariates that are the subject-specific random effects of longitudinal measurements. The usual assumption that the effects of the longitudinal covariate processes are linear in the GLM may be unrealistic and if this happens it can cast doubt on the inference of observed covariate effects. Allowing the regression functions to be unknown, we propose to apply Bayesian nonparametric methods including cubic smoothing splines or P-splines for the possible nonlinearity and use an additive model in this complex setting. To improve computational efficiency, we propose the use of data-augmentation schemes. The approach allows flexible covariance structures for the random effects and within-subject measurement errors of the longitudinal processes. The posterior model space is explored through a Markov chain Monte Carlo (MCMC) sampler. The proposed methods are illustrated and compared to other approaches, the "naive" approach and the regression calibration, via simulations and by an application that investigates the relationship between obesity in adulthood and childhood growth curves. © 2010, The International Biometric Society.

  9. Bayesian nonparametric regression analysis of data with random effects covariates from longitudinal measurements.

    Science.gov (United States)

    Ryu, Duchwan; Li, Erning; Mallick, Bani K

    2011-06-01

    We consider nonparametric regression analysis in a generalized linear model (GLM) framework for data with covariates that are the subject-specific random effects of longitudinal measurements. The usual assumption that the effects of the longitudinal covariate processes are linear in the GLM may be unrealistic and if this happens it can cast doubt on the inference of observed covariate effects. Allowing the regression functions to be unknown, we propose to apply Bayesian nonparametric methods including cubic smoothing splines or P-splines for the possible nonlinearity and use an additive model in this complex setting. To improve computational efficiency, we propose the use of data-augmentation schemes. The approach allows flexible covariance structures for the random effects and within-subject measurement errors of the longitudinal processes. The posterior model space is explored through a Markov chain Monte Carlo (MCMC) sampler. The proposed methods are illustrated and compared to other approaches, the "naive" approach and the regression calibration, via simulations and by an application that investigates the relationship between obesity in adulthood and childhood growth curves.

  10. Risk evaluation with enhaced covariance matrix

    CERN Document Server

    Urbanowicz, K; Richmond, P; Holyst, Janusz A.; Richmond, Peter; Urbanowicz, Krzysztof

    2006-01-01

    We propose a route for the evaluation of risk based on a transformation of the covariance matrix. The approach uses a `potential' or `objective' function. This allows us to rescale data from diferent assets (or sources) such that each set then has similar statistical properties in terms of their probability distributions. The method is tested using historical data from both the New York and Warsaw Stock Exchanges.

  11. ANL Critical Assembly Covariance Matrix Generation

    Energy Technology Data Exchange (ETDEWEB)

    McKnight, Richard D. [Argonne National Lab. (ANL), Argonne, IL (United States); Grimm, Karl N. [Argonne National Lab. (ANL), Argonne, IL (United States)

    2014-01-15

    This report discusses the generation of a covariance matrix for selected critical assemblies that were carried out by Argonne National Laboratory (ANL) using four critical facilities-all of which are now decommissioned. The four different ANL critical facilities are: ZPR-3 located at ANL-West (now Idaho National Laboratory- INL), ZPR-6 and ZPR-9 located at ANL-East (Illinois) and ZPPr located at ANL-West.

  12. Shrinkage covariance matrix approach for microarray data

    Science.gov (United States)

    Karjanto, Suryaefiza; Aripin, Rasimah

    2013-04-01

    Microarray technology was developed for the purpose of monitoring the expression levels of thousands of genes. A microarray data set typically consists of tens of thousands of genes (variables) from just dozens of samples due to various constraints including the high cost of producing microarray chips. As a result, the widely used standard covariance estimator is not appropriate for this purpose. One such technique is the Hotelling's T2 statistic which is a multivariate test statistic for comparing means between two groups. It requires that the number of observations (n) exceeds the number of genes (p) in the set but in microarray studies it is common that n Hotelling's T2 statistic with the shrinkage approach is proposed to estimate the covariance matrix for testing differential gene expression. The performance of this approach is then compared with other commonly used multivariate tests using a widely analysed diabetes data set as illustrations. The results across the methods are consistent, implying that this approach provides an alternative to existing techniques.

  13. Perturbative approach to covariance matrix of the matter power spectrum

    Energy Technology Data Exchange (ETDEWEB)

    Mohammed, Irshad [Fermilab; Seljak, Uros [UC, Berkeley, Astron. Dept.; Vlah, Zvonimir [Stanford U., ITP

    2016-06-30

    We evaluate the covariance matrix of the matter power spectrum using perturbation theory up to dominant terms at 1-loop order and compare it to numerical simulations. We decompose the covariance matrix into the disconnected (Gaussian) part, trispectrum from the modes outside the survey (beat coupling or super-sample variance), and trispectrum from the modes inside the survey, and show how the different components contribute to the overall covariance matrix. We find the agreement with the simulations is at a 10\\% level up to $k \\sim 1 h {\\rm Mpc^{-1}}$. We show that all the connected components are dominated by the large-scale modes ($k<0.1 h {\\rm Mpc^{-1}}$), regardless of the value of the wavevectors $k,\\, k'$ of the covariance matrix, suggesting that one must be careful in applying the jackknife or bootstrap methods to the covariance matrix. We perform an eigenmode decomposition of the connected part of the covariance matrix, showing that at higher $k$ it is dominated by a single eigenmode. The full covariance matrix can be approximated as the disconnected part only, with the connected part being treated as an external nuisance parameter with a known scale dependence, and a known prior on its variance for a given survey volume. Finally, we provide a prescription for how to evaluate the covariance matrix from small box simulations without the need to simulate large volumes.

  14. Perturbative approach to covariance matrix of the matter power spectrum

    Science.gov (United States)

    Mohammed, Irshad; Seljak, Uroš; Vlah, Zvonimir

    2017-04-01

    We evaluate the covariance matrix of the matter power spectrum using perturbation theory up to dominant terms at 1-loop order and compare it to numerical simulations. We decompose the covariance matrix into the disconnected (Gaussian) part, trispectrum from the modes outside the survey (supersample variance) and trispectrum from the modes inside the survey, and show how the different components contribute to the overall covariance matrix. We find the agreement with the simulations is at a 10 per cent level up to k ˜ 1 h Mpc-1. We show that all the connected components are dominated by the large-scale modes (k covariance matrix, suggesting that one must be careful in applying the jackknife or bootstrap methods to the covariance matrix. We perform an eigenmode decomposition of the connected part of the covariance matrix, showing that at higher k, it is dominated by a single eigenmode. The full covariance matrix can be approximated as the disconnected part only, with the connected part being treated as an external nuisance parameter with a known scale dependence, and a known prior on its variance for a given survey volume. Finally, we provide a prescription for how to evaluate the covariance matrix from small box simulations without the need to simulate large volumes.

  15. Comparison of Rank Analysis of Covariance and Nonparametric Randomized Blocks Analysis.

    Science.gov (United States)

    Porter, Andrew C.; McSweeney, Maryellen

    The relative power of three possible experimental designs under the condition that data is to be analyzed by nonparametric techniques; the comparison of the power of each nonparametric technique to its parametric analogue; and the comparison of relative powers using nonparametric and parametric techniques are discussed. The three nonparametric…

  16. Estimating the power spectrum covariance matrix with fewer mock samples

    CERN Document Server

    Pearson, David W

    2015-01-01

    The covariance matrices of power-spectrum (P(k)) measurements from galaxy surveys are difficult to compute theoretically. The current best practice is to estimate covariance matrices by computing a sample covariance of a large number of mock catalogues. The next generation of galaxy surveys will require thousands of large volume mocks to determine the covariance matrices to desired accuracy. The errors in the inverse covariance matrix are larger and scale with the number of P(k) bins, making the problem even more acute. We develop a method of estimating covariance matrices using a theoretically justified, few-parameter model, calibrated with mock catalogues. Using a set of 600 BOSS DR11 mock catalogues, we show that a seven parameter model is sufficient to fit the covariance matrix of BOSS DR11 P(k) measurements. The covariance computed with this method is better than the sample covariance at any number of mocks and only ~100 mocks are required for it to fully converge and the inverse covariance matrix conver...

  17. HIGH DIMENSIONAL COVARIANCE MATRIX ESTIMATION IN APPROXIMATE FACTOR MODELS.

    Science.gov (United States)

    Fan, Jianqing; Liao, Yuan; Mincheva, Martina

    2011-01-01

    The variance covariance matrix plays a central role in the inferential theories of high dimensional factor models in finance and economics. Popular regularization methods of directly exploiting sparsity are not directly applicable to many financial problems. Classical methods of estimating the covariance matrices are based on the strict factor models, assuming independent idiosyncratic components. This assumption, however, is restrictive in practical applications. By assuming sparse error covariance matrix, we allow the presence of the cross-sectional correlation even after taking out common factors, and it enables us to combine the merits of both methods. We estimate the sparse covariance using the adaptive thresholding technique as in Cai and Liu (2011), taking into account the fact that direct observations of the idiosyncratic components are unavailable. The impact of high dimensionality on the covariance matrix estimation based on the factor structure is then studied.

  18. High-dimensional covariance matrix estimation in approximate factor models

    CERN Document Server

    Fan, Jianqing; Mincheva, Martina; 10.1214/11-AOS944

    2012-01-01

    The variance--covariance matrix plays a central role in the inferential theories of high-dimensional factor models in finance and economics. Popular regularization methods of directly exploiting sparsity are not directly applicable to many financial problems. Classical methods of estimating the covariance matrices are based on the strict factor models, assuming independent idiosyncratic components. This assumption, however, is restrictive in practical applications. By assuming sparse error covariance matrix, we allow the presence of the cross-sectional correlation even after taking out common factors, and it enables us to combine the merits of both methods. We estimate the sparse covariance using the adaptive thresholding technique as in Cai and Liu [J. Amer. Statist. Assoc. 106 (2011) 672--684], taking into account the fact that direct observations of the idiosyncratic components are unavailable. The impact of high dimensionality on the covariance matrix estimation based on the factor structure is then studi...

  19. NParCov3: A SAS/IML Macro for Nonparametric Randomization-Based Analysis of Covariance

    Directory of Open Access Journals (Sweden)

    Richard C. Zink

    2012-07-01

    Full Text Available Analysis of covariance serves two important purposes in a randomized clinical trial. First, there is a reduction of variance for the treatment effect which provides more powerful statistical tests and more precise confidence intervals. Second, it provides estimates of the treatment effect which are adjusted for random imbalances of covariates between the treatment groups. The nonparametric analysis of covariance method of Koch, Tangen, Jung, and Amara (1998 defines a very general methodology using weighted least-squares to generate covariate-adjusted treatment effects with minimal assumptions. This methodology is general in its applicability to a variety of outcomes, whether continuous, binary, ordinal, incidence density or time-to-event. Further, its use has been illustrated in many clinical trial settings, such as multi-center, dose-response and non-inferiority trials.NParCov3 is a SAS/IML macro written to conduct the nonparametric randomization-based covariance analyses of Koch et al. (1998. The software can analyze a variety of outcomes and can account for stratification. Data from multiple clinical trials will be used for illustration.

  20. High-dimensional covariance matrix estimation with missing observations

    OpenAIRE

    Lounici, Karim

    2014-01-01

    In this paper, we study the problem of high-dimensional covariance matrix estimation with missing observations. We propose a simple procedure computationally tractable in high-dimension and that does not require imputation of the missing data. We establish non-asymptotic sparsity oracle inequalities for the estimation of the covariance matrix involving the Frobenius and the spectral norms which are valid for any setting of the sample size, probability of a missing observation and the dimensio...

  1. Perturbative approach to covariance matrix of the matter power spectrum

    CERN Document Server

    Mohammed, Irshad; Vlah, Zvonimir

    2016-01-01

    We evaluate the covariance matrix of the matter power spectrum using perturbation theory up to dominant terms at 1-loop order and compare it to numerical simulations. We decompose the covariance matrix into the disconnected (Gaussian) part, trispectrum from the modes outside the survey (beat coupling or super-sample variance), and trispectrum from the modes inside the survey, and show how the different components contribute to the overall covariance matrix. We find the agreement with the simulations is at a 10\\% level up to $k \\sim 1 h {\\rm Mpc^{-1}}$. We show that all the connected components are dominated by the large-scale modes ($k<0.1 h {\\rm Mpc^{-1}}$), regardless of the value of the wavevectors $k,\\, k'$ of the covariance matrix, suggesting that one must be careful in applying the jackknife or bootstrap methods to the covariance matrix. We perform an eigenmode decomposition of the connected part of the covariance matrix, showing that at higher $k$ it is dominated by a single eigenmode. The full cova...

  2. The Performance Analysis Based on SAR Sample Covariance Matrix

    Directory of Open Access Journals (Sweden)

    Esra Erten

    2012-03-01

    Full Text Available Multi-channel systems appear in several fields of application in science. In the Synthetic Aperture Radar (SAR context, multi-channel systems may refer to different domains, as multi-polarization, multi-interferometric or multi-temporal data, or even a combination of them. Due to the inherent speckle phenomenon present in SAR images, the statistical description of the data is almost mandatory for its utilization. The complex images acquired over natural media present in general zero-mean circular Gaussian characteristics. In this case, second order statistics as the multi-channel covariance matrix fully describe the data. For practical situations however, the covariance matrix has to be estimated using a limited number of samples, and this sample covariance matrix follow the complex Wishart distribution. In this context, the eigendecomposition of the multi-channel covariance matrix has been shown in different areas of high relevance regarding the physical properties of the imaged scene. Specifically, the maximum eigenvalue of the covariance matrix has been frequently used in different applications as target or change detection, estimation of the dominant scattering mechanism in polarimetric data, moving target indication, etc. In this paper, the statistical behavior of the maximum eigenvalue derived from the eigendecomposition of the sample multi-channel covariance matrix in terms of multi-channel SAR images is simplified for SAR community. Validation is performed against simulated data and examples of estimation and detection problems using the analytical expressions are as well given.

  3. Covariance, correlation matrix, and the multiscale community structure of networks.

    Science.gov (United States)

    Shen, Hua-Wei; Cheng, Xue-Qi; Fang, Bin-Xing

    2010-07-01

    Empirical studies show that real world networks often exhibit multiple scales of topological descriptions. However, it is still an open problem how to identify the intrinsic multiple scales of networks. In this paper, we consider detecting the multiscale community structure of network from the perspective of dimension reduction. According to this perspective, a covariance matrix of network is defined to uncover the multiscale community structure through the translation and rotation transformations. It is proved that the covariance matrix is the unbiased version of the well-known modularity matrix. We then point out that the translation and rotation transformations fail to deal with the heterogeneous network, which is very common in nature and society. To address this problem, a correlation matrix is proposed through introducing the rescaling transformation into the covariance matrix. Extensive tests on real world and artificial networks demonstrate that the correlation matrix significantly outperforms the covariance matrix, identically the modularity matrix, as regards identifying the multiscale community structure of network. This work provides a novel perspective to the identification of community structure and thus various dimension reduction methods might be used for the identification of community structure. Through introducing the correlation matrix, we further conclude that the rescaling transformation is crucial to identify the multiscale community structure of network, as well as the translation and rotation transformations.

  4. A New Test for a Normal Covariance Matrix

    Institute of Scientific and Technical Information of China (English)

    禹建奇

    2015-01-01

    The problem of testing the normal covariance matrix equal to a specified matrix is considered.A new Chi-Square test statistic is derived for multivariate normal population.Unlike the likelihood ratio test,the new test is an exact one.

  5. Structural damage detection based on covariance of covariance matrix with general white noise excitation

    Science.gov (United States)

    Hui, Yi; Law, Siu Seong; Ku, Chiu Jen

    2017-02-01

    Covariance of the auto/cross-covariance matrix based method is studied for the damage identification of a structure with illustrations on its advantages and limitations. The original method is extended for structures under direct white noise excitations. The auto/cross-covariance function of the measured acceleration and its corresponding derivatives are formulated analytically, and the method is modified in two new strategies to enable successful identification with much fewer sensors. Numerical examples are adopted to illustrate the improved method, and the effects of sampling frequency and sampling duration are discussed. Results show that the covariance of covariance calculated from responses of higher order modes of a structure play an important role to the accurate identification of local damage in a structure.

  6. An Empirical State Error Covariance Matrix for Batch State Estimation

    Science.gov (United States)

    Frisbee, Joseph H., Jr.

    2011-01-01

    State estimation techniques serve effectively to provide mean state estimates. However, the state error covariance matrices provided as part of these techniques suffer from some degree of lack of confidence in their ability to adequately describe the uncertainty in the estimated states. A specific problem with the traditional form of state error covariance matrices is that they represent only a mapping of the assumed observation error characteristics into the state space. Any errors that arise from other sources (environment modeling, precision, etc.) are not directly represented in a traditional, theoretical state error covariance matrix. Consider that an actual observation contains only measurement error and that an estimated observation contains all other errors, known and unknown. It then follows that a measurement residual (the difference between expected and observed measurements) contains all errors for that measurement. Therefore, a direct and appropriate inclusion of the actual measurement residuals in the state error covariance matrix will result in an empirical state error covariance matrix. This empirical state error covariance matrix will fully account for the error in the state estimate. By way of a literal reinterpretation of the equations involved in the weighted least squares estimation algorithm, it is possible to arrive at an appropriate, and formally correct, empirical state error covariance matrix. The first specific step of the method is to use the average form of the weighted measurement residual variance performance index rather than its usual total weighted residual form. Next it is helpful to interpret the solution to the normal equations as the average of a collection of sample vectors drawn from a hypothetical parent population. From here, using a standard statistical analysis approach, it directly follows as to how to determine the standard empirical state error covariance matrix. This matrix will contain the total uncertainty in the

  7. Web Tool for Constructing a Covariance Matrix from EXFOR Uncertainties

    Directory of Open Access Journals (Sweden)

    Zerkin V.

    2012-05-01

    Full Text Available The experimental nuclear reaction database EXFOR contains almost no covariance data because most experimentalists provide experimental data only with uncertainties. With the tool described here a user can construct an experimental covariance matrix from uncertainties using general assumptions when uncertainty information given in EXFOR is poor (or even absent. The tool is publically available in the IAEA EXFOR Web retrieval system [1].

  8. Bayes linear covariance matrix adjustment for multivariate dynamic linear models

    CERN Document Server

    Wilkinson, Darren J

    2008-01-01

    A methodology is developed for the adjustment of the covariance matrices underlying a multivariate constant time series dynamic linear model. The covariance matrices are embedded in a distribution-free inner-product space of matrix objects which facilitates such adjustment. This approach helps to make the analysis simple, tractable and robust. To illustrate the methods, a simple model is developed for a time series representing sales of certain brands of a product from a cash-and-carry depot. The covariance structure underlying the model is revised, and the benefits of this revision on first order inferences are then examined.

  9. Some Algorithms for the Conditional Mean Vector and Covariance Matrix

    Directory of Open Access Journals (Sweden)

    John F. Monahan

    2006-08-01

    Full Text Available We consider here the problem of computing the mean vector and covariance matrix for a conditional normal distribution, considering especially a sequence of problems where the conditioning variables are changing. The sweep operator provides one simple general approach that is easy to implement and update. A second, more goal-oriented general method avoids explicit computation of the vector and matrix, while enabling easy evaluation of the conditional density for likelihood computation or easy generation from the conditional distribution. The covariance structure that arises from the special case of an ARMA(p, q time series can be exploited for substantial improvements in computational efficiency.

  10. Neutron Resonance Parameters and Covariance Matrix of 239Pu

    Energy Technology Data Exchange (ETDEWEB)

    Derrien, Herve [ORNL; Leal, Luiz C [ORNL; Larson, Nancy M [ORNL

    2008-08-01

    In order to obtain the resonance parameters in a single energy range and the corresponding covariance matrix, a reevaluation of 239Pu was performed with the code SAMMY. The most recent experimental data were analyzed or reanalyzed in the energy range thermal to 2.5 keV. The normalization of the fission cross section data was reconsidered by taking into account the most recent measurements of Weston et al. and Wagemans et al. A full resonance parameter covariance matrix was generated. The method used to obtain realistic uncertainties on the average cross section calculated by SAMMY or other processing codes was examined.

  11. High-dimensional covariance matrix estimation with missing observations

    CERN Document Server

    Lounici, Karim

    2012-01-01

    In this paper, we study the problem of high-dimensional approximately low-rank covariance matrix estimation with missing observations. We propose a simple procedure computationally tractable in high-dimension and that does not require imputation of the missing data. We establish non-asymptotic sparsity oracle inequalities for the estimation of the covariance matrix with the Frobenius and spectral norms, valid for any setting of the sample size and the dimension of the observations. We further establish minimax lower bounds showing that our rates are minimax optimal up to a logarithmic factor.

  12. Nonparametric randomization-based covariate adjustment for stratified analysis of time-to-event or dichotomous outcomes.

    Science.gov (United States)

    Hussey, Michael A; Koch, Gary G; Preisser, John S; Saville, Benjamin R

    2016-01-01

    Time-to-event or dichotomous outcomes in randomized clinical trials often have analyses using the Cox proportional hazards model or conditional logistic regression, respectively, to obtain covariate-adjusted log hazard (or odds) ratios. Nonparametric Randomization-Based Analysis of Covariance (NPANCOVA) can be applied to unadjusted log hazard (or odds) ratios estimated from a model containing treatment as the only explanatory variable. These adjusted estimates are stratified population-averaged treatment effects and only require a valid randomization to the two treatment groups and avoid key modeling assumptions (e.g., proportional hazards in the case of a Cox model) for the adjustment variables. The methodology has application in the regulatory environment where such assumptions cannot be verified a priori. Application of the methodology is illustrated through three examples on real data from two randomized trials.

  13. Data Covariances from R-Matrix Analyses of Light Nuclei

    Energy Technology Data Exchange (ETDEWEB)

    Hale, G.M., E-mail: ghale@lanl.gov; Paris, M.W.

    2015-01-15

    After first reviewing the parametric description of light-element reactions in multichannel systems using R-matrix theory and features of the general LANL R-matrix analysis code EDA, we describe how its chi-square minimization procedure gives parameter covariances. This information is used, together with analytically calculated sensitivity derivatives, to obtain cross section covariances for all reactions included in the analysis by first-order error propagation. Examples are given of the covariances obtained for systems with few resonances ({sup 5}He) and with many resonances ({sup 13}C ). We discuss the prevalent problem of this method leading to cross section uncertainty estimates that are unreasonably small for large data sets. The answer to this problem appears to be using parameter confidence intervals in place of standard errors.

  14. An Empirical State Error Covariance Matrix Orbit Determination Example

    Science.gov (United States)

    Frisbee, Joseph H., Jr.

    2015-01-01

    State estimation techniques serve effectively to provide mean state estimates. However, the state error covariance matrices provided as part of these techniques suffer from some degree of lack of confidence in their ability to adequately describe the uncertainty in the estimated states. A specific problem with the traditional form of state error covariance matrices is that they represent only a mapping of the assumed observation error characteristics into the state space. Any errors that arise from other sources (environment modeling, precision, etc.) are not directly represented in a traditional, theoretical state error covariance matrix. First, consider that an actual observation contains only measurement error and that an estimated observation contains all other errors, known and unknown. Then it follows that a measurement residual (the difference between expected and observed measurements) contains all errors for that measurement. Therefore, a direct and appropriate inclusion of the actual measurement residuals in the state error covariance matrix of the estimate will result in an empirical state error covariance matrix. This empirical state error covariance matrix will fully include all of the errors in the state estimate. The empirical error covariance matrix is determined from a literal reinterpretation of the equations involved in the weighted least squares estimation algorithm. It is a formally correct, empirical state error covariance matrix obtained through use of the average form of the weighted measurement residual variance performance index rather than the usual total weighted residual form. Based on its formulation, this matrix will contain the total uncertainty in the state estimate, regardless as to the source of the uncertainty and whether the source is anticipated or not. It is expected that the empirical error covariance matrix will give a better, statistical representation of the state error in poorly modeled systems or when sensor performance

  15. Central limit theorem of nonparametric estimate of spectral density functions of sample covariance matrices

    CERN Document Server

    Pan, Guangming; Zhou, Wang

    2010-01-01

    A consistent kernel estimator of the limiting spectral distribution of general sample covariance matrices was introduced in Jing, Pan, Shao and Zhou (2010). The central limit theorem of the kernel estimator is proved in this paper.

  16. Neutron Multiplicity: LANL W Covariance Matrix for Curve Fitting

    Energy Technology Data Exchange (ETDEWEB)

    Wendelberger, James G. [Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

    2016-12-08

    In neutron multiplicity counting one may fit a curve by minimizing an objective function, χ$2\\atop{n}$. The objective function includes the inverse of an n by n matrix of covariances, W. The inverse of the W matrix has a closed form solution. In addition W-1 is a tri-diagonal matrix. The closed form and tridiagonal nature allows for a simpler expression of the objective function χ$2\\atop{n}$. Minimization of this simpler expression will provide the optimal parameters for the fitted curve.

  17. MIMO Radar Transmit Beampattern Design Without Synthesising the Covariance Matrix

    KAUST Repository

    Ahmed, Sajid

    2013-10-28

    Compared to phased-array, multiple-input multiple-output (MIMO) radars provide more degrees-offreedom (DOF) that can be exploited for improved spatial resolution, better parametric identifiability, lower side-lobe levels at the transmitter/receiver, and design variety of transmit beampatterns. The design of the transmit beampattern generally requires the waveforms to have arbitrary auto- and crosscorrelation properties. The generation of such waveforms is a two step complicated process. In the first step a waveform covariance matrix is synthesised, which is a constrained optimisation problem. In the second step, to realise this covariance matrix actual waveforms are designed, which is also a constrained optimisation problem. Our proposed scheme converts this two step constrained optimisation problem into a one step unconstrained optimisation problem. In the proposed scheme, in contrast to synthesising the covariance matrix for the desired beampattern, nT independent finite-alphabet constantenvelope waveforms are generated and pre-processed, with weight matrix W, before transmitting from the antennas. In this work, two weight matrices are proposed that can be easily optimised for the desired symmetric and non-symmetric beampatterns and guarantee equal average power transmission from each antenna. Simulation results validate our claims.

  18. Nonparametric estimate of spectral density functions of sample covariance matrices: A first step

    OpenAIRE

    2012-01-01

    The density function of the limiting spectral distribution of general sample covariance matrices is usually unknown. We propose to use kernel estimators which are proved to be consistent. A simulation study is also conducted to show the performance of the estimators.

  19. A covariance matrix test for high-dimensional data

    Directory of Open Access Journals (Sweden)

    Saowapha Chaipitak

    2016-10-01

    Full Text Available For the multivariate normally distributed data with the dimension larger than or equal to the number of observations, or the sample size, called high-dimensional normal data, we proposed a test for testing the null hypothesis that the covariance matrix of a normal population is proportional to a given matrix on some conditions when the dimension goes to infinity. We showed that this test statistic is consistent. The asymptotic null and non-null distribution of the test statistic is also given. The performance of the proposed test is evaluated via simulation study and its application.

  20. Large-scale portfolios using realized covariance matrix: evidence from the Japanese stock market

    OpenAIRE

    Masato Ubukata

    2009-01-01

    The objective of this paper is to examine effects of realized covariance matrix estimators based on intraday returns on large-scale minimum-variance equity portfolio optimization. We empirically assess out-of-sample performance of portfolios with different covariance matrix estimators: the realized covariance matrix estimators and Bayesian shrinkage estimators based on the past monthly and daily returns. The main results are: (1) the realized covariance matrix estimators using the past intrad...

  1. Eigenvectors of some large sample covariance matrix ensembles

    CERN Document Server

    Ledoit, Olivier

    2009-01-01

    We consider sample covariance matrices $S_N=\\frac{1}{p}\\Sigma_N^{1/2}X_NX_N^* \\Sigma_N^{1/2}$ where $X_N$ is a $N \\times p$ real or complex matrix with i.i.d. entries with finite $12^{\\rm th}$ moment and $\\Sigma_N$ is a $N \\times N$ positive definite matrix. In addition we assume that the spectral measure of $\\Sigma_N$ almost surely converges to some limiting probability distribution as $N \\to \\infty$ and $p/N \\to \\gamma >0.$ We quantify the relationship between sample and population eigenvectors by studying the asymptotics of functionals of the type $\\frac{1}{N} \\text{Tr} (g(\\Sigma_N) (S_N-zI)^{-1})),$ where $I$ is the identity matrix, $g$ is a bounded function and $z$ is a complex number. This is then used to compute the asymptotically optimal bias correction for sample eigenvalues, paving the way for a new generation of improved estimators of the covariance matrix and its inverse.

  2. Improving on the empirical covariance matrix using truncated PCA with white noise residuals

    CERN Document Server

    Jewson, S

    2005-01-01

    The empirical covariance matrix is not necessarily the best estimator for the population covariance matrix: we describe a simple method which gives better estimates in two examples. The method models the covariance matrix using truncated PCA with white noise residuals. Jack-knife cross-validation is used to find the truncation that maximises the out-of-sample likelihood score.

  3. Random matrix theory and robust covariance matrix estimation for financial data

    CERN Document Server

    Frahm, G; Frahm, Gabriel; Jaekel, Uwe

    2005-01-01

    The traditional class of elliptical distributions is extended to allow for asymmetries. A completely robust dispersion matrix estimator (the `spectral estimator') for the new class of `generalized elliptical distributions' is presented. It is shown that the spectral estimator corresponds to an M-estimator proposed by Tyler (1983) in the context of elliptical distributions. Both the generalization of elliptical distributions and the development of a robust dispersion matrix estimator are motivated by the stylized facts of empirical finance. Random matrix theory is used for analyzing the linear dependence structure of high-dimensional data. It is shown that the Marcenko-Pastur law fails if the sample covariance matrix is considered as a random matrix in the context of elliptically distributed and heavy tailed data. But substituting the sample covariance matrix by the spectral estimator resolves the problem and the Marcenko-Pastur law remains valid.

  4. USING COVARIANCE MATRIX FOR CHANGE DETECTION OF POLARIMETRIC SAR DATA

    Directory of Open Access Journals (Sweden)

    M. Esmaeilzade

    2017-09-01

    Full Text Available Nowadays change detection is an important role in civil and military fields. The Synthetic Aperture Radar (SAR images due to its independent of atmospheric conditions and cloud cover, have attracted much attention in the change detection applications. When the SAR data are used, one of the appropriate ways to display the backscattered signal is using covariance matrix that follows the Wishart distribution. Based on this distribution a statistical test for equality of two complex variance-covariance matrices can be used. In this study, two full polarization data in band L from UAVSAR are used for change detection in agricultural fields and urban areas in the region of United States which the first image belong to 2014 and the second one is from 2017. To investigate the effect of polarization on the rate of change, full polarization data and dual polarization data were used and the results were compared. According to the results, full polarization shows more changes than dual polarization.

  5. A New Heteroskedastic Consistent Covariance Matrix Estimator using Deviance Measure

    Directory of Open Access Journals (Sweden)

    Nuzhat Aftab

    2016-06-01

    Full Text Available In this article we propose a new heteroskedastic consistent covariance matrix estimator, HC6, based on deviance measure. We have studied and compared the finite sample behavior of the new test and compared it with other this kind of estimators, HC1, HC3 and HC4m, which are used in case of leverage observations. Simulation study is conducted to study the effect of various levels of heteroskedasticity on the size and power of quasi-t test with HC estimators. Results show that the test statistic based on our new suggested estimator has better asymptotic approximation and less size distortion as compared to other estimators for small sample sizes when high level ofheteroskedasticity is present in data.

  6. Analysis of gene set using shrinkage covariance matrix approach

    Science.gov (United States)

    Karjanto, Suryaefiza; Aripin, Rasimah

    2013-09-01

    Microarray methodology has been exploited for different applications such as gene discovery and disease diagnosis. This technology is also used for quantitative and highly parallel measurements of gene expression. Recently, microarrays have been one of main interests of statisticians because they provide a perfect example of the paradigms of modern statistics. In this study, the alternative approach to estimate the covariance matrix has been proposed to solve the high dimensionality problem in microarrays. The extension of traditional Hotelling's T2 statistic is constructed for determining the significant gene sets across experimental conditions using shrinkage approach. Real data sets were used as illustrations to compare the performance of the proposed methods with other methods. The results across the methods are consistent, implying that this approach provides an alternative to existing techniques.

  7. Multiple mainlobe interferences suppression based on subspace matrix filtering and covariance matrix reconstruction

    Science.gov (United States)

    Wang, Yasen; Bao, Qinglong; Chen, Zengping

    2016-07-01

    In order to suppress multiple mainlobe interferences and sidelobe interferences simultaneously, a mainlobe interference suppression algorithm is proposed. In this algorithm, the number of mainlobe interferences is estimated through a matrix filter at first. Then, the eigenvectors associated with mainlobe interference are determined and the eigen-projection matrix can be calculated. Next, the sidelobe-interference-plus-noise covariance matrix is reconstructed through eigenvalue replacement procedure. Finally, we can get the adaptive weight vector. Simulation results demonstrate the effectiveness of the proposed method when multiple mainlobe interferences exist.

  8. Using Analysis State to Construct a Forecast Error Covariance Matrix in Ensemble Kalman Filter Assimilation

    Institute of Scientific and Technical Information of China (English)

    ZHENG Xiaogu; WU Guocan; ZHANG Shupeng; LIANG Xiao; DAI Yongjiu; LI Yong

    2013-01-01

    Correctly estimating the forecast error covariance matrix is a key step in any data assimilation scheme.If it is not correctly estimated,the assimilated states could be far from the true states.A popular method to address this problem is error covariance matrix inflation.That is,to multiply the forecast error covariance matrix by an appropriate factor.In this paper,analysis states are used to construct the forecast error covariance matrix and an adaptive estimation procedure associated with the error covariance matrix inflation technique is developed.The proposed assimilation scheme was tested on the Lorenz-96 model and 2D Shallow Water Equation model,both of which are associated with spatially correlated observational systems.The experiments showed that by introducing the proposed structure of the forecast error covariance matrix and applying its adaptive estimation procedure,the assimilation results were further improved.

  9. Large-scale portfolios using realized covariance matrix: evidence from the Japanese stock market

    OpenAIRE

    Masato Ubukata

    2010-01-01

    This paper examines effects of realized covariance matrix estimators based on high-frequency data on large-scale minimum-variance equity portfolio optimization. The main results are: (i) the realized covariance matrix estimators yield a lower standard deviation of large-scale portfolio returns than Bayesian shrinkage estimators based on monthly and daily historical returns; (ii) gains to switching to strategies using the realized covariance matrix estimators are higher for an investor with hi...

  10. Ocean spectral data assimilation without background error covariance matrix

    Science.gov (United States)

    Chu, Peter C.; Fan, Chenwu; Margolina, Tetyana

    2016-09-01

    Predetermination of background error covariance matrix B is challenging in existing ocean data assimilation schemes such as the optimal interpolation (OI). An optimal spectral decomposition (OSD) has been developed to overcome such difficulty without using the B matrix. The basis functions are eigenvectors of the horizontal Laplacian operator, pre-calculated on the base of ocean topography, and independent on any observational data and background fields. Minimization of analysis error variance is achieved by optimal selection of the spectral coefficients. Optimal mode truncation is dependent on the observational data and observational error variance and determined using the steep-descending method. Analytical 2D fields of large and small mesoscale eddies with white Gaussian noises inside a domain with four rigid and curved boundaries are used to demonstrate the capability of the OSD method. The overall error reduction using the OSD is evident in comparison to the OI scheme. Synoptic monthly gridded world ocean temperature, salinity, and absolute geostrophic velocity datasets produced with the OSD method and quality controlled by the NOAA National Centers for Environmental Information (NCEI) are also presented.

  11. Accuracy of Pseudo-Inverse Covariance Learning--A Random Matrix Theory Analysis.

    Science.gov (United States)

    Hoyle, David C

    2011-07-01

    For many learning problems, estimates of the inverse population covariance are required and often obtained by inverting the sample covariance matrix. Increasingly for modern scientific data sets, the number of sample points is less than the number of features and so the sample covariance is not invertible. In such circumstances, the Moore-Penrose pseudo-inverse sample covariance matrix, constructed from the eigenvectors corresponding to nonzero sample covariance eigenvalues, is often used as an approximation to the inverse population covariance matrix. The reconstruction error of the pseudo-inverse sample covariance matrix in estimating the true inverse covariance can be quantified via the Frobenius norm of the difference between the two. The reconstruction error is dominated by the smallest nonzero sample covariance eigenvalues and diverges as the sample size becomes comparable to the number of features. For high-dimensional data, we use random matrix theory techniques and results to study the reconstruction error for a wide class of population covariance matrices. We also show how bagging and random subspace methods can result in a reduction in the reconstruction error and can be combined to improve the accuracy of classifiers that utilize the pseudo-inverse sample covariance matrix. We test our analysis on both simulated and benchmark data sets.

  12. Covariance matrix adaptation for multi-objective optimization.

    Science.gov (United States)

    Igel, Christian; Hansen, Nikolaus; Roth, Stefan

    2007-01-01

    The covariance matrix adaptation evolution strategy (CMA-ES) is one of the most powerful evolutionary algorithms for real-valued single-objective optimization. In this paper, we develop a variant of the CMA-ES for multi-objective optimization (MOO). We first introduce a single-objective, elitist CMA-ES using plus-selection and step size control based on a success rule. This algorithm is compared to the standard CMA-ES. The elitist CMA-ES turns out to be slightly faster on unimodal functions, but is more prone to getting stuck in sub-optimal local minima. In the new multi-objective CMAES (MO-CMA-ES) a population of individuals that adapt their search strategy as in the elitist CMA-ES is maintained. These are subject to multi-objective selection. The selection is based on non-dominated sorting using either the crowding-distance or the contributing hypervolume as second sorting criterion. Both the elitist single-objective CMA-ES and the MO-CMA-ES inherit important invariance properties, in particular invariance against rotation of the search space, from the original CMA-ES. The benefits of the new MO-CMA-ES in comparison to the well-known NSGA-II and to NSDE, a multi-objective differential evolution algorithm, are experimentally shown.

  13. On the regularity of the covariance matrix of a discretized scalar field on the sphere

    Science.gov (United States)

    Bilbao-Ahedo, J. D.; Barreiro, R. B.; Herranz, D.; Vielva, P.; Martínez-González, E.

    2017-02-01

    We present a comprehensive study of the regularity of the covariance matrix of a discretized field on the sphere. In a particular situation, the rank of the matrix depends on the number of pixels, the number of spherical harmonics, the symmetries of the pixelization scheme and the presence of a mask. Taking into account the above mentioned components, we provide analytical expressions that constrain the rank of the matrix. They are obtained by expanding the determinant of the covariance matrix as a sum of determinants of matrices made up of spherical harmonics. We investigate these constraints for five different pixelizations that have been used in the context of Cosmic Microwave Background (CMB) data analysis: Cube, Icosahedron, Igloo, GLESP and HEALPix, finding that, at least in the considered cases, the HEALPix pixelization tends to provide a covariance matrix with a rank closer to the maximum expected theoretical value than the other pixelizations. The effect of the propagation of numerical errors in the regularity of the covariance matrix is also studied for different computational precisions, as well as the effect of adding a certain level of noise in order to regularize the matrix. In addition, we investigate the application of the previous results to a particular example that requires the inversion of the covariance matrix: the estimation of the CMB temperature power spectrum through the Quadratic Maximum Likelihood algorithm. Finally, some general considerations in order to achieve a regular covariance matrix are also presented.

  14. Rapid Divergence of Genetic Variance-Covariance Matrix within a Natural Population

    NARCIS (Netherlands)

    Doroszuk, A.; Wojewodzic, M.W.; Gort, G.; Kammenga, J.E.

    2008-01-01

    The matrix of genetic variances and covariances (G matrix) represents the genetic architecture of multiple traits sharing developmental and genetic processes and is central for predicting phenotypic evolution. These predictions require that the G matrix be stable. Yet the timescale and conditions pr

  15. Testing Constancy of the Error Covariance Matrix in Vector Models against Parametric Alternatives using a Spectral Decomposition

    DEFF Research Database (Denmark)

    Yang, Yukay

    I consider multivariate (vector) time series models in which the error covariance matrix may be time-varying. I derive a test of constancy of the error covariance matrix against the alternative that the covariance matrix changes over time. I design a new family of Lagrange-multiplier tests against...

  16. Preconditioning of Nonlinear Mixed Effects Models for Stabilisation of Variance-Covariance Matrix Computations

    National Research Council Canada - National Science Library

    Aoki, Yasunori; Nordgren, Rikard; Hooker, Andrew C

    2016-01-01

    ... a bottleneck in the analysis. We propose a preconditioning method for non-linear mixed effects models used in pharmacometric analyses to stabilise the computation of the variance-covariance matrix...

  17. Improved covariance matrix estimation in spectrally inhomogeneous sea clutter with application to adaptive small boat detection.

    CSIR Research Space (South Africa)

    Herselman, PL

    2008-09-01

    Full Text Available Asymptotically optimal coherent detection techniques yield sub-clutter visibility in heavy-tailed sea clutter. The adaptive linear quadratic detector inherently assumes spectral homogeneity for the reference window of the covariance matrix estimator...

  18. ℋ-matrix techniques for approximating large covariance matrices and estimating its parameters

    KAUST Repository

    Litvinenko, Alexander

    2016-10-25

    In this work the task is to use the available measurements to estimate unknown hyper-parameters (variance, smoothness parameter and covariance length) of the covariance function. We do it by maximizing the joint log-likelihood function. This is a non-convex and non-linear problem. To overcome cubic complexity in linear algebra, we approximate the discretised covariance function in the hierarchical (ℋ-) matrix format. The ℋ-matrix format has a log-linear computational cost and storage O(knlogn), where rank k is a small integer. On each iteration step of the optimization procedure the covariance matrix itself, its determinant and its Cholesky decomposition are recomputed within ℋ-matrix format. (© 2016 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)

  19. Spatio-Temporal Audio Enhancement Based on IAA Noise Covariance Matrix Estimates

    DEFF Research Database (Denmark)

    Nørholm, Sidsel Marie; Jensen, Jesper Rindom; Christensen, Mads Græsbøll

    2014-01-01

    to an amplitude and phase estimation (APES) based filter. For a fixed number of samples, the performance in terms of signal-to-noise ratio can be increased by using the IAA method, whereas if the filter size is fixed and the number of samples in the APES based filter is increased, the APES based filter performs......A method for estimating the noise covariance matrix in a mul- tichannel setup is proposed. The method is based on the iter- ative adaptive approach (IAA), which only needs short seg- ments of data to estimate the covariance matrix. Therefore, the method can be used for fast varying signals....... The method is based on an assumption of the desired signal being harmonic, which is used for estimating the noise covariance matrix from the covariance matrix of the observed signal. The noise co- variance estimate is used in the linearly constrained minimum variance (LCMV) filter and compared...

  20. Weak signal detection in hyperspectral imagery using sparse matrix transform (SMT) covariance estimation

    Energy Technology Data Exchange (ETDEWEB)

    Theiler, James P [Los Alamos National Laboratory; Cao, Guangzhi [PURDUE UNIV; Bouman, Charles A [PURDUE UNIV

    2009-01-01

    Many detection algorithms in hyperspectral image analysis, from well-characterized gaseous and solid targets to deliberately uncharacterized anomalies and anomlous changes, depend on accurately estimating the covariance matrix of the background. In practice, the background covariance is estimated from samples in the image, and imprecision in this estimate can lead to a loss of detection power. In this paper, we describe the sparse matrix transform (SMT) and investigate its utility for estimating the covariance matrix from a limited number of samples. The SMT is formed by a product of pairwise coordinate (Givens) rotations, which can be efficiently estimated using greedy optimization. Experiments on hyperspectral data show that the estimate accurately reproduces even small eigenvalues and eigenvectors. In particular, we find that using the SMT to estimate the covariance matrix used in the adaptive matched filter leads to consistently higher signal-to-noise ratios.

  1. Construction and use of gene expression covariation matrix

    Directory of Open Access Journals (Sweden)

    Bellis Michel

    2009-07-01

    Full Text Available Abstract Background One essential step in the massive analysis of transcriptomic profiles is the calculation of the correlation coefficient, a value used to select pairs of genes with similar or inverse transcriptional profiles across a large fraction of the biological conditions examined. Until now, the choice between the two available methods for calculating the coefficient has been dictated mainly by technological considerations. Specifically, in analyses based on double-channel techniques, researchers have been required to use covariation correlation, i.e. the correlation between gene expression changes measured between several pairs of biological conditions, expressed for example as fold-change. In contrast, in analyses of single-channel techniques scientists have been restricted to the use of coexpression correlation, i.e. correlation between gene expression levels. To our knowledge, nobody has ever examined the possible benefits of using covariation instead of coexpression in massive analyses of single channel microarray results. Results We describe here how single-channel techniques can be treated like double-channel techniques and used to generate both gene expression changes and covariation measures. We also present a new method that allows the calculation of both positive and negative correlation coefficients between genes. First, we perform systematic comparisons between two given biological conditions and classify, for each comparison, genes as increased (I, decreased (D, or not changed (N. As a result, the original series of n gene expression level measures assigned to each gene is replaced by an ordered string of n(n-1/2 symbols, e.g. IDDNNIDID....DNNNNNNID, with the length of the string corresponding to the number of comparisons. In a second step, positive and negative covariation matrices (CVM are constructed by calculating statistically significant positive or negative correlation scores for any pair of genes by comparing their

  2. Exploring Eddy-Covariance Measurements Using a Spatial Approach: The Eddy Matrix

    Science.gov (United States)

    Engelmann, Christian; Bernhofer, Christian

    2016-10-01

    Taylor's frozen turbulence hypothesis states that "standard" eddy-covariance measurements of fluxes at a fixed location can replace a spatial ensemble of instantaneous values at multiple locations. For testing this hypothesis, a unique turbulence measurement set-up was used for two measurement campaigns over desert (Namibia) and grassland (Germany) in 2012. This "Eddy Matrix" combined nine ultrasonic anemometer-thermometers and 17 thermocouples in a 10 m × 10 m regular grid with 2.5-m grid distance. The instantaneous buoyancy flux derived from the spatial eddy covariance of the Eddy Matrix was highly variable in time (from -0.3 to 1 m K s^{-1}). However, the 10-min average reflected 83 % of the reference eddy-covariance flux with a good correlation. By introducing a combined eddy-covariance method (the spatial eddy covariance plus the additional flux of the temporal eddy covariance of the spatial mean values), the mean flux increases by 9 % relative to the eddy-covariance reference. Considering the typical underestimation of fluxes by the standard eddy-covariance method, this is seen as an improvement. Within the limits of the Eddy Matrix, Taylor's hypothesis is supported by the results.

  3. Covariant 4-dimensional fuzzy spheres, matrix models and higher spin

    Science.gov (United States)

    Sperling, Marcus; Steinacker, Harold C.

    2017-09-01

    We study in detail generalized 4-dimensional fuzzy spheres with twisted extra dimensions. These spheres can be viewed as SO(5) -equivariant projections of quantized coadjoint orbits of SO(6) . We show that they arise as solutions in Yang-Mills matrix models, which naturally leads to higher-spin gauge theories on S 4. Several types of embeddings in matrix models are found, including one with self-intersecting fuzzy extra dimensions \

  4. An Alternative Method for Computing Mean and Covariance Matrix of Some Multivariate Distributions

    Science.gov (United States)

    Radhakrishnan, R.; Choudhury, Askar

    2009-01-01

    Computing the mean and covariance matrix of some multivariate distributions, in particular, multivariate normal distribution and Wishart distribution are considered in this article. It involves a matrix transformation of the normal random vector into a random vector whose components are independent normal random variables, and then integrating…

  5. Empirical data and the variance-covariance matrix for the 1969 Smithsonian Standard Earth (2)

    Science.gov (United States)

    Gaposchkin, E. M.

    1972-01-01

    The empirical data used in the 1969 Smithsonian Standard Earth (2) are presented. The variance-covariance matrix, or the normal equations, used for correlation analysis, are considered. The format and contents of the matrix, available on magnetic tape, are described and a sample printout is given.

  6. A synthetic covariance matrix for monitoring by terrestrial laser scanning

    Science.gov (United States)

    Kauker, Stephanie; Schwieger, Volker

    2017-06-01

    Modelling correlations within laser scanning point clouds can be achieved by using synthetic covariance matrices. These are based on the elementary error model which contains different groups of correlations: non-correlating, functional correlating and stochastic correlating. By applying the elementary error model on terrestrial laser scanning several groups of error sources should be considered: instrumental, atmospheric and object based. This contribution presents calculations for the Leica HDS 7000. The determined variances and the spatial correlations of the points are estimated and discussed. Hereby, the mean standard deviation of the point cloud is up to 0.6 mm and the mean correlation is about 0.6 with respect to 5 m scanning range. The change of these numerical values compared to previous publications as Kauker and Schwieger [17] is mainly caused by the complete consideration of the object related error sources.

  7. Bootstrapping integrated covariance matrix estimators in noisy jump-diffusion models with non-synchronous trading

    DEFF Research Database (Denmark)

    Hounyo, Ulrich

    We propose a bootstrap mehtod for estimating the distribution (and functionals of it such as the variance) of various integrated covariance matrix estimators. In particular, we first adapt the wild blocks of blocks bootsratp method suggested for the pre-averaged realized volatility estimator......-studentized statistics, our results justify using the bootstrap to esitmate the covariance matrix of a broad class of covolatility estimators. The bootstrap variance estimator is positive semi-definite by construction, an appealing feature that is not always shared by existing variance estimators of the integrated...

  8. The eigenvalues of the sample covariance matrix of a multivariate heavy-tailed stochastic volatility model

    DEFF Research Database (Denmark)

    Janssen, Anja; Mikosch, Thomas Valentin; Rezapour, Mohsen

    2017-01-01

    We consider a multivariate heavy-tailed stochastic volatility model and analyze the large-sample behavior of its sample covariance matrix. We study the limiting behavior of its entries in the infinite-variance case and derive results for the ordered eigenvalues and corresponding eigenvectors...... of the sample covariance matrix. While we show that in the case of heavy-tailed innovations the limiting behavior resembles that of completely independent observations, we also derive that in the case of a heavy-tailed volatility sequence the possible limiting behavior is more diverse, i.e. allowing...

  9. Shrinkage covariance matrix approach based on robust trimmed mean in gene sets detection

    Science.gov (United States)

    Karjanto, Suryaefiza; Ramli, Norazan Mohamed; Ghani, Nor Azura Md; Aripin, Rasimah; Yusop, Noorezatty Mohd

    2015-02-01

    Microarray involves of placing an orderly arrangement of thousands of gene sequences in a grid on a suitable surface. The technology has made a novelty discovery since its development and obtained an increasing attention among researchers. The widespread of microarray technology is largely due to its ability to perform simultaneous analysis of thousands of genes in a massively parallel manner in one experiment. Hence, it provides valuable knowledge on gene interaction and function. The microarray data set typically consists of tens of thousands of genes (variables) from just dozens of samples due to various constraints. Therefore, the sample covariance matrix in Hotelling's T2 statistic is not positive definite and become singular, thus it cannot be inverted. In this research, the Hotelling's T2 statistic is combined with a shrinkage approach as an alternative estimation to estimate the covariance matrix to detect significant gene sets. The use of shrinkage covariance matrix overcomes the singularity problem by converting an unbiased to an improved biased estimator of covariance matrix. Robust trimmed mean is integrated into the shrinkage matrix to reduce the influence of outliers and consequently increases its efficiency. The performance of the proposed method is measured using several simulation designs. The results are expected to outperform existing techniques in many tested conditions.

  10. A Note on the Eigensystem of the Covariance Matrix of Dichotomous Guttman Items.

    Science.gov (United States)

    Davis-Stober, Clintin P; Doignon, Jean-Paul; Suck, Reinhard

    2015-01-01

    We consider the covariance matrix for dichotomous Guttman items under a set of uniformity conditions, and obtain closed-form expressions for the eigenvalues and eigenvectors of the matrix. In particular, we describe the eigenvalues and eigenvectors of the matrix in terms of trigonometric functions of the number of items. Our results parallel those of Zwick (1987) for the correlation matrix under the same uniformity conditions. We provide an explanation for certain properties of principal components under Guttman scalability which have been first reported by Guttman (1950).

  11. A note on the eigensystem of the covariance matrix of dichotomous Guttman items

    Directory of Open Access Journals (Sweden)

    Clintin P Davis-Stober

    2015-12-01

    Full Text Available We consider the sample covariance matrix for dichotomous Guttman items under a set of uniformity conditions, and obtain closed-form expressions for the eigenvalues and eigenvectors of the matrix. In particular, we describe the eigenvalues and eigenvectors of the matrix in terms of trigonometric functions of the number of items. Our results parallel those of Zwick (1987 for the correlation matrix under the same uniformity conditions. We provide an explanation for certain properties of principal components under Guttman scalability which have been first reported by Guttman (1950.

  12. Preconditioning of Nonlinear Mixed Effects Models for Stabilisation of Variance-Covariance Matrix Computations.

    Science.gov (United States)

    Aoki, Yasunori; Nordgren, Rikard; Hooker, Andrew C

    2016-03-01

    As the importance of pharmacometric analysis increases, more and more complex mathematical models are introduced and computational error resulting from computational instability starts to become a bottleneck in the analysis. We propose a preconditioning method for non-linear mixed effects models used in pharmacometric analyses to stabilise the computation of the variance-covariance matrix. Roughly speaking, the method reparameterises the model with a linear combination of the original model parameters so that the Hessian matrix of the likelihood of the reparameterised model becomes close to an identity matrix. This approach will reduce the influence of computational error, for example rounding error, to the final computational result. We present numerical experiments demonstrating that the stabilisation of the computation using the proposed method can recover failed variance-covariance matrix computations, and reveal non-identifiability of the model parameters.

  13. HLIBCov: Parallel Hierarchical Matrix Approximation of Large Covariance Matrices and Likelihoods with Applications in Parameter Identification

    KAUST Repository

    Litvinenko, Alexander

    2017-09-26

    The main goal of this article is to introduce the parallel hierarchical matrix library HLIBpro to the statistical community. We describe the HLIBCov package, which is an extension of the HLIBpro library for approximating large covariance matrices and maximizing likelihood functions. We show that an approximate Cholesky factorization of a dense matrix of size $2M\\\\times 2M$ can be computed on a modern multi-core desktop in few minutes. Further, HLIBCov is used for estimating the unknown parameters such as the covariance length, variance and smoothness parameter of a Mat\\\\\\'ern covariance function by maximizing the joint Gaussian log-likelihood function. The computational bottleneck here is expensive linear algebra arithmetics due to large and dense covariance matrices. Therefore covariance matrices are approximated in the hierarchical ($\\\\H$-) matrix format with computational cost $\\\\mathcal{O}(k^2n \\\\log^2 n/p)$ and storage $\\\\mathcal{O}(kn \\\\log n)$, where the rank $k$ is a small integer (typically $k<25$), $p$ the number of cores and $n$ the number of locations on a fairly general mesh. We demonstrate a synthetic example, where the true values of known parameters are known. For reproducibility we provide the C++ code, the documentation, and the synthetic data.

  14. Asymptotic properties of eigenmatrices of a large sample covariance matrix

    CERN Document Server

    Bai, Z D; Wong, W K; 10.1214/10-AAP748

    2012-01-01

    Let $S_n=\\frac{1}{n}X_nX_n^*$ where $X_n=\\{X_{ij}\\}$ is a $p\\times n$ matrix with i.i.d. complex standardized entries having finite fourth moments. Let $Y_n(\\mathbf {t}_1,\\mathbf {t}_2,\\sigma)=\\sqrt{p}({\\mathbf {x}}_n(\\mathbf {t}_1)^*(S_n+\\sigma I)^{-1}{\\mathbf {x}}_n(\\mathbf {t}_2)-{\\mathbf {x}}_n(\\mathbf {t}_1)^*{\\mathbf {x}}_n(\\mathbf {t}_2)m_n(\\sigma))$ in which $\\sigma>0$ and $m_n(\\sigma)=\\int\\frac{dF_{y_n}(x)}{x+\\sigma}$ where $F_{y_n}(x)$ is the Mar\\v{c}enko--Pastur law with parameter $y_n=p/n$; which converges to a positive constant as $n\\to\\infty$, and ${\\mathbf {x}}_n(\\mathbf {t}_1)$ and ${\\mathbf {x}}_n(\\mathbf {t}_2)$ are unit vectors in ${\\Bbb{C}}^p$, having indices $\\mathbf {t}_1$ and $\\mathbf {t}_2$, ranging in a compact subset of a finite-dimensional Euclidean space. In this paper, we prove that the sequence $Y_n(\\mathbf {t}_1,\\mathbf {t}_2,\\sigma)$ converges weakly to a $(2m+1)$-dimensional Gaussian process. This result provides further evidence in support of the conjecture that the distribut...

  15. Asymptotic theory for the sample covariance matrix of a heavy-tailed multivariate time series

    DEFF Research Database (Denmark)

    Davis, Richard A.; Mikosch, Thomas Valentin; Pfaffel, Olivier

    2016-01-01

    particular, the time series has infinite fourth moment. We derive the limiting behavior for the largest eigenvalues of the sample covariance matrix and show point process convergence of the normalized eigenvalues. The limiting process has an explicit form involving points of a Poisson process and eigenvalues...

  16. Quantifying lost information due to covariance matrix estimation in parameter inference

    Science.gov (United States)

    Sellentin, Elena; Heavens, Alan F.

    2017-02-01

    Parameter inference with an estimated covariance matrix systematically loses information due to the remaining uncertainty of the covariance matrix. Here, we quantify this loss of precision and develop a framework to hypothetically restore it, which allows to judge how far away a given analysis is from the ideal case of a known covariance matrix. We point out that it is insufficient to estimate this loss by debiasing the Fisher matrix as previously done, due to a fundamental inequality that describes how biases arise in non-linear functions. We therefore develop direct estimators for parameter credibility contours and the figure of merit, finding that significantly fewer simulations than previously thought are sufficient to reach satisfactory precisions. We apply our results to DES Science Verification weak lensing data, detecting a 10 per cent loss of information that increases their credibility contours. No significant loss of information is found for KiDS. For a Euclid-like survey, with about 10 nuisance parameters we find that 2900 simulations are sufficient to limit the systematically lost information to 1 per cent, with an additional uncertainty of about 2 per cent. Without any nuisance parameters, 1900 simulations are sufficient to only lose 1 per cent of information. We further derive estimators for all quantities needed for forecasting with estimated covariance matrices. Our formalism allows to determine the sweetspot between running sophisticated simulations to reduce the number of nuisance parameters, and running as many fast simulations as possible.

  17. A low-complexity adaptive beamformer for ultrasound imaging using structured covariance matrix.

    Science.gov (United States)

    Asl, Babak Mohammadzadeh; Mahloojifar, Ali

    2012-04-01

    In recent years, adaptive beamforming methods have been successfully applied to medical ultrasound imaging, resulting in simultaneous improvement in imaging resolution and contrast. These improvements have been achieved at the expense of higher computational complexity, with respect to the conventional non-adaptive delay-and-sum (DAS) beamformer, in which computational complexity is proportional to the number of elements, O(M). The computational overhead results from the covariance matrix inversion needed for computation of the adaptive weights, the complexity of which is cubic with the subarray size, O(L(3)). This is a computationally intensive procedure, which makes the implementation of adaptive beamformers less attractive in spite of their advantages. Considering that, in medical ultrasound applications, most of the energy is scattered from angles close to the steering angle, assuming spatial stationarity is a good approximation, allowing us to assume the Toeplitz structure for the estimated covariance matrix. Based on this idea, in this paper, we have applied the Toeplitz structure to the spatially smoothed covariance matrix by averaging the entries along all subdiagonals. Because the inverse of the resulting Toeplitz covariance matrix can be computed in O(L(2)) operations, this technique results in a greatly reduced computational complexity. By using simulated and experimental RF data-point targets as well as cyst phantoms-we show that the proposed low-complexity adaptive beamformer significantly outperforms the DAS and its performance is comparable to that of the minimum variance beamformer, with reduced computational complexity.

  18. A numerical approach to the approximate and the exact minimum rank of a covariance matrix

    NARCIS (Netherlands)

    ten Berge, Jos M.F.; Kiers, Henk A.L.

    A concept of approximate minimum rank for a covariance matrix is defined, which contains the (exact) minimum rank as a special case. A computational procedure to evaluate the approximate minimum rank is offered. The procedure yields those proper communalities for which the unexplained common

  19. STRONGLY CONSISTENT ESTIMATION FOR A MULTIVARIATE LINEAR RELATIONSHIP MODEL WITH ESTIMATED COVARIANCES MATRIX

    Institute of Scientific and Technical Information of China (English)

    Yee LEUNG; WU Kefa; DONG Tianxin

    2001-01-01

    In this paper, a multivariate linear functional relationship model, where the covariance matrix of the observational errors is not restricted, is considered. The parameter estimation of this model is discussed. The estimators are shown to be a strongly consistent estimation under some mild conditions on the incidental parameters.

  20. Simultaneous Multiple Response Regression and Inverse Covariance Matrix Estimation via Penalized Gaussian Maximum Likelihood.

    Science.gov (United States)

    Lee, Wonyul; Liu, Yufeng

    2012-10-01

    Multivariate regression is a common statistical tool for practical problems. Many multivariate regression techniques are designed for univariate response cases. For problems with multiple response variables available, one common approach is to apply the univariate response regression technique separately on each response variable. Although it is simple and popular, the univariate response approach ignores the joint information among response variables. In this paper, we propose three new methods for utilizing joint information among response variables. All methods are in a penalized likelihood framework with weighted L(1) regularization. The proposed methods provide sparse estimators of conditional inverse co-variance matrix of response vector given explanatory variables as well as sparse estimators of regression parameters. Our first approach is to estimate the regression coefficients with plug-in estimated inverse covariance matrices, and our second approach is to estimate the inverse covariance matrix with plug-in estimated regression parameters. Our third approach is to estimate both simultaneously. Asymptotic properties of these methods are explored. Our numerical examples demonstrate that the proposed methods perform competitively in terms of prediction, variable selection, as well as inverse covariance matrix estimation.

  1. Self-Calibration of Radio Astronomical Arrays With Non-Diagonal Noise Covariance Matrix

    CERN Document Server

    Wijnholds, Stefan J

    2010-01-01

    The radio astronomy community is currently building a number of phased array telescopes. The calibration of these telescopes is hampered by the fact that covariances of signals from closely spaced antennas are sensitive to noise coupling and to variations in sky brightness on large spatial scales. These effects are difficult and computationally expensive to model. We propose to model them phenomenologically using a non-diagonal noise covariance matrix. The parameters can be estimated using a weighted alternating least squares (WALS) algorithm iterating between the calibration parameters and the additive nuisance parameters. We demonstrate the effectiveness of our method using data from the low frequency array (LOFAR) prototype station.

  2. Quantifying lost information due to covariance matrix estimation in parameter inference

    CERN Document Server

    Sellentin, Elena

    2016-01-01

    Parameter inference with an estimated covariance matrix systematically loses information due to the remaining uncertainty of the covariance matrix. Here, we quantify this loss of precision and develop a framework to hypothetically restore it, which allows to judge how far away a given analysis is from the ideal case of a known covariance matrix. We point out that it is insufficient to estimate this loss by debiasing a Fisher matrix as previously done, due to a fundamental inequality that describes how biases arise in non-linear functions. We therefore develop direct estimators for parameter credibility contours and the figure of merit. We apply our results to DES Science Verification weak lensing data, detecting a 10% loss of information that increases their credibility contours. No significant loss of information is found for KiDS. For a Euclid-like survey, with about 10 nuisance parameters we find that 2900 simulations are sufficient to limit the systematically lost information to 1%, with an additional unc...

  3. APPLICATION OF RESTART COVARIANCE MATRIX ADAPTATION EVOLUTION STRATEGY (RCMA-ES TO GENERATION EXPANSION PLANNING PROBLEM

    Directory of Open Access Journals (Sweden)

    K. Karthikeyan

    2012-10-01

    Full Text Available This paper describes the application of an evolutionary algorithm, Restart Covariance Matrix Adaptation Evolution Strategy (RCMA-ES to the Generation Expansion Planning (GEP problem. RCMA-ES is a class of continuous Evolutionary Algorithm (EA derived from the concept of self-adaptation in evolution strategies, which adapts the covariance matrix of a multivariate normal search distribution. The original GEP problem is modified by incorporating Virtual Mapping Procedure (VMP. The GEP problem of a synthetic test systems for 6-year, 14-year and 24-year planning horizons having five types of candidate units is considered. Two different constraint-handling methods are incorporated and impact of each method has been compared. In addition, comparison and validation has also made with dynamic programming method.

  4. Synthesis of linear regression coefficients by recovering the within-study covariance matrix from summary statistics.

    Science.gov (United States)

    Yoneoka, Daisuke; Henmi, Masayuki

    2017-06-01

    Recently, the number of regression models has dramatically increased in several academic fields. However, within the context of meta-analysis, synthesis methods for such models have not been developed in a commensurate trend. One of the difficulties hindering the development is the disparity in sets of covariates among literature models. If the sets of covariates differ across models, interpretation of coefficients will differ, thereby making it difficult to synthesize them. Moreover, previous synthesis methods for regression models, such as multivariate meta-analysis, often have problems because covariance matrix of coefficients (i.e. within-study correlations) or individual patient data are not necessarily available. This study, therefore, proposes a brief explanation regarding a method to synthesize linear regression models under different covariate sets by using a generalized least squares method involving bias correction terms. Especially, we also propose an approach to recover (at most) threecorrelations of covariates, which is required for the calculation of the bias term without individual patient data. Copyright © 2016 John Wiley & Sons, Ltd. Copyright © 2016 John Wiley & Sons, Ltd.

  5. Estimation of genetic connectedness diagnostics based on prediction errors without the prediction error variance-covariance matrix.

    Science.gov (United States)

    Holmes, John B; Dodds, Ken G; Lee, Michael A

    2017-03-02

    An important issue in genetic evaluation is the comparability of random effects (breeding values), particularly between pairs of animals in different contemporary groups. This is usually referred to as genetic connectedness. While various measures of connectedness have been proposed in the literature, there is general agreement that the most appropriate measure is some function of the prediction error variance-covariance matrix. However, obtaining the prediction error variance-covariance matrix is computationally demanding for large-scale genetic evaluations. Many alternative statistics have been proposed that avoid the computational cost of obtaining the prediction error variance-covariance matrix, such as counts of genetic links between contemporary groups, gene flow matrices, and functions of the variance-covariance matrix of estimated contemporary group fixed effects. In this paper, we show that a correction to the variance-covariance matrix of estimated contemporary group fixed effects will produce the exact prediction error variance-covariance matrix averaged by contemporary group for univariate models in the presence of single or multiple fixed effects and one random effect. We demonstrate the correction for a series of models and show that approximations to the prediction error matrix based solely on the variance-covariance matrix of estimated contemporary group fixed effects are inappropriate in certain circumstances. Our method allows for the calculation of a connectedness measure based on the prediction error variance-covariance matrix by calculating only the variance-covariance matrix of estimated fixed effects. Since the number of fixed effects in genetic evaluation is usually orders of magnitudes smaller than the number of random effect levels, the computational requirements for our method should be reduced.

  6. A new procedure to built a model covariance matrix: first results

    Science.gov (United States)

    Barzaghi, R.; Marotta, A. M.; Splendore, R.; Borghi, A.

    2012-04-01

    In order to validate the results of geophysical models a common procedure is to compare model predictions with observations by means of statistical tests. A limit of this approach is the lack of a covariance matrix associated to model results, that may frustrate the achievement of a confident statistical significance of the results. Trying to overcome this limit, we have implemented a new procedure to build a model covariance matrix that could allow a more reliable statistical analysis. This procedure has been developed in the frame of the thermo-mechanical model described in Splendore et al. (2010), that predicts the present-day crustal velocity field in the Tyrrhenian due to Africa-Eurasia convergence and to lateral rheological heterogeneities of the lithosphere. Modelled tectonic velocity field has been compared to the available surface velocity field based on GPS observation, determining the best fit model and the degree of fitting, through the use of a χ2 test. Once we have identified the key models parameters and defined their appropriate ranges of variability, we have run 100 different models for 100 sets of randomly values of the parameters extracted within the corresponding interval, obtaining a stack of 100 velocity fields. Then, we calculated variance and empirical covariance for the stack of results, taking into account also cross-correlation, obtaining a positive defined, diagonal matrix that represents the covariance matrix of the model. This empirical approach allows us to define a more robust statistical analysis with respect the classic approach. Reference Splendore, Marotta, Barzaghi, Borghi and Cannizzaro, 2010. Block model versus thermomechanical model: new insights on the present-day regional deformation in the surroundings of the Calabrian Arc. In: Spalla, Marotta and Gosso (Eds) Advances in Interpretation of Geological Processes: Refinement of Multi scale Data and Integration in Numerical Modelling. Geological Society, London, Special

  7. Cloud-Based DDoS HTTP Attack Detection Using Covariance Matrix Approach

    Directory of Open Access Journals (Sweden)

    Abdulaziz Aborujilah

    2017-01-01

    Full Text Available In this era of technology, cloud computing technology has become essential part of the IT services used the daily life. In this regard, website hosting services are gradually moving to the cloud. This adds new valued feature to the cloud-based websites and at the same time introduces new threats for such services. DDoS attack is one such serious threat. Covariance matrix approach is used in this article to detect such attacks. The results were encouraging, according to confusion matrix and ROC descriptors.

  8. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES).

    Science.gov (United States)

    Hansen, Nikolaus; Müller, Sibylle D; Koumoutsakos, Petros

    2003-01-01

    This paper presents a novel evolutionary optimization strategy based on the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). This new approach is intended to reduce the number of generations required for convergence to the optimum. Reducing the number of generations, i.e., the time complexity of the algorithm, is important if a large population size is desired: (1) to reduce the effect of noise; (2) to improve global search properties; and (3) to implement the algorithm on (highly) parallel machines. Our method results in a highly parallel algorithm which scales favorably with large numbers of processors. This is accomplished by efficiently incorporating the available information from a large population, thus significantly reducing the number of generations needed to adapt the covariance matrix. The original version of the CMA-ES was designed to reliably adapt the covariance matrix in small populations but it cannot exploit large populations efficiently. Our modifications scale up the efficiency to population sizes of up to 10n, where n is the problem dimension. This method has been applied to a large number of test problems, demonstrating that in many cases the CMA-ES can be advanced from quadratic to linear time complexity.

  9. A comparison of likelihood ratio tests and Rao's score test for three separable covariance matrix structures.

    Science.gov (United States)

    Filipiak, Katarzyna; Klein, Daniel; Roy, Anuradha

    2017-01-01

    The problem of testing the separability of a covariance matrix against an unstructured variance-covariance matrix is studied in the context of multivariate repeated measures data using Rao's score test (RST). The RST statistic is developed with the first component of the separable structure as a first-order autoregressive (AR(1)) correlation matrix or an unstructured (UN) covariance matrix under the assumption of multivariate normality. It is shown that the distribution of the RST statistic under the null hypothesis of any separability does not depend on the true values of the mean or the unstructured components of the separable structure. A significant advantage of the RST is that it can be performed for small samples, even smaller than the dimension of the data, where the likelihood ratio test (LRT) cannot be used, and it outperforms the standard LRT in a number of contexts. Monte Carlo simulations are then used to study the comparative behavior of the null distribution of the RST statistic, as well as that of the LRT statistic, in terms of sample size considerations, and for the estimation of the empirical percentiles. Our findings are compared with existing results where the first component of the separable structure is a compound symmetry (CS) correlation matrix. It is also shown by simulations that the empirical null distribution of the RST statistic converges faster than the empirical null distribution of the LRT statistic to the limiting χ(2) distribution. The tests are implemented on a real dataset from medical studies. © 2016 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

  10. Noncommutative Field Theory on Yang's Space-Time Algebra, Covariant Moyal Star Product and Matrix Model

    CERN Document Server

    Tanaka, S

    2004-01-01

    Noncommutative field theory on Yang's quantized space-time algebra (YSTA) is studied. It gives a theoretical framework to reformulate the matrix model as quantum mechanics of $D_0$ branes in a Lorentz-covariant form. The so-called kinetic term ($\\sim {\\hat{P_i}}^2)$ and potential term ($\\sim {[\\hat{X_i},\\hat{X_j}]}^2)$ of $D_0$ branes in the matrix model are described now in terms of Casimir operator of $SO(D,1)$, a subalgebra of the primary algebra $SO(D+1,1)$ which underlies YSTA with two contraction- parameters, $\\lambda$ and $R$. $D$-dimensional noncommutative space-time and momentum operators $\\hat{X_\\mu}$ and $\\hat{P_\\mu}$ in YSTA show a distinctive spectral structure, that is, space-components $\\hat{X_i}$ and $\\hat{P_i}$ have discrete eigenvalues, and time-components $\\hat{X_0}$ and $\\hat{P_0}$ continuous eigenvalues, consistently with Lorentz-covariance. According to the method of Lorentz-covariant Moyal star product proper to YSTA, the field equation of $D_0$ brane on YSTA is derived in a nontrivial ...

  11. A Concise Method for Storing and Communicating the Data Covariance Matrix

    Energy Technology Data Exchange (ETDEWEB)

    Larson, Nancy M [ORNL

    2008-10-01

    The covariance matrix associated with experimental cross section or transmission data consists of several components. Statistical uncertainties on the measured quantity (counts) provide a diagonal contribution. Off-diagonal components arise from uncertainties on the parameters (such as normalization or background) that figure into the data reduction process; these are denoted systematic or common uncertainties, since they affect all data points. The full off-diagonal data covariance matrix (DCM) can be extremely large, since the size is the square of the number of data points. Fortunately, it is not necessary to explicitly calculate, store, or invert the DCM. Likewise, it is not necessary to explicitly calculate, store, or use the inverse of the DCM. Instead, it is more efficient to accomplish the same results using only the various component matrices that appear in the definition of the DCM. Those component matrices are either diagonal or small (the number of data points times the number of data-reduction parameters); hence, this implicit data covariance method requires far less array storage and far fewer computations while producing more accurate results.

  12. Exact distribution of MLE of covariance matrix in a GMANOVA-MANOVA model

    Institute of Scientific and Technical Information of China (English)

    BAI; Peng

    2005-01-01

    For a GMANOVA-MANOVA model with normal error: Y = XB1Z1T + B2Z2T +E, E ~ Nq×n(0, In(×) ∑), the present paper is devoted to the study of distribution of MLE,Σ, of covariance matrix ∑. The main results obtained are stated as follows: (1) When rk(Z) -rk(Z2) ≥ q-rk(X), the exact distribution of (Σ) is derived, where Z = (Z1, Z2), rk(A)denotes the rank of matrix A. (2) The exact distribution of |(Σ)| is gained. (3) It is proved that ntr{[∑-1 - ∑-1XM(MTXT∑-1XM)-1MTXT∑-1]Σ} has x2(q-rk(X))(n-rk(Z2)) distribution, where M is the matrix whose columns are the standardized orthogonal eigenvectors corresponding to the nonzero eigenvalues of XT∑-1X.

  13. Obtaining the Hessian from the force covariance matrix: Application to crystalline explosives PETN and RDX

    Science.gov (United States)

    Pereverzev, Andrey; Sewell, Thomas D.

    2015-04-01

    We show that for solids the effective Hessian matrix, averaged over the canonical ensemble, can be calculated from the force covariance matrix. This effective Hessian reduces to the standard Hessian as the temperature approaches zero, while at finite temperatures it implicitly includes anharmonic corrections. As a case study, we calculate the effective Hessians and the corresponding normal mode eigenvectors and frequencies for the crystalline organic explosives pentaerythritol tetranitrate and α-1,3,5-trinitro-1,3,5-triazacyclohexane. The resulting normal mode frequencies are compared to those obtained by diagonalizing the standard Hessian matrix of second derivatives in Cartesian displacements about the potential energy minimum. Effects of temperature and statistical noise on the effective Hessians and normal mode frequencies are discussed.

  14. Estimation of genetic connectedness diagnostics based on prediction errors without the prediction error variance-covariance matrix

    National Research Council Canada - National Science Library

    John B Holmes; Ken G Dodds; Michael A Lee

    2017-01-01

    .... While various measures of connectedness have been proposed in the literature, there is general agreement that the most appropriate measure is some function of the prediction error variance-covariance matrix...

  15. Variational Assimilation for Xenon Dynamical Forecasts in Neutronic using Advanced Background Error Covariance Matrix

    CERN Document Server

    Ponçot, Angélique; Bouriquet, Bertrand; Erhard, Patrick; Gratton, Serge; Thual, Olivier

    2013-01-01

    Data assimilation method consists in combining all available pieces of information about a system to obtain optimal estimates of initial states. The different sources of information are weighted according to their accuracy by the means of error covariance matrices. Our purpose here is to evaluate the efficiency of variational data assimilation for the xenon induced oscillations forecasts in nuclear cores. In this paper we focus on the comparison between 3DVAR schemes with optimised background error covariance matrix B and a 4DVAR scheme. Tests were made in twin experiments using a simulation code which implements a mono-dimensional coupled model of xenon dynamics, thermal, and thermal-hydraulic processes. We enlighten the very good efficiency of the 4DVAR scheme as well as good results with the 3DVAR one using a careful multivariate modelling of B.

  16. Employment of the covariance matrix in parameter estimation for stochastic processes in cell biology

    Science.gov (United States)

    Preuss, R.; Dieterich, P.

    2013-08-01

    The dynamics of movements of biological cells can be described with models from correlated stochastic processes. In order to overcome problems from correlated and insufficient data in the determination of the model parameters of such processes we employ the covariance matrix of the data. Since the covariance suffers itself from statistical uncertainty it is corrected by a renormalization treatment [1]. For the example of normal and fractional Brownian motion, which allows both to access all quantities on full theoretical grounds and to generate data similar to experiment, we discuss our results and those of previous works by Gregory [2] and Sivia [3]. The presented approach has the potential to estimate the aging correlation function of observed cell paths and can be applied to more complicated models.

  17. Numerical Reconstruction of the Covariance Matrix of a Spherically Truncated Multinormal Distribution

    Directory of Open Access Journals (Sweden)

    Filippo Palombi

    2017-01-01

    Full Text Available We relate the matrix SB of the second moments of a spherically truncated normal multivariate to its full covariance matrix Σ and present an algorithm to invert the relation and reconstruct Σ from SB. While the eigenvectors of Σ are left invariant by the truncation, its eigenvalues are nonuniformly damped. We show that the eigenvalues of Σ can be reconstructed from their truncated counterparts via a fixed point iteration, whose convergence we prove analytically. The procedure requires the computation of multidimensional Gaussian integrals over an Euclidean ball, for which we extend a numerical technique, originally proposed by Ruben in 1962, based on a series expansion in chi-square distributions. In order to study the feasibility of our approach, we examine the convergence rate of some iterative schemes on suitably chosen ensembles of Wishart matrices. We finally discuss the practical difficulties arising in sample space and outline a regularization of the problem based on perturbation theory.

  18. Accuracy of whole-genome prediction using a genetic architecture-enhanced variance-covariance matrix.

    Science.gov (United States)

    Zhang, Zhe; Erbe, Malena; He, Jinlong; Ober, Ulrike; Gao, Ning; Zhang, Hao; Simianer, Henner; Li, Jiaqi

    2015-02-09

    Obtaining accurate predictions of unobserved genetic or phenotypic values for complex traits in animal, plant, and human populations is possible through whole-genome prediction (WGP), a combined analysis of genotypic and phenotypic data. Because the underlying genetic architecture of the trait of interest is an important factor affecting model selection, we propose a new strategy, termed BLUP|GA (BLUP-given genetic architecture), which can use genetic architecture information within the dataset at hand rather than from public sources. This is achieved by using a trait-specific covariance matrix ( T: ), which is a weighted sum of a genetic architecture part ( S: matrix) and the realized relationship matrix ( G: ). The algorithm of BLUP|GA (BLUP-given genetic architecture) is provided and illustrated with real and simulated datasets. Predictive ability of BLUP|GA was validated with three model traits in a dairy cattle dataset and 11 traits in three public datasets with a variety of genetic architectures and compared with GBLUP and other approaches. Results show that BLUP|GA outperformed GBLUP in 20 of 21 scenarios in the dairy cattle dataset and outperformed GBLUP, BayesA, and BayesB in 12 of 13 traits in the analyzed public datasets. Further analyses showed that the difference of accuracies for BLUP|GA and GBLUP significantly correlate with the distance between the T: and G: matrices. The new strategy applied in BLUP|GA is a favorable and flexible alternative to the standard GBLUP model, allowing to account for the genetic architecture of the quantitative trait under consideration when necessary. This feature is mainly due to the increased similarity between the trait-specific relationship matrix ( T: matrix) and the genetic relationship matrix at unobserved causal loci. Applying BLUP|GA in WGP would ease the burden of model selection. Copyright © 2015 Zhang et al.

  19. Design Of Multivariable Fractional Order Pid Controller Using Covariance Matrix Adaptation Evolution Strategy

    Directory of Open Access Journals (Sweden)

    Sivananaithaperumal Sudalaiandi

    2014-06-01

    Full Text Available This paper presents an automatic tuning of multivariable Fractional-Order Proportional, Integral and Derivative controller (FO-PID parameters using Covariance Matrix Adaptation Evolution Strategy (CMAES algorithm. Decoupled multivariable FO-PI and FO-PID controller structures are considered. Oustaloup integer order approximation is used for the fractional integrals and derivatives. For validation, two Multi-Input Multi- Output (MIMO distillation columns described byWood and Berry and Ogunnaike and Ray are considered for the design of multivariable FO-PID controller. Optimal FO-PID controller is designed by minimizing Integral Absolute Error (IAE as objective function. The results of previously reported PI/PID controller are considered for comparison purposes. Simulation results reveal that the performance of FOPI and FO-PID controller is better than integer order PI/PID controller in terms of IAE. Also, CMAES algorithm is suitable for the design of FO-PI / FO-PID controller.

  20. Pre-processing ambient noise cross-correlations with equalizing the covariance matrix eigenspectrum

    Science.gov (United States)

    Seydoux, Léonard; de Rosny, Julien; Shapiro, Nikolai M.

    2017-09-01

    Passive imaging techniques from ambient seismic noise requires a nearly isotropic distribution of the noise sources in order to ensure reliable traveltime measurements between seismic stations. However, real ambient seismic noise often partially fulfils this condition. It is generated in preferential areas (in deep ocean or near continental shores), and some highly coherent pulse-like signals may be present in the data such as those generated by earthquakes. Several pre-processing techniques have been developed in order to attenuate the directional and deterministic behaviour of this real ambient noise. Most of them are applied to individual seismograms before cross-correlation computation. The most widely used techniques are the spectral whitening and temporal smoothing of the individual seismic traces. We here propose an additional pre-processing to be used together with the classical ones, which is based on the spatial analysis of the seismic wavefield. We compute the cross-spectra between all available stations pairs in spectral domain, leading to the data covariance matrix. We apply a one-bit normalization to the covariance matrix eigenspectrum before extracting the cross-correlations in the time domain. The efficiency of the method is shown with several numerical tests. We apply the method to the data collected by the USArray, when the M8.8 Maule earthquake occurred on 2010 February 27. The method shows a clear improvement compared with the classical equalization to attenuate the highly energetic and coherent waves incoming from the earthquake, and allows to perform reliable traveltime measurement even in the presence of the earthquake.

  1. Multimodal Optimization by Covariance Matrix Self-Adaptation Evolution Strategy with Repelling Subpopulations.

    Science.gov (United States)

    Ahrari, Ali; Deb, Kalyanmoy; Preuss, Mike

    2017-01-01

    During the recent decades, many niching methods have been proposed and empirically verified on some available test problems. They often rely on some particular assumptions associated with the distribution, shape, and size of the basins, which can seldom be made in practical optimization problems. This study utilizes several existing concepts and techniques, such as taboo points, normalized Mahalanobis distance, and the Ursem's hill-valley function in order to develop a new tool for multimodal optimization, which does not make any of these assumptions. In the proposed method, several subpopulations explore the search space in parallel. Offspring of a subpopulation are forced to maintain a sufficient distance to the center of fitter subpopulations and the previously identified basins, which are marked as taboo points. The taboo points repel the subpopulation to prevent convergence to the same basin. A strategy to update the repelling power of the taboo points is proposed to address the challenge of basins of dissimilar size. The local shape of a basin is also approximated by the distribution of the subpopulation members converging to that basin. The proposed niching strategy is incorporated into the covariance matrix self-adaptation evolution strategy (CMSA-ES), a potent global optimization method. The resultant method, called the covariance matrix self-adaptation with repelling subpopulations (RS-CMSA), is assessed and compared to several state-of-the-art niching methods on a standard test suite for multimodal optimization. An organized procedure for parameter setting is followed which assumes a rough estimation of the desired/expected number of minima available. Performance sensitivity to the accuracy of this estimation is also studied by introducing the concept of robust mean peak ratio. Based on the numerical results using the available and the introduced performance measures, RS-CMSA emerges as the most successful method when robustness and efficiency are

  2. Finding Imaging Patterns of Structural Covariance via Non-Negative Matrix Factorization

    Science.gov (United States)

    Sotiras, Aristeidis; Resnick, Susan M.; Davatzikos, Christos

    2015-01-01

    In this paper, we investigate the use of Non-Negative Matrix Factorization (NNMF) for the analysis of structural neuroimaging data. The goal is to identify the brain regions that co-vary across individuals in a consistent way, hence potentially being part of underlying brain networks or otherwise influenced by underlying common mechanisms such as genetics and pathologies. NNMF offers a directly data-driven way of extracting relatively localized co-varying structural regions, thereby transcending limitations of Principal Component Analysis (PCA), Independent Component Analysis (ICA) and other related methods that tend to produce dispersed components of positive and negative loadings. In particular, leveraging upon the well known ability of NNMF to produce parts-based representations of image data, we derive decompositions that partition the brain into regions that vary in consistent ways across individuals. Importantly, these decompositions achieve dimensionality reduction via highly interpretable ways and generalize well to new data as shown via split-sample experiments. We empirically validate NNMF in two data sets: i) a Diffusion Tensor (DT) mouse brain development study, and ii) a structural Magnetic Resonance (sMR) study of human brain aging. We demonstrate the ability of NNMF to produce sparse parts-based representations of the data at various resolutions. These representations seem to follow what we know about the underlying functional organization of the brain and also capture some pathological processes. Moreover, we show that these low dimensional representations favorably compare to descriptions obtained with more commonly used matrix factorization methods like PCA and ICA. PMID:25497684

  3. Pre-Processing Noise Cross-Correlations with Equalizing the Network Covariance Matrix Eigen-Spectrum

    Science.gov (United States)

    Seydoux, L.; de Rosny, J.; Shapiro, N.

    2016-12-01

    Theoretically, the extraction of Green functions from noise cross-correlation requires the ambient seismic wavefield to be generated by uncorrelated sources evenly distributed in the medium. Yet, this condition is often not verified. Strong events such as earthquakes often produce highly coherent transient signals. Also, the microseismic noise is generated at specific places on the Earth's surface with source regions often very localized in space. Different localized and persistent seismic sources may contaminate the cross-correlations of continuous records resulting in spurious arrivals or asymmetry and, finally, in biased travel-time measurements. Pre-processing techniques therefore must be applied to the seismic data in order to reduce the effect of noise anisotropy and the influence of strong localized events. Here we describe a pre-processing approach that uses the covariance matrix computed from signals recorded by a network of seismographs. We extend the widely used time and spectral equalization pre-processing to the equalization of the covariance matrix spectrum (i.e., its ordered eigenvalues). This approach can be considered as a spatial equalization. This method allows us to correct for the wavefield anisotropy in two ways: (1) the influence of strong directive sources is substantially attenuated, and (2) the weakly excited modes are reinforced, allowing to partially recover the conditions required for the Green's function retrieval. We also present an eigenvector-based spatial filter used to distinguish between surface and body waves. This last filter is used together with the equalization of the eigenvalue spectrum. We simulate two-dimensional wavefield in a heterogeneous medium with strongly dominating source. We show that our method greatly improves the travel-time measurements obtained from the inter-station cross-correlation functions. Also, we apply the developed method to the USArray data and pre-process the continuous records strongly influenced

  4. The Covariance matrix Analysis Of Log Data%测井资料的协方差阵分析

    Institute of Scientific and Technical Information of China (English)

    秦海菲; 陈超慧

    2012-01-01

    协方差阵是多元分析中一个重要的组成部分。协方差分析是方差分析和回归分析结合而产生的一种数据分析方法。本文利用协方差阵对测并资料中油层、水层、干层的数据进行分析,通过准备数据,建立油层、水层、干层的矩阵,分析矩阵并建立相应的协方差阵,计算协方差阵和聚类中心,比较每一个层与聚类中心的距离,判断出所要判断的层是油,水,干层中的哪一层,再与实际层对比,得出符合率。通过证明,说明了用协方差阵对测井数据进行分析是可行的,也是可信的。%Covariance matrix is a essential part of multivariate analysis .Covariance matrix analysis is a method of data analysis that mixed the Variance analysis and Regression analysis.This paper using Covariance matrix analysis Logging data. According to data prepare, build the matrix of layer in oil, water, dry. Analysis matrix and build Covariance matrix, computer Covariance matrix, build the center of cluster. Compare the distance of the layer to the center of cluster, discriminanted the layer is that is oil,water or dry layer. Compare the discriminant layer and actual layer,obtain the coincidence rate.

  5. Adaptation of the optimal fingerprint method for climate change detection using a well-conditioned covariance matrix estimate

    Energy Technology Data Exchange (ETDEWEB)

    Ribes, Aurelien; Planton, Serge [CNRM-GAME, Meteo France-CNRS, Toulouse (France); Azais, Jean-Marc [Universite de Toulouse, UPS, IMT, LSP, Toulouse (France)

    2009-10-15

    The ''optimal fingerprint'' method, usually used for detection and attribution studies, requires to know, or, in practice, to estimate the covariance matrix of the internal climate variability. In this work, a new adaptation of the ''optimal fingerprints'' method is presented. The main goal is to allow the use of a covariance matrix estimate based on an observation dataset in which the number of years used for covariance estimation is close to the number of observed time series. Our adaptation is based on the use of a regularized estimate of the covariance matrix, that is well-conditioned, and asymptotically more precise, in the sense of the mean square error. This method is shown to be more powerful than the basic ''guess pattern fingerprint'', and than the classical use of a pseudo-inverted truncation of the empirical covariance matrix. The construction of the detection test is achieved by using a bootstrap technique particularly well-suited to estimate the internal climate variability in real world observations. In order to validate the efficiency of the detection algorithm with climate data, the methodology presented here is first applied with pseudo-observations derived from transient regional climate change scenarios covering the 1960-2099 period. It is then used to perform a formal detection study of climate change over France, analyzing homogenized observed temperature series from 1900 to 2006. In this case, the estimation of the covariance matrix is only based on a part of the observation dataset. This new approach allows the confirmation and extension of previous results regarding the detection of an anthropogenic climate change signal over the country. (orig.)

  6. Direct closed-form covariance matrix and finite alphabet constant-envelope waveforms for planar array beampatterns

    KAUST Repository

    Ahmed, Sajid

    2016-11-24

    Various examples of methods and systems are provided for direct closed-form finite alphabet constant-envelope waveforms for planar array beampatterns. In one example, a method includes defining a waveform covariance matrix based at least in part upon a two-dimensional fast Fourier transform (2D-FFT) analysis of a frequency domain matrix Hf associated with a planar array of antennas. Symbols can be encoded based upon the waveform covariance matrix and the encoded symbols can be transmitted via the planar array of antennas. In another embodiment, a system comprises an N x M planar array of antennas and transmission circuitry configured to transmit symbols via a two-dimensional waveform beampattern defined based at least in part upon a 2D-FFT analysis of a frequency domain matrix Hf associated with the planar array of antennas.

  7. Covariance and crossover matrix guided differential evolution for global numerical optimization.

    Science.gov (United States)

    Li, YongLi; Feng, JinFu; Hu, JunHua

    2016-01-01

    Differential evolution (DE) is an efficient and robust evolutionary algorithm and has wide application in various science and engineering fields. DE is sensitive to the selection of mutation and crossover strategies and their associated control parameters. However, the structure and implementation of DEs are becoming more complex because of the diverse mutation and crossover strategies that use distinct parameter settings during the different stages of the evolution. A novel strategy is used in this study to improve the crossover and mutation operations. The crossover matrix, instead of a crossover operator and its control parameter CR, is proposed to implement the function of the crossover operation. Meanwhile, Gaussian distribution centers the best individuals found in each generation based on the proposed covariance matrix, which is generated between the best individual and several better individuals. Improved mutation operator based on the crossover matrix is randomly selected to generate the trial population. This operator is used to generate high-quality solutions to improve the capability of exploitation and enhance the preference of exploration. In addition, the memory population is randomly chosen from previous generation and used to control the search direction in the novel mutation strategy. Accordingly, the diversity of the population is improved. Thus, CCDE, which is a novel efficient and simple DE variant, is presented in this paper. CCDE has been tested on 30 benchmarks and 5 real-world optimization problems from the IEEE Congress on Evolutionary Computation (CEC) 2014 and CEC 2011, respectively. Experimental and statistical results demonstrate the effectiveness of CCDE for global numerical and engineering optimization. CCDE can solve the test benchmark functions and engineering problems more successfully than the other DE variants and algorithms from CEC 2014.

  8. DFT-Based Closed-form Covariance Matrix and Direct Waveforms Design for MIMO Radar to Achieve Desired Beampatterns

    KAUST Repository

    Bouchoucha, Taha

    2017-01-23

    In multiple-input multiple-out (MIMO) radar, for desired transmit beampatterns, appropriate correlated waveforms are designed. To design such waveforms, conventional MIMO radar methods use two steps. In the first step, the waveforms covariance matrix, R, is synthesized to achieve the desired beampattern. While in the second step, to realize the synthesized covariance matrix, actual waveforms are designed. Most of the existing methods use iterative algorithms to solve these constrained optimization problems. The computational complexity of these algorithms is very high, which makes them difficult to use in practice. In this paper, to achieve the desired beampattern, a low complexity discrete-Fourier-transform based closed-form covariance matrix design technique is introduced for a MIMO radar. The designed covariance matrix is then exploited to derive a novel closed-form algorithm to directly design the finite-alphabet constant-envelope waveforms for the desired beampattern. The proposed technique can be used to design waveforms for large antenna array to change the beampattern in real time. It is also shown that the number of transmitted symbols from each antenna depends on the beampattern and is less than the total number of transmit antenna elements.

  9. The potential of more accurate InSAR covariance matrix estimation for land cover mapping

    Science.gov (United States)

    Jiang, Mi; Yong, Bin; Tian, Xin; Malhotra, Rakesh; Hu, Rui; Li, Zhiwei; Yu, Zhongbo; Zhang, Xinxin

    2017-04-01

    Synthetic aperture radar (SAR) and Interferometric SAR (InSAR) provide both structural and electromagnetic information for the ground surface and therefore have been widely used for land cover classification. However, relatively few studies have developed analyses that investigate SAR datasets over richly textured areas where heterogeneous land covers exist and intermingle over short distances. One of main difficulties is that the shapes of the structures in a SAR image cannot be represented in detail as mixed pixels are likely to occur when conventional InSAR parameter estimation methods are used. To solve this problem and further extend previous research into remote monitoring of urban environments, we address the use of accurate InSAR covariance matrix estimation to improve the accuracy of land cover mapping. The standard and updated methods were tested using the HH-polarization TerraSAR-X dataset and compared with each other using the random forest classifier. A detailed accuracy assessment complied for six types of surfaces shows that the updated method outperforms the standard approach by around 9%, with an overall accuracy of 82.46% over areas with rich texture in Zhuhai, China. This paper demonstrates that the accuracy of land cover mapping can benefit from the 3 enhancement of the quality of the observations in addition to classifiers selection and multi-source data ingratiation reported in previous studies.

  10. Analytic model for the matter power spectrum, its covariance matrix, and baryonic effects

    CERN Document Server

    Mohammed, Irshad

    2014-01-01

    We develop a model for the matter power spectrum as the sum of quasi-linear Zeldovich approximation and even powers of $k$, i.e., $A_0 - A_2k^2 + A_4k^4 - ...$, compensated at low $k$. The model can predict the true power spectrum to a few percent accuracy up to $k \\sim 0.7\\ h \\rm{Mpc}^{-1}$, over a wide range of redshifts and models, including massive neutrino models. We write a simple form of the covariance matrix as a sum of Gaussian part and $A_0$ variance and we find that it reproduces well the simulations. We investigate the super-sample variance effect and show it induces a relation between the Zeldovich term and $A_0$ that differs from the amplitude change, allowing it to be modeled as an additional parameter that can be determined from the data. The $A_n$ coefficients contain information about cosmology, in particular the amplitude of fluctuations $\\sigma_8$. We explore their information content, showing that $A_0$ contains the bulk of amplitude information, scaling as $\\sigma_8^{3.9}$, which allows ...

  11. Four-Component Scattering Power Decomposition Algorithm with Rotation of Covariance Matrix Using ALOS-PALSAR Polarimetric Data

    Directory of Open Access Journals (Sweden)

    Yasuhiro Nakamura

    2012-07-01

    Full Text Available The present study introduces the four-component scattering power decomposition (4-CSPD algorithm with rotation of covariance matrix, and presents an experimental proof of the equivalence between the 4-CSPD algorithms based on rotation of covariance matrix and coherency matrix. From a theoretical point of view, the 4-CSPD algorithms with rotation of the two matrices are identical. Although it seems obvious, no experimental evidence has yet been presented. In this paper, using polarimetric synthetic aperture radar (POLSAR data acquired by Phased Array L-band SAR (PALSAR on board of Advanced Land Observing Satellite (ALOS, an experimental proof is presented to show that both algorithms indeed produce identical results.

  12. A heteroskedastic error covariance matrix estimator using a first-order conditional autoregressive Markov simulation for deriving asympotical efficient estimates from ecological sampled Anopheles arabiensis aquatic habitat covariates

    Directory of Open Access Journals (Sweden)

    Githure John I

    2009-09-01

    Full Text Available Abstract Background Autoregressive regression coefficients for Anopheles arabiensis aquatic habitat models are usually assessed using global error techniques and are reported as error covariance matrices. A global statistic, however, will summarize error estimates from multiple habitat locations. This makes it difficult to identify where there are clusters of An. arabiensis aquatic habitats of acceptable prediction. It is therefore useful to conduct some form of spatial error analysis to detect clusters of An. arabiensis aquatic habitats based on uncertainty residuals from individual sampled habitats. In this research, a method of error estimation for spatial simulation models was demonstrated using autocorrelation indices and eigenfunction spatial filters to distinguish among the effects of parameter uncertainty on a stochastic simulation of ecological sampled Anopheles aquatic habitat covariates. A test for diagnostic checking error residuals in an An. arabiensis aquatic habitat model may enable intervention efforts targeting productive habitats clusters, based on larval/pupal productivity, by using the asymptotic distribution of parameter estimates from a residual autocovariance matrix. The models considered in this research extends a normal regression analysis previously considered in the literature. Methods Field and remote-sampled data were collected during July 2006 to December 2007 in Karima rice-village complex in Mwea, Kenya. SAS 9.1.4® was used to explore univariate statistics, correlations, distributions, and to generate global autocorrelation statistics from the ecological sampled datasets. A local autocorrelation index was also generated using spatial covariance parameters (i.e., Moran's Indices in a SAS/GIS® database. The Moran's statistic was decomposed into orthogonal and uncorrelated synthetic map pattern components using a Poisson model with a gamma-distributed mean (i.e. negative binomial regression. The eigenfunction

  13. Estimating Cosmological Parameter Covariance

    CERN Document Server

    Taylor, Andy

    2014-01-01

    We investigate the bias and error in estimates of the cosmological parameter covariance matrix, due to sampling or modelling the data covariance matrix, for likelihood width and peak scatter estimators. We show that these estimators do not coincide unless the data covariance is exactly known. For sampled data covariances, with Gaussian distributed data and parameters, the parameter covariance matrix estimated from the width of the likelihood has a Wishart distribution, from which we derive the mean and covariance. This mean is biased and we propose an unbiased estimator of the parameter covariance matrix. Comparing our analytic results to a numerical Wishart sampler of the data covariance matrix we find excellent agreement. An accurate ansatz for the mean parameter covariance for the peak scatter estimator is found, and we fit its covariance to our numerical analysis. The mean is again biased and we propose an unbiased estimator for the peak parameter covariance. For sampled data covariances the width estimat...

  14. Estimation of Circular Error Probability of Strapped Down Inertial Navigation System by Propagation of Error Covariance Matrix

    Directory of Open Access Journals (Sweden)

    S. Vathsal

    1994-01-01

    Full Text Available This paper provides an error model of the strapped down inertial navigation system in the state space format. A method to estimate the circular error probability is presented using time propagation of error covariance matrix. Numerical results have been obtained for a typical flight trajectory. Sensitivity studies have also been conducted for variation of sensor noise covariances and initial state uncertainty. This methodology seems to work in all the practical cases considered so far. Software has been tested for both the local vertical frame and the inertial frame. The covariance propagation technique provides accurate estimation of dispersions of position at impact. This in turn enables to estimate the circular error probability (CEP very accurately.

  15. Threat Object Detection using Covariance Matrix Modeling in X-ray Images

    Energy Technology Data Exchange (ETDEWEB)

    Jeon, Byoun Gil; Kim, Jong Yul; Moon, Myung Kook [KAERI, Daejeon (Korea, Republic of)

    2016-05-15

    The X-ray imaging system for the aviation security is one of the applications. In airports, all passengers and properties should be inspected and accepted by security machines before boarding on aircrafts to avoid all treat factors. That treat factors might be directly connected on terrorist threats awfully hazardous to not only passengers but also people in highly populated area such as major cities or buildings. Because the performance of the system is increasing along with the growth of IT technology, information that has various type and good quality can be provided for security check. However, human factors are mainly affected on the inspections. It means that human inspectors should be proficient corresponding to the growth of technology for efficient and effective inspection but there is clear limit of proficiency. Human being is not a computer. Because of the limitation, the aviation security techniques have the tendencies to provide not only numerous and nice information but also effective assistance for security inspectors. Many image processing applications already have been developed to provide efficient assistance for the security systems. Naturally, the security check procedure should not be altered by automatic software because it's not guaranteed that the automatic system will never make any mistake. This paper addressed an application of threat object detection using the covariance matrix modeling. The algorithm is implemented in MATLAB environment and evaluated the performance by comparing with other detection algorithms. Considering the shape of an object on an image is changed by the attitude of that to the imaging machine, the implemented detector has the robustness for rotation and scale of an object.

  16. A special form of SPD covariance matrix for interpretation and visualization of data manipulated with Riemannian geometry

    Science.gov (United States)

    Congedo, Marco; Barachant, Alexandre

    2015-01-01

    Currently the Riemannian geometry of symmetric positive definite (SPD) matrices is gaining momentum as a powerful tool in a wide range of engineering applications such as image, radar and biomedical data signal processing. If the data is not natively represented in the form of SPD matrices, typically we may summarize them in such form by estimating covariance matrices of the data. However once we manipulate such covariance matrices on the Riemannian manifold we lose the representation in the original data space. For instance, we can evaluate the geometric mean of a set of covariance matrices, but not the geometric mean of the data generating the covariance matrices, the space of interest in which the geometric mean can be interpreted. As a consequence, Riemannian information geometry is often perceived by non-experts as a "black-box" tool and this perception prevents a wider adoption in the scientific community. Hereby we show that we can overcome this limitation by constructing a special form of SPD matrix embedding both the covariance structure of the data and the data itself. Incidentally, whenever the original data can be represented in the form of a generic data matrix (not even square), this special SPD matrix enables an exhaustive and unique description of the data up to second-order statistics. This is achieved embedding the covariance structure of both the rows and columns of the data matrix, allowing naturally a wide range of possible applications and bringing us over and above just an interpretability issue. We demonstrate the method by manipulating satellite images (pansharpening) and event-related potentials (ERPs) of an electroencephalography brain-computer interface (BCI) study. The first example illustrates the effect of moving along geodesics in the original data space and the second provides a novel estimation of ERP average (geometric mean), showing that, in contrast to the usual arithmetic mean, this estimation is robust to outliers. In

  17. Information content of weak lensing bispectrum: including the non-Gaussian error covariance matrix

    CERN Document Server

    Kayo, Issha; Jain, Bhuvnesh

    2013-01-01

    We address a long-standing problem, how can we extract information in the non-Gaussian regime of weak lensing surveys, by accurate modeling of all relevant covariances between the power spectra and bispectra. We use 1000 ray-tracing simulation realizations for a Lambda-CDM model and an analytical halo model. We develop a formalism to describe the covariance matrices of power spectra and bispectra of all triangle configurations, which extend to 6-point correlation functions. We include a new contribution arising from coupling of the lensing Fourier modes with large-scale mass fluctuations on scales comparable with the survey region via halo bias theory, which we call the halo sample variance (HSV) effect. We show that the model predictions are in excellent agreement with the simulation results for the power spectrum and bispectrum covariances. The HSV effect gives a dominant contribution to the covariances at multipoles l > 10^3, which arise from massive halos with masses of about 10^14 solar masses and at rel...

  18. Few Group Collapsing of Covariance Matrix Data Based on a Conservation Principle

    Energy Technology Data Exchange (ETDEWEB)

    H. Hiruta; G. Palmiotti; M. Salvatores; R. Arcilla, Jr.; R. D. McKnight; G. Aliberti; P. Oblozinsky; W. S. Yang

    2008-12-01

    A new algorithm for a rigorous collapsing of covariance data is proposed, derived, implemented, and tested. The method is based on a conservation principle that allows the uncertainty calculated in a fine group energy structure for a specific integral parameter, using as weights the associated sensitivity coefficients, to be preserved at a broad energy group structure.

  19. Detecting land cover change by evaluating the internal covariance matrix of the extended Kalman filter

    CSIR Research Space (South Africa)

    Salmon, BP

    2012-07-01

    Full Text Available In this paper, the internal operations of an Extended Kalman Filter is investigated to see if any useful information can be derived to detect land cover change in a MODIS time series. The Extended Kalman Filter expands its internal covariance if a...

  20. Epistasis and the temporal change in the additive variance-covariance matrix induced by drift.

    Science.gov (United States)

    López-Fanjul, Carlos; Fernández, Almudena; Toro, Miguel A

    2004-08-01

    The effect of population bottlenecks on the components of the genetic covariance generated by two neutral independent epistatic loci has been studied theoretically (additive, covA; dominance, covD; additive-by-additive, covAA; additive-by-dominance, covAD; and dominance-by-dominance, covDD). The additive-by-additive model and a more general model covering all possible types of marginal gene action at the single-locus level (additive/dominance epistatic model) were considered. The covariance components in an infinitely large panmictic population (ancestral components) were compared with their expected values at equilibrium over replicates randomly derived from the base population, after t consecutive bottlenecks of equal size N (derived components). Formulae were obtained in terms of the allele frequencies and effects at each locus, the corresponding epistatic effects and the inbreeding coefficient Ft. These expressions show that the contribution of nonadditive loci to the derived additive covariance (covAt) does not linearly decrease with inbreeding, as in the pure additive case, and may initially increase or even change sign in specific situations. Numerical examples were also analyzed, restricted for simplicity to the case of all covariance components being positive. For additive-by-additive epistasis, the condition covAt > covA only holds for high frequencies of the allele decreasing the metric traits at each locus (negative allele) if epistasis is weak, or for intermediate allele frequencies if it is strong. For the additive/dominance epistatic model, however, covAt > covA applies for low frequencies of the negative alleles at one or both loci and mild epistasis, but this result can be progressively extended to intermediate frequencies as epistasis becomes stronger. Without epistasis the same qualitative results were found, indicating that marginal dominance induced by epistasis can be considered as the primary cause of an increase of the additive covariance

  1. Evolution of the additive genetic variance–covariance matrix under continuous directional selection on a complex behavioural phenotype

    Science.gov (United States)

    Careau, Vincent; Wolak, Matthew E.; Carter, Patrick A.; Garland, Theodore

    2015-01-01

    Given the pace at which human-induced environmental changes occur, a pressing challenge is to determine the speed with which selection can drive evolutionary change. A key determinant of adaptive response to multivariate phenotypic selection is the additive genetic variance–covariance matrix (G). Yet knowledge of G in a population experiencing new or altered selection is not sufficient to predict selection response because G itself evolves in ways that are poorly understood. We experimentally evaluated changes in G when closely related behavioural traits experience continuous directional selection. We applied the genetic covariance tensor approach to a large dataset (n = 17 328 individuals) from a replicated, 31-generation artificial selection experiment that bred mice for voluntary wheel running on days 5 and 6 of a 6-day test. Selection on this subset of G induced proportional changes across the matrix for all 6 days of running behaviour within the first four generations. The changes in G induced by selection resulted in a fourfold slower-than-predicted rate of response to selection. Thus, selection exacerbated constraints within G and limited future adaptive response, a phenomenon that could have profound consequences for populations facing rapid environmental change. PMID:26582016

  2. Evolution of the additive genetic variance-covariance matrix under continuous directional selection on a complex behavioural phenotype.

    Science.gov (United States)

    Careau, Vincent; Wolak, Matthew E; Carter, Patrick A; Garland, Theodore

    2015-11-22

    Given the pace at which human-induced environmental changes occur, a pressing challenge is to determine the speed with which selection can drive evolutionary change. A key determinant of adaptive response to multivariate phenotypic selection is the additive genetic variance-covariance matrix ( G: ). Yet knowledge of G: in a population experiencing new or altered selection is not sufficient to predict selection response because G: itself evolves in ways that are poorly understood. We experimentally evaluated changes in G: when closely related behavioural traits experience continuous directional selection. We applied the genetic covariance tensor approach to a large dataset (n = 17 328 individuals) from a replicated, 31-generation artificial selection experiment that bred mice for voluntary wheel running on days 5 and 6 of a 6-day test. Selection on this subset of G: induced proportional changes across the matrix for all 6 days of running behaviour within the first four generations. The changes in G: induced by selection resulted in a fourfold slower-than-predicted rate of response to selection. Thus, selection exacerbated constraints within G: and limited future adaptive response, a phenomenon that could have profound consequences for populations facing rapid environmental change. © 2015 The Author(s).

  3. Calculation of Covariance Matrix for Multi-mode Gaussian States in Decoherence Processes

    Institute of Scientific and Technical Information of China (English)

    XIANG Shao-Hua; SHAO Bin; SONG Ke-Hui

    2009-01-01

    We investigate the dynamics of n single-mode continuous variable systems in a generic Gaussian state under the influence of the independent and correlated noises making use of the characteristic function method.In two models the bath is assumed to be a squeezed thermal one.We derive an explicit input-output expression between the initial and final covariance matrices.As an example,we study the evolution of entanglement of three-mode Gaussian state embedded in two noisy models.

  4. A Fast Algorithm for the Computation of HAC Covariance Matrix Estimators

    Directory of Open Access Journals (Sweden)

    Jochen Heberle

    2017-01-01

    Full Text Available This paper considers the algorithmic implementation of the heteroskedasticity and autocorrelation consistent (HAC estimation problem for covariance matrices of parameter estimators. We introduce a new algorithm, mainly based on the fast Fourier transform, and show via computer simulation that our algorithm is up to 20 times faster than well-established alternative algorithms. The cumulative effect is substantial if the HAC estimation problem has to be solved repeatedly. Moreover, the bandwidth parameter has no impact on this performance. We provide a general description of the new algorithm as well as code for a reference implementation in R.

  5. The choice of prior distribution for a covariance matrix in multivariate meta-analysis: a simulation study.

    Science.gov (United States)

    Hurtado Rúa, Sandra M; Mazumdar, Madhu; Strawderman, Robert L

    2015-12-30

    Bayesian meta-analysis is an increasingly important component of clinical research, with multivariate meta-analysis a promising tool for studies with multiple endpoints. Model assumptions, including the choice of priors, are crucial aspects of multivariate Bayesian meta-analysis (MBMA) models. In a given model, two different prior distributions can lead to different inferences about a particular parameter. A simulation study was performed in which the impact of families of prior distributions for the covariance matrix of a multivariate normal random effects MBMA model was analyzed. Inferences about effect sizes were not particularly sensitive to prior choice, but the related covariance estimates were. A few families of prior distributions with small relative biases, tight mean squared errors, and close to nominal coverage for the effect size estimates were identified. Our results demonstrate the need for sensitivity analysis and suggest some guidelines for choosing prior distributions in this class of problems. The MBMA models proposed here are illustrated in a small meta-analysis example from the periodontal field and a medium meta-analysis from the study of stroke. Copyright © 2015 John Wiley & Sons, Ltd.

  6. Calculation of the covariant matrix elements, and cross-sections of Compton diffusion on a bound electron

    Science.gov (United States)

    Al Saleh, Salwa

    2016-10-01

    This paper completes a previous published work that calculated analytically the relativistic wavefunctions for bound electron in a Compton diffusion process. This work calculates the relativistic propagator and the Wronskian of the two associated Feynman diagrams of Compton diffusion (emission first and absorption first). Then find an explicit expression for the covariant matrix elements separated into two parts: spin-angular part and radial part. Using these explicit expressions, the effective cross-section for Compton diffusion in the most general form is obtained in terms of basic dynamical and static quantities, like electron's and photon's 4-momenta and atomic number. The form of the cross-section is put ready for numerical calculations.

  7. Simulations of Baryon Acoustic Oscillations II: Covariance matrix of the matter power spectrum

    CERN Document Server

    Takahashi, Ryuichi; Takada, Masahiro; Matsubara, Takahiko; Sugiyama, Naoshi; Kayo, Issha; Nishizawa, Atsushi J; Nishimichi, Takahiro; Saito, Shun; Taruya, Atsushi

    2009-01-01

    We use 5000 cosmological N-body simulations of 1(Gpc/h)^3 box for the concordance LCDM model in order to study the sampling variances of nonlinear matter power spectrum. We show that the non-Gaussian errors can be important even on large length scales relevant for baryon acoustic oscillations (BAO). Our findings are (1) the non-Gaussian errors degrade the cumulative signal-to-noise ratios (S/N) for the power spectrum amplitude by up to a factor of 2 and 4 for redshifts z=1 and 0, respectively. (2) There is little information on the power spectrum amplitudes in the quasi-nonlinear regime, confirming the previous results. (3) The distribution of power spectrum estimators at BAO scales, among the realizations, is well approximated by a Gaussian distribution with variance that is given by the diagonal covariance component. (4) For the redshift-space power spectrum, the degradation in S/N by non-Gaussian errors is mitigated due to nonlinear redshift distortions. (5) For an actual galaxy survey, the additional shot...

  8. The impact of the rank of marker variance-covariance matrix in principal component evaluation for genomic selection applications.

    Science.gov (United States)

    Dimauro, C; Cellesi, M; Pintus, M A; Macciotta, N P P

    2011-12-01

    In genomic selection (GS) programmes, direct genomic values (DGV) are evaluated using information provided by high-density SNP chip. Being DGV accuracy strictly dependent on SNP density, it is likely that an increase in the number of markers per chip will result in severe computational consequences. Aim of present work was to test the effectiveness of principal component analysis (PCA) carried out by chromosome in reducing the marker dimensionality for GS purposes. A simulated data set of 5700 individuals with an equal number of SNP distributed over six chromosomes was used. PCs were extracted both genome-wide (ALL) and separately by chromosome (CHR) and used to predict DGVs. In the ALL scenario, the SNP variance-covariance matrix (S) was singular, positive semi-definite and contained null information which introduces 'spuriousness' in the derived results. On the contrary, the S matrix for each chromosome (CHR scenario) had a full rank. Obtained DGV accuracies were always better for CHR than ALL. Moreover, in the latter scenario, DGV accuracies became soon unsettled as the number of animals decreases, whereas in CHR, they remain stable till 900-1000 individuals. In real applications where a 54k SNP chip is used, the largest number of markers per chromosome is approximately 2500. Thus, a number of around 3000 genotyped animals could lead to reliable results when the original SNP variables are replaced by a reduced number of PCs. © 2011 Blackwell Verlag GmbH.

  9. Predicting the required number of training samples. [for remotely sensed image data based on covariance matrix estimate quality criterion of normal distribution

    Science.gov (United States)

    Kalayeh, H. M.; Landgrebe, D. A.

    1983-01-01

    A criterion which measures the quality of the estimate of the covariance matrix of a multivariate normal distribution is developed. Based on this criterion, the necessary number of training samples is predicted. Experimental results which are used as a guide for determining the number of training samples are included. Previously announced in STAR as N82-28109

  10. Robust design of multiple-input multiple-output radar waveform covariance matrix in the presence of clutter

    Science.gov (United States)

    Guo, Rongyan; Wang, Hongyan

    2016-07-01

    In this work, the issue of robust waveform optimization is addressed in the presence of clutter to improve the worst-case estimation accuracy for collocated multiple-input multiple-output (MIMO) radar. Robust design is necessary due to the fact that waveform design may be sensitive to uncertainties in the initial parameter estimates. Following the min-max approach, the robust waveform covariance matrix design is formulated here on the basis of Cramér-Rao Bound to ease this sensitivity systematically for improving the worst-case accuracy. To tackle the resultant complicated and nonlinear problem, a new diagonal loading (DL)-based iterative approach is developed, in which the inner optimization problem can first be decomposed to some independent subproblems by using the Hadamard's inequality, and then these subproblems can be reformulated into convex issues by using DL method, as well as the outer optimization problem can also be relaxed to a convex issue by translating the nonlinear function into a linear one, and, hence, both of them can be solved very effectively. An optimal solution to the original problem can be obtained via the least-squares fitting of the solution acquired by the iterative approach. Numerical simulations show the efficiency of the proposed method.

  11. Limited plasticity in the phenotypic variance-covariance matrix for male advertisement calls in the black field cricket, Teleogryllus commodus.

    Science.gov (United States)

    Pitchers, W R; Brooks, R; Jennions, M D; Tregenza, T; Dworkin, I; Hunt, J

    2013-05-01

    Phenotypic integration and plasticity are central to our understanding of how complex phenotypic traits evolve. Evolutionary change in complex quantitative traits can be predicted using the multivariate breeders' equation, but such predictions are only accurate if the matrices involved are stable over evolutionary time. Recent study, however, suggests that these matrices are temporally plastic, spatially variable and themselves evolvable. The data available on phenotypic variance-covariance matrix (P) stability are sparse, and largely focused on morphological traits. Here, we compared P for the structure of the complex sexual advertisement call of six divergent allopatric populations of the Australian black field cricket, Teleogryllus commodus. We measured a subset of calls from wild-caught crickets from each of the populations and then a second subset after rearing crickets under common-garden conditions for three generations. In a second experiment, crickets from each population were reared in the laboratory on high- and low-nutrient diets and their calls recorded. In both experiments, we estimated P for call traits and used multiple methods to compare them statistically (Flury hierarchy, geometric subspace comparisons and random skewers). Despite considerable variation in means and variances of individual call traits, the structure of P was largely conserved among populations, across generations and between our rearing diets. Our finding that P remains largely stable, among populations and between environmental conditions, suggests that selection has preserved the structure of call traits in order that they can function as an integrated unit.

  12. Trace evaluation of matrix determinants and inversion of 4 $\\times$ 4 matrices in terms of Dirac covariants

    OpenAIRE

    1998-01-01

    In the following short paper we list some useful results concerning determinants and inverses of matrices. First we show, how to calculate determinants of $d \\times d$ matrices, if their traces are known. As a next step $4 \\times 4$ matrices are expressed in terms of Dirac covariants. The third step is the calculation of the corresponding inverse matrices in terms of Dirac covariants.

  13. Efficient covariance matrix algorithm and realization for images on GPU%基于GPU的高效图像协方差矩阵算法与实现

    Institute of Scientific and Technical Information of China (English)

    陈彬; 陈和平; 李晓卉

    2014-01-01

    To improve the efficiency of covariance matrix computation for image processing to adapt to real‐time or near real‐time requirements of practical applications ,with the general purpose of GPU technology ,the covariance matrix computation was opti‐mized for parallel computation and a novel parallelization approach for covariance matrix computation based on CUDA program‐ming model was proposed .Image processing applications based on covariance matrices can be processed in real time on general personal PC platforms .Moreover ,as to image processing ,there are many other real‐time requirements based on covariance ma‐trices can also be satisfied .Compared to the original CPU implementation ,the proposed GPU implementation improves the effi‐ciency by thousands of times on average .%为提高图像处理领域协方差矩阵的计算效率,满足其在实时要求下的应用,借助GPU 通用计算技术,结合CU‐DA编程模型,对协方差矩阵的计算进行有针对性的并行化优化,设计并实现一种高效的并行图像协方差矩阵算法。为在通用PC平台上使用协方差矩阵并满足实时性需求的各种图像处理应用提供了一个可行的解决方法,对其它领域涉及到协方差矩阵的实时计算也有良好的借鉴作用。与原有的CPU实现方法相比, GPU的效率有了平均数千倍的提升。

  14. Matter power spectrum covariance matrix from the DEUS-PUR ΛCDM simulations: mass resolution and non-Gaussian errors

    Science.gov (United States)

    Blot, L.; Corasaniti, P. S.; Alimi, J.-M.; Reverdy, V.; Rasera, Y.

    2015-01-01

    The upcoming generation of galaxy surveys will probe the distribution of matter in the Universe with unprecedented accuracy. Measurements of the matter power spectrum at different scales and red shifts will provide stringent constraints on the cosmological parameters. However, on non-linear scales this will require an accurate evaluation of the covariance matrix. Here, we compute the covariance matrix of the three-dimensional matter density power spectrum for the concordance ΛCDM cosmology from an ensemble of N-body simulations of the Dark Energy Universe Simulation - Parallel Universe Runs (DEUS-PUR). This consists of 12 288 realizations of a (656 h-1 Mpc)3 simulation box with 2563 particles. We combine this set with an auxiliary sample of 96 simulations of the same volume with 10243 particles. We find the N-body mass resolution effect to be an important source of systematic errors on the covariance at high redshift and small intermediate scales. We correct for this effect by introducing an empirical statistical method which provide an accurate determination of the covariance matrix over a wide range of scales including the baryon oscillations interval. Contrary to previous studies that used smaller N-body ensembles, we find the power spectrum distribution to significantly deviate from expectations of a Gaussian random density field at k ≳ 0.25 h Mpc-1 and z < 0.5. This suggests that for the finite-volume surveys, an unbiased estimate of the ensemble-averaged band power at these scales and red shifts may require a more careful assessment of non-Gaussian errors than previously considered.

  15. Nonparametric statistical inference

    CERN Document Server

    Gibbons, Jean Dickinson

    2010-01-01

    Overall, this remains a very fine book suitable for a graduate-level course in nonparametric statistics. I recommend it for all people interested in learning the basic ideas of nonparametric statistical inference.-Eugenia Stoimenova, Journal of Applied Statistics, June 2012… one of the best books available for a graduate (or advanced undergraduate) text for a theory course on nonparametric statistics. … a very well-written and organized book on nonparametric statistics, especially useful and recommended for teachers and graduate students.-Biometrics, 67, September 2011This excellently presente

  16. COVAR: Computer Program for Multifactor Relative Risks and Tests of Hypotheses Using a Variance-Covariance Matrix from Linear and Log-Linear Regression

    Directory of Open Access Journals (Sweden)

    Leif E. Peterson

    1997-11-01

    Full Text Available A computer program for multifactor relative risks, confidence limits, and tests of hypotheses using regression coefficients and a variance-covariance matrix obtained from a previous additive or multiplicative regression analysis is described in detail. Data used by the program can be stored and input from an external disk-file or entered via the keyboard. The output contains a list of the input data, point estimates of single or joint effects, confidence intervals and tests of hypotheses based on a minimum modified chi-square statistic. Availability of the program is also discussed.

  17. Quantal Response: Nonparametric Modeling

    Science.gov (United States)

    2017-01-01

    spline N−spline Fig. 3 Logistic regression 7 Approved for public release; distribution is unlimited. 5. Nonparametric QR Models Nonparametric linear ...stimulus and probability of response. The Generalized Linear Model approach does not make use of the limit distribution but allows arbitrary functional...7. Conclusions and Recommendations 18 8. References 19 Appendix A. The Linear Model 21 Appendix B. The Generalized Linear Model 33 Appendix C. B

  18. Matter Power Spectrum Covariance Matrix from the DEUS-PUR {\\Lambda}CDM simulations: Mass Resolution and non-Gaussian Errors

    CERN Document Server

    Blot, Linda; Alimi, Jean-Michel; Reverdy, Vincent; Rasera, Yann

    2014-01-01

    The upcoming generation of galaxy surveys will probe the distribution of matter in the universe with unprecedented accuracy. Measurements of the matter power spectrum at different scales and redshifts will provide stringent constraints on the cosmological parameters. However, on non-linear scales this will require an accurate evaluation of the covariance matrix. Here, we compute the covariance matrix of the matter power spectrum for the concordance $\\Lambda$CDM cosmology from an ensemble of N-body simulations of the Dark Energy Universe Simulation - Parallel Universe Runs (DEUS-PUR). This consists of 12288 realizations of a $(656\\,h^{-1}\\,\\textrm{Mpc})^3$ simulation box with $256^3$ particles. We combine this set with an auxiliary sample of 96 simulations of the same volume with $1024^3$ particles to assess the impact of non-Gaussian uncertainties due to mass resolution effects. We find this to be an important source of systematic errors at high redshift and small intermediate scales. We introduce an empirica...

  19. Convergence of the standard RLS method and UDUT factorisation of covariance matrix for solving the algebraic Riccati equation of the DLQR via heuristic approximate dynamic programming

    Science.gov (United States)

    Moraes Rêgo, Patrícia Helena; Viana da Fonseca Neto, João; Ferreira, Ernesto M.

    2015-08-01

    The main focus of this article is to present a proposal to solve, via UDUT factorisation, the convergence and numerical stability problems that are related to the covariance matrix ill-conditioning of the recursive least squares (RLS) approach for online approximations of the algebraic Riccati equation (ARE) solution associated with the discrete linear quadratic regulator (DLQR) problem formulated in the actor-critic reinforcement learning and approximate dynamic programming context. The parameterisations of the Bellman equation, utility function and dynamic system as well as the algebra of Kronecker product assemble a framework for the solution of the DLQR problem. The condition number and the positivity parameter of the covariance matrix are associated with statistical metrics for evaluating the approximation performance of the ARE solution via RLS-based estimators. The performance of RLS approximators is also evaluated in terms of consistence and polarisation when associated with reinforcement learning methods. The used methodology contemplates realisations of online designs for DLQR controllers that is evaluated in a multivariable dynamic system model.

  20. Two-sided bounds on the mean vector and covariance matrix in linear stochastically excited vibration systems with application of the differential calculus of norms

    Directory of Open Access Journals (Sweden)

    Ludwig Kohaupt

    2015-12-01

    Full Text Available For a linear stochastic vibration model in state-space form, $ \\dot{x}(t = A x(t+b(t, \\, x(0=x_0, $ with system matrix A and white noise excitation $ b(t $, under certain conditions, the solution $ x(t $ is a random vector that can be completely described by its mean vector, $ m_x(t:=m_{x(t} $, and its covariance matrix, $ P_x(t:=P_{x(t} $. If matrix $ A $ is asymptotically stable, then $ m_x(t \\rightarrow 0 \\, (t \\rightarrow \\infty $ and $ P_x(t \\rightarrow P \\, (t \\rightarrow \\infty $, where $ P $ is a positive (semi-definite matrix. As the main new points, in this paper, we derive two-sided bounds on $ \\Vert m_x(t\\Vert _2 $ and $ \\Vert P_x(t- P\\Vert _2 $ as well as formulas for the right norm derivatives $ D_+^k \\Vert P_x(t- P\\Vert _2, \\, k=0,1,2 $, and apply these results to the computation of the best constants in the two-sided bounds. The obtained results are of special interest to applied mathematicians and engineers.

  1. Estimation of the parameter covariance matrix for aone-compartment cardiac perfusion model estimated from a dynamic sequencereconstructed using map iterative reconstruction algorithms

    Energy Technology Data Exchange (ETDEWEB)

    Gullberg, Grant T.; Huesman, Ronald H.; Reutter, Bryan W.; Qi,Jinyi; Ghosh Roy, Dilip N.

    2004-01-01

    In dynamic cardiac SPECT estimates of kinetic parameters ofa one-compartment perfusion model are usually obtained in a two stepprocess: 1) first a MAP iterative algorithm, which properly models thePoisson statistics and the physics of the data acquisition, reconstructsa sequence of dynamic reconstructions, 2) then kinetic parameters areestimated from time activity curves generated from the dynamicreconstructions. This paper provides a method for calculating thecovariance matrix of the kinetic parameters, which are determined usingweighted least squares fitting that incorporates the estimated varianceand covariance of the dynamic reconstructions. For each transaxial slicesets of sequential tomographic projections are reconstructed into asequence of transaxial reconstructions usingfor each reconstruction inthe time sequence an iterative MAP reconstruction to calculate themaximum a priori reconstructed estimate. Time-activity curves for a sumof activity in a blood region inside the left ventricle and a sum in acardiac tissue region are generated. Also, curves for the variance of thetwo estimates of the sum and for the covariance between the two ROIestimates are generated as a function of time at convergence using anexpression obtained from the fixed-point solution of the statisticalerror of the reconstruction. A one-compartment model is fit to the tissueactivity curves assuming a noisy blood input function to give weightedleast squares estimates of blood volume fraction, wash-in and wash-outrate constants specifying the kinetics of 99mTc-teboroxime for theleftventricular myocardium. Numerical methods are used to calculate thesecond derivative of the chi-square criterion to obtain estimates of thecovariance matrix for the weighted least square parameter estimates. Eventhough the method requires one matrix inverse for each time interval oftomographic acquisition, efficient estimates of the tissue kineticparameters in a dynamic cardiac SPECT study can be obtained with

  2. Nonparametric correlation models for portfolio allocation

    DEFF Research Database (Denmark)

    Aslanidis, Nektarios; Casas, Isabel

    2013-01-01

    This article proposes time-varying nonparametric and semiparametric estimators of the conditional cross-correlation matrix in the context of portfolio allocation. Simulations results show that the nonparametric and semiparametric models are best in DGPs with substantial variability or structural...... breaks in correlations. Only when correlations are constant does the parametric DCC model deliver the best outcome. The methodologies are illustrated by evaluating two interesting portfolios. The first portfolio consists of the equity sector SPDRs and the S&P 500, while the second one contains major...

  3. Nonparametric correlation models for portfolio allocation

    DEFF Research Database (Denmark)

    Aslanidis, Nektarios; Casas, Isabel

    2013-01-01

    breaks in correlations. Only when correlations are constant does the parametric DCC model deliver the best outcome. The methodologies are illustrated by evaluating two interesting portfolios. The first portfolio consists of the equity sector SPDRs and the S&P 500, while the second one contains major......This article proposes time-varying nonparametric and semiparametric estimators of the conditional cross-correlation matrix in the context of portfolio allocation. Simulations results show that the nonparametric and semiparametric models are best in DGPs with substantial variability or structural...... currencies. Results show the nonparametric model generally dominates the others when evaluating in-sample. However, the semiparametric model is best for out-of-sample analysis....

  4. Nonparametric statistical methods

    CERN Document Server

    Hollander, Myles; Chicken, Eric

    2013-01-01

    Praise for the Second Edition"This book should be an essential part of the personal library of every practicing statistician."-Technometrics  Thoroughly revised and updated, the new edition of Nonparametric Statistical Methods includes additional modern topics and procedures, more practical data sets, and new problems from real-life situations. The book continues to emphasize the importance of nonparametric methods as a significant branch of modern statistics and equips readers with the conceptual and technical skills necessary to select and apply the appropriate procedures for any given sit

  5. Bayesian nonparametric data analysis

    CERN Document Server

    Müller, Peter; Jara, Alejandro; Hanson, Tim

    2015-01-01

    This book reviews nonparametric Bayesian methods and models that have proven useful in the context of data analysis. Rather than providing an encyclopedic review of probability models, the book’s structure follows a data analysis perspective. As such, the chapters are organized by traditional data analysis problems. In selecting specific nonparametric models, simpler and more traditional models are favored over specialized ones. The discussed methods are illustrated with a wealth of examples, including applications ranging from stylized examples to case studies from recent literature. The book also includes an extensive discussion of computational methods and details on their implementation. R code for many examples is included in on-line software pages.

  6. A Bayesian nonparametric meta-analysis model.

    Science.gov (United States)

    Karabatsos, George; Talbott, Elizabeth; Walker, Stephen G

    2015-03-01

    In a meta-analysis, it is important to specify a model that adequately describes the effect-size distribution of the underlying population of studies. The conventional normal fixed-effect and normal random-effects models assume a normal effect-size population distribution, conditionally on parameters and covariates. For estimating the mean overall effect size, such models may be adequate, but for prediction, they surely are not if the effect-size distribution exhibits non-normal behavior. To address this issue, we propose a Bayesian nonparametric meta-analysis model, which can describe a wider range of effect-size distributions, including unimodal symmetric distributions, as well as skewed and more multimodal distributions. We demonstrate our model through the analysis of real meta-analytic data arising from behavioral-genetic research. We compare the predictive performance of the Bayesian nonparametric model against various conventional and more modern normal fixed-effects and random-effects models.

  7. Cholesky Factorization for Covariance Matrix Recovery%Cholesky分解在协方差矩阵恢复中的使用

    Institute of Scientific and Technical Information of China (English)

    杜航原; 郝燕玲; 赵玉新; 陈立娟

    2012-01-01

    For simultaneous localization and mapping based on sparse extended information filter, we compare the principles of nearest neighbor data association, maximum likelihood data association and joint compatibility test data association, and discuss the requirements of marginal covariance matrix recovering in data association. A computationally efficient approach based on Cholesky factorization is proposed to exactly recover the marginal covariance from information matrix. In the simulation, we compare the proposed algorithm with covariance bound approximation, and analyze three common data association approaches using the proposed algorithm in SLAM based on a sparse extended information filter. The results show that the proposed recovery algorithm is suitable for various data association approaches, leading to high localization accuracy and reduced computational complexity. Performance of different data association approaches in SEIF-SLAM are discussed.%针对基于稀疏扩展信息滤波的同步定位与地图创建(simultaneous localization and mapping,SLAM)问题,分析并比较了最近邻数据关联、极大似然数据关联以及联合相容性检验数据关联的原理,阐述了边缘协方差矩阵恢复的必要性.在此基础上提出一种利用Cholesky分解由信息矩阵准确恢复协方差任意元素的方法,该方法具有较高的计算效率.在仿真实验中将该方法与协方差边界估计法比较,并分别用于3种数据关联算法的比较分析,表明所提出的方法适用于多种数据关联方法,能在保证定位精度的同时有效控制算法复杂度.最后对各种数据关联算法在稀疏扩展信息滤波SLAM中的性能进行了讨论.

  8. Bilateral filtering using the full noise covariance matrix applied to x-ray phase-contrast computed tomography

    Science.gov (United States)

    Allner, S.; Koehler, T.; Fehringer, A.; Birnbacher, L.; Willner, M.; Pfeiffer, F.; Noël, P. B.

    2016-05-01

    The purpose of this work is to develop an image-based de-noising algorithm that exploits complementary information and noise statistics from multi-modal images, as they emerge in x-ray tomography techniques, for instance grating-based phase-contrast CT and spectral CT. Among the noise reduction methods, image-based de-noising is one popular approach and the so-called bilateral filter is a well known algorithm for edge-preserving filtering. We developed a generalization of the bilateral filter for the case where the imaging system provides two or more perfectly aligned images. The proposed generalization is statistically motivated and takes the full second order noise statistics of these images into account. In particular, it includes a noise correlation between the images and spatial noise correlation within the same image. The novel generalized three-dimensional bilateral filter is applied to the attenuation and phase images created with filtered backprojection reconstructions from grating-based phase-contrast tomography. In comparison to established bilateral filters, we obtain improved noise reduction and at the same time a better preservation of edges in the images on the examples of a simulated soft-tissue phantom, a human cerebellum and a human artery sample. The applied full noise covariance is determined via cross-correlation of the image noise. The filter results yield an improved feature recovery based on enhanced noise suppression and edge preservation as shown here on the example of attenuation and phase images captured with grating-based phase-contrast computed tomography. This is supported by quantitative image analysis. Without being bound to phase-contrast imaging, this generalized filter is applicable to any kind of noise-afflicted image data with or without noise correlation. Therefore, it can be utilized in various imaging applications and fields.

  9. Bilateral filtering using the full noise covariance matrix applied to x-ray phase-contrast computed tomography.

    Science.gov (United States)

    Allner, S; Koehler, T; Fehringer, A; Birnbacher, L; Willner, M; Pfeiffer, F; Noël, P B

    2016-05-21

    The purpose of this work is to develop an image-based de-noising algorithm that exploits complementary information and noise statistics from multi-modal images, as they emerge in x-ray tomography techniques, for instance grating-based phase-contrast CT and spectral CT. Among the noise reduction methods, image-based de-noising is one popular approach and the so-called bilateral filter is a well known algorithm for edge-preserving filtering. We developed a generalization of the bilateral filter for the case where the imaging system provides two or more perfectly aligned images. The proposed generalization is statistically motivated and takes the full second order noise statistics of these images into account. In particular, it includes a noise correlation between the images and spatial noise correlation within the same image. The novel generalized three-dimensional bilateral filter is applied to the attenuation and phase images created with filtered backprojection reconstructions from grating-based phase-contrast tomography. In comparison to established bilateral filters, we obtain improved noise reduction and at the same time a better preservation of edges in the images on the examples of a simulated soft-tissue phantom, a human cerebellum and a human artery sample. The applied full noise covariance is determined via cross-correlation of the image noise. The filter results yield an improved feature recovery based on enhanced noise suppression and edge preservation as shown here on the example of attenuation and phase images captured with grating-based phase-contrast computed tomography. This is supported by quantitative image analysis. Without being bound to phase-contrast imaging, this generalized filter is applicable to any kind of noise-afflicted image data with or without noise correlation. Therefore, it can be utilized in various imaging applications and fields.

  10. Technical Note: Variance-covariance matrix and averaging kernels for the Levenberg-Marquardt solution of the retrieval of atmospheric vertical profiles

    Directory of Open Access Journals (Sweden)

    S. Ceccherini

    2010-03-01

    Full Text Available The variance-covariance matrix (VCM and the averaging kernel matrix (AKM are widely used tools to characterize atmospheric vertical profiles retrieved from remote sensing measurements. Accurate estimation of these quantities is essential for both the evaluation of the quality of the retrieved profiles and for the correct use of the profiles themselves in subsequent applications such as data comparison, data assimilation and data fusion. We propose a new method to estimate the VCM and AKM of vertical profiles retrieved using the Levenberg-Marquardt iterative technique. We apply the new method to the inversion of simulated limb emission measurements. Then we compare the obtained VCM and AKM with those resulting from other methods already published in the literature and with accurate estimates derived using statistical and numerical estimators. The proposed method accounts for all the iterations done in the inversion and provides the most accurate VCM and AKM. Furthermore, it correctly estimates the VCM and the AKM also if the retrieval iterations are stopped when a physically meaningful convergence criterion is fulfilled, i.e. before achievement of the numerical convergence at machine precision. The method can be easily implemented in any Levenberg-Marquardt iterative retrieval scheme, either constrained or unconstrained, without significant computational overhead.

  11. Status of the R-matrix Code AMUR toward a consistent cross-section evaluation and covariance analysis for the light nuclei

    Science.gov (United States)

    Kunieda, Satoshi

    2017-09-01

    We report the status of the R-matrix code AMUR toward consistent cross-section evaluation and covariance analysis for the light-mass nuclei. The applicable limit of the code is extended by including computational capability for the charged-particle elastic scattering cross-sections and the neutron capture cross-sections as example results are shown in the main texts. A simultaneous analysis is performed on the 17O compound system including the 16O(n,tot) and 13C(α,n)16O reactions together with the 16O(n,n) and 13C(α,α) scattering cross-sections. It is found that a large theoretical background is required for each reaction process to obtain a simultaneous fit with all the experimental cross-sections we analyzed. Also, the hard-sphere radii should be assumed to be different from the channel radii. Although these are technical approaches, we could learn roles and sources of the theoretical background in the standard R-matrix.

  12. Nonparametric Predictive Regression

    OpenAIRE

    Ioannis Kasparis; Elena Andreou; Phillips, Peter C.B.

    2012-01-01

    A unifying framework for inference is developed in predictive regressions where the predictor has unknown integration properties and may be stationary or nonstationary. Two easily implemented nonparametric F-tests are proposed. The test statistics are related to those of Kasparis and Phillips (2012) and are obtained by kernel regression. The limit distribution of these predictive tests holds for a wide range of predictors including stationary as well as non-stationary fractional and near unit...

  13. Covariance specification and estimation to improve top-down Green House Gas emission estimates

    Science.gov (United States)

    Ghosh, S.; Lopez-Coto, I.; Prasad, K.; Whetstone, J. R.

    2015-12-01

    The National Institute of Standards and Technology (NIST) operates the North-East Corridor (NEC) project and the Indianapolis Flux Experiment (INFLUX) in order to develop measurement methods to quantify sources of Greenhouse Gas (GHG) emissions as well as their uncertainties in urban domains using a top down inversion method. Top down inversion updates prior knowledge using observations in a Bayesian way. One primary consideration in a Bayesian inversion framework is the covariance structure of (1) the emission prior residuals and (2) the observation residuals (i.e. the difference between observations and model predicted observations). These covariance matrices are respectively referred to as the prior covariance matrix and the model-data mismatch covariance matrix. It is known that the choice of these covariances can have large effect on estimates. The main objective of this work is to determine the impact of different covariance models on inversion estimates and their associated uncertainties in urban domains. We use a pseudo-data Bayesian inversion framework using footprints (i.e. sensitivities of tower measurements of GHGs to surface emissions) and emission priors (based on Hestia project to quantify fossil-fuel emissions) to estimate posterior emissions using different covariance schemes. The posterior emission estimates and uncertainties are compared to the hypothetical truth. We find that, if we correctly specify spatial variability and spatio-temporal variability in prior and model-data mismatch covariances respectively, then we can compute more accurate posterior estimates. We discuss few covariance models to introduce space-time interacting mismatches along with estimation of the involved parameters. We then compare several candidate prior spatial covariance models from the Matern covariance class and estimate their parameters with specified mismatches. We find that best-fitted prior covariances are not always best in recovering the truth. To achieve

  14. Nonparametric statistical inference

    CERN Document Server

    Gibbons, Jean Dickinson

    2014-01-01

    Thoroughly revised and reorganized, the fourth edition presents in-depth coverage of the theory and methods of the most widely used nonparametric procedures in statistical analysis and offers example applications appropriate for all areas of the social, behavioral, and life sciences. The book presents new material on the quantiles, the calculation of exact and simulated power, multiple comparisons, additional goodness-of-fit tests, methods of analysis of count data, and modern computer applications using MINITAB, SAS, and STATXACT. It includes tabular guides for simplified applications of tests and finding P values and confidence interval estimates.

  15. Multilevel covariance regression with correlated random effects in the mean and variance structure.

    Science.gov (United States)

    Quintero, Adrian; Lesaffre, Emmanuel

    2017-09-01

    Multivariate regression methods generally assume a constant covariance matrix for the observations. In case a heteroscedastic model is needed, the parametric and nonparametric covariance regression approaches can be restrictive in the literature. We propose a multilevel regression model for the mean and covariance structure, including random intercepts in both components and allowing for correlation between them. The implied conditional covariance function can be different across clusters as a result of the random effect in the variance structure. In addition, allowing for correlation between the random intercepts in the mean and covariance makes the model convenient for skewedly distributed responses. Furthermore, it permits us to analyse directly the relation between the mean response level and the variability in each cluster. Parameter estimation is carried out via Gibbs sampling. We compare the performance of our model to other covariance modelling approaches in a simulation study. Finally, the proposed model is applied to the RN4CAST dataset to identify the variables that impact burnout of nurses in Belgium. © 2017 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

  16. Parameter inference with estimated covariance matrices

    CERN Document Server

    Sellentin, Elena

    2015-01-01

    When inferring parameters from a Gaussian-distributed data set by computing a likelihood, a covariance matrix is needed that describes the data errors and their correlations. If the covariance matrix is not known a priori, it may be estimated and thereby becomes a random object with some intrinsic uncertainty itself. We show how to infer parameters in the presence of such an estimated covariance matrix, by marginalising over the true covariance matrix, conditioned on its estimated value. This leads to a likelihood function that is no longer Gaussian, but rather an adapted version of a multivariate $t$-distribution, which has the same numerical complexity as the multivariate Gaussian. As expected, marginalisation over the true covariance matrix improves inference when compared with Hartlap et al.'s method, which uses an unbiased estimate of the inverse covariance matrix but still assumes that the likelihood is Gaussian.

  17. Covariance Applications with Kiwi

    Science.gov (United States)

    Mattoon, C. M.; Brown, D.; Elliott, J. B.

    2012-05-01

    The Computational Nuclear Physics group at Lawrence Livermore National Laboratory (LLNL) is developing a new tool, named `Kiwi', that is intended as an interface between the covariance data increasingly available in major nuclear reaction libraries (including ENDF and ENDL) and large-scale Uncertainty Quantification (UQ) studies. Kiwi is designed to integrate smoothly into large UQ studies, using the covariance matrix to generate multiple variations of nuclear data. The code has been tested using critical assemblies as a test case, and is being integrated into LLNL's quality assurance and benchmarking for nuclear data.

  18. Covariance Applications with Kiwi

    Directory of Open Access Journals (Sweden)

    Elliott J.B.

    2012-05-01

    Full Text Available The Computational Nuclear Physics group at Lawrence Livermore National Laboratory (LLNL is developing a new tool, named ‘Kiwi’, that is intended as an interface between the covariance data increasingly available in major nuclear reaction libraries (including ENDF and ENDL and large-scale Uncertainty Quantification (UQ studies. Kiwi is designed to integrate smoothly into large UQ studies, using the covariance matrix to generate multiple variations of nuclear data. The code has been tested using critical assemblies as a test case, and is being integrated into LLNL's quality assurance and benchmarking for nuclear data.

  19. Nonparametric tests for censored data

    CERN Document Server

    Bagdonavicus, Vilijandas; Nikulin, Mikhail

    2013-01-01

    This book concerns testing hypotheses in non-parametric models. Generalizations of many non-parametric tests to the case of censored and truncated data are considered. Most of the test results are proved and real applications are illustrated using examples. Theories and exercises are provided. The incorrect use of many tests applying most statistical software is highlighted and discussed.

  20. Nonparametric additive regression for repeatedly measured data

    KAUST Repository

    Carroll, R. J.

    2009-05-20

    We develop an easily computed smooth backfitting algorithm for additive model fitting in repeated measures problems. Our methodology easily copes with various settings, such as when some covariates are the same over repeated response measurements. We allow for a working covariance matrix for the regression errors, showing that our method is most efficient when the correct covariance matrix is used. The component functions achieve the known asymptotic variance lower bound for the scalar argument case. Smooth backfitting also leads directly to design-independent biases in the local linear case. Simulations show our estimator has smaller variance than the usual kernel estimator. This is also illustrated by an example from nutritional epidemiology. © 2009 Biometrika Trust.

  1. CURRENT STATUS OF NONPARAMETRIC STATISTICS

    Directory of Open Access Journals (Sweden)

    Orlov A. I.

    2015-02-01

    Full Text Available Nonparametric statistics is one of the five points of growth of applied mathematical statistics. Despite the large number of publications on specific issues of nonparametric statistics, the internal structure of this research direction has remained undeveloped. The purpose of this article is to consider its division into regions based on the existing practice of scientific activity determination of nonparametric statistics and classify investigations on nonparametric statistical methods. Nonparametric statistics allows to make statistical inference, in particular, to estimate the characteristics of the distribution and testing statistical hypotheses without, as a rule, weakly proven assumptions about the distribution function of samples included in a particular parametric family. For example, the widespread belief that the statistical data are often have the normal distribution. Meanwhile, analysis of results of observations, in particular, measurement errors, always leads to the same conclusion - in most cases the actual distribution significantly different from normal. Uncritical use of the hypothesis of normality often leads to significant errors, in areas such as rejection of outlying observation results (emissions, the statistical quality control, and in other cases. Therefore, it is advisable to use nonparametric methods, in which the distribution functions of the results of observations are imposed only weak requirements. It is usually assumed only their continuity. On the basis of generalization of numerous studies it can be stated that to date, using nonparametric methods can solve almost the same number of tasks that previously used parametric methods. Certain statements in the literature are incorrect that nonparametric methods have less power, or require larger sample sizes than parametric methods. Note that in the nonparametric statistics, as in mathematical statistics in general, there remain a number of unresolved problems

  2. Nonparametric statistical methods using R

    CERN Document Server

    Kloke, John

    2014-01-01

    A Practical Guide to Implementing Nonparametric and Rank-Based ProceduresNonparametric Statistical Methods Using R covers traditional nonparametric methods and rank-based analyses, including estimation and inference for models ranging from simple location models to general linear and nonlinear models for uncorrelated and correlated responses. The authors emphasize applications and statistical computation. They illustrate the methods with many real and simulated data examples using R, including the packages Rfit and npsm.The book first gives an overview of the R language and basic statistical c

  3. Bayesian Nonparametric Mixture Estimation for Time-Indexed Functional Data in R

    Directory of Open Access Journals (Sweden)

    Terrance D. Savitsky

    2016-08-01

    Full Text Available We present growfunctions for R that offers Bayesian nonparametric estimation models for analysis of dependent, noisy time series data indexed by a collection of domains. This data structure arises from combining periodically published government survey statistics, such as are reported in the Current Population Study (CPS. The CPS publishes monthly, by-state estimates of employment levels, where each state expresses a noisy time series. Published state-level estimates from the CPS are composed from household survey responses in a model-free manner and express high levels of volatility due to insufficient sample sizes. Existing software solutions borrow information over a modeled time-based dependence to extract a de-noised time series for each domain. These solutions, however, ignore the dependence among the domains that may be additionally leveraged to improve estimation efficiency. The growfunctions package offers two fully nonparametric mixture models that simultaneously estimate both a time and domain-indexed dependence structure for a collection of time series: (1 A Gaussian process (GP construction, which is parameterized through the covariance matrix, estimates a latent function for each domain. The covariance parameters of the latent functions are indexed by domain under a Dirichlet process prior that permits estimation of the dependence among functions across the domains: (2 An intrinsic Gaussian Markov random field prior construction provides an alternative to the GP that expresses different computation and estimation properties. In addition to performing denoised estimation of latent functions from published domain estimates, growfunctions allows estimation of collections of functions for observation units (e.g., households, rather than aggregated domains, by accounting for an informative sampling design under which the probabilities for inclusion of observation units are related to the response variable. growfunctions includes plot

  4. High resolution MUSIC algorithm reconstructing covariance matrix in low SNR%低信噪比下重构协方差矩阵的高分辨MUSIC算法

    Institute of Scientific and Technical Information of China (English)

    杜梓冰; 杨坤德

    2013-01-01

    在满足对称分布的海洋噪声场中,为提高低信噪比条件下目标方位估计性能,提出一种重构信号协方差矩阵的MUSIC算法。利用数据协方差矩阵虚部与对称噪声无关的性质,根据协方差矩阵虚部和虚部MUSIC算法的预估角重构出信号协方差矩阵,在此基础上实现MUSIC算法。仿真结果表明,所提算法相比常规MUSIC算法能有效降低对称噪声的影响,提高方位估计性能,并避免双边谱的出现,有更高的分辨率和更低的分辨门限。还研究了协方差矩阵的Toeplitz修正处理对于MUSIC类算法的改善作用。仿真表明,Toeplitz修正处理能显著提高MUSIC类算法的分辨性能。%In the symmetrical distribution ocean noise field, in order to improve the direction-of-arrival (DOA) estima-tion performance in low signal-to-noise (SNR) conditions, a novel MUSIC algorithm that reconstructs the signal cova-riance matrix is presented. Based on the properties that the image part of data covariance matrix has no relation with the symmetric noise, the signal covariance matrix is reconstructed by using the image part of data covariance matrix and the DOA which is estimated by Image-MUSIC algorithm. Thus, the MUSIC algorithm is implemented. Simulation results show that the new algorithm effectively reduces the influence of symmetrical noise on DOA estimation and avoids bi-lateral spectrum arising, moreover the algorithm achieves higher resolution and lower resolution threshold in low SNR cases. Covariance matrix’s Toeplitz modification that aims to improve the DOA estimation performance of MUSIC algorithm has also been studied, and simulation results show that the resolution of MUSIC algorithm is improved sig-nificantly.

  5. Parametrically guided estimation in nonparametric varying coefficient models with quasi-likelihood.

    Science.gov (United States)

    Davenport, Clemontina A; Maity, Arnab; Wu, Yichao

    2015-04-01

    Varying coefficient models allow us to generalize standard linear regression models to incorporate complex covariate effects by modeling the regression coefficients as functions of another covariate. For nonparametric varying coefficients, we can borrow the idea of parametrically guided estimation to improve asymptotic bias. In this paper, we develop a guided estimation procedure for the nonparametric varying coefficient models. Asymptotic properties are established for the guided estimators and a method of bandwidth selection via bias-variance tradeoff is proposed. We compare the performance of the guided estimator with that of the unguided estimator via both simulation and real data examples.

  6. Partially linear varying coefficient models stratified by a functional covariate

    KAUST Repository

    Maity, Arnab

    2012-10-01

    We consider the problem of estimation in semiparametric varying coefficient models where the covariate modifying the varying coefficients is functional and is modeled nonparametrically. We develop a kernel-based estimator of the nonparametric component and a profiling estimator of the parametric component of the model and derive their asymptotic properties. Specifically, we show the consistency of the nonparametric functional estimates and derive the asymptotic expansion of the estimates of the parametric component. We illustrate the performance of our methodology using a simulation study and a real data application.

  7. Partially Linear Varying Coefficient Models Stratified by a Functional Covariate.

    Science.gov (United States)

    Maity, Arnab; Huang, Jianhua Z

    2012-10-01

    We consider the problem of estimation in semiparametric varying coefficient models where the covariate modifying the varying coefficients is functional and is modeled nonparametrically. We develop a kernel-based estimator of the nonparametric component and a profiling estimator of the parametric component of the model and derive their asymptotic properties. Specifically, we show the consistency of the nonparametric functional estimates and derive the asymptotic expansion of the estimates of the parametric component. We illustrate the performance of our methodology using a simulation study and a real data application.

  8. A new noise covariance matrix estimation metho d of Kalman filter for satellite clo ck errors%一种新的卫星钟差Kalman滤波噪声协方差估计方法∗

    Institute of Scientific and Technical Information of China (English)

    林旭; 罗志才

    2015-01-01

    采用Kalman滤波方法进行钟差参数计算和预报时,需确定Kalman滤波噪声协方差矩阵。针对这一问题,提出了一种新的卫星钟差Kalman滤波噪声协方差估计方法,通过建立新息的相关函数序列与未知的噪声参数间的线性函数模型,采用最小二乘法进行噪声参数估计。采用精密钟差数据进行钟差参数估计和预报分析,结果表明,该方法具有较好的收敛性,并与顾及随机噪声模型的开窗分类因子自适应抗差估计方法进行对比分析,验证了新方法的正确性和有效性。%The satellite clock plays a key role in the global navigation satellite system (GNSS). The accuracy of GNSS and its applications depend on the quality of the satellite clock. Therefore, precisely estimating and predicting the satellite clock is an important issue in the fields of GNSS and its application. As an optimal estimation algorithm, Kalman filter has been used to estimate and predict the satellite clock. However, in a conventional Kalman filter algorithm, the noise covariance matrices of satellite clock need to be predetermined, which restricts its further applications since the noise covariance matrices, especially the process noise covariance matrix, are usually unknown in the real cases. With inappropriate noise covariance matrices, the state estimation of conventional Kalman filter is suboptimal. To cope with this problem, a new noise covariance matrix estimation method of Kalman filter is proposed, and then we apply it to the problem of satellite clock estimation and prediction. Considering the fact that the process noise covariance matrix depends on the unknown noise parameters, the problem of estimating process noise covariance matrix can be solved by estimating the unknown noise parameters. First, the correlation between the Kalman innovations is used to establish a linear relationship with the unknown noise parameters. Then the unknown parameters can be

  9. Nonparametric inference procedures for multistate life table analysis.

    Science.gov (United States)

    Dow, M M

    1985-01-01

    Recent generalizations of the classical single state life table procedures to the multistate case provide the means to analyze simultaneously the mobility and mortality experience of 1 or more cohorts. This paper examines fairly general nonparametric combinatorial matrix procedures, known as quadratic assignment, as an analysis technic of various transitional patterns commonly generated by cohorts over the life cycle course. To some degree, the output from a multistate life table analysis suggests inference procedures. In his discussion of multstate life table construction features, the author focuses on the matrix formulation of the problem. He then presents several examples of the proposed nonparametric procedures. Data for the mobility and life expectancies at birth matrices come from the 458 member Cayo Santiago rhesus monkey colony. The author's matrix combinatorial approach to hypotheses testing may prove to be a useful inferential strategy in several multidimensional demographic areas.

  10. Semi- and Nonparametric ARCH Processes

    Directory of Open Access Journals (Sweden)

    Oliver B. Linton

    2011-01-01

    Full Text Available ARCH/GARCH modelling has been successfully applied in empirical finance for many years. This paper surveys the semiparametric and nonparametric methods in univariate and multivariate ARCH/GARCH models. First, we introduce some specific semiparametric models and investigate the semiparametric and nonparametrics estimation techniques applied to: the error density, the functional form of the volatility function, the relationship between mean and variance, long memory processes, locally stationary processes, continuous time processes and multivariate models. The second part of the paper is about the general properties of such processes, including stationary conditions, ergodic conditions and mixing conditions. The last part is on the estimation methods in ARCH/GARCH processes.

  11. Universality of Covariance Matrices

    CERN Document Server

    Pillai, Natesh S

    2011-01-01

    We prove the universality of covariance matrices of the form $H_{N \\times N} = {1 \\over N} \\tp{X}X$ where $[X]_{M \\times N}$ is a rectangular matrix with independent real valued entries $[x_{ij}]$ satisfying $\\E \\,x_{ij} = 0$ and $\\E \\,x^2_{ij} = {1 \\over M}$, $N, M\\to \\infty$. Furthermore it is assumed that these entries have sub-exponential tails. We will study the asymptotics in the regime $N/M = d_N \\in (0,\\infty), \\lim_{N\\to \\infty}d_N \

  12. Levy Matrices and Financial Covariances

    Science.gov (United States)

    Burda, Zdzislaw; Jurkiewicz, Jerzy; Nowak, Maciej A.; Papp, Gabor; Zahed, Ismail

    2003-10-01

    In a given market, financial covariances capture the intra-stock correlations and can be used to address statistically the bulk nature of the market as a complex system. We provide a statistical analysis of three SP500 covariances with evidence for raw tail distributions. We study the stability of these tails against reshuffling for the SP500 data and show that the covariance with the strongest tails is robust, with a spectral density in remarkable agreement with random Lévy matrix theory. We study the inverse participation ratio for the three covariances. The strong localization observed at both ends of the spectral density is analogous to the localization exhibited in the random Lévy matrix ensemble. We discuss two competitive mechanisms responsible for the occurrence of an extensive and delocalized eigenvalue at the edge of the spectrum: (a) the Lévy character of the entries of the correlation matrix and (b) a sort of off-diagonal order induced by underlying inter-stock correlations. (b) can be destroyed by reshuffling, while (a) cannot. We show that the stocks with the largest scattering are the least susceptible to correlations, and likely candidates for the localized states. We introduce a simple model for price fluctuations which captures behavior of the SP500 covariances. It may be of importance for assets diversification.

  13. Nonparametric estimation of ultrasound pulses

    DEFF Research Database (Denmark)

    Jensen, Jørgen Arendt; Leeman, Sidney

    1994-01-01

    An algorithm for nonparametric estimation of 1D ultrasound pulses in echo sequences from human tissues is derived. The technique is a variation of the homomorphic filtering technique using the real cepstrum, and the underlying basis of the method is explained. The algorithm exploits a priori...

  14. Testing discontinuities in nonparametric regression

    KAUST Repository

    Dai, Wenlin

    2017-01-19

    In nonparametric regression, it is often needed to detect whether there are jump discontinuities in the mean function. In this paper, we revisit the difference-based method in [13 H.-G. Müller and U. Stadtmüller, Discontinuous versus smooth regression, Ann. Stat. 27 (1999), pp. 299–337. doi: 10.1214/aos/1018031100

  15. Non-Parametric Inference in Astrophysics

    CERN Document Server

    Wasserman, L H; Nichol, R C; Genovese, C; Jang, W; Connolly, A J; Moore, A W; Schneider, J; Wasserman, Larry; Miller, Christopher J.; Nichol, Robert C.; Genovese, Chris; Jang, Woncheol; Connolly, Andrew J.; Moore, Andrew W.; Schneider, Jeff; group, the PICA

    2001-01-01

    We discuss non-parametric density estimation and regression for astrophysics problems. In particular, we show how to compute non-parametric confidence intervals for the location and size of peaks of a function. We illustrate these ideas with recent data on the Cosmic Microwave Background. We also briefly discuss non-parametric Bayesian inference.

  16. Non-parametric estimation of Fisher information from real data

    CERN Document Server

    Shemesh, Omri Har; Miñano, Borja; Hoekstra, Alfons G; Sloot, Peter M A

    2015-01-01

    The Fisher Information matrix is a widely used measure for applications ranging from statistical inference, information geometry, experiment design, to the study of criticality in biological systems. Yet there is no commonly accepted non-parametric algorithm to estimate it from real data. In this rapid communication we show how to accurately estimate the Fisher information in a nonparametric way. We also develop a numerical procedure to minimize the errors by choosing the interval of the finite difference scheme necessary to compute the derivatives in the definition of the Fisher information. Our method uses the recently published "Density Estimation using Field Theory" algorithm to compute the probability density functions for continuous densities. We use the Fisher information of the normal distribution to validate our method and as an example we compute the temperature component of the Fisher Information Matrix in the two dimensional Ising model and show that it obeys the expected relation to the heat capa...

  17. Adjusting for covariate effects on classification accuracy using the covariate-adjusted receiver operating characteristic curve.

    Science.gov (United States)

    Janes, Holly; Pepe, Margaret S

    2009-06-01

    Recent scientific and technological innovations have produced an abundance of potential markers that are being investigated for their use in disease screening and diagnosis. In evaluating these markers, it is often necessary to account for covariates associated with the marker of interest. Covariates may include subject characteristics, expertise of the test operator, test procedures or aspects of specimen handling. In this paper, we propose the covariate-adjusted receiver operating characteristic curve, a measure of covariate-adjusted classification accuracy. Nonparametric and semiparametric estimators are proposed, asymptotic distribution theory is provided and finite sample performance is investigated. For illustration we characterize the age-adjusted discriminatory accuracy of prostate-specific antigen as a biomarker for prostate cancer.

  18. Nonparametric Inference for Periodic Sequences

    KAUST Repository

    Sun, Ying

    2012-02-01

    This article proposes a nonparametric method for estimating the period and values of a periodic sequence when the data are evenly spaced in time. The period is estimated by a "leave-out-one-cycle" version of cross-validation (CV) and complements the periodogram, a widely used tool for period estimation. The CV method is computationally simple and implicitly penalizes multiples of the smallest period, leading to a "virtually" consistent estimator of integer periods. This estimator is investigated both theoretically and by simulation.We also propose a nonparametric test of the null hypothesis that the data have constantmean against the alternative that the sequence of means is periodic. Finally, our methodology is demonstrated on three well-known time series: the sunspots and lynx trapping data, and the El Niño series of sea surface temperatures. © 2012 American Statistical Association and the American Society for Quality.

  19. Nonparametric Econometrics: The np Package

    Directory of Open Access Journals (Sweden)

    Tristen Hayfield

    2008-07-01

    Full Text Available We describe the R np package via a series of applications that may be of interest to applied econometricians. The np package implements a variety of nonparametric and semiparametric kernel-based estimators that are popular among econometricians. There are also procedures for nonparametric tests of significance and consistent model specification tests for parametric mean regression models and parametric quantile regression models, among others. The np package focuses on kernel methods appropriate for the mix of continuous, discrete, and categorical data often found in applied settings. Data-driven methods of bandwidth selection are emphasized throughout, though we caution the user that data-driven bandwidth selection methods can be computationally demanding.

  20. Astronomical Methods for Nonparametric Regression

    Science.gov (United States)

    Steinhardt, Charles L.; Jermyn, Adam

    2017-01-01

    I will discuss commonly used techniques for nonparametric regression in astronomy. We find that several of them, particularly running averages and running medians, are generically biased, asymmetric between dependent and independent variables, and perform poorly in recovering the underlying function, even when errors are present only in one variable. We then examine less-commonly used techniques such as Multivariate Adaptive Regressive Splines and Boosted Trees and find them superior in bias, asymmetry, and variance both theoretically and in practice under a wide range of numerical benchmarks. In this context the chief advantage of the common techniques is runtime, which even for large datasets is now measured in microseconds compared with milliseconds for the more statistically robust techniques. This points to a tradeoff between bias, variance, and computational resources which in recent years has shifted heavily in favor of the more advanced methods, primarily driven by Moore's Law. Along these lines, we also propose a new algorithm which has better overall statistical properties than all techniques examined thus far, at the cost of significantly worse runtime, in addition to providing guidance on choosing the nonparametric regression technique most suitable to any specific problem. We then examine the more general problem of errors in both variables and provide a new algorithm which performs well in most cases and lacks the clear asymmetry of existing non-parametric methods, which fail to account for errors in both variables.

  1. Out-of-Sample Extensions for Non-Parametric Kernel Methods.

    Science.gov (United States)

    Pan, Binbin; Chen, Wen-Sheng; Chen, Bo; Xu, Chen; Lai, Jianhuang

    2017-02-01

    Choosing suitable kernels plays an important role in the performance of kernel methods. Recently, a number of studies were devoted to developing nonparametric kernels. Without assuming any parametric form of the target kernel, nonparametric kernel learning offers a flexible scheme to utilize the information of the data, which may potentially characterize the data similarity better. The kernel methods using nonparametric kernels are referred to as nonparametric kernel methods. However, many nonparametric kernel methods are restricted to transductive learning, where the prediction function is defined only over the data points given beforehand. They have no straightforward extension for the out-of-sample data points, and thus cannot be applied to inductive learning. In this paper, we show how to make the nonparametric kernel methods applicable to inductive learning. The key problem of out-of-sample extension is how to extend the nonparametric kernel matrix to the corresponding kernel function. A regression approach in the hyper reproducing kernel Hilbert space is proposed to solve this problem. Empirical results indicate that the out-of-sample performance is comparable to the in-sample performance in most cases. Experiments on face recognition demonstrate the superiority of our nonparametric kernel method over the state-of-the-art parametric kernel methods.

  2. Nonparametric Independence Screening in Sparse Ultra-High Dimensional Varying Coefficient Models.

    Science.gov (United States)

    Fan, Jianqing; Ma, Yunbei; Dai, Wei

    2014-01-01

    The varying-coefficient model is an important class of nonparametric statistical model that allows us to examine how the effects of covariates vary with exposure variables. When the number of covariates is large, the issue of variable selection arises. In this paper, we propose and investigate marginal nonparametric screening methods to screen variables in sparse ultra-high dimensional varying-coefficient models. The proposed nonparametric independence screening (NIS) selects variables by ranking a measure of the nonparametric marginal contributions of each covariate given the exposure variable. The sure independent screening property is established under some mild technical conditions when the dimensionality is of nonpolynomial order, and the dimensionality reduction of NIS is quantified. To enhance the practical utility and finite sample performance, two data-driven iterative NIS methods are proposed for selecting thresholding parameters and variables: conditional permutation and greedy methods, resulting in Conditional-INIS and Greedy-INIS. The effectiveness and flexibility of the proposed methods are further illustrated by simulation studies and real data applications.

  3. Deriving covariant holographic entanglement

    Science.gov (United States)

    Dong, Xi; Lewkowycz, Aitor; Rangamani, Mukund

    2016-11-01

    We provide a gravitational argument in favour of the covariant holographic entanglement entropy proposal. In general time-dependent states, the proposal asserts that the entanglement entropy of a region in the boundary field theory is given by a quarter of the area of a bulk extremal surface in Planck units. The main element of our discussion is an implementation of an appropriate Schwinger-Keldysh contour to obtain the reduced density matrix (and its powers) of a given region, as is relevant for the replica construction. We map this contour into the bulk gravitational theory, and argue that the saddle point solutions of these replica geometries lead to a consistent prescription for computing the field theory Rényi entropies. In the limiting case where the replica index is taken to unity, a local analysis suffices to show that these saddles lead to the extremal surfaces of interest. We also comment on various properties of holographic entanglement that follow from this construction.

  4. Deriving covariant holographic entanglement

    CERN Document Server

    Dong, Xi; Rangamani, Mukund

    2016-01-01

    We provide a gravitational argument in favour of the covariant holographic entanglement entropy proposal. In general time-dependent states, the proposal asserts that the entanglement entropy of a region in the boundary field theory is given by a quarter of the area of a bulk extremal surface in Planck units. The main element of our discussion is an implementation of an appropriate Schwinger-Keldysh contour to obtain the reduced density matrix (and its powers) of a given region, as is relevant for the replica construction. We map this contour into the bulk gravitational theory, and argue that the saddle point solutions of these replica geometries lead to a consistent prescription for computing the field theory Renyi entropies. In the limiting case where the replica index is taken to unity, a local analysis suffices to show that these saddles lead to the extremal surfaces of interest. We also comment on various properties of holographic entanglement that follow from this construction.

  5. Multivariate covariance generalized linear models

    DEFF Research Database (Denmark)

    Bonat, W. H.; Jørgensen, Bent

    2016-01-01

    We propose a general framework for non-normal multivariate data analysis called multivariate covariance generalized linear models, designed to handle multivariate response variables, along with a wide range of temporal and spatial correlation structures defined in terms of a covariance link...... function combined with a matrix linear predictor involving known matrices. The method is motivated by three data examples that are not easily handled by existing methods. The first example concerns multivariate count data, the second involves response variables of mixed types, combined with repeated...... are fitted by using an efficient Newton scoring algorithm based on quasi-likelihood and Pearson estimating functions, using only second-moment assumptions. This provides a unified approach to a wide variety of types of response variables and covariance structures, including multivariate extensions...

  6. Nonparametric regression with filtered data

    CERN Document Server

    Linton, Oliver; Nielsen, Jens Perch; Van Keilegom, Ingrid; 10.3150/10-BEJ260

    2011-01-01

    We present a general principle for estimating a regression function nonparametrically, allowing for a wide variety of data filtering, for example, repeated left truncation and right censoring. Both the mean and the median regression cases are considered. The method works by first estimating the conditional hazard function or conditional survivor function and then integrating. We also investigate improved methods that take account of model structure such as independent errors and show that such methods can improve performance when the model structure is true. We establish the pointwise asymptotic normality of our estimators.

  7. Nonparametric identification of copula structures

    KAUST Repository

    Li, Bo

    2013-06-01

    We propose a unified framework for testing a variety of assumptions commonly made about the structure of copulas, including symmetry, radial symmetry, joint symmetry, associativity and Archimedeanity, and max-stability. Our test is nonparametric and based on the asymptotic distribution of the empirical copula process.We perform simulation experiments to evaluate our test and conclude that our method is reliable and powerful for assessing common assumptions on the structure of copulas, particularly when the sample size is moderately large. We illustrate our testing approach on two datasets. © 2013 American Statistical Association.

  8. Condition Number Regularized Covariance Estimation.

    Science.gov (United States)

    Won, Joong-Ho; Lim, Johan; Kim, Seung-Jean; Rajaratnam, Bala

    2013-06-01

    Estimation of high-dimensional covariance matrices is known to be a difficult problem, has many applications, and is of current interest to the larger statistics community. In many applications including so-called the "large p small n" setting, the estimate of the covariance matrix is required to be not only invertible, but also well-conditioned. Although many regularization schemes attempt to do this, none of them address the ill-conditioning problem directly. In this paper, we propose a maximum likelihood approach, with the direct goal of obtaining a well-conditioned estimator. No sparsity assumption on either the covariance matrix or its inverse are are imposed, thus making our procedure more widely applicable. We demonstrate that the proposed regularization scheme is computationally efficient, yields a type of Steinian shrinkage estimator, and has a natural Bayesian interpretation. We investigate the theoretical properties of the regularized covariance estimator comprehensively, including its regularization path, and proceed to develop an approach that adaptively determines the level of regularization that is required. Finally, we demonstrate the performance of the regularized estimator in decision-theoretic comparisons and in the financial portfolio optimization setting. The proposed approach has desirable properties, and can serve as a competitive procedure, especially when the sample size is small and when a well-conditioned estimator is required.

  9. Condition Number Regularized Covariance Estimation*

    Science.gov (United States)

    Won, Joong-Ho; Lim, Johan; Kim, Seung-Jean; Rajaratnam, Bala

    2012-01-01

    Estimation of high-dimensional covariance matrices is known to be a difficult problem, has many applications, and is of current interest to the larger statistics community. In many applications including so-called the “large p small n” setting, the estimate of the covariance matrix is required to be not only invertible, but also well-conditioned. Although many regularization schemes attempt to do this, none of them address the ill-conditioning problem directly. In this paper, we propose a maximum likelihood approach, with the direct goal of obtaining a well-conditioned estimator. No sparsity assumption on either the covariance matrix or its inverse are are imposed, thus making our procedure more widely applicable. We demonstrate that the proposed regularization scheme is computationally efficient, yields a type of Steinian shrinkage estimator, and has a natural Bayesian interpretation. We investigate the theoretical properties of the regularized covariance estimator comprehensively, including its regularization path, and proceed to develop an approach that adaptively determines the level of regularization that is required. Finally, we demonstrate the performance of the regularized estimator in decision-theoretic comparisons and in the financial portfolio optimization setting. The proposed approach has desirable properties, and can serve as a competitive procedure, especially when the sample size is small and when a well-conditioned estimator is required. PMID:23730197

  10. Multiatlas segmentation as nonparametric regression.

    Science.gov (United States)

    Awate, Suyash P; Whitaker, Ross T

    2014-09-01

    This paper proposes a novel theoretical framework to model and analyze the statistical characteristics of a wide range of segmentation methods that incorporate a database of label maps or atlases; such methods are termed as label fusion or multiatlas segmentation. We model these multiatlas segmentation problems as nonparametric regression problems in the high-dimensional space of image patches. We analyze the nonparametric estimator's convergence behavior that characterizes expected segmentation error as a function of the size of the multiatlas database. We show that this error has an analytic form involving several parameters that are fundamental to the specific segmentation problem (determined by the chosen anatomical structure, imaging modality, registration algorithm, and label-fusion algorithm). We describe how to estimate these parameters and show that several human anatomical structures exhibit the trends modeled analytically. We use these parameter estimates to optimize the regression estimator. We show that the expected error for large database sizes is well predicted by models learned on small databases. Thus, a few expert segmentations can help predict the database sizes required to keep the expected error below a specified tolerance level. Such cost-benefit analysis is crucial for deploying clinical multiatlas segmentation systems.

  11. A contingency table approach to nonparametric testing

    CERN Document Server

    Rayner, JCW

    2000-01-01

    Most texts on nonparametric techniques concentrate on location and linear-linear (correlation) tests, with less emphasis on dispersion effects and linear-quadratic tests. Tests for higher moment effects are virtually ignored. Using a fresh approach, A Contingency Table Approach to Nonparametric Testing unifies and extends the popular, standard tests by linking them to tests based on models for data that can be presented in contingency tables.This approach unifies popular nonparametric statistical inference and makes the traditional, most commonly performed nonparametric analyses much more comp

  12. Nonparametric statistics for social and behavioral sciences

    CERN Document Server

    Kraska-MIller, M

    2013-01-01

    Introduction to Research in Social and Behavioral SciencesBasic Principles of ResearchPlanning for ResearchTypes of Research Designs Sampling ProceduresValidity and Reliability of Measurement InstrumentsSteps of the Research Process Introduction to Nonparametric StatisticsData AnalysisOverview of Nonparametric Statistics and Parametric Statistics Overview of Parametric Statistics Overview of Nonparametric StatisticsImportance of Nonparametric MethodsMeasurement InstrumentsAnalysis of Data to Determine Association and Agreement Pearson Chi-Square Test of Association and IndependenceContingency

  13. Covariant Formulations of Superstring Theories.

    Science.gov (United States)

    Mikovic, Aleksandar Radomir

    1990-01-01

    Chapter 1 contains a brief introduction to the subject of string theory, and tries to motivate the study of superstrings and covariant formulations. Chapter 2 describes the Green-Schwarz formulation of the superstrings. The Hamiltonian and BRST structure of the theory is analysed in the case of the superparticle. Implications for the superstring case are discussed. Chapter 3 describes the Siegel's formulation of the superstring, which contains only the first class constraints. It is shown that the physical spectrum coincides with that of the Green-Schwarz formulation. In chapter 4 we analyse the BRST structure of the Siegel's formulation. We show that the BRST charge has the wrong cohomology, and propose a modification, called first ilk, which gives the right cohomology. We also propose another superparticle model, called second ilk, which has infinitely many coordinates and constraints. We construct the complete BRST charge for it, and show that it gives the correct cohomology. In chapter 5 we analyse the properties of the covariant vertex operators and the corresponding S-matrix elements by using the Siegel's formulation. We conclude that the knowledge of the ghosts is necessary, even at the tree level, in order to obtain the correct S-matrix. In chapter 6 we attempt to calculate the superstring loops, in a covariant gauge. We calculate the vacuum-to -vacuum amplitude, which is also the cosmological constant. We show that it vanishes to all loop orders, under the assumption that the free covariant gauge-fixed action exists. In chapter 7 we present our conclusions, and briefly discuss the random lattice approach to the string theory, as a possible way of resolving the problem of the covariant quantization and the nonperturbative definition of the superstrings.

  14. Nonparametric Bayesian inference in biostatistics

    CERN Document Server

    Müller, Peter

    2015-01-01

    As chapters in this book demonstrate, BNP has important uses in clinical sciences and inference for issues like unknown partitions in genomics. Nonparametric Bayesian approaches (BNP) play an ever expanding role in biostatistical inference from use in proteomics to clinical trials. Many research problems involve an abundance of data and require flexible and complex probability models beyond the traditional parametric approaches. As this book's expert contributors show, BNP approaches can be the answer. Survival Analysis, in particular survival regression, has traditionally used BNP, but BNP's potential is now very broad. This applies to important tasks like arrangement of patients into clinically meaningful subpopulations and segmenting the genome into functionally distinct regions. This book is designed to both review and introduce application areas for BNP. While existing books provide theoretical foundations, this book connects theory to practice through engaging examples and research questions. Chapters c...

  15. Nonparametric Regression with Common Shocks

    Directory of Open Access Journals (Sweden)

    Eduardo A. Souza-Rodrigues

    2016-09-01

    Full Text Available This paper considers a nonparametric regression model for cross-sectional data in the presence of common shocks. Common shocks are allowed to be very general in nature; they do not need to be finite dimensional with a known (small number of factors. I investigate the properties of the Nadaraya-Watson kernel estimator and determine how general the common shocks can be while still obtaining meaningful kernel estimates. Restrictions on the common shocks are necessary because kernel estimators typically manipulate conditional densities, and conditional densities do not necessarily exist in the present case. By appealing to disintegration theory, I provide sufficient conditions for the existence of such conditional densities and show that the estimator converges in probability to the Kolmogorov conditional expectation given the sigma-field generated by the common shocks. I also establish the rate of convergence and the asymptotic distribution of the kernel estimator.

  16. Nonparametric Bayesian Modeling of Complex Networks

    DEFF Research Database (Denmark)

    Schmidt, Mikkel Nørgaard; Mørup, Morten

    2013-01-01

    Modeling structure in complex networks using Bayesian nonparametrics makes it possible to specify flexible model structures and infer the adequate model complexity from the observed data. This article provides a gentle introduction to nonparametric Bayesian modeling of complex networks: Using...... for complex networks can be derived and point out relevant literature....

  17. An asymptotically optimal nonparametric adaptive controller

    Institute of Scientific and Technical Information of China (English)

    郭雷; 谢亮亮

    2000-01-01

    For discrete-time nonlinear stochastic systems with unknown nonparametric structure, a kernel estimation-based nonparametric adaptive controller is constructed based on truncated certainty equivalence principle. Global stability and asymptotic optimality of the closed-loop systems are established without resorting to any external excitations.

  18. Covariance NMR spectroscopy by singular value decomposition.

    Science.gov (United States)

    Trbovic, Nikola; Smirnov, Serge; Zhang, Fengli; Brüschweiler, Rafael

    2004-12-01

    Covariance NMR is demonstrated for homonuclear 2D NMR data collected using the hypercomplex and TPPI methods. Absorption mode 2D spectra are obtained by application of the square-root operation to the covariance matrices. The resulting spectra closely resemble the 2D Fourier transformation spectra, except that they are fully symmetric with the spectral resolution along both dimensions determined by the favorable resolution achievable along omega2. An efficient method is introduced for the calculation of the square root of the covariance spectrum by applying a singular value decomposition (SVD) directly to the mixed time-frequency domain data matrix. Applications are shown for 2D NOESY and 2QF-COSY data sets and computational benchmarks are given for data matrix dimensions typically encountered in practice. The SVD implementation makes covariance NMR amenable to routine applications.

  19. Manifestly covariant electromagnetism

    Energy Technology Data Exchange (ETDEWEB)

    Hillion, P. [Institut Henri Poincare' , Le Vesinet (France)

    1999-03-01

    The conventional relativistic formulation of electromagnetism is covariant under the full Lorentz group. But relativity requires covariance only under the proper Lorentz group and the authors present here the formalism covariant under the complex rotation group isomorphic to the proper Lorentz group. The authors discuss successively Maxwell's equations, constitutive relations and potential functions. A comparison is made with the usual formulation.

  20. 修正协方差阵的投资组合方案及其稳定性%Optimal portfolio project with modified covariance matrix and its stability

    Institute of Scientific and Technical Information of China (English)

    储晨; 方兆本

    2011-01-01

    The reasons why the risk of optimal portfolio is unstable was analyzed.Based on this analysis a new method was tried,which was proposed by the previous researchers,to dispose the sample covariance matrix based on random matrix theory (RMT),to get a better result.Generally,we first give more sufficient and practical evidence to support the method's reasonableness and power,then apply this idea to simulate the “real investment”,using real data of the Chinese stock market.It is found that the results have better mean return and stability than those of the traditional method,and can be explained by the analysis of the prediction to the risk proposed by the previous researchers.%分析了最优化投资组合不稳定的原因,在此基础上,推广了由前人提出的一个新方法,即根据随机矩阵定理(RMT)对样本协差阵进行变化以得到一个更好的投资方案.首先对该方法中未证明的部分给出了更充分的实证理由,并运用这种思想,结合中国证券市场的实际数据,来模拟“真实的投资”.而这种方法得到的投资方案,较之传统方法有较好的收益率和方差,且可被前人对风险预测的分析方法所解释.

  1. 任意收益率分布和奇导协方差矩阵下的均值-风险模型研究%The Mean-Risk Model under any Asset Return Distribution and Singular Covariance Matrix

    Institute of Scientific and Technical Information of China (English)

    姚海祥; 马庆华

    2011-01-01

    本文在一般均值-风险模型的框架下,在无套利假设下研究了奇异协方差矩阵和任意收益率分布情形下的投资组合问题,得到了模型有效边界的本质特征,并给出了极大线性无关组的确定方法及表示系数的求解方法,最后根据这些结论提出了有效的、操作性强的投资策略.%Under general mean-risk framework and arbitrage-free market hypothesis, this paper study the portfolio selection problem with any asset return distribution and singular covariance matrix. First we obtain the characteristic of the effcient frontier of the mean-risk model, further propose a method for obtaining the maximal linearly independent group. Finally, as an application of our results, we present some investment strategies.

  2. Forecasting Covariance Matrices: A Mixed Frequency Approach

    DEFF Research Database (Denmark)

    Halbleib, Roxana; Voev, Valeri

    This paper proposes a new method for forecasting covariance matrices of financial returns. The model mixes volatility forecasts from a dynamic model of daily realized volatilities estimated with high-frequency data with correlation forecasts based on daily data. This new approach allows...... for flexible dependence patterns for volatilities and correlations, and can be applied to covariance matrices of large dimensions. The separate modeling of volatility and correlation forecasts considerably reduces the estimation and measurement error implied by the joint estimation and modeling of covariance...... matrix dynamics. Our empirical results show that the new mixing approach provides superior forecasts compared to multivariate volatility specifications using single sources of information....

  3. Representations of Inverse Covariances by Differential Operators

    Institute of Scientific and Technical Information of China (English)

    Qin XU

    2005-01-01

    In the cost function of three- or four-dimensional variational data assimilation, each term is weighted by the inverse of its associated error covariance matrix and the background error covariance matrix is usually much larger than the other covariance matrices. Although the background error covariances are traditionally normalized and parameterized by simple smooth homogeneous correlation functions, the covariance matrices constructed from these correlation functions are often too large to be inverted or even manipulated. It is thus desirable to find direct representations of the inverses of background errorcorrelations. This problem is studied in this paper. In particular, it is shown that the background term can be written into ∫ dx|Dv(x)|2, that is, a squared L2 norm of a vector differential operator D, called the D-operator, applied to the field of analysis increment v(x). For autoregressive correlation functions, the Doperators are of finite orders. For Gaussian correlation functions, the D-operators are of infinite order. For practical applications, the Gaussian D-operators must be truncated to finite orders. The truncation errors are found to be small even when the Gaussian D-operators are truncated to low orders. With a truncated D-operator, the background term can be easily constructed with neither inversion nor direct calculation of the covariance matrix. D-operators are also derived for non-Gaussian correlations and transformed into non-isotropic forms.

  4. Nonparametric estimation of Fisher information from real data

    Science.gov (United States)

    Har-Shemesh, Omri; Quax, Rick; Miñano, Borja; Hoekstra, Alfons G.; Sloot, Peter M. A.

    2016-02-01

    The Fisher information matrix (FIM) is a widely used measure for applications including statistical inference, information geometry, experiment design, and the study of criticality in biological systems. The FIM is defined for a parametric family of probability distributions and its estimation from data follows one of two paths: either the distribution is assumed to be known and the parameters are estimated from the data or the parameters are known and the distribution is estimated from the data. We consider the latter case which is applicable, for example, to experiments where the parameters are controlled by the experimenter and a complicated relation exists between the input parameters and the resulting distribution of the data. Since we assume that the distribution is unknown, we use a nonparametric density estimation on the data and then compute the FIM directly from that estimate using a finite-difference approximation to estimate the derivatives in its definition. The accuracy of the estimate depends on both the method of nonparametric estimation and the difference Δ θ between the densities used in the finite-difference formula. We develop an approach for choosing the optimal parameter difference Δ θ based on large deviations theory and compare two nonparametric density estimation methods, the Gaussian kernel density estimator and a novel density estimation using field theory method. We also compare these two methods to a recently published approach that circumvents the need for density estimation by estimating a nonparametric f divergence and using it to approximate the FIM. We use the Fisher information of the normal distribution to validate our method and as a more involved example we compute the temperature component of the FIM in the two-dimensional Ising model and show that it obeys the expected relation to the heat capacity and therefore peaks at the phase transition at the correct critical temperature.

  5. Performance evaluation of sensor allocation algorithm based on covariance control

    Institute of Scientific and Technical Information of China (English)

    2005-01-01

    The covariance control capability of sensor allocation algorithms based on covariance control strategy is an important index to evaluate the performance of these algorithms. Owing to lack of standard performance metric indices to evaluate covariance control capability, sensor allocation ratio, etc, there are no guides to follow in the design procedure of sensor allocation algorithm in practical applications. To meet these demands, three quantified performance metric indices are presented, which are average covariance misadjustment quantity (ACMQ), average sensor allocation ratio (ASAR) and matrix metric influence factor (MMIF), where ACMQ, ASAR and MMIF quantify the covariance control capability, the usage of sensor resources and the robustness of sensor allocation algorithm, respectively. Meanwhile, a covariance adaptive sensor allocation algorithm based on a new objective function is proposed to improve the covariance control capability of the algorithm based on information gain. The experiment results show that the proposed algorithm have the advantage over the preceding sensor allocation algorithm in covariance control capability and robustness.

  6. Kriging approach for the experimental cross-section covariances estimation

    Directory of Open Access Journals (Sweden)

    Garlaud A.

    2013-03-01

    Full Text Available In the classical use of a generalized χ2 to determine the evaluated cross section uncertainty, we need the covariance matrix of the experimental cross sections. The usual propagation error method to estimate the covariances is hardly usable and the lack of data prevents from using the direct empirical estimator. We propose in this paper to apply the kriging method which allows to estimate the covariances via the distances between the points and with some assumptions on the covariance matrix structure. All the results are illustrated with the 2555Mn nucleus measurements.

  7. Parametric and Non-Parametric System Modelling

    DEFF Research Database (Denmark)

    Nielsen, Henrik Aalborg

    1999-01-01

    considered. It is shown that adaptive estimation in conditional parametric models can be performed by combining the well known methods of local polynomial regression and recursive least squares with exponential forgetting. The approach used for estimation in conditional parametric models also highlights how....... For this purpose non-parametric methods together with additive models are suggested. Also, a new approach specifically designed to detect non-linearities is introduced. Confidence intervals are constructed by use of bootstrapping. As a link between non-parametric and parametric methods a paper dealing with neural...... the focus is on combinations of parametric and non-parametric methods of regression. This combination can be in terms of additive models where e.g. one or more non-parametric term is added to a linear regression model. It can also be in terms of conditional parametric models where the coefficients...

  8. Bayesian nonparametric duration model with censorship

    Directory of Open Access Journals (Sweden)

    Joseph Hakizamungu

    2007-10-01

    Full Text Available This paper is concerned with nonparametric i.i.d. durations models censored observations and we establish by a simple and unified approach the general structure of a bayesian nonparametric estimator for a survival function S. For Dirichlet prior distributions, we describe completely the structure of the posterior distribution of the survival function. These results are essentially supported by prior and posterior independence properties.

  9. Bootstrap Estimation for Nonparametric Efficiency Estimates

    OpenAIRE

    1995-01-01

    This paper develops a consistent bootstrap estimation procedure to obtain confidence intervals for nonparametric measures of productive efficiency. Although the methodology is illustrated in terms of technical efficiency measured by output distance functions, the technique can be easily extended to other consistent nonparametric frontier models. Variation in estimated efficiency scores is assumed to result from variation in empirical approximations to the true boundary of the production set. ...

  10. Covariant Hamiltonian field theory

    CERN Document Server

    Giachetta, G; Sardanashvily, G

    1999-01-01

    We study the relationship between the equations of first order Lagrangian field theory on fiber bundles and the covariant Hamilton equations on the finite-dimensional polysymplectic phase space of covariant Hamiltonian field theory. The main peculiarity of these Hamilton equations lies in the fact that, for degenerate systems, they contain additional gauge fixing conditions. We develop the BRST extension of the covariant Hamiltonian formalism, characterized by a Lie superalgebra of BRST and anti-BRST symmetries.

  11. Stability of covariance matrix for generalized transmission control protocol flow control equations%推广形式传输控制协议流量控制方程的方差稳定性

    Institute of Scientific and Technical Information of China (English)

    樊华; 山秀明; 任勇; 袁坚

    2011-01-01

    neglects the random variations in the controlled TCP flows. This paper aims at revealing the un-negligible influence from such random variations to the system stability. Using TCP/RED(TCP flows with random early detection) as an example, based on the stochastic differential equations of the system, the system is converted into a multi-dimensional linear time-invariant system with mixed additive and multiplicative noises by linearization at the equilibrium point. Then, generalized TCP flow control equations for continuous-time and discrete-time cases respectively are given, which are the first-degree time-invariant stochastic differential or difference equations with multi-noise-inputs. After that, the covariance matrix equation for such generalized system is derived; and based on this matrix equation, the sufficient and necessary condition when the covariance matrix has an asymptotically stable limit is presented, together with the expression of this limit. In engineering design, this condition can be regarded as a substitute criterion for estimating the motion domain. Finally, this general condition is applied to a specific example for demonstrating the change of the stability range when the stability of covariance is considered. Moreover, the results in this paper can be extended to the nonlinear system or time-varying system when treated by similar methods used in deterministic cases.

  12. Group Lasso estimation of high-dimensional covariance matrices

    CERN Document Server

    Bigot, Jérémie; Loubes, Jean-Michel; Alvarez, Lilian Muniz

    2010-01-01

    In this paper, we consider the Group Lasso estimator of the covariance matrix of a stochastic process corrupted by an additive noise. We propose to estimate the covariance matrix in a high-dimensional setting under the assumption that the process has a sparse representation in a large dictionary of basis functions. Using a matrix regression model, we propose a new methodology for high-dimensional covariance matrix estimation based on empirical contrast regularization by a group Lasso penalty. Using such a penalty, the method selects a sparse set of basis functions in the dictionary used to approximate the process, leading to an approximation of the covariance matrix into a low dimensional space. Consistency of the estimator is studied in Frobenius and operator norms and an application to sparse PCA is proposed.

  13. Covariant w∞ gravity

    NARCIS (Netherlands)

    Bergshoeff, E.; Pope, C.N.; Stelle, K.S.

    1990-01-01

    We discuss the notion of higher-spin covariance in w∞ gravity. We show how a recently proposed covariant w∞ gravity action can be obtained from non-chiral w∞ gravity by making field redefinitions that introduce new gauge-field components with corresponding new gauge transformations.

  14. Gráfico de controle MCMAX para o monitoramento simultâneo do vetor de médias e da matriz de covariâncias MCMAX control chart for the joint monitoring of the mean vector and the covariance matrix

    Directory of Open Access Journals (Sweden)

    Antônio Fernando Branco Costa

    2010-01-01

    Full Text Available O gráfico T² de Hotelling e o gráfico |S| da variância generalizada são utilizados para monitorar o vetor de médias e a matriz de covariâncias de processos multivariados. Neste artigo, propõe-se o uso de um único gráfico de controle para o monitoramento de processos bivariados, isto é, o gráfico de controle MCMAX cujo valor da estatística de monitoramento corresponde ao maior valor em módulo de quatro medidas amostrais das duas características de qualidade sob monitoramento, isto é, as suas médias e variâncias padronizadas. O usuário de gráficos de controle já está bem familiarizado com médias e variâncias amostrais; o mesmo não pode ser dito a respeito da estatística de Hotelling ou da variância generalizada. Conseqüentemente, ele preferirá usar o gráfico de controle proposto ao invés dos gráficos conjuntos de T² e |S|. Além disso, o usuário, em geral, se sente mais seguro em intervir no processo somente após a ocorrência de um segundo ponto na região de ação do gráfico. Se o sinal for dado por dois pontos, não necessariamente vizinhos, porém próximos e na região de ação, o gráfico proposto terá um desempenho geral superior ao dos gráficos conjuntos de T² e |S| na detecção de desajustes do processo, exceto quando a correlação entre as duas características de qualidade for muito alta. Quando a correlação é muito alta e a causa especial desloca a média e/ou aumenta a variância de apenas uma das variáveis, X ou Y, os gráficos de T² e |S| são, em geral, mais ágeis do que gráfico MCMAX.The control chart and the generalized variance |S| chart are used for monitoring the mean vector and the covariance matrix of multivariate processes. In this article, we propose the use of a single chart for monitoring bivariate processes, that is, the MCMAX control chart based on a new statistic, which corresponds to the maximum among four sample values: the standardized sample means (in module and

  15. Why preferring parametric forecasting to nonparametric methods?

    Science.gov (United States)

    Jabot, Franck

    2015-05-07

    A recent series of papers by Charles T. Perretti and collaborators have shown that nonparametric forecasting methods can outperform parametric methods in noisy nonlinear systems. Such a situation can arise because of two main reasons: the instability of parametric inference procedures in chaotic systems which can lead to biased parameter estimates, and the discrepancy between the real system dynamics and the modeled one, a problem that Perretti and collaborators call "the true model myth". Should ecologists go on using the demanding parametric machinery when trying to forecast the dynamics of complex ecosystems? Or should they rely on the elegant nonparametric approach that appears so promising? It will be here argued that ecological forecasting based on parametric models presents two key comparative advantages over nonparametric approaches. First, the likelihood of parametric forecasting failure can be diagnosed thanks to simple Bayesian model checking procedures. Second, when parametric forecasting is diagnosed to be reliable, forecasting uncertainty can be estimated on virtual data generated with the fitted to data parametric model. In contrast, nonparametric techniques provide forecasts with unknown reliability. This argumentation is illustrated with the simple theta-logistic model that was previously used by Perretti and collaborators to make their point. It should convince ecologists to stick to standard parametric approaches, until methods have been developed to assess the reliability of nonparametric forecasting. Copyright © 2015 Elsevier Ltd. All rights reserved.

  16. Progress on Nuclear Data Covariances: AFCI-1.2 Covariance Library

    Energy Technology Data Exchange (ETDEWEB)

    Oblozinsky,P.; Oblozinsky,P.; Mattoon,C.M.; Herman,M.; Mughabghab,S.F.; Pigni,M.T.; Talou,P.; Hale,G.M.; Kahler,A.C.; Kawano,T.; Little,R.C.; Young,P.G

    2009-09-28

    Improved neutron cross section covariances were produced for 110 materials including 12 light nuclei (coolants and moderators), 78 structural materials and fission products, and 20 actinides. Improved covariances were organized into AFCI-1.2 covariance library in 33-energy groups, from 10{sup -5} eV to 19.6 MeV. BNL contributed improved covariance data for the following materials: {sup 23}Na and {sup 55}Mn where more detailed evaluation was done; improvements in major structural materials {sup 52}Cr, {sup 56}Fe and {sup 58}Ni; improved estimates for remaining structural materials and fission products; improved covariances for 14 minor actinides, and estimates of mubar covariances for {sup 23}Na and {sup 56}Fe. LANL contributed improved covariance data for {sup 235}U and {sup 239}Pu including prompt neutron fission spectra and completely new evaluation for {sup 240}Pu. New R-matrix evaluation for {sup 16}O including mubar covariances is under completion. BNL assembled the library and performed basic testing using improved procedures including inspection of uncertainty and correlation plots for each material. The AFCI-1.2 library was released to ANL and INL in August 2009.

  17. Accurate covariance estimation of galaxy-galaxy weak lensing: limitations of jackknife covariance

    CERN Document Server

    Shirasaki, Masato; Miyatake, Hironao; Takahashi, Ryuichi; Hamana, Takashi; Nishimichi, Takahiro; Murata, Ryoma

    2016-01-01

    We develop a method to simulate galaxy-galaxy weak lensing by utilizing all-sky, light-cone simulations. We populate a real catalog of source galaxies into a light-cone simulation realization, simulate the lensing effect on each galaxy, and then identify lensing halos that are considered to host galaxies or clusters of interest. We use the mock catalog to study the error covariance matrix of galaxy-galaxy weak lensing and find that the super-sample covariance (SSC), which arises from density fluctuations with length scales comparable with or greater than a size of survey area, gives a dominant source of the sample variance. We then compare the full covariance with the jackknife (JK) covariance, the method that estimates the covariance from the resamples of the data itself. We show that, although the JK method gives an unbiased estimator of the covariance in the shot noise or Gaussian regime, it always over-estimates the true covariance in the sample variance regime, because the JK covariance turns out to be a...

  18. Covariance fitting of highly correlated $B_K$ data

    CERN Document Server

    Yoon, Boram; Jung, Chulwoo; Lee, Weonjong

    2011-01-01

    We present the reason why we use the diagonal approximation (uncorrelated fitting) when we perform the data analysis of highly correlated $B_K$ data on the basis of the SU(2) staggered chiral perturbation theory. Basically, the essence of the problem is that we do not have enough statistics to determine the small eigenvalues of the covariance matrix with a high precision. As a result, we have the smallest eigenvalue, which is smaller than the statistical error of the covariance matrix, corresponding to an unphysical eigenmode. We have applied a number of prescriptions available in the market such as the cutoff method and modified covariance matrix method. It turns out that the cutoff method is not a good prescription and the modified covariance matrix method is an even worse one. The diagonal approximation turns out to be a good prescription if the data points are somehow correlated and the statistics are relatively poor.

  19. Wishart distributions for decomposable covariance graph models

    CERN Document Server

    Khare, Kshitij; 10.1214/10-AOS841

    2011-01-01

    Gaussian covariance graph models encode marginal independence among the components of a multivariate random vector by means of a graph $G$. These models are distinctly different from the traditional concentration graph models (often also referred to as Gaussian graphical models or covariance selection models) since the zeros in the parameter are now reflected in the covariance matrix $\\Sigma$, as compared to the concentration matrix $\\Omega =\\Sigma^{-1}$. The parameter space of interest for covariance graph models is the cone $P_G$ of positive definite matrices with fixed zeros corresponding to the missing edges of $G$. As in Letac and Massam [Ann. Statist. 35 (2007) 1278--1323], we consider the case where $G$ is decomposable. In this paper, we construct on the cone $P_G$ a family of Wishart distributions which serve a similar purpose in the covariance graph setting as those constructed by Letac and Massam [Ann. Statist. 35 (2007) 1278--1323] and Dawid and Lauritzen [Ann. Statist. 21 (1993) 1272--1317] do in ...

  20. Recent Advances and Trends in Nonparametric Statistics

    CERN Document Server

    Akritas, MG

    2003-01-01

    The advent of high-speed, affordable computers in the last two decades has given a new boost to the nonparametric way of thinking. Classical nonparametric procedures, such as function smoothing, suddenly lost their abstract flavour as they became practically implementable. In addition, many previously unthinkable possibilities became mainstream; prime examples include the bootstrap and resampling methods, wavelets and nonlinear smoothers, graphical methods, data mining, bioinformatics, as well as the more recent algorithmic approaches such as bagging and boosting. This volume is a collection o

  1. Correlated Non-Parametric Latent Feature Models

    CERN Document Server

    Doshi-Velez, Finale

    2012-01-01

    We are often interested in explaining data through a set of hidden factors or features. When the number of hidden features is unknown, the Indian Buffet Process (IBP) is a nonparametric latent feature model that does not bound the number of active features in dataset. However, the IBP assumes that all latent features are uncorrelated, making it inadequate for many realworld problems. We introduce a framework for correlated nonparametric feature models, generalising the IBP. We use this framework to generate several specific models and demonstrate applications on realworld datasets.

  2. A Censored Nonparametric Software Reliability Model

    Institute of Scientific and Technical Information of China (English)

    2006-01-01

    This paper analyses the effct of censoring on the estimation of failure rate, and presents a framework of a censored nonparametric software reliability model. The model is based on nonparametric testing of failure rate monotonically decreasing and weighted kernel failure rate estimation under the constraint of failure rate monotonically decreasing. Not only does the model have the advantages of little assumptions and weak constraints, but also the residual defects number of the software system can be estimated. The numerical experiment and real data analysis show that the model performs well with censored data.

  3. Linear transformations of variance/covariance matrices.

    Science.gov (United States)

    Parois, Pascal; Lutz, Martin

    2011-07-01

    Many applications in crystallography require the use of linear transformations on parameters and their standard uncertainties. While the transformation of the parameters is textbook knowledge, the transformation of the standard uncertainties is more complicated and needs the full variance/covariance matrix. For the transformation of second-rank tensors it is suggested that the 3 × 3 matrix is re-written into a 9 × 1 vector. The transformation of the corresponding variance/covariance matrix is then straightforward and easily implemented into computer software. This method is applied in the transformation of anisotropic displacement parameters, the calculation of equivalent isotropic displacement parameters, the comparison of refinements in different space-group settings and the calculation of standard uncertainties of eigenvalues.

  4. Covariate analysis of bivariate survival data

    Energy Technology Data Exchange (ETDEWEB)

    Bennett, L.E.

    1992-01-01

    The methods developed are used to analyze the effects of covariates on bivariate survival data when censoring and ties are present. The proposed method provides models for bivariate survival data that include differential covariate effects and censored observations. The proposed models are based on an extension of the univariate Buckley-James estimators which replace censored data points by their expected values, conditional on the censoring time and the covariates. For the bivariate situation, it is necessary to determine the expectation of the failure times for one component conditional on the failure or censoring time of the other component. Two different methods have been developed to estimate these expectations. In the semiparametric approach these expectations are determined from a modification of Burke's estimate of the bivariate empirical survival function. In the parametric approach censored data points are also replaced by their conditional expected values where the expected values are determined from a specified parametric distribution. The model estimation will be based on the revised data set, comprised of uncensored components and expected values for the censored components. The variance-covariance matrix for the estimated covariate parameters has also been derived for both the semiparametric and parametric methods. Data from the Demographic and Health Survey was analyzed by these methods. The two outcome variables are post-partum amenorrhea and breastfeeding; education and parity were used as the covariates. Both the covariate parameter estimates and the variance-covariance estimates for the semiparametric and parametric models will be compared. In addition, a multivariate test statistic was used in the semiparametric model to examine contrasts. The significance of the statistic was determined from a bootstrap distribution of the test statistic.

  5. Thirty years of nonparametric item response theory

    NARCIS (Netherlands)

    Molenaar, W.

    2001-01-01

    Relationships between a mathematical measurement model and its real-world applications are discussed. A distinction is made between large data matrices commonly found in educational measurement and smaller matrices found in attitude and personality measurement. Nonparametric methods are evaluated fo

  6. A Bayesian Nonparametric Approach to Test Equating

    Science.gov (United States)

    Karabatsos, George; Walker, Stephen G.

    2009-01-01

    A Bayesian nonparametric model is introduced for score equating. It is applicable to all major equating designs, and has advantages over previous equating models. Unlike the previous models, the Bayesian model accounts for positive dependence between distributions of scores from two tests. The Bayesian model and the previous equating models are…

  7. How Are Teachers Teaching? A Nonparametric Approach

    Science.gov (United States)

    De Witte, Kristof; Van Klaveren, Chris

    2014-01-01

    This paper examines which configuration of teaching activities maximizes student performance. For this purpose a nonparametric efficiency model is formulated that accounts for (1) self-selection of students and teachers in better schools and (2) complementary teaching activities. The analysis distinguishes both individual teaching (i.e., a…

  8. Nonparametric confidence intervals for monotone functions

    NARCIS (Netherlands)

    Groeneboom, P.; Jongbloed, G.

    2015-01-01

    We study nonparametric isotonic confidence intervals for monotone functions. In [Ann. Statist. 29 (2001) 1699–1731], pointwise confidence intervals, based on likelihood ratio tests using the restricted and unrestricted MLE in the current status model, are introduced. We extend the method to the trea

  9. Decompounding random sums: A nonparametric approach

    DEFF Research Database (Denmark)

    Hansen, Martin Bøgsted; Pitts, Susan M.

    review a number of applications and consider the nonlinear inverse problem of inferring the cumulative distribution function of the components in the random sum. We review the existing literature on non-parametric approaches to the problem. The models amenable to the analysis are generalized considerably...

  10. Nonparametric confidence intervals for monotone functions

    NARCIS (Netherlands)

    Groeneboom, P.; Jongbloed, G.

    2015-01-01

    We study nonparametric isotonic confidence intervals for monotone functions. In [Ann. Statist. 29 (2001) 1699–1731], pointwise confidence intervals, based on likelihood ratio tests using the restricted and unrestricted MLE in the current status model, are introduced. We extend the method to the

  11. Panel data specifications in nonparametric kernel regression

    DEFF Research Database (Denmark)

    Czekaj, Tomasz Gerard; Henningsen, Arne

    parametric panel data estimators to analyse the production technology of Polish crop farms. The results of our nonparametric kernel regressions generally differ from the estimates of the parametric models but they only slightly depend on the choice of the kernel functions. Based on economic reasoning, we...

  12. How Are Teachers Teaching? A Nonparametric Approach

    Science.gov (United States)

    De Witte, Kristof; Van Klaveren, Chris

    2014-01-01

    This paper examines which configuration of teaching activities maximizes student performance. For this purpose a nonparametric efficiency model is formulated that accounts for (1) self-selection of students and teachers in better schools and (2) complementary teaching activities. The analysis distinguishes both individual teaching (i.e., a…

  13. Covariance mapping techniques

    Science.gov (United States)

    Frasinski, Leszek J.

    2016-08-01

    Recent technological advances in the generation of intense femtosecond pulses have made covariance mapping an attractive analytical technique. The laser pulses available are so intense that often thousands of ionisation and Coulomb explosion events will occur within each pulse. To understand the physics of these processes the photoelectrons and photoions need to be correlated, and covariance mapping is well suited for operating at the high counting rates of these laser sources. Partial covariance is particularly useful in experiments with x-ray free electron lasers, because it is capable of suppressing pulse fluctuation effects. A variety of covariance mapping methods is described: simple, partial (single- and multi-parameter), sliced, contingent and multi-dimensional. The relationship to coincidence techniques is discussed. Covariance mapping has been used in many areas of science and technology: inner-shell excitation and Auger decay, multiphoton and multielectron ionisation, time-of-flight and angle-resolved spectrometry, infrared spectroscopy, nuclear magnetic resonance imaging, stimulated Raman scattering, directional gamma ray sensing, welding diagnostics and brain connectivity studies (connectomics). This review gives practical advice for implementing the technique and interpreting the results, including its limitations and instrumental constraints. It also summarises recent theoretical studies, highlights unsolved problems and outlines a personal view on the most promising research directions.

  14. Covariant Bardeen perturbation formalism

    Science.gov (United States)

    Vitenti, S. D. P.; Falciano, F. T.; Pinto-Neto, N.

    2014-05-01

    In a previous work we obtained a set of necessary conditions for the linear approximation in cosmology. Here we discuss the relations of this approach with the so-called covariant perturbations. It is often argued in the literature that one of the main advantages of the covariant approach to describe cosmological perturbations is that the Bardeen formalism is coordinate dependent. In this paper we will reformulate the Bardeen approach in a completely covariant manner. For that, we introduce the notion of pure and mixed tensors, which yields an adequate language to treat both perturbative approaches in a common framework. We then stress that in the referred covariant approach, one necessarily introduces an additional hypersurface choice to the problem. Using our mixed and pure tensors approach, we are able to construct a one-to-one map relating the usual gauge dependence of the Bardeen formalism with the hypersurface dependence inherent to the covariant approach. Finally, through the use of this map, we define full nonlinear tensors that at first order correspond to the three known gauge invariant variables Φ, Ψ and Ξ, which are simultaneously foliation and gauge invariant. We then stress that the use of the proposed mixed tensors allows one to construct simultaneously gauge and hypersurface invariant variables at any order.

  15. On the Estimation of Integrated Covariance Matrices of High Dimensional Diffusion Processes

    CERN Document Server

    Zheng, Xinghua

    2010-01-01

    We consider the estimation of integrated covariance matrices of high dimensional diffusion processes by using high frequency data. We start by studying the most commonly used estimator, the realized covariance matrix (RCV). We show that in the high dimensional case when the dimension p and the observation frequency n grow in the same rate, the limiting empirical spectral distribution of RCV depends on the covolatility processes not only through the underlying integrated covariance matrix Sigma, but also on how the covolatility processes vary in time. In particular, for two high dimensional diffusion processes with the same integrated covariance matrix, the empirical spectral distributions of their RCVs can be very different. Hence in terms of making inference about the spectrum of the integrated covariance matrix, the RCV is in general \\emph{not} a good proxy to rely on in the high dimensional case. We then propose an alternative estimator, the time-variation adjusted realized covariance matrix (TVARCV), for ...

  16. Covariant canonical quantization

    Energy Technology Data Exchange (ETDEWEB)

    Hippel, G.M. von [University of Regina, Department of Physics, Regina, Saskatchewan (Canada); Wohlfarth, M.N.R. [Universitaet Hamburg, Institut fuer Theoretische Physik, Hamburg (Germany)

    2006-09-15

    We present a manifestly covariant quantization procedure based on the de Donder-Weyl Hamiltonian formulation of classical field theory. This procedure agrees with conventional canonical quantization only if the parameter space is d=1 dimensional time. In d>1 quantization requires a fundamental length scale, and any bosonic field generates a spinorial wave function, leading to the purely quantum-theoretical emergence of spinors as a byproduct. We provide a probabilistic interpretation of the wave functions for the fields, and we apply the formalism to a number of simple examples. These show that covariant canonical quantization produces both the Klein-Gordon and the Dirac equation, while also predicting the existence of discrete towers of identically charged fermions with different masses. Covariant canonical quantization can thus be understood as a ''first'' or pre-quantization within the framework of conventional QFT. (orig.)

  17. Covariant canonical quantization

    CERN Document Server

    Von Hippel, G M; Hippel, Georg M. von; Wohlfarth, Mattias N.R.

    2006-01-01

    We present a manifestly covariant quantization procedure based on the de Donder-Weyl Hamiltonian formulation of classical field theory. Covariant canonical quantization agrees with conventional canonical quantization only if the parameter space is d=1 dimensional time. In d>1 quantization requires a fundamental length scale, and any bosonic field generates a spinorial wave function, leading to the purely quantum-theoretical emergence of spinors as a byproduct. We provide a probabilistic interpretation of the wave functions for the fields, and apply the formalism to a number of simple examples. These show that covariant canonical quantization produces both the Klein-Gordon and the Dirac equation, while also predicting the existence of discrete towers of identically charged fermions with different masses.

  18. Trouble shooting for covariance fitting in highly correlated data

    CERN Document Server

    Yoon, Boram; Lee, Weonjong; Jung, Chulwoo

    2011-01-01

    We report a possible solution to the trouble that the covariance fitting fails when the data is highly correlated and the covariance matrix has small eigenvalues. As an example, we choose the data analysis of highly correlated $B_K$ data on the basis of the SU(2) staggered chiral perturbation theory. Basically, the essence of the problem is that we do not have an accurate fitting function so that we cannot fit the highly correlated and precise data. When some eigenvalues of the covariance matrix are small, even a tiny error of fitting function can produce large chi-square and spoil the fitting procedure. We have applied a number of prescriptions available in the market such as diagonal approximation and cutoff method. In addition, we present a new method, the eigenmode shift method which fine-tunes the fitting function while keeping the covariance matrix untouched.

  19. On covariance structure in noisy, big data

    Science.gov (United States)

    Paffenroth, Randy C.; Nong, Ryan; Du Toit, Philip C.

    2013-09-01

    Herein we describe theory and algorithms for detecting covariance structures in large, noisy data sets. Our work uses ideas from matrix completion and robust principal component analysis to detect the presence of low-rank covariance matrices, even when the data is noisy, distorted by large corruptions, and only partially observed. In fact, the ability to handle partial observations combined with ideas from randomized algorithms for matrix decomposition enables us to produce asymptotically fast algorithms. Herein we will provide numerical demonstrations of the methods and their convergence properties. While such methods have applicability to many problems, including mathematical finance, crime analysis, and other large-scale sensor fusion problems, our inspiration arises from applying these methods in the context of cyber network intrusion detection.

  20. Cross-covariance functions for multivariate geostatistics

    KAUST Repository

    Genton, Marc G.

    2015-05-01

    Continuously indexed datasets with multiple variables have become ubiquitous in the geophysical, ecological, environmental and climate sciences, and pose substantial analysis challenges to scientists and statisticians. For many years, scientists developed models that aimed at capturing the spatial behavior for an individual process; only within the last few decades has it become commonplace to model multiple processes jointly. The key difficulty is in specifying the cross-covariance function, that is, the function responsible for the relationship between distinct variables. Indeed, these cross-covariance functions must be chosen to be consistent with marginal covariance functions in such a way that the second-order structure always yields a nonnegative definite covariance matrix. We review the main approaches to building cross-covariance models, including the linear model of coregionalization, convolution methods, the multivariate Matérn and nonstationary and space-time extensions of these among others. We additionally cover specialized constructions, including those designed for asymmetry, compact support and spherical domains, with a review of physics-constrained models. We illustrate select models on a bivariate regional climate model output example for temperature and pressure, along with a bivariate minimum and maximum temperature observational dataset; we compare models by likelihood value as well as via cross-validation co-kriging studies. The article closes with a discussion of unsolved problems. © Institute of Mathematical Statistics, 2015.

  1. Conditioning of the stationary kriging matrices for some well-known covariance models

    Energy Technology Data Exchange (ETDEWEB)

    Posa, D. (IRMA-CNR, Bari (Italy))

    1989-10-01

    In this paper, the condition number of the stationary kriging matrix is studied for some well-known covariance models. Indeed, the robustness of the kriging weights is strongly affected by this measure. Such an analysis can justify the choice of a covariance function among other admissible models which could fit a given experimental covariance equally well.

  2. Electromagnetic Radiation and Equation of Motion for Really Shaped Particle -- New Covariant Formulation

    CERN Document Server

    Klacka, J

    2001-01-01

    Relativistically covariant form of equation of motion for real particle (body) under the action of electromagnetic radiation is derived. Equation of motion in the proper frame of the particle uses the radiation pressure cross section 3 $\\times$ 3 matrix. Obtained covariant equation of motion is compared with another covariant equation of motion which was presented more than one year ago.

  3. Generalized Linear Covariance Analysis

    Science.gov (United States)

    Carpenter, James R.; Markley, F. Landis

    2014-01-01

    This talk presents a comprehensive approach to filter modeling for generalized covariance analysis of both batch least-squares and sequential estimators. We review and extend in two directions the results of prior work that allowed for partitioning of the state space into solve-for'' and consider'' parameters, accounted for differences between the formal values and the true values of the measurement noise, process noise, and textita priori solve-for and consider covariances, and explicitly partitioned the errors into subspaces containing only the influence of the measurement noise, process noise, and solve-for and consider covariances. In this work, we explicitly add sensitivity analysis to this prior work, and relax an implicit assumption that the batch estimator's epoch time occurs prior to the definitive span. We also apply the method to an integrated orbit and attitude problem, in which gyro and accelerometer errors, though not estimated, influence the orbit determination performance. We illustrate our results using two graphical presentations, which we call the variance sandpile'' and the sensitivity mosaic,'' and we compare the linear covariance results to confidence intervals associated with ensemble statistics from a Monte Carlo analysis.

  4. Nonparametric tests for pathwise properties of semimartingales

    CERN Document Server

    Cont, Rama; 10.3150/10-BEJ293

    2011-01-01

    We propose two nonparametric tests for investigating the pathwise properties of a signal modeled as the sum of a L\\'{e}vy process and a Brownian semimartingale. Using a nonparametric threshold estimator for the continuous component of the quadratic variation, we design a test for the presence of a continuous martingale component in the process and a test for establishing whether the jumps have finite or infinite variation, based on observations on a discrete-time grid. We evaluate the performance of our tests using simulations of various stochastic models and use the tests to investigate the fine structure of the DM/USD exchange rate fluctuations and SPX futures prices. In both cases, our tests reveal the presence of a non-zero Brownian component and a finite variation jump component.

  5. Nonparametric Transient Classification using Adaptive Wavelets

    CERN Document Server

    Varughese, Melvin M; Stephanou, Michael; Bassett, Bruce A

    2015-01-01

    Classifying transients based on multi band light curves is a challenging but crucial problem in the era of GAIA and LSST since the sheer volume of transients will make spectroscopic classification unfeasible. Here we present a nonparametric classifier that uses the transient's light curve measurements to predict its class given training data. It implements two novel components: the first is the use of the BAGIDIS wavelet methodology - a characterization of functional data using hierarchical wavelet coefficients. The second novelty is the introduction of a ranked probability classifier on the wavelet coefficients that handles both the heteroscedasticity of the data in addition to the potential non-representativity of the training set. The ranked classifier is simple and quick to implement while a major advantage of the BAGIDIS wavelets is that they are translation invariant, hence they do not need the light curves to be aligned to extract features. Further, BAGIDIS is nonparametric so it can be used for blind ...

  6. Nonparametric Bayes analysis of social science data

    Science.gov (United States)

    Kunihama, Tsuyoshi

    Social science data often contain complex characteristics that standard statistical methods fail to capture. Social surveys assign many questions to respondents, which often consist of mixed-scale variables. Each of the variables can follow a complex distribution outside parametric families and associations among variables may have more complicated structures than standard linear dependence. Therefore, it is not straightforward to develop a statistical model which can approximate structures well in the social science data. In addition, many social surveys have collected data over time and therefore we need to incorporate dynamic dependence into the models. Also, it is standard to observe massive number of missing values in the social science data. To address these challenging problems, this thesis develops flexible nonparametric Bayesian methods for the analysis of social science data. Chapter 1 briefly explains backgrounds and motivations of the projects in the following chapters. Chapter 2 develops a nonparametric Bayesian modeling of temporal dependence in large sparse contingency tables, relying on a probabilistic factorization of the joint pmf. Chapter 3 proposes nonparametric Bayes inference on conditional independence with conditional mutual information used as a measure of the strength of conditional dependence. Chapter 4 proposes a novel Bayesian density estimation method in social surveys with complex designs where there is a gap between sample and population. We correct for the bias by adjusting mixture weights in Bayesian mixture models. Chapter 5 develops a nonparametric model for mixed-scale longitudinal surveys, in which various types of variables can be induced through latent continuous variables and dynamic latent factors lead to flexibly time-varying associations among variables.

  7. Bayesian nonparametric estimation for Quantum Homodyne Tomography

    OpenAIRE

    Naulet, Zacharie; Barat, Eric

    2016-01-01

    We estimate the quantum state of a light beam from results of quantum homodyne tomography noisy measurements performed on identically prepared quantum systems. We propose two Bayesian nonparametric approaches. The first approach is based on mixture models and is illustrated through simulation examples. The second approach is based on random basis expansions. We study the theoretical performance of the second approach by quantifying the rate of contraction of the posterior distribution around ...

  8. NONPARAMETRIC ESTIMATION OF CHARACTERISTICS OF PROBABILITY DISTRIBUTIONS

    Directory of Open Access Journals (Sweden)

    Orlov A. I.

    2015-10-01

    Full Text Available The article is devoted to the nonparametric point and interval estimation of the characteristics of the probabilistic distribution (the expectation, median, variance, standard deviation, variation coefficient of the sample results. Sample values are regarded as the implementation of independent and identically distributed random variables with an arbitrary distribution function having the desired number of moments. Nonparametric analysis procedures are compared with the parametric procedures, based on the assumption that the sample values have a normal distribution. Point estimators are constructed in the obvious way - using sample analogs of the theoretical characteristics. Interval estimators are based on asymptotic normality of sample moments and functions from them. Nonparametric asymptotic confidence intervals are obtained through the use of special output technology of the asymptotic relations of Applied Statistics. In the first step this technology uses the multidimensional central limit theorem, applied to the sums of vectors whose coordinates are the degrees of initial random variables. The second step is the conversion limit multivariate normal vector to obtain the interest of researcher vector. At the same considerations we have used linearization and discarded infinitesimal quantities. The third step - a rigorous justification of the results on the asymptotic standard for mathematical and statistical reasoning level. It is usually necessary to use the necessary and sufficient conditions for the inheritance of convergence. This article contains 10 numerical examples. Initial data - information about an operating time of 50 cutting tools to the limit state. Using the methods developed on the assumption of normal distribution, it can lead to noticeably distorted conclusions in a situation where the normality hypothesis failed. Practical recommendations are: for the analysis of real data we should use nonparametric confidence limits

  9. portfolio optimization based on nonparametric estimation methods

    Directory of Open Access Journals (Sweden)

    mahsa ghandehari

    2017-03-01

    Full Text Available One of the major issues investors are facing with in capital markets is decision making about select an appropriate stock exchange for investing and selecting an optimal portfolio. This process is done through the risk and expected return assessment. On the other hand in portfolio selection problem if the assets expected returns are normally distributed, variance and standard deviation are used as a risk measure. But, the expected returns on assets are not necessarily normal and sometimes have dramatic differences from normal distribution. This paper with the introduction of conditional value at risk ( CVaR, as a measure of risk in a nonparametric framework, for a given expected return, offers the optimal portfolio and this method is compared with the linear programming method. The data used in this study consists of monthly returns of 15 companies selected from the top 50 companies in Tehran Stock Exchange during the winter of 1392 which is considered from April of 1388 to June of 1393. The results of this study show the superiority of nonparametric method over the linear programming method and the nonparametric method is much faster than the linear programming method.

  10. Introduction to nonparametric statistics for the biological sciences using R

    CERN Document Server

    MacFarland, Thomas W

    2016-01-01

    This book contains a rich set of tools for nonparametric analyses, and the purpose of this supplemental text is to provide guidance to students and professional researchers on how R is used for nonparametric data analysis in the biological sciences: To introduce when nonparametric approaches to data analysis are appropriate To introduce the leading nonparametric tests commonly used in biostatistics and how R is used to generate appropriate statistics for each test To introduce common figures typically associated with nonparametric data analysis and how R is used to generate appropriate figures in support of each data set The book focuses on how R is used to distinguish between data that could be classified as nonparametric as opposed to data that could be classified as parametric, with both approaches to data classification covered extensively. Following an introductory lesson on nonparametric statistics for the biological sciences, the book is organized into eight self-contained lessons on various analyses a...

  11. Statistical inference based on the nonparametric maximum likelihood estimator under double-truncation.

    Science.gov (United States)

    Emura, Takeshi; Konno, Yoshihiko; Michimae, Hirofumi

    2015-07-01

    Doubly truncated data consist of samples whose observed values fall between the right- and left- truncation limits. With such samples, the distribution function of interest is estimated using the nonparametric maximum likelihood estimator (NPMLE) that is obtained through a self-consistency algorithm. Owing to the complicated asymptotic distribution of the NPMLE, the bootstrap method has been suggested for statistical inference. This paper proposes a closed-form estimator for the asymptotic covariance function of the NPMLE, which is computationally attractive alternative to bootstrapping. Furthermore, we develop various statistical inference procedures, such as confidence interval, goodness-of-fit tests, and confidence bands to demonstrate the usefulness of the proposed covariance estimator. Simulations are performed to compare the proposed method with both the bootstrap and jackknife methods. The methods are illustrated using the childhood cancer dataset.

  12. Photo-z Estimation: An Example of Nonparametric Conditional Density Estimation under Selection Bias

    CERN Document Server

    Izbicki, Rafael; Freeman, Peter E

    2016-01-01

    Redshift is a key quantity for inferring cosmological model parameters. In photometric redshift estimation, cosmologists use the coarse data collected from the vast majority of galaxies to predict the redshift of individual galaxies. To properly quantify the uncertainty in the predictions, however, one needs to go beyond standard regression and instead estimate the full conditional density f(z|x) of a galaxy's redshift z given its photometric covariates x. The problem is further complicated by selection bias: usually only the rarest and brightest galaxies have known redshifts, and these galaxies have characteristics and measured covariates that do not necessarily match those of more numerous and dimmer galaxies of unknown redshift. Unfortunately, there is not much research on how to best estimate complex multivariate densities in such settings. Here we describe a general framework for properly constructing and assessing nonparametric conditional density estimators under selection bias, and for combining two o...

  13. An Evaluation of Parametric and Nonparametric Models of Fish Population Response.

    Energy Technology Data Exchange (ETDEWEB)

    Haas, Timothy C.; Peterson, James T.; Lee, Danny C.

    1999-11-01

    Predicting the distribution or status of animal populations at large scales often requires the use of broad-scale information describing landforms, climate, vegetation, etc. These data, however, often consist of mixtures of continuous and categorical covariates and nonmultiplicative interactions among covariates, complicating statistical analyses. Using data from the interior Columbia River Basin, USA, we compared four methods for predicting the distribution of seven salmonid taxa using landscape information. Subwatersheds (mean size, 7800 ha) were characterized using a set of 12 covariates describing physiography, vegetation, and current land-use. The techniques included generalized logit modeling, classification trees, a nearest neighbor technique, and a modular neural network. We evaluated model performance using out-of-sample prediction accuracy via leave-one-out cross-validation and introduce a computer-intensive Monte Carlo hypothesis testing approach for examining the statistical significance of landscape covariates with the non-parametric methods. We found the modular neural network and the nearest-neighbor techniques to be the most accurate, but were difficult to summarize in ways that provided ecological insight. The modular neural network also required the most extensive computer resources for model fitting and hypothesis testing. The generalized logit models were readily interpretable, but were the least accurate, possibly due to nonlinear relationships and nonmultiplicative interactions among covariates. Substantial overlap among the statistically significant (P<0.05) covariates for each method suggested that each is capable of detecting similar relationships between responses and covariates. Consequently, we believe that employing one or more methods may provide greater biological insight without sacrificing prediction accuracy.

  14. A cautionary note on generalized linear models for covariance of unbalanced longitudinal data

    KAUST Repository

    Huang, Jianhua Z.

    2012-03-01

    Missing data in longitudinal studies can create enormous challenges in data analysis when coupled with the positive-definiteness constraint on a covariance matrix. For complete balanced data, the Cholesky decomposition of a covariance matrix makes it possible to remove the positive-definiteness constraint and use a generalized linear model setup to jointly model the mean and covariance using covariates (Pourahmadi, 2000). However, this approach may not be directly applicable when the longitudinal data are unbalanced, as coherent regression models for the dependence across all times and subjects may not exist. Within the existing generalized linear model framework, we show how to overcome this and other challenges by embedding the covariance matrix of the observed data for each subject in a larger covariance matrix and employing the familiar EM algorithm to compute the maximum likelihood estimates of the parameters and their standard errors. We illustrate and assess the methodology using real data sets and simulations. © 2011 Elsevier B.V.

  15. Bayesian Nonparametric Estimation for Dynamic Treatment Regimes with Sequential Transition Times.

    Science.gov (United States)

    Xu, Yanxun; Müller, Peter; Wahed, Abdus S; Thall, Peter F

    2016-01-01

    We analyze a dataset arising from a clinical trial involving multi-stage chemotherapy regimes for acute leukemia. The trial design was a 2 × 2 factorial for frontline therapies only. Motivated by the idea that subsequent salvage treatments affect survival time, we model therapy as a dynamic treatment regime (DTR), that is, an alternating sequence of adaptive treatments or other actions and transition times between disease states. These sequences may vary substantially between patients, depending on how the regime plays out. To evaluate the regimes, mean overall survival time is expressed as a weighted average of the means of all possible sums of successive transitions times. We assume a Bayesian nonparametric survival regression model for each transition time, with a dependent Dirichlet process prior and Gaussian process base measure (DDP-GP). Posterior simulation is implemented by Markov chain Monte Carlo (MCMC) sampling. We provide general guidelines for constructing a prior using empirical Bayes methods. The proposed approach is compared with inverse probability of treatment weighting, including a doubly robust augmented version of this approach, for both single-stage and multi-stage regimes with treatment assignment depending on baseline covariates. The simulations show that the proposed nonparametric Bayesian approach can substantially improve inference compared to existing methods. An R program for implementing the DDP-GP-based Bayesian nonparametric analysis is freely available at https://www.ma.utexas.edu/users/yxu/.

  16. Algorithm-Eigenvalue Estimation of Hyperspectral Wishart Covariance Matrices from a Limited Number of Samples

    Science.gov (United States)

    2015-03-01

    ALGORITHM—EIGENVALUE ESTIMATION OF HYPERSPECTRAL WISHART COVARIANCE MATRICES FROM A LIMITED NUMBER OF SAMPLES ECBC-TN-067 Avishai Ben-David...Estimation of Hyperspectral Wishart Covariance Matrices from a Limited Number of Samples 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT...covariance matrices and to recompute a revised covariance matrix from the eigenvalues. The MATLAB function is an implementation of the procedure developed

  17. Covariantly Quantum Galileon

    CERN Document Server

    Saltas, Ippocratis D

    2016-01-01

    We derive the 1-loop effective action of the cubic Galileon coupled to quantum-gravitational fluctuations in a background and gauge-independent manner, employing the covariant framework of DeWitt and Vilkovisky. Although the bare action respects shift symmetry, the coupling to gravity induces an effective mass to the scalar, of the order of the cosmological constant, as a direct result of the non-flat field-space metric, the latter ensuring the field-reparametrization invariance of the formalism. Within a gauge-invariant regularization scheme, we discover novel, gravitationally induced non-Galileon higher-derivative interactions in the effective action. These terms, previously unnoticed within standard, non-covariant frameworks, are not Planck suppressed. Unless tuned to be sub-dominant, their presence could have important implications for the classical and quantum phenomenology of the theory.

  18. Covariant approximation averaging

    CERN Document Server

    Shintani, Eigo; Blum, Thomas; Izubuchi, Taku; Jung, Chulwoo; Lehner, Christoph

    2014-01-01

    We present a new class of statistical error reduction techniques for Monte-Carlo simulations. Using covariant symmetries, we show that correlation functions can be constructed from inexpensive approximations without introducing any systematic bias in the final result. We introduce a new class of covariant approximation averaging techniques, known as all-mode averaging (AMA), in which the approximation takes account of contributions of all eigenmodes through the inverse of the Dirac operator computed from the conjugate gradient method with a relaxed stopping condition. In this paper we compare the performance and computational cost of our new method with traditional methods using correlation functions and masses of the pion, nucleon, and vector meson in $N_f=2+1$ lattice QCD using domain-wall fermions. This comparison indicates that AMA significantly reduces statistical errors in Monte-Carlo calculations over conventional methods for the same cost.

  19. Using Analysis of Covariance (ANCOVA) with Fallible Covariates

    Science.gov (United States)

    Culpepper, Steven Andrew; Aguinis, Herman

    2011-01-01

    Analysis of covariance (ANCOVA) is used widely in psychological research implementing nonexperimental designs. However, when covariates are fallible (i.e., measured with error), which is the norm, researchers must choose from among 3 inadequate courses of action: (a) know that the assumption that covariates are perfectly reliable is violated but…

  20. Using Analysis of Covariance (ANCOVA) with Fallible Covariates

    Science.gov (United States)

    Culpepper, Steven Andrew; Aguinis, Herman

    2011-01-01

    Analysis of covariance (ANCOVA) is used widely in psychological research implementing nonexperimental designs. However, when covariates are fallible (i.e., measured with error), which is the norm, researchers must choose from among 3 inadequate courses of action: (a) know that the assumption that covariates are perfectly reliable is violated but…

  1. Two new covariate adjustment methods for non-inferiority assessment of binary clinical trials data.

    Science.gov (United States)

    Hou, Yan; Ding, Victoria; Li, Kang; Zhou, Xiao-Hua

    2011-01-01

    In clinical trials, examining the adjusted treatment difference has become the preferred way to establish non-inferiority (NI) in cases involving a binary endpoint. However, current methods are inadequate in the area of covariate adjustment. In this paper, we introduce two new methods, nonparametric and parametric, of using the probability and probability (P-P) curve to address the issue of unadjusted categorical covariates in the traditional assessment of NI in clinical trials. We also show that the area under the P-P curve is a valid alternative for assessing NI using the adjusted treatment difference, and we compute this area using Mann-Whitney nonparametric statistics. Our simulation studies demonstrate that our proposed methods can not only control type I error at a predefined significance level but also achieve higher statistical power than those of traditional parametric and nonparametric methods that overlook covariate adjustment, especially when covariates are unbalanced in the two treatment groups. We illustrate the effectiveness of our methodology with data from clinical trials of a therapy for coronary heart disease.

  2. A nonparametric and diversified portfolio model

    Science.gov (United States)

    Shirazi, Yasaman Izadparast; Sabiruzzaman, Md.; Hamzah, Nor Aishah

    2014-07-01

    Traditional portfolio models, like mean-variance (MV) suffer from estimation error and lack of diversity. Alternatives, like mean-entropy (ME) or mean-variance-entropy (MVE) portfolio models focus independently on the issue of either a proper risk measure or the diversity. In this paper, we propose an asset allocation model that compromise between risk of historical data and future uncertainty. In the new model, entropy is presented as a nonparametric risk measure as well as an index of diversity. Our empirical evaluation with a variety of performance measures shows that this model has better out-of-sample performances and lower portfolio turnover than its competitors.

  3. Non-Parametric Estimation of Correlation Functions

    DEFF Research Database (Denmark)

    Brincker, Rune; Rytter, Anders; Krenk, Steen

    In this paper three methods of non-parametric correlation function estimation are reviewed and evaluated: the direct method, estimation by the Fast Fourier Transform and finally estimation by the Random Decrement technique. The basic ideas of the techniques are reviewed, sources of bias are pointed...... out, and methods to prevent bias are presented. The techniques are evaluated by comparing their speed and accuracy on the simple case of estimating auto-correlation functions for the response of a single degree-of-freedom system loaded with white noise....

  4. Lottery spending: a non-parametric analysis.

    Science.gov (United States)

    Garibaldi, Skip; Frisoli, Kayla; Ke, Li; Lim, Melody

    2015-01-01

    We analyze the spending of individuals in the United States on lottery tickets in an average month, as reported in surveys. We view these surveys as sampling from an unknown distribution, and we use non-parametric methods to compare properties of this distribution for various demographic groups, as well as claims that some properties of this distribution are constant across surveys. We find that the observed higher spending by Hispanic lottery players can be attributed to differences in education levels, and we dispute previous claims that the top 10% of lottery players consistently account for 50% of lottery sales.

  5. Lottery spending: a non-parametric analysis.

    Directory of Open Access Journals (Sweden)

    Skip Garibaldi

    Full Text Available We analyze the spending of individuals in the United States on lottery tickets in an average month, as reported in surveys. We view these surveys as sampling from an unknown distribution, and we use non-parametric methods to compare properties of this distribution for various demographic groups, as well as claims that some properties of this distribution are constant across surveys. We find that the observed higher spending by Hispanic lottery players can be attributed to differences in education levels, and we dispute previous claims that the top 10% of lottery players consistently account for 50% of lottery sales.

  6. Nonparametric inferences for kurtosis and conditional kurtosis

    Institute of Scientific and Technical Information of China (English)

    XIE Xiao-heng; HE You-hua

    2009-01-01

    Under the assumption of strictly stationary process, this paper proposes a nonparametric model to test the kurtosis and conditional kurtosis for risk time series. We apply this method to the daily returns of S&P500 index and the Shanghai Composite Index, and simulate GARCH data for verifying the efficiency of the presented model. Our results indicate that the risk series distribution is heavily tailed, but the historical information can make its future distribution light-tailed. However the far future distribution's tails are little affected by the historical data.

  7. Parametric versus non-parametric simulation

    OpenAIRE

    Dupeux, Bérénice; Buysse, Jeroen

    2014-01-01

    Most of ex-ante impact assessment policy models have been based on a parametric approach. We develop a novel non-parametric approach, called Inverse DEA. We use non parametric efficiency analysis for determining the farm’s technology and behaviour. Then, we compare the parametric approach and the Inverse DEA models to a known data generating process. We use a bio-economic model as a data generating process reflecting a real world situation where often non-linear relationships exist. Results s...

  8. Preliminary results on nonparametric facial occlusion detection

    Directory of Open Access Journals (Sweden)

    Daniel LÓPEZ SÁNCHEZ

    2016-10-01

    Full Text Available The problem of face recognition has been extensively studied in the available literature, however, some aspects of this field require further research. The design and implementation of face recognition systems that can efficiently handle unconstrained conditions (e.g. pose variations, illumination, partial occlusion... is still an area under active research. This work focuses on the design of a new nonparametric occlusion detection technique. In addition, we present some preliminary results that indicate that the proposed technique might be useful to face recognition systems, allowing them to dynamically discard occluded face parts.

  9. Covariate Adjustment in Receiver Operating Characteristic Curve Analysis for PSA in Diagnosis of Prostate Cancer

    Directory of Open Access Journals (Sweden)

    Emre DEMIR

    2016-10-01

    Full Text Available Objective: Markers which are used for classification into two groups, such as patient / healthy, benign/malignant or prediction of optimal cut off value for diagnostic test and evaluating the performance of diagnostic tests is evaluated by Receiver Operating Characteristic (ROC curve in the diagnostic test researches. In classification accuracy research, some variables such as gender and age, commonly is not similar in groups. In these cases, covariates should be considered to estimate in the area under ROC and covariate adjustment for ROC should be performed. This study aims to introduce methods in the literature for the effect of covariate adjustment and to present an application with sample from the health field. Material and Methods: In the study, we introduced methods used in the literatüre for covariate adjustment and prediction of the area under ROC curves as well as an application with data from the field of urology. In this study, 105 PSA (prostate specific antigen measurements were taken in order to examine the covariate effect for the age variable and to assess the diagnostic performance of PSA measurements with regard to pathologic methods. Results: Covariate effect were found statistically significant with 0.733 parameter estimation of the age in ROC curves analysis with PSA data (p<0.001. According to the methods (Non-parametric (empirical, non-parametric (normal, semi-parametric (empirical, parametric (normal that estimates of the area under ROC curves which is obtained without covariate effect were found 0.708, 0.629, 0.709 and 0.628, respectively, by using PSA measurements. Area under the curve that obtained by covariate adjustment were significantly lower as compared to the traditional ROC with estimation 0.580, 0.577, 0.582 and 0.579. Conclusion: Area under the ROC curves should be estimated with adjustment according to the covariates that could affect the markers value of diagnostic tests performed in concert with matching

  10. Nonparametric feature extraction for classification of hyperspectral images with limited training samples

    Science.gov (United States)

    Kianisarkaleh, Azadeh; Ghassemian, Hassan

    2016-09-01

    Feature extraction plays a crucial role in improvement of hyperspectral images classification. Nonparametric feature extraction methods show better performance compared to parametric ones when distribution of classes is non normal-like. Moreover, they can extract more features than parametric methods do. In this paper, a new nonparametric linear feature extraction method is introduced for classification of hyperspectral images. The proposed method has no free parameter and its novelty can be discussed in two parts. First, neighbor samples are specified by using Parzen window idea for determining local mean. Second, two new weighting functions are used. Samples close to class boundaries will have more weight in the between-class scatter matrix formation and samples close to class mean will have more weight in the within-class scatter matrix formation. The experimental results on three real hyperspectral data sets, Indian Pines, Salinas and Pavia University, demonstrate that the proposed method has better performance in comparison with some other nonparametric and parametric feature extraction methods.

  11. Covariant Magnetic Connection Hypersurfaces

    CERN Document Server

    Pegoraro, F

    2016-01-01

    In the single fluid, nonrelativistic, ideal-Magnetohydrodynamic (MHD) plasma description magnetic field lines play a fundamental role by defining dynamically preserved "magnetic connections" between plasma elements. Here we show how the concept of magnetic connection needs to be generalized in the case of a relativistic MHD description where we require covariance under arbitrary Lorentz transformations. This is performed by defining 2-D {\\it magnetic connection hypersurfaces} in the 4-D Minkowski space. This generalization accounts for the loss of simultaneity between spatially separated events in different frames and is expected to provide a powerful insight into the 4-D geometry of electromagnetic fields when ${\\bf E} \\cdot {\\bf B} = 0$.

  12. Covariant Projective Extensions

    Institute of Scientific and Technical Information of China (English)

    许天周; 梁洁

    2003-01-01

    @@ The theory of crossed products of C*-algebras by groups of automorphisms is a well-developed area of the theory of operator algebras. Given the importance and the success ofthat theory, it is natural to attempt to extend it to a more general situation by, for example,developing a theory of crossed products of C*-algebras by semigroups of automorphisms, or evenof endomorphisms. Indeed, in recent years a number of papers have appeared that are concernedwith such non-classicaltheories of covariance algebras, see, for instance [1-3].

  13. Covariance fitting of highly-correlated data in lattice QCD

    Science.gov (United States)

    Yoon, Boram; Jang, Yong-Chull; Jung, Chulwoo; Lee, Weonjong

    2013-07-01

    We address a frequently-asked question on the covariance fitting of highly-correlated data such as our B K data based on the SU(2) staggered chiral perturbation theory. Basically, the essence of the problem is that we do not have a fitting function accurate enough to fit extremely precise data. When eigenvalues of the covariance matrix are small, even a tiny error in the fitting function yields a large chi-square value and spoils the fitting procedure. We have applied a number of prescriptions available in the market, such as the cut-off method, modified covariance matrix method, and Bayesian method. We also propose a brand new method, the eigenmode shift (ES) method, which allows a full covariance fitting without modifying the covariance matrix at all. We provide a pedagogical example of data analysis in which the cut-off method manifestly fails in fitting, but the rest work well. In our case of the B K fitting, the diagonal approximation, the cut-off method, the ES method, and the Bayesian method work reasonably well in an engineering sense. However, interpreting the meaning of χ 2 is easier in the case of the ES method and the Bayesian method in a theoretical sense aesthetically. Hence, the ES method can be a useful alternative optional tool to check the systematic error caused by the covariance fitting procedure.

  14. Earth Observing System Covariance Realism

    Science.gov (United States)

    Zaidi, Waqar H.; Hejduk, Matthew D.

    2016-01-01

    The purpose of covariance realism is to properly size a primary object's covariance in order to add validity to the calculation of the probability of collision. The covariance realism technique in this paper consists of three parts: collection/calculation of definitive state estimates through orbit determination, calculation of covariance realism test statistics at each covariance propagation point, and proper assessment of those test statistics. An empirical cumulative distribution function (ECDF) Goodness-of-Fit (GOF) method is employed to determine if a covariance is properly sized by comparing the empirical distribution of Mahalanobis distance calculations to the hypothesized parent 3-DoF chi-squared distribution. To realistically size a covariance for collision probability calculations, this study uses a state noise compensation algorithm that adds process noise to the definitive epoch covariance to account for uncertainty in the force model. Process noise is added until the GOF tests pass a group significance level threshold. The results of this study indicate that when outliers attributed to persistently high or extreme levels of solar activity are removed, the aforementioned covariance realism compensation method produces a tuned covariance with up to 80 to 90% of the covariance propagation timespan passing (against a 60% minimum passing threshold) the GOF tests-a quite satisfactory and useful result.

  15. Bayesian Nonparametric Clustering for Positive Definite Matrices.

    Science.gov (United States)

    Cherian, Anoop; Morellas, Vassilios; Papanikolopoulos, Nikolaos

    2016-05-01

    Symmetric Positive Definite (SPD) matrices emerge as data descriptors in several applications of computer vision such as object tracking, texture recognition, and diffusion tensor imaging. Clustering these data matrices forms an integral part of these applications, for which soft-clustering algorithms (K-Means, expectation maximization, etc.) are generally used. As is well-known, these algorithms need the number of clusters to be specified, which is difficult when the dataset scales. To address this issue, we resort to the classical nonparametric Bayesian framework by modeling the data as a mixture model using the Dirichlet process (DP) prior. Since these matrices do not conform to the Euclidean geometry, rather belongs to a curved Riemannian manifold,existing DP models cannot be directly applied. Thus, in this paper, we propose a novel DP mixture model framework for SPD matrices. Using the log-determinant divergence as the underlying dissimilarity measure to compare these matrices, and further using the connection between this measure and the Wishart distribution, we derive a novel DPM model based on the Wishart-Inverse-Wishart conjugate pair. We apply this model to several applications in computer vision. Our experiments demonstrate that our model is scalable to the dataset size and at the same time achieves superior accuracy compared to several state-of-the-art parametric and nonparametric clustering algorithms.

  16. The Covariant Stark Effect

    CERN Document Server

    Land, M C

    2001-01-01

    This paper examines the Stark effect, as a first order perturbation of manifestly covariant hydrogen-like bound states. These bound states are solutions to a relativistic Schr\\"odinger equation with invariant evolution parameter, and represent mass eigenstates whose eigenvalues correspond to the well-known energy spectrum of the non-relativistic theory. In analogy to the nonrelativistic case, the off-diagonal perturbation leads to a lifting of the degeneracy in the mass spectrum. In the covariant case, not only do the spectral lines split, but they acquire an imaginary part which is lnear in the applied electric field, thus revealing induced bound state decay in first order perturbation theory. This imaginary part results from the coupling of the external field to the non-compact boost generator. In order to recover the conventional first order Stark splitting, we must include a scalar potential term. This term may be understood as a fifth gauge potential, which compensates for dependence of gauge transformat...

  17. Selection method of Toeplitz matrix row number based on covariance driven stochastic subspace identification%协方差驱动随机子空间的Toeplitz矩阵行数选择方法

    Institute of Scientific and Technical Information of China (English)

    王燕; 杭晓晨; 姜东; 韩晓林; 费庆国

    2015-01-01

    Stochastic Subspace Identification is a parameter identification method,which can effectively obtain modal parameters from the structural signal under ambient excitation.The choice of Toeplitz matrix row number directly influences the accuracy of identification.By constructing a correlation matrix,the influence of the dimension of Toeplitz matrix i on the denoising ability via SVD was derived.The concept of condition number was introduced in solving the system matrix.According to the relationship between i and condition number of Toeplitz matrix,it is proved once again that i has influence on identification accuracy.Then the selection method of Toeplitz matrix row number i was studied. Two examplic simulations in regard to a two-degree spring mass vibration system and a cropped delta wing model were presented to show the method in the selection of i.The results show that on the premise of determining a suitable system order,the smaller the Toeplitz matrix condition number is,the higher the identification accuracy is.%随机子空间识别算法是一种基于环境激励的模态参数识别方法,仅需要响应时程便可识别模态参数。其中,协方差驱动随机子空间方法中Toeplitz矩阵行数的选取直接影响识别精度。通过构造相关矩阵,研究了Toeplitz矩阵行数i对协方差驱动随机子空间方法中奇异值分解去噪能力的影响。引入Toeplitz矩阵条件数,根据i与Toeplitz矩阵条件数的关系再次证明了i对识别精度的影响。研究了Toeplitz矩阵行数i的选择方法。采用两自由度弹簧振子系统和切尖三角翼模型两个仿真算例研究了Toeplitz矩阵行数i的选择方法。结果表明:在确定合适的系统阶数的前提下,Toeplitz矩阵的条件数越小识别精度越高。

  18. A Bayesian Nonparametric Meta-Analysis Model

    Science.gov (United States)

    Karabatsos, George; Talbott, Elizabeth; Walker, Stephen G.

    2015-01-01

    In a meta-analysis, it is important to specify a model that adequately describes the effect-size distribution of the underlying population of studies. The conventional normal fixed-effect and normal random-effects models assume a normal effect-size population distribution, conditionally on parameters and covariates. For estimating the mean overall…

  19. Covariant holography of a tachyonic accelerating universe

    CERN Document Server

    Rozas-Fernández, Alberto

    2014-01-01

    We apply the holographic principle to a flat dark energy dominated Friedmann-Robertson-Walker spacetime filled with a tachyon scalar field with constant equation of state $w=p/\\rho$, both for $w>-1$ and $w<-1$. By using a geometrical covariant procedure, which allows the construction of holographic hypersurfaces, we have obtained for each case the position of the preferred screen and have then compared these with those obtained by using the holographic dark energy model with the future event horizon as the infrared cutoff. In the phantom scenario, one of the two obtained holographic screens is placed on the big rip hypersurface, both for the covariant holographic formalism and the holographic phantom model. It is also analysed whether the existence of these preferred screens allows a mathematically consistent formulation of fundamental theories based on the existence of a S matrix at infinite distances.

  20. Covariant holography of a tachyonic accelerating universe

    Energy Technology Data Exchange (ETDEWEB)

    Rozas-Fernandez, Alberto [Consejo Superior de Investigaciones Cientificas, Instituto de Fisica Fundamental, Madrid (Spain); University of Portsmouth, Institute of Cosmology and Gravitation, Portsmouth (United Kingdom)

    2014-08-15

    We apply the holographic principle to a flat dark energy dominated Friedmann-Robertson-Walker spacetime filled with a tachyon scalar field with constant equation of state w = p/ρ, both for w > -1 and w < -1. By using a geometrical covariant procedure, which allows the construction of holographic hypersurfaces, we have obtained for each case the position of the preferred screen and have then compared these with those obtained by using the holographic dark energy model with the future event horizon as the infrared cutoff. In the phantom scenario, one of the two obtained holographic screens is placed on the big rip hypersurface, both for the covariant holographic formalism and the holographic phantom model. It is also analyzed whether the existence of these preferred screens allows a mathematically consistent formulation of fundamental theories based on the existence of an S-matrix at infinite distances. (orig.)

  1. On spectral distribution of high dimensional covariation matrices

    DEFF Research Database (Denmark)

    Heinrich, Claudio; Podolskij, Mark

    In this paper we present the asymptotic theory for spectral distributions of high dimensional covariation matrices of Brownian diffusions. More specifically, we consider N-dimensional Itô integrals with time varying matrix-valued integrands. We observe n equidistant high frequency data points...... of the underlying Brownian diffusion and we assume that N/n -> c in (0,oo). We show that under a certain mixed spectral moment condition the spectral distribution of the empirical covariation matrix converges in distribution almost surely. Our proof relies on method of moments and applications of graph theory....

  2. A Statistical Investigation of the Sensitivity of Ensemble-Based Kalman Filters to Covariance Filtering

    Science.gov (United States)

    2011-09-01

    several in- dependent, locally stationary processes with simple parametric stationary (or isotropic) covariance func- tions ( Fuentes 2001). Parametric...230, 99–111. ——, and S. L. Anderson, 1999: A Monte Carlo implementation of the nonlinear filtering problem to produce ensemble assimi- lations and...Q. Yao, 2003: Nonlinear Time Series: Nonparametric and Parametric Methods. Springer-Verlag, 552 pp. Fuentes , M., 2001: A high frequency kriging

  3. Testing for Change in Mean of Independent Multivariate Observations with Time Varying Covariance

    Directory of Open Access Journals (Sweden)

    Mohamed Boutahar

    2012-01-01

    Full Text Available We consider a nonparametric CUSUM test for change in the mean of multivariate time series with time varying covariance. We prove that under the null, the test statistic has a Kolmogorov limiting distribution. The asymptotic consistency of the test against a large class of alternatives which contains abrupt, smooth and continuous changes is established. We also perform a simulation study to analyze the size distortion and the power of the proposed test.

  4. Covariant Residual Entropy

    CERN Document Server

    Hubeny, Veronika E

    2014-01-01

    A recently explored interesting quantity in AdS/CFT, dubbed 'residual entropy', characterizes the amount of collective ignorance associated with either boundary observers restricted to finite time duration, or bulk observers who lack access to a certain spacetime region. However, the previously-proposed expression for this quantity involving variation of boundary entanglement entropy (subsequently renamed to 'differential entropy') works only in a severely restrictive context. We explain the key limitations, arguing that in general, differential entropy does not correspond to residual entropy. Given that the concept of residual entropy as collective ignorance transcends these limitations, we identify two correspondingly robust, covariantly-defined constructs: a 'strip wedge' associated with boundary observers and a 'rim wedge' associated with bulk observers. These causal sets are well-defined in arbitrary time-dependent asymptotically AdS spacetimes in any number of dimensions. We discuss their relation, spec...

  5. Covariant Macroscopic Quantum Geometry

    CERN Document Server

    Hogan, Craig J

    2012-01-01

    A covariant noncommutative algebra of position operators is presented, and interpreted as the macroscopic limit of a geometry that describes a collective quantum behavior of the positions of massive bodies in a flat emergent space-time. The commutator defines a quantum-geometrical relationship between world lines that depends on their separation and relative velocity, but on no other property of the bodies, and leads to a transverse uncertainty of the geometrical wave function that increases with separation. The number of geometrical degrees of freedom in a space-time volume scales holographically, as the surface area in Planck units. Ongoing branching of the wave function causes fluctuations in transverse position, shared coherently among bodies with similar trajectories. The theory can be tested using appropriately configured Michelson interferometers.

  6. Covariantly quantum Galileon

    Science.gov (United States)

    Saltas, Ippocratis D.; Vitagliano, Vincenzo

    2017-05-01

    We derive the 1-loop effective action of the cubic Galileon coupled to quantum-gravitational fluctuations in a background and gauge-independent manner, employing the covariant framework of DeWitt and Vilkovisky. Although the bare action respects shift symmetry, the coupling to gravity induces an effective mass to the scalar, of the order of the cosmological constant, as a direct result of the nonflat field-space metric, the latter ensuring the field-reparametrization invariance of the formalism. Within a gauge-invariant regularization scheme, we discover novel, gravitationally induced non-Galileon higher-derivative interactions in the effective action. These terms, previously unnoticed within standard, noncovariant frameworks, are not Planck suppressed. Unless tuned to be subdominant, their presence could have important implications for the classical and quantum phenomenology of the theory.

  7. Covariant holographic entanglement negativity

    CERN Document Server

    Chaturvedi, Pankaj; Sengupta, Gautam

    2016-01-01

    We conjecture a holographic prescription for the covariant entanglement negativity of $d$-dimensional conformal field theories dual to non static bulk $AdS_{d+1}$ gravitational configurations in the framework of the $AdS/CFT$ correspondence. Application of our conjecture to a $AdS_3/CFT_2$ scenario involving bulk rotating BTZ black holes exactly reproduces the entanglement negativity of the corresponding $(1+1)$ dimensional conformal field theories and precisely captures the distillable quantum entanglement. Interestingly our conjecture for the scenario involving dual bulk extremal rotating BTZ black holes also accurately leads to the entanglement negativity for the chiral half of the corresponding $(1+1)$ dimensional conformal field theory at zero temperature.

  8. Nonparametric dark energy reconstruction from supernova data.

    Science.gov (United States)

    Holsclaw, Tracy; Alam, Ujjaini; Sansó, Bruno; Lee, Herbert; Heitmann, Katrin; Habib, Salman; Higdon, David

    2010-12-10

    Understanding the origin of the accelerated expansion of the Universe poses one of the greatest challenges in physics today. Lacking a compelling fundamental theory to test, observational efforts are targeted at a better characterization of the underlying cause. If a new form of mass-energy, dark energy, is driving the acceleration, the redshift evolution of the equation of state parameter w(z) will hold essential clues as to its origin. To best exploit data from observations it is necessary to develop a robust and accurate reconstruction approach, with controlled errors, for w(z). We introduce a new, nonparametric method for solving the associated statistical inverse problem based on Gaussian process modeling and Markov chain Monte Carlo sampling. Applying this method to recent supernova measurements, we reconstruct the continuous history of w out to redshift z=1.5.

  9. Local Component Analysis for Nonparametric Bayes Classifier

    CERN Document Server

    Khademi, Mahmoud; safayani, Meharn

    2010-01-01

    The decision boundaries of Bayes classifier are optimal because they lead to maximum probability of correct decision. It means if we knew the prior probabilities and the class-conditional densities, we could design a classifier which gives the lowest probability of error. However, in classification based on nonparametric density estimation methods such as Parzen windows, the decision regions depend on the choice of parameters such as window width. Moreover, these methods suffer from curse of dimensionality of the feature space and small sample size problem which severely restricts their practical applications. In this paper, we address these problems by introducing a novel dimension reduction and classification method based on local component analysis. In this method, by adopting an iterative cross-validation algorithm, we simultaneously estimate the optimal transformation matrices (for dimension reduction) and classifier parameters based on local information. The proposed method can classify the data with co...

  10. Nonparametric k-nearest-neighbor entropy estimator.

    Science.gov (United States)

    Lombardi, Damiano; Pant, Sanjay

    2016-01-01

    A nonparametric k-nearest-neighbor-based entropy estimator is proposed. It improves on the classical Kozachenko-Leonenko estimator by considering nonuniform probability densities in the region of k-nearest neighbors around each sample point. It aims to improve the classical estimators in three situations: first, when the dimensionality of the random variable is large; second, when near-functional relationships leading to high correlation between components of the random variable are present; and third, when the marginal variances of random variable components vary significantly with respect to each other. Heuristics on the error of the proposed and classical estimators are presented. Finally, the proposed estimator is tested for a variety of distributions in successively increasing dimensions and in the presence of a near-functional relationship. Its performance is compared with a classical estimator, and a significant improvement is demonstrated.

  11. Nonparametric estimation of location and scale parameters

    KAUST Repository

    Potgieter, C.J.

    2012-12-01

    Two random variables X and Y belong to the same location-scale family if there are constants μ and σ such that Y and μ+σX have the same distribution. In this paper we consider non-parametric estimation of the parameters μ and σ under minimal assumptions regarding the form of the distribution functions of X and Y. We discuss an approach to the estimation problem that is based on asymptotic likelihood considerations. Our results enable us to provide a methodology that can be implemented easily and which yields estimators that are often near optimal when compared to fully parametric methods. We evaluate the performance of the estimators in a series of Monte Carlo simulations. © 2012 Elsevier B.V. All rights reserved.

  12. Nonparametric Maximum Entropy Estimation on Information Diagrams

    CERN Document Server

    Martin, Elliot A; Meinke, Alexander; Děchtěrenko, Filip; Davidsen, Jörn

    2016-01-01

    Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of random discrete variables when conditioning on bivariate mutual informations and univariate entropies. Specifically, we show how to apply the concept to continuous random variables and vastly expand the types of information-theoretic quantities one can condition on. This allows us to establish a number of significant advantages of our approach over existing ones. Not only does our method perform favorably in the undersampled regime, where existing methods fail, but it also can be dramatically less computationally expensive as the cardinality of the variables increases. In addition, we propose a nonparametric formulation of connected informations and give an illustrative example showing how this agrees with the existing parametric formulation in cases of interest. We furthe...

  13. Nonparametric estimation of employee stock options

    Institute of Scientific and Technical Information of China (English)

    FU Qiang; LIU Li-an; LIU Qian

    2006-01-01

    We proposed a new model to price employee stock options (ESOs). The model is based on nonparametric statistical methods with market data. It incorporates the kernel estimator and employs a three-step method to modify BlackScholes formula. The model overcomes the limits of Black-Scholes formula in handling option prices with varied volatility. It disposes the effects of ESOs self-characteristics such as non-tradability, the longer term for expiration, the early exercise feature, the restriction on shorting selling and the employee's risk aversion on risk neutral pricing condition, and can be applied to ESOs valuation with the explanatory variable in no matter the certainty case or random case.

  14. On Parametric (and Non-Parametric Variation

    Directory of Open Access Journals (Sweden)

    Neil Smith

    2009-11-01

    Full Text Available This article raises the issue of the correct characterization of ‘Parametric Variation’ in syntax and phonology. After specifying their theoretical commitments, the authors outline the relevant parts of the Principles–and–Parameters framework, and draw a three-way distinction among Universal Principles, Parameters, and Accidents. The core of the contribution then consists of an attempt to provide identity criteria for parametric, as opposed to non-parametric, variation. Parametric choices must be antecedently known, and it is suggested that they must also satisfy seven individually necessary and jointly sufficient criteria. These are that they be cognitively represented, systematic, dependent on the input, deterministic, discrete, mutually exclusive, and irreversible.

  15. Estimation of Esfarayen Farmers Risk Aversion Coefficient and Its Influencing Factors (Nonparametric Approach

    Directory of Open Access Journals (Sweden)

    Z. Nematollahi

    2016-03-01

    Full Text Available Introduction: Due to existence of the risk and uncertainty in agriculture, risk management is crucial for management in agriculture. Therefore the present study was designed to determine the risk aversion coefficient for Esfarayens farmers. Materials and Methods: The following approaches have been utilized to assess risk attitudes: (1 direct elicitation of utility functions, (2 experimental procedures in which individuals are presented with hypothetical questionnaires regarding risky alternatives with or without real payments and (3: Inference from observation of economic behavior. In this paper, we focused on approach (3: inference from observation of economic behavior, based on this assumption of existence of the relationship between the actual behavior of a decision maker and the behavior predicted from empirically specified models. A new non-parametric method and the QP method were used to calculate the coefficient of risk aversion. We maximized the decision maker expected utility with the E-V formulation (Freund, 1956. Ideally, in constructing a QP model, the variance-covariance matrix should be formed for each individual farmer. For this purpose, a sample of 100 farmers was selected using random sampling and their data about 14 products of years 2008- 2012 were assembled. The lowlands of Esfarayen were used since within this area, production possibilities are rather homogeneous. Results and Discussion: The results of this study showed that there was low correlation between some of the activities, which implies opportunities for income stabilization through diversification. With respect to transitory income, Ra, vary from 0.000006 to 0.000361 and the absolute coefficient of risk aversion in our sample were 0.00005. The estimated Ra values vary considerably from farm to farm. The results showed that the estimated Ra for the subsample existing of 'non-wealthy' farmers was 0.00010. The subsample with farmers in the 'wealthy' group had an

  16. Efficient retrieval of landscape Hessian: forced optimal covariance adaptive learning.

    Science.gov (United States)

    Shir, Ofer M; Roslund, Jonathan; Whitley, Darrell; Rabitz, Herschel

    2014-06-01

    Knowledge of the Hessian matrix at the landscape optimum of a controlled physical observable offers valuable information about the system robustness to control noise. The Hessian can also assist in physical landscape characterization, which is of particular interest in quantum system control experiments. The recently developed landscape theoretical analysis motivated the compilation of an automated method to learn the Hessian matrix about the global optimum without derivative measurements from noisy data. The current study introduces the forced optimal covariance adaptive learning (FOCAL) technique for this purpose. FOCAL relies on the covariance matrix adaptation evolution strategy (CMA-ES) that exploits covariance information amongst the control variables by means of principal component analysis. The FOCAL technique is designed to operate with experimental optimization, generally involving continuous high-dimensional search landscapes (≳30) with large Hessian condition numbers (≳10^{4}). This paper introduces the theoretical foundations of the inverse relationship between the covariance learned by the evolution strategy and the actual Hessian matrix of the landscape. FOCAL is presented and demonstrated to retrieve the Hessian matrix with high fidelity on both model landscapes and quantum control experiments, which are observed to possess nonseparable, nonquadratic search landscapes. The recovered Hessian forms were corroborated by physical knowledge of the systems. The implications of FOCAL extend beyond the investigated studies to potentially cover other physically motivated multivariate landscapes.

  17. Nonparametric inference of network structure and dynamics

    Science.gov (United States)

    Peixoto, Tiago P.

    The network structure of complex systems determine their function and serve as evidence for the evolutionary mechanisms that lie behind them. Despite considerable effort in recent years, it remains an open challenge to formulate general descriptions of the large-scale structure of network systems, and how to reliably extract such information from data. Although many approaches have been proposed, few methods attempt to gauge the statistical significance of the uncovered structures, and hence the majority cannot reliably separate actual structure from stochastic fluctuations. Due to the sheer size and high-dimensionality of many networks, this represents a major limitation that prevents meaningful interpretations of the results obtained with such nonstatistical methods. In this talk, I will show how these issues can be tackled in a principled and efficient fashion by formulating appropriate generative models of network structure that can have their parameters inferred from data. By employing a Bayesian description of such models, the inference can be performed in a nonparametric fashion, that does not require any a priori knowledge or ad hoc assumptions about the data. I will show how this approach can be used to perform model comparison, and how hierarchical models yield the most appropriate trade-off between model complexity and quality of fit based on the statistical evidence present in the data. I will also show how this general approach can be elegantly extended to networks with edge attributes, that are embedded in latent spaces, and that change in time. The latter is obtained via a fully dynamic generative network model, based on arbitrary-order Markov chains, that can also be inferred in a nonparametric fashion. Throughout the talk I will illustrate the application of the methods with many empirical networks such as the internet at the autonomous systems level, the global airport network, the network of actors and films, social networks, citations among

  18. A nonparametric dynamic additive regression model for longitudinal data

    DEFF Research Database (Denmark)

    Martinussen, Torben; Scheike, Thomas H.

    2000-01-01

    dynamic linear models, estimating equations, least squares, longitudinal data, nonparametric methods, partly conditional mean models, time-varying-coefficient models......dynamic linear models, estimating equations, least squares, longitudinal data, nonparametric methods, partly conditional mean models, time-varying-coefficient models...

  19. Nonparametric Bayesian inference for multidimensional compound Poisson processes

    NARCIS (Netherlands)

    S. Gugushvili; F. van der Meulen; P. Spreij

    2015-01-01

    Given a sample from a discretely observed multidimensional compound Poisson process, we study the problem of nonparametric estimation of its jump size density r0 and intensity λ0. We take a nonparametric Bayesian approach to the problem and determine posterior contraction rates in this context, whic

  20. Asymptotic theory of nonparametric regression estimates with censored data

    Institute of Scientific and Technical Information of China (English)

    施沛德; 王海燕; 张利华

    2000-01-01

    For regression analysis, some useful Information may have been lost when the responses are right censored. To estimate nonparametric functions, several estimates based on censored data have been proposed and their consistency and convergence rates have been studied in literat黵e, but the optimal rates of global convergence have not been obtained yet. Because of the possible Information loss, one may think that it is impossible for an estimate based on censored data to achieve the optimal rates of global convergence for nonparametric regression, which were established by Stone based on complete data. This paper constructs a regression spline estimate of a general nonparametric regression f unction based on right-censored response data, and proves, under some regularity condi-tions, that this estimate achieves the optimal rates of global convergence for nonparametric regression. Since the parameters for the nonparametric regression estimate have to be chosen based on a data driven criterion, we also obtai

  1. 2nd Conference of the International Society for Nonparametric Statistics

    CERN Document Server

    Manteiga, Wenceslao; Romo, Juan

    2016-01-01

    This volume collects selected, peer-reviewed contributions from the 2nd Conference of the International Society for Nonparametric Statistics (ISNPS), held in Cádiz (Spain) between June 11–16 2014, and sponsored by the American Statistical Association, the Institute of Mathematical Statistics, the Bernoulli Society for Mathematical Statistics and Probability, the Journal of Nonparametric Statistics and Universidad Carlos III de Madrid. The 15 articles are a representative sample of the 336 contributed papers presented at the conference. They cover topics such as high-dimensional data modelling, inference for stochastic processes and for dependent data, nonparametric and goodness-of-fit testing, nonparametric curve estimation, object-oriented data analysis, and semiparametric inference. The aim of the ISNPS 2014 conference was to bring together recent advances and trends in several areas of nonparametric statistics in order to facilitate the exchange of research ideas, promote collaboration among researchers...

  2. A simple procedure for the comparison of covariance matrices.

    Science.gov (United States)

    Garcia, Carlos

    2012-11-21

    Comparing the covariation patterns of populations or species is a basic step in the evolutionary analysis of quantitative traits. Here I propose a new, simple method to make this comparison in two population samples that is based on comparing the variance explained in each sample by the eigenvectors of its own covariance matrix with that explained by the covariance matrix eigenvectors of the other sample. The rationale of this procedure is that the matrix eigenvectors of two similar samples would explain similar amounts of variance in the two samples. I use computer simulation and morphological covariance matrices from the two morphs in a marine snail hybrid zone to show how the proposed procedure can be used to measure the contribution of the matrices orientation and shape to the overall differentiation. I show how this procedure can detect even modest differences between matrices calculated with moderately sized samples, and how it can be used as the basis for more detailed analyses of the nature of these differences. The new procedure constitutes a useful resource for the comparison of covariance matrices. It could fill the gap between procedures resulting in a single, overall measure of differentiation, and analytical methods based on multiple model comparison not providing such a measure.

  3. A simple procedure for the comparison of covariance matrices

    Science.gov (United States)

    2012-01-01

    Background Comparing the covariation patterns of populations or species is a basic step in the evolutionary analysis of quantitative traits. Here I propose a new, simple method to make this comparison in two population samples that is based on comparing the variance explained in each sample by the eigenvectors of its own covariance matrix with that explained by the covariance matrix eigenvectors of the other sample. The rationale of this procedure is that the matrix eigenvectors of two similar samples would explain similar amounts of variance in the two samples. I use computer simulation and morphological covariance matrices from the two morphs in a marine snail hybrid zone to show how the proposed procedure can be used to measure the contribution of the matrices orientation and shape to the overall differentiation. Results I show how this procedure can detect even modest differences between matrices calculated with moderately sized samples, and how it can be used as the basis for more detailed analyses of the nature of these differences. Conclusions The new procedure constitutes a useful resource for the comparison of covariance matrices. It could fill the gap between procedures resulting in a single, overall measure of differentiation, and analytical methods based on multiple model comparison not providing such a measure. PMID:23171139

  4. A simple procedure for the comparison of covariance matrices

    Directory of Open Access Journals (Sweden)

    Garcia Carlos

    2012-11-01

    Full Text Available Abstract Background Comparing the covariation patterns of populations or species is a basic step in the evolutionary analysis of quantitative traits. Here I propose a new, simple method to make this comparison in two population samples that is based on comparing the variance explained in each sample by the eigenvectors of its own covariance matrix with that explained by the covariance matrix eigenvectors of the other sample. The rationale of this procedure is that the matrix eigenvectors of two similar samples would explain similar amounts of variance in the two samples. I use computer simulation and morphological covariance matrices from the two morphs in a marine snail hybrid zone to show how the proposed procedure can be used to measure the contribution of the matrices orientation and shape to the overall differentiation. Results I show how this procedure can detect even modest differences between matrices calculated with moderately sized samples, and how it can be used as the basis for more detailed analyses of the nature of these differences. Conclusions The new procedure constitutes a useful resource for the comparison of covariance matrices. It could fill the gap between procedures resulting in a single, overall measure of differentiation, and analytical methods based on multiple model comparison not providing such a measure.

  5. Classifying Algorithm of Mouse Behavior Using Distance of Log-covariance Matrix%应用L og-协方差矩阵距离的实验鼠行为分类方法

    Institute of Scientific and Technical Information of China (English)

    马玲玲; 洪留荣; 胡倩

    2014-01-01

    实验鼠行为分类在神经科学、生物科学、药物开发等领域的研究中十分重要.针对行为分析中实验鼠肢体短小,提取相关信息困难问题,应用实验鼠轮廓抽取特征,进行实验鼠的行为分类.首先从视频每一帧图像中分割出实验鼠轮廓,并提取轮廓中心从八个方向上到轮廓边缘的距离以及轮廓区域的两个主分量,组成一个10维向量作为特征向量,最后应用协方差距离进行分类.实验结果显示,分类正确率达87.6%.%The classification of mice behavior is very important in the study fields of neuroscience, biological science, drug development and so on. In behavioral analysis,according to the short limbs of mice, it is difficult to extract relevant information. Mouse behavior is classified using characteristics of mouse contour. Firstly, mouse contour is extracted from each frame behavior video. Then, the distance of eight directions are calculated which are from contour center to edge and calculate two main components of contour. Using these values to form ten dimensional vectors as feature vector. Finally, calculate the log-covariance distance and classify. Experimental results show that the correct rate of classification is 87.6%.

  6. Progress of Covariance Evaluation at the China Nuclear Data Center

    Energy Technology Data Exchange (ETDEWEB)

    Xu, R., E-mail: xuruirui@ciae.ac.cn [China Nuclear Data Center, P.O. Box, 275(41), Beijing 102413 (China); Zhang, Q. [China Nuclear Data Center, P.O. Box, 275(41), Beijing 102413 (China); Shanxi Normal University, Linfen, Shanxi Province 041004 (China); Zhang, Y.; Liu, T.; Ge, Z.; Lu, H.; Sun, Z.; Yu, B. [China Nuclear Data Center, P.O. Box, 275(41), Beijing 102413 (China); Tang, G. [Peking University, Beijing 100871 (China)

    2015-01-15

    Covariance evaluations at the China Nuclear Data Center focus on the cross sections of structural materials and actinides in the fast neutron energy range. In addition to the well-known Least-squares approach, a method based on the analysis of the sources of experimental uncertainties is especially introduced to generate a covariance matrix for a particular reaction for which multiple measurements are available. The scheme of the covariance evaluation flow is presented, and an example of n+{sup 90}Zr is given to illuminate the whole procedure. It is proven that the accuracy of measurements can be properly incorporated into the covariance and the long-standing small uncertainty problem can be avoided.

  7. How covariant is the galaxy luminosity function?

    CERN Document Server

    Smith, Robert E

    2012-01-01

    We investigate the error properties of certain galaxy luminosity function (GLF) estimators. Using a cluster expansion of the density field, we show how, for both volume and flux limited samples, the GLF estimates are covariant. The covariance matrix can be decomposed into three pieces: a diagonal term arising from Poisson noise; a sample variance term arising from large-scale structure in the survey volume; an occupancy covariance term arising due to galaxies of different luminosities inhabiting the same cluster. To evaluate the theory one needs: the mass function and bias of clusters, and the conditional luminosity function (CLF). We use a semi-analytic model (SAM) galaxy catalogue from the Millennium run N-body simulation and the CLF of Yang et al. (2003) to explore these effects. The GLF estimates from the SAM and the CLF qualitatively reproduce results from the 2dFGRS. We also measure the luminosity dependence of clustering in the SAM and find reasonable agreement with 2dFGRS results for bright galaxies. ...

  8. Covariant electromagnetic field lines

    Science.gov (United States)

    Hadad, Y.; Cohen, E.; Kaminer, I.; Elitzur, A. C.

    2017-08-01

    Faraday introduced electric field lines as a powerful tool for understanding the electric force, and these field lines are still used today in classrooms and textbooks teaching the basics of electromagnetism within the electrostatic limit. However, despite attempts at generalizing this concept beyond the electrostatic limit, such a fully relativistic field line theory still appears to be missing. In this work, we propose such a theory and define covariant electromagnetic field lines that naturally extend electric field lines to relativistic systems and general electromagnetic fields. We derive a closed-form formula for the field lines curvature in the vicinity of a charge, and show that it is related to the world line of the charge. This demonstrates how the kinematics of a charge can be derived from the geometry of the electromagnetic field lines. Such a theory may also provide new tools in modeling and analyzing electromagnetic phenomena, and may entail new insights regarding long-standing problems such as radiation-reaction and self-force. In particular, the electromagnetic field lines curvature has the attractive property of being non-singular everywhere, thus eliminating all self-field singularities without using renormalization techniques.

  9. Sparse reduced-rank regression with covariance estimation

    KAUST Repository

    Chen, Lisha

    2014-12-08

    Improving the predicting performance of the multiple response regression compared with separate linear regressions is a challenging question. On the one hand, it is desirable to seek model parsimony when facing a large number of parameters. On the other hand, for certain applications it is necessary to take into account the general covariance structure for the errors of the regression model. We assume a reduced-rank regression model and work with the likelihood function with general error covariance to achieve both objectives. In addition we propose to select relevant variables for reduced-rank regression by using a sparsity-inducing penalty, and to estimate the error covariance matrix simultaneously by using a similar penalty on the precision matrix. We develop a numerical algorithm to solve the penalized regression problem. In a simulation study and real data analysis, the new method is compared with two recent methods for multivariate regression and exhibits competitive performance in prediction and variable selection.

  10. Extreme eigenvalues of sample covariance and correlation matrices

    DEFF Research Database (Denmark)

    Heiny, Johannes

    This thesis is concerned with asymptotic properties of the eigenvalues of high-dimensional sample covariance and correlation matrices under an infinite fourth moment of the entries. In the first part, we study the joint distributional convergence of the largest eigenvalues of the sample covariance...... of the problem at hand. We develop a theory for the point process of the normalized eigenvalues of the sample covariance matrix in the case where rows and columns of the data are linearly dependent. Based on the weak convergence of this point process we derive the limit laws of various functionals...... of the eigenvalues. In the second part, we show that the largest and smallest eigenvalues of a highdimensional sample correlation matrix possess almost sure non-random limits if the truncated variance of the entry distribution is “almost slowly varying”, a condition we describe via moment properties of self...

  11. Nonparametric Bayesian Sparse Factor Models with application to Gene Expression modelling

    CERN Document Server

    Knowles, David

    2010-01-01

    A nonparametric Bayesian extension of Factor Analysis (FA) is proposed where observed data Y is modeled as a linear superposition, G, of a potentially infinite number of hidden factors, X. The Indian Buffet Process (IBP) is used as a prior on G to incorporate sparsity and to allow the number of latent features to be inferred. The model's utility for modeling gene expression data is investigated using randomly generated datasets based on a known sparse connectivity matrix for E. Coli, and on three biological datasets of increasing complexity.

  12. Spectral Density of Sample Covariance Matrices of Colored Noise

    CERN Document Server

    Dolezal, Emil

    2008-01-01

    We study the dependence of the spectral density of the covariance matrix ensemble on the power spectrum of the underlying multivariate signal. The white noise signal leads to the celebrated Marchenko-Pastur formula. We demonstrate results for some colored noise signals.

  13. Modeling the Conditional Covariance between Stock and Bond Returns

    NARCIS (Netherlands)

    P. de Goeij (Peter); W.A. Marquering (Wessel)

    2002-01-01

    textabstractTo analyze the intertemporal interaction between the stock and bond market returns, we allow the conditional covariance matrix to vary over time according to a multivariate GARCH model similar to Bollerslev, Engle and Wooldridge (1988). We extend the model such that it allows for asymmet

  14. Improved Multiple Signal Classification Algorithm for Direction of Arrival Estimation Based on Covariance Matrix of Cross-correlation%基于互相关协方差矩阵的改进多重信号分类高分辨波达方位估计方法

    Institute of Scientific and Technical Information of China (English)

    2015-01-01

    针对经典高分辨波达方位(DOA)估计方法在低信噪比下分辨性能较差的问题,该文提出一种适用于主动探测系统的基于互相关矩阵的改进多重信号分类(MUSIC)高分辨方位估计方法(I-MUSIC)。该方法首先利用主动声呐发射信号已知的特性,将发射信号与阵元接收信号进行互相关,利用互相关序列形成新的空域协方差矩阵,再进行特征分解。理论分析表明,互相关处理在抑制噪声的同时保留了阵元之间的相位信息,可以得到比MUSIC方法更准确的子空间划分,进而提高低信噪比方位估计性能。在此基础上,提出一种基于相关时间门限的改进MUSIC高分辨方位估计(T-MUSIC)方法,通过对互相关序列设置时间门限进一步提高方位估计信噪比。仿真结果表明,与MUSIC方法相比,I-MUSIC与T-MUSIC可以分别使低信噪比时的估计性能提高3 dB和6 dB,相应平均估计误差分别为原方法的77%和53%。在阵元间接收噪声存在相关性时,T-MUSIC与I-MUSIC方法相比可获得8 dB的估计增益,估计性能更优。I-MUSIC 与 T-MUSIC 应用于多目标主动探测,可大幅提高探测系统在低信噪比下的方位估计性能。%In view of the poor performance of traditional Direction of Arrival (DOA) methods at low signal-to-noise ratios, an improved MUltiple SIgnal Classification (MUSIC) algorithm for DOA estimation applied to active detection system based on covariance matrix decomposition of cross-correlation (I-MUSIC) is proposed. Exploiting the transmission feature of active sonar, cross-correlation sequence between the transmitted signal and the array output is formulated. The spatial covariance matrix is then constructed from the sequence. Then matrix decomposition is implemented over the new spatial covariance matrix to estimate the DOA. It is proved that cross-correlation can suppress noise while preserving the phase information between array

  15. Enveloping Spectral Surfaces: Covariate Dependent Spectral Analysis of Categorical Time Series.

    Science.gov (United States)

    Krafty, Robert T; Xiong, Shuangyan; Stoffer, David S; Buysse, Daniel J; Hall, Martica

    2012-09-01

    Motivated by problems in Sleep Medicine and Circadian Biology, we present a method for the analysis of cross-sectional categorical time series collected from multiple subjects where the effect of static continuous-valued covariates is of interest. Toward this goal, we extend the spectral envelope methodology for the frequency domain analysis of a single categorical process to cross-sectional categorical processes that are possibly covariate dependent. The analysis introduces an enveloping spectral surface for describing the association between the frequency domain properties of qualitative time series and covariates. The resulting surface offers an intuitively interpretable measure of association between covariates and a qualitative time series by finding the maximum possible conditional power at a given frequency from scalings of the qualitative time series conditional on the covariates. The optimal scalings that maximize the power provide scientific insight by identifying the aspects of the qualitative series which have the most pronounced periodic features at a given frequency conditional on the value of the covariates. To facilitate the assessment of the dependence of the enveloping spectral surface on the covariates, we include a theory for analyzing the partial derivatives of the surface. Our approach is entirely nonparametric, and we present estimation and asymptotics in the setting of local polynomial smoothing.

  16. Nonparametric methods in actigraphy: An update

    Directory of Open Access Journals (Sweden)

    Bruno S.B. Gonçalves

    2014-09-01

    Full Text Available Circadian rhythmicity in humans has been well studied using actigraphy, a method of measuring gross motor movement. As actigraphic technology continues to evolve, it is important for data analysis to keep pace with new variables and features. Our objective is to study the behavior of two variables, interdaily stability and intradaily variability, to describe rest activity rhythm. Simulated data and actigraphy data of humans, rats, and marmosets were used in this study. We modified the method of calculation for IV and IS by modifying the time intervals of analysis. For each variable, we calculated the average value (IVm and ISm results for each time interval. Simulated data showed that (1 synchronization analysis depends on sample size, and (2 fragmentation is independent of the amplitude of the generated noise. We were able to obtain a significant difference in the fragmentation patterns of stroke patients using an IVm variable, while the variable IV60 was not identified. Rhythmic synchronization of activity and rest was significantly higher in young than adults with Parkinson׳s when using the ISM variable; however, this difference was not seen using IS60. We propose an updated format to calculate rhythmic fragmentation, including two additional optional variables. These alternative methods of nonparametric analysis aim to more precisely detect sleep–wake cycle fragmentation and synchronization.

  17. Bayesian nonparametric adaptive control using Gaussian processes.

    Science.gov (United States)

    Chowdhary, Girish; Kingravi, Hassan A; How, Jonathan P; Vela, Patricio A

    2015-03-01

    Most current model reference adaptive control (MRAC) methods rely on parametric adaptive elements, in which the number of parameters of the adaptive element are fixed a priori, often through expert judgment. An example of such an adaptive element is radial basis function networks (RBFNs), with RBF centers preallocated based on the expected operating domain. If the system operates outside of the expected operating domain, this adaptive element can become noneffective in capturing and canceling the uncertainty, thus rendering the adaptive controller only semiglobal in nature. This paper investigates a Gaussian process-based Bayesian MRAC architecture (GP-MRAC), which leverages the power and flexibility of GP Bayesian nonparametric models of uncertainty. The GP-MRAC does not require the centers to be preallocated, can inherently handle measurement noise, and enables MRAC to handle a broader set of uncertainties, including those that are defined as distributions over functions. We use stochastic stability arguments to show that GP-MRAC guarantees good closed-loop performance with no prior domain knowledge of the uncertainty. Online implementable GP inference methods are compared in numerical simulations against RBFN-MRAC with preallocated centers and are shown to provide better tracking and improved long-term learning.

  18. Nonparametric methods in actigraphy: An update

    Science.gov (United States)

    Gonçalves, Bruno S.B.; Cavalcanti, Paula R.A.; Tavares, Gracilene R.; Campos, Tania F.; Araujo, John F.

    2014-01-01

    Circadian rhythmicity in humans has been well studied using actigraphy, a method of measuring gross motor movement. As actigraphic technology continues to evolve, it is important for data analysis to keep pace with new variables and features. Our objective is to study the behavior of two variables, interdaily stability and intradaily variability, to describe rest activity rhythm. Simulated data and actigraphy data of humans, rats, and marmosets were used in this study. We modified the method of calculation for IV and IS by modifying the time intervals of analysis. For each variable, we calculated the average value (IVm and ISm) results for each time interval. Simulated data showed that (1) synchronization analysis depends on sample size, and (2) fragmentation is independent of the amplitude of the generated noise. We were able to obtain a significant difference in the fragmentation patterns of stroke patients using an IVm variable, while the variable IV60 was not identified. Rhythmic synchronization of activity and rest was significantly higher in young than adults with Parkinson׳s when using the ISM variable; however, this difference was not seen using IS60. We propose an updated format to calculate rhythmic fragmentation, including two additional optional variables. These alternative methods of nonparametric analysis aim to more precisely detect sleep–wake cycle fragmentation and synchronization. PMID:26483921

  19. Nonparametric Detection of Geometric Structures Over Networks

    Science.gov (United States)

    Zou, Shaofeng; Liang, Yingbin; Poor, H. Vincent

    2017-10-01

    Nonparametric detection of existence of an anomalous structure over a network is investigated. Nodes corresponding to the anomalous structure (if one exists) receive samples generated by a distribution q, which is different from a distribution p generating samples for other nodes. If an anomalous structure does not exist, all nodes receive samples generated by p. It is assumed that the distributions p and q are arbitrary and unknown. The goal is to design statistically consistent tests with probability of errors converging to zero as the network size becomes asymptotically large. Kernel-based tests are proposed based on maximum mean discrepancy that measures the distance between mean embeddings of distributions into a reproducing kernel Hilbert space. Detection of an anomalous interval over a line network is first studied. Sufficient conditions on minimum and maximum sizes of candidate anomalous intervals are characterized in order to guarantee the proposed test to be consistent. It is also shown that certain necessary conditions must hold to guarantee any test to be universally consistent. Comparison of sufficient and necessary conditions yields that the proposed test is order-level optimal and nearly optimal respectively in terms of minimum and maximum sizes of candidate anomalous intervals. Generalization of the results to other networks is further developed. Numerical results are provided to demonstrate the performance of the proposed tests.

  20. Contribution to the Nonparametric Estimation of the Density of the Regression Errors (Doctoral Thesis)

    CERN Document Server

    LSTA, Rawane Samb

    2010-01-01

    This thesis deals with the nonparametric estimation of density f of the regression error term E of the model Y=m(X)+E, assuming its independence with the covariate X. The difficulty linked to this study is the fact that the regression error E is not observed. In a such setup, it would be unwise, for estimating f, to use a conditional approach based upon the probability distribution function of Y given X. Indeed, this approach is affected by the curse of dimensionality, so that the resulting estimator of the residual term E would have considerably a slow rate of convergence if the dimension of X is very high. Two approaches are proposed in this thesis to avoid the curse of dimensionality. The first approach uses the estimated residuals, while the second integrates a nonparametric conditional density estimator of Y given X. If proceeding so can circumvent the curse of dimensionality, a challenging issue is to evaluate the impact of the estimated residuals on the final estimator of the density f. We will also at...

  1. Directional Variance Adjustment: improving covariance estimates for high-dimensional portfolio optimization

    OpenAIRE

    Daniel Bartz; Kerr Hatrick; Hesse, Christian W.; Klaus-Robert M\\"uller; Steven Lemm

    2011-01-01

    Robust and reliable covariance estimates play a decisive role in financial and many other applications. An important class of estimators is based on Factor models. Here, we show by extensive Monte Carlo simulations that covariance matrices derived from the statistical Factor Analysis model exhibit a systematic error, which is similar to the well-known systematic error of the spectrum of the sample covariance matrix. Moreover, we introduce the Directional Variance Adjustment (DVA) algorithm, w...

  2. Numerical Differentiation Methods for Computing Error Covariance Matrices in Item Response Theory Modeling: An Evaluation and a New Proposal

    Science.gov (United States)

    Tian, Wei; Cai, Li; Thissen, David; Xin, Tao

    2013-01-01

    In item response theory (IRT) modeling, the item parameter error covariance matrix plays a critical role in statistical inference procedures. When item parameters are estimated using the EM algorithm, the parameter error covariance matrix is not an automatic by-product of item calibration. Cai proposed the use of Supplemented EM algorithm for…

  3. Evaluation of covariance and information performance measures for dynamic object tracking

    Science.gov (United States)

    Yang, Chun; Blasch, Erik; Douville, Phil; Kaplan, Lance; Qiu, Di

    2010-04-01

    In surveillance and reconnaissance applications, dynamic objects are dynamically followed by track filters with sequential measurements. There are two popular implementations of tracking filters: one is the covariance or Kalman filter and the other is the information filter. Evaluation of tracking filters is important in performance optimization not only for tracking filter design but also for resource management. Typically, the information matrix is the inverse of the covariance matrix. The covariance filter-based approaches attempt to minimize the covariance matrix-based scalar indexes whereas the information filter-based methods aim at maximizing the information matrix-based scalar indexes. Such scalar performance measures include the trace, determinant, norms (1-norm, 2-norm, infinite-norm, and Forbenius norm), and eigenstructure of the covariance matrix or the information matrix and their variants. One natural question to ask is if the scalar track filter performance measures applied to the covariance matrix are equivalent to those applied to the information matrix? In this paper we show most of the scalar performance indexes are equivalent yet some are not. As a result, the indexes if used improperly would provide an "optimized" solution but in the wrong sense relative to track accuracy. The simulation indicated that all the seven indexes were successful when applied to the covariance matrix. However, the failed indexes for the information filter include the trace and the four norms (as defined in MATLAB) of the information matrix. Nevertheless, the determinant and the properly selected eigenvalue of the information matrix were successful to select the optimal sensor update configuration. The evaluation analysis of track measures can serve as a guideline to determine the suitability of performance measures for tracking filter design and resource management.

  4. msSurv: An R Package for Nonparametric Estimation of Multistate Models

    Directory of Open Access Journals (Sweden)

    Nicole Ferguson

    2012-09-01

    Full Text Available We present an R package, msSurv, to calculate the marginal (that is, not conditional on any covariates state occupation probabilities, the state entry and exit time distributions, and the marginal integrated transition hazard for a general, possibly non-Markov, multistate system under left-truncation and right censoring. For a Markov model, msSurv also calculates and returns the transition probability matrix between any two states. Dependent censoring is handled via modeling the censoring hazard through observable covariates. Pointwise confidence intervals for the above mentioned quantities are obtained and returned for independent censoring from closed-form variance estimators and for dependent censoring using the bootstrap.

  5. Covariant representations of subproduct systems

    CERN Document Server

    Viselter, Ami

    2010-01-01

    A celebrated theorem of Pimsner states that a covariant representation $T$ of a $C^*$-correspondence $E$ extends to a $C^*$-representation of the Toeplitz algebra of $E$ if and only if $T$ is isometric. This paper is mainly concerned with finding conditions for a covariant representation of a \\emph{subproduct system} to extend to a $C^*$-representation of the Toeplitz algebra. This framework is much more general than the former. We are able to find sufficient conditions, and show that in important special cases, they are also necessary. Further results include the universality of the tensor algebra, dilations of completely contractive covariant representations, Wold decompositions and von Neumann inequalities.

  6. Disintegrating the fly: A mutational perspective on phenotypic integration and covariation.

    Science.gov (United States)

    Haber, Annat; Dworkin, Ian

    2017-01-01

    The structure of environmentally induced phenotypic covariation can influence the effective strength and magnitude of natural selection. Yet our understanding of the factors that contribute to and influence the evolutionary lability of such covariation is poor. Most studies have either examined environmental variation without accounting for covariation, or examined phenotypic and genetic covariation without distinguishing the environmental component. In this study, we examined the effect of mutational perturbations on different properties of environmental covariation, as well as mean shape. We use strains of Drosophila melanogaster bearing well-characterized mutations known to influence wing shape, as well as naturally derived strains, all reared under carefully controlled conditions and with the same genetic background. We find that mean shape changes more freely than the covariance structure, and that different properties of the covariance matrix change independently from each other. The perturbations affect matrix orientation more than they affect matrix eccentricity or total variance. Yet, mutational effects on matrix orientation do not cluster according to the developmental pathway that they target. These results suggest that it might be useful to consider a more general concept of "decanalization," involving all aspects of variation and covariation.

  7. Nonparametric Bayesian drift estimation for multidimensional stochastic differential equations

    NARCIS (Netherlands)

    Gugushvili, S.; Spreij, P.

    2014-01-01

    We consider nonparametric Bayesian estimation of the drift coefficient of a multidimensional stochastic differential equation from discrete-time observations on the solution of this equation. Under suitable regularity conditions, we establish posterior consistency in this context.

  8. Homothetic Efficiency and Test Power: A Non-Parametric Approach

    NARCIS (Netherlands)

    J. Heufer (Jan); P. Hjertstrand (Per)

    2015-01-01

    markdownabstract__Abstract__ We provide a nonparametric revealed preference approach to demand analysis based on homothetic efficiency. Homotheticity is a useful restriction but data rarely satisfies testable conditions. To overcome this we provide a way to estimate homothetic efficiency of

  9. A non-parametric approach to investigating fish population dynamics

    National Research Council Canada - National Science Library

    Cook, R.M; Fryer, R.J

    2001-01-01

    .... Using a non-parametric model for the stock-recruitment relationship it is possible to avoid defining specific functions relating recruitment to stock size while also providing a natural framework to model process error...

  10. Non-parametric approach to the study of phenotypic stability.

    Science.gov (United States)

    Ferreira, D F; Fernandes, S B; Bruzi, A T; Ramalho, M A P

    2016-02-19

    The aim of this study was to undertake the theoretical derivations of non-parametric methods, which use linear regressions based on rank order, for stability analyses. These methods were extension different parametric methods used for stability analyses and the result was compared with a standard non-parametric method. Intensive computational methods (e.g., bootstrap and permutation) were applied, and data from the plant-breeding program of the Biology Department of UFLA (Minas Gerais, Brazil) were used to illustrate and compare the tests. The non-parametric stability methods were effective for the evaluation of phenotypic stability. In the presence of variance heterogeneity, the non-parametric methods exhibited greater power of discrimination when determining the phenotypic stability of genotypes.

  11. Nonparametric Bayesian Modeling for Automated Database Schema Matching

    Energy Technology Data Exchange (ETDEWEB)

    Ferragut, Erik M [ORNL; Laska, Jason A [ORNL

    2015-01-01

    The problem of merging databases arises in many government and commercial applications. Schema matching, a common first step, identifies equivalent fields between databases. We introduce a schema matching framework that builds nonparametric Bayesian models for each field and compares them by computing the probability that a single model could have generated both fields. Our experiments show that our method is more accurate and faster than the existing instance-based matching algorithms in part because of the use of nonparametric Bayesian models.

  12. PV power forecast using a nonparametric PV model

    OpenAIRE

    Almeida, Marcelo Pinho; Perpiñan Lamigueiro, Oscar; Narvarte Fernández, Luis

    2015-01-01

    Forecasting the AC power output of a PV plant accurately is important both for plant owners and electric system operators. Two main categories of PV modeling are available: the parametric and the nonparametric. In this paper, a methodology using a nonparametric PV model is proposed, using as inputs several forecasts of meteorological variables from a Numerical Weather Forecast model, and actual AC power measurements of PV plants. The methodology was built upon the R environment and uses Quant...

  13. General covariance in computational electrodynamics

    DEFF Research Database (Denmark)

    Shyroki, Dzmitry; Lægsgaard, Jesper; Bang, Ole;

    2007-01-01

    We advocate the generally covariant formulation of Maxwell equations as underpinning some recent advances in computational electrodynamics—in the dimensionality reduction for separable structures; in mesh truncation for finite-difference computations; and in adaptive coordinate mapping as opposed...

  14. Nonparametric Cooperative Spectrum Sensing Algorithm Based on Friedman Test%基于Friedman检验的非参数协作频谱感知方法

    Institute of Scientific and Technical Information of China (English)

    王炯滔; 金明; 李有明; 高洋

    2014-01-01

    Covariance matrix based spectrum sensing encounters performance degradation when there the antenna correlation is low. To overcome this drawback, a nonparametric cooperative spectrum sensing algorithm based on Friedman test is proposed. Distributed sensors possess the effect of space diversity, so that the signal power among the sensors at the same time may not be completely equal. Based on this feature, the spectrum sensing is realized by comparing signal powers among the sensors. For the nonparametric approach is adopted, the proposed algorithm is robust to noise uncertainty and is suitable for noise of any statistical distribution. The theoretical expression of decision threshold is also derived, which shows that the decision threshold has no relationship with the sample number. As a result, the threshold does not need to be reset when the sample number changes. Simulation results demonstrate the effectiveness of the algorithm.%协方差矩阵频谱感知方法在天线相关性低时感知性能较差,该文针对这一问题提出一种基于Friedman检验的非参数协作频谱感知方法。分布式放置的感知节点具有空间分集的特性,因此在同一时刻感知节点上的信号功率不完全相同。利用这一特点,提出通过比较各感知节点的信号功率水平来实现频谱感知。由于采用了非参数化表示,该方法对噪声不确定性稳定,且适用于任意统计分布的噪声。另外,推导了所提方法判决门限的理论表达式,结果显示判决门限与采样点数无关,因此在采样点数变化的情况下无需重新设置判决门限。仿真结果验证了上述理论分析的有效性。

  15. Sparse Inverse Covariance Estimation via an Adaptive Gradient-Based Method

    OpenAIRE

    Sra, Suvrit; Kim, Dongmin

    2011-01-01

    We study the problem of estimating from data, a sparse approximation to the inverse covariance matrix. Estimating a sparsity constrained inverse covariance matrix is a key component in Gaussian graphical model learning, but one that is numerically very challenging. We address this challenge by developing a new adaptive gradient-based method that carefully combines gradient information with an adaptive step-scaling strategy, which results in a scalable, highly competitive method. Our algorithm...

  16. High-dimensional Sparse Inverse Covariance Estimation using Greedy Methods

    CERN Document Server

    Johnson, Christopher C; Ravikumar, Pradeep

    2011-01-01

    In this paper we consider the task of estimating the non-zero pattern of the sparse inverse covariance matrix of a zero-mean Gaussian random vector from a set of iid samples. Note that this is also equivalent to recovering the underlying graph structure of a sparse Gaussian Markov Random Field (GMRF). We present two novel greedy approaches to solving this problem. The first estimates the non-zero covariates of the overall inverse covariance matrix using a series of global forward and backward greedy steps. The second estimates the neighborhood of each node in the graph separately, again using greedy forward and backward steps, and combines the intermediate neighborhoods to form an overall estimate. The principal contribution of this paper is a rigorous analysis of the sparsistency, or consistency in recovering the sparsity pattern of the inverse covariance matrix. Surprisingly, we show that both the local and global greedy methods learn the full structure of the model with high probability given just $O(d\\log...

  17. Positive semidefinite integrated covariance estimation, factorizations and asynchronicity

    DEFF Research Database (Denmark)

    Sauri, Orimar; Lunde, Asger; Laurent, Sébastien;

    2017-01-01

    An estimator of the ex-post covariation of log-prices under asynchronicity and microstructure noise is proposed. It uses the Cholesky factorization of the covariance matrix in order to exploit the heterogeneity in trading intensities to estimate the different parameters sequentially with as many...... observations as possible. The estimator is positive semidefinite by construction. We derive asymptotic results and confirm their good finite sample properties by means of a Monte Carlo simulation. In the application we forecast portfolio Value-at-Risk and sector risk exposures for a portfolio of 52 stocks. We...

  18. A Blind Detection Algorithm Utilizing Statistical Covariance in Cognitive Radio

    Directory of Open Access Journals (Sweden)

    Yingxue Li

    2012-11-01

    Full Text Available As the expression of performance parameters are obtained using asymptotic method in most blind covariance detection algorithm, the paper presented a new blind detection algorithm using cholesky factorization. Utilizing random matrix theory, we derived the performance parameters using non-asymptotic method. The proposed method overcomes the noise uncertainty problem and performs well without any information about the channel, primary user and noise. Numerical simulation results demonstrate that the performance parameters expressions are correct and the new detector outperforms the other blind covariance detectors.

  19. A scale invariant covariance structure on jet space

    DEFF Research Database (Denmark)

    Pedersen, Kim Steenstrup; Loog, Marco; Markussen, Bo

    2005-01-01

    This paper considers scale invariance of statistical image models. We study statistical scale invariance of the covariance structure of jet space under scale space blurring and derive the necessary structure and conditions of the jet covariance matrix in order for it to be scale invariant. As part...... of the derivation, we introduce a blurring operator At that acts on jet space contrary to doing spatial filtering and a scaling operator Ss. The stochastic Brownian image model is an example of a class of functions which are scale invariant with respect to the operators At and Ss. This paper also includes empirical...

  20. An Econometric Analysis of Modulated Realised Covariance, Regression and Correlation in Noisy Diffusion Models

    DEFF Research Database (Denmark)

    Kinnebrock, Silja; Podolskij, Mark

    This paper introduces a new estimator to measure the ex-post covariation between high-frequency financial time series under market microstructure noise. We provide an asymptotic limit theory (including feasible central limit theorems) for standard methods such as regression, correlation analysis...... and covariance, for which we obtain the optimal rate of convergence. We demonstrate some positive semidefinite estimators of the covariation and construct a positive semidefinite estimator of the conditional covariance matrix in the central limit theorem. Furthermore, we indicate how the assumptions on the noise...

  1. An Econometric Analysis of Modulated Realised Covariance, Regression and Correlation in Noisy Diffusion Models

    DEFF Research Database (Denmark)

    Kinnebrock, Silja; Podolskij, Mark

    and covariance, for which we obtain the optimal rate of convergence. We demonstrate some positive semidefinite estimators of the covariation and construct a positive semidefinite estimator of the conditional covariance matrix in the central limit theorem. Furthermore, we indicate how the assumptions on the noise......This paper introduces a new estimator to measure the ex-post covariation between high-frequency financial time series under market microstructure noise. We provide an asymptotic limit theory (including feasible central limit theorems) for standard methods such as regression, correlation analysis...

  2. Non-parametric Bayesian mixture of sparse regressions with application towards feature selection for statistical downscaling

    Directory of Open Access Journals (Sweden)

    D. Das

    2014-04-01

    Full Text Available Climate projections simulated by Global Climate Models (GCM are often used for assessing the impacts of climate change. However, the relatively coarse resolutions of GCM outputs often precludes their application towards accurately assessing the effects of climate change on finer regional scale phenomena. Downscaling of climate variables from coarser to finer regional scales using statistical methods are often performed for regional climate projections. Statistical downscaling (SD is based on the understanding that the regional climate is influenced by two factors – the large scale climatic state and the regional or local features. A transfer function approach of SD involves learning a regression model which relates these features (predictors to a climatic variable of interest (predictand based on the past observations. However, often a single regression model is not sufficient to describe complex dynamic relationships between the predictors and predictand. We focus on the covariate selection part of the transfer function approach and propose a nonparametric Bayesian mixture of sparse regression models based on Dirichlet Process (DP, for simultaneous clustering and discovery of covariates within the clusters while automatically finding the number of clusters. Sparse linear models are parsimonious and hence relatively more generalizable than non-sparse alternatives, and lends to domain relevant interpretation. Applications to synthetic data demonstrate the value of the new approach and preliminary results related to feature selection for statistical downscaling shows our method can lead to new insights.

  3. Calcul Stochastique Covariant à Sauts & Calcul Stochastique à Sauts Covariants

    OpenAIRE

    Maillard-Teyssier, Laurence

    2003-01-01

    We propose a stochastic covariant calculus forcàdlàg semimartingales in the tangent bundle $TM$ over a manifold $M$. A connection on $M$ allows us to define an intrinsic derivative ofa $C^1$ curve $(Y_t)$ in $TM$, the covariantderivative. More precisely, it is the derivative of$(Y_t)$ seen in a frame moving parallelly along its projection curve$(x_t)$ on $M$. With the transfer principle, Norris defined thestochastic covariant integration along a continuous semimartingale in$TM$. We describe t...

  4. Covariate-free and Covariate-dependent Reliability.

    Science.gov (United States)

    Bentler, Peter M

    2016-12-01

    Classical test theory reliability coefficients are said to be population specific. Reliability generalization, a meta-analysis method, is the main procedure for evaluating the stability of reliability coefficients across populations. A new approach is developed to evaluate the degree of invariance of reliability coefficients to population characteristics. Factor or common variance of a reliability measure is partitioned into parts that are, and are not, influenced by control variables, resulting in a partition of reliability into a covariate-dependent and a covariate-free part. The approach can be implemented in a single sample and can be applied to a variety of reliability coefficients.

  5. PENERAPAN METODE LEAST MEDIAN SQUARE-MINIMUM COVARIANCE DETERMINANT (LMS-MCD DALAM REGRESI KOMPONEN UTAMA

    Directory of Open Access Journals (Sweden)

    I PUTU EKA IRAWAN

    2014-01-01

    Full Text Available Principal Component Regression is a method to overcome multicollinearity techniques by combining principal component analysis with regression analysis. The calculation of classical principal component analysis is based on the regular covariance matrix. The covariance matrix is optimal if the data originated from a multivariate normal distribution, but is very sensitive to the presence of outliers. Alternatives are used to overcome this problem the method of Least Median Square-Minimum Covariance Determinant (LMS-MCD. The purpose of this research is to conduct a comparison between Principal Component Regression (RKU and Method of Least Median Square - Minimum Covariance Determinant (LMS-MCD in dealing with outliers. In this study, Method of Least Median Square - Minimum Covariance Determinant (LMS-MCD has a bias and mean square error (MSE is smaller than the parameter RKU. Based on the difference of parameter estimators, still have a test that has a difference of parameter estimators method LMS-MCD greater than RKU method.

  6. Matrix algebra for higher order moments

    NARCIS (Netherlands)

    Meijer, Erik

    2005-01-01

    A large part of statistics is devoted to the estimation of models from the sample covariance matrix. The development of the statistical theory and estimators has been greatly facilitated by the introduction of special matrices, such as the commutation matrix and the duplication matrix, and the corre

  7. Galaxy-galaxy lensing estimators and their covariance properties

    CERN Document Server

    Singh, Sukhdeep; Seljak, Uroš; Slosar, Anže; Gonzalez, Jose Vazquez

    2016-01-01

    We study the covariance properties of real space correlation function estimators -- primarily galaxy-shear correlations, or galaxy-galaxy lensing -- using SDSS data for both shear catalogs and lenses (specifically the BOSS LOWZ sample). Using mock catalogs of lenses and sources, we disentangle the various contributions to the covariance matrix and compare them with a simple analytical model. We show that not subtracting the lensing measurement around random points from the measurement around the lens sample is equivalent to performing the measurement using the density field instead of the over-density field, and that this leads to a significant error increase due to an additional term in the covariance. Therefore, this subtraction should be performed regardless of its beneficial effects on systematics. Comparing the error estimates from data and mocks for estimators that involve the over-density, we find that the errors are dominated by the shape noise and lens clustering, that empirically estimated covarianc...

  8. Asymptotic theory of nonparametric regression estimates with censored data

    Institute of Scientific and Technical Information of China (English)

    2000-01-01

    For regression analysis, some useful information may have been lost when the responses are right censored. To estimate nonparametric functions, several estimates based on censored data have been proposed and their consistency and convergence rates have been studied in literature, but the optimal rates of global convergence have not been obtained yet. Because of the possible information loss, one may think that it is impossible for an estimate based on censored data to achieve the optimal rates of global convergence for nonparametric regression, which were established by Stone based on complete data. This paper constructs a regression spline estimate of a general nonparametric regression function based on right_censored response data, and proves, under some regularity conditions, that this estimate achieves the optimal rates of global convergence for nonparametric regression. Since the parameters for the nonparametric regression estimate have to be chosen based on a data driven criterion, we also obtain the asymptotic optimality of AIC, AICC, GCV, Cp and FPE criteria in the process of selecting the parameters.

  9. Rediscovery of Good-Turing estimators via Bayesian nonparametrics.

    Science.gov (United States)

    Favaro, Stefano; Nipoti, Bernardo; Teh, Yee Whye

    2016-03-01

    The problem of estimating discovery probabilities originated in the context of statistical ecology, and in recent years it has become popular due to its frequent appearance in challenging applications arising in genetics, bioinformatics, linguistics, designs of experiments, machine learning, etc. A full range of statistical approaches, parametric and nonparametric as well as frequentist and Bayesian, has been proposed for estimating discovery probabilities. In this article, we investigate the relationships between the celebrated Good-Turing approach, which is a frequentist nonparametric approach developed in the 1940s, and a Bayesian nonparametric approach recently introduced in the literature. Specifically, under the assumption of a two parameter Poisson-Dirichlet prior, we show that Bayesian nonparametric estimators of discovery probabilities are asymptotically equivalent, for a large sample size, to suitably smoothed Good-Turing estimators. As a by-product of this result, we introduce and investigate a methodology for deriving exact and asymptotic credible intervals to be associated with the Bayesian nonparametric estimators of discovery probabilities. The proposed methodology is illustrated through a comprehensive simulation study and the analysis of Expressed Sequence Tags data generated by sequencing a benchmark complementary DNA library.

  10. Comparing parametric and nonparametric regression methods for panel data

    DEFF Research Database (Denmark)

    Czekaj, Tomasz Gerard; Henningsen, Arne

    We investigate and compare the suitability of parametric and non-parametric stochastic regression methods for analysing production technologies and the optimal firm size. Our theoretical analysis shows that the most commonly used functional forms in empirical production analysis, Cobb-Douglas and......We investigate and compare the suitability of parametric and non-parametric stochastic regression methods for analysing production technologies and the optimal firm size. Our theoretical analysis shows that the most commonly used functional forms in empirical production analysis, Cobb......-Douglas and Translog, are unsuitable for analysing the optimal firm size. We show that the Translog functional form implies an implausible linear relationship between the (logarithmic) firm size and the elasticity of scale, where the slope is artificially related to the substitutability between the inputs...... rejects both the Cobb-Douglas and the Translog functional form, while a recently developed nonparametric kernel regression method with a fully nonparametric panel data specification delivers plausible results. On average, the nonparametric regression results are similar to results that are obtained from...

  11. Szekeres models: a covariant approach

    CERN Document Server

    Apostolopoulos, Pantelis S

    2016-01-01

    We exploit the 1+1+2 formalism to covariantly describe the inhomogeneous and anisotropic Szekeres models. It is shown that an \\emph{average scale length} can be defined \\emph{covariantly} which satisfies a 2d equation of motion driven from the \\emph{effective gravitational mass} (EGM) contained in the dust cloud. The contributions to the EGM are encoded to the energy density of the dust fluid and the free gravitational field $E_{ab}$. In addition the notions of the Apparent and Absolute Apparent Horizons are briefly discussed and we give an alternative gauge-invariant form to define them in terms of the kinematical variables of the spacelike congruences. We argue that the proposed program can be used in order to express the Sachs optical equations in a covariant form and analyze the confrontation of a spatially inhomogeneous irrotational overdense fluid model with the observational data.

  12. Covariance evaluation work at LANL

    Energy Technology Data Exchange (ETDEWEB)

    Kawano, Toshihiko [Los Alamos National Laboratory; Talou, Patrick [Los Alamos National Laboratory; Young, Phillip [Los Alamos National Laboratory; Hale, Gerald [Los Alamos National Laboratory; Chadwick, M B [Los Alamos National Laboratory; Little, R C [Los Alamos National Laboratory

    2008-01-01

    Los Alamos evaluates covariances for nuclear data library, mainly for actinides above the resonance regions and light elements in the enUre energy range. We also develop techniques to evaluate the covariance data, like Bayesian and least-squares fitting methods, which are important to explore the uncertainty information on different types of physical quantities such as elastic scattering angular distribution, or prompt neutron fission spectra. This paper summarizes our current activities of the covariance evaluation work at LANL, including the actinide and light element data mainly for the criticality safety study and transmutation technology. The Bayesian method based on the Kalman filter technique, which combines uncertainties in the theoretical model and experimental data, is discussed.

  13. Cosmic Censorship Conjecture revisited: Covariantly

    CERN Document Server

    Hamid, Aymen I M; Maharaj, Sunil D

    2014-01-01

    In this paper we study the dynamics of the trapped region using a frame independent semi-tetrad covariant formalism for general Locally Rotationally Symmetric (LRS) class II spacetimes. We covariantly prove some important geometrical results for the apparent horizon, and state the necessary and sufficient conditions for a singularity to be locally naked. These conditions bring out, for the first time in a quantitative and transparent manner, the importance of the Weyl curvature in deforming and delaying the trapped region during continual gravitational collapse, making the central singularity locally visible.

  14. MIMO-radar Waveform Covariance Matrices for High SINR and Low Side-lobe Levels

    KAUST Repository

    Ahmed, Sajid

    2012-12-29

    MIMO-radar has better parametric identifiability but compared to phased-array radar it shows loss in signal-to-noise ratio due to non-coherent processing. To exploit the benefits of both MIMO-radar and phased-array two transmit covariance matrices are found. Both of the covariance matrices yield gain in signal-to-interference-plus-noise ratio (SINR) compared to MIMO-radar and have lower side-lobe levels (SLL)\\'s compared to phased-array and MIMO-radar. Moreover, in contrast to recently introduced phased-MIMO scheme, where each antenna transmit different power, our proposed schemes allows same power transmission from each antenna. The SLL\\'s of the proposed first covariance matrix are higher than the phased-MIMO scheme while the SLL\\'s of the second proposed covariance matrix are lower than the phased-MIMO scheme. The first covariance matrix is generated using an auto-regressive process, which allow us to change the SINR and side lobe levels by changing the auto-regressive parameter, while to generate the second covariance matrix the values of sine function between 0 and $\\\\pi$ with the step size of $\\\\pi/n_T$ are used to form a positive-semidefinite Toeplitiz matrix, where $n_T$ is the number of transmit antennas. Simulation results validate our analytical results.

  15. AFCI-2.0 Neutron Cross Section Covariance Library

    Energy Technology Data Exchange (ETDEWEB)

    Herman, M.; Herman, M; Oblozinsky, P.; Mattoon, C.M.; Pigni, M.; Hoblit, S.; Mughabghab, S.F.; Sonzogni, A.; Talou, P.; Chadwick, M.B.; Hale, G.M.; Kahler, A.C.; Kawano, T.; Little, R.C.; Yount, P.G.

    2011-03-01

    The cross section covariance library has been under development by BNL-LANL collaborative effort over the last three years. The project builds on two covariance libraries developed earlier, with considerable input from BNL and LANL. In 2006, international effort under WPEC Subgroup 26 produced BOLNA covariance library by putting together data, often preliminary, from various sources for most important materials for nuclear reactor technology. This was followed in 2007 by collaborative effort of four US national laboratories to produce covariances, often of modest quality - hence the name low-fidelity, for virtually complete set of materials included in ENDF/B-VII.0. The present project is focusing on covariances of 4-5 major reaction channels for 110 materials of importance for power reactors. The work started under Global Nuclear Energy Partnership (GNEP) in 2008, which changed to Advanced Fuel Cycle Initiative (AFCI) in 2009. With the 2011 release the name has changed to the Covariance Multigroup Matrix for Advanced Reactor Applications (COMMARA) version 2.0. The primary purpose of the library is to provide covariances for AFCI data adjustment project, which is focusing on the needs of fast advanced burner reactors. Responsibility of BNL was defined as developing covariances for structural materials and fission products, management of the library and coordination of the work; LANL responsibility was defined as covariances for light nuclei and actinides. The COMMARA-2.0 covariance library has been developed by BNL-LANL collaboration for Advanced Fuel Cycle Initiative applications over the period of three years, 2008-2010. It contains covariances for 110 materials relevant to fast reactor R&D. The library is to be used together with the ENDF/B-VII.0 central values of the latest official release of US files of evaluated neutron cross sections. COMMARA-2.0 library contains neutron cross section covariances for 12 light nuclei (coolants and moderators), 78 structural

  16. AFCI-2.0 Neutron Cross Section Covariance Library

    Energy Technology Data Exchange (ETDEWEB)

    Herman, M.; Herman, M; Oblozinsky, P.; Mattoon, C.M.; Pigni, M.; Hoblit, S.; Mughabghab, S.F.; Sonzogni, A.; Talou, P.; Chadwick, M.B.; Hale, G.M.; Kahler, A.C.; Kawano, T.; Little, R.C.; Yount, P.G.

    2011-03-01

    The cross section covariance library has been under development by BNL-LANL collaborative effort over the last three years. The project builds on two covariance libraries developed earlier, with considerable input from BNL and LANL. In 2006, international effort under WPEC Subgroup 26 produced BOLNA covariance library by putting together data, often preliminary, from various sources for most important materials for nuclear reactor technology. This was followed in 2007 by collaborative effort of four US national laboratories to produce covariances, often of modest quality - hence the name low-fidelity, for virtually complete set of materials included in ENDF/B-VII.0. The present project is focusing on covariances of 4-5 major reaction channels for 110 materials of importance for power reactors. The work started under Global Nuclear Energy Partnership (GNEP) in 2008, which changed to Advanced Fuel Cycle Initiative (AFCI) in 2009. With the 2011 release the name has changed to the Covariance Multigroup Matrix for Advanced Reactor Applications (COMMARA) version 2.0. The primary purpose of the library is to provide covariances for AFCI data adjustment project, which is focusing on the needs of fast advanced burner reactors. Responsibility of BNL was defined as developing covariances for structural materials and fission products, management of the library and coordination of the work; LANL responsibility was defined as covariances for light nuclei and actinides. The COMMARA-2.0 covariance library has been developed by BNL-LANL collaboration for Advanced Fuel Cycle Initiative applications over the period of three years, 2008-2010. It contains covariances for 110 materials relevant to fast reactor R&D. The library is to be used together with the ENDF/B-VII.0 central values of the latest official release of US files of evaluated neutron cross sections. COMMARA-2.0 library contains neutron cross section covariances for 12 light nuclei (coolants and moderators), 78 structural

  17. Predicting Market Impact Costs Using Nonparametric Machine Learning Models.

    Directory of Open Access Journals (Sweden)

    Saerom Park

    Full Text Available Market impact cost is the most significant portion of implicit transaction costs that can reduce the overall transaction cost, although it cannot be measured directly. In this paper, we employed the state-of-the-art nonparametric machine learning models: neural networks, Bayesian neural network, Gaussian process, and support vector regression, to predict market impact cost accurately and to provide the predictive model that is versatile in the number of variables. We collected a large amount of real single transaction data of US stock market from Bloomberg Terminal and generated three independent input variables. As a result, most nonparametric machine learning models outperformed a-state-of-the-art benchmark parametric model such as I-star model in four error measures. Although these models encounter certain difficulties in separating the permanent and temporary cost directly, nonparametric machine learning models can be good alternatives in reducing transaction costs by considerably improving in prediction performance.

  18. Predicting Market Impact Costs Using Nonparametric Machine Learning Models.

    Science.gov (United States)

    Park, Saerom; Lee, Jaewook; Son, Youngdoo

    2016-01-01

    Market impact cost is the most significant portion of implicit transaction costs that can reduce the overall transaction cost, although it cannot be measured directly. In this paper, we employed the state-of-the-art nonparametric machine learning models: neural networks, Bayesian neural network, Gaussian process, and support vector regression, to predict market impact cost accurately and to provide the predictive model that is versatile in the number of variables. We collected a large amount of real single transaction data of US stock market from Bloomberg Terminal and generated three independent input variables. As a result, most nonparametric machine learning models outperformed a-state-of-the-art benchmark parametric model such as I-star model in four error measures. Although these models encounter certain difficulties in separating the permanent and temporary cost directly, nonparametric machine learning models can be good alternatives in reducing transaction costs by considerably improving in prediction performance.

  19. Comparing parametric and nonparametric regression methods for panel data

    DEFF Research Database (Denmark)

    Czekaj, Tomasz Gerard; Henningsen, Arne

    We investigate and compare the suitability of parametric and non-parametric stochastic regression methods for analysing production technologies and the optimal firm size. Our theoretical analysis shows that the most commonly used functional forms in empirical production analysis, Cobb......-Douglas and Translog, are unsuitable for analysing the optimal firm size. We show that the Translog functional form implies an implausible linear relationship between the (logarithmic) firm size and the elasticity of scale, where the slope is artificially related to the substitutability between the inputs....... The practical applicability of the parametric and non-parametric regression methods is scrutinised and compared by an empirical example: we analyse the production technology and investigate the optimal size of Polish crop farms based on a firm-level balanced panel data set. A nonparametric specification test...

  20. Transformation rule for covariance matrices under Bell-like detections

    CERN Document Server

    Spedalieri, Gaetana; Pirandola, Stefano

    2013-01-01

    Starting from the transformation rule of a covariance matrix under homodyne detections, we can easily derive a formula for the transformation of a covariance matrix of (n+2) bosonic modes under Bell-like detections, where the last two modes are combined in an arbitrary beam splitter (i.e., with arbitrary transmissivity) and then homodyned. This formula can be specialized to describe the standard Bell detection and the heterodyne measurement, which are exploited in many contexts, including protocols of quantum teleportation, entanglement swapping and quantum cryptography. Our general formula can be adopted to study these protocols in the presence of experimental imperfections or asymmetric setups, e.g., deriving from the use of unbalanced beam splitters.

  1. Sparse Inverse Covariance Selection via Alternating Linearization Methods

    CERN Document Server

    Scheinberg, Katya; Goldfarb, Donald

    2010-01-01

    Gaussian graphical models are of great interest in statistical learning. Because the conditional independencies between different nodes correspond to zero entries in the inverse covariance matrix of the Gaussian distribution, one can learn the structure of the graph by estimating a sparse inverse covariance matrix from sample data, by solving a convex maximum likelihood problem with an $\\ell_1$-regularization term. In this paper, we propose a first-order method based on an alternating linearization technique that exploits the problem's special structure; in particular, the subproblems solved in each iteration have closed-form solutions. Moreover, our algorithm obtains an $\\epsilon$-optimal solution in $O(1/\\epsilon)$ iterations. Numerical experiments on both synthetic and real data from gene association networks show that a practical version of this algorithm outperforms other competitive algorithms.

  2. Fast Component Pursuit for Large-Scale Inverse Covariance Estimation.

    Science.gov (United States)

    Han, Lei; Zhang, Yu; Zhang, Tong

    2016-08-01

    The maximum likelihood estimation (MLE) for the Gaussian graphical model, which is also known as the inverse covariance estimation problem, has gained increasing interest recently. Most existing works assume that inverse covariance estimators contain sparse structure and then construct models with the ℓ1 regularization. In this paper, different from existing works, we study the inverse covariance estimation problem from another perspective by efficiently modeling the low-rank structure in the inverse covariance, which is assumed to be a combination of a low-rank part and a diagonal matrix. One motivation for this assumption is that the low-rank structure is common in many applications including the climate and financial analysis, and another one is that such assumption can reduce the computational complexity when computing its inverse. Specifically, we propose an efficient COmponent Pursuit (COP) method to obtain the low-rank part, where each component can be sparse. For optimization, the COP method greedily learns a rank-one component in each iteration by maximizing the log-likelihood. Moreover, the COP algorithm enjoys several appealing properties including the existence of an efficient solution in each iteration and the theoretical guarantee on the convergence of this greedy approach. Experiments on large-scale synthetic and real-world datasets including thousands of millions variables show that the COP method is faster than the state-of-the-art techniques for the inverse covariance estimation problem when achieving comparable log-likelihood on test data.

  3. Nonparametric estimation of a convex bathtub-shaped hazard function.

    Science.gov (United States)

    Jankowski, Hanna K; Wellner, Jon A

    2009-11-01

    In this paper, we study the nonparametric maximum likelihood estimator (MLE) of a convex hazard function. We show that the MLE is consistent and converges at a local rate of n(2/5) at points x(0) where the true hazard function is positive and strictly convex. Moreover, we establish the pointwise asymptotic distribution theory of our estimator under these same assumptions. One notable feature of the nonparametric MLE studied here is that no arbitrary choice of tuning parameter (or complicated data-adaptive selection of the tuning parameter) is required.

  4. Covariant description of isothermic surfaces

    CERN Document Server

    Tafel, Jacek

    2014-01-01

    We present a covariant formulation of the Gauss-Weingarten equations and the Gauss-Mainardi-Codazzi equations for surfaces in 3-dimensional curved spaces. We derive a coordinate invariant condition on the first and second fundamental form which is necessary and sufficient for the surface to be isothermic.

  5. Covariation Neglect among Novice Investors

    Science.gov (United States)

    Hedesstrom, Ted Martin; Svedsater, Henrik; Garling, Tommy

    2006-01-01

    In 4 experiments, undergraduates made hypothetical investment choices. In Experiment 1, participants paid more attention to the volatility of individual assets than to the volatility of aggregated portfolios. The results of Experiment 2 show that most participants diversified even when this increased risk because of covariation between the returns…

  6. Do Time-Varying Covariances, Volatility Comovement and Spillover Matter?

    OpenAIRE

    Lakshmi Balasubramanyan

    2005-01-01

    Financial markets and their respective assets are so intertwined; analyzing any single market in isolation ignores important information. We investigate whether time varying volatility comovement and spillover impact the true variance-covariance matrix under a time-varying correlation set up. Statistically significant volatility spillover and comovement between US, UK and Japan is found. To demonstrate the importance of modelling volatility comovement and spillover, we look at a simple portfo...

  7. Testing and Estimating Shape-Constrained Nonparametric Density and Regression in the Presence of Measurement Error

    KAUST Repository

    Carroll, Raymond J.

    2011-03-01

    In many applications we can expect that, or are interested to know if, a density function or a regression curve satisfies some specific shape constraints. For example, when the explanatory variable, X, represents the value taken by a treatment or dosage, the conditional mean of the response, Y , is often anticipated to be a monotone function of X. Indeed, if this regression mean is not monotone (in the appropriate direction) then the medical or commercial value of the treatment is likely to be significantly curtailed, at least for values of X that lie beyond the point at which monotonicity fails. In the case of a density, common shape constraints include log-concavity and unimodality. If we can correctly guess the shape of a curve, then nonparametric estimators can be improved by taking this information into account. Addressing such problems requires a method for testing the hypothesis that the curve of interest satisfies a shape constraint, and, if the conclusion of the test is positive, a technique for estimating the curve subject to the constraint. Nonparametric methodology for solving these problems already exists, but only in cases where the covariates are observed precisely. However in many problems, data can only be observed with measurement errors, and the methods employed in the error-free case typically do not carry over to this error context. In this paper we develop a novel approach to hypothesis testing and function estimation under shape constraints, which is valid in the context of measurement errors. Our method is based on tilting an estimator of the density or the regression mean until it satisfies the shape constraint, and we take as our test statistic the distance through which it is tilted. Bootstrap methods are used to calibrate the test. The constrained curve estimators that we develop are also based on tilting, and in that context our work has points of contact with methodology in the error-free case.

  8. Application of the LSQR algorithm in non-parametric estimation of aerosol size distribution

    Science.gov (United States)

    He, Zhenzong; Qi, Hong; Lew, Zhongyuan; Ruan, Liming; Tan, Heping; Luo, Kun

    2016-05-01

    Based on the Least Squares QR decomposition (LSQR) algorithm, the aerosol size distribution (ASD) is retrieved in non-parametric approach. The direct problem is solved by the Anomalous Diffraction Approximation (ADA) and the Lambert-Beer Law. An optimal wavelength selection method is developed to improve the retrieval accuracy of the ASD. The proposed optimal wavelength set is selected by the method which can make the measurement signals sensitive to wavelength and decrease the degree of the ill-condition of coefficient matrix of linear systems effectively to enhance the anti-interference ability of retrieval results. Two common kinds of monomodal and bimodal ASDs, log-normal (L-N) and Gamma distributions, are estimated, respectively. Numerical tests show that the LSQR algorithm can be successfully applied to retrieve the ASD with high stability in the presence of random noise and low susceptibility to the shape of distributions. Finally, the experimental measurement ASD over Harbin in China is recovered reasonably. All the results confirm that the LSQR algorithm combined with the optimal wavelength selection method is an effective and reliable technique in non-parametric estimation of ASD.

  9. Pu239 Cross-Section Variations Based on Experimental Uncertainties and Covariances

    Energy Technology Data Exchange (ETDEWEB)

    Sigeti, David Edward [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Williams, Brian J. [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Parsons, D. Kent [Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

    2016-10-18

    Algorithms and software have been developed for producing variations in plutonium 239 neutron cross sections based on experimental uncertainties and covariances. The varied cross- section sets may be produced as random samples from the multi- variate normal distribution defined by an experimental mean vector and covariance matrix, or they may be produced as Latin- Hypercube/Orthogonal-Array samples (based on the same means and covariances) for use in parametrized studies. The variations obey two classes of constraints that are obligatory for cross-section sets and which put related constraints on the mean vector and covariance matrix that detemine the sampling. Because the experimental means and covariances do not obey some of these constraints to sufficient precision, imposing the constraints requires modifying the experimental mean vector and covariance matrix. Modification is done with an algorithm based on linear algebra that minimizes changes to the means and covariances while insuring that the operations that impose the different constraints do not conflict with each other.

  10. Pu239 Cross-Section Variations Based on Experimental Uncertainties and Covariances

    Energy Technology Data Exchange (ETDEWEB)

    Sigeti, David Edward [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Williams, Brian J. [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Parsons, D. Kent [Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

    2016-10-18

    Algorithms and software have been developed for producing variations in plutonium-239 neutron cross sections based on experimental uncertainties and covariances. The varied cross-section sets may be produced as random samples from the multi-variate normal distribution defined by an experimental mean vector and covariance matrix, or they may be produced as Latin-Hypercube/Orthogonal-Array samples (based on the same means and covariances) for use in parametrized studies. The variations obey two classes of constraints that are obligatory for cross-section sets and which put related constraints on the mean vector and covariance matrix that detemine the sampling. Because the experimental means and covariances do not obey some of these constraints to sufficient precision, imposing the constraints requires modifying the experimental mean vector and covariance matrix. Modification is done with an algorithm based on linear algebra that minimizes changes to the means and covariances while insuring that the operations that impose the different constraints do not conflict with each other.

  11. IMPROVED COVARIANCE DRIVEN BLIND SUBSPACE IDENTIFICATION METHOD

    Institute of Scientific and Technical Information of China (English)

    ZHANG Zhiyi; FAN Jiangling; HUA Hongxing

    2006-01-01

    An improved covariance driven subspace identification method is presented to identify the weakly excited modes. In this method, the traditional Hankel matrix is replaced by a reformed one to enhance the identifiability of weak characteristics. The robustness of eigenparameter estimation to noise contamination is reinforced by the improved Hankel matrix. In combination with component energy index (CEI) which indicates the vibration intensity of signal components, an alternative stabilization diagram is adopted to effectively separate spurious and physical modes. Simulation of a vibration system of multiple-degree-of-freedom and experiment of a frame structure subject to wind excitation are presented to demonstrate the improvement of the proposed blind method. The performance of this blind method is assessed in terms of its capability in extracting the weak modes as well as the accuracy of estimated parameters. The results have shown that the proposed blind method gives a better estimation of the weak modes from response signals of small signal to noise ratio (SNR)and gives a reliable separation of spurious and physical estimates.

  12. IIB Matrix Model

    CERN Document Server

    Aoki, H; Kawai, H; Kitazawa, Y; Tada, T; Tsuchiya, A

    1999-01-01

    We review our proposal for a constructive definition of superstring, type IIB matrix model. The IIB matrix model is a manifestly covariant model for space-time and matter which possesses N=2 supersymmetry in ten dimensions. We refine our arguments to reproduce string perturbation theory based on the loop equations. We emphasize that the space-time is dynamically determined from the eigenvalue distributions of the matrices. We also explain how matter, gauge fields and gravitation appear as fluctuations around dynamically determined space-time.

  13. Multi-level restricted maximum likelihood covariance estimation and kriging for large non-gridded spatial datasets

    KAUST Repository

    Castrillon, Julio

    2015-11-10

    We develop a multi-level restricted Gaussian maximum likelihood method for estimating the covariance function parameters and computing the best unbiased predictor. Our approach produces a new set of multi-level contrasts where the deterministic parameters of the model are filtered out thus enabling the estimation of the covariance parameters to be decoupled from the deterministic component. Moreover, the multi-level covariance matrix of the contrasts exhibit fast decay that is dependent on the smoothness of the covariance function. Due to the fast decay of the multi-level covariance matrix coefficients only a small set is computed with a level dependent criterion. We demonstrate our approach on problems of up to 512,000 observations with a Matérn covariance function and highly irregular placements of the observations. In addition, these problems are numerically unstable and hard to solve with traditional methods.

  14. Nonparametric Cointegration Analysis of Fractional Systems With Unknown Integration Orders

    DEFF Research Database (Denmark)

    Nielsen, Morten Ørregaard

    2009-01-01

    In this paper a nonparametric variance ratio testing approach is proposed for determining the number of cointegrating relations in fractionally integrated systems. The test statistic is easily calculated without prior knowledge of the integration order of the data, the strength of the cointegrating...

  15. Non-parametric analysis of rating transition and default data

    DEFF Research Database (Denmark)

    Fledelius, Peter; Lando, David; Perch Nielsen, Jens

    2004-01-01

    We demonstrate the use of non-parametric intensity estimation - including construction of pointwise confidence sets - for analyzing rating transition data. We find that transition intensities away from the class studied here for illustration strongly depend on the direction of the previous move b...... but that this dependence vanishes after 2-3 years....

  16. A non-parametric model for the cosmic velocity field

    NARCIS (Netherlands)

    Branchini, E; Teodoro, L; Frenk, CS; Schmoldt, [No Value; Efstathiou, G; White, SDM; Saunders, W; Sutherland, W; Rowan-Robinson, M; Keeble, O; Tadros, H; Maddox, S; Oliver, S

    1999-01-01

    We present a self-consistent non-parametric model of the local cosmic velocity field derived from the distribution of IRAS galaxies in the PSCz redshift survey. The survey has been analysed using two independent methods, both based on the assumptions of gravitational instability and linear biasing.

  17. Influence of test and person characteristics on nonparametric appropriateness measurement

    NARCIS (Netherlands)

    Meijer, Rob R.; Molenaar, Ivo W.; Sijtsma, Klaas

    1994-01-01

    Appropriateness measurement in nonparametric item response theory modeling is affected by the reliability of the items, the test length, the type of aberrant response behavior, and the percentage of aberrant persons in the group. The percentage of simulees defined a priori as aberrant responders tha

  18. Influence of Test and Person Characteristics on Nonparametric Appropriateness Measurement

    NARCIS (Netherlands)

    Meijer, Rob R; Molenaar, Ivo W; Sijtsma, Klaas

    1994-01-01

    Appropriateness measurement in nonparametric item response theory modeling is affected by the reliability of the items, the test length, the type of aberrant response behavior, and the percentage of aberrant persons in the group. The percentage of simulees defined a priori as aberrant responders tha

  19. Estimation of Spatial Dynamic Nonparametric Durbin Models with Fixed Effects

    Science.gov (United States)

    Qian, Minghui; Hu, Ridong; Chen, Jianwei

    2016-01-01

    Spatial panel data models have been widely studied and applied in both scientific and social science disciplines, especially in the analysis of spatial influence. In this paper, we consider the spatial dynamic nonparametric Durbin model (SDNDM) with fixed effects, which takes the nonlinear factors into account base on the spatial dynamic panel…

  20. Uniform Consistency for Nonparametric Estimators in Null Recurrent Time Series

    DEFF Research Database (Denmark)

    Gao, Jiti; Kanaya, Shin; Li, Degui

    2015-01-01

    This paper establishes uniform consistency results for nonparametric kernel density and regression estimators when time series regressors concerned are nonstationary null recurrent Markov chains. Under suitable regularity conditions, we derive uniform convergence rates of the estimators. Our...... results can be viewed as a nonstationary extension of some well-known uniform consistency results for stationary time series....

  1. Non-parametric Bayesian inference for inhomogeneous Markov point processes

    DEFF Research Database (Denmark)

    Berthelsen, Kasper Klitgaard; Møller, Jesper

    With reference to a specific data set, we consider how to perform a flexible non-parametric Bayesian analysis of an inhomogeneous point pattern modelled by a Markov point process, with a location dependent first order term and pairwise interaction only. A priori we assume that the first order term...

  2. Investigating the cultural patterns of corruption: A nonparametric analysis

    OpenAIRE

    Halkos, George; Tzeremes, Nickolaos

    2011-01-01

    By using a sample of 77 countries our analysis applies several nonparametric techniques in order to reveal the link between national culture and corruption. Based on Hofstede’s cultural dimensions and the corruption perception index, the results reveal that countries with higher levels of corruption tend to have higher power distance and collectivism values in their society.

  3. Coverage Accuracy of Confidence Intervals in Nonparametric Regression

    Institute of Scientific and Technical Information of China (English)

    Song-xi Chen; Yong-song Qin

    2003-01-01

    Point-wise confidence intervals for a nonparametric regression function with random design points are considered. The confidence intervals are those based on the traditional normal approximation and the empirical likelihood. Their coverage accuracy is assessed by developing the Edgeworth expansions for the coverage probabilities. It is shown that the empirical likelihood confidence intervals are Bartlett correctable.

  4. Homothetic Efficiency and Test Power: A Non-Parametric Approach

    NARCIS (Netherlands)

    J. Heufer (Jan); P. Hjertstrand (Per)

    2015-01-01

    markdownabstract__Abstract__ We provide a nonparametric revealed preference approach to demand analysis based on homothetic efficiency. Homotheticity is a useful restriction but data rarely satisfies testable conditions. To overcome this we provide a way to estimate homothetic efficiency of consump

  5. Non-parametric analysis of rating transition and default data

    DEFF Research Database (Denmark)

    Fledelius, Peter; Lando, David; Perch Nielsen, Jens

    2004-01-01

    We demonstrate the use of non-parametric intensity estimation - including construction of pointwise confidence sets - for analyzing rating transition data. We find that transition intensities away from the class studied here for illustration strongly depend on the direction of the previous move...

  6. Background error covariance modelling for convective-scale variational data assimilation

    Science.gov (United States)

    Petrie, R. E.

    An essential component in data assimilation is the background error covariance matrix (B). This matrix regularizes the ill-posed data assimilation problem, describes the confidence of the background state and spreads information. Since the B-matrix is too large to represent explicitly it must be modelled. In variational data assimilation it is essentially a climatological approximation of the true covariances. Such a conventional covariance model additionally relies on the imposition of balance conditions. A toy model which is derived from the Euler equations (by making appropriate simplifications and introducing tuneable parameters) is used as a convective-scale system to investigate these issues. Its behaviour is shown to exhibit large-scale geostrophic and hydrostatic balance while permitting small-scale imbalance. A control variable transform (CVT) approach to modelling the B-matrix where the control variables are taken to be the normal modes (NM) of the linearized model is investigated. This approach is attractive for convective-scale covariance modelling as it allows for unbalanced as well as appropriately balanced relationships. Although the NM-CVT is not applied to a data assimilation problem directly, it is shown to be a viable approach to convective-scale covariance modelling. A new mathematically rigorous method to incorporate flow-dependent error covariances with the otherwise static B-matrix estimate is also proposed. This is an extension to the reduced rank Kalman filter (RRKF) where its Hessian singular vector calculation is replaced by an ensemble estimate of the covariances, and is known as the ensemble RRKF (EnRRKF). Ultimately it is hoped that together the NM-CVT and the EnRRKF would improve the predictability of small-scale features in convective-scale weather forecasting through the relaxation of inappropriate balance and the inclusion of flow-dependent covariances.

  7. Prediction intervals for future BMI values of individual children: a non-parametric approach by quantile boosting.

    Science.gov (United States)

    Mayr, Andreas; Hothorn, Torsten; Fenske, Nora

    2012-01-25

    The construction of prediction intervals (PIs) for future body mass index (BMI) values of individual children based on a recent German birth cohort study with n = 2007 children is problematic for standard parametric approaches, as the BMI distribution in childhood is typically skewed depending on age. We avoid distributional assumptions by directly modelling the borders of PIs by additive quantile regression, estimated by boosting. We point out the concept of conditional coverage to prove the accuracy of PIs. As conditional coverage can hardly be evaluated in practical applications, we conduct a simulation study before fitting child- and covariate-specific PIs for future BMI values and BMI patterns for the present data. The results of our simulation study suggest that PIs fitted by quantile boosting cover future observations with the predefined coverage probability and outperform the benchmark approach. For the prediction of future BMI values, quantile boosting automatically selects informative covariates and adapts to the age-specific skewness of the BMI distribution. The lengths of the estimated PIs are child-specific and increase, as expected, with the age of the child. Quantile boosting is a promising approach to construct PIs with correct conditional coverage in a non-parametric way. It is in particular suitable for the prediction of BMI patterns depending on covariates, since it provides an interpretable predictor structure, inherent variable selection properties and can even account for longitudinal data structures.

  8. Discrete Symmetries in Covariant LQG

    CERN Document Server

    Rovelli, Carlo

    2012-01-01

    We study time-reversal and parity ---on the physical manifold and in internal space--- in covariant loop gravity. We consider a minor modification of the Holst action which makes it transform coherently under such transformations. The classical theory is not affected but the quantum theory is slightly different. In particular, the simplicity constraints are slightly modified and this restricts orientation flips in a spinfoam to occur only across degenerate regions, thus reducing the sources of potential divergences.

  9. Phenotypic covariance at species’ borders

    Science.gov (United States)

    2013-01-01

    Background Understanding the evolution of species limits is important in ecology, evolution, and conservation biology. Despite its likely importance in the evolution of these limits, little is known about phenotypic covariance in geographically marginal populations, and the degree to which it constrains, or facilitates, responses to selection. We investigated phenotypic covariance in morphological traits at species’ borders by comparing phenotypic covariance matrices (P), including the degree of shared structure, the distribution of strengths of pair-wise correlations between traits, the degree of morphological integration of traits, and the ranks of matricies, between central and marginal populations of three species-pairs of coral reef fishes. Results Greater structural differences in P were observed between populations close to range margins and conspecific populations toward range centres, than between pairs of conspecific populations that were both more centrally located within their ranges. Approximately 80% of all pair-wise trait correlations within populations were greater in the north, but these differences were unrelated to the position of the sampled population with respect to the geographic range of the species. Conclusions Neither the degree of morphological integration, nor ranks of P, indicated greater evolutionary constraint at range edges. Characteristics of P observed here provide no support for constraint contributing to the formation of these species’ borders, but may instead reflect structural change in P caused by selection or drift, and their potential to evolve in the future. PMID:23714580

  10. COVARIANCE CORRECTION FOR ESTIMATING GROUNDWATER ...

    African Journals Online (AJOL)

    2015-01-15

    Jan 15, 2015 ... and V, which is known as observation noise, is a Gaussian process vector with ... where Kalman gain matrix (K ) which is the weighting matrix for ... The most widely used correlation function is compactly supported 5th order ...

  11. Nonparametric Estimation of Mean and Variance and Pricing of Securities Nonparametric Estimation of Mean and Variance and Pricing of Sec

    Directory of Open Access Journals (Sweden)

    Akhtar R. Siddique

    2000-03-01

    Full Text Available This paper develops a filtering-based framework of non-parametric estimation of parameters of a diffusion process from the conditional moments of discrete observations of the process. This method is implemented for interest rate data in the Eurodollar and long term bond markets. The resulting estimates are then used to form non-parametric univariate and bivariate interest rate models and compute prices for the short term Eurodollar interest rate futures options and long term discount bonds. The bivariate model produces prices substantially closer to the market prices. This paper develops a filtering-based framework of non-parametric estimation of parameters of a diffusion process from the conditional moments of discrete observations of the process. This method is implemented for interest rate data in the Eurodollar and long term bond markets. The resulting estimates are then used to form non-parametric univariate and bivariate interest rate models and compute prices for the short term Eurodollar interest rate futures options and long term discount bonds. The bivariate model produces prices substantially closer to the market prices.

  12. Examination of various roles for covariance matrices in the development, evaluation, and application of nuclear data

    Energy Technology Data Exchange (ETDEWEB)

    Smith, D.L.

    1988-01-01

    The last decade has been a period of rapid development in the implementation of covariance-matrix methodology in nuclear data research. This paper offers some perspective on the progress which has been made, on some of the unresolved problems, and on the potential yet to be realized. These discussions address a variety of issues related to the development of nuclear data. Topics examined are: the importance of designing and conducting experiments so that error information is conveniently generated; the procedures for identifying error sources and quantifying their magnitudes and correlations; the combination of errors; the importance of consistent and well-characterized measurement standards; the role of covariances in data parameterization (fitting); the estimation of covariances for values calculated from mathematical models; the identification of abnormalities in covariance matrices and the analysis of their consequences; the problems encountered in representing covariance information in evaluated files; the role of covariances in the weighting of diverse data sets; the comparison of various evaluations; the influence of primary-data covariance in the analysis of covariances for derived quantities (sensitivity); and the role of covariances in the merging of the diverse nuclear data information. 226 refs., 2 tabs.

  13. Competing risks and time-dependent covariates

    DEFF Research Database (Denmark)

    Cortese, Giuliana; Andersen, Per K

    2010-01-01

    Time-dependent covariates are frequently encountered in regression analysis for event history data and competing risks. They are often essential predictors, which cannot be substituted by time-fixed covariates. This study briefly recalls the different types of time-dependent covariates...

  14. Variations of cosmic large-scale structure covariance matrices across parameter space

    CERN Document Server

    Reischke, Robert; Schäfer, Björn Malte

    2016-01-01

    The likelihood function for cosmological parameters, given by e.g. weak lensing shear measurements, depends on contributions to the covariance induced by the nonlinear evolution of the cosmic web. As nonlinear clustering to date has only been described by numerical $N$-body simulations in a reliable and sufficiently precise way, the necessary computational costs for estimating those covariances at different points in parameter space are tremendous. In this work we describe the change of the matter covariance and of the weak lensing covariance matrix as a function of cosmological parameters by constructing a suitable basis, where we model the contribution to the covariance from nonlinear structure formation using Eulerian perturbation theory at third order. We show that our formalism is capable of dealing with large matrices and reproduces expected degeneracies and scaling with cosmological parameters in a reliable way. Comparing our analytical results to numerical simulations we find that the method describes...

  15. Survival trees for left-truncated and right-censored data, with application to time-varying covariate data.

    Science.gov (United States)

    Fu, Wei; Simonoff, Jeffrey S

    2016-12-26

    SUMMARYTree methods (recursive partitioning) are a popular class of nonparametric methods for analyzing data. One extension of the basic tree methodology is the survival tree, which applies recursive partitioning to censored survival data. There are several existing survival tree methods in the literature, which are mainly designed for right-censored data. We propose two new survival trees for left-truncated and right-censored (LTRC) data, which can be seen as a generalization of the traditional survival tree for right-censored data. Further, we show that such trees can be used to analyze survival data with time-varying covariates, essentially building a time-varying covariates survival tree. Implementation of the methods is easy, and simulations and real data analysis results show that the proposed methods work well for LTRC data and survival data with time-varying covariates, respectively.

  16. Covariant constraints for generic massive gravity and analysis of its characteristics

    CERN Document Server

    Deser, S; Waldron, A; Zahariade, G

    2014-01-01

    We perform a covariant constraint analysis of massive gravity valid for its entire parameter space, demonstrating that the model generically propagates five degrees of freedom; this is also verified by a new and streamlined Hamiltonian description. The constraint's covariant expression permits computation of the model's caustics. Although new features such as the dynamical Riemann tensor appear in the characteristic matrix, the model still exhibits the pathologies uncovered in earlier work: superluminality and likely acausalities.

  17. Genome-Wide Scan for Adaptive Divergence and Association with Population-Specific Covariates

    OpenAIRE

    Gautier, Mathieu

    2015-01-01

    In population genomics studies, accounting for the neutral covariance structure across population allele frequencies is critical to improve the robustness of genome-wide scan approaches. Elaborating on the BayEnv model, this study investigates several modeling extensions (i) to improve the estimation accuracy of the population covariance matrix and all the related measures, (ii) to identify significantly overly differentiated SNPs based on a calibration procedure of the XtX statistics, and (i...

  18. Covariance Evaluation Methodology for Neutron Cross Sections

    Energy Technology Data Exchange (ETDEWEB)

    Herman,M.; Arcilla, R.; Mattoon, C.M.; Mughabghab, S.F.; Oblozinsky, P.; Pigni, M.; Pritychenko, b.; Songzoni, A.A.

    2008-09-01

    We present the NNDC-BNL methodology for estimating neutron cross section covariances in thermal, resolved resonance, unresolved resonance and fast neutron regions. The three key elements of the methodology are Atlas of Neutron Resonances, nuclear reaction code EMPIRE, and the Bayesian code implementing Kalman filter concept. The covariance data processing, visualization and distribution capabilities are integral components of the NNDC methodology. We illustrate its application on examples including relatively detailed evaluation of covariances for two individual nuclei and massive production of simple covariance estimates for 307 materials. Certain peculiarities regarding evaluation of covariances for resolved resonances and the consistency between resonance parameter uncertainties and thermal cross section uncertainties are also discussed.

  19. Useful and little-known applications of the Least Square Method and some consequences of covariances

    Science.gov (United States)

    Helene, Otaviano; Mariano, Leandro; Guimarães-Filho, Zwinglio

    2016-10-01

    Covariances are as important as variances when dealing with experimental data and they must be considered in fitting procedures and adjustments in order to preserve the statistical properties of the adjusted quantities. In this paper, we apply the Least Square Method in matrix form to several simple problems in order to evaluate the consequences of covariances in the fitting procedure. Among the examples, we demonstrate how a measurement of a physical quantity can change the adopted value of all other covariant quantities and how a new single point (x , y) improves the parameters of a previously adjusted straight-line.

  20. International Conference on Robust Rank-Based and Nonparametric Methods

    CERN Document Server

    McKean, Joseph

    2016-01-01

    The contributors to this volume include many of the distinguished researchers in this area. Many of these scholars have collaborated with Joseph McKean to develop underlying theory for these methods, obtain small sample corrections, and develop efficient algorithms for their computation. The papers cover the scope of the area, including robust nonparametric rank-based procedures through Bayesian and big data rank-based analyses. Areas of application include biostatistics and spatial areas. Over the last 30 years, robust rank-based and nonparametric methods have developed considerably. These procedures generalize traditional Wilcoxon-type methods for one- and two-sample location problems. Research into these procedures has culminated in complete analyses for many of the models used in practice including linear, generalized linear, mixed, and nonlinear models. Settings are both multivariate and univariate. With the development of R packages in these areas, computation of these procedures is easily shared with r...

  1. Nonparametric instrumental regression with non-convex constraints

    Science.gov (United States)

    Grasmair, M.; Scherzer, O.; Vanhems, A.

    2013-03-01

    This paper considers the nonparametric regression model with an additive error that is dependent on the explanatory variables. As is common in empirical studies in epidemiology and economics, it also supposes that valid instrumental variables are observed. A classical example in microeconomics considers the consumer demand function as a function of the price of goods and the income, both variables often considered as endogenous. In this framework, the economic theory also imposes shape restrictions on the demand function, such as integrability conditions. Motivated by this illustration in microeconomics, we study an estimator of a nonparametric constrained regression function using instrumental variables by means of Tikhonov regularization. We derive rates of convergence for the regularized model both in a deterministic and stochastic setting under the assumption that the true regression function satisfies a projected source condition including, because of the non-convexity of the imposed constraints, an additional smallness condition.

  2. Combined parametric-nonparametric identification of block-oriented systems

    CERN Document Server

    Mzyk, Grzegorz

    2014-01-01

    This book considers a problem of block-oriented nonlinear dynamic system identification in the presence of random disturbances. This class of systems includes various interconnections of linear dynamic blocks and static nonlinear elements, e.g., Hammerstein system, Wiener system, Wiener-Hammerstein ("sandwich") system and additive NARMAX systems with feedback. Interconnecting signals are not accessible for measurement. The combined parametric-nonparametric algorithms, proposed in the book, can be selected dependently on the prior knowledge of the system and signals. Most of them are based on the decomposition of the complex system identification task into simpler local sub-problems by using non-parametric (kernel or orthogonal) regression estimation. In the parametric stage, the generalized least squares or the instrumental variables technique is commonly applied to cope with correlated excitations. Limit properties of the algorithms have been shown analytically and illustrated in simple experiments.

  3. Estimation of Stochastic Volatility Models by Nonparametric Filtering

    DEFF Research Database (Denmark)

    Kanaya, Shin; Kristensen, Dennis

    2016-01-01

    /estimated volatility process replacing the latent process. Our estimation strategy is applicable to both parametric and nonparametric stochastic volatility models, and can handle both jumps and market microstructure noise. The resulting estimators of the stochastic volatility model will carry additional biases......A two-step estimation method of stochastic volatility models is proposed: In the first step, we nonparametrically estimate the (unobserved) instantaneous volatility process. In the second step, standard estimation methods for fully observed diffusion processes are employed, but with the filtered...... and variances due to the first-step estimation, but under regularity conditions we show that these vanish asymptotically and our estimators inherit the asymptotic properties of the infeasible estimators based on observations of the volatility process. A simulation study examines the finite-sample properties...

  4. Nonparametric Regression Estimation for Multivariate Null Recurrent Processes

    Directory of Open Access Journals (Sweden)

    Biqing Cai

    2015-04-01

    Full Text Available This paper discusses nonparametric kernel regression with the regressor being a \\(d\\-dimensional \\(\\beta\\-null recurrent process in presence of conditional heteroscedasticity. We show that the mean function estimator is consistent with convergence rate \\(\\sqrt{n(Th^{d}}\\, where \\(n(T\\ is the number of regenerations for a \\(\\beta\\-null recurrent process and the limiting distribution (with proper normalization is normal. Furthermore, we show that the two-step estimator for the volatility function is consistent. The finite sample performance of the estimate is quite reasonable when the leave-one-out cross validation method is used for bandwidth selection. We apply the proposed method to study the relationship of Federal funds rate with 3-month and 5-year T-bill rates and discover the existence of nonlinearity of the relationship. Furthermore, the in-sample and out-of-sample performance of the nonparametric model is far better than the linear model.

  5. Using non-parametric methods in econometric production analysis

    DEFF Research Database (Denmark)

    Czekaj, Tomasz Gerard; Henningsen, Arne

    -Douglas function nor the Translog function are consistent with the “true” relationship between the inputs and the output in our data set. We solve this problem by using non-parametric regression. This approach delivers reasonable results, which are on average not too different from the results of the parametric......Econometric estimation of production functions is one of the most common methods in applied economic production analysis. These studies usually apply parametric estimation techniques, which obligate the researcher to specify the functional form of the production function. Most often, the Cobb...... results—including measures that are of interest of applied economists, such as elasticities. Therefore, we propose to use nonparametric econometric methods. First, they can be applied to verify the functional form used in parametric estimations of production functions. Second, they can be directly used...

  6. Right-Censored Nonparametric Regression: A Comparative Simulation Study

    Directory of Open Access Journals (Sweden)

    Dursun Aydın

    2016-11-01

    Full Text Available This paper introduces the operating of the selection criteria for right-censored nonparametric regression using smoothing spline. In order to transform the response variable into a variable that contains the right-censorship, we used the KaplanMeier weights proposed by [1], and [2]. The major problem in smoothing spline method is to determine a smoothing parameter to obtain nonparametric estimates of the regression function. In this study, the mentioned parameter is chosen based on censored data by means of the criteria such as improved Akaike information criterion (AICc, Bayesian (or Schwarz information criterion (BIC and generalized crossvalidation (GCV. For this purpose, a Monte-Carlo simulation study is carried out to illustrate which selection criterion gives the best estimation for censored data.

  7. Using non-parametric methods in econometric production analysis

    DEFF Research Database (Denmark)

    Czekaj, Tomasz Gerard; Henningsen, Arne

    2012-01-01

    by investigating the relationship between the elasticity of scale and the farm size. We use a balanced panel data set of 371~specialised crop farms for the years 2004-2007. A non-parametric specification test shows that neither the Cobb-Douglas function nor the Translog function are consistent with the "true......Econometric estimation of production functions is one of the most common methods in applied economic production analysis. These studies usually apply parametric estimation techniques, which obligate the researcher to specify a functional form of the production function of which the Cobb...... parameter estimates, but also in biased measures which are derived from the parameters, such as elasticities. Therefore, we propose to use non-parametric econometric methods. First, these can be applied to verify the functional form used in parametric production analysis. Second, they can be directly used...

  8. A novel nonparametric measure of explained variation for survival data with an easy graphical interpretation

    Directory of Open Access Journals (Sweden)

    Weiß, Verena

    2015-10-01

    Full Text Available Introduction: For survival data the coefficient of determination cannot be used to describe how good a model fits to the data. Therefore, several measures of explained variation for survival data have been proposed in recent years.Methods: We analyse an existing measure of explained variation with regard to minimisation aspects and demonstrate that these are not fulfilled for the measure.Results: In analogy to the least squares method from linear regression analysis we develop a novel measure for categorical covariates which is based only on the Kaplan-Meier estimator. Hence, the novel measure is a completely nonparametric measure with an easy graphical interpretation. For the novel measure different weighting possibilities are available and a statistical test of significance can be performed. Eventually, we apply the novel measure and further measures of explained variation to a dataset comprising persons with a histopathological papillary thyroid carcinoma.Conclusion: We propose a novel measure of explained variation with a comprehensible derivation as well as a graphical interpretation, which may be used in further analyses with survival data.

  9. Poverty and life cycle effects: A nonparametric analysis for Germany

    OpenAIRE

    Stich, Andreas

    1996-01-01

    Most empirical studies on poverty consider the extent of poverty either for the entire society or for separate groups like elderly people.However, these papers do not show what the situation looks like for persons of a certain age. In this paper poverty measures depending on age are derived using the joint density of income and age. The density is nonparametrically estimated by weighted Gaussian kernel density estimation. Applying the conditional density of income to several poverty measures ...

  10. ANALYSIS OF TIED DATA: AN ALTERNATIVE NON-PARAMETRIC APPROACH

    Directory of Open Access Journals (Sweden)

    I. C. A. OYEKA

    2012-02-01

    Full Text Available This paper presents a non-parametric statistical method of analyzing two-sample data that makes provision for the possibility of ties in the data. A test statistic is developed and shown to be free of the effect of any possible ties in the data. An illustrative example is provided and the method is shown to compare favourably with its competitor; the Mann-Whitney test and is more powerful than the latter when there are ties.

  11. Nonparametric test for detecting change in distribution with panel data

    CERN Document Server

    Pommeret, Denys; Ghattas, Badih

    2011-01-01

    This paper considers the problem of comparing two processes with panel data. A nonparametric test is proposed for detecting a monotone change in the link between the two process distributions. The test statistic is of CUSUM type, based on the empirical distribution functions. The asymptotic distribution of the proposed statistic is derived and its finite sample property is examined by bootstrap procedures through Monte Carlo simulations.

  12. A Bayesian nonparametric method for prediction in EST analysis

    Directory of Open Access Journals (Sweden)

    Prünster Igor

    2007-09-01

    Full Text Available Abstract Background Expressed sequence tags (ESTs analyses are a fundamental tool for gene identification in organisms. Given a preliminary EST sample from a certain library, several statistical prediction problems arise. In particular, it is of interest to estimate how many new genes can be detected in a future EST sample of given size and also to determine the gene discovery rate: these estimates represent the basis for deciding whether to proceed sequencing the library and, in case of a positive decision, a guideline for selecting the size of the new sample. Such information is also useful for establishing sequencing efficiency in experimental design and for measuring the degree of redundancy of an EST library. Results In this work we propose a Bayesian nonparametric approach for tackling statistical problems related to EST surveys. In particular, we provide estimates for: a the coverage, defined as the proportion of unique genes in the library represented in the given sample of reads; b the number of new unique genes to be observed in a future sample; c the discovery rate of new genes as a function of the future sample size. The Bayesian nonparametric model we adopt conveys, in a statistically rigorous way, the available information into prediction. Our proposal has appealing properties over frequentist nonparametric methods, which become unstable when prediction is required for large future samples. EST libraries, previously studied with frequentist methods, are analyzed in detail. Conclusion The Bayesian nonparametric approach we undertake yields valuable tools for gene capture and prediction in EST libraries. The estimators we obtain do not feature the kind of drawbacks associated with frequentist estimators and are reliable for any size of the additional sample.

  13. Fusion of Hard and Soft Information in Nonparametric Density Estimation

    Science.gov (United States)

    2015-06-10

    estimation exploiting, in concert, hard and soft information. Although our development, theoretical and numerical, makes no distinction based on sample...Fusion of Hard and Soft Information in Nonparametric Density Estimation∗ Johannes O. Royset Roger J-B Wets Department of Operations Research...univariate density estimation in situations when the sample ( hard information) is supplemented by “soft” information about the random phenomenon. These

  14. Nonparametric estimation for hazard rate monotonously decreasing system

    Institute of Scientific and Technical Information of China (English)

    Han Fengyan; Li Weisong

    2005-01-01

    Estimation of density and hazard rate is very important to the reliability analysis of a system. In order to estimate the density and hazard rate of a hazard rate monotonously decreasing system, a new nonparametric estimator is put forward. The estimator is based on the kernel function method and optimum algorithm. Numerical experiment shows that the method is accurate enough and can be used in many cases.

  15. Non-parametric versus parametric methods in environmental sciences

    Directory of Open Access Journals (Sweden)

    Muhammad Riaz

    2016-01-01

    Full Text Available This current report intends to highlight the importance of considering background assumptions required for the analysis of real datasets in different disciplines. We will provide comparative discussion of parametric methods (that depends on distributional assumptions (like normality relative to non-parametric methods (that are free from many distributional assumptions. We have chosen a real dataset from environmental sciences (one of the application areas. The findings may be extended to the other disciplines following the same spirit.

  16. The impact of covariance misspecification in group-based trajectory models for longitudinal data with non-stationary covariance structure.

    Science.gov (United States)

    Davies, Christopher E; Glonek, Gary Fv; Giles, Lynne C

    2017-08-01

    One purpose of a longitudinal study is to gain a better understanding of how an outcome of interest changes among a given population over time. In what follows, a trajectory will be taken to mean the series of measurements of the outcome variable for an individual. Group-based trajectory modelling methods seek to identify subgroups of trajectories within a population, such that trajectories that are grouped together are more similar to each other than to trajectories in distinct groups. Group-based trajectory models generally assume a certain structure in the covariances between measurements, for example conditional independence, homogeneous variance between groups or stationary variance over time. Violations of these assumptions could be expected to result in poor model performance. We used simulation to investigate the effect of covariance misspecification on misclassification of trajectories in commonly used models under a range of scenarios. To do this we defined a measure of performance relative to the ideal Bayesian correct classification rate. We found that the more complex models generally performed better over a range of scenarios. In particular, incorrectly specified covariance matrices could significantly bias the results but using models with a correct but more complicated than necessary covariance matrix incurred little cost.

  17. Least-Squares Data Adjustment with Rank-Deficient Data Covariance Matrices

    Energy Technology Data Exchange (ETDEWEB)

    Williams, J.G. [The University of Arizona, Tucson, AZ 85721-0119 (United States)

    2011-07-01

    A derivation of the linear least-squares adjustment formulae is required that avoids the assumption that the covariance matrix of prior parameters can be inverted. Possible proofs are of several kinds, including: (i) extension of standard results for the linear regression formulae, and (ii) minimization by differentiation of a quadratic form of the deviations in parameters and responses. In this paper, the least-squares adjustment equations are derived in both these ways, while explicitly assuming that the covariance matrix of prior parameters is singular. It will be proved that the solutions are unique and that, contrary to statements that have appeared in the literature, the least-squares adjustment problem is not ill-posed. No modification is required to the adjustment formulae that have been used in the past in the case of a singular covariance matrix for the priors. In conclusion: The linear least-squares adjustment formula that has been used in the past is valid in the case of a singular covariance matrix for the covariance matrix of prior parameters. Furthermore, it provides a unique solution. Statements in the literature, to the effect that the problem is ill-posed are wrong. No regularization of the problem is required. This has been proved in the present paper by two methods, while explicitly assuming that the covariance matrix of prior parameters is singular: i) extension of standard results for the linear regression formulae, and (ii) minimization by differentiation of a quadratic form of the deviations in parameters and responses. No modification is needed to the adjustment formulae that have been used in the past. (author)

  18. Multigroup covariance matrices for fast-reactor studies

    Energy Technology Data Exchange (ETDEWEB)

    Smith, J.D. III; Broadhead, B.L.

    1981-04-01

    This report presents the multigroup covariance matrices based on the ENDF/B-V nuclear data evaluations. The materials and reactions have been chosen according to the specifications of ORNL-5517. Several cross section covariances, other than those specified by that report, are included due to the derived nature of the uncertainty files in ENDF/B-V. The materials represented are Ni, Cr, /sup 16/O, /sup 12/C, Fe, Na, /sup 235/U, /sup 238/U, /sup 239/Pu, /sup 240/Pu, /sup 241/Pu, and /sup 10/B (present due to its correlation to /sup 238/U). The data have been originally processed into a 52-group energy structure by PUFF-II and subsequently collapsed to smaller subgroup strutures. The results are illustrated in 52-group correlation matrix plots and tabulated into thirteen groups for convenience.

  19. Model Order Selection Rules for Covariance Structure Classification in Radar

    Science.gov (United States)

    Carotenuto, Vincenzo; De Maio, Antonio; Orlando, Danilo; Stoica, Petre

    2017-10-01

    The adaptive classification of the interference covariance matrix structure for radar signal processing applications is addressed in this paper. This represents a key issue because many detection architectures are synthesized assuming a specific covariance structure which may not necessarily coincide with the actual one due to the joint action of the system and environment uncertainties. The considered classification problem is cast in terms of a multiple hypotheses test with some nested alternatives and the theory of Model Order Selection (MOS) is exploited to devise suitable decision rules. Several MOS techniques, such as the Akaike, Takeuchi, and Bayesian information criteria are adopted and the corresponding merits and drawbacks are discussed. At the analysis stage, illustrating examples for the probability of correct model selection are presented showing the effectiveness of the proposed rules.

  20. General Covariance from the Quantum Renormalization Group

    CERN Document Server

    Shyam, Vasudev

    2016-01-01

    The Quantum renormalization group (QRG) is a realisation of holography through a coarse graining prescription that maps the beta functions of a quantum field theory thought to live on the `boundary' of some space to holographic actions in the `bulk' of this space. A consistency condition will be proposed that translates into general covariance of the gravitational theory in the $D + 1$ dimensional bulk. This emerges from the application of the QRG on a planar matrix field theory living on the $D$ dimensional boundary. This will be a particular form of the Wess--Zumino consistency condition that the generating functional of the boundary theory needs to satisfy. In the bulk, this condition forces the Poisson bracket algebra of the scalar and vector constraints of the dual gravitational theory to close in a very specific manner, namely, the manner in which the corresponding constraints of general relativity do. A number of features of the gravitational theory will be fixed as a consequence of this form of the Po...

  1. An Adaptive Estimation of Forecast Error Covariance Parameters for Kalman Filtering Data Assimilation

    Institute of Scientific and Technical Information of China (English)

    Xiaogu ZHENG

    2009-01-01

    An adaptive estimation of forecast error covariance matrices is proposed for Kalman filtering data assimilation. A forecast error covariance matrix is initially estimated using an ensemble of perturbation forecasts. This initially estimated matrix is then adjusted with scale parameters that are adaptively estimated by minimizing -2log-likelihood of observed-minus-forecast residuals. The proposed approach could be applied to Kalman filtering data assimilation with imperfect models when the model error statistics are not known. A simple nonlinear model (Burgers' equation model) is used to demonstrate the efficacy of the proposed approach.

  2. Covariant diagrams for one-loop matching

    CERN Document Server

    Zhang, Zhengkang

    2016-01-01

    We present a diagrammatic formulation of recently-revived covariant functional approaches to one-loop matching from an ultraviolet (UV) theory to a low-energy effective field theory. Various terms following from a covariant derivative expansion (CDE) are represented by diagrams which, unlike conventional Feynman diagrams, involve gauge-covariant quantities and are thus dubbed "covariant diagrams." The use of covariant diagrams helps organize and simplify one-loop matching calculations, which we illustrate with examples. Of particular interest is the derivation of UV model-independent universal results, which reduce matching calculations of specific UV models to applications of master formulas. We show how such derivation can be done in a more concise manner than the previous literature, and discuss how additional structures that are not directly captured by existing universal results, including mixed heavy-light loops, open covariant derivatives, and mixed statistics, can be easily accounted for.

  3. ISSUES IN NEUTRON CROSS SECTION COVARIANCES

    Energy Technology Data Exchange (ETDEWEB)

    Mattoon, C.M.; Oblozinsky,P.

    2010-04-30

    We review neutron cross section covariances in both the resonance and fast neutron regions with the goal to identify existing issues in evaluation methods and their impact on covariances. We also outline ideas for suitable covariance quality assurance procedures.We show that the topic of covariance data remains controversial, the evaluation methodologies are not fully established and covariances produced by different approaches have unacceptable spread. The main controversy is in very low uncertainties generated by rigorous evaluation methods and much larger uncertainties based on simple estimates from experimental data. Since the evaluators tend to trust the former, while the users tend to trust the latter, this controversy has considerable practical implications. Dedicated effort is needed to arrive at covariance evaluation methods that would resolve this issue and produce results accepted internationally both by evaluators and users.

  4. ISSUES IN NEUTRON CROSS SECTION COVARIANCES

    Energy Technology Data Exchange (ETDEWEB)

    Mattoon, C.M.; Oblozinsky,P.

    2010-04-30

    We review neutron cross section covariances in both the resonance and fast neutron regions with the goal to identify existing issues in evaluation methods and their impact on covariances. We also outline ideas for suitable covariance quality assurance procedures.We show that the topic of covariance data remains controversial, the evaluation methodologies are not fully established and covariances produced by different approaches have unacceptable spread. The main controversy is in very low uncertainties generated by rigorous evaluation methods and much larger uncertainties based on simple estimates from experimental data. Since the evaluators tend to trust the former, while the users tend to trust the latter, this controversy has considerable practical implications. Dedicated effort is needed to arrive at covariance evaluation methods that would resolve this issue and produce results accepted internationally both by evaluators and users.

  5. Treatment Effects with Many Covariates and Heteroskedasticity

    DEFF Research Database (Denmark)

    Cattaneo, Matias D.; Jansson, Michael; Newey, Whitney K.

    The linear regression model is widely used in empirical work in Economics. Researchers often include many covariates in their linear model specification in an attempt to control for confounders. We give inference methods that allow for many covariates and heteroskedasticity. Our results are obtai......The linear regression model is widely used in empirical work in Economics. Researchers often include many covariates in their linear model specification in an attempt to control for confounders. We give inference methods that allow for many covariates and heteroskedasticity. Our results...... then propose a new heteroskedasticity consistent standard error formula that is fully automatic and robust to both (conditional) heteroskedasticity of unknown form and the inclusion of possibly many covariates. We apply our findings to three settings: (i) parametric linear models with many covariates, (ii...

  6. TRANSPOSABLE REGULARIZED COVARIANCE MODELS WITH AN APPLICATION TO MISSING DATA IMPUTATION.

    Science.gov (United States)

    Allen, Genevera I; Tibshirani, Robert

    2010-06-01

    Missing data estimation is an important challenge with high-dimensional data arranged in the form of a matrix. Typically this data matrix is transposable, meaning that either the rows, columns or both can be treated as features. To model transposable data, we present a modification of the matrix-variate normal, the mean-restricted matrix-variate normal, in which the rows and columns each have a separate mean vector and covariance matrix. By placing additive penalties on the inverse covariance matrices of the rows and columns, these so called transposable regularized covariance models allow for maximum likelihood estimation of the mean and non-singular covariance matrices. Using these models, we formulate EM-type algorithms for missing data imputation in both the multivariate and transposable frameworks. We present theoretical results exploiting the structure of our transposable models that allow these models and imputation methods to be applied to high-dimensional data. Simulations and results on microarray data and the Netflix data show that these imputation techniques often outperform existing methods and offer a greater degree of flexibility.

  7. Singular Value Decomposition, Hessian Errors, and Linear Algebra of Non-parametric Extraction of Partons from DIS

    CERN Document Server

    Goshtasbpour, Mehrdad

    2014-01-01

    By singular value decomposition (SVD) of a numerically singular Hessian matrix and a numerically singular system of linear equations for the experimental data (accumulated in the respective ${\\chi ^2}$ function) and constraints, least square solutions and their propagated errors for the non-parametric extraction of Partons from $F_2$ are obtained. SVD and its physical application is phenomenologically described in the two cases. Among the subjects covered are: identification and properties of the boundary between the two subsets of ordered eigenvalues corresponding to range and null space, and the eigenvalue structure of the null space of the singular matrix, including a second boundary separating the smallest eigenvalues of essentially no information, in a particular case. The eigenvector-eigenvalue structure of "redundancy and smallness" of the errors of two pdf sets, in our simplified Hessian model, is described by a secondary manifestation of deeper null space, in the context of SVD.

  8. Eigenvalue variance bounds for covariance matrices

    OpenAIRE

    Dallaporta, Sandrine

    2013-01-01

    This work is concerned with finite range bounds on the variance of individual eigenvalues of random covariance matrices, both in the bulk and at the edge of the spectrum. In a preceding paper, the author established analogous results for Wigner matrices and stated the results for covariance matrices. They are proved in the present paper. Relying on the LUE example, which needs to be investigated first, the main bounds are extended to complex covariance matrices by means of the Tao, Vu and Wan...

  9. Covariance of Light-Front Models Pair Current

    CERN Document Server

    Pacheco-Bicudo-Cabral de Melo, J; Naus, H W L; Sauer, P U

    1999-01-01

    We compute the "+" component of the electromagnetic current of a composite spin-one two-fermion system for vanishing momentum transfer component $q^+=q^0+q^3$. In particular, we extract the nonvanishing pair production amplitude on the light-front. It is a consequence of the longitudinal zero momentum mode, contributing to the light-front current in the Breit-frame. The covariance of the current is violated, if such pair terms are not included in its matrix elements. We illustrate our discussion with some numerical examples.

  10. Gaussian Fluctuations for Sample Covariance Matrices with Dependent Data

    CERN Document Server

    Friesen, Olga; Stolz, Michael

    2012-01-01

    It is known (Hofmann-Credner and Stolz (2008)) that the convergence of the mean empirical spectral distribution of a sample covariance matrix W_n = 1/n Y_n Y_n^t to the Mar\\v{c}enko-Pastur law remains unaffected if the rows and columns of Y_n exhibit some dependence, where only the growth of the number of dependent entries, but not the joint distribution of dependent entries needs to be controlled. In this paper we show that the well-known CLT for traces of powers of W_n also extends to the dependent case.

  11. a Multivariate Downscaling Model for Nonparametric Simulation of Daily Flows

    Science.gov (United States)

    Molina, J. M.; Ramirez, J. A.; Raff, D. A.

    2011-12-01

    A multivariate, stochastic nonparametric framework for stepwise disaggregation of seasonal runoff volumes to daily streamflow is presented. The downscaling process is conditional on volumes of spring runoff and large-scale ocean-atmosphere teleconnections and includes a two-level cascade scheme: seasonal-to-monthly disaggregation first followed by monthly-to-daily disaggregation. The non-parametric and assumption-free character of the framework allows consideration of the random nature and nonlinearities of daily flows, which parametric models are unable to account for adequately. This paper examines statistical links between decadal/interannual climatic variations in the Pacific Ocean and hydrologic variability in US northwest region, and includes a periodicity analysis of climate patterns to detect coherences of their cyclic behavior in the frequency domain. We explore the use of such relationships and selected signals (e.g., north Pacific gyre oscillation, southern oscillation, and Pacific decadal oscillation indices, NPGO, SOI and PDO, respectively) in the proposed data-driven framework by means of a combinatorial approach with the aim of simulating improved streamflow sequences when compared with disaggregated series generated from flows alone. A nearest neighbor time series bootstrapping approach is integrated with principal component analysis to resample from the empirical multivariate distribution. A volume-dependent scaling transformation is implemented to guarantee the summability condition. In addition, we present a new and simple algorithm, based on nonparametric resampling, that overcomes the common limitation of lack of preservation of historical correlation between daily flows across months. The downscaling framework presented here is parsimonious in parameters and model assumptions, does not generate negative values, and produces synthetic series that are statistically indistinguishable from the observations. We present evidence showing that both

  12. A trade-off solution between model resolution and covariance in surface-wave inversion

    Science.gov (United States)

    Xia, J.; Xu, Y.; Miller, R.D.; Zeng, C.

    2010-01-01

    Regularization is necessary for inversion of ill-posed geophysical problems. Appraisal of inverse models is essential for meaningful interpretation of these models. Because uncertainties are associated with regularization parameters, extra conditions are usually required to determine proper parameters for assessing inverse models. Commonly used techniques for assessment of a geophysical inverse model derived (generally iteratively) from a linear system are based on calculating the model resolution and the model covariance matrices. Because the model resolution and the model covariance matrices of the regularized solutions are controlled by the regularization parameter, direct assessment of inverse models using only the covariance matrix may provide incorrect results. To assess an inverted model, we use the concept of a trade-off between model resolution and covariance to find a proper regularization parameter with singular values calculated in the last iteration. We plot the singular values from large to small to form a singular value plot. A proper regularization parameter is normally the first singular value that approaches zero in the plot. With this regularization parameter, we obtain a trade-off solution between model resolution and model covariance in the vicinity of a regularized solution. The unit covariance matrix can then be used to calculate error bars of the inverse model at a resolution level determined by the regularization parameter. We demonstrate this approach with both synthetic and real surface-wave data. ?? 2010 Birkh??user / Springer Basel AG.

  13. Variations of cosmic large-scale structure covariance matrices across parameter space

    Science.gov (United States)

    Reischke, Robert; Kiessling, Alina; Schäfer, Björn Malte

    2017-03-01

    The likelihood function for cosmological parameters, given by e.g. weak lensing shear measurements, depends on contributions to the covariance induced by the non-linear evolution of the cosmic web. As highly non-linear clustering to date has only been described by numerical N-body simulations in a reliable and sufficiently precise way, the necessary computational costs for estimating those covariances at different points in parameter space are tremendous. In this work, we describe the change of the matter covariance and the weak lensing covariance matrix as a function of cosmological parameters by constructing a suitable basis, where we model the contribution to the covariance from non-linear structure formation using Eulerian perturbation theory at third order. We show that our formalism is capable of dealing with large matrices and reproduces expected degeneracies and scaling with cosmological parameters in a reliable way. Comparing our analytical results to numerical simulations, we find that the method describes the variation of the covariance matrix found in the SUNGLASS weak lensing simulation pipeline within the errors at one-loop and tree-level for the spectrum and the trispectrum, respectively, for multipoles up to ℓ ≤ 1300. We show that it is possible to optimize the sampling of parameter space where numerical simulations should be carried out by minimizing interpolation errors and propose a corresponding method to distribute points in parameter space in an economical way.

  14. Panel data nonparametric estimation of production risk and risk preferences

    DEFF Research Database (Denmark)

    Czekaj, Tomasz Gerard; Henningsen, Arne

    We apply nonparametric panel data kernel regression to investigate production risk, out-put price uncertainty, and risk attitudes of Polish dairy farms based on a firm-level unbalanced panel data set that covers the period 2004–2010. We compare different model specifications and different...... approaches for obtaining firm-specific measures of risk attitudes. We found that Polish dairy farmers are risk averse regarding production risk and price uncertainty. According to our results, Polish dairy farmers perceive the production risk as being more significant than the risk related to output price...

  15. Digital spectral analysis parametric, non-parametric and advanced methods

    CERN Document Server

    Castanié, Francis

    2013-01-01

    Digital Spectral Analysis provides a single source that offers complete coverage of the spectral analysis domain. This self-contained work includes details on advanced topics that are usually presented in scattered sources throughout the literature.The theoretical principles necessary for the understanding of spectral analysis are discussed in the first four chapters: fundamentals, digital signal processing, estimation in spectral analysis, and time-series models.An entire chapter is devoted to the non-parametric methods most widely used in industry.High resolution methods a

  16. Nonparametric statistics a step-by-step approach

    CERN Document Server

    Corder, Gregory W

    2014-01-01

    "…a very useful resource for courses in nonparametric statistics in which the emphasis is on applications rather than on theory.  It also deserves a place in libraries of all institutions where introductory statistics courses are taught."" -CHOICE This Second Edition presents a practical and understandable approach that enhances and expands the statistical toolset for readers. This book includes: New coverage of the sign test and the Kolmogorov-Smirnov two-sample test in an effort to offer a logical and natural progression to statistical powerSPSS® (Version 21) software and updated screen ca

  17. Categorical and nonparametric data analysis choosing the best statistical technique

    CERN Document Server

    Nussbaum, E Michael

    2014-01-01

    Featuring in-depth coverage of categorical and nonparametric statistics, this book provides a conceptual framework for choosing the most appropriate type of test in various research scenarios. Class tested at the University of Nevada, the book's clear explanations of the underlying assumptions, computer simulations, and Exploring the Concept boxes help reduce reader anxiety. Problems inspired by actual studies provide meaningful illustrations of the techniques. The underlying assumptions of each test and the factors that impact validity and statistical power are reviewed so readers can explain

  18. Nonparametric statistical structuring of knowledge systems using binary feature matches

    DEFF Research Database (Denmark)

    Mørup, Morten; Glückstad, Fumiko Kano; Herlau, Tue

    2014-01-01

    statistical support and how this approach generalizes to the structuring and alignment of knowledge systems. We propose a non-parametric Bayesian generative model for structuring binary feature data that does not depend on a specific choice of similarity measure. We jointly model all combinations of binary......Structuring knowledge systems with binary features is often based on imposing a similarity measure and clustering objects according to this similarity. Unfortunately, such analyses can be heavily influenced by the choice of similarity measure. Furthermore, it is unclear at which level clusters have...

  19. Testing for a constant coefficient of variation in nonparametric regression

    OpenAIRE

    Dette, Holger; Marchlewski, Mareen; Wagener, Jens

    2010-01-01

    In the common nonparametric regression model Y_i=m(X_i)+sigma(X_i)epsilon_i we consider the problem of testing the hypothesis that the coefficient of the scale and location function is constant. The test is based on a comparison of the observations Y_i=\\hat{sigma}(X_i) with their mean by a smoothed empirical process, where \\hat{sigma} denotes the local linear estimate of the scale function. We show weak convergence of a centered version of this process to a Gaussian process under the null ...

  20. Generative Temporal Modelling of Neuroimaging - Decomposition and Nonparametric Testing

    DEFF Research Database (Denmark)

    Hald, Ditte Høvenhoff

    The goal of this thesis is to explore two improvements for functional magnetic resonance imaging (fMRI) analysis; namely our proposed decomposition method and an extension to the non-parametric testing framework. Analysis of fMRI allows researchers to investigate the functional processes...... of the brain, and provides insight into neuronal coupling during mental processes or tasks. The decomposition method is a Gaussian process-based independent components analysis (GPICA), which incorporates a temporal dependency in the sources. A hierarchical model specification is used, featuring both...

  1. A multivariate multilevel Gaussian model with a mixed effects structure in the mean and covariance part.

    Science.gov (United States)

    Li, Baoyue; Bruyneel, Luk; Lesaffre, Emmanuel

    2014-05-20

    A traditional Gaussian hierarchical model assumes a nested multilevel structure for the mean and a constant variance at each level. We propose a Bayesian multivariate multilevel factor model that assumes a multilevel structure for both the mean and the covariance matrix. That is, in addition to a multilevel structure for the mean we also assume that the covariance matrix depends on covariates and random effects. This allows to explore whether the covariance structure depends on the values of the higher levels and as such models heterogeneity in the variances and correlation structure of the multivariate outcome across the higher level values. The approach is applied to the three-dimensional vector of burnout measurements collected on nurses in a large European study to answer the research question whether the covariance matrix of the outcomes depends on recorded system-level features in the organization of nursing care, but also on not-recorded factors that vary with countries, hospitals, and nursing units. Simulations illustrate the performance of our modeling approach. Copyright © 2013 John Wiley & Sons, Ltd.

  2. Cox regression with missing covariate data using a modified partial likelihood method

    DEFF Research Database (Denmark)

    Martinussen, Torben; Holst, Klaus K.; Scheike, Thomas H.

    2016-01-01

    us to calculate estimators without having to assume anything about the distribution of the covariates. We show that the proposed estimator is consistent and asymptotically normal, and derive a consistent estimator of the variance-covariance matrix that does not involve any choice of a perturbation......Missing covariate values is a common problem in survival analysis. In this paper we propose a novel method for the Cox regression model that is close to maximum likelihood but avoids the use of the EM-algorithm. It exploits that the observed hazard function is multiplicative in the baseline hazard...... function with the idea being to profile out this function before carrying out the estimation of the parameter of interest. In this step one uses a Breslow type estimator to estimate the cumulative baseline hazard function. We focus on the situation where the observed covariates are categorical which allows...

  3. Scaling dimensions of manifestly generally covariant operators in two-dimensional quantum gravity

    CERN Document Server

    Nishimura, J; Tsuchiya, A; Jun Nishimura; Shinya Tamura; Asato Tsuchiya

    1994-01-01

    Using (2+$\\epsilon$)-dimensional quantum gravity recently formulated by Kawai, Kitazawa and Ninomiya, we calculate the scaling dimensions of manifestly generally covariant operators in two-dimensional quantum gravity coupled to $(p,q)$ minimal conformal matter. In the spectrum appear all the scaling dimensions of the scaling operators in the matrix model except the boundary operators, while there are also many others which have no corresponding scaling dimensions in the matrix model.

  4. Covariance structure models of expectancy.

    Science.gov (United States)

    Henderson, M J; Goldman, M S; Coovert, M D; Carnevalla, N

    1994-05-01

    Antecedent variables under the broad categories of genetic, environmental and cultural influences have been linked to the risk for alcohol abuse. Such risk factors have not been shown to result in high correlations with alcohol consumption and leave unclear an understanding of the mechanism by which these variables lead to increased risk. This study employed covariance structure modeling to examine the mediational influence of stored information in memory about alcohol, alcohol expectancies in relation to two biologically and environmentally driven antecedent variables, family history of alcohol abuse and a sensation-seeking temperament in a college population. We also examined the effect of criterion contamination on the relationship between sensation-seeking and alcohol consumption. Results indicated that alcohol expectancy acts as a significant, partial mediator of the relationship between sensation-seeking and consumption, that family history of alcohol abuse is not related to drinking outcome and that overlap in items on sensation-seeking and alcohol consumption measures may falsely inflate their relationship.

  5. On the Origin of Gravitational Lorentz Covariance

    CERN Document Server

    Khoury, Justin; Tolley, Andrew J

    2013-01-01

    We provide evidence that general relativity is the unique spatially covariant effective field theory of the transverse, traceless graviton degrees of freedom. The Lorentz covariance of general relativity, having not been assumed in our analysis, is thus plausibly interpreted as an accidental or emergent symmetry of the gravitational sector.

  6. COVARIATION BIAS AND THE RETURN OF FEAR

    NARCIS (Netherlands)

    de Jong, Peter; VANDENHOUT, MA; MERCKELBACH, H

    1995-01-01

    Several studies have indicated that phobic fear is accompanied by a covariation bias, i.e. that phobic Ss tend to overassociate fear relevant stimuli and aversive outcomes. Such a covariation bias seems to be a fairly direct and powerful way to confirm danger expectations and enhance fear. Therefore

  7. Covariant derivative of fermions and all that

    CERN Document Server

    Shapiro, Ilya L

    2016-01-01

    We present detailed pedagogical derivation of covariant derivative of fermions and some related expressions, including commutator of covariant derivatives and energy-momentum tensor of a free Dirac field. The text represents a part of the initial chapter of a one-semester course on semiclassical gravity.

  8. An estimator for the quadratic covariation of asynchronously observed It\\^o processes with noise: Asymptotic distribution theory

    CERN Document Server

    Bibinger, Markus

    2011-01-01

    The article is devoted to the nonparametric estimation of the quadratic covariation of non-synchronously observed It\\^o processes in an additive microstructure noise model. In a high-frequency setting, we aim at establishing an asymptotic distribution theory for a generalized multiscale estimator including a feasible central limit theorem with optimal convergence rate on convenient regularity assumptions. The inevitably remaining impact of asynchronous deterministic sampling schemes and noise corruption on the asymptotic distribution is precisely elucidated. A case study for various important examples, several generalizations of the model and an algorithm for the implementation warrant the utility of the estimation method in applications.

  9. 高频价格数据信息对资产组合动态风险测度的影响研究--基于实现协方差矩阵模型的分析%The Study on the Influence of High-Frequency Prices Information on the Portfolio's Dynamic Risk Measurement--Based on the Realized Co-variance Matrix Model

    Institute of Scientific and Technical Information of China (English)

    殷炼乾

    2016-01-01

    The estimation and forecast of portfolio market risks is always a very important aspect of risk manage-ment. This paper employs the realized co-variance matrix model, DCC-MVGARCH model, RiskMetrics model and multi-variants Orthogonal GARCH model to compare their forecast failure ratios of the value at risk of the Shanghai and Shenzhen stock index portfolio and also compare these models with a dynamic quantile test for the forecasting robust-ness. The results show that the realized co-variance matrix model based on high-frequency prices data can significantly improve the forecast accuracy of the portfolio market risk,and its failure rates are also strictly consistent with the corre-sponding confidence levels. Hence this model has achieved a good balance between high utilization of money and also its risk exposures of portfolio management.%组合风险的估计和预测一直都是风险管理中非常重要的一个方面。本文使用了利用高频数据信息的实现协方差矩阵、DCC-MVGARCH多元波动率模型、RiskMetrics模型和多元正交GARCH模型对沪深两市的指数资产组合风险在险价值的预测失败率进行了对比,并利用动态分位数检验方法对各模型的组合风险测度稳健性进行了对比研究。研究结果证明,基于高频数据的实现协方差矩阵模型能够显著提高组合风险测度的预测精度,且严格符合VaR置信区间所要求的失败率,能够很好地在提高资金使用效率与管理资产组合风险敞口间取得平衡。

  10. Using Mathematica to build Non-parametric Statistical Tables

    Directory of Open Access Journals (Sweden)

    Gloria Perez Sainz de Rozas

    2003-01-01

    Full Text Available In this paper, I present computational procedures to obtian statistical tables. The tables of the asymptotic distribution and the exact distribution of Kolmogorov-Smirnov statistic Dn for one population, the table of the distribution of the runs R, the table of the distribution of Wilcoxon signed-rank statistic W+ and the table of the distribution of Mann-Whitney statistic Ux using Mathematica, Version 3.9 under Window98. I think that it is an interesting cuestion because many statistical packages give the asymptotic significance level in the statistical tests and with these porcedures one can easily calculate the exact significance levels and the left-tail and right-tail probabilities with non-parametric distributions. I have used mathematica to make these calculations because one can use symbolic language to solve recursion relations. It's very easy to generate the format of the tables, and it's possible to obtain any table of the mentioned non-parametric distributions with any precision, not only with the standard parameters more used in Statistics, and without transcription mistakes. Furthermore, using similar procedures, we can generate tables for the following distribution functions: Binomial, Poisson, Hypergeometric, Normal, x2 Chi-Square, T-Student, F-Snedecor, Geometric, Gamma and Beta.

  11. 1st Conference of the International Society for Nonparametric Statistics

    CERN Document Server

    Lahiri, S; Politis, Dimitris

    2014-01-01

    This volume is composed of peer-reviewed papers that have developed from the First Conference of the International Society for NonParametric Statistics (ISNPS). This inaugural conference took place in Chalkidiki, Greece, June 15-19, 2012. It was organized with the co-sponsorship of the IMS, the ISI, and other organizations. M.G. Akritas, S.N. Lahiri, and D.N. Politis are the first executive committee members of ISNPS, and the editors of this volume. ISNPS has a distinguished Advisory Committee that includes Professors R.Beran, P.Bickel, R. Carroll, D. Cook, P. Hall, R. Johnson, B. Lindsay, E. Parzen, P. Robinson, M. Rosenblatt, G. Roussas, T. SubbaRao, and G. Wahba. The Charting Committee of ISNPS consists of more than 50 prominent researchers from all over the world.   The chapters in this volume bring forth recent advances and trends in several areas of nonparametric statistics. In this way, the volume facilitates the exchange of research ideas, promotes collaboration among researchers from all over the wo...

  12. Non-parametric Morphologies of Mergers in the Illustris Simulation

    CERN Document Server

    Bignone, Lucas A; Sillero, Emanuel; Pedrosa, Susana E; Pellizza, Leonardo J; Lambas, Diego G

    2016-01-01

    We study non-parametric morphologies of mergers events in a cosmological context, using the Illustris project. We produce mock g-band images comparable to observational surveys from the publicly available Illustris simulation idealized mock images at $z=0$. We then measure non parametric indicators: asymmetry, Gini, $M_{20}$, clumpiness and concentration for a set of galaxies with $M_* >10^{10}$ M$_\\odot$. We correlate these automatic statistics with the recent merger history of galaxies and with the presence of close companions. Our main contribution is to assess in a cosmological framework, the empirically derived non-parametric demarcation line and average time-scales used to determine the merger rate observationally. We found that 98 per cent of galaxies above the demarcation line have a close companion or have experienced a recent merger event. On average, merger signatures obtained from the $G-M_{20}$ criteria anticorrelate clearly with the elapsing time to the last merger event. We also find that the a...

  13. Genomic breeding value estimation using nonparametric additive regression models

    Directory of Open Access Journals (Sweden)

    Solberg Trygve

    2009-01-01

    Full Text Available Abstract Genomic selection refers to the use of genomewide dense markers for breeding value estimation and subsequently for selection. The main challenge of genomic breeding value estimation is the estimation of many effects from a limited number of observations. Bayesian methods have been proposed to successfully cope with these challenges. As an alternative class of models, non- and semiparametric models were recently introduced. The present study investigated the ability of nonparametric additive regression models to predict genomic breeding values. The genotypes were modelled for each marker or pair of flanking markers (i.e. the predictors separately. The nonparametric functions for the predictors were estimated simultaneously using additive model theory, applying a binomial kernel. The optimal degree of smoothing was determined by bootstrapping. A mutation-drift-balance simulation was carried out. The breeding values of the last generation (genotyped was predicted using data from the next last generation (genotyped and phenotyped. The results show moderate to high accuracies of the predicted breeding values. A determination of predictor specific degree of smoothing increased the accuracy.

  14. Nonparametric Analyses of Log-Periodic Precursors to Financial Crashes

    Science.gov (United States)

    Zhou, Wei-Xing; Sornette, Didier

    We apply two nonparametric methods to further test the hypothesis that log-periodicity characterizes the detrended price trajectory of large financial indices prior to financial crashes or strong corrections. The term "parametric" refers here to the use of the log-periodic power law formula to fit the data; in contrast, "nonparametric" refers to the use of general tools such as Fourier transform, and in the present case the Hilbert transform and the so-called (H, q)-analysis. The analysis using the (H, q)-derivative is applied to seven time series ending with the October 1987 crash, the October 1997 correction and the April 2000 crash of the Dow Jones Industrial Average (DJIA), the Standard & Poor 500 and Nasdaq indices. The Hilbert transform is applied to two detrended price time series in terms of the ln(tc-t) variable, where tc is the time of the crash. Taking all results together, we find strong evidence for a universal fundamental log-frequency f=1.02±0.05 corresponding to the scaling ratio λ=2.67±0.12. These values are in very good agreement with those obtained in earlier works with different parametric techniques. This note is extracted from a long unpublished report with 58 figures available at , which extensively describes the evidence we have accumulated on these seven time series, in particular by presenting all relevant details so that the reader can judge for himself or herself the validity and robustness of the results.

  15. Stochastic Earthquake Rupture Modeling Using Nonparametric Co-Regionalization

    Science.gov (United States)

    Lee, Kyungbook; Song, Seok Goo

    2016-10-01

    Accurate predictions of the intensity and variability of ground motions are essential in simulation-based seismic hazard assessment. Advanced simulation-based ground motion prediction methods have been proposed to complement the empirical approach, which suffers from the lack of observed ground motion data, especially in the near-source region for large events. It is important to quantify the variability of the earthquake rupture process for future events and to produce a number of rupture scenario models to capture the variability in simulation-based ground motion predictions. In this study, we improved the previously developed stochastic earthquake rupture modeling method by applying the nonparametric co-regionalization, which was proposed in geostatistics, to the correlation models estimated from dynamically derived earthquake rupture models. The nonparametric approach adopted in this study is computationally efficient and, therefore, enables us to simulate numerous rupture scenarios, including large events (M > 7.0). It also gives us an opportunity to check the shape of true input correlation models in stochastic modeling after being deformed for permissibility. We expect that this type of modeling will improve our ability to simulate a wide range of rupture scenario models and thereby predict ground motions and perform seismic hazard assessment more accurately.

  16. A non-parametric framework for estimating threshold limit values

    Directory of Open Access Journals (Sweden)

    Ulm Kurt

    2005-11-01

    Full Text Available Abstract Background To estimate a threshold limit value for a compound known to have harmful health effects, an 'elbow' threshold model is usually applied. We are interested on non-parametric flexible alternatives. Methods We describe how a step function model fitted by isotonic regression can be used to estimate threshold limit values. This method returns a set of candidate locations, and we discuss two algorithms to select the threshold among them: the reduced isotonic regression and an algorithm considering the closed family of hypotheses. We assess the performance of these two alternative approaches under different scenarios in a simulation study. We illustrate the framework by analysing the data from a study conducted by the German Research Foundation aiming to set a threshold limit value in the exposure to total dust at workplace, as a causal agent for developing chronic bronchitis. Results In the paper we demonstrate the use and the properties of the proposed methodology along with the results from an application. The method appears to detect the threshold with satisfactory success. However, its performance can be compromised by the low power to reject the constant risk assumption when the true dose-response relationship is weak. Conclusion The estimation of thresholds based on isotonic framework is conceptually simple and sufficiently powerful. Given that in threshold value estimation context there is not a gold standard method, the proposed model provides a useful non-parametric alternative to the standard approaches and can corroborate or challenge their findings.

  17. Using non-parametric methods in econometric production analysis

    DEFF Research Database (Denmark)

    Czekaj, Tomasz Gerard; Henningsen, Arne

    2012-01-01

    Econometric estimation of production functions is one of the most common methods in applied economic production analysis. These studies usually apply parametric estimation techniques, which obligate the researcher to specify a functional form of the production function of which the Cobb-Douglas a......Econometric estimation of production functions is one of the most common methods in applied economic production analysis. These studies usually apply parametric estimation techniques, which obligate the researcher to specify a functional form of the production function of which the Cobb...... parameter estimates, but also in biased measures which are derived from the parameters, such as elasticities. Therefore, we propose to use non-parametric econometric methods. First, these can be applied to verify the functional form used in parametric production analysis. Second, they can be directly used...... to estimate production functions without the specification of a functional form. Therefore, they avoid possible misspecification errors due to the use of an unsuitable functional form. In this paper, we use parametric and non-parametric methods to identify the optimal size of Polish crop farms...

  18. Bayesian nonparametric centered random effects models with variable selection.

    Science.gov (United States)

    Yang, Mingan

    2013-03-01

    In a linear mixed effects model, it is common practice to assume that the random effects follow a parametric distribution such as a normal distribution with mean zero. However, in the case of variable selection, substantial violation of the normality assumption can potentially impact the subset selection and result in poor interpretation and even incorrect results. In nonparametric random effects models, the random effects generally have a nonzero mean, which causes an identifiability problem for the fixed effects that are paired with the random effects. In this article, we focus on a Bayesian method for variable selection. We characterize the subject-specific random effects nonparametrically with a Dirichlet process and resolve the bias simultaneously. In particular, we propose flexible modeling of the conditional distribution of the random effects with changes across the predictor space. The approach is implemented using a stochastic search Gibbs sampler to identify subsets of fixed effects and random effects to be included in the model. Simulations are provided to evaluate and compare the performance of our approach to the existing ones. We then apply the new approach to a real data example, cross-country and interlaboratory rodent uterotrophic bioassay.

  19. Wavelet Estimators in Nonparametric Regression: A Comparative Simulation Study

    Directory of Open Access Journals (Sweden)

    Anestis Antoniadis

    2001-06-01

    Full Text Available Wavelet analysis has been found to be a powerful tool for the nonparametric estimation of spatially-variable objects. We discuss in detail wavelet methods in nonparametric regression, where the data are modelled as observations of a signal contaminated with additive Gaussian noise, and provide an extensive review of the vast literature of wavelet shrinkage and wavelet thresholding estimators developed to denoise such data. These estimators arise from a wide range of classical and empirical Bayes methods treating either individual or blocks of wavelet coefficients. We compare various estimators in an extensive simulation study on a variety of sample sizes, test functions, signal-to-noise ratios and wavelet filters. Because there is no single criterion that can adequately summarise the behaviour of an estimator, we use various criteria to measure performance in finite sample situations. Insight into the performance of these estimators is obtained from graphical outputs and numerical tables. In order to provide some hints of how these estimators should be used to analyse real data sets, a detailed practical step-by-step illustration of a wavelet denoising analysis on electrical consumption is provided. Matlab codes are provided so that all figures and tables in this paper can be reproduced.

  20. Computing Economies of Scope Using Robust Partial Frontier Nonparametric Methods

    Directory of Open Access Journals (Sweden)

    Pedro Carvalho

    2016-03-01

    Full Text Available This paper proposes a methodology to examine economies of scope using the recent order-α nonparametric method. It allows us to investigate economies of scope by comparing the efficient order-α frontiers of firms that produce two or more goods with the efficient order-α frontiers of firms that produce only one good. To accomplish this, and because the order-α frontiers are irregular, we suggest to linearize them by the DEA estimator. The proposed methodology uses partial frontier nonparametric methods that are more robust than the traditional full frontier methods. By using a sample of 67 Portuguese water utilities for the period 2002–2008 and, also, a simulated sample, we prove the usefulness of the approach adopted and show that if only the full frontier methods were used, they would lead to different results. We found evidence of economies of scope in the provision of water supply and wastewater services simultaneously by water utilities in Portugal.

  1. Bayesian nonparametric dictionary learning for compressed sensing MRI.

    Science.gov (United States)

    Huang, Yue; Paisley, John; Lin, Qin; Ding, Xinghao; Fu, Xueyang; Zhang, Xiao-Ping

    2014-12-01

    We develop a Bayesian nonparametric model for reconstructing magnetic resonance images (MRIs) from highly undersampled k -space data. We perform dictionary learning as part of the image reconstruction process. To this end, we use the beta process as a nonparametric dictionary learning prior for representing an image patch as a sparse combination of dictionary elements. The size of the dictionary and patch-specific sparsity pattern are inferred from the data, in addition to other dictionary learning variables. Dictionary learning is performed directly on the compressed image, and so is tailored to the MRI being considered. In addition, we investigate a total variation penalty term in combination with the dictionary learning model, and show how the denoising property of dictionary learning removes dependence on regularization parameters in the noisy setting. We derive a stochastic optimization algorithm based on Markov chain Monte Carlo for the Bayesian model, and use the alternating direction method of multipliers for efficiently performing total variation minimization. We present empirical results on several MRI, which show that the proposed regularization framework can improve reconstruction accuracy over other methods.

  2. Covariant diagrams for one-loop matching

    Energy Technology Data Exchange (ETDEWEB)

    Zhang, Zhengkang [Michigan Univ., Ann Arbor, MI (United States). Michigan Center for Theoretical Physics; Deutsches Elektronen-Synchrotron (DESY), Hamburg (Germany)

    2016-10-15

    We present a diagrammatic formulation of recently-revived covariant functional approaches to one-loop matching from an ultraviolet (UV) theory to a low-energy effective field theory. Various terms following from a covariant derivative expansion (CDE) are represented by diagrams which, unlike conventional Feynman diagrams, involve gaugecovariant quantities and are thus dubbed ''covariant diagrams.'' The use of covariant diagrams helps organize and simplify one-loop matching calculations, which we illustrate with examples. Of particular interest is the derivation of UV model-independent universal results, which reduce matching calculations of specific UV models to applications of master formulas. We show how such derivation can be done in a more concise manner than the previous literature, and discuss how additional structures that are not directly captured by existing universal results, including mixed heavy-light loops, open covariant derivatives, and mixed statistics, can be easily accounted for.

  3. Nonparametric Estimation of Cumulative Incidence Functions for Competing Risks Data with Missing Cause of Failure

    DEFF Research Database (Denmark)

    Effraimidis, Georgios; Dahl, Christian Møller

    In this paper, we develop a fully nonparametric approach for the estimation of the cumulative incidence function with Missing At Random right-censored competing risks data. We obtain results on the pointwise asymptotic normality as well as the uniform convergence rate of the proposed nonparametric...... estimator. A simulation study that serves two purposes is provided. First, it illustrates in details how to implement our proposed nonparametric estimator. Secondly, it facilitates a comparison of the nonparametric estimator to a parametric counterpart based on the estimator of Lu and Liang (2008...

  4. Incorporation of covariates in simultaneous localization of two linked loci using affected relative pairs

    Directory of Open Access Journals (Sweden)

    Liang Kung-Yee

    2010-07-01

    Full Text Available Abstract Background Many dichotomous traits for complex diseases are often involved more than one locus and/or associated with quantitative biomarkers or environmental factors. Incorporating these quantitative variables into linkage analysis as well as localizing two linked disease loci simultaneously could therefore improve the efficiency in mapping genes. We extended the robust multipoint Identity-by-Descent (IBD approach with incorporation of covariates developed previously to simultaneously estimate two linked loci using different types of affected relative pairs (ARPs. Results We showed that the efficiency was enhanced by incorporating a quantitative covariate parametrically or non-parametrically while localizing two disease loci using ARPs. In addition to its help in identifying factors associated with the disease and in improving the efficiency in estimating disease loci, this extension also allows investigators to account for heterogeneity in risk-ratios for different ARPs. Data released from the collaborative study on the genetics of alcoholism (COGA for Genetic Analysis Workshop 14 (GAW 14 were used to illustrate the application of this extended method. Conclusions The simulation studies and example illustrated that the efficiency in estimating disease loci was demonstratively enhanced by incorporating a quantitative covariate and by using all relative pairs while mapping two linked loci simultaneously.

  5. Improved Mainlobe Interference Suppression Based on Blocking Matrix Preprocess

    National Research Council Canada - National Science Library

    Yang, Jie; Liu, Congfeng

    2015-01-01

    ... on the combination of diagonal loading and linear constraints. Therein, the reason for mainlobe direction shifting is analyzed and found to be that the covariance matrix mismatch exists in the realization of the adaptive beamforming...

  6. The covariate-adjusted frequency plot.

    Science.gov (United States)

    Holling, Heinz; Böhning, Walailuck; Böhning, Dankmar; Formann, Anton K

    2016-04-01

    Count data arise in numerous fields of interest. Analysis of these data frequently require distributional assumptions. Although the graphical display of a fitted model is straightforward in the univariate scenario, this becomes more complex if covariate information needs to be included into the model. Stratification is one way to proceed, but has its limitations if the covariate has many levels or the number of covariates is large. The article suggests a marginal method which works even in the case that all possible covariate combinations are different (i.e. no covariate combination occurs more than once). For each covariate combination the fitted model value is computed and then summed over the entire data set. The technique is quite general and works with all count distributional models as well as with all forms of covariate modelling. The article provides illustrations of the method for various situations and also shows that the proposed estimator as well as the empirical count frequency are consistent with respect to the same parameter.

  7. Solving the differential biochemical Jacobian from metabolomics covariance data.

    Science.gov (United States)

    Nägele, Thomas; Mair, Andrea; Sun, Xiaoliang; Fragner, Lena; Teige, Markus; Weckwerth, Wolfram

    2014-01-01

    High-throughput molecular analysis has become an integral part in organismal systems biology. In contrast, due to a missing systematic linkage of the data with functional and predictive theoretical models of the underlying metabolic network the understanding of the resulting complex data sets is lacking far behind. Here, we present a biomathematical method addressing this problem by using metabolomics data for the inverse calculation of a biochemical Jacobian matrix, thereby linking computer-based genome-scale metabolic reconstruction and in vivo metabolic dynamics. The incongruity of metabolome coverage by typical metabolite profiling approaches and genome-scale metabolic reconstruction was solved by the design of superpathways to define a metabolic interaction matrix. A differential biochemical Jacobian was calculated using an approach which links this metabolic interaction matrix and the covariance of metabolomics data satisfying a Lyapunov equation. The predictions of the differential Jacobian from real metabolomic data were found to be correct by testing the corresponding enzymatic activities. Moreover it is demonstrated that the predictions of the biochemical Jacobian matrix allow for the design of parameter optimization strategies for ODE-based kinetic models of the system. The presented concept combines dynamic modelling strategies with large-scale steady state profiling approaches without the explicit knowledge of individual kinetic parameters. In summary, the presented strategy allows for the identification of regulatory key processes in the biochemical network directly from metabolomics data and is a fundamental achievement for the functional interpretation of metabolomics data.

  8. Estimation of Low-Rank Covariance Function

    OpenAIRE

    Koltchinskii, Vladimir; Lounici, Karim; Tsybakov, Alexander B.

    2015-01-01

    We consider the problem of estimating a low rank covariance function $K(t,u)$ of a Gaussian process $S(t), t\\in [0,1]$ based on $n$ i.i.d. copies of $S$ observed in a white noise. We suggest a new estimation procedure adapting simultaneously to the low rank structure and the smoothness of the covariance function. The new procedure is based on nuclear norm penalization and exhibits superior performances as compared to the sample covariance function by a polynomial factor in the sample size $n$...

  9. Robust Depth-Weighted Wavelet for Nonparametric Regression Models

    Institute of Scientific and Technical Information of China (English)

    Lu LIN

    2005-01-01

    In the nonpaxametric regression models, the original regression estimators including kernel estimator, Fourier series estimator and wavelet estimator are always constructed by the weighted sum of data, and the weights depend only on the distance between the design points and estimation points. As a result these estimators are not robust to the perturbations in data. In order to avoid this problem, a new nonparametric regression model, called the depth-weighted regression model, is introduced and then the depth-weighted wavelet estimation is defined. The new estimation is robust to the perturbations in data, which attains very high breakdown value close to 1/2. On the other hand, some asymptotic behaviours such as asymptotic normality are obtained. Some simulations illustrate that the proposed wavelet estimator is more robust than the original wavelet estimator and, as a price to pay for the robustness, the new method is slightly less efficient than the original method.

  10. Nonparametric Bayesian inference of the microcanonical stochastic block model

    CERN Document Server

    Peixoto, Tiago P

    2016-01-01

    A principled approach to characterize the hidden modular structure of networks is to formulate generative models, and then infer their parameters from data. When the desired structure is composed of modules or "communities", a suitable choice for this task is the stochastic block model (SBM), where nodes are divided into groups, and the placement of edges is conditioned on the group memberships. Here, we present a nonparametric Bayesian method to infer the modular structure of empirical networks, including the number of modules and their hierarchical organization. We focus on a microcanonical variant of the SBM, where the structure is imposed via hard constraints. We show how this simple model variation allows simultaneously for two important improvements over more traditional inference approaches: 1. Deeper Bayesian hierarchies, with noninformative priors replaced by sequences of priors and hyperpriors, that not only remove limitations that seriously degrade the inference on large networks, but also reveal s...

  11. A Non-Parametric Spatial Independence Test Using Symbolic Entropy

    Directory of Open Access Journals (Sweden)

    López Hernández, Fernando

    2008-01-01

    Full Text Available In the present paper, we construct a new, simple, consistent and powerful test forspatial independence, called the SG test, by using symbolic dynamics and symbolic entropyas a measure of spatial dependence. We also give a standard asymptotic distribution of anaffine transformation of the symbolic entropy under the null hypothesis of independencein the spatial process. The test statistic and its standard limit distribution, with theproposed symbolization, are invariant to any monotonuous transformation of the data.The test applies to discrete or continuous distributions. Given that the test is based onentropy measures, it avoids smoothed nonparametric estimation. We include a MonteCarlo study of our test, together with the well-known Moran’s I, the SBDS (de Graaffet al, 2001 and (Brett and Pinkse, 1997 non parametric test, in order to illustrate ourapproach.

  12. Analyzing single-molecule time series via nonparametric Bayesian inference.

    Science.gov (United States)

    Hines, Keegan E; Bankston, John R; Aldrich, Richard W

    2015-02-03

    The ability to measure the properties of proteins at the single-molecule level offers an unparalleled glimpse into biological systems at the molecular scale. The interpretation of single-molecule time series has often been rooted in statistical mechanics and the theory of Markov processes. While existing analysis methods have been useful, they are not without significant limitations including problems of model selection and parameter nonidentifiability. To address these challenges, we introduce the use of nonparametric Bayesian inference for the analysis of single-molecule time series. These methods provide a flexible way to extract structure from data instead of assuming models beforehand. We demonstrate these methods with applications to several diverse settings in single-molecule biophysics. This approach provides a well-constrained and rigorously grounded method for determining the number of biophysical states underlying single-molecule data. Copyright © 2015 Biophysical Society. Published by Elsevier Inc. All rights reserved.

  13. Analyzing multiple spike trains with nonparametric Granger causality.

    Science.gov (United States)

    Nedungadi, Aatira G; Rangarajan, Govindan; Jain, Neeraj; Ding, Mingzhou

    2009-08-01

    Simultaneous recordings of spike trains from multiple single neurons are becoming commonplace. Understanding the interaction patterns among these spike trains remains a key research area. A question of interest is the evaluation of information flow between neurons through the analysis of whether one spike train exerts causal influence on another. For continuous-valued time series data, Granger causality has proven an effective method for this purpose. However, the basis for Granger causality estimation is autoregressive data modeling, which is not directly applicable to spike trains. Various filtering options distort the properties of spike trains as point processes. Here we propose a new nonparametric approach to estimate Granger causality directly from the Fourier transforms of spike train data. We validate the method on synthetic spike trains generated by model networks of neurons with known connectivity patterns and then apply it to neurons simultaneously recorded from the thalamus and the primary somatosensory cortex of a squirrel monkey undergoing tactile stimulation.

  14. Prior processes and their applications nonparametric Bayesian estimation

    CERN Document Server

    Phadia, Eswar G

    2016-01-01

    This book presents a systematic and comprehensive treatment of various prior processes that have been developed over the past four decades for dealing with Bayesian approach to solving selected nonparametric inference problems. This revised edition has been substantially expanded to reflect the current interest in this area. After an overview of different prior processes, it examines the now pre-eminent Dirichlet process and its variants including hierarchical processes, then addresses new processes such as dependent Dirichlet, local Dirichlet, time-varying and spatial processes, all of which exploit the countable mixture representation of the Dirichlet process. It subsequently discusses various neutral to right type processes, including gamma and extended gamma, beta and beta-Stacy processes, and then describes the Chinese Restaurant, Indian Buffet and infinite gamma-Poisson processes, which prove to be very useful in areas such as machine learning, information retrieval and featural modeling. Tailfree and P...

  15. Using non-parametric methods in econometric production analysis

    DEFF Research Database (Denmark)

    Czekaj, Tomasz Gerard; Henningsen, Arne

    Econometric estimation of production functions is one of the most common methods in applied economic production analysis. These studies usually apply parametric estimation techniques, which obligate the researcher to specify the functional form of the production function. Most often, the Cobb......-Douglas or the Translog production function is used. However, the specification of a functional form for the production function involves the risk of specifying a functional form that is not similar to the “true” relationship between the inputs and the output. This misspecification might result in biased estimation...... results—including measures that are of interest of applied economists, such as elasticities. Therefore, we propose to use nonparametric econometric methods. First, they can be applied to verify the functional form used in parametric estimations of production functions. Second, they can be directly used...

  16. Nonparametric Estimation of Distributions in Random Effects Models

    KAUST Repository

    Hart, Jeffrey D.

    2011-01-01

    We propose using minimum distance to obtain nonparametric estimates of the distributions of components in random effects models. A main setting considered is equivalent to having a large number of small datasets whose locations, and perhaps scales, vary randomly, but which otherwise have a common distribution. Interest focuses on estimating the distribution that is common to all datasets, knowledge of which is crucial in multiple testing problems where a location/scale invariant test is applied to every small dataset. A detailed algorithm for computing minimum distance estimates is proposed, and the usefulness of our methodology is illustrated by a simulation study and an analysis of microarray data. Supplemental materials for the article, including R-code and a dataset, are available online. © 2011 American Statistical Association.

  17. Curve registration by nonparametric goodness-of-fit testing

    CERN Document Server

    Dalalyan, Arnak

    2011-01-01

    The problem of curve registration appears in many different areas of applications ranging from neuroscience to road traffic modeling. In the present work, we propose a nonparametric testing framework in which we develop a generalized likelihood ratio test to perform curve registration. We first prove that, under the null hypothesis, the resulting test statistic is asymptotically distributed as a chi-squared random variable. This result, often referred to as Wilks' phenomenon, provides a natural threshold for the test of a prescribed asymptotic significance level and a natural measure of lack-of-fit in terms of the p-value of the chi squared test. We also prove that the proposed test is consistent, i.e., its power is asymptotically equal to 1. Some numerical experiments on synthetic datasets are reported as well.

  18. Nonparametric forecasting of low-dimensional dynamical systems.

    Science.gov (United States)

    Berry, Tyrus; Giannakis, Dimitrios; Harlim, John

    2015-03-01

    This paper presents a nonparametric modeling approach for forecasting stochastic dynamical systems on low-dimensional manifolds. The key idea is to represent the discrete shift maps on a smooth basis which can be obtained by the diffusion maps algorithm. In the limit of large data, this approach converges to a Galerkin projection of the semigroup solution to the underlying dynamics on a basis adapted to the invariant measure. This approach allows one to quantify uncertainties (in fact, evolve the probability distribution) for nontrivial dynamical systems with equation-free modeling. We verify our approach on various examples, ranging from an inhomogeneous anisotropic stochastic differential equation on a torus, the chaotic Lorenz three-dimensional model, and the Niño-3.4 data set which is used as a proxy of the El Niño Southern Oscillation.

  19. Nonparametric Model of Smooth Muscle Force Production During Electrical Stimulation.

    Science.gov (United States)

    Cole, Marc; Eikenberry, Steffen; Kato, Takahide; Sandler, Roman A; Yamashiro, Stanley M; Marmarelis, Vasilis Z

    2017-03-01

    A nonparametric model of smooth muscle tension response to electrical stimulation was estimated using the Laguerre expansion technique of nonlinear system kernel estimation. The experimental data consisted of force responses of smooth muscle to energy-matched alternating single pulse and burst current stimuli. The burst stimuli led to at least a 10-fold increase in peak force in smooth muscle from Mytilus edulis, despite the constant energy constraint. A linear model did not fit the data. However, a second-order model fit the data accurately, so the higher-order models were not required to fit the data. Results showed that smooth muscle force response is not linearly related to the stimulation power.

  20. Nonparametric estimation of stochastic differential equations with sparse Gaussian processes

    Science.gov (United States)

    García, Constantino A.; Otero, Abraham; Félix, Paulo; Presedo, Jesús; Márquez, David G.

    2017-08-01

    The application of stochastic differential equations (SDEs) to the analysis of temporal data has attracted increasing attention, due to their ability to describe complex dynamics with physically interpretable equations. In this paper, we introduce a nonparametric method for estimating the drift and diffusion terms of SDEs from a densely observed discrete time series. The use of Gaussian processes as priors permits working directly in a function-space view and thus the inference takes place directly in this space. To cope with the computational complexity that requires the use of Gaussian processes, a sparse Gaussian process approximation is provided. This approximation permits the efficient computation of predictions for the drift and diffusion terms by using a distribution over a small subset of pseudosamples. The proposed method has been validated using both simulated data and real data from economy and paleoclimatology. The application of the method to real data demonstrates its ability to capture the behavior of complex systems.

  1. Indoor Positioning Using Nonparametric Belief Propagation Based on Spanning Trees

    Directory of Open Access Journals (Sweden)

    Savic Vladimir

    2010-01-01

    Full Text Available Nonparametric belief propagation (NBP is one of the best-known methods for cooperative localization in sensor networks. It is capable of providing information about location estimation with appropriate uncertainty and to accommodate non-Gaussian distance measurement errors. However, the accuracy of NBP is questionable in loopy networks. Therefore, in this paper, we propose a novel approach, NBP based on spanning trees (NBP-ST created by breadth first search (BFS method. In addition, we propose a reliable indoor model based on obtained measurements in our lab. According to our simulation results, NBP-ST performs better than NBP in terms of accuracy and communication cost in the networks with high connectivity (i.e., highly loopy networks. Furthermore, the computational and communication costs are nearly constant with respect to the transmission radius. However, the drawbacks of proposed method are a little bit higher computational cost and poor performance in low-connected networks.

  2. Revealing components of the galaxy population through nonparametric techniques

    CERN Document Server

    Bamford, Steven P; Nichol, Robert C; Miller, Christopher J; Wasserman, Larry; Genovese, Christopher R; Freeman, Peter E

    2008-01-01

    The distributions of galaxy properties vary with environment, and are often multimodal, suggesting that the galaxy population may be a combination of multiple components. The behaviour of these components versus environment holds details about the processes of galaxy development. To release this information we apply a novel, nonparametric statistical technique, identifying four components present in the distribution of galaxy H$\\alpha$ emission-line equivalent-widths. We interpret these components as passive, star-forming, and two varieties of active galactic nuclei. Independent of this interpretation, the properties of each component are remarkably constant as a function of environment. Only their relative proportions display substantial variation. The galaxy population thus appears to comprise distinct components which are individually independent of environment, with galaxies rapidly transitioning between components as they move into denser environments.

  3. Multi-Directional Non-Parametric Analysis of Agricultural Efficiency

    DEFF Research Database (Denmark)

    Balezentis, Tomas

    This thesis seeks to develop methodologies for assessment of agricultural efficiency and employ them to Lithuanian family farms. In particular, we focus on three particular objectives throughout the research: (i) to perform a fully non-parametric analysis of efficiency effects, (ii) to extend...... relative to labour, intermediate consumption and land (in some cases land was not treated as a discretionary input). These findings call for further research on relationships among financial structure, investment decisions, and efficiency in Lithuanian family farms. Application of different techniques...... of stochasticity associated with Lithuanian family farm performance. The former technique showed that the farms differed in terms of the mean values and variance of the efficiency scores over time with some clear patterns prevailing throughout the whole research period. The fuzzy Free Disposal Hull showed...

  4. Binary Classifier Calibration Using a Bayesian Non-Parametric Approach.

    Science.gov (United States)

    Naeini, Mahdi Pakdaman; Cooper, Gregory F; Hauskrecht, Milos

    Learning probabilistic predictive models that are well calibrated is critical for many prediction and decision-making tasks in Data mining. This paper presents two new non-parametric methods for calibrating outputs of binary classification models: a method based on the Bayes optimal selection and a method based on the Bayesian model averaging. The advantage of these methods is that they are independent of the algorithm used to learn a predictive model, and they can be applied in a post-processing step, after the model is learned. This makes them applicable to a wide variety of machine learning models and methods. These calibration methods, as well as other methods, are tested on a variety of datasets in terms of both discrimination and calibration performance. The results show the methods either outperform or are comparable in performance to the state-of-the-art calibration methods.

  5. Parametric or nonparametric? A parametricness index for model selection

    CERN Document Server

    Liu, Wei; 10.1214/11-AOS899

    2012-01-01

    In model selection literature, two classes of criteria perform well asymptotically in different situations: Bayesian information criterion (BIC) (as a representative) is consistent in selection when the true model is finite dimensional (parametric scenario); Akaike's information criterion (AIC) performs well in an asymptotic efficiency when the true model is infinite dimensional (nonparametric scenario). But there is little work that addresses if it is possible and how to detect the situation that a specific model selection problem is in. In this work, we differentiate the two scenarios theoretically under some conditions. We develop a measure, parametricness index (PI), to assess whether a model selected by a potentially consistent procedure can be practically treated as the true model, which also hints on AIC or BIC is better suited for the data for the goal of estimating the regression function. A consequence is that by switching between AIC and BIC based on the PI, the resulting regression estimator is si...

  6. Nonparametric reconstruction of the Om diagnostic to test LCDM

    CERN Document Server

    Escamilla-Rivera, Celia

    2015-01-01

    Cosmic acceleration is usually related with the unknown dark energy, which equation of state, w(z), is constrained and numerically confronted with independent astrophysical data. In order to make a diagnostic of w(z), the introduction of a null test of dark energy can be done using a diagnostic function of redshift, Om. In this work we present a nonparametric reconstruction of this diagnostic using the so-called Loess-Simex factory to test the concordance model with the advantage that this approach offers an alternative way to relax the use of priors and find a possible 'w' that reliably describe the data with no previous knowledge of a cosmological model. Our results demonstrate that the method applied to the dynamical Om diagnostic finds a preference for a dark energy model with equation of state w =-2/3, which correspond to a static domain wall network.

  7. Evaluation of Nonparametric Probabilistic Forecasts of Wind Power

    DEFF Research Database (Denmark)

    Pinson, Pierre; Møller, Jan Kloppenborg; Nielsen, Henrik Aalborg, orlov 31.07.2008;

    likely outcome for each look-ahead time, but also with uncertainty estimates given by probabilistic forecasts. In order to avoid assumptions on the shape of predictive distributions, these probabilistic predictions are produced from nonparametric methods, and then take the form of a single or a set...... of quantile forecasts. The required and desirable properties of such probabilistic forecasts are defined and a framework for their evaluation is proposed. This framework is applied for evaluating the quality of two statistical methods producing full predictive distributions from point predictions of wind......Predictions of wind power production for horizons up to 48-72 hour ahead comprise a highly valuable input to the methods for the daily management or trading of wind generation. Today, users of wind power predictions are not only provided with point predictions, which are estimates of the most...

  8. Matrix theory

    CERN Document Server

    Franklin, Joel N

    2003-01-01

    Mathematically rigorous introduction covers vector and matrix norms, the condition-number of a matrix, positive and irreducible matrices, much more. Only elementary algebra and calculus required. Includes problem-solving exercises. 1968 edition.

  9. Equity and efficiency in private and public education: a nonparametric comparison

    NARCIS (Netherlands)

    L. Cherchye; K. de Witte; E. Ooghe; I. Nicaise

    2007-01-01

    We present a nonparametric approach for the equity and efficiency evaluation of (private and public) primary schools in Flanders. First, we use a nonparametric (Data Envelopment Analysis) model that is specially tailored to assess educational efficiency at the pupil level. The model accounts for the

  10. Parametric and non-parametric modeling of short-term synaptic plasticity. Part II: Experimental study.

    Science.gov (United States)

    Song, Dong; Wang, Zhuo; Marmarelis, Vasilis Z; Berger, Theodore W

    2009-02-01

    This paper presents a synergistic parametric and non-parametric modeling study of short-term plasticity (STP) in the Schaffer collateral to hippocampal CA1 pyramidal neuron (SC) synapse. Parametric models in the form of sets of differential and algebraic equations have been proposed on the basis of the current understanding of biological mechanisms active within the system. Non-parametric Poisson-Volterra models are obtained herein from broadband experimental input-output data. The non-parametric model is shown to provide better prediction of the experimental output than a parametric model with a single set of facilitation/depression (FD) process. The parametric model is then validated in terms of its input-output transformational properties using the non-parametric model since the latter constitutes a canonical and more complete representation of the synaptic nonlinear dynamics. Furthermore, discrepancies between the experimentally-derived non-parametric model and the equivalent non-parametric model of the parametric model suggest the presence of multiple FD processes in the SC synapses. Inclusion of an additional set of FD process in the parametric model makes it replicate better the characteristics of the experimentally-derived non-parametric model. This improved parametric model in turn provides the requisite biological interpretability that the non-parametric model lacks.

  11. Non-parametric tests of productive efficiency with errors-in-variables

    NARCIS (Netherlands)

    Kuosmanen, T.K.; Post, T.; Scholtes, S.

    2007-01-01

    We develop a non-parametric test of productive efficiency that accounts for errors-in-variables, following the approach of Varian. [1985. Nonparametric analysis of optimizing behavior with measurement error. Journal of Econometrics 30(1/2), 445-458]. The test is based on the general Pareto-Koopmans

  12. Equity and efficiency in private and public education: a nonparametric comparison

    NARCIS (Netherlands)

    Cherchye, L.; de Witte, K.; Ooghe, E.; Nicaise, I.

    2007-01-01

    We present a nonparametric approach for the equity and efficiency evaluation of (private and public) primary schools in Flanders. First, we use a nonparametric (Data Envelopment Analysis) model that is specially tailored to assess educational efficiency at the pupil level. The model accounts for the

  13. Earth Observing System Covariance Realism Updates

    Science.gov (United States)

    Ojeda Romero, Juan A.; Miguel, Fred

    2017-01-01

    This presentation will be given at the International Earth Science Constellation Mission Operations Working Group meetings June 13-15, 2017 to discuss the Earth Observing System Covariance Realism updates.

  14. Covariant Quantization with Extended BRST Symmetry

    CERN Document Server

    Geyer, B; Lavrov, P M

    1999-01-01

    A short rewiev of covariant quantization methods based on BRST-antiBRST symmetry is given. In particular problems of correct definition of Sp(2) symmetric quantization scheme known as triplectic quantization are considered.

  15. Conformally covariant parametrizations for relativistic initial data

    Science.gov (United States)

    Delay, Erwann

    2017-01-01

    We revisit the Lichnerowicz-York method, and an alternative method of York, in order to obtain some conformally covariant systems. This type of parametrization is certainly more natural for non constant mean curvature initial data.

  16. Semi-parametric regression: Efficiency gains from modeling the nonparametric part

    CERN Document Server

    Yu, Kyusang; Park, Byeong U; 10.3150/10-BEJ296

    2011-01-01

    It is widely admitted that structured nonparametric modeling that circumvents the curse of dimensionality is important in nonparametric estimation. In this paper we show that the same holds for semi-parametric estimation. We argue that estimation of the parametric component of a semi-parametric model can be improved essentially when more structure is put into the nonparametric part of the model. We illustrate this for the partially linear model, and investigate efficiency gains when the nonparametric part of the model has an additive structure. We present the semi-parametric Fisher information bound for estimating the parametric part of the partially linear additive model and provide semi-parametric efficient estimators for which we use a smooth backfitting technique to deal with the additive nonparametric part. We also present the finite sample performances of the proposed estimators and analyze Boston housing data as an illustration.

  17. Covariant action for type IIB supergravity

    Science.gov (United States)

    Sen, Ashoke

    2016-07-01

    Taking clues from the recent construction of the covariant action for type II and heterotic string field theories, we construct a manifestly Lorentz covariant action for type IIB supergravity, and discuss its gauge fixing maintaining manifest Lorentz invariance. The action contains a (non-gravitating) free 4-form field besides the usual fields of type IIB supergravity. This free field, being completely decoupled from the interacting sector, has no physical consequence.

  18. Functional CLT for sample covariance matrices

    CERN Document Server

    Bai, Zhidong; Zhou, Wang; 10.3150/10-BEJ250

    2010-01-01

    Using Bernstein polynomial approximations, we prove the central limit theorem for linear spectral statistics of sample covariance matrices, indexed by a set of functions with continuous fourth order derivatives on an open interval including $[(1-\\sqrt{y})^2,(1+\\sqrt{y})^2]$, the support of the Mar\\u{c}enko--Pastur law. We also derive the explicit expressions for asymptotic mean and covariance functions.

  19. On the covariance of residual lives

    Directory of Open Access Journals (Sweden)

    N. Unnikrishnan Nair

    2007-10-01

    Full Text Available Various properties of residual life such as mean, median, percentiles, variance etc have been discussed in literature on reliability and survival analysis. However a detailed study on covariance between residual lives in a two component system does not seem to have been undertaken. The present paper discusses various properties of product moment and covariance of residual lives. Relationships the product moment has with mean residual life and failure rate are studied and some characterizations are established.

  20. Covariant Hamilton equations for field theory

    Energy Technology Data Exchange (ETDEWEB)

    Giachetta, Giovanni [Department of Mathematics and Physics, University of Camerino, Camerino (Italy); Mangiarotti, Luigi [Department of Mathematics and Physics, University of Camerino, Camerino (Italy)]. E-mail: mangiaro@camserv.unicam.it; Sardanashvily, Gennadi [Department of Theoretical Physics, Physics Faculty, Moscow State University, Moscow (Russian Federation)]. E-mail: sard@grav.phys.msu.su

    1999-09-24

    We study the relations between the equations of first-order Lagrangian field theory on fibre bundles and the covariant Hamilton equations on the finite-dimensional polysymplectic phase space of covariant Hamiltonian field theory. If a Lagrangian is hyperregular, these equations are equivalent. A degenerate Lagrangian requires a set of associated Hamiltonian forms in order to exhaust all solutions of the Euler-Lagrange equations. The case of quadratic degenerate Lagrangians is studied in detail. (author)