Exascale Data Analysis

CERN Multimedia

CERN. Geneva; Fitch, Blake

2011-01-01

Traditionaly, the primary role of supercomputers was to create data, primarily for simulation applications. Due to usage and technology trends, supercomputers are increasingly also used for data analysis. Some of this data is from simulations, but there is also a rapidly increasingly amount of real-world science and business data to be analyzed. We briefly overview Blue Gene and other current supercomputer architectures. We outline future architectures, up to the Exascale supercomputers expected in the 2020 time frame. We focus on the data analysis challenges and opportunites, especially those concerning Flash and other up-and-coming storage class memory. About the speakers Blake G. Fitch has been with IBM Research, Yorktown Heights, NY since 1987, mainly pursuing interests in parallel systems. He joined the Scalable Parallel Systems Group in 1990, contributing to research and development that culminated in the IBM scalable parallel system (SP*) product. His research interests have focused on applicatio...

Python data analysis

CERN Document Server

Idris, Ivan

2014-01-01

This book is for programmers, scientists, and engineers who have knowledge of the Python language and know the basics of data science. It is for those who wish to learn different data analysis methods using Python and its libraries. This book contains all the basic ingredients you need to become an expert data analyst.

Virtual Data in CMS Analysis

CERN Document Server

Arbree, A; Bourilkov, D; Cavanaugh, R J; Graham, G; Rodríguez, J; Wilde, M; Zhao, Y

2003-01-01

The use of virtual data for enhancing the collaboration between large groups of scientists is explored in several ways: - by defining ``virtual'' parameter spaces which can be searched and shared in an organized way by a collaboration of scientists in the course of their analysis - by providing a mechanism to log the provenance of results and the ability to trace them back to the various stages in the analysis of real or simulated data - by creating ``check points'' in the course of an analysis to permit collaborators to explore their own analysis branches by refining selections, improving the signal to background ratio, varying the estimation of parameters, etc. - by facilitating the audit of an analysis and the reproduction of its results by a different group, or in a peer review context. We describe a prototype for the analysis of data from the CMS experiment based on the virtual data system Chimera and the object-oriented data analysis framework ROOT. The Chimera system is used to chain together several s...

Bayesian nonparametric data analysis

CERN Document Server

Müller, Peter; Jara, Alejandro; Hanson, Tim

2015-01-01

This book reviews nonparametric Bayesian methods and models that have proven useful in the context of data analysis. Rather than providing an encyclopedic review of probability models, the book’s structure follows a data analysis perspective. As such, the chapters are organized by traditional data analysis problems. In selecting specific nonparametric models, simpler and more traditional models are favored over specialized ones. The discussed methods are illustrated with a wealth of examples, including applications ranging from stylized examples to case studies from recent literature. The book also includes an extensive discussion of computational methods and details on their implementation. R code for many examples is included in on-line software pages.

The data analysis handbook

CERN Document Server

Frank, IE

1994-01-01

Analyzing observed or measured data is an important step in applied sciences. The recent increase in computer capacity has resulted in a revolution both in data collection and data analysis. An increasing number of scientists, researchers and students are venturing into statistical data analysis; hence the need for more guidance in this field, which was previously dominated mainly by statisticians. This handbook fills the gap in the range of textbooks on data analysis. Written in a dictionary format, it will serve as a comprehensive reference book in a rapidly growing field. However, this book is more structured than an ordinary dictionary, where each entry is a separate, self-contained entity. The authors provide not only definitions and short descriptions, but also offer an overview of the different topics. Therefore, the handbook can also be used as a companion to textbooks for undergraduate or graduate courses. 1700 entries are given in alphabetical order grouped into 20 topics and each topic is organized...

Data near processing support for climate data analysis

Science.gov (United States)

Kindermann, Stephan; Ehbrecht, Carsten; Hempelmann, Nils

2016-04-01

Climate data repositories grow in size exponentially. Scalable data near processing capabilities are required to meet future data analysis requirements and to replace current "data download and process at home" workflows and approaches. On one hand side, these processing capabilities should be accessible via standardized interfaces (e.g. OGC WPS), on the other side a large variety of processing tools, toolboxes and deployment alternatives have to be supported and maintained at the data/processing center. We present a community approach of a modular and flexible system supporting the development, deployment and maintenace of OGC-WPS based web processing services. This approach is organized in an open source github project (called "bird-house") supporting individual processing services ("birds", e.g. climate index calculations, model data ensemble calculations), which rely on basic common infrastructural components (e.g. installation and deployment recipes, analysis code dependencies management). To support easy deployment at data centers as well as home institutes (e.g. for testing and development) the system supports the management of the often very complex package dependency chain of climate data analysis packages as well as docker based packaging and installation. We present a concrete deployment scenario at the German Climate Computing Center (DKRZ). The DKRZ one hand side hosts a multi-petabyte climate archive which is integrated e.g. into the european ENES and worldwide ESGF data infrastructure, and on the other hand hosts an HPC center supporting (model) data production and data analysis. The deployment scenario also includes openstack based data cloud services to support data import and data distribution for bird-house based WPS web processing services. Current challenges for inter-institutionnal deployments of web processing services supporting the european and international climate modeling community as well as the climate impact community are highlighted

Dynamic data analysis modeling data with differential equations

CERN Document Server

Ramsay, James

2017-01-01

This text focuses on the use of smoothing methods for developing and estimating differential equations following recent developments in functional data analysis and building on techniques described in Ramsay and Silverman (2005) Functional Data Analysis. The central concept of a dynamical system as a buffer that translates sudden changes in input into smooth controlled output responses has led to applications of previously analyzed data, opening up entirely new opportunities for dynamical systems. The technical level has been kept low so that those with little or no exposure to differential equations as modeling objects can be brought into this data analysis landscape. There are already many texts on the mathematical properties of ordinary differential equations, or dynamic models, and there is a large literature distributed over many fields on models for real world processes consisting of differential equations. However, a researcher interested in fitting such a model to data, or a statistician interested in...

Abstract Interfaces for Data Analysis Component Architecture for Data Analysis Tools

CERN Document Server

Barrand, G; Dönszelmann, M; Johnson, A; Pfeiffer, A

2001-01-01

The fast turnover of software technologies, in particular in the domain of interactivity (covering user interface and visualisation), makes it difficult for a small group of people to produce complete and polished software-tools before the underlying technologies make them obsolete. At the HepVis '99 workshop, a working group has been formed to improve the production of software tools for data analysis in HENP. Beside promoting a distributed development organisation, one goal of the group is to systematically design a set of abstract interfaces based on using modern OO analysis and OO design techniques. An initial domain analysis has come up with several categories (components) found in typical data analysis tools: Histograms, Ntuples, Functions, Vectors, Fitter, Plotter, Analyzer and Controller. Special emphasis was put on reducing the couplings between the categories to a minimum, thus optimising re-use and maintainability of any component individually. The interfaces have been defined in Java and C++ and i...

Beginning statistics with data analysis

CERN Document Server

Mosteller, Frederick; Rourke, Robert EK

2013-01-01

This introduction to the world of statistics covers exploratory data analysis, methods for collecting data, formal statistical inference, and techniques of regression and analysis of variance. 1983 edition.

Workbook on data analysis

International Nuclear Information System (INIS)

Hopke, P.K.

2000-01-01

As a consequence of various IAEA programmes to sample airborne particulate matter and determine its elemental composition, the participating research groups are accumulating data on the composition of the atmospheric aerosol. It is necessary to consider ways in which these data can be utilized in order to be certain that the data obtained are correct and that the information then being transmitted to others who may make decisions based on such information is as representative and correct as possible. In order to both examine the validity of those data and extract appropriate information from them, it is necessary to utilize a variety of data analysis methods. The objective of this workbook is to provide a guide with examples of utilizing data analysis on airborne particle composition data using a spreadsheet program (EXCEL) and a personal computer based statistical package (StatGraphics)

Workbook on data analysis

Energy Technology Data Exchange (ETDEWEB)

Hopke, P K [Department of Chemistry, Clarkson Univ., Potsdam, NY (United States)

2000-07-01

As a consequence of various IAEA programmes to sample airborne particulate matter and determine its elemental composition, the participating research groups are accumulating data on the composition of the atmospheric aerosol. It is necessary to consider ways in which these data can be utilized in order to be certain that the data obtained are correct and that the information then being transmitted to others who may make decisions based on such information is as representative and correct as possible. In order to both examine the validity of those data and extract appropriate information from them, it is necessary to utilize a variety of data analysis methods. The objective of this workbook is to provide a guide with examples of utilizing data analysis on airborne particle composition data using a spreadsheet program (EXCEL) and a personal computer based statistical package (StatGraphics)

Qualitative data analysis: conceptual and practical considerations.

Science.gov (United States)

Liamputtong, Pranee

2009-08-01

Qualitative inquiry requires that collected data is organised in a meaningful way, and this is referred to as data analysis. Through analytic processes, researchers turn what can be voluminous data into understandable and insightful analysis. This paper sets out the different approaches that qualitative researchers can use to make sense of their data including thematic analysis, narrative analysis, discourse analysis and semiotic analysis and discusses the ways that qualitative researchers can analyse their data. I first discuss salient issues in performing qualitative data analysis, and then proceed to provide some suggestions on different methods of data analysis in qualitative research. Finally, I provide some discussion on the use of computer-assisted data analysis.

SP mountain data analysis

Science.gov (United States)

Rawson, R. F.; Hamilton, R. E.; Liskow, C. L.; Dias, A. R.; Jackson, P. L.

1981-01-01

An analysis of synthetic aperture radar data of SP Mountain was undertaken to demonstrate the use of digital image processing techniques to aid in geologic interpretation of SAR data. These data were collected with the ERIM X- and L-band airborne SAR using like- and cross-polarizations. The resulting signal films were used to produce computer compatible tapes, from which four-channel imagery was generated. Slant range-to-ground range and range-azimuth-scale corrections were made in order to facilitate image registration; intensity corrections were also made. Manual interpretation of the imagery showed that L-band represented the geology of the area better than X-band. Several differences between the various images were also noted. Further digital analysis of the corrected data was done for enhancement purposes. This analysis included application of an MSS differencing routine and development of a routine for removal of relief displacement. It was found that accurate registration of the SAR channels is critical to the effectiveness of the differencing routine. Use of the relief displacement algorithm on the SP Mountain data demonstrated the feasibility of the technique.

Expediting Scientific Data Analysis with Reorganization of Data

Energy Technology Data Exchange (ETDEWEB)

Byna, Surendra; Wu, Kesheng

2013-08-19

Data producers typically optimize the layout of data files to minimize the write time. In most cases, data analysis tasks read these files in access patterns different from the write patterns causing poor read performance. In this paper, we introduce Scientific Data Services (SDS), a framework for bridging the performance gap between writing and reading scientific data. SDS reorganizes data to match the read patterns of analysis tasks and enables transparent data reads from the reorganized data. We implemented a HDF5 Virtual Object Layer (VOL) plugin to redirect the HDF5 dataset read calls to the reorganized data. To demonstrate the effectiveness of SDS, we applied two parallel data organization techniques: a sort-based organization on a plasma physics data and a transpose-based organization on mass spectrometry imaging data. We also extended the HDF5 data access API to allow selection of data based on their values through a query interface, called SDS Query. We evaluated the execution time in accessing various subsets of data through existing HDF5 Read API and SDS Query. We showed that reading the reorganized data using SDS is up to 55X faster than reading the original data.

DATA ANALYSIS BY SQL-MAPREDUCE PLATFORM

Directory of Open Access Journals (Sweden)

A. A. A. Dergachev

2014-01-01

Full Text Available The paper deals with the problems related to the usage of relational database management system (RDBMS, mainly in the analysis of large data content, including data analysis based on web services in the Internet. A solution of these problems can be represented as a web-oriented distributed system of the data analysis with the processor of service requests as an executive kernel. The functions of such system are similar to the functions of relational DBMS, only with the usage of web services. The processor of service requests is responsible for planning of data analysis web services calls and their execution. The efficiency of such web-oriented system depends on the efficiency of web services calls plan and their program implementation where the basic element is the facilities of analyzed data storage – relational DBMS. The main attention is given to extension of functionality of relational DBMS for the analysis of large data content, in particular, the perspective estimation of web services data analysis implementation on the basis of SQL/MapReduce platform. With a view of obtaining this result, analytical task was chosen as an application-oriented part, typical for data analysis in various social networks and web portals, based on data analysis of users’ attendance. In the practical part of this research the algorithm for planning of web services calls was implemented for application-oriented task solution. SQL/MapReduce platform efficiency is confirmed by experimental results that show the opportunity of effective application for data analysis web services.

Learning Haskell data analysis

CERN Document Server

Church, James

2015-01-01

If you are a developer, analyst, or data scientist who wants to learn data analysis methods using Haskell and its libraries, then this book is for you. Prior experience with Haskell and a basic knowledge of data science will be beneficial.

Analysis of neural data

CERN Document Server

Kass, Robert E; Brown, Emery N

2014-01-01

Continual improvements in data collection and processing have had a huge impact on brain research, producing data sets that are often large and complicated. By emphasizing a few fundamental principles, and a handful of ubiquitous techniques, Analysis of Neural Data provides a unified treatment of analytical methods that have become essential for contemporary researchers. Throughout the book ideas are illustrated with more than 100 examples drawn from the literature, ranging from electrophysiology, to neuroimaging, to behavior. By demonstrating the commonality among various statistical approaches the authors provide the crucial tools for gaining knowledge from diverse types of data. Aimed at experimentalists with only high-school level mathematics, as well as computationally-oriented neuroscientists who have limited familiarity with statistics, Analysis of Neural Data serves as both a self-contained introduction and a reference work.

Computerized ECT data analysis system

International Nuclear Information System (INIS)

Miyake, Y.; Fukui, S.; Iwahashi, Y.; Matsumoto, M.; Koyama, K.

1988-01-01

For the analytical method of the eddy current testing (ECT) of steam generator tubes in nuclear power plants, the authors have developed the computerized ECT data analysis system using a large-scale computer with a high-resolution color graphic display. This system can store acquired ECT data up to 15 steam generators, and ECT data can be analyzed immediately on the monitor in dialogue communication with a computer. Analyzed results of ECT data are stored and registered in the data base. This system enables an analyst to perform sorting and collecting of data under various conditions and obtain the results automatically, and also to make a plan of tube repair works. This system has completed the test run, and has been used for data analysis at the annual inspection of domestic plants. This paper describes an outline, features and examples of the computerized eddy current data analysis system for steam generator tubes in PWR nuclear power plants

Excel data analysis for dummies

CERN Document Server

Nelson, Stephen L

2014-01-01

Harness the power of Excel to discover what your numbers are hiding Excel Data Analysis For Dummies, 2nd Edition is the ultimate guide to getting the most out of your data. Veteran Dummies author Stephen L. Nelson guides you through the basic and not-so-basic features of Excel to help you discover the gems hidden in your rough data. From input, to analysis, to visualization, the book walks you through the steps that lead to superior data analysis. Excel is the number-one spreadsheet application, with ever-expanding capabilities. If you're only using it to balance the books, you're missing out

On Survey Data Analysis in Corporate Finance

OpenAIRE

Serita, Toshio

2008-01-01

Recently, survey data analysis has emerged as a new method for testing hypotheses andfor clarifying the relative importance of different factors in corporate finance decisions. This paper investigates the advantages and drawbacks of survey data analysis, methodology of survey data analysis such as questionnaire design, and analytical methods for survey data, incomparison with traditional large sample analysis. We show that survey data analysis does not replace traditional large sample analysi...

Reporting Data with "Over-the-Counter" Data Analysis Supports Increases Educators' Analysis Accuracy

Science.gov (United States)

Rankin, Jenny Grant

2013-01-01

There is extensive research on the benefits of making data-informed decisions to improve learning, but these benefits rely on the data being effectively interpreted. Despite educators' above-average intellect and education levels, there is evidence many educators routinely misinterpret student data. Data analysis problems persist even at districts…

Collective Analysis of Qualitative Data

DEFF Research Database (Denmark)

Simonsen, Jesper; Friberg, Karin

2014-01-01

What. Many students and practitioners do not know how to systematically process qualitative data once it is gathered—at least not as a collective effort. This chapter presents two workshop techniques, affinity diagramming and diagnostic mapping, that support collective analysis of large amounts...... of qualitative data. Affinity diagramming is used to make collective analysis and interpretations of qualitative data to identify core problems that need to be addressed in the design process. Diagnostic mapping supports collective interpretation and description of these problems and how to intervene in them. We....... In particular, collective analysis can be used to identify, understand, and act on complex design problems that emerge, for example, after the introduction of new tech- nologies. Such problems might be hard to clarify, and the basis for the analysis often involves large amounts of unstructured qualitative data...

Statistical analysis of medical data using SAS

CERN Document Server

Der, Geoff

2005-01-01

An Introduction to SASDescribing and Summarizing DataBasic InferenceScatterplots Correlation: Simple Regression and SmoothingAnalysis of Variance and CovarianceMultiple RegressionLogistic RegressionThe Generalized Linear ModelGeneralized Additive ModelsNonlinear Regression ModelsThe Analysis of Longitudinal Data IThe Analysis of Longitudinal Data II: Models for Normal Response VariablesThe Analysis of Longitudinal Data III: Non-Normal ResponseSurvival AnalysisAnalysis Multivariate Date: Principal Components and Cluster AnalysisReferences

Mobile networks for biometric data analysis

CERN Document Server

Madrid, Natividad; Seepold, Ralf; Orcioni, Simone

2016-01-01

This book showcases new and innovative approaches to biometric data capture and analysis, focusing especially on those that are characterized by non-intrusiveness, reliable prediction algorithms, and high user acceptance. It comprises the peer-reviewed papers from the international workshop on the subject that was held in Ancona, Italy, in October 2014 and featured sessions on ICT for health care, biometric data in automotive and home applications, embedded systems for biometric data analysis, biometric data analysis: EMG and ECG, and ICT for gait analysis. The background to the book is the challenge posed by the prevention and treatment of common, widespread chronic diseases in modern, aging societies. Capture of biometric data is a cornerstone for any analysis and treatment strategy. The latest advances in sensor technology allow accurate data measurement in a non-intrusive way, and in many cases it is necessary to provide online monitoring and real-time data capturing to support a patient’s prevention pl...

Data Analysis in Experimental Biomedical Research

DEFF Research Database (Denmark)

Markovich, Dmitriy

This thesis covers two non-related topics in experimental biomedical research: data analysis in thrombin generation experiments (collaboration with Novo Nordisk A/S), and analysis of images and physiological signals in the context of neurovascular signalling and blood flow regulation in the brain...... to critically assess and compare obtained results. We reverse engineered the data analysis performed by CAT, a de facto standard assay in the field. This revealed a number of possibilities to improve its methods of data analysis. We found that experimental calibration data is described well with textbook...

Open Data and Data Analysis Preservation Services for LHC Experiments

CERN Document Server

Cowton, J; Fokianos, P; Rueda, L; Herterich, P; Kunčar, J; Šimko, T; Smith, T

2015-01-01

In this paper we present newly launched services for open data and for long-term preservation and reuse of high-energy-physics data analyses based on the digital library software Invenio. We track the ”data continuum” practices through several progressive data analysis phases up to the final publication. The aim is to capture for subsequent generations all digital assets and associated knowledge inherent in the data analysis process, and to make a subset available rapidly to the public. The ultimate goal of the analysis preservation platform is to capture enough information about the processing steps in order to facilitate reproduction of an analysis even many years after its initial publication, permitting to extend the impact of preserved analyses through future revalidation and recasting services. A related ”open data” service was launched for the benefit of the general public.

Functional and shape data analysis

CERN Document Server

Srivastava, Anuj

2016-01-01

This textbook for courses on function data analysis and shape data analysis describes how to define, compare, and mathematically represent shapes, with a focus on statistical modeling and inference. It is aimed at graduate students in analysis in statistics, engineering, applied mathematics, neuroscience, biology, bioinformatics, and other related areas. The interdisciplinary nature of the broad range of ideas covered—from introductory theory to algorithmic implementations and some statistical case studies—is meant to familiarize graduate students with an array of tools that are relevant in developing computational solutions for shape and related analyses. These tools, gleaned from geometry, algebra, statistics, and computational science, are traditionally scattered across different courses, departments, and disciplines; Functional and Shape Data Analysis offers a unified, comprehensive solution by integrating the registration problem into shape analysis, better preparing graduate students for handling fu...

The Ultra-scale Visualization Climate Data Analysis Tools (UV-CDAT): Data Analysis and Visualization for Geoscience Data

Energy Technology Data Exchange (ETDEWEB)

Williams, Dean [Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States); Doutriaux, Charles [Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States); Patchett, John [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Williams, Sean [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Shipman, Galen [Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Miller, Ross [Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Steed, Chad [Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Krishnan, Harinarayan [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Silva, Claudio [NYU Polytechnic School of Engineering, New York, NY (United States); Chaudhary, Aashish [Kitware, Inc., Clifton Park, NY (United States); Bremer, Peer-Timo [Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States); Pugmire, David [Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Bethel, E. Wes [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Childs, Hank [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Prabhat, Mr. [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Geveci, Berk [Kitware, Inc., Clifton Park, NY (United States); Bauer, Andrew [Kitware, Inc., Clifton Park, NY (United States); Pletzer, Alexander [Tech-X Corp., Boulder, CO (United States); Poco, Jorge [NYU Polytechnic School of Engineering, New York, NY (United States); Ellqvist, Tommy [NYU Polytechnic School of Engineering, New York, NY (United States); Santos, Emanuele [Federal Univ. of Ceara, Fortaleza (Brazil); Potter, Gerald [NASA Johnson Space Center, Houston, TX (United States); Smith, Brian [Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Maxwell, Thomas [NASA Johnson Space Center, Houston, TX (United States); Kindig, David [Tech-X Corp., Boulder, CO (United States); Koop, David [NYU Polytechnic School of Engineering, New York, NY (United States)

2013-05-01

To support interactive visualization and analysis of complex, large-scale climate data sets, UV-CDAT integrates a powerful set of scientific computing libraries and applications to foster more efficient knowledge discovery. Connected through a provenance framework, the UV-CDAT components can be loosely coupled for fast integration or tightly coupled for greater functionality and communication with other components. This framework addresses many challenges in the interactive visual analysis of distributed large-scale data for the climate community.

Exploring functional data analysis and wavelet principal component analysis on ecstasy (MDMA wastewater data

Directory of Open Access Journals (Sweden)

Stefania Salvatore

2016-07-01

Full Text Available Abstract Background Wastewater-based epidemiology (WBE is a novel approach in drug use epidemiology which aims to monitor the extent of use of various drugs in a community. In this study, we investigate functional principal component analysis (FPCA as a tool for analysing WBE data and compare it to traditional principal component analysis (PCA and to wavelet principal component analysis (WPCA which is more flexible temporally. Methods We analysed temporal wastewater data from 42 European cities collected daily over one week in March 2013. The main temporal features of ecstasy (MDMA were extracted using FPCA using both Fourier and B-spline basis functions with three different smoothing parameters, along with PCA and WPCA with different mother wavelets and shrinkage rules. The stability of FPCA was explored through bootstrapping and analysis of sensitivity to missing data. Results The first three principal components (PCs, functional principal components (FPCs and wavelet principal components (WPCs explained 87.5-99.6 % of the temporal variation between cities, depending on the choice of basis and smoothing. The extracted temporal features from PCA, FPCA and WPCA were consistent. FPCA using Fourier basis and common-optimal smoothing was the most stable and least sensitive to missing data. Conclusion FPCA is a flexible and analytically tractable method for analysing temporal changes in wastewater data, and is robust to missing data. WPCA did not reveal any rapid temporal changes in the data not captured by FPCA. Overall the results suggest FPCA with Fourier basis functions and common-optimal smoothing parameter as the most accurate approach when analysing WBE data.

Computer-assisted qualitative data analysis software.

Science.gov (United States)

Cope, Diane G

2014-05-01

Advances in technology have provided new approaches for data collection methods and analysis for researchers. Data collection is no longer limited to paper-and-pencil format, and numerous methods are now available through Internet and electronic resources. With these techniques, researchers are not burdened with entering data manually and data analysis is facilitated by software programs. Quantitative research is supported by the use of computer software and provides ease in the management of large data sets and rapid analysis of numeric statistical methods. New technologies are emerging to support qualitative research with the availability of computer-assisted qualitative data analysis software (CAQDAS).CAQDAS will be presented with a discussion of advantages, limitations, controversial issues, and recommendations for this type of software use.

AGR-1 Thermocouple Data Analysis

International Nuclear Information System (INIS)

Einerson, Jeff

2012-01-01

This report documents an effort to analyze measured and simulated data obtained in the Advanced Gas Reactor (AGR) fuel irradiation test program conducted in the INL's Advanced Test Reactor (ATR) to support the Next Generation Nuclear Plant (NGNP) R and D program. The work follows up on a previous study (Pham and Einerson, 2010), in which statistical analysis methods were applied for AGR-1 thermocouple data qualification. The present work exercises the idea that, while recognizing uncertainties inherent in physics and thermal simulations of the AGR-1 test, results of the numerical simulations can be used in combination with the statistical analysis methods to further improve qualification of measured data. Additionally, the combined analysis of measured and simulation data can generate insights about simulation model uncertainty that can be useful for model improvement. This report also describes an experimental control procedure to maintain fuel target temperature in the future AGR tests using regression relationships that include simulation results. The report is organized into four chapters. Chapter 1 introduces the AGR Fuel Development and Qualification program, AGR-1 test configuration and test procedure, overview of AGR-1 measured data, and overview of physics and thermal simulation, including modeling assumptions and uncertainties. A brief summary of statistical analysis methods developed in (Pham and Einerson 2010) for AGR-1 measured data qualification within NGNP Data Management and Analysis System (NDMAS) is also included for completeness. Chapters 2-3 describe and discuss cases, in which the combined use of experimental and simulation data is realized. A set of issues associated with measurement and modeling uncertainties resulted from the combined analysis are identified. This includes demonstration that such a combined analysis led to important insights for reducing uncertainty in presentation of AGR-1 measured data (Chapter 2) and interpretation of

AGR-1 Thermocouple Data Analysis

Energy Technology Data Exchange (ETDEWEB)

Jeff Einerson

2012-05-01

This report documents an effort to analyze measured and simulated data obtained in the Advanced Gas Reactor (AGR) fuel irradiation test program conducted in the INL's Advanced Test Reactor (ATR) to support the Next Generation Nuclear Plant (NGNP) R&D program. The work follows up on a previous study (Pham and Einerson, 2010), in which statistical analysis methods were applied for AGR-1 thermocouple data qualification. The present work exercises the idea that, while recognizing uncertainties inherent in physics and thermal simulations of the AGR-1 test, results of the numerical simulations can be used in combination with the statistical analysis methods to further improve qualification of measured data. Additionally, the combined analysis of measured and simulation data can generate insights about simulation model uncertainty that can be useful for model improvement. This report also describes an experimental control procedure to maintain fuel target temperature in the future AGR tests using regression relationships that include simulation results. The report is organized into four chapters. Chapter 1 introduces the AGR Fuel Development and Qualification program, AGR-1 test configuration and test procedure, overview of AGR-1 measured data, and overview of physics and thermal simulation, including modeling assumptions and uncertainties. A brief summary of statistical analysis methods developed in (Pham and Einerson 2010) for AGR-1 measured data qualification within NGNP Data Management and Analysis System (NDMAS) is also included for completeness. Chapters 2-3 describe and discuss cases, in which the combined use of experimental and simulation data is realized. A set of issues associated with measurement and modeling uncertainties resulted from the combined analysis are identified. This includes demonstration that such a combined analysis led to important insights for reducing uncertainty in presentation of AGR-1 measured data (Chapter 2) and interpretation of

Integrative Analysis of Omics Big Data.

Science.gov (United States)

Yu, Xiang-Tian; Zeng, Tao

2018-01-01

The diversity and huge omics data take biology and biomedicine research and application into a big data era, just like that popular in human society a decade ago. They are opening a new challenge from horizontal data ensemble (e.g., the similar types of data collected from different labs or companies) to vertical data ensemble (e.g., the different types of data collected for a group of person with match information), which requires the integrative analysis in biology and biomedicine and also asks for emergent development of data integration to address the great changes from previous population-guided to newly individual-guided investigations.Data integration is an effective concept to solve the complex problem or understand the complicate system. Several benchmark studies have revealed the heterogeneity and trade-off that existed in the analysis of omics data. Integrative analysis can combine and investigate many datasets in a cost-effective reproducible way. Current integration approaches on biological data have two modes: one is "bottom-up integration" mode with follow-up manual integration, and the other one is "top-down integration" mode with follow-up in silico integration.This paper will firstly summarize the combinatory analysis approaches to give candidate protocol on biological experiment design for effectively integrative study on genomics and then survey the data fusion approaches to give helpful instruction on computational model development for biological significance detection, which have also provided newly data resources and analysis tools to support the precision medicine dependent on the big biomedical data. Finally, the problems and future directions are highlighted for integrative analysis of omics big data.

Data management, archiving, visualization and analysis of space physics data

Science.gov (United States)

Russell, C. T.

1995-01-01

A series of programs for the visualization and analysis of space physics data has been developed at UCLA. In the course of those developments, a number of lessons have been learned regarding data management and data archiving, as well as data analysis. The issues now facing those wishing to develop such software, as well as the lessons learned, are reviewed. Modern media have eased many of the earlier problems of the physical volume required to store data, the speed of access, and the permanence of the records. However, the ultimate longevity of these media is still a question of debate. Finally, while software development has become easier, cost is still a limiting factor in developing visualization and analysis software.

Data Analysis and Data Mining: Current Issues in Biomedical Informatics

Science.gov (United States)

Bellazzi, Riccardo; Diomidous, Marianna; Sarkar, Indra Neil; Takabayashi, Katsuhiko; Ziegler, Andreas; McCray, Alexa T.

2011-01-01

Summary Background Medicine and biomedical sciences have become data-intensive fields, which, at the same time, enable the application of data-driven approaches and require sophisticated data analysis and data mining methods. Biomedical informatics provides a proper interdisciplinary context to integrate data and knowledge when processing available information, with the aim of giving effective decision-making support in clinics and translational research. Objectives To reflect on different perspectives related to the role of data analysis and data mining in biomedical informatics. Methods On the occasion of the 50th year of Methods of Information in Medicine a symposium was organized, that reflected on opportunities, challenges and priorities of organizing, representing and analysing data, information and knowledge in biomedicine and health care. The contributions of experts with a variety of backgrounds in the area of biomedical data analysis have been collected as one outcome of this symposium, in order to provide a broad, though coherent, overview of some of the most interesting aspects of the field. Results The paper presents sections on data accumulation and data-driven approaches in medical informatics, data and knowledge integration, statistical issues for the evaluation of data mining models, translational bioinformatics and bioinformatics aspects of genetic epidemiology. Conclusions Biomedical informatics represents a natural framework to properly and effectively apply data analysis and data mining methods in a decision-making context. In the future, it will be necessary to preserve the inclusive nature of the field and to foster an increasing sharing of data and methods between researchers. PMID:22146916

Exact analysis of discrete data

CERN Document Server

Hirji, Karim F

2005-01-01

Researchers in fields ranging from biology and medicine to the social sciences, law, and economics regularly encounter variables that are discrete or categorical in nature. While there is no dearth of books on the analysis and interpretation of such data, these generally focus on large sample methods. When sample sizes are not large or the data are otherwise sparse, exact methods--methods not based on asymptotic theory--are more accurate and therefore preferable.This book introduces the statistical theory, analysis methods, and computation techniques for exact analysis of discrete data. After reviewing the relevant discrete distributions, the author develops the exact methods from the ground up in a conceptually integrated manner. The topics covered range from univariate discrete data analysis, a single and several 2 x 2 tables, a single and several 2 x K tables, incidence density and inverse sampling designs, unmatched and matched case -control studies, paired binary and trinomial response models, and Markov...

2nd European Conference on Data Analysis

CERN Document Server

Wilhelm, Adalbert FX

2016-01-01

This book offers a snapshot of the state-of-the-art in classification at the interface between statistics, computer science and application fields. The contributions span a broad spectrum, from theoretical developments to practical applications; they all share a strong computational component. The topics addressed are from the following fields: Statistics and Data Analysis; Machine Learning and Knowledge Discovery; Data Analysis in Marketing; Data Analysis in Finance and Economics; Data Analysis in Medicine and the Life Sciences; Data Analysis in the Social, Behavioural, and Health Care Sciences; Data Analysis in Interdisciplinary Domains; Classification and Subject Indexing in Library and Information Science. The book presents selected papers from the Second European Conference on Data Analysis, held at Jacobs University Bremen in July 2014. This conference unites diverse researchers in the pursuit of a common topic, creating truly unique synergies in the process.

Grain-A Java data analysis system for Total Data Readout

International Nuclear Information System (INIS)

Rahkila, P.

2008-01-01

Grain is a data analysis system developed to be used with the novel Total Data Readout data acquisition system. In Total Data Readout all the electronics channels are read out asynchronously in singles mode and each data item is timestamped. Event building and analysis has to be done entirely in the software post-processing the data stream. A flexible and efficient event parser and the accompanying software system have been written entirely in Java. The design and implementation of the software are discussed along with experiences gained in running real-life experiments

Chapter 8. Data Analysis

Science.gov (United States)

Lyman L. McDonald; Christina D. Vojta; Kevin S. McKelvey

2013-01-01

Perhaps the greatest barrier between monitoring and management is data analysis. Data languish in drawers and spreadsheets because those who collect or maintain monitoring data lack training in how to effectively summarize and analyze their findings. This chapter serves as a first step to surmounting that barrier by empowering any monitoring team with the basic...

Project MOHAVE data analysis plan

International Nuclear Information System (INIS)

Watson, J.G.; Green, M.; Hoffer, T.E.; Lawson, D.R.; Pitchford, M.; Eatough, D.J.; Farber, R.J.; Malm, W.C.; McDade, C.E.

1993-01-01

Project MOHAVE is intended to develop ambient and source emissions data for use with source models, receptor models, and data analysis methods in order to explain the nature and causes of visibility degradation in the Grand Canyon. Approximately 50% of the modeling and data analysis effort will be directed toward understanding the contributions from the Mohave Power Project to haze in the Grand Canyon and other nearby Class areas; the remaining resources will be used to understand the contribution from other sources. The major goals of Project MOHAVE and data analysis are: to evaluate the measurement for applicability to modeling and data analysis activities; to describe the visibility, air quality and meteorology during the field study period and to determine the degree to which these measurements represent typical visibility events at the Grand Canyon; to further develop conceptual models of physical and chemical processes which affect visibility impairment at the Grand Canyon; to estimate the contributions from different emission sources to visibility impairment at the Grand Canyon, and to quantitatively evaluate the uncertainties of those estimates; to reconcile different scientific interpretations of the same data and to present this reconciliation to decision-makers. Several different approaches will be applied. Each approach will involve explicit examination of measurement uncertainties, compliance with implicit and explicit assumptions, and representativeness of the measurements. Scientific disagreements will be sought, expressed, explained, quantified, and presented. Data which can be used to verify methods will be withheld for independent evaluation of the validity of those methods. All assumptions will be stated and evaluated against reality. Data analysis results not supporting hypotheses will be presented with those results supporting the hypotheses. Uncertainty statements will be quantitative and consistent with decision-making needs

Tornado detection data reduction and analysis

Science.gov (United States)

Davisson, L. D.

1977-01-01

Data processing and analysis was provided in support of tornado detection by analysis of radio frequency interference in various frequency bands. Sea state determination data from short pulse radar measurements were also processed and analyzed. A backscatter simulation was implemented to predict radar performance as a function of wind velocity. Computer programs were developed for the various data processing and analysis goals of the effort.

Emotion Analysis on Social Big Data

Institute of Scientific and Technical Information of China (English)

REN Fuji; Kazuyuki Matsumoto

2017-01-01

In this paper, we describe a method of emotion analysis on social big data. Social big data means text data that is emerging on In-ternet social networking services.We collect multilingual web corpora and annotated emotion tags to these corpora for the purpose of emotion analysis. Because these data are constructed by manual annotation, their quality is high but their quantity is low. If we create an emotion analysis model based on this corpus with high quality and use the model for the analysis of social big data, we might be able to statistically analyze emotional sensesand behavior of the people in Internet communications, which we could not know before. In this paper, we create an emotion analysis model that integrate the high-quality emotion corpus and the automatic-constructed corpus that we created in our past studies, and then analyze a large-scale corpus consisting of Twitter tweets based on the model. As the result of time-series analysis on the large-scale corpus and the result of model evaluation, we show the effective-ness of our proposed method.

Selected topics on data analysis in astronomy

International Nuclear Information System (INIS)

Scarsi, L.

1987-01-01

The contents of this book are: General Lectures Given at the Erice II Workshop on Data Analysis in Astronomy: Fundamentals in Data Analysis in Astronomy; Computational Techniques; Evolution of Architectures for Data Processing; Hardware for Graphics and Image Display; and Data Analysis Systems

Roadside video data analysis deep learning

CERN Document Server

Verma, Brijesh; Stockwell, David

2017-01-01

This book highlights the methods and applications for roadside video data analysis, with a particular focus on the use of deep learning to solve roadside video data segmentation and classification problems. It describes system architectures and methodologies that are specifically built upon learning concepts for roadside video data processing, and offers a detailed analysis of the segmentation, feature extraction and classification processes. Lastly, it demonstrates the applications of roadside video data analysis including scene labelling, roadside vegetation classification and vegetation biomass estimation in fire risk assessment.

NVivo 8 and consistency in data analysis: reflecting on the use of a qualitative data analysis program.

Science.gov (United States)

Bergin, Michael

2011-01-01

Qualitative data analysis is a complex process and demands clear thinking on the part of the analyst. However, a number of deficiencies may obstruct the research analyst during the process, leading to inconsistencies occurring. This paper is a reflection on the use of a qualitative data analysis program, NVivo 8, and its usefulness in identifying consistency and inconsistency during the coding process. The author was conducting a large-scale study of providers and users of mental health services in Ireland. He used NVivo 8 to store, code and analyse the data and this paper reflects some of his observations during the study. The demands placed on the analyst in trying to balance the mechanics of working through a qualitative data analysis program, while simultaneously remaining conscious of the value of all sources are highlighted. NVivo 8 as a qualitative data analysis program is a challenging but valuable means for advancing the robustness of qualitative research. Pitfalls can be avoided during analysis by running queries as the analyst progresses from tree node to tree node rather than leaving it to a stage whereby data analysis is well advanced.

Analysis of irradiation disordering data

Energy Technology Data Exchange (ETDEWEB)

Schwartz, D L [Jet Propulsion Lab., Pasadena, CA (USA); Schwartz, D M

1978-08-01

The analysis of irradiation disordering data in ordered Ni/sub 3/Mn is discussed. An analytical expression relating observed irradiation induced magnetic changes in this material to the number of alternating site <110> replacements is derived. This expression is then employed to analyze previous experimental results. This analysis gives results which appear to be consistent with a previous Monte Carlo data analysis and indicates that the expected number of alternating site <110> replacements is 66.4 per 450 eV recoil.

A Disciplined Architectural Approach to Scaling Data Analysis for Massive, Scientific Data

Science.gov (United States)

Crichton, D. J.; Braverman, A. J.; Cinquini, L.; Turmon, M.; Lee, H.; Law, E.

2014-12-01

Data collections across remote sensing and ground-based instruments in astronomy, Earth science, and planetary science are outpacing scientists' ability to analyze them. Furthermore, the distribution, structure, and heterogeneity of the measurements themselves pose challenges that limit the scalability of data analysis using traditional approaches. Methods for developing science data processing pipelines, distribution of scientific datasets, and performing analysis will require innovative approaches that integrate cyber-infrastructure, algorithms, and data into more systematic approaches that can more efficiently compute and reduce data, particularly distributed data. This requires the integration of computer science, machine learning, statistics and domain expertise to identify scalable architectures for data analysis. The size of data returned from Earth Science observing satellites and the magnitude of data from climate model output, is predicted to grow into the tens of petabytes challenging current data analysis paradigms. This same kind of growth is present in astronomy and planetary science data. One of the major challenges in data science and related disciplines defining new approaches to scaling systems and analysis in order to increase scientific productivity and yield. Specific needs include: 1) identification of optimized system architectures for analyzing massive, distributed data sets; 2) algorithms for systematic analysis of massive data sets in distributed environments; and 3) the development of software infrastructures that are capable of performing massive, distributed data analysis across a comprehensive data science framework. NASA/JPL has begun an initiative in data science to address these challenges. Our goal is to evaluate how scientific productivity can be improved through optimized architectural topologies that identify how to deploy and manage the access, distribution, computation, and reduction of massive, distributed data, while

The ACIGA data analysis programme

International Nuclear Information System (INIS)

Scott, Susan M; Searle, Antony C; Cusack, Benedict J; McClelland, David E

2004-01-01

The data analysis programme of the Australian Consortium for Interferometric Gravitational Astronomy (ACIGA) was set up in 1998 by Scott to complement the then existing ACIGA programmes working on suspension systems, lasers and optics and detector configurations. The ACIGA data analysis programme continues to contribute significantly in the field; we present an overview of our activities

Scientific data analysis on data-parallel platforms.

Energy Technology Data Exchange (ETDEWEB)

Ulmer, Craig D.; Bayer, Gregory W.; Choe, Yung Ryn; Roe, Diana C.

2010-09-01

As scientific computing users migrate to petaflop platforms that promise to generate multi-terabyte datasets, there is a growing need in the community to be able to embed sophisticated analysis algorithms in the computing platforms' storage systems. Data Warehouse Appliances (DWAs) are attractive for this work, due to their ability to store and process massive datasets efficiently. While DWAs have been utilized effectively in data-mining and informatics applications, they remain largely unproven in scientific workloads. In this paper we present our experiences in adapting two mesh analysis algorithms to function on five different DWA architectures: two Netezza database appliances, an XtremeData dbX database, a LexisNexis DAS, and multiple Hadoop MapReduce clusters. The main contribution of this work is insight into the differences between these DWAs from a user's perspective. In addition, we present performance measurements for ten DWA systems to help understand the impact of different architectural trade-offs in these systems.

Analysis of successive data sets

NARCIS (Netherlands)

Spreeuwers, Lieuwe Jan; Breeuwer, Marcel; Haselhoff, Eltjo Hans

2008-01-01

The invention relates to the analysis of successive data sets. A local intensity variation is formed from such successive data sets, that is, from data values in successive data sets at corresponding positions in each of the data sets. A region of interest is localized in the individual data sets on

Analysis of successive data sets

NARCIS (Netherlands)

Spreeuwers, Lieuwe Jan; Breeuwer, Marcel; Haselhoff, Eltjo Hans

2002-01-01

The invention relates to the analysis of successive data sets. A local intensity variation is formed from such successive data sets, that is, from data values in successive data sets at corresponding positions in each of the data sets. A region of interest is localized in the individual data sets on

Analysis of Hydrologic Properties Data

International Nuclear Information System (INIS)

Liu, H.H.; Ahlers, C.F.

2001-01-01

The purpose of this Analysis/Model Report (AMR) is to describe the methods used to determine hydrologic properties based on the available field data from the unsaturated zone at Yucca Mountain, Nevada. This is in accordance with the AMR Development Plan (DP) for U0090 Analysis of Hydrologic Properties Data (CRWMS M and O 1999c). Fracture and matrix properties are developed by compiling and analyzing available survey data from the Exploratory Studies Facility (ESF), Cross Drift of Enhanced Characterization of Repository Block (ECRB), and/or boreholes; air injection testing data from surface boreholes and from boreholes in ESF; in-situ measurements of water potential; and data from laboratory testing of core samples

Clinical trial data analysis using R

National Research Council Canada - National Science Library

Chen, Ding-Geng; Peace, Karl E

2011-01-01

.... Case studies demonstrate how to select the appropriate clinical trial data. The authors introduce the corresponding biostatistical analysis methods, followed by the step-by-step data analysis using R...

40 CFR 92.131 - Smoke, data analysis.

Science.gov (United States)

2010-07-01

... 40 Protection of Environment 20 2010-07-01 2010-07-01 false Smoke, data analysis. 92.131 Section... analysis. The following procedure shall be used to analyze the smoke test data: (a) Locate each throttle... performed by direct analysis of the recorder traces, or by computer analysis of data collected by automatic...

40 CFR 86.884-13 - Data analysis.

Science.gov (United States)

2010-07-01

... 40 Protection of Environment 19 2010-07-01 2010-07-01 false Data analysis. 86.884-13 Section 86... New Diesel Heavy-Duty Engines; Smoke Exhaust Test Procedure § 86.884-13 Data analysis. The following... linearity check may be performed by direct analysis of the recorder traces, or by computer analysis of data...

European Conference on Data Analysis

CERN Document Server

Krolak-Schwerdt, Sabine; Böhmer, Matthias; Data Science, Learning by Latent Structures, and Knowledge Discovery; ECDA 2013

2015-01-01

This volume comprises papers dedicated to data science and the extraction of knowledge from many types of data: structural, quantitative, or statistical approaches for the analysis of data; advances in classification, clustering, and pattern recognition methods; strategies for modeling complex data and mining large data sets; applications of advanced methods in specific domains of practice. The contributions offer interesting applications to various disciplines such as psychology, biology, medical and health sciences; economics, marketing, banking, and finance; engineering; geography and geology;  archeology, sociology, educational sciences, linguistics, and musicology; library science. The book contains the selected and peer-reviewed papers presented during the European Conference on Data Analysis (ECDA 2013) which was jointly held by the German Classification Society (GfKl) and the French-speaking Classification Society (SFC) in July 2013 at the University of Luxembourg.

DataSHIELD : taking the analysis to the data, not the data to the analysis

NARCIS (Netherlands)

Gaye, Amadou; Marcon, Yannick; Isaeva, Julia; LaFlamme, Philippe; Turner, Andrew; Jones, Elinor M.; Minion, Joel; Boyd, Andrew W.; Newby, Christopher J.; Nuotio, Marja-Liisa; Wilson, Rebecca; Butters, Oliver; Murtagh, Barnaby; Demir, Ipek; Doiron, Dany; Giepmans, Lisette; Wallace, Susan E.; Budin-Ljosne, Isabelle; Schmidt, Carsten Oliver; Boffetta, Paolo; Boniol, Mathieu; Bota, Maria; Carter, Kim W.; deKlerk, Nick; Dibben, Chris; Francis, Richard W.; Hiekkalinna, Tero; Hveem, Kristian; Kvaloy, Kirsti; Millar, Sean; Perry, Ivan J.; Peters, Annette; Phillips, Catherine M.; Popham, Frank; Raab, Gillian; Reischl, Eva; Sheehan, Nuala; Waldenberger, Melanie; Perola, Markus; van den Heuvel, Edwin; Macleod, John; Knoppers, Bartha M.; Stolk, Ronald P.; Fortier, Isabel; Harris, Jennifer R.; Woffenbuttel, Bruce H. R.; Murtagh, Madeleine J.; Ferretti, Vincent; Burton, Paul R.

2014-01-01

Background: Research in modern biomedicine and social science requires sample sizes so large that they can often only be achieved through a pooled co-analysis of data from several studies. But the pooling of information from individuals in a central database that may be queried by researchers raises

Nuclear data needs for material analysis

International Nuclear Information System (INIS)

Molnar, Gabor L.

2001-01-01

Nuclear data for material analysis using neutron-based methods are examined. Besides a critical review of the available data, emphasis is given to emerging application areas and new experimental techniques. Neutron scattering and reaction data, as well as decay data for delayed and prompt gamma activation analysis are all discussed in detail. Conclusions are formed concerning the need of new measurement, calculation, evaluation and dissemination activities. (author)

Correspondence analysis of longitudinal data

NARCIS (Netherlands)

Van der Heijden, P.G.M.|info:eu-repo/dai/nl/073087998

2005-01-01

Correspondence analysis is an exploratory tool for the analysis of associations between categorical variables, the results of which may be displayed graphically. For longitudinal data with two time points, an analysis of the transition matrix (showing the relative frequencies for pairs of

Data analysis using a data base driven graphics animation system

International Nuclear Information System (INIS)

Schwieder, D.H.; Stewart, H.D.; Curtis, J.N.

1985-01-01

A graphics animation system has been developed at the Idaho National Engineering Laboratory (INEL) to assist engineers in the analysis of large amounts of time series data. Most prior attempts at computer animation of data involve the development of large and expensive problem-specific systems. This paper discusses a generalized interactive computer animation system designed to be used in a wide variety of data analysis applications. By using relational data base storage of graphics and control information, considerable flexibility in design and development of animated displays is achieved

Efficient Incremental Data Analysis

OpenAIRE

Nikolic, Milos

2016-01-01

Many data-intensive applications require real-time analytics over streaming data. In a growing number of domains -- sensor network monitoring, social web applications, clickstream analysis, high-frequency algorithmic trading, and fraud detections to name a few -- applications continuously monitor stream events to promptly react to certain data conditions. These applications demand responsive analytics even when faced with high volume and velocity of incoming changes, large numbers of users, a...

Online Interactive Data Analysis of Multi-Sensor Data Using Giovanni

Science.gov (United States)

Berrick, S.; Leptoukh, G.; Liu, Z.; Rui, H.; Shen, S.; Teng, W.; Zhu, T.

2005-12-01

The goal of the GES-DISC Interactive Online Visualization and Analysis System (Giovanni) is to provide earth science users a means for performing data analysis on data in the Goddard Earth Sciences (GES) Distributed Active Archive Center (DAAC) without having to download the data. Through Giovanni, users are able to apply statistical analysis on many individual gridded global data products across multiple instruments and even inter-compare parameters from more than one instrument. Giovanni currently allows users to select a time window and a region of interest to generate many graphical output types including area plots (time-averaged), time-series (area-averaged), Hovmoller (latitude vs. time, longitude vs. time), and animations for area plots. A number of graphical output types are also available for parameter inter-comparisons. ASCII output is also available for those who want to apply their own analysis software. Using the knowledge gained from Giovanni, a user can minimize the amount of data they need to download while maximizing the amount of relevant content in those data. The design challenges of Giovanni are (1) to successfully balance a simple, intuitive Web interface with the complexity and heterogeneity of our data, (2) to have a simple and flexible configuration so that new data sets and parameters can be added and organized for particular user communities, (3) to be agnostic with respect to the analysis software and graphing software and, (4) scalability. In a short time, the original Giovanni (Giovanni 1) has grown from two instances to eight (Giovanni 2), each tailored for a specific user community. The demand, however, for Giovanni and its capabilities continues to increase and in order to meet those demands, a redesign effort of Giovanni, which we call Giovanni 3, is being undertaken.

IAGA Geomagnetic Data Analysis format - Analysis_IAGA

Science.gov (United States)

-Emilian Toader, Victorin; Marmureanu, Alexandru

2013-04-01

Geomagnetic research involves a continuous Earth's magnetic field monitoring and software for processing large amounts of data. The Analysis_IAGA program reads and analyses files in IAGA2002 format used within the INTERMAGNET observer network. The data is made available by INTERMAGNET (http://www.intermagnet.org/Data_e.php) and NOAA - National Geophysical Data Center (ftp://ftp.ngdc.noaa.gov/wdc/geomagnetism/data/observatories/definitive) cost free for scientific use. The users of this software are those who study geomagnetism or use this data along with other atmospheric or seismic factors. Analysis_IAGA allows the visualization of files for the same station, with the feature of merging data for analyzing longer time intervals. Each file contains data collected within a 24 hour time interval with a sampling rate of 60 seconds or 1 second. Adding a large number of files may be done by dividing the sampling frequency. Also, the program has the feature of combining data files gathered from multiple stations as long as the sampling rate and time intervals are the same. Different channels may be selected, visualized and filtered individually. Channel properties can be saved and edited in a file. Data can be processed (spectral power, P / F, estimated frequency, Bz/Bx, Bz/By, convolutions and correlations on pairs of axis, discrete differentiation) and visualized along with the original signals on the same panel. With the help of cursors/magnifiers time differences can be calculated. Each channel can be analyzed separately. Signals can be filtered using bandpass, lowpass, highpass (Butterworth, Chebyshev, Inver Chebyshev, Eliptic, Bessel, Median, ZeroPath). Separate graphics visualize the spectral power, frequency spectrum histogram, the evolution of the estimated frequency, P/H, the spectral power. Adaptive JTFA spectrograms can be selected: CSD (Cone-Shaped Distribution), CWD (Choi-Williams Distribution), Gabor, STFT (short-time Fourier transform), WVD (Wigner

Topological data analysis for scientific visualization

CERN Document Server

Tierny, Julien

2017-01-01

Combining theoretical and practical aspects of topology, this book delivers a comprehensive and self-contained introduction to topological methods for the analysis and visualization of scientific data. Theoretical concepts are presented in a thorough but intuitive manner, with many high-quality color illustrations. Key algorithms for the computation and simplification of topological data representations are described in details, and their application is carefully illustrated in a chapter dedicated to concrete use cases. With its fine balance between theory and practice, "Topological Data Analysis for Scientific Visualization" constitutes an appealing introduction to the increasingly important topic of topological data analysis, for lecturers, students and researchers.

Columbia River Component Data Gap Analysis

Energy Technology Data Exchange (ETDEWEB)

L. C. Hulstrom

2007-10-23

This Data Gap Analysis report documents the results of a study conducted by Washington Closure Hanford (WCH) to compile and reivew the currently available surface water and sediment data for the Columbia River near and downstream of the Hanford Site. This Data Gap Analysis study was conducted to review the adequacy of the existing surface water and sediment data set from the Columbia River, with specific reference to the use of the data in future site characterization and screening level risk assessments.

Privacy protected text analysis in DataSHIELD

Directory of Open Access Journals (Sweden)

Rebecca Wilson

2017-04-01

Whilst it is possible to analyse free text within a DataSHIELD infrastructure, the challenge is creating generalised and resilient anti-disclosure methods for free text analysis. There are a range of biomedical and health sciences applications for DataSHIELD methods of privacy protected analysis of free text including analysis of electronic health records and analysis of qualitative data e.g. from social media.

Adaptive Analysis of Functional MRI Data

International Nuclear Information System (INIS)

Friman, Ola

2003-01-01

Functional Magnetic Resonance Imaging (fMRI) is a recently developed neuro-imaging technique with capacity to map neural activity with high spatial precision. To locate active brain areas, the method utilizes local blood oxygenation changes which are reflected as small intensity changes in a special type of MR images. The ability to non-invasively map brain functions provides new opportunities to unravel the mysteries and advance the understanding of the human brain, as well as to perform pre-surgical examinations in order to optimize surgical interventions. This dissertation introduces new approaches for the analysis of fMRI data. The detection of active brain areas is a challenging problem due to high noise levels and artifacts present in the data. A fundamental tool in the developed methods is Canonical Correlation Analysis (CCA). CCA is used in two novel ways. First as a method with the ability to fully exploit the spatio-temporal nature of fMRI data for detecting active brain areas. Established analysis approaches mainly focus on the temporal dimension of the data and they are for this reason commonly referred to as being mass-univariate. The new CCA detection method encompasses and generalizes the traditional mass-univariate methods and can in this terminology be viewed as a mass-multivariate approach. The concept of spatial basis functions is introduced as a spatial counterpart of the temporal basis functions already in use in fMRI analysis. The spatial basis functions implicitly perform an adaptive spatial filtering of the fMRI images, which significantly improves detection performance. It is also shown how prior information can be incorporated into the analysis by imposing constraints on the temporal and spatial models and a constrained version of CCA is devised to this end. A general Principal Component Analysis technique for generating and constraining temporal and spatial subspace models is proposed to be used in combination with the constrained CCA

A Multimodal Data Analysis Approach for Targeted Drug Discovery Involving Topological Data Analysis (TDA).

Science.gov (United States)

Alagappan, Muthuraman; Jiang, Dadi; Denko, Nicholas; Koong, Albert C

In silico drug discovery refers to a combination of computational techniques that augment our ability to discover drug compounds from compound libraries. Many such techniques exist, including virtual high-throughput screening (vHTS), high-throughput screening (HTS), and mechanisms for data storage and querying. However, presently these tools are often used independent of one another. In this chapter, we describe a new multimodal in silico technique for the hit identification and lead generation phases of traditional drug discovery. Our technique leverages the benefits of three independent methods-virtual high-throughput screening, high-throughput screening, and structural fingerprint analysis-by using a fourth technique called topological data analysis (TDA). We describe how a compound library can be independently tested with vHTS, HTS, and fingerprint analysis, and how the results can be transformed into a topological data analysis network to identify compounds from a diverse group of structural families. This process of using TDA or similar clustering methods to identify drug leads is advantageous because it provides a mechanism for choosing structurally diverse compounds while maintaining the unique advantages of already established techniques such as vHTS and HTS.

Modeling data irregularities and structural complexities in data envelopment analysis

CERN Document Server

Zhu, Joe

2007-01-01

In a relatively short period of time, Data Envelopment Analysis (DEA) has grown into a powerful quantitative, analytical tool for measuring and evaluating performance. It has been successfully applied to a whole variety of problems in many different contexts worldwide. This book deals with the micro aspects of handling and modeling data issues in modeling DEA problems. DEA's use has grown with its capability of dealing with complex "service industry" and the "public service domain" types of problems that require modeling of both qualitative and quantitative data. This handbook treatment deals with specific data problems including: imprecise or inaccurate data; missing data; qualitative data; outliers; undesirable outputs; quality data; statistical analysis; software and other data aspects of modeling complex DEA problems. In addition, the book will demonstrate how to visualize DEA results when the data is more than 3-dimensional, and how to identify efficiency units quickly and accurately.

Integrated analysis of genetic data with R

Directory of Open Access Journals (Sweden)

Zhao Jing

2006-01-01

Full Text Available Abstract Genetic data are now widely available. There is, however, an apparent lack of concerted effort to produce software systems for statistical analysis of genetic data compared with other fields of statistics. It is often a tremendous task for end-users to tailor them for particular data, especially when genetic data are analysed in conjunction with a large number of covariates. Here, R http://www.r-project.org, a free, flexible and platform-independent environment for statistical modelling and graphics is explored as an integrated system for genetic data analysis. An overview of some packages currently available for analysis of genetic data is given. This is followed by examples of package development and practical applications. With clear advantages in data management, graphics, statistical analysis, programming, internet capability and use of available codes, it is a feasible, although challenging, task to develop it into an integrated platform for genetic analysis; this will require the joint efforts of many researchers.

Advanced Excel for scientific data analysis

CERN Document Server

De Levie, Robert

2004-01-01

Excel is by far the most widely distributed data analysis software but few users are aware of its full powers. Advanced Excel For Scientific Data Analysis takes off from where most books dealing with scientific applications of Excel end. It focuses on three areas-least squares, Fourier transformation, and digital simulation-and illustrates these with extensive examples, often taken from the literature. It also includes and describes a number of sample macros and functions to facilitate common data analysis tasks. These macros and functions are provided in uncompiled, computer-readable, easily

Panel data analysis of cardiotocograph (CTG) data.

Science.gov (United States)

Horio, Hiroyuki; Kikuchi, Hitomi; Ikeda, Tomoaki

2013-01-01

Panel data analysis is a statistical method, widely used in econometrics, which deals with two-dimensional panel data collected over time and over individuals. Cardiotocograph (CTG) which monitors fetal heart rate (FHR) using Doppler ultrasound and uterine contraction by strain gage is commonly used in intrapartum treatment of pregnant women. Although the relationship between FHR waveform pattern and the outcome such as umbilical blood gas data at delivery has long been analyzed, there exists no accumulated FHR patterns from large number of cases. As time-series economic fluctuations in econometrics such as consumption trend has been studied using panel data which consists of time-series and cross-sectional data, we tried to apply this method to CTG data. The panel data composed of a symbolized segment of FHR pattern can be easily handled, and a perinatologist can get the whole FHR pattern view from the microscopic level of time-series FHR data.

Haskell data analysis cookbook

CERN Document Server

Shukla, Nishant

2014-01-01

Step-by-step recipes filled with practical code samples and engaging examples demonstrate Haskell in practice, and then the concepts behind the code. This book shows functional developers and analysts how to leverage their existing knowledge of Haskell specifically for high-quality data analysis. A good understanding of data sets and functional programming is assumed.

DataToText: A Consumer-Oriented Approach to Data Analysis

Science.gov (United States)

Kenny, David A.

2010-01-01

DataToText is a project developed where the user communicates the relevant information for an analysis and DataToText computer routine produces text output that describes in words, tables, and figures the results from the analyses. Two extended examples are given, one an example of a moderator analysis and the other an example of a dyadic data…

Secondary data analysis of large data sets in urology: successes and errors to avoid.

Science.gov (United States)

Schlomer, Bruce J; Copp, Hillary L

2014-03-01

Secondary data analysis is the use of data collected for research by someone other than the investigator. In the last several years there has been a dramatic increase in the number of these studies being published in urological journals and presented at urological meetings, especially involving secondary data analysis of large administrative data sets. Along with this expansion, skepticism for secondary data analysis studies has increased for many urologists. In this narrative review we discuss the types of large data sets that are commonly used for secondary data analysis in urology, and discuss the advantages and disadvantages of secondary data analysis. A literature search was performed to identify urological secondary data analysis studies published since 2008 using commonly used large data sets, and examples of high quality studies published in high impact journals are given. We outline an approach for performing a successful hypothesis or goal driven secondary data analysis study and highlight common errors to avoid. More than 350 secondary data analysis studies using large data sets have been published on urological topics since 2008 with likely many more studies presented at meetings but never published. Nonhypothesis or goal driven studies have likely constituted some of these studies and have probably contributed to the increased skepticism of this type of research. However, many high quality, hypothesis driven studies addressing research questions that would have been difficult to conduct with other methods have been performed in the last few years. Secondary data analysis is a powerful tool that can address questions which could not be adequately studied by another method. Knowledge of the limitations of secondary data analysis and of the data sets used is critical for a successful study. There are also important errors to avoid when planning and performing a secondary data analysis study. Investigators and the urological community need to strive to use

Advantages of Integrative Data Analysis for Developmental Research

Science.gov (United States)

Bainter, Sierra A.; Curran, Patrick J.

2015-01-01

Amid recent progress in cognitive development research, high-quality data resources are accumulating, and data sharing and secondary data analysis are becoming increasingly valuable tools. Integrative data analysis (IDA) is an exciting analytical framework that can enhance secondary data analysis in powerful ways. IDA pools item-level data across…

GaggleBridge: collaborative data analysis.

Science.gov (United States)

Battke, Florian; Symons, Stephan; Herbig, Alexander; Nieselt, Kay

2011-09-15

Tools aiding in collaborative data analysis are becoming ever more important as researchers work together over long distances. We present an extension to the Gaggle framework, which has been widely adopted as a tool to enable data exchange between different analysis programs on one computer. Our program, GaggleBridge, transparently extends this functionality to allow data exchange between Gaggle users at different geographic locations using network communication. GaggleBridge can automatically set up SSH tunnels to traverse firewalls while adding some security features to the Gaggle communication. GaggleBridge is available as open-source software implemented in the Java language at http://it.inf.uni-tuebingen.de/gb. florian.battke@uni-tuebingen.de Supplementary data are available at Bioinformatics online.

40 CFR 51.366 - Data analysis and reporting.

Science.gov (United States)

2010-07-01

... 40 Protection of Environment 2 2010-07-01 2010-07-01 false Data analysis and reporting. 51.366... Requirements § 51.366 Data analysis and reporting. Data analysis and reporting are required to allow for..., including the results of an analysis of the registration data base; (ii) The percentage of motorist...

Long-term Preservation of Data Analysis Capabilities

Science.gov (United States)

Gabriel, C.; Arviset, C.; Ibarra, A.; Pollock, A.

2015-09-01

While the long-term preservation of scientific data obtained by large astrophysics missions is ensured through science archives, the issue of data analysis software preservation has hardly been addressed. Efforts by large data centres have contributed so far to maintain some instrument or mission-specific data reduction packages on top of high-level general purpose data analysis software. However, it is always difficult to keep software alive without support and maintenance once the active phase of a mission is over. This is especially difficult in the budgetary model followed by space agencies. We discuss the importance of extending the lifetime of dedicated data analysis packages and review diverse strategies under development at ESA using new paradigms such as Virtual Machines, Cloud Computing, and Software as a Service for making possible full availability of data analysis and calibration software for decades at minimal cost.

An Automated Data Analysis Tool for Livestock Market Data

Science.gov (United States)

Williams, Galen S.; Raper, Kellie Curry

2011-01-01

This article describes an automated data analysis tool that allows Oklahoma Cooperative Extension Service educators to disseminate results in a timely manner. Primary data collected at Oklahoma Quality Beef Network (OQBN) certified calf auctions across the state results in a large amount of data per sale site. Sale summaries for an individual sale…

Common Data Format (CDF) and Coordinated Data Analysis Web (CDAWeb)

Science.gov (United States)

Candey, Robert M.

2010-01-01

The Coordinated Data Analysis Web (CDAWeb) data browsing system provides plotting, listing and open access v ia FTP, HTTP, and web services (REST, SOAP, OPeNDAP) for data from mo st NASA Heliophysics missions and is heavily used by the community. C ombining data from many instruments and missions enables broad resear ch analysis and correlation and coordination with other experiments a nd missions. Crucial to its effectiveness is the use of a standard se lf-describing data format, in this case, the Common Data Format (CDF) , also developed at the Space Physics Data facility , and the use of metadata standa rds (easily edited with SKTeditor ). CDAweb is based on a set of IDL routines, CDAWlib . . The CDF project also maintains soft ware and services for translating between many standard formats (CDF. netCDF, HDF, FITS, XML) .

Robust statistics and geochemical data analysis

International Nuclear Information System (INIS)

Di, Z.

1987-01-01

Advantages of robust procedures over ordinary least-squares procedures in geochemical data analysis is demonstrated using NURE data from the Hot Springs Quadrangle, South Dakota, USA. Robust principal components analysis with 5% multivariate trimming successfully guarded the analysis against perturbations by outliers and increased the number of interpretable factors. Regression with SINE estimates significantly increased the goodness-of-fit of the regression and improved the correspondence of delineated anomalies with known uranium prospects. Because of the ubiquitous existence of outliers in geochemical data, robust statistical procedures are suggested as routine procedures to replace ordinary least-squares procedures

TREX13 Data Analysis/Modeling

Science.gov (United States)

2018-03-29

From: Dajun Tang, Principal Investigator Subj: ONR Grant# N00014-14-1-0239 & N00014-16-1-2371, “TREX 13 Data analysis /Modeling” Encl: (1) Final...Performance/ Technical Report with SF298 The attached enclosures constitute the final technical report for ONR Grant# N00014-14-1-0239 & N00014-16-1-2371...TREX 13 Data analysis /Modeling” cc: Grant & Contract Administrator, APL-UW Office of Sponsor Programs, UW ONR Seattle – Robert Rice and

Data engineering systems: Computerized modeling and data bank capabilities for engineering analysis

Science.gov (United States)

Kopp, H.; Trettau, R.; Zolotar, B.

1984-01-01

The Data Engineering System (DES) is a computer-based system that organizes technical data and provides automated mechanisms for storage, retrieval, and engineering analysis. The DES combines the benefits of a structured data base system with automated links to large-scale analysis codes. While the DES provides the user with many of the capabilities of a computer-aided design (CAD) system, the systems are actually quite different in several respects. A typical CAD system emphasizes interactive graphics capabilities and organizes data in a manner that optimizes these graphics. On the other hand, the DES is a computer-aided engineering system intended for the engineer who must operationally understand an existing or planned design or who desires to carry out additional technical analysis based on a particular design. The DES emphasizes data retrieval in a form that not only provides the engineer access to search and display the data but also links the data automatically with the computer analysis codes.

Wavelets in functional data analysis

CERN Document Server

Morettin, Pedro A; Vidakovic, Brani

2017-01-01

Wavelet-based procedures are key in many areas of statistics, applied mathematics, engineering, and science. This book presents wavelets in functional data analysis, offering a glimpse of problems in which they can be applied, including tumor analysis, functional magnetic resonance and meteorological data. Starting with the Haar wavelet, the authors explore myriad families of wavelets and how they can be used. High-dimensional data visualization (using Andrews' plots), wavelet shrinkage (a simple, yet powerful, procedure for nonparametric models) and a selection of estimation and testing techniques (including a discussion on Stein’s Paradox) make this a highly valuable resource for graduate students and experienced researchers alike.

Enabling High-performance Interactive Geoscience Data Analysis Through Data Placement and Movement Optimization

Science.gov (United States)

Zhu, F.; Yu, H.; Rilee, M. L.; Kuo, K. S.; Yu, L.; Pan, Y.; Jiang, H.

2017-12-01

Since the establishment of data archive centers and the standardization of file formats, scientists are required to search metadata catalogs for data needed and download the data files to their local machines to carry out data analysis. This approach has facilitated data discovery and access for decades, but it inevitably leads to data transfer from data archive centers to scientists' computers through low-bandwidth Internet connections. Data transfer becomes a major performance bottleneck in such an approach. Combined with generally constrained local compute/storage resources, they limit the extent of scientists' studies and deprive them of timely outcomes. Thus, this conventional approach is not scalable with respect to both the volume and variety of geoscience data. A much more viable solution is to couple analysis and storage systems to minimize data transfer. In our study, we compare loosely coupled approaches (exemplified by Spark and Hadoop) and tightly coupled approaches (exemplified by parallel distributed database management systems, e.g., SciDB). In particular, we investigate the optimization of data placement and movement to effectively tackle the variety challenge, and boost the popularization of parallelization to address the volume challenge. Our goal is to enable high-performance interactive analysis for a good portion of geoscience data analysis exercise. We show that tightly coupled approaches can concentrate data traffic between local storage systems and compute units, and thereby optimizing bandwidth utilization to achieve a better throughput. Based on our observations, we develop a geoscience data analysis system that tightly couples analysis engines with storages, which has direct access to the detailed map of data partition locations. Through an innovation data partitioning and distribution scheme, our system has demonstrated scalable and interactive performance in real-world geoscience data analysis applications.

Data Analysis Facility (DAF)

Science.gov (United States)

1991-01-01

NASA-Dryden's Data Analysis Facility (DAF) provides a variety of support services to the entire Dryden community. It provides state-of-the-art hardware and software systems, available to any Dryden engineer for pre- and post-flight data processing and analysis, plus supporting all archival and general computer use. The Flight Data Access System (FDAS) is one of the advanced computer systems in the DAF, providing for fast engineering unit conversion and archival processing of flight data delivered from the Western Aeronautical Test Range. Engineering unit conversion and archival formatting of flight data is performed by the DRACO program on a Sun 690MP and an E-5000 computer. Time history files produced by DRACO are then moved to a permanent magneto-optical archive, where they are network-accessible 24 hours a day, 7 days a week. Pertinent information about the individual flights is maintained in a relational (Sybase) database. The DAF also houses all general computer services, including; the Compute Server 1 and 2 (CS1 and CS2), the server for the World Wide Web, overall computer operations support, courier service, a CD-ROM Writer system, a Technical Support Center, the NASA Dryden Phone System (NDPS), and Hardware Maintenance.

An integrative data analysis platform for gene set analysis and knowledge discovery in a data warehouse framework.

Science.gov (United States)

Chen, Yi-An; Tripathi, Lokesh P; Mizuguchi, Kenji

2016-01-01

Data analysis is one of the most critical and challenging steps in drug discovery and disease biology. A user-friendly resource to visualize and analyse high-throughput data provides a powerful medium for both experimental and computational biologists to understand vastly different biological data types and obtain a concise, simplified and meaningful output for better knowledge discovery. We have previously developed TargetMine, an integrated data warehouse optimized for target prioritization. Here we describe how upgraded and newly modelled data types in TargetMine can now survey the wider biological and chemical data space, relevant to drug discovery and development. To enhance the scope of TargetMine from target prioritization to broad-based knowledge discovery, we have also developed a new auxiliary toolkit to assist with data analysis and visualization in TargetMine. This toolkit features interactive data analysis tools to query and analyse the biological data compiled within the TargetMine data warehouse. The enhanced system enables users to discover new hypotheses interactively by performing complicated searches with no programming and obtaining the results in an easy to comprehend output format. Database URL: http://targetmine.mizuguchilab.org. © The Author(s) 2016. Published by Oxford University Press.

Classification, (big) data analysis and statistical learning

CERN Document Server

Conversano, Claudio; Vichi, Maurizio

2018-01-01

This edited book focuses on the latest developments in classification, statistical learning, data analysis and related areas of data science, including statistical analysis of large datasets, big data analytics, time series clustering, integration of data from different sources, as well as social networks. It covers both methodological aspects as well as applications to a wide range of areas such as economics, marketing, education, social sciences, medicine, environmental sciences and the pharmaceutical industry. In addition, it describes the basic features of the software behind the data analysis results, and provides links to the corresponding codes and data sets where necessary. This book is intended for researchers and practitioners who are interested in the latest developments and applications in the field. The peer-reviewed contributions were presented at the 10th Scientific Meeting of the Classification and Data Analysis Group (CLADAG) of the Italian Statistical Society, held in Santa Margherita di Pul...

Survival analysis using S analysis of time-to-event data

CERN Document Server

Tableman, Mara

2003-01-01

Survival Analysis Using S: Analysis of Time-to-Event Data is designed as a text for a one-semester or one-quarter course in survival analysis for upper-level or graduate students in statistics, biostatistics, and epidemiology. Prerequisites are a standard pre-calculus first course in probability and statistics, and a course in applied linear regression models. No prior knowledge of S or R is assumed. A wide choice of exercises is included, some intended for more advanced students with a first course in mathematical statistics. The authors emphasize parametric log-linear models, while also detailing nonparametric procedures along with model building and data diagnostics. Medical and public health researchers will find the discussion of cut point analysis with bootstrap validation, competing risks and the cumulative incidence estimator, and the analysis of left-truncated and right-censored data invaluable. The bootstrap procedure checks robustness of cut point analysis and determines cut point(s). In a chapter ...

NGNP Data Management and Analysis System Analysis and Web Delivery Capabilities

Energy Technology Data Exchange (ETDEWEB)

Cynthia D. Gentillon

2011-09-01

Projects for the Very High Temperature Reactor (VHTR) Technology Development Office provide data in support of Nuclear Regulatory Commission licensing of the very high temperature reactor. Fuel and materials to be used in the reactor are tested and characterized to quantify performance in high-temperature and high-fluence environments. The NGNP Data Management and Analysis System (NDMAS) at the Idaho National Laboratory has been established to ensure that VHTR data are (1) qualified for use, (2) stored in a readily accessible electronic form, and (3) analyzed to extract useful results. This document focuses on the third NDMAS objective. It describes capabilities for displaying the data in meaningful ways and for data analysis to identify useful relationships among the measured quantities. The capabilities are described from the perspective of NDMAS users, starting with those who just view experimental data and analytical results on the INL NDMAS web portal. Web display and delivery capabilities are described in detail. Also the current web pages that show Advanced Gas Reactor, Advanced Graphite Capsule, and High Temperature Materials test results are itemized. Capabilities available to NDMAS developers are more extensive, and are described using a second series of examples. Much of the data analysis efforts focus on understanding how thermocouple measurements relate to simulated temperatures and other experimental parameters. Statistical control charts and correlation monitoring provide an ongoing assessment of instrument accuracy. Data analysis capabilities are virtually unlimited for those who use the NDMAS web data download capabilities and the analysis software of their choice. Overall, the NDMAS provides convenient data analysis and web delivery capabilities for studying a very large and rapidly increasing database of well-documented, pedigreed data.

VESUVIO Data Analysis Goes MANTID

International Nuclear Information System (INIS)

Jackson, S; Krzystyniak, M; Seel, A G; Gigg, M; Richards, S E; Fernandez-Alonso, F

2014-01-01

This paper describes ongoing efforts to implement the reduction and analysis of neutron Compton scattering data within the MANTID framework. Recently, extensive work has been carried out to integrate the bespoke data reduction and analysis routines written for VESUVIO with the MANTID framework. While the programs described in this document are designed to replicate the functionality of the Fortran and Genie routines already in use, most of them have been written from scratch and are not based on the original code base

VESUVIO Data Analysis Goes MANTID

Science.gov (United States)

Jackson, S.; Krzystyniak, M.; Seel, A. G.; Gigg, M.; Richards, S. E.; Fernandez-Alonso, F.

2014-12-01

This paper describes ongoing efforts to implement the reduction and analysis of neutron Compton scattering data within the MANTID framework. Recently, extensive work has been carried out to integrate the bespoke data reduction and analysis routines written for VESUVIO with the MANTID framework. While the programs described in this document are designed to replicate the functionality of the Fortran and Genie routines already in use, most of them have been written from scratch and are not based on the original code base.

An Analysis of the Climate Data Initiative's Data Collection

Science.gov (United States)

Ramachandran, R.; Bugbee, K.

2015-12-01

The Climate Data Initiative (CDI) is a broad multi-agency effort of the U.S. government that seeks to leverage the extensive existing federal climate-relevant data to stimulate innovation and private-sector entrepreneurship to support national climate-change preparedness. The CDI project is a systematic effort to manually curate and share openly available climate data from various federal agencies. To date, the CDI has curated seven themes, or topics, relevant to climate change resiliency. These themes include Coastal Flooding, Food Resilience, Water, Ecosystem Vulnerability, Human Health, Energy Infrastructure, and Transportation. Each theme was curated by subject matter experts who selected datasets relevant to the topic at hand. An analysis of the entire Climate Data Initiative data collection and the data curated for each theme offers insights into which datasets are considered most relevant in addressing climate resiliency. Other aspects of the data collection will be examined including which datasets were the most visited or popular and which datasets were the most sought after for curation by the theme teams. Results from the analysis of the CDI collection will be presented in this talk.

Critical Data Analysis Precedes Soft Computing Of Medical Data

DEFF Research Database (Denmark)

Keyserlingk, Diedrich Graf von; Jantzen, Jan; Berks, G.

2000-01-01

extracted. The factors had different relationships (loadings) to the symptoms. Although the factors were gained only by computations, they seemed to express some modular features of the language disturbances. This phenomenon, that factors represent superior aspects of data, is well known in factor analysis...... the deficits in communication. Sets of symptoms corresponding to the traditional symptoms in Broca and Wernicke aphasia may be represented in the factors, but the factor itself does not represent a syndrome. It is assumed that this kind of data analysis shows a new approach to the understanding of language...

Substituting missing data in compositional analysis

Energy Technology Data Exchange (ETDEWEB)

Real, Carlos, E-mail: carlos.real@usc.es [Area de Ecologia, Departamento de Biologia Celular y Ecologia, Escuela Politecnica Superior, Universidad de Santiago de Compostela, 27002 Lugo (Spain); Angel Fernandez, J.; Aboal, Jesus R.; Carballeira, Alejo [Area de Ecologia, Departamento de Biologia Celular y Ecologia, Facultad de Biologia, Universidad de Santiago de Compostela, 15782 Santiago de Compostela (Spain)

2011-10-15

Multivariate analysis of environmental data sets requires the absence of missing values or their substitution by small values. However, if the data is transformed logarithmically prior to the analysis, this solution cannot be applied because the logarithm of a small value might become an outlier. Several methods for substituting the missing values can be found in the literature although none of them guarantees that no distortion of the structure of the data set is produced. We propose a method for the assessment of these distortions which can be used for deciding whether to retain or not the samples or variables containing missing values and for the investigation of the performance of different substitution techniques. The method analyzes the structure of the distances among samples using Mantel tests. We present an application of the method to PCDD/F data measured in samples of terrestrial moss as part of a biomonitoring study. - Highlights: > Missing values in multivariate data sets must be substituted prior to analysis. > The substituted values can modify the structure of the data set. > We developed a method to estimate the magnitude of the alterations. > The method is simple and based on the Mantel test. > The method allowed the identification of problematic variables in a sample data set. - A method is presented for the assessment of the possible distortions in multivariate analysis caused by the substitution of missing values.

A Proposed Data Fusion Architecture for Micro-Zone Analysis and Data Mining

Energy Technology Data Exchange (ETDEWEB)

Kevin McCarthy; Milos Manic

2012-08-01

Data Fusion requires the ability to combine or “fuse” date from multiple data sources. Time Series Analysis is a data mining technique used to predict future values from a data set based upon past values. Unlike other data mining techniques, however, Time Series places special emphasis on periodicity and how seasonal and other time-based factors tend to affect trends over time. One of the difficulties encountered in developing generic time series techniques is the wide variability of the data sets available for analysis. This presents challenges all the way from the data gathering stage to results presentation. This paper presents an architecture designed and used to facilitate the collection of disparate data sets well suited to Time Series analysis as well as other predictive data mining techniques. Results show this architecture provides a flexible, dynamic framework for the capture and storage of a myriad of dissimilar data sets and can serve as a foundation from which to build a complete data fusion architecture.

CMS Data Analysis School Model

CERN Document Server

Malik, Sudhir; Cavanaugh, R; Bloom, K; Chan, Kai-Feng; D'Hondt, J; Klima, B; Narain, M; Palla, F; Rolandi, G; Schörner-Sadenius, T

2014-01-01

To impart hands-on training in physics analysis, CMS experiment initiated the  concept of CMS Data Analysis School (CMSDAS). It was born three years ago at the LPC (LHC Physics Center), Fermilab and is based on earlier workshops held at the LPC and CLEO Experiment. As CMS transitioned from construction to the data taking mode, the nature of earlier training also evolved to include more of analysis tools, software tutorials and physics analysis. This effort epitomized as CMSDAS has proven to be a key for the new and young physicists to jump start and contribute to the physics goals of CMS by looking for new physics with the collision data. With over 400 physicists trained in six CMSDAS around the globe , CMS is trying to  engage the collaboration discovery potential and maximize the physics output. As a bigger goal, CMS is striving to nurture and increase engagement of the myriad talents of CMS, in the development of physics, service, upgrade, education of those new to CMS and the caree...

Post-Flight Data Analysis Tool

Science.gov (United States)

George, Marina

2018-01-01

A software tool that facilitates the retrieval and analysis of post-flight data. This allows our team and other teams to effectively and efficiently analyze and evaluate post-flight data in order to certify commercial providers.

Factor analysis of multivariate data

Digital Repository Service at National Institute of Oceanography (India)

Fernandes, A.A.; Mahadevan, R.

A brief introduction to factor analysis is presented. A FORTRAN program, which can perform the Q-mode and R-mode factor analysis and the singular value decomposition of a given data matrix is presented in Appendix B. This computer program, uses...

NeoAnalysis: a Python-based toolbox for quick electrophysiological data processing and analysis.

Science.gov (United States)

Zhang, Bo; Dai, Ji; Zhang, Tao

2017-11-13

In a typical electrophysiological experiment, especially one that includes studying animal behavior, the data collected normally contain spikes, local field potentials, behavioral responses and other associated data. In order to obtain informative results, the data must be analyzed simultaneously with the experimental settings. However, most open-source toolboxes currently available for data analysis were developed to handle only a portion of the data and did not take into account the sorting of experimental conditions. Additionally, these toolboxes require that the input data be in a specific format, which can be inconvenient to users. Therefore, the development of a highly integrated toolbox that can process multiple types of data regardless of input data format and perform basic analysis for general electrophysiological experiments is incredibly useful. Here, we report the development of a Python based open-source toolbox, referred to as NeoAnalysis, to be used for quick electrophysiological data processing and analysis. The toolbox can import data from different data acquisition systems regardless of their formats and automatically combine different types of data into a single file with a standardized format. In cases where additional spike sorting is needed, NeoAnalysis provides a module to perform efficient offline sorting with a user-friendly interface. Then, NeoAnalysis can perform regular analog signal processing, spike train, and local field potentials analysis, behavioral response (e.g. saccade) detection and extraction, with several options available for data plotting and statistics. Particularly, it can automatically generate sorted results without requiring users to manually sort data beforehand. In addition, NeoAnalysis can organize all of the relevant data into an informative table on a trial-by-trial basis for data visualization. Finally, NeoAnalysis supports analysis at the population level. With the multitude of general-purpose functions provided

A practical guide to scientific data analysis

CERN Document Server

Livingstone, David J

2009-01-01

Inspired by the author's need for practical guidance in the processes of data analysis, A Practical Guide to Scientific Data Analysis has been written as a statistical companion for the working scientist.  This handbook of data analysis with worked examples focuses on the application of mathematical and statistical techniques and the interpretation of their results. Covering the most common statistical methods for examining and exploring relationships in data, the text includes extensive examples from a variety of scientific disciplines. The chapters are organised logically, from pl

Data analysis for the LISA Technology Package

Energy Technology Data Exchange (ETDEWEB)

Hewitson, M; Danzmann, K; Diepholz, I; GarcIa, A [Albert-Einstein-Institut, Max-Planck-Institut fuer Gravitationsphysik und Universitaet Hannover, 30167 Hannover (Germany); Armano, M; Fauste, J [European Space Agency, ESAC, Villanueva de la Canada, 28692 Madrid (Spain); Benedetti, M [Dipartimento di Ingegneria dei Materiali e Tecnologie Industriali, Universita di Trento and INFN, Gruppo Collegato di Trento, Mesiano, Trento (Italy); Bogenstahl, J [Department of Physics and Astronomy, University of Glasgow, Glasgow (United Kingdom); Bortoluzzi, D; Bosetti, P; Cristofolini, I [Dipartimento di Ingegneria Meccanica e Strutturale, Universita di Trento and INFN, Gruppo Collegato di Trento, Mesiano, Trento (Italy); Brandt, N [Astrium GmbH, 88039 Friedrichshafen (Germany); Cavalleri, A; Ciani, G; Dolesi, R; Ferraioli, L [Dipartimento di Fisica, Universita di Trento and INFN, Gruppo Collegato di Trento, 38050 Povo, Trento (Italy); Cruise, M [Department of Physics and Astronomy, University of Birmingham, Birmingham (United Kingdom); Fertin, D; GarcIa, C [European Space Agency, ESTEC, 2200 AG Noordwijk (Netherlands); Fichter, W, E-mail: martin.hewitson@aei.mpg.d [Institut fuer Flugmechanik und Flugregelung, 70569 Stuttgart (Germany)

2009-05-07

The LISA Technology Package (LTP) on board the LISA Pathfinder mission aims to demonstrate some key concepts for LISA which cannot be tested on ground. The mission consists of a series of preplanned experimental runs. The data analysis for each experiment must be designed in advance of the mission. During the mission, the analysis must be carried out promptly so that the results can be fed forward into subsequent experiments. As such a robust and flexible data analysis environment needs to be put in place. Since this software is used during mission operations and effects the mission timeline, it must be very robust and tested to a high degree. This paper presents the requirements, design and implementation of the data analysis environment (LTPDA) that will be used for analysing the data from LTP. The use of the analysis software to perform mock data challenges (MDC) is also discussed, and some highlights from the first MDC are presented.

Data analysis for the LISA Technology Package

International Nuclear Information System (INIS)

Hewitson, M; Danzmann, K; Diepholz, I; GarcIa, A; Armano, M; Fauste, J; Benedetti, M; Bogenstahl, J; Bortoluzzi, D; Bosetti, P; Cristofolini, I; Brandt, N; Cavalleri, A; Ciani, G; Dolesi, R; Ferraioli, L; Cruise, M; Fertin, D; GarcIa, C; Fichter, W

2009-01-01

The LISA Technology Package (LTP) on board the LISA Pathfinder mission aims to demonstrate some key concepts for LISA which cannot be tested on ground. The mission consists of a series of preplanned experimental runs. The data analysis for each experiment must be designed in advance of the mission. During the mission, the analysis must be carried out promptly so that the results can be fed forward into subsequent experiments. As such a robust and flexible data analysis environment needs to be put in place. Since this software is used during mission operations and effects the mission timeline, it must be very robust and tested to a high degree. This paper presents the requirements, design and implementation of the data analysis environment (LTPDA) that will be used for analysing the data from LTP. The use of the analysis software to perform mock data challenges (MDC) is also discussed, and some highlights from the first MDC are presented.

Quantitative Data Analysis--In the Graduate Curriculum

Science.gov (United States)

Albers, Michael J.

2017-01-01

A quantitative research study collects numerical data that must be analyzed to help draw the study's conclusions. Teaching quantitative data analysis is not teaching number crunching, but teaching a way of critical thinking for how to analyze the data. The goal of data analysis is to reveal the underlying patterns, trends, and relationships of a…

Collecting operational event data for statistical analysis

International Nuclear Information System (INIS)

Atwood, C.L.

1994-09-01

This report gives guidance for collecting operational data to be used for statistical analysis, especially analysis of event counts. It discusses how to define the purpose of the study, the unit (system, component, etc.) to be studied, events to be counted, and demand or exposure time. Examples are given of classification systems for events in the data sources. A checklist summarizes the essential steps in data collection for statistical analysis

EXPLORATORY DATA ANALYSIS AND MULTIVARIATE STRATEGIES FOR REVEALING MULTIVARIATE STRUCTURES IN CLIMATE DATA

Directory of Open Access Journals (Sweden)

2016-12-01

Full Text Available This paper is on data analysis strategy in a complex, multidimensional, and dynamic domain. The focus is on the use of data mining techniques to explore the importance of multivariate structures; using climate variables which influences climate change. Techniques involved in data mining exercise vary according to the data structures. The multivariate analysis strategy considered here involved choosing an appropriate tool to analyze a process. Factor analysis is introduced into data mining technique in order to reveal the influencing impacts of factors involved as well as solving for multicolinearity effect among the variables. The temporal nature and multidimensionality of the target variables is revealed in the model using multidimensional regression estimates. The strategy of integrating the method of several statistical techniques, using climate variables in Nigeria was employed. R2 of 0.518 was obtained from the ordinary least square regression analysis carried out and the test was not significant at 5% level of significance. However, factor analysis regression strategy gave a good fit with R2 of 0.811 and the test was significant at 5% level of significance. Based on this study, model building should go beyond the usual confirmatory data analysis (CDA, rather it should be complemented with exploratory data analysis (EDA in order to achieve a desired result.

Power analysis of trials with multilevel data

CERN Document Server

Moerbeek, Mirjam

2015-01-01

Power Analysis of Trials with Multilevel Data covers using power and sample size calculations to design trials that involve nested data structures. The book gives a thorough overview of power analysis that details terminology and notation, outlines key concepts of statistical power and power analysis, and explains why they are necessary in trial design. It guides you in performing power calculations with hierarchical data, which enables more effective trial design.The authors are leading experts in the field who recognize that power analysis has attracted attention from applied statisticians i

Multiple-user data acquisition and analysis system

International Nuclear Information System (INIS)

Manzella, V.; Chrien, R.E.; Gill, R.L.; Liou, H.I.; Stelts, M.L.

1981-01-01

The nuclear physics program at the Brookhaven National Laboratory High Flux Beam Reactor (HFBR) employs a pair of PDP-11 computers for the dual functions of data acquisition and analysis. The data acquisition is accomplished through CAMAC and features a microprogrammed branch driver to accommodate various experimental inputs. The acquisition computer performs the functions of multi-channel analyzers, multiscaling and time-sequenced multichannel analyzers and gamma-ray coincidence analyzers. The data analysis computer is available for rapid processing of data tapes written by the acquisition computer. The ability to accommodate many users is facilitated by separating the data acquisition and analysis functions, and allowing each user to tailor the analysis to the specific requirements of his own experiment. The system is to be upgraded soon by the introduction of a dual port disk to allow a data base to be available to each computer

Anticipated Changes in Conducting Scientific Data-Analysis Research in the Big-Data Era

Science.gov (United States)

Kuo, Kwo-Sen; Seablom, Michael; Clune, Thomas; Ramachandran, Rahul

2014-05-01

A Big-Data environment is one that is capable of orchestrating quick-turnaround analyses involving large volumes of data for numerous simultaneous users. Based on our experiences with a prototype Big-Data analysis environment, we anticipate some important changes in research behaviors and processes while conducting scientific data-analysis research in the near future as such Big-Data environments become the mainstream. The first anticipated change will be the reduced effort and difficulty in most parts of the data management process. A Big-Data analysis environment is likely to house most of the data required for a particular research discipline along with appropriate analysis capabilities. This will reduce the need for researchers to download local copies of data. In turn, this also reduces the need for compute and storage procurement by individual researchers or groups, as well as associated maintenance and management afterwards. It is almost certain that Big-Data environments will require a different "programming language" to fully exploit the latent potential. In addition, the process of extending the environment to provide new analysis capabilities will likely be more involved than, say, compiling a piece of new or revised code. We thus anticipate that researchers will require support from dedicated organizations associated with the environment that are composed of professional software engineers and data scientists. A major benefit will likely be that such extensions are of higher-quality and broader applicability than ad hoc changes by physical scientists. Another anticipated significant change is improved collaboration among the researchers using the same environment. Since the environment is homogeneous within itself, many barriers to collaboration are minimized or eliminated. For example, data and analysis algorithms can be seamlessly shared, reused and re-purposed. In conclusion, we will be able to achieve a new level of scientific productivity in the

Anticipated Changes in Conducting Scientific Data-Analysis Research in the Big-Data Era

Science.gov (United States)

Kuo, Kwo-Sen; Seablom, Michael; Clune, Thomas; Ramachandran, Rahul

2014-01-01

A Big-Data environment is one that is capable of orchestrating quick-turnaround analyses involving large volumes of data for numerous simultaneous users. Based on our experiences with a prototype Big-Data analysis environment, we anticipate some important changes in research behaviors and processes while conducting scientific data-analysis research in the near future as such Big-Data environments become the mainstream. The first anticipated change will be the reduced effort and difficulty in most parts of the data management process. A Big-Data analysis environment is likely to house most of the data required for a particular research discipline along with appropriate analysis capabilities. This will reduce the need for researchers to download local copies of data. In turn, this also reduces the need for compute and storage procurement by individual researchers or groups, as well as associated maintenance and management afterwards. It is almost certain that Big-Data environments will require a different "programming language" to fully exploit the latent potential. In addition, the process of extending the environment to provide new analysis capabilities will likely be more involved than, say, compiling a piece of new or revised code.We thus anticipate that researchers will require support from dedicated organizations associated with the environment that are composed of professional software engineers and data scientists. A major benefit will likely be that such extensions are of higherquality and broader applicability than ad hoc changes by physical scientists. Another anticipated significant change is improved collaboration among the researchers using the same environment. Since the environment is homogeneous within itself, many barriers to collaboration are minimized or eliminated. For example, data and analysis algorithms can be seamlessly shared, reused and re-purposed. In conclusion, we will be able to achieve a new level of scientific productivity in the Big-Data

Data analysis with Mplus

CERN Document Server

Geiser, Christian

2012-01-01

A practical introduction to using Mplus for the analysis of multivariate data, this volume provides step-by-step guidance, complete with real data examples, numerous screen shots, and output excerpts. The author shows how to prepare a data set for import in Mplus using SPSS. He explains how to specify different types of models in Mplus syntax and address typical caveats--for example, assessing measurement invariance in longitudinal SEMs. Coverage includes path and factor analytic models as well as mediational, longitudinal, multilevel, and latent class models. Specific programming tips an

Data Analysis Strategies in Medical Imaging.

Science.gov (United States)

Parmar, Chintan; Barry, Joseph D; Hosny, Ahmed; Quackenbush, John; Aerts, Hugo Jwl

2018-03-26

Radiographic imaging continues to be one of the most effective and clinically useful tools within oncology. Sophistication of artificial intelligence (AI) has allowed for detailed quantification of radiographic characteristics of tissues using predefined engineered algorithms or deep learning methods. Precedents in radiology as well as a wealth of research studies hint at the clinical relevance of these characteristics. However, there are critical challenges associated with the analysis of medical imaging data. While some of these challenges are specific to the imaging field, many others like reproducibility and batch effects are generic and have already been addressed in other quantitative fields such as genomics. Here, we identify these pitfalls and provide recommendations for analysis strategies of medical imaging data including data normalization, development of robust models, and rigorous statistical analyses. Adhering to these recommendations will not only improve analysis quality, but will also enhance precision medicine by allowing better integration of imaging data with other biomedical data sources. Copyright ©2018, American Association for Cancer Research.

Intelligent data-acquisition instrumentation for special nuclear material assay data analysis

International Nuclear Information System (INIS)

Ethridge, C.D.

1980-01-01

The Detection, Surveillance, Verification, and Recovery Group of the Los Alamos Scientific Laboratory Energy Division/Nuclear Safeguards Programs is now utilizing intelligent data-acquisition instrumentation for assay data analysis of special nuclear material. The data acquisition and analysis are enabled by the incorporation of a number-crunching microprocessor sequenced by a single component microcomputer. Microcomputer firmware establishes the capability for processing the computation of several selected functions and also the ability of instrumentation self-diagnostics

Substituting missing data in compositional analysis

International Nuclear Information System (INIS)

Real, Carlos; Angel Fernandez, J.; Aboal, Jesus R.; Carballeira, Alejo

2011-01-01

Multivariate analysis of environmental data sets requires the absence of missing values or their substitution by small values. However, if the data is transformed logarithmically prior to the analysis, this solution cannot be applied because the logarithm of a small value might become an outlier. Several methods for substituting the missing values can be found in the literature although none of them guarantees that no distortion of the structure of the data set is produced. We propose a method for the assessment of these distortions which can be used for deciding whether to retain or not the samples or variables containing missing values and for the investigation of the performance of different substitution techniques. The method analyzes the structure of the distances among samples using Mantel tests. We present an application of the method to PCDD/F data measured in samples of terrestrial moss as part of a biomonitoring study. - Highlights: → Missing values in multivariate data sets must be substituted prior to analysis. → The substituted values can modify the structure of the data set. → We developed a method to estimate the magnitude of the alterations. → The method is simple and based on the Mantel test. → The method allowed the identification of problematic variables in a sample data set. - A method is presented for the assessment of the possible distortions in multivariate analysis caused by the substitution of missing values.

Advances in Risk Analysis with Big Data.

Science.gov (United States)

Choi, Tsan-Ming; Lambert, James H

2017-08-01

With cloud computing, Internet-of-things, wireless sensors, social media, fast storage and retrieval, etc., organizations and enterprises have access to unprecedented amounts and varieties of data. Current risk analysis methodology and applications are experiencing related advances and breakthroughs. For example, highway operations data are readily available, and making use of them reduces risks of traffic crashes and travel delays. Massive data of financial and enterprise systems support decision making under risk by individuals, industries, regulators, etc. In this introductory article, we first discuss the meaning of big data for risk analysis. We then examine recent advances in risk analysis with big data in several topic areas. For each area, we identify and introduce the relevant articles that are featured in the special issue. We conclude with a discussion on future research opportunities. © 2017 Society for Risk Analysis.

Computer system for environmental sample analysis and data storage and analysis

International Nuclear Information System (INIS)

Brauer, F.P.; Fager, J.E.

1976-01-01

A mini-computer based environmental sample analysis and data storage system has been developed. The system is used for analytical data acquisition, computation, storage of analytical results, and tabulation of selected or derived results for data analysis, interpretation and reporting. This paper discussed the structure, performance and applications of the system

QUAGOL: a guide for qualitative data analysis.

Science.gov (United States)

Dierckx de Casterlé, Bernadette; Gastmans, Chris; Bryon, Els; Denier, Yvonne

2012-03-01

Data analysis is a complex and contested part of the qualitative research process, which has received limited theoretical attention. Researchers are often in need of useful instructions or guidelines on how to analyze the mass of qualitative data, but face the lack of clear guidance for using particular analytic methods. The aim of this paper is to propose and discuss the Qualitative Analysis Guide of Leuven (QUAGOL), a guide that was developed in order to be able to truly capture the rich insights of qualitative interview data. The article describes six major problems researchers are often struggling with during the process of qualitative data analysis. Consequently, the QUAGOL is proposed as a guide to facilitate the process of analysis. Challenges emerged and lessons learned from own extensive experiences with qualitative data analysis within the Grounded Theory Approach, as well as from those of other researchers (as described in the literature), were discussed and recommendations were presented. Strengths and pitfalls of the proposed method were discussed in detail. The Qualitative Analysis Guide of Leuven (QUAGOL) offers a comprehensive method to guide the process of qualitative data analysis. The process consists of two parts, each consisting of five stages. The method is systematic but not rigid. It is characterized by iterative processes of digging deeper, constantly moving between the various stages of the process. As such, it aims to stimulate the researcher's intuition and creativity as optimal as possible. The QUAGOL guide is a theory and practice-based guide that supports and facilitates the process of analysis of qualitative interview data. Although the method can facilitate the process of analysis, it cannot guarantee automatic quality. The skills of the researcher and the quality of the research team remain the most crucial components of a successful process of analysis. Additionally, the importance of constantly moving between the various stages

Geographical data structures supporting regional analysis

International Nuclear Information System (INIS)

Edwards, R.G.; Durfee, R.C.

1978-01-01

In recent years the computer has become a valuable aid in solving regional environmental problems. Over a hundred different geographic information systems have been developed to digitize, store, analyze, and display spatially distributed data. One important aspect of these systems is the data structure (e.g. grids, polygons, segments) used to model the environment being studied. This paper presents eight common geographic data structures and their use in studies of coal resources, power plant siting, population distributions, LANDSAT imagery analysis, and landuse analysis

msBiodat analysis tool, big data analysis for high-throughput experiments.

Science.gov (United States)

Muñoz-Torres, Pau M; Rokć, Filip; Belužic, Robert; Grbeša, Ivana; Vugrek, Oliver

2016-01-01

Mass spectrometry (MS) are a group of a high-throughput techniques used to increase knowledge about biomolecules. They produce a large amount of data which is presented as a list of hundreds or thousands of proteins. Filtering those data efficiently is the first step for extracting biologically relevant information. The filtering may increase interest by merging previous data with the data obtained from public databases, resulting in an accurate list of proteins which meet the predetermined conditions. In this article we present msBiodat Analysis Tool, a web-based application thought to approach proteomics to the big data analysis. With this tool, researchers can easily select the most relevant information from their MS experiments using an easy-to-use web interface. An interesting feature of msBiodat analysis tool is the possibility of selecting proteins by its annotation on Gene Ontology using its Gene Id, ensembl or UniProt codes. The msBiodat analysis tool is a web-based application that allows researchers with any programming experience to deal with efficient database querying advantages. Its versatility and user-friendly interface makes easy to perform fast and accurate data screening by using complex queries. Once the analysis is finished, the result is delivered by e-mail. msBiodat analysis tool is freely available at http://msbiodata.irb.hr.

The Data Party: Involving Stakeholders in Meaningful Data Analysis

Science.gov (United States)

Franz, Nancy K.

2013-01-01

A hallmark of Extension includes the involvement of stakeholders in research and program needs assessment, design, implementation, evaluation, and reporting. A data party can be used to enhance this stakeholder involvement specifically in data analysis. This type of event can not only increase client participation in Extension programming and…

Qualitative Data Analysis Strategies

OpenAIRE

Greaves, Kristoffer

2014-01-01

A set of concept maps for qualitative data analysis strategies, inspired by Corbin, JM & Strauss, AL 2008, Basics of qualitative research: Techniques and procedures for developing grounded theory, 3rd edn, Sage Publications, Inc, Thousand Oaks, California.

A method for data base management and analysis for wind tunnel data

Science.gov (United States)

Biser, Aileen O.

1987-01-01

To respond to the need for improved data base management and analysis capabilities for wind-tunnel data at the Langley 16-Foot Transonic Tunnel, research was conducted into current methods of managing wind-tunnel data and a method was developed as a solution to this need. This paper describes the development of the data base management and analysis method for wind-tunnel data. The design and implementation of the software system are discussed and examples of its use are shown.

Methods for Mediation Analysis with Missing Data

Science.gov (United States)

Zhang, Zhiyong; Wang, Lijuan

2013-01-01

Despite wide applications of both mediation models and missing data techniques, formal discussion of mediation analysis with missing data is still rare. We introduce and compare four approaches to dealing with missing data in mediation analysis including list wise deletion, pairwise deletion, multiple imputation (MI), and a two-stage maximum…

Graph-Based Analysis of Nuclear Smuggling Data

International Nuclear Information System (INIS)

Cook, Diane; Holder, Larry; Thompson, Sandra E.; Whitney, Paul D.; Chilton, Lawrence

2009-01-01

Much of the data that is collected and analyzed today is structural, consisting not only of entities but also of relationships between the entities. As a result, analysis applications rely upon automated structural data mining approaches to find patterns and concepts of interest. This ability to analyze structural data has become a particular challenge in many security-related domains. In these domains, focusing on the relationships between entities in the data is critical to detect important underlying patterns. In this study we apply structural data mining techniques to automate analysis of nuclear smuggling data. In particular, we choose to model the data as a graph and use graph-based relational learning to identify patterns and concepts of interest in the data. In this paper, we identify the analysis questions that are of importance to security analysts and describe the knowledge representation and data mining approach that we adopt for this challenge. We analyze the results using the Russian nuclear smuggling event database.

A Statistical Toolkit for Data Analysis

International Nuclear Information System (INIS)

Donadio, S.; Guatelli, S.; Mascialino, B.; Pfeiffer, A.; Pia, M.G.; Ribon, A.; Viarengo, P.

2006-01-01

The present project aims to develop an open-source and object-oriented software Toolkit for statistical data analysis. Its statistical testing component contains a variety of Goodness-of-Fit tests, from Chi-squared to Kolmogorov-Smirnov, to less known, but generally much more powerful tests such as Anderson-Darling, Goodman, Fisz-Cramer-von Mises, Kuiper, Tiku. Thanks to the component-based design and the usage of the standard abstract interfaces for data analysis, this tool can be used by other data analysis systems or integrated in experimental software frameworks. This Toolkit has been released and is downloadable from the web. In this paper we describe the statistical details of the algorithms, the computational features of the Toolkit and describe the code validation

Thematic mapper data analysis

Science.gov (United States)

Settle, M.; Chavez, P.; Kieffer, H. H.; Everett, J. R.; Kahle, A. B.; Kitcho, C. A.; Milton, N. M.; Mouat, D. A.

1983-01-01

The geological applications of remote sensing technology are discussed, with emphasis given to the analysis of data from the Thematic Mapper (TM) instrument onboard the Landsat 4 satellite. The flight history and design characteristics of the Landsat 4/TM are reviewed, and some difficulties endountered in the interpretation of raw TM data are discussed, including: the volume of data; residual noise; detector-to-detector striping; and spatial misregistration between measurements. Preliminary results of several geological, lithological, geobotanical mapping experiments are presented as examples of the geological applications of the TM, and some areas for improving the guality of TM imagery are identified.

XML-based analysis interface for particle physics data analysis

International Nuclear Information System (INIS)

Hu Jifeng; Lu Xiaorui; Zhang Yangheng

2011-01-01

The letter emphasizes on an XML-based interface and its framework for particle physics data analysis. The interface uses a concise XML syntax to describe, in data analysis, the basic tasks: event-selection, kinematic fitting, particle identification, etc. and a basic processing logic: the next step goes on if and only if this step succeeds. The framework can perform an analysis without compiling by loading the XML-interface file, setting p in run-time and running dynamically. An analysis coding in XML instead of C++, easy-to-understood arid use, effectively reduces the work load, and enables users to carry out their analyses quickly. The framework has been developed on the BESⅢ offline software system (BOSS) with the object-oriented C++ programming. These functions, required by the regular tasks and the basic processing logic, are implemented with both standard modules or inherited from the modules in BOSS. The interface and its framework have been tested to perform physics analysis. (authors)

Control, data acquisition, data analysis and remote participation in LHD

International Nuclear Information System (INIS)

Nagayama, Y.; Emoto, M.; Nakanishi, H.; Sudo, S.; Imazu, S.; Inagaki, S.; Iwata, C.; Kojima, M.; Nonomura, M.; Ohsuna, M.; Tsuda, K.; Yoshida, M.; Chikaraishi, H.; Funaba, H.; Horiuchi, R.; Ishiguro, S.; Ito, Y.; Kubo, S.; Mase, A.; Mito, T.

2008-01-01

This paper presents the control, data acquisition, data analysis and remote participation facilities of the Large Helical Device (LHD), which is designed to confine the plasma in steady state. In LHD the plasma duration exceeds 3000 s by controlling the plasma position, the density and the ICRF heating. The 'LABCOM' data acquisition system takes both the short-pulse and the steady-state data. A two-layer Mass Storage System with RAIDs and Blu-ray Disk jukeboxes in a storage area network has been developed to increase capacity of storage. The steady-state data can be monitored with a Web browser in real time. A high-level data analysis system with Web interfaces is being developed in order to provide easier usage of LHD data and large FORTRAN codes in a supercomputer. A virtual laboratory system for the Japanese fusion community has been developed with Multi-protocol Label Switching Virtual Private Network Technology. Collaborators at remote sites can join the LHD experiment or use the NIFS supercomputer system as if they were working in the LHD control room

Structural Dynamics and Data Analysis

Science.gov (United States)

Luthman, Briana L.

2013-01-01

This project consists of two parts, the first will be the post-flight analysis of data from a Delta IV launch vehicle, and the second will be a Finite Element Analysis of a CubeSat. Shock and vibration data was collected on WGS-5 (Wideband Global SATCOM- 5) which was launched on a Delta IV launch vehicle. Using CAM (CAlculation with Matrices) software, the data is to be plotted into Time History, Shock Response Spectrum, and SPL (Sound Pressure Level) curves. In this format the data is to be reviewed and compared to flight instrumentation data from previous flights of the same launch vehicle. This is done to ensure the current mission environments, such as shock, random vibration, and acoustics, are not out of family with existing flight experience. In family means the peaks on the SRS curve for WGS-5 are similar to the peaks from the previous flights and there are no major outliers. The curves from the data will then be compiled into a useful format so that is can be peer reviewed then presented before an engineering review board if required. Also, the reviewed data will be uploaded to the Engineering Review Board Information System (ERBIS) to archive. The second part of this project is conducting Finite Element Analysis of a CubeSat. In 2010, Merritt Island High School partnered with NASA to design, build and launch a CubeSat. The team is now called StangSat in honor of their mascot, the mustang. Over the past few years, the StangSat team has built a satellite and has now been manifested for flight on a SpaceX Falcon 9 launch in 2014. To prepare for the final launch, a test flight was conducted in Mojave, California. StangSat was launched on a Prospector 18D, a high altitude rocket made by Garvey Spacecraft Corporation, along with their sister satellite CP9 built by California Polytechnic University. However, StangSat was damaged during an off nominal landing and this project will give beneficial insights into what loads the CubeSat experienced during the crash

Gravity Probe B data analysis: II. Science data and their handling prior to the final analysis

International Nuclear Information System (INIS)

Silbergleit, A S; Conklin, J W; Heifetz, M I; Holmes, T; Li, J; Mandel, I; Solomonik, V G; Stahl, K; P W Worden Jr; Everitt, C W F; Adams, M; Berberian, J E; Bencze, W; Clarke, B; Al-Jadaan, A; Keiser, G M; Kozaczuk, J A; Al-Meshari, M; Muhlfelder, B; Salomon, M

2015-01-01

The results of the Gravity Probe B relativity science mission published in Everitt et al (2011 Phys. Rev. Lett. 106 221101) required a rather sophisticated analysis of experimental data due to several unexpected complications discovered on-orbit. We give a detailed description of the Gravity Probe B data reduction. In the first paper (Silbergleit et al Class. Quantum Grav. 22 224018) we derived the measurement models, i.e., mathematical expressions for all the signals to analyze. In the third paper (Conklin et al Class. Quantum Grav. 22 224020) we explain the estimation algorithms and their program implementation, and discuss the experiment results obtained through data reduction. This paper deals with the science data preparation for the main analysis yielding the relativistic drift estimates. (paper)

Statistical data analysis using SAS intermediate statistical methods

CERN Document Server

Marasinghe, Mervyn G

2018-01-01

The aim of this textbook (previously titled SAS for Data Analytics) is to teach the use of SAS for statistical analysis of data for advanced undergraduate and graduate students in statistics, data science, and disciplines involving analyzing data. The book begins with an introduction beyond the basics of SAS, illustrated with non-trivial, real-world, worked examples. It proceeds to SAS programming and applications, SAS graphics, statistical analysis of regression models, analysis of variance models, analysis of variance with random and mixed effects models, and then takes the discussion beyond regression and analysis of variance to conclude. Pedagogically, the authors introduce theory and methodological basis topic by topic, present a problem as an application, followed by a SAS analysis of the data provided and a discussion of results. The text focuses on applied statistical problems and methods. Key features include: end of chapter exercises, downloadable SAS code and data sets, and advanced material suitab...

Enhancing yeast transcription analysis through integration of heterogeneous data

DEFF Research Database (Denmark)

Grotkjær, Thomas; Nielsen, Jens

2004-01-01

of Saccharomyces cerevisiae whole genome transcription data. A special focus is on the quantitative aspects of normalisation and mathematical modelling approaches, since they are expected to play an increasing role in future DNA microarray analysis studies. Data analysis is exemplified with cluster analysis......DNA microarray technology enables the simultaneous measurement of the transcript level of thousands of genes. Primary analysis can be done with basic statistical tools and cluster analysis, but effective and in depth analysis of the vast amount of transcription data requires integration with data...... from several heterogeneous data Sources, such as upstream promoter sequences, genome-scale metabolic models, annotation databases and other experimental data. In this review, we discuss how experimental design, normalisation, heterogeneous data and mathematical modelling can enhance analysis...

Statistical analysis of environmental data

International Nuclear Information System (INIS)

Beauchamp, J.J.; Bowman, K.O.; Miller, F.L. Jr.

1975-10-01

This report summarizes the analyses of data obtained by the Radiological Hygiene Branch of the Tennessee Valley Authority from samples taken around the Browns Ferry Nuclear Plant located in Northern Alabama. The data collection was begun in 1968 and a wide variety of types of samples have been gathered on a regular basis. The statistical analysis of environmental data involving very low-levels of radioactivity is discussed. Applications of computer calculations for data processing are described

Statistical data analysis handbook

National Research Council Canada - National Science Library

Wall, Francis J

1986-01-01

It must be emphasized that this is not a text book on statistics. Instead it is a working tool that presents data analysis in clear, concise terms which can be readily understood even by those without formal training in statistics...

FERRET data analysis code

International Nuclear Information System (INIS)

Schmittroth, F.

1979-09-01

A documentation of the FERRET data analysis code is given. The code provides a way to combine related measurements and calculations in a consistent evaluation. Basically a very general least-squares code, it is oriented towards problems frequently encountered in nuclear data and reactor physics. A strong emphasis is on the proper treatment of uncertainties and correlations and in providing quantitative uncertainty estimates. Documentation includes a review of the method, structure of the code, input formats, and examples

Teaching Data Analysis with Interactive Visual Narratives

Science.gov (United States)

Saundage, Dilal; Cybulski, Jacob L.; Keller, Susan; Dharmasena, Lasitha

2016-01-01

Data analysis is a major part of business analytics (BA), which refers to the skills, methods, and technologies that enable managers to make swift, quality decisions based on large amounts of data. BA has become a major component of Information Systems (IS) courses all over the world. The challenge for IS educators is to teach data analysis--the…

Data archiving and analysis for CWDD

International Nuclear Information System (INIS)

Coleman, T.A.; Novick, A.H.; Meystrik, C.C.; Marselle, J.R.

1992-01-01

A computer system has been developed to handle archiving and analysis of data acquired during operations of the Continuous Wave Deuterium Demonstrator (CWDD). Data files generated by the CWDD Instrumentation and Control system are transferred across a local area network to the CWDD Archive system where they are enlisted into the archive and stored on removeable media optical disk drives. A relational database management system maintains an on-line database catalog of all archived files. This database contains information about file contents and formats, and holds signal parameter configuration tables needed to extract and interpret data from the files. Software has been developed to assist the selection and retrieval of data on demand based upon references in the catalog. Data retrieved from the archive is transferred to commercial data visualization applications for viewing, plotting and analysis

Beyond Constant Comparison Qualitative Data Analysis: Using NVivo

Science.gov (United States)

Leech, Nancy L.; Onwuegbuzie, Anthony J.

2011-01-01

The purposes of this paper are to outline seven types of qualitative data analysis techniques, to present step-by-step guidance for conducting these analyses via a computer-assisted qualitative data analysis software program (i.e., NVivo9), and to present screenshots of the data analysis process. Specifically, the following seven analyses are…

Spectral map-analysis: a method to analyze gene expression data

OpenAIRE

Bijnens, Luc J.M.; Lewi, Paul J.; Göhlmann, Hinrich W.; Molenberghs, Geert; Wouters, Luc

2004-01-01

bioinformatics; biplot; correspondence factor analysis; data mining; data visualization; gene expression data; microarray data; multivariate exploratory data analysis; principal component analysis; Spectral map analysis

A Hierarchical Visualization Analysis Model of Power Big Data

Science.gov (United States)

Li, Yongjie; Wang, Zheng; Hao, Yang

2018-01-01

Based on the conception of integrating VR scene and power big data analysis, a hierarchical visualization analysis model of power big data is proposed, in which levels are designed, targeting at different abstract modules like transaction, engine, computation, control and store. The regularly departed modules of power data storing, data mining and analysis, data visualization are integrated into one platform by this model. It provides a visual analysis solution for the power big data.

Pengembangan Aplikasi Antarmuka Layanan Big Data Analysis

Directory of Open Access Journals (Sweden)

Gede Karya

2017-11-01

Full Text Available In the 2016 Higher Competitive Grants Research (Hibah Bersaing Dikti, we have been successfully developed models, infrastructure and modules of Hadoop-based big data analysis application. It has also successfully developed a virtual private network (VPN network that allows integration and access to the infrastructure from outside the FTIS Computer Laboratorium. Infrastructure and application modules of analysis are then wanted to be presented as services to small and medium enterprises (SMEs in Indonesia. This research aims to develop application of big data analysis service interface integrated with Hadoop-Cluster. The research begins with finding appropriate methods and techniques for scheduling jobs, calling for ready-made Java Map-Reduce (MR application modules, and techniques for tunneling input / output and meta-data construction of service request (input and service output. The above methods and techniques are then developed into a web-based service application, as well as an executable module that runs on Java and J2EE based programming environment and can access Hadoop-Cluster in the FTIS Computer Lab. The resulting application can be accessed by the public through the site http://bigdata.unpar.ac.id. Based on the test results, the application has functioned well in accordance with the specifications and can be used to perform big data analysis. Keywords: web based service, big data analysis, Hadop, J2EE Abstrak Pada penelitian Hibah Bersaing Dikti tahun 2016 telah berhasil dikembangkan model, infrastruktur dan modul-modul aplikasi big data analysis berbasis Hadoop. Selain itu juga telah berhasil dikembangkan jaringan virtual private network (VPN yang memungkinkan integrasi dan akses infrastruktur tersebut dari luar Laboratorium Komputer FTIS. Infrastruktur dan modul aplikasi analisis tersebut selanjutnya ingin dipresentasikan sebagai layanan kepada usaha kecil dan menengah (UKM di Indonesia. Penelitian ini bertujuan untuk mengembangkan

A Web Services Data Analysis Grid

Energy Technology Data Exchange (ETDEWEB)

William A Watson III; Ian Bird; Jie Chen; Bryan Hess; Andy Kowalski; Ying Chen

2002-07-01

The trend in large-scale scientific data analysis is to exploit compute, storage and other resources located at multiple sites, and to make those resources accessible to the scientist as if they were a single, coherent system. Web technologies driven by the huge and rapidly growing electronic commerce industry provide valuable components to speed the deployment of such sophisticated systems. Jefferson Lab, where several hundred terabytes of experimental data are acquired each year, is in the process of developing a web-based distributed system for data analysis and management. The essential aspects of this system are a distributed data grid (site independent access to experiment, simulation and model data) and a distributed batch system, augmented with various supervisory and management capabilities, and integrated using Java and XML-based web services.

A Web Services Data Analysis Grid

International Nuclear Information System (INIS)

William A Watson III; Ian Bird; Jie Chen; Bryan Hess; Andy Kowalski; Ying Chen

2002-01-01

The trend in large-scale scientific data analysis is to exploit compute, storage and other resources located at multiple sites, and to make those resources accessible to the scientist as if they were a single, coherent system. Web technologies driven by the huge and rapidly growing electronic commerce industry provide valuable components to speed the deployment of such sophisticated systems. Jefferson Lab, where several hundred terabytes of experimental data are acquired each year, is in the process of developing a web-based distributed system for data analysis and management. The essential aspects of this system are a distributed data grid (site independent access to experiment, simulation and model data) and a distributed batch system, augmented with various supervisory and management capabilities, and integrated using Java and XML-based web services

Geotechnical field data and analysis report

International Nuclear Information System (INIS)

1990-09-01

The geotechnical Field Data and Analysis Report documents the geomechanical data collected at the Waste Isolation Pilot Plant up to June 30, 1989 and describes the conditions of underground openings from July 1, 1988 to June 30, 1989. The data is required to understand performance during operations and does not include data from tests performed to support performance assessment. In summary, the underground openings have performed in a satisfactory manner during the reporting period. This analysis is based primarily on the evaluation of instrumentation data, in particular the comparison of measured convergence with predictions, and the observations of exposed rock surfaces. The main concerns during this period have been the deterioration found in Site Preliminary Design Validation Test Rooms 1 and 2 and some spalling found in Panel 1. 14 refs., 45 figs., 11 tabs

Analysis of longitudinal data from animals where some data are missing in SPSS

Science.gov (United States)

Duricki, DA; Soleman, S; Moon, LDF

2017-01-01

Testing of therapies for disease or injury often involves analysis of longitudinal data from animals. Modern analytical methods have advantages over conventional methods (particularly where some data are missing) yet are not used widely by pre-clinical researchers. We provide here an easy to use protocol for analysing longitudinal data from animals and present a click-by-click guide for performing suitable analyses using the statistical package SPSS. We guide readers through analysis of a real-life data set obtained when testing a therapy for brain injury (stroke) in elderly rats. We show that repeated measures analysis of covariance failed to detect a treatment effect when a few data points were missing (due to animal drop-out) whereas analysis using an alternative method detected a beneficial effect of treatment; specifically, we demonstrate the superiority of linear models (with various covariance structures) analysed using Restricted Maximum Likelihood estimation (to include all available data). This protocol takes two hours to follow. PMID:27196723

SIMONE: Tool for Data Analysis and Simulation

International Nuclear Information System (INIS)

Chudoba, V.; Hnatio, B.; Sharov, P.; Papka, Paul

2013-06-01

SIMONE is a software tool based on the ROOT Data Analysis Framework and developed in collaboration of FLNR JINR and iThemba LABS. It is intended for physicists planning experiments and analysing experimental data. The goal of the SIMONE framework is to provide a flexible system, user friendly, efficient and well documented. It is intended for simulation of a wide range of Nuclear Physics experiments. The most significant conditions and physical processes can be taken into account during simulation of the experiment. The user can create his own experimental setup through the access of predefined detector geometries. Simulated data is made available in the same format as for the real experiment for identical analysis of both experimental and simulated data. Significant time reduction is expected during experiment planning and data analysis. (authors)

KDD for science data analysis: Issues and examples

International Nuclear Information System (INIS)

Fayyad, U.; Haussler, D.; Stolorz, P.

1996-01-01

The analysis of the massive data sets collected by scientific instruments demands automation as a prerequisite to analysis. There is an urgent need to create an intermediate level at which scientists can operate effectively; isolating them from the massive sizes and harnessing human analysis capabilities to focus on tasks in which machines do not even remotely approach humans-namely, creative data analysis, theory and hypothesis formation, and drawing insights into underlying phenomena. We give an overview of the main issues in the exploitation of scientific datasets, present five case studies where KDD tools play important and enabling roles, and conclude with future challenges for data mining and KDD techniques in science data analysis

Lipidomic data analysis: Tutorial, practical guidelines and applications

Energy Technology Data Exchange (ETDEWEB)

Checa, Antonio [Department of Medical Biochemistry and Biophysics, Karolinska Institutet, Scheeles väg 2, SE-171 77 Stockholm (Sweden); Bedia, Carmen [Department of Environmental Chemistry, IDAEA-CSIC, Jordi Girona 18–26, Barcelona 08034 (Spain); Jaumot, Joaquim, E-mail: joaquim.jaumot@idaea.csic.es [Department of Environmental Chemistry, IDAEA-CSIC, Jordi Girona 18–26, Barcelona 08034 (Spain)

2015-07-23

Highlights: • An overview of chemometric methods applied to lipidomic data analysis is presented. • A lipidomic data set is analyzed showing the strengths of the introduced methods. • Practical guidelines for lipidomic data analysis are discussed. • Examples of applications of lipidomic data analysis in different fields are provided. - Abstract: Lipids are a broad group of biomolecules involved in diverse critical biological roles such as cellular membrane structure, energy storage or cell signaling and homeostasis. Lipidomics is the -omics science that pursues the comprehensive characterization of lipids present in a biological sample. Different analytical strategies such as nuclear magnetic resonance or mass spectrometry with or without previous chromatographic separation are currently used to analyze the lipid composition of a sample. However, current analytical techniques provide a vast amount of data which complicates the interpretation of results without the use of advanced data analysis tools. The choice of the appropriate chemometric method is essential to extract valuable information from the crude data as well as to interpret the lipidomic results in the biological context studied. The present work summarizes the diverse methods of analysis than can be used to study lipidomic data, from statistical inference tests to more sophisticated multivariate analysis methods. In addition to the theoretical description of the methods, application of various methods to a particular lipidomic data set as well as literature examples are presented.

Contracting Data Analysis: Assessment of Government-Wide Trends

Science.gov (United States)

2017-03-01

CONTRACTING DATA ANALYSIS Assessment of Government -wide Trends Report to Congressional Addressees March 2017...Office Highlights of GAO-17-244SP, a report to congressional addressees March 2017 CONTRACTING DATA ANALYSIS Assessment of Government -wide...Trends What GAO Found GAO’s analysis of government -wide contracting data found that while defense obligations to buy products and services

Archiving, Distribution and Analysis of Solar-B Data

Science.gov (United States)

Shimojo, M.

2007-10-01

The Solar-B Mission Operation and Data Analysis (MODA) working group has been discussing the data analysis system for Solar-B data since 2001. In the paper, based on the Solar-B MODA document and the recent work in Japan, we introduce the dataflow from Solar-B to scientists, the data format and data-level of Solar-B data, and the data searching/providing system.

The JASMIN Analysis Platform - bridging the gap between traditional climate data practicies and data-centric analysis paradigms

Science.gov (United States)

Pascoe, Stephen; Iwi, Alan; kershaw, philip; Stephens, Ag; Lawrence, Bryan

2014-05-01

The advent of large-scale data and the consequential analysis problems have led to two new challenges for the research community: how to share such data to get the maximum value and how to carry out efficient analysis. Solving both challenges require a form of parallelisation: the first is social parallelisation (involving trust and information sharing), the second data parallelisation (involving new algorithms and tools). The JASMIN infrastructure supports both kinds of parallelism by providing a multi-tennent environment with petabyte-scale storage, VM provisioning and batch cluster facilities. The JASMIN Analysis Platform (JAP) is an analysis software layer for JASMIN which emphasises ease of transition from a researcher's local environment to JASMIN. JAP brings together tools traditionally used by multiple communities and configures them to work together, enabling users to move analysis from their local environment to JASMIN without rewriting code. JAP also provides facilities to exploit JASMIN's parallel capabilities whilst maintaining their familiar analysis environment where ever possible. Modern opensource analysis tools typically have multiple dependent packages, increasing the installation burden on system administrators. When you consider a suite of tools, often with both common and conflicting dependencies, analysis pipelines can become locked to a particular installation simply because of the effort required to reconstruct the dependency tree. JAP addresses this problem by providing a consistent suite of RPMs compatible with RedHat Enterprise Linux and CentOS 6.4. Researchers can install JAP locally, either as RPMs or through a pre-built VM image, giving them the confidence to know moving analysis to JASMIN will not disrupt their environment. Analysis parallelisation is in it's infancy in climate sciences, with few tools capable of exploiting any parallel environment beyond manual scripting of the use of multiple processors. JAP begins to bridge this

EBT data acquisition and analysis system

International Nuclear Information System (INIS)

Burris, R.D.; Greenwood, D.E.; Stanton, J.S.; Geoffroy, K.A.

1980-10-01

This document describes the design and implementation of a data acquisition and analysis system for the EBT fusion experiment. The system includes data acquisition on five computers, automatic transmission of that data to a large, central data base, and a powerful data retrieval system. The system is flexible and easy to use, and it provides a fully documented record of the experiments

Analysis of mixed data methods & applications

CERN Document Server

de Leon, Alexander R

2013-01-01

A comprehensive source on mixed data analysis, Analysis of Mixed Data: Methods & Applications summarizes the fundamental developments in the field. Case studies are used extensively throughout the book to illustrate interesting applications from economics, medicine and health, marketing, and genetics. Carefully edited for smooth readability and seamless transitions between chaptersAll chapters follow a common structure, with an introduction and a concluding summary, and include illustrative examples from real-life case studies in developmental toxicolog

On vehicular traffic data analysis

Energy Technology Data Exchange (ETDEWEB)

Brics, Martins; Mahnke, Reinhard [Institute of Physics, Rostock University (Germany)

2011-07-01

This contribution consists of analysis of empirical vehicular traffic flow data. The main focus lies on the Next Generation Simulation (NGSIM) data. The first findings show that there are artificial structures within the data due to errors of monitoring as well as smoothing position measurement data. As a result speed data show discretisation in 5 feet per second. The aim of this investigation is to construct microscopic traffic flow models which are in agreement to the analysed empirical data. The ongoing work follows the subject of research summarized by Christof Liebe in his PhD thesis entitled ''Physics of traffic flow: Empirical data and dynamical models'' (Rostock, 2010).

Object-oriented data analysis framework for neutron scattering experiments

International Nuclear Information System (INIS)

Suzuki, Jiro; Nakatani, Takeshi; Ohhara, Takashi; Inamura, Yasuhiro; Yonemura, Masao; Morishima, Takahiro; Aoyagi, Tetsuo; Manabe, Atsushi; Otomo, Toshiya

2009-01-01

Materials and Life Science Facility (MLF) of Japan Proton Accelerator Research Complex (J-PARC) is one of the facilities that provided the highest intensity pulsed neutron and muon beams. The MLF computing environment design group organizes the computing environments of MLF and instruments. It is important that the computing environment is provided by the facility side, because meta-data formats, the analysis functions and also data analysis strategy should be shared among many instruments in MLF. The C++ class library, named Manyo-lib, is a framework software for developing data reduction and analysis softwares. The framework is composed of the class library for data reduction and analysis operators, network distributed data processing modules and data containers. The class library is wrapped by the Python interface created by SWIG. All classes of the framework can be called from Python language, and Manyo-lib will be cooperated with the data acquisition and data-visualization components through the MLF-platform, a user interface unified in MLF, which is working on Python language. Raw data in the event-data format obtained by data acquisition systems will be converted into histogram format data on Manyo-lib in high performance, and data reductions and analysis are performed with user-application software developed based on Manyo-lib. We enforce standardization of data containers with Manyo-lib, and many additional fundamental data containers in Manyo-lib have been designed and developed. Experimental and analysis data in the data containers can be converted into NeXus file. Manyo-lib is the standard framework for developing analysis software in MLF, and prototypes of data-analysis softwares for each instrument are being developed by the instrument teams.

Analysis of mass spectrometry data in proteomics

DEFF Research Database (Denmark)

Matthiesen, Rune; Jensen, Ole N

2008-01-01

The systematic study of proteins and protein networks, that is, proteomics, calls for qualitative and quantitative analysis of proteins and peptides. Mass spectrometry (MS) is a key analytical technology in current proteomics and modern mass spectrometers generate large amounts of high-quality data...... that in turn allow protein identification, annotation of secondary modifications, and determination of the absolute or relative abundance of individual proteins. Advances in mass spectrometry-driven proteomics rely on robust bioinformatics tools that enable large-scale data analysis. This chapter describes...... some of the basic concepts and current approaches to the analysis of MS and MS/MS data in proteomics....

Distributed Data Analysis in ATLAS

CERN Document Server

Nilsson, P; The ATLAS collaboration

2012-01-01

Data analysis using grid resources is one of the fundamental challenges to be addressed before the start of LHC data taking. The ATLAS detector will produce petabytes of data per year, and roughly one thousand users will need to run physics analyses on this data. Appropriate user interfaces and helper applications have been made available to ensure that the grid resources can be used without requiring expertise in grid technology. These tools enlarge the number of grid users from a few production administrators to potentially all participating physicists. ATLAS makes use of three grid infrastructures for the distributed analysis: the EGEE sites, the Open Science Grid, and NorduGrid. These grids are managed by the gLite workload management system, the PanDA workload management system, and ARC middleware; many sites can be accessed via both the gLite WMS and PanDA. Users can choose between two front-end tools to access the distributed resources. Ganga is a tool co-developed with LHCb to provide a common interfa...

Analysis of biomarker data a practical guide

CERN Document Server

Looney, Stephen W

2015-01-01

A "how to" guide for applying statistical methods to biomarker data analysis Presenting a solid foundation for the statistical methods that are used to analyze biomarker data, Analysis of Biomarker Data: A Practical Guide features preferred techniques for biomarker validation. The authors provide descriptions of select elementary statistical methods that are traditionally used to analyze biomarker data with a focus on the proper application of each method, including necessary assumptions, software recommendations, and proper interpretation of computer output. In addition, the book discusses

Intelligent Data Analysis in the 21st Century

Science.gov (United States)

Cohen, Paul; Adams, Niall

When IDA began, data sets were small and clean, data provenance and management were not significant issues, workflows and grid computing and cloud computing didn’t exist, and the world was not populated with billions of cellphone and computer users. The original conception of intelligent data analysis — automating some of the reasoning of skilled data analysts — has not been updated to account for the dramatic changes in what skilled data analysis means, today. IDA might update its mission to address pressing problems in areas such as climate change, habitat loss, education, and medicine. It might anticipate data analysis opportunities five to ten years out, such as customizing educational trajectories to individual students, and personalizing medical protocols. Such developments will elevate the conference and our community by shifting our focus from arbitrary measures of the performance of isolated algorithms to the practical, societal value of intelligent data analysis systems.

The ASDEX integrated data analysis system AIDA

International Nuclear Information System (INIS)

Grassie, K.; Gruber, O.; Kardaun, O.; Kaufmann, M.; Lackner, K.; Martin, P.; Mast, K.F.; McCarthy, P.J.; Mertens, V.; Pohl, D.; Rang, U.; Wunderlich, R.

1989-11-01

Since about two years, the ASDEX integrated data analysis system (AIDA), which combines the database (DABA) and the statistical analysis system (SAS), is successfully in operation. Besides a considerable, but meaningful, reduction of the 'raw' shot data, it offers the advantage of carefully selected and precisely defined datasets, which are easily accessible for informative tabular data overviews (DABA), and multi-shot analysis (SAS). Even rather complicated, statistical analyses can be performed efficiently within this system. In this report, we want to summarise AIDA's main features, give some details on its set-up and on the physical models which have been used for the derivation of the processed data. We also give short introduction how to use DABA and SAS. (orig.)

Coupling Visualization and Data Analysis for Knowledge Discovery from Multi-dimensional Scientific Data

International Nuclear Information System (INIS)

Rubel, Oliver; Ahern, Sean; Bethel, E. Wes; Biggin, Mark D.; Childs, Hank; Cormier-Michel, Estelle; DePace, Angela; Eisen, Michael B.; Fowlkes, Charless C.; Geddes, Cameron G.R.; Hagen, Hans; Hamann, Bernd; Huang, Min-Yu; Keranen, Soile V.E.; Knowles, David W.; Hendriks, Chris L. Luengo; Malik, Jitendra; Meredith, Jeremy; Messmer, Peter; Prabhat; Ushizima, Daniela; Weber, Gunther H.; Wu, Kesheng

2010-01-01

Knowledge discovery from large and complex scientific data is a challenging task. With the ability to measure and simulate more processes at increasingly finer spatial and temporal scales, the growing number of data dimensions and data objects presents tremendous challenges for effective data analysis and data exploration methods and tools. The combination and close integration of methods from scientific visualization, information visualization, automated data analysis, and other enabling technologies 'such as efficient data management' supports knowledge discovery from multi-dimensional scientific data. This paper surveys two distinct applications in developmental biology and accelerator physics, illustrating the effectiveness of the described approach.

A SWOT Analysis of Big Data

Science.gov (United States)

Ahmadi, Mohammad; Dileepan, Parthasarati; Wheatley, Kathleen K.

2016-01-01

This is the decade of data analytics and big data, but not everyone agrees with the definition of big data. Some researchers see it as the future of data analysis, while others consider it as hype and foresee its demise in the near future. No matter how it is defined, big data for the time being is having its glory moment. The most important…

Information, Privacy and Stability in Adaptive Data Analysis

OpenAIRE

Smith, Adam

2017-01-01

Traditional statistical theory assumes that the analysis to be performed on a given data set is selected independently of the data themselves. This assumption breaks downs when data are re-used across analyses and the analysis to be performed at a given stage depends on the results of earlier stages. Such dependency can arise when the same data are used by several scientific studies, or when a single analysis consists of multiple stages. How can we draw statistically valid conclusions when da...

Data Analysis of Cybercrimes in Businesses

Directory of Open Access Journals (Sweden)

Balan Shilpa

2017-12-01

Full Text Available In the current digital age, most people have become very dependent on technology for their daily work tasks. With the rise of the technological advancements, cyber-attacks have also increased. Over the past few years, there have been several security breaches. When sensitive data are breached, both organisations and consumers are affected. In the present research, we analyse the cyber security risks and its impact on organisations. To perform the analysis, a big data technology such as R programming is used. For example, using a big data analysis, it was found that the majority of businesses detected at least one incident involving a local area network (LAN breach.

SWISH DataLab: A Web Interface for Data Exploration and Analysis

NARCIS (Netherlands)

T. Bogaard (Tessel); J. Wielemaker (Jan); L. Hollink (Laura); J.R. van Ossenbruggen (Jacco)

2017-01-01

textabstractSWISH DataLab is a single integrated collaborative environment for data processing, exploration and analysis combining Prolog and R. The web interface makes it possible to share the data, the code of all processing steps and the results among researchers; and a versioning system

On the Impact of Inhomogeneities in Meteorological Data on VLBI Data Analysis

Science.gov (United States)

Balidakis, Kyriakos; Heinkelmann, Robert; Phogat, Apurva; Soja, Benedikt; Glaser, Susanne; Nilsson, Tobias; Karbon, Maria; Schuh, Harald

2016-12-01

In this study, we address the issue of the quality of meteorological data employed for VLBI data analysis. We use data from six numerical weather models (NWMs) to form references on which the homogenization process is based. We explore the impact of the choice of NWM as well as the way to extract data from it. Among our findings is that data from the surface fields of NWMs are not suitable for either geodetic analysis or homogenization efforts, whether they are in their original form or after they have been compensated for the height difference between the orography of the NWM and the actual elevation. The reason lies in the fact that for 77% of the VLBI stations a height bias larger than 2.5 mm appears, as well as an average bias in the zenith wet delay estimates of 12.2 mm. Should the proposed extraction approach be followed, the difference between operational and reanalysis NWMs is not significant for such an application. Our conclusions are based on the analysis of VLBI data over 13 years.

SOLE: enhanced FIA data analysis capabilities

Science.gov (United States)

Michael Spinney; Paul Van Deusen

2009-01-01

The Southern On Line Estimator (SOLE), is an Internet-based annual forest inventory and analysis (FIA) data analysis tool developed cooperatively by the National Council for Air and Stream Improvement and the Forest Service, U.S. Department of Agriculture's Forest Inventory and Analysis program at the Southern Research Station. Recent development of SOLE has...

Leveraging Data Analysis for Domain Experts: An Embeddable Framework for Basic Data Science Tasks

Science.gov (United States)

Lohrer, Johannes-Y.; Kaltenthaler, Daniel; Kröger, Peer

2016-01-01

In this paper, we describe a framework for data analysis that can be embedded into a base application. Since it is important to analyze the data directly inside the application where the data is entered, a tool that allows the scientists to easily work with their data, supports and motivates the execution of further analysis of their data, which…

Spatiotemporal Data Mining, Analysis, and Visualization of Human Activity Data

Science.gov (United States)

Li, Xun

2012-01-01

This dissertation addresses the research challenge of developing efficient new methods for discovering useful patterns and knowledge in large volumes of electronically collected spatiotemporal activity data. I propose to analyze three types of such spatiotemporal activity data in a methodological framework that integrates spatial analysis, data…

Proteomics wants cRacker: automated standardized data analysis of LC-MS derived proteomic data.

Science.gov (United States)

Zauber, Henrik; Schulze, Waltraud X

2012-11-02

The large-scale analysis of thousands of proteins under various experimental conditions or in mutant lines has gained more and more importance in hypothesis-driven scientific research and systems biology in the past years. Quantitative analysis by large scale proteomics using modern mass spectrometry usually results in long lists of peptide ion intensities. The main interest for most researchers, however, is to draw conclusions on the protein level. Postprocessing and combining peptide intensities of a proteomic data set requires expert knowledge, and the often repetitive and standardized manual calculations can be time-consuming. The analysis of complex samples can result in very large data sets (lists with several 1000s to 100,000 entries of different peptides) that cannot easily be analyzed using standard spreadsheet programs. To improve speed and consistency of the data analysis of LC-MS derived proteomic data, we developed cRacker. cRacker is an R-based program for automated downstream proteomic data analysis including data normalization strategies for metabolic labeling and label free quantitation. In addition, cRacker includes basic statistical analysis, such as clustering of data, or ANOVA and t tests for comparison between treatments. Results are presented in editable graphic formats and in list files.

Integrated Data Collection Analysis (IDCA) Program — RDX Standard Data Sets

Energy Technology Data Exchange (ETDEWEB)

Sandstrom, Mary M. [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Brown, Geoffrey W. [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Preston, Daniel N. [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Pollard, Colin J. [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Warner, Kirstin F. [Naval Surface Warfare Center (NSWC), Indian Head, MD (United States). Indian Head Division; Sorensen, Daniel N. [Naval Surface Warfare Center (NSWC), Indian Head, MD (United States). Indian Head Division; Remmers, Daniel L. [Naval Surface Warfare Center (NSWC), Indian Head, MD (United States). Indian Head Division; Phillips, Jason J. [Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Shelley, Timothy J. [Bureau of Alcohol, Tobacco and Firearms, Huntsville, AL (United States); Reyes, Jose A. [Applied Research Associates, Tyndall AFB, FL (United States); Hsu, Peter C. [Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States); Reynolds, John G. [Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)

2013-03-04

The Integrated Data Collection Analysis (IDCA) program is conducting a proficiency study for Small- Scale Safety and Thermal (SSST) testing of homemade explosives (HMEs). Described here are the results for impact, friction, electrostatic discharge, and differential scanning calorimetry analysis of the RDX Type II Class 5 standard, for a third and fourth time in the Proficiency Test and averaged with the analysis results from the first and second time. The results, from averaging all four sets (1, 2, 3 and 4) of data suggest a material to have slightly more impact sensitivity, more BAM friction sensitivity, less ABL friction sensitivity, similar ESD sensitivity, and same DSC sensitivity, compared to the results from Set 1, which was used previously as the values for the RDX standard in IDCA Analysis Reports.

Analysis of high-fold gamma data

International Nuclear Information System (INIS)

Radford, D. C.; Cromaz, M.; Beyer, C. J.

1999-01-01

Historically, γ-γ and γ-γ-γ coincidence spectra were utilized to build nuclear level schemes. With the development of large detector arrays, it has became possible to analyze higher fold coincidence data sets. This paper briefly reports on software to analyze 4-fold coincidence data sets that allows creation of 4-fold histograms (hypercubes) of at least 1024 channels per side (corresponding to a 43 gigachannel data space) that will fit onto a few gigabytes of disk space, and extraction of triple-gated spectra in a few seconds. Future detector arrays may have even much higher efficiencies, and detect as many as 15 or 20 γ rays simultaneously; such data will require very different algorithms for storage and analysis. Difficulties inherent in the analysis of such data are discussed, and two possible new solutions are presented, namely adaptive list-mode systems and 'list-list-mode' storage

Statistical methods for astronomical data analysis

CERN Document Server

Chattopadhyay, Asis Kumar

2014-01-01

This book introduces “Astrostatistics” as a subject in its own right with rewarding examples, including work by the authors with galaxy and Gamma Ray Burst data to engage the reader. This includes a comprehensive blending of Astrophysics and Statistics. The first chapter’s coverage of preliminary concepts and terminologies for astronomical phenomenon will appeal to both Statistics and Astrophysics readers as helpful context. Statistics concepts covered in the book provide a methodological framework. A unique feature is the inclusion of different possible sources of astronomical data, as well as software packages for converting the raw data into appropriate forms for data analysis. Readers can then use the appropriate statistical packages for their particular data analysis needs. The ideas of statistical inference discussed in the book help readers determine how to apply statistical tests. The authors cover different applications of statistical techniques already developed or specifically introduced for ...

Textbooks for Responsible Data Analysis in Excel

Science.gov (United States)

Garrett, Nathan

2015-01-01

With 27 million users, Excel (Microsoft Corporation, Seattle, WA) is the most common business data analysis software. However, audits show that almost all complex spreadsheets have errors. The author examined textbooks to understand why responsible data analysis is taught. A purposeful sample of 10 textbooks was coded, and then compared against…

Data analysis techniques for gravitational wave observations

Indian Academy of Sciences (India)

Astrophysical sources of gravitational waves fall broadly into three categories: (i) transient and bursts, (ii) periodic or continuous wave and (iii) stochastic. Each type of source requires a different type of data analysis strategy. In this talk various data analysis strategies will be reviewed. Optimal filtering is used for extracting ...

Compositional Data Analysis Theory and Applications

CERN Document Server

Pawlowsky-Glahn, Vera

2011-01-01

This book presents the state-of-the-art in compositional data analysis and will feature a collection of papers covering theory, applications to various fields of science and software. Areas covered will range from geology, biology, environmental sciences, forensic sciences, medicine and hydrology. Key features:Provides the state-of-the-art text in compositional data analysisCovers a variety of subject areas, from geology to medicineWritten by leading researchers in the fieldIs supported by a website featuring R code

Analysis of Ordinal Categorical Data

CERN Document Server

Agresti, Alan

2012-01-01

Statistical science's first coordinated manual of methods for analyzing ordered categorical data, now fully revised and updated, continues to present applications and case studies in fields as diverse as sociology, public health, ecology, marketing, and pharmacy. Analysis of Ordinal Categorical Data, Second Edition provides an introduction to basic descriptive and inferential methods for categorical data, giving thorough coverage of new developments and recent methods. Special emphasis is placed on interpretation and application of methods including an integrated comparison of the available st

Chaotic data analysis of heart R-R interval EKG data

International Nuclear Information System (INIS)

Frison, T.W.; Peng, C.K.; Goldberger, A.; Katz, R.A.

1996-01-01

Cardiac beat-to-beat interval data is analyzed with a chaotic data analysis toolkit. The embedding dimension of ten data sets from healthy subjects is 7 or at most 8. Ten of the eleven pathological data sets have an embedding dimension of 9 or greater. Statistically, the first local minimum of average mutual information for healthy hearts is larger than the pathological cases. But, there is a large standard deviation for this metric that blurs the distinction between the healthy and pathological data. copyright 1996 American Institute of Physics

Lectures on categorical data analysis

CERN Document Server

Rudas, Tamás

2018-01-01

This book offers a relatively self-contained presentation of the fundamental results in categorical data analysis, which plays a central role among the statistical techniques applied in the social, political and behavioral sciences, as well as in marketing and medical and biological research. The methods applied are mainly aimed at understanding the structure of associations among variables and the effects of other variables on these interactions. A great advantage of studying categorical data analysis is that many concepts in statistics become transparent when discussed in a categorical data context, and, in many places, the book takes this opportunity to comment on general principles and methods in statistics, addressing not only the “how” but also the “why.” Assuming minimal background in calculus, linear algebra, probability theory and statistics, the book is designed to be used in upper-undergraduate and graduate-level courses in the field and in more general statistical methodology courses, as w...

LISA Pathfinder instrument data analysis

Science.gov (United States)

Guzman, Felipe

LISA Pathfinder (LPF) is an ESA-launched demonstration mission of key technologies required for the joint NASA-ESA gravitational wave observatory in space, LISA. As part of the LPF interferometry investigations, analytic models of noise sources and corresponding noise subtrac-tion techniques have been developed to correct for effects like the coupling of test mass jitter into displacement readout, and fluctuations of the laser frequency or optical pathlength difference. Ground testing of pre-flight hardware of the Optical Metrology Subsystem is currently ongoing at the Albert Einstein Institute Hannover. In collaboration with NASA Goddard Space Flight Center, the LPF mission data analysis tool LTPDA is being used to analyze the data product of these tests. Furthermore, the noise subtraction techniques and in-flight experiment runs for noise characterization are being defined as part of the mission experiment master plan. We will present the data analysis outcome of pre-flight hardware ground tests and possible noise subtraction strategies for in-flight instrument operations.

Structural-Vibration-Response Data Analysis

Science.gov (United States)

Smith, W. R.; Hechenlaible, R. N.; Perez, R. C.

1983-01-01

Computer program developed as structural-vibration-response data analysis tool for use in dynamic testing of Space Shuttle. Program provides fast and efficient time-domain least-squares curve-fitting procedure for reducing transient response data to obtain structural model frequencies and dampings from free-decay records. Procedure simultaneously identifies frequencies, damping values, and participation factors for noisy multiple-response records.

Radiation and environmental data analysis computer (REDAC) hardware, software band analysis procedures

International Nuclear Information System (INIS)

Hendricks, T.J.

1985-01-01

The REDAC was conceived originally as a tape verifier for the Radiation and Environmental Data Acquisition Recorder (REDAR). From that simple beginning in 1971, the REDAC has evolved into a family of systems used for complete analysis of data obtained by the REDAR and other acquisition systems. Portable or mobile REDACs are deployed to support checkout and analysis tasks in the field. Laboratory systems are additionally used for software development, physics investigations, data base management and graphics. System configurations range from man-portable systems to a large laboratory-based system which supports time-shared analysis and development tasks. Custom operating software allows the analyst to process data either interactively or by batch procedures. Analysis packages are provided for numerous necessary functions. All these analysis procedures can be performed even on the smallest man-portable REDAC. Examples of the multi-isotope stripping and radiation isopleth mapping are presented. Techniques utilized for these operations are also presented

CMS Analysis and Data Reduction with Apache Spark

Energy Technology Data Exchange (ETDEWEB)

Gutsche, Oliver [Fermilab; Canali, Luca [CERN; Cremer, Illia [Magnetic Corp., Waltham; Cremonesi, Matteo [Fermilab; Elmer, Peter [Princeton U.; Fisk, Ian [Flatiron Inst., New York; Girone, Maria [CERN; Jayatilaka, Bo [Fermilab; Kowalkowski, Jim [Fermilab; Khristenko, Viktor [CERN; Motesnitsalis, Evangelos [CERN; Pivarski, Jim [Princeton U.; Sehrish, Saba [Fermilab; Surdy, Kacper [CERN; Svyatkovskiy, Alexey [Princeton U.

2017-10-31

Experimental Particle Physics has been at the forefront of analyzing the world's largest datasets for decades. The HEP community was among the first to develop suitable software and computing tools for this task. In recent times, new toolkits and systems for distributed data processing, collectively called "Big Data" technologies have emerged from industry and open source projects to support the analysis of Petabyte and Exabyte datasets in industry. While the principles of data analysis in HEP have not changed (filtering and transforming experiment-specific data formats), these new technologies use different approaches and tools, promising a fresh look at analysis of very large datasets that could potentially reduce the time-to-physics with increased interactivity. Moreover these new tools are typically actively developed by large communities, often profiting of industry resources, and under open source licensing. These factors result in a boost for adoption and maturity of the tools and for the communities supporting them, at the same time helping in reducing the cost of ownership for the end-users. In this talk, we are presenting studies of using Apache Spark for end user data analysis. We are studying the HEP analysis workflow separated into two thrusts: the reduction of centrally produced experiment datasets and the end-analysis up to the publication plot. Studying the first thrust, CMS is working together with CERN openlab and Intel on the CMS Big Data Reduction Facility. The goal is to reduce 1 PB of official CMS data to 1 TB of ntuple output for analysis. We are presenting the progress of this 2-year project with first results of scaling up Spark-based HEP analysis. Studying the second thrust, we are presenting studies on using Apache Spark for a CMS Dark Matter physics search, comparing Spark's feasibility, usability and performance to the ROOT-based analysis.

Gaussian process regression analysis for functional data

CERN Document Server

Shi, Jian Qing

2011-01-01

Gaussian Process Regression Analysis for Functional Data presents nonparametric statistical methods for functional regression analysis, specifically the methods based on a Gaussian process prior in a functional space. The authors focus on problems involving functional response variables and mixed covariates of functional and scalar variables.Covering the basics of Gaussian process regression, the first several chapters discuss functional data analysis, theoretical aspects based on the asymptotic properties of Gaussian process regression models, and new methodological developments for high dime

Guide on reflectivity data analysis

International Nuclear Information System (INIS)

Lee, Jeong Soo; Ku, Ja Seung; Seong, Baek Seok; Lee, Chang Hee; Hong, Kwang Pyo; Choi, Byung Hoon

2004-09-01

This report contains reduction and fitting process of neutron reflectivity data by REFLRED and REFLFIT in NIST. Because the detail of data reduction like BKG, footprint and data normalization was described, it will be useful to the user who has no experience in this field. Also, reflectivity and BKG of d-PS thin film were measured by HANARO neutron reflectometer. From these, the structure of d-PS thin film was analyzed with REFLRED and REFLFIT. Because the structure of thin film such as thickness, roughness and SLD was attained in the work, the possibility of data analysis with REFLRED and REFLFIT was certified

DATA ENVELOPMENT ANALYSIS OF BANKING SECTOR IN BANGLADESH

Directory of Open Access Journals (Sweden)

Md. Rashedul Hoque

2012-05-01

Full Text Available Banking sector of Bangladesh is flourishing and contributing to its economy. In this aspect measuring efficiency is important. Data Envelopment Analysis technique is used for this purpose. The data are collected from the annual reports of twenty four different banks in Bangladesh. Data Envelopment Analysis is mainly of two types - constant returns to scale and variable returns to scale. Since this study attempts to maximize output, so the output oriented Data Envelopment Analysis is used. The most efficient bank is one that obtains the highest efficiency score.

A Probabilistic Analysis of Data Popularity in ATLAS Data Caching

International Nuclear Information System (INIS)

Titov, M; Záruba, G; De, K; Klimentov, A

2012-01-01

One of the most important aspects in any computing distribution system is efficient data replication over storage or computing centers, that guarantees high data availability and low cost for resource utilization. In this paper we propose a data distribution scheme for the production and distributed analysis system PanDA at the ATLAS experiment. Our proposed scheme is based on the investigation of data usage. Thus, the paper is focused on the main concepts of data popularity in the PanDA system and their utilization. Data popularity is represented as the set of parameters that are used to predict the future data state in terms of popularity levels.

ADAGE signature analysis: differential expression analysis with data-defined gene sets.

Science.gov (United States)

Tan, Jie; Huyck, Matthew; Hu, Dongbo; Zelaya, René A; Hogan, Deborah A; Greene, Casey S

2017-11-22

Gene set enrichment analysis and overrepresentation analyses are commonly used methods to determine the biological processes affected by a differential expression experiment. This approach requires biologically relevant gene sets, which are currently curated manually, limiting their availability and accuracy in many organisms without extensively curated resources. New feature learning approaches can now be paired with existing data collections to directly extract functional gene sets from big data. Here we introduce a method to identify perturbed processes. In contrast with methods that use curated gene sets, this approach uses signatures extracted from public expression data. We first extract expression signatures from public data using ADAGE, a neural network-based feature extraction approach. We next identify signatures that are differentially active under a given treatment. Our results demonstrate that these signatures represent biological processes that are perturbed by the experiment. Because these signatures are directly learned from data without supervision, they can identify uncurated or novel biological processes. We implemented ADAGE signature analysis for the bacterial pathogen Pseudomonas aeruginosa. For the convenience of different user groups, we implemented both an R package (ADAGEpath) and a web server ( http://adage.greenelab.com ) to run these analyses. Both are open-source to allow easy expansion to other organisms or signature generation methods. We applied ADAGE signature analysis to an example dataset in which wild-type and ∆anr mutant cells were grown as biofilms on the Cystic Fibrosis genotype bronchial epithelial cells. We mapped active signatures in the dataset to KEGG pathways and compared with pathways identified using GSEA. The two approaches generally return consistent results; however, ADAGE signature analysis also identified a signature that revealed the molecularly supported link between the MexT regulon and Anr. We designed

Bridging ImmunoGenomic Data Analysis Workflow Gaps (BIGDAWG): An integrated case-control analysis pipeline.

Science.gov (United States)

Pappas, Derek J; Marin, Wesley; Hollenbach, Jill A; Mack, Steven J

2016-03-01

Bridging ImmunoGenomic Data-Analysis Workflow Gaps (BIGDAWG) is an integrated data-analysis pipeline designed for the standardized analysis of highly-polymorphic genetic data, specifically for the HLA and KIR genetic systems. Most modern genetic analysis programs are designed for the analysis of single nucleotide polymorphisms, but the highly polymorphic nature of HLA and KIR data require specialized methods of data analysis. BIGDAWG performs case-control data analyses of highly polymorphic genotype data characteristic of the HLA and KIR loci. BIGDAWG performs tests for Hardy-Weinberg equilibrium, calculates allele frequencies and bins low-frequency alleles for k×2 and 2×2 chi-squared tests, and calculates odds ratios, confidence intervals and p-values for each allele. When multi-locus genotype data are available, BIGDAWG estimates user-specified haplotypes and performs the same binning and statistical calculations for each haplotype. For the HLA loci, BIGDAWG performs the same analyses at the individual amino-acid level. Finally, BIGDAWG generates figures and tables for each of these comparisons. BIGDAWG obviates the error-prone reformatting needed to traffic data between multiple programs, and streamlines and standardizes the data-analysis process for case-control studies of highly polymorphic data. BIGDAWG has been implemented as the bigdawg R package and as a free web application at bigdawg.immunogenomics.org. Copyright © 2015 American Society for Histocompatibility and Immunogenetics. Published by Elsevier Inc. All rights reserved.

Qualitative case study data analysis: an example from practice.

Science.gov (United States)

Houghton, Catherine; Murphy, Kathy; Shaw, David; Casey, Dympna

2015-05-01

To illustrate an approach to data analysis in qualitative case study methodology. There is often little detail in case study research about how data were analysed. However, it is important that comprehensive analysis procedures are used because there are often large sets of data from multiple sources of evidence. Furthermore, the ability to describe in detail how the analysis was conducted ensures rigour in reporting qualitative research. The research example used is a multiple case study that explored the role of the clinical skills laboratory in preparing students for the real world of practice. Data analysis was conducted using a framework guided by the four stages of analysis outlined by Morse ( 1994 ): comprehending, synthesising, theorising and recontextualising. The specific strategies for analysis in these stages centred on the work of Miles and Huberman ( 1994 ), which has been successfully used in case study research. The data were managed using NVivo software. Literature examining qualitative data analysis was reviewed and strategies illustrated by the case study example provided. Discussion Each stage of the analysis framework is described with illustration from the research example for the purpose of highlighting the benefits of a systematic approach to handling large data sets from multiple sources. By providing an example of how each stage of the analysis was conducted, it is hoped that researchers will be able to consider the benefits of such an approach to their own case study analysis. This paper illustrates specific strategies that can be employed when conducting data analysis in case study research and other qualitative research designs.

Hurricane Data Analysis Tool

Science.gov (United States)

Liu, Zhong; Ostrenga, Dana; Leptoukh, Gregory

2011-01-01

In order to facilitate Earth science data access, the NASA Goddard Earth Sciences Data Information Services Center (GES DISC) has developed a web prototype, the Hurricane Data Analysis Tool (HDAT; URL: http://disc.gsfc.nasa.gov/HDAT), to allow users to conduct online visualization and analysis of several remote sensing and model datasets for educational activities and studies of tropical cyclones and other weather phenomena. With a web browser and few mouse clicks, users can have a full access to terabytes of data and generate 2-D or time-series plots and animation without downloading any software and data. HDAT includes data from the NASA Tropical Rainfall Measuring Mission (TRMM), the NASA Quick Scatterometer(QuikSCAT) and NECP Reanalysis, and the NCEP/CPC half-hourly, 4-km Global (60 N - 60 S) IR Dataset. The GES DISC archives TRMM data. The daily global rainfall product derived from the 3-hourly multi-satellite precipitation product (3B42 V6) is available in HDAT. The TRMM Microwave Imager (TMI) sea surface temperature from the Remote Sensing Systems is in HDAT as well. The NASA QuikSCAT ocean surface wind and the NCEP Reanalysis provide ocean surface and atmospheric conditions, respectively. The global merged IR product, also known as, the NCEP/CPC half-hourly, 4-km Global (60 N -60 S) IR Dataset, is one of TRMM ancillary datasets. They are globally-merged pixel-resolution IR brightness temperature data (equivalent blackbody temperatures), merged from all available geostationary satellites (GOES-8/10, METEOSAT-7/5 & GMS). The GES DISC has collected over 10 years of the data beginning from February of 2000. This high temporal resolution (every 30 minutes) dataset not only provides additional background information to TRMM and other satellite missions, but also allows observing a wide range of meteorological phenomena from space, such as, hurricanes, typhoons, tropical cyclones, mesoscale convection system, etc. Basic functions include selection of area of

Analysis of Home Health Sensor Data

NARCIS (Netherlands)

Kröse, B.; van Hoof, J.; Demiris, G.; Wouters, E.J.M.

2014-01-01

This chapter focuses on the analysis of data that is collected from sensors in the home environment. First we discuss the need for a good model that relates sensor data (or features derived from the data) to indicators of health and well-being. Then we present several methods for model building. We

Processing of pulse oximeter data using discrete wavelet analysis.

Science.gov (United States)

Lee, Seungjoon; Ibey, Bennett L; Xu, Weijian; Wilson, Mark A; Ericson, M Nance; Coté, Gerard L

2005-07-01

A wavelet-based signal processing technique was employed to improve an implantable blood perfusion monitoring system. Data was acquired from both in vitro and in vivo sources: a perfusion model and the proximal jejunum of an adult pig. Results showed that wavelet analysis could isolate perfusion signals from raw, periodic, in vitro data as well as fast Fourier transform (FFT) methods. However, for the quasi-periodic in vivo data segments, wavelet analysis provided more consistent results than the FFT analysis for data segments of 50, 10, and 5 s in length. Wavelet analysis has thus been shown to require less data points for quasi-periodic data than FFT analysis making it a good choice for an indwelling perfusion monitor where power consumption and reaction time are paramount.

Data formats design of laser irradiation experiments in view of data analysis

International Nuclear Information System (INIS)

Su Chunxiao; Yu Xiaoqi; Yang Cunbang; Guo Su; Chen Hongsu

2002-01-01

The designing rules of new data file formats of laser irradiation experiments are introduced. Object-oriented programs are designed in studying experimental data of the laser facilities. The new format data files are combinations of the experiment data and diagnostic configuration data, which are applied in data processing and analysis. The edit of diagnostic configuration data in data acquisition program is also described

CMS Data Analysis: Current Status and Future Strategy

CERN Document Server

Innocente, V

2003-01-01

We present the current status of CMS data analysis architecture and describe work on future Grid-based distributed analysis prototypes. CMS has two main software frameworks related to data analysis: COBRA, the main framework, and IGUANA, the interactive visualisation framework. Software using these frameworks is used today in the world-wide production and analysis of CMS data. We describe their overall design and present examples of their current use with emphasis on interactive analysis. CMS is currently developing remote analysis prototypes, including one based on Clarens, a Grid-enabled client-server tool. Use of the prototypes by CMS physicists will guide us in forming a Grid-enriched analysis strategy. The status of this work is presented, as is an outline of how we plan to leverage the power of our existing frameworks in the migration of CMS software to the Grid.

Quality Analysis of Open Street Map Data

Science.gov (United States)

Wang, M.; Li, Q.; Hu, Q.; Zhou, M.

2013-05-01

Crowd sourcing geographic data is an opensource geographic data which is contributed by lots of non-professionals and provided to the public. The typical crowd sourcing geographic data contains GPS track data like OpenStreetMap, collaborative map data like Wikimapia, social websites like Twitter and Facebook, POI signed by Jiepang user and so on. These data will provide canonical geographic information for pubic after treatment. As compared with conventional geographic data collection and update method, the crowd sourcing geographic data from the non-professional has characteristics or advantages of large data volume, high currency, abundance information and low cost and becomes a research hotspot of international geographic information science in the recent years. Large volume crowd sourcing geographic data with high currency provides a new solution for geospatial database updating while it need to solve the quality problem of crowd sourcing geographic data obtained from the non-professionals. In this paper, a quality analysis model for OpenStreetMap crowd sourcing geographic data is proposed. Firstly, a quality analysis framework is designed based on data characteristic analysis of OSM data. Secondly, a quality assessment model for OSM data by three different quality elements: completeness, thematic accuracy and positional accuracy is presented. Finally, take the OSM data of Wuhan for instance, the paper analyses and assesses the quality of OSM data with 2011 version of navigation map for reference. The result shows that the high-level roads and urban traffic network of OSM data has a high positional accuracy and completeness so that these OSM data can be used for updating of urban road network database.

Inspection, visualisation and analysis of quantitative proteomics data

OpenAIRE

Gatto, Laurent

2016-01-01

Material Quantitative Proteomics and Data Analysis Course. 4 - 5 April 2016, Queen Hotel, Chester, UK Table D - Inspection, visualisation and analysis of quantitative proteomics data, Laurent Gatto (University of Cambridge)

Data Analysis and reduction in Hanford's corrosion monitoring systems

International Nuclear Information System (INIS)

EDGEMON, G.L.

1999-01-01

A project to improve the Hanford Site's corrosion monitoring strategy was started in 1995. The project is designed to integrate EN-based corrosion monitoring into the site's corrosion monitoring strategy. In order to monitor multiple tanks, a major focus of this project has been to automate the data collection and analysis process. Data collection and analysis from the early EN corrosion monitoring equipment (241-AZ-101 and 241-AN-107) was primarily performed manually by a trained operator skilled in the analysis of EN data. Thousands of raw data files were collected, manually sorted and stored. Further statistical analysis of these files was performed by manually stripping out data from thousands of raw data files and calculating statistics in a spreadsheet format. Plotting and other graphical display analyses were performed by manually exporting data from the data files or spreadsheet into another plotting or presentation software package. In 1999, an Amulet/PRP system was procured and employed on the 241-AN-102 corrosion monitoring system. A duplicate system was purchased for use on the upcoming 241-AN-105 system. A third system has been procured and will eventually be used to upgrade the 241-AN-107 system. The Amulet software has greatly improved the automation of waste tank EN data analysis. In contrast with previous systems, the Amulet operator no longer has to manually collect, sort, store, and analyze thousands of raw EN data files. Amulet writes all data to a single database. Statistical analysis, uniform corrosion rate, and other derived parameters are automatically calculated in Amulet from the raw data while the raw data are being collected. Other improvements in plotting and presentation make inspection of the data a much quicker and relatively easy task. These and other improvements have greatly improved the speed at which EN data can be analyzed in addition to improving the quality of the final interpretation. The increase in data automation offered

Earth Science Data Analysis in the Era of Big Data

Science.gov (United States)

Kuo, K.-S.; Clune, T. L.; Ramachandran, R.

2014-01-01

Anyone with even a cursory interest in information technology cannot help but recognize that "Big Data" is one of the most fashionable catchphrases of late. From accurate voice and facial recognition, language translation, and airfare prediction and comparison, to monitoring the real-time spread of flu, Big Data techniques have been applied to many seemingly intractable problems with spectacular successes. They appear to be a rewarding way to approach many currently unsolved problems. Few fields of research can claim a longer history with problems involving voluminous data than Earth science. The problems we are facing today with our Earth's future are more complex and carry potentially graver consequences than the examples given above. How has our climate changed? Beside natural variations, what is causing these changes? What are the processes involved and through what mechanisms are these connected? How will they impact life as we know it? In attempts to answer these questions, we have resorted to observations and numerical simulations with ever-finer resolutions, which continue to feed the "data deluge." Plausibly, many Earth scientists are wondering: How will Big Data technologies benefit Earth science research? As an example from the global water cycle, one subdomain among many in Earth science, how would these technologies accelerate the analysis of decades of global precipitation to ascertain the changes in its characteristics, to validate these changes in predictive climate models, and to infer the implications of these changes to ecosystems, economies, and public health? Earth science researchers need a viable way to harness the power of Big Data technologies to analyze large volumes and varieties of data with velocity and veracity. Beyond providing speedy data analysis capabilities, Big Data technologies can also play a crucial, albeit indirect, role in boosting scientific productivity by facilitating effective collaboration within an analysis environment

Abnormal traffic flow data detection based on wavelet analysis

Directory of Open Access Journals (Sweden)

Xiao Qian

2016-01-01

Full Text Available In view of the traffic flow data of non-stationary, the abnormal data detection is difficult.proposed basing on the wavelet analysis and least squares method of abnormal traffic flow data detection in this paper.First using wavelet analysis to make the traffic flow data of high frequency and low frequency component and separation, and then, combined with least square method to find abnormal points in the reconstructed signal data.Wavelet analysis and least square method, the simulation results show that using wavelet analysis of abnormal traffic flow data detection, effectively reduce the detection results of misjudgment rate and false negative rate.

Full second order chromatographic/spectrometric data matrices for automated sample identification and component analysis by non-data-reducing image analysis

DEFF Research Database (Denmark)

Nielsen, Niles-Peter Vest; Smedsgaard, Jørn; Frisvad, Jens Christian

1999-01-01

A data analysis method is proposed for identification and for confirmation of classification schemes, based on single- or multiple-wavelength chromatographic profiles. The proposed method works directly on the chromatographic data without data reduction procedures such as peak area or retention...... classes from the reference chromatograms, This feature is a valuable aid in selecting components for further analysis, The identification method is demonstrated on two data sets: 212 isolates from 41 food-borne Penicillium species and 61 isolates from 6 soil-borne Penicillium species. Both data sets...

A program for activation analysis data processing

International Nuclear Information System (INIS)

Janczyszyn, J.; Loska, L.; Taczanowski, S.

1978-01-01

An ALGOL program for activation analysis data handling is presented. The program may be used either for single channel spectrometry data or for multichannel spectrometry. The calculation of instrumental error and of analysis standard deviation is carried out. The outliers are tested, and the regression line diagram with the related observations are plotted by the program. (author)

Functional data analysis of sleeping energy expenditure.

Science.gov (United States)

Lee, Jong Soo; Zakeri, Issa F; Butte, Nancy F

2017-01-01

Adequate sleep is crucial during childhood for metabolic health, and physical and cognitive development. Inadequate sleep can disrupt metabolic homeostasis and alter sleeping energy expenditure (SEE). Functional data analysis methods were applied to SEE data to elucidate the population structure of SEE and to discriminate SEE between obese and non-obese children. Minute-by-minute SEE in 109 children, ages 5-18, was measured in room respiration calorimeters. A smoothing spline method was applied to the calorimetric data to extract the true smoothing function for each subject. Functional principal component analysis was used to capture the important modes of variation of the functional data and to identify differences in SEE patterns. Combinations of functional principal component analysis and classifier algorithm were used to classify SEE. Smoothing effectively removed instrumentation noise inherent in the room calorimeter data, providing more accurate data for analysis of the dynamics of SEE. SEE exhibited declining but subtly undulating patterns throughout the night. Mean SEE was markedly higher in obese than non-obese children, as expected due to their greater body mass. SEE was higher among the obese than non-obese children (p0.1, after post hoc testing). Functional principal component scores for the first two components explained 77.8% of the variance in SEE and also differed between groups (p = 0.037). Logistic regression, support vector machine or random forest classification methods were able to distinguish weight-adjusted SEE between obese and non-obese participants with good classification rates (62-64%). Our results implicate other factors, yet to be uncovered, that affect the weight-adjusted SEE of obese and non-obese children. Functional data analysis revealed differences in the structure of SEE between obese and non-obese children that may contribute to disruption of metabolic homeostasis.

Visual modeling in an analysis of multidimensional data

Science.gov (United States)

Zakharova, A. A.; Vekhter, E. V.; Shklyar, A. V.; Pak, A. J.

2018-01-01

The article proposes an approach to solve visualization problems and the subsequent analysis of multidimensional data. Requirements to the properties of visual models, which were created to solve analysis problems, are described. As a perspective direction for the development of visual analysis tools for multidimensional and voluminous data, there was suggested an active use of factors of subjective perception and dynamic visualization. Practical results of solving the problem of multidimensional data analysis are shown using the example of a visual model of empirical data on the current state of studying processes of obtaining silicon carbide by an electric arc method. There are several results of solving this problem. At first, an idea of possibilities of determining the strategy for the development of the domain, secondly, the reliability of the published data on this subject, and changes in the areas of attention of researchers over time.

Data management and statistical analysis for environmental assessment

International Nuclear Information System (INIS)

Wendelberger, J.R.; McVittie, T.I.

1995-01-01

Data management and statistical analysis for environmental assessment are important issues on the interface of computer science and statistics. Data collection for environmental decision making can generate large quantities of various types of data. A database/GIS system developed is described which provides efficient data storage as well as visualization tools which may be integrated into the data analysis process. FIMAD is a living database and GIS system. The system has changed and developed over time to meet the needs of the Los Alamos National Laboratory Restoration Program. The system provides a repository for data which may be accessed by different individuals for different purposes. The database structure is driven by the large amount and varied types of data required for environmental assessment. The integration of the database with the GIS system provides the foundation for powerful visualization and analysis capabilities

Method for statistical data analysis of multivariate observations

CERN Document Server

Gnanadesikan, R

1997-01-01

A practical guide for multivariate statistical techniques-- now updated and revised In recent years, innovations in computer technology and statistical methodologies have dramatically altered the landscape of multivariate data analysis. This new edition of Methods for Statistical Data Analysis of Multivariate Observations explores current multivariate concepts and techniques while retaining the same practical focus of its predecessor. It integrates methods and data-based interpretations relevant to multivariate analysis in a way that addresses real-world problems arising in many areas of inte

ADAS: Atomic data, modelling and analysis for fusion

International Nuclear Information System (INIS)

Summers, H. P.; O'Mullane, M. G.; Whiteford, A. D.; Badnell, N. R.; Loch, S. D.

2007-01-01

The Atomic Data and Analysis Structure, ADAS, comprises extensive fundamental and derived atomic data collections, interactive codes for the manipulation and generation of collisional-radiative data and models, off-line codes for large scale fundamental atomic data production and codes for diagnostic analysis in the fusion and astrophysical environments. ADAS data are organized according to precise specifications, tuned to application and are assigned to numbered ADAS data formats. Some of these formats contain very large quantities of data and some have achieved wide-scale adoption in the fusion community.The paper focuses on recent extensions of ADAS designed to orient ADAS to the needs of ITER. The issue of heavy atomic species, expected to be present as ITER wall and divertor materials, dopants or control species, will be addressed with a view to the economized handling of the emission and ionisation state data needed for diagnostic spectral analysis. Charge exchange and beam emission spectroscopic capabilities and developments in ADAS will be reviewed from an ITER perspective and in the context of a shared analysis between fusion laboratories. Finally an overview and summary of current large scale fundamental data production in the framework of the ADAS project will be given and its intended availability in both fusion and astrophysics noted

Status of MTP Data Analysis for TCSP

Science.gov (United States)

Mahoney, Michael J.

2006-01-01

Topics covered include: a) MTP temperature calibration and data analysis; b) Background for interpreting MTP data; c) Large amplitude temperature structure; d) Gravity waves (GWs) in MTP data; and e) Subsidence over hurricanes.

A data skimming service for locally resident analysis data

International Nuclear Information System (INIS)

Cranshaw, J; Gieraltowski, J; Malon, D; May, E; Gardner, R W; Mambelli, M

2008-01-01

A Data Skimming Service (DSS) is a site-level service for rapid event filtering and selection from locally resident datasets based on metadata queries to associated 'tag' databases. In US ATLAS, we expect most if not all of the AOD-based datasets to be replicated to each of the five Tier 2 regional facilities in the US Tier 1 'cloud' coordinated by Brookhaven National Laboratory. Entire datasets will consist of on the order of several terabytes of data, and providing easy, quick access to skimmed subsets of these data will be vital to physics working groups. Typically, physicists will be interested in portions of the complete datasets, selected according to event-level attributes (number of jets, missing Et, etc) and content (specific analysis objects for subsequent processing). In this paper we describe methods used to classify data (metadata tag generation) and to store these results in a local database. Next we discuss a general framework which includes methods for accessing this information, defining skims, specifying event output content, accessing locally available storage through a variety of interfaces (SRM, dCache/dccp, gridftp), accessing remote storage elements as specified, and user job submission tools through local or grid schedulers. The advantages of the DSS are the ability to quickly 'browse' datasets and design skims, for example, pre-adjusting cuts to get to a desired skim level with minimal use of compute resources, and to encode these analysis operations in a database for re-analysis and archival purposes. Additionally the framework has provisions to operate autonomously in the event that external, central resources are not available, and to provide, as a reduced package, a minimal skimming service tailored to the needs of small Tier 3 centres or individual users

Hierarchical modeling and analysis for spatial data

CERN Document Server

Banerjee, Sudipto; Gelfand, Alan E

2003-01-01

Among the many uses of hierarchical modeling, their application to the statistical analysis of spatial and spatio-temporal data from areas such as epidemiology And environmental science has proven particularly fruitful. Yet to date, the few books that address the subject have been either too narrowly focused on specific aspects of spatial analysis, or written at a level often inaccessible to those lacking a strong background in mathematical statistics.Hierarchical Modeling and Analysis for Spatial Data is the first accessible, self-contained treatment of hierarchical methods, modeling, and dat

Conducting Qualitative Data Analysis: Managing Dynamic Tensions within

Science.gov (United States)

Chenail, Ronald J.

2012-01-01

In the third of a series of "how-to" essays on conducting qualitative data analysis, Ron Chenail examines the dynamic tensions within the process of qualitative data analysis that qualitative researchers must manage in order to produce credible and creative results. These tensions include (a) the qualities of the data and the qualitative data…

The Analysis of Polyploid Genetic Data.

Science.gov (United States)

Meirmans, Patrick G; Liu, Shenglin; van Tienderen, Peter H

2018-03-16

Though polyploidy is an important aspect of the evolutionary genetics of both plants and animals, the development of population genetic theory of polyploids has seriously lagged behind that of diploids. This is unfortunate since the analysis of polyploid genetic data-and the interpretation of the results-requires even more scrutiny than with diploid data. This is because of several polyploidy-specific complications in segregation and genotyping such as tetrasomy, double reduction, and missing dosage information. Here, we review the theoretical and statistical aspects of the population genetics of polyploids. We discuss several widely used types of inferences, including genetic diversity, Hardy-Weinberg equilibrium, population differentiation, genetic distance, and detecting population structure. For each, we point out how the statistical approach, expected result, and interpretation differ between different ploidy levels. We also discuss for each type of inference what biases may arise from the polyploid-specific complications and how these biases can be overcome. From our overview, it is clear that the statistical toolbox that is available for the analysis of genetic data is flexible and still expanding. Modern sequencing techniques will soon be able to overcome some of the current limitations to the analysis of polyploid data, though the techniques are lagging behind those available for diploids. Furthermore, the availability of more data may aggravate the biases that can arise, and increase the risk of false inferences. Therefore, simulations such as we used throughout this review are an important tool to verify the results of analyses of polyploid genetic data.

Statistical Analysis of Research Data | Center for Cancer Research

Science.gov (United States)

Recent advances in cancer biology have resulted in the need for increased statistical analysis of research data. The Statistical Analysis of Research Data (SARD) course will be held on April 5-6, 2018 from 9 a.m.-5 p.m. at the National Institutes of Health's Natcher Conference Center, Balcony C on the Bethesda Campus. SARD is designed to provide an overview on the general principles of statistical analysis of research data.  The first day will feature univariate data analysis, including descriptive statistics, probability distributions, one- and two-sample inferential statistics.

Mining survey data for SWOT analysis

OpenAIRE

Phadermrod, Boonyarat

2016-01-01

Strengths, Weaknesses, Opportunities and Threats (SWOT) analysis is one of the most important tools for strategic planning. The traditional method of conducting SWOT analysis does not prioritize and is likely to hold subjective views that may result in an improper strategic action. Accordingly, this research exploits Importance-Performance Analysis (IPA), a technique for measuring customers’ satisfaction based on survey data, to systematically generate prioritized SWOT factors based on custom...

Data Analysis for the LISA Pathfinder Mission

Science.gov (United States)

Thorpe, James Ira

2009-01-01

The LTP (LISA Technology Package) is the core part of the Laser Interferometer Space Antenna (LISA) Pathfinder mission. The main goal of the mission is to study the sources of any disturbances that perturb the motion of the freely-falling test masses from their geodesic trajectories as well as 10 test various technologies needed for LISA. The LTP experiment is designed as a sequence of experimental runs in which the performance of the instrument is studied and characterized under different operating conditions. In order to best optimize subsequent experimental runs, each run must be promptly analysed to ensure that the following ones make best use of the available knowledge of the instrument ' In order to do this, all analyses must be designed and tested in advance of the mission and have sufficient built-in flexibility to account for unexpected results or behaviour. To support this activity, a robust and flexible data analysis software package is also required. This poster presents two of the main components that make up the data analysis effort: the data analysis software and the mock-data challenges used to validate analysis procedures and experiment designs.

Matrix-based introduction to multivariate data analysis

CERN Document Server

Adachi, Kohei

2016-01-01

This book enables readers who may not be familiar with matrices to understand a variety of multivariate analysis procedures in matrix forms. Another feature of the book is that it emphasizes what model underlies a procedure and what objective function is optimized for fitting the model to data. The author believes that the matrix-based learning of such models and objective functions is the fastest way to comprehend multivariate data analysis. The text is arranged so that readers can intuitively capture the purposes for which multivariate analysis procedures are utilized: plain explanations of the purposes with numerical examples precede mathematical descriptions in almost every chapter. This volume is appropriate for undergraduate students who already have studied introductory statistics. Graduate students and researchers who are not familiar with matrix-intensive formulations of multivariate data analysis will also find the book useful, as it is based on modern matrix formulations with a special emphasis on ...

A Probabilistic Analysis of Data Popularity in ATLAS Data Caching

CERN Document Server

Titov, M; The ATLAS collaboration; Záruba, G; De, K

2012-01-01

Efficient distribution of physics data over ATLAS grid sites is one of the most important tasks for user data processing. ATLAS' initial static data distribution model over-replicated some unpopular data and under-replicated popular data, creating heavy disk space loads while underutilizing some processing resources due to low data availability. Thus, a new data distribution mechanism was implemented, PD2P (PanDA Dynamic Data Placement) within the production and distributed analysis system PanDA that dynamically reacts to user data needs, basing dataset distribution principally on user demand. Data deletion is also demand driven, reducing replica counts for unpopular data. This dynamic model has led to substantial improvements in efficient utilization of storage and processing resources. Based on this experience, in this work we seek to further improve data placement policy by investigating in detail how data popularity is calculated. For this it is necessary to precisely define what data popularity means, wh...

Data-driven security analysis, visualization and dashboards

CERN Document Server

Jacobs, Jay

2014-01-01

Uncover hidden patterns of data and respond with countermeasures Security professionals need all the tools at their disposal to increase their visibility in order to prevent security breaches and attacks. This careful guide explores two of the most powerful ? data analysis and visualization. You'll soon understand how to harness and wield data, from collection and storage to management and analysis as well as visualization and presentation. Using a hands-on approach with real-world examples, this book shows you how to gather feedback, measure the effectiveness of your security methods, and ma

Data structures and algorithm analysis in C++

CERN Document Server

Shaffer, Clifford A

2011-01-01

With its focus on creating efficient data structures and algorithms, this comprehensive text helps readers understand how to select or design the tools that will best solve specific problems. It uses Microsoft C++ as the programming language and is suitable for second-year data structure courses and computer science courses in algorithm analysis.Techniques for representing data are presented within the context of assessing costs and benefits, promoting an understanding of the principles of algorithm analysis and the effects of a chosen physical medium. The text also explores tradeoff issues, f

Data structures and algorithm analysis in Java

CERN Document Server

Shaffer, Clifford A

2011-01-01

With its focus on creating efficient data structures and algorithms, this comprehensive text helps readers understand how to select or design the tools that will best solve specific problems. It uses Java as the programming language and is suitable for second-year data structure courses and computer science courses in algorithm analysis. Techniques for representing data are presented within the context of assessing costs and benefits, promoting an understanding of the principles of algorithm analysis and the effects of a chosen physical medium. The text also explores tradeoff issues, familiari

Simulation and analysis of plutonium reprocessing plant data

International Nuclear Information System (INIS)

Burr, T.; Coulter, A.; Wangen, L.

1996-01-01

It will be difficult for large-throughput reprocessing plants to meet International Atomic Energy Agency (IAEA) detection goals for protracted diversion of plutonium by materials accounting alone. Therefore, the IAEA is considering supplementing traditional material balance analysis with analysis of solution monitoring data (frequent snapshots of such solution parameters as level, density, and temperature for all major process vessels). Analysis of solution monitoring data will enhance safeguards by improving anomaly detection and resolution, maintaining continuity of knowledge, and validating and improving measurement error models. However, there are costs associated with accessing and analyzing the data. To minimize these costs, analysis methods should be as complete as possible simple to implement, and require little human effort. As a step toward that goal, the authors have implemented simple analysis methods for use in an off-line situation. These methods use solution level to recognize major tank activities, such as tank-to-tank transfers and sampling. In this paper, the authors describe their application to realistic simulated data (the methods were developed by using both real and simulated data), and they present some quantifiable benefits of solution monitoring

Approaches to data analysis of multiple-choice questions

OpenAIRE

Lin Ding; Robert Beichner

2009-01-01

This paper introduces five commonly used approaches to analyzing multiple-choice test data. They are classical test theory, factor analysis, cluster analysis, item response theory, and model analysis. Brief descriptions of the goals and algorithms of these approaches are provided, together with examples illustrating their applications in physics education research. We minimize mathematics, instead placing emphasis on data interpretation using these approaches.

High-Level Overview of Data Needs for RE Analysis

Energy Technology Data Exchange (ETDEWEB)

Lopez, Anthony

2016-12-22

This presentation provides a high level overview of analysis topics and associated data needs. Types of renewable energy analysis are grouped into two buckets: First, analysis for renewable energy potential, and second, analysis for other goals. Data requirements are similar but and they build upon one another.

R data analysis without programming

CERN Document Server

Gerbing, David W

2013-01-01

This book prepares readers to analyze data and interpret statistical results using R more quickly than other texts. R is a challenging program to learn because code must be created to get started. To alleviate that challenge, Professor Gerbing developed lessR. LessR extensions remove the need to program. By introducing R through less R, readers learn how to organize data for analysis, read the data into R, and produce output without performing numerous functions and programming exercises first. With lessR, readers can select the necessary procedure and change the relevant variables without pro

Probabilistic Principal Component Analysis for Metabolomic Data.

LENUS (Irish Health Repository)

Nyamundanda, Gift

2010-11-23

Abstract Background Data from metabolomic studies are typically complex and high-dimensional. Principal component analysis (PCA) is currently the most widely used statistical technique for analyzing metabolomic data. However, PCA is limited by the fact that it is not based on a statistical model. Results Here, probabilistic principal component analysis (PPCA) which addresses some of the limitations of PCA, is reviewed and extended. A novel extension of PPCA, called probabilistic principal component and covariates analysis (PPCCA), is introduced which provides a flexible approach to jointly model metabolomic data and additional covariate information. The use of a mixture of PPCA models for discovering the number of inherent groups in metabolomic data is demonstrated. The jackknife technique is employed to construct confidence intervals for estimated model parameters throughout. The optimal number of principal components is determined through the use of the Bayesian Information Criterion model selection tool, which is modified to address the high dimensionality of the data. Conclusions The methods presented are illustrated through an application to metabolomic data sets. Jointly modeling metabolomic data and covariates was successfully achieved and has the potential to provide deeper insight to the underlying data structure. Examination of confidence intervals for the model parameters, such as loadings, allows for principled and clear interpretation of the underlying data structure. A software package called MetabolAnalyze, freely available through the R statistical software, has been developed to facilitate implementation of the presented methods in the metabolomics field.

Big Data Analysis of Human Genome Variations

KAUST Repository

Gojobori, Takashi

2016-01-25

Since the human genome draft sequence was in public for the first time in 2000, genomic analyses have been intensively extended to the population level. The following three international projects are good examples for large-scale studies of human genome variations: 1) HapMap Data (1,417 individuals) (http://hapmap.ncbi.nlm.nih.gov/downloads/genotypes/2010-08_phaseII+III/forward/), 2) HGDP (Human Genome Diversity Project) Data (940 individuals) (http://www.hagsc.org/hgdp/files.html), 3) 1000 genomes Data (2,504 individuals) http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ If we can integrate all three data into a single volume of data, we should be able to conduct a more detailed analysis of human genome variations for a total number of 4,861 individuals (= 1,417+940+2,504 individuals). In fact, we successfully integrated these three data sets by use of information on the reference human genome sequence, and we conducted the big data analysis. In particular, we constructed a phylogenetic tree of about 5,000 human individuals at the genome level. As a result, we were able to identify clusters of ethnic groups, with detectable admixture, that were not possible by an analysis of each of the three data sets. Here, we report the outcome of this kind of big data analyses and discuss evolutionary significance of human genomic variations. Note that the present study was conducted in collaboration with Katsuhiko Mineta and Kosuke Goto at KAUST.

Licensing Support System: Preliminary data scope analysis

International Nuclear Information System (INIS)

1989-01-01

The purpose of this analysis is to determine the content and scope of the Licensing Support System (LSS) data base. Both user needs and currently available data bases that, at least in part, address those needs have been analyzed. This analysis, together with the Preliminary Needs Analysis (DOE, 1988d) is a first effort under the LSS Design and Implementation Contract toward developing a sound requirements foundation for subsequent design work. These reports are preliminary. Further refinements must be made before requirements can be specified in sufficient detail to provide a basis for suitably specific system specifications. This document provides a baseline for what is known at this time. Additional analyses, currently being conducted, will provide more precise information on the content and scope of the LSS data base. 23 refs., 4 figs., 8 tabs

Complex Visual Data Analysis, Uncertainty, and Representation

National Research Council Canada - National Science Library

Schunn, Christian D; Saner, Lelyn D; Kirschenbaum, Susan K; Trafton, J. G; Littleton, Eliza B

2007-01-01

... (weather forecasting, submarine target motion analysis, and fMRI data analysis). Internal spatial representations are coded from spontaneous gestures made during cued-recall summaries of problem solving activities...

The Role of Microsimulation in Longitudinal Data Analysis

Directory of Open Access Journals (Sweden)

Douglas A. Wolf

2001-12-01

microsimulation also has the potential to contribute to longitudinal data analysis in several ways, including extending the range of outputs generated by a model, addressing several defective-data problems, and serving as a vehicle for missing-data imputation. This paper discusses microsimulation procedures suitable for several commonly-used statistical models applied to longitudinal data. It also addresses the unique role that can be played by microsimulation in longitudinal data analysis, and the problem of accounting for the several sources of variability associated with microsimulation procedures.

Utilization of Integrated Process Control, Data Capture, and Data Analysis in Construction of Accelerator Systems

International Nuclear Information System (INIS)

Bonnie Madre; Charles Reece; Joseph Ozelis; Valerie Bookwalter

2003-01-01

Jefferson Lab has developed a web-based system that integrates commercial database, data analysis, document archiving and retrieval, and user interface software, into a coherent knowledge management product (Pansophy). This product provides important tools for the successful pursuit of major projects such as accelerator system development and construction, by offering elements of process and procedure control, data capture and review, and data mining and analysis. After a period of initial development, Pansophy is now being used in Jefferson Lab's SNS superconducting linac construction effort, as a means for structuring and implementing the QA program, for process control and tracking, and for cryomodule test data capture and presentation/analysis. Development of Pansophy is continuing, in particular data queries and analysis functions that are the cornerstone of its utility

Estimating Most Productive Scale Size in Data Envelopment Analysis with Integer Value Data

Science.gov (United States)

Dwi Sari, Yunita; Angria S, Layla; Efendi, Syahril; Zarlis, Muhammad

2018-01-01

The most productive scale size (MPSS) is a measurement that states how resources should be organized and utilized to achieve optimal results. The most productive scale size (MPSS) can be used as a benchmark for the success of an industry or company in producing goods or services. To estimate the most productive scale size (MPSS), each decision making unit (DMU) should pay attention the level of input-output efficiency, by data envelopment analysis (DEA) method decision making unit (DMU) can identify units used as references that can help to find the cause and solution from inefficiencies can optimize productivity that main advantage in managerial applications. Therefore, data envelopment analysis (DEA) is chosen to estimating most productive scale size (MPSS) that will focus on the input of integer value data with the CCR model and the BCC model. The purpose of this research is to find the best solution for estimating most productive scale size (MPSS) with input of integer value data in data envelopment analysis (DEA) method.

RCRA groundwater data analysis protocol for the Hanford Site, Washington

International Nuclear Information System (INIS)

Chou, C.J.; Jackson, R.L.

1992-04-01

The Resource Conservation and Recovery Act of 1976 (RCRA) groundwater monitoring program currently involves site-specific monitoring of 20 facilities on the Hanford Site in southeastern Washington. The RCRA groundwater monitoring program has collected abundant data on groundwater quality. These data are used to assess the impact of a facility on groundwater quality or whether remediation efforts under RCRA corrective action programs are effective. Both evaluations rely on statistical analysis of groundwater monitoring data. The need for information on groundwater quality by regulators and environmental managers makes statistical analysis of monitoring data an important part of RCRA groundwater monitoring programs. The complexity of groundwater monitoring programs and variabilities (spatial, temporal, and analytical) exhibited in groundwater quality variables indicate the need for a data analysis protocol to guide statistical analysis. A data analysis protocol was developed from the perspective of addressing regulatory requirements, data quality, and management information needs. This data analysis protocol contains four elements: data handling methods; graphical evaluation techniques; statistical tests for trend, central tendency, and excursion analysis; and reporting procedures for presenting results to users

DARHT Multi-intelligence Seismic and Acoustic Data Analysis

Energy Technology Data Exchange (ETDEWEB)

Stevens, Garrison Nicole [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Van Buren, Kendra Lu [Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Hemez, Francois M. [Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

2016-07-21

The purpose of this report is to document the analysis of seismic and acoustic data collected at the Dual-Axis Radiographic Hydrodynamic Test (DARHT) facility at Los Alamos National Laboratory for robust, multi-intelligence decision making. The data utilized herein is obtained from two tri-axial seismic sensors and three acoustic sensors, resulting in a total of nine data channels. The goal of this analysis is to develop a generalized, automated framework to determine internal operations at DARHT using informative features extracted from measurements collected external of the facility. Our framework involves four components: (1) feature extraction, (2) data fusion, (3) classification, and finally (4) robustness analysis. Two approaches are taken for extracting features from the data. The first of these, generic feature extraction, involves extraction of statistical features from the nine data channels. The second approach, event detection, identifies specific events relevant to traffic entering and leaving the facility as well as explosive activities at DARHT and nearby explosive testing sites. Event detection is completed using a two stage method, first utilizing signatures in the frequency domain to identify outliers and second extracting short duration events of interest among these outliers by evaluating residuals of an autoregressive exogenous time series model. Features extracted from each data set are then fused to perform analysis with a multi-intelligence paradigm, where information from multiple data sets are combined to generate more information than available through analysis of each independently. The fused feature set is used to train a statistical classifier and predict the state of operations to inform a decision maker. We demonstrate this classification using both generic statistical features and event detection and provide a comparison of the two methods. Finally, the concept of decision robustness is presented through a preliminary analysis where

Network analysis for the visualization and analysis of qualitative data.

Science.gov (United States)

Pokorny, Jennifer J; Norman, Alex; Zanesco, Anthony P; Bauer-Wu, Susan; Sahdra, Baljinder K; Saron, Clifford D

2018-03-01

We present a novel manner in which to visualize the coding of qualitative data that enables representation and analysis of connections between codes using graph theory and network analysis. Network graphs are created from codes applied to a transcript or audio file using the code names and their chronological location. The resulting network is a representation of the coding data that characterizes the interrelations of codes. This approach enables quantification of qualitative codes using network analysis and facilitates examination of associations of network indices with other quantitative variables using common statistical procedures. Here, as a proof of concept, we applied this method to a set of interview transcripts that had been coded in 2 different ways and the resultant network graphs were examined. The creation of network graphs allows researchers an opportunity to view and share their qualitative data in an innovative way that may provide new insights and enhance transparency of the analytical process by which they reach their conclusions. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Integrative sparse principal component analysis of gene expression data.

Science.gov (United States)

Liu, Mengque; Fan, Xinyan; Fang, Kuangnan; Zhang, Qingzhao; Ma, Shuangge

2017-12-01

In the analysis of gene expression data, dimension reduction techniques have been extensively adopted. The most popular one is perhaps the PCA (principal component analysis). To generate more reliable and more interpretable results, the SPCA (sparse PCA) technique has been developed. With the "small sample size, high dimensionality" characteristic of gene expression data, the analysis results generated from a single dataset are often unsatisfactory. Under contexts other than dimension reduction, integrative analysis techniques, which jointly analyze the raw data of multiple independent datasets, have been developed and shown to outperform "classic" meta-analysis and other multidatasets techniques and single-dataset analysis. In this study, we conduct integrative analysis by developing the iSPCA (integrative SPCA) method. iSPCA achieves the selection and estimation of sparse loadings using a group penalty. To take advantage of the similarity across datasets and generate more accurate results, we further impose contrasted penalties. Different penalties are proposed to accommodate different data conditions. Extensive simulations show that iSPCA outperforms the alternatives under a wide spectrum of settings. The analysis of breast cancer and pancreatic cancer data further shows iSPCA's satisfactory performance. © 2017 WILEY PERIODICALS, INC.

Time series analysis methods and applications for flight data

CERN Document Server

Zhang, Jianye

2017-01-01

This book focuses on different facets of flight data analysis, including the basic goals, methods, and implementation techniques. As mass flight data possesses the typical characteristics of time series, the time series analysis methods and their application for flight data have been illustrated from several aspects, such as data filtering, data extension, feature optimization, similarity search, trend monitoring, fault diagnosis, and parameter prediction, etc. An intelligent information-processing platform for flight data has been established to assist in aircraft condition monitoring, training evaluation and scientific maintenance. The book will serve as a reference resource for people working in aviation management and maintenance, as well as researchers and engineers in the fields of data analysis and data mining.

Big data analysis new algorithms for a new society

CERN Document Server

Stefanowski, Jerzy

2016-01-01

This edited volume is devoted to Big Data Analysis from a Machine Learning standpoint as presented by some of the most eminent researchers in this area. It demonstrates that Big Data Analysis opens up new research problems which were either never considered before, or were only considered within a limited range. In addition to providing methodological discussions on the principles of mining Big Data and the difference between traditional statistical data analysis and newer computing frameworks, this book presents recently developed algorithms affecting such areas as business, financial forecasting, human mobility, the Internet of Things, information networks, bioinformatics, medical systems and life science. It explores, through a number of specific examples, how the study of Big Data Analysis has evolved and how it has started and will most likely continue to affect society. While the benefits brought upon by Big Data Analysis are underlined, the book also discusses some of the warnings that have been issued...

Bayesian networks for omics data analysis

NARCIS (Netherlands)

Gavai, A.K.

2009-01-01

This thesis focuses on two aspects of high throughput technologies, i.e. data storage and data analysis, in particular in transcriptomics and metabolomics. Both technologies are part of a research field that is generally called ‘omics’ (or ‘-omics’, with a leading hyphen), which refers to genomics,

Analyzing how we do Analysis and Consume Data, Results from the SciDAC-Data Project

Science.gov (United States)

Ding, P.; Aliaga, L.; Mubarak, M.; Tsaris, A.; Norman, A.; Lyon, A.; Ross, R.

2017-10-01

One of the main goals of the Dept. of Energy funded SciDAC-Data project is to analyze the more than 410,000 high energy physics datasets that have been collected, generated and defined over the past two decades by experiments using the Fermilab storage facilities. These datasets have been used as the input to over 5.6 million recorded analysis projects, for which detailed analytics have been gathered. The analytics and meta information for these datasets and analysis projects are being combined with knowledge of their part of the HEP analysis chains for major experiments to understand how modern computing and data delivery is being used. We present the first results of this project, which examine in detail how the CDF, D0, NOvA, MINERvA and MicroBooNE experiments have organized, classified and consumed petascale datasets to produce their physics results. The results include analysis of the correlations in dataset/file overlap, data usage patterns, data popularity, dataset dependency and temporary dataset consumption. The results provide critical insight into how workflows and data delivery schemes can be combined with different caching strategies to more efficiently perform the work required to mine these large HEP data volumes and to understand the physics analysis requirements for the next generation of HEP computing facilities. In particular we present a detailed analysis of the NOvA data organization and consumption model corresponding to their first and second oscillation results (2014-2016) and the first look at the analysis of the Tevatron Run II experiments. We present statistical distributions for the characterization of these data and data driven models describing their consumption.

Approaches to data analysis of multiple-choice questions

Directory of Open Access Journals (Sweden)

Lin Ding

2009-09-01

Full Text Available This paper introduces five commonly used approaches to analyzing multiple-choice test data. They are classical test theory, factor analysis, cluster analysis, item response theory, and model analysis. Brief descriptions of the goals and algorithms of these approaches are provided, together with examples illustrating their applications in physics education research. We minimize mathematics, instead placing emphasis on data interpretation using these approaches.

Toward improved analysis of concentration data: Embracing nondetects.

Science.gov (United States)

Shoari, Niloofar; Dubé, Jean-Sébastien

2018-03-01

Various statistical tests on concentration data serve to support decision-making regarding characterization and monitoring of contaminated media, assessing exposure to a chemical, and quantifying the associated risks. However, the routine statistical protocols cannot be directly applied because of challenges arising from nondetects or left-censored observations, which are concentration measurements below the detection limit of measuring instruments. Despite the existence of techniques based on survival analysis that can adjust for nondetects, these are seldom taken into account properly. A comprehensive review of the literature showed that managing policies regarding analysis of censored data do not always agree and that guidance from regulatory agencies may be outdated. Therefore, researchers and practitioners commonly resort to the most convenient way of tackling the censored data problem by substituting nondetects with arbitrary constants prior to data analysis, although this is generally regarded as a bias-prone approach. Hoping to improve the interpretation of concentration data, the present article aims to familiarize researchers in different disciplines with the significance of left-censored observations and provides theoretical and computational recommendations (under both frequentist and Bayesian frameworks) for adequate analysis of censored data. In particular, the present article synthesizes key findings from previous research with respect to 3 noteworthy aspects of inferential statistics: estimation of descriptive statistics, hypothesis testing, and regression analysis. Environ Toxicol Chem 2018;37:643-656. © 2017 SETAC. © 2017 SETAC.

Data analysis and approximate models model choice, location-scale, analysis of variance, nonparametric regression and image analysis

CERN Document Server

Davies, Patrick Laurie

2014-01-01

Introduction IntroductionApproximate Models Notation Two Modes of Statistical AnalysisTowards One Mode of Analysis Approximation, Randomness, Chaos, Determinism ApproximationA Concept of Approximation Approximation Approximating a Data Set by a Model Approximation Regions Functionals and EquivarianceRegularization and Optimality Metrics and DiscrepanciesStrong and Weak Topologies On Being (almost) Honest Simulations and Tables Degree of Approximation and p-values ScalesStability of Analysis The Choice of En(α, P) Independence Procedures, Approximation and VaguenessDiscrete Models The Empirical Density Metrics and Discrepancies The Total Variation Metric The Kullback-Leibler and Chi-Squared Discrepancies The Po(λ) ModelThe b(k, p) and nb(k, p) Models The Flying Bomb Data The Student Study Times Data OutliersOutliers, Data Analysis and Models Breakdown Points and Equivariance Identifying Outliers and Breakdown Outliers in Multivariate Data Outliers in Linear Regression Outliers in Structured Data The Location...

Analysis of the real EADGENE data set:

DEFF Research Database (Denmark)

Jaffrézic, Florence; de Koning, Dirk-Jan; Boettcher, Paul J

2007-01-01

A large variety of methods has been proposed in the literature for microarray data analysis. The aim of this paper was to present techniques used by the EADGENE (European Animal Disease Genomics Network of Excellence) WP1.4 participants for data quality control, normalisation and statistical...... methods for the detection of differentially expressed genes in order to provide some more general data analysis guidelines. All the workshop participants were given a real data set obtained in an EADGENE funded microarray study looking at the gene expression changes following artificial infection with two...... quarters. Very little transcriptional variation was observed for the bacteria S. aureus. Lists of differentially expressed genes found by the different research teams were, however, quite dependent on the method used, especially concerning the data quality control step. These analyses also emphasised...

Data analysis of event tape and connection

International Nuclear Information System (INIS)

Gong Huili

1995-01-01

The data analysis on the VAX-11/780 computer is briefly described, the data is from the recorded event tape of JUHU data acquisition system on the PDP-11/44 computer. The connection of the recorded event tapes of the XSYS data acquisition system on VAX computer is also introduced

Multiregion analysis of creep rupture data of 316 stainless steel

International Nuclear Information System (INIS)

Maruyama, Kouichi; Armaki, Hassan Ghassemi; Yoshimi, Kyosuke

2007-01-01

A creep rupture data set of 316 stainless steel containing 319 data points at nine heats was subjected to a conventional single-region analysis and a multiregion analysis. In the former, the conventional Larson-Miller analysis was applied to the whole data set. In the latter, a data set of a single heat is divided into several data sets, so that the Orr-Sherby-Dorn (OSD) constant Q takes a unique value in each data set, and the conventional OSD analysis was applied to each divided data set. A region with a low value of Q appears in long-term creep of eight heats. Predicted values of the 10 5 h creep rupture stress of three heats were lower than the 99% confidence limit evaluated by the single-region analysis, suggesting that the single-region analysis is error prone. The multiregion analysis is necessary for the correct evaluation of the long-term creep properties of 316 stainless steel

Remote Sensing Data Visualization, Fusion and Analysis via Giovanni

Science.gov (United States)

Leptoukh, G.; Zubko, V.; Gopalan, A.; Khayat, M.

2007-01-01

We describe Giovanni, the NASA Goddard developed online visualization and analysis tool that allows users explore various phenomena without learning remote sensing data formats and downloading voluminous data. Using MODIS aerosol data as an example, we formulate an approach to the data fusion for Giovanni to further enrich online multi-sensor remote sensing data comparison and analysis.

Python for data analysis data wrangling with Pandas, NumPy, and IPython

CERN Document Server

McKinney, Wes

2017-01-01

Get complete instructions for manipulating, processing, cleaning, and crunching datasets in Python. Updated for Python 3.6, the second edition of this hands-on guide is packed with practical case studies that show you how to solve a broad set of data analysis problems effectively. You’ll learn the latest versions of pandas, NumPy, IPython, and Jupyter in the process. Written by Wes McKinney, the creator of the Python pandas project, this book is a practical, modern introduction to data science tools in Python. It’s ideal for analysts new to Python and for Python programmers new to data science and scientific computing. Data files and related material are available on GitHub. Use the IPython shell and Jupyter notebook for exploratory computing Learn basic and advanced features in NumPy (Numerical Python) Get started with data analysis tools in the pandas library Use flexible tools to load, clean, transform, merge, and reshape data Create informative visualizations with matplotlib ...

Data Mining and Analysis

Science.gov (United States)

Samms, Kevin O.

2015-01-01

The Data Mining project seeks to bring the capability of data visualization to NASA anomaly and problem reporting systems for the purpose of improving data trending, evaluations, and analyses. Currently NASA systems are tailored to meet the specific needs of its organizations. This tailoring has led to a variety of nomenclatures and levels of annotation for procedures, parts, and anomalies making difficult the realization of the common causes for anomalies. Making significant observations and realizing the connection between these causes without a common way to view large data sets is difficult to impossible. In the first phase of the Data Mining project a portal was created to present a common visualization of normalized sensitive data to customers with the appropriate security access. The tool of the visualization itself was also developed and fine-tuned. In the second phase of the project we took on the difficult task of searching and analyzing the target data set for common causes between anomalies. In the final part of the second phase we have learned more about how much of the analysis work will be the job of the Data Mining team, how to perform that work, and how that work may be used by different customers in different ways. In this paper I detail how our perspective has changed after gaining more insight into how the customers wish to interact with the output and how that has changed the product.

Data analysis and interpretation for environmental surveillance

International Nuclear Information System (INIS)

1992-06-01

The Data Analysis and Interpretation for Environmental Surveillance Conference was held in Lexington, Kentucky, February 5--7, 1990. The conference was sponsored by what is now the Office of Environmental Compliance and Documentation, Oak Ridge National Laboratory. Participants included technical professionals from all Martin Marietta Energy Systems facilities, Westinghouse Materials Company of Ohio, Pacific Northwest Laboratory, and several technical support contractors. Presentations at the conference ranged the full spectrum of issues that effect the analysis and interpretation of environmental data. Topics included tracking systems for samples and schedules associated with ongoing programs; coalescing data from a variety of sources and pedigrees into integrated data bases; methods for evaluating the quality of environmental data through empirical estimates of parameters such as charge balance, pH, and specific conductance; statistical applications to the interpretation of environmental information; and uses of environmental information in risk and dose assessments. Hearing about and discussing this wide variety of topics provided an opportunity to capture the subtlety of each discipline and to appreciate the continuity that is required among the disciplines in order to perform high-quality environmental information analysis

Using influence diagrams for data worth analysis

International Nuclear Information System (INIS)

Sharif Heger, A.; White, Janis E.

1997-01-01

Decision-making under uncertainty describes most environmental remediation and waste management problems. Inherent limitations in knowledge concerning contaminants, environmental fate and transport, remedies, and risks force decision-makers to select a course of action based on uncertain and incomplete information. Because uncertainties can be reduced by collecting additional data., uncertainty and sensitivity analysis techniques have received considerable attention. When costs associated with reducing uncertainty are considered in a decision problem, the objective changes; rather than determine what data to collect to reduce overall uncertainty, the goal is to determine what data to collect to best differentiate between possible courses of action or decision alternatives. Environmental restoration and waste management requires cost-effective methods for characterization and monitoring, and these methods must also satisfy regulatory requirements. Characterization and monitoring activities imply that, sooner or later, a decision must be made about collecting new field data. Limited fiscal resources for data collection should be committed only to those data that have the most impact on the decision at lowest possible cost. Applying influence diagrams in combination with data worth analysis produces a method which not only satisfies these requirements but also gives rise to an intuitive representation of complex structures not possible in the more traditional decision tree representation. This paper demonstrates the use of influence diagrams in data worth analysis by applying to a monitor-and-treat problem often encountered in environmental decision problems

Data Processing and Analysis Systems for JT-60U

International Nuclear Information System (INIS)

Matsuda, T.; Totsuka, T.; Tsugita, T.; Oshima, T.; Sakata, S.; Sato, M.; Iwasaki, K.

2002-01-01

The JT-60U data processing system is a large computer complex gradually modernized by utilizing progressive computer and network technology. A main computer using state-of-the-art CMOS technology can handle ∼550 MB of data per discharge. A gigabit ethernet switch with FDDI ports has been introduced to cope with the increase of handling data. Workstation systems with VMEbus serial highway drivers for CAMAC have been developed and used to replace many minicomputer systems. VMEbus-based fast data acquisition systems have also been developed to enlarge and replace a minicomputer system for mass data.The JT-60U data analysis system is composed of a JT-60U database server and a JT-60U analysis server, which are distributed UNIX servers. The experimental database is stored in the 1TB RAID disk of the JT-60U database server and is composed of ZENKEI and diagnostic databases. Various data analysis tools are available on the JT-60U analysis server. For the remote collaboration, technical features of the data analysis system have been applied to the computer system to access JT-60U data via the Internet. Remote participation in JT-60U experiments has been successfully conducted since 1996

System for the analysis of cohort mortality data

International Nuclear Information System (INIS)

McLain, R.; Frome, E.L.

1986-01-01

A system is developed for the analysis of cohort mortality data. This Mortality Analysis System (MAS) is designed as a research tool in epidemiologic studies. The system allows a researcher to investigate the effect of one or more factors on the mortality of a study cohort. Variables can be categorized as factors to allow for stratification in the analysis. DATA steps and PROC MATRIX are incorporated in the system to produce the output. Person-years, observed deaths, and expected deaths are calculated and cross-classified by the levels of the factors. The resulting data set can be used to compute the standardized mortality ratios (SMR) for each stratum level. Poisson regression models can then be used for further statistical analysis

Parallel interactive data analysis with PROOF

International Nuclear Information System (INIS)

Ballintijn, Maarten; Biskup, Marek; Brun, Rene; Canal, Philippe; Feichtinger, Derek; Ganis, Gerardo; Kickinger, Guenter; Peters, Andreas; Rademakers, Fons

2006-01-01

The Parallel ROOT Facility, PROOF, enables the analysis of much larger data sets on a shorter time scale. It exploits the inherent parallelism in data of uncorrelated events via a multi-tier architecture that optimizes I/O and CPU utilization in heterogeneous clusters with distributed storage. The system provides transparent and interactive access to gigabytes today. Being part of the ROOT framework PROOF inherits the benefits of a performant object storage system and a wealth of statistical and visualization tools. This paper describes the data analysis model of ROOT and the latest developments on closer integration of PROOF into that model and the ROOT user environment, e.g. support for PROOF-based browsing of trees stored remotely, and the popular TTree::Draw() interface. We also outline the ongoing developments aimed to improve the flexibility and user-friendliness of the system

Advances in Moessbauer data analysis

International Nuclear Information System (INIS)

Souza, Paulo A. de

1998-01-01

The whole Moessbauer community generates a huge amount of data in several fields of human knowledge since the first publication of Rudolf Moessbauer. Interlaboratory measurements of the same substance may result in minor differences in the Moessbauer Parameters (MP) of isomer shift, quadrupole splitting and internal magnetic field. Therefore, a conventional data bank of published MP will be of limited help in identification of substances. Data bank search for exact information became incapable to differentiate the values of Moessbauer parameters within the experimental errors (e.g., IS = 0.22 mm/s from IS = 0.23 mm/s), but physically both values may be considered the same. An artificial neural network (ANN) is able to identify a substance and its crystalline structure from measured MP, and its slight variations do not represent an obstacle for the ANN identification. A barrier to the popularization of Moessbauer spectroscopy as an analytical technique is the absence of a full automated equipment, since the analysis of a Moessbauer spectrum normally is time-consuming and requires a specialist. In this work, the fitting process of a Moessbauer spectrum was completely automated through the use of genetic algorithms and fuzzy logic. Both software and hardware systems were implemented turning out to be a fully automated Moessbauer data analysis system. The developed system will be presented

PATTER, Pattern Recognition Data Analysis

International Nuclear Information System (INIS)

Cox, L.C. Jr.; Bender, C.F.

1986-01-01

1 - Description of program or function: PATTER is an interactive program with extensive facilities for modeling analytical processes and solving complex data analysis problems using statistical methods, spectral analysis, and pattern recognition techniques. PATTER addresses the type of problem generally stated as follows: given a set of objects and a list of measurements made on these objects, is it possible to find or predict a property of the objects which is not directly measurable but is known to define some unknown relationship? When employed intelligently, PATTER will act upon a data set in such a way it becomes apparent if useful information, beyond that already discerned, is contained in the data. 2 - Method of solution: In order to solve the general problem, PATTER contains preprocessing techniques to produce new variables that are related to the values of the measurements which may reduce the number of variables and/or reveal useful information about the 'obscure' property; display techniques to represent the variable space in some way that can be easily projected onto a two- or three-dimensional plot for human observation to see if any significant clustering of points occurs; and learning techniques based on both unsupervised and supervised methods, to extract as much information from the data as possible so that the optimum solution can be found

Data analysis facility at LAMPF

International Nuclear Information System (INIS)

Perry, D.G.; Amann, J.F.; Butler, H.S.; Hoffman, C.J.; Mischke, R.E.; Shera, E.B.; Thiessen, H.A.

1977-11-01

This report documents the discussions and conclusions of a study held in July 1977 to develop the requirements for a data analysis facility to support the experimental program in medium-energy physics at the Clinton P. Anderson Meson Physics Facility (LAMPF). 2 tables

An evaluation of Oracle for persistent data storage and analysis of LHC physics data

International Nuclear Information System (INIS)

Grancher, E.; Marczukajtis, M.

2001-01-01

CERN's IT/DB group is currently exploring the possibility of using oracle to store LHC physics data. It presents preliminary results from this work, concentrating on two aspects: the storage of RAW and the analysis of TAG data. The RAW data part of the study discusses the throughput that one can achieve with the oracle database system, the options for storing the data and an estimation of the associated overheads. The TAG data analysis focuses on the use of new and extended indexing features of oracle to perform efficient cuts on the data. The tests were performed with Oracle 8.1.7

Microrheology with optical tweezers: data analysis

International Nuclear Information System (INIS)

Tassieri, Manlio; Warren, Rebecca L; Cooper, Jonathan M; Evans, R M L; Bailey, Nicholas J

2012-01-01

We present a data analysis procedure that provides the solution to a long-standing issue in microrheology studies, i.e. the evaluation of the fluids' linear viscoelastic properties from the analysis of a finite set of experimental data, describing (for instance) the time-dependent mean-square displacement of suspended probe particles experiencing Brownian fluctuations. We report, for the first time in the literature, the linear viscoelastic response of an optically trapped bead suspended in a Newtonian fluid, over the entire range of experimentally accessible frequencies. The general validity of the proposed method makes it transferable to the majority of microrheology and rheology techniques. (paper)

Game data analysis tools and methods

CERN Document Server

Coupart, Thibault

2013-01-01

This book features an introduction to the basic theoretical tenets of data analysis from a game developer's point of view, as well as a practical guide to performing gameplay analysis on a real-world game.This book is ideal for video game developers who want to try and experiment with the game analytics approach for their own productions. It will provide a good overview of the themes you need to pay attention to, and will pave the way for success. Furthermore, the book also provides a wide range of concrete examples that will be useful for any game data analysts or scientists who want to impro

[Preliminarily application of content analysis to qualitative nursing data].

Science.gov (United States)

Liang, Shu-Yuan; Chuang, Yeu-Hui; Wu, Shu-Fang

2012-10-01

Content analysis is a methodology for objectively and systematically studying the content of communication in various formats. Content analysis in nursing research and nursing education is called qualitative content analysis. Qualitative content analysis is frequently applied to nursing research, as it allows researchers to determine categories inductively and deductively. This article examines qualitative content analysis in nursing research from theoretical and practical perspectives. We first describe how content analysis concepts such as unit of analysis, meaning unit, code, category, and theme are used. Next, we describe the basic steps involved in using content analysis, including data preparation, data familiarization, analysis unit identification, creating tentative coding categories, category refinement, and establishing category integrity. Finally, this paper introduces the concept of content analysis rigor, including dependability, confirmability, credibility, and transferability. This article elucidates the content analysis method in order to help professionals conduct systematic research that generates data that are informative and useful in practical application.

Application of Ontology Technology in Health Statistic Data Analysis.

Science.gov (United States)

Guo, Minjiang; Hu, Hongpu; Lei, Xingyun

2017-01-01

Research Purpose: establish health management ontology for analysis of health statistic data. Proposed Methods: this paper established health management ontology based on the analysis of the concepts in China Health Statistics Yearbook, and used protégé to define the syntactic and semantic structure of health statistical data. six classes of top-level ontology concepts and their subclasses had been extracted and the object properties and data properties were defined to establish the construction of these classes. By ontology instantiation, we can integrate multi-source heterogeneous data and enable administrators to have an overall understanding and analysis of the health statistic data. ontology technology provides a comprehensive and unified information integration structure of the health management domain and lays a foundation for the efficient analysis of multi-source and heterogeneous health system management data and enhancement of the management efficiency.

Design database for quantitative trait loci (QTL) data warehouse, data mining, and meta-analysis.

Science.gov (United States)

Hu, Zhi-Liang; Reecy, James M; Wu, Xiao-Lin

2012-01-01

A database can be used to warehouse quantitative trait loci (QTL) data from multiple sources for comparison, genomic data mining, and meta-analysis. A robust database design involves sound data structure logistics, meaningful data transformations, normalization, and proper user interface designs. This chapter starts with a brief review of relational database basics and concentrates on issues associated with curation of QTL data into a relational database, with emphasis on the principles of data normalization and structure optimization. In addition, some simple examples of QTL data mining and meta-analysis are included. These examples are provided to help readers better understand the potential and importance of sound database design.

Genetic data analysis for plant and animal breeding

Science.gov (United States)

This book is an advanced textbook covering the application of quantitative genetics theory to analysis of actual data (both trait and DNA marker information) for breeding populations of crops, trees, and animals. Chapter 1 is an introduction to basic software used for trait data analysis. Chapter 2 ...

Statistical analysis of network data with R

CERN Document Server

Kolaczyk, Eric D

2014-01-01

Networks have permeated everyday life through everyday realities like the Internet, social networks, and viral marketing. As such, network analysis is an important growth area in the quantitative sciences, with roots in social network analysis going back to the 1930s and graph theory going back centuries. Measurement and analysis are integral components of network research. As a result, statistical methods play a critical role in network analysis. This book is the first of its kind in network research. It can be used as a stand-alone resource in which multiple R packages are used to illustrate how to conduct a wide range of network analyses, from basic manipulation and visualization, to summary and characterization, to modeling of network data. The central package is igraph, which provides extensive capabilities for studying network graphs in R. This text builds on Eric D. Kolaczyk’s book Statistical Analysis of Network Data (Springer, 2009).

Challenges of Big Data Analysis.

Science.gov (United States)

Fan, Jianqing; Han, Fang; Liu, Han

2014-06-01

Big Data bring new opportunities to modern society and challenges to data scientists. On one hand, Big Data hold great promises for discovering subtle population patterns and heterogeneities that are not possible with small-scale data. On the other hand, the massive sample size and high dimensionality of Big Data introduce unique computational and statistical challenges, including scalability and storage bottleneck, noise accumulation, spurious correlation, incidental endogeneity, and measurement errors. These challenges are distinguished and require new computational and statistical paradigm. This article gives overviews on the salient features of Big Data and how these features impact on paradigm change on statistical and computational methods as well as computing architectures. We also provide various new perspectives on the Big Data analysis and computation. In particular, we emphasize on the viability of the sparsest solution in high-confidence set and point out that exogeneous assumptions in most statistical methods for Big Data can not be validated due to incidental endogeneity. They can lead to wrong statistical inferences and consequently wrong scientific conclusions.

MAGMA: generalized gene-set analysis of GWAS data.

Science.gov (United States)

de Leeuw, Christiaan A; Mooij, Joris M; Heskes, Tom; Posthuma, Danielle

2015-04-01

By aggregating data for complex traits in a biologically meaningful way, gene and gene-set analysis constitute a valuable addition to single-marker analysis. However, although various methods for gene and gene-set analysis currently exist, they generally suffer from a number of issues. Statistical power for most methods is strongly affected by linkage disequilibrium between markers, multi-marker associations are often hard to detect, and the reliance on permutation to compute p-values tends to make the analysis computationally very expensive. To address these issues we have developed MAGMA, a novel tool for gene and gene-set analysis. The gene analysis is based on a multiple regression model, to provide better statistical performance. The gene-set analysis is built as a separate layer around the gene analysis for additional flexibility. This gene-set analysis also uses a regression structure to allow generalization to analysis of continuous properties of genes and simultaneous analysis of multiple gene sets and other gene properties. Simulations and an analysis of Crohn's Disease data are used to evaluate the performance of MAGMA and to compare it to a number of other gene and gene-set analysis tools. The results show that MAGMA has significantly more power than other tools for both the gene and the gene-set analysis, identifying more genes and gene sets associated with Crohn's Disease while maintaining a correct type 1 error rate. Moreover, the MAGMA analysis of the Crohn's Disease data was found to be considerably faster as well.

SECIMTools: a suite of metabolomics data analysis tools.

Science.gov (United States)

Kirpich, Alexander S; Ibarra, Miguel; Moskalenko, Oleksandr; Fear, Justin M; Gerken, Joseph; Mi, Xinlei; Ashrafi, Ali; Morse, Alison M; McIntyre, Lauren M

2018-04-20

Metabolomics has the promise to transform the area of personalized medicine with the rapid development of high throughput technology for untargeted analysis of metabolites. Open access, easy to use, analytic tools that are broadly accessible to the biological community need to be developed. While technology used in metabolomics varies, most metabolomics studies have a set of features identified. Galaxy is an open access platform that enables scientists at all levels to interact with big data. Galaxy promotes reproducibility by saving histories and enabling the sharing workflows among scientists. SECIMTools (SouthEast Center for Integrated Metabolomics) is a set of Python applications that are available both as standalone tools and wrapped for use in Galaxy. The suite includes a comprehensive set of quality control metrics (retention time window evaluation and various peak evaluation tools), visualization techniques (hierarchical cluster heatmap, principal component analysis, modular modularity clustering), basic statistical analysis methods (partial least squares - discriminant analysis, analysis of variance, t-test, Kruskal-Wallis non-parametric test), advanced classification methods (random forest, support vector machines), and advanced variable selection tools (least absolute shrinkage and selection operator LASSO and Elastic Net). SECIMTools leverages the Galaxy platform and enables integrated workflows for metabolomics data analysis made from building blocks designed for easy use and interpretability. Standard data formats and a set of utilities allow arbitrary linkages between tools to encourage novel workflow designs. The Galaxy framework enables future data integration for metabolomics studies with other omics data.

Big climate data analysis

Science.gov (United States)

Mudelsee, Manfred

2015-04-01

The Big Data era has begun also in the climate sciences, not only in economics or molecular biology. We measure climate at increasing spatial resolution by means of satellites and look farther back in time at increasing temporal resolution by means of natural archives and proxy data. We use powerful supercomputers to run climate models. The model output of the calculations made for the IPCC's Fifth Assessment Report amounts to ~650 TB. The 'scientific evolution' of grid computing has started, and the 'scientific revolution' of quantum computing is being prepared. This will increase computing power, and data amount, by several orders of magnitude in the future. However, more data does not automatically mean more knowledge. We need statisticians, who are at the core of transforming data into knowledge. Statisticians notably also explore the limits of our knowledge (uncertainties, that is, confidence intervals and P-values). Mudelsee (2014 Climate Time Series Analysis: Classical Statistical and Bootstrap Methods. Second edition. Springer, Cham, xxxii + 454 pp.) coined the term 'optimal estimation'. Consider the hyperspace of climate estimation. It has many, but not infinite, dimensions. It consists of the three subspaces Monte Carlo design, method and measure. The Monte Carlo design describes the data generating process. The method subspace describes the estimation and confidence interval construction. The measure subspace describes how to detect the optimal estimation method for the Monte Carlo experiment. The envisaged large increase in computing power may bring the following idea of optimal climate estimation into existence. Given a data sample, some prior information (e.g. measurement standard errors) and a set of questions (parameters to be estimated), the first task is simple: perform an initial estimation on basis of existing knowledge and experience with such types of estimation problems. The second task requires the computing power: explore the hyperspace to

Database tools for enhanced analysis of TMX-U data

International Nuclear Information System (INIS)

Stewart, M.E.; Carter, M.R.; Casper, T.A.; Meyer, W.H.; Perkins, D.E.; Whitney, D.M.

1986-01-01

A commercial database software package has been used to create several databases and tools that assist and enhance the ability of experimental physicists to analyze data from the Tandem Mirror Experiment-Upgrade (TMX-U) experiment. This software runs on a DEC-20 computer in M-Divisions's User Service Center at Lawrence Livermore National Laboratory (LLNL), where data can be analyzed off line from the main TMX-U acquisition computers. When combined with interactive data analysis programs, these tools provide the capability to do batch-style processing or interactive data analysis on the computers in the USC or the supercomputers of the National Magnetic Fusion Energy Computer Center (NMFECC) in addition to the normal processing done by the TMX-U acquisition system. One database tool provides highly reduced data for searching and correlation analysis of several diagnostic signals within a single shot or over many shots. A second database tool provides retrieval and storage of unreduced data for use in detailed analysis of one or more diagnostic signals. We will show how these database tools form the core of an evolving off-line data analysis environment on the USC computers

Database tools for enhanced analysis of TMX-U data

International Nuclear Information System (INIS)

Stewart, M.E.; Carter, M.R.; Casper, T.A.; Meyer, W.H.; Perkins, D.E.; Whitney, D.M.

1986-01-01

A commercial database software package has been used to create several databases and tools that assist and enhance the ability of experimental physicists to analyze data from the Tandem Mirror Experiment-Upgrade (TMX-U) experiment. This software runs on a DEC-20 computer in M-Division's User Service Center at Lawrence Livermore National Laboratory (LLNL), where data can be analyzed offline from the main TMX-U acquisition computers. When combined with interactive data analysis programs, these tools provide the capability to do batch-style processing or interactive data analysis on the computers in the USC or the supercomputers of the National Magnetic Fusion Energy Computer Center (NMFECC) in addition to the normal processing done by the TMX-U acquisition system. One database tool provides highly reduced data for searching and correlation analysis of several diagnostic signals within a single shot or over many shots. A second database tool provides retrieval and storage of unreduced data for use in detailed analysis of one or more diagnostic signals. We will show how these database tools form the core of an evolving offline data analysis environment on the USC computers

Analysis of facility-monitoring data

Energy Technology Data Exchange (ETDEWEB)

Howell, J.A.

1996-09-01

This paper discusses techniques for analysis of data collected from nuclear-safeguards facility-monitoring systems. These methods can process information gathered from sensors and make interpretations that are in the best interests of the facility or agency, thereby enhancing safeguards while shortening inspection time.

Handbook on data envelopment analysis

CERN Document Server

Cooper, William W; Zhu, Joe

2011-01-01

Focusing on extensively used Data Envelopment Analysis topics, this volume aims to both describe the state of the field and extend the frontier of DEA research. New chapters include DEA models for DMUs, network DEA, models for supply chain operations and applications, and new developments.

Big data analysis for smart farming

NARCIS (Netherlands)

Kempenaar, C.; Lokhorst, C.; Bleumer, E.J.B.; Veerkamp, R.F.; Been, Th.; Evert, van F.K.; Boogaardt, M.J.; Ge, L.; Wolfert, J.; Verdouw, C.N.; Bekkum, van Michael; Feldbrugge, L.; Verhoosel, Jack P.C.; Waaij, B.D.; Persie, van M.; Noorbergen, H.

2016-01-01

In this report we describe results of a one-year TO2 institutes project on the development of big data technologies within the milk production chain. The goal of this project is to ‘create’ an integration platform for big data analysis for smart farming and to develop a show case. This includes both

A novel water quality data analysis framework based on time-series data mining.

Science.gov (United States)

Deng, Weihui; Wang, Guoyin

2017-07-01

The rapid development of time-series data mining provides an emerging method for water resource management research. In this paper, based on the time-series data mining methodology, we propose a novel and general analysis framework for water quality time-series data. It consists of two parts: implementation components and common tasks of time-series data mining in water quality data. In the first part, we propose to granulate the time series into several two-dimensional normal clouds and calculate the similarities in the granulated level. On the basis of the similarity matrix, the similarity search, anomaly detection, and pattern discovery tasks in the water quality time-series instance dataset can be easily implemented in the second part. We present a case study of this analysis framework on weekly Dissolve Oxygen time-series data collected from five monitoring stations on the upper reaches of Yangtze River, China. It discovered the relationship of water quality in the mainstream and tributary as well as the main changing patterns of DO. The experimental results show that the proposed analysis framework is a feasible and efficient method to mine the hidden and valuable knowledge from water quality historical time-series data. Copyright © 2017 Elsevier Ltd. All rights reserved.

Enabling Analysis of Big, Thick, Long, and Wide Data: Data Management for the Analysis of a Large Longitudinal and Cross-National Narrative Data Set.

Science.gov (United States)

Winskell, Kate; Singleton, Robyn; Sabben, Gaelle

2018-03-01

Distinctive longitudinal narrative data, collected during a critical 18-year period in the history of the HIV epidemic, offer a unique opportunity to examine how young Africans are making sense of evolving developments in HIV prevention and treatment. More than 200,000 young people from across sub-Saharan Africa took part in HIV-themed scriptwriting contests held at eight discrete time points between 1997 and 2014, creating more than 75,000 narratives. This article describes the data reduction and management strategies developed for our cross-national and longitudinal study of these qualitative data. The study aims to inform HIV communication practice by identifying cultural meanings and contextual factors that inform sexual behaviors and social practices, and also to help increase understanding of processes of sociocultural change. We describe our sampling strategies and our triangulating methodologies, combining in-depth narrative analysis, thematic qualitative analysis, and quantitative analysis, which are designed to enable systematic comparison without sacrificing ethnographic richness.

Planning representation for automated exploratory data analysis

Science.gov (United States)

St. Amant, Robert; Cohen, Paul R.

1994-03-01

Igor is a knowledge-based system for exploratory statistical analysis of complex systems and environments. Igor has two related goals: to help automate the search for interesting patterns in data sets, and to help develop models that capture significant relationships in the data. We outline a language for Igor, based on techniques of opportunistic planning, which balances control and opportunism. We describe the application of Igor to the analysis of the behavior of Phoenix, an artificial intelligence planning system.

NGNP Data Management and Analysis System Analysis and Web Delivery Capabilities

Energy Technology Data Exchange (ETDEWEB)

Cynthia D. Gentillon

2010-09-01

Projects for the Very High Temperature Reactor Technology Development Office provide data in support of Nuclear Regulatory Commission licensing of the very high temperature reactor. Fuel and materials to be used in the reactor are tested and characterized to quantify performance in high-temperature and high-fluence environments. In addition, thermal-hydraulic experiments are conducted to validate codes used to assess reactor safety. The Very High Temperature Reactor Technology Development Office has established the NGNP Data Management and Analysis System (NDMAS) at the Idaho National Laboratory to ensure that very high temperature reactor data are (1) qualified for use, (2) stored in a readily accessible electronic form, and (3) analyzed to extract useful results. This document focuses on the third NDMAS objective. It describes capabilities for displaying the data in meaningful ways and for data analysis to identify useful relationships among the measured quantities.

Multivariate statistical analysis of atom probe tomography data

International Nuclear Information System (INIS)

Parish, Chad M.; Miller, Michael K.

2010-01-01

The application of spectrum imaging multivariate statistical analysis methods, specifically principal component analysis (PCA), to atom probe tomography (APT) data has been investigated. The mathematical method of analysis is described and the results for two example datasets are analyzed and presented. The first dataset is from the analysis of a PM 2000 Fe-Cr-Al-Ti steel containing two different ultrafine precipitate populations. PCA properly describes the matrix and precipitate phases in a simple and intuitive manner. A second APT example is from the analysis of an irradiated reactor pressure vessel steel. Fine, nm-scale Cu-enriched precipitates having a core-shell structure were identified and qualitatively described by PCA. Advantages, disadvantages, and future prospects for implementing these data analysis methodologies for APT datasets, particularly with regard to quantitative analysis, are also discussed.

Using MetaboAnalyst 3.0 for Comprehensive Metabolomics Data Analysis.

Science.gov (United States)

Xia, Jianguo; Wishart, David S

2016-09-07

MetaboAnalyst (http://www.metaboanalyst.ca) is a comprehensive Web application for metabolomic data analysis and interpretation. MetaboAnalyst handles most of the common metabolomic data types from most kinds of metabolomics platforms (MS and NMR) for most kinds of metabolomics experiments (targeted, untargeted, quantitative). In addition to providing a variety of data processing and normalization procedures, MetaboAnalyst also supports a number of data analysis and data visualization tasks using a range of univariate, multivariate methods such as PCA (principal component analysis), PLS-DA (partial least squares discriminant analysis), heatmap clustering and machine learning methods. MetaboAnalyst also offers a variety of tools for metabolomic data interpretation including MSEA (metabolite set enrichment analysis), MetPA (metabolite pathway analysis), and biomarker selection via ROC (receiver operating characteristic) curve analysis, as well as time series and power analysis. This unit provides an overview of the main functional modules and the general workflow of the latest version of MetaboAnalyst (MetaboAnalyst 3.0), followed by eight detailed protocols. © 2016 by John Wiley & Sons, Inc. Copyright © 2016 John Wiley & Sons, Inc.

Data Analysis with Open Source Tools

CERN Document Server

Janert, Philipp

2010-01-01

Collecting data is relatively easy, but turning raw information into something useful requires that you know how to extract precisely what you need. With this insightful book, intermediate to experienced programmers interested in data analysis will learn techniques for working with data in a business environment. You'll learn how to look at data to discover what it contains, how to capture those ideas in conceptual models, and then feed your understanding back into the organization through business plans, metrics dashboards, and other applications. Along the way, you'll experiment with conce

Qualitative data analysis a methods sourcebook

CERN Document Server

Miles, Matthew B; Saldana, Johnny

2014-01-01

The Third Edition of Miles & Huberman's classic research methods text is updated and streamlined by Johnny SaldaNa, author of The Coding Manual for Qualitative Researchers. Several of the data display strategies from previous editions are now presented in re-envisioned and reorganized formats to enhance reader accessibility and comprehension. The Third Edition's presentation of the fundamentals of research design and data management is followed by five distinct methods of analysis: exploring, describing, ordering, explaining, and predicting. Miles and Huberman's original research studies are profiled and accompanied with new examples from SaldaNa's recent qualitative work. The book's most celebrated chapter, "Drawing and Verifying Conclusions," is retained and revised, and the chapter on report writing has been greatly expanded, and is now called "Writing About Qualitative Research." Comprehensive and authoritative, Qualitative Data Analysis has been elegantly revised for a new generation of qualitative r...

Statistics and analysis of scientific data

CERN Document Server

Bonamente, Massimiliano

2017-01-01

The revised second edition of this textbook provides the reader with a solid foundation in probability theory and statistics as applied to the physical sciences, engineering and related fields. It covers a broad range of numerical and analytical methods that are essential for the correct analysis of scientific data, including probability theory, distribution functions of statistics, fits to two-dimensional data and parameter estimation, Monte Carlo methods and Markov chains. Features new to this edition include: • a discussion of statistical techniques employed in business science, such as multiple regression analysis of multivariate datasets. • a new chapter on the various measures of the mean including logarithmic averages. • new chapters on systematic errors and intrinsic scatter, and on the fitting of data with bivariate errors. • a new case study and additional worked examples. • mathematical derivations and theoretical background material have been appropriately marked,to improve the readabili...

Essentials of multivariate data analysis

CERN Document Server

Spencer, Neil H

2013-01-01

""… this text provides an overview at an introductory level of several methods in multivariate data analysis. It contains in-depth examples from one data set woven throughout the text, and a free [Excel] Add-In to perform the analyses in Excel, with step-by-step instructions provided for each technique. … could be used as a text (possibly supplemental) for courses in other fields where researchers wish to apply these methods without delving too deeply into the underlying statistics.""-The American Statistician, February 2015

Analysis of Functional Data with Focus on Multinomial Regression and Multilevel Data

DEFF Research Database (Denmark)

Mousavi, Seyed Nourollah

Functional data analysis (FDA) is a fast growing area in statistical research with increasingly diverse range of application from economics, medicine, agriculture, chemometrics, etc. Functional regression is an area of FDA which has received the most attention both in aspects of application...... and methodological development. Our main Functional data analysis (FDA) is a fast growing area in statistical research with increasingly diverse range of application from economics, medicine, agriculture, chemometrics, etc. Functional regression is an area of FDA which has received the most attention both in aspects...

Language workbench user interfaces for data analysis

Directory of Open Access Journals (Sweden)

Victoria M. Benson

2015-02-01

Full Text Available Biological data analysis is frequently performed with command line software. While this practice provides considerable flexibility for computationally savy individuals, such as investigators trained in bioinformatics, this also creates a barrier to the widespread use of data analysis software by investigators trained as biologists and/or clinicians. Workflow systems such as Galaxy and Taverna have been developed to try and provide generic user interfaces that can wrap command line analysis software. These solutions are useful for problems that can be solved with workflows, and that do not require specialized user interfaces. However, some types of analyses can benefit from custom user interfaces. For instance, developing biomarker models from high-throughput data is a type of analysis that can be expressed more succinctly with specialized user interfaces. Here, we show how Language Workbench (LW technology can be used to model the biomarker development and validation process. We developed a language that models the concepts of Dataset, Endpoint, Feature Selection Method and Classifier. These high-level language concepts map directly to abstractions that analysts who develop biomarker models are familiar with. We found that user interfaces developed in the Meta-Programming System (MPS LW provide convenient means to configure a biomarker development project, to train models and view the validation statistics. We discuss several advantages of developing user interfaces for data analysis with a LW, including increased interface consistency, portability and extension by language composition. The language developed during this experiment is distributed as an MPS plugin (available at http://campagnelab.org/software/bdval-for-mps/.

Language workbench user interfaces for data analysis

Science.gov (United States)

Benson, Victoria M.

2015-01-01

Biological data analysis is frequently performed with command line software. While this practice provides considerable flexibility for computationally savy individuals, such as investigators trained in bioinformatics, this also creates a barrier to the widespread use of data analysis software by investigators trained as biologists and/or clinicians. Workflow systems such as Galaxy and Taverna have been developed to try and provide generic user interfaces that can wrap command line analysis software. These solutions are useful for problems that can be solved with workflows, and that do not require specialized user interfaces. However, some types of analyses can benefit from custom user interfaces. For instance, developing biomarker models from high-throughput data is a type of analysis that can be expressed more succinctly with specialized user interfaces. Here, we show how Language Workbench (LW) technology can be used to model the biomarker development and validation process. We developed a language that models the concepts of Dataset, Endpoint, Feature Selection Method and Classifier. These high-level language concepts map directly to abstractions that analysts who develop biomarker models are familiar with. We found that user interfaces developed in the Meta-Programming System (MPS) LW provide convenient means to configure a biomarker development project, to train models and view the validation statistics. We discuss several advantages of developing user interfaces for data analysis with a LW, including increased interface consistency, portability and extension by language composition. The language developed during this experiment is distributed as an MPS plugin (available at http://campagnelab.org/software/bdval-for-mps/). PMID:25755929

Federal metering data analysis needs and existing tools

Energy Technology Data Exchange (ETDEWEB)

Henderson, Jordan W. [Pacific Northwest National Lab. (PNNL), Richland, WA (United States); Fowler, Kimberly M. [Pacific Northwest National Lab. (PNNL), Richland, WA (United States)

2015-07-01

Agencies have been working to improve their metering data collection, management, and analysis efforts over the last decade (since EPAct 2005) and will continue to address these challenges as new requirements and data needs come into place. Unfortunately there is no “one-size-fits-all” solution. As agencies continue to expand their capabilities to use metered consumption data to reducing resource use and improve operations, the hope is that shared knowledge will empower others to follow suit. This paper discusses the Federal metering data analysis needs and some existing tools.

User analysis of LHCb data with Ganga

CERN Document Server

Maier, A; Cowan, G; Egede, U; Elmsheuser, J; Gaidioz, B; Harrison, K; Lee, H -C; Liko, D; Moscicki, J; Muraru, A; Pajchel, K; Reece, W; Samset, B; Slater, M; Soroko, A; van der Ster, D; Williams, M; Tan, C L

2010-01-01

GANGA (http://cern.ch/ganga) is a job-management tool that offers a simple, efficient and consistent user analysis tool in a variety of heterogeneous environments: from local clusters to global Grid systems. Experiment specific plug-ins allow GANGA to be customised for each experiment. For LHCb users GANGA is the officially supported and advertised tool for job submission to the Grid. The LHCb specific plug-ins allow support for end-to-end analysis helping the user to perform his complete analysis with the help of GANGA. This starts with the support for data selection, where a user can select data sets from the LHCb Bookkeeping system. Next comes the set up for large analysis jobs: with tailored plug-ins for the LHCb core software, jobs can be managed by the splitting of these analysis jobs with the subsequent merging of the resulting files. Furthermore, GANGA offers support for Toy Monte-Carlos to help the user tune their analysis. In addition to describing the GANGA architecture, typical usage patterns with...

User analysis of LHCb data with Ganga

International Nuclear Information System (INIS)

Maier, Andrew; Gaidioz, Benjamin; Moscicki, Jakub; Muraru, Adrian; Ster, Daniel van der; Brochu, Frederic; Cowan, Greg; Egede, Ulrik; Reece, Will; Williams, Mike; Elmsheuser, Johannes; Harrison, Karl; Slater, Mark; Tan, Chun Lik; Lee, Hurng-Chun; Liko, Dietrich; Pajchel, Katarina; Samset, Bjoern; Soroko, Alexander

2010-01-01

GANGA (http://cern.ch/ganga) is a job-management tool that offers a simple, efficient and consistent user analysis tool in a variety of heterogeneous environments: from local clusters to global Grid systems. Experiment specific plug-ins allow GANGA to be customised for each experiment. For LHCb users GANGA is the officially supported and advertised tool for job submission to the Grid. The LHCb specific plug-ins allow support for end-to-end analysis helping the user to perform his complete analysis with the help of GANGA. This starts with the support for data selection, where a user can select data sets from the LHCb Bookkeeping system. Next comes the set up for large analysis jobs: with tailored plug-ins for the LHCb core software, jobs can be managed by the splitting of these analysis jobs with the subsequent merging of the resulting files. Furthermore, GANGA offers support for Toy Monte-Carlos to help the user tune their analysis. In addition to describing the GANGA architecture, typical usage patterns within LHCb and experience with the updated LHCb DIRAC workload management system are presented.

RECOG-ORNL, Pattern Recognition Data Analysis

International Nuclear Information System (INIS)

Begovich, C.L.; Larson, N.M.

2000-01-01

Description of program or function: RECOG-ORNL, a general-purpose pattern recognition code, is a modification of the RECOG program, written at Lawrence Livermore National Laboratory. RECOG-ORNL contains techniques for preprocessing, analyzing, and displaying data, and for unsupervised and supervised learning. Data preprocessing routines transform the data into useful representations by auto-calling, selecting important variables, and/or adding products or transformations of the variables of the data set. Data analysis routines use correlations to evaluate the data and interrelationships among the data. Display routines plot the multidimensional patterns in two dimensions or plot histograms, patterns, or one variable versus another. Unsupervised learning techniques search for classes contained inherently in the data. Supervised learning techniques use known information about some of the data to generate predicted properties for an unknown set

Integrating Data Clustering and Visualization for the Analysis of 3D Gene Expression Data

Energy Technology Data Exchange (ETDEWEB)

Data Analysis and Visualization (IDAV) and the Department of Computer Science, University of California, Davis, One Shields Avenue, Davis CA 95616, USA,; nternational Research Training Group ``Visualization of Large and Unstructured Data Sets,' ' University of Kaiserslautern, Germany; Computational Research Division, Lawrence Berkeley National Laboratory, One Cyclotron Road, Berkeley, CA 94720, USA; Genomics Division, Lawrence Berkeley National Laboratory, One Cyclotron Road, Berkeley CA 94720, USA; Life Sciences Division, Lawrence Berkeley National Laboratory, One Cyclotron Road, Berkeley CA 94720, USA,; Computer Science Division,University of California, Berkeley, CA, USA,; Computer Science Department, University of California, Irvine, CA, USA,; All authors are with the Berkeley Drosophila Transcription Network Project, Lawrence Berkeley National Laboratory,; Rubel, Oliver; Weber, Gunther H.; Huang, Min-Yu; Bethel, E. Wes; Biggin, Mark D.; Fowlkes, Charless C.; Hendriks, Cris L. Luengo; Keranen, Soile V. E.; Eisen, Michael B.; Knowles, David W.; Malik, Jitendra; Hagen, Hans; Hamann, Bernd

2008-05-12

The recent development of methods for extracting precise measurements of spatial gene expression patterns from three-dimensional (3D) image data opens the way for new analyses of the complex gene regulatory networks controlling animal development. We present an integrated visualization and analysis framework that supports user-guided data clustering to aid exploration of these new complex datasets. The interplay of data visualization and clustering-based data classification leads to improved visualization and enables a more detailed analysis than previously possible. We discuss (i) integration of data clustering and visualization into one framework; (ii) application of data clustering to 3D gene expression data; (iii) evaluation of the number of clusters k in the context of 3D gene expression clustering; and (iv) improvement of overall analysis quality via dedicated post-processing of clustering results based on visualization. We discuss the use of this framework to objectively define spatial pattern boundaries and temporal profiles of genes and to analyze how mRNA patterns are controlled by their regulatory transcription factors.

Analysis of CERN computing infrastructure and monitoring data

Science.gov (United States)

Nieke, C.; Lassnig, M.; Menichetti, L.; Motesnitsalis, E.; Duellmann, D.

2015-12-01

Optimizing a computing infrastructure on the scale of LHC requires a quantitative understanding of a complex network of many different resources and services. For this purpose the CERN IT department and the LHC experiments are collecting a large multitude of logs and performance probes, which are already successfully used for short-term analysis (e.g. operational dashboards) within each group. The IT analytics working group has been created with the goal to bring data sources from different services and on different abstraction levels together and to implement a suitable infrastructure for mid- to long-term statistical analysis. It further provides a forum for joint optimization across single service boundaries and the exchange of analysis methods and tools. To simplify access to the collected data, we implemented an automated repository for cleaned and aggregated data sources based on the Hadoop ecosystem. This contribution describes some of the challenges encountered, such as dealing with heterogeneous data formats, selecting an efficient storage format for map reduce and external access, and will describe the repository user interface. Using this infrastructure we were able to quantitatively analyze the relationship between CPU/wall fraction, latency/throughput constraints of network and disk and the effective job throughput. In this contribution we will first describe the design of the shared analysis infrastructure and then present a summary of first analysis results from the combined data sources.

Analysis of ChIP-seq Data in R/Bioconductor.

Science.gov (United States)

de Santiago, Ines; Carroll, Thomas

2018-01-01

The development of novel high-throughput sequencing methods for ChIP (chromatin immunoprecipitation) has provided a very powerful tool to study gene regulation in multiple conditions at unprecedented resolution and scale. Proactive quality-control and appropriate data analysis techniques are of critical importance to extract the most meaningful results from the data. Over the last years, an array of R/Bioconductor tools has been developed allowing researchers to process and analyze ChIP-seq data. This chapter provides an overview of the methods available to analyze ChIP-seq data based primarily on software packages from the open-source Bioconductor project. Protocols described in this chapter cover basic steps including data alignment, peak calling, quality control and data visualization, as well as more complex methods such as the identification of differentially bound regions and functional analyses to annotate regulatory regions. The steps in the data analysis process were demonstrated on publicly available data sets and will serve as a demonstration of the computational procedures routinely used for the analysis of ChIP-seq data in R/Bioconductor, from which readers can construct their own analysis pipelines.

Big and complex data analysis methodologies and applications

CERN Document Server

2017-01-01

This volume conveys some of the surprises, puzzles and success stories in high-dimensional and complex data analysis and related fields. Its peer-reviewed contributions showcase recent advances in variable selection, estimation and prediction strategies for a host of useful models, as well as essential new developments in the field. The continued and rapid advancement of modern technology now allows scientists to collect data of increasingly unprecedented size and complexity. Examples include epigenomic data, genomic data, proteomic data, high-resolution image data, high-frequency financial data, functional and longitudinal data, and network data. Simultaneous variable selection and estimation is one of the key statistical problems involved in analyzing such big and complex data. The purpose of this book is to stimulate research and foster interaction between researchers in the area of high-dimensional data analysis. More concretely, its goals are to: 1) highlight and expand the breadth of existing methods in...

Data analysis & probability drill sheets : grades 6-8

CERN Document Server

Forest, Chris

2011-01-01

For grades 6-8, our Common Core State Standards-based resource meets the data analysis & probability concepts addressed by the NCTM standards and encourages your students to review the concepts in unique ways. Each drill sheet contains warm-up and timed drill activities for the student to practice data analysis & probability concepts.

Integrating Data Transformation in Principal Components Analysis

KAUST Repository

Maadooliat, Mehdi; Huang, Jianhua Z.; Hu, Jianhua

2015-01-01

Principal component analysis (PCA) is a popular dimension reduction method to reduce the complexity and obtain the informative aspects of high-dimensional datasets. When the data distribution is skewed, data transformation is commonly used prior

Data Collection, Collaboration, Analysis, and Publication Using the Open Data Repository's (ODR) Data Publisher

Science.gov (United States)

Lafuente, B.; Stone, N.; Bristow, T.; Keller, R. M.; Blake, D. F.; Downs, R. T.; Pires, A.; Dateo, C. E.; Fonda, M.

2017-12-01

In development for nearly four years, the Open Data Repository's (ODR) Data Publisher software has become a useful tool for researchers' data needs. Data Publisher facilitates the creation of customized databases with flexible permission sets that allow researchers to share data collaboratively while improving data discovery and maintaining ownership rights. The open source software provides an end-to-end solution from collection to final repository publication. A web-based interface allows researchers to enter data, view data, and conduct analysis using any programming language supported by JupyterHub (http://www.jupyterhub.org). This toolset makes it possible for a researcher to store and manipulate their data in the cloud from any internet capable device. Data can be embargoed in the system until a date selected by the researcher. For instance, open publication can be set to a date that coincides with publication of data analysis in a third party journal. In conjunction with teams at NASA Ames and the University of Arizona, a number of pilot studies are being conducted to guide the software development so that it allows them to publish and share their data. These pilots include (1) the Astrobiology Habitable Environments Database (AHED), a central searchable repository designed to promote and facilitate the integration and sharing of all the data generated by the diverse disciplines in astrobiology; (2) a database containing the raw and derived data products from the CheMin instrument on the MSL rover Curiosity (http://odr.io/CheMin), featuring a versatile graphing system, instructions and analytical tools to process the data, and a capability to download data in different formats; and (3) the Mineral Evolution project, which by correlating the diversity of mineral species with their ages, localities, and other measurable properties aims to understand how the episodes of planetary accretion and differentiation, plate tectonics, and origin of life lead to a

Data Farming Process and Initial Network Analysis Capabilities

Directory of Open Access Journals (Sweden)

Gary Horne

2016-01-01

Full Text Available Data Farming, network applications and approaches to integrate network analysis and processes to the data farming paradigm are presented as approaches to address complex system questions. Data Farming is a quantified approach that examines questions in large possibility spaces using modeling and simulation. It evaluates whole landscapes of outcomes to draw insights from outcome distributions and outliers. Social network analysis and graph theory are widely used techniques for the evaluation of social systems. Incorporation of these techniques into the data farming process provides analysts examining complex systems with a powerful new suite of tools for more fully exploring and understanding the effect of interactions in complex systems. The integration of network analysis with data farming techniques provides modelers with the capability to gain insight into the effect of network attributes, whether the network is explicitly defined or emergent, on the breadth of the model outcome space and the effect of model inputs on the resultant network statistics.

The Run 2 ATLAS Analysis Event Data Model

CERN Document Server

SNYDER, S; The ATLAS collaboration; NOWAK, M; EIFERT, T; BUCKLEY, A; ELSING, M; GILLBERG, D; MOYSE, E; KOENEKE, K; KRASZNAHORKAY, A

2014-01-01

During the LHC's first Long Shutdown (LS1) ATLAS set out to establish a new analysis model, based on the experience gained during Run 1. A key component of this is a new Event Data Model (EDM), called the xAOD. This format, which is now in production, provides the following features: A separation of the EDM into interface classes that the user code directly interacts with, and data storage classes that hold the payload data. The user sees an Array of Structs (AoS) interface, while the data is stored in a Struct of Arrays (SoA) format in memory, thus making it possible to efficiently auto-vectorise reconstruction code. A simple way of augmenting and reducing the information saved for different data objects. This makes it possible to easily decorate objects with new properties during data analysis, and to remove properties that the analysis does not need. A persistent file format that can be explored directly with ROOT, either with or without loading any additional libraries. This allows fast interactive naviga...

CMS Analysis and Data Reduction with Apache Spark

OpenAIRE

Gutsche, Oliver; Canali, Luca; Cremer, Illia; Cremonesi, Matteo; Elmer, Peter; Fisk, Ian; Girone, Maria; Jayatilaka, Bo; Kowalkowski, Jim; Khristenko, Viktor; Motesnitsalis, Evangelos; Pivarski, Jim; Sehrish, Saba; Surdy, Kacper; Svyatkovskiy, Alexey

2017-01-01

Experimental Particle Physics has been at the forefront of analyzing the world's largest datasets for decades. The HEP community was among the first to develop suitable software and computing tools for this task. In recent times, new toolkits and systems for distributed data processing, collectively called "Big Data" technologies have emerged from industry and open source projects to support the analysis of Petabyte and Exabyte datasets in industry. While the principles of data analysis in HE...

Oasis: online analysis of small RNA deep sequencing data.

Science.gov (United States)

Capece, Vincenzo; Garcia Vizcaino, Julio C; Vidal, Ramon; Rahman, Raza-Ur; Pena Centeno, Tonatiuh; Shomroni, Orr; Suberviola, Irantzu; Fischer, Andre; Bonn, Stefan

2015-07-01

Oasis is a web application that allows for the fast and flexible online analysis of small-RNA-seq (sRNA-seq) data. It was designed for the end user in the lab, providing an easy-to-use web frontend including video tutorials, demo data and best practice step-by-step guidelines on how to analyze sRNA-seq data. Oasis' exclusive selling points are a differential expression module that allows for the multivariate analysis of samples, a classification module for robust biomarker detection and an advanced programming interface that supports the batch submission of jobs. Both modules include the analysis of novel miRNAs, miRNA targets and functional analyses including GO and pathway enrichment. Oasis generates downloadable interactive web reports for easy visualization, exploration and analysis of data on a local system. Finally, Oasis' modular workflow enables for the rapid (re-) analysis of data. Oasis is implemented in Python, R, Java, PHP, C++ and JavaScript. It is freely available at http://oasis.dzne.de. stefan.bonn@dzne.de Supplementary data are available at Bioinformatics online. © The Author 2015. Published by Oxford University Press.

[The meta-analysis of data from individual patients].

NARCIS (Netherlands)

Rovers, M.M.; Reitsma, J.B.

2012-01-01

- An IPD (Individual Participant Data) meta-analysis requires collecting original individual patient data and calculating an estimated effect based on these data.- The use of individual patient data has various advantages: the original data and the results of published analyses are verified,

National survey on dose data analysis in computed tomography.

Science.gov (United States)

Heilmaier, Christina; Treier, Reto; Merkle, Elmar Max; Alkhadi, Hatem; Weishaupt, Dominik; Schindera, Sebastian

2018-05-28

A nationwide survey was performed assessing current practice of dose data analysis in computed tomography (CT). All radiological departments in Switzerland were asked to participate in the on-line survey composed of 19 questions (16 multiple choice, 3 free text). It consisted of four sections: (1) general information on the department, (2) dose data analysis, (3) use of a dose management software (DMS) and (4) radiation protection activities. In total, 152 out of 241 Swiss radiological departments filled in the whole questionnaire (return rate, 63%). Seventy-nine per cent of the departments (n = 120/152) analyse dose data on a regular basis with considerable heterogeneity in the frequency (1-2 times per year, 45%, n = 54/120; every month, 35%, n = 42/120) and method of analysis. Manual analysis is carried out by 58% (n = 70/120) compared with 42% (n = 50/120) of departments using a DMS. Purchase of a DMS is planned by 43% (n = 30/70) of the departments with manual analysis. Real-time analysis of dose data is performed by 42% (n = 21/50) of the departments with a DMS; however, residents can access the DMS in clinical routine only in 20% (n = 10/50) of the departments. An interdisciplinary dose team, which among other things communicates dose data internally (63%, n = 76/120) and externally, is already implemented in 57% (n = 68/120) departments. Swiss radiological departments are committed to radiation safety. However, there is high heterogeneity among them regarding the frequency and method of dose data analysis as well as the use of DMS and radiation protection activities. • Swiss radiological departments are committed to and interest in radiation safety as proven by a 63% return rate of the survey. • Seventy-nine per cent of departments analyse dose data on a regular basis with differences in the frequency and method of analysis: 42% use a dose management software, while 58% currently perform manual dose data analysis. Of the latter, 43% plan to buy a dose

Statistics and analysis of scientific data

CERN Document Server

Bonamente, Massimiliano

2013-01-01

Statistics and Analysis of Scientific Data covers the foundations of probability theory and statistics, and a number of numerical and analytical methods that are essential for the present-day analyst of scientific data. Topics covered include probability theory, distribution functions of statistics, fits to two-dimensional datasheets and parameter estimation, Monte Carlo methods and Markov chains. Equal attention is paid to the theory and its practical application, and results from classic experiments in various fields are used to illustrate the importance of statistics in the analysis of scientific data. The main pedagogical method is a theory-then-application approach, where emphasis is placed first on a sound understanding of the underlying theory of a topic, which becomes the basis for an efficient and proactive use of the material for practical applications. The level is appropriate for undergraduates and beginning graduate students, and as a reference for the experienced researcher. Basic calculus is us...

Secondary Data Analysis in Family Research

Science.gov (United States)

Hofferth, Sandra L.

2005-01-01

This article first provides an overview of the part that secondary data analysis plays in the field of family studies in the early 21st century. It addresses changes over time in the use of existing omnibus data sets and discusses their advantages and disadvantages. The second part of the article focuses on the elements that make a study a…

QUALITATIVE DATA AND ERROR MEASUREMENT IN INPUT-OUTPUT-ANALYSIS

NARCIS (Netherlands)

NIJKAMP, P; OOSTERHAVEN, J; OUWERSLOOT, H; RIETVELD, P

1992-01-01

This paper is a contribution to the rapidly emerging field of qualitative data analysis in economics. Ordinal data techniques and error measurement in input-output analysis are here combined in order to test the reliability of a low level of measurement and precision of data by means of a stochastic

Plasma data analysis using statistical analysis system

International Nuclear Information System (INIS)

Yoshida, Z.; Iwata, Y.; Fukuda, Y.; Inoue, N.

1987-01-01

Multivariate factor analysis has been applied to a plasma data base of REPUTE-1. The characteristics of the reverse field pinch plasma in REPUTE-1 are shown to be explained by four independent parameters which are described in the report. The well known scaling laws F/sub chi/ proportional to I/sub p/, T/sub e/ proportional to I/sub p/, and tau/sub E/ proportional to N/sub e/ are also confirmed. 4 refs., 8 figs., 1 tab

SEURAT: visual analytics for the integrated analysis of microarray data.

Science.gov (United States)

Gribov, Alexander; Sill, Martin; Lück, Sonja; Rücker, Frank; Döhner, Konstanze; Bullinger, Lars; Benner, Axel; Unwin, Antony

2010-06-03

In translational cancer research, gene expression data is collected together with clinical data and genomic data arising from other chip based high throughput technologies. Software tools for the joint analysis of such high dimensional data sets together with clinical data are required. We have developed an open source software tool which provides interactive visualization capability for the integrated analysis of high-dimensional gene expression data together with associated clinical data, array CGH data and SNP array data. The different data types are organized by a comprehensive data manager. Interactive tools are provided for all graphics: heatmaps, dendrograms, barcharts, histograms, eventcharts and a chromosome browser, which displays genetic variations along the genome. All graphics are dynamic and fully linked so that any object selected in a graphic will be highlighted in all other graphics. For exploratory data analysis the software provides unsupervised data analytics like clustering, seriation algorithms and biclustering algorithms. The SEURAT software meets the growing needs of researchers to perform joint analysis of gene expression, genomical and clinical data.

SEURAT: Visual analytics for the integrated analysis of microarray data

Directory of Open Access Journals (Sweden)

Bullinger Lars

2010-06-01

Full Text Available Abstract Background In translational cancer research, gene expression data is collected together with clinical data and genomic data arising from other chip based high throughput technologies. Software tools for the joint analysis of such high dimensional data sets together with clinical data are required. Results We have developed an open source software tool which provides interactive visualization capability for the integrated analysis of high-dimensional gene expression data together with associated clinical data, array CGH data and SNP array data. The different data types are organized by a comprehensive data manager. Interactive tools are provided for all graphics: heatmaps, dendrograms, barcharts, histograms, eventcharts and a chromosome browser, which displays genetic variations along the genome. All graphics are dynamic and fully linked so that any object selected in a graphic will be highlighted in all other graphics. For exploratory data analysis the software provides unsupervised data analytics like clustering, seriation algorithms and biclustering algorithms. Conclusions The SEURAT software meets the growing needs of researchers to perform joint analysis of gene expression, genomical and clinical data.

Application of Open Source Technologies for Oceanographic Data Analysis

Science.gov (United States)

Huang, T.; Gangl, M.; Quach, N. T.; Wilson, B. D.; Chang, G.; Armstrong, E. M.; Chin, T. M.; Greguska, F.

2015-12-01

NEXUS is a data-intensive analysis solution developed with a new approach for handling science data that enables large-scale data analysis by leveraging open source technologies such as Apache Cassandra, Apache Spark, Apache Solr, and Webification. NEXUS has been selected to provide on-the-fly time-series and histogram generation for the Soil Moisture Active Passive (SMAP) mission for Level 2 and Level 3 Active, Passive, and Active Passive products. It also provides an on-the-fly data subsetting capability. NEXUS is designed to scale horizontally, enabling it to handle massive amounts of data in parallel. It takes a new approach on managing time and geo-referenced array data by dividing data artifacts into chunks and stores them in an industry-standard, horizontally scaled NoSQL database. This approach enables the development of scalable data analysis services that can infuse and leverage the elastic computing infrastructure of the Cloud. It is equipped with a high-performance geospatial and indexed data search solution, coupled with a high-performance data Webification solution free from file I/O bottlenecks, as well as a high-performance, in-memory data analysis engine. In this talk, we will focus on the recently funded AIST 2014 project by using NEXUS as the core for oceanographic anomaly detection service and web portal. We call it, OceanXtremes

Opening up to Big Data: Computer-Assisted Analysis of Textual Data in Social Sciences

Directory of Open Access Journals (Sweden)

Gregor Wiedemann

2013-05-01

Full Text Available Two developments in computational text analysis may change the way qualitative data analysis in social sciences is performed: 1. the availability of digital text worth to investigate is growing rapidly, and 2. the improvement of algorithmic information extraction approaches, also called text mining, allows for further bridging the gap between qualitative and quantitative text analysis. The key factor hereby is the inclusion of context into computational linguistic models which extends conventional computational content analysis towards the extraction of meaning. To clarify methodological differences of various computer-assisted text analysis approaches the article suggests a typology from the perspective of a qualitative researcher. This typology shows compatibilities between manual qualitative data analysis methods and computational, rather quantitative approaches for large scale mixed method text analysis designs. URN: http://nbn-resolving.de/urn:nbn:de:0114-fqs1302231

Data analysis through interactive computer animation method (DATICAM)

International Nuclear Information System (INIS)

Curtis, J.N.; Schwieder, D.H.

1983-01-01

DATICAM is an interactive computer animation method designed to aid in the analysis of nuclear research data. DATICAM was developed at the Idaho National Engineering Laboratory (INEL) by EG and G Idaho, Inc. INEL analysts use DATICAM to produce computer codes that are better able to predict the behavior of nuclear power reactors. In addition to increased code accuracy, DATICAM has saved manpower and computer costs. DATICAM has been generalized to assist in the data analysis of virtually any data-producing dynamic process

Visualization and Data Analysis for High-Performance Computing

Energy Technology Data Exchange (ETDEWEB)

Sewell, Christopher Meyer [Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

2016-09-27

This is a set of slides from a guest lecture for a class at the University of Texas, El Paso on visualization and data analysis for high-performance computing. The topics covered are the following: trends in high-performance computing; scientific visualization, such as OpenGL, ray tracing and volume rendering, VTK, and ParaView; data science at scale, such as in-situ visualization, image databases, distributed memory parallelism, shared memory parallelism, VTK-m, "big data", and then an analysis example.

Network-Based Visual Analysis of Tabular Data

Science.gov (United States)

Liu, Zhicheng

2012-01-01

Tabular data is pervasive in the form of spreadsheets and relational databases. Although tables often describe multivariate data without explicit network semantics, it may be advantageous to explore the data modeled as a graph or network for analysis. Even when a given table design conveys some static network semantics, analysts may want to look…

Analysis of metabolomic data: tools, current strategies and future challenges for omics data integration.

Science.gov (United States)

Cambiaghi, Alice; Ferrario, Manuela; Masseroli, Marco

2017-05-01

Metabolomics is a rapidly growing field consisting of the analysis of a large number of metabolites at a system scale. The two major goals of metabolomics are the identification of the metabolites characterizing each organism state and the measurement of their dynamics under different situations (e.g. pathological conditions, environmental factors). Knowledge about metabolites is crucial for the understanding of most cellular phenomena, but this information alone is not sufficient to gain a comprehensive view of all the biological processes involved. Integrated approaches combining metabolomics with transcriptomics and proteomics are thus required to obtain much deeper insights than any of these techniques alone. Although this information is available, multilevel integration of different 'omics' data is still a challenge. The handling, processing, analysis and integration of these data require specialized mathematical, statistical and bioinformatics tools, and several technical problems hampering a rapid progress in the field exist. Here, we review four main tools for number of users or provided features (MetaCoreTM, MetaboAnalyst, InCroMAP and 3Omics) out of the several available for metabolomic data analysis and integration with other 'omics' data, highlighting their strong and weak aspects; a number of related issues affecting data analysis and integration are also identified and discussed. Overall, we provide an objective description of how some of the main currently available software packages work, which may help the experimental practitioner in the choice of a robust pipeline for metabolomic data analysis and integration. © The Author 2016. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com.

Identification of noise in linear data sets by factor analysis

International Nuclear Information System (INIS)

Roscoe, B.A.; Hopke, Ph.K.

1982-01-01

A technique which has the ability to identify bad data points, after the data has been generated, is classical factor analysis. The ability of classical factor analysis to identify two different types of data errors make it ideally suited for scanning large data sets. Since the results yielded by factor analysis indicate correlations between parameters, one must know something about the nature of the data set and the analytical techniques used to obtain it to confidentially isolate errors. (author)

Best practice for analysis of shared clinical trial data

Directory of Open Access Journals (Sweden)

Sally Hollis

2016-07-01

Full Text Available Abstract Background Greater transparency, including sharing of patient-level data for further research, is an increasingly important topic for organisations who sponsor, fund and conduct clinical trials. This is a major paradigm shift with the aim of maximising the value of patient-level data from clinical trials for the benefit of future patients and society. We consider the analysis of shared clinical trial data in three broad categories: (1 reanalysis - further investigation of the efficacy and safety of the randomized intervention, (2 meta-analysis, and (3 supplemental analysis for a research question that is not directly assessing the randomized intervention. Discussion In order to support appropriate interpretation and limit the risk of misleading findings, analysis of shared clinical trial data should have a pre-specified analysis plan. However, it is not generally possible to limit bias and control multiplicity to the extent that is possible in the original trial design, conduct and analysis, and this should be acknowledged and taken into account when interpreting results. We highlight a number of areas where specific considerations arise in planning, conducting, interpreting and reporting analyses of shared clinical trial data. A key issue is that that these analyses essentially share many of the limitations of any post hoc analyses beyond the original specified analyses. The use of individual patient data in meta-analysis can provide increased precision and reduce bias. Supplemental analyses are subject to many of the same issues that arise in broader epidemiological analyses. Specific discussion topics are addressed within each of these areas. Summary Increased provision of patient-level data from industry and academic-led clinical trials for secondary research can benefit future patients and society. Responsible data sharing, including transparency of the research objectives, analysis plans and of the results will support appropriate

Application of Workflow Technology for Big Data Analysis Service

Directory of Open Access Journals (Sweden)

Bin Zhang

2018-04-01

Full Text Available This study presents a lightweight representational state transfer-based cloud workflow system to construct a big data intelligent software-as-a-service (SaaS platform. The system supports the dynamic construction and operation of an intelligent data analysis application, and realizes rapid development and flexible deployment of the business analysis process that can improve the interaction and response time of the process. The proposed system integrates offline-batch and online-streaming analysis models that allow users to conduct batch and streaming computing simultaneously. Users can rend cloud capabilities and customize a set of big data analysis applications in the form of workflow processes. This study elucidates the architecture and application modeling, customization, dynamic construction, and scheduling of a cloud workflow system. A chain workflow foundation mechanism is proposed to combine several analysis components into a chain component that can promote efficiency. Four practical application cases are provided to verify the analysis capability of the system. Experimental results show that the proposed system can support multiple users in accessing the system concurrently and effectively uses data analysis algorithms. The proposed SaaS workflow system has been used in network operators and has achieved good results.

Analysis strategies for high-resolution UHF-fMRI data.

Science.gov (United States)

Polimeni, Jonathan R; Renvall, Ville; Zaretskaya, Natalia; Fischl, Bruce

2018-03-01

Functional MRI (fMRI) benefits from both increased sensitivity and specificity with increasing magnetic field strength, making it a key application for Ultra-High Field (UHF) MRI scanners. Most UHF-fMRI studies utilize the dramatic increases in sensitivity and specificity to acquire high-resolution data reaching sub-millimeter scales, which enable new classes of experiments to probe the functional organization of the human brain. This review article surveys advanced data analysis strategies developed for high-resolution fMRI at UHF. These include strategies designed to mitigate distortion and artifacts associated with higher fields in ways that attempt to preserve spatial resolution of the fMRI data, as well as recently introduced analysis techniques that are enabled by these extremely high-resolution data. Particular focus is placed on anatomically-informed analyses, including cortical surface-based analysis, which are powerful techniques that can guide each step of the analysis from preprocessing to statistical analysis to interpretation and visualization. New intracortical analysis techniques for laminar and columnar fMRI are also reviewed and discussed. Prospects for single-subject individualized analyses are also presented and discussed. Altogether, there are both specific challenges and opportunities presented by UHF-fMRI, and the use of proper analysis strategies can help these valuable data reach their full potential. Copyright © 2017 Elsevier Inc. All rights reserved.

Data fusion qualitative sensitivity analysis

International Nuclear Information System (INIS)

Clayton, E.A.; Lewis, R.E.

1995-09-01

Pacific Northwest Laboratory was tasked with testing, debugging, and refining the Hanford Site data fusion workstation (DFW), with the assistance of Coleman Research Corporation (CRC), before delivering the DFW to the environmental restoration client at the Hanford Site. Data fusion is the mathematical combination (or fusion) of disparate data sets into a single interpretation. The data fusion software used in this study was developed by CRC. The data fusion software developed by CRC was initially demonstrated on a data set collected at the Hanford Site where three types of data were combined. These data were (1) seismic reflection, (2) seismic refraction, and (3) depth to geologic horizons. The fused results included a contour map of the top of a low-permeability horizon. This report discusses the results of a sensitivity analysis of data fusion software to variations in its input parameters. The data fusion software developed by CRC has a large number of input parameters that can be varied by the user and that influence the results of data fusion. Many of these parameters are defined as part of the earth model. The earth model is a series of 3-dimensional polynomials with horizontal spatial coordinates as the independent variables and either subsurface layer depth or values of various properties within these layers (e.g., compression wave velocity, resistivity) as the dependent variables

Frontier Assignment for Sensitivity Analysis of Data Envelopment Analysis

Science.gov (United States)

Naito, Akio; Aoki, Shingo; Tsuji, Hiroshi

To extend the sensitivity analysis capability for DEA (Data Envelopment Analysis), this paper proposes frontier assignment based DEA (FA-DEA). The basic idea of FA-DEA is to allow a decision maker to decide frontier intentionally while the traditional DEA and Super-DEA decide frontier computationally. The features of FA-DEA are as follows: (1) provides chances to exclude extra-influential DMU (Decision Making Unit) and finds extra-ordinal DMU, and (2) includes the function of the traditional DEA and Super-DEA so that it is able to deal with sensitivity analysis more flexibly. Simple numerical study has shown the effectiveness of the proposed FA-DEA and the difference from the traditional DEA.

Automating dChip: toward reproducible sharing of microarray data analysis

Directory of Open Access Journals (Sweden)

Li Cheng

2008-05-01

Full Text Available Abstract Background During the past decade, many software packages have been developed for analysis and visualization of various types of microarrays. We have developed and maintained the widely used dChip as a microarray analysis software package accessible to both biologist and data analysts. However, challenges arise when dChip users want to analyze large number of arrays automatically and share data analysis procedures and parameters. Improvement is also needed when the dChip user support team tries to identify the causes of reported analysis errors or bugs from users. Results We report here implementation and application of the dChip automation module. Through this module, dChip automation files can be created to include menu steps, parameters, and data viewpoints to run automatically. A data-packaging function allows convenient transfer from one user to another of the dChip software, microarray data, and analysis procedures, so that the second user can reproduce the entire analysis session of the first user. An analysis report file can also be generated during an automated run, including analysis logs, user comments, and viewpoint screenshots. Conclusion The dChip automation module is a step toward reproducible research, and it can prompt a more convenient and reproducible mechanism for sharing microarray software, data, and analysis procedures and results. Automation data packages can also be used as publication supplements. Similar automation mechanisms could be valuable to the research community if implemented in other genomics and bioinformatics software packages.

Dealing with poor data quality of OSINT data in fraud risk analysis

NARCIS (Netherlands)

van Keulen, Maurice

2015-01-01

Governmental organizations responsible for keeping certain types of fraud under control, often use data-driven methods for both immediate detection of fraud, or for fraud risk analysis aimed at more effectively targeting inspections. A blind spot in such methods, is that the source data often

Application of computer intensive data analysis methods to the analysis of digital images and spatial data

DEFF Research Database (Denmark)

Windfeld, Kristian

1992-01-01

Computer-intensive methods for data analysis in a traditional setting has developed rapidly in the last decade. The application of and adaption of some of these methods to the analysis of multivariate digital images and spatial data are explored, evaluated and compared to well established classical...... into the projection pursuit is presented. Examples from remote sensing are given. The ACE algorithm for computing non-linear transformations for maximizing correlation is extended and applied to obtain a non-linear transformation that maximizes autocorrelation or 'signal' in a multivariate image....... This is a generalization of the minimum /maximum autocorrelation factors (MAF's) which is a linear method. The non-linear method is compared to the linear method when analyzing a multivariate TM image from Greenland. The ACE method is shown to give a more detailed decomposition of the image than the MAF-transformation...

Interactive analysis of systems biology molecular expression data

Directory of Open Access Journals (Sweden)

Prabhakar Sunil

2008-02-01

Full Text Available Abstract Background Systems biology aims to understand biological systems on a comprehensive scale, such that the components that make up the whole are connected to one another and work through dependent interactions. Molecular correlations and comparative studies of molecular expression are crucial to establishing interdependent connections in systems biology. The existing software packages provide limited data mining capability. The user must first generate visualization data with a preferred data mining algorithm and then upload the resulting data into the visualization package for graphic visualization of molecular relations. Results Presented is a novel interactive visual data mining application, SysNet that provides an interactive environment for the analysis of high data volume molecular expression information of most any type from biological systems. It integrates interactive graphic visualization and statistical data mining into a single package. SysNet interactively presents intermolecular correlation information with circular and heatmap layouts. It is also applicable to comparative analysis of molecular expression data, such as time course data. Conclusion The SysNet program has been utilized to analyze elemental profile changes in response to an increasing concentration of iron (Fe in growth media (an ionomics dataset. This study case demonstrates that the SysNet software is an effective platform for interactive analysis of molecular expression information in systems biology.

airborne data analysis/monitor system

Science.gov (United States)

Stephison, D. B.

1981-01-01

An Airborne Data Analysis/Monitor System (ADAMS), a ROLM 1666 computer based system installed onboard test airplanes used during experimental testing is evaluated. In addition to the 1666 computer, the ADAMS hardware includes a DDC System 90 fixed head disk and a Miltape DD400 floppy disk. Boeing designed a DMA interface to the data acquisition system and an intelligent terminal to reduce system overhead and simplify operator commands. The ADAMS software includes RMX/RTOS and both ROLM FORTRAN and assembly language are used. The ADAMS provides real time displays that enable onboard test engineers to make rapid decisions about test conduct thus reducing the cost and time required to certify new model airplanes, and improved the quality of data derived from the test, leading to more rapid development of improvements resulting in quieter, safer, and more efficient airplanes. The availability of airborne data processing removes most of the weather and geographical restrictions imposed by telemetered flight test data systems. A data base is maintained to describe the airplane, the data acquisition system, the type of testing, and the conditions under which the test is performed.

Functional data analysis of sleeping energy expenditure

Science.gov (United States)

Adequate sleep is crucial during childhood for metabolic health, and physical and cognitive development. Inadequate sleep can disrupt metabolic homeostasis and alter sleeping energy expenditure (SEE). Functional data analysis methods were applied to SEE data to elucidate the population structure of ...

PIVOT: platform for interactive analysis and visualization of transcriptomics data.

Science.gov (United States)

Zhu, Qin; Fisher, Stephen A; Dueck, Hannah; Middleton, Sarah; Khaladkar, Mugdha; Kim, Junhyong

2018-01-05

Many R packages have been developed for transcriptome analysis but their use often requires familiarity with R and integrating results of different packages requires scripts to wrangle the datatypes. Furthermore, exploratory data analyses often generate multiple derived datasets such as data subsets or data transformations, which can be difficult to track. Here we present PIVOT, an R-based platform that wraps open source transcriptome analysis packages with a uniform user interface and graphical data management that allows non-programmers to interactively explore transcriptomics data. PIVOT supports more than 40 popular open source packages for transcriptome analysis and provides an extensive set of tools for statistical data manipulations. A graph-based visual interface is used to represent the links between derived datasets, allowing easy tracking of data versions. PIVOT further supports automatic report generation, publication-quality plots, and program/data state saving, such that all analysis can be saved, shared and reproduced. PIVOT will allow researchers with broad background to easily access sophisticated transcriptome analysis tools and interactively explore transcriptome datasets.

Advances in statistical models for data analysis

CERN Document Server

Minerva, Tommaso; Vichi, Maurizio

2015-01-01

This edited volume focuses on recent research results in classification, multivariate statistics and machine learning and highlights advances in statistical models for data analysis. The volume provides both methodological developments and contributions to a wide range of application areas such as economics, marketing, education, social sciences and environment. The papers in this volume were first presented at the 9th biannual meeting of the Classification and Data Analysis Group (CLADAG) of the Italian Statistical Society, held in September 2013 at the University of Modena and Reggio Emilia, Italy.

Lag profile inversion method for EISCAT data analysis

Directory of Open Access Journals (Sweden)

I. I. Virtanen

2008-03-01

Full Text Available The present standard EISCAT incoherent scatter experiments are based on alternating codes that are decoded in power domain by simple summation and subtraction operations. The signal is first digitised and then different lagged products are calculated and decoded in real time. Only the decoded lagged products are saved for further analysis so that both the original data samples and the undecoded lagged products are lost. A fit of plasma parameters can be later performed using the recorded lagged products. In this paper we describe a different analysis method, which makes use of statistical inversion in removing range ambiguities from the lag profiles. An analysis program carrying out both the lag profile inversion and the fit of the plasma parameters has been constructed. Because recording the received signal itself instead of the lagged products allows very flexible data analysis, the program is constructed to use raw data, i.e. IQ-sampled signal recorded from an IF stage of the radar. The program is now capable of analysing standard alternating-coded EISCAT experiments as well as experiments with any other kind of radar modulation if raw data is available. The program calculates the ambiguous lag profiles and is capable of inverting them as such but, for analysis in real time, time integration is needed before inversion. We demonstrate the method using alternating code experiments in the EISCAT UHF radar and specific hardware connected to the second IF stage of the receiver. This method produces a data stream of complex samples, which are stored for later processing. The raw data is analysed with lag profile inversion and the results are compared to those given by the standard method.

Meta-Analysis for Primary and Secondary Data Analysis: The Super-Experiment Metaphor.

Science.gov (United States)

Jackson, Sally

1991-01-01

Considers the relation between meta-analysis statistics and analysis of variance statistics. Discusses advantages and disadvantages as a primary data analysis tool. Argues that the two approaches are partial paraphrases of one another. Advocates an integrative approach that introduces the best of meta-analytic thinking into primary analysis…

Statistical methods for categorical data analysis

CERN Document Server

Powers, Daniel

2008-01-01

This book provides a comprehensive introduction to methods and models for categorical data analysis and their applications in social science research. Companion website also available, at https://webspace.utexas.edu/dpowers/www/

Basic statistical tools in research and data analysis

Directory of Open Access Journals (Sweden)

Zulfiqar Ali

2016-01-01

Full Text Available Statistical methods involved in carrying out a study include planning, designing, collecting data, analysing, drawing meaningful interpretation and reporting of the research findings. The statistical analysis gives meaning to the meaningless numbers, thereby breathing life into a lifeless data. The results and inferences are precise only if proper statistical tests are used. This article will try to acquaint the reader with the basic research tools that are utilised while conducting various studies. The article covers a brief outline of the variables, an understanding of quantitative and qualitative variables and the measures of central tendency. An idea of the sample size estimation, power analysis and the statistical errors is given. Finally, there is a summary of parametric and non-parametric tests used for data analysis.

[Quantitative data analysis for live imaging of bone.

Science.gov (United States)

Seno, Shigeto

Bone tissue is a hard tissue, it was difficult to observe the interior of the bone tissue alive. With the progress of microscopic technology and fluorescent probe technology in recent years, it becomes possible to observe various activities of various cells forming bone society. On the other hand, the quantitative increase in data and the diversification and complexity of the images makes it difficult to perform quantitative analysis by visual inspection. It has been expected to develop a methodology for processing microscopic images and data analysis. In this article, we introduce the research field of bioimage informatics which is the boundary area of biology and information science, and then outline the basic image processing technology for quantitative analysis of live imaging data of bone.

User's manual of JT-60 experimental data analysis system

International Nuclear Information System (INIS)

Hirayama, Takashi; Morishima, Soichi; Yoshioka, Yuji

2010-02-01

In the Japan Atomic Energy Agency Naka Fusion Institute, a lot of experiments have been conducted by using the large tokamak device JT-60 aiming to realize fusion power plant. In order to optimize the JT-60 experiment and to investigate complex characteristics of plasma, JT-60 experimental data analysis system was developed and used for collecting, referring and analyzing the JT-60 experimental data. Main components of the system are a data analysis server and a database server for the analyses and accumulation of the experimental data respectively. Other peripheral devices of the system are magnetic disk units, NAS (Network Attached Storage) device, and a backup tape drive. This is a user's manual of the JT-60 experimental data analysis system. (author)

Rethinking Meta-Analysis: Applications for Air Pollution Data and Beyond

Science.gov (United States)

Goodman, Julie E; Petito Boyce, Catherine; Sax, Sonja N; Beyer, Leslie A; Prueitt, Robyn L

2015-01-01

Meta-analyses offer a rigorous and transparent systematic framework for synthesizing data that can be used for a wide range of research areas, study designs, and data types. Both the outcome of meta-analyses and the meta-analysis process itself can yield useful insights for answering scientific questions and making policy decisions. Development of the National Ambient Air Quality Standards illustrates many potential applications of meta-analysis. These applications demonstrate the strengths and limitations of meta-analysis, issues that arise in various data realms, how meta-analysis design choices can influence interpretation of results, and how meta-analysis can be used to address bias and heterogeneity. Reviewing available data from a meta-analysis perspective can provide a useful framework and impetus for identifying and refining strategies for future research. Moreover, increased pervasiveness of a meta-analysis mindset—focusing on how the pieces of the research puzzle fit together—would benefit scientific research and data syntheses regardless of whether or not a quantitative meta-analysis is undertaken. While an individual meta-analysis can only synthesize studies addressing the same research question, the results of separate meta-analyses can be combined to address a question encompassing multiple data types. This observation applies to any scientific or policy area where information from a variety of disciplines must be considered to address a broader research question. PMID:25969128

Integration, warehousing, and analysis strategies of Omics data.

Science.gov (United States)

Gedela, Srinubabu

2011-01-01

"-Omics" is a current suffix for numerous types of large-scale biological data generation procedures, which naturally demand the development of novel algorithms for data storage and analysis. With next generation genome sequencing burgeoning, it is pivotal to decipher a coding site on the genome, a gene's function, and information on transcripts next to the pure availability of sequence information. To explore a genome and downstream molecular processes, we need umpteen results at the various levels of cellular organization by utilizing different experimental designs, data analysis strategies and methodologies. Here comes the need for controlled vocabularies and data integration to annotate, store, and update the flow of experimental data. This chapter explores key methodologies to merge Omics data by semantic data carriers, discusses controlled vocabularies as eXtensible Markup Languages (XML), and provides practical guidance, databases, and software links supporting the integration of Omics data.

Status of data and data needs for XRF and PIXE based element analysis

International Nuclear Information System (INIS)

Kapoor, S.S.; Choudhary, R.K.

1986-01-01

The status of data and data needs for X-ray fluorescence (XRF) and particle induced X-ray (PIXE) are examined to determine the areas where additional and improved data are required to improve the accuracy, precision and sensitivity of quantitative element analysis by the above techniques. (author)

Astronomical Image and Data Analysis

CERN Document Server

Starck, J.-L

2006-01-01

With information and scale as central themes, this comprehensive survey explains how to handle real problems in astronomical data analysis using a modern arsenal of powerful techniques. It treats those innovative methods of image, signal, and data processing that are proving to be both effective and widely relevant. The authors are leaders in this rapidly developing field and draw upon decades of experience. They have been playing leading roles in international projects such as the Virtual Observatory and the Grid. The book addresses not only students and professional astronomers and astrophysicists, but also serious amateur astronomers and specialists in earth observation, medical imaging, and data mining. The coverage includes chapters or appendices on: detection and filtering; image compression; multichannel, multiscale, and catalog data analytical methods; wavelets transforms, Picard iteration, and software tools. This second edition of Starck and Murtagh's highly appreciated reference again deals with to...

Multi-Dimensional Customer Data Analysis in Online Auctions

Institute of Scientific and Technical Information of China (English)

LAO Guoling; XIONG Kuan; QIN Zheng

2007-01-01

In this paper, we designed a customer-centered data warehouse system with five subjects: listing, bidding, transaction,accounts, and customer contact based on the business process of online auction companies. For each subject, we analyzed its fact indexes and dimensions. Then take transaction subject as example,analyzed the data warehouse model in detail, and got the multi-dimensional analysis structure of transaction subject. At last, using data mining to do customer segmentation, we divided customers into four types: impulse customer, prudent customer, potential customer, and ordinary customer. By the result of multi-dimensional customer data analysis, online auction companies can do more target marketing and increase customer loyalty.

Data analysis for physical scientists featuring Excel

CERN Document Server

Kirkup, Les

2012-01-01

The ability to summarise data, compare models and apply computer-based analysis tools are vital skills necessary for studying and working in the physical sciences. This textbook supports undergraduate students as they develop and enhance these skills. Introducing data analysis techniques, this textbook pays particular attention to the internationally recognised guidelines for calculating and expressing measurement uncertainty. This new edition has been revised to incorporate Excel® 2010. It also provides a practical approach to fitting models to data using non-linear least squares, a powerful technique which can be applied to many types of model. Worked examples using actual experimental data help students understand how the calculations apply to real situations. Over 200 in-text exercises and end-of-chapter problems give students the opportunity to use the techniques themselves and gain confidence in applying them. Answers to the exercises and problems are given at the end of the book.

Data preparation for functional data analysis of PM10 in Peninsular Malaysia

Science.gov (United States)

Shaadan, Norshahida; Jemain, Abdul Aziz; Deni, Sayang Mohd

2014-07-01

The use of curves or functional data in the study analysis is increasingly gaining momentum in the various fields of research. The statistical method to analyze such data is known as functional data analysis (FDA). The first step in FDA is to convert the observed data points which are repeatedly recorded over a period of time or space into either a rough (raw) or smooth curve. In the case of the smooth curve, basis functions expansion is one of the methods used for the data conversion. The data can be converted into a smooth curve either by using the regression smoothing or roughness penalty smoothing approach. By using the regression smoothing approach, the degree of curve's smoothness is very dependent on k number of basis functions; meanwhile for the roughness penalty approach, the smoothness is dependent on a roughness coefficient given by parameter λ Based on previous studies, researchers often used the rather time-consuming trial and error or cross validation method to estimate the appropriate number of basis functions. Thus, this paper proposes a statistical procedure to construct functional data or curves for the hourly and daily recorded data. The Bayesian Information Criteria is used to determine the number of basis functions while the Generalized Cross Validation criteria is used to identify the parameter λ The proposed procedure is then applied on a ten year (2001-2010) period of PM10 data from 30 air quality monitoring stations that are located in Peninsular Malaysia. It was found that the number of basis functions required for the construction of the PM10 daily curve in Peninsular Malaysia was in the interval of between 14 and 20 with an average value of 17; the first percentile is 15 and the third percentile is 19. Meanwhile the initial value of the roughness coefficient was in the interval of between 10-5 and 10-7 and the mode was 10-6. An example of the functional descriptive analysis is also shown.

Compass 2011 data analysis and reporting.

Science.gov (United States)

2013-05-01

Past efforts include data analysis and reporting performance and outcomes for signs, pavement, shoulders, roadsides, drainage, traffic, and bridges. In : the 2005 Compass report, measures for bridge inspection and maintenance were added, and historic...

Compass 2012 data analysis and reporting.

Science.gov (United States)

2014-05-01

Past efforts include data analysis and reporting performance and outcomes for signs, pavement, shoulders, roadsides, drainage, traffic, and bridges. In : the 2005 Compass report, measures for bridge inspection and maintenance were added, and historic...

The EADGENE Microarray Data Analysis Workshop

DEFF Research Database (Denmark)

de Koning, Dirk-Jan; Jaffrézic, Florence; Lund, Mogens Sandø

2007-01-01

Microarray analyses have become an important tool in animal genomics. While their use is becoming widespread, there is still a lot of ongoing research regarding the analysis of microarray data. In the context of a European Network of Excellence, 31 researchers representing 14 research groups from...... 10 countries performed and discussed the statistical analyses of real and simulated 2-colour microarray data that were distributed among participants. The real data consisted of 48 microarrays from a disease challenge experiment in dairy cattle, while the simulated data consisted of 10 microarrays...... statistical weights, to omitting a large number of spots or omitting entire slides. Surprisingly, these very different approaches gave quite similar results when applied to the simulated data, although not all participating groups analysed both real and simulated data. The workshop was very successful...

Further Development of Rotating Rake Mode Measurement Data Analysis

Science.gov (United States)

Dahl, Milo D.; Hixon, Ray; Sutliff, Daniel L.

2013-01-01