Streaming nested data parallelism on multicores

DEFF Research Database (Denmark)

Madsen, Frederik Meisner; Filinski, Andrzej

2016-01-01

The paradigm of nested data parallelism (NDP) allows a variety of semi-regular computation tasks to be mapped onto SIMD-style hardware, including GPUs and vector units. However, some care is needed to keep down space consumption in situations where the available parallelism may vastly exceed...

MoviCompile : An LLVM based compiler for heterogeneous SIMD code generation

NARCIS (Netherlands)

Diken, E.; Jordans, R.; O'Riordan, M.

2015-01-01

Numerous applications in communication and multimedia domains show significant data-level parallelism (DLP). The amount of DLP varies between applications in the same domain or even within a single application. Most architectures support a single vector-, SIMD-width which may not be optimal. This

OpenCL code generation for low energy wide SIMD architectures with explicit datapath.

NARCIS (Netherlands)

She, D.; He, Y.; Waeijen, L.J.W.; Corporaal, H.; Jeschke, H.; Silvén, O.

2013-01-01

Energy efficiency is one of the most important aspects in designing embedded processors. The use of a wide SIMD processor architecture is a promising approach to build energy-efficient high performance embedded processors. In this paper, we propose a configurable wide SIMD architecture that utilizes

Filtering Redundant Data from RFID Data Streams

Directory of Open Access Journals (Sweden)

Hazalila Kamaludin

2016-01-01

Full Text Available Radio Frequency Identification (RFID enabled systems are evolving in many applications that need to know the physical location of objects such as supply chain management. Naturally, RFID systems create large volumes of duplicate data. As the duplicate data wastes communication, processing, and storage resources as well as delaying decision-making, filtering duplicate data from RFID data stream is an important and challenging problem. Existing Bloom Filter-based approaches for filtering duplicate RFID data streams are complex and slow as they use multiple hash functions. In this paper, we propose an approach for filtering duplicate data from RFID data streams. The proposed approach is based on modified Bloom Filter and uses only a single hash function. We performed extensive empirical study of the proposed approach and compared it against the Bloom Filter, d-Left Time Bloom Filter, and the Count Bloom Filter approaches. The results show that the proposed approach outperforms the baseline approaches in terms of false positive rate, execution time, and true positive rate.

Search of molecular ground state via genetic algorithm: Implementation on a hybrid SIMD-MIMD platform

International Nuclear Information System (INIS)

Pucello, N.; D'Agostino, G.; Pisacane, F.

1997-01-01

A genetic algorithm for the optimization of the ground-state structure of a metallic cluster has been developed and ported on a SIMD-MIMD parallel platform. The SIMD part of the parallel platform is represented by a Quadrics/APE100 consisting of 512 floating point units, while the MIMD part is formed by a cluster of workstations. The proposed algorithm is composed by a part where the genetic operators are applied to the elements of the population and a part which performs a further local relaxation and the fitness calculation via Molecular Dynamics. These parts have been implemented on the MIMD and on the SIMD part, respectively. Results have been compared to those generated by using Simulated Annealing

N-body simulation for self-gravitating collisional systems with a new SIMD instruction set extension to the x86 architecture, Advanced Vector eXtensions

Science.gov (United States)

Tanikawa, Ataru; Yoshikawa, Kohji; Okamoto, Takashi; Nitadori, Keigo

2012-02-01

We present a high-performance N-body code for self-gravitating collisional systems accelerated with the aid of a new SIMD instruction set extension of the x86 architecture: Advanced Vector eXtensions (AVX), an enhanced version of the Streaming SIMD Extensions (SSE). With one processor core of Intel Core i7-2600 processor (8 MB cache and 3.40 GHz) based on Sandy Bridge micro-architecture, we implemented a fourth-order Hermite scheme with individual timestep scheme ( Makino and Aarseth, 1992), and achieved the performance of ˜20 giga floating point number operations per second (GFLOPS) for double-precision accuracy, which is two times and five times higher than that of the previously developed code implemented with the SSE instructions ( Nitadori et al., 2006b), and that of a code implemented without any explicit use of SIMD instructions with the same processor core, respectively. We have parallelized the code by using so-called NINJA scheme ( Nitadori et al., 2006a), and achieved ˜90 GFLOPS for a system containing more than N = 8192 particles with 8 MPI processes on four cores. We expect to achieve about 10 tera FLOPS (TFLOPS) for a self-gravitating collisional system with N ˜ 10 5 on massively parallel systems with at most 800 cores with Sandy Bridge micro-architecture. This performance will be comparable to that of Graphic Processing Unit (GPU) cluster systems, such as the one with about 200 Tesla C1070 GPUs ( Spurzem et al., 2010). This paper offers an alternative to collisional N-body simulations with GRAPEs and GPUs.

Vectorization with SIMD extensions speeds up reconstruction in electron tomography.

Science.gov (United States)

Agulleiro, J I; Garzón, E M; García, I; Fernández, J J

2010-06-01

Electron tomography allows structural studies of cellular structures at molecular detail. Large 3D reconstructions are needed to meet the resolution requirements. The processing time to compute these large volumes may be considerable and so, high performance computing techniques have been used traditionally. This work presents a vector approach to tomographic reconstruction that relies on the exploitation of the SIMD extensions available in modern processors in combination to other single processor optimization techniques. This approach succeeds in producing full resolution tomograms with an important reduction in processing time, as evaluated with the most common reconstruction algorithms, namely WBP and SIRT. The main advantage stems from the fact that this approach is to be run on standard computers without the need of specialized hardware, which facilitates the development, use and management of programs. Future trends in processor design open excellent opportunities for vector processing with processor's SIMD extensions in the field of 3D electron microscopy.

Time-Based Data Streams: Fundamental Concepts for a Data Resource for Streams

Energy Technology Data Exchange (ETDEWEB)

Beth A. Plale

2009-10-10

Real time data, which we call data streams, are readings from instruments, environmental, bodily or building sensors that are generated at regular intervals and often, due to their volume, need to be processed in real time. Often a single pass is all that can be made on the data, and a decision to discard or keep the instance is made on the spot. Too, the stream is for all practical purposes indefinite, so decisions must be made on incomplete knowledge. This notion of data streams has a different set of issues from a file, for instance, that is byte streamed to a reader. The file is finite, so the byte stream is becomes a processing convenience more than a fundamentally different kind of data. Through the duration of the project we examined three aspects of streaming data: the first, techniques to handle streaming data in a distributed system organized as a collection of web services, the second, the notion of the dashboard and real time controllable analysis constructs in the context of the Fermi Tevatron Beam Position Monitor, and third and finally, we examined provenance collection of stream processing such as might occur as raw observational data flows from the source and undergoes correction, cleaning, and quality control. The impact of this work is severalfold. We were one of the first to advocate that streams had little value unless aggregated, and that notion is now gaining general acceptance. We were one of the first groups to grapple with the notion of provenance of stream data also.

Real time image synthesis on a SIMD linear array processor: algorithms and architectures

International Nuclear Information System (INIS)

Letellier, Laurent

1993-01-01

Nowadays, image synthesis has become a widely used technique. The impressive computing power required for real time applications necessitates the use of parallel architectures. In this context, we evaluate an SIMD linear parallel architecture, SYMPATI2, dedicated to image processing. The objective of this study is to propose a cost-effective graphics accelerator relying on SYMPATI2's modular and programmable structure. The parallelization of basic image synthesis algorithms on SYMPATI2 enables us to determine its limits in this application field. These limits lead us to evaluate a new structure with a fast intercommunication network between processors, but processors have to support the message consistency, which brings about a strong decrease in performance. To solve this problem, we suggest a simple network whose access priorities are represented by tokens. The simulations of this new architecture indicate that the SIMD mode causes a drastic cut in parallelism. To cope with this drawback, we propose a context switching procedure which reduces the SIMD rigidity and increases the parallelism rate significantly. Then, the graphics accelerator we propose is compared with existing graphics workstations. This comparison indicates that our structure, which is able to accelerate both image synthesis and image processing, is competitive and well-suited for multimedia applications. (author) [fr

Optimizing Cost of Continuous Overlapping Queries over Data Streams by Filter Adaption

KAUST Repository

Xie, Qing; Zhang, Xiangliang; Li, Zhixu; Zhou, Xiaofang

2016-01-01

The problem we aim to address is the optimization of cost management for executing multiple continuous queries on data streams, where each query is defined by several filters, each of which monitors certain status of the data stream. Specially

Some multigrid algorithms for SIMD machines

Energy Technology Data Exchange (ETDEWEB)

Dendy, J.E. Jr. [Los Alamos National Lab., NM (United States)

1996-12-31

Previously a semicoarsening multigrid algorithm suitable for use on SIMD architectures was investigated. Through the use of new software tools, the performance of this algorithm has been considerably improved. The method has also been extended to three space dimensions. The method performs well for strongly anisotropic problems and for problems with coefficients jumping by orders of magnitude across internal interfaces. The parallel efficiency of this method is analyzed, and its actual performance on the CM-5 is compared with its performance on the CRAY-YMP. A standard coarsening multigrid algorithm is also considered, and we compare its performance on these two platforms as well.

Use of multiple dispersal pathways facilitates amphibian persistence in stream networks

Science.gov (United States)

Campbell, Grant E.H.; Nichols, J.D.; Lowe, W.H.; Fagan, W.F.

2010-01-01

Although populations of amphibians are declining worldwide, there is no evidence that salamanders occupying small streams are experiencing enigmatic declines, and populations of these species seem stable. Theory predicts that dispersal through multiple pathways can stabilize populations, preventing extinction in habitat networks. However, empirical data to support this prediction are absent for most species, especially those at risk of decline. Our mark-recapture study of stream salamanders reveals both a strong upstream bias in dispersal and a surprisingly high rate of overland dispersal to adjacent headwater streams. This evidence of route-dependent variation in dispersal rates suggests a spatial mechanism for population stability in headwater-stream salamanders. Our results link the movement behavior of stream salamanders to network topology, and they underscore the importance of identifying and protecting critical dispersal pathways when addressing region-wide population declines.

Use of multiple dispersal pathways facilitates amphibian persistence in stream networks.

Science.gov (United States)

Campbell Grant, Evan H; Nichols, James D; Lowe, Winsor H; Fagan, William F

2010-04-13

Although populations of amphibians are declining worldwide, there is no evidence that salamanders occupying small streams are experiencing enigmatic declines, and populations of these species seem stable. Theory predicts that dispersal through multiple pathways can stabilize populations, preventing extinction in habitat networks. However, empirical data to support this prediction are absent for most species, especially those at risk of decline. Our mark-recapture study of stream salamanders reveals both a strong upstream bias in dispersal and a surprisingly high rate of overland dispersal to adjacent headwater streams. This evidence of route-dependent variation in dispersal rates suggests a spatial mechanism for population stability in headwater-stream salamanders. Our results link the movement behavior of stream salamanders to network topology, and they underscore the importance of identifying and protecting critical dispersal pathways when addressing region-wide population declines.

Flexbar 3.0 - SIMD and multicore parallelization.

Science.gov (United States)

Roehr, Johannes T; Dieterich, Christoph; Reinert, Knut

2017-09-15

High-throughput sequencing machines can process many samples in a single run. For Illumina systems, sequencing reads are barcoded with an additional DNA tag that is contained in the respective sequencing adapters. The recognition of barcode and adapter sequences is hence commonly needed for the analysis of next-generation sequencing data. Flexbar performs demultiplexing based on barcodes and adapter trimming for such data. The massive amounts of data generated on modern sequencing machines demand that this preprocessing is done as efficiently as possible. We present Flexbar 3.0, the successor of the popular program Flexbar. It employs now twofold parallelism: multi-threading and additionally SIMD vectorization. Both types of parallelism are used to speed-up the computation of pair-wise sequence alignments, which are used for the detection of barcodes and adapters. Furthermore, new features were included to cover a wide range of applications. We evaluated the performance of Flexbar based on a simulated sequencing dataset. Our program outcompetes other tools in terms of speed and is among the best tools in the presented quality benchmark. https://github.com/seqan/flexbar. johannes.roehr@fu-berlin.de or knut.reinert@fu-berlin.de. © The Author (2017). Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Combining multiple approaches and optimized data resolution for an improved understanding of stream temperature dynamics of a forested headwater basin in the Southern Appalachians

Science.gov (United States)

Belica, L.; Mitasova, H.; Caldwell, P.; McCarter, J. B.; Nelson, S. A. C.

2017-12-01

Thermal regimes of forested headwater streams continue to be an area of active research as climatic, hydrologic, and land cover changes can influence water temperature, a key aspect of aquatic ecosystems. Widespread monitoring of stream temperatures have provided an important data source, yielding insights on the temporal and spatial patterns and the underlying processes that influence stream temperature. However, small forested streams remain challenging to model due to the high spatial and temporal variability of stream temperatures and the climatic and hydrologic conditions that drive them. Technological advances and increased computational power continue to provide new tools and measurement methods and have allowed spatially explicit analyses of dynamic natural systems at greater temporal resolutions than previously possible. With the goal of understanding how current stream temperature patterns and processes may respond to changing landcover and hydroclimatoligical conditions, we combined high-resolution, spatially explicit geospatial modeling with deterministic heat flux modeling approaches using data sources that ranged from traditional hydrological and climatological measurements to emerging remote sensing techniques. Initial analyses of stream temperature monitoring data revealed that high temporal resolution (5 minutes) and measurement resolutions (guide field data collection for further heat flux modeling. By integrating multiple approaches and optimizing data resolution for the processes being investigated, small, but ecologically significant differences in stream thermal regimes were revealed. In this case, multi-approach research contributed to the identification of the dominant mechanisms driving stream temperature in the study area and advanced our understanding of the current thermal fluxes and how they may change as environmental conditions change in the future.

Parallel computing and networking; Heiretsu keisanki to network

Energy Technology Data Exchange (ETDEWEB)

Asakawa, E; Tsuru, T [Japan National Oil Corp., Tokyo (Japan); Matsuoka, T [Japan Petroleum Exploration Co. Ltd., Tokyo (Japan)

1996-05-01

This paper describes the trend of parallel computers used in geophysical exploration. Around 1993 was the early days when the parallel computers began to be used for geophysical exploration. Classification of these computers those days was mainly MIMD (multiple instruction stream, multiple data stream), SIMD (single instruction stream, multiple data stream) and the like. Parallel computers were publicized in the 1994 meeting of the Geophysical Exploration Society as a `high precision imaging technology`. Concerning the library of parallel computers, there was a shift to PVM (parallel virtual machine) in 1993 and to MPI (message passing interface) in 1995. In addition, the compiler of FORTRAN90 was released with support implemented for data parallel and vector computers. In 1993, networks used were Ethernet, FDDI, CDDI and HIPPI. In 1995, the OC-3 products under ATM began to propagate. However, ATM remains to be an interoffice high speed network because the ATM service has not spread yet for the public network. 1 ref.

HydroCloud: A Web Application for Exploring Stream Gage Data

Directory of Open Access Journals (Sweden)

Martin C. Roberge

2017-08-01

Full Text Available HydroCloud (hydrocloud.org is a mobile-friendly web application for visually browsing hydrology data from multiple sources. Data providers such as the US Geological Survey (USGS and the German 'Wasserstraßen- und Schifffahrtsverwaltung des Bundes' (WSV currently serve stream discharge data from more than 10,000 stream gages around the world. HydroCloud allows users to plot these data while out in the field, while also providing contextual information such as the current NEXRAD weather imagery or descriptive information about the stream gage and its watershed. Additional features include a chat mechanism for contacting developers, and the use of local storage for saving data.   Funding Statement: This project was supported in part by a grant from the Towson University School of Emerging Technology.

CAMS: OLAPing Multidimensional Data Streams Efficiently

Science.gov (United States)

Cuzzocrea, Alfredo

In the context of data stream research, taming the multidimensionality of real-life data streams in order to efficiently support OLAP analysis/mining tasks is a critical challenge. Inspired by this fundamental motivation, in this paper we introduce CAMS (C ube-based A cquisition model for M ultidimensional S treams), a model for efficiently OLAPing multidimensional data streams. CAMS combines a set of data stream processing methodologies, namely (i) the OLAP dimension flattening process, which allows us to obtain dimensionality reduction of multidimensional data streams, and (ii) the OLAP stream aggregation scheme, which aggregates data stream readings according to an OLAP-hierarchy-based membership approach. We complete our analytical contribution by means of experimental assessment and analysis of both the efficiency and the scalability of OLAPing capabilities of CAMS on synthetic multidimensional data streams. Both analytical and experimental results clearly connote CAMS as an enabling component for next-generation Data Stream Management Systems.

Parallel sorting algorithms

CERN Document Server

Akl, Selim G

1985-01-01

Parallel Sorting Algorithms explains how to use parallel algorithms to sort a sequence of items on a variety of parallel computers. The book reviews the sorting problem, the parallel models of computation, parallel algorithms, and the lower bounds on the parallel sorting problems. The text also presents twenty different algorithms, such as linear arrays, mesh-connected computers, cube-connected computers. Another example where algorithm can be applied is on the shared-memory SIMD (single instruction stream multiple data stream) computers in which the whole sequence to be sorted can fit in the

Architecture and VHDL behavioural validation of a parallel processor dedicated to computer vision

International Nuclear Information System (INIS)

Collette, Thierry

1992-01-01

Speeding up image processing is mainly obtained using parallel computers; SIMD processors (single instruction stream, multiple data stream) have been developed, and have proven highly efficient regarding low-level image processing operations. Nevertheless, their performances drop for most intermediate of high level operations, mainly when random data reorganisations in processor memories are involved. The aim of this thesis was to extend the SIMD computer capabilities to allow it to perform more efficiently at the image processing intermediate level. The study of some representative algorithms of this class, points out the limits of this computer. Nevertheless, these limits can be erased by architectural modifications. This leads us to propose SYMPATIX, a new SIMD parallel computer. To valid its new concept, a behavioural model written in VHDL - Hardware Description Language - has been elaborated. With this model, the new computer performances have been estimated running image processing algorithm simulations. VHDL modeling approach allows to perform the system top down electronic design giving an easy coupling between system architectural modifications and their electronic cost. The obtained results show SYMPATIX to be an efficient computer for low and intermediate level image processing. It can be connected to a high level computer, opening up the development of new computer vision applications. This thesis also presents, a top down design method, based on the VHDL, intended for electronic system architects. (author) [fr

The online performance estimation framework: heterogeneous ensemble learning for data streams

NARCIS (Netherlands)

van Rijn, J.N.; Holmes, G.; Pfahringer, B.; Vanschoren, J.

2018-01-01

Ensembles of classifiers are among the best performing classifiers available in many data mining applications, including the mining of data streams. Rather than training one classifier, multiple classifiers are trained, and their predictions are combined according to a given voting schedule. An

What Can Hierarchies Do for Data Streams?

DEFF Research Database (Denmark)

Yin, Xuepeng; Pedersen, Torben Bach

Much effort has been put into building data streams management systems for querying data streams. Here, data streams have been viewed as a flow of low-level data items, e.g., sensor readings or IP packet data. Stream query languages have mostly been SQL-based, with the STREAM and TelegraphCQ lang...

Knowledge discovery from data streams

CERN Document Server

Gama, Joao

2010-01-01

Since the beginning of the Internet age and the increased use of ubiquitous computing devices, the large volume and continuous flow of distributed data have imposed new constraints on the design of learning algorithms. Exploring how to extract knowledge structures from evolving and time-changing data, Knowledge Discovery from Data Streams presents a coherent overview of state-of-the-art research in learning from data streams.The book covers the fundamentals that are imperative to understanding data streams and describes important applications, such as TCP/IP traffic, GPS data, sensor networks,

Wadeable Streams Assessment Data

Science.gov (United States)

The Wadeable Streams Assessment (WSA) is a first-ever statistically-valid survey of the biological condition of small streams throughout the U.S. The U.S. Environmental Protection Agency (EPA) worked with the states to conduct the assessment in 2004-2005. Data for each parameter sampled in the Wadeable Streams Assessment (WSA) are available for downloading in a series of files as comma separated values (*.csv). Each *.csv data file has a companion text file (*.txt) that lists a dataset label and individual descriptions for each variable. Users should view the *.txt files first to help guide their understanding and use of the data.

Data Stream Processing Study in a Multichannel Telemetry Data Registering System

Directory of Open Access Journals (Sweden)

I. M. Sidyakin

2015-01-01

Full Text Available The paper presents the results of research that is aimed to improve the reliability of transmission of telemetry information (TMI through a communication channel with noise from the object of telemeasurements to the telemetry system for collecting and processing data. It considers the case where the quality of received information changes over time, due to movement of the object relative to the receiving station, or other factors that cause changes in the characteristics of noise in the channel, up to the total loss due to some temporary sites. To improve the reliability of transmission and ensure continuous communication with the object, it is proposed to use a multi-channel system to record the TMI. This system consists of several telemetry stations, which simultaneously register data stream transmitted from the telemetry object. The multichannel system generates a single stream of TMI for the user at the output. The stream comprises the most reliable pieces of information, being received at all inputs of the system.The paper investigates the task of constructing a multi-channel registration scheme for telemetry information (TMI to provide a simultaneous reception of the telemeasurement data by multiple telemetry stations and to form a single TMI stream containing the most reliable pieces of received data on the basis of quality analysis of information being received.In a multichannel registering system of TMI there are three main factors affecting the quality of the output of a single stream of information: 1 quality of the method used for protecting against errors during transmission over the communication channel with noise; 2 efficiency of the synchronization process of telemetry frames in the received flow of information; 3 efficiency of the applied criteria to form a single output stream from multiple input streams coming from different stations in the discussed multichannel registering system of TMI.In the paper, in practical

Multiple stress response of lowland stream benthic macroinvertebrates is dependent on habitat type

DEFF Research Database (Denmark)

Graeber, Daniel; Jensen, Tinna M.; Rasmussen, Jes

2017-01-01

Worldwide, lowland stream ecosystems are exposed to multiple anthropogenic stress due to the combination of water scarcity, eutrophication and fine sedimentation. The understanding of the effects of such multiple stress on stream benthic macroinvertebrates has been growing in the recent years...

Introduction to stream: An Extensible Framework for Data Stream Clustering Research with R

Directory of Open Access Journals (Sweden)

Michael Hahsler

2017-02-01

Full Text Available In recent years, data streams have become an increasingly important area of research for the computer science, database and statistics communities. Data streams are ordered and potentially unbounded sequences of data points created by a typically non-stationary data generating process. Common data mining tasks associated with data streams include clustering, classification and frequent pattern mining. New algorithms for these types of data are proposed regularly and it is important to evaluate them thoroughly under standardized conditions. In this paper we introduce stream, a research tool that includes modeling and simulating data streams as well as an extensible framework for implementing, interfacing and experimenting with algorithms for various data stream mining tasks. The main advantage of stream is that it seamlessly integrates with the large existing infrastructure provided by R. In addition to data handling, plotting and easy scripting capabilities, R also provides many existing algorithms and enables users to interface code written in many programming languages popular among data mining researchers (e.g., C/C++, Java and Python. In this paper we describe the architecture of stream and focus on its use for data stream clustering research. stream was implemented with extensibility in mind and will be extended in the future to cover additional data stream mining tasks like classification and frequent pattern mining.

On the Organization of Parallel Operation of Some Algorithms for Finding the Shortest Path on a Graph on a Computer System with Multiple Instruction Stream and Single Data Stream

Directory of Open Access Journals (Sweden)

V. E. Podol'skii

2015-01-01

Full Text Available The paper considers the implementing Bellman-Ford and Lee algorithms to find the shortest graph path on a computer system with multiple instruction stream and single data stream (MISD. The MISD computer is a computer that executes commands of arithmetic-logical processing (on the CPU and commands of structures processing (on the structures processor in parallel on a single data stream. Transformation of sequential programs into the MISD programs is a labor intensity process because it requires a stream of the arithmetic-logical processing to be manually separated from that of the structures processing. Algorithms based on the processing of data structures (e.g., algorithms on graphs show high performance on a MISD computer. Bellman-Ford and Lee algorithms for finding the shortest path on a graph are representatives of these algorithms. They are applied to robotics for automatic planning of the robot movement in-situ. Modification of Bellman-Ford and Lee algorithms for finding the shortest graph path in coprocessor MISD mode and the parallel MISD modification of these algorithms were first obtained in this article. Thus, this article continues a series of studies on the transformation of sequential algorithms into MISD ones (Dijkstra and Ford-Fulkerson 's algorithms and has a pronouncedly applied nature. The article also presents the analysis results of Bellman-Ford and Lee algorithms in MISD mode. The paper formulates the basic trends of a technique for parallelization of algorithms into arithmetic-logical processing stream and structures processing stream. Among the key areas for future research, development of the mathematical approach to provide a subsequently formalized and automated process of parallelizing sequential algorithms between the CPU and structures processor is highlighted. Among the mathematical models that can be used in future studies there are graph models of algorithms (e.g., dependency graph of a program. Due to the high

Handling multiple metadata streams regarding digital learning material

NARCIS (Netherlands)

Roes, J.B.M.; Vuuren, J. van; Verbeij, N.; Nijstad, H.

2010-01-01

This paper presents the outcome of a study performed in the Netherlands on handling multiple metadata streams regarding digital learning material. The paper describes the present metadata architecture in the Netherlands, the present suppliers and users of metadata and digital learning materials. It

Data streams: algorithms and applications

National Research Council Canada - National Science Library

Muthukrishnan, S

2005-01-01

... massive data sets in general. Researchers in Theoretical Computer Science, Databases, IP Networking and Computer Systems are working on the data stream challenges. This article is an overview and survey of data stream algorithmics and is an updated version of [175]. S. Muthukrishnan Rutgers University, New Brunswick, NJ, USA, muthu@cs...

Comparing the performance of SIMD computers by running large air pollution models

DEFF Research Database (Denmark)

Brown, J.; Hansen, Per Christian; Wasniewski, J.

1996-01-01

To compare the performance and use of three massively parallel SIMD computers, we implemented a large air pollution model on these computers. Using a realistic large-scale model, we gained detailed insight about the performance of the computers involved when used to solve large-scale scientific...... problems that involve several types of numerical computations. The computers used in our study are the Connection Machines CM-200 and CM-5, and the MasPar MP-2216...

Faster Smith-Waterman database searches with inter-sequence SIMD parallelisation.

Science.gov (United States)

Rognes, Torbjørn

2011-06-01

The Smith-Waterman algorithm for local sequence alignment is more sensitive than heuristic methods for database searching, but also more time-consuming. The fastest approach to parallelisation with SIMD technology has previously been described by Farrar in 2007. The aim of this study was to explore whether further speed could be gained by other approaches to parallelisation. A faster approach and implementation is described and benchmarked. In the new tool SWIPE, residues from sixteen different database sequences are compared in parallel to one query residue. Using a 375 residue query sequence a speed of 106 billion cell updates per second (GCUPS) was achieved on a dual Intel Xeon X5650 six-core processor system, which is over six times more rapid than software based on Farrar's 'striped' approach. SWIPE was about 2.5 times faster when the programs used only a single thread. For shorter queries, the increase in speed was larger. SWIPE was about twice as fast as BLAST when using the BLOSUM50 score matrix, while BLAST was about twice as fast as SWIPE for the BLOSUM62 matrix. The software is designed for 64 bit Linux on processors with SSSE3. Source code is available from http://dna.uio.no/swipe/ under the GNU Affero General Public License. Efficient parallelisation using SIMD on standard hardware makes it possible to run Smith-Waterman database searches more than six times faster than before. The approach described here could significantly widen the potential application of Smith-Waterman searches. Other applications that require optimal local alignment scores could also benefit from improved performance.

Guidelines for the collection of continuous stream water-temperature data in Alaska

Science.gov (United States)

Toohey, Ryan C.; Neal, Edward G.; Solin, Gary L.

2014-01-01

Objectives of stream monitoring programs differ considerably among many of the academic, Federal, state, tribal, and non-profit organizations in the state of Alaska. Broad inclusion of stream-temperature monitoring can provide an opportunity for collaboration in the development of a statewide stream-temperature database. Statewide and regional coordination could reduce overall monitoring cost, while providing better analyses at multiple spatial and temporal scales to improve resource decision-making. Increased adoption of standardized protocols and data-quality standards may allow for validation of historical modeling efforts with better projection calibration. For records of stream water temperature to be generally consistent, unbiased, and reproducible, data must be collected and analyzed according to documented protocols. Collection of water-temperature data requires definition of data-quality objectives, good site selection, proper selection of instrumentation, proper installation of sensors, periodic site visits to maintain sensors and download data, pre- and post-deployment verification against an NIST-certified thermometer, potential data corrections, and proper documentation, review, and approval. A study created to develop a quality-assurance project plan, data-quality objectives, and a database management plan that includes procedures for data archiving and dissemination could provide a means to standardize a statewide stream-temperature database in Alaska. Protocols can be modified depending on desired accuracy or specific needs of data collected. This document is intended to guide users in collecting time series water-temperature data in Alaskan streams and draws extensively on the broader protocols already published by the U.S. Geological Survey.

Attending to Multiple Visual Streams: Interactions between Location-Based and Category-Based Attentional Selection

Science.gov (United States)

Fagioli, Sabrina; Macaluso, Emiliano

2009-01-01

Behavioral studies indicate that subjects are able to divide attention between multiple streams of information at different locations. However, it is still unclear to what extent the observed costs reflect processes specifically associated with spatial attention, versus more general interference due the concurrent monitoring of multiple streams of…

Toward 3D-IPTV: design and implementation of a stereoscopic and multiple-perspective video streaming system

Science.gov (United States)

Petrovic, Goran; Farin, Dirk; de With, Peter H. N.

2008-02-01

3D-Video systems allow a user to perceive depth in the viewed scene and to display the scene from arbitrary viewpoints interactively and on-demand. This paper presents a prototype implementation of a 3D-video streaming system using an IP network. The architecture of our streaming system is layered, where each information layer conveys a single coded video signal or coded scene-description data. We demonstrate the benefits of a layered architecture with two examples: (a) stereoscopic video streaming, (b) monoscopic video streaming with remote multiple-perspective rendering. Our implementation experiments confirm that prototyping 3D-video streaming systems is possible with today's software and hardware. Furthermore, our current operational prototype demonstrates that highly heterogeneous clients can coexist in the system, ranging from auto-stereoscopic 3D displays to resource-constrained mobile devices.

Faster Smith-Waterman database searches with inter-sequence SIMD parallelisation

Directory of Open Access Journals (Sweden)

Rognes Torbjørn

2011-06-01

Full Text Available Abstract Background The Smith-Waterman algorithm for local sequence alignment is more sensitive than heuristic methods for database searching, but also more time-consuming. The fastest approach to parallelisation with SIMD technology has previously been described by Farrar in 2007. The aim of this study was to explore whether further speed could be gained by other approaches to parallelisation. Results A faster approach and implementation is described and benchmarked. In the new tool SWIPE, residues from sixteen different database sequences are compared in parallel to one query residue. Using a 375 residue query sequence a speed of 106 billion cell updates per second (GCUPS was achieved on a dual Intel Xeon X5650 six-core processor system, which is over six times more rapid than software based on Farrar's 'striped' approach. SWIPE was about 2.5 times faster when the programs used only a single thread. For shorter queries, the increase in speed was larger. SWIPE was about twice as fast as BLAST when using the BLOSUM50 score matrix, while BLAST was about twice as fast as SWIPE for the BLOSUM62 matrix. The software is designed for 64 bit Linux on processors with SSSE3. Source code is available from http://dna.uio.no/swipe/ under the GNU Affero General Public License. Conclusions Efficient parallelisation using SIMD on standard hardware makes it possible to run Smith-Waterman database searches more than six times faster than before. The approach described here could significantly widen the potential application of Smith-Waterman searches. Other applications that require optimal local alignment scores could also benefit from improved performance.

High-Performance Parallel and Stream Processing of X-ray Microdiffraction Data on Multicores

International Nuclear Information System (INIS)

Bauer, Michael A; McIntyre, Stewart; Xie Yuzhen; Biem, Alain; Tamura, Nobumichi

2012-01-01

We present the design and implementation of a high-performance system for processing synchrotron X-ray microdiffraction (XRD) data in IBM InfoSphere Streams on multicore processors. We report on the parallel and stream processing techniques that we use to harvest the power of clusters of multicores to analyze hundreds of gigabytes of synchrotron XRD data in order to reveal the microtexture of polycrystalline materials. The timing to process one XRD image using one pipeline is about ten times faster than the best C program at present. With the support of InfoSphere Streams platform, our software is able to be scaled up to operate on clusters of multi-cores for processing multiple images concurrently. This system provides a high-performance processing kernel to achieve near real-time data analysis of image data from synchrotron experiments.

Optimizing Cost of Continuous Overlapping Queries over Data Streams by Filter Adaption

KAUST Repository

Xie, Qing

2016-01-12

The problem we aim to address is the optimization of cost management for executing multiple continuous queries on data streams, where each query is defined by several filters, each of which monitors certain status of the data stream. Specially the filter can be shared by different queries and expensive to evaluate. The conventional objective for such a problem is to minimize the overall execution cost to solve all queries, by planning the order of filter evaluation in shared strategy. However, in streaming scenario, the characteristics of data items may change in process, which can bring some uncertainty to the outcome of individual filter evaluation, and affect the plan of query execution as well as the overall execution cost. In our work, considering the influence of the uncertain variation of data characteristics, we propose a framework to deal with the dynamic adjustment of filter ordering for query execution on data stream, and focus on the issues of cost management. By incrementally monitoring and analyzing the results of filter evaluation, our proposed approach can be effectively adaptive to the varied stream behavior and adjust the optimal ordering of filter evaluation, so as to optimize the execution cost. In order to achieve satisfactory performance and efficiency, we also discuss the trade-off between the adaptivity of our framework and the overhead incurred by filter adaption. The experimental results on synthetic and two real data sets (traffic and multimedia) show that our framework can effectively reduce and balance the overall query execution cost and keep high adaptivity in streaming scenario.

A real time sorting algorithm to time sort any deterministic time disordered data stream

Science.gov (United States)

Saini, J.; Mandal, S.; Chakrabarti, A.; Chattopadhyay, S.

2017-12-01

In new generation high intensity high energy physics experiments, millions of free streaming high rate data sources are to be readout. Free streaming data with associated time-stamp can only be controlled by thresholds as there is no trigger information available for the readout. Therefore, these readouts are prone to collect large amount of noise and unwanted data. For this reason, these experiments can have output data rate of several orders of magnitude higher than the useful signal data rate. It is therefore necessary to perform online processing of the data to extract useful information from the full data set. Without trigger information, pre-processing on the free streaming data can only be done with time based correlation among the data set. Multiple data sources have different path delays and bandwidth utilizations and therefore the unsorted merged data requires significant computational efforts for real time manifestation of sorting before analysis. Present work reports a new high speed scalable data stream sorting algorithm with its architectural design, verified through Field programmable Gate Array (FPGA) based hardware simulation. Realistic time based simulated data likely to be collected in an high energy physics experiment have been used to study the performance of the algorithm. The proposed algorithm uses parallel read-write blocks with added memory management and zero suppression features to make it efficient for high rate data-streams. This algorithm is best suited for online data streams with deterministic time disorder/unsorting on FPGA like hardware.

Effective Vectorization with OpenMP 4.5

Energy Technology Data Exchange (ETDEWEB)

Huber, Joseph N. [Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Hernandez, Oscar R. [Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Lopez, Matthew Graham [Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

2017-03-01

This paper describes how the Single Instruction Multiple Data (SIMD) model and its extensions in OpenMP work, and how these are implemented in different compilers. Modern processors are highly parallel computational machines which often include multiple processors capable of executing several instructions in parallel. Understanding SIMD and executing instructions in parallel allows the processor to achieve higher performance without increasing the power required to run it. SIMD instructions can significantly reduce the runtime of code by executing a single operation on large groups of data. The SIMD model is so integral to the processor s potential performance that, if SIMD is not utilized, less than half of the processor is ever actually used. Unfortunately, using SIMD instructions is a challenge in higher level languages because most programming languages do not have a way to describe them. Most compilers are capable of vectorizing code by using the SIMD instructions, but there are many code features important for SIMD vectorization that the compiler cannot determine at compile time. OpenMP attempts to solve this by extending the C++/C and Fortran programming languages with compiler directives that express SIMD parallelism. OpenMP is used to pass hints to the compiler about the code to be executed in SIMD. This is a key resource for making optimized code, but it does not change whether or not the code can use SIMD operations. However, in many cases critical functions are limited by a poor understanding of how SIMD instructions are actually implemented, as SIMD can be implemented through vector instructions or simultaneous multi-threading (SMT). We have found that it is often the case that code cannot be vectorized, or is vectorized poorly, because the programmer does not have sufficient knowledge of how SIMD instructions work.

StreamQRE: Modular Specification and Efficient Evaluation of Quantitative Queries over Streaming Data.

Science.gov (United States)

Mamouras, Konstantinos; Raghothaman, Mukund; Alur, Rajeev; Ives, Zachary G; Khanna, Sanjeev

2017-06-01

Real-time decision making in emerging IoT applications typically relies on computing quantitative summaries of large data streams in an efficient and incremental manner. To simplify the task of programming the desired logic, we propose StreamQRE, which provides natural and high-level constructs for processing streaming data. Our language has a novel integration of linguistic constructs from two distinct programming paradigms: streaming extensions of relational query languages and quantitative extensions of regular expressions. The former allows the programmer to employ relational constructs to partition the input data by keys and to integrate data streams from different sources, while the latter can be used to exploit the logical hierarchy in the input stream for modular specifications. We first present the core language with a small set of combinators, formal semantics, and a decidable type system. We then show how to express a number of common patterns with illustrative examples. Our compilation algorithm translates the high-level query into a streaming algorithm with precise complexity bounds on per-item processing time and total memory footprint. We also show how to integrate approximation algorithms into our framework. We report on an implementation in Java, and evaluate it with respect to existing high-performance engines for processing streaming data. Our experimental evaluation shows that (1) StreamQRE allows more natural and succinct specification of queries compared to existing frameworks, (2) the throughput of our implementation is higher than comparable systems (for example, two-to-four times greater than RxJava), and (3) the approximation algorithms supported by our implementation can lead to substantial memory savings.

Accelerating finite-rate chemical kinetics with coprocessors: Comparing vectorization methods on GPUs, MICs, and CPUs

Science.gov (United States)

Stone, Christopher P.; Alferman, Andrew T.; Niemeyer, Kyle E.

2018-05-01

Accurate and efficient methods for solving stiff ordinary differential equations (ODEs) are a critical component of turbulent combustion simulations with finite-rate chemistry. The ODEs governing the chemical kinetics at each mesh point are decoupled by operator-splitting allowing each to be solved concurrently. An efficient ODE solver must then take into account the available thread and instruction-level parallelism of the underlying hardware, especially on many-core coprocessors, as well as the numerical efficiency. A stiff Rosenbrock and a nonstiff Runge-Kutta ODE solver are both implemented using the single instruction, multiple thread (SIMT) and single instruction, multiple data (SIMD) paradigms within OpenCL. Both methods solve multiple ODEs concurrently within the same instruction stream. The performance of these parallel implementations was measured on three chemical kinetic models of increasing size across several multicore and many-core platforms. Two separate benchmarks were conducted to clearly determine any performance advantage offered by either method. The first benchmark measured the run-time of evaluating the right-hand-side source terms in parallel and the second benchmark integrated a series of constant-pressure, homogeneous reactors using the Rosenbrock and Runge-Kutta solvers. The right-hand-side evaluations with SIMD parallelism on the host multicore Xeon CPU and many-core Xeon Phi co-processor performed approximately three times faster than the baseline multithreaded C++ code. The SIMT parallel model on the host and Phi was 13%-35% slower than the baseline while the SIMT model on the NVIDIA Kepler GPU provided approximately the same performance as the SIMD model on the Phi. The runtimes for both ODE solvers decreased significantly with the SIMD implementations on the host CPU (2.5-2.7 ×) and Xeon Phi coprocessor (4.7-4.9 ×) compared to the baseline parallel code. The SIMT implementations on the GPU ran 1.5-1.6 times faster than the baseline

Responses of stream microbes to multiple anthropogenic stressors in a mesocosm study.

Science.gov (United States)

Nuy, Julia K; Lange, Anja; Beermann, Arne J; Jensen, Manfred; Elbrecht, Vasco; Röhl, Oliver; Peršoh, Derek; Begerow, Dominik; Leese, Florian; Boenigk, Jens

2018-08-15

Stream ecosystems are affected by multiple anthropogenic stressors worldwide. Even though effects of many single stressors are comparatively well studied, the effects of multiple stressors are difficult to predict. In particular bacteria and protists, which are responsible for the majority of ecosystem respiration and element flows, are infrequently studied with respect to multiple stressors responses. We conducted a stream mesocosm experiment to characterize the responses of single and multiple stressors on microbiota. Two functionally important stream habitats, leaf litter and benthic phototrophic rock biofilms, were exposed to three stressors in a full factorial design: fine sediment deposition, increased chloride concentration (salinization) and reduced flow velocity. We analyzed the microbial composition in the two habitat types of the mesocosms using an amplicon sequencing approach. Community analysis on different taxonomic levels as well as principle component analyses (PCoAs) based on realtive abundances of operational taxonomic units (OTUs) showed treatment specific shifts in the eukaryotic biofilm community. Analysis of variance (ANOVA) revealed that Bacillariophyta responded positively salinity and sediment increase, while the relative read abundance of chlorophyte taxa decreased. The combined effects of multiple stressors were mainly antagonistic. Therefore, the community composition in multiply stressed environments resembled the composition of the unstressed control community in terms of OTU occurrence and relative abundances. Copyright © 2018 The Authors. Published by Elsevier B.V. All rights reserved.

Efficient, symmetry-driven SIMD access patterns for 3D PET image reconstruction applicable for CPUs and GPUs

Energy Technology Data Exchange (ETDEWEB)

Scheins, J.J.; Garcia Lucio, L.F.; Herzog, H.; Shah, N.J. [Forschungszentrum Juelich GmbH (Germany). Inst. of Neuroscience and Medicine (INM-4)

2011-07-01

Fully 3D PET image reconstruction still remains a challenging computational task due to the tremendous number of registered Lines-of-Response. Typically, billions of geometrical weights have to be repeatedly calculated and evaluated for iterative algorithms. In this context, the reconstruction software PRESTO (PET REconstruction Software TOolkit) provides accurate geometrical weighting schemes for the forward projection and backward projection, e.g. Volume-of-Intersection, while using all measured LORs separately. PRESTO exploits redundancies to realise a strongly compressed, memory-resident system matrix. Consequently, the needed time to calculate matrix weights no longer influences the reconstruction time. Very high compression factors (>300) are achieved by using unconventional non-cartesian voxel patterns. However, in the original implementation the addressing of matrix weights, projection values and voxel values happens in disfavoured memory access patterns. This causes severe computational inefficiencies due to the limited memory bandwidth using CPUs. In this work, the image data and projection data in memory as well as the order of mathematical operations have been completely re-organised to provide an optimal merit for the Single Instruction Multiple Data (SIMD) approach. This reorganisation is directly driven by the induced symmetries of PRESTO. A global speedup factor of 15 for has been achieved for the CPU-based implementation while obtaining identical results. In addition, a GPU-based implementation using CUDA on Nvidia TESLA C1060/S1070 hardware provides another speed up factor of 4 compared to single core CPU processing. (orig.)

Efficient, symmetry-driven SIMD access patterns for 3D PET image reconstruction applicable for CPUs and GPUs

International Nuclear Information System (INIS)

Scheins, J.J.; Garcia Lucio, L.F.; Herzog, H.; Shah, N.J.

2011-01-01

Fully 3D PET image reconstruction still remains a challenging computational task due to the tremendous number of registered Lines-of-Response. Typically, billions of geometrical weights have to be repeatedly calculated and evaluated for iterative algorithms. In this context, the reconstruction software PRESTO (PET REconstruction Software TOolkit) provides accurate geometrical weighting schemes for the forward projection and backward projection, e.g. Volume-of-Intersection, while using all measured LORs separately. PRESTO exploits redundancies to realise a strongly compressed, memory-resident system matrix. Consequently, the needed time to calculate matrix weights no longer influences the reconstruction time. Very high compression factors (>300) are achieved by using unconventional non-cartesian voxel patterns. However, in the original implementation the addressing of matrix weights, projection values and voxel values happens in disfavoured memory access patterns. This causes severe computational inefficiencies due to the limited memory bandwidth using CPUs. In this work, the image data and projection data in memory as well as the order of mathematical operations have been completely re-organised to provide an optimal merit for the Single Instruction Multiple Data (SIMD) approach. This reorganisation is directly driven by the induced symmetries of PRESTO. A global speedup factor of 15 for has been achieved for the CPU-based implementation while obtaining identical results. In addition, a GPU-based implementation using CUDA on Nvidia TESLA C1060/S1070 hardware provides another speed up factor of 4 compared to single core CPU processing. (orig.)

Data Stream Classification Based on the Gamma Classifier

Directory of Open Access Journals (Sweden)

Abril Valeria Uriarte-Arcia

2015-01-01

Full Text Available The ever increasing data generation confronts us with the problem of handling online massive amounts of information. One of the biggest challenges is how to extract valuable information from these massive continuous data streams during single scanning. In a data stream context, data arrive continuously at high speed; therefore the algorithms developed to address this context must be efficient regarding memory and time management and capable of detecting changes over time in the underlying distribution that generated the data. This work describes a novel method for the task of pattern classification over a continuous data stream based on an associative model. The proposed method is based on the Gamma classifier, which is inspired by the Alpha-Beta associative memories, which are both supervised pattern recognition models. The proposed method is capable of handling the space and time constrain inherent to data stream scenarios. The Data Streaming Gamma classifier (DS-Gamma classifier implements a sliding window approach to provide concept drift detection and a forgetting mechanism. In order to test the classifier, several experiments were performed using different data stream scenarios with real and synthetic data streams. The experimental results show that the method exhibits competitive performance when compared to other state-of-the-art algorithms.

Analytic Strategies of Streaming Data for eHealth.

Science.gov (United States)

Yoon, Sunmoo

2016-01-01

New analytic strategies for streaming big data from wearable devices and social media are emerging in ehealth. We face challenges to find meaningful patterns from big data because researchers face difficulties to process big volume of streaming data using traditional processing applications.1 This introductory 180 minutes tutorial offers hand-on instruction on analytics2 (e.g., topic modeling, social network analysis) of streaming data. This tutorial aims to provide practical strategies of information on reducing dimensionality using examples of big data. This tutorial will highlight strategies of incorporating domain experts and a comprehensive approach to streaming social media data.

Human Factors in Streaming Data Analysis: Challenges and Opportunities for Information Visualization: Human Factors in Streaming Data Analysis

Energy Technology Data Exchange (ETDEWEB)

Dasgupta, Aritra [Pacific Northwest National Laboratory, Richland Washington USA; Arendt, Dustin L. [Pacific Northwest National Laboratory, Richland Washington USA; Franklin, Lyndsey R. [Pacific Northwest National Laboratory, Richland Washington USA; Wong, Pak Chung [Pacific Northwest National Laboratory, Richland Washington USA; Cook, Kristin A. [Pacific Northwest National Laboratory, Richland Washington USA

2017-09-01

Real-world systems change continuously and across domains like traffic monitoring, cyber security, etc., such changes occur within short time scales. This leads to a streaming data problem and produces unique challenges for the human in the loop, as analysts have to ingest and make sense of dynamic patterns in real time. In this paper, our goal is to study how the state-of-the-art in streaming data visualization handles these challenges and reflect on the gaps and opportunities. To this end, we have three contributions: i) problem characterization for identifying domain-specific goals and challenges for handling streaming data, ii) a survey and analysis of the state-of-the-art in streaming data visualization research with a focus on the visualization design space, and iii) reflections on the perceptually motivated design challenges and potential research directions for addressing them.

Data Stream Clustering With Affinity Propagation

KAUST Repository

Zhang, Xiangliang

2014-07-09

Data stream clustering provides insights into the underlying patterns of data flows. This paper focuses on selecting the best representatives from clusters of streaming data. There are two main challenges: how to cluster with the best representatives and how to handle the evolving patterns that are important characteristics of streaming data with dynamic distributions. We employ the Affinity Propagation (AP) algorithm presented in 2007 by Frey and Dueck for the first challenge, as it offers good guarantees of clustering optimality for selecting exemplars. The second challenging problem is solved by change detection. The presented StrAP algorithm combines AP with a statistical change point detection test; the clustering model is rebuilt whenever the test detects a change in the underlying data distribution. Besides the validation on two benchmark data sets, the presented algorithm is validated on a real-world application, monitoring the data flow of jobs submitted to the EGEE grid.

Data Stream Clustering With Affinity Propagation

KAUST Repository

Zhang, Xiangliang; Furtlehner, Cyril; Germain-Renaud, Cecile; Sebag, Michele

2014-01-01

Data stream clustering provides insights into the underlying patterns of data flows. This paper focuses on selecting the best representatives from clusters of streaming data. There are two main challenges: how to cluster with the best representatives and how to handle the evolving patterns that are important characteristics of streaming data with dynamic distributions. We employ the Affinity Propagation (AP) algorithm presented in 2007 by Frey and Dueck for the first challenge, as it offers good guarantees of clustering optimality for selecting exemplars. The second challenging problem is solved by change detection. The presented StrAP algorithm combines AP with a statistical change point detection test; the clustering model is rebuilt whenever the test detects a change in the underlying data distribution. Besides the validation on two benchmark data sets, the presented algorithm is validated on a real-world application, monitoring the data flow of jobs submitted to the EGEE grid.

An Approach for Removing Redundant Data from RFID Data Streams

Science.gov (United States)

Mahdin, Hairulnizam; Abawajy, Jemal

2011-01-01

Radio frequency identification (RFID) systems are emerging as the primary object identification mechanism, especially in supply chain management. However, RFID naturally generates a large amount of duplicate readings. Removing these duplicates from the RFID data stream is paramount as it does not contribute new information to the system and wastes system resources. Existing approaches to deal with this problem cannot fulfill the real time demands to process the massive RFID data stream. We propose a data filtering approach that efficiently detects and removes duplicate readings from RFID data streams. Experimental results show that the proposed approach offers a significant improvement as compared to the existing approaches. PMID:22163730

An Approach for Removing Redundant Data from RFID Data Streams

Directory of Open Access Journals (Sweden)

Hairulnizam Mahdin

2011-10-01

Full Text Available Radio frequency identification (RFID systems are emerging as the primary object identification mechanism, especially in supply chain management. However, RFID naturally generates a large amount of duplicate readings. Removing these duplicates from the RFID data stream is paramount as it does not contribute new information to the system and wastes system resources. Existing approaches to deal with this problem cannot fulfill the real time demands to process the massive RFID data stream. We propose a data filtering approach that efficiently detects and removes duplicate readings from RFID data streams. Experimental results show that the proposed approach offers a significant improvement as compared to the existing approaches.

A Multiple Streams analysis of the decisions to fund gender-neutral HPV vaccination in Canada.

Science.gov (United States)

Shapiro, Gilla K; Guichon, Juliet; Prue, Gillian; Perez, Samara; Rosberger, Zeev

2017-07-01

In Canada, the human papillomavirus (HPV) vaccine is licensed and recommended for females and males. Although all Canadian jurisdictions fund school-based HPV vaccine programs for girls, only six jurisdictions fund school-based HPV vaccination for boys. The research aimed to analyze the factors that underpin government decisions to fund HPV vaccine for boys using a theoretical policy model, Kingdon's Multiple Streams framework. This approach assesses policy development by examining three concurrent, but independent, streams that guide analysis: Problem Stream, Policy Stream, and Politics Stream. Analysis from the Problem Stream highlights that males are affected by HPV-related diseases and are involved in transmitting HPV infection to their sexual partners. Policy Stream analysis makes clear that while the inclusion of males in HPV vaccine programs is suitable, equitable, and acceptable; there is debate regarding cost-effectiveness. Politics Stream analysis identifies the perspectives of six different stakeholder groups and highlights the contribution of government officials at the provincial and territorial level. Kingdon's Multiple Streams framework helps clarify the opportunities and barriers for HPV vaccine policy change. This analysis identified that the interpretation of cost-effectiveness models and advocacy of stakeholders such as citizen-advocates and HPV-affected politicians have been particularly important in galvanizing policy change. Copyright © 2017 Elsevier Inc. All rights reserved.

A survey on Big Data Stream Mining

African Journals Online (AJOL)

pc

2018-03-05

Mar 5, 2018 ... huge amount of stream like telecommunication systems. So, there ... streams have many challenges for data mining algorithm design like using of ..... A. Bifet and R. Gavalda, "Learning from Time-Changing Data with. Adaptive ...

Variation partitioning of diatom species data matrices: Understanding the influence of multiple factors on benthic diatom communities in tropical streams

Energy Technology Data Exchange (ETDEWEB)

Bere, Taurai, E-mail: tbere2015@gmail.com; Mangadze, Tinotenda; Mwedzi, Tongai

2016-10-01

Elucidating the confounding influence of multiple environmental factors on benthic diatom communities is important in developing water quality predictive models for better guidance of stream management efforts. The objective of this study was to explore the relative impact of metal pollution and hydromorphological alterations in, addition to nutrient enrichment and organic pollution, on diatom taxonomic composition with the view to improve stream diatom-based water quality inference models. Samples were collected twice at 20 sampling stations in the tropical Manyame Catchment, Zimbabwe. Diatom, macroinvertebrate communities and environmental factors were sampled and analysed. The variations in diatom community composition explained by different categories of environmental factors were analysed using canonical correspondence analysis using variance partitioning (partial CCA). The following variations were explained by the different predictor matrices: nutrient levels and organic pollution - 10.4%, metal pollution - 8.3% and hydromorphological factors - 7.9%. Thus, factors other than nutrient levels and organic pollution explain additional significant variation in these diatom communities. Development of diatom-based stream water quality inference models that incorporate metal pollution and hydromorphological alterations, where these are key issues, is thus deemed necessary. - Highlights: • Confounding influences of multiple environmental factors on diatom communities are elucidated. • Variation explained: nutrients + organic pollution - 10.4%, metals - 8.3% and hydromorphological factors - 7.9%. • Calibration of existing or development of new indices may be necessary.

Variation partitioning of diatom species data matrices: Understanding the influence of multiple factors on benthic diatom communities in tropical streams

International Nuclear Information System (INIS)

Bere, Taurai; Mangadze, Tinotenda; Mwedzi, Tongai

2016-01-01

Elucidating the confounding influence of multiple environmental factors on benthic diatom communities is important in developing water quality predictive models for better guidance of stream management efforts. The objective of this study was to explore the relative impact of metal pollution and hydromorphological alterations in, addition to nutrient enrichment and organic pollution, on diatom taxonomic composition with the view to improve stream diatom-based water quality inference models. Samples were collected twice at 20 sampling stations in the tropical Manyame Catchment, Zimbabwe. Diatom, macroinvertebrate communities and environmental factors were sampled and analysed. The variations in diatom community composition explained by different categories of environmental factors were analysed using canonical correspondence analysis using variance partitioning (partial CCA). The following variations were explained by the different predictor matrices: nutrient levels and organic pollution - 10.4%, metal pollution - 8.3% and hydromorphological factors - 7.9%. Thus, factors other than nutrient levels and organic pollution explain additional significant variation in these diatom communities. Development of diatom-based stream water quality inference models that incorporate metal pollution and hydromorphological alterations, where these are key issues, is thus deemed necessary. - Highlights: • Confounding influences of multiple environmental factors on diatom communities are elucidated. • Variation explained: nutrients + organic pollution - 10.4%, metals - 8.3% and hydromorphological factors - 7.9%. • Calibration of existing or development of new indices may be necessary.

What Can Hierarchies Do for Data Streams?

DEFF Research Database (Denmark)

Yin, Xuepeng; Pedersen, Torben Bach

2007-01-01

Abstract. Much effort has been put into building data streams management systems for querying data streams. However, the query languages have mostly been SQL-based and aimed for low-level analysis of base data; therefore, there has been little work on supporting OLAP-like queries that provide rea...

Towards adaptive, streaming analysis of x-ray tomography data

Energy Technology Data Exchange (ETDEWEB)

Thomas, Mathew; Kleese van Dam, Kerstin; Marshall, Matthew J.; Kuprat, Andrew P.; Carson, James P.; Lansing, Carina S.; Guillen, Zoe C.; Miller, Erin A.; Lanekoff, Ingela; Laskin, Julia

2015-03-04

Temporal and spatial resolution of chemical imaging methodologies such as x-ray tomography are rapidly increasing, leading to more complex experimental procedures and fast growing data volumes. Automated analysis pipelines and big data analytics are becoming essential to effectively evaluate the results of such experiments. Offering those data techniques in an adaptive, streaming environment can further substantially improve the scientific discovery process, by enabling experimental control and steering based on the evaluation of emerging phenomena as they are observed by the experiment. Pacific Northwest National Laboratory (PNNL)’ Chemical Imaging Initiative (CII - http://imaging.pnnl.gov/ ) has worked since 2011 towards developing a framework that allows users to rapidly compose and customize high throughput experimental analysis pipelines for multiple instrument types. The framework, named ‘Rapid Experimental Analysis’ (REXAN) Framework [1], is based on the idea of reusable component libraries and utilizes the PNNL developed collaborative data management and analysis environment ‘Velo’, to provide a user friendly analysis and data management environment for experimental facilities. This article will, discuss the capabilities established for X-Ray tomography, discuss lessons learned, and provide an overview of our more recent work in the Analysis in Motion Initiative (AIM - http://aim.pnnl.gov/ ) at PNNL to provide REXAN capabilities in a streaming environment.

Prediction and explanation over DL-Lite data streams

CSIR Research Space (South Africa)

Klarman, S

2013-12-01

Full Text Available the popular DL-Lite family, and study the logic foundations of prediction and explanation over DL-Lite data streams, i.e., reasoning from finite segments of streaming data to conjectures about the content of the streams in the future or in the past. We propose...

Alignment data streams for the ATLAS inner detector

CERN Document Server

Pinto, B; Pereira, P; Elsing, M; Hawkings, R; Schieck, J; García, S; Schaffer, A; Ma, H; Anjos, A

2008-01-01

The ATLAS experiment uses a complex trigger strategy to be able to reduce the Event Filter rate output, down to a level that allows the storage and processing of these data. These concepts are described in the ATLAS Computing Model which embraces Grid paradigm. The output coming from the Event Filter consists of four main streams: physical stream, express stream, calibration stream, and diagnostic stream. The calibration stream will be transferred to the Tier-0 facilities that will provide the prompt reconstruction of this stream with a minimum latency of 8 hours, producing calibration constants of sufficient quality to allow a first-pass processing. The Inner Detector community is developing and testing an independent common calibration stream selected at the Event Filter after track reconstruction. It is composed of raw data, in byte-stream format, contained in Readout Buffers (ROBs) with hit information of the selected tracks, and it will be used to derive and update a set of calibration and alignment cons...

Streaming Model Based Volume Ray Casting Implementation for Cell Broadband Engine

Directory of Open Access Journals (Sweden)

Jusub Kim

2009-01-01

Full Text Available Interactive high quality volume rendering is becoming increasingly more important as the amount of more complex volumetric data steadily grows. While a number of volumetric rendering techniques have been widely used, ray casting has been recognized as an effective approach for generating high quality visualization. However, for most users, the use of ray casting has been limited to datasets that are very small because of its high demands on computational power and memory bandwidth. However the recent introduction of the Cell Broadband Engine (Cell B.E. processor, which consists of 9 heterogeneous cores designed to handle extremely demanding computations with large streams of data, provides an opportunity to put the ray casting into practical use. In this paper, we introduce an efficient parallel implementation of volume ray casting on the Cell B.E. The implementation is designed to take full advantage of the computational power and memory bandwidth of the Cell B.E. using an intricate orchestration of the ray casting computation on the available heterogeneous resources. Specifically, we introduce streaming model based schemes and techniques to efficiently implement acceleration techniques for ray casting on Cell B.E. In addition to ensuring effective SIMD utilization, our method provides two key benefits: there is no cost for empty space skipping and there is no memory bottleneck on moving volumetric data for processing. Our experimental results show that we can interactively render practical datasets on a single Cell B.E. processor.

Continuous sampling from distributed streams

DEFF Research Database (Denmark)

Graham, Cormode; Muthukrishnan, S.; Yi, Ke

2012-01-01

A fundamental problem in data management is to draw and maintain a sample of a large data set, for approximate query answering, selectivity estimation, and query planning. With large, streaming data sets, this problem becomes particularly difficult when the data is shared across multiple distribu......A fundamental problem in data management is to draw and maintain a sample of a large data set, for approximate query answering, selectivity estimation, and query planning. With large, streaming data sets, this problem becomes particularly difficult when the data is shared across multiple...... distributed sites. The main challenge is to ensure that a sample is drawn uniformly across the union of the data while minimizing the communication needed to run the protocol on the evolving data. At the same time, it is also necessary to make the protocol lightweight, by keeping the space and time costs low...... for each participant. In this article, we present communication-efficient protocols for continuously maintaining a sample (both with and without replacement) from k distributed streams. These apply to the case when we want a sample from the full streams, and to the sliding window cases of only the W most...

Interaction sorting method for molecular dynamics on multi-core SIMD CPU architecture.

Science.gov (United States)

Matvienko, Sergey; Alemasov, Nikolay; Fomin, Eduard

2015-02-01

Molecular dynamics (MD) is widely used in computational biology for studying binding mechanisms of molecules, molecular transport, conformational transitions, protein folding, etc. The method is computationally expensive; thus, the demand for the development of novel, much more efficient algorithms is still high. Therefore, the new algorithm designed in 2007 and called interaction sorting (IS) clearly attracted interest, as it outperformed the most efficient MD algorithms. In this work, a new IS modification is proposed which allows the algorithm to utilize SIMD processor instructions. This paper shows that the improvement provides an additional gain in performance, 9% to 45% in comparison to the original IS method.

Mining Building Metadata by Data Stream Comparison

DEFF Research Database (Denmark)

Holmegaard, Emil; Kjærgaard, Mikkel Baun

2016-01-01

to handle data streams with only slightly similar patterns. We have evaluated Metafier with points and data from one building located in Denmark. We have evaluated Metafier with 903 points, and the overall accuracy, with only 3 known examples, was 94.71%. Furthermore we found that using DTW for mining...... ways to annotate sensor and actuation points. This makes it difficult to create intuitive queries for retrieving data streams from points. Another problem is the amount of insufficient or missing metadata. We introduce Metafier, a tool for extracting metadata from comparing data streams. Metafier...... enables a semi-automatic labeling of metadata to building instrumentation. Metafier annotates points with metadata by comparing the data from a set of validated points with unvalidated points. Metafier has three different algorithms to compare points with based on their data. The three algorithms...

A Context-Aware Adaptive Streaming Media Distribution System in a Heterogeneous Network with Multiple Terminals

Directory of Open Access Journals (Sweden)

Yepeng Ni

2016-01-01

Full Text Available We consider the problem of streaming media transmission in a heterogeneous network from a multisource server to home multiple terminals. In wired network, the transmission performance is limited by network state (e.g., the bandwidth variation, jitter, and packet loss. In wireless network, the multiple user terminals can cause bandwidth competition. Thus, the streaming media distribution in a heterogeneous network becomes a severe challenge which is critical for QoS guarantee. In this paper, we propose a context-aware adaptive streaming media distribution system (CAASS, which implements the context-aware module to perceive the environment parameters and use the strategy analysis (SA module to deduce the most suitable service level. This approach is able to improve the video quality for guarantying streaming QoS. We formulate the optimization problem of QoS relationship with the environment parameters based on the QoS testing algorithm for IPTV in ITU-T G.1070. We evaluate the performance of the proposed CAASS through 12 types of experimental environments using a prototype system. Experimental results show that CAASS can dynamically adjust the service level according to the environment variation (e.g., network state and terminal performances and outperforms the existing streaming approaches in adaptive streaming media distribution according to peak signal-to-noise ratio (PSNR.

AMIDST: Analysis of MassIve Data STreams

DEFF Research Database (Denmark)

Masegosa, Andres; Martinez, Ana Maria; Borchani, Hanen

2015-01-01

The Analysis of MassIve Data STreams (AMIDST) Java toolbox provides a collection of scalable and parallel algorithms for inference and learning of hybrid Bayesian networks from data streams. The toolbox, available at http://amidst.github.io/toolbox/ under the Apache Software License version 2.......0, also efficiently leverages existing functionalities and algorithms by interfacing to software tools such as HUGIN and MOA....

Scenario driven data modelling: a method for integrating diverse sources of data and data streams

Science.gov (United States)

2011-01-01

Background Biology is rapidly becoming a data intensive, data-driven science. It is essential that data is represented and connected in ways that best represent its full conceptual content and allows both automated integration and data driven decision-making. Recent advancements in distributed multi-relational directed graphs, implemented in the form of the Semantic Web make it possible to deal with complicated heterogeneous data in new and interesting ways. Results This paper presents a new approach, scenario driven data modelling (SDDM), that integrates multi-relational directed graphs with data streams. SDDM can be applied to virtually any data integration challenge with widely divergent types of data and data streams. In this work, we explored integrating genetics data with reports from traditional media. SDDM was applied to the New Delhi metallo-beta-lactamase gene (NDM-1), an emerging global health threat. The SDDM process constructed a scenario, created a RDF multi-relational directed graph that linked diverse types of data to the Semantic Web, implemented RDF conversion tools (RDFizers) to bring content into the Sematic Web, identified data streams and analytical routines to analyse those streams, and identified user requirements and graph traversals to meet end-user requirements. Conclusions We provided an example where SDDM was applied to a complex data integration challenge. The process created a model of the emerging NDM-1 health threat, identified and filled gaps in that model, and constructed reliable software that monitored data streams based on the scenario derived multi-relational directed graph. The SDDM process significantly reduced the software requirements phase by letting the scenario and resulting multi-relational directed graph define what is possible and then set the scope of the user requirements. Approaches like SDDM will be critical to the future of data intensive, data-driven science because they automate the process of converting

StreamSqueeze: a dynamic stream visualization for monitoring of event data

Science.gov (United States)

Mansmann, Florian; Krstajic, Milos; Fischer, Fabian; Bertini, Enrico

2012-01-01

While in clear-cut situations automated analytical solution for data streams are already in place, only few visual approaches have been proposed in the literature for exploratory analysis tasks on dynamic information. However, due to the competitive or security-related advantages that real-time information gives in domains such as finance, business or networking, we are convinced that there is a need for exploratory visualization tools for data streams. Under the conditions that new events have higher relevance and that smooth transitions enable traceability of items, we propose a novel dynamic stream visualization called StreamSqueeze. In this technique the degree of interest of recent items is expressed through an increase in size and thus recent events can be shown with more details. The technique has two main benefits: First, the layout algorithm arranges items in several lists of various sizes and optimizes the positions within each list so that the transition of an item from one list to the other triggers least visual changes. Second, the animation scheme ensures that for 50 percent of the time an item has a static screen position where reading is most effective and then continuously shrinks and moves to the its next static position in the subsequent list. To demonstrate the capability of our technique, we apply it to large and high-frequency news and syslog streams and show how it maintains optimal stability of the layout under the conditions given above.

Analyzing indicators of stream health for Minnesota streams

Science.gov (United States)

Singh, U.; Kocian, M.; Wilson, B.; Bolton, A.; Nieber, J.; Vondracek, B.; Perry, J.; Magner, J.

2005-01-01

Recent research has emphasized the importance of using physical, chemical, and biological indicators of stream health for diagnosing impaired watersheds and their receiving water bodies. A multidisciplinary team at the University of Minnesota is carrying out research to develop a stream classification system for Total Maximum Daily Load (TMDL) assessment. Funding for this research is provided by the United States Environmental Protection Agency and the Minnesota Pollution Control Agency. One objective of the research study involves investigating the relationships between indicators of stream health and localized stream characteristics. Measured data from Minnesota streams collected by various government and non-government agencies and research institutions have been obtained for the research study. Innovative Geographic Information Systems tools developed by the Environmental Science Research Institute and the University of Texas are being utilized to combine and organize the data. Simple linear relationships between index of biological integrity (IBI) and channel slope, two-year stream flow, and drainage area are presented for the Redwood River and the Snake River Basins. Results suggest that more rigorous techniques are needed to successfully capture trends in IBI scores. Additional analyses will be done using multiple regression, principal component analysis, and clustering techniques. Uncovering key independent variables and understanding how they fit together to influence stream health are critical in the development of a stream classification for TMDL assessment.

Event Streams Clustering Using Machine Learning Techniques

Directory of Open Access Journals (Sweden)

Hanen Bouali

2015-10-01

Full Text Available Data streams are usually of unbounded lengths which push users to consider only recent observations by focusing on a time window, and ignore past data. However, in many real world applications, past data must be taken in consideration to guarantee the efficiency, the performance of decision making and to handle data streams evolution over time. In order to build a selectively history to track the underlying event streams changes, we opt for the continuously data of the sliding window which increases the time window based on changes over historical data. In this paper, to have the ability to access to historical data without requiring any significant storage or multiple passes over the data. In this paper, we propose a new algorithm for clustering multiple data streams using incremental support vector machine and data representative points’ technique. The algorithm uses a sliding window model for the most recent clustering results and data representative points to model the old data clustering results. Our experimental results on electromyography signal show a better clustering than other present in the literature

A High-Performance Parallel FDTD Method Enhanced by Using SSE Instruction Set

Directory of Open Access Journals (Sweden)

Dau-Chyrh Chang

2012-01-01

Full Text Available We introduce a hardware acceleration technique for the parallel finite difference time domain (FDTD method using the SSE (streaming (single instruction multiple data SIMD extensions instruction set. The implementation of SSE instruction set to parallel FDTD method has achieved the significant improvement on the simulation performance. The benchmarks of the SSE acceleration on both the multi-CPU workstation and computer cluster have demonstrated the advantages of (vector arithmetic logic unit VALU acceleration over GPU acceleration. Several engineering applications are employed to demonstrate the performance of parallel FDTD method enhanced by SSE instruction set.

Alignment data streams for the ATLAS inner detector

International Nuclear Information System (INIS)

Pinto, B; Amorim, A; Pereira, P; Elsing, M; Hawkings, R; Schieck, J; Garcia, S; Schaffer, A; Ma, H; Anjos, A

2008-01-01

The ATLAS experiment uses a complex trigger strategy to be able to reduce the Event Filter rate output, down to a level that allows the storage and processing of these data. These concepts are described in the ATLAS Computing Model which embraces Grid paradigm. The output coming from the Event Filter consists of four main streams: physical stream, express stream, calibration stream, and diagnostic stream. The calibration stream will be transferred to the Tier-0 facilities that will provide the prompt reconstruction of this stream with a minimum latency of 8 hours, producing calibration constants of sufficient quality to allow a first-pass processing. The Inner Detector community is developing and testing an independent common calibration stream selected at the Event Filter after track reconstruction. It is composed of raw data, in byte-stream format, contained in Readout Buffers (ROBs) with hit information of the selected tracks, and it will be used to derive and update a set of calibration and alignment constants. This option was selected because it makes use of the Byte Stream Converter infrastructure and possibly gives better bandwidth usage and storage optimization. Processing is done using specialized algorithms running in the Athena framework in dedicated Tier-0 resources, and the alignment constants will be stored and distributed using the COOL conditions database infrastructure. This work is addressing in particular the alignment requirements, the needs for track and hit selection, and the performance issues

Alignment data stream for the ATLAS inner detector

International Nuclear Information System (INIS)

Pinto, B

2010-01-01

The ATLAS experiment uses a complex trigger strategy to be able to achieve the necessary Event Filter rate output, making possible to optimize the storage and processing needs of these data. These needs are described in the ATLAS Computing Model, which embraces Grid concepts. The output coming from the Event Filter will consist of three main streams: a primary stream, the express stream and the calibration stream. The calibration stream will be transferred to the Tier-0 facilities which will allow the prompt reconstruction of this stream with an admissible latency of 8 hours, producing calibration constants of sufficient quality to permit a first-pass processing. An independent calibration stream is developed and tested, which selects tracks at the level-2 trigger (LVL2) after the reconstruction. The stream is composed of raw data, in byte-stream format, and contains only information of the relevant parts of the detector, in particular the hit information of the selected tracks. This leads to a significantly improved bandwidth usage and storage capability. The stream will be used to derive and update the calibration and alignment constants if necessary every 24h. Processing is done using specialized algorithms running in Athena framework in dedicated Tier-0 resources, and the alignment constants will be stored and distributed using the COOL conditions database infrastructure. The work is addressing in particular the alignment requirements, the needs for track and hit selection, timing and bandwidth issues.

A catchment scale evaluation of multiple stressor effects in headwater streams.

Science.gov (United States)

Rasmussen, Jes J; McKnight, Ursula S; Loinaz, Maria C; Thomsen, Nanna I; Olsson, Mikael E; Bjerg, Poul L; Binning, Philip J; Kronvang, Brian

2013-01-01

Mitigation activities to improve water quality and quantity in streams as well as stream management and restoration efforts are conducted in the European Union aiming to improve the chemical, physical and ecological status of streams. Headwater streams are often characterised by impairment of hydromorphological, chemical, and ecological conditions due to multiple anthropogenic impacts. However, they are generally disregarded as water bodies for mitigation activities in the European Water Framework Directive despite their importance for supporting a higher ecological quality in higher order streams. We studied 11 headwater streams in the Hove catchment in the Copenhagen region. All sites had substantial physical habitat and water quality impairments due to anthropogenic influence (intensive agriculture, urban settlements, contaminated sites and low base-flow due to water abstraction activities in the catchment). We aimed to identify the dominating anthropogenic stressors at the catchment scale causing ecological impairment of benthic macroinvertebrate communities and provide a rank-order of importance that could help in prioritising mitigation activities. We identified numerous chemical and hydromorphological impacts of which several were probably causing major ecological impairments, but we were unable to provide a robust rank-ordering of importance suggesting that targeted mitigation efforts on single anthropogenic stressors in the catchment are unlikely to have substantial effects on the ecological quality in these streams. The SPEcies At Risk (SPEAR) index explained most of the variability in the macroinvertebrate community structure, and notably, SPEAR index scores were often very low (<10% SPEAR abundance). An extensive re-sampling of a subset of the streams provided evidence that especially insecticides were probably essential contributors to the overall ecological impairment of these streams. Our results suggest that headwater streams should be considered in

Simulation of a two-dimensional dipolar system on a APE100/quadrics SIMD architecture

International Nuclear Information System (INIS)

Bruno, A.; Pisacane, F.; Rosato, V.

1997-01-01

The temperature behavior of a system of dipoles with long-range interactions has been simulated via a two-dimensional lattice Monte Carlo on a massively (SIMD) platform (Quadrics/APE100). Thermodynamic quantities have been evaluated in order to locate and to characterize the phase transition in absence of applied field. Emphasis is given to the code implementation on the SIMD architecture and to the relevant features which have been used to improve code capabilities and performances. The probability of simultaneous occurrence of at least k spanning clusters has been studied by Monte Carlo simulations on the 2D square lattice with free boundaries at the bond percolation threshold p c = 1/2. It is found that the probability of k and more Incipient Spanning Clusters (ISC) have the values P(k > 1) ∼ 0.00658(3) and P(k > 2) ∼ 0.00000148(21) provided that the limit of these probabilities for infinite lattices exists. The probability P(k > 3) of more than three ISC could be estimated to be of the order of 10 -11 and is beyond the possibility to compute such a value by nowadays computers. So, it is impossible to check in simulations the Aizenman law for the probabilities when k much-gt 1. We have detected a single sample with four ISC in a total number of about 1010 samples investigated. The probability of this single event is 1/10 for that number of samples. The influence of boundary conditions is discussed in the last section

Stream invertebrate productivity linked to forest subsidies: 37 stream-years of reference and experimental data

Science.gov (United States)

J. Bruce Wallace; Susan L Eggert; Judy L. Meyer; Jackson R. Webster

2015-01-01

Riparian habitats provide detrital subsidies of varying quantities and qualities to recipient ecosystems. We used long-term data from three reference streams (covering 24 stream-years) and 13-year whole-stream organic matter manipulations to investigate the influence of terrestrial detrital quantity and quality on benthic invertebrate community structure, abundance,...

Information-Theoretic Data Discarding for Dynamic Trees on Data Streams

Directory of Open Access Journals (Sweden)

Christoforos Anagnostopoulos

2013-12-01

Full Text Available Ubiquitous automated data collection at an unprecedented scale is making available streaming, real-time information flows in a wide variety of settings, transforming both science and industry. Learning algorithms deployed in such contexts often rely on single-pass inference, where the data history is never revisited. Learning may also need to be temporally adaptive to remain up-to-date against unforeseen changes in the data generating mechanism. Online Bayesian inference remains challenged by such transient, evolving data streams. Nonparametric modeling techniques can prove particularly ill-suited, as the complexity of the model is allowed to increase with the sample size. In this work, we take steps to overcome these challenges by porting information theoretic heuristics, such as exponential forgetting and active learning, into a fully Bayesian framework. We showcase our methods by augmenting a modern non-parametric modeling framework, dynamic trees, and illustrate its performance on a number of practical examples. The end product is a powerful streaming regression and classification tool, whose performance compares favorably to the state-of-the-art.

Exploring the connectome: Petascale volume visualization of microscopy data streams

KAUST Repository

Beyer, Johanna; Hadwiger, Markus; Al-Awami, Ali K.; Jeong, Wonki; Kasthuri, Narayanan; Lichtman, Jeff W M D; Pfister, Hanspeter

2013-01-01

Recent advances in high-resolution microscopy let neuroscientists acquire neural-tissue volume data of extremely large sizes. However, the tremendous resolution and the high complexity of neural structures present big challenges to storage, processing, and visualization at interactive rates. A proposed system provides interactive exploration of petascale (petavoxel) volumes resulting from high-throughput electron microscopy data streams. The system can concurrently handle multiple volumes and can support the simultaneous visualization of high-resolution voxel segmentation data. Its visualization-driven design restricts most computations to a small subset of the data. It employs a multiresolution virtual-memory architecture for better scalability than previous approaches and for handling incomplete data. Researchers have employed it for a 1-teravoxel mouse cortex volume, of which several hundred axons and dendrites as well as synapses have been segmented and labeled. © 1981-2012 IEEE.

Exploring the connectome: Petascale volume visualization of microscopy data streams

KAUST Repository

Beyer, Johanna

2013-07-01

Recent advances in high-resolution microscopy let neuroscientists acquire neural-tissue volume data of extremely large sizes. However, the tremendous resolution and the high complexity of neural structures present big challenges to storage, processing, and visualization at interactive rates. A proposed system provides interactive exploration of petascale (petavoxel) volumes resulting from high-throughput electron microscopy data streams. The system can concurrently handle multiple volumes and can support the simultaneous visualization of high-resolution voxel segmentation data. Its visualization-driven design restricts most computations to a small subset of the data. It employs a multiresolution virtual-memory architecture for better scalability than previous approaches and for handling incomplete data. Researchers have employed it for a 1-teravoxel mouse cortex volume, of which several hundred axons and dendrites as well as synapses have been segmented and labeled. © 1981-2012 IEEE.

KDE-Track: An Efficient Dynamic Density Estimator for Data Streams

KAUST Repository

Qahtan, Abdulhakim Ali Ali; Wang, Suojin; Zhang, Xiangliang

2016-01-01

Recent developments in sensors, global positioning system devices and smart phones have increased the availability of spatiotemporal data streams. Developing models for mining such streams is challenged by the huge amount of data that cannot be stored in the memory, the high arrival speed and the dynamic changes in the data distribution. Density estimation is an important technique in stream mining for a wide variety of applications. The construction of kernel density estimators is well studied and documented. However, existing techniques are either expensive or inaccurate and unable to capture the changes in the data distribution. In this paper, we present a method called KDE-Track to estimate the density of spatiotemporal data streams. KDE-Track can efficiently estimate the density function with linear time complexity using interpolation on a kernel model, which is incrementally updated upon the arrival of new samples from the stream. We also propose an accurate and efficient method for selecting the bandwidth value for the kernel density estimator, which increases its accuracy significantly. Both theoretical analysis and experimental validation show that KDE-Track outperforms a set of baseline methods on the estimation accuracy and computing time of complex density structures in data streams.

KDE-Track: An Efficient Dynamic Density Estimator for Data Streams

KAUST Repository

Qahtan, Abdulhakim Ali Ali

2016-11-08

Recent developments in sensors, global positioning system devices and smart phones have increased the availability of spatiotemporal data streams. Developing models for mining such streams is challenged by the huge amount of data that cannot be stored in the memory, the high arrival speed and the dynamic changes in the data distribution. Density estimation is an important technique in stream mining for a wide variety of applications. The construction of kernel density estimators is well studied and documented. However, existing techniques are either expensive or inaccurate and unable to capture the changes in the data distribution. In this paper, we present a method called KDE-Track to estimate the density of spatiotemporal data streams. KDE-Track can efficiently estimate the density function with linear time complexity using interpolation on a kernel model, which is incrementally updated upon the arrival of new samples from the stream. We also propose an accurate and efficient method for selecting the bandwidth value for the kernel density estimator, which increases its accuracy significantly. Both theoretical analysis and experimental validation show that KDE-Track outperforms a set of baseline methods on the estimation accuracy and computing time of complex density structures in data streams.

Biotic interactions modify multiple-stressor effects on juvenile brown trout in an experimental stream food web.

Science.gov (United States)

Bruder, Andreas; Salis, Romana K; Jones, Peter E; Matthaei, Christoph D

2017-09-01

Agricultural land use results in multiple stressors affecting stream ecosystems. Flow reduction due to water abstraction, elevated levels of nutrients and chemical contaminants are common agricultural stressors worldwide. Concurrently, stream ecosystems are also increasingly affected by climate change. Interactions among multiple co-occurring stressors result in biological responses that cannot be predicted from single-stressor effects (i.e. synergisms and antagonisms). At the ecosystem level, multiple-stressor effects can be further modified by biotic interactions (e.g. trophic interactions). We conducted a field experiment using 128 flow-through stream mesocosms to examine the individual and combined effects of water abstraction, nutrient enrichment and elevated levels of the nitrification inhibitor dicyandiamide (DCD) on survival, condition and gut content of juvenile brown trout and on benthic abundance of their invertebrate prey. Flow velocity reduction decreased fish survival (-12% compared to controls) and condition (-8% compared to initial condition), whereas effects of nutrient and DCD additions and interactions among these stressors were not significant. Negative effects of flow velocity reduction on fish survival and condition were consistent with effects on fish gut content (-25% compared to controls) and abundance of dominant invertebrate prey (-30% compared to controls), suggesting a negative metabolic balance driving fish mortality and condition decline, which was confirmed by structural equation modelling. Fish mortality under reduced flow velocity increased as maximal daily water temperatures approached the upper limit of their tolerance range, reflecting synergistic interactions between these stressors. Our study highlights the importance of indirect stressor effects such as those transferred through trophic interactions, which need to be considered when assessing and managing fish populations and stream food webs in multiple-stressor situations

A Streaming PCA VLSI Chip for Neural Data Compression.

Science.gov (United States)

Wu, Tong; Zhao, Wenfeng; Guo, Hongsun; Lim, Hubert H; Yang, Zhi

2017-12-01

Neural recording system miniaturization and integration with low-power wireless technologies require compressing neural data before transmission. Feature extraction is a procedure to represent data in a low-dimensional space; its integration into a recording chip can be an efficient approach to compress neural data. In this paper, we propose a streaming principal component analysis algorithm and its microchip implementation to compress multichannel local field potential (LFP) and spike data. The circuits have been designed in a 65-nm CMOS technology and occupy a silicon area of 0.06 mm. Throughout the experiments, the chip compresses LFPs by 10 at the expense of as low as 1% reconstruction errors and 144-nW/channel power consumption; for spikes, the achieved compression ratio is 25 with 8% reconstruction errors and 3.05-W/channel power consumption. In addition, the algorithm and its hardware architecture can swiftly adapt to nonstationary spiking activities, which enables efficient hardware sharing among multiple channels to support a high-channel count recorder.

Change Semantic Constrained Online Data Cleaning Method for Real-Time Observational Data Stream

Science.gov (United States)

Ding, Yulin; Lin, Hui; Li, Rongrong

2016-06-01

Recent breakthroughs in sensor networks have made it possible to collect and assemble increasing amounts of real-time observational data by observing dynamic phenomena at previously impossible time and space scales. Real-time observational data streams present potentially profound opportunities for real-time applications in disaster mitigation and emergency response, by providing accurate and timeliness estimates of environment's status. However, the data are always subject to inevitable anomalies (including errors and anomalous changes/events) caused by various effects produced by the environment they are monitoring. The "big but dirty" real-time observational data streams can rarely achieve their full potential in the following real-time models or applications due to the low data quality. Therefore, timely and meaningful online data cleaning is a necessary pre-requisite step to ensure the quality, reliability, and timeliness of the real-time observational data. In general, a straightforward streaming data cleaning approach, is to define various types of models/classifiers representing normal behavior of sensor data streams and then declare any deviation from this model as normal or erroneous data. The effectiveness of these models is affected by dynamic changes of deployed environments. Due to the changing nature of the complicated process being observed, real-time observational data is characterized by diversity and dynamic, showing a typical Big (Geo) Data characters. Dynamics and diversity is not only reflected in the data values, but also reflected in the complicated changing patterns of the data distributions. This means the pattern of the real-time observational data distribution is not stationary or static but changing and dynamic. After the data pattern changed, it is necessary to adapt the model over time to cope with the changing patterns of real-time data streams. Otherwise, the model will not fit the following observational data streams, which may led

CHANGE SEMANTIC CONSTRAINED ONLINE DATA CLEANING METHOD FOR REAL-TIME OBSERVATIONAL DATA STREAM

Directory of Open Access Journals (Sweden)

Y. Ding

2016-06-01

Full Text Available Recent breakthroughs in sensor networks have made it possible to collect and assemble increasing amounts of real-time observational data by observing dynamic phenomena at previously impossible time and space scales. Real-time observational data streams present potentially profound opportunities for real-time applications in disaster mitigation and emergency response, by providing accurate and timeliness estimates of environment’s status. However, the data are always subject to inevitable anomalies (including errors and anomalous changes/events caused by various effects produced by the environment they are monitoring. The “big but dirty” real-time observational data streams can rarely achieve their full potential in the following real-time models or applications due to the low data quality. Therefore, timely and meaningful online data cleaning is a necessary pre-requisite step to ensure the quality, reliability, and timeliness of the real-time observational data. In general, a straightforward streaming data cleaning approach, is to define various types of models/classifiers representing normal behavior of sensor data streams and then declare any deviation from this model as normal or erroneous data. The effectiveness of these models is affected by dynamic changes of deployed environments. Due to the changing nature of the complicated process being observed, real-time observational data is characterized by diversity and dynamic, showing a typical Big (Geo Data characters. Dynamics and diversity is not only reflected in the data values, but also reflected in the complicated changing patterns of the data distributions. This means the pattern of the real-time observational data distribution is not stationary or static but changing and dynamic. After the data pattern changed, it is necessary to adapt the model over time to cope with the changing patterns of real-time data streams. Otherwise, the model will not fit the following observational

Applications of spatial statistical network models to stream data

Science.gov (United States)

Daniel J. Isaak; Erin E. Peterson; Jay M. Ver Hoef; Seth J. Wenger; Jeffrey A. Falke; Christian E. Torgersen; Colin Sowder; E. Ashley Steel; Marie-Josee Fortin; Chris E. Jordan; Aaron S. Ruesch; Nicholas Som; Pascal. Monestiez

2014-01-01

Streams and rivers host a significant portion of Earth's biodiversity and provide important ecosystem services for human populations. Accurate information regarding the status and trends of stream resources is vital for their effective conservation and management. Most statistical techniques applied to data measured on stream networks were developed for...

A framework to preserve the privacy of electronic health data streams.

Science.gov (United States)

Kim, Soohyung; Sung, Min Kyoung; Chung, Yon Dohn

2014-08-01

The anonymization of health data streams is important to protect these data against potential privacy breaches. A large number of research studies aiming at offering privacy in the context of data streams has been recently conducted. However, the techniques that have been proposed in these studies generate a significant delay during the anonymization process, since they concentrate on applying existing privacy models (e.g., k-anonymity and l-diversity) to batches of data extracted from data streams in a period of time. In this paper, we present delay-free anonymization, a framework for preserving the privacy of electronic health data streams. Unlike existing works, our method does not generate an accumulation delay, since input streams are anonymized immediately with counterfeit values. We further devise late validation for increasing the data utility of the anonymization results and managing the counterfeit values. Through experiments, we show the efficiency and effectiveness of the proposed method for the real-time release of data streams. Copyright © 2014 Elsevier Inc. All rights reserved.

A SWAT model validation of nested-scale contemporaneous stream flow, suspended sediment and nutrients from a multiple-land-use watershed of the central USA.

Science.gov (United States)

Zeiger, Sean J; Hubbart, Jason A

2016-12-01

There is an ongoing need to validate the accuracy of predictive model simulated pollutant yields, particularly from multiple-land-use (i.e. forested, agricultural, and urban) watersheds. However, there are seldom sufficient observed data sets available that supply requisite spatial and temporal resolution and coupled multi-parameter constituents for rigorous model performance assessment. Four years of hydroclimate and water quality data were used to validate SWAT model estimates of monthly stream flow, suspended sediment, total phosphorus, nitrate, nitrite, ammonium, and total inorganic nitrogen from 5 nested-scale gauging sites located in a multiple-land-use watershed of the central USA. The uncalibrated SWAT model satisfactorily simulated monthly stream flow with Nash-Sutcliffe efficiency (NSE) values ranging from 0.50 near the headwaters, to 0.75 near the watershed outlet. However, the uncalibrated model did not accurately simulate monthly sediment, total phosphorus, nitrate, nitrite, ammonium, and total inorganic nitrogen with NSE valuesSWAT model to multiple gauging sites within the watershed improved estimates of monthly stream flow (NSE=0.83), sediment (NSE=0.78), total phosphorus (NSE=0.81), nitrate (NSE=0.90), and total inorganic nitrogen (NSE=0.86). However, NSE values were model performance decreased for sediment, nitrate, and total inorganic nitrogen during the validation period with NSE valuesSWAT model to multiple gauging sites and provide guidance to SWAT model (or similar models) users wishing to improve model performance at multiple scales. Copyright © 2016 Elsevier B.V. All rights reserved.

Fast algorithm for automatically computing Strahler stream order

Science.gov (United States)

Lanfear, Kenneth J.

1990-01-01

An efficient algorithm was developed to determine Strahler stream order for segments of stream networks represented in a Geographic Information System (GIS). The algorithm correctly assigns Strahler stream order in topologically complex situations such as braided streams and multiple drainage outlets. Execution time varies nearly linearly with the number of stream segments in the network. This technique is expected to be particularly useful for studying the topology of dense stream networks derived from digital elevation model data.

Elastic execution of continuous mapreduce jobs over data streams

DEFF Research Database (Denmark)

2015-01-01

There is provided a set of methods describing how to elastically change the resources used by a MapReduce job on streaming data while executing......There is provided a set of methods describing how to elastically change the resources used by a MapReduce job on streaming data while executing...

Efficient Estimation of Dynamic Density Functions with Applications in Streaming Data

KAUST Repository

Qahtan, Abdulhakim

2016-05-11

Recent advances in computing technology allow for collecting vast amount of data that arrive continuously in the form of streams. Mining data streams is challenged by the speed and volume of the arriving data. Furthermore, the underlying distribution of the data changes over the time in unpredicted scenarios. To reduce the computational cost, data streams are often studied in forms of condensed representation, e.g., Probability Density Function (PDF). This thesis aims at developing an online density estimator that builds a model called KDE-Track for characterizing the dynamic density of the data streams. KDE-Track estimates the PDF of the stream at a set of resampling points and uses interpolation to estimate the density at any given point. To reduce the interpolation error and computational complexity, we introduce adaptive resampling where more/less resampling points are used in high/low curved regions of the PDF. The PDF values at the resampling points are updated online to provide up-to-date model of the data stream. Comparing with other existing online density estimators, KDE-Track is often more accurate (as reflected by smaller error values) and more computationally efficient (as reflected by shorter running time). The anytime available PDF estimated by KDE-Track can be applied for visualizing the dynamic density of data streams, outlier detection and change detection in data streams. In this thesis work, the first application is to visualize the taxi traffic volume in New York city. Utilizing KDE-Track allows for visualizing and monitoring the traffic flow on real time without extra overhead and provides insight analysis of the pick up demand that can be utilized by service providers to improve service availability. The second application is to detect outliers in data streams from sensor networks based on the estimated PDF. The method detects outliers accurately and outperforms baseline methods designed for detecting and cleaning outliers in sensor data. The

Stream invertebrate productivity linked to forest subsidies: 37 stream-years of reference and experimental data.

Science.gov (United States)

Wallace, J Bruce; Eggert, Susan L; Meyer, Judy L; Webster, Jackson R

2015-05-01

Riparian habitats provide detrital subsidies of varying quantities and qualities to recipient ecosystems. We used long-term data from three reference streams (covering 24 stream-years) and 13-year whole-stream organic matter manipulations to investigate the influence of terrestrial detrital quantity and quality on benthic invertebrate community structure, abundance, biomass, and secondary production in rockface (RF) and mixed substrates (MS) of forested headwater streams. Using a mesh canopy covering the entire treatment stream, we examined effects of litter ex'clusion, small- and large-wood removal, and addition of artificial wood (PVC) and leaves of varying quality on organic matter standing crops and invertebrate community structure and function. We assessed differences in functional feeding group distribution between substrate types as influenced by organic matter manipulations and long-term patterns of predator and prey production in manipulated vs. reference years. Particulate organic matter standing crops in MS of the treatment stream declined drastically with each successive year of litter exclusion, approaching zero after three years. Monthly invertebrate biomass and annual secondary production was positively related to benthic organic matter in the MS habitats. Rockface habitats exhibited fewer changes than MS habitats across all organic matter manipulations. With leaf addition, the patterns of functional group distribution among MS and RF habitats returned to patterns seen in reference streams. Secondary production per unit organic matter standing crop was greatest for the leaf addition period, followed by the reference streams, and significantly less for the litter exclusion and wood removal periods. These data indicate that the limited organic matter remaining in the stream following litter exclusion and wood removal was more refractory than that in the reference streams, whereas the added leaf material was more labile and readily converted into

Developing a national stream morphology data exchange: Needs, challenges, and opportunities

Science.gov (United States)

Collins, Mathias J.; Gray, John R.; Peppler, Marie C.; Fitzpatrick, Faith A.; Schubauer-Berigan, Joseph P.

2012-05-01

Stream morphology data, primarily consisting of channel and foodplain geometry and bed material size measurements, historically have had a wide range of applications and uses including culvert/ bridge design, rainfall- runoff modeling, food inundation mapping (e.g., U.S. Federal Emergency Management Agency food insurance studies), climate change studies, channel stability/sediment source investigations, navigation studies, habitat assessments, and landscape change research. The need for stream morphology data in the United States, and thus the quantity of data collected, has grown substantially over the past 2 decades because of the expanded interests of resource management agencies in watershed management and restoration. The quantity of stream morphology data collected has also increased because of state-of-the-art technologies capable of rapidly collecting high-resolution data over large areas with heretofore unprecedented precision. Despite increasing needs for and the expanding quantity of stream morphology data, neither common reporting standards nor a central data archive exist for storing and serving these often large and spatially complex data sets. We are proposing an open- access data exchange for archiving and disseminating stream morphology data.

Monitoring forest cover loss using multiple data streams, a case study of a tropical dry forest in Bolivia

Science.gov (United States)

Dutrieux, Loïc Paul; Verbesselt, Jan; Kooistra, Lammert; Herold, Martin

2015-09-01

Automatically detecting forest disturbances as they occur can be extremely challenging for certain types of environments, particularly those presenting strong natural variations. Here, we use a generic structural break detection framework (BFAST) to improve the monitoring of forest cover loss by combining multiple data streams. Forest change monitoring is performed using Landsat data in combination with MODIS or rainfall data to further improve the modelling and monitoring. We tested the use of the Normalized Difference Vegetation Index (NDVI) from the Moderate Resolution Imaging Spectroradiometer (MODIS) with varying spatial aggregation window sizes as well as a rainfall derived index as external regressors. The method was evaluated on a dry tropical forest area in lowland Bolivia where forest cover loss is known to occur, and we validated the results against a set of ground truth samples manually interpreted using the TimeSync environment. We found that the addition of an external regressor allows to take advantage of the difference in spatial extent between human induced and naturally induced variations and only detect the processes of interest. Of all configurations, we found the 13 by 13 km MODIS NDVI window to be the most successful, with an overall accuracy of 87%. Compared with a single pixel approach, the proposed method produced better time-series model fits resulting in increases of overall accuracy (from 82% to 87%), and decrease in omission and commission errors (from 33% to 24% and from 3% to 0% respectively). The presented approach seems particularly relevant for areas with high inter-annual natural variability, such as forests regularly experiencing exceptional drought events.

Frequent Pairs in Data Streams: Exploiting Parallelism and Skew

DEFF Research Database (Denmark)

Campagna, Andrea; Kutzkow, Konstantin; Pagh, Rasmus

2011-01-01

We introduce the Pair Streaming Engine (PairSE) that detects frequent pairs in a data stream of transactions. Our algorithm finds the most frequent pairs with high probability, and gives tight bounds on their frequency. It is particularly space efficient for skewed distribution of pair supports...... items mining in data streams. We show how to efficiently scale these approaches to handle large transactions. We report experimental results showcasing precision and recall of our method. In particular, we find that often our method achieves excellent precision, returning identical upper and lower...... bounds on the supports of the most frequent pairs....

Self-adaptive change detection in streaming data with non-stationary distribution

KAUST Repository

Zhang, Xiangliang

2010-01-01

Non-stationary distribution, in which the data distribution evolves over time, is a common issue in many application fields, e.g., intrusion detection and grid computing. Detecting the changes in massive streaming data with a non-stationary distribution helps to alarm the anomalies, to clean the noises, and to report the new patterns. In this paper, we employ a novel approach for detecting changes in streaming data with the purpose of improving the quality of modeling the data streams. Through observing the outliers, this approach of change detection uses a weighted standard deviation to monitor the evolution of the distribution of data streams. A cumulative statistical test, Page-Hinkley, is employed to collect the evidence of changes in distribution. The parameter used for reporting the changes is self-adaptively adjusted according to the distribution of data streams, rather than set by a fixed empirical value. The self-adaptability of the novel approach enhances the effectiveness of modeling data streams by timely catching the changes of distributions. We validated the approach on an online clustering framework with a benchmark KDDcup 1999 intrusion detection data set as well as with a real-world grid data set. The validation results demonstrate its better performance on achieving higher accuracy and lower percentage of outliers comparing to the other change detection approaches. © 2010 Springer-Verlag.

Analysis of sound data streamed over the network

Directory of Open Access Journals (Sweden)

Jiří Fejfar

2013-01-01

Full Text Available In this paper we inspect a difference between original sound recording and signal captured after streaming this original recording over a network loaded with a heavy traffic. There are several kinds of failures occurring in the captured recording caused by network congestion. We try to find a method how to evaluate correctness of streamed audio. Usually there are metrics based on a human perception of a signal such as “signal is clear, without audible failures”, “signal is having some failures but it is understandable”, or “signal is inarticulate”. These approaches need to be statistically evaluated on a broad set of respondents, which is time and resource consuming. We try to propose some metrics based on signal properties allowing us to compare the original and captured recording. We use algorithm called Dynamic Time Warping (Müller, 2007 commonly used for time series comparison in this paper. Some other time series exploration approaches can be found in (Fejfar, 2011 and (Fejfar, 2012. The data was acquired in our network laboratory simulating network traffic by downloading files, streaming audio and video simultaneously. Our former experiment inspected Quality of Service (QoS and its impact on failures of received audio data stream. This experiment is focused on the comparison of sound recordings rather than network mechanism.We focus, in this paper, on a real time audio stream such as a telephone call, where it is not possible to stream audio in advance to a “pool”. Instead it is necessary to achieve as small delay as possible (between speaker voice recording and listener voice replay. We are using RTP protocol for streaming audio.

A Quality-Centric Data Model for Distributed Stream Management Systems

OpenAIRE

Pietzuch, P; Fiscato, M; Vu, QH

2009-01-01

21.10.14 KB Ok to add published version to spiral. It is challenging for large-scale stream management systems to return always perfect results when processing data streams originating from distributed sources. Data sources and intermediate processing nodes may fail during the lifetime of a stream query. In addition, individual nodes may become overloaded due to processing demands. In practice, users have to accept incomplete or inaccurate query results because of failure or overload. In t...

Ten years of multiple data stream assimilation with the ORCHIDEE land surface model to improve regional to global simulated carbon budgets: synthesis and perspectives on directions for the future

Science.gov (United States)

Peylin, P. P.; Bacour, C.; MacBean, N.; Maignan, F.; Bastrikov, V.; Chevallier, F.

2017-12-01

Predicting the fate of carbon stocks and their sensitivity to climate change and land use/management strongly relies on our ability to accurately model net and gross carbon fluxes. However, simulated carbon and water fluxes remain subject to large uncertainties, partly because of unknown or poorly calibrated parameters. Over the past ten years, the carbon cycle data assimilation system at the Laboratoire des Sciences du Climat et de l'Environnement has investigated the benefit of assimilating multiple carbon cycle data streams into the ORCHIDEE LSM, the land surface component of the Institut Pierre Simon Laplace Earth System Model. These datasets have included FLUXNET eddy covariance data (net CO2 flux and latent heat flux) to constrain hourly to seasonal time-scale carbon cycle processes, remote sensing of the vegetation activity (MODIS NDVI) to constrain the leaf phenology, biomass data to constrain "slow" (yearly to decadal) processes of carbon allocation, and atmospheric CO2 concentrations to provide overall large scale constraints on the land carbon sink. Furthermore, we have investigated technical issues related to multiple data stream assimilation and choice of optimization algorithm. This has provided a wide-ranging perspective on the challenges we face in constraining model parameters and thus better quantifying, and reducing, model uncertainty in projections of the future global carbon sink. We review our past studies in terms of the impact of the optimization on key characteristics of the carbon cycle, e.g. the partition of the northern latitudes vs tropical land carbon sink, and compare to the classic atmospheric flux inversion approach. Throughout, we discuss our work in context of the abovementioned challenges, and propose solutions for the community going forward, including the potential of new observations such as atmospheric COS concentrations and satellite-derived Solar Induced Fluorescence to constrain the gross carbon fluxes of the ORCHIDEE

Clustering big data streams : recent challenges and contributions

NARCIS (Netherlands)

Hassani, M.; Seidl, T.

Traditional clustering algorithms merely considered static data. Today's various applications and research issues in big data mining have however to deal with continuous, possibly infinite streams of data, arriving at high velocity. Web traffic data, surveillance data, sensor measurements and stock

CUDASW++2.0: enhanced Smith-Waterman protein database search on CUDA-enabled GPUs based on SIMT and virtualized SIMD abstractions

Directory of Open Access Journals (Sweden)

Schmidt Bertil

2010-04-01

Full Text Available Abstract Background Due to its high sensitivity, the Smith-Waterman algorithm is widely used for biological database searches. Unfortunately, the quadratic time complexity of this algorithm makes it highly time-consuming. The exponential growth of biological databases further deteriorates the situation. To accelerate this algorithm, many efforts have been made to develop techniques in high performance architectures, especially the recently emerging many-core architectures and their associated programming models. Findings This paper describes the latest release of the CUDASW++ software, CUDASW++ 2.0, which makes new contributions to Smith-Waterman protein database searches using compute unified device architecture (CUDA. A parallel Smith-Waterman algorithm is proposed to further optimize the performance of CUDASW++ 1.0 based on the single instruction, multiple thread (SIMT abstraction. For the first time, we have investigated a partitioned vectorized Smith-Waterman algorithm using CUDA based on the virtualized single instruction, multiple data (SIMD abstraction. The optimized SIMT and the partitioned vectorized algorithms were benchmarked, and remarkably, have similar performance characteristics. CUDASW++ 2.0 achieves performance improvement over CUDASW++ 1.0 as much as 1.74 (1.72 times using the optimized SIMT algorithm and up to 1.77 (1.66 times using the partitioned vectorized algorithm, with a performance of up to 17 (30 billion cells update per second (GCUPS on a single-GPU GeForce GTX 280 (dual-GPU GeForce GTX 295 graphics card. Conclusions CUDASW++ 2.0 is publicly available open-source software, written in CUDA and C++ programming languages. It obtains significant performance improvement over CUDASW++ 1.0 using either the optimized SIMT algorithm or the partitioned vectorized algorithm for Smith-Waterman protein database searches by fully exploiting the compute capability of commonly used CUDA-enabled low-cost GPUs.

Stream Kriging: Incremental and recursive ordinary Kriging over spatiotemporal data streams

Science.gov (United States)

Zhong, Xu; Kealy, Allison; Duckham, Matt

2016-05-01

Ordinary Kriging is widely used for geospatial interpolation and estimation. Due to the O (n3) time complexity of solving the system of linear equations, ordinary Kriging for a large set of source points is computationally intensive. Conducting real-time Kriging interpolation over continuously varying spatiotemporal data streams can therefore be especially challenging. This paper develops and tests two new strategies for improving the performance of an ordinary Kriging interpolator adapted to a stream-processing environment. These strategies rely on the expectation that, over time, source data points will frequently refer to the same spatial locations (for example, where static sensor nodes are generating repeated observations of a dynamic field). First, an incremental strategy improves efficiency in cases where a relatively small proportion of previously processed spatial locations are absent from the source points at any given iteration. Second, a recursive strategy improves efficiency in cases where there is substantial set overlap between the sets of spatial locations of source points at the current and previous iterations. These two strategies are evaluated in terms of their computational efficiency in comparison to ordinary Kriging algorithm. The results show that these two strategies can reduce the time taken to perform the interpolation by up to 90%, and approach average-case time complexity of O (n2) when most but not all source points refer to the same locations over time. By combining the approaches developed in this paper with existing heuristic ordinary Kriging algorithms, the conclusions indicate how further efficiency gains could potentially be accrued. The work ultimately contributes to the development of online ordinary Kriging interpolation algorithms, capable of real-time spatial interpolation with large streaming data sets.

rEMM: Extensible Markov Model for Data Stream Clustering in R

Directory of Open Access Journals (Sweden)

Michael Hahsler

2010-10-01

Full Text Available Clustering streams of continuously arriving data has become an important application of data mining in recent years and efficient algorithms have been proposed by several researchers. However, clustering alone neglects the fact that data in a data stream is not only characterized by the proximity of data points which is used by clustering, but also by a temporal component. The extensible Markov model (EMM adds the temporal component to data stream clustering by superimposing a dynamically adapting Markov chain. In this paper we introduce the implementation of the R extension package rEMM which implements EMM and we discuss some examples and applications.

Sampling the stream landscape: Improving the applicability of an ecoregion-level capture probability model for stream fishes

Science.gov (United States)

Mollenhauer, Robert; Mouser, Joshua B.; Brewer, Shannon K.

2018-01-01

Temporal and spatial variability in streams result in heterogeneous gear capture probability (i.e., the proportion of available individuals identified) that confounds interpretation of data used to monitor fish abundance. We modeled tow-barge electrofishing capture probability at multiple spatial scales for nine Ozark Highland stream fishes. In addition to fish size, we identified seven reach-scale environmental characteristics associated with variable capture probability: stream discharge, water depth, conductivity, water clarity, emergent vegetation, wetted width–depth ratio, and proportion of riffle habitat. The magnitude of the relationship between capture probability and both discharge and depth varied among stream fishes. We also identified lithological characteristics among stream segments as a coarse-scale source of variable capture probability. The resulting capture probability model can be used to adjust catch data and derive reach-scale absolute abundance estimates across a wide range of sampling conditions with similar effort as used in more traditional fisheries surveys (i.e., catch per unit effort). Adjusting catch data based on variable capture probability improves the comparability of data sets, thus promoting both well-informed conservation and management decisions and advances in stream-fish ecology.

A data-stream classification system for investigating terrorist threats

Science.gov (United States)

Schulz, Alexia; Dettman, Joshua; Gottschalk, Jeffrey; Kotson, Michael; Vuksani, Era; Yu, Tamara

2016-05-01

The role of cyber forensics in criminal investigations has greatly increased in recent years due to the wealth of data that is collected and available to investigators. Physical forensics has also experienced a data volume and fidelity revolution due to advances in methods for DNA and trace evidence analysis. Key to extracting insight is the ability to correlate across multi-modal data, which depends critically on identifying a touch-point connecting the separate data streams. Separate data sources may be connected because they refer to the same individual, entity or event. In this paper we present a data source classification system tailored to facilitate the investigation of potential terrorist activity. This taxonomy is structured to illuminate the defining characteristics of a particular terrorist effort and designed to guide reporting to decision makers that is complete, concise, and evidence-based. The classification system has been validated and empirically utilized in the forensic analysis of a simulated terrorist activity. Next-generation analysts can use this schema to label and correlate across existing data streams, assess which critical information may be missing from the data, and identify options for collecting additional data streams to fill information gaps.

New Splitting Criteria for Decision Trees in Stationary Data Streams.

Science.gov (United States)

Jaworski, Maciej; Duda, Piotr; Rutkowski, Leszek; Jaworski, Maciej; Duda, Piotr; Rutkowski, Leszek; Rutkowski, Leszek; Duda, Piotr; Jaworski, Maciej

2018-06-01

The most popular tools for stream data mining are based on decision trees. In previous 15 years, all designed methods, headed by the very fast decision tree algorithm, relayed on Hoeffding's inequality and hundreds of researchers followed this scheme. Recently, we have demonstrated that although the Hoeffding decision trees are an effective tool for dealing with stream data, they are a purely heuristic procedure; for example, classical decision trees such as ID3 or CART cannot be adopted to data stream mining using Hoeffding's inequality. Therefore, there is an urgent need to develop new algorithms, which are both mathematically justified and characterized by good performance. In this paper, we address this problem by developing a family of new splitting criteria for classification in stationary data streams and investigating their probabilistic properties. The new criteria, derived using appropriate statistical tools, are based on the misclassification error and the Gini index impurity measures. The general division of splitting criteria into two types is proposed. Attributes chosen based on type- splitting criteria guarantee, with high probability, the highest expected value of split measure. Type- criteria ensure that the chosen attribute is the same, with high probability, as it would be chosen based on the whole infinite data stream. Moreover, in this paper, two hybrid splitting criteria are proposed, which are the combinations of single criteria based on the misclassification error and Gini index.

An efficient reversible privacy-preserving data mining technology over data streams.

Science.gov (United States)

Lin, Chen-Yi; Kao, Yuan-Hung; Lee, Wei-Bin; Chen, Rong-Chang

2016-01-01

With the popularity of smart handheld devices and the emergence of cloud computing, users and companies can save various data, which may contain private data, to the cloud. Topics relating to data security have therefore received much attention. This study focuses on data stream environments and uses the concept of a sliding window to design a reversible privacy-preserving technology to process continuous data in real time, known as a continuous reversible privacy-preserving (CRP) algorithm. Data with CRP algorithm protection can be accurately recovered through a data recovery process. In addition, by using an embedded watermark, the integrity of the data can be verified. The results from the experiments show that, compared to existing algorithms, CRP is better at preserving knowledge and is more effective in terms of reducing information loss and privacy disclosure risk. In addition, it takes far less time for CRP to process continuous data than existing algorithms. As a result, CRP is confirmed as suitable for data stream environments and fulfills the requirements of being lightweight and energy-efficient for smart handheld devices.

Encrypted data stream identification using randomness sparse representation and fuzzy Gaussian mixture model

Science.gov (United States)

Zhang, Hong; Hou, Rui; Yi, Lei; Meng, Juan; Pan, Zhisong; Zhou, Yuhuan

2016-07-01

The accurate identification of encrypted data stream helps to regulate illegal data, detect network attacks and protect users' information. In this paper, a novel encrypted data stream identification algorithm is introduced. The proposed method is based on randomness characteristics of encrypted data stream. We use a l1-norm regularized logistic regression to improve sparse representation of randomness features and Fuzzy Gaussian Mixture Model (FGMM) to improve identification accuracy. Experimental results demonstrate that the method can be adopted as an effective technique for encrypted data stream identification.

Learning from medical data streams: an introduction

NARCIS (Netherlands)

Pereira Rodrigues, P.; Pechenizkiy, M.; Gaber, M.M.; Gama, J.

2011-01-01

Clinical practice and research are facing a new challenge created by the rapid growth of health information science and technology, and the complexity and volume of biomedical data. Machine learning from medical data streams is a recent area of research that aims to provide better knowledge

National Aquatic Resource Survey Rivers and Streams Data

Data.gov (United States)

U.S. Environmental Protection Agency — Data are from 1,000 river and stream sites across the conterminous US where consistent biological, chemical, physical and watershed data were gathered. The sites...

Local correlation detection with linearity enhancement in streaming data

KAUST Repository

Xie, Qing

2013-01-01

This paper addresses the challenges in detecting the potential correlation between numerical data streams, which facilitates the research of data stream mining and pattern discovery. We focus on local correlation with delay, which may occur in burst at different time in different streams, and last for a limited period. The uncertainty on the correlation occurrence and the time delay make it diff cult to monitor the correlation online. Furthermore, the conventional correlation measure lacks the ability of ref ecting visual linearity, which is more desirable in reality. This paper proposes effective methods to continuously detect the correlation between data streams. Our approach is based on the Discrete Fourier Transform to make rapid cross-correlation calculation with time delay allowed. In addition, we introduce a shape-based similarity measure into the framework, which ref nes the results by representative trend patterns to enhance the signif cance of linearity. The similarity of proposed linear representations can quickly estimate the correlation, and the window sliding strategy in segment level improves the eff ciency for online detection. The empirical study demonstrates the accuracy of our detection approach, as well as more than 30% improvement of eff ciency. Copyright 2013 ACM.

Real-Time Clinical Decision Support System with Data Stream Mining

Directory of Open Access Journals (Sweden)

Yang Zhang

2012-01-01

Full Text Available This research aims to describe a new design of data stream mining system that can analyze medical data stream and make real-time prediction. The motivation of the research is due to a growing concern of combining software technology and medical functions for the development of software application that can be used in medical field of chronic disease prognosis and diagnosis, children healthcare, diabetes diagnosis, and so forth. Most of the existing software technologies are case-based data mining systems. They only can analyze finite and structured data set and can only work well in their early years and can hardly meet today's medical requirement. In this paper, we describe a clinical-support-system based data stream mining technology; the design has taken into account all the shortcomings of the existing clinical support systems.

Real-time analytics techniques to analyze and visualize streaming data

CERN Document Server

Ellis, Byron

2014-01-01

Construct a robust end-to-end solution for analyzing and visualizing streaming data Real-time analytics is the hottest topic in data analytics today. In Real-Time Analytics: Techniques to Analyze and Visualize Streaming Data, expert Byron Ellis teaches data analysts technologies to build an effective real-time analytics platform. This platform can then be used to make sense of the constantly changing data that is beginning to outpace traditional batch-based analysis platforms. The author is among a very few leading experts in the field. He has a prestigious background in research, development,

Mutual Information Based Dynamic Integration of Multiple Feature Streams for Robust Real-Time LVCSR

Science.gov (United States)

Sato, Shoei; Kobayashi, Akio; Onoe, Kazuo; Homma, Shinichi; Imai, Toru; Takagi, Tohru; Kobayashi, Tetsunori

We present a novel method of integrating the likelihoods of multiple feature streams, representing different acoustic aspects, for robust speech recognition. The integration algorithm dynamically calculates a frame-wise stream weight so that a higher weight is given to a stream that is robust to a variety of noisy environments or speaking styles. Such a robust stream is expected to show discriminative ability. A conventional method proposed for the recognition of spoken digits calculates the weights front the entropy of the whole set of HMM states. This paper extends the dynamic weighting to a real-time large-vocabulary continuous speech recognition (LVCSR) system. The proposed weight is calculated in real-time from mutual information between an input stream and active HMM states in a searchs pace without an additional likelihood calculation. Furthermore, the mutual information takes the width of the search space into account by calculating the marginal entropy from the number of active states. In this paper, we integrate three features that are extracted through auditory filters by taking into account the human auditory system's ability to extract amplitude and frequency modulations. Due to this, features representing energy, amplitude drift, and resonant frequency drifts, are integrated. These features are expected to provide complementary clues for speech recognition. Speech recognition experiments on field reports and spontaneous commentary from Japanese broadcast news showed that the proposed method reduced error words by 9.2% in field reports and 4.7% in spontaneous commentaries relative to the best result obtained from a single stream.

A PCA-Based Change Detection Framework for Multidimensional Data Streams

KAUST Repository

Qahtan, Abdulhakim Ali Ali

2015-08-10

Detecting changes in multidimensional data streams is an important and challenging task. In unsupervised change detection, changes are usually detected by comparing the distribution in a current (test) window with a reference window. It is thus essential to design divergence metrics and density estimators for comparing the data distributions, which are mostly done for univariate data. Detecting changes in multidimensional data streams brings difficulties to the density estimation and comparisons. In this paper, we propose a framework for detecting changes in multidimensional data streams based on principal component analysis, which is used for projecting data into a lower dimensional space, thus facilitating density estimation and change-score calculations. The proposed framework also has advantages over existing approaches by reducing computational costs with an efficient density estimator, promoting the change-score calculation by introducing effective divergence metrics, and by minimizing the efforts required from users on the threshold parameter setting by using the Page-Hinkley test. The evaluation results on synthetic and real data show that our framework outperforms two baseline methods in terms of both detection accuracy and computational costs.

FACT. Streamed data analysis and online application of machine learning models

Energy Technology Data Exchange (ETDEWEB)

Bruegge, Kai Arno; Buss, Jens [Technische Universitaet Dortmund (Germany). Astroteilchenphysik; Collaboration: FACT-Collaboration

2016-07-01

Imaging Atmospheric Cherenkov Telescopes (IACTs) like FACT produce a continuous flow of data during measurements. Analyzing the data in near real time is essential for monitoring sources. One major task of a monitoring system is to detect changes in the gamma-ray flux of a source, and to alert other experiments if some predefined limit is reached. In order to calculate the flux of an observed source, it is necessary to run an entire data analysis process including calibration, image cleaning, parameterization, signal-background separation and flux estimation. Software built on top of a data streaming framework has been implemented for FACT and generalized to work with the data acquisition framework of the Cherenkov Telescope Array (CTA). We present how the streams-framework is used to apply supervised machine learning models to an online data stream from the telescope.

Object tracking using multiple camera video streams

Science.gov (United States)

Mehrubeoglu, Mehrube; Rojas, Diego; McLauchlan, Lifford

2010-05-01

Two synchronized cameras are utilized to obtain independent video streams to detect moving objects from two different viewing angles. The video frames are directly correlated in time. Moving objects in image frames from the two cameras are identified and tagged for tracking. One advantage of such a system involves overcoming effects of occlusions that could result in an object in partial or full view in one camera, when the same object is fully visible in another camera. Object registration is achieved by determining the location of common features in the moving object across simultaneous frames. Perspective differences are adjusted. Combining information from images from multiple cameras increases robustness of the tracking process. Motion tracking is achieved by determining anomalies caused by the objects' movement across frames in time in each and the combined video information. The path of each object is determined heuristically. Accuracy of detection is dependent on the speed of the object as well as variations in direction of motion. Fast cameras increase accuracy but limit the speed and complexity of the algorithm. Such an imaging system has applications in traffic analysis, surveillance and security, as well as object modeling from multi-view images. The system can easily be expanded by increasing the number of cameras such that there is an overlap between the scenes from at least two cameras in proximity. An object can then be tracked long distances or across multiple cameras continuously, applicable, for example, in wireless sensor networks for surveillance or navigation.

Fault-Tolerant and Elastic Streaming MapReduce with Decentralized Coordination

Energy Technology Data Exchange (ETDEWEB)

Kumbhare, Alok [Univ. of Southern California, Los Angeles, CA (United States); Frincu, Marc [Univ. of Southern California, Los Angeles, CA (United States); Simmhan, Yogesh [Indian Inst. of Technology (IIT), Bangalore (India); Prasanna, Viktor K. [Univ. of Southern California, Los Angeles, CA (United States)

2015-06-29

The MapReduce programming model, due to its simplicity and scalability, has become an essential tool for processing large data volumes in distributed environments. Recent Stream Processing Systems (SPS) extend this model to provide low-latency analysis of high-velocity continuous data streams. However, integrating MapReduce with streaming poses challenges: first, the runtime variations in data characteristics such as data-rates and key-distribution cause resource overload, that inturn leads to fluctuations in the Quality of the Service (QoS); and second, the stateful reducers, whose state depends on the complete tuple history, necessitates efficient fault-recovery mechanisms to maintain the desired QoS in the presence of resource failures. We propose an integrated streaming MapReduce architecture leveraging the concept of consistent hashing to support runtime elasticity along with locality-aware data and state replication to provide efficient load-balancing with low-overhead fault-tolerance and parallel fault-recovery from multiple simultaneous failures. Our evaluation on a private cloud shows up to 2:8 improvement in peak throughput compared to Apache Storm SPS, and a low recovery latency of 700 -1500 ms from multiple failures.

A Stream Tilling Approach to Surface Area Estimation for Large Scale Spatial Data in a Shared Memory System

Directory of Open Access Journals (Sweden)

Liu Jiping

2017-12-01

Full Text Available Surface area estimation is a widely used tool for resource evaluation in the physical world. When processing large scale spatial data, the input/output (I/O can easily become the bottleneck in parallelizing the algorithm due to the limited physical memory resources and the very slow disk transfer rate. In this paper, we proposed a stream tilling approach to surface area estimation that first decomposed a spatial data set into tiles with topological expansions. With these tiles, the one-to-one mapping relationship between the input and the computing process was broken. Then, we realized a streaming framework towards the scheduling of the I/O processes and computing units. Herein, each computing unit encapsulated a same copy of the estimation algorithm, and multiple asynchronous computing units could work individually in parallel. Finally, the performed experiment demonstrated that our stream tilling estimation can efficiently alleviate the heavy pressures from the I/O-bound work, and the measured speedup after being optimized have greatly outperformed the directly parallel versions in shared memory systems with multi-core processors.

StreamNet, 1999-2000 Annual Report.

Energy Technology Data Exchange (ETDEWEB)

Schmidt, Bruce; Roger, Phil; Butterfield, Bart (Pacific States Marine Fisheries Commission, Gladstone, OR)

2001-09-01

The StreamNet Project is a cooperative project that provides basic fishery management data in a consistent format across the Columbia Basin region, with some data from outside the region. Specific categories of data are acquired from the multiple data generating agencies in the Columbia Basin, converted into a standardized data exchange format (DEF) and distributed to fish researchers, managers and decision makers directly or through an on-line data retrieval system (www.streamnet.org). The project is funded by the Bonneville Power Administration (BPA) as part of the Northwest Power Planning Council's (NWPPC) Fish and Wildlife Program. This cooperative effort is composed of a region-wide project administered by the Pacific States Marine Fisheries Commission (PSMFC) that is responsible for project management, regional data management and data delivery (Region), plus seven contributing projects within the data generating entities: Columbia River Intertribal Fish Commission (CRITFC); Idaho Department of Fish and Game (IDFG); Montana Fish, Wildlife and Parks (MFWP); Oregon Department of Fish and Wildlife (ODFW); Shoshone-Bannock Tribes; U. S. Fish and Wildlife Service (FWS); and Washington Department of Fish and Wildlife (WDFW). The contributing projects are funded through the StreamNet contract but work within their respective agencies and are referred to here as the agency's StreamNet project (for example, ''IDFG StreamNet'' for Idaho's project). The StreamNet Project provides an important link in the chain of data flow in the Columbia Basin, with specific emphasis on data collected routinely over time by management agencies. Basic fish related data are collected in the field by the various state, tribal and federal agencies in the basin for purposes related to each agency's individual mission and responsibility. As a result, there often is a lack of standardization among agencies in field methodology or data management. To be

INCREMENTAL PRINCIPAL COMPONENT ANALYSIS BASED OUTLIER DETECTION METHODS FOR SPATIOTEMPORAL DATA STREAMS

Directory of Open Access Journals (Sweden)

A. Bhushan

2015-07-01

Full Text Available In this paper, we address outliers in spatiotemporal data streams obtained from sensors placed across geographically distributed locations. Outliers may appear in such sensor data due to various reasons such as instrumental error and environmental change. Real-time detection of these outliers is essential to prevent propagation of errors in subsequent analyses and results. Incremental Principal Component Analysis (IPCA is one possible approach for detecting outliers in such type of spatiotemporal data streams. IPCA has been widely used in many real-time applications such as credit card fraud detection, pattern recognition, and image analysis. However, the suitability of applying IPCA for outlier detection in spatiotemporal data streams is unknown and needs to be investigated. To fill this research gap, this paper contributes by presenting two new IPCA-based outlier detection methods and performing a comparative analysis with the existing IPCA-based outlier detection methods to assess their suitability for spatiotemporal sensor data streams.

Open Source Initiative Powers Real-Time Data Streams

Science.gov (United States)

2014-01-01

Under an SBIR contract with Dryden Flight Research Center, Creare Inc. developed a data collection tool called the Ring Buffered Network Bus. The technology has now been released under an open source license and is hosted by the Open Source DataTurbine Initiative. DataTurbine allows anyone to stream live data from sensors, labs, cameras, ocean buoys, cell phones, and more.

Mining Frequent Item Sets in Asynchronous Transactional Data Streams over Time Sensitive Sliding Windows Model

International Nuclear Information System (INIS)

Javaid, Q.; Memon, F.; Talpur, S.; Arif, M.; Awan, M.D.

2016-01-01

EPs (Extracting Frequent Patterns) from the continuous transactional data streams is a challenging and critical task in some of the applications, such as web mining, data analysis and retail market, prediction and network monitoring, or analysis of stock market exchange data. Many algorithms have been developed previously for mining FPs (Frequent Patterns) from a data stream. Such algorithms are currently highly required to develop new solutions and approaches to the precise handling of data streams. New techniques, solutions, or approaches are developed to address unbounded, ordered, and continuous sequences of data and for the generation of data at a rapid speed from data streams. Hence, extracting FPs using fresh or recent data involves the high-level analysis of data streams. We have suggested an efficient technique for the window sliding model; this technique extracts new and fresh FPs from high-speed data streams. In this study, a CPILT (Compacted Tree Compact Pattern Tree) is developed to capture the latest contents in the stream and to efficiently remove outdated contents from the data stream. The main concept introduced in this work on CPILT is the dynamic restructuring of a tree, which is helpful in producing a compacted tree and the frequency descending structure of a tree on runtime. With the help of the mining technique of FP growth, a complete list of new and fresh FPs is obtained from a CPILT using an existing window. The memory usage and time complexity of the latest FPs in high-speed data streams can efficiently be determined through proper experimentation and analysis. (author)

Indexing Density Models for Incremental Learning and Anytime Classification on Data Streams

DEFF Research Database (Denmark)

Seidl, Thomas; Assent, Ira; Kranen, Philipp

2009-01-01

Classification of streaming data faces three basic challenges: it has to deal with huge amounts of data, the varying time between two stream data items must be used best possible (anytime classification) and additional training data must be incrementally learned (anytime learning) for applying...... to the individual object to be classified) a hierarchy of mixture densities that represent kernel density estimators at successively coarser levels. Our probability density queries together with novel classification improvement strategies provide the necessary information for very effective classification at any...... point of interruption. Moreover, we propose a novel evaluation method for anytime classification using Poisson streams and demonstrate the anytime learning performance of the Bayes tree....

SWAMP+: multiple subsequence alignment using associative massive parallelism

Energy Technology Data Exchange (ETDEWEB)

Steinfadt, Shannon Irene [Los Alamos National Laboratory; Baker, Johnnie W [KENT STATE UNIV.

2010-10-18

A new parallel algorithm SWAMP+ incorporates the Smith-Waterman sequence alignment on an associative parallel model known as ASC. It is a highly sensitive parallel approach that expands traditional pairwise sequence alignment. This is the first parallel algorithm to provide multiple non-overlapping, non-intersecting subsequence alignments with the accuracy of Smith-Waterman. The efficient algorithm provides multiple alignments similar to BLAST while creating a better workflow for the end users. The parallel portions of the code run in O(m+n) time using m processors. When m = n, the algorithmic analysis becomes O(n) with a coefficient of two, yielding a linear speedup. Implementation of the algorithm on the SIMD ClearSpeed CSX620 confirms this theoretical linear speedup with real timings.

Hydrogeochemical and stream sediment reconnaissance basic data for Seguin NTMS quadrangle, Texas

International Nuclear Information System (INIS)

1978-01-01

Results of a reconnaissance geochemical survey of the Sequin Quadrangle, Texas are reported. Field and laboratory data are presented for 848 groundwater, 950 stream sediment, and 406 stream water samples. Statistical and areal distributions of uranium and other possible uranium-related variables are displayed. A generalized geologic map of the survey area is provided, and pertinent geologic factors which may be of significance in evaluating the potential for uranium mineralization are briefly discussed. Groundwater data indicate that uranium concentrations above the 85th percentile occur along several northeast-southwest trends paralleling the regional strike of the major formations located within the survey area. The stream sediment data indicate that uranium is associated with heavy and/or resistate minerals in the Carrizo Sand and certain members of the Claiborne Group. Soluble uranium is primarily associated with the Cretaceous Formations, the Whitsett and Catahoula Formations, and sections of the Oakville and Fleming Formations. Stream water data corroborate well with both groundwater and stream sediment data. Anomalous values for uranium and associated pathfinder elements indicate that the Whitsett and Catahoula Formations and sections of the Oakville and Fleming Formations are potentially favorable for uranium mineralization. Anomalous values for certain pathfinder elements also occur in basins draining from the Beaumont Formation and may warrant further investigation

Design for real-time data acquisition based on streaming technology

International Nuclear Information System (INIS)

Nakanishi, Hideya; Kojima, Mamoru

2001-04-01

For the LHD project a long-pulse plasma experiment of one-hour duration is planned. In this quasi steady-state operation, the data acquisition system will be required to continuously transfer the diagnostic data from the digitizer front-end and display them in real-time. The Compact PCI standard is used to replace the conventional CAMAC digitizers in LHD, because it provides good functionality for real-time data streaming and also a connectivity with modern PC technology. The digitizer scheme, interface to the host computer, adoption of data compression, and downstream applications are discussed in detail to design and implement this new real-time data streaming system for LHD plasma diagnostics. (author)

Adaptive SVM for Data Stream Classification

Directory of Open Access Journals (Sweden)

Isah A. Lawal

2017-07-01

Full Text Available In this paper, we address the problem of learning an adaptive classifier for the classification of continuous streams of data. We present a solution based on incremental extensions of the Support Vector Machine (SVM learning paradigm that updates an existing SVM whenever new training data are acquired. To ensure that the SVM effectiveness is guaranteed while exploiting the newly gathered data, we introduce an on-line model selection approach in the incremental learning process. We evaluated the proposed method on real world applications including on-line spam email filtering and human action classification from videos. Experimental results show the effectiveness and the potential of the proposed approach.

Stream on the Sky: Outsourcing Access Control Enforcement for Stream Data to the Cloud

OpenAIRE

Dinh, Tien Tuan Anh; Datta, Anwitaman

2012-01-01

There is an increasing trend for businesses to migrate their systems towards the cloud. Security concerns that arise when outsourcing data and computation to the cloud include data confidentiality and privacy. Given that a tremendous amount of data is being generated everyday from plethora of devices equipped with sensing capabilities, we focus on the problem of access controls over live streams of data based on triggers or sliding windows, which is a distinct and more challenging problem tha...

Simulation and reconstruction of free-streaming data in CBM

International Nuclear Information System (INIS)

Friese, Volker

2011-01-01

The CBM experiment will investigate heavy-ion reactions at the FAIR facility at unprecedented interaction rates. This implies a novel read-out and data acquisition concept with self-triggered front-end electronics and free-streaming data. Event association must be performed in software on-line, and may require four-dimensional reconstruction routines. In order to study the problem of event association and to develop proper algorithms, simulations must be performed which go beyond the normal event-by-event processing as available from most experimental simulation frameworks. In this article, we discuss the challenges and concepts for the reconstruction of such free-streaming data and present first steps for a time-based simulation which is necessary for the development and validation of the reconstruction algorithms, and which requires modifications to the current software framework FAIRROOT as well as to the data model.

Improved streaming analysis technique: spherical harmonics expansion of albedo data

International Nuclear Information System (INIS)

Albert, T.E.; Simmons, G.L.

1979-01-01

An improved albedo scattering technique was implemented with a three-dimensional Monte Carlo transport code for use in analyzing radiation streaming problems. The improvement was based on a shifted spherical Harmonics expansion of the doubly differential albedo data base. The result of the improvement was a factor of 3 to 10 reduction in data storage requirements and approximately a factor of 3 to 6 increase in computational speed. Comparisons of results obtained using the technique with measurements are shown for neutron streaming in one- and two-legged square concrete ducts

Expanded stream gauging includes groundwater data and trends

Science.gov (United States)

Constantz, James E.; Barlow, Jeannie R.; Eddy-Miller, Cheryl; Caldwell, Rodney R.; Wheeler, Jerrod D.

2012-01-01

Population growth has increased water scarcity to the point that documenting current amounts of worldwide water resources is now as critical as any data collection in the Earth sciences. As a key element of this data collection, stream gauges yield continuous hydrologic information and document long-term trends, recording high-frequency hydrologic information over decadal to centennial time frames.

StreamNet, annual report FY 2000; ANNUAL

International Nuclear Information System (INIS)

Schmidt, Bruce R.

2001-01-01

The StreamNet Project is a cooperative project that provides basic fishery management data in a consistent format across the Columbia Basin region, with some data from outside the region. Specific categories of data are acquired from the multiple data generating agencies in the Columbia Basin, converted into a standardized data exchange format (DEF) and distributed to fish researchers, managers and decision makers directly or through an on-line data retrieval system (www.streamnet.org). The project is funded by the Bonneville Power Administration (BPA) as part of the Northwest Power Planning Council's (NWPPC) Fish and Wildlife Program. This cooperative effort is composed of a region-wide project administered by the Pacific States Marine Fisheries Commission (PSMFC) that is responsible for project management, regional data management and data delivery (Region), plus seven contributing projects within the data generating entities: Columbia River Intertribal Fish Commission (CRITFC); Idaho Department of Fish and Game (IDFG); Montana Fish, Wildlife and Parks (MFWP); Oregon Department of Fish and Wildlife (ODFW); Shoshone-Bannock Tribes; U. S. Fish and Wildlife Service (FWS); and Washington Department of Fish and Wildlife (WDFW). The contributing projects are funded through the StreamNet contract but work within their respective agencies and are referred to here as the agency's StreamNet project (for example, ''IDFG StreamNet'' for Idaho's project). The StreamNet Project provides an important link in the chain of data flow in the Columbia Basin, with specific emphasis on data collected routinely over time by management agencies. Basic fish related data are collected in the field by the various state, tribal and federal agencies in the basin for purposes related to each agency's individual mission and responsibility. As a result, there often is a lack of standardization among agencies in field methodology or data management. To be able to utilize data for comparison or

Real-time change detection in data streams with FPGAs

International Nuclear Information System (INIS)

Vega, J.; Dormido-Canto, S.; Cruz, T.; Ruiz, M.; Barrera, E.; Castro, R.; Murari, A.; Ochando, M.

2014-01-01

Highlights: • Automatic recognition of changes in data streams of multidimensional signals. • Detection algorithm based on testing exchangeability on-line. • Real-time and off-line applicability. • Real-time implementation in FPGAs. - Abstract: The automatic recognition of changes in data streams is useful in both real-time and off-line data analyses. This article shows several effective change-detecting algorithms (based on martingales) and describes their real-time applicability in the data acquisition systems through the use of Field Programmable Gate Arrays (FPGA). The automatic event recognition system is absolutely general and it does not depend on either the particular event to detect or the specific data representation (waveforms, images or multidimensional signals). The developed approach provides good results for change detection in both the temporal evolution of profiles and the two-dimensional spatial distribution of volume emission intensity. The average computation time in the FPGA is 210 μs per profile

Recognition of periodic behavioral patterns from streaming mobility data

NARCIS (Netherlands)

Baratchi, Mitra; Meratnia, Nirvana; Havinga, Paul J.M.; Stojmenovic, Ivan; Cheng, Zixue; Guo, Song

2014-01-01

Ubiquitous location-aware sensing devices have facilitated collection of large volumes of mobility data streams from moving entities such as people and animals, among others. Extraction of various types of periodic behavioral patterns hidden in such large volume of mobility data helps in

Streaming Pool: reuse, combine and create reactive streams with pleasure

CERN Multimedia

CERN. Geneva

2017-01-01

When connecting together heterogeneous and complex systems, it is not easy to exchange data between components. Streams of data are successfully used in industry in order to overcome this problem, especially in the case of "live" data. Streams are a specialization of the Observer design pattern and they provide asynchronous and non-blocking data flow. The ongoing effort of the ReactiveX initiative is one example that demonstrates how demanding this technology is even for big companies. Bridging the discrepancies of different technologies with common interfaces is already done by the Reactive Streams initiative and, in the JVM world, via reactive-streams-jvm interfaces. Streaming Pool is a framework for providing and discovering reactive streams. Through the mechanism of dependency injection provided by the Spring Framework, Streaming Pool provides a so called Discovery Service. This object can discover and chain streams of data that are technologically agnostic, through the use of Stream IDs. The stream to ...

Method and apparatus of prefetching streams of varying prefetch depth

Science.gov (United States)

Gara, Alan [Mount Kisco, NY; Ohmacht, Martin [Yorktown Heights, NY; Salapura, Valentina [Chappaqua, NY; Sugavanam, Krishnan [Mahopac, NY; Hoenicke, Dirk [Seebruck-Seeon, DE

2012-01-24

Method and apparatus of prefetching streams of varying prefetch depth dynamically changes the depth of prefetching so that the number of multiple streams as well as the hit rate of a single stream are optimized. The method and apparatus in one aspect monitor a plurality of load requests from a processing unit for data in a prefetch buffer, determine an access pattern associated with the plurality of load requests and adjust a prefetch depth according to the access pattern.

A fast calculating two-stream-like multiple scattering algorithm that captures azimuthal and elevation variations

Science.gov (United States)

Fiorino, Steven T.; Elmore, Brannon; Schmidt, Jaclyn; Matchefts, Elizabeth; Burley, Jarred L.

2016-05-01

Properly accounting for multiple scattering effects can have important implications for remote sensing and possibly directed energy applications. For example, increasing path radiance can affect signal noise. This study describes the implementation of a fast-calculating two-stream-like multiple scattering algorithm that captures azimuthal and elevation variations into the Laser Environmental Effects Definition and Reference (LEEDR) atmospheric characterization and radiative transfer code. The multiple scattering algorithm fully solves for molecular, aerosol, cloud, and precipitation single-scatter layer effects with a Mie algorithm at every calculation point/layer rather than an interpolated value from a pre-calculated look-up-table. This top-down cumulative diffusivity method first considers the incident solar radiance contribution to a given layer accounting for solid angle and elevation, and it then measures the contribution of diffused energy from previous layers based on the transmission of the current level to produce a cumulative radiance that is reflected from a surface and measured at the aperture at the observer. Then a unique set of asymmetry and backscattering phase function parameter calculations are made which account for the radiance loss due to the molecular and aerosol constituent reflectivity within a level and allows for a more accurate characterization of diffuse layers that contribute to multiple scattered radiances in inhomogeneous atmospheres. The code logic is valid for spectral bands between 200 nm and radio wavelengths, and the accuracy is demonstrated by comparing the results from LEEDR to observed sky radiance data.

Mining top-k frequent closed itemsets in data streams using sliding window

International Nuclear Information System (INIS)

Rehman, Z.; Shahbaz, M.

2013-01-01

Frequent itemset mining has become a popular research area in data mining community since the last few years. T here are two main technical hitches while finding frequent itemsets. First, to provide an appropriate minimum support value to start and user need to tune this minimum support value by running the algorithm again and again. Secondly, generated frequent itemsets are mostly numerous and as a result a number of association rules generated are also very large in numbers. Applications dealing with streaming environment need to process the data received at high rate, therefore, finding frequent itemsets in data streams becomes complex. In this paper, we present an algorithm to mine top-k frequent closed itemsets using sliding window approach from streaming data. We developed a single-pass algorithm to find frequent closed itemsets of length between user's defined minimum and maximum- length. To improve the performance of algorithm and to avoid rescanning of data, we have transformed data into bitmap based tree data structure. (author)

Scalable IC Platform for Smart Cameras

Directory of Open Access Journals (Sweden)

Harry Broers

2005-08-01

Full Text Available Smart cameras are among the emerging new fields of electronics. The points of interest are in the application areas, software and IC development. In order to reduce cost, it is worthwhile to invest in a single architecture that can be scaled for the various application areas in performance (and resulting power consumption. In this paper, we show that the combination of an SIMD (single-instruction multiple-data processor and a general-purpose DSP is very advantageous for the image processing tasks encountered in smart cameras. While the SIMD processor gives the very high performance necessary by exploiting the inherent data parallelism found in the pixel crunching part of the algorithms, the DSP offers a friendly approach to the more complex tasks. The paper continues to motivate that SIMD processors have very convenient scaling properties in silicon, making the complete, SIMD-DSP architecture suitable for different application areas without changing the software suite. Analysis of the changes in power consumption due to scaling shows that for typical image processing tasks, it is beneficial to scale the SIMD processor to use the maximum level of parallelism available in the algorithm if the IC supply voltage can be lowered. If silicon cost is of importance, the parallelism of the processor should be scaled to just reach the desired performance given the speed of the silicon.

CC_TRS: Continuous Clustering of Trajectory Stream Data Based on Micro Cluster Life

Directory of Open Access Journals (Sweden)

Musaab Riyadh

2017-01-01

Full Text Available The rapid spreading of positioning devices leads to the generation of massive spatiotemporal trajectories data. In some scenarios, spatiotemporal data are received in stream manner. Clustering of stream data is beneficial for different applications such as traffic management and weather forecasting. In this article, an algorithm for Continuous Clustering of Trajectory Stream Data Based on Micro Cluster Life is proposed. The algorithm consists of two phases. There is the online phase where temporal micro clusters are used to store summarized spatiotemporal information for each group of similar segments. The clustering task in online phase is based on temporal micro cluster lifetime instead of time window technique which divides stream data into time bins and clusters each bin separately. For offline phase, a density based clustering approach is used to generate macro clusters depending on temporal micro clusters. The evaluation of the proposed algorithm on real data sets shows the efficiency and the effectiveness of the proposed algorithm and proved it is efficient alternative to time window technique.

Longitudinal structure in temperate stream fish communities: evaluating conceptual models with temporal data

Science.gov (United States)

Roberts, James H.; Hitt, Nathaniel P.

2010-01-01

Five conceptual models of longitudinal fish community organization in streams were examined: (1) niche diversity model (NDM), (2) stream continuum model (SCM), (3) immigrant accessibility model (IAM), (4) environmental stability model (ESM), and (5) adventitious stream model (ASM). We used differences among models in their predictions about temporal species turnover, along with five spatiotemporal fish community data sets, to evaluate model applicability. Models were similar in predicting a positive species richness–stream size relationship and longitudinal species nestedness, but differed in predicting either similar temporal species turnover throughout the stream continuum (NDM, SCM), higher turnover upstream (IAM, ESM), or higher turnover downstream (ASM). We calculated measures of spatial and temporal variation from spatiotemporal fish data in five wadeable streams in central and eastern North America spanning 34–68 years (French Creek [New York], Piasa Creek [Illinois], Spruce Run [Virginia], Little Stony Creek [Virginia], and Sinking Creek [Virginia]). All streams exhibited substantial species turnover (i.e., at least 27% turnover in stream-scale species pools), in contrast to the predictions of the SCM. Furthermore, community change was greater in downstream than upstream reaches in four of five streams. This result is most consistent with the ASM and suggests that downstream communities are strongly influenced by migrants to and from species pools outside the focal stream. In Sinking Creek, which is isolated from external species pools, temporal species turnover (via increased richness) was higher upstream than downstream, which is a pattern most consistent with the IAM or ESM. These results corroborate the hypothesis that temperate stream habitats and fish communities are temporally dynamic and that fish migration and environmental disturbances play fundamental roles in stream fish community organization.

Mining Twitter Data Stream to Augment NASA GPM Validation

Science.gov (United States)

Teng, W. L.; Albayrak, A.; Huffman, G. J.; Vollmer, B.

2017-12-01

The Twitter data stream is an important new source of real-time and historical global information for potentially augmenting the validation program of NASA's Global Precipitation Measurement (GPM) mission. There have been other similar uses of Twitter, though mostly related to natural hazards monitoring and management. The validation of satellite precipitation estimates is challenging, because many regions lack data or access to data, especially outside of the U.S. and in remote and developing areas. The time-varying set of "precipitation" tweets can be thought of as an organic network of rain gauges, potentially providing a widespread view of precipitation occurrence. Twitter provides a large source of crowd for crowdsourcing. During a 24-hour period in the middle of the snow storm this past March in the U.S. Northeast, we collected more than 13,000 relevant precipitation tweets with exact geolocation. The overall objective of our project is to determine the extent to which processed tweets can provide additional information that improves the validation of GPM data. Though our current effort focuses on tweets and precipitation, our approach is general and applicable to other social media and other geophysical measurements. Specifically, we have developed an operational infrastructure for processing tweets, in a format suitable for analysis with GPM data; engaged with potential participants, both passive and active, to "enrich" the Twitter stream; and inter-compared "precipitation" tweet data, ground station data, and GPM retrievals. In this presentation, we detail the technical capabilities of our tweet processing infrastructure, including data abstraction, feature extraction, search engine, context-awareness, real-time processing, and high volume (big) data processing; various means for "enriching" the Twitter stream; and results of inter-comparisons. Our project should bring a new kind of visibility to Twitter and engender a new kind of appreciation of the value

Observed TRU data from nuclear utility waste streams

International Nuclear Information System (INIS)

Wessman, R.A.; Floyd, J.G.; Leventhal, L.

1990-01-01

TMA/Norcal has performed 10CFR61 analysis of radioactive waste streams from BWR's and PWR's since 1983. Many standard and non-routine sample types have been received for analysis from nuclear power plants nation-wide. In addition to the 10CFR61 Tables I and II analyses, we also have analyzed for many of the supplementary isotopes. As part of this program TRU analyses are required. As a result, have accumulated a significant amount of data for plutonium, americium, and curium in radioactive waste for many different sample matrices from many different waste streams. This paper will present our analytical program for 10CFR61 TRU. The laboratory methodology including chemical and radiometric procedures is discussed. The sensitivity of our measurements and ability to meet the lower limits of detection is also discussed. Secondly, a review of TRU data is presented. Scaling factors and their ranges from selected PWR stations are included. We discuss some features of, and limits to, interpretation of these data. 8 refs., 3 tabs

Assessing the influence of multiple stressors on stream diatom metrics in the upper Midwest, USA

Science.gov (United States)

Munn, Mark D.; Waite, Ian R.; Konrad, Christopher P.

2018-01-01

Water resource managers face increasing challenges in identifying what physical and chemical stressors are responsible for the alteration of biological conditions in streams. The objective of this study was to assess the comparative influence of multiple stressors on benthic diatoms at 98 sites that spanned a range of stressors in an agriculturally dominated region in the upper Midwest, USA. The primary stressors of interest included: nutrients, herbicides and fungicides, sediment, and streamflow; although the influence of physical habitat was incorporated in the assessment. Boosted Regression Tree was used to examine both the sensitivity of various diatom metrics and the relative importance of the primary stressors. Percent Sensitive Taxa, percent Highly Motile Taxa, and percent High Phosphorus Taxa had the strongest response to stressors. Habitat and total phosphorous were the most common discriminators of diatom metrics, with herbicides as secondary factors. A Classification and Regression Tree (CART) model was used to examine conditional relations among stressors and indicated that fine-grain streams had a lower percentage of Sensitive Taxa than coarse-grain streams, with Sensitive Taxa decreasing further with increased water temperature (>30 °C) and triazine concentrations (>1500 ng/L). In contrast, streams dominated by coarse-grain substrate contained a higher percentage of Sensitive Taxa, with relative abundance increasing with lower water temperatures (water depth (water temperature appears to be a major limiting factor in Midwest streams; whereas both total phosphorus and percent fines showed a slight subsidy-stress response. While using benthic algae for assessing stream quality can be challenging, field-based studies can elucidate stressor effects and interactions when the response variables are appropriate, sufficient stressor resolution is achieved, and the number and type of sites represent a gradient of stressor conditions and at least a quasi

PROXY-BASED PATCHING STREAM TRANSMISSION STRATEGY IN MOBILE STREAMING MEDIA SYSTEM

Institute of Scientific and Technical Information of China (English)

Liao Jianxin; Lei Zhengxiong; Ma Xutao; Zhu Xiaomin

2006-01-01

A mobile transmission strategy, PMPatching (Proxy-based Mobile Patching) transmission strategy is proposed, it applies to the proxy-based mobile streaming media system in Wideband Code Division Multiple Access (WCDMA) network. Performance of the whole system can be improved by using patching stream to transmit anterior part of the suffix that had been played back, and by batching all the demands for the suffix arrived in prefix period and patching stream transmission threshold period. Experimental results show that this strategy can efficiently reduce average network transmission cost and number of channels consumed in central streaming media server.

Guidelines and Procedures for Computing Time-Series Suspended-Sediment Concentrations and Loads from In-Stream Turbidity-Sensor and Streamflow Data

Science.gov (United States)

Rasmussen, Patrick P.; Gray, John R.; Glysson, G. Douglas; Ziegler, Andrew C.

2009-01-01

In-stream continuous turbidity and streamflow data, calibrated with measured suspended-sediment concentration data, can be used to compute a time series of suspended-sediment concentration and load at a stream site. Development of a simple linear (ordinary least squares) regression model for computing suspended-sediment concentrations from instantaneous turbidity data is the first step in the computation process. If the model standard percentage error (MSPE) of the simple linear regression model meets a minimum criterion, this model should be used to compute a time series of suspended-sediment concentrations. Otherwise, a multiple linear regression model using paired instantaneous turbidity and streamflow data is developed and compared to the simple regression model. If the inclusion of the streamflow variable proves to be statistically significant and the uncertainty associated with the multiple regression model results in an improvement over that for the simple linear model, the turbidity-streamflow multiple linear regression model should be used to compute a suspended-sediment concentration time series. The computed concentration time series is subsequently used with its paired streamflow time series to compute suspended-sediment loads by standard U.S. Geological Survey techniques. Once an acceptable regression model is developed, it can be used to compute suspended-sediment concentration beyond the period of record used in model development with proper ongoing collection and analysis of calibration samples. Regression models to compute suspended-sediment concentrations are generally site specific and should never be considered static, but they represent a set period in a continually dynamic system in which additional data will help verify any change in sediment load, type, and source.

Process recognition in multi-element soil and stream-sediment geochemical data

Science.gov (United States)

Grunsky, E.C.; Drew, L.J.; Sutphin, D.M.

2009-01-01

Stream-sediment and soil geochemical data from the Upper and Lower Coastal Plains of South Carolina (USA) were studied to determine relationships between soils and stream sediments. From multi-element associations, characteristic compositions were determined for both media. Primary associations of elements reflect mineralogy, including heavy minerals, carbonates and clays, and the effects of groundwater. The effects of groundwater on element concentrations are more evident in soils than stream sediments. A "winnowing index" was created using ratios of Th to Al that revealed differing erosional and depositional environments. Both soils and stream sediments from the Upper and Lower Coastal Plains show derivation from similar materials and subsequent similar multi-element relationships, but have some distinct differences. In the Lower Coastal Plain, soils have high values of elements concentrated in heavy minerals (Ce, Y, Th) that grade into high values of elements concentrated into finer-grain-size, lower-density materials, primarily comprised of carbonates and feldspar minerals (Mg, Ca, Na, K, Al). These gradational trends in mineralogy and geochemistry are inferred to reflect reworking of materials during marine transgressions and regressions. Upper Coastal Plain stream-sediment geochemistry shows a higher winnowing index relative to soil geochemistry. A comparison of the 4 media (Upper Coastal Plain soils and stream sediments and Lower Coastal Plain soils and stream sediments) shows that Upper Coastal Plain stream sediments have a higher winnowing index and a higher concentration of elements contained within heavy minerals, whereas Lower Coastal Plain stream sediments show a strong correlation between elements typically contained within clays. It is not possible to calculate a functional relationship between stream sediment-soil compositions for all elements due to the complex history of weathering, deposition, reworking and re-deposition. However, depending on

Efficient Processing of Continuous Skyline Query over Smarter Traffic Data Stream for Cloud Computing

Directory of Open Access Journals (Sweden)

Wang Hanning

2013-01-01

Full Text Available The analyzing and processing of multisource real-time transportation data stream lay a foundation for the smart transportation's sensibility, interconnection, integration, and real-time decision making. Strong computing ability and valid mass data management mode provided by the cloud computing, is feasible for handling Skyline continuous query in the mass distributed uncertain transportation data stream. In this paper, we gave architecture of layered smart transportation about data processing, and we formalized the description about continuous query over smart transportation data Skyline. Besides, we proposed mMR-SUDS algorithm (Skyline query algorithm of uncertain transportation stream data based on micro-batchinMap Reduce based on sliding window division and architecture.

SmartCell: An Energy Efficient Coarse-Grained Reconfigurable Architecture for Stream-Based Applications

Directory of Open Access Journals (Sweden)

Liang Cao

2009-01-01

Full Text Available This paper presents SmartCell, a novel coarse-grained reconfigurable architecture, which tiles a large number of processor elements with reconfigurable interconnection fabrics on a single chip. SmartCell is able to provide high performance and energy efficient processing for stream-based applications. It can be configured to operate in various modes, such as SIMD, MIMD, and systolic array. This paper describes the SmartCell architecture design, including processing element, reconfigurable interconnection fabrics, instruction and control process, and configuration scheme. The SmartCell prototype with 64 PEs is implemented using 0.13  m CMOS standard cell technology. The core area is about 8.5  , and the power consumption is about 1.6 mW/MHz. The performance is evaluated through a set of benchmark applications, and then compared with FPGA, ASIC, and two well-known reconfigurable architectures including RaPiD and Montium. The results show that the SmartCell can bridge the performance and flexibility gap between ASIC and FPGA. It is also about 8% and 69% more energy efficient than Montium and RaPiD systems for evaluated benchmarks. Meanwhile, SmartCell can achieve 4 and 2 times more throughput gains when comparing with Montium and RaPiD, respectively. It is concluded that SmartCell system is a promising reconfigurable and energy efficient architecture for stream processing.

DataCell: Exploiting the Power of Relational Databases for Efficient Stream Processing

NARCIS (Netherlands)

E. Liarou (Erietta); M.L. Kersten (Martin)

2009-01-01

htmlabstractDesigned for complex event processing, DataCell is a research prototype database system in the area of sensor stream systems. Under development at CWI, it belongs to the MonetDB database system family. CWI researchers innovatively built a stream engine directly on top of a database

Bivariate functional data clustering: grouping streams based on a varying coefficient model of the stream water and air temperature relationship

Science.gov (United States)

H. Li; X. Deng; Andy Dolloff; E. P. Smith

2015-01-01

A novel clustering method for bivariate functional data is proposed to group streams based on their water–air temperature relationship. A distance measure is developed for bivariate curves by using a time-varying coefficient model and a weighting scheme. This distance is also adjusted by spatial correlation of streams via the variogram. Therefore, the proposed...

Efficient Streaming Mass Spatio-Temporal Vehicle Data Access in Urban Sensor Networks Based on Apache Storm.

Science.gov (United States)

Zhou, Lianjie; Chen, Nengcheng; Chen, Zeqiang

2017-04-10

The efficient data access of streaming vehicle data is the foundation of analyzing, using and mining vehicle data in smart cities, which is an approach to understand traffic environments. However, the number of vehicles in urban cities has grown rapidly, reaching hundreds of thousands in number. Accessing the mass streaming data of vehicles is hard and takes a long time due to limited computation capability and backward modes. We propose an efficient streaming spatio-temporal data access based on Apache Storm (ESDAS) to achieve real-time streaming data access and data cleaning. As a popular streaming data processing tool, Apache Storm can be applied to streaming mass data access and real time data cleaning. By designing the Spout/bolt workflow of topology in ESDAS and by developing the speeding bolt and other bolts, Apache Storm can achieve the prospective aim. In our experiments, Taiyuan BeiDou bus location data is selected as the mass spatio-temporal data source. In the experiments, the data access results with different bolts are shown in map form, and the filtered buses' aggregation forms are different. In terms of performance evaluation, the consumption time in ESDAS for ten thousand records per second for a speeding bolt is approximately 300 milliseconds, and that for MongoDB is approximately 1300 milliseconds. The efficiency of ESDAS is approximately three times higher than that of MongoDB.

Efficient Streaming Mass Spatio-Temporal Vehicle Data Access in Urban Sensor Networks Based on Apache Storm

Directory of Open Access Journals (Sweden)

Lianjie Zhou

2017-04-01

Full Text Available The efficient data access of streaming vehicle data is the foundation of analyzing, using and mining vehicle data in smart cities, which is an approach to understand traffic environments. However, the number of vehicles in urban cities has grown rapidly, reaching hundreds of thousands in number. Accessing the mass streaming data of vehicles is hard and takes a long time due to limited computation capability and backward modes. We propose an efficient streaming spatio-temporal data access based on Apache Storm (ESDAS to achieve real-time streaming data access and data cleaning. As a popular streaming data processing tool, Apache Storm can be applied to streaming mass data access and real time data cleaning. By designing the Spout/bolt workflow of topology in ESDAS and by developing the speeding bolt and other bolts, Apache Storm can achieve the prospective aim. In our experiments, Taiyuan BeiDou bus location data is selected as the mass spatio-temporal data source. In the experiments, the data access results with different bolts are shown in map form, and the filtered buses’ aggregation forms are different. In terms of performance evaluation, the consumption time in ESDAS for ten thousand records per second for a speeding bolt is approximately 300 milliseconds, and that for MongoDB is approximately 1300 milliseconds. The efficiency of ESDAS is approximately three times higher than that of MongoDB.

Capabilities of Raspberry Pi 2 for Big Data and Video Streaming Applications in Data Centres

NARCIS (Netherlands)

Schot, Nick J.; Velthuis, Paul J.E.; Postema, Björn Frits; Remke, Anne Katharina Ingrid; Remke, A.K.I.; Haverkort, Boudewijn R.H.M.; Haverkort, B.R.H.M.

Many new data centres have been built in recent years in order to keep up with the rising demand for server capacity. These data centres require a lot of electrical energy and cooling. Big data and video streaming are two heavily used applications in data centres. This paper experimentally

Multiple bio-monitoring system using visible light for electromagnetic-wave free indoor healthcare

Science.gov (United States)

An, Jinyoung; Pham, Ngoc Quan; Chung, Wan-Young

2017-12-01

In this paper, a multiple biomedical data transmission system with visible light communication (VLC) is proposed for an electromagnetic-wave-free indoor healthcare. VLC technology has emerged as an alternative solution to radio-frequency (RF) wireless systems, due to its various merits, e.g., ubiquity, power efficiency, no RF radiation, and security. With VLC, critical bio-medical signals, including electrocardiography (ECG), can be transmitted in places where RF radiation is restricted. This potential advantage of VLC could save more lives in emergency situations. A time hopping (TH) scheme is employed to transfer multiple medical-data streams in real time with a simple system design. Multiple data streams are transmitted using identical color LEDs and go into an optical detector. The received multiple data streams are demodulated and rearranged using a TH-based demodulator. The medical data is then monitored and managed to provide the necessary medical care for each patient.

Cooperative Coding and Caching for Streaming Data in Multihop Wireless Networks

Directory of Open Access Journals (Sweden)

Liu Jiangchuan

2010-01-01

Full Text Available This paper studies the distributed caching managements for the current flourish of the streaming applications in multihop wireless networks. Many caching managements to date use randomized network coding approach, which provides an elegant solution for ubiquitous data accesses in such systems. However, the encoding, essentially a combination operation, makes the coded data difficult to be changed. In particular, to accommodate new data, the system may have to first decode all the combined data segments, remove some unimportant ones, and then reencode the data segments again. This procedure is clearly expensive for continuously evolving data storage. As such, we introduce a novel Cooperative Coding and Caching ( scheme, which allows decoding-free data removal through a triangle-like codeword organization. Its decoding performance is very close to the conventional network coding with only a sublinear overhead. Our scheme offers a promising solution to the caching management for streaming data.

A high performance data parallel tensor contraction framework: Application to coupled electro-mechanics

Science.gov (United States)

Poya, Roman; Gil, Antonio J.; Ortigosa, Rogelio

2017-07-01

The paper presents aspects of implementation of a new high performance tensor contraction framework for the numerical analysis of coupled and multi-physics problems on streaming architectures. In addition to explicit SIMD instructions and smart expression templates, the framework introduces domain specific constructs for the tensor cross product and its associated algebra recently rediscovered by Bonet et al. (2015, 2016) in the context of solid mechanics. The two key ingredients of the presented expression template engine are as follows. First, the capability to mathematically transform complex chains of operations to simpler equivalent expressions, while potentially avoiding routes with higher levels of computational complexity and, second, to perform a compile time depth-first or breadth-first search to find the optimal contraction indices of a large tensor network in order to minimise the number of floating point operations. For optimisations of tensor contraction such as loop transformation, loop fusion and data locality optimisations, the framework relies heavily on compile time technologies rather than source-to-source translation or JIT techniques. Every aspect of the framework is examined through relevant performance benchmarks, including the impact of data parallelism on the performance of isomorphic and nonisomorphic tensor products, the FLOP and memory I/O optimality in the evaluation of tensor networks, the compilation cost and memory footprint of the framework and the performance of tensor cross product kernels. The framework is then applied to finite element analysis of coupled electro-mechanical problems to assess the speed-ups achieved in kernel-based numerical integration of complex electroelastic energy functionals. In this context, domain-aware expression templates combined with SIMD instructions are shown to provide a significant speed-up over the classical low-level style programming techniques.

Geospatial Image Stream Processing: Models, techniques, and applications in remote sensing change detection

Science.gov (United States)

Rueda-Velasquez, Carlos Alberto

Detection of changes in environmental phenomena using remotely sensed data is a major requirement in the Earth sciences, especially in natural disaster related scenarios where real-time detection plays a crucial role in the saving of human lives and the preservation of natural resources. Although various approaches formulated to model multidimensional data can in principle be applied to the inherent complexity of remotely sensed geospatial data, there are still challenging peculiarities that demand a precise characterization in the context of change detection, particularly in scenarios of fast changes. In the same vein, geospatial image streams do not fit appropriately in the standard Data Stream Management System (DSMS) approach because these systems mainly deal with tuple-based streams. Recognizing the necessity for a systematic effort to address the above issues, the work presented in this thesis is a concrete step toward the foundation and construction of an integrated Geospatial Image Stream Processing framework, GISP. First, we present a data and metadata model for remotely sensed image streams. We introduce a precise characterization of images and image streams in the context of remotely sensed geospatial data. On this foundation, we define spatially-aware temporal operators with a consistent semantics for change analysis tasks. We address the change detection problem in settings where multiple image stream sources are available, and thus we introduce an architectural design for the processing of geospatial image streams from multiple sources. With the aim of targeting collaborative scientific environments, we construct a realization of our architecture based on Kepler, a robust and widely used scientific workflow management system, as the underlying computational support; and open data and Web interface standards, as a means to facilitate the interoperability of GISP instances with other processing infrastructures and client applications. We demonstrate our

Crowdsourcing Stream Stage in Data Scarce Regions: Applications of CrowdHydrology

Science.gov (United States)

Lowry, C.; Fienen, M. N.

2013-12-01

Crowdsourced data collection using citizen scientists and mobile phones is a promising way to collect supplemental information in data scarce or remote regions. The research presented here explore the possibilities and pitfalls of crowdsourcing hydrologic data via mobile phone text messaging through the example of CrowdHydrology, a distributed network of over 40 stream gages in four states. Signage at the CrowdHydrology gages ask citizen scientists to answer to a simple question via text message: 'What is the water height?'. While these data in no way replace more traditional measurements of stream stage, they do provide low cost supplemental measurements in data scarce regions. Results demonstrate the accuracy of crowdsourced data and provide insight for successful future crowdsourced data collection efforts. A less recognized benefit is that even in data rich areas, crowdsourced data collection is a cost-effective way to perform quality assurance on more sophisticated, and costly, data collection efforts.

Data quality objectives for the B-Cell waste stream classification sampling

International Nuclear Information System (INIS)

Barnett, J.M.

1998-01-01

This document defines the data quality objectives, (DQOS) for sampling the B-Cell racks waste stream. The sampling effort is concentrated on determining a ratio of Cs-137 to Sr-90 and Cs-137 to transuranics (TRU). Figure 1.0 shows the logic path of sampling effort. The flow chart begins with sample and data acquisition and progresses toward (a) statistical confidence and waste classification boundaries, (b) management decisions based on the input parameters and technical methods available, and (c) grout container volume/weight limits and radiation limits. The end result will be accurately classifying the B-Cell rack waste stream

Enhancing Network Data Obliviousness in Trusted Execution Environment-based Stream Processing Systems

KAUST Repository

Alsibyani, Hassan M.

2018-05-15

Cloud computing usage is increasing and a common concern is the privacy and security of the data and computation. Third party cloud environments are not considered fit for processing private information because the data will be revealed to the cloud provider. However, Trusted Execution Environments (TEEs), such as Intel SGX, provide a way for applications to run privately and securely on untrusted platforms. Nonetheless, using a TEE by itself for stream processing systems is not sufficient since network communication patterns may leak properties of the data under processing. This work addresses leaky topology structures and suggests mitigation techniques for each of these. We create specific metrics to evaluate leaks occurring from the network patterns; the metrics measure information leaked when the stream processing system is running. We consider routing techniques for inter-stage communication in a streaming application to mitigate this data leakage. We consider a dynamic policy to change the mitigation technique depending on how much information is currently leaking. Additionally, we consider techniques to hide irregularities resulting from a filtering stage in a topology. We also consider leakages resulting from applications containing cycles. For each of the techniques, we explore their effectiveness in terms of the advantage they provide in overcoming the network leakage. The techniques are tested partly using simulations and some were implemented in a prototype SGX-based stream processing system.

Implicit Unstructured Aerodynamics on Emerging Multi- and Many-Core HPC Architectures

KAUST Repository

Al Farhan, Mohammed A.; Kaushik, Dinesh K.; Keyes, David E.

2017-01-01

Instruction, Multiple Data (SIMD) for hundreds of threads per node. We explore thread-level performance optimizations on state-of-the-art multi- and many-core Intel processors, including the second generation of Xeon Phi, Knights Landing (KNL). We study

Quantifying in-stream nitrate reaction rates using continuously-collected water quality data

Science.gov (United States)

Matthew Miller; Anthony Tesoriero; Paul Capel

2016-01-01

High frequency in situ nitrate data from three streams of varying hydrologic condition, land use, and watershed size were used to quantify the mass loading of nitrate to streams from two sources – groundwater discharge and event flow – at a daily time step for one year. These estimated loadings were used to quantify temporally-variable in-stream nitrate processing ...

Cooperative Coding and Caching for Streaming Data in Multihop Wireless Networks

Directory of Open Access Journals (Sweden)

Dan Wang

2010-01-01

Full Text Available This paper studies the distributed caching managements for the current flourish of the streaming applications in multihop wireless networks. Many caching managements to date use randomized network coding approach, which provides an elegant solution for ubiquitous data accesses in such systems. However, the encoding, essentially a combination operation, makes the coded data difficult to be changed. In particular, to accommodate new data, the system may have to first decode all the combined data segments, remove some unimportant ones, and then reencode the data segments again. This procedure is clearly expensive for continuously evolving data storage. As such, we introduce a novel Cooperative Coding and Caching (C3 scheme, which allows decoding-free data removal through a triangle-like codeword organization. Its decoding performance is very close to the conventional network coding with only a sublinear overhead. Our scheme offers a promising solution to the caching management for streaming data.

FORTRAN computer programs to process Savannah River Laboratory hydrogeochemical and stream-sediment reconnaissance data

International Nuclear Information System (INIS)

Zinkl, R.J.; Shettel, D.L. Jr.; D'Andrea, R.F. Jr.

1980-03-01

FORTRAN computer programs have been written to read, edit, and reformat the hydrogeochemical and stream-sediment reconnaissance data produced by Savannah River Laboratory for the National Uranium Resource Evaluation program. The data are presorted by Savannah River Laboratory into stream sediment, ground water, and stream water for each 1 0 x 2 0 quadrangle. Extraneous information is eliminated, and missing analyses are assigned a specific value (-99999.0). Negative analyses are below the detection limit; the absolute value of a negative analysis is assumed to be the detection limit

Approximating Frequent Items in Asynchronous Data Stream over a Sliding Window

Directory of Open Access Journals (Sweden)

Ho-Leung Chan

2011-09-01

Full Text Available In an asynchronous data stream, the data items may be out of order with respect to their original timestamps. This paper studies the space complexity required by a data structure to maintain such a data stream so that it can approximate the set of frequent items over a sliding time window with sufficient accuracy. Prior to our work, the best solution is given by Cormode et al. [1], who gave an O (1/ε log W log (εB/ log W min {log W, 1/ε} log |U|- space data structure that can approximate the frequent items within an ε error bound, where W and B are parameters of the sliding window, and U is the set of all possible item names. We gave a more space-efficient data structure that only requires O (1/ε log W log (εB/ logW log log W space.

Self-adaptive change detection in streaming data with non-stationary distribution

KAUST Repository

Zhang, Xiangliang; Wang, Wei

2010-01-01

Non-stationary distribution, in which the data distribution evolves over time, is a common issue in many application fields, e.g., intrusion detection and grid computing. Detecting the changes in massive streaming data with a non

A multimetric approach for predicting the ecological integrity of New Zealand streams

Directory of Open Access Journals (Sweden)

Clapcott J.E.

2014-01-01

Full Text Available Integrating multiple measures of stream health into a combined metric can provide a holistic assessment of the ecological integrity of a stream. The aim of this study was to develop a multimetric index (MMI of stream integrity based on predictive modelling of national data sets of water quality, macroinvertebrates, fish and ecosystem process metrics. We used a boosted regression tree approach to calculate an observed/expected score for each metric prior to combining metrics in a MMI based on data availability and the strength of predictive models. The resulting MMI provides a geographically meaningful prediction of the ecological integrity of rivers in New Zealand, but identifies limitations in data and approach, providing focus for ongoing research.

StreamMap: Smooth Dynamic Visualization of High-Density Streaming Points.

Science.gov (United States)

Li, Chenhui; Baciu, George; Han, Yu

2018-03-01

Interactive visualization of streaming points for real-time scatterplots and linear blending of correlation patterns is increasingly becoming the dominant mode of visual analytics for both big data and streaming data from active sensors and broadcasting media. To better visualize and interact with inter-stream patterns, it is generally necessary to smooth out gaps or distortions in the streaming data. Previous approaches either animate the points directly or present a sampled static heat-map. We propose a new approach, called StreamMap, to smoothly blend high-density streaming points and create a visual flow that emphasizes the density pattern distributions. In essence, we present three new contributions for the visualization of high-density streaming points. The first contribution is a density-based method called super kernel density estimation that aggregates streaming points using an adaptive kernel to solve the overlapping problem. The second contribution is a robust density morphing algorithm that generates several smooth intermediate frames for a given pair of frames. The third contribution is a trend representation design that can help convey the flow directions of the streaming points. The experimental results on three datasets demonstrate the effectiveness of StreamMap when dynamic visualization and visual analysis of trend patterns on streaming points are required.

Real-Time Joint Streaming Data Processing from Social and Physical Sensors

Science.gov (United States)

Kropivnitskaya, Y. Y.; Qin, J.; Tiampo, K. F.; Bauer, M.

2014-12-01

The results of the technological breakthroughs in computing that have taken place over the last few decades makes it possible to achieve emergency management objectives that focus on saving human lives and decreasing economic effects. In particular, the integration of a wide variety of information sources, including observations from spatially-referenced physical sensors and new social media sources, enables better real-time seismic hazard analysis through distributed computing networks. The main goal of this work is to utilize innovative computational algorithms for better real-time seismic risk analysis by integrating different data sources and processing tools into streaming and cloud computing applications. The Geological Survey of Canada operates the Canadian National Seismograph Network (CNSN) with over 100 high-gain instruments and 60 low-gain or strong motion seismographs. The processing of the continuous data streams from each station of the CNSN provides the opportunity to detect possible earthquakes in near real-time. The information from physical sources is combined to calculate a location and magnitude for an earthquake. The automatically calculated results are not always sufficiently precise and prompt that can significantly reduce the response time to a felt or damaging earthquake. Social sensors, here represented as Twitter users, can provide information earlier to the general public and more rapidly to the emergency planning and disaster relief agencies. We introduce joint streaming data processing from social and physical sensors in real-time based on the idea that social media observations serve as proxies for physical sensors. By using the streams of data in the form of Twitter messages, each of which has an associated time and location, we can extract information related to a target event and perform enhanced analysis by combining it with physical sensor data. Results of this work suggest that the use of data from social media, in conjunction

Studying the effect of the shape parameters on the performance of the darrieus wind turbine using the multiple double disk stream tube theory

International Nuclear Information System (INIS)

Elmabrok, Ali Mohamed; Al-makhlufi, Ahmed A.

2006-01-01

The performance of the Darrieus vertical axis turbine is comparable with that of the more common horizontal axis machines. It has a number of aerodynamic and structural advantages over HAWTS. However the darrieus turbines are not self-starting at low wind speeds which is a considerable disadvantage for a simple small scale installation. Generally, papers concerning vertical axis turbine do not study the behavior of the rotor at low tip speed ratios. Therefore they do not deal with the self starting problems. A number of analytical methods were investigated to see whether they could predict the starting performance of vertical axis turbines. The chosen methods and 'actuator disc theory' for multiple stream tubes. In this paper the multiple stream tube model is applied using two discs in tandem. The computational analysis of all models simulates the blade aerodynamics throughout the full range of incidence from 180 degree centigrade. The effects of varying various geometric parameters of the windmill upon the performance of the rotor are investigated to find a design with improved self starting characteristics. The best agreement between theory and experiment was obtained using the multiple stream tube (double disc) models.(Author)

Efficient Estimation of Dynamic Density Functions with Applications in Streaming Data

KAUST Repository

Qahtan, Abdulhakim Ali Ali

2016-01-01

application is to detect outliers in data streams from sensor networks based on the estimated PDF. The method detects outliers accurately and outperforms baseline methods designed for detecting and cleaning outliers in sensor data. The third application

Predictive analytics on evolving data streams anticipating and adapting to changes in known and unknown contexts

NARCIS (Netherlands)

Pechenizkiy, M.

2015-01-01

Ever increasing volumes of sensor readings, transactional records, web data and event logs call for next generation of big data mining technology providing effective and efficient tools for making use of the streaming data. Predictive analytics on data streams is actively studied in research

Association rule extraction from XML stream data for wireless sensor networks.

Science.gov (United States)

Paik, Juryon; Nam, Junghyun; Kim, Ung Mo; Won, Dongho

2014-07-18

With the advances of wireless sensor networks, they yield massive volumes of disparate, dynamic and geographically-distributed and heterogeneous data. The data mining community has attempted to extract knowledge from the huge amount of data that they generate. However, previous mining work in WSNs has focused on supporting simple relational data structures, like one table per network, while there is a need for more complex data structures. This deficiency motivates XML, which is the current de facto format for the data exchange and modeling of a wide variety of data sources over the web, to be used in WSNs in order to encourage the interchangeability of heterogeneous types of sensors and systems. However, mining XML data for WSNs has two challenging issues: one is the endless data flow; and the other is the complex tree structure. In this paper, we present several new definitions and techniques related to association rule mining over XML data streams in WSNs. To the best of our knowledge, this work provides the first approach to mining XML stream data that generates frequent tree items without any redundancy.

Shifting stream planform state decreases stream productivity yet increases riparian animal production

Science.gov (United States)

Venarsky, Michael P.; Walters, David M.; Hall, Robert O.; Livers, Bridget; Wohl, Ellen

2018-01-01

In the Colorado Front Range (USA), disturbance history dictates stream planform. Undisturbed, old-growth streams have multiple channels and large amounts of wood and depositional habitat. Disturbed streams (wildfires and logging production, emerging aquatic insect flux, and riparian spider biomass. Organic matter and macroinvertebrate production did not differ among sites per unit area (m−2), but values were 2 ×–21 × higher in undisturbed reaches per unit of stream valley (m−1 valley) because total stream area was higher in undisturbed reaches. Insect emergence was similar among streams at the per unit area and per unit of stream valley. However, rescaling insect emergence to per meter of stream bank showed that the emerging insect biomass reaching the stream bank was lower in undisturbed sites because multi-channel reaches had 3 × more stream bank than single-channel reaches. Riparian spider biomass followed the same pattern as emerging aquatic insects, and we attribute this to bottom-up limitation caused by the multi-channeled undisturbed sites diluting prey quantity (emerging insects) reaching the stream bank (riparian spider habitat). These results show that historic landscape disturbances continue to influence stream and riparian communities in the Colorado Front Range. However, these legacy effects are only weakly influencing habitat-specific function and instead are primarily influencing stream–riparian community productivity by dictating both stream planform (total stream area, total stream bank length) and the proportional distribution of specific habitat types (pools vs riffles).

On-stream chemical element monitor

International Nuclear Information System (INIS)

Averitt, O.R.; Dorsch, R.R.

1979-01-01

An apparatus and method for on-stream chemical element monitoring are described wherein a multiplicity of sample streams are flowed continuously through individual analytical cells and fluorescence analyses are performed on the sample streams in sequence, together with a method of controlling the time duration of each analysis as a function of the concomitant radiation exposure of a preselected perforate reference material interposed in the sample-radiation source path

Vectorization for Molecular Dynamics on Intel Xeon Phi Corpocessors

Science.gov (United States)

Yi, Hongsuk

2014-03-01

Many modern processors are capable of exploiting data-level parallelism through the use of single instruction multiple data (SIMD) execution. The new Intel Xeon Phi coprocessor supports 512 bit vector registers for the high performance computing. In this paper, we have developed a hierarchical parallelization scheme for accelerated molecular dynamics simulations with the Terfoff potentials for covalent bond solid crystals on Intel Xeon Phi coprocessor systems. The scheme exploits multi-level parallelism computing. We combine thread-level parallelism using a tightly coupled thread-level and task-level parallelism with 512-bit vector register. The simulation results show that the parallel performance of SIMD implementations on Xeon Phi is apparently superior to their x86 CPU architecture.

An Approximate L p Difference Algorithm for Massive Data Streams

Directory of Open Access Journals (Sweden)

Jessica H. Fong

2001-12-01

Full Text Available Several recent papers have shown how to approximate the difference ∑ i |a i-b i | or ∑|a i-b i | 2 between two functions, when the function values a i and b i are given in a data stream, and their order is chosen by an adversary. These algorithms use little space (much less than would be needed to store the entire stream and little time to process each item in the stream. They approximate with small relative error. Using different techniques, we show how to approximate the L p-difference ∑ i |a i-b i | p for any rational-valued p∈(0,2], with comparable efficiency and error. We also show how to approximate ∑ i |a i-b i | p for larger values of p but with a worse error guarantee. Our results fill in gaps left by recent work, by providing an algorithm that is precisely tunable for the application at hand. These results can be used to assess the difference between two chronologically or physically separated massive data sets, making one quick pass over each data set, without buffering the data or requiring the data source to pause. For example, one can use our techniques to judge whether the traffic on two remote network routers are similar without requiring either router to transmit a copy of its traffic. A web search engine could use such algorithms to construct a library of small ``sketches,'' one for each distinct page on the web; one can approximate the extent to which new web pages duplicate old ones by comparing the sketches of the web pages. Such techniques will become increasingly important as the enormous scale, distributional nature, and one-pass processing requirements of data sets become more commonplace.

Implementation of a Multichannel Serial Data Streaming Algorithm using the Xilinx Serial RapidIO Solution

Science.gov (United States)

Doxley, Charles A.

2016-01-01

In the current world of applications that use reconfigurable technology implemented on field programmable gate arrays (FPGAs), there is a need for flexible architectures that can grow as the systems evolve. A project has limited resources and a fixed set of requirements that development efforts are tasked to meet. Designers must develop robust solutions that practically meet the current customer demands and also have the ability to grow for future performance. This paper describes the development of a high speed serial data streaming algorithm that allows for transmission of multiple data channels over a single serial link. The technique has the ability to change to meet new applications developed for future design considerations. This approach uses the Xilinx Serial RapidIO LOGICORE Solution to implement a flexible infrastructure to meet the current project requirements with the ability to adapt future system designs.

Shifting stream planform state decreases stream productivity yet increases riparian animal production

Science.gov (United States)

Venarsky, Michael P.; Walters, David M.; Hall, Robert O.; Livers, Bridget; Wohl, Ellen

2018-01-01

In the Colorado Front Range (USA), disturbance history dictates stream planform. Undisturbed, old-growth streams have multiple channels and large amounts of wood and depositional habitat. Disturbed streams (wildfires and logging tested how these opposing stream states influenced organic matter, benthic macroinvertebrate secondary production, emerging aquatic insect flux, and riparian spider biomass. Organic matter and macroinvertebrate production did not differ among sites per unit area (m−2), but values were 2 ×–21 × higher in undisturbed reaches per unit of stream valley (m−1 valley) because total stream area was higher in undisturbed reaches. Insect emergence was similar among streams at the per unit area and per unit of stream valley. However, rescaling insect emergence to per meter of stream bank showed that the emerging insect biomass reaching the stream bank was lower in undisturbed sites because multi-channel reaches had 3 × more stream bank than single-channel reaches. Riparian spider biomass followed the same pattern as emerging aquatic insects, and we attribute this to bottom-up limitation caused by the multi-channeled undisturbed sites diluting prey quantity (emerging insects) reaching the stream bank (riparian spider habitat). These results show that historic landscape disturbances continue to influence stream and riparian communities in the Colorado Front Range. However, these legacy effects are only weakly influencing habitat-specific function and instead are primarily influencing stream–riparian community productivity by dictating both stream planform (total stream area, total stream bank length) and the proportional distribution of specific habitat types (pools vs riffles).

Functional trait composition of aquatic plants can serve to disentangle multiple interacting stressors in lowland streams

Energy Technology Data Exchange (ETDEWEB)

Baattrup-Pedersen, Annette, E-mail: abp@bios.au.dk [Department of Bioscience, Aarhus University, Vejlsøvej 25, P.O. Box 314, DK-8600 Silkeborg (Denmark); Göthe, Emma [Department of Bioscience, Aarhus University, Vejlsøvej 25, P.O. Box 314, DK-8600 Silkeborg (Denmark); Riis, Tenna [Department of Bioscience, Aarhus University, Ole Worms Allé 1, Building 1135, Room 217, DK-8000 Aarhus C (Denmark); O' Hare, Matthew T. [Centre for Ecology and Hydrology, Bush Estate, Penicuik EH26 0QB (United Kingdom)

2016-02-01

Historically, close attention has been paid to negative impacts associated with nutrient loads to streams and rivers, but today hydromorphological alterations are considered increasingly implicated when lowland streams do not achieve good ecological status. Here, we explore if trait-abundance patterns of aquatic plants change along gradients in hydromorphological degradation and eutrophication in lowland stream sites located in Denmark. Specifically, we hypothesised that: i) changes in trait-abundance patterns occur along gradients in hydromorphological degradation and ii) trait-abundance patterns can serve to disentangle effects of eutrophication and hydromorphological degradation in lowland streams reflecting that the mechanisms behind changes differ. We used monitoring data from a total of 147 stream reaches with combined data on aquatic plant species abundance, catchment land use, hydromorphological alterations (i.e. planform, cross section, weed cutting) and water chemistry parameters. Traits related to life form, dispersal, reproduction and survival together with ecological preference values for nutrients and light (Ellenberg N and L) were allocated to 41 species representing 79% of the total species pool. We found clear evidence that habitat degradation (hydromorphological alterations and eutrophication) mediated selective changes in the trait-abundance patterns of the plant community. Specific traits could distinguish hydromorphological degradation (free-floating, surface; anchored floating leaves; anchored heterophylly) from eutrophication (free-floating, submerged; leaf area). We provide a conceptual framework for interpretation of how eutrophication and hydromorphological degradation interact and how this is reflected in trait-abundance patterns in aquatic plant communities in lowland streams. Our findings support the merit of trait-based approaches in biomonitoring as they shed light on mechanisms controlling structural changes under environmental

Functional trait composition of aquatic plants can serve to disentangle multiple interacting stressors in lowland streams

International Nuclear Information System (INIS)

Baattrup-Pedersen, Annette; Göthe, Emma; Riis, Tenna; O'Hare, Matthew T.

2016-01-01

Historically, close attention has been paid to negative impacts associated with nutrient loads to streams and rivers, but today hydromorphological alterations are considered increasingly implicated when lowland streams do not achieve good ecological status. Here, we explore if trait-abundance patterns of aquatic plants change along gradients in hydromorphological degradation and eutrophication in lowland stream sites located in Denmark. Specifically, we hypothesised that: i) changes in trait-abundance patterns occur along gradients in hydromorphological degradation and ii) trait-abundance patterns can serve to disentangle effects of eutrophication and hydromorphological degradation in lowland streams reflecting that the mechanisms behind changes differ. We used monitoring data from a total of 147 stream reaches with combined data on aquatic plant species abundance, catchment land use, hydromorphological alterations (i.e. planform, cross section, weed cutting) and water chemistry parameters. Traits related to life form, dispersal, reproduction and survival together with ecological preference values for nutrients and light (Ellenberg N and L) were allocated to 41 species representing 79% of the total species pool. We found clear evidence that habitat degradation (hydromorphological alterations and eutrophication) mediated selective changes in the trait-abundance patterns of the plant community. Specific traits could distinguish hydromorphological degradation (free-floating, surface; anchored floating leaves; anchored heterophylly) from eutrophication (free-floating, submerged; leaf area). We provide a conceptual framework for interpretation of how eutrophication and hydromorphological degradation interact and how this is reflected in trait-abundance patterns in aquatic plant communities in lowland streams. Our findings support the merit of trait-based approaches in biomonitoring as they shed light on mechanisms controlling structural changes under environmental

Geospatial data stream processing in Python using FOSS4G components

CSIR Research Space (South Africa)

McFerren, Graeme

2016-07-01

Full Text Available that Swordfish is stable for long running applications, though its early deployment in particular use cases suggest it is reasonably stable. Swordfish is currently limited to holding state in memory; further work may be necessary to develop... mechanisms to serialise state, especially in use cases where recovery of the streaming topology state may be necessary. A long term view of Swordfish development is the provision of a streaming data management platform, (as is provided by ESRI Geo...

MonetDB/DataCell: Online Analytics in a Streaming Column-Store

NARCIS (Netherlands)

E. Liarou (Erietta); S. Idreos (Stratos); S. Manegold (Stefan); M.L. Kersten (Martin)

2012-01-01

textabstractIn DataCell, we design streaming functionalities in a mod- ern relational database kernel which targets big data analyt- ics. This includes exploitation of both its storage/execution engine and its optimizer infrastructure. We investigate the opportunities and challenges that arise with

MonetDB/DataCell: online analytics in a streaming column-store

NARCIS (Netherlands)

Liarou, E.; Idreos, S.; Manegold, S.; Kersten, M.

2012-01-01

In DataCell, we design streaming functionalities in a modern relational database kernel which targets big data analytics. This includes exploitation of both its storage/execution engine and its optimizer infrastructure. We investigate the opportunities and challenges that arise with such a direction

Towards the Development of a Taxonomy for Visualisation of Streamed Geospatial Data

Science.gov (United States)

Sibolla, B. H.; Van Zyl, T.; Coetzee, S.

2016-06-01

Geospatial data has very specific characteristics that need to be carefully captured in its visualisation, in order for the user and the viewer to gain knowledge from it. The science of visualisation has gained much traction over the last decade as a response to various visualisation challenges. During the development of an open source based, dynamic two-dimensional visualisation library, that caters for geospatial streaming data, it was found necessary to conduct a review of existing geospatial visualisation taxonomies. The review was done in order to inform the design phase of the library development, such that either an existing taxonomy can be adopted or extended to fit the needs at hand. The major challenge in this case is to develop dynamic two dimensional visualisations that enable human interaction in order to assist the user to understand the data streams that are continuously being updated. This paper reviews the existing geospatial data visualisation taxonomies that have been developed over the years. Based on the review, an adopted taxonomy for visualisation of geospatial streaming data is presented. Example applications of this taxonomy are also provided. The adopted taxonomy will then be used to develop the information model for the visualisation library in a further study.

Pilot-Streaming: Design Considerations for a Stream Processing Framework for High-Performance Computing

OpenAIRE

Andre Luckow; Peter Kasson; Shantenu Jha

2016-01-01

This White Paper (submitted to STREAM 2016) identifies an approach to integrate streaming data with HPC resources. The paper outlines the design of Pilot-Streaming, which extends the concept of Pilot-abstraction to streaming real-time data.

Incremental temporal pattern mining using efficient batch-free stream clustering

NARCIS (Netherlands)

Lu, Y.; Hassani, M.; Seidl, T.

2017-01-01

This paper address the problem of temporal pattern mining from multiple data streams containing temporal events. Temporal events are considered as real world events aligned with comprehensive starting and ending timing information rather than simple integer timestamps. Predefined relations, such as

Association Rule Extraction from XML Stream Data for Wireless Sensor Networks

Science.gov (United States)

Paik, Juryon; Nam, Junghyun; Kim, Ung Mo; Won, Dongho

2014-01-01

With the advances of wireless sensor networks, they yield massive volumes of disparate, dynamic and geographically-distributed and heterogeneous data. The data mining community has attempted to extract knowledge from the huge amount of data that they generate. However, previous mining work in WSNs has focused on supporting simple relational data structures, like one table per network, while there is a need for more complex data structures. This deficiency motivates XML, which is the current de facto format for the data exchange and modeling of a wide variety of data sources over the web, to be used in WSNs in order to encourage the interchangeability of heterogeneous types of sensors and systems. However, mining XML data for WSNs has two challenging issues: one is the endless data flow; and the other is the complex tree structure. In this paper, we present several new definitions and techniques related to association rule mining over XML data streams in WSNs. To the best of our knowledge, this work provides the first approach to mining XML stream data that generates frequent tree items without any redundancy. PMID:25046017

A catchment scale evaluation of multiple stressor effects in headwater streams

DEFF Research Database (Denmark)

Rasmussen, J. J.; McKnight, Ursula S.; Loinaz, Maria Christina

2013-01-01

studied 11 headwater streams in the Hove catchment in the Copenhagen region. All sites had substantial physical habitat and water quality impairments due to anthropogenic influence (intensive agriculture, urban settlements, contaminated sites and low base-flow due to water abstraction activities...... insecticides were probably essential contributors to the overall ecological impairment of these streams. Our results suggest that headwater streams should be considered in future management and mitigation plans. Catchment-based management is necessary because several anthropogenic stressors exceeded...

Hydrogeochemical and stream sediment reconnaissance basic data for Lawton NTMS quadrangle, Oklahoma; Texas

International Nuclear Information System (INIS)

1978-01-01

Field and laboratory data are presented for 703 groundwater and 782 stream sediment samples. Statistical and areal distributions of uranium and possible uranium-related variables are displayed. Groundwater data indicate that the most promising areas for potential uranium mineralization occur in the Lower Permian units surrounding the granite outcrops of the Wichita Mountains. Waters from the Hennessey and Clearfork Groups and the Garber Sandstone contain the highest uranium values. Elements associated with the uranium are arsenic, boron, barium, molybdenum, sodium, selenium, and vandium. Stream sediment data indicate that the promising areas for potential uranium mineralization occur around the Wichita Mountains where stream sediments are derived from the Lower Permian Post Oak Conglomerate, Hennessey Group, and Garber Sandstone and from the Cambrian igneous rocks. Other areas of interest occur (1) in the western part of the quadrangle where the sediments are derived from rocks of the El Reno Group, and (2) along the southern border of the quadrangle where the sediments are derived from the Wichita Group

Truck Route Choice Modeling using Large Streams of GPS Data

Science.gov (United States)

2017-07-31

The primary goal of this research was to use large streams of truck-GPS data to analyze travel routes (or paths) chosen by freight trucks to travel between different origin and destination (OD) location pairs in metropolitan regions of Florida. Two s...

Dynamically Scaling Apache Storm for the Analysis of Streaming Data

NARCIS (Netherlands)

Veen, J.S. van der; Waaij, B.D. van der; Lazovik, E.; Wijbrandi, W.E.; Meijer, R.J.

2015-01-01

Stream processing platforms allow applications to analyse incoming data continuously. Several use cases exist that make use of these capabilities, ranging from monitoring of physical infrastructures to pre selecting video surveillance feeds for human inspection. It is difficult to predict how much

Assessing the impact of groundwater contamination on stream water quality by multiple approaches at the groundwater-surface water interface (Invited Presentation)

DEFF Research Database (Denmark)

Bjerg, Poul Løgstrup; Rønde, Vinni Kampman; Balbarini, Nicola

Contaminants such as chlorinated solvents and pesticides, as well as new classes of compounds or emerging micropollutants are extensively produced, utilized and then discarded in society and subsequently released to streams from multiple point and diffuse sources. Sustainable management of water...

Design of Optimized Multimedia Data Streaming Management Using OMDSM over Mobile Networks

Directory of Open Access Journals (Sweden)

Byungjoo Park

2017-01-01

Full Text Available Mobility management is an essential challenge for supporting reliable multimedia data streaming over wireless and mobile networks in the Internet of Things (IoT for location-based mobile marketing applications. The communications among mobile nodes for IoT need to have a seamless handover for delivering high quality multimedia services. The Internet Engineering Task Force (IETF mobility management schemes are the proposals for handling the routing of IPv6 packets to mobile nodes that have moved away from their home network. However, the standard mobility management scheme cannot prevent packet losses due to longer handover latency. In this article, a new enhanced data streaming route optimization scheme is introduced that uses an optimized Transmission Control Protocol (TCP realignment algorithm in order to prevent the packet disordering problem whenever the nodes in the IoT environment are communicating with each other. With the proposed scheme, data packets sequence realignment can be prevented, the packet traffic speed can be controlled, and the TCP performance can be improved. The experimental results show that managing the packet order in proposed new scheme remarkably increases the overall TCP performance over mobile networks within the IoT environment thus ensuring the high quality of service (QoS for multimedia data streaming in location-based mobile marketing applications.

Relative performance of three stream bed stability indices as indicators of stream health.

Science.gov (United States)

Kusnierz, Paul C; Holbrook, Christopher M

2017-10-16

Bed stability is an important stream habitat attribute because it affects geomorphology and biotic communities. Natural resource managers desire indices of bed stability that can be used under a wide range of geomorphic conditions, are biologically meaningful, and are easily incorporated into sampling protocols. To eliminate potential bias due to presence of instream wood and increase precision of stability values, we modified a stream bed instability index (ISI) to include measurements of bankfull depth (d bf ) and median particle diameter (D 50 ) only in riffles and increased the pebble count to decrease variability (i.e., increase precision) in D 50 . The new riffle-based instability index (RISI) was compared to two established indices: ISI and the riffle stability index (RSI). RISI and ISI were strongly associated with each other but neither was closely associated with RSI. RISI and ISI were closely associated with both a diatom- and two macrovertebrate-based stream health indices, but RSI was only weakly associated with the macroinvertebrate indices. Unexpectedly, precision of D 50 did not differ between RISI and ISI. Results suggest that RISI is a viable alternative to both ISI and RSI for evaluating bed stability in multiple stream types. With few data requirements and a simple protocol, RISI may also better conform to riffle-based sampling methods used by some water quality practitioners.

The influence of data characteristics on detecting wetland/stream surface-water connections in the Delmarva Peninsula, Maryland and Delaware

Science.gov (United States)

Vanderhoof, Melanie; Distler, Hayley; Lang, Megan W.; Alexander, Laurie C.

2018-01-01

The dependence of downstream waters on upstream ecosystems necessitates an improved understanding of watershed-scale hydrological interactions including connections between wetlands and streams. An evaluation of such connections is challenging when, (1) accurate and complete datasets of wetland and stream locations are often not available and (2) natural variability in surface-water extent influences the frequency and duration of wetland/stream connectivity. The Upper Choptank River watershed on the Delmarva Peninsula in eastern Maryland and Delaware is dominated by a high density of small, forested wetlands. In this analysis, wetland/stream surface water connections were quantified using multiple wetland and stream datasets, including headwater streams and depressions mapped from a lidar-derived digital elevation model. Surface-water extent was mapped across the watershed for spring 2015 using Landsat-8, Radarsat-2 and Worldview-3 imagery. The frequency of wetland/stream connections increased as a more complete and accurate stream dataset was used and surface-water extent was included, in particular when the spatial resolution of the imagery was finer (i.e., watershed contributing direct surface water runoff to streamflow. This finding suggests that our interpretation of the frequency and duration of wetland/stream connections will be influenced not only by the spatial and temporal characteristics of wetlands, streams and potential flowpaths, but also by the completeness, accuracy and resolution of input datasets.

STREAM2016: Streaming Requirements, Experience, Applications and Middleware Workshop

Energy Technology Data Exchange (ETDEWEB)

Fox, Geoffrey [Indiana Univ., Bloomington, IN (United States); Jha, Shantenu [Rutgers Univ., New Brunswick, NJ (United States); Ramakrishnan, Lavanya [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

2016-10-01

The Department of Energy (DOE) Office of Science (SC) facilities including accelerators, light sources and neutron sources and sensors that study, the environment, and the atmosphere, are producing streaming data that needs to be analyzed for next-generation scientific discoveries. There has been an explosion of new research and technologies for stream analytics arising from the academic and private sectors. However, there has been no corresponding effort in either documenting the critical research opportunities or building a community that can create and foster productive collaborations. The two-part workshop series, STREAM: Streaming Requirements, Experience, Applications and Middleware Workshop (STREAM2015 and STREAM2016), were conducted to bring the community together and identify gaps and future efforts needed by both NSF and DOE. This report describes the discussions, outcomes and conclusions from STREAM2016: Streaming Requirements, Experience, Applications and Middleware Workshop, the second of these workshops held on March 22-23, 2016 in Tysons, VA. STREAM2016 focused on the Department of Energy (DOE) applications, computational and experimental facilities, as well software systems. Thus, the role of “streaming and steering” as a critical mode of connecting the experimental and computing facilities was pervasive through the workshop. Given the overlap in interests and challenges with industry, the workshop had significant presence from several innovative companies and major contributors. The requirements that drive the proposed research directions, identified in this report, show an important opportunity for building competitive research and development program around streaming data. These findings and recommendations are consistent with vision outlined in NRC Frontiers of Data and National Strategic Computing Initiative (NCSI) [1, 2]. The discussions from the workshop are captured as topic areas covered in this report's sections. The report

Simultaneous heat integration and techno-economic optimization of Organic Rankine Cycle (ORC) for multiple waste heat stream recovery

International Nuclear Information System (INIS)

Yu, Haoshui; Eason, John; Biegler, Lorenz T.; Feng, Xiao

2017-01-01

In the past decades, the Organic Rankine Cycle (ORC) has become a promising technology for low and medium temperature energy utilization. In refineries, there are usually multiple waste heat streams to be recovered. From a safety and controllability perspective, using an intermedium (hot water) to recover waste heat before releasing heat to the ORC system is more favorable than direct integration. The mass flowrate of the intermediate hot water stream determines the amount of waste heat recovered and the final hot water temperature affects the thermal efficiency of ORC. Both, in turn, exert great influence on the power output. Therefore, the hot water mass flowrate is a critical decision variable for the optimal design of the system. This study develops a model for techno-economic optimization of an ORC with simultaneous heat recovery and capital cost optimization. The ORC is modeled using rigorous thermodynamics with the concept of state points. The task of waste heat recovery using the hot water intermedium is modeled using the Duran-Grossmann model for simultaneous heat integration and process optimization. The combined model determines the optimal design of an ORC that recovers multiple waste heat streams in a large scale background process using an intermediate heat transfer stream. In particular, the model determines the optimal heat recovery approach temperature (HRAT), the utility load of the background process, and the optimal operating conditions of the ORC simultaneously. The effectiveness of this method is demonstrated with a case study that uses a refinery as the background process. Sensitivity of the optimal solution to the parameters (electricity price, utility cost) is quantified in this paper. - Highlights: • A new model for Organic Rankine cycle design optimization is presented. • Process heat integration and ORC are considered simultaneously. • Rigorous equation oriented models of the ORC are used for accurate results. • Impact of working

Deploy Nalu/Kokkos algorithmic infrastructure with performance benchmarking.

Energy Technology Data Exchange (ETDEWEB)

Domino, Stefan P. [Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Ananthan, Shreyas [Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Knaus, Robert C. [Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Williams, Alan B. [Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

2017-09-29

The former Nalu interior heterogeneous algorithm design, which was originally designed to manage matrix assembly operations over all elemental topology types, has been modified to operate over homogeneous collections of mesh entities. This newly templated kernel design allows for removal of workset variable resize operations that were formerly required at each loop over a Sierra ToolKit (STK) bucket (nominally, 512 entities in size). Extensive usage of the Standard Template Library (STL) std::vector has been removed in favor of intrinsic Kokkos memory views. In this milestone effort, the transition to Kokkos as the underlying infrastructure to support performance and portability on many-core architectures has been deployed for key matrix algorithmic kernels. A unit-test driven design effort has developed a homogeneous entity algorithm that employs a team-based thread parallelism construct. The STK Single Instruction Multiple Data (SIMD) infrastructure is used to interleave data for improved vectorization. The collective algorithm design, which allows for concurrent threading and SIMD management, has been deployed for the core low-Mach element- based algorithm. Several tests to ascertain SIMD performance on Intel KNL and Haswell architectures have been carried out. The performance test matrix includes evaluation of both low- and higher-order methods. The higher-order low-Mach methodology builds on polynomial promotion of the core low-order control volume nite element method (CVFEM). Performance testing of the Kokkos-view/SIMD design indicates low-order matrix assembly kernel speed-up ranging between two and four times depending on mesh loading and node count. Better speedups are observed for higher-order meshes (currently only P=2 has been tested) especially on KNL. The increased workload per element on higher-order meshes bene ts from the wide SIMD width on KNL machines. Combining multiple threads with SIMD on KNL achieves a 4.6x speedup over the baseline, with

Hydrogeochemical and stream sediment reconnaissance basic data for Dickinson NTMS Quadrangle, North Dakota

International Nuclear Information System (INIS)

1980-01-01

Results of a reconnaissance geochemical survey of the Dickinson Quadrangle, North Dakota are reported. Field and laboratory data are presented for 544 groundwater and 554 stream sediment samples. Statistical and areal distributions of uranium and possible uranium-related variables are displayed. A generalized geologic map of the survey area is provided, and pertinent geologic factors which may be of significance in evaluating the potential for uranium mineralization are briefly discussed. Interpretation of the groundwater data indicates that scattered localities in the central portion of the quadrangle appear most promising for uranium mineralization. High values of uranium in this area are usually found in waters of the Sentinel Butte and Tongue River Formations. Uranium is believed to be concentrated in the lignite beds of the Fort Union Group, with concentrations increasing with proximity to the pre-Oligocene unconformity. Stream sediment data indicate high uranium values distributed over the central area of the quadrangle. Uranium in stream sediments does not appear to be associated with any particular geologic unit and is perhaps following a structural trend

Autonomic intrusion detection: Adaptively detecting anomalies over unlabeled audit data streams in computer networks

KAUST Repository

Wang, Wei; Guyet, Thomas; Quiniou, René ; Cordier, Marie-Odile; Masseglia, Florent; Zhang, Xiangliang

2014-01-01

In this work, we propose a novel framework of autonomic intrusion detection that fulfills online and adaptive intrusion detection over unlabeled HTTP traffic streams in computer networks. The framework holds potential for self-managing: self-labeling, self-updating and self-adapting. Our framework employs the Affinity Propagation (AP) algorithm to learn a subject’s behaviors through dynamical clustering of the streaming data. It automatically labels the data and adapts to normal behavior changes while identifies anomalies. Two large real HTTP traffic streams collected in our institute as well as a set of benchmark KDD’99 data are used to validate the framework and the method. The test results show that the autonomic model achieves better results in terms of effectiveness and efficiency compared to adaptive Sequential Karhunen–Loeve method and static AP as well as three other static anomaly detection methods, namely, k-NN, PCA and SVM.

Autonomic intrusion detection: Adaptively detecting anomalies over unlabeled audit data streams in computer networks

KAUST Repository

Wang, Wei

2014-06-22

In this work, we propose a novel framework of autonomic intrusion detection that fulfills online and adaptive intrusion detection over unlabeled HTTP traffic streams in computer networks. The framework holds potential for self-managing: self-labeling, self-updating and self-adapting. Our framework employs the Affinity Propagation (AP) algorithm to learn a subject’s behaviors through dynamical clustering of the streaming data. It automatically labels the data and adapts to normal behavior changes while identifies anomalies. Two large real HTTP traffic streams collected in our institute as well as a set of benchmark KDD’99 data are used to validate the framework and the method. The test results show that the autonomic model achieves better results in terms of effectiveness and efficiency compared to adaptive Sequential Karhunen–Loeve method and static AP as well as three other static anomaly detection methods, namely, k-NN, PCA and SVM.

Towards a streaming model for nested data parallelism

DEFF Research Database (Denmark)

Madsen, Frederik Meisner; Filinski, Andrzej

2013-01-01

The language-integrated cost semantics for nested data parallelism pioneered by NESL provides an intuitive, high-level model for predicting performance and scalability of parallel algorithms with reasonable accuracy. However, this predictability, obtained through a uniform, parallelism-flattening......The language-integrated cost semantics for nested data parallelism pioneered by NESL provides an intuitive, high-level model for predicting performance and scalability of parallel algorithms with reasonable accuracy. However, this predictability, obtained through a uniform, parallelism......-processable in a streaming fashion. This semantics is directly compatible with previously proposed piecewise execution models for nested data parallelism, but allows the expected space usage to be reasoned about directly at the source-language level. The language definition and implementation are still very much work...

Gesture Recognition from Data Streams of Human Motion Sensor Using Accelerated PSO Swarm Search Feature Selection Algorithm

Directory of Open Access Journals (Sweden)

Simon Fong

2015-01-01

Full Text Available Human motion sensing technology gains tremendous popularity nowadays with practical applications such as video surveillance for security, hand signing, and smart-home and gaming. These applications capture human motions in real-time from video sensors, the data patterns are nonstationary and ever changing. While the hardware technology of such motion sensing devices as well as their data collection process become relatively mature, the computational challenge lies in the real-time analysis of these live feeds. In this paper we argue that traditional data mining methods run short of accurately analyzing the human activity patterns from the sensor data stream. The shortcoming is due to the algorithmic design which is not adaptive to the dynamic changes in the dynamic gesture motions. The successor of these algorithms which is known as data stream mining is evaluated versus traditional data mining, through a case of gesture recognition over motion data by using Microsoft Kinect sensors. Three different subjects were asked to read three comic strips and to tell the stories in front of the sensor. The data stream contains coordinates of articulation points and various positions of the parts of the human body corresponding to the actions that the user performs. In particular, a novel technique of feature selection using swarm search and accelerated PSO is proposed for enabling fast preprocessing for inducing an improved classification model in real-time. Superior result is shown in the experiment that runs on this empirical data stream. The contribution of this paper is on a comparative study between using traditional and data stream mining algorithms and incorporation of the novel improved feature selection technique with a scenario where different gesture patterns are to be recognized from streaming sensor data.

Performance Modeling in CUDA Streams - A Means for High-Throughput Data Processing.

Science.gov (United States)

Li, Hao; Yu, Di; Kumar, Anand; Tu, Yi-Cheng

2014-10-01

Push-based database management system (DBMS) is a new type of data processing software that streams large volume of data to concurrent query operators. The high data rate of such systems requires large computing power provided by the query engine. In our previous work, we built a push-based DBMS named G-SDMS to harness the unrivaled computational capabilities of modern GPUs. A major design goal of G-SDMS is to support concurrent processing of heterogenous query processing operations and enable resource allocation among such operations. Understanding the performance of operations as a result of resource consumption is thus a premise in the design of G-SDMS. With NVIDIA's CUDA framework as the system implementation platform, we present our recent work on performance modeling of CUDA kernels running concurrently under a runtime mechanism named CUDA stream . Specifically, we explore the connection between performance and resource occupancy of compute-bound kernels and develop a model that can predict the performance of such kernels. Furthermore, we provide an in-depth anatomy of the CUDA stream mechanism and summarize the main kernel scheduling disciplines in it. Our models and derived scheduling disciplines are verified by extensive experiments using synthetic and real-world CUDA kernels.

Troubled Waters: where Multiple Streams of Inequality Converge in the Math and Science Experiences of Nonprivileged Girls

Science.gov (United States)

Parrott, Laurel; Spatig, Linda; Kusimo, Patricia S.; Carter, Carolyn C.; Keyes, Marian

Water is often hardest to navigate at the confluence of individual streams. As they experience math and science, nonprivileged girls maneuver through roiling waters where the streams of gender, ethnicity, poverty, place, and teaching practices converge. Just as waters of separate streams blend, these issues - too often considered separate factors - become blended and difficult to isolate, and the resulting turbulence produces a bumpy ride. We draw on 3 years of qualitative data collected as part of an intervention program to explore the math and science experiences and perceptions of a group of ethnically diverse, low socioeconomic status rural and urban adolescent Appalachian girls. After describing program and community contexts, we explore "opportunity to leant" issues - specifically, expectations, access to content, and support networks - and examine their schooling experiences against visions of science and math reform and pressures for accountability. Data are discussed within a framework of critical educational theory.

Mahanaxar: quality of service guarantees in high-bandwidth, real-time streaming data storage

Energy Technology Data Exchange (ETDEWEB)

Bigelow, David [Los Alamos National Laboratory; Bent, John [Los Alamos National Laboratory; Chen, Hsing-Bung [Los Alamos National Laboratory; Brandt, Scott [UCSC

2010-04-05

Large radio telescopes, cyber-security systems monitoring real-time network traffic, and others have specialized data storage needs: guaranteed capture of an ultra-high-bandwidth data stream, retention of the data long enough to determine what is 'interesting,' retention of interesting data indefinitely, and concurrent read/write access to determine what data is interesting, without interrupting the ongoing capture of incoming data. Mahanaxar addresses this problem. Mahanaxar guarantees streaming real-time data capture at (nearly) the full rate of the raw device, allows concurrent read and write access to the device on a best-effort basis without interrupting the data capture, and retains data as long as possible given the available storage. It has built in mechanisms for reliability and indexing, can scale to meet arbitrary bandwidth requirements, and handles both small and large data elements equally well. Results from our prototype implementation shows that Mahanaxar provides both better guarantees and better performance than traditional file systems.

2008-09 National Rivers and Streams Assessment Fish Tissue Data Dictionary

Science.gov (United States)

The Office of Science and Technology (OST) is providing the fish tissue results from the 2008-09 National Rivers and Streams Assessment (NRSA). This document includes the “data dictionary” for Mercury, Selenium, PBDEs, PCBs, Pesticides and PFCs.

A reference web architecture and patterns for real-time visual analytics on large streaming data

Science.gov (United States)

Kandogan, Eser; Soroker, Danny; Rohall, Steven; Bak, Peter; van Ham, Frank; Lu, Jie; Ship, Harold-Jeffrey; Wang, Chun-Fu; Lai, Jennifer

2013-12-01

Monitoring and analysis of streaming data, such as social media, sensors, and news feeds, has become increasingly important for business and government. The volume and velocity of incoming data are key challenges. To effectively support monitoring and analysis, statistical and visual analytics techniques need to be seamlessly integrated; analytic techniques for a variety of data types (e.g., text, numerical) and scope (e.g., incremental, rolling-window, global) must be properly accommodated; interaction, collaboration, and coordination among several visualizations must be supported in an efficient manner; and the system should support the use of different analytics techniques in a pluggable manner. Especially in web-based environments, these requirements pose restrictions on the basic visual analytics architecture for streaming data. In this paper we report on our experience of building a reference web architecture for real-time visual analytics of streaming data, identify and discuss architectural patterns that address these challenges, and report on applying the reference architecture for real-time Twitter monitoring and analysis.

Knoxville 10 x 20 NTMS area, North Carolina, South Carolina, and Tennessee: data release. Hydrogeochemical and stream sediment reconnaissance

International Nuclear Information System (INIS)

Baucom, E.I.; Ferguson, R.B.

1979-05-01

Stream sediment and stream water samples were collected from small streams at 1430 sites or at a nominal density of one site per 14 square kilometers (five square miles) in rural areas. Ground water samples were collected at 791 sites or at a nominal density of one site per 25 square kilometers (ten square miles). Neutron activation analysis (NAA) results are given for uranium and 16 other elements in sediments, and for uranium and 9 other elements in ground water. Key data from ground water sites include (1) water chemistry measurements (pH, conductivity, and alkalinity), (2) well depth, (3) elemental analyses (U, Br, Cl, F, Mg, Mn, Na, and V). Supplementary data include site descriptors (well age, frequency of use of well, etc.) and tabulated analytical data for Al and Dy. Key data from stream sediment sites include (1) water quality measurements (pH, conductivity, and alkalinity), and (2) important elemental analyses (U, Th, Hf, Al, Ce, Fe, Mn, Na, Sc, Ti, and V). Supplementary data from stream sediment sites include sample site descriptors (stream characteristics, vegetation, etc.) and additional elemental analyses

A programming framework for data streaming on the Xeon Phi

Science.gov (United States)

Chapeland, S.; ALICE Collaboration

2017-10-01

ALICE (A Large Ion Collider Experiment) is the dedicated heavy-ion detector studying the physics of strongly interacting matter and the quark-gluon plasma at the CERN LHC (Large Hadron Collider). After the second long shut-down of the LHC, the ALICE detector will be upgraded to cope with an interaction rate of 50 kHz in Pb-Pb collisions, producing in the online computing system (O2) a sustained throughput of 3.4 TB/s. This data will be processed on the fly so that the stream to permanent storage does not exceed 90 GB/s peak, the raw data being discarded. In the context of assessing different computing platforms for the O2 system, we have developed a framework for the Intel Xeon Phi processors (MIC). It provides the components to build a processing pipeline streaming the data from the PC memory to a pool of permanent threads running on the MIC, and back to the host after processing. It is based on explicit offloading mechanisms (data transfer, asynchronous tasks) and basic building blocks (FIFOs, memory pools, C++11 threads). The user only needs to implement the processing method to be run on the MIC. We present in this paper the architecture, implementation, and performance of this system.

Effects of forest harvesting on summer stream temperatures in New Brunswick, Canada: an inter-catchment, multiple-year comparison

Directory of Open Access Journals (Sweden)

C. P.-A. Bourque

2001-01-01

Full Text Available This paper presents a pre- and post-harvest comparison of stream temperatures collected in five neighbouring streams (sub-catchments over a period of five years (1994-1998. The aim of the study was to determine whether land cover changes from clear cutting in areas outside forest buffer zones (applied to streams >0.5 m wide might contribute to an increase in summer mean stream temperatures in buffered streams downslope by infusion of warmed surface and sub-surface water into the streams. Specific relationships were observed in all five forest streams investigated. To assist in the analysis, several spatially-relevant variables, such as land cover change, mid-summer potential solar radiation, flow accumulation, stream location and slope of the land were determined, in part, from existing aerial photographs, GIS-archived forest inventory data and a digital terrain model of the study area. Spatial calculations of insolation levels for July 15th were used as an index of mid-summer solar heating across sub-catchments. Analysis indicated that prior to the 1995 harvest, differences in stream temperature could be attributed to (i topographic position and catchment-to-sun orientation, (ii the level of cutting that occurred in the upper catchment prior to the start of the study, and (iii the average slope within harvested areas. Compared to the pre-harvest mean stream temperatures in 1994, mean temperatures in the three streams downslope from the 1995 harvest areas increased by 0.3 to 0.7°C (representing a 4-8% increase; p-value of normalised temperatures Keywords: terrain attributes, solar radiation, land cover, forest buffers, New Brunswick regulations, spatial modelling, DEM, forest covertypes

Stream processing health card application.

Science.gov (United States)

Polat, Seda; Gündem, Taflan Imre

2012-10-01

In this paper, we propose a data stream management system embedded to a smart card for handling and storing user specific summaries of streaming data coming from medical sensor measurements and/or other medical measurements. The data stream management system that we propose for a health card can handle the stream data rates of commonly known medical devices and sensors. It incorporates a type of context awareness feature that acts according to user specific information. The proposed system is cheap and provides security for private data by enhancing the capabilities of smart health cards. The stream data management system is tested on a real smart card using both synthetic and real data.

No double-dissociation between optic ataxia and visual agnosia: Multiple sub-streams for multiple visuo-manual integrations

NARCIS (Netherlands)

Pisella, L.; Binkofski, F.; Lasek, K.; Toni, I.; Rossetti, Y.

2006-01-01

The current dominant view of the visual system is marked by the functional and anatomical dissociation between a ventral stream specialised for perception and a dorsal stream specialised for action. The "double-dissociation" between visual agnosia (VA), a deficit of visual recognition, and optic

The Open Source DataTurbine Initiative: Streaming Data Middleware for Environmental Observing Systems

Science.gov (United States)

Fountain T.; Tilak, S.; Shin, P.; Hubbard, P.; Freudinger, L.

2009-01-01

The Open Source DataTurbine Initiative is an international community of scientists and engineers sharing a common interest in real-time streaming data middleware and applications. The technology base of the OSDT Initiative is the DataTurbine open source middleware. Key applications of DataTurbine include coral reef monitoring, lake monitoring and limnology, biodiversity and animal tracking, structural health monitoring and earthquake engineering, airborne environmental monitoring, and environmental sustainability. DataTurbine software emerged as a commercial product in the 1990 s from collaborations between NASA and private industry. In October 2007, a grant from the USA National Science Foundation (NSF) Office of Cyberinfrastructure allowed us to transition DataTurbine from a proprietary software product into an open source software initiative. This paper describes the DataTurbine software and highlights key applications in environmental monitoring.

Scalable Stream Coding for Adaptive Foveation Enhanced Percept Multimedia Information Communication for Interactive Medical Applications

National Research Council Canada - National Science Library

Khan, Javed

2003-01-01

.... The demonstrated systems include interactive perceptual transcoding where real-time eye-tracker data fuses with a passing stream, the active subnet diffusion coding-- where multiple active nodes...

Construction of a digital elevation model: methods and parallelization

International Nuclear Information System (INIS)

Mazzoni, Christophe

1995-01-01

The aim of this work is to reduce the computation time needed to produce the Digital Elevation Models (DEM) by using a parallel machine. It is made in collaboration between the French 'Institut Geographique National' (IGN) and the Laboratoire d'Electronique de Technologie et d'Instrumentation (LETI) of the French Atomic Energy Commission (CEA). The IGN has developed a system which provides DEM that is used to produce topographic maps. The kernel of this system is the correlator, a software which automatically matches pairs of homologous points of a stereo-pair of photographs. Nevertheless the correlator is expensive In computing time. In order to reduce computation time and to produce the DEM with same accuracy that the actual system, we have parallelized the IGN's correlator on the OPENVISION system. This hardware solution uses a SIMD (Single Instruction Multiple Data) parallel machine SYMPATI-2, developed by the LETI that is involved in parallel architecture and image processing. Our analysis of the implementation has demonstrated the difficulty of efficient coupling between scalar and parallel structure. So we propose solutions to reinforce this coupling. In order to accelerate more the processing we evaluate SYMPHONIE, a SIMD calculator, successor of SYMPATI-2. On an other hand, we developed a multi-agent approach for what a MIMD (Multiple Instruction, Multiple Data) architecture is available. At last, we describe a Multi-SIMD architecture that conciliates our two approaches. This architecture offers a capacity to apprehend efficiently multi-level treatment image. It is flexible by its modularity, and its communication network supplies reliability that interest sensible systems. (author) [fr

Baseline Channel Geometry and Aquatic Habitat Data for Selected Streams in the Matanuska-Susitna Valley, Alaska

Science.gov (United States)

Curran, Janet H.; Rice, William J.

2009-01-01

Small streams in the rapidly developing Matanuska-Susitna Valley in south-central Alaska are known to support anadromous and resident fish but little is known about their hydrologic and riparian conditions, or their sensitivity to the rapid development of the area or climate variability. To help address this need, channel geometry and aquatic habitat data were collected in 2005 as a baseline of stream conditions for selected streams. Three streams were selected as representative of various stream types, and one drainage network, the Big Lake drainage basin, was selected for a systematic assessment. Streams in the Big Lake basin were drawn in a Geographic Information System (GIS), and 55 reaches along 16 miles of Meadow Creek and its primary tributary Little Meadow Creek were identified from orthoimagery and field observations on the basis of distinctive physical and habitat parameters, most commonly gradient, substrate, and vegetation. Data-collection methods for sites at the three representative reaches and the 55 systematically studied reaches consisted of a field survey of channel and flood-plain geometry and collection of 14 habitat attributes using published protocols or slight modifications. Width/depth and entrenchment ratios along the Meadow-Little Meadow Creek corridor were large and highly variable upstream of Parks Highway and lower and more consistent downstream of Parks Highway. Channel width was strongly correlated with distance, increasing downstream in a log-linear relation. Runs formed the most common habitat type, and instream vegetation dominated the habitat cover types, which collectively covered 53 percent of the channel. Gravel suitable for spawning covered isolated areas along Meadow Creek and about 29 percent of Little Meadow Creek. Broad wetlands were common along both streams. For a comprehensive assessment of small streams in the Mat-Su Valley, critical additional data needs include hydrologic, geologic and geomorphic, and biologic data

The Pacific northwest stream quality assessment

Science.gov (United States)

Van Metre, Peter C.; Morace, Jennifer L.; Sheibley, Rich W.

2015-01-01

In 2015, the U.S. Geological Survey (USGS) National Water-Quality Assessment (NAWQA) program is assessing stream quality in the Pacific Northwest. The goals of the Pacific Northwest Stream Quality Assessment (Pacific Northwest study) are to assess the quality of streams in the region by characterizing multiple water-quality factors that are stressors to aquatic life and to evaluate the relation between these stressors and biological communities. The effects of urbanization and agriculture on stream quality for the Puget Lowlands and Willamette Valley are the focus of this regional study. Findings will provide the public and policymakers with information regarding which human and environmental factors are the most critical in affecting stream quality and, thus, provide insights about possible approaches to protect or improve the health of streams in the region.

A New Study of Two Divergence Metrics for Change Detection in Data Streams

KAUST Repository

Qahtan, Abdulhakim Ali Ali; Wang, Suojin; Carroll, Raymond; Zhang, Xiangliang

2014-01-01

Streaming data are dynamic in nature with frequent changes. To detect such changes, most methods measure the difference between the data distributions in a current time window and a reference window. Divergence metrics and density estimation are required to measure the difference between the data distributions. Our study shows that the Kullback-Leibler (KL) divergence, the most popular metric for comparing distributions, fails to detect certain changes due to its asymmetric property and its dependence on the variance of the data. We thus consider two metrics for detecting changes in univariate data streams: a symmetric KL-divergence and a divergence metric measuring the intersection area of two distributions. The experimental results show that these two metrics lead to more accurate results in change detection than baseline methods such as Change Finder and using conventional KL-divergence.

A New Study of Two Divergence Metrics for Change Detection in Data Streams

KAUST Repository

Qahtan, Abdulhakim Ali Ali

2014-08-01

Streaming data are dynamic in nature with frequent changes. To detect such changes, most methods measure the difference between the data distributions in a current time window and a reference window. Divergence metrics and density estimation are required to measure the difference between the data distributions. Our study shows that the Kullback-Leibler (KL) divergence, the most popular metric for comparing distributions, fails to detect certain changes due to its asymmetric property and its dependence on the variance of the data. We thus consider two metrics for detecting changes in univariate data streams: a symmetric KL-divergence and a divergence metric measuring the intersection area of two distributions. The experimental results show that these two metrics lead to more accurate results in change detection than baseline methods such as Change Finder and using conventional KL-divergence.

High-speed packet filtering utilizing stream processors

Science.gov (United States)

Hummel, Richard J.; Fulp, Errin W.

2009-04-01

Parallel firewalls offer a scalable architecture for the next generation of high-speed networks. While these parallel systems can be implemented using multiple firewalls, the latest generation of stream processors can provide similar benefits with a significantly reduced latency due to locality. This paper describes how the Cell Broadband Engine (CBE), a popular stream processor, can be used as a high-speed packet filter. Results show the CBE can potentially process packets arriving at a rate of 1 Gbps with a latency less than 82 μ-seconds. Performance depends on how well the packet filtering process is translated to the unique stream processor architecture. For example the method used for transmitting data and control messages among the pseudo-independent processor cores has a significant impact on performance. Experimental results will also show the current limitations of a CBE operating system when used to process packets. Possible solutions to these issues will be discussed.

Evaluating habitat associations of a fish assemblage at multiple spatial scales in a minimally disturbed stream using low-cost remote sensing

Science.gov (United States)

Cheek, Brandon D.; Grabowski, Timothy B.; Bean, Preston T.; Groeschel, Jillian R.; Magnelia, Stephan J.

2016-01-01

Habitat heterogeneity at multiple scales is a major factor affecting fish assemblage structure. However, assessments that examine these relationships at multiple scales concurrently are lacking. The lack of assessments at these scales is a critical gap in understanding as conservation and restoration efforts typically work at these levels.A combination of low-cost side-scan sonar surveys, aerial imagery using an unmanned aerial vehicle, and fish collections were used to evaluate the relationship between physicochemical and landscape variables at various spatial scales (e.g. micro-mesohabitat, mesohabitat, channel unit, stream reach) and stream–fish assemblage structure and habitat associations in the South Llano River, a spring-fed second-order stream on the Edwards Plateau in central Texas during 2012–2013.Low-cost side-scan sonar surveys have not typically been used to generate data for riverscape assessments of assemblage structure, thus the secondary objective was to assess the efficacy of this approach.The finest spatial scale (micro-mesohabitat) and the intermediate scale (channel unit) had the greatest explanatory power for variation in fish assemblage structure.Many of the fish endemic to the Edwards Plateau showed similar associations with physicochemical and landscape variables suggesting that conservation and restoration actions targeting a single endemic species may provide benefits to a large proportion of the endemic species in this system.Low-cost side-scan sonar proved to be a cost-effective means of acquiring information on the habitat availability of the entire river length and allowed the assessment of how a full suite of riverscape-level variables influenced local fish assemblage structure.

Analysis of massive data streams using R and AMIDST

DEFF Research Database (Denmark)

Madsen, Anders Læsø; Salmerón, Antonio

methods able to handle massive data streams based on Bayesian networks technology. All of the developed methods are available through the AMIDST toolbox, implemented in Java 8. We show how the functionality of the AMIDST toolbox can be accessed from R. Available AMIDST objects include variables......, distributions and Bayesian networks, as well as those devoted to inference and learning. The interaction between both platforms relies on the rJava package....

Prioritized Contact Transport Stream

Science.gov (United States)

Hunt, Walter Lee, Jr. (Inventor)

2015-01-01

A detection process, contact recognition process, classification process, and identification process are applied to raw sensor data to produce an identified contact record set containing one or more identified contact records. A prioritization process is applied to the identified contact record set to assign a contact priority to each contact record in the identified contact record set. Data are removed from the contact records in the identified contact record set based on the contact priorities assigned to those contact records. A first contact stream is produced from the resulting contact records. The first contact stream is streamed in a contact transport stream. The contact transport stream may include and stream additional contact streams. The contact transport stream may be varied dynamically over time based on parameters such as available bandwidth, contact priority, presence/absence of contacts, system state, and configuration parameters.

Flood-frequency characteristics of Wisconsin streams

Science.gov (United States)

Walker, John F.; Peppler, Marie C.; Danz, Mari E.; Hubbard, Laura E.

2017-05-22

Flood-frequency characteristics for 360 gaged sites on unregulated rural streams in Wisconsin are presented for percent annual exceedance probabilities ranging from 0.2 to 50 using a statewide skewness map developed for this report. Equations of the relations between flood-frequency and drainage-basin characteristics were developed by multiple-regression analyses. Flood-frequency characteristics for ungaged sites on unregulated, rural streams can be estimated by use of the equations presented in this report. The State was divided into eight areas of similar physiographic characteristics. The most significant basin characteristics are drainage area, soil saturated hydraulic conductivity, main-channel slope, and several land-use variables. The standard error of prediction for the equation for the 1-percent annual exceedance probability flood ranges from 56 to 70 percent for Wisconsin Streams; these values are larger than results presented in previous reports. The increase in the standard error of prediction is likely due to increased variability of the annual-peak discharges, resulting in increased variability in the magnitude of flood peaks at higher frequencies. For each of the unregulated rural streamflow-gaging stations, a weighted estimate based on the at-site log Pearson type III analysis and the multiple regression results was determined. The weighted estimate generally has a lower uncertainty than either the Log Pearson type III or multiple regression estimates. For regulated streams, a graphical method for estimating flood-frequency characteristics was developed from the relations of discharge and drainage area for selected annual exceedance probabilities. Graphs for the major regulated streams in Wisconsin are presented in the report.

Multiple-jet thermal mixing in a piping tee

International Nuclear Information System (INIS)

Lykoudis, P.S.; Hagar, R.C.

1979-01-01

Piping tees that are used to mix fluid streams at different temperatures are subjected to possibly severe thermal and mechanical stresses. There is reason to suspect that mixing in a piping tee could be improved by injecting the fluid streams into the tee through multiple jets. This paper reports the results of an experimental investigation of the effects of multiple-jet injection on mixing in a piping tee. The experimental work involves the measurement of the temperature fluctuation intensity with a hot-film sensor downstream of a simple 22.22-mm(7/8-in.)-diam tee with mixed multiple-jet injected hot and cold streams of water. The jets were provided by holes drilled in plates that partially blocked the inlet streams; 26 pairs of plates were investigated. The number of holes per plate varied from 1 to 51; the jet diameters ranged from 5 to 68% of the tee diameter. The inlet stream Reynolds number upstream of the jet plates was roughly 15 500 for each stream. The data indicated that the root mean square (rms) temperature fluctuation intensity measured at the tee outlet decreased dramatically as the jet plate cross-sectional area void fraction was decreased. When the jets emanating from the tee plates were misaligned, the reduction of the rms temperature fluctuation was not as high as when the jets were aligned. The rate of decay of the intensity downstream of the tee for most ofthe plates investigated was found to agree well with the -3/4 power decay law predicted by Corrsin's theory of scalar decay. However, unusual features in the intensity decay data were also observed, such as an increase of the intensity several diameters downstream before continuing to decay

Stream Classification Tool User Manual: For Use in Applications in Hydropower-Related Evironmental Mitigation

Energy Technology Data Exchange (ETDEWEB)

McManamay, Ryan A. [Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Troia, Matthew J. [Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); DeRolph, Christopher R. [Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States); Samu, Nicole M. [Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

2016-01-01

Stream classifications are an inventory of different types of streams. Classifications help us explore similarities and differences among different types of streams, make inferences regarding stream ecosystem behavior, and communicate the complexities of ecosystems. We developed a nested, layered, and spatially contiguous stream classification to characterize the biophysical settings of stream reaches within the Eastern United States (~ 900,000 reaches). The classification is composed of five natural characteristics (hydrology, temperature, size, confinement, and substrate) along with several disturbance regime layers, and each was selected because of their relevance to hydropower mitigation. We developed the classification at the stream reach level using the National Hydrography Dataset Plus Version 1 (1:100k scale). The stream classification is useful to environmental mitigation for hydropower dams in multiple ways. First, it creates efficiency in the regulatory process by creating an objective and data-rich means to address meaningful mitigation actions. Secondly, the SCT addresses data gaps as it quickly provides an inventory of hydrology, temperature, morphology, and ecological communities for the immediate project area, but also surrounding streams. This includes identifying potential reference streams as those that are proximate to the hydropower facility and fall within the same class. These streams can potentially be used to identify ideal environmental conditions or identify desired ecological communities. In doing so, the stream provides some context for how streams may function, respond to dam regulation, and an overview of specific mitigation needs. Herein, we describe the methodology in developing each stream classification layer and provide a tutorial to guide applications of the classification (and associated data) in regulatory settings, such as hydropower (re)licensing.

Multicast Delayed Authentication For Streaming Synchrophasor Data in the Smart Grid.

Science.gov (United States)

Câmara, Sérgio; Anand, Dhananjay; Pillitteri, Victoria; Carmo, Luiz

2016-01-01

Multicast authentication of synchrophasor data is challenging due to the design requirements of Smart Grid monitoring systems such as low security overhead, tolerance of lossy networks, time-criticality and high data rates. In this work, we propose inf -TESLA, Infinite Timed Efficient Stream Loss-tolerant Authentication, a multicast delayed authentication protocol for communication links used to stream synchrophasor data for wide area control of electric power networks. Our approach is based on the authentication protocol TESLA but is augmented to accommodate high frequency transmissions of unbounded length. inf TESLA protocol utilizes the Dual Offset Key Chains mechanism to reduce authentication delay and computational cost associated with key chain commitment. We provide a description of the mechanism using two different modes for disclosing keys and demonstrate its security against a man-in-the-middle attack attempt. We compare our approach against the TESLA protocol in a 2-day simulation scenario, showing a reduction of 15.82% and 47.29% in computational cost, sender and receiver respectively, and a cumulative reduction in the communication overhead.

Reducing False Negative Reads in RFID Data Streams Using an Adaptive Sliding-Window Approach

Directory of Open Access Journals (Sweden)

Herman Vermaak

2012-03-01

Full Text Available Unreliability of the data streams generated by RFID readers is among the primary factors which limit the widespread adoption of the RFID technology. RFID data cleaning is, therefore, an essential task in the RFID middleware systems in order to reduce reading errors, and to allow these data streams to be used to make a correct interpretation and analysis of the physical world they are representing. In this paper we propose an adaptive sliding-window based approach called WSTD which is capable of efficiently coping with both environmental variation and tag dynamics. Our experimental results demonstrate the efficacy of the proposed approach.

Groundwater data improve modelling of headwater stream CO2 outgassing with a stable DIC isotope approach

Science.gov (United States)

Marx, Anne; Conrad, Marcus; Aizinger, Vadym; Prechtel, Alexander; van Geldern, Robert; Barth, Johannes A. C.

2018-05-01

A large portion of terrestrially derived carbon outgasses as carbon dioxide (CO2) from streams and rivers to the atmosphere. Particularly, the amount of CO2 outgassing from small headwater streams is highly uncertain. Conservative estimates suggest that they contribute 36 % (i.e. 0.93 petagrams (Pg) C yr-1) of total CO2 outgassing from all fluvial ecosystems on the globe. In this study, stream pCO2, dissolved inorganic carbon (DIC), and δ13CDIC data were used to determine CO2 outgassing from an acidic headwater stream in the Uhlířská catchment (Czech Republic). This stream drains a catchment with silicate bedrock. The applied stable isotope model is based on the principle that the 13C / 12C ratio of its sources and the intensity of CO2 outgassing control the isotope ratio of DIC in stream water. It avoids the use of the gas transfer velocity parameter (k), which is highly variable and mostly difficult to constrain. Model results indicate that CO2 outgassing contributed more than 80 % to the annual stream inorganic carbon loss in the Uhlířská catchment. This translated to a CO2 outgassing rate from the stream of 34.9 kg C m-2 yr-1 when normalised to the stream surface area. Large temporal variations with maximum values shortly before spring snowmelt and in summer emphasise the need for investigations at higher temporal resolution. We improved the model uncertainty by incorporating groundwater data to better constrain the isotope compositions of initial DIC. Due to the large global abundance of acidic, humic-rich headwaters, we underline the importance of this integral approach for global applications.

Towards automatic parameter tuning of stream processing systems

KAUST Repository

Bilal, Muhammad

2017-09-27

Optimizing the performance of big-data streaming applications has become a daunting and time-consuming task: parameters may be tuned from a space of hundreds or even thousands of possible configurations. In this paper, we present a framework for automating parameter tuning for stream-processing systems. Our framework supports standard black-box optimization algorithms as well as a novel gray-box optimization algorithm. We demonstrate the multiple benefits of automated parameter tuning in optimizing three benchmark applications in Apache Storm. Our results show that a hill-climbing algorithm that uses a new heuristic sampling approach based on Latin Hypercube provides the best results. Our gray-box algorithm provides comparable results while being two to five times faster.

LHCb trigger streams optimization

Science.gov (United States)

Derkach, D.; Kazeev, N.; Neychev, R.; Panin, A.; Trofimov, I.; Ustyuzhanin, A.; Vesterinen, M.

2017-10-01

The LHCb experiment stores around 1011 collision events per year. A typical physics analysis deals with a final sample of up to 107 events. Event preselection algorithms (lines) are used for data reduction. Since the data are stored in a format that requires sequential access, the lines are grouped into several output file streams, in order to increase the efficiency of user analysis jobs that read these data. The scheme efficiency heavily depends on the stream composition. By putting similar lines together and balancing the stream sizes it is possible to reduce the overhead. We present a method for finding an optimal stream composition. The method is applied to a part of the LHCb data (Turbo stream) on the stage where it is prepared for user physics analysis. This results in an expected improvement of 15% in the speed of user analysis jobs, and will be applied on data to be recorded in 2017.

Development of a cross-section based stream package for MODFLOW

Science.gov (United States)

Ou, G.; Chen, X.; Irmak, A.

2012-12-01

Accurate simulation of stream-aquifer interactions for wide rivers using the streamflow routing package in MODFLOW is very challenging. To better represent a wide river spanning over multiple model grid cells, a Cross-Section based streamflow Routing (CSR) package is developed and incorporated into MODFLOW to simulate the interaction between streams and aquifers. In the CSR package, a stream segment is represented as a four-point polygon instead of a polyline which is traditionally used in streamflow routing simulation. Each stream segment is composed of upstream and downstream cross-sections. A cross-section consists of a number of streambed points possessing coordinates, streambed thicknesses and streambed hydraulic conductivities to describe the streambed geometry and hydraulic properties. The left and right end points are used to determine the locations of the stream segments. According to the cross-section geometry and hydraulic properties, CSR calculates the new stream stage at the cross-section using the Brent's method to solve the Manning's Equation. A module is developed to automatically compute the area of the stream segment polygon on each intersected MODFLOW grid cell as the upstream and downstream stages change. The stream stage and streambed hydraulic properties of model grids are interpolated based on the streambed points. Streambed leakage is computed as a function of streambed conductance and difference between the groundwater level and stream stage. The Muskingum-Cunge flow routing scheme with variable parameters is used to simulate the streamflow as the groundwater (discharge or recharge) contributes as lateral flows. An example is used to illustrate the capabilities of the CSR package. The result shows that the CSR is applicable to describing the spatial and temporal variation in the interaction between streams and aquifers. The input data become simple due to that the internal program automatically interpolates the cross-section data to each

The Relationship Between Grazing, Er osion and Adult Aquatic Insects in Streams in Mongolia.

Directory of Open Access Journals (Sweden)

Barbara Hayford

2010-06-01

Full Text Available Overgrazing along stream channels in Mongolia may impact streams by increasing stream channel erosion and in-stream sediments, water temperature, pH, and conductivity. Grazing and erosion impacts may impair stream insects. The Mongolian Aquatic Insect Survey sampled 250 streams during summer seasons in 2003-2006 and 2008. On-site identifi cations of aquatic insect families mostly based on collections of adults were recorded for each site, leading us to ask whether the family-level data were useful in biological assessment related to impacts and impairment from grazing and erosion. A double dendrogram based on hierarchical cluster analysis was used to fi nd patterns in sites and aquatic insect communities. Sites did not group by sampling period, but some sites did group by stream size and elevation. However, elevation was not a signifi cant predictor of variation in aquatic insect metrics. Analysis of variance was used to determine whether insect metrics and water quality variables varied signifi cantly between categories of erosion in the stream channel. Plecoptera and Diptera richness decreased with increased erosion and Percent Diptera Richness was the only aquatic insect metric to vary signifi cantly between categories of erosion along the stream channel. Water temperature, conductivity, and pH also signifi cantly increased with increased erosion. Multiple regression analysis was used to determine whether aquatic insect metrics could be predicted by variation in landscape, water quality and stream reach variables. Trichoptera, Ephemeroptera, and Coleoptera richness increased with increased erosion, conductivity, and pH, but not signifi cantly. Percent Diptera Richness formed the only signifi cant model in the multiple regression analysis, with conductivity the only signifi cant predictor of variation in Percent Diptera Richness. Family-level data generated in the fi eld indicated that sampling for Trichoptera and Ephemeroptera diversity would

Groundwater data improve modelling of headwater stream CO2 outgassing with a stable DIC isotope approach

Directory of Open Access Journals (Sweden)

A. Marx

2018-05-01

Full Text Available A large portion of terrestrially derived carbon outgasses as carbon dioxide (CO2 from streams and rivers to the atmosphere. Particularly, the amount of CO2 outgassing from small headwater streams is highly uncertain. Conservative estimates suggest that they contribute 36 % (i.e. 0.93 petagrams (Pg C yr−1 of total CO2 outgassing from all fluvial ecosystems on the globe. In this study, stream pCO2, dissolved inorganic carbon (DIC, and δ13CDIC data were used to determine CO2 outgassing from an acidic headwater stream in the Uhlířská catchment (Czech Republic. This stream drains a catchment with silicate bedrock. The applied stable isotope model is based on the principle that the 13C ∕ 12C ratio of its sources and the intensity of CO2 outgassing control the isotope ratio of DIC in stream water. It avoids the use of the gas transfer velocity parameter (k, which is highly variable and mostly difficult to constrain. Model results indicate that CO2 outgassing contributed more than 80 % to the annual stream inorganic carbon loss in the Uhlířská catchment. This translated to a CO2 outgassing rate from the stream of 34.9 kg C m−2 yr−1 when normalised to the stream surface area. Large temporal variations with maximum values shortly before spring snowmelt and in summer emphasise the need for investigations at higher temporal resolution. We improved the model uncertainty by incorporating groundwater data to better constrain the isotope compositions of initial DIC. Due to the large global abundance of acidic, humic-rich headwaters, we underline the importance of this integral approach for global applications.

Ensemble Classification of Data Streams Based on Attribute Reduction and a Sliding Window

Directory of Open Access Journals (Sweden)

Yingchun Chen

2018-04-01

Full Text Available With the current increasing volume and dimensionality of data, traditional data classification algorithms are unable to satisfy the demands of practical classification applications of data streams. To deal with noise and concept drift in data streams, we propose an ensemble classification algorithm based on attribute reduction and a sliding window in this paper. Using mutual information, an approximate attribute reduction algorithm based on rough sets is used to reduce data dimensionality and increase the diversity of reduced results in the algorithm. A double-threshold concept drift detection method and a three-stage sliding window control strategy are introduced to improve the performance of the algorithm when dealing with both noise and concept drift. The classification precision is further improved by updating the base classifiers and their nonlinear weights. Experiments on synthetic datasets and actual datasets demonstrate the performance of the algorithm in terms of classification precision, memory use, and time efficiency.

Using macroinvertebrate assemblages and multiple stressors to infer urban stream system condition: A case study in the central US

Science.gov (United States)

Nichols, John W.; Hubbart, Jason A.; Poulton, Barry C.

2016-01-01

Characterizing the impacts of hydrologic alterations, pollutants, and habitat degradation on macroinvertebrate species assemblages is of critical value for managers wishing to categorize stream ecosystem condition. A combination of approaches including trait-based metrics and traditional bioassessments provides greater information, particularly in anthropogenic stream ecosystems where traditional approaches can be confounded by variously interacting land use impacts. Macroinvertebrates were collected from two rural and three urban nested study sites in central Missouri, USA during the spring and fall seasons of 2011. Land use responses of conventional taxonomic and trait-based metrics were compared to streamflow indices, physical habitat metrics, and water quality indices. Results show that biotic index was significantly different (p habitats in urban reaches contained 21 % more (p = 0.03) multivoltine organisms, which was positively correlated to the magnitude of peak flows (r2 = 0.91, p = 0.012) suggesting that high flow events may serve as a disturbance in those areas. Results support the use of macroinvertebrate assemblages and multiple stressors to characterize urban stream system condition and highlight the need to better understand the complex interactions of trait-based metrics and anthropogenic aquatic ecosystem stressors.

The Northeast Stream Quality Assessment

Science.gov (United States)

Van Metre, Peter C.; Riva-Murray, Karen; Coles, James F.

2016-04-22

In 2016, the U.S. Geological Survey (USGS) National Water-Quality Assessment (NAWQA) is assessing stream quality in the northeastern United States. The goal of the Northeast Stream Quality Assessment (NESQA) is to assess the quality of streams in the region by characterizing multiple water-quality factors that are stressors to aquatic life and evaluating the relation between these stressors and biological communities. The focus of NESQA in 2016 will be on the effects of urbanization and agriculture on stream quality in all or parts of eight states: Connecticut, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, and Vermont.Findings will provide the public and policymakers with information about the most critical factors affecting stream quality, thus providing insights about possible approaches to protect the health of streams in the region. The NESQA study will be the fourth regional study conducted as part of NAWQA and will be of similar design and scope to the first three, in the Midwest in 2013, the Southeast in 2014, and the Pacific Northwest in 2015 (http://txpub.usgs.gov/RSQA/).

Hydrogeologic data for the Big River-Mishnock River stream-aquifer system, central Rhode Island

Science.gov (United States)

Craft, P.A.

2001-01-01

Hydrogeology, ground-water development alternatives, and water quality in the BigMishnock stream-aquifer system in central Rhode Island are being investigated as part of a long-term cooperative program between the Rhode Island Water Resources Board and the U.S. Geological Survey to evaluate the ground-water resources throughout Rhode Island. The study area includes the Big River drainage basin and that portion of the Mishnock River drainage basin upstream from the Mishnock River at State Route 3. This report presents geologic data and hydrologic and water-quality data for ground and surface water. Ground-water data were collected from July 1996 through September 1998 from a network of observation wells consisting of existing wells and wells installed for this study, which provided a broad distribution of data-collection sites throughout the study area. Streambed piezometers were used to obtain differences in head data between surface-water levels and ground-water levels to help evaluate stream-aquifer interactions throughout the study area. The types of data presented include monthly ground-water levels, average daily ground-water withdrawals, drawdown data from aquifer tests, and water-quality data. Historical water-level data from other wells within the study area also are presented in this report. Surface-water data were obtained from a network consisting of surface-water impoundments, such as ponds and reservoirs, existing and newly established partial-record stream-discharge sites, and synoptic surface-water-quality sites. Water levels were collected monthly from the surface-water impoundments. Stream-discharge measurements were made at partial-record sites to provide measurements of inflow, outflow, and internal flow throughout the study area. Specific conductance was measured monthly at partial-record sites during the study, and also during the fall and spring of 1997 and 1998 at 41 synoptic sites throughout the study area. General geologic data, such as

Low-dimensional and Data Fusion Techniques Applied to a Rectangular Supersonic Multi-stream Jet

Science.gov (United States)

Berry, Matthew; Stack, Cory; Magstadt, Andrew; Ali, Mohd; Gaitonde, Datta; Glauser, Mark

2017-11-01

Low-dimensional models of experimental and simulation data for a complex supersonic jet were fused to reconstruct time-dependent proper orthogonal decomposition (POD) coefficients. The jet consists of a multi-stream rectangular single expansion ramp nozzle, containing a core stream operating at Mj , 1 = 1.6 , and bypass stream at Mj , 3 = 1.0 with an underlying deck. POD was applied to schlieren and PIV data to acquire the spatial basis functions. These eigenfunctions were projected onto their corresponding time-dependent large eddy simulation (LES) fields to reconstruct the temporal POD coefficients. This reconstruction was able to resolve spectral peaks that were previously aliased due to the slower sampling rates of the experiments. Additionally, dynamic mode decomposition (DMD) was applied to the experimental and LES datasets, and the spatio-temporal characteristics were compared to POD. The authors would like to acknowledge AFOSR, program manager Dr. Doug Smith, for funding this research, Grant No. FA9550-15-1-0435.

Adaptive Framework for Classification and Novel Class Detection over Evolving Data Streams with Limited Labeled Data.

Energy Technology Data Exchange (ETDEWEB)

Haque, Ahsanul [Univ. of Texas, Dallas, TX (United States); Khan, Latifur [Univ. of Texas, Dallas, TX (United States); Baron, Michael [Univ. of Texas, Dallas, TX (United States); Ingram, Joey Burton [Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

2015-09-01

Most approaches to classifying evolving data streams either divide the stream of data into fixed-size chunks or use gradual forgetting to address the problems of infinite length and concept drift. Finding the fixed size of the chunks or choosing a forgetting rate without prior knowledge about time-scale of change is not a trivial task. As a result, these approaches suffer from a trade-off between performance and sensitivity. To address this problem, we present a framework which uses change detection techniques on the classifier performance to determine chunk boundaries dynamically. Though this framework exhibits good performance, it is heavily dependent on the availability of true labels of data instances. However, labeled data instances are scarce in realistic settings and not readily available. Therefore, we present a second framework which is unsupervised in nature, and exploits change detection on classifier confidence values to determine chunk boundaries dynamically. In this way, it avoids the use of labeled data while still addressing the problems of infinite length and concept drift. Moreover, both of our proposed frameworks address the concept evolution problem by detecting outliers having similar values for the attributes. We provide theoretical proof that our change detection method works better than other state-of-the-art approaches in this particular scenario. Results from experiments on various benchmark and synthetic data sets also show the efficiency of our proposed frameworks.

Analysis of hydraulic characteristics for stream diversion in small stream

Energy Technology Data Exchange (ETDEWEB)

Ahn, Sang-Jin; Jun, Kye-Won [Chungbuk National University, Cheongju(Korea)

2001-10-31

This study is the analysis of hydraulic characteristics for stream diversion reach by numerical model test. Through it we can provide the basis data in flood, and in grasping stream flow characteristics. Analysis of hydraulic characteristics in Seoknam stream were implemented by using computer model HEC-RAS(one-dimensional model) and RMA2(two-dimensional finite element model). As a result we became to know that RMA2 to simulate left, main channel, right in stream is more effective method in analysing flow in channel bends, steep slope, complex bed form effect stream flow characteristics, than HEC-RAS. (author). 13 refs., 3 tabs., 5 figs.

Short-term stream flow forecasting at Australian river sites using data-driven regression techniques

CSIR Research Space (South Africa)

Steyn, Melise

2017-09-01

Full Text Available This study proposes a computationally efficient solution to stream flow forecasting for river basins where historical time series data are available. Two data-driven modeling techniques are investigated, namely support vector regression...

The end of polling: why and how to transform a REST API into a Data Streaming API?

CERN Multimedia

CERN. Geneva

2017-01-01

We know interactivity is the key to keep our user’s interest alive but we can’t reduce animation to UI anymore. Twitter, Waze, Slack… users are used to have real-time data in applications they love. But how can you turn your static API into a stream of data? By pulling? Pushing? Webhook-ing? When talking about data streaming, we often think about WebSockets. But have you ever heard of Server-Sent Events? In this tools-in-action we will compare those technologies to understand which one you should opt for depending on your usecase, and I’ll show you how we have been reducing the amount of data to transfer even further with JSON-Patch. And because real-time data is not only needed by web (and because it’s much more fun), I’ll show you how we can make a drone dance on streamed APIs.

Deep SOMs for automated feature extraction and classification from big data streaming

Science.gov (United States)

Sakkari, Mohamed; Ejbali, Ridha; Zaied, Mourad

2017-03-01

In this paper, we proposed a deep self-organizing map model (Deep-SOMs) for automated features extracting and learning from big data streaming which we benefit from the framework Spark for real time streams and highly parallel data processing. The SOMs deep architecture is based on the notion of abstraction (patterns automatically extract from the raw data, from the less to more abstract). The proposed model consists of three hidden self-organizing layers, an input and an output layer. Each layer is made up of a multitude of SOMs, each map only focusing at local headmistress sub-region from the input image. Then, each layer trains the local information to generate more overall information in the higher layer. The proposed Deep-SOMs model is unique in terms of the layers architecture, the SOMs sampling method and learning. During the learning stage we use a set of unsupervised SOMs for feature extraction. We validate the effectiveness of our approach on large data sets such as Leukemia dataset and SRBCT. Results of comparison have shown that the Deep-SOMs model performs better than many existing algorithms for images classification.

A stream temperature model for the Peace-Athabasca River basin

Science.gov (United States)

Morales-Marin, L. A.; Rokaya, P.; Wheater, H. S.; Lindenschmidt, K. E.

2017-12-01

Water temperature plays a fundamental role in water ecosystem functioning. Because it regulates flow energy and metabolic rates in organism productivity over a broad spectrum of space and time scales, water temperature constitutes an important indicator of aquatic ecosystems health. In cold region basins, stream water temperature modelling is also fundamental to predict ice freeze-up and break-up events in order to improve flood management. Multiple model approaches such as linear and multivariable regression methods, neural network and thermal energy budged models have been developed and implemented to simulate stream water temperature. Most of these models have been applied to specific stream reaches and trained using observed data, but very little has been done to simulate water temperature in large catchment river networks. We present the coupling of RBM model, a semi-Lagrangian water temperature model for advection-dominated river system, and MESH, a semi-distributed hydrological model, to simulate stream water temperature in river catchments. The coupled models are implemented in the Peace-Athabasca River basin in order to analyze the variation in stream temperature regimes under changing hydrological and meteorological conditions. Uncertainty of stream temperature simulations is also assessed in order to determine the degree of reliability of the estimates.

Energy Efficiency Effects of Vectorization in Data Reuse Transformations for Many-Core Processors—A Case Study †

Directory of Open Access Journals (Sweden)

Abdullah Al Hasib

2017-02-01

Full Text Available Thread-level and data-level parallel architectures have become the design of choice in many of today’s energy-efficient computing systems. However, these architectures put substantially higher requirements on the memory subsystem than scalar architectures, making memory latency and bandwidth critical in their overall efficiency. Data reuse exploration aims at reducing the pressure on the memory subsystem by exploiting the temporal locality in data accesses. In this paper, we investigate the effects on performance and energy from a data reuse methodology combined with parallelization and vectorization in multi- and many-core processors. As a test case, a full-search motion estimation kernel is evaluated on Intel® CoreTM i7-4700K (Haswell and i7-2600K (Sandy Bridge multi-core processors, as well as on an Intel® Xeon PhiTM many-core processor (Knights Landing with Streaming Single Instruction Multiple Data (SIMD Extensions (SSE and Advanced Vector Extensions (AVX instruction sets. Results using a single-threaded execution on the Haswell and Sandy Bridge systems show that performance and EDP (Energy Delay Product can be improved through data reuse transformations on the scalar code by a factor of ≈3× and ≈6×, respectively. Compared to scalar code without data reuse optimization, the SSE/AVX2 version achieves ≈10×/17× better performance and ≈92×/307× better EDP, respectively. These results can be improved by 10% to 15% using data reuse techniques. Finally, the most optimized version using data reuse and AVX512 achieves a speedup of ≈35× and an EDP improvement of ≈1192× on the Xeon Phi system. While single-threaded execution serves as a common reference point for all architectures to analyze the effects of data reuse on both scalar and vector codes, scalability with thread count is also discussed in the paper.

Sensitivity Sampling Over Dynamic Geometric Data Streams with Applications to $k$-Clustering

OpenAIRE

Song, Zhao; Yang, Lin F.; Zhong, Peilin

2018-01-01

Sensitivity based sampling is crucial for constructing nearly-optimal coreset for $k$-means / median clustering. In this paper, we provide a novel data structure that enables sensitivity sampling over a dynamic data stream, where points from a high dimensional discrete Euclidean space can be either inserted or deleted. Based on this data structure, we provide a one-pass coreset construction for $k$-means %and M-estimator clustering using space $\\widetilde{O}(k\\mathrm{poly}(d))$ over $d$-dimen...

Data Reduction of Laser Ablation Split-Stream (LASS) Analyses Using Newly Developed Features Within Iolite: With Applications to Lu-Hf + U-Pb in Detrital Zircon and Sm-Nd +U-Pb in Igneous Monazite

Science.gov (United States)

Fisher, Christopher M.; Paton, Chad; Pearson, D. Graham; Sarkar, Chiranjeeb; Luo, Yan; Tersmette, Daniel B.; Chacko, Thomas

2017-12-01

A robust platform to view and integrate multiple data sets collected simultaneously is required to realize the utility and potential of the Laser Ablation Split-Stream (LASS) method. This capability, until now, has been unavailable and practitioners have had to laboriously process each data set separately, making it challenging to take full advantage of the benefits of LASS. We describe a new program for handling multiple mass spectrometric data sets collected simultaneously, designed specifically for the LASS technique, by which a laser aerosol is been split into two or more separate "streams" to be measured on separate mass spectrometers. New features within Iolite (https://iolite-software.com) enable the capability of loading, synchronizing, viewing, and reducing two or more data sets acquired simultaneously, as multiple DRSs (data reduction schemes) can be run concurrently. While this version of Iolite accommodates any combination of simultaneously collected mass spectrometer data, we demonstrate the utility using case studies where U-Pb and Lu-Hf isotope composition of zircon, and U-Pb and Sm-Nd isotope composition of monazite were analyzed simultaneously, in crystals showing complex isotopic zonation. These studies demonstrate the importance of being able to view and integrate simultaneously acquired data sets, especially for samples with complicated zoning and decoupled isotope systematics, in order to extract accurate and geologically meaningful isotopic and compositional data. This contribution provides instructions and examples for handling simultaneously collected laser ablation data. An instructional video is also provided. The updated Iolite software will help to fully develop the applications of both LASS and multi-instrument mass spectrometric measurement capabilities.

Methods of natural gas liquefaction and natural gas liquefaction plants utilizing multiple and varying gas streams

Science.gov (United States)

Wilding, Bruce M; Turner, Terry D

2014-12-02

A method of natural gas liquefaction may include cooling a gaseous NG process stream to form a liquid NG process stream. The method may further include directing the first tail gas stream out of a plant at a first pressure and directing a second tail gas stream out of the plant at a second pressure. An additional method of natural gas liquefaction may include separating CO.sub.2 from a liquid NG process stream and processing the CO.sub.2 to provide a CO.sub.2 product stream. Another method of natural gas liquefaction may include combining a marginal gaseous NG process stream with a secondary substantially pure NG stream to provide an improved gaseous NG process stream. Additionally, a NG liquefaction plant may include a first tail gas outlet, and at least a second tail gas outlet, the at least a second tail gas outlet separate from the first tail gas outlet.

Models of Tidally Induced Gas Filaments in the Magellanic Stream

Science.gov (United States)

Pardy, Stephen A.; D’Onghia, Elena; Fox, Andrew J.

2018-04-01

The Magellanic Stream and Leading Arm of H I that stretches from the Large and Small Magellanic Clouds (LMC and SMC) and over 200° of the Southern sky is thought to be formed from multiple encounters between the LMC and SMC. In this scenario, most of the gas in the Stream and Leading Arm is stripped from the SMC, yet recent observations have shown a bifurcation of the Trailing Arm that reveals LMC origins for some of the gas. Absorption measurements in the Stream also reveal an order of magnitude more gas than in current tidal models. We present hydrodynamical simulations of the multiple encounters between the LMC and SMC at their first pass around the Milky Way, assuming that the Clouds were more extended and gas-rich in the past. Our models create filamentary structures of gas in the Trailing Stream from both the LMC and SMC. While the SMC trailing filament matches the observed Stream location, the LMC filament is offset. In addition, the total observed mass of the Stream in these models is underestimated by a factor of four when the ionized component is accounted for. Our results suggest that there should also be gas stripped from both the LMC and SMC in the Leading Arm, mirroring the bifurcation in the Trailing Stream. This prediction is consistent with recent measurements of spatial variation in chemical abundances in the Leading Arm, which show that gas from multiple sources is present, although its nature is still uncertain.

Linking land-use type and stream water quality using spatial data of fecal indicator bacteria and heavy metals in the Yeongsan river basin.

Science.gov (United States)

Kang, Joo-Hyon; Lee, Seung Won; Cho, Kyung Hwa; Ki, Seo Jin; Cha, Sung Min; Kim, Joon Ha

2010-07-01

This study reveals land-use factors that explain stream water quality during wet and dry weather conditions in a large river basin using two different linear models-multiple linear regression (MLR) models and constrained least squares (CLS) models. Six land-use types and three topographical parameters (size, slope, and permeability) of the watershed were incorporated into the models as explanatory variables. The suggested models were then demonstrated using a digitized elevation map in conjunction with the land-use and the measured concentration data for Escherichia coli (EC), Enterococci bacteria (ENT), and six heavy metal species collected monthly during 2007-2008 at 50 monitoring sites in the Yeongsan Watershed, Korea. The results showed that the MLR models can be a powerful tool for predicting the average concentrations of pollutants in stream water (the Nash-Sutcliffe (NS) model efficiency coefficients ranged from 0.67 to 0.95). On the other hand, the CLS models, with moderately good prediction performance (the NS coefficients ranged 0.28-0.85), were more suitable for quantifying contributions of respective land-uses to the stream water quality. The CLS models suggested that industrial and urban land-uses are major contributors to the stream concentrations of EC and ENT, whereas agricultural, industrial, and mining areas were significant sources of many heavy metal species. In addition, the slope, size, and permeability of the watershed were found to be important factors determining the extent of the contribution from each land-use type to the stream water quality. The models proposed in this paper can be considered useful tools for developing land cover guidelines and for prioritizing locations for implementing management practices to maintain stream water quality standard in a large river basin. Copyright 2010 Elsevier Ltd. All rights reserved.

The role of international policy transfer within the Multiple Streams Approach: the case of smart electricity metering in Australia

OpenAIRE

Lovell, Heather

2016-01-01

This paper draws on Kingdon’s Multiple Streams Approach (MSA) to consider international flows of policy, not just domestic. It is argued that using the MSA in conjunction with international policy transfer and mobility theories allows for a fuller explanation of the development of smart electricity metering policy in Australia. The MSA is based originally on empirical research within a single country - the USA - in the late 1970s, and all three of the ‘streams’ identified as important to poli...

Spatial patterns of stream temperatures and electric conductivity in a mesoscale catchment

Science.gov (United States)

Lieder, Ernestine; Weiler, Markus; Blume, Theresa

2017-04-01

Stream temperature and electric conductivity (EC) are both relatively easily measured and can provide valuable information on runoff generation processes and catchment storage.This study investigates the spatial variability of stream temperature and EC in a mesoscale basin. We focus on the mesoscale (sub-catchments and reach scale), and long term (seasonal / annual) stream temperature and EC patterns. Our study basin is the Attert catchment in Luxembourg (288km2), which contains multiple sub-catchments of different geology, topography and land use patterns. We installed 90 stream temperature and EC sensors at sites across the basin in summer 2015. The collected data is complemented by land use and discharge data and an extensive climate data set. Thermal sensitivity was calculated as the slope of daily air temperature-water-temperature regression line and describes the sensitivity of stream temperature to long term environmental change. Amplitude sensitivity was calculated as slope of the daily air and water temperature amplitude regression and describes the short term warming capacity of the stream. We found that groups with similar long term thermal and EC patterns are strongly related to different geological units. The sandstone reaches show the coldest temperatures and lowest annual thermal sensitivity to air temperature. The slate reaches are characterized by comparably low EC and high daily temperature amplitudes and amplitude sensitivity. Furthermore, mean annual temperatures and thermal sensitivities increase exponentially with drainage area, which can be attributed to the accumulation of heat throughout the system. On the reach scale, daily stream temperature fluctuations or sensitivities were strongly influenced by land cover distribution, stream shading and runoff volume. Daily thermal sensitivities were low for headwater streams; peaked for intermediate reaches in the middle of the catchment and then decreased again further downstream with increasing

Jet stream winds - Comparisons of analyses with independent aircraft data over Southwest Asia

Science.gov (United States)

Tenenbaum, J.

1991-01-01

Cruise-level wind data from commercial aircraft are obtained, and these data are compared with operational jet stream analyses over southwest Asia, an area of limited conventional data. Results from an ensemble of 11 cases during January 1989 and individual cases during December 1988-March 1989 are presented. The key results are: (1) European Centre for Medium-Range Weather Forecasts (ECMWF), National Meteorological Center, and United Kingdom Meteorological Office analyses of the subtropical jet in southwest Asia are 11 percent, 17 percent, and 17 percent weaker, respectively, than aircraft observations; (2) analyzed poleward shears range up to 1 f (0.00007/s) compared with up to 3 f (0.00021/s) in the aircraft observations where f is the local Coriolis parameters; (3) the ECMWF errors are largest at the base of the jet; (4) the mean ECMWF core location is latitudinally correct but has an rms latitude variance of 1.5 deg; (5) isolated erroneous radiosondes produce unmeteorological structures in the analyzed subtropical jet stream; and (6) the increased utilization of automated aircraft reports is likely to produce a spurious secular increase in the apparent strength of the jets. The magnitude and spatial extent of the errors seen are near limits of current GCM resolution (100 km) but should be resolvable. The results imply that studies of GCM systematic jet stream wind errors in weather and climate forecasts must be interpreted with caution in this region.

Reconfigurable Multicore Architectures for Streaming Applications

NARCIS (Netherlands)

Smit, Gerardus Johannes Maria; Kokkeler, Andre B.J.; Rauwerda, G.K.; Jacobs, J.W.M.; Nicolescu, G.; Mosterman, P.J.

2009-01-01

This chapter addresses reconfigurable heterogenous and homogeneous multicore system-on-chip (SoC) platforms for streaming digital signal processing applications, also called DSP applications. In streaming DSP applications, computations can be specified as a data flow graph with streams of data items

Data Centric Sensor Stream Reduction for Real-Time Applications in Wireless Sensor Networks

Science.gov (United States)

Aquino, Andre Luiz Lins; Nakamura, Eduardo Freire

2009-01-01

This work presents a data-centric strategy to meet deadlines in soft real-time applications in wireless sensor networks. This strategy considers three main aspects: (i) The design of real-time application to obtain the minimum deadlines; (ii) An analytic model to estimate the ideal sample size used by data-reduction algorithms; and (iii) Two data-centric stream-based sampling algorithms to perform data reduction whenever necessary. Simulation results show that our data-centric strategies meet deadlines without loosing data representativeness. PMID:22303145

Unequal Protection of Video Streaming through Adaptive Modulation with a Trizone Buffer over Bluetooth Enhanced Data Rate

Directory of Open Access Journals (Sweden)

Razavi Rouzbeh

2008-01-01

Full Text Available Abstract Bluetooth enhanced data rate wireless channel can support higher-quality video streams compared to previous versions of Bluetooth. Packet loss when transmitting compressed data has an effect on the delivered video quality that endures over multiple frames. To reduce the impact of radio frequency noise and interference, this paper proposes adaptive modulation based on content type at the video frame level and content importance at the macroblock level. Because the bit rate of protected data is reduced, the paper proposes buffer management to reduce the risk of buffer overflow. A trizone buffer is introduced, with a varying unequal protection policy in each zone. Application of this policy together with adaptive modulation results in up to 4 dB improvement in objective video quality compared to fixed rate scheme for an additive white Gaussian noise channel and around 10 dB for a Gilbert-Elliott channel. The paper also reports a consistent improvement in video quality over a scheme that adapts to channel conditions by varying the data rate without accounting for the video frame packet type or buffer congestion.

Unequal Protection of Video Streaming through Adaptive Modulation with a Trizone Buffer over Bluetooth Enhanced Data Rate

Directory of Open Access Journals (Sweden)

Rouzbeh Razavi

2007-12-01

Full Text Available Bluetooth enhanced data rate wireless channel can support higher-quality video streams compared to previous versions of Bluetooth. Packet loss when transmitting compressed data has an effect on the delivered video quality that endures over multiple frames. To reduce the impact of radio frequency noise and interference, this paper proposes adaptive modulation based on content type at the video frame level and content importance at the macroblock level. Because the bit rate of protected data is reduced, the paper proposes buffer management to reduce the risk of buffer overflow. A trizone buffer is introduced, with a varying unequal protection policy in each zone. Application of this policy together with adaptive modulation results in up to 4 dB improvement in objective video quality compared to fixed rate scheme for an additive white Gaussian noise channel and around 10 dB for a Gilbert-Elliott channel. The paper also reports a consistent improvement in video quality over a scheme that adapts to channel conditions by varying the data rate without accounting for the video frame packet type or buffer congestion.

Validating alternative methodologies to estimate the hydrological regime of temporary streams when flow data are unavailable

Science.gov (United States)

Llorens, Pilar; Gallart, Francesc; Latron, Jérôme; Cid, Núria; Rieradevall, Maria; Prat, Narcís

2016-04-01

Aquatic life in temporary streams is strongly conditioned by the temporal variability of the hydrological conditions that control the occurrence and connectivity of diverse mesohabitats. In this context, the software TREHS (Temporary Rivers' Ecological and Hydrological Status) has been developed, in the framework of the LIFE Trivers project, to help managers for adequately implement the Water Framework Directive in this type of water bodies. TREHS, using the methodology described in Gallart et al (2012), defines six temporal 'aquatic states', based on the hydrological conditions representing different mesohabitats, for a given reach at a particular moment. Nevertheless, hydrological data for assessing the regime of temporary streams are often non-existent or scarce. The scarcity of flow data makes frequently impossible the characterization of temporary streams hydrological regimes and, as a consequence, the selection of the correct periods and methods to determine their ecological status. Because of its qualitative nature, the TREHS approach allows the use of alternative methodologies to assess the regime of temporary streams in the lack of observed flow data. However, to adapt the TREHS to this qualitative data both the temporal scheme (from monthly to seasonal) as well as the number of aquatic states (from 6 to 3) have been modified. Two alternatives complementary methodologies were tested within the TREHS framework to assess the regime of temporary streams: interviews and aerial photographs. All the gauging stations (13) belonging to the Catalan Internal Catchments (NE, Spain) with recurrent zero flows periods were selected to validate both methodologies. On one hand, non-structured interviews were carried out to inhabitants of villages and small towns near the gauging stations. Flow permanence metrics for input into TREHS were drawn from the notes taken during the interviews. On the other hand, the historical series of available aerial photographs (typically 10

Grid-based Parallel Data Streaming Implemented for the Gyrokinetic Toroidal Code

International Nuclear Information System (INIS)

Klasky, S.; Ethier, S.; Lin, Z.; Martins, K.; McCune, D.; Samtaney, R.

2003-01-01

We have developed a threaded parallel data streaming approach using Globus to transfer multi-terabyte simulation data from a remote supercomputer to the scientist's home analysis/visualization cluster, as the simulation executes, with negligible overhead. Data transfer experiments show that this concurrent data transfer approach is more favorable compared with writing to local disk and then transferring this data to be post-processed. The present approach is conducive to using the grid to pipeline the simulation with post-processing and visualization. We have applied this method to the Gyrokinetic Toroidal Code (GTC), a 3-dimensional particle-in-cell code used to study microturbulence in magnetic confinement fusion from first principles plasma theory

Parasail: SIMD C library for global, semi-global, and local pairwise sequence alignments.

Science.gov (United States)

Daily, Jeff

2016-02-10

Sequence alignment algorithms are a key component of many bioinformatics applications. Though various fast Smith-Waterman local sequence alignment implementations have been developed for x86 CPUs, most are embedded into larger database search tools. In addition, fast implementations of Needleman-Wunsch global sequence alignment and its semi-global variants are not as widespread. This article presents the first software library for local, global, and semi-global pairwise intra-sequence alignments and improves the performance of previous intra-sequence implementations. A faster intra-sequence local pairwise alignment implementation is described and benchmarked, including new global and semi-global variants. Using a 375 residue query sequence a speed of 136 billion cell updates per second (GCUPS) was achieved on a dual Intel Xeon E5-2670 24-core processor system, the highest reported for an implementation based on Farrar's 'striped' approach. Rognes's SWIPE optimal database search application is still generally the fastest available at 1.2 to at best 2.4 times faster than Parasail for sequences shorter than 500 amino acids. However, Parasail was faster for longer sequences. For global alignments, Parasail's prefix scan implementation is generally the fastest, faster even than Farrar's 'striped' approach, however the opal library is faster for single-threaded applications. The software library is designed for 64 bit Linux, OS X, or Windows on processors with SSE2, SSE41, or AVX2. Source code is available from https://github.com/jeffdaily/parasail under the Battelle BSD-style license. Applications that require optimal alignment scores could benefit from the improved performance. For the first time, SIMD global, semi-global, and local alignments are available in a stand-alone C library.

Matisse: A Visual Analytics System for Exploring Emotion Trends in Social Media Text Streams

Energy Technology Data Exchange (ETDEWEB)

Steed, Chad A [ORNL; Drouhard, Margaret MEG G [ORNL; Beaver, Justin M [ORNL; Pyle, Joshua M [ORNL; BogenII, Paul L. [Google Inc.

2015-01-01

Dynamically mining textual information streams to gain real-time situational awareness is especially challenging with social media systems where throughput and velocity properties push the limits of a static analytical approach. In this paper, we describe an interactive visual analytics system, called Matisse, that aids with the discovery and investigation of trends in streaming text. Matisse addresses the challenges inherent to text stream mining through the following technical contributions: (1) robust stream data management, (2) automated sentiment/emotion analytics, (3) interactive coordinated visualizations, and (4) a flexible drill-down interaction scheme that accesses multiple levels of detail. In addition to positive/negative sentiment prediction, Matisse provides fine-grained emotion classification based on Valence, Arousal, and Dominance dimensions and a novel machine learning process. Information from the sentiment/emotion analytics are fused with raw data and summary information to feed temporal, geospatial, term frequency, and scatterplot visualizations using a multi-scale, coordinated interaction model. After describing these techniques, we conclude with a practical case study focused on analyzing the Twitter sample stream during the week of the 2013 Boston Marathon bombings. The case study demonstrates the effectiveness of Matisse at providing guided situational awareness of significant trends in social media streams by orchestrating computational power and human cognition.

Streaming support for data intensive cloud-based sequence analysis.

Science.gov (United States)

Issa, Shadi A; Kienzler, Romeo; El-Kalioby, Mohamed; Tonellato, Peter J; Wall, Dennis; Bruggmann, Rémy; Abouelhoda, Mohamed

2013-01-01

Cloud computing provides a promising solution to the genomics data deluge problem resulting from the advent of next-generation sequencing (NGS) technology. Based on the concepts of "resources-on-demand" and "pay-as-you-go", scientists with no or limited infrastructure can have access to scalable and cost-effective computational resources. However, the large size of NGS data causes a significant data transfer latency from the client's site to the cloud, which presents a bottleneck for using cloud computing services. In this paper, we provide a streaming-based scheme to overcome this problem, where the NGS data is processed while being transferred to the cloud. Our scheme targets the wide class of NGS data analysis tasks, where the NGS sequences can be processed independently from one another. We also provide the elastream package that supports the use of this scheme with individual analysis programs or with workflow systems. Experiments presented in this paper show that our solution mitigates the effect of data transfer latency and saves both time and cost of computation.

Streaming Support for Data Intensive Cloud-Based Sequence Analysis

Directory of Open Access Journals (Sweden)

Shadi A. Issa

2013-01-01

Full Text Available Cloud computing provides a promising solution to the genomics data deluge problem resulting from the advent of next-generation sequencing (NGS technology. Based on the concepts of “resources-on-demand” and “pay-as-you-go”, scientists with no or limited infrastructure can have access to scalable and cost-effective computational resources. However, the large size of NGS data causes a significant data transfer latency from the client’s site to the cloud, which presents a bottleneck for using cloud computing services. In this paper, we provide a streaming-based scheme to overcome this problem, where the NGS data is processed while being transferred to the cloud. Our scheme targets the wide class of NGS data analysis tasks, where the NGS sequences can be processed independently from one another. We also provide the elastream package that supports the use of this scheme with individual analysis programs or with workflow systems. Experiments presented in this paper show that our solution mitigates the effect of data transfer latency and saves both time and cost of computation.

Streaming Support for Data Intensive Cloud-Based Sequence Analysis

Science.gov (United States)

Issa, Shadi A.; Kienzler, Romeo; El-Kalioby, Mohamed; Tonellato, Peter J.; Wall, Dennis; Bruggmann, Rémy; Abouelhoda, Mohamed

2013-01-01

Cloud computing provides a promising solution to the genomics data deluge problem resulting from the advent of next-generation sequencing (NGS) technology. Based on the concepts of “resources-on-demand” and “pay-as-you-go”, scientists with no or limited infrastructure can have access to scalable and cost-effective computational resources. However, the large size of NGS data causes a significant data transfer latency from the client's site to the cloud, which presents a bottleneck for using cloud computing services. In this paper, we provide a streaming-based scheme to overcome this problem, where the NGS data is processed while being transferred to the cloud. Our scheme targets the wide class of NGS data analysis tasks, where the NGS sequences can be processed independently from one another. We also provide the elastream package that supports the use of this scheme with individual analysis programs or with workflow systems. Experiments presented in this paper show that our solution mitigates the effect of data transfer latency and saves both time and cost of computation. PMID:23710461

Hydrogeochemical and stream sediment reconnaissance basic data for Lubbock NTMS Quadrangle, Texas

International Nuclear Information System (INIS)

1979-01-01

Field and laboratory data are presented for 994 groundwater and 602 stream sediment samples. Statistical and areal distributions of uranium and possible uranium-related variables are displayed. A generalized geologic map of the survey area is provided, and pertinent geologic factors which may be of significance in evaluating the potential for uranium mineralization are briefly discussed. Interpretation of the groundwater data indicate that the area which appears most promising for uranium mineralization is located in the southwestern part of the quadrangle, particularly in Crosby, Garza, Lynn, and Lubbock Counties. The waters produced from the Ogallala Formation in this area have high values for arsenic, molybdenum, selenium, and vanadium. Groundwaters from the Dockum Group in Garza County where uranium is associated with selenium, molybdenum, and copper indicate potential for uranium mineralization. Uranium is generally associated with copper, iron, and sulfate in the Permian aquifers reflecting the red bed evaporite lithology of those units. The stream sediment data indicate that the Dockum Group has the highest potential for uranium mineralization, particularly in and around Garza County. Associated elements indicate that uranium may occur in residual minerals or in hydrous manganese oxides. Sediment data also indicate that the Blaine Formation shows limited potential for small red bed copper-uranium deposits

Optimal Multi-Interface Selection for Mobile Video Streaming in Efficient Battery Consumption and Data Usage

Directory of Open Access Journals (Sweden)

Seonghoon Moon

2016-01-01

Full Text Available With the proliferation of high-performance, large-screen mobile devices, users’ expectations of having access to high-resolution video content in smooth network environments are steadily growing. To guarantee such stable streaming, a high cellular network bandwidth is required; yet network providers often charge high prices for even limited data plans. Moreover, the costs of smoothly streaming high-resolution videos are not merely monetary; the device’s battery life must also be accounted for. To resolve these problems, we design an optimal multi-interface selection system for streaming video over HTTP/TCP. An optimization problem including battery life and LTE data constraints is derived and then solved using binary integer programming. Additionally, the system is designed with an adoption of split-layer scalable video coding, which provides direct adaptations of video quality and prevents out-of-order packet delivery problems. The proposed system is evaluated using a prototype application in a real, iOS-based device as well as through experiments conducted in heterogeneous mobile scenarios. Results show that the system not only guarantees the highest-possible video quality, but also prevents reckless consumption of LTE data and battery life.

A survey of systems for massive stream analytics

OpenAIRE

Singh, Maninder Pal; Hoque, Mohammad A.; Tarkoma, Sasu

2016-01-01

The immense growth of data demands switching from traditional data processing solutions to systems, which can process a continuous stream of real time data. Various applications employ stream processing systems to provide solutions to emerging Big Data problems. Open-source solutions such as Storm, Spark Streaming, and S4 are the attempts to answer key stream processing questions. The recent introduction of real time stream processing commercial solutions such as Amazon Kinesis, IBM Infospher...

Rapid Bioassessment Methods for Assessing Stream Macroinvertebrate Community on the Savannah River Site

International Nuclear Information System (INIS)

Specht, W.L.

1999-01-01

Macroinvertebrate sampling was performed at 16 locations in the Savannah River Site (SRS) streams using Hester-Dendy multiplate samplers and EPA Rapid Bioassessment Protocols (RBP). Some of the sampling locations were unimpacted, while other locations had been subject to various forms of perturbation by SRS activities. In general, the data from the Hester-Dendy multiplate samplers were more sensitive at detecting impacts than were the RBP data. We developed a Biotic Index for the Hester-Dendy data which incorporated eight community structure, function, and balance parameters. when tested using a data set that was unrelated to the data set that was used in developing the Biotic Index, the index was very successful at detecting impact

Rapid Bioassessment Methods for Assessing Stream Macroinvertebrate Community on the Savannah River Site

Energy Technology Data Exchange (ETDEWEB)

Specht, W.L.

1999-11-22

Macroinvertebrate sampling was performed at 16 locations in the Savannah River Site (SRS) streams using Hester-Dendy multiplate samplers and EPA Rapid Bioassessment Protocols (RBP). Some of the sampling locations were unimpacted, while other locations had been subject to various forms of perturbation by SRS activities. In general, the data from the Hester-Dendy multiplate samplers were more sensitive at detecting impacts than were the RBP data. We developed a Biotic Index for the Hester-Dendy data which incorporated eight community structure, function, and balance parameters. when tested using a data set that was unrelated to the data set that was used in developing the Biotic Index, the index was very successful at detecting impact.

Dynamical modeling of tidal streams

International Nuclear Information System (INIS)

Bovy, Jo

2014-01-01

I present a new framework for modeling the dynamics of tidal streams. The framework consists of simple models for the initial action-angle distribution of tidal debris, which can be straightforwardly evolved forward in time. Taking advantage of the essentially one-dimensional nature of tidal streams, the transformation to position-velocity coordinates can be linearized and interpolated near a small number of points along the stream, thus allowing for efficient computations of a stream's properties in observable quantities. I illustrate how to calculate the stream's average location (its 'track') in different coordinate systems, how to quickly estimate the dispersion around its track, and how to draw mock stream data. As a generative model, this framework allows one to compute the full probability distribution function and marginalize over or condition it on certain phase-space dimensions as well as convolve it with observational uncertainties. This will be instrumental in proper data analysis of stream data. In addition to providing a computationally efficient practical tool for modeling the dynamics of tidal streams, the action-angle nature of the framework helps elucidate how the observed width of the stream relates to the velocity dispersion or mass of the progenitor, and how the progenitors of 'orphan' streams could be located. The practical usefulness of the proposed framework crucially depends on the ability to calculate action-angle variables for any orbit in any gravitational potential. A novel method for calculating actions, frequencies, and angles in any static potential using a single orbit integration is described in the Appendix.

Time course of auditory streaming: Do CI users differ from normal-hearing listeners?

Directory of Open Access Journals (Sweden)

Martin eBöckmann-Barthel

2014-07-01

Full Text Available In a complex acoustical environment with multiple sound sources the auditory system uses streaming as a tool to organize the incoming sounds in one or more streams depending on the stimulus parameters. Streaming is commonly studied by alternating sequences of signals. These are often tones with different frequencies. The present study investigates stream segregation in cochlear implant (CI users, where hearing is restored by electrical stimulation of the auditory nerve. CI users listened to 30-s long sequences of alternating A and B harmonic complexes at four different fundamental frequency separations, ranging from 2 to 14 semitones. They had to indicate as promptly as possible after sequence onset, if they perceived one stream or two streams and, in addition, any changes of the percept throughout the rest of the sequence. The conventional view is that the initial percept is always that of a single stream which may after some time change to a percept of two streams. This general build-up hypothesis has recently been challenged on the basis of a new analysis of data of normal-hearing listeners which showed a build-up response only for an intermediate frequency separation. Using the same experimental paradigm and analysis, the present study found that the results of CI users agree with those of the normal-hearing listeners: (i the probability of the first decision to be a one-stream percept decreased and that of a two-stream percept increased as Δf increased, and (ii a build-up was only found for 6 semitones. Only the time elapsed before the listeners made their first decision of the percept was prolonged as compared to normal-hearing listeners. The similarity in the data of the CI user and the normal-hearing listeners indicates that the quality of stream formation is similar in these groups of listeners.

Multi-scale window specification over streaming trajectories

Directory of Open Access Journals (Sweden)

Kostas Patroumpas

2013-12-01

Full Text Available Enormous amounts of positional information are collected by monitoring applications in domains such as fleet management, cargo transport, wildlife protection, etc. With the advent of modern location-based services, processing such data mostly focuses on providing real-time response to a variety of user requests in continuous and scalable fashion. An important class of such queries concerns evolving trajectories that continuously trace the streaming locations of moving objects, like GPS-equipped vehicles, commodities with RFID's, people with smartphones etc. In this work, we propose an advanced windowing operator that enables online, incremental examination of recent motion paths at multiple resolutions for numerous point entities. When applied against incoming positions, this window can abstract trajectories at coarser representations towards the past, while retaining progressively finer features closer to the present. We explain the semantics of such multi-scale sliding windows through parameterized functions that reflect the sequential nature of trajectories and can effectively capture their spatiotemporal properties. Such window specification goes beyond its usual role for non-blocking processing of multiple concurrent queries. Actually, it can offer concrete subsequences from each trajectory, thus preserving continuity in time and contiguity in space along the respective segments. Further, we suggest language extensions in order to express characteristic spatiotemporal queries using windows. Finally, we discuss algorithms for nested maintenance of multi-scale windows and evaluate their efficiency against streaming positional data, offering empirical evidence of their benefits to online trajectory processing.

Estimating Discharge and Nonpoint Source Nitrate Loading to Streams From Three End-Member Pathways Using High-Frequency Water Quality Data

Science.gov (United States)

Miller, Matthew P.; Tesoriero, Anthony J.; Hood, Krista; Terziotti, Silvia; Wolock, David M.

2017-12-01

The myriad hydrologic and biogeochemical processes taking place in watersheds occurring across space and time are integrated and reflected in the quantity and quality of water in streams and rivers. Collection of high-frequency water quality data with sensors in surface waters provides new opportunities to disentangle these processes and quantify sources and transport of water and solutes in the coupled groundwater-surface water system. A new approach for separating the streamflow hydrograph into three components was developed and coupled with high-frequency nitrate data to estimate time-variable nitrate loads from chemically dilute quick flow, chemically concentrated quick flow, and slowflow groundwater end-member pathways for periods of up to 2 years in a groundwater-dominated and a quick-flow-dominated stream in central Wisconsin, using only streamflow and in-stream water quality data. The dilute and concentrated quick flow end-members were distinguished using high-frequency specific conductance data. Results indicate that dilute quick flow contributed less than 5% of the nitrate load at both sites, whereas 89 ± 8% of the nitrate load at the groundwater-dominated stream was from slowflow groundwater, and 84 ± 25% of the nitrate load at the quick-flow-dominated stream was from concentrated quick flow. Concentrated quick flow nitrate concentrations varied seasonally at both sites, with peak concentrations in the winter that were 2-3 times greater than minimum concentrations during the growing season. Application of this approach provides an opportunity to assess stream vulnerability to nonpoint source nitrate loading and expected stream responses to current or changing conditions and practices in watersheds.

Real-Time Earthquake Intensity Estimation Using Streaming Data Analysis of Social and Physical Sensors

Science.gov (United States)

Kropivnitskaya, Yelena; Tiampo, Kristy F.; Qin, Jinhui; Bauer, Michael A.

2017-06-01

Earthquake intensity is one of the key components of the decision-making process for disaster response and emergency services. Accurate and rapid intensity calculations can help to reduce total loss and the number of casualties after an earthquake. Modern intensity assessment procedures handle a variety of information sources, which can be divided into two main categories. The first type of data is that derived from physical sensors, such as seismographs and accelerometers, while the second type consists of data obtained from social sensors, such as witness observations of the consequences of the earthquake itself. Estimation approaches using additional data sources or that combine sources from both data types tend to increase intensity uncertainty due to human factors and inadequate procedures for temporal and spatial estimation, resulting in precision errors in both time and space. Here we present a processing approach for the real-time analysis of streams of data from both source types. The physical sensor data is acquired from the U.S. Geological Survey (USGS) seismic network in California and the social sensor data is based on Twitter user observations. First, empirical relationships between tweet rate and observed Modified Mercalli Intensity (MMI) are developed using data from the M6.0 South Napa, CAF earthquake that occurred on August 24, 2014. Second, the streams of both data types are analyzed together in simulated real-time to produce one intensity map. The second implementation is based on IBM InfoSphere Streams, a cloud platform for real-time analytics of big data. To handle large processing workloads for data from various sources, it is deployed and run on a cloud-based cluster of virtual machines. We compare the quality and evolution of intensity maps from different data sources over 10-min time intervals immediately following the earthquake. Results from the joint analysis shows that it provides more complete coverage, with better accuracy and higher

Passing a smoke-free law in a pro-tobacco culture: a multiple streams approach.

Science.gov (United States)

Greathouse, Lisa W; Hahn, Ellen J; Okoli, Chizimuzo T C; Warnick, Todd A; Riker, Carol A

2005-08-01

This article describes a case study of the policy development and political decision-making process involved in the enactment of Lexington, Kentucky's smoke-free law. The multiple streams framework is used to analyze the development of the law in a seemingly unlikely and challenging political environment. Proponents developed a dissemination research plan targeted at policy makers and the public to demonstrate the need for a comprehensive law. The existence of a strong coalition of health care providers and health care systems including the board of health, as well as long-standing tobacco control expertise and a strong legal team, were essential ingredients for success. A deliberate strategy to expose the tobacco industry was effective in preparing policy makers for the opponents' policy arguments. As expected, a hospitality industry association was formed to oppose the ordinance, resulting in a legal challenge that delayed enactment of the law.

Automatic event-based synchronization of multimodal data streams from wearable and ambient sensors

NARCIS (Netherlands)

Bannach, D.; Amft, O.D.; Lukowicz, P.; Barnaghi, P.; Moessner, K.; Presser, M.; Meissner, S.

2009-01-01

A major challenge in using multi-modal, distributed sensor systems for activity recognition is to maintain a temporal synchronization between individually recorded data streams. A common approach is to use well defined ‘synchronization actions’ performed by the user to generate, easily identifiable

Stream Tracker: Crowd sourcing and remote sensing to monitor stream flow intermittence

Science.gov (United States)

Puntenney, K.; Kampf, S. K.; Newman, G.; Lefsky, M. A.; Weber, R.; Gerlich, J.

2017-12-01

Streams that do not flow continuously in time and space support diverse aquatic life and can be critical contributors to downstream water supply. However, these intermittent streams are rarely monitored and poorly mapped. Stream Tracker is a community powered stream monitoring project that pairs citizen contributed observations of streamflow presence or absence with a network of streamflow sensors and remotely sensed data from satellites to track when and where water is flowing in intermittent stream channels. Citizens can visit sites on roads and trails to track flow and contribute their observations to the project site hosted by CitSci.org. Data can be entered using either a mobile application with offline capabilities or an online data entry portal. The sensor network provides a consistent record of streamflow and flow presence/absence across a range of elevations and drainage areas. Capacitance, resistance, and laser sensors have been deployed to determine the most reliable, low cost sensor that could be mass distributed to track streamflow intermittence over a larger number of sites. Streamflow presence or absence observations from the citizen and sensor networks are then compared to satellite imagery to improve flow detection algorithms using remotely sensed data from Landsat. In the first two months of this project, 1,287 observations have been made at 241 sites by 24 project members across northern and western Colorado.

Phase II in-stream sediment measurements in the Talakhaya watershed on Rota, Commonwealth of the Northern Mariana Islands from 2016-03-09 to 2017-05-11 (NCEI Accession 0166379)

Data.gov (United States)

National Oceanic and Atmospheric Administration, Department of Commerce — The dataset includes TSS (mg/L), turbidity (NTU), stream level (ft), and precipitation (in) data collected at multiple stations across five streams and at four rain...

Use of NTRIP for Optimizing the Decoding Algorithm for Real-Time Data Streams

Directory of Open Access Journals (Sweden)

Zhanke He

2014-10-01

Full Text Available As a network transmission protocol, Networked Transport of RTCM via Internet Protocol (NTRIP is widely used in GPS and Global Orbiting Navigational Satellite System (GLONASS Augmentation systems, such as Continuous Operational Reference System (CORS, Wide Area Augmentation System (WAAS and Satellite Based Augmentation Systems (SBAS. With the deployment of BeiDou Navigation Satellite system(BDS to serve the Asia-Pacific region, there are increasing needs for ground monitoring of the BeiDou Navigation Satellite system and the development of the high-precision real-time BeiDou products. This paper aims to optimize the decoding algorithm of NTRIP Client data streams and the user authentication strategies of the NTRIP Caster based on NTRIP. The proposed method greatly enhances the handling efficiency and significantly reduces the data transmission delay compared with the Federal Agency for Cartography and Geodesy (BKG NTRIP. Meanwhile, a transcoding method is proposed to facilitate the data transformation from the BINary EXchange (BINEX format to the RTCM format. The transformation scheme thus solves the problem of handing real-time data streams from Trimble receivers in the BeiDou Navigation Satellite System indigenously developed by China.

Use of NTRIP for optimizing the decoding algorithm for real-time data streams.

Science.gov (United States)

He, Zhanke; Tang, Wenda; Yang, Xuhai; Wang, Liming; Liu, Jihua

2014-10-10

As a network transmission protocol, Networked Transport of RTCM via Internet Protocol (NTRIP) is widely used in GPS and Global Orbiting Navigational Satellite System (GLONASS) Augmentation systems, such as Continuous Operational Reference System (CORS), Wide Area Augmentation System (WAAS) and Satellite Based Augmentation Systems (SBAS). With the deployment of BeiDou Navigation Satellite system(BDS) to serve the Asia-Pacific region, there are increasing needs for ground monitoring of the BeiDou Navigation Satellite system and the development of the high-precision real-time BeiDou products. This paper aims to optimize the decoding algorithm of NTRIP Client data streams and the user authentication strategies of the NTRIP Caster based on NTRIP. The proposed method greatly enhances the handling efficiency and significantly reduces the data transmission delay compared with the Federal Agency for Cartography and Geodesy (BKG) NTRIP. Meanwhile, a transcoding method is proposed to facilitate the data transformation from the BINary EXchange (BINEX) format to the RTCM format. The transformation scheme thus solves the problem of handing real-time data streams from Trimble receivers in the BeiDou Navigation Satellite System indigenously developed by China.

The Midwest Stream Quality Assessment—Influences of human activities on streams

Science.gov (United States)

Van Metre, Peter C.; Mahler, Barbara J.; Carlisle, Daren M.; Coles, James F.

2018-04-16

Healthy streams and the fish and other organisms that live in them contribute to our quality of life. Extensive modification of the landscape in the Midwestern United States, however, has profoundly affected the condition of streams. Row crops and pavement have replaced grasslands and woodlands, streams have been straightened, and wetlands and fields have been drained. Runoff from agricultural and urban land brings sediment and chemicals to streams. What is the chemical, physical, and biological condition of Midwestern streams? Which physical and chemical stressors are adversely affecting biological communities, what are their origins, and how might we lessen or avoid their adverse effects?In 2013, the U.S. Geological Survey (USGS) conducted the Midwest Stream Quality Assessment to evaluate how human activities affect the biological condition of Midwestern streams. In collaboration with the U.S. Environmental Protection Agency National Rivers and Streams Assessment, the USGS sampled 100 streams, chosen to be representative of the different types of watersheds in the region. Biological condition was evaluated based on the number and diversity of fish, algae, and invertebrates in the streams. Changes to the physical habitat and chemical characteristics of the streams—“stressors”—were assessed, and their relation to landscape factors and biological condition was explored by using mathematical models. The data and models help us to better understand how the human activities on the landscape are affecting streams in the region.

The Stream-Catchment (StreamCat) Dataset: A database of watershed metrics for the conterminous USA

Science.gov (United States)

We developed an extensive database of landscape metrics for ~2.65 million streams, and their associated catchments, within the conterminous USA: The Stream-Catchment (StreamCat) Dataset. These data are publically available and greatly reduce the specialized geospatial expertise n...

Results of Macroinvertebrate Sampling Conducted at 33 SRS Stream Locations, July--August 1993

Energy Technology Data Exchange (ETDEWEB)

Specht, W.L.

1994-12-01

In order to assess the health of the macroinvertebrate communities of SRS streams, the macroinvertebrate communities at 30 stream locations on SRS were sampled during the summer of 1993, using Hester-Dendy multiplate samplers. In addition, three off-site locations in the Upper Three Runs drainage were sampled in order to assess the potential for impact from off-site activities. In interpreting the data, it is important to recognize that these data were from a single set of collections. Macroinvertebrate communities often undergo considerable temporal variation, and are also greatly influenced by such factors as water depth, water velocity, and available habitat. These stations were selected with the intent of developing an on-going sampling program at a smaller number of stations, with the selection of the stations to be based largely upon the results of this preliminary sampling program. When stations within a given stream showed similar results, fewer stations would be sampled in the future. Similarly, if a stream appeared to be perturbed, additional stations or chemical analyses might be added so that the source of the perturbation could be identified. In general, unperturbed streams will contain more taxa than perturbed streams, and the distribution of taxa among orders or families will differ. Some groups of macroinvertebrates, such as Ephemeroptera (mayflies), Plecoptera (stoneflies) and Trichoptera (caddisflies), which are collectively called EPT taxa, are considered to be relatively sensitive to most kinds of stream perturbation; therefore a reduced number of EPT taxa generally indicates that the stream has been subject to chemical or physical stressors. In coastal plain streams, EPT taxa are generally less dominant than in streams with rocky substrates, while Chironomidae (midges) are more abundant. (Abstract Truncated)

A decision support system using combined-classifier for high-speed data stream in smart grid

Science.gov (United States)

Yang, Hang; Li, Peng; He, Zhian; Guo, Xiaobin; Fong, Simon; Chen, Huajun

2016-11-01

Large volume of high-speed streaming data is generated by big power grids continuously. In order to detect and avoid power grid failure, decision support systems (DSSs) are commonly adopted in power grid enterprises. Among all the decision-making algorithms, incremental decision tree is the most widely used one. In this paper, we propose a combined classifier that is a composite of a cache-based classifier (CBC) and a main tree classifier (MTC). We integrate this classifier into a stream processing engine on top of the DSS such that high-speed steaming data can be transformed into operational intelligence efficiently. Experimental results show that our proposed classifier can return more accurate answers than other existing ones.

Linking Stream Dissolved Oxygen with the Dynamic Environmental Drivers across the Pacific Coast of U.S.A.

Science.gov (United States)

Araya, F. Z.; Abdul-Aziz, O. I.

2017-12-01

This study utilized a systematic data analytics approach to determine the relative linkages of stream dissolved oxygen (DO) with the hydro-climatic and biogeochemical drivers across the U.S. Pacific Coast. Multivariate statistical techniques of Pearson correlation matrix, principal component analysis, and factor analysis were applied to a complex water quality dataset (1998-2015) at 35 water quality monitoring stations of USGS NWIS and EPA STORET. Power-law based partial least squares regression (PLSR) models with a bootstrap Monte Carlo procedure (1000 iterations) were developed to reliably estimate the relative linkages by resolving multicollinearity (Nash-Sutcliffe Efficiency, NSE = 0.50-0.94). Based on the dominant drivers, four environmental regimes have been identified and adequately described the system-data variances. In Pacific North West and Southern California, water temperature was the most dominant driver of DO in majority of the streams. However, in Central and Northern California, stream DO was controlled by multiple drivers (i.e., water temperature, pH, stream flow, and total phosphorus), exhibiting a transitional environmental regime. Further, total phosphorus (TP) appeared to be the limiting nutrient for most streams. The estimated linkages and insights would be useful to identify management priorities to achieve healthy coastal stream ecosystems across the Pacific Coast of U.S.A. and similar regions around the world. Keywords: Data analytics, water quality, coastal streams, dissolved oxygen, environmental regimes, Pacific Coast, United States.

Development of Ecogeomorphological (EGM Stream Design and Assessment Tools for the Piedmont of Alabama, USA

Directory of Open Access Journals (Sweden)

Brian Helms

2016-04-01

Full Text Available Regional data needed for effective stream restoration include hydraulic geometry relationships (i.e., regional curves and reference channel morphology parameters. Increasingly ecological conditions are being considered when designing, implementing, and assessing restoration efforts. We provide morphology relationships and associated ecological endpoint curves for reference streams in the Alabama piedmont. Twenty-one reference stream reaches were identified in the Tallapoosa drainage of Alabama, ranging from 0.2 to 242 km2 drainage area. Geomorphic surveys were conducted in each stream to measure riffle cross-sections and longitudinal profiles and related to drainage area to develop regional curves. Fish, crayfish, and benthic macroinvertebrates were collected from each surveyed reach and related to drainage area and geomorphic data to provide associated biological community endpoints. Bankfull channel cross-section area, width, mean depth, and estimated discharge were strongly correlated to watershed drainage area, similar to efforts in other areas of the Piedmont ecoregion. Multiple measures of fish assemblages and crayfish size were strongly predicted by drainage area and geomorphic dimensions. Macroinvertebrates showed no taxonomic and limited functional relationships with drainage area and geomorphic dimension. These tools, which integrate geomorphological and ecological conditions, can result in improved stream evaluations and designs increasing the effectiveness of stream restoration projects.

Precipitation and stream water stable isotope data from the Marys River, Oregon in water year 2015.

Data.gov (United States)

U.S. Environmental Protection Agency — Water stable isotope data collected from a range of streams throughout the Marys River basin in water year 2015, and precipitation data collected within the basin at...

Extending Data Worth Analyses to Select Multiple Observations Targeting Multiple Forecasts

DEFF Research Database (Denmark)

Vilhelmsen, Troels Norvin; Ferre, Ty Paul

2017-01-01

. In the present study, we extend previous data worth analyses to include: simultaneous selection of multiple new measurements and consideration of multiple forecasts of interest. We show how the suggested approach can be used to optimize data collection. This can be used in a manner that suggests specific...... measurement sets or that produces probability maps indicating areas likely to be informative for specific forecasts. Moreover, we provide examples documenting that sequential measurement election approaches often lead to suboptimal designs and that estimates of data covariance should be included when...

Multiple drivers, scales, and interactions influence southern Appalachian stream salamander occupancy

Science.gov (United States)

Cecala, Kristen K.; Maerz, John C.; Halstead, Brian J.; Frisch, John R.; Gragson, Ted L.; Hepinstall-Cymerman, Jeffrey; Leigh, David S.; Jackson, C. Rhett; Peterson, James T.; Pringle, Catherine M.

2018-01-01

Understanding how factors that vary in spatial scale relate to population abundance is vital to forecasting species responses to environmental change. Stream and river ecosystems are inherently hierarchical, potentially resulting in organismal responses to fine‐scale changes in patch characteristics that are conditional on the watershed context. Here, we address how populations of two salamander species are affected by interactions among hierarchical processes operating at different scales within a rapidly changing landscape of the southern Appalachian Mountains. We modeled reach‐level occupancy of larval and adult black‐bellied salamanders (Desmognathus quadramaculatus) and larval Blue Ridge two‐lined salamanders (Eurycea wilderae) as a function of 17 different terrestrial and aquatic predictor variables that varied in spatial extent. We found that salamander occurrence varied widely among streams within fully forested catchments, but also exhibited species‐specific responses to changes in local conditions. While D. quadramaculatus declined predictably in relation to losses in forest cover, larval occupancy exhibited the strongest negative response to forest loss as well as decreases in elevation. Conversely, occupancy of E. wilderae was unassociated with watershed conditions, only responding negatively to higher proportions of fast‐flowing stream habitat types. Evaluation of hierarchical relationships demonstrated that most fine‐scale variables were closely correlated with broad watershed‐scale variables, suggesting that local reach‐scale factors have relatively smaller effects within the context of the larger landscape. Our results imply that effective management of southern Appalachian stream salamanders must first focus on the larger scale condition of watersheds before management of local‐scale conditions should proceed. Our findings confirm the results of some studies while refuting the results of others, which may indicate that

Towards Cache-Enabled, Order-Aware, Ontology-Based Stream Reasoning Framework

Energy Technology Data Exchange (ETDEWEB)

Yan, Rui; Praggastis, Brenda L.; Smith, William P.; McGuinness, Deborah L.

2016-08-16

While streaming data have become increasingly more popular in business and research communities, semantic models and processing software for streaming data have not kept pace. Traditional semantic solutions have not addressed transient data streams. Semantic web languages (e.g., RDF, OWL) have typically addressed static data settings and linked data approaches have predominantly addressed static or growing data repositories. Streaming data settings have some fundamental differences; in particular, data are consumed on the fly and data may expire. Stream reasoning, a combination of stream processing and semantic reasoning, has emerged with the vision of providing "smart" processing of streaming data. C-SPARQL is a prominent stream reasoning system that handles semantic (RDF) data streams. Many stream reasoning systems including C-SPARQL use a sliding window and use data arrival time to evict data. For data streams that include expiration times, a simple arrival time scheme is inadequate if the window size does not match the expiration period. In this paper, we propose a cache-enabled, order-aware, ontology-based stream reasoning framework. This framework consumes RDF streams with expiration timestamps assigned by the streaming source. Our framework utilizes both arrival and expiration timestamps in its cache eviction policies. In addition, we introduce the notion of "semantic importance" which aims to address the relevance of data to the expected reasoning, thus enabling the eviction algorithms to be more context- and reasoning-aware when choosing what data to maintain for question answering. We evaluate this framework by implementing three different prototypes and utilizing five metrics. The trade-offs of deploying the proposed framework are also discussed.

RStorm: Developing and Testing Streaming Algorithms in R

NARCIS (Netherlands)

Kaptein, M.C.

2014-01-01

Streaming data, consisting of indefinitely evolving sequences, are becoming ubiquitous in many branches of science and in various applications. Computer scientists have developed streaming applications such as Storm and the S4 distributed stream computing platform1 to deal with data streams.

RStorm : Developing and testing streaming algorithms in R

NARCIS (Netherlands)

Kaptein, M.C.

2014-01-01

Streaming data, consisting of indefinitely evolving sequences, are becoming ubiquitous in many branches of science and in various applications. Computer scientists have developed streaming applications such as Storm and the S4 distributed stream computing platform1 to deal with data streams.

Multiple-Features-Based Semisupervised Clustering DDoS Detection Method

Directory of Open Access Journals (Sweden)

Yonghao Gu

2017-01-01

Full Text Available DDoS attack stream from different agent host converged at victim host will become very large, which will lead to system halt or network congestion. Therefore, it is necessary to propose an effective method to detect the DDoS attack behavior from the massive data stream. In order to solve the problem that large numbers of labeled data are not provided in supervised learning method, and the relatively low detection accuracy and convergence speed of unsupervised k-means algorithm, this paper presents a semisupervised clustering detection method using multiple features. In this detection method, we firstly select three features according to the characteristics of DDoS attacks to form detection feature vector. Then, Multiple-Features-Based Constrained-K-Means (MF-CKM algorithm is proposed based on semisupervised clustering. Finally, using MIT Laboratory Scenario (DDoS 1.0 data set, we verify that the proposed method can improve the convergence speed and accuracy of the algorithm under the condition of using a small amount of labeled data sets.

A novel WDM passive optical network architecture supporting two independent multicast data streams

Science.gov (United States)

Qiu, Yang; Chan, Chun-Kit

2012-01-01

We propose a novel scheme to perform optical multicast overlay of two independent multicast data streams on a wavelength-division-multiplexed (WDM) passive optical network. By controlling a sinusoidal clock signal and shifting the wavelength at the optical line terminal (OLT), the delivery of the two multicast data, being carried by the generated optical tones, can be independently and flexibly controlled. Simultaneous transmission of 10-Gb/s unicast downstream and upstream data as well as two independent 10-Gb/s multicast data was successfully demonstrated.

MIVIS image geocoding experience on merging position attitude system data and public domain GPS stream (ASI-GeoDAF

Directory of Open Access Journals (Sweden)

S. Pignatti

2006-06-01

Full Text Available The use of airborne scanners involves geo-referencing problems, which are difficult because of the need to know the exact platform position and attitude for each scan line. The errors of the onboard navigation system are normally corrected using ground control point on the image. This post-processing correction procedure is too long in case of multiple flight campaigns, and besides it implies the need to have available 1:10000 orthophotoimages or maps in digital format. To optimize the above procedure a new method to correct MIVIS navigational data in the post-processing phase has been implemented. The procedure takes into consideration the GPS stream in Rinex format of common knowledge and findable on the web, acquired at the ground stations of the Geodetic Data Archiving Facilities provided by ASI. The application of this correction entails the assumption that the environmental variables affecting both onboard and geodetic GPS equally affect the position measurements. The airborne data correction was carried out merging the two data sets (onboard and ground station GPS to achieve a more precise aircraft trajectory. The present study compares the geo-coded images obtained by means of the two post-processing methods.

Estimating stream discharge from a Himalayan Glacier using coupled satellite sensor data

Science.gov (United States)

Child, S. F.; Stearns, L. A.; van der Veen, C. J.; Haritashya, U. K.; Tarpanelli, A.

2015-12-01

The 4th IPCC report highlighted our limited understanding of Himalayan glacier behavior and contribution to the region's hydrology. Seasonal snow and glacier melt in the Himalayas are important sources of water, but estimates greatly differ about the actual contribution of melted glacier ice to stream discharge. A more comprehensive understanding of the contribution of glaciers to stream discharge is needed because streams being fed by glaciers affect the livelihoods of a large part of the world's population. Most of the streams in the Himalayas are unmonitored because in situ measurements are logistically difficult and costly. This necessitates the use of remote sensing platforms to obtain estimates of river discharge for validating hydrological models. In this study, we estimate stream discharge using cost-effective methods via repeat satellite imagery from Landsat-8 and SENTINEL-1A sensors. The methodology is based on previous studies, which show that ratio values from optical satellite bands correlate well with measured stream discharge. While similar, our methodology relies on significantly higher resolution imagery (30 m) and utilizes bands that are in the blue and near-infrared spectrum as opposed to previous studies using 250 m resolution imagery and spectral bands only in the near-infrared. Higher resolution imagery is necessary for streams where the source is a glacier's terminus because the width of the stream is often only 10s of meters. We validate our methodology using two rivers in the state of Kansas, where stream gauges are plentiful. We then apply our method to the Bhagirathi River, in the North-Central Himalayas, which is fed by the Gangotri Glacier and has a well monitored stream gauge. The analysis will later be used to couple river discharge and glacier flow and mass balance through an integrated hydrologic model in the Bhagirathi Basin.

Tryton Supercomputer Capabilities for Analysis of Massive Data Streams

Directory of Open Access Journals (Sweden)

Krawczyk Henryk

2015-09-01

Full Text Available The recently deployed supercomputer Tryton, located in the Academic Computer Center of Gdansk University of Technology, provides great means for massive parallel processing. Moreover, the status of the Center as one of the main network nodes in the PIONIER network enables the fast and reliable transfer of data produced by miscellaneous devices scattered in the area of the whole country. The typical examples of such data are streams containing radio-telescope and satellite observations. Their analysis, especially with real-time constraints, can be challenging and requires the usage of dedicated software components. We propose a solution for such parallel analysis using the supercomputer, supervised by the KASKADA platform, which with the conjunction with immerse 3D visualization techniques can be used to solve problems such as pulsar detection and chronometric or oil-spill simulation on the sea surface.

Developing semi-analytical solution for multiple-zone transient storage model with spatially non-uniform storage

Science.gov (United States)

Deng, Baoqing; Si, Yinbing; Wang, Jia

2017-12-01

Transient storages may vary along the stream due to stream hydraulic conditions and the characteristics of storage. Analytical solutions of transient storage models in literature didn't cover the spatially non-uniform storage. A novel integral transform strategy is presented that simultaneously performs integral transforms to the concentrations in the stream and in storage zones by using the single set of eigenfunctions derived from the advection-diffusion equation of the stream. The semi-analytical solution of the multiple-zone transient storage model with the spatially non-uniform storage is obtained by applying the generalized integral transform technique to all partial differential equations in the multiple-zone transient storage model. The derived semi-analytical solution is validated against the field data in literature. Good agreement between the computed data and the field data is obtained. Some illustrative examples are formulated to demonstrate the applications of the present solution. It is shown that solute transport can be greatly affected by the variation of mass exchange coefficient and the ratio of cross-sectional areas. When the ratio of cross-sectional areas is big or the mass exchange coefficient is small, more reaches are recommended to calibrate the parameter.

Multiple attenuation to reflection seismic data using Radon filter and Wave Equation Multiple Rejection (WEMR) method

Energy Technology Data Exchange (ETDEWEB)

Erlangga, Mokhammad Puput [Geophysical Engineering, Institut Teknologi Bandung, Ganesha Street no.10 Basic Science B Buliding fl.2-3 Bandung, 40132, West Java Indonesia puput.erlangga@gmail.com (Indonesia)

2015-04-16

Separation between signal and noise, incoherent or coherent, is important in seismic data processing. Although we have processed the seismic data, the coherent noise is still mixing with the primary signal. Multiple reflections are a kind of coherent noise. In this research, we processed seismic data to attenuate multiple reflections in the both synthetic and real seismic data of Mentawai. There are several methods to attenuate multiple reflection, one of them is Radon filter method that discriminates between primary reflection and multiple reflection in the τ-p domain based on move out difference between primary reflection and multiple reflection. However, in case where the move out difference is too small, the Radon filter method is not enough to attenuate the multiple reflections. The Radon filter also produces the artifacts on the gathers data. Except the Radon filter method, we also use the Wave Equation Multiple Elimination (WEMR) method to attenuate the long period multiple reflection. The WEMR method can attenuate the long period multiple reflection based on wave equation inversion. Refer to the inversion of wave equation and the magnitude of the seismic wave amplitude that observed on the free surface, we get the water bottom reflectivity which is used to eliminate the multiple reflections. The WEMR method does not depend on the move out difference to attenuate the long period multiple reflection. Therefore, the WEMR method can be applied to the seismic data which has small move out difference as the Mentawai seismic data. The small move out difference on the Mentawai seismic data is caused by the restrictiveness of far offset, which is only 705 meter. We compared the real free multiple stacking data after processing with Radon filter and WEMR process. The conclusion is the WEMR method can more attenuate the long period multiple reflection than the Radon filter method on the real (Mentawai) seismic data.

Performance Evaluation of Concurrent Multipath Video Streaming in Multihomed Mobile Networks

Directory of Open Access Journals (Sweden)

James Nightingale

2013-01-01

Full Text Available High-quality real-time video streaming to users in mobile networks is challenging due to the dynamically changing nature of the network paths, particularly the limited bandwidth and varying end-to-end delay. In this paper, we empirically investigate the performance of multipath streaming in the context of multihomed mobile networks. Existing schemes that make use of the aggregated bandwidth of multiple paths can overcome bandwidth limitations on a single path but suffer an efficiency penalty caused by retransmission of lost packets in reliable transport schemes or path switching overheads in unreliable transport schemes. This work focuses on the evaluation of schemes to permit concurrent use of multiple paths to deliver video streams. A comprehensive streaming framework for concurrent multipath video streaming is proposed and experimentally evaluated, using current state-of-the-art H.264 Scalable Video Coding (H.264/SVC and the next generation High Efficiency Video Coding (HEVC standards. It provides a valuable insight into the benefit of using such schemes in conjunction with encoder specific packet prioritisation mechanisms for quality-aware packet scheduling and scalable streaming. The remaining obstacles to deployment of concurrent multipath schemes are identified, and the challenges in realising HEVC based concurrent multipath streaming are highlighted.

StreamAR: incremental and active learning with evolving sensory data for activity recognition

OpenAIRE

Abdallah, Z.; Gaber, M.; Srinivasan, B.; Krishnaswamy, S.

2012-01-01

Activity recognition focuses on inferring current user activities by leveraging sensory data available on today’s sensor rich environment. Supervised learning has been applied pervasively for activity recognition. Typical activity recognition techniques process sensory data based on point-by-point approaches. In this paper, we propose a novel cluster-based classification for activity recognition Systems, termed StreamAR. The system incorporates incremental and active learning for mining user ...

The role of the geophysical template and environmental regimes in controlling stream-living trout populations

Science.gov (United States)

Penaluna, Brooke E.; Railsback, Steve F.; Dunham, Jason B.; Johnson, S.; Bilby, Richard E.; Skaugset, Arne E.

2015-01-01

The importance of multiple processes and instream factors to aquatic biota has been explored extensively, but questions remain about how local spatiotemporal variability of aquatic biota is tied to environmental regimes and the geophysical template of streams. We used an individual-based trout model to explore the relative role of the geophysical template versus environmental regimes on biomass of trout (Oncorhynchus clarkii clarkii). We parameterized the model with observed data from each of the four headwater streams (their local geophysical template and environmental regime) and then ran 12 simulations where we replaced environmental regimes (stream temperature, flow, turbidity) of a given stream with values from each neighboring stream while keeping the geophysical template fixed. We also performed single-parameter sensitivity analyses on the model results from each of the four streams. Although our modeled findings show that trout biomass is most responsive to changes in the geophysical template of streams, they also reveal that biomass is restricted by available habitat during seasonal low flow, which is a product of both the stream’s geophysical template and flow regime. Our modeled results suggest that differences in the geophysical template among streams render trout more or less sensitive to environmental change, emphasizing the importance of local fish–habitat relationships in streams.

A Statistical Method to Predict Flow Permanence in Dryland Streams from Time Series of Stream Temperature

Directory of Open Access Journals (Sweden)

Ivan Arismendi

2017-12-01

Full Text Available Intermittent and ephemeral streams represent more than half of the length of the global river network. Dryland freshwater ecosystems are especially vulnerable to changes in human-related water uses as well as shifts in terrestrial climates. Yet, the description and quantification of patterns of flow permanence in these systems is challenging mostly due to difficulties in instrumentation. Here, we took advantage of existing stream temperature datasets in dryland streams in the northwest Great Basin desert, USA, to extract critical information on climate-sensitive patterns of flow permanence. We used a signal detection technique, Hidden Markov Models (HMMs, to extract information from daily time series of stream temperature to diagnose patterns of stream drying. Specifically, we applied HMMs to time series of daily standard deviation (SD of stream temperature (i.e., dry stream channels typically display highly variable daily temperature records compared to wet stream channels between April and August (2015–2016. We used information from paired stream and air temperature data loggers as well as co-located stream temperature data loggers with electrical resistors as confirmatory sources of the timing of stream drying. We expanded our approach to an entire stream network to illustrate the utility of the method to detect patterns of flow permanence over a broader spatial extent. We successfully identified and separated signals characteristic of wet and dry stream conditions and their shifts over time. Most of our study sites within the entire stream network exhibited a single state over the entire season (80%, but a portion of them showed one or more shifts among states (17%. We provide recommendations to use this approach based on a series of simple steps. Our findings illustrate a successful method that can be used to rigorously quantify flow permanence regimes in streams using existing records of stream temperature.

A statistical method to predict flow permanence in dryland streams from time series of stream temperature

Science.gov (United States)

Arismendi, Ivan; Dunham, Jason B.; Heck, Michael; Schultz, Luke; Hockman-Wert, David

2017-01-01

Intermittent and ephemeral streams represent more than half of the length of the global river network. Dryland freshwater ecosystems are especially vulnerable to changes in human-related water uses as well as shifts in terrestrial climates. Yet, the description and quantification of patterns of flow permanence in these systems is challenging mostly due to difficulties in instrumentation. Here, we took advantage of existing stream temperature datasets in dryland streams in the northwest Great Basin desert, USA, to extract critical information on climate-sensitive patterns of flow permanence. We used a signal detection technique, Hidden Markov Models (HMMs), to extract information from daily time series of stream temperature to diagnose patterns of stream drying. Specifically, we applied HMMs to time series of daily standard deviation (SD) of stream temperature (i.e., dry stream channels typically display highly variable daily temperature records compared to wet stream channels) between April and August (2015–2016). We used information from paired stream and air temperature data loggers as well as co-located stream temperature data loggers with electrical resistors as confirmatory sources of the timing of stream drying. We expanded our approach to an entire stream network to illustrate the utility of the method to detect patterns of flow permanence over a broader spatial extent. We successfully identified and separated signals characteristic of wet and dry stream conditions and their shifts over time. Most of our study sites within the entire stream network exhibited a single state over the entire season (80%), but a portion of them showed one or more shifts among states (17%). We provide recommendations to use this approach based on a series of simple steps. Our findings illustrate a successful method that can be used to rigorously quantify flow permanence regimes in streams using existing records of stream temperature.

Relationship between bifenthrin sediment toxic units and benthic community metrics in urban California streams.

Science.gov (United States)

Hall, Lenwood W; Anderson, Ronald D

2013-08-01

The objective of this study was to use ecologically relevant field measurements for determining the relationship between bifenthrin sediment toxic units (TUs) (environmental concentrations/Hyalella acute LC50 value) and 15 benthic metrics in four urban California streams sampled from 2006 to 2011. Data from the following four California streams were used in the analysis: Kirker Creek (2006, 2007), Pleasant Grove Creek (2006, 2007, and 2008), Arcade Creek (2009, 2010, and 2011), and Salinas streams (2009, 2010, and 2011). The results from univariate analysis of benthic metrics versus bifenthrin TU calculations for the four California streams with multiple-year datasets combined by stream showed that there were either nonsignificant relationships or lack of metric data for 93 % of cases. For 7 % of the data (4 cases) where significant relationships were reported between benthic metrics and bifenthrin TUs, these relationships were ecologically meaningful. Three of these significant direct relationships were an expression of tolerant benthic taxa (either % tolerant taxa or tolerance values, which are similar metrics), which would be expected to increase in a stressed environment. These direct significant tolerance relationships were reported for Kirker Creek, Pleasant Grove Creek, and Arcade Creek. The fourth significant relationship was an inverse relationship between taxa richness and bifenthrin TUs for the 3-year Pleasant Grove Creek dataset. In summary, only a small percent of the benthic metric × bifenthrin TU relationships were significant for the four California streams. Therefore, the general summary conclusion from this analysis is that there is no strong case for showing consistent meaningful relationships between various benthic metrics used to characterize the status of benthic communities and bifenthrin TUs for these four California streams.

Recurrent and Dynamic Models for Predicting Streaming Video Quality of Experience.

Science.gov (United States)

Bampis, Christos G; Li, Zhi; Katsavounidis, Ioannis; Bovik, Alan C

2018-07-01

Streaming video services represent a very large fraction of global bandwidth consumption. Due to the exploding demands of mobile video streaming services, coupled with limited bandwidth availability, video streams are often transmitted through unreliable, low-bandwidth networks. This unavoidably leads to two types of major streaming-related impairments: compression artifacts and/or rebuffering events. In streaming video applications, the end-user is a human observer; hence being able to predict the subjective Quality of Experience (QoE) associated with streamed videos could lead to the creation of perceptually optimized resource allocation strategies driving higher quality video streaming services. We propose a variety of recurrent dynamic neural networks that conduct continuous-time subjective QoE prediction. By formulating the problem as one of time-series forecasting, we train a variety of recurrent neural networks and non-linear autoregressive models to predict QoE using several recently developed subjective QoE databases. These models combine multiple, diverse neural network inputs, such as predicted video quality scores, rebuffering measurements, and data related to memory and its effects on human behavioral responses, using them to predict QoE on video streams impaired by both compression artifacts and rebuffering events. Instead of finding a single time-series prediction model, we propose and evaluate ways of aggregating different models into a forecasting ensemble that delivers improved results with reduced forecasting variance. We also deploy appropriate new evaluation metrics for comparing time-series predictions in streaming applications. Our experimental results demonstrate improved prediction performance that approaches human performance. An implementation of this work can be found at https://github.com/christosbampis/NARX_QoE_release.

Numeric Analysis for Relationship-Aware Scalable Streaming Scheme

Directory of Open Access Journals (Sweden)

Heung Ki Lee

2014-01-01

Full Text Available Frequent packet loss of media data is a critical problem that degrades the quality of streaming services over mobile networks. Packet loss invalidates frames containing lost packets and other related frames at the same time. Indirect loss caused by losing packets decreases the quality of streaming. A scalable streaming service can decrease the amount of dropped multimedia resulting from a single packet loss. Content providers typically divide one large media stream into several layers through a scalable streaming service and then provide each scalable layer to the user depending on the mobile network. Also, a scalable streaming service makes it possible to decode partial multimedia data depending on the relationship between frames and layers. Therefore, a scalable streaming service provides a way to decrease the wasted multimedia data when one packet is lost. However, the hierarchical structure between frames and layers of scalable streams determines the service quality of the scalable streaming service. Even if whole packets of layers are transmitted successfully, they cannot be decoded as a result of the absence of reference frames and layers. Therefore, the complicated relationship between frames and layers in a scalable stream increases the volume of abandoned layers. For providing a high-quality scalable streaming service, we choose a proper relationship between scalable layers as well as the amount of transmitted multimedia data depending on the network situation. We prove that a simple scalable scheme outperforms a complicated scheme in an error-prone network. We suggest an adaptive set-top box (AdaptiveSTB to lower the dependency between scalable layers in a scalable stream. Also, we provide a numerical model to obtain the indirect loss of multimedia data and apply it to various multimedia streams. Our AdaptiveSTB enhances the quality of a scalable streaming service by removing indirect loss.

Standards for Multi-Stream and Multi-Device Media Synchronisation

NARCIS (Netherlands)

Deventer, M.O. van; Stokking, H.M.; Hammond, M.; Cesar, P.

2016-01-01

Media synchronization is getting renewed attention with ecosystems of connected devices enabling novel media consumption paradigms. Social TV, hybrid TV, and companion screens are examples that are enabling people to consume multiple media streams at multiple devices together. These novel use cases

Extending Data Worth Analyses to Select Multiple Observations Targeting Multiple Forecasts.

Science.gov (United States)

Vilhelmsen, Troels N; Ferré, Ty P A

2017-09-15

Hydrological models are often set up to provide specific forecasts of interest. Owing to the inherent uncertainty in data used to derive model structure and used to constrain parameter variations, the model forecasts will be uncertain. Additional data collection is often performed to minimize this forecast uncertainty. Given our common financial restrictions, it is critical that we identify data with maximal information content with respect to forecast of interest. In practice, this often devolves to qualitative decisions based on expert opinion. However, there is no assurance that this will lead to optimal design, especially for complex hydrogeological problems. Specifically, these complexities include considerations of multiple forecasts, shared information among potential observations, information content of existing data, and the assumptions and simplifications underlying model construction. In the present study, we extend previous data worth analyses to include: simultaneous selection of multiple new measurements and consideration of multiple forecasts of interest. We show how the suggested approach can be used to optimize data collection. This can be used in a manner that suggests specific measurement sets or that produces probability maps indicating areas likely to be informative for specific forecasts. Moreover, we provide examples documenting that sequential measurement election approaches often lead to suboptimal designs and that estimates of data covariance should be included when selecting future measurement sets. © 2017, National Ground Water Association.

Yosemite Hydroclimate Network: Distributed stream and atmospheric data for the Tuolumne River watershed and surroundings

Science.gov (United States)

Lundquist, Jessica D.; Roche, James W.; Forrester, Harrison; Moore, Courtney; Keenan, Eric; Perry, Gwyneth; Cristea, Nicoleta; Henn, Brian; Lapo, Karl; McGurk, Bruce; Cayan, Daniel R.; Dettinger, Michael D.

2016-01-01

Regions of complex topography and remote wilderness terrain have spatially varying patterns of temperature and streamflow, but due to inherent difficulties of access, are often very poorly sampled. Here we present a data set of distributed stream stage, streamflow, stream temperature, barometric pressure, and air temperature from the Tuolumne River Watershed in Yosemite National Park, Sierra Nevada, California, USA, for water years 2002–2015, as well as a quality-controlled hourly meteorological forcing time series for use in hydrologic modeling. We also provide snow data and daily inflow to the Hetch Hetchy Reservoir for 1970–2015. This paper describes data collected using low-visibility and low-impact installations for wilderness locations and can be used alone or as a critical supplement to ancillary data sets collected by cooperating agencies, referenced herein. This data set provides a unique opportunity to understand spatial patterns and scaling of hydroclimatic processes in complex terrain and can be used to evaluate downscaling techniques or distributed modeling. The paper also provides an example methodology and lessons learned in conducting hydroclimatic monitoring in remote wilderness.

How wide is a stream? Spatial extent of the potential "stream signature" in terrestrial food webs using meta-analysis.

Science.gov (United States)

Muehlbauer, Jeffrey D; Collins, Scott F; Doyle, Martin W; Tockner, Klement

2014-01-01

The magnitude of cross-ecosystem resource subsidies is increasingly well recognized; however, less is known about the distance these subsidies travel into the recipient landscape. In streams and rivers, this distance can delimit the "biological stream width," complementary to hydro-geomorphic measures (e.g., channel banks) that have typically defined stream ecosystem boundaries. In this study we used meta-analysis to define a "stream signature" on land that relates the stream-to-land subsidy to distance. The 50% stream signature, for example, identifies the point on the landscape where subsidy resources are still at half of their maximum (in- or near-stream) level. The decay curve for these data was best fit by a negative power function in which the 50% stream signature was concentrated near stream banks (1.5 m), but a non-trivial (10%) portion of the maximum subsidy level was still found > 0.5 km from the water's edge. The meta-analysis also identified explanatory variables that affect the stream signature. This improves our understanding of ecosystem conditions that permit spatially extensive subsidy transmission, such as in highly productive, middle-order streams and rivers. Resultant multivariate models from this analysis may be useful to managers implementing buffer rules and conservation strategies for stream and riparian function, as they facilitate prediction of the extent of subsidies. Our results stress that much of the subsidy remains near the stream, but also that subsidies (and aquatic organisms) are capable of long-distance dispersal into adjacent environments, and that the effective "biological stream width" of stream and river ecosystems is often much larger than has been defined by hydro-geomorphic metrics alone. Limited data available from marine and lake sources overlap well with the stream signature data, indicating that the "signature" approach may also be applicable to subsidy spatial dynamics across other ecosystems.

Modeling and clustering users with evolving profiles in usage streams

KAUST Repository

Zhang, Chongsheng; Masseglia, Florent; Zhang, Xiangliang

2012-01-01

Today, there is an increasing need of data stream mining technology to discover important patterns on the fly. Existing data stream models and algorithms commonly assume that users' records or profiles in data streams will not be updated or revised once they arrive. Nevertheless, in various applications such asWeb usage, the records/profiles of the users can evolve along time. This kind of streaming data evolves in two forms, the streaming of tuples or transactions as in the case of traditional data streams, and more importantly, the evolving of user records/profiles inside the streams. Such data streams bring difficulties on modeling and clustering for exploring users' behaviors. In this paper, we propose three models to summarize this kind of data streams, which are the batch model, the Evolving Objects (EO) model and the Dynamic Data Stream (DDS) model. Through creating, updating and deleting user profiles, these models summarize the behaviors of each user as a profile object. Based upon these models, clustering algorithms are employed to discover interesting user groups from the profile objects. We have evaluated all the proposed models on a large real-world data set, showing that the DDS model summarizes the data streams with evolving tuples more efficiently and effectively, and provides better basis for clustering users than the other two models. © 2012 IEEE.

Modeling and clustering users with evolving profiles in usage streams

KAUST Repository

Zhang, Chongsheng

2012-09-01

Today, there is an increasing need of data stream mining technology to discover important patterns on the fly. Existing data stream models and algorithms commonly assume that users\\' records or profiles in data streams will not be updated or revised once they arrive. Nevertheless, in various applications such asWeb usage, the records/profiles of the users can evolve along time. This kind of streaming data evolves in two forms, the streaming of tuples or transactions as in the case of traditional data streams, and more importantly, the evolving of user records/profiles inside the streams. Such data streams bring difficulties on modeling and clustering for exploring users\\' behaviors. In this paper, we propose three models to summarize this kind of data streams, which are the batch model, the Evolving Objects (EO) model and the Dynamic Data Stream (DDS) model. Through creating, updating and deleting user profiles, these models summarize the behaviors of each user as a profile object. Based upon these models, clustering algorithms are employed to discover interesting user groups from the profile objects. We have evaluated all the proposed models on a large real-world data set, showing that the DDS model summarizes the data streams with evolving tuples more efficiently and effectively, and provides better basis for clustering users than the other two models. © 2012 IEEE.

Population persistence of stream fish in response to environmental change: integrating data and models across space

Science.gov (United States)

Letcher, B. H.; Schueller, P.; Bassar, R.; Coombs, J.; Rosner, A.; Sakrejda, K.; Kanno, Y.; Whiteley, A.; Nislow, K. H.

2013-12-01

For stream fishes, environmental variation is a key driver of individual body growth/movement/survival and, by extension, population dynamics. Identifying how stream fish respond to environmental variation can help clarify mechanisms responsible for population dynamics and can help provide tools to forecast relative resilience of populations across space. Forecasting dynamics across space is challenging, however, because it can be difficult to conduct enough studies with enough intensity to fully characterize broad-scale population response to environmental change. We have adopted a multi-scale approach, using detailed individual-based studies and analyses (integral projection matrix) to determine sensitivities of population growth to environmental variation combined with broad spatial data and analyses (occupancy and abundance models) to estimate patterns of population response across space. Population growth of brook trout was most sensitive to stream flow in the spring and winter, most sensitive to stream temperature in the fall and sensitive to both flow and temperature in the summer. High flow in the spring and winter had negative effects on population growth while high temperature had a negative effect in the fall. Flow had no effect when it was cold, but a positive effect when it was warm in the summer. Combined with occupancy and abundance models, these data give insight into the spatial structure of resilient populations and can help guide prioritization of management actions.

Hydrogeochemical and stream sediment reconnaissance basic data for Dallas NTMS Quadrangle, Texas

International Nuclear Information System (INIS)

1981-01-01

Results of a reconnaissance geochemical survey of the Dallas Quadrangle, Texas are reported. Field and laboratory data are presented for 284 groundwater and 545 stream sediment samples. Statistical and areal distribution plots of uranium and possible uranium-related variables are displayed. A generalized geologic map of the survey area is provided. Groundwater produced from the Navarro Group, Neylandville Formation, Marlbrook Marl, and the Glen Rose and Twin Mountains Formations exhibit anomalous uranium (> 9.05 ppB) and specific conductance (> 1871 μmhos/cm) values. The anomalies represent a southern extension of a similar trend observed in the Sherman Quadrangle, K/UR-110. Stream sediments representing the Eagle Ford Group and Woodbine Formation exhibit the highest concentrations of total and hot-acid-soluble uranium and thorium of samples collected in the Dallas Quadrangle. The U/TU value indicates that > 80% of this uranium is present in a soluble form

Computer analysis to the geochemical interpretation of soil and stream sediment data in an area of Southern Uruguay

International Nuclear Information System (INIS)

Spangenberg, J.

2010-01-01

In southern Uruguay there are several known occurrences of base metal sulphide mineralization within an area of Precambrian volcanic sedimentary rocks. Regional geochemical stream sediment reconnaissance surveys revealed new polymetallic anomalies in the same stratigraphic zone. Geochemical interpretation of multi-element data from a soil and stream sediment survey carried out in one of these anomalous areas is presented.

Multiple streams approach to tobacco control policymaking in a tobacco-growing state.

Science.gov (United States)

Mamudu, Hadii M; Dadkar, Sumati; Veeranki, Sreenivas P; He, Yi; Barnes, Richard; Glantz, Stanton A

2014-08-01

Smokefree policies (SFPs) have diffused throughout the US and worldwide. However, the development of SFPs in the difficult policy environment of tobacco-producing states and economies worldwide has not been well-explored. In 2007, Tennessee, the third largest tobacco producer in the US, enacted the Non-Smoker Protection Act (NSPA). This study utilizes the multiple streams model to provide understanding of why and how this policy was developed by triangulating interviews with key stakeholders and legislative debates with archival documents. In June 2006, the Governor unexpectedly announced support for SFP, which created a window of opportunity for policy change. The Campaign for Healthy and Responsible Tennessee, a health coalition, seized this opportunity and worked with the administration and the Tennessee Restaurant Association to negotiate a comprehensive SFP, however, a weaker bill was used by the legislative leadership to develop the NSPA. Although the Governor and the Tennessee Restaurant Association's support generated an environment for 100% SFP, health groups did not fully capitalize on this environmental change and settled for a weak policy with several exemptions. This study suggests the importance for proponents of policy change to understand changes in their environment and be willing and able to capitalize on these changes.

Real-Time Management of Multimodal Streaming Data for Monitoring of Epileptic Patients.

Science.gov (United States)

Triantafyllopoulos, Dimitrios; Korvesis, Panagiotis; Mporas, Iosif; Megalooikonomou, Vasileios

2016-03-01

New generation of healthcare is represented by wearable health monitoring systems, which provide real-time monitoring of patient's physiological parameters. It is expected that continuous ambulatory monitoring of vital signals will improve treatment of patients and enable proactive personal health management. In this paper, we present the implementation of a multimodal real-time system for epilepsy management. The proposed methodology is based on a data streaming architecture and efficient management of a big flow of physiological parameters. The performance of this architecture is examined for varying spatial resolution of the recorded data.

HUNTING THE PARENT OF THE ORPHAN STREAM: IDENTIFYING STREAM MEMBERS FROM LOW-RESOLUTION SPECTROSCOPY

International Nuclear Information System (INIS)

Casey, Andrew R.; Da Costa, Gary; Keller, Stefan C.; Maunder, Elizabeth

2013-01-01

We present candidate K-giant members in the Orphan Stream that have been identified from low-resolution data taken with the AAOmega spectrograph on the Anglo-Australian Telescope. From modest signal-to-noise spectra and independent cuts in photometry, kinematics, gravity, and metallicity we yield self-consistent, highly probable stream members. We find a revised stream distance of 22.5 ± 2.0 kpc near the celestial equator and our kinematic signature peaks at V GSR = 82.1 ± 1.4 km s –1 . The observed velocity dispersion of our most probable members is consistent with arising from the velocity uncertainties alone. This indicates that at least along this line of sight, the Orphan Stream is kinematically cold. Our data indicate an overall stream metallicity of [Fe/H] = –1.63 ± 0.19 dex which is more metal-rich than previously found and unbiased by spectral type. Furthermore, the significant metallicity dispersion displayed by our most probable members, σ([Fe/H]) = 0.56 dex, suggests that the unidentified Orphan Stream parent is a dSph satellite. We highlight likely members for high-resolution spectroscopic follow-up.

Analytical data and sample locality map for aqua-regia leachates of stream sediments analyzed by ICP, and emission spectrographic and ICP results for many NURE stream sediments from the Killik River Quadrangle, Alaska

International Nuclear Information System (INIS)

Motooka, J.M.; Adrian, B.M.; Church, S.E.; McDougal, C.M.; Fife, J.B.

1989-01-01

A U.S. Geological Survey report is presented giving analytical data and sample locality map for aqua-regia leachates of stream sediments analyzed by ICP, and emission spectrographic and ICP results for many NURE stream sediments from the Killik River Quadrangle, Alaska

Death Valley 10 x 20 NTMS area, California and Nevada. Data report: National Uranium Resource Evaluation program, hydrogeochemical and stream sediment reconnaissance

International Nuclear Information System (INIS)

Cook, J.R.

1980-04-01

Results of ground water and stream sediment reconnaissance in the National Topographic Map Series (NTMS) Death Valley 1 0 x 2 0 quadrangle are presented. Stream sediment samples were collected from small streams at 649 sites or at a nominal density of one site per 20 square kilometers. Ground water samples were collected at 62 sites or at a nominal density of one site per 220 square kilometers. Neutron activation analysis results are given for uranium and 16 other elements in sediments, and for uranium and 8 other elements in ground water and surface water. Mass spectrometry results are given for helium in ground water. Field measurements and observations are reported for each site. Analytical data and field measurements are presented in tables and maps. Statistical summaries of data and a brief description of results are given. A generalized geologic map and a summary of the geology of the area are included. Key data from ground water sites include (1) water chemistry measurements (pH, conductivity, and alkalinity), (2) scintillometer readings, and (3) elemental analyses (U, Br, Cl, F, He, Mn, Na, and V). Supplementary data include site descriptors, tabulated analytical data for Al, Dy, and Mg, and histograms and cumulative frequency plots for all elements. Key data from stream sediment sites include (1) water quality measurements (2) important elemental analyses, (U, Th, Hf, Ce, Fe, Mn, Sc, Na, Ti, and V), and (3) scintillometer readings. Supplementary data from stream sediment sites include sample site descriptors (stream characteristics, vegetation, etc.), additional elemental analyses (Dy, Eu, La, Lu, Sm, and Yb), and histograms and cumulative frequency plots for all elements

ThermoData Engine: Extension to Solvent Design and Multi-component Process Stream Property Calculations with Uncertainty Analysis

DEFF Research Database (Denmark)

Diky, Vladimir; Chirico, Robert D.; Muzny, Chris

ThermoData Engine (TDE, NIST Standard Reference Databases 103a and 103b) is the first product that implements the concept of Dynamic Data Evaluation in the fields of thermophysics and thermochemistry, which includes maintaining the comprehensive and up-to-date database of experimentally measured...... property values and expert system for data analysis and generation of recommended property values at the specified conditions along with uncertainties on demand. The most recent extension of TDE covers solvent design and multi-component process stream property calculations with uncertainty analysis...... variations). Predictions can be compared to the available experimental data, and uncertainties are estimated for all efficiency criteria. Calculations of the properties of multi-component streams including composition at phase equilibria (flash calculations) are at the heart of process simulation engines...

Can Low-Resolution Airborne Laser Scanning Data Be Used to Model Stream Rating Curves?

Directory of Open Access Journals (Sweden)

Steve W. Lyon

2015-03-01

Full Text Available This pilot study explores the potential of using low-resolution (0.2 points/m2 airborne laser scanning (ALS-derived elevation data to model stream rating curves. Rating curves, which allow the functional translation of stream water depth into discharge, making them integral to water resource monitoring efforts, were modeled using a physics-based approach that captures basic geometric measurements to establish flow resistance due to implicit channel roughness. We tested synthetically thinned high-resolution (more than 2 points/m2 ALS data as a proxy for low-resolution data at a point density equivalent to that obtained within most national-scale ALS strategies. Our results show that the errors incurred due to the effect of low-resolution versus high-resolution ALS data were less than those due to flow measurement and empirical rating curve fitting uncertainties. As such, although there likely are scale and technical limitations to consider, it is theoretically possible to generate rating curves in a river network from ALS data of the resolution anticipated within national-scale ALS schemes (at least for rivers with relatively simple geometries. This is promising, since generating rating curves from ALS scans would greatly enhance our ability to monitor streamflow by simplifying the overall effort required.

Can low-resolution airborne laser scanning data be used to model stream rating curves?

Science.gov (United States)

Lyon, Steve; Nathanson, Marcus; Lam, Norris; Dahlke, Helen; Rutzinger, Martin; Kean, Jason W.; Laudon, Hjalmar

2015-01-01

This pilot study explores the potential of using low-resolution (0.2 points/m2) airborne laser scanning (ALS)-derived elevation data to model stream rating curves. Rating curves, which allow the functional translation of stream water depth into discharge, making them integral to water resource monitoring efforts, were modeled using a physics-based approach that captures basic geometric measurements to establish flow resistance due to implicit channel roughness. We tested synthetically thinned high-resolution (more than 2 points/m2) ALS data as a proxy for low-resolution data at a point density equivalent to that obtained within most national-scale ALS strategies. Our results show that the errors incurred due to the effect of low-resolution versus high-resolution ALS data were less than those due to flow measurement and empirical rating curve fitting uncertainties. As such, although there likely are scale and technical limitations to consider, it is theoretically possible to generate rating curves in a river network from ALS data of the resolution anticipated within national-scale ALS schemes (at least for rivers with relatively simple geometries). This is promising, since generating rating curves from ALS scans would greatly enhance our ability to monitor streamflow by simplifying the overall effort required.

Streams with Strahler Stream Order

Data.gov (United States)

Minnesota Department of Natural Resources — Stream segments with Strahler stream order values assigned. As of 01/08/08 the linework is from the DNR24K stream coverages and will not match the updated...

Contaminant Dynamics and Trends in Hyperalkaline Urban Streams

Science.gov (United States)

Riley, Alex; Mayes, William

2015-04-01

Streams in post-industrial urban areas can have multiple contemporary and historic pressures impacting upon their chemical and ecological status. This paper presents analysis of long term data series (up to 36 years in length) from two small streams in northern England (catchment areas 0.5-0.6km2). Around 3.5 million m3 of steel making slags and other wastes were deposited in the headwater areas of the Howden Burn and Dene Burn in northeast England up to the closure of the workings in the early 1980s. This has led to streams draining from the former workings which have a hyperalkaline ambient pH (mean of 10.3 in both streams), elevated alkalinity (up to 487 mg/L as CaCO3) from leaching of lime and other calcium oxides / silicates within the slag, and enrichment of some trace elements (e.g. aluminium (Al), lithium (Li) and zinc (Zn)) including those which form oxyanions mobile at high pH such as vanadium (V). The high ionic strength of the waters and calcium enrichment also leads to waters highly supersaturated with calcium carbonate. Trace contaminant concentrations are strongly positively correlated, and concentrations generally diminish with increased flow rate suggesting the key source of metals in the system is the highly alkaline groundwater draining from the slag mounds. Some contaminants (notably Cr and ammonium) increase with high flow suggesting sources related to urban runoff and drainage from combined sewer overflows into one of the catchments. Loading estimates instream show that many of the contaminants (e.g. Al, V and Zn) are rapidly attenuated in secondary calcium carbonate-dominated deposits that precipitate vigorously on the streambeds with rates of up to 250 g CaCO3/m2/day. These secondary sinks limit the mobility of many contaminants in the water column, while concentrations in secondary deposits are relatively low given the rapid rates at which Ca is also attenuated. Long-term trend analysis showed modest declines in calcium and alkalinity over

Network Transfer of Control Data: An Application of the NIST SMART DATA FLOW

Directory of Open Access Journals (Sweden)

Vincent Stanford

2004-12-01

Full Text Available Pervasive Computing environments range from basic mobile point of sale terminal systems, to rich Smart Spaces with many devices and sensors such as lapel microphones, audio and video sensor arrays and multiple interactive PDA acting as electronic brief cases, providing authentication, and user preference data to the environment. These systems present new challenges in distributed human-computer interfaces such as how to best use sensor streams, distribute interfaces across multiple devices, and dynamic network management as users come an go, and as devices are added or fail. The NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY SMART DATA FLOW system is a low overhead, high bandwidth transport mechanism for standardized multi-modal data streams. It is designed to allow integration of multiple sensors with distributed processing needed for the sense-recognize-respond cycle of multi modal user interfaces. Its core is a server/client architecture, allowing clients to produce or subscribe to data flows, and supporting steps toward scalable processing, distributing the computing requirements among many network connected computers and pervasive devices. This article introduces the communication broker and provides an example of an effective real time sensor fusion to track a speaker with a video camera using data captured from multi-channel microphone array.

The kinematic footprints of five stellar streams in Andromeda's halo

Science.gov (United States)

Chapman, S. C.; Ibata, R.; Irwin, M.; Koch, A.; Letarte, B.; Martin, N.; Collins, M.; Lewis, G. F.; McConnachie, A.; Peñarrubia, J.; Rich, R. M.; Trethewey, D.; Ferguson, A.; Huxor, A.; Tanvir, N.

2008-11-01

halo is largely made up of multiple kinematically cold streams. The data presented herein were obtained at the W. M. Keck Observatory, which is operated as a scientific partnership among the California Institute of Technology, the University of California and the National Aeronautics and Space Administration. The Observatory was made possible by the generous financial support of the W. M. Keck Foundation. E-mail: schapman@ast.cam.ac.uk ‡ Canadian Space Agency, Space Science Fellow.

Autonomous Byte Stream Randomizer

Science.gov (United States)

Paloulian, George K.; Woo, Simon S.; Chow, Edward T.

2013-01-01

Net-centric networking environments are often faced with limited resources and must utilize bandwidth as efficiently as possible. In networking environments that span wide areas, the data transmission has to be efficient without any redundant or exuberant metadata. The Autonomous Byte Stream Randomizer software provides an extra level of security on top of existing data encryption methods. Randomizing the data s byte stream adds an extra layer to existing data protection methods, thus making it harder for an attacker to decrypt protected data. Based on a generated crypto-graphically secure random seed, a random sequence of numbers is used to intelligently and efficiently swap the organization of bytes in data using the unbiased and memory-efficient in-place Fisher-Yates shuffle method. Swapping bytes and reorganizing the crucial structure of the byte data renders the data file unreadable and leaves the data in a deconstructed state. This deconstruction adds an extra level of security requiring the byte stream to be reconstructed with the random seed in order to be readable. Once the data byte stream has been randomized, the software enables the data to be distributed to N nodes in an environment. Each piece of the data in randomized and distributed form is a separate entity unreadable on its own right, but when combined with all N pieces, is able to be reconstructed back to one. Reconstruction requires possession of the key used for randomizing the bytes, leading to the generation of the same cryptographically secure random sequence of numbers used to randomize the data. This software is a cornerstone capability possessing the ability to generate the same cryptographically secure sequence on different machines and time intervals, thus allowing this software to be used more heavily in net-centric environments where data transfer bandwidth is limited.

Combining Empirical Relationships with Data Based Mechanistic Modeling to Inform Solute Tracer Investigations across Stream Orders

Science.gov (United States)

Herrington, C.; Gonzalez-Pinzon, R.; Covino, T. P.; Mortensen, J.

2015-12-01

Solute transport studies in streams and rivers often begin with the introduction of conservative and reactive tracers into the water column. Information on the transport of these substances is then captured within tracer breakthrough curves (BTCs) and used to estimate, for instance, travel times and dissolved nutrient and carbon dynamics. Traditionally, these investigations have been limited to systems with small discharges (turbidity (e.g., nitrate signals with SUNA instruments or fluorescence measures) and/or high total dissolved solids (e.g., making prohibitively expensive the use of salt tracers such as NaCl) in larger systems. Additionally, a successful time-of-travel study is valuable for only a single discharge and river stage. We have developed a method to predict tracer BTCs to inform sampling frequencies at small and large stream orders using empirical relationships developed from multiple tracer injections spanning several orders of magnitude in discharge and reach length. This method was successfully tested in 1st to 8th order systems along the Middle Rio Grande River Basin in New Mexico, USA.

Stream Habitat Reach Summary - NCWAP [ds158

Data.gov (United States)

California Natural Resource Agency — The Stream Habitat - NCWAP - Reach Summary [ds158] shapefile contains in-stream habitat survey data summarized to the stream reach level. It is a derivative of the...

DEVELOPMENT OF TRANSPORT SUBSYSTEM STREAMING DATA REPLICATION CLUSTER IN CORBA-SYSTEM WITH ZEROMQ TECHNOLOGY

Directory of Open Access Journals (Sweden)

F. A. Kozlov

2013-03-01

Full Text Available The article deals with the peculiarities of distributed cluster system creation with streaming data replication. Ways of replication cluster implementation in CORBA-systems with ZeroMq technology are presented. Major advantages of ZeroMQ technology over similar technologies are considered in this type distributed systems creation.

Estimating discharge and non-point source nitrate loading to streams from three end-member pathways using high-frequency water quality and streamflow data

Science.gov (United States)

Miller, M. P.; Tesoriero, A. J.; Hood, K.; Terziotti, S.; Wolock, D.

2017-12-01

The myriad hydrologic and biogeochemical processes taking place in watersheds occurring across space and time are integrated and reflected in the quantity and quality of water in streams and rivers. Collection of high-frequency water quality data with sensors in surface waters provides new opportunities to disentangle these processes and quantify sources and transport of water and solutes in the coupled groundwater-surface water system. A new approach for separating the streamflow hydrograph into three components was developed and coupled with high-frequency specific conductance and nitrate data to estimate time-variable watershed-scale nitrate loading from three end-member pathways - dilute quickflow, concentrated quickflow, and slowflow groundwater - to two streams in central Wisconsin. Time-variable nitrate loads from the three pathways were estimated for periods of up to two years in a groundwater-dominated and a quickflow-dominated stream, using only streamflow and in-stream water quality data. The dilute and concentrated quickflow end-members were distinguished using high-frequency specific conductance data. Results indicate that dilute quickflow contributed less than 5% of the nitrate load at both sites, whereas 89±5% of the nitrate load at the groundwater-dominated stream was from slowflow groundwater, and 84±13% of the nitrate load at the quickflow-dominated stream was from concentrated quickflow. Concentrated quickflow nitrate concentrations varied seasonally at both sites, with peak concentrations in the winter that were 2-3 times greater than minimum concentrations during the growing season. Application of this approach provides an opportunity to assess stream vulnerability to non-point source nitrate loading and expected stream responses to current or changing conditions and practices in watersheds.

Multiple variables data sets visualization in ROOT

International Nuclear Information System (INIS)

Couet, O

2008-01-01

The ROOT graphical framework provides support for many different functions including basic graphics, high-level visualization techniques, output on files, 3D viewing etc. They use well-known world standards to render graphics on screen, to produce high-quality output files, and to generate images for Web publishing. Many techniques allow visualization of all the basic ROOT data types, but the graphical framework was still a bit weak in the visualization of multiple variables data sets. This paper presents latest developments done in the ROOT framework to visualize multiple variables (>4) data sets

Relative Linkages of Stream Dissolved Oxygen with the Hydroclimatic and Biogeochemical Drivers across the Gulf Coast of U.S.A.

Science.gov (United States)

Gebreslase, A. K.; Abdul-Aziz, O. I.

2017-12-01

Dynamics of coastal stream water quality is influenced by a multitude of interacting environmental drivers. A systematic data analytics approach was employed to determine the relative linkages of stream dissolved oxygen (DO) with the hydroclimatic and biogeochemical variables across the Gulf Coast of U.S.A. Multivariate pattern recognition techniques of PCA and FA, alongside Pearson's correlation matrix, were utilized to examine the interrelation of variables at 36 water quality monitoring stations from USGS NWIS and EPA STORET databases. Power-law based partial least square regression models with a bootstrap Monte Carlo procedure (1000 iterations) were developed to estimate the relative linkages of dissolved oxygen with the hydroclimatic and biogeochemical variables by appropriately resolving multicollinearity (Nash-Sutcliffe efficiency = 0.58-0.94). Based on the dominant drivers, stations were divided into four environmental regimes. Water temperature was the dominant driver of DO in the majority of streams, representing most the northern part of Gulf Coast states. However, streams in the southern part of Texas and Florida showed a dominant pH control on stream DO. Further, streams representing the transition zone of the two environmental regimes showed notable controls of multiple drivers (i.e., water temperature, stream flow, and specific conductance) on the stream DO. The data analytics research provided profound insight to understand the dynamics of stream DO with the hydroclimatic and biogeochemical variables. The knowledge can help water quality managers in formulating plans for effective stream water quality and watershed management in the U.S. Gulf Coast. Keywords Data analytics, coastal streams, relative linkages, dissolved oxygen, environmental regimes, Gulf Coast, United States.

Convergent Time-Varying Regression Models for Data Streams: Tracking Concept Drift by the Recursive Parzen-Based Generalized Regression Neural Networks.

Science.gov (United States)

Duda, Piotr; Jaworski, Maciej; Rutkowski, Leszek

2018-03-01

One of the greatest challenges in data mining is related to processing and analysis of massive data streams. Contrary to traditional static data mining problems, data streams require that each element is processed only once, the amount of allocated memory is constant and the models incorporate changes of investigated streams. A vast majority of available methods have been developed for data stream classification and only a few of them attempted to solve regression problems, using various heuristic approaches. In this paper, we develop mathematically justified regression models working in a time-varying environment. More specifically, we study incremental versions of generalized regression neural networks, called IGRNNs, and we prove their tracking properties - weak (in probability) and strong (with probability one) convergence assuming various concept drift scenarios. First, we present the IGRNNs, based on the Parzen kernels, for modeling stationary systems under nonstationary noise. Next, we extend our approach to modeling time-varying systems under nonstationary noise. We present several types of concept drifts to be handled by our approach in such a way that weak and strong convergence holds under certain conditions. Finally, in the series of simulations, we compare our method with commonly used heuristic approaches, based on forgetting mechanism or sliding windows, to deal with concept drift. Finally, we apply our concept in a real life scenario solving the problem of currency exchange rates prediction.

Nutrient spiraling in streams and river networks

Science.gov (United States)

Ensign, Scott H.; Doyle, Martin W.

2006-12-01

Over the past 3 decades, nutrient spiraling has become a unifying paradigm for stream biogeochemical research. This paper presents (1) a quantitative synthesis of the nutrient spiraling literature and (2) application of these data to elucidate trends in nutrient spiraling within stream networks. Results are based on 404 individual experiments on ammonium (NH4), nitrate (NO3), and phosphate (PO4) from 52 published studies. Sixty-nine percent of the experiments were performed in first- and second-order streams, and 31% were performed in third- to fifth-order streams. Uptake lengths, Sw, of NH4 (median = 86 m) and PO4 (median = 96 m) were significantly different (α = 0.05) than NO3 (median = 236 m). Areal uptake rates of NH4 (median = 28 μg m-2 min-1) were significantly different than NO3 and PO4 (median = 15 and 14 μg m-2 min-1, respectively). There were significant differences among NH4, NO3, and PO4 uptake velocity (median = 5, 1, and 2 mm min-1, respectively). Correlation analysis results were equivocal on the effect of transient storage on nutrient spiraling. Application of these data to a stream network model showed that recycling (defined here as stream length ÷ Sw) of NH4 and NO3 generally increased with stream order, while PO4 recycling remained constant along a first- to fifth-order stream gradient. Within this hypothetical stream network, cumulative NH4 uptake decreased slightly with stream order, while cumulative NO3 and PO4 uptake increased with stream order. These data suggest the importance of larger rivers to nutrient spiraling and the need to consider how stream networks affect nutrient flux between terrestrial and marine ecosystems.

Microtubule–microtubule sliding by kinesin-1 is essential for normal cytoplasmic streaming in Drosophila oocytes

Science.gov (United States)

Lu, Wen; Winding, Michael; Lakonishok, Margot; Wildonger, Jill

2016-01-01

Cytoplasmic streaming in Drosophila oocytes is a microtubule-based bulk cytoplasmic movement. Streaming efficiently circulates and localizes mRNAs and proteins deposited by the nurse cells across the oocyte. This movement is driven by kinesin-1, a major microtubule motor. Recently, we have shown that kinesin-1 heavy chain (KHC) can transport one microtubule on another microtubule, thus driving microtubule–microtubule sliding in multiple cell types. To study the role of microtubule sliding in oocyte cytoplasmic streaming, we used a Khc mutant that is deficient in microtubule sliding but able to transport a majority of cargoes. We demonstrated that streaming is reduced by genomic replacement of wild-type Khc with this sliding-deficient mutant. Streaming can be fully rescued by wild-type KHC and partially rescued by a chimeric motor that cannot move organelles but is active in microtubule sliding. Consistent with these data, we identified two populations of microtubules in fast-streaming oocytes: a network of stable microtubules anchored to the actin cortex and free cytoplasmic microtubules that moved in the ooplasm. We further demonstrated that the reduced streaming in sliding-deficient oocytes resulted in posterior determination defects. Together, we propose that kinesin-1 slides free cytoplasmic microtubules against cortically immobilized microtubules, generating forces that contribute to cytoplasmic streaming and are essential for the refinement of posterior determinants. PMID:27512034

Stream fish colonization but not persistence varies regionally across a large North American river basin

Science.gov (United States)

Wheeler, Kit; Wengerd, Seth J.; Walsh, Stephen J.; Martin, Zachary P.; Jelks, Howard L.; Freeman, Mary C.

2018-01-01

Many species have distributions that span distinctly different physiographic regions, and effective conservation of such taxa will require a full accounting of all factors that potentially influence populations. Ecologists recognize effects of physiographic differences in topography, geology and climate on local habitat configurations, and thus the relevance of landscape heterogeneity to species distributions and abundances. However, research is lacking that examines how physiography affects the processes underlying metapopulation dynamics. We used data describing occupancy dynamics of stream fishes to evaluate evidence that physiography influences rates at which individual taxa persist in or colonize stream reaches under different flow conditions. Using periodic survey data from a stream fish assemblage in a large river basin that encompasses multiple physiographic regions, we fit multi-species dynamic occupancy models. Our modeling results suggested that stream fish colonization but not persistence was strongly governed by physiography, with estimated colonization rates considerably higher in Coastal Plain streams than in Piedmont and Blue Ridge systems. Like colonization, persistence was positively related to an index of stream flow magnitude, but the relationship between flow and persistence did not depend on physiography. Understanding the relative importance of colonization and persistence, and how one or both processes may change across the landscape, is critical information for the conservation of broadly distributed taxa, and conservation strategies explicitly accounting for spatial variation in these processes are likely to be more successful for such taxa.

Fish populations in Plynlimon streams

Directory of Open Access Journals (Sweden)

D. T. Crisp

1997-01-01

Full Text Available In Plynlimon streams, brown trout (Salmo trutta L. are widespread in the upper Wye at population densities of 0.03 to 0.32 fish m-2 and show evidence of successful recruitment in most years. In the upper Severn, brown trout are found only in an area of c. 1670 -2 downstream of Blaenhafren Falls at densities of 0.03 to 0.24 fish -2 and the evidence suggests very variable year to year success in recruitment (Crisp & Beaumont, 1996. Analyses of the data show that temperature differences between afforested and unafforested streams may affect the rates of trout incubation and growth but are not likely to influence species survival. Simple analyses of stream discharge data suggest, but do not prove, that good years for recruitment in the Hafren population were years of low stream discharge. This may be linked to groundwater inputs detected in other studies in this stream. More research is needed to explain the survival of the apparently isolated trout population in the Hafren.

Stream Deniable-Encryption Algorithms

Directory of Open Access Journals (Sweden)

N.A. Moldovyan

2016-04-01

Full Text Available A method for stream deniable encryption of secret message is proposed, which is computationally indistinguishable from the probabilistic encryption of some fake message. The method uses generation of two key streams with some secure block cipher. One of the key streams is generated depending on the secret key and the other one is generated depending on the fake key. The key streams are mixed with the secret and fake data streams so that the output ciphertext looks like the ciphertext produced by some probabilistic encryption algorithm applied to the fake message, while using the fake key. When the receiver or/and sender of the ciphertext are coerced to open the encryption key and the source message, they open the fake key and the fake message. To disclose their lie the coercer should demonstrate possibility of the alternative decryption of the ciphertext, however this is a computationally hard problem.

Performance of the air2stream model that relates air and stream water temperatures depends on the calibration method

Science.gov (United States)

Piotrowski, Adam P.; Napiorkowski, Jaroslaw J.

2018-06-01

A number of physical or data-driven models have been proposed to evaluate stream water temperatures based on hydrological and meteorological observations. However, physical models require a large amount of information that is frequently unavailable, while data-based models ignore the physical processes. Recently the air2stream model has been proposed as an intermediate alternative that is based on physical heat budget processes, but it is so simplified that the model may be applied like data-driven ones. However, the price for simplicity is the need to calibrate eight parameters that, although have some physical meaning, cannot be measured or evaluated a priori. As a result, applicability and performance of the air2stream model for a particular stream relies on the efficiency of the calibration method. The original air2stream model uses an inefficient 20-year old approach called Particle Swarm Optimization with inertia weight. This study aims at finding an effective and robust calibration method for the air2stream model. Twelve different optimization algorithms are examined on six different streams from northern USA (states of Washington, Oregon and New York), Poland and Switzerland, located in both high mountains, hilly and lowland areas. It is found that the performance of the air2stream model depends significantly on the calibration method. Two algorithms lead to the best results for each considered stream. The air2stream model, calibrated with the chosen optimization methods, performs favorably against classical streamwater temperature models. The MATLAB code of the air2stream model and the chosen calibration procedure (CoBiDE) are available as Supplementary Material on the Journal of Hydrology web page.

Workflow-Oriented Cyberinfrastructure for Sensor Data Analytics

Science.gov (United States)

Orcutt, J. A.; Rajasekar, A.; Moore, R. W.; Vernon, F.

2015-12-01

Sensor streams comprise an increasingly large part of Earth Science data. Analytics based on sensor data require an easy way to perform operations such as acquisition, conversion to physical units, metadata linking, sensor fusion, analysis and visualization on distributed sensor streams. Furthermore, embedding real-time sensor data into scientific workflows is of growing interest. We have implemented a scalable networked architecture that can be used to dynamically access packets of data in a stream from multiple sensors, and perform synthesis and analysis across a distributed network. Our system is based on the integrated Rule Oriented Data System (irods.org), which accesses sensor data from the Antelope Real Time Data System (brtt.com), and provides virtualized access to collections of data streams. We integrate real-time data streaming from different sources, collected for different purposes, on different time and spatial scales, and sensed by different methods. iRODS, noted for its policy-oriented data management, brings to sensor processing features and facilities such as single sign-on, third party access control lists ( ACLs), location transparency, logical resource naming, and server-side modeling capabilities while reducing the burden on sensor network operators. Rich integrated metadata support also makes it straightforward to discover data streams of interest and maintain data provenance. The workflow support in iRODS readily integrates sensor processing into any analytical pipeline. The system is developed as part of the NSF-funded Datanet Federation Consortium (datafed.org). APIs for selecting, opening, reaping and closing sensor streams are provided, along with other helper functions to associate metadata and convert sensor packets into NetCDF and JSON formats. Near real-time sensor data including seismic sensors, environmental sensors, LIDAR and video streams are available through this interface. A system for archiving sensor data and metadata in Net

Quality of streams in Johnson County, Kansas, 2002--10

Science.gov (United States)

Rasmussen, Teresa J.; Stone, Mandy S.; Poulton, Barry C.; Graham, Jennifer L.

2012-01-01

Stream quality in Johnson County, northeastern Kansas, was assessed on the basis of land use, hydrology, stream-water and streambed-sediment chemistry, riparian and in-stream habitat, and periphyton and macroinvertebrate community data collected from 22 sites during 2002 through 2010. Stream conditions at the end of the study period are evaluated and compared to previous years, stream biological communities and physical and chemical conditions are characterized, streams are described relative to Kansas Department of Health and Environment impairment categories and water-quality standards, and environmental factors that most strongly correlate with biological stream quality are evaluated. The information is useful for improving water-quality management programs, documenting changing conditions with time, and evaluating compliance with water-quality standards, total maximum daily loads (TMDLs), National Pollutant Discharge Elimination System (NPDES) permit conditions, and other established guidelines and goals. Constituent concentrations in water during base flow varied across the study area and 2010 conditions were not markedly different from those measured in 2003, 2004, and 2007. Generally the highest specific conductance and concentrations of dissolved solids and major ions in water occurred at urban sites except the upstream Cedar Creek site, which is rural and has a large area of commercial and industrial land less than 1 mile upstream on both sides of the creek. The highest base-flow nutrient concentrations in water occurred downstream from wastewater treatment facilities. Water chemistry data represent base-flow conditions only, and do not show the variability in concentrations that occurs during stormwater runoff. Constituent concentrations in streambed sediment also varied across the study area and some notable changes occurred from previously collected data. High organic carbon and nutrient concentrations at the rural Big Bull Creek site in 2003 decreased

A Critical Examination of the Introduction of Drug Detection Dogs for Policing of Illicit Drugs in New South Wales, Australia Using Kingdon's "Multiple Streams" Heuristic

Science.gov (United States)

Lancaster, Kari; Ritter, Alison; Hughes, Caitlin; Hoppe, Robert

2017-01-01

This paper critically analyses the introduction of drug detection dogs as a tool for policing of illicit drugs in New South Wales, Australia. Using Kingdon's "multiple streams" heuristic as a lens for analysis, we identify how the issue of drugs policing became prominent on the policy agenda, and the conditions under which the…

Stream Evaluation

Data.gov (United States)

Kansas Data Access and Support Center — Digital representation of the map accompanying the "Kansas stream and river fishery resource evaluation" (R.E. Moss and K. Brunson, 1981.U.S. Fish and Wildlife...

Sampling, Splitting and Merging in Coinductive Stream Calculus

NARCIS (Netherlands)

M. Niqui (Milad); J.J.M.M. Rutten (Jan); C. Bolduc; J. Desharnais; B. Ktari

2010-01-01

textabstractWe study various operations for partitioning, projecting and merging streams of data. These operations are motivated by their use in dataflow programming and the stream processing languages. We use the framework of \\emph{stream calculus} and \\emph{stream circuits} for defining and

Sampling, splitting and merging in coinductive stream calculus

NARCIS (Netherlands)

M. Niqui (Milad); J.J.M.M. Rutten (Jan)

2009-01-01

htmlabstractWe study various operations for partitioning, projecting and merging streams of data. These operations are motivated by their use in dataflow programming and the stream processing languages. We use the framework of stream calculus and stream circuits for defining and proving properties

SPEAR indicates pesticide effects in streams - Comparative use of species- and family-level biomonitoring data

International Nuclear Information System (INIS)

Beketov, M.A.; Foit, K.; Schaefer, R.B.; Schriever, C.A.; Sacchi, A.; Capri, E.; Biggs, J.; Wells, C.; Liess, M.

2009-01-01

To detect effects of pesticides on non-target freshwater organisms the Species at risk (SPEAR pesticides ) bioindicator based on biological traits was previously developed and successfully validated over different biogeographical regions of Europe using species-level data on stream invertebrates. Since many freshwater biomonitoring programmes have family-level taxonomic resolution we tested the applicability of SPEAR pesticides with family-level biomonitoring data to indicate pesticide effects in streams (i.e. insecticide toxicity of pesticides). The study showed that the explanatory power of the family-level SPEAR(fm) pesticides is not significantly lower than the species-level index. The results suggest that the family-level SPEAR(fm) pesticides is a sensitive, cost-effective, and potentially European-wide bioindicator of pesticide contamination in flowing waters. Class boundaries for SPEAR pesticides according to EU Water Framework Directive are defined to contribute to the assessment of ecological status of water bodies. - We show that SPEAR pesticides can be based on family-level biomonitoring data and is applicable for large-scale monitoring programmes to detect and quantify pesticide contamination.

SPEAR indicates pesticide effects in streams - Comparative use of species- and family-level biomonitoring data

Energy Technology Data Exchange (ETDEWEB)

Beketov, M.A., E-mail: mikhail.beketov@ufz.d [UFZ - Helmholtz Centre for Environmental Research, Department System Ecotoxicology, Permoserstrasse 15, D-04318 Leipzig (Germany); Foit, K.; Schaefer, R.B.; Schriever, C.A. [UFZ - Helmholtz Centre for Environmental Research, Department System Ecotoxicology, Permoserstrasse 15, D-04318 Leipzig (Germany); Sacchi, A.; Capri, E. [Universita Cattolica del Sacro Cuore, Istituto di Chimica Agraria ed Ambientale, Piacenza (Italy); Biggs, J. [Pond Conservation, c/o Oxford Brookes University, Headington (United Kingdom); Wells, C. [Environment Agency of England and Wales, Science Department, Bristol (United Kingdom); Liess, M. [UFZ - Helmholtz Centre for Environmental Research, Department System Ecotoxicology, Permoserstrasse 15, D-04318 Leipzig (Germany)

2009-06-15

To detect effects of pesticides on non-target freshwater organisms the Species at risk (SPEAR{sub pesticides}) bioindicator based on biological traits was previously developed and successfully validated over different biogeographical regions of Europe using species-level data on stream invertebrates. Since many freshwater biomonitoring programmes have family-level taxonomic resolution we tested the applicability of SPEAR{sub pesticides} with family-level biomonitoring data to indicate pesticide effects in streams (i.e. insecticide toxicity of pesticides). The study showed that the explanatory power of the family-level SPEAR(fm){sub pesticides} is not significantly lower than the species-level index. The results suggest that the family-level SPEAR(fm){sub pesticides} is a sensitive, cost-effective, and potentially European-wide bioindicator of pesticide contamination in flowing waters. Class boundaries for SPEAR{sub pesticides} according to EU Water Framework Directive are defined to contribute to the assessment of ecological status of water bodies. - We show that SPEAR{sub pesticides} can be based on family-level biomonitoring data and is applicable for large-scale monitoring programmes to detect and quantify pesticide contamination.

Comprehensive Protection of Data-Partitioned Video for Broadband Wireless IPTV Streaming

Directory of Open Access Journals (Sweden)

Laith Al-Jobouri

2012-01-01

Full Text Available This paper examines the threat to video streaming from slow and fast fading, traffic congestion, and channel packet drops. The proposed response is a combination of: rateless channel coding, which is adaptively applied; data-partitioned source coding to exploit prioritized packetization; and duplicate slice provision, which is the focus of the evaluation in this paper. The paper also considers the distribution of intra-refresh macroblocks as a means of avoiding sudden data rate increases. When error bursts occur, this paper shows that duplicate slices are certainly necessary but this provision is more effective for medium quality video than it is for high quality video. The percentage of intra-refresh macroblocks can be low and still reduce the impact of temporal error propagation.

Comparison of animated jet stream visualizations

Science.gov (United States)

Nocke, Thomas; Hoffmann, Peter

2016-04-01

The visualization of 3D atmospheric phenomena in space and time is still a challenging problem. In particular, multiple solutions of animated jet stream visualizations have been produced in recent years, which were designed to visually analyze and communicate the jet and related impacts on weather circulation patterns and extreme weather events. This PICO integrates popular and new jet animation solutions and inter-compares them. The applied techniques (e.g. stream lines or line integral convolution) and parametrizations (color mapping, line lengths) are discussed with respect to visualization quality criteria and their suitability for certain visualization tasks (e.g. jet patterns and jet anomaly analysis, communicating its relevance for climate change).

Quantifying Forested Riparian Buffer Ability to Ameliorate Stream Temperature in a Missouri Ozark Border Stream of the Central U.S

Science.gov (United States)

Bulliner, E. A.; Hubbart, J. A.

2009-12-01

Riparian buffers play an important role in modulating stream water quality, including temperature. There is a need to better understand riparian form and function to validate and improve contemporary management practices. Further studies are warranted to characterize energy attenuation by forested riparian canopy layers that normally buffer stream temperature, particularly in the central hardwood forest regions of the United States where relationships between canopy density and stream temperature are unknown. To quantify these complex processes, two intensively instrumented hydroclimate stations were installed along two stream reaches of a riparian stream in central Missouri, USA in the winter of 2008. Hydroclimate stations are located along stream reaches oriented in both cardinal directions, which will allow interpolation of results to other orientations. Each station consists of an array of instrumentation that senses the flux of water and energy into and out of the riparian zone. Reference data are supplied from a nearby flux tower (US DOE) located on top of a forested ridge. The study sites are located within a University of Missouri preserved wildland area on the border of the southern Missouri’s Ozark region, an ecologically distinct region in the central United States. Limestone underlies the study area, resulting in a distinct semi-Karst hydrologic system. Vegetation forms a complex, multi-layered canopy extending from the stream edge through the riparian zone and into surrounding hills. Climate is classified as humid continental, with approximate average annual temperature and precipitation of 13.2°C and 970mm, respectively. Preliminary results (summer 2009 data) indicate incoming short-wave radiation is 24.9% higher at the N-S oriented stream reach relative to the E-W oriented reach. Maximum incoming short wave radiation during the period was 64.5% lower at the N-S reach relative to E-W reach. Average air temperature for the E-W reach was 0.3°C lower

Data matching for free-surface multiple attenuation by multidimensional deconvolution

Science.gov (United States)

van der Neut, Joost; Frijlink, Martijn; van Borselen, Roald

2012-09-01

A common strategy for surface-related multiple elimination of seismic data is to predict multiples by a convolutional model and subtract these adaptively from the input gathers. Problems can be posed by interfering multiples and primaries. Removing multiples by multidimensional deconvolution (MDD) (inversion) does not suffer from these problems. However, this approach requires data to be consistent, which is often not the case, especially not at interpolated near-offsets. A novel method is proposed to improve data consistency prior to inversion. This is done by backpropagating first-order multiples with a time-gated reference primary event and matching these with early primaries in the input gather. After data matching, multiple elimination by MDD can be applied with a deterministic inversion scheme.

The study of Kruskal's and Prim's algorithms on the Multiple Instruction and Single Data stream computer system

Directory of Open Access Journals (Sweden)

A. Yu. Popov

2015-01-01

Full Text Available Bauman Moscow State Technical University is implementing a project to develop operating principles of computer system having radically new architecture. A developed working model of the system allowed us to evaluate an efficiency of developed hardware and software. The experimental results presented in previous studies, as well as the analysis of operating principles of new computer system permit to draw conclusions regarding its efficiency in solving discrete optimization problems related to processing of sets.The new architecture is based on a direct hardware support of operations of discrete mathematics, which is reflected in using the special facilities for processing of sets and data structures. Within the framework of the project a special device was designed, i.e. a structure processor (SP, which improved the performance, without limiting the scope of applications of such a computer system.The previous works presented the basic principles of the computational process organization in MISD (Multiple Instructions, Single Data system, showed the structure and features of the structure processor and the general principles to solve discrete optimization problems on graphs.This paper examines two search algorithms of the minimum spanning tree, namely Kruskal's and Prim's algorithms. It studies the implementations of algorithms for two SP operation modes: coprocessor mode and MISD one. The paper presents results of experimental comparison of MISD system performance in coprocessor mode with mainframes.

ACTION-SPACE CLUSTERING OF TIDAL STREAMS TO INFER THE GALACTIC POTENTIAL

Energy Technology Data Exchange (ETDEWEB)

Sanderson, Robyn E.; Helmi, Amina [Kapteyn Astronomical Institute, P.O. Box 800, 9700 AV Groningen (Netherlands); Hogg, David W., E-mail: robyn@astro.columbia.edu [Center for Cosmology and Particle Physics, Department of Physics, New York University, 4 Washington Place, New York, NY 10003 (United States)

2015-03-10

We present a new method for constraining the Milky Way halo gravitational potential by simultaneously fitting multiple tidal streams. This method requires three-dimensional positions and velocities for all stars to be fit, but does not require identification of any specific stream or determination of stream membership for any star. We exploit the principle that the action distribution of stream stars is most clustered when the potential used to calculate the actions is closest to the true potential. Clustering is quantified with the Kullback-Leibler Divergence (KLD), which also provides conditional uncertainties for our parameter estimates. We show, for toy Gaia-like data in a spherical isochrone potential, that maximizing the KLD of the action distribution relative to a smoother distribution recovers the input potential. The precision depends on the observational errors and number of streams; using K III giants as tracers, we measure the enclosed mass at the average radius of the sample stars accurate to 3% and precise to 20%-40%. Recovery of the scale radius is precise to 25%, biased 50% high by the small galactocentric distance range of stars in our mock sample (1-25 kpc, or about three scale radii, with mean 6.5 kpc). 20-25 streams with at least 100 stars each are required for a stable confidence interval. With radial velocities (RVs) to 100 kpc, all parameters are determined with ∼10% accuracy and 20% precision (1.3% accuracy for the enclosed mass), underlining the need to complete the RV catalog for faint halo stars observed by Gaia.

Hydrogeochemical and stream sediment reconnaissance basic data for Dodge City NTMS Quadrangle, Kansas

International Nuclear Information System (INIS)

1980-01-01

Results of a reconnaissance geochemical survey of the Dodge City Quadrangle are reported. Field and laboratory data are presented for 756 groundwater and 321 stream sediment samples. Statistical and areal distributions of uranium and possible uranium-related variables are displayed. A generalized geologic map of the survey area is provided, and pertinent geologic factors which may be of significance in evaluating the potential for uranium mineralization are briefly discussed. Groundwater data indicate that the most promising areas for uranium mineralization are as follows: (1) in the north central area of the quadrangle within close proximity to the Arkansas River, mostly from waters of the Ogallala Formation; (2) in the west central area, from groundwater samples of the Dakota and the Ogallala Formations; and (3) between the North Fork of the Cimarron River and the main Cimarron River, mostly in waters from the Ogallala Formation. Associated with the high uranium values are high concentrations for magnesium, strontium, and sulfate. Of the groundwater samples taken 81% were collected from the Ogallala Formation. Stream sediment data indicate high uranium concentrations in scattered samples in the northwestern, central, and southwestern areas of the quadrangle. Most of the samples with high uranium values were collected from the Quaternary alluvium. Associated with the high uranium values are high concentrations of barium, cerium, iron, manganese, titanium, vanadium, yttrium, and zirconium

Modeling transient streaming potentials in falling-head permeameter tests.

Science.gov (United States)

Malama, Bwalya; Revil, André

2014-01-01

We present transient streaming potential data collected during falling-head permeameter tests performed on samples of two sands with different physical and chemical properties. The objective of the work is to estimate hydraulic conductivity (K) and the electrokinetic coupling coefficient (Cl ) of the sand samples. A semi-empirical model based on the falling-head permeameter flow model and electrokinetic coupling is used to analyze the streaming potential data and to estimate K and Cl . The values of K estimated from head data are used to validate the streaming potential method. Estimates of K from streaming potential data closely match those obtained from the associated head data, with less than 10% deviation. The electrokinetic coupling coefficient was estimated from streaming potential vs. (1) time and (2) head data for both sands. The results indicate that, within limits of experimental error, the values of Cl estimated by the two methods are essentially the same. The results of this work demonstrate that a temporal record of the streaming potential response in falling-head permeameter tests can be used to estimate both K and Cl . They further indicate the potential for using transient streaming potential data as a proxy for hydraulic head in hydrogeology applications. © 2013, National Ground Water Association.