Prostate multimodality image registration based on B-splines and quadrature local energy.

Science.gov (United States)

Mitra, Jhimli; Martí, Robert; Oliver, Arnau; Lladó, Xavier; Ghose, Soumya; Vilanova, Joan C; Meriaudeau, Fabrice

2012-05-01

Needle biopsy of the prostate is guided by Transrectal Ultrasound (TRUS) imaging. The TRUS images do not provide proper spatial localization of malignant tissues due to the poor sensitivity of TRUS to visualize early malignancy. Magnetic Resonance Imaging (MRI) has been shown to be sensitive for the detection of early stage malignancy, and therefore, a novel 2D deformable registration method that overlays pre-biopsy MRI onto TRUS images has been proposed. The registration method involves B-spline deformations with Normalized Mutual Information (NMI) as the similarity measure computed from the texture images obtained from the amplitude responses of the directional quadrature filter pairs. Registration accuracy of the proposed method is evaluated by computing the Dice Similarity coefficient (DSC) and 95% Hausdorff Distance (HD) values for 20 patients prostate mid-gland slices and Target Registration Error (TRE) for 18 patients only where homologous structures are visible in both the TRUS and transformed MR images. The proposed method and B-splines using NMI computed from intensities provide average TRE values of 2.64 ± 1.37 and 4.43 ± 2.77 mm respectively. Our method shows statistically significant improvement in TRE when compared with B-spline using NMI computed from intensities with Student's t test p = 0.02. The proposed method shows 1.18 times improvement over thin-plate splines registration with average TRE of 3.11 ± 2.18 mm. The mean DSC and the mean 95% HD values obtained with the proposed method of B-spline with NMI computed from texture are 0.943 ± 0.039 and 4.75 ± 2.40 mm respectively. The texture energy computed from the quadrature filter pairs provides better registration accuracy for multimodal images than raw intensities. Low TRE values of the proposed registration method add to the feasibility of it being used during TRUS-guided biopsy.

Multimodal Registration of gated cardiac PET, CT and MR sequences

International Nuclear Information System (INIS)

Baty, X.

2007-07-01

The research described in this manuscript deals with the multimodal registration of cardiac images from Magnetic Resonance Imaging (MRI), Position Emission Tomography (PET) and Computerized Tomography (CT). All these modalities are gated to the Electrocardiogram (ECG) and provide information to evaluate cardiac function, and to diagnose and to follow-up cardiovascular pathologies. PET imaging allows the evaluation of ventricular function and MRI is a gold standard for the study of the left ventricular function. The goal of our registration process is to merge functional (from PET) and anatomical images (from CT and MRI). Our process is adapted to the modalities used and is divided in two steps: (i) a global rigid 3-dimensional model-based ICP (Iterative Closest Point) registration between CT and MR data and (ii) an iconic 2-dimensional registration based on Free Form Deformations and Mutual Information. This last step presents an original contribution by using a composite image of CT (which presents epicardic contours) and PET (where endocardic contours are partially visible) data to make mutual information more accurate in representing the similarity with the MR data. To speed up the whole process, we also present a transformation initialization scheme using displacement field obtained form MR data only. The obtained results have been evaluated by experts. (author)

Image fusion between whole body FDG PET images and whole body MRI images using a full-automatic mutual information-based multimodality image registration software

International Nuclear Information System (INIS)

Uchida, Yoshitaka; Nakano, Yoshitada; Fujibuchi, Toshiou; Isobe, Tomoko; Kazama, Toshiki; Ito, Hisao

2006-01-01

We attempted image fusion between whole body PET and whole body MRI of thirty patients using a full-automatic mutual information (MI) -based multimodality image registration software and evaluated accuracy of this method and impact of the coregistrated imaging on diagnostic accuracy. For 25 of 30 fused images in body area, translating gaps were within 6 mm in all axes and rotating gaps were within 2 degrees around all axes. In head and neck area, considerably much gaps caused by difference of head inclination at imaging occurred in 16 patients, however these gaps were able to decrease by fused separately. In 6 patients, diagnostic accuracy using PET/MRI fused images was superior compared by PET image alone. This work shows that whole body FDG PET images and whole body MRI images can be automatically fused using MI-based multimodality image registration software accurately and this technique can add useful information when evaluating FDG PET images. (author)

Multi-Modality Registration And Fusion Of Medical Image Data

International Nuclear Information System (INIS)

Kassak, P.; Vencko, D.; Cerovsky, I.

2008-01-01

Digitalisation of health care providing facilities allows US to maximize the usage of digital data from one patient obtained by various modalities. Complex view on to the problem can be achieved from the site of morphology as well as functionality. Multi-modal registration and fusion of medical image data is one of the examples that provides improved insight and allows more precise approach and treatment. (author)

Improving efficiency of multi-modality registration of brain scans based on mutual information

International Nuclear Information System (INIS)

Thurfjell, L.; Lau, Y.; Hutton, B.; Westmead Hospital, Sydney, NSW; University of Technology, Sydney, NSW

1999-01-01

Full text: One approach for multi-modality registration uses a similarity measure based on mutual information (MI) of voxel intensities. MI measures the statistical dependence between two images by comparing the joint probability distribution (approximated by the 2D joint histogram), with the distribution in the case of complete independence (approximated from the I D histograms). The MI measure reaches a maximum when the images are aligned. The purpose of the current work was to investigate if the registration process could be accelerated through subsampling, i.e. by using only a subset of all voxels for the calculations. The behaviour of the MI measure at different subsampling factors was studied. It was observed that subsampling caused MI to exhibit multiple local maxima unless it was accompanied by a reduction in the number of bins used for the histograms. However, too few bins in the histograms made the peak of the MI measure broader. It was therefore concluded that a coarse-to-fine subsampling procedure, followed by a corresponding increase in the number of bins in the histogram, would be the best choice. The method was validated on SPET-MRI data from seven healthy volunteers. Using a 64:1, 32:1 and 16:1 subsampling scheme with a corresponding bin size of 24, 32 and 48, the new method converged in an average time of 2.5 min as compared to 46 min for the original method (PC Pentium 200). The average absolute differences were 0.24, 0.34, 0.30 mm translation and 0.58, 0.41, 0.66 degrees rotation. We conclude that the suggested scheme renders the registration method sufficiently rapid for routine use in the clinical setting

Surface-based prostate registration with biomechanical regularization

Science.gov (United States)

van de Ven, Wendy J. M.; Hu, Yipeng; Barentsz, Jelle O.; Karssemeijer, Nico; Barratt, Dean; Huisman, Henkjan J.

2013-03-01

Adding MR-derived information to standard transrectal ultrasound (TRUS) images for guiding prostate biopsy is of substantial clinical interest. A tumor visible on MR images can be projected on ultrasound by using MRUS registration. A common approach is to use surface-based registration. We hypothesize that biomechanical modeling will better control deformation inside the prostate than a regular surface-based registration method. We developed a novel method by extending a surface-based registration with finite element (FE) simulation to better predict internal deformation of the prostate. For each of six patients, a tetrahedral mesh was constructed from the manual prostate segmentation. Next, the internal prostate deformation was simulated using the derived radial surface displacement as boundary condition. The deformation field within the gland was calculated using the predicted FE node displacements and thin-plate spline interpolation. We tested our method on MR guided MR biopsy imaging data, as landmarks can easily be identified on MR images. For evaluation of the registration accuracy we used 45 anatomical landmarks located in all regions of the prostate. Our results show that the median target registration error of a surface-based registration with biomechanical regularization is 1.88 mm, which is significantly different from 2.61 mm without biomechanical regularization. We can conclude that biomechanical FE modeling has the potential to improve the accuracy of multimodal prostate registration when comparing it to regular surface-based registration.

Hierarchical patch-based co-registration of differently stained histopathology slides

Science.gov (United States)

Yigitsoy, Mehmet; Schmidt, Günter

2017-03-01

Over the past decades, digital pathology has emerged as an alternative way of looking at the tissue at subcellular level. It enables multiplexed analysis of different cell types at micron level. Information about cell types can be extracted by staining sections of a tissue block using different markers. However, robust fusion of structural and functional information from different stains is necessary for reproducible multiplexed analysis. Such a fusion can be obtained via image co-registration by establishing spatial correspondences between tissue sections. Spatial correspondences can then be used to transfer various statistics about cell types between sections. However, the multi-modal nature of images and sparse distribution of interesting cell types pose several challenges for the registration of differently stained tissue sections. In this work, we propose a co-registration framework that efficiently addresses such challenges. We present a hierarchical patch-based registration of intensity normalized tissue sections. Preliminary experiments demonstrate the potential of the proposed technique for the fusion of multi-modal information from differently stained digital histopathology sections.

A strategy for multimodal deformable image registration to integrate PET/MR into radiotherapy treatment planning

International Nuclear Information System (INIS)

Leibfarth, Sara; Moennich, David; Thorwarth, Daniela; Welz, Stefan; Siegel, Christine; Zips, Daniel; Schwenzer, Nina; Holger Schmidt, Holger

2013-01-01

Background: Combined positron emission tomography (PET)/magnetic resonance imaging (MRI) is highly promising for biologically individualized radiotherapy (RT). Hence, the purpose of this work was to develop an accurate and robust registration strategy to integrate combined PET/MR data into RT treatment planning. Material and methods: Eight patient datasets consisting of an FDG PET/computed tomography (CT) and a subsequently acquired PET/MR of the head and neck (HN) region were available. Registration strategies were developed based on CT and MR data only, whereas the PET components were fused with the resulting deformation field. Following a rigid registration, deformable registration was performed with a transform parametrized by B-splines. Three different optimization metrics were investigated: global mutual information (GMI), GMI combined with a bending energy penalty (BEP) for regularization (GMI + BEP) and localized mutual information with BEP (LMI + BEP). Different quantitative registration quality measures were developed, including volumetric overlap and mean distance measures for structures segmented on CT and MR as well as anatomical landmark distances. Moreover, the local registration quality in the tumor region was assessed by the normalized cross correlation (NCC) of the two PET datasets. Results: LMI + BEP yielded the most robust and accurate registration results. For GMI, GMI + BEP and LMI + BEP, mean landmark distances (standard deviations) were 23.9 mm (15.5 mm), 4.8 mm (4.0 mm) and 3.0 mm (1.0 mm), and mean NCC values (standard deviations) were 0.29 (0.29), 0.84 (0.14) and 0.88 (0.06), respectively. Conclusion: Accurate and robust multimodal deformable image registration of CT and MR in the HN region can be performed using a B-spline parametrized transform and LMI + BEP as optimization metric. With this strategy, biologically individualized RT based on combined PET/MRI in terms of dose painting is possible

Registration for Optical Multimodal Remote Sensing Images Based on FAST Detection, Window Selection, and Histogram Specification

Directory of Open Access Journals (Sweden)

Xiaoyang Zhao

2018-04-01

Full Text Available In recent years, digital frame cameras have been increasingly used for remote sensing applications. However, it is always a challenge to align or register images captured with different cameras or different imaging sensor units. In this research, a novel registration method was proposed. Coarse registration was first applied to approximately align the sensed and reference images. Window selection was then used to reduce the search space and a histogram specification was applied to optimize the grayscale similarity between the images. After comparisons with other commonly-used detectors, the fast corner detector, FAST (Features from Accelerated Segment Test, was selected to extract the feature points. The matching point pairs were then detected between the images, the outliers were eliminated, and geometric transformation was performed. The appropriate window size was searched and set to one-tenth of the image width. The images that were acquired by a two-camera system, a camera with five imaging sensors, and a camera with replaceable filters mounted on a manned aircraft, an unmanned aerial vehicle, and a ground-based platform, respectively, were used to evaluate the performance of the proposed method. The image analysis results showed that, through the appropriate window selection and histogram specification, the number of correctly matched point pairs had increased by 11.30 times, and that the correct matching rate had increased by 36%, compared with the results based on FAST alone. The root mean square error (RMSE in the x and y directions was generally within 0.5 pixels. In comparison with the binary robust invariant scalable keypoints (BRISK, curvature scale space (CSS, Harris, speed up robust features (SURF, and commercial software ERDAS and ENVI, this method resulted in larger numbers of correct matching pairs and smaller, more consistent RMSE. Furthermore, it was not necessary to choose any tie control points manually before registration

Multimodal registration of three-dimensional maxillodental cone beam CT and photogrammetry data over time.

Science.gov (United States)

Bolandzadeh, N; Bischof, W; Flores-Mir, C; Boulanger, P

2013-01-01

In recent years, one of the foci of orthodontics has been on systems for the evaluation of treatment results and the tracking of tissue variations over time. This can be accomplished through analysing three-dimensional orthodontic images obtained before and after the treatments. Since complementary information is achieved by integrating multiple imaging modalities, cone beam CT (CBCT) and stereophotogrammetry technologies are used in this study to develop a method for tracking bone, teeth and facial soft-tissue variations over time. We propose a two-phase procedure of multimodal (Phase 1) and multitemporal (Phase 2) registration which aligns images taken from the same patient by different imaging modalities and at different times. Extrinsic (for Phase 1) and intrinsic (for Phase 2) landmark-based registration methods are employed as an initiation for a robust iterative closest points algorithm. Since the mandible moves independently of the upper skull, the registration procedure is applied separately on the mandible and the upper skull. The results show that the signed error distributions of both mandible and skull registrations follow a mixture of two Gaussian distributions, corresponding to alignment errors (due to our method) and temporal change over time. We suggest that the large values among the total registration errors correspond to the temporal change resulting from (1) the effect of treatment (i.e. the orthodontic changes of teeth positions); (2) the biological changes such as teeth growth over time, especially for teenagers; and (3) the segmentation procedure and CBCT precision change over time.

An embedded system for image segmentation and multimodal registration in noninvasive skin cancer screening.

Science.gov (United States)

Diaz, Silvana; Soto, Javier E; Inostroza, Fabian; Godoy, Sebastian E; Figueroa, Miguel

2017-07-01

We present a heterogeneous architecture for image registration and multimodal segmentation on an embedded system for noninvasive skin cancer screening. The architecture combines Otsu thresholding and the random walker algorithm to perform image segmentation, and features a hardware implementation of the Harris corner detection algorithm to perform region-of-interest detection and image registration. Running on a Xilinx XC7Z020 reconfigurable system-on-a-chip, our prototype computes the initial segmentation of a 400×400-pixel region of interest in the visible spectrum in 12.1 seconds, and registers infrared images against this region at 540 frames per second, while consuming 1.9W.

Multiscale registration of medical images based on edge preserving scale space with application in image-guided radiation therapy

Science.gov (United States)

Li, Dengwang; Li, Hongsheng; Wan, Honglin; Chen, Jinhu; Gong, Guanzhong; Wang, Hongjun; Wang, Liming; Yin, Yong

2012-08-01

Mutual information (MI) is a well-accepted similarity measure for image registration in medical systems. However, MI-based registration faces the challenges of high computational complexity and a high likelihood of being trapped into local optima due to an absence of spatial information. In order to solve these problems, multi-scale frameworks can be used to accelerate registration and improve robustness. Traditional Gaussian pyramid representation is one such technique but it suffers from contour diffusion at coarse levels which may lead to unsatisfactory registration results. In this work, a new multi-scale registration framework called edge preserving multiscale registration (EPMR) was proposed based upon an edge preserving total variation L1 norm (TV-L1) scale space representation. TV-L1 scale space is constructed by selecting edges and contours of images according to their size rather than the intensity values of the image features. This ensures more meaningful spatial information with an EPMR framework for MI-based registration. Furthermore, we design an optimal estimation of the TV-L1 parameter in the EPMR framework by training and minimizing the transformation offset between the registered pairs for automated registration in medical systems. We validated our EPMR method on both simulated mono- and multi-modal medical datasets with ground truth and clinical studies from a combined positron emission tomography/computed tomography (PET/CT) scanner. We compared our registration framework with other traditional registration approaches. Our experimental results demonstrated that our method outperformed other methods in terms of the accuracy and robustness for medical images. EPMR can always achieve a small offset value, which is closer to the ground truth both for mono-modality and multi-modality, and the speed can be increased 5-8% for mono-modality and 10-14% for multi-modality registration under the same condition. Furthermore, clinical application by adaptive

Multiscale registration of medical images based on edge preserving scale space with application in image-guided radiation therapy

International Nuclear Information System (INIS)

Li Dengwang; Wan Honglin; Li Hongsheng; Chen Jinhu; Gong Guanzhong; Yin Yong; Wang Hongjun; Wang Liming

2012-01-01

Mutual information (MI) is a well-accepted similarity measure for image registration in medical systems. However, MI-based registration faces the challenges of high computational complexity and a high likelihood of being trapped into local optima due to an absence of spatial information. In order to solve these problems, multi-scale frameworks can be used to accelerate registration and improve robustness. Traditional Gaussian pyramid representation is one such technique but it suffers from contour diffusion at coarse levels which may lead to unsatisfactory registration results. In this work, a new multi-scale registration framework called edge preserving multiscale registration (EPMR) was proposed based upon an edge preserving total variation L1 norm (TV-L1) scale space representation. TV-L1 scale space is constructed by selecting edges and contours of images according to their size rather than the intensity values of the image features. This ensures more meaningful spatial information with an EPMR framework for MI-based registration. Furthermore, we design an optimal estimation of the TV-L1 parameter in the EPMR framework by training and minimizing the transformation offset between the registered pairs for automated registration in medical systems. We validated our EPMR method on both simulated mono- and multi-modal medical datasets with ground truth and clinical studies from a combined positron emission tomography/computed tomography (PET/CT) scanner. We compared our registration framework with other traditional registration approaches. Our experimental results demonstrated that our method outperformed other methods in terms of the accuracy and robustness for medical images. EPMR can always achieve a small offset value, which is closer to the ground truth both for mono-modality and multi-modality, and the speed can be increased 5–8% for mono-modality and 10–14% for multi-modality registration under the same condition. Furthermore, clinical application by

Automated Registration of Multimodal Optic Disc Images: Clinical Assessment of Alignment Accuracy.

Science.gov (United States)

Ng, Wai Siene; Legg, Phil; Avadhanam, Venkat; Aye, Kyaw; Evans, Steffan H P; North, Rachel V; Marshall, Andrew D; Rosin, Paul; Morgan, James E

2016-04-01

To determine the accuracy of automated alignment algorithms for the registration of optic disc images obtained by 2 different modalities: fundus photography and scanning laser tomography. Images obtained with the Heidelberg Retina Tomograph II and paired photographic optic disc images of 135 eyes were analyzed. Three state-of-the-art automated registration techniques Regional Mutual Information, rigid Feature Neighbourhood Mutual Information (FNMI), and nonrigid FNMI (NRFNMI) were used to align these image pairs. Alignment of each composite picture was assessed on a 5-point grading scale: "Fail" (no alignment of vessels with no vessel contact), "Weak" (vessels have slight contact), "Good" (vessels with 50% contact), and "Excellent" (complete alignment). Custom software generated an image mosaic in which the modalities were interleaved as a series of alternate 5×5-pixel blocks. These were graded independently by 3 clinically experienced observers. A total of 810 image pairs were assessed. All 3 registration techniques achieved a score of "Good" or better in >95% of the image sets. NRFNMI had the highest percentage of "Excellent" (mean: 99.6%; range, 95.2% to 99.6%), followed by Regional Mutual Information (mean: 81.6%; range, 86.3% to 78.5%) and FNMI (mean: 73.1%; range, 85.2% to 54.4%). Automated registration of optic disc images by different modalities is a feasible option for clinical application. All 3 methods provided useful levels of alignment, but the NRFNMI technique consistently outperformed the others and is recommended as a practical approach to the automated registration of multimodal disc images.

Evaluation of whole-body MR to CT deformable image registration

NARCIS (Netherlands)

Akbarzadeh, A.; Gutierrez, D.; Baskin, A.; Ay, M. R.; Ahmadian, A.; Alam, N. Riahi; Loevblad, K. O.; Zaidi, H.

2013-01-01

Multimodality image registration plays a crucial role in various clinical and research applications. The aim of this study is to present an optimized MR to CT whole-body deformable image registration algorithm and its validation using clinical studies. A 3D intermodality registration technique based

A Robust and Accurate Two-Step Auto-Labeling Conditional Iterative Closest Points (TACICP Algorithm for Three-Dimensional Multi-Modal Carotid Image Registration.

Directory of Open Access Journals (Sweden)

Hengkai Guo

Full Text Available Atherosclerosis is among the leading causes of death and disability. Combining information from multi-modal vascular images is an effective and efficient way to diagnose and monitor atherosclerosis, in which image registration is a key technique. In this paper a feature-based registration algorithm, Two-step Auto-labeling Conditional Iterative Closed Points (TACICP algorithm, is proposed to align three-dimensional carotid image datasets from ultrasound (US and magnetic resonance (MR. Based on 2D segmented contours, a coarse-to-fine strategy is employed with two steps: rigid initialization step and non-rigid refinement step. Conditional Iterative Closest Points (CICP algorithm is given in rigid initialization step to obtain the robust rigid transformation and label configurations. Then the labels and CICP algorithm with non-rigid thin-plate-spline (TPS transformation model is introduced to solve non-rigid carotid deformation between different body positions. The results demonstrate that proposed TACICP algorithm has achieved an average registration error of less than 0.2mm with no failure case, which is superior to the state-of-the-art feature-based methods.

A Joint Land Cover Mapping and Image Registration Algorithm Based on a Markov Random Field Model

Directory of Open Access Journals (Sweden)

Apisit Eiumnoh

2013-10-01

Full Text Available Traditionally, image registration of multi-modal and multi-temporal images is performed satisfactorily before land cover mapping. However, since multi-modal and multi-temporal images are likely to be obtained from different satellite platforms and/or acquired at different times, perfect alignment is very difficult to achieve. As a result, a proper land cover mapping algorithm must be able to correct registration errors as well as perform an accurate classification. In this paper, we propose a joint classification and registration technique based on a Markov random field (MRF model to simultaneously align two or more images and obtain a land cover map (LCM of the scene. The expectation maximization (EM algorithm is employed to solve the joint image classification and registration problem by iteratively estimating the map parameters and approximate posterior probabilities. Then, the maximum a posteriori (MAP criterion is used to produce an optimum land cover map. We conducted experiments on a set of four simulated images and one pair of remotely sensed images to investigate the effectiveness and robustness of the proposed algorithm. Our results show that, with proper selection of a critical MRF parameter, the resulting LCMs derived from an unregistered image pair can achieve an accuracy that is as high as when images are perfectly aligned. Furthermore, the registration error can be greatly reduced.

A novel 3D volumetric voxel registration technique for volume-view-guided image registration of multiple imaging modalities

International Nuclear Information System (INIS)

Li Guang; Xie Huchen; Ning, Holly; Capala, Jacek; Arora, Barbara C.; Coleman, C. Norman; Camphausen, Kevin; Miller, Robert W.

2005-01-01

Purpose: To provide more clinically useful image registration with improved accuracy and reduced time, a novel technique of three-dimensional (3D) volumetric voxel registration of multimodality images is developed. Methods and Materials: This technique can register up to four concurrent images from multimodalities with volume view guidance. Various visualization effects can be applied, facilitating global and internal voxel registration. Fourteen computed tomography/magnetic resonance (CT/MR) image sets and two computed tomography/positron emission tomography (CT/PET) image sets are used. For comparison, an automatic registration technique using maximization of mutual information (MMI) and a three-orthogonal-planar (3P) registration technique are used. Results: Visually sensitive registration criteria for CT/MR and CT/PET have been established, including the homogeneity of color distribution. Based on the registration results of 14 CT/MR images, the 3D voxel technique is in excellent agreement with the automatic MMI technique and is indicatory of a global positioning error (defined as the means and standard deviations of the error distribution) using the 3P pixel technique: 1.8 deg ± 1.2 deg in rotation and 2.0 ± 1.3 (voxel unit) in translation. To the best of our knowledge, this is the first time that such positioning error has been addressed. Conclusion: This novel 3D voxel technique establishes volume-view-guided image registration of up to four modalities. It improves registration accuracy with reduced time, compared with the 3P pixel technique. This article suggests that any interactive and automatic registration should be safeguarded using the 3D voxel technique

Multimodal Registration and Fusion for 3D Thermal Imaging

Directory of Open Access Journals (Sweden)

Moulay A. Akhloufi

2015-01-01

Full Text Available 3D vision is an area of computer vision that has attracted a lot of research interest and has been widely studied. In recent years we witness an increasing interest from the industrial community. This interest is driven by the recent advances in 3D technologies, which enable high precision measurements at an affordable cost. With 3D vision techniques we can conduct advanced manufactured parts inspections and metrology analysis. However, we are not able to detect subsurface defects. This kind of detection is achieved by other techniques, like infrared thermography. In this work, we present a new registration framework for 3D and thermal infrared multimodal fusion. The resulting fused data can be used for advanced 3D inspection in Nondestructive Testing and Evaluation (NDT&E applications. The fusion permits the simultaneous visible surface and subsurface inspections to be conducted in the same process. Experimental tests were conducted with different materials. The obtained results are promising and show how these new techniques can be used efficiently in a combined NDT&E-Metrology analysis of manufactured parts, in areas such as aerospace and automotive.

Comparison of manual vs. automated multimodality (CT-MRI) image registration for brain tumors

International Nuclear Information System (INIS)

Sarkar, Abhirup; Santiago, Roberto J.; Smith, Ryan; Kassaee, Alireza

2005-01-01

Computed tomgoraphy-magnetic resonance imaging (CT-MRI) registrations are routinely used for target-volume delineation of brain tumors. We clinically use 2 software packages based on manual operation and 1 automated package with 2 different algorithms: chamfer matching using bony structures, and mutual information using intensity patterns. In all registration algorithms, a minimum of 3 pairs of identical anatomical and preferably noncoplanar landmarks is used on each of the 2 image sets. In manual registration, the program registers these points and links the image sets using a 3-dimensional (3D) transformation. In automated registration, the 3 landmarks are used as an initial starting point and further processing is done to complete the registration. Using our registration packages, registration of CT and MRI was performed on 10 patients. We scored the results of each registration set based on the amount of time spent, the accuracy reported by the software, and a final evaluation. We evaluated each software program by measuring the residual error between 'matched' points on the right and left globes and the posterior fossa for fused image slices. In general, manual registration showed higher misalignment between corresponding points compared to automated registration using intensity matching. This error had no directional dependence and was, most of the time, larger for a larger structure in both registration techniques. Automated algorithm based on intensity matching also gave the best results in terms of registration accuracy, irrespective of whether or not the initial landmarks were chosen carefully, when compared to that done using bone matching algorithm. Intensity-matching algorithm required the least amount of user-time and provided better accuracy

Feasibility of Multimodal Deformable Registration for Head and Neck Tumor Treatment Planning

Energy Technology Data Exchange (ETDEWEB)

Fortunati, Valerio, E-mail: v.fortunati@erasmusmc.nl [Biomedical Imaging Group Rotterdam, Department of Medical Informatics and Radiology, Erasmus MC University Medical Center, Rotterdam (Netherlands); Verhaart, René F. [Hyperthermia Unit, Department of Radiation Oncology, Erasmus MC University Medical Center Cancer Institute, Rotterdam (Netherlands); Angeloni, Francesco [Istituto di Ricovero e Cura a Carattere Scientifico Foundation SDN for Research and High Education in Nuclear Diagnostics, Naples (Italy); Lugt, Aad van der [Department of Radiology, Erasmus MC University Medical Center, Rotterdam (Netherlands); Niessen, Wiro J. [Biomedical Imaging Group Rotterdam, Department of Medical Informatics and Radiology, Erasmus MC University Medical Center, Rotterdam (Netherlands); Faculty of Applied Sciences, Delft University of Technology, Delft (Netherlands); Veenland, Jifke F. [Biomedical Imaging Group Rotterdam, Department of Medical Informatics and Radiology, Erasmus MC University Medical Center, Rotterdam (Netherlands); Paulides, Margarethus M. [Hyperthermia Unit, Department of Radiation Oncology, Erasmus MC University Medical Center Cancer Institute, Rotterdam (Netherlands); Walsum, Theo van [Biomedical Imaging Group Rotterdam, Department of Medical Informatics and Radiology, Erasmus MC University Medical Center, Rotterdam (Netherlands)

2014-09-01

Purpose: To investigate the feasibility of using deformable registration in clinical practice to fuse MR and CT images of the head and neck for treatment planning. Method and Materials: A state-of-the-art deformable registration algorithm was optimized, evaluated, and compared with rigid registration. The evaluation was based on manually annotated anatomic landmarks and regions of interest in both modalities. We also developed a multiparametric registration approach, which simultaneously aligns T1- and T2-weighted MR sequences to CT. This was evaluated and compared with single-parametric approaches. Results: Our results show that deformable registration yielded a better accuracy than rigid registration, without introducing unrealistic deformations. For deformable registration, an average landmark alignment of approximatively 1.7 mm was obtained. For all the regions of interest excluding the cerebellum and the parotids, deformable registration provided a median modified Hausdorff distance of approximatively 1 mm. Similar accuracies were obtained for the single-parameter and multiparameter approaches. Conclusions: This study demonstrates that deformable registration of head-and-neck CT and MR images is feasible, with overall a significanlty higher accuracy than for rigid registration.

Feasibility of Multimodal Deformable Registration for Head and Neck Tumor Treatment Planning

International Nuclear Information System (INIS)

Fortunati, Valerio; Verhaart, René F.; Angeloni, Francesco; Lugt, Aad van der; Niessen, Wiro J.; Veenland, Jifke F.; Paulides, Margarethus M.; Walsum, Theo van

2014-01-01

Purpose: To investigate the feasibility of using deformable registration in clinical practice to fuse MR and CT images of the head and neck for treatment planning. Method and Materials: A state-of-the-art deformable registration algorithm was optimized, evaluated, and compared with rigid registration. The evaluation was based on manually annotated anatomic landmarks and regions of interest in both modalities. We also developed a multiparametric registration approach, which simultaneously aligns T1- and T2-weighted MR sequences to CT. This was evaluated and compared with single-parametric approaches. Results: Our results show that deformable registration yielded a better accuracy than rigid registration, without introducing unrealistic deformations. For deformable registration, an average landmark alignment of approximatively 1.7 mm was obtained. For all the regions of interest excluding the cerebellum and the parotids, deformable registration provided a median modified Hausdorff distance of approximatively 1 mm. Similar accuracies were obtained for the single-parameter and multiparameter approaches. Conclusions: This study demonstrates that deformable registration of head-and-neck CT and MR images is feasible, with overall a significanlty higher accuracy than for rigid registration

Deformable image registration for cone-beam CT guided transoral robotic base-of-tongue surgery

International Nuclear Information System (INIS)

Reaungamornrat, S; Liu, W P; Otake, Y; Uneri, A; Siewerdsen, J H; Taylor, R H; Wang, A S; Nithiananthan, S; Schafer, S; Tryggestad, E; Richmon, J; Sorger, J M

2013-01-01

Transoral robotic surgery (TORS) offers a minimally invasive approach to resection of base-of-tongue tumors. However, precise localization of the surgical target and adjacent critical structures can be challenged by the highly deformed intraoperative setup. We propose a deformable registration method using intraoperative cone-beam computed tomography (CBCT) to accurately align preoperative CT or MR images with the intraoperative scene. The registration method combines a Gaussian mixture (GM) model followed by a variation of the Demons algorithm. First, following segmentation of the volume of interest (i.e. volume of the tongue extending to the hyoid), a GM model is applied to surface point clouds for rigid initialization (GM rigid) followed by nonrigid deformation (GM nonrigid). Second, the registration is refined using the Demons algorithm applied to distance map transforms of the (GM-registered) preoperative image and intraoperative CBCT. Performance was evaluated in repeat cadaver studies (25 image pairs) in terms of target registration error (TRE), entropy correlation coefficient (ECC) and normalized pointwise mutual information (NPMI). Retraction of the tongue in the TORS operative setup induced gross deformation >30 mm. The mean TRE following the GM rigid, GM nonrigid and Demons steps was 4.6, 2.1 and 1.7 mm, respectively. The respective ECC was 0.57, 0.70 and 0.73, and NPMI was 0.46, 0.57 and 0.60. Registration accuracy was best across the superior aspect of the tongue and in proximity to the hyoid (by virtue of GM registration of surface points on these structures). The Demons step refined registration primarily in deeper portions of the tongue further from the surface and hyoid bone. Since the method does not use image intensities directly, it is suitable to multi-modality registration of preoperative CT or MR with intraoperative CBCT. Extending the 3D image registration to the fusion of image and planning data in stereo-endoscopic video is anticipated to

Multi-modal image registration: matching MRI with histology

Science.gov (United States)

Alic, Lejla; Haeck, Joost C.; Klein, Stefan; Bol, Karin; van Tiel, Sandra T.; Wielopolski, Piotr A.; Bijster, Magda; Niessen, Wiro J.; Bernsen, Monique; Veenland, Jifke F.; de Jong, Marion

2010-03-01

Spatial correspondence between histology and multi sequence MRI can provide information about the capabilities of non-invasive imaging to characterize cancerous tissue. However, shrinkage and deformation occurring during the excision of the tumor and the histological processing complicate the co registration of MR images with histological sections. This work proposes a methodology to establish a detailed 3D relation between histology sections and in vivo MRI tumor data. The key features of the methodology are a very dense histological sampling (up to 100 histology slices per tumor), mutual information based non-rigid B-spline registration, the utilization of the whole 3D data sets, and the exploitation of an intermediate ex vivo MRI. In this proof of concept paper, the methodology was applied to one tumor. We found that, after registration, the visual alignment of tumor borders and internal structures was fairly accurate. Utilizing the intermediate ex vivo MRI, it was possible to account for changes caused by the excision of the tumor: we observed a tumor expansion of 20%. Also the effects of fixation, dehydration and histological sectioning could be determined: 26% shrinkage of the tumor was found. The annotation of viable tissue, performed in histology and transformed to the in vivo MRI, matched clearly with high intensity regions in MRI. With this methodology, histological annotation can be directly related to the corresponding in vivo MRI. This is a vital step for the evaluation of the feasibility of multi-spectral MRI to depict histological groundtruth.

A comparative study of surface- and volume-based techniques for the automatic registration between CT and SPECT brain images

International Nuclear Information System (INIS)

Kagadis, George C.; Delibasis, Konstantinos K.; Matsopoulos, George K.; Mouravliansky, Nikolaos A.; Asvestas, Pantelis A.; Nikiforidis, George C.

2002-01-01

Image registration of multimodality images is an essential task in numerous applications in three-dimensional medical image processing. Medical diagnosis can benefit from the complementary information in different modality images. Surface-based registration techniques, while still widely used, were succeeded by volume-based registration algorithms that appear to be theoretically advantageous in terms of reliability and accuracy. Several applications of such algorithms for the registration of CT-MRI, CT-PET, MRI-PET, and SPECT-MRI images have emerged in the literature, using local optimization techniques for the matching of images. Our purpose in this work is the development of automatic techniques for the registration of real CT and SPECT images, based on either surface- or volume-based algorithms. Optimization is achieved using genetic algorithms that are known for their robustness. The two techniques are compared against a well-established method, the Iterative Closest Point--ICP. The correlation coefficient was employed as an independent measure of spatial match, to produce unbiased results. The repeated measures ANOVA indicates the significant impact of the choice of registration method on the magnitude of the correlation (F=4.968, p=0.0396). The volume-based method achieves an average correlation coefficient value of 0.454 with a standard deviation of 0.0395, as opposed to an average of 0.380 with a standard deviation of 0.0603 achieved by the surface-based method and an average of 0.396 with a standard deviation equal to 0.0353 achieved by ICP. The volume-based technique performs significantly better compared to both ICP (p<0.05, Neuman Keuls test) and the surface-based technique (p<0.05, Neuman-Keuls test). Surface-based registration and ICP do not differ significantly in performance

Markerless registration for image guided surgery. Preoperative image, intraoperative video image, and patient

International Nuclear Information System (INIS)

Kihara, Tomohiko; Tanaka, Yuko

1998-01-01

Real-time and volumetric acquisition of X-ray CT, MR, and SPECT is the latest trend of the medical imaging devices. A clinical challenge is to use these multi-modality volumetric information complementary on patient in the entire diagnostic and surgical processes. The intraoperative image and patient integration intents to establish a common reference frame by image in diagnostic and surgical processes. This provides a quantitative measure during surgery, for which we have been relied mostly on doctors' skills and experiences. The intraoperative image and patient integration involves various technologies, however, we think one of the most important elements is the development of markerless registration, which should be efficient and applicable to the preoperative multi-modality data sets, intraoperative image, and patient. We developed a registration system which integrates preoperative multi-modality images, intraoperative video image, and patient. It consists of a real-time registration of video camera for intraoperative use, a markerless surface sampling matching of patient and image, our previous works of markerless multi-modality image registration of X-ray CT, MR, and SPECT, and an image synthesis on video image. We think these techniques can be used in many applications which involve video camera like devices such as video camera, microscope, and image Intensifier. (author)

LINE-BASED REGISTRATION OF DSM AND HYPERSPECTRAL IMAGES

Directory of Open Access Journals (Sweden)

J. Avbelj

2013-04-01

Full Text Available Data fusion techniques require a good registration of all the used datasets. In remote sensing, images are usually geo-referenced using the GPS and IMU data. However, if more precise registration is required, image processing techniques can be employed. We propose a method for multi-modal image coregistration between hyperspectral images (HSI and digital surface models (DSM. The method is divided in three parts: object and line detection of the same object in HSI and DSM, line matching and determination of transformation parameters. Homogeneous coordinates are used to implement matching and adjustment of transformation parameters. The common object in HSI and DSM are building boundaries. They have apparent change in height and material, that can be detected in DSM and HSI, respectively. Thus, before the matching and transformation parameter computation, building outlines are detected and adjusted in HSI and DSM. We test the method on a HSI and two DSM, using extracted building outbounds and for comparison also extracted lines with a line detector. The results show that estimated building boundaries provide more line assignments, than using line detector.

Demonstration of accuracy and clinical versatility of mutual information for automatic multimodality image fusion using affine and thin-plate spline warped geometric deformations.

Science.gov (United States)

Meyer, C R; Boes, J L; Kim, B; Bland, P H; Zasadny, K R; Kison, P V; Koral, K; Frey, K A; Wahl, R L

1997-04-01

This paper applies and evaluates an automatic mutual information-based registration algorithm across a broad spectrum of multimodal volume data sets. The algorithm requires little or no pre-processing, minimal user input and easily implements either affine, i.e. linear or thin-plate spline (TPS) warped registrations. We have evaluated the algorithm in phantom studies as well as in selected cases where few other algorithms could perform as well, if at all, to demonstrate the value of this new method. Pairs of multimodal gray-scale volume data sets were registered by iteratively changing registration parameters to maximize mutual information. Quantitative registration errors were assessed in registrations of a thorax phantom using PET/CT and in the National Library of Medicine's Visible Male using MRI T2-/T1-weighted acquisitions. Registrations of diverse clinical data sets were demonstrated including rotate-translate mapping of PET/MRI brain scans with significant missing data, full affine mapping of thoracic PET/CT and rotate-translate mapping of abdominal SPECT/CT. A five-point thin-plate spline (TPS) warped registration of thoracic PET/CT is also demonstrated. The registration algorithm converged in times ranging between 3.5 and 31 min for affine clinical registrations and 57 min for TPS warping. Mean error vector lengths for rotate-translate registrations were measured to be subvoxel in phantoms. More importantly the rotate-translate algorithm performs well even with missing data. The demonstrated clinical fusions are qualitatively excellent at all levels. We conclude that such automatic, rapid, robust algorithms significantly increase the likelihood that multimodality registrations will be routinely used to aid clinical diagnoses and post-therapeutic assessment in the near future.

3D/2D model-to-image registration by imitation learning for cardiac procedures.

Science.gov (United States)

Toth, Daniel; Miao, Shun; Kurzendorfer, Tanja; Rinaldi, Christopher A; Liao, Rui; Mansi, Tommaso; Rhode, Kawal; Mountney, Peter

2018-05-12

In cardiac interventions, such as cardiac resynchronization therapy (CRT), image guidance can be enhanced by involving preoperative models. Multimodality 3D/2D registration for image guidance, however, remains a significant research challenge for fundamentally different image data, i.e., MR to X-ray. Registration methods must account for differences in intensity, contrast levels, resolution, dimensionality, field of view. Furthermore, same anatomical structures may not be visible in both modalities. Current approaches have focused on developing modality-specific solutions for individual clinical use cases, by introducing constraints, or identifying cross-modality information manually. Machine learning approaches have the potential to create more general registration platforms. However, training image to image methods would require large multimodal datasets and ground truth for each target application. This paper proposes a model-to-image registration approach instead, because it is common in image-guided interventions to create anatomical models for diagnosis, planning or guidance prior to procedures. An imitation learning-based method, trained on 702 datasets, is used to register preoperative models to intraoperative X-ray images. Accuracy is demonstrated on cardiac models and artificial X-rays generated from CTs. The registration error was [Formula: see text] on 1000 test cases, superior to that of manual ([Formula: see text]) and gradient-based ([Formula: see text]) registration. High robustness is shown in 19 clinical CRT cases. Besides the proposed methods feasibility in a clinical environment, evaluation has shown good accuracy and high robustness indicating that it could be applied in image-guided interventions.

Robust Multimodal Dictionary Learning

Science.gov (United States)

Cao, Tian; Jojic, Vladimir; Modla, Shannon; Powell, Debbie; Czymmek, Kirk; Niethammer, Marc

2014-01-01

We propose a robust multimodal dictionary learning method for multimodal images. Joint dictionary learning for both modalities may be impaired by lack of correspondence between image modalities in training data, for example due to areas of low quality in one of the modalities. Dictionaries learned with such non-corresponding data will induce uncertainty about image representation. In this paper, we propose a probabilistic model that accounts for image areas that are poorly corresponding between the image modalities. We cast the problem of learning a dictionary in presence of problematic image patches as a likelihood maximization problem and solve it with a variant of the EM algorithm. Our algorithm iterates identification of poorly corresponding patches and re-finements of the dictionary. We tested our method on synthetic and real data. We show improvements in image prediction quality and alignment accuracy when using the method for multimodal image registration. PMID:24505674

An efficient strategy based on an individualized selection of registration methods. Application to the coregistration of MR and SPECT images in neuro-oncology

International Nuclear Information System (INIS)

Tacchella, Jean-Marc; Lefort, Muriel; Habert, Marie-Odile; Yeni, Nathanaëlle; Kas, Aurélie; Frouin, Frédérique; Roullot, Elodie; Cohen, Mike-Ely; Guillevin, Rémy; Petrirena, Grégorio; Delattre, Jean-Yves

2014-01-01

An efficient registration strategy is described that aims to help solve delicate medical imaging registration problems. It consists of running several registration methods for each dataset and selecting the best one for each specific dataset, according to an evaluation criterion. Finally, the quality of the registration results, obtained with the best method, is visually scored by an expert as excellent, correct or poor. The strategy was applied to coregister Technetium-99m Sestamibi SPECT and MRI data in the framework of a follow-up protocol in patients with high grade gliomas receiving antiangiogenic therapy. To adapt the strategy to this clinical context, a robust semi-automatic evaluation criterion based on the physiological uptake of the Sestamibi tracer was defined. A panel of eighteen multimodal registration algorithms issued from BrainVisa, SPM or AIR software environments was systematically applied to the clinical database composed of sixty-two datasets. According to the expert visual validation, this new strategy provides 85% excellent registrations, 12% correct ones and only 3% poor ones. These results compare favorably to the ones obtained by the globally most efficient registration method over the whole database, for which only 61% of excellent registration results have been reported. Thus the registration strategy in its current implementation proves to be suitable for clinical application. (paper)

Robust multi-site MR data processing: iterative optimization of bias correction, tissue classification, and registration.

Science.gov (United States)

Young Kim, Eun; Johnson, Hans J

2013-01-01

A robust multi-modal tool, for automated registration, bias correction, and tissue classification, has been implemented for large-scale heterogeneous multi-site longitudinal MR data analysis. This work focused on improving the an iterative optimization framework between bias-correction, registration, and tissue classification inspired from previous work. The primary contributions are robustness improvements from incorporation of following four elements: (1) utilize multi-modal and repeated scans, (2) incorporate high-deformable registration, (3) use extended set of tissue definitions, and (4) use of multi-modal aware intensity-context priors. The benefits of these enhancements were investigated by a series of experiments with both simulated brain data set (BrainWeb) and by applying to highly-heterogeneous data from a 32 site imaging study with quality assessments through the expert visual inspection. The implementation of this tool is tailored for, but not limited to, large-scale data processing with great data variation with a flexible interface. In this paper, we describe enhancements to a joint registration, bias correction, and the tissue classification, that improve the generalizability and robustness for processing multi-modal longitudinal MR scans collected at multi-sites. The tool was evaluated by using both simulated and simulated and human subject MRI images. With these enhancements, the results showed improved robustness for large-scale heterogeneous MRI processing.

Feature and Intensity Based Medical Image Registration Using Particle Swarm Optimization.

Science.gov (United States)

Abdel-Basset, Mohamed; Fakhry, Ahmed E; El-Henawy, Ibrahim; Qiu, Tie; Sangaiah, Arun Kumar

2017-11-03

Image registration is an important aspect in medical image analysis, and kinds use in a variety of medical applications. Examples include diagnosis, pre/post surgery guidance, comparing/merging/integrating images from multi-modal like Magnetic Resonance Imaging (MRI), and Computed Tomography (CT). Whether registering images across modalities for a single patient or registering across patients for a single modality, registration is an effective way to combine information from different images into a normalized frame for reference. Registered datasets can be used for providing information relating to the structure, function, and pathology of the organ or individual being imaged. In this paper a hybrid approach for medical images registration has been developed. It employs a modified Mutual Information (MI) as a similarity metric and Particle Swarm Optimization (PSO) method. Computation of mutual information is modified using a weighted linear combination of image intensity and image gradient vector flow (GVF) intensity. In this manner, statistical as well as spatial image information is included into the image registration process. Maximization of the modified mutual information is effected using the versatile Particle Swarm Optimization which is developed easily with adjusted less parameter. The developed approach has been tested and verified successfully on a number of medical image data sets that include images with missing parts, noise contamination, and/or of different modalities (CT, MRI). The registration results indicate the proposed model as accurate and effective, and show the posture contribution in inclusion of both statistical and spatial image data to the developed approach.

Subspace-Based Holistic Registration for Low-Resolution Facial Images

Directory of Open Access Journals (Sweden)

Boom BJ

2010-01-01

Full Text Available Subspace-based holistic registration is introduced as an alternative to landmark-based face registration, which has a poor performance on low-resolution images, as obtained in camera surveillance applications. The proposed registration method finds the alignment by maximizing the similarity score between a probe and a gallery image. We use a novel probabilistic framework for both user-independent as well as user-specific face registration. The similarity is calculated using the probability that the face image is correctly aligned in a face subspace, but additionally we take the probability into account that the face is misaligned based on the residual error in the dimensions perpendicular to the face subspace. We perform extensive experiments on the FRGCv2 database to evaluate the impact that the face registration methods have on face recognition. Subspace-based holistic registration on low-resolution images can improve face recognition in comparison with landmark-based registration on high-resolution images. The performance of the tested face recognition methods after subspace-based holistic registration on a low-resolution version of the FRGC database is similar to that after manual registration.

Consistency of parametric registration in serial MRI studies of brain tumor progression

International Nuclear Information System (INIS)

Mang, Andreas; Buzug, Thorsten M.; Schnabel, Julia A.; Crum, William R.; Modat, Marc; Ourselin, Sebastien; Hawkes, David J.; Camara-Rey, Oscar; Palm, Christoph; Caseiras, Gisele Brasil; Jaeger, H.R.

2008-01-01

The consistency of parametric registration in multi-temporal magnetic resonance (MR) imaging studies was evaluated. Serial MRI scans of adult patients with a brain tumor (glioma) were aligned by parametric registration. The performance of low-order spatial alignment (6/9/12 degrees of freedom) of different 3D serial MR-weighted images is evaluated. A registration protocol for the alignment of all images to one reference coordinate system at baseline is presented. Registration results were evaluated for both, multimodal intra-timepoint and mono-modal multi-temporal registration. The latter case might present a challenge to automatic intensity-based registration algorithms due to ill-defined correspondences. The performance of our algorithm was assessed by testing the inverse registration consistency. Four different similarity measures were evaluated to assess consistency. Careful visual inspection suggests that images are well aligned, but their consistency may be imperfect. Sub-voxel inconsistency within the brain was found for allsimilarity measures used for parametric multi-temporal registration. T1-weighted images were most reliable for establishing spatial correspondence between different timepoints. The parametric registration algorithm is feasible for use in this application. The sub-voxel resolution mean displacement error of registration transformations demonstrates that the algorithm converges to an almost identical solution for forward and reverse registration. (orig.)

Fiber-Optic Vibration Sensor Based on Multimode Fiber

Directory of Open Access Journals (Sweden)

I. Lujo

2008-06-01

Full Text Available The purpose of this paper is to present a fiberoptic vibration sensor based on the monitoring of the mode distribution in a multimode optical fiber. Detection of vibrations and their parameters is possible through observation of the output speckle pattern from the multimode optical fiber. A working experimental model has been built in which all used components are widely available and cheap: a CCD camera (a simple web-cam, a multimode laser in visible range as a light source, a length of multimode optical fiber, and a computer for signal processing. Measurements have shown good agreement with the actual frequency of vibrations, and promising results were achieved with the amplitude measurements although they require some adaptation of the experimental model. Proposed sensor is cheap and lightweight and therefore presents an interesting alternative for monitoring large smart structures.

Semiautomated Multimodal Breast Image Registration

Directory of Open Access Journals (Sweden)

Charlotte Curtis

2012-01-01

However, due to the highly deformable nature of breast tissues, comparison of 3D and 2D modalities is a challenge. To enable this comparison, a registration technique was developed to map features from 2D mammograms to locations in the 3D image space. This technique was developed and tested using magnetic resonance (MR images as a reference 3D modality, as MR breast imaging is an established technique in clinical practice. The algorithm was validated using a numerical phantom then successfully tested on twenty-four image pairs. Dice's coefficient was used to measure the external goodness of fit, resulting in an excellent overall average of 0.94. Internal agreement was evaluated by examining internal features in consultation with a radiologist, and subjective assessment concludes that reasonable alignment was achieved.

Wavelet based free-form deformations for nonrigid registration

Science.gov (United States)

Sun, Wei; Niessen, Wiro J.; Klein, Stefan

2014-03-01

In nonrigid registration, deformations may take place on the coarse and fine scales. For the conventional B-splines based free-form deformation (FFD) registration, these coarse- and fine-scale deformations are all represented by basis functions of a single scale. Meanwhile, wavelets have been proposed as a signal representation suitable for multi-scale problems. Wavelet analysis leads to a unique decomposition of a signal into its coarse- and fine-scale components. Potentially, this could therefore be useful for image registration. In this work, we investigate whether a wavelet-based FFD model has advantages for nonrigid image registration. We use a B-splines based wavelet, as defined by Cai and Wang.1 This wavelet is expressed as a linear combination of B-spline basis functions. Derived from the original B-spline function, this wavelet is smooth, differentiable, and compactly supported. The basis functions of this wavelet are orthogonal across scales in Sobolev space. This wavelet was previously used for registration in computer vision, in 2D optical flow problems,2 but it was not compared with the conventional B-spline FFD in medical image registration problems. An advantage of choosing this B-splines based wavelet model is that the space of allowable deformation is exactly equivalent to that of the traditional B-spline. The wavelet transformation is essentially a (linear) reparameterization of the B-spline transformation model. Experiments on 10 CT lung and 18 T1-weighted MRI brain datasets show that wavelet based registration leads to smoother deformation fields than traditional B-splines based registration, while achieving better accuracy.

A data fusion environment for multimodal and multi-informational neuronavigation.

Science.gov (United States)

Jannin, P; Fleig, O J; Seigneuret, E; Grova, C; Morandi, X; Scarabin, J M

2000-01-01

Part of the planning and performance of neurosurgery consists of determining target areas, areas to be avoided, landmark areas, and trajectories, all of which are components of the surgical script. Nowadays, neurosurgeons have access to multimodal medical imaging to support the definition of the surgical script. The purpose of this paper is to present a software environment developed by the authors that allows full multimodal and multi-informational planning as well as neuronavigation for epilepsy and tumor surgery. We have developed a data fusion environment dedicated to neuronavigation around the Surgical Microscope Neuronavigator system (Carl Zeiss, Oberkochen, Germany). This environment includes registration, segmentation, 3D visualization, and interaction-applied tools. It provides the neuronavigation system with the multimodal information involved in the definition of the surgical script: lesional areas, sulci, ventricles segmented from magnetic resonance imaging (MRI), vessels segmented from magnetic resonance angiography (MRA), functional areas from magneto-encephalography (MEG), and functional magnetic resonance imaging (fMRI) for somatosensory, motor, or language activation. These data are considered to be relevant for the performance of the surgical procedure. The definition of each entity results from the same procedure: registration to the anatomical MRI data set (defined as the reference data set), segmentation, fused 3D display, selection of the relevant entities for the surgical step, encoding in 3D surface-based representation, and storage of the 3D surfaces in a file recognized by the neuronavigation software (STP 3.4, Leibinger; Freiburg, Germany). Multimodal neuronavigation is illustrated with two clinical cases for which multimodal information was introduced into the neuronavigation system. Lesional areas were used to define and follow the surgical path, sulci and vessels helped identify the anatomical environment of the surgical field, and

Fiducial-based registration with a touchable region model.

Science.gov (United States)

Kim, Sungmin; Kazanzides, Peter

2017-02-01

Image-guided surgery requires registration between an image coordinate system and an intraoperative coordinate system that is typically referenced to a tracking device. In fiducial-based registration methods, this is achieved by localizing points (fiducials) in each coordinate system. Often, both localizations are performed manually, first by picking a fiducial point in the image and then by using a hand-held tracked pointer to physically touch the corresponding fiducial on the patient. These manual procedures introduce localization error that is user-dependent and can significantly decrease registration accuracy. Thus, there is a need for a registration method that is tolerant of imprecise fiducial localization in the preoperative and intraoperative phases. We propose the iterative closest touchable point (ICTP) registration framework, which uses model-based localization and a touchable region model. This method consists of three stages: (1) fiducial marker localization in image space, using a fiducial marker model, (2) initial registration with paired-point registration, and (3) fine registration based on the iterative closest point method. We perform phantom experiments with a fiducial marker design that is commonly used in neurosurgery. The results demonstrate that ICTP can provide accuracy improvements compared to the standard paired-point registration method that is widely used for surgical navigation and surgical robot systems, especially in cases where the surgeon introduces large localization errors. The results demonstrate that the proposed method can reduce the effect of the surgeon's localization performance on the accuracy of registration, thereby producing more consistent and less user-dependent registration outcomes.

MO-DE-202-02: Advances in Image Registration and Reconstruction for Image-Guided Neurosurgery

Energy Technology Data Exchange (ETDEWEB)

Siewerdsen, J. [Johns Hopkins University (United States)

2016-06-15

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guided neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504

MO-DE-202-02: Advances in Image Registration and Reconstruction for Image-Guided Neurosurgery

International Nuclear Information System (INIS)

Siewerdsen, J.

2016-01-01

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guided neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504

MO-DE-202-04: Multimodality Image-Guided Surgery and Intervention: For the Rest of Us

Energy Technology Data Exchange (ETDEWEB)

Shekhar, R. [Children’s National Health System (United States)

2016-06-15

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guided neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504

MO-DE-202-04: Multimodality Image-Guided Surgery and Intervention: For the Rest of Us

International Nuclear Information System (INIS)

Shekhar, R.

2016-01-01

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guided neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504

A general technique for interstudy registration of multifunction and multimodality images

International Nuclear Information System (INIS)

Lin, K.P.; Huang, S.C.; Bacter, L.R.; Phelps, M.E.

1994-01-01

A technique that can register anatomic/structural brain images (e.g., MRI) with various functional images (e.g., PET-FDG and PET-FDOPA) of the same subject has been developed. The procedure of this technique includes the following steps: (1) segmentation of MRI brain images into gray matter (GM), white matter (WM), cerebral spinal fluid (CSF), and, muscle (MS) components, (2) assignment of appropriate radio-tracer concentrations to various components depending on the kind of functional image that is being registered, (3) generation of simulated functional images to have a spatial resolution that is comparable to that of the measured ones, (4) alignment of the measured functional images to the simulated ones that are based on MRI images. A self-organization clustering method is used to segment the MRI images. The image alignment is based on the criterion of least squares of the pixel-by-pixel differences between the two sets of images that are being matched and on the Powell's algorithm for minimization. The technique was applied successfully for registering the MRI, PET-FDG, and PET-FDOPA images. This technique offers a general solution to the registration of structural images to functional images and to the registration of different functional images of markedly different distributions

Reliability-Based Decision Fusion in Multimodal Biometric Verification Systems

Directory of Open Access Journals (Sweden)

Kryszczuk Krzysztof

2007-01-01

Full Text Available We present a methodology of reliability estimation in the multimodal biometric verification scenario. Reliability estimation has shown to be an efficient and accurate way of predicting and correcting erroneous classification decisions in both unimodal (speech, face, online signature and multimodal (speech and face systems. While the initial research results indicate the high potential of the proposed methodology, the performance of the reliability estimation in a multimodal setting has not been sufficiently studied or evaluated. In this paper, we demonstrate the advantages of using the unimodal reliability information in order to perform an efficient biometric fusion of two modalities. We further show the presented method to be superior to state-of-the-art multimodal decision-level fusion schemes. The experimental evaluation presented in this paper is based on the popular benchmarking bimodal BANCA database.

Co-registration of the BNCT treatment planning images for clinical practice

International Nuclear Information System (INIS)

Salli, Eero; Seppaelae, Tiina; Kankaanranta, Leena; Asikainen, Sami; Savolainen, Sauli; Koivunoro, Hanna

2006-01-01

We have co-registered MRI, CT and FBPA-PET images for BNCT in clinical practice. Co-registration improves the spatial accuracy of the treatment planning by enabling use of information from all the co-registered modalities. The multimodal co-registration has been implemented as a service product provided by the Imaging Center of Helsinki University Central Hospital to other departments. To increase the accuracy of co-registration and patient positioning in the head area BNCT, a patient-specific fixation mask suitable for PET, MRI and CT was developed. The goal of the fixation mask is to normalize the orientation of the patient's head and neck. Co-registration is performed at the image processing unit by using a rigid body model, mutual-information based algorithms and partly in-house developed software tools. The accuracy of co-registration is verified by comparing the locations of the external skin markers and anatomical landmarks in different modalities. After co-registration, the images are transformed and covered into a format required by the BNCT dose-planning software and set to the dose-planning unit of the hospital. So far co-registration has been done for 22 patients. The co-registration protocol has proved to be reliable and efficient. Some registration errors are seen on some patients in the neck area because the rigid-body model used in co-registration is not fully valid for the brain-neck entity. The registration accuracy in this area could likely be improved by implementing a co-registration procedure utilizing a partly non-rigid body model. (author)

A multicore based parallel image registration method.

Science.gov (United States)

Yang, Lin; Gong, Leiguang; Zhang, Hong; Nosher, John L; Foran, David J

2009-01-01

Image registration is a crucial step for many image-assisted clinical applications such as surgery planning and treatment evaluation. In this paper we proposed a landmark based nonlinear image registration algorithm for matching 2D image pairs. The algorithm was shown to be effective and robust under conditions of large deformations. In landmark based registration, the most important step is establishing the correspondence among the selected landmark points. This usually requires an extensive search which is often computationally expensive. We introduced a nonregular data partition algorithm using the K-means clustering algorithm to group the landmarks based on the number of available processing cores. The step optimizes the memory usage and data transfer. We have tested our method using IBM Cell Broadband Engine (Cell/B.E.) platform.

A multimodal 3D framework for fire characteristics estimation

Science.gov (United States)

Toulouse, T.; Rossi, L.; Akhloufi, M. A.; Pieri, A.; Maldague, X.

2018-02-01

In the last decade we have witnessed an increasing interest in using computer vision and image processing in forest fire research. Image processing techniques have been successfully used in different fire analysis areas such as early detection, monitoring, modeling and fire front characteristics estimation. While the majority of the work deals with the use of 2D visible spectrum images, recent work has introduced the use of 3D vision in this field. This work proposes a new multimodal vision framework permitting the extraction of the three-dimensional geometrical characteristics of fires captured by multiple 3D vision systems. The 3D system is a multispectral stereo system operating in both the visible and near-infrared (NIR) spectral bands. The framework supports the use of multiple stereo pairs positioned so as to capture complementary views of the fire front during its propagation. Multimodal registration is conducted using the captured views in order to build a complete 3D model of the fire front. The registration process is achieved using multisensory fusion based on visual data (2D and NIR images), GPS positions and IMU inertial data. Experiments were conducted outdoors in order to show the performance of the proposed framework. The obtained results are promising and show the potential of using the proposed framework in operational scenarios for wildland fire research and as a decision management system in fighting.

Coordinate Systems Integration for Craniofacial Database from Multimodal Devices

Directory of Open Access Journals (Sweden)

Deni Suwardhi

2005-05-01

Full Text Available This study presents a data registration method for craniofacial spatial data of different modalities. The data consists of three dimensional (3D vector and raster data models. The data is stored in object relational database. The data capture devices are Laser scanner, CT (Computed Tomography scan and CR (Close Range Photogrammetry. The objective of the registration is to transform the data from various coordinate systems into a single 3-D Cartesian coordinate system. The standard error of the registration obtained from multimodal imaging devices using 3D affine transformation is in the ranged of 1-2 mm. This study is a step forward for storing the craniofacial spatial data in one reference system in database.

Optical sensor in planar configuration based on multimode interference

Science.gov (United States)

Blahut, Marek

2017-08-01

In the paper a numerical analysis of optical sensors based on multimode interference in planar one-dimensional step-index configuration is presented. The structure consists in single-mode input and output waveguides and multimode waveguide which guide only few modes. Material parameters discussed refer to a SU8 polymer waveguide on SiO2 substrate. The optical system described will be designed to the analysis of biological substances.

Content-based TV sports video retrieval using multimodal analysis

Science.gov (United States)

Yu, Yiqing; Liu, Huayong; Wang, Hongbin; Zhou, Dongru

2003-09-01

In this paper, we propose content-based video retrieval, which is a kind of retrieval by its semantical contents. Because video data is composed of multimodal information streams such as video, auditory and textual streams, we describe a strategy of using multimodal analysis for automatic parsing sports video. The paper first defines the basic structure of sports video database system, and then introduces a new approach that integrates visual stream analysis, speech recognition, speech signal processing and text extraction to realize video retrieval. The experimental results for TV sports video of football games indicate that the multimodal analysis is effective for video retrieval by quickly browsing tree-like video clips or inputting keywords within predefined domain.

Feasibility of a novel deformable image registration technique to facilitate classification, targeting, and monitoring of tumor and normal tissue

International Nuclear Information System (INIS)

Brock, Kristy K.; Dawson, Laura A.; Sharpe, Michael B.; Moseley, Douglas J.; Jaffray, David A.

2006-01-01

Purpose: To investigate the feasibility of a biomechanical-based deformable image registration technique for the integration of multimodality imaging, image guided treatment, and response monitoring. Methods and Materials: A multiorgan deformable image registration technique based on finite element modeling (FEM) and surface projection alignment of selected regions of interest with biomechanical material and interface models has been developed. FEM also provides an inherent method for direct tracking specified regions through treatment and follow-up. Results: The technique was demonstrated on 5 liver cancer patients. Differences of up to 1 cm of motion were seen between the diaphragm and the tumor center of mass after deformable image registration of exhale and inhale CT scans. Spatial differences of 5 mm or more were observed for up to 86% of the surface of the defined tumor after deformable image registration of the computed tomography (CT) and magnetic resonance images. Up to 6.8 mm of motion was observed for the tumor after deformable image registration of the CT and cone-beam CT scan after rigid registration of the liver. Deformable registration of the CT to the follow-up CT allowed a more accurate assessment of tumor response. Conclusions: This biomechanical-based deformable image registration technique incorporates classification, targeting, and monitoring of tumor and normal tissue using one methodology

MO-DE-202-03: Image-Guided Surgery and Interventions in the Advanced Multimodality Image-Guided Operating (AMIGO) Suite

Energy Technology Data Exchange (ETDEWEB)

Kapur, T. [Brigham & Women’s Hospital (United States)

2016-06-15

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guided neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504

MO-DE-202-03: Image-Guided Surgery and Interventions in the Advanced Multimodality Image-Guided Operating (AMIGO) Suite

International Nuclear Information System (INIS)

Kapur, T.

2016-01-01

At least three major trends in surgical intervention have emerged over the last decade: a move toward more minimally invasive (or non-invasive) approach to the surgical target; the development of high-precision treatment delivery techniques; and the increasing role of multi-modality intraoperative imaging in support of such procedures. This symposium includes invited presentations on recent advances in each of these areas and the emerging role for medical physics research in the development and translation of high-precision interventional techniques. The four speakers are: Keyvan Farahani, “Image-guided focused ultrasound surgery and therapy” Jeffrey H. Siewerdsen, “Advances in image registration and reconstruction for image-guided neurosurgery” Tina Kapur, “Image-guided surgery and interventions in the advanced multimodality image-guided operating (AMIGO) suite” Raj Shekhar, “Multimodality image-guided interventions: Multimodality for the rest of us” Learning Objectives: Understand the principles and applications of HIFU in surgical ablation. Learn about recent advances in 3D–2D and 3D deformable image registration in support of surgical safety and precision. Learn about recent advances in model-based 3D image reconstruction in application to intraoperative 3D imaging. Understand the multi-modality imaging technologies and clinical applications investigated in the AMIGO suite. Understand the emerging need and techniques to implement multi-modality image guidance in surgical applications such as neurosurgery, orthopaedic surgery, vascular surgery, and interventional radiology. Research supported by the NIH and Siemens Healthcare.; J. Siewerdsen; Grant Support - National Institutes of Health; Grant Support - Siemens Healthcare; Grant Support - Carestream Health; Advisory Board - Carestream Health; Licensing Agreement - Carestream Health; Licensing Agreement - Elekta Oncology.; T. Kapur, P41EB015898; R. Shekhar, Funding: R42CA137886 and R41CA192504

Canny edge-based deformable image registration.

Science.gov (United States)

Kearney, Vasant; Huang, Yihui; Mao, Weihua; Yuan, Baohong; Tang, Liping

2017-02-07

This work focuses on developing a 2D Canny edge-based deformable image registration (Canny DIR) algorithm to register in vivo white light images taken at various time points. This method uses a sparse interpolation deformation algorithm to sparsely register regions of the image with strong edge information. A stability criterion is enforced which removes regions of edges that do not deform in a smooth uniform manner. Using a synthetic mouse surface ground truth model, the accuracy of the Canny DIR algorithm was evaluated under axial rotation in the presence of deformation. The accuracy was also tested using fluorescent dye injections, which were then used for gamma analysis to establish a second ground truth. The results indicate that the Canny DIR algorithm performs better than rigid registration, intensity corrected Demons, and distinctive features for all evaluation matrices and ground truth scenarios. In conclusion Canny DIR performs well in the presence of the unique lighting and shading variations associated with white-light-based image registration.

Comparison of carina-based versus bony anatomy-based registration for setup verification in esophageal cancer radiotherapy.

Science.gov (United States)

Machiels, Mélanie; Jin, Peng; van Gurp, Christianne H; van Hooft, Jeanin E; Alderliesten, Tanja; Hulshof, Maarten C C M

2018-03-21

To investigate the feasibility and geometric accuracy of carina-based registration for CBCT-guided setup verification in esophageal cancer IGRT, compared with current practice bony anatomy-based registration. Included were 24 esophageal cancer patients with 65 implanted fiducial markers, visible on planning CTs and follow-up CBCTs. All available CBCT scans (n = 236) were rigidly registered to the planning CT with respect to the bony anatomy and the carina. Target coverage was visually inspected and marker position variation was quantified relative to both registration approaches; the variation of systematic (Σ) and random errors (σ) was estimated. Automatic carina-based registration was feasible in 94.9% of the CBCT scans, with an adequate target coverage in 91.1% compared to 100% after bony anatomy-based registration. Overall, Σ (σ) in the LR/CC/AP direction was 2.9(2.4)/4.1(2.4)/2.2(1.8) mm using the bony anatomy registration compared to 3.3(3.0)/3.6(2.6)/3.9(3.1) mm for the carina. Mid-thoracic placed markers showed a non-significant but smaller Σ in CC and AP direction when using the carina-based registration. Compared with a bony anatomy-based registration, carina-based registration for esophageal cancer IGRT results in inadequate target coverage in 8.9% of cases. Furthermore, large Σ and σ, requiring larger anisotropic margins, were seen after carina-based registration. Only for tumors entirely confined to the mid-thoracic region the carina-based registration might be slightly favorable.

Overlay improvement by exposure map based mask registration optimization

Science.gov (United States)

Shi, Irene; Guo, Eric; Chen, Ming; Lu, Max; Li, Gordon; Li, Rivan; Tian, Eric

2015-03-01

Along with the increased miniaturization of semiconductor electronic devices, the design rules of advanced semiconductor devices shrink dramatically. [1] One of the main challenges of lithography step is the layer-to-layer overlay control. Furthermore, DPT (Double Patterning Technology) has been adapted for the advanced technology node like 28nm and 14nm, corresponding overlay budget becomes even tighter. [2][3] After the in-die mask registration (pattern placement) measurement is introduced, with the model analysis of a KLA SOV (sources of variation) tool, it's observed that registration difference between masks is a significant error source of wafer layer-to-layer overlay at 28nm process. [4][5] Mask registration optimization would highly improve wafer overlay performance accordingly. It was reported that a laser based registration control (RegC) process could be applied after the pattern generation or after pellicle mounting and allowed fine tuning of the mask registration. [6] In this paper we propose a novel method of mask registration correction, which can be applied before mask writing based on mask exposure map, considering the factors of mask chip layout, writing sequence, and pattern density distribution. Our experiment data show if pattern density on the mask keeps at a low level, in-die mask registration residue error in 3sigma could be always under 5nm whatever blank type and related writer POSCOR (position correction) file was applied; it proves random error induced by material or equipment would occupy relatively fixed error budget as an error source of mask registration. On the real production, comparing the mask registration difference through critical production layers, it could be revealed that registration residue error of line space layers with higher pattern density is always much larger than the one of contact hole layers with lower pattern density. Additionally, the mask registration difference between layers with similar pattern density

Automated landmark-guided deformable image registration.

Science.gov (United States)

Kearney, Vasant; Chen, Susie; Gu, Xuejun; Chiu, Tsuicheng; Liu, Honghuan; Jiang, Lan; Wang, Jing; Yordy, John; Nedzi, Lucien; Mao, Weihua

2015-01-07

The purpose of this work is to develop an automated landmark-guided deformable image registration (LDIR) algorithm between the planning CT and daily cone-beam CT (CBCT) with low image quality. This method uses an automated landmark generation algorithm in conjunction with a local small volume gradient matching search engine to map corresponding landmarks between the CBCT and the planning CT. The landmarks act as stabilizing control points in the following Demons deformable image registration. LDIR is implemented on graphics processing units (GPUs) for parallel computation to achieve ultra fast calculation. The accuracy of the LDIR algorithm has been evaluated on a synthetic case in the presence of different noise levels and data of six head and neck cancer patients. The results indicate that LDIR performed better than rigid registration, Demons, and intensity corrected Demons for all similarity metrics used. In conclusion, LDIR achieves high accuracy in the presence of multimodality intensity mismatch and CBCT noise contamination, while simultaneously preserving high computational efficiency.

Automated landmark-guided deformable image registration

International Nuclear Information System (INIS)

Kearney, Vasant; Chen, Susie; Gu, Xuejun; Chiu, Tsuicheng; Liu, Honghuan; Jiang, Lan; Wang, Jing; Yordy, John; Nedzi, Lucien; Mao, Weihua

2015-01-01

The purpose of this work is to develop an automated landmark-guided deformable image registration (LDIR) algorithm between the planning CT and daily cone-beam CT (CBCT) with low image quality. This method uses an automated landmark generation algorithm in conjunction with a local small volume gradient matching search engine to map corresponding landmarks between the CBCT and the planning CT. The landmarks act as stabilizing control points in the following Demons deformable image registration. LDIR is implemented on graphics processing units (GPUs) for parallel computation to achieve ultra fast calculation. The accuracy of the LDIR algorithm has been evaluated on a synthetic case in the presence of different noise levels and data of six head and neck cancer patients. The results indicate that LDIR performed better than rigid registration, Demons, and intensity corrected Demons for all similarity metrics used. In conclusion, LDIR achieves high accuracy in the presence of multimodality intensity mismatch and CBCT noise contamination, while simultaneously preserving high computational efficiency. (paper)

An agent-based architecture for multimodal interaction

NARCIS (Netherlands)

Jonker, C.M.; Treur, J.; Wijngaards, W.C.A.

In this paper, an executable generic process model is proposed for combined verbal and non-verbal communication processes and their interaction. The agent-based architecture can be used to create multimodal interaction. The generic process model has been designed, implemented and used to simulate

An agent-based architecture for multimodal interaction

NARCIS (Netherlands)

Jonker, C.M.; Treur, J.; Wijngaards, W.C.A.

2001-01-01

In this paper, an executable generic process model is proposed for combined verbal and non-verbal communication processes and their interaction. The agent-based architecture can be used to create multimodal interaction. The generic process model has been designed, implemented and used to simulate

Multi-modal imaging, model-based tracking, and mixed reality visualisation for orthopaedic surgery

Science.gov (United States)

Fuerst, Bernhard; Tateno, Keisuke; Johnson, Alex; Fotouhi, Javad; Osgood, Greg; Tombari, Federico; Navab, Nassir

2017-01-01

Orthopaedic surgeons are still following the decades old workflow of using dozens of two-dimensional fluoroscopic images to drill through complex 3D structures, e.g. pelvis. This Letter presents a mixed reality support system, which incorporates multi-modal data fusion and model-based surgical tool tracking for creating a mixed reality environment supporting screw placement in orthopaedic surgery. A red–green–blue–depth camera is rigidly attached to a mobile C-arm and is calibrated to the cone-beam computed tomography (CBCT) imaging space via iterative closest point algorithm. This allows real-time automatic fusion of reconstructed surface and/or 3D point clouds and synthetic fluoroscopic images obtained through CBCT imaging. An adapted 3D model-based tracking algorithm with automatic tool segmentation allows for tracking of the surgical tools occluded by hand. This proposed interactive 3D mixed reality environment provides an intuitive understanding of the surgical site and supports surgeons in quickly localising the entry point and orienting the surgical tool during screw placement. The authors validate the augmentation by measuring target registration error and also evaluate the tracking accuracy in the presence of partial occlusion. PMID:29184659

Evaluation of whole-body MR to CT deformable image registration.

Science.gov (United States)

Akbarzadeh, A; Gutierrez, D; Baskin, A; Ay, M R; Ahmadian, A; Riahi Alam, N; Lövblad, K O; Zaidi, H

2013-07-08

Multimodality image registration plays a crucial role in various clinical and research applications. The aim of this study is to present an optimized MR to CT whole-body deformable image registration algorithm and its validation using clinical studies. A 3D intermodality registration technique based on B-spline transformation was performed using optimized parameters of the elastix package based on the Insight Toolkit (ITK) framework. Twenty-eight (17 male and 11 female) clinical studies were used in this work. The registration was evaluated using anatomical landmarks and segmented organs. In addition to 16 anatomical landmarks, three key organs (brain, lungs, and kidneys) and the entire body volume were segmented for evaluation. Several parameters--such as the Euclidean distance between anatomical landmarks, target overlap, Dice and Jaccard coefficients, false positives and false negatives, volume similarity, distance error, and Hausdorff distance--were calculated to quantify the quality of the registration algorithm. Dice coefficients for the majority of patients (> 75%) were in the 0.8-1 range for the whole body, brain, and lungs, which satisfies the criteria to achieve excellent alignment. On the other hand, for kidneys, Dice coefficients for volumes of 25% of the patients meet excellent volume agreement requirement, while the majority of patients satisfy good agreement criteria (> 0.6). For all patients, the distance error was in 0-10 mm range for all segmented organs. In summary, we optimized and evaluated the accuracy of an MR to CT deformable registration algorithm. The registered images constitute a useful 3D whole-body MR-CT atlas suitable for the development and evaluation of novel MR-guided attenuation correction procedures on hybrid PET-MR systems.

TU-CD-BRA-01: A Novel 3D Registration Method for Multiparametric Radiological Images

International Nuclear Information System (INIS)

Akhbardeh, A; Parekth, VS; Jacobs, MA

2015-01-01

Purpose: Multiparametric and multimodality radiological imaging methods, such as, magnetic resonance imaging(MRI), computed tomography(CT), and positron emission tomography(PET), provide multiple types of tissue contrast and anatomical information for clinical diagnosis. However, these radiological modalities are acquired using very different technical parameters, e.g.,field of view(FOV), matrix size, and scan planes, which, can lead to challenges in registering the different data sets. Therefore, we developed a hybrid registration method based on 3D wavelet transformation and 3D interpolations that performs 3D resampling and rotation of the target radiological images without loss of information Methods: T1-weighted, T2-weighted, diffusion-weighted-imaging(DWI), dynamic-contrast-enhanced(DCE) MRI and PET/CT were used in the registration algorithm from breast and prostate data at 3T MRI and multimodality(PET/CT) cases. The hybrid registration scheme consists of several steps to reslice and match each modality using a combination of 3D wavelets, interpolations, and affine registration steps. First, orthogonal reslicing is performed to equalize FOV, matrix sizes and the number of slices using wavelet transformation. Second, angular resampling of the target data is performed to match the reference data. Finally, using optimized angles from resampling, 3D registration is performed using similarity transformation(scaling and translation) between the reference and resliced target volume is performed. After registration, the mean-square-error(MSE) and Dice Similarity(DS) between the reference and registered target volumes were calculated. Results: The 3D registration method registered synthetic and clinical data with significant improvement(p<0.05) of overlap between anatomical structures. After transforming and deforming the synthetic data, the MSE and Dice similarity were 0.12 and 0.99. The average improvement of the MSE in breast was 62%(0.27 to 0.10) and prostate was

Graph-based surface extraction of the liver using locally adaptive priors for multimodal interventional image registration

NARCIS (Netherlands)

Kadoury, S.; Wood, B.; Venkatesan, A.; Ardon, R.; Jago, J.; Kruecker, J.

2011-01-01

The 3D fusion of tracked ultrasound with a diagnostic CT image has multiple benefits in a variety of interventional applications for oncology. Still, manual registration is a considerable drawback to theclinical workflow and hinders the widespread clinical adoption of this technique. In this paper,

Determination of optimal ultrasound planes for the initialisation of image registration during endoscopic ultrasound-guided procedures.

Science.gov (United States)

Bonmati, Ester; Hu, Yipeng; Gibson, Eli; Uribarri, Laura; Keane, Geri; Gurusami, Kurinchi; Davidson, Brian; Pereira, Stephen P; Clarkson, Matthew J; Barratt, Dean C

2018-06-01

Navigation of endoscopic ultrasound (EUS)-guided procedures of the upper gastrointestinal (GI) system can be technically challenging due to the small fields-of-view of ultrasound and optical devices, as well as the anatomical variability and limited number of orienting landmarks during navigation. Co-registration of an EUS device and a pre-procedure 3D image can enhance the ability to navigate. However, the fidelity of this contextual information depends on the accuracy of registration. The purpose of this study was to develop and test the feasibility of a simulation-based planning method for pre-selecting patient-specific EUS-visible anatomical landmark locations to maximise the accuracy and robustness of a feature-based multimodality registration method. A registration approach was adopted in which landmarks are registered to anatomical structures segmented from the pre-procedure volume. The predicted target registration errors (TREs) of EUS-CT registration were estimated using simulated visible anatomical landmarks and a Monte Carlo simulation of landmark localisation error. The optimal planes were selected based on the 90th percentile of TREs, which provide a robust and more accurate EUS-CT registration initialisation. The method was evaluated by comparing the accuracy and robustness of registrations initialised using optimised planes versus non-optimised planes using manually segmented CT images and simulated ([Formula: see text]) or retrospective clinical ([Formula: see text]) EUS landmarks. The results show a lower 90th percentile TRE when registration is initialised using the optimised planes compared with a non-optimised initialisation approach (p value [Formula: see text]). The proposed simulation-based method to find optimised EUS planes and landmarks for EUS-guided procedures may have the potential to improve registration accuracy. Further work will investigate applying the technique in a clinical setting.

Relationship between pre-reconstruction filter and accuracy of registration software based on mutual-information maximization. A study of SPECT-MR brain phantom images

International Nuclear Information System (INIS)

Mito, Suzuko; Magota, Keiichi; Arai, Hiroshi; Omote, Hidehiko; Katsuura, Hidenori; Suzuki, Kotaro; Kubo Naoki

2005-01-01

Image registration technique is becoming an increasingly important tool in SPECT. Recently, software based on mutual-information maximization has been developed for automatic multimodality image registration. The accuracy of the software is important for its application to image registration. During SPECT reconstruction, the projection data are pre-filtered in order to reduce Poisson noise, commonly using a Butterworth filter. We have investigated the dependence of the absolute accuracy of MRI-SPECT registration on the cut-off frequencies of a range of Butterworth filters. This study used a 3D Hoffman phantom (Model No. 9000, Data-spectrum Co.). For the reference volume, an magnetization prepared rapid gradient echo (MPRage) sequence was performed on a Vision MRI (Siemence, 1.5 T). For the floating volumes, SPECT data of a phantom including 99m Tc 85 kBq/mL were acquired by a GCA-9300 (Toshiba Medical Systems Co.). During SPECT, the orbito-meatal (OM) line of the phantom was tilted by 5 deg and 15 deg to mimic the incline of a patient's head. The projection data were pre-filtered with Butterworth filters (cut-off frequency varying between 0.24 to 0.94 cycles/cm in 0.02 steps, order 8). The automated registrations were performed using iNRT β version software (Nihon Medi. Co.) and the rotation angles of SPECT for registration were noted. In this study, the registrations of all SPECT data were successful. Graphs of registration rotation angles against cut-off frequencies were scattered and showed no correlation between the two. The registration rotation angles ranged with changing cut-off frequency from -0.4 deg to +3.8 deg at a 5 deg tilt and from +12.7 deg to +19.6 deg at a 15 deg tilt. The registration rotation angles showed variation even for slight differences in cut-off frequencies. The absolute errors were a few degrees for any cut-off frequency. Regardless of the cut-off frequency, automatic registration using this software provides similar results. (author)

A Gradient-Based Multistart Algorithm for Multimodal Aerodynamic Shape Optimization Problems Based on Free-Form Deformation

Science.gov (United States)

Streuber, Gregg Mitchell

Environmental and economic factors motivate the pursuit of more fuel-efficient aircraft designs. Aerodynamic shape optimization is a powerful tool in this effort, but is hampered by the presence of multimodality in many design spaces. Gradient-based multistart optimization uses a sampling algorithm and multiple parallel optimizations to reliably apply fast gradient-based optimization to moderately multimodal problems. Ensuring that the sampled geometries remain physically realizable requires manually developing specialized linear constraints for each class of problem. Utilizing free-form deformation geometry control allows these linear constraints to be written in a geometry-independent fashion, greatly easing the process of applying the algorithm to new problems. This algorithm was used to assess the presence of multimodality when optimizing a wing in subsonic and transonic flows, under inviscid and viscous conditions, and a blended wing-body under transonic, viscous conditions. Multimodality was present in every wing case, while the blended wing-body was found to be generally unimodal.

Development and application of pulmonary structure-function registration methods: towards pulmonary image-guidance tools for improved airway targeted therapies and outcomes

Science.gov (United States)

Guo, Fumin; Pike, Damien; Svenningsen, Sarah; Coxson, Harvey O.; Drozd, John J.; Yuan, Jing; Fenster, Aaron; Parraga, Grace

2014-03-01

Objectives: We aimed to develop a way to rapidly generate multi-modality (MRI-CT) pulmonary imaging structurefunction maps using novel non-rigid image registration methods. This objective is part of our overarching goal to provide an image processing pipeline to generate pulmonary structure-function maps and guide airway-targeted therapies. Methods: Anatomical 1H and functional 3He MRI were acquired in 5 healthy asymptomatic ex-smokers and 7 ex-smokers with chronic obstructive pulmonary disease (COPD) at inspiration breath-hold. Thoracic CT was performed within ten minutes of MRI using the same breath-hold volume. Landmark-based affine registration methods previously validated for imaging of COPD, was based on corresponding fiducial markers located in both CT and 1H MRI coronal slices and compared with shape-based CT-MRI non-rigid registration. Shape-based CT-MRI registration was developed by first identifying the shapes of the lung cavities manually, and then registering the two shapes using affine and thin-plate spline algorithms. We compared registration accuracy using the fiducial localization error (FLE) and target registration error (TRE). Results: For landmark-based registration, the TRE was 8.4±5.3 mm for whole lung and 7.8±4.6 mm for the R and L lungs registered independently (p=0.4). For shape-based registration, the TRE was 8.0±4.6 mm for whole lung as compared to 6.9±4.4 mm for the R and L lung registered independently and this difference was significant (p=0.01). The difference for shape-based (6.9±4.4 mm) and landmark-based R and L lung registration (7.8±4.6 mm) was also significant (p=.04) Conclusion: Shape-based registration TRE was significantly improved compared to landmark-based registration when considering L and R lungs independently.

A Remote Registration Based on MIDAS

Science.gov (United States)

JIN, Xin

2017-04-01

We often need for software registration to protect the interests of the software developers. This article narrated one kind of software long-distance registration technology. The registration method is: place the registration information in a database table, after the procedure starts in check table registration information, if it has registered then the procedure may the normal operation; Otherwise, the customer must input the sequence number and registers through the network on the long-distance server. If it registers successfully, then records the registration information in the database table. This remote registration method can protect the rights of software developers.

Feature-based Alignment of Volumetric Multi-modal Images

Science.gov (United States)

Toews, Matthew; Zöllei, Lilla; Wells, William M.

2014-01-01

This paper proposes a method for aligning image volumes acquired from different imaging modalities (e.g. MR, CT) based on 3D scale-invariant image features. A novel method for encoding invariant feature geometry and appearance is developed, based on the assumption of locally linear intensity relationships, providing a solution to poor repeatability of feature detection in different image modalities. The encoding method is incorporated into a probabilistic feature-based model for multi-modal image alignment. The model parameters are estimated via a group-wise alignment algorithm, that iteratively alternates between estimating a feature-based model from feature data, then realigning feature data to the model, converging to a stable alignment solution with few pre-processing or pre-alignment requirements. The resulting model can be used to align multi-modal image data with the benefits of invariant feature correspondence: globally optimal solutions, high efficiency and low memory usage. The method is tested on the difficult RIRE data set of CT, T1, T2, PD and MP-RAGE brain images of subjects exhibiting significant inter-subject variability due to pathology. PMID:24683955

Evaluation of whole‐body MR to CT deformable image registration

Science.gov (United States)

Akbarzadeh, A.; Gutierrez, D.; Baskin, A.; Ay, M.R.; Ahmadian, A.; Alam, N. Riahi; Lövblad, KO

2013-01-01

Multimodality image registration plays a crucial role in various clinical and research applications. The aim of this study is to present an optimized MR to CT whole‐body deformable image registration algorithm and its validation using clinical studies. A 3D intermodality registration technique based on B‐spline transformation was performed using optimized parameters of the elastix package based on the Insight Toolkit (ITK) framework. Twenty‐eight (17 male and 11 female) clinical studies were used in this work. The registration was evaluated using anatomical landmarks and segmented organs. In addition to 16 anatomical landmarks, three key organs (brain, lungs, and kidneys) and the entire body volume were segmented for evaluation. Several parameters — such as the Euclidean distance between anatomical landmarks, target overlap, Dice and Jaccard coefficients, false positives and false negatives, volume similarity, distance error, and Hausdorff distance — were calculated to quantify the quality of the registration algorithm. Dice coefficients for the majority of patients (>75%) were in the 0.8–1 range for the whole body, brain, and lungs, which satisfies the criteria to achieve excellent alignment. On the other hand, for kidneys, Dice coefficients for volumes of 25% of the patients meet excellent volume agreement requirement, while the majority of patients satisfy good agreement criteria (>0.6). For all patients, the distance error was in 0–10 mm range for all segmented organs. In summary, we optimized and evaluated the accuracy of an MR to CT deformable registration algorithm. The registered images constitute a useful 3D whole‐body MR‐CT atlas suitable for the development and evaluation of novel MR‐guided attenuation correction procedures on hybrid PET‐MR systems. PACS number: 07.05.Pj PMID:23835382

Improving treatment planning accuracy through multimodality imaging

International Nuclear Information System (INIS)

Sailer, Scott L.; Rosenman, Julian G.; Soltys, Mitchel; Cullip, Tim J.; Chen, Jun

1996-01-01

Purpose: In clinical practice, physicians are constantly comparing multiple images taken at various times during the patient's treatment course. One goal of such a comparison is to accurately define the gross tumor volume (GTV). The introduction of three-dimensional treatment planning has greatly enhanced the ability to define the GTV, but there are times when the GTV is not visible on the treatment-planning computed tomography (CT) scan. We have modified our treatment-planning software to allow for interactive display of multiple, registered images that enhance the physician's ability to accurately determine the GTV. Methods and Materials: Images are registered using interactive tools developed at the University of North Carolina at Chapel Hill (UNC). Automated methods are also available. Images registered with the treatment-planning CT scan are digitized from film. After a physician has approved the registration, the registered images are made available to the treatment-planning software. Structures and volumes of interest are contoured on all images. In the beam's eye view, wire loop representations of these structures can be visualized from all image types simultaneously. Each registered image can be seamlessly viewed during the treatment-planning process, and all contours from all image types can be seen on any registered image. A beam may, therefore, be designed based on any contour. Results: Nineteen patients have been planned and treated using multimodality imaging from November 1993 through August 1994. All registered images were digitized from film, and many were from outside institutions. Brain has been the most common site (12), but the techniques of registration and image display have also been used for the thorax (4), abdomen (2), and extremity (1). The registered image has been an magnetic resonance (MR) scan in 15 cases and a diagnostic CT scan in 5 cases. In one case, sequential MRs, one before treatment and another after 30 Gy, were used to plan

Evaluation of multi-modality CT-MRI-SPECT registration tools for radiotherapy treatment planning purposes

International Nuclear Information System (INIS)

Bianchini, S.; Alfonso, R.; Castillo, J.; Coca, M.; Torres, L.

2013-01-01

A qualitative and quantitative comparison of registration CT-CT, CT-MR and CT-SPECT performed by the different software and algorithms studies is presented. Only two studied software were full DICOM RT compatible while accepting DICOM images in any layout. Quantitative results of fiducial displacement errors were calculated for all software and available registration methods. The presented methodology demonstrated being effective for assessing the quality of studied image registration tools in the radiotherapy planning context, provided the images are free of significant geometric deformation. When implementing this methodology in real patients, the use of immobilization devices, such as thermoplastic masks, is recommended for enhanced quality of image registration. (Author)

Multi-Modality Cascaded Convolutional Neural Networks for Alzheimer's Disease Diagnosis.

Science.gov (United States)

Liu, Manhua; Cheng, Danni; Wang, Kundong; Wang, Yaping

2018-03-23

Accurate and early diagnosis of Alzheimer's disease (AD) plays important role for patient care and development of future treatment. Structural and functional neuroimages, such as magnetic resonance images (MRI) and positron emission tomography (PET), are providing powerful imaging modalities to help understand the anatomical and functional neural changes related to AD. In recent years, machine learning methods have been widely studied on analysis of multi-modality neuroimages for quantitative evaluation and computer-aided-diagnosis (CAD) of AD. Most existing methods extract the hand-craft imaging features after image preprocessing such as registration and segmentation, and then train a classifier to distinguish AD subjects from other groups. This paper proposes to construct cascaded convolutional neural networks (CNNs) to learn the multi-level and multimodal features of MRI and PET brain images for AD classification. First, multiple deep 3D-CNNs are constructed on different local image patches to transform the local brain image into more compact high-level features. Then, an upper high-level 2D-CNN followed by softmax layer is cascaded to ensemble the high-level features learned from the multi-modality and generate the latent multimodal correlation features of the corresponding image patches for classification task. Finally, these learned features are combined by a fully connected layer followed by softmax layer for AD classification. The proposed method can automatically learn the generic multi-level and multimodal features from multiple imaging modalities for classification, which are robust to the scale and rotation variations to some extent. No image segmentation and rigid registration are required in pre-processing the brain images. Our method is evaluated on the baseline MRI and PET images of 397 subjects including 93 AD patients, 204 mild cognitive impairment (MCI, 76 pMCI +128 sMCI) and 100 normal controls (NC) from Alzheimer's Disease Neuroimaging

Quality assurance of CT-PET alignment and image registration for radiation treatment planning

International Nuclear Information System (INIS)

Gong, S.J.; O'Keefe, G.J.; Gunawardana, D.H.

2005-01-01

A multi-layer point source phantom was first used to calibrate and verify the CT-PET system alignment. A partial whole-body Aldcrson RANDO Man Phantom (head through mid-femur) was externally and internally marked with small metal cannulas filled with 18F-FDG and then scanned with both modalities. Six series of phantom studies with different acquisition settings and scan positions were performed to reveal possible system bias and evaluate the accuracy and reliabilities of Philips Syntegra program in image alignment, coregistration and fusion. The registration error was assessed quantitatively by measuring the root-mean-square distance between the iso-centers of corresponding fiducial marker geometries in reference CT volumes and transformed CT or PET volumes. Results: Experimental data confirms the accuracy of manual, parameter, point and image-based registration using Syntegra is better than 2 mm. Comparisons between blind and cross definition of iso-centers of fiducial marks indicate that the fused CT and PET is superior to visual correlation of CT and PET side-by-side. Conclusion: In this work we demonstrate the QA procedures of Gemini image alignment and registration. Syntegra produces intrinsic and robust multi-modality image registration and fusion with careful user interaction. The registration accuracy is generally better than the spatial resolution of the PET scanner used and this appears to be sufficient for most RTP CT-PET registration procedures

Edge-based correlation image registration for multispectral imaging

Science.gov (United States)

Nandy, Prabal [Albuquerque, NM

2009-11-17

Registration information for images of a common target obtained from a plurality of different spectral bands can be obtained by combining edge detection and phase correlation. The images are edge-filtered, and pairs of the edge-filtered images are then phase correlated to produce phase correlation images. The registration information can be determined based on these phase correlation images.

FEM-based evaluation of deformable image registration for radiation therapy

International Nuclear Information System (INIS)

Zhong Hualiang; Peters, Terry; Siebers, Jeffrey V

2007-01-01

This paper presents a new concept to automatically detect the neighborhood in an image where deformable registration is mis-performing. Specifically, the displacement vector field (DVF) from a deformable image registration is substituted into a finite-element-based elastic framework to calculate unbalanced energy in each element. The value of the derived energy indicates the quality of the DVF in its neighborhood. The new voxel-based evaluation approach is compared with three other validation criteria: landmark measurement, a finite element approach and visual comparison, for deformable registrations performed with the optical-flow-based 'demons' algorithm as well as thin-plate spline interpolation. This analysis was performed on three pairs of prostate CT images. The results of the analysis show that the four criteria give mutually comparable quantitative assessments on the six registration instances. As an objective concept, the unbalanced energy presents no requirement on boundary constraints in its calculation, different from traditional mechanical modeling. This method is automatic, and at voxel level suitable to evaluate deformable registration in a clinical setting

elastix: a toolbox for intensity-based medical image registration.

Science.gov (United States)

Klein, Stefan; Staring, Marius; Murphy, Keelin; Viergever, Max A; Pluim, Josien P W

2010-01-01

Medical image registration is an important task in medical image processing. It refers to the process of aligning data sets, possibly from different modalities (e.g., magnetic resonance and computed tomography), different time points (e.g., follow-up scans), and/or different subjects (in case of population studies). A large number of methods for image registration are described in the literature. Unfortunately, there is not one method that works for all applications. We have therefore developed elastix, a publicly available computer program for intensity-based medical image registration. The software consists of a collection of algorithms that are commonly used to solve medical image registration problems. The modular design of elastix allows the user to quickly configure, test, and compare different registration methods for a specific application. The command-line interface enables automated processing of large numbers of data sets, by means of scripting. The usage of elastix for comparing different registration methods is illustrated with three example experiments, in which individual components of the registration method are varied.

Palmprint and face multi-modal biometric recognition based on SDA-GSVD and its kernelization.

Science.gov (United States)

Jing, Xiao-Yuan; Li, Sheng; Li, Wen-Qian; Yao, Yong-Fang; Lan, Chao; Lu, Jia-Sen; Yang, Jing-Yu

2012-01-01

When extracting discriminative features from multimodal data, current methods rarely concern themselves with the data distribution. In this paper, we present an assumption that is consistent with the viewpoint of discrimination, that is, a person's overall biometric data should be regarded as one class in the input space, and his different biometric data can form different Gaussians distributions, i.e., different subclasses. Hence, we propose a novel multimodal feature extraction and recognition approach based on subclass discriminant analysis (SDA). Specifically, one person's different bio-data are treated as different subclasses of one class, and a transformed space is calculated, where the difference among subclasses belonging to different persons is maximized, and the difference within each subclass is minimized. Then, the obtained multimodal features are used for classification. Two solutions are presented to overcome the singularity problem encountered in calculation, which are using PCA preprocessing, and employing the generalized singular value decomposition (GSVD) technique, respectively. Further, we provide nonlinear extensions of SDA based multimodal feature extraction, that is, the feature fusion based on KPCA-SDA and KSDA-GSVD. In KPCA-SDA, we first apply Kernel PCA on each single modal before performing SDA. While in KSDA-GSVD, we directly perform Kernel SDA to fuse multimodal data by applying GSVD to avoid the singular problem. For simplicity two typical types of biometric data are considered in this paper, i.e., palmprint data and face data. Compared with several representative multimodal biometrics recognition methods, experimental results show that our approaches outperform related multimodal recognition methods and KSDA-GSVD achieves the best recognition performance.

Palmprint and Face Multi-Modal Biometric Recognition Based on SDA-GSVD and Its Kernelization

Science.gov (United States)

Jing, Xiao-Yuan; Li, Sheng; Li, Wen-Qian; Yao, Yong-Fang; Lan, Chao; Lu, Jia-Sen; Yang, Jing-Yu

2012-01-01

When extracting discriminative features from multimodal data, current methods rarely concern themselves with the data distribution. In this paper, we present an assumption that is consistent with the viewpoint of discrimination, that is, a person's overall biometric data should be regarded as one class in the input space, and his different biometric data can form different Gaussians distributions, i.e., different subclasses. Hence, we propose a novel multimodal feature extraction and recognition approach based on subclass discriminant analysis (SDA). Specifically, one person's different bio-data are treated as different subclasses of one class, and a transformed space is calculated, where the difference among subclasses belonging to different persons is maximized, and the difference within each subclass is minimized. Then, the obtained multimodal features are used for classification. Two solutions are presented to overcome the singularity problem encountered in calculation, which are using PCA preprocessing, and employing the generalized singular value decomposition (GSVD) technique, respectively. Further, we provide nonlinear extensions of SDA based multimodal feature extraction, that is, the feature fusion based on KPCA-SDA and KSDA-GSVD. In KPCA-SDA, we first apply Kernel PCA on each single modal before performing SDA. While in KSDA-GSVD, we directly perform Kernel SDA to fuse multimodal data by applying GSVD to avoid the singular problem. For simplicity two typical types of biometric data are considered in this paper, i.e., palmprint data and face data. Compared with several representative multimodal biometrics recognition methods, experimental results show that our approaches outperform related multimodal recognition methods and KSDA-GSVD achieves the best recognition performance. PMID:22778600

Bi-objective optimization for multi-modal transportation routing planning problem based on Pareto optimality

Directory of Open Access Journals (Sweden)

Yan Sun

2015-09-01

Full Text Available Purpose: The purpose of study is to solve the multi-modal transportation routing planning problem that aims to select an optimal route to move a consignment of goods from its origin to its destination through the multi-modal transportation network. And the optimization is from two viewpoints including cost and time. Design/methodology/approach: In this study, a bi-objective mixed integer linear programming model is proposed to optimize the multi-modal transportation routing planning problem. Minimizing the total transportation cost and the total transportation time are set as the optimization objectives of the model. In order to balance the benefit between the two objectives, Pareto optimality is utilized to solve the model by gaining its Pareto frontier. The Pareto frontier of the model can provide the multi-modal transportation operator (MTO and customers with better decision support and it is gained by the normalized normal constraint method. Then, an experimental case study is designed to verify the feasibility of the model and Pareto optimality by using the mathematical programming software Lingo. Finally, the sensitivity analysis of the demand and supply in the multi-modal transportation organization is performed based on the designed case. Findings: The calculation results indicate that the proposed model and Pareto optimality have good performance in dealing with the bi-objective optimization. The sensitivity analysis also shows the influence of the variation of the demand and supply on the multi-modal transportation organization clearly. Therefore, this method can be further promoted to the practice. Originality/value: A bi-objective mixed integer linear programming model is proposed to optimize the multi-modal transportation routing planning problem. The Pareto frontier based sensitivity analysis of the demand and supply in the multi-modal transportation organization is performed based on the designed case.

Modification of species-based differential evolution for multimodal optimization

Science.gov (United States)

Idrus, Said Iskandar Al; Syahputra, Hermawan; Firdaus, Muliawan

2015-12-01

At this time optimization has an important role in various fields as well as between other operational research, industry, finance and management. Optimization problem is the problem of maximizing or minimizing a function of one variable or many variables, which include unimodal and multimodal functions. Differential Evolution (DE), is a random search technique using vectors as an alternative solution in the search for the optimum. To localize all local maximum and minimum on multimodal function, this function can be divided into several domain of fitness using niching method. Species-based niching method is one of method that build sub-populations or species in the domain functions. This paper describes the modification of species-based previously to reduce the computational complexity and run more efficiently. The results of the test functions show species-based modifications able to locate all the local optima in once run the program.

Palmprint and Face Multi-Modal Biometric Recognition Based on SDA-GSVD and Its Kernelization

Directory of Open Access Journals (Sweden)

Jing-Yu Yang

2012-04-01

Full Text Available When extracting discriminative features from multimodal data, current methods rarely concern themselves with the data distribution. In this paper, we present an assumption that is consistent with the viewpoint of discrimination, that is, a person’s overall biometric data should be regarded as one class in the input space, and his different biometric data can form different Gaussians distributions, i.e., different subclasses. Hence, we propose a novel multimodal feature extraction and recognition approach based on subclass discriminant analysis (SDA. Specifically, one person’s different bio-data are treated as different subclasses of one class, and a transformed space is calculated, where the difference among subclasses belonging to different persons is maximized, and the difference within each subclass is minimized. Then, the obtained multimodal features are used for classification. Two solutions are presented to overcome the singularity problem encountered in calculation, which are using PCA preprocessing, and employing the generalized singular value decomposition (GSVD technique, respectively. Further, we provide nonlinear extensions of SDA based multimodal feature extraction, that is, the feature fusion based on KPCA-SDA and KSDA-GSVD. In KPCA-SDA, we first apply Kernel PCA on each single modal before performing SDA. While in KSDA-GSVD, we directly perform Kernel SDA to fuse multimodal data by applying GSVD to avoid the singular problem. For simplicity two typical types of biometric data are considered in this paper, i.e., palmprint data and face data. Compared with several representative multimodal biometrics recognition methods, experimental results show that our approaches outperform related multimodal recognition methods and KSDA-GSVD achieves the best recognition performance.

Free-Form Deformation Approach for Registration of Visible and Infrared Facial Images in Fever Screening

Directory of Open Access Journals (Sweden)

Yedukondala Narendra Dwith Chenna

2018-01-01

Full Text Available Fever screening based on infrared (IR thermographs (IRTs is an approach that has been implemented during infectious disease pandemics, such as Ebola and Severe Acute Respiratory Syndrome. A recently published international standard indicates that regions medially adjacent to the inner canthi provide accurate estimates of core body temperature and are preferred sites for fever screening. Therefore, rapid, automated identification of the canthi regions within facial IR images may greatly facilitate rapid fever screening of asymptomatic travelers. However, it is more difficult to accurately identify the canthi regions from IR images than from visible images that are rich with exploitable features. In this study, we developed and evaluated techniques for multi-modality image registration (MMIR of simultaneously captured visible and IR facial images for fever screening. We used free form deformation (FFD models based on edge maps to improve registration accuracy after an affine transformation. Two widely used FFD models in medical image registration based on the Demons and cubic B-spline algorithms were qualitatively compared. The results showed that the Demons algorithm outperformed the cubic B-spline algorithm, likely due to overfitting of outliers by the latter method. The quantitative measure of registration accuracy, obtained through selected control point correspondence, was within 2.8 ± 1.2 mm, which enables accurate and automatic localization of canthi regions in the IR images for temperature measurement.

Outcome of transarterial chemoembolization-based multi-modal treatment in patients with unresectable hepatocellular carcinoma.

Science.gov (United States)

Song, Do Seon; Nam, Soon Woo; Bae, Si Hyun; Kim, Jin Dong; Jang, Jeong Won; Song, Myeong Jun; Lee, Sung Won; Kim, Hee Yeon; Lee, Young Joon; Chun, Ho Jong; You, Young Kyoung; Choi, Jong Young; Yoon, Seung Kew

2015-02-28

To investigate the efficacy and safety of transarterial chemoembolization (TACE)-based multimodal treatment in patients with large hepatocellular carcinoma (HCC). A total of 146 consecutive patients were included in the analysis, and their medical records and radiological data were reviewed retrospectively. In total, 119 patients received TACE-based multi-modal treatments, and the remaining 27 received conservative management. Overall survival (P<0.001) and objective tumor response (P=0.003) were significantly better in the treatment group than in the conservative group. After subgroup analysis, survival benefits were observed not only in the multi-modal treatment group compared with the TACE-only group (P=0.002) but also in the surgical treatment group compared with the loco-regional treatment-only group (P<0.001). Multivariate analysis identified tumor stage (P<0.001) and tumor type (P=0.009) as two independent pre-treatment factors for survival. After adjusting for significant pre-treatment prognostic factors, objective response (P<0.001), surgical treatment (P=0.009), and multi-modal treatment (P=0.002) were identified as independent post-treatment prognostic factors. TACE-based multi-modal treatments were safe and more beneficial than conservative management. Salvage surgery after successful downstaging resulted in long-term survival in patients with large, unresectable HCC.

3D ultrasound-CT registration of the liver using combined landmark-intensity information

International Nuclear Information System (INIS)

Lange, Thomas; Schlag, Peter M.; Papenberg, Nils; Heldmann, Stefan; Modersitzki, Jan; Fischer, Bernd; Lamecker, Hans

2009-01-01

An important issue in computer-assisted surgery of the liver is a fast and reliable transfer of preoperative resection plans to the intraoperative situation. One problem is to match the planning data, derived from preoperative CT or MR images, with 3D ultrasound images of the liver, acquired during surgery. As the liver deforms significantly in the intraoperative situation non-rigid registration is necessary. This is a particularly challenging task because pre- and intraoperative image data stem from different modalities and ultrasound images are generally very noisy. One way to overcome these problems is to incorporate prior knowledge into the registration process. We propose a method of combining anatomical landmark information with a fast non-parametric intensity registration approach. Mathematically, this leads to a constrained optimization problem. As distance measure we use the normalized gradient field which allows for multimodal image registration. A qualitative and quantitative validation on clinical liver data sets of three different patients has been performed. We used the distance of dense corresponding points on vessel center lines for quantitative validation. The combined landmark and intensity approach improves the mean and percentage of point distances above 3 mm compared to rigid and thin-plate spline registration based only on landmarks. The proposed algorithm offers the possibility to incorporate additional a priori knowledge - in terms of few landmarks - provided by a human expert into a non-rigid registration process. (orig.)

A spatiotemporal-based scheme for efficient registration-based segmentation of thoracic 4-D MRI.

Science.gov (United States)

Yang, Y; Van Reeth, E; Poh, C L; Tan, C H; Tham, I W K

2014-05-01

Dynamic three-dimensional (3-D) (four-dimensional, 4-D) magnetic resonance (MR) imaging is gaining importance in the study of pulmonary motion for respiratory diseases and pulmonary tumor motion for radiotherapy. To perform quantitative analysis using 4-D MR images, segmentation of anatomical structures such as the lung and pulmonary tumor is required. Manual segmentation of entire thoracic 4-D MRI data that typically contains many 3-D volumes acquired over several breathing cycles is extremely tedious, time consuming, and suffers high user variability. This requires the development of new automated segmentation schemes for 4-D MRI data segmentation. Registration-based segmentation technique that uses automatic registration methods for segmentation has been shown to be an accurate method to segment structures for 4-D data series. However, directly applying registration-based segmentation to segment 4-D MRI series lacks efficiency. Here we propose an automated 4-D registration-based segmentation scheme that is based on spatiotemporal information for the segmentation of thoracic 4-D MR lung images. The proposed scheme saved up to 95% of computation amount while achieving comparable accurate segmentations compared to directly applying registration-based segmentation to 4-D dataset. The scheme facilitates rapid 3-D/4-D visualization of the lung and tumor motion and potentially the tracking of tumor during radiation delivery.

Unified voxel- and tensor-based morphometry (UVTBM) using registration confidence.

Science.gov (United States)

Khan, Ali R; Wang, Lei; Beg, Mirza Faisal

2015-01-01

Voxel-based morphometry (VBM) and tensor-based morphometry (TBM) both rely on spatial normalization to a template and yet have different requirements for the level of registration accuracy. VBM requires only global alignment of brain structures, with limited degrees of freedom in transformation, whereas TBM performs best when the registration is highly deformable and can achieve higher registration accuracy. In addition, the registration accuracy varies over the whole brain, with higher accuracy typically observed in subcortical areas and lower accuracy seen in cortical areas. Hence, even the determinant of Jacobian of registration maps is spatially varying in their accuracy, and combining these with VBM by direct multiplication introduces errors in VBM maps where the registration is inaccurate. We propose a unified approach to combining these 2 morphometry methods that is motivated by these differing requirements for registration and our interest in harnessing the advantages of both. Our novel method uses local estimates of registration confidence to determine how to weight the influence of VBM- and TBM-like approaches. Results are shown on healthy and mild Alzheimer's subjects (N = 150) investigating age and group differences, and potential of differential diagnosis is shown on a set of Alzheimer's disease (N = 34) and frontotemporal dementia (N = 30) patients compared against controls (N = 14). These show that the group differences detected by our proposed approach are more descriptive than those detected from VBM, Jacobian-modulated VBM, and TBM separately, hence leveraging the advantages of both approaches in a unified framework. Copyright © 2015 Elsevier Inc. All rights reserved.

Alcohol sensor based on single-mode-multimode-single-mode fiber structure

Science.gov (United States)

Mefina Yulias, R.; Hatta, A. M.; Sekartedjo, Sekartedjo

2016-11-01

Alcohol sensor based on Single-mode -Multimode-Single-mode (SMS) fiber structure is being proposed to sense alcohol concentration in alcohol-water mixtures. This proposed sensor uses refractive index sensing as its sensing principle. Fabricated SMS fiber structure had 40 m of multimode length. With power input -6 dBm and wavelength 1550 nm, the proposed sensor showed good response with sensitivity 1,983 dB per % v/v with measurement range 05 % v/v and measurement span 0,5% v/v.

Inorganic Nanoparticles for Multimodal Molecular Imaging

Directory of Open Access Journals (Sweden)

Magdalena Swierczewska

2011-01-01

Full Text Available Multimodal molecular imaging can offer a synergistic improvement of diagnostic ability over a single imaging modality. Recent development of hybrid imaging systems has profoundly impacted the pool of available multimodal imaging probes. In particular, much interest has been focused on biocompatible, inorganic nanoparticle-based multimodal probes. Inorganic nanoparticles offer exceptional advantages to the field of multimodal imaging owing to their unique characteristics, such as nanometer dimensions, tunable imaging properties, and multifunctionality. Nanoparticles mainly based on iron oxide, quantum dots, gold, and silica have been applied to various imaging modalities to characterize and image specific biologic processes on a molecular level. A combination of nanoparticles and other materials such as biomolecules, polymers, and radiometals continue to increase functionality for in vivo multimodal imaging and therapeutic agents. In this review, we discuss the unique concepts, characteristics, and applications of the various multimodal imaging probes based on inorganic nanoparticles.

Fast time-of-flight camera based surface registration for radiotherapy patient positioning.

Science.gov (United States)

Placht, Simon; Stancanello, Joseph; Schaller, Christian; Balda, Michael; Angelopoulou, Elli

2012-01-01

This work introduces a rigid registration framework for patient positioning in radiotherapy, based on real-time surface acquisition by a time-of-flight (ToF) camera. Dynamic properties of the system are also investigated for future gating/tracking strategies. A novel preregistration algorithm, based on translation and rotation-invariant features representing surface structures, was developed. Using these features, corresponding three-dimensional points were computed in order to determine initial registration parameters. These parameters became a robust input to an accelerated version of the iterative closest point (ICP) algorithm for the fine-tuning of the registration result. Distance calibration and Kalman filtering were used to compensate for ToF-camera dependent noise. Additionally, the advantage of using the feature based preregistration over an "ICP only" strategy was evaluated, as well as the robustness of the rigid-transformation-based method to deformation. The proposed surface registration method was validated using phantom data. A mean target registration error (TRE) for translations and rotations of 1.62 ± 1.08 mm and 0.07° ± 0.05°, respectively, was achieved. There was a temporal delay of about 65 ms in the registration output, which can be seen as negligible considering the dynamics of biological systems. Feature based preregistration allowed for accurate and robust registrations even at very large initial displacements. Deformations affected the accuracy of the results, necessitating particular care in cases of deformed surfaces. The proposed solution is able to solve surface registration problems with an accuracy suitable for radiotherapy cases where external surfaces offer primary or complementary information to patient positioning. The system shows promising dynamic properties for its use in gating/tracking applications. The overall system is competitive with commonly-used surface registration technologies. Its main benefit is the

Fast time-of-flight camera based surface registration for radiotherapy patient positioning

International Nuclear Information System (INIS)

Placht, Simon; Stancanello, Joseph; Schaller, Christian; Balda, Michael; Angelopoulou, Elli

2012-01-01

Purpose: This work introduces a rigid registration framework for patient positioning in radiotherapy, based on real-time surface acquisition by a time-of-flight (ToF) camera. Dynamic properties of the system are also investigated for future gating/tracking strategies. Methods: A novel preregistration algorithm, based on translation and rotation-invariant features representing surface structures, was developed. Using these features, corresponding three-dimensional points were computed in order to determine initial registration parameters. These parameters became a robust input to an accelerated version of the iterative closest point (ICP) algorithm for the fine-tuning of the registration result. Distance calibration and Kalman filtering were used to compensate for ToF-camera dependent noise. Additionally, the advantage of using the feature based preregistration over an ''ICP only'' strategy was evaluated, as well as the robustness of the rigid-transformation-based method to deformation. Results: The proposed surface registration method was validated using phantom data. A mean target registration error (TRE) for translations and rotations of 1.62 ± 1.08 mm and 0.07 deg. ± 0.05 deg., respectively, was achieved. There was a temporal delay of about 65 ms in the registration output, which can be seen as negligible considering the dynamics of biological systems. Feature based preregistration allowed for accurate and robust registrations even at very large initial displacements. Deformations affected the accuracy of the results, necessitating particular care in cases of deformed surfaces. Conclusions: The proposed solution is able to solve surface registration problems with an accuracy suitable for radiotherapy cases where external surfaces offer primary or complementary information to patient positioning. The system shows promising dynamic properties for its use in gating/tracking applications. The overall system is competitive with commonly-used surface registration

Intensity-based hierarchical elastic registration using approximating splines.

Science.gov (United States)

Serifovic-Trbalic, Amira; Demirovic, Damir; Cattin, Philippe C

2014-01-01

We introduce a new hierarchical approach for elastic medical image registration using approximating splines. In order to obtain the dense deformation field, we employ Gaussian elastic body splines (GEBS) that incorporate anisotropic landmark errors and rotation information. Since the GEBS approach is based on a physical model in form of analytical solutions of the Navier equation, it can very well cope with the local as well as global deformations present in the images by varying the standard deviation of the Gaussian forces. The proposed GEBS approximating model is integrated into the elastic hierarchical image registration framework, which decomposes a nonrigid registration problem into numerous local rigid transformations. The approximating GEBS registration scheme incorporates anisotropic landmark errors as well as rotation information. The anisotropic landmark localization uncertainties can be estimated directly from the image data, and in this case, they represent the minimal stochastic localization error, i.e., the Cramér-Rao bound. The rotation information of each landmark obtained from the hierarchical procedure is transposed in an additional angular landmark, doubling the number of landmarks in the GEBS model. The modified hierarchical registration using the approximating GEBS model is applied to register 161 image pairs from a digital mammogram database. The obtained results are very encouraging, and the proposed approach significantly improved all registrations comparing the mean-square error in relation to approximating TPS with the rotation information. On artificially deformed breast images, the newly proposed method performed better than the state-of-the-art registration algorithm introduced by Rueckert et al. (IEEE Trans Med Imaging 18:712-721, 1999). The average error per breast tissue pixel was less than 2.23 pixels compared to 2.46 pixels for Rueckert's method. The proposed hierarchical elastic image registration approach incorporates the GEBS

Multiresolution image registration for multimodal brain images and fusion for better neurosurgical planning

Directory of Open Access Journals (Sweden)

Siddeshappa Nandish

2017-12-01

Conclusion: The end resultant fused images are validated by the radiologists and mutual information measure is used to validate registration results. It is found that CT and MRI sequence with more number of slices gave promising results. Few cases with deformation during misregistrations recorded with low mutual information of about 0.3 and which is not acceptable and few recorded with 0.6 and above mutual information during registration gives promising results.

Multi-band Image Registration Method Based on Fourier Transform

Institute of Scientific and Technical Information of China (English)

庹红娅; 刘允才

2004-01-01

This paper presented a registration method based on Fourier transform for multi-band images which is involved in translation and small rotation. Although different band images differ a lot in the intensity and features,they contain certain common information which we can exploit. A model was given that the multi-band images have linear correlations under the least-square sense. It is proved that the coefficients have no effect on the registration progress if two images have linear correlations. Finally, the steps of the registration method were proposed. The experiments show that the model is reasonable and the results are satisfying.

A dental implant-based registration method for measuring mandibular kinematics using cone beam computed tomography-based fluoroscopy.

Science.gov (United States)

Lin, Cheng-Chung; Chen, Chien-Chih; Chen, Yunn-Jy; Lu, Tung-Wu; Hong, Shih-Wun

2014-01-01

This study aimed to develop and evaluate experimentally an implant-based registration method for measuring three-dimensional (3D) kinematics of the mandible and dental implants in the mandible based on dental cone beam computed tomography (CBCT), modified to include fluoroscopic function. The proposed implant-based registration method was based on the registration of CBCT data of implants/bones with single-plane fluoroscopy images. Seven registration conditions that included one to three implants were evaluated experimentally for their performance in a cadaveric porcine headmodel. The implant-based registration method was shown to have measurement errors (SD) of less than -0.2 (0.3) mm, 1.1 (2.2) mm, and 0.7 degrees (1.3 degrees) for the in-plane translation, out-of-plane translation, and all angular components, respectively, regardless of the number of implants used. The corresponding errors were reduced to less than -0.1 (0.1) mm, -0.3 (1.7) mm, and 0.5 degree (0.4 degree) when three implants were used. An implant-based registration method was developed to measure the 3D kinematics of the mandible/implants. With its high accuracy and reliability, the new method will be useful for measuring the 3D motion of the bones/implants for relevant applications.

Multimodal image registration of the scoliotic torso for surgical planning

International Nuclear Information System (INIS)

Harmouche, Rola; Cheriet, Farida; Labelle, Hubert; Dansereau, Jean

2013-01-01

This paper presents a method that registers MRIs acquired in prone position, with surface topography (TP) and X-ray reconstructions acquired in standing position, in order to obtain a 3D representation of a human torso incorporating the external surface, bone structures, and soft tissues. TP and X-ray data are registered using landmarks. Bone structures are used to register each MRI slice using an articulated model, and the soft tissue is confined to the volume delimited by the trunk and bone surfaces using a constrained thin-plate spline. The method is tested on 3 pre-surgical patients with scoliosis and shows a significant improvement, qualitatively and using the Dice similarity coefficient, in fitting the MRI into the standing patient model when compared to rigid and articulated model registration. The determinant of the Jacobian of the registration deformation shows higher variations in the deformation in areas closer to the surface of the torso. The novel, resulting 3D full torso model can provide a more complete representation of patient geometry to be incorporated in surgical simulators under development that aim at predicting the effect of scoliosis surgery on the external appearance of the patient’s torso

Log-Gabor Energy Based Multimodal Medical Image Fusion in NSCT Domain

Directory of Open Access Journals (Sweden)

Yong Yang

2014-01-01

Full Text Available Multimodal medical image fusion is a powerful tool in clinical applications such as noninvasive diagnosis, image-guided radiotherapy, and treatment planning. In this paper, a novel nonsubsampled Contourlet transform (NSCT based method for multimodal medical image fusion is presented, which is approximately shift invariant and can effectively suppress the pseudo-Gibbs phenomena. The source medical images are initially transformed by NSCT followed by fusing low- and high-frequency components. The phase congruency that can provide a contrast and brightness-invariant representation is applied to fuse low-frequency coefficients, whereas the Log-Gabor energy that can efficiently determine the frequency coefficients from the clear and detail parts is employed to fuse the high-frequency coefficients. The proposed fusion method has been compared with the discrete wavelet transform (DWT, the fast discrete curvelet transform (FDCT, and the dual tree complex wavelet transform (DTCWT based image fusion methods and other NSCT-based methods. Visually and quantitatively experimental results indicate that the proposed fusion method can obtain more effective and accurate fusion results of multimodal medical images than other algorithms. Further, the applicability of the proposed method has been testified by carrying out a clinical example on a woman affected with recurrent tumor images.

Multimodal label-free microscopy

Directory of Open Access Journals (Sweden)

Nicolas Pavillon

2014-09-01

Full Text Available This paper reviews the different multimodal applications based on a large extent of label-free imaging modalities, ranging from linear to nonlinear optics, while also including spectroscopic measurements. We put specific emphasis on multimodal measurements going across the usual boundaries between imaging modalities, whereas most multimodal platforms combine techniques based on similar light interactions or similar hardware implementations. In this review, we limit the scope to focus on applications for biology such as live cells or tissues, since by their nature of being alive or fragile, we are often not free to take liberties with the image acquisition times and are forced to gather the maximum amount of information possible at one time. For such samples, imaging by a given label-free method usually presents a challenge in obtaining sufficient optical signal or is limited in terms of the types of observable targets. Multimodal imaging is then particularly attractive for these samples in order to maximize the amount of measured information. While multimodal imaging is always useful in the sense of acquiring additional information from additional modes, at times it is possible to attain information that could not be discovered using any single mode alone, which is the essence of the progress that is possible using a multimodal approach.

Three-dimensional registration methods for multi-modal magnetic resonance neuroimages

International Nuclear Information System (INIS)

Triantafyllou, C.

2001-08-01

In this thesis, image alignment techniques are developed and evaluated for applications in neuroimaging. In particular, the problem of combining cross-sequence MRI (Magnetic Resonance Imaging) intra-subject scans is considered. The challenge in this case is to find topographically uniform mappings in order to register (find a mapping between) low resolution echo-planar images and their high resolution structural counterparts. Such an approach enables us to effectually fuse, in a clinically useful way, information across scans. This dissertation devises a new framework by which this may be achieved, involving appropriate optimisation of the required mapping functions, which turn out to be non-linear and high-dimensional in nature. Novel ways to constrain and regularise these functions to enhance the computational speed of the process and the accuracy of the solution are also studied. The algorithms, whose characteristics are demonstrated for this specific application should be fully generalisable to other medical imaging modalities and potentially, other areas of image processing. To begin with, some existing registration methods are reviewed, followed by the introduction of an automated global 3-D registration method. Its performance is investigated on extracted cortical and ventricular surfaces by utilising the principles of the chamfer matching approach. Evaluations on synthetic and real data-sets, are performed to show that removal of global image differences is possible in principle, although the true accuracy of the method depends on the type of geometrical distortions present. These results also reveal that this class of algorithm is unable to solve more localised variations and higher order magnetic field distortions between the images. These facts motivate the development of a high-dimensional 3-D registration method capable of effecting a one-to-one correspondence by capturing the localised differences. This method was seen to account not only for

Assessing Digital Student Productions, a Design-Based Research Study on the Development of a Criteria-Based Assessment Tool for Students’ Digital Multimodal Productions

DEFF Research Database (Denmark)

Hoffmeyer, Mikkeline; Jensen, Jesper Juellund; Olsen, Marie Veisegaard

2017-01-01

Digital multimodal production is becoming increasingly important as a 21st century skill and as a learning condition in school (K-12). Moreover, there is a growing attention to the significance of criteria-based assessment for learning. Nevertheless, assessment of students’ digital multimodal...... productions is often vague or lacking. Therefore, the research project aims at developing a tool to support assessment of student’s digital multimodal productions through a design-based research method. This paper presents a proposal for issues to be considered through a prototyping phase, based on interviews...

MRI and CBCT image registration of temporomandibular joint: a systematic review.

Science.gov (United States)

Al-Saleh, Mohammed A Q; Alsufyani, Noura A; Saltaji, Humam; Jaremko, Jacob L; Major, Paul W

2016-05-10

The purpose of the present review is to systematically and critically analyze the available literature regarding the importance, applicability, and practicality of (MRI), computerized tomography (CT) or cone-beam CT (CBCT) image registration for TMJ anatomy and assessment. A systematic search of 4 databases; MEDLINE, EMBASE, EBM reviews and Scopus, was conducted by 2 reviewers. An additional manual search of the bibliography was performed. All articles discussing the magnetic resonance imaging MRI and CT or CBCT image registration for temporomandibular joint (TMJ) visualization or assessment were included. Only 3 articles satisfied the inclusion criteria. All included articles were published within the last 7 years. Two articles described MRI to CT multimodality image registration as a complementary tool to visualize TMJ. Both articles used images of one patient only to introduce the complementary concept of MRI-CT fused image. One article assessed the reliability of using MRI-CBCT registration to evaluate the TMJ disc position and osseous pathology for 10 temporomandibular disorder (TMD) patients. There are very limited studies of MRI-CT/CBCT registration to reach a conclusion regarding its accuracy or clinical use in the temporomandibular joints.

Multimodal image registration of the scoliotic torso for surgical planning

Science.gov (United States)

2013-01-01

Background This paper presents a method that registers MRIs acquired in prone position, with surface topography (TP) and X-ray reconstructions acquired in standing position, in order to obtain a 3D representation of a human torso incorporating the external surface, bone structures, and soft tissues. Methods TP and X-ray data are registered using landmarks. Bone structures are used to register each MRI slice using an articulated model, and the soft tissue is confined to the volume delimited by the trunk and bone surfaces using a constrained thin-plate spline. Results The method is tested on 3 pre-surgical patients with scoliosis and shows a significant improvement, qualitatively and using the Dice similarity coefficient, in fitting the MRI into the standing patient model when compared to rigid and articulated model registration. The determinant of the Jacobian of the registration deformation shows higher variations in the deformation in areas closer to the surface of the torso. Conclusions The novel, resulting 3D full torso model can provide a more complete representation of patient geometry to be incorporated in surgical simulators under development that aim at predicting the effect of scoliosis surgery on the external appearance of the patient’s torso. PMID:23289431

Constrained non-rigid registration for whole body image registration: method and validation

Science.gov (United States)

Li, Xia; Yankeelov, Thomas E.; Peterson, Todd E.; Gore, John C.; Dawant, Benoit M.

2007-03-01

3D intra- and inter-subject registration of image volumes is important for tasks that include measurements and quantification of temporal/longitudinal changes, atlas-based segmentation, deriving population averages, or voxel and tensor-based morphometry. A number of methods have been proposed to tackle this problem but few of them have focused on the problem of registering whole body image volumes acquired either from humans or small animals. These image volumes typically contain a large number of articulated structures, which makes registration more difficult than the registration of head images, to which the vast majority of registration algorithms have been applied. To solve this problem, we have previously proposed an approach, which initializes an intensity-based non-rigid registration algorithm with a point based registration technique [1, 2]. In this paper, we introduce new constraints into our non-rigid registration algorithm to prevent the bones from being deformed inaccurately. Results we have obtained show that the new constrained algorithm leads to better registration results than the previous one.

Non-rigid contour-to-pixel registration of photographic and quantitative light-induced fluorescence imaging of decalcified teeth

Science.gov (United States)

Berkels, Benjamin; Deserno, Thomas; Ehrlich, Eva E.; Fritz, Ulrike B.; Sirazitdinova, Ekaterina; Tatano, Rosalia

2016-03-01

Quantitative light-induced fluorescence (QLF) is widely used to assess the damage of a tooth due to decalcification. In digital photographs, decalcification appears as white spot lesions, i.e. white spots on the tooth surface. We propose a novel multimodal registration approach for the matching of digital photographs and QLF images of decalcified teeth. The registration is based on the idea of contour-to-pixel matching. Here, the curve, which represents the shape of the tooth, is extracted from the QLF image using a contour segmentation by binarization and morphological processing. This curve is aligned to the photo with a non-rigid variational registration approach. Thus, the registration problem is formulated as minimization problem with an objective function that consists of a data term and a regularizer for the deformation. To construct the data term, the photo is pointwise classified into tooth and non-tooth regions. Then, the signed distance function of the tooth region allows to measure the mismatch between curve and photo. As regularizer a higher order, linear elastic prior is used. The resulting minimization problem is solved numerically using bilinear Finite Elements for the spatial discretization and the Gauss-Newton algorithm. The evaluation is based on 150 image pairs, where an average of 5 teeth have been captured from 32 subjects. All registrations have been confirmed correctly by a dental expert. The contour-to-pixel methods can directly be used in 3D for surface-to-voxel tasks.

Improved image registration by sparse patch-based deformation estimation.

Science.gov (United States)

Kim, Minjeong; Wu, Guorong; Wang, Qian; Lee, Seong-Whan; Shen, Dinggang

2015-01-15

Despite intensive efforts for decades, deformable image registration is still a challenging problem due to the potential large anatomical differences across individual images, which limits the registration performance. Fortunately, this issue could be alleviated if a good initial deformation can be provided for the two images under registration, which are often termed as the moving subject and the fixed template, respectively. In this work, we present a novel patch-based initial deformation prediction framework for improving the performance of existing registration algorithms. Our main idea is to estimate the initial deformation between subject and template in a patch-wise fashion by using the sparse representation technique. We argue that two image patches should follow the same deformation toward the template image if their patch-wise appearance patterns are similar. To this end, our framework consists of two stages, i.e., the training stage and the application stage. In the training stage, we register all training images to the pre-selected template, such that the deformation of each training image with respect to the template is known. In the application stage, we apply the following four steps to efficiently calculate the initial deformation field for the new test subject: (1) We pick a small number of key points in the distinctive regions of the test subject; (2) for each key point, we extract a local patch and form a coupled appearance-deformation dictionary from training images where each dictionary atom consists of the image intensity patch as well as their respective local deformations; (3) a small set of training image patches in the coupled dictionary are selected to represent the image patch of each subject key point by sparse representation. Then, we can predict the initial deformation for each subject key point by propagating the pre-estimated deformations on the selected training patches with the same sparse representation coefficients; and (4) we

INVITED REVIEW--IMAGE REGISTRATION IN VETERINARY RADIATION ONCOLOGY: INDICATIONS, IMPLICATIONS, AND FUTURE ADVANCES.

Science.gov (United States)

Feng, Yang; Lawrence, Jessica; Cheng, Kun; Montgomery, Dean; Forrest, Lisa; Mclaren, Duncan B; McLaughlin, Stephen; Argyle, David J; Nailon, William H

2016-01-01

The field of veterinary radiation therapy (RT) has gained substantial momentum in recent decades with significant advances in conformal treatment planning, image-guided radiation therapy (IGRT), and intensity-modulated (IMRT) techniques. At the root of these advancements lie improvements in tumor imaging, image alignment (registration), target volume delineation, and identification of critical structures. Image registration has been widely used to combine information from multimodality images such as computerized tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET) to improve the accuracy of radiation delivery and reliably identify tumor-bearing areas. Many different techniques have been applied in image registration. This review provides an overview of medical image registration in RT and its applications in veterinary oncology. A summary of the most commonly used approaches in human and veterinary medicine is presented along with their current use in IGRT and adaptive radiation therapy (ART). It is important to realize that registration does not guarantee that target volumes, such as the gross tumor volume (GTV), are correctly identified on the image being registered, as limitations unique to registration algorithms exist. Research involving novel registration frameworks for automatic segmentation of tumor volumes is ongoing and comparative oncology programs offer a unique opportunity to test the efficacy of proposed algorithms. © 2016 American College of Veterinary Radiology.

Hierarchical and successive approximate registration of the non-rigid medical image based on thin-plate splines

Science.gov (United States)

Hu, Jinyan; Li, Li; Yang, Yunfeng

2017-06-01

The hierarchical and successive approximate registration method of non-rigid medical image based on the thin-plate splines is proposed in the paper. There are two major novelties in the proposed method. First, the hierarchical registration based on Wavelet transform is used. The approximate image of Wavelet transform is selected as the registered object. Second, the successive approximation registration method is used to accomplish the non-rigid medical images registration, i.e. the local regions of the couple images are registered roughly based on the thin-plate splines, then, the current rough registration result is selected as the object to be registered in the following registration procedure. Experiments show that the proposed method is effective in the registration process of the non-rigid medical images.

Superparamagnetic nanoparticles for enhanced magnetic resonance and multimodal imaging

Science.gov (United States)

Sikma, Elise Ann Schultz

Magnetic resonance imaging (MRI) is a powerful tool for noninvasive tomographic imaging of biological systems with high spatial and temporal resolution. Superparamagnetic (SPM) nanoparticles have emerged as highly effective MR contrast agents due to their biocompatibility, ease of surface modification and magnetic properties. Conventional nanoparticle contrast agents suffer from difficult synthetic reproducibility, polydisperse sizes and weak magnetism. Numerous synthetic techniques and nanoparticle formulations have been developed to overcome these barriers. However, there are still major limitations in the development of new nanoparticle-based probes for MR and multimodal imaging including low signal amplification and absence of biochemical reporters. To address these issues, a set of multimodal (T2/optical) and dual contrast (T1/T2) nanoparticle probes has been developed. Their unique magnetic properties and imaging capabilities were thoroughly explored. An enzyme-activatable contrast agent is currently being developed as an innovative means for early in vivo detection of cancer at the cellular level. Multimodal probes function by combining the strengths of multiple imaging techniques into a single agent. Co-registration of data obtained by multiple imaging modalities validates the data, enhancing its quality and reliability. A series of T2/optical probes were successfully synthesized by attachment of a fluorescent dye to the surface of different types of nanoparticles. The multimodal nanoparticles generated sufficient MR and fluorescence signal to image transplanted islets in vivo. Dual contrast T1/T2 imaging probes were designed to overcome disadvantages inherent in the individual T1 and T2 components. A class of T1/T2 agents was developed consisting of a gadolinium (III) complex (DTPA chelate or DO3A macrocycle) conjugated to a biocompatible silica-coated metal oxide nanoparticle through a disulfide linker. The disulfide linker has the ability to be reduced

Extended feature-fusion guidelines to improve image-based multi-modal biometrics

CSIR Research Space (South Africa)

Brown, Dane

2016-09-01

Full Text Available The feature-level, unlike the match score-level, lacks multi-modal fusion guidelines. This work demonstrates a practical approach for improved image-based biometric feature-fusion. The approach extracts and combines the face, fingerprint...

A Novel Multimodal Biometrics Recognition Model Based on Stacked ELM and CCA Methods

Directory of Open Access Journals (Sweden)

Jucheng Yang

2018-04-01

Full Text Available Multimodal biometrics combine a variety of biological features to have a significant impact on identification performance, which is a newly developed trend in biometrics identification technology. This study proposes a novel multimodal biometrics recognition model based on the stacked extreme learning machines (ELMs and canonical correlation analysis (CCA methods. The model, which has a symmetric structure, is found to have high potential for multimodal biometrics. The model works as follows. First, it learns the hidden-layer representation of biological images using extreme learning machines layer by layer. Second, the canonical correlation analysis method is applied to map the representation to a feature space, which is used to reconstruct the multimodal image feature representation. Third, the reconstructed features are used as the input of a classifier for supervised training and output. To verify the validity and efficiency of the method, we adopt it for new hybrid datasets obtained from typical face image datasets and finger-vein image datasets. Our experimental results demonstrate that our model performs better than traditional methods.

Multimodality localization of epileptic foci

Science.gov (United States)

Desco, Manuel; Pascau, Javier; Pozo, M. A.; Santos, Andres; Reig, Santiago; Gispert, Juan D.; Garcia-Barreno, Pedro

2001-05-01

This paper presents a multimodality approach for the localization of epileptic foci using PET, MRI and EEG combined without the need of external markers. Mutual Information algorithm is used for MRI-PET registration. Dipole coordinates (provided by BESA software) are projected onto the MRI using a specifically developed algorithm. The four anatomical references used for electrode positioning (nasion, inion and two preauricular points) are located on the MRI using a triplanar viewer combined with a surface-rendering tool. Geometric transformation using deformation of the ideal sphere used for dipole calculations is then applied to match the patient's brain size and shape. Eight treatment-refractory epileptic patients have been studied. The combination of the anatomical information from the MRI, hipoperfusion areas in PET and dipole position and orientation helped the physician in the diagnosis of epileptic focus location. Neurosurgery was not indicated for patients where PET and dipole results were inconsistent; in two cases it was clinically indicated despite the mismatch, showing a negative follow up. The multimodality approach presented does not require external markers for dipole projection onto the MRI, this being the main difference with previous methods. The proposed method may play an important role in the indication of surgery for treatment- refractory epileptic patients.

Photoacoustic-Based Multimodal Nanoprobes: from Constructing to Biological Applications.

Science.gov (United States)

Gao, Duyang; Yuan, Zhen

2017-01-01

Multimodal nanoprobes have attracted intensive attentions since they can integrate various imaging modalities to obtain complementary merits of single modality. Meanwhile, recent interest in laser-induced photoacoustic imaging is rapidly growing due to its unique advantages in visualizing tissue structure and function with high spatial resolution and satisfactory imaging depth. In this review, we summarize multimodal nanoprobes involving photoacoustic imaging. In particular, we focus on the method to construct multimodal nanoprobes. We have divided the synthetic methods into two types. First, we call it "one for all" concept, which involves intrinsic properties of the element in a single particle. Second, "all in one" concept, which means integrating different functional blocks in one particle. Then, we simply introduce the applications of the multifunctional nanoprobes for in vivo imaging and imaging-guided tumor therapy. At last, we discuss the advantages and disadvantages of the present methods to construct the multimodal nanoprobes and share our viewpoints in this area.

A robust probabilistic collaborative representation based classification for multimodal biometrics

Science.gov (United States)

Zhang, Jing; Liu, Huanxi; Ding, Derui; Xiao, Jianli

2018-04-01

Most of the traditional biometric recognition systems perform recognition with a single biometric indicator. These systems have suffered noisy data, interclass variations, unacceptable error rates, forged identity, and so on. Due to these inherent problems, it is not valid that many researchers attempt to enhance the performance of unimodal biometric systems with single features. Thus, multimodal biometrics is investigated to reduce some of these defects. This paper proposes a new multimodal biometric recognition approach by fused faces and fingerprints. For more recognizable features, the proposed method extracts block local binary pattern features for all modalities, and then combines them into a single framework. For better classification, it employs the robust probabilistic collaborative representation based classifier to recognize individuals. Experimental results indicate that the proposed method has improved the recognition accuracy compared to the unimodal biometrics.

A community intervention trial of multimodal suicide prevention program in Japan: A Novel multimodal Community Intervention program to prevent suicide and suicide attempt in Japan, NOCOMIT-J

Directory of Open Access Journals (Sweden)

Suzuki Yuriko

2008-09-01

Full Text Available Abstract Background To respond to the rapid surge in the incidence of suicide in Japan, which appears to be an ongoing trend, the Japanese Multimodal Intervention Trials for Suicide Prevention (J-MISP have launched a multimodal community-based suicide prevention program, NOCOMIT-J. The primary aim of this study is to examine whether NOCOMIT-J is effective in reducing suicidal behavior in the community. Methods/DesignThis study is a community intervention trial involving seven intervention regions with accompanying control regions, all with populations of statistically sufficient size. The program focuses on building social support networks in the public health system for suicide prevention and mental health promotion, intending to reinforce human relationships in the community. The intervention program components includes a primary prevention measures of awareness campaign for the public and key personnel, secondary prevention measures for screening of, and assisting, high-risk individuals, after-care for individuals bereaved by suicide, and other measures. The intervention started in July 2006, and will continue for 3.5 years. Participants are Japanese and foreign residents living in the intervention and control regions (a total of population of 2,120,000 individuals. Discussion The present study is designed to evaluate the effectiveness of the community-based suicide prevention program in the seven participating areas. Trial registration UMIN Clinical Trials Registry (UMIN-CTR UMIN000000460.

Learning multimodal dictionaries.

Science.gov (United States)

Monaci, Gianluca; Jost, Philippe; Vandergheynst, Pierre; Mailhé, Boris; Lesage, Sylvain; Gribonval, Rémi

2007-09-01

Real-world phenomena involve complex interactions between multiple signal modalities. As a consequence, humans are used to integrate at each instant perceptions from all their senses in order to enrich their understanding of the surrounding world. This paradigm can be also extremely useful in many signal processing and computer vision problems involving mutually related signals. The simultaneous processing of multimodal data can, in fact, reveal information that is otherwise hidden when considering the signals independently. However, in natural multimodal signals, the statistical dependencies between modalities are in general not obvious. Learning fundamental multimodal patterns could offer deep insight into the structure of such signals. In this paper, we present a novel model of multimodal signals based on their sparse decomposition over a dictionary of multimodal structures. An algorithm for iteratively learning multimodal generating functions that can be shifted at all positions in the signal is proposed, as well. The learning is defined in such a way that it can be accomplished by iteratively solving a generalized eigenvector problem, which makes the algorithm fast, flexible, and free of user-defined parameters. The proposed algorithm is applied to audiovisual sequences and it is able to discover underlying structures in the data. The detection of such audio-video patterns in audiovisual clips allows to effectively localize the sound source on the video in presence of substantial acoustic and visual distractors, outperforming state-of-the-art audiovisual localization algorithms.

Reducing uncertainties in volumetric image based deformable organ registration

International Nuclear Information System (INIS)

Liang, J.; Yan, D.

2003-01-01

Applying volumetric image feedback in radiotherapy requires image based deformable organ registration. The foundation of this registration is the ability of tracking subvolume displacement in organs of interest. Subvolume displacement can be calculated by applying biomechanics model and the finite element method to human organs manifested on the multiple volumetric images. The calculation accuracy, however, is highly dependent on the determination of the corresponding organ boundary points. Lacking sufficient information for such determination, uncertainties are inevitable--thus diminishing the registration accuracy. In this paper, a method of consuming energy minimization was developed to reduce these uncertainties. Starting from an initial selection of organ boundary point correspondence on volumetric image sets, the subvolume displacement and stress distribution of the whole organ are calculated and the consumed energy due to the subvolume displacements is computed accordingly. The corresponding positions of the initially selected boundary points are then iteratively optimized to minimize the consuming energy under geometry and stress constraints. In this study, a rectal wall delineated from patient CT image was artificially deformed using a computer simulation and utilized to test the optimization. Subvolume displacements calculated based on the optimized boundary point correspondence were compared to the true displacements, and the calculation accuracy was thereby evaluated. Results demonstrate that a significant improvement on the accuracy of the deformable organ registration can be achieved by applying the consuming energy minimization in the organ deformation calculation

Real-time estimation of FLE for point-based registration

Science.gov (United States)

Wiles, Andrew D.; Peters, Terry M.

2009-02-01

In image-guide surgery, optimizing the accuracy in localizing the surgical tools within the virtual reality environment or 3D image is vitally important, significant effort has been spent reducing the measurement errors at the point of interest or target. This target registration error (TRE) is often defined by a root-mean-square statistic which reduces the vector data to a single term that can be minimized. However, lost in the data reduction is the directionality of the error which, can be modelled using a 3D covariance matrix. Recently, we developed a set of expressions that modeled the TRE statistics for point-based registrations as a function of the fiducial marker geometry, target location and the fiducial localizer error (FLE). Unfortunately, these expressions are only as good as the definition of the FLE. In order to close the gap, we have subsequently developed a closed form expression that estimates the FLE as a function of the estimated fiducial registration error (FRE, the error between the measured fiducials and the best fit locations of those fiducials). The FRE covariance matrix is estimated using a sliding window technique and used as input into the closed form expression to estimate the FLE. The estimated FLE can then used to estimate the TRE which, can be given to the surgeon to permit the procedure to be designed such that the errors associated with the point-based registrations are minimized.

Single-pulse CARS based multimodal nonlinear optical microscope for bioimaging.

Science.gov (United States)

Kumar, Sunil; Kamali, Tschackad; Levitte, Jonathan M; Katz, Ori; Hermann, Boris; Werkmeister, Rene; Považay, Boris; Drexler, Wolfgang; Unterhuber, Angelika; Silberberg, Yaron

2015-05-18

Noninvasive label-free imaging of biological systems raises demand not only for high-speed three-dimensional prescreening of morphology over a wide-field of view but also it seeks to extract the microscopic functional and molecular details within. Capitalizing on the unique advantages brought out by different nonlinear optical effects, a multimodal nonlinear optical microscope can be a powerful tool for bioimaging. Bringing together the intensity-dependent contrast mechanisms via second harmonic generation, third harmonic generation and four-wave mixing for structural-sensitive imaging, and single-beam/single-pulse coherent anti-Stokes Raman scattering technique for chemical sensitive imaging in the finger-print region, we have developed a simple and nearly alignment-free multimodal nonlinear optical microscope that is based on a single wide-band Ti:Sapphire femtosecond pulse laser source. Successful imaging tests have been realized on two exemplary biological samples, a canine femur bone and collagen fibrils harvested from a rat tail. Since the ultra-broad band-width femtosecond laser is a suitable source for performing high-resolution optical coherence tomography, a wide-field optical coherence tomography arm can be easily incorporated into the presented multimodal microscope making it a versatile optical imaging tool for noninvasive label-free bioimaging.

Fast Rotation-Free Feature-Based Image Registration Using Improved N-SIFT and GMM-Based Parallel Optimization.

Science.gov (United States)

Yu, Dongdong; Yang, Feng; Yang, Caiyun; Leng, Chengcai; Cao, Jian; Wang, Yining; Tian, Jie

2016-08-01

Image registration is a key problem in a variety of applications, such as computer vision, medical image processing, pattern recognition, etc., while the application of registration is limited by time consumption and the accuracy in the case of large pose differences. Aimed at these two kinds of problems, we propose a fast rotation-free feature-based rigid registration method based on our proposed accelerated-NSIFT and GMM registration-based parallel optimization (PO-GMMREG). Our method is accelerated by using the GPU/CUDA programming and preserving only the location information without constructing the descriptor of each interest point, while its robustness to missing correspondences and outliers is improved by converting the interest point matching to Gaussian mixture model alignment. The accuracy in the case of large pose differences is settled by our proposed PO-GMMREG algorithm by constructing a set of initial transformations. Experimental results demonstrate that our proposed algorithm can fast rigidly register 3-D medical images and is reliable for aligning 3-D scans even when they exhibit a poor initialization.

Feature-based US to CT registration of the aortic root

Science.gov (United States)

Lang, Pencilla; Chen, Elvis C. S.; Guiraudon, Gerard M.; Jones, Doug L.; Bainbridge, Daniel; Chu, Michael W.; Drangova, Maria; Hata, Noby; Jain, Ameet; Peters, Terry M.

2011-03-01

A feature-based registration was developed to align biplane and tracked ultrasound images of the aortic root with a preoperative CT volume. In transcatheter aortic valve replacement, a prosthetic valve is inserted into the aortic annulus via a catheter. Poor anatomical visualization of the aortic root region can result in incorrect positioning, leading to significant morbidity and mortality. Registration of pre-operative CT to transesophageal ultrasound and fluoroscopy images is a major step towards providing augmented image guidance for this procedure. The proposed registration approach uses an iterative closest point algorithm to register a surface mesh generated from CT to 3D US points reconstructed from a single biplane US acquisition, or multiple tracked US images. The use of a single simultaneous acquisition biplane image eliminates reconstruction error introduced by cardiac gating and TEE probe tracking, creating potential for real-time intra-operative registration. A simple initialization procedure is used to minimize changes to operating room workflow. The algorithm is tested on images acquired from excised porcine hearts. Results demonstrate a clinically acceptable accuracy of 2.6mm and 5mm for tracked US to CT and biplane US to CT registration respectively.

Multimodal Discourse Analysis of the Movie "Argo"

Science.gov (United States)

Bo, Xu

2018-01-01

Based on multimodal discourse theory, this paper makes a multimodal discourse analysis of some shots in the movie "Argo" from the perspective of context of culture, context of situation and meaning of image. Results show that this movie constructs multimodal discourse through particular context, language and image, and successfully…

Seizure Onset Detection based on a Uni- or Multi-modal Intelligent Seizure Acquisition (UISA/MISA) System

DEFF Research Database (Denmark)

Conradsen, Isa; Beniczky, Sándor; Wolf, Peter

2010-01-01

An automatic Uni- or Multi-modal Inteligent Seizure Acquisition (UISA/MISA) system is highly applicable for onset detection of epileptic seizures based on motion data. The modalities used are surface electromyography (sEMG), acceleration (ACC) and angular velocity (ANG). The new proposed automatic...... algorithm on motion data is extracting features as “log-sum” measures of discrete wavelet components. Classification into the two groups “seizure” versus “nonseizure” is made based on the support vector machine (SVM) algorithm. The algorithm performs with a sensitivity of 91-100%, a median latency of 1...... second and a specificity of 100% on multi-modal data from five healthy subjects simulating seizures. The uni-modal algorithm based on sEMG data from the subjects and patients performs satisfactorily in some cases. As expected, our results clearly show superiority of the multimodal approach, as compared...

Arbitrary-ratio power splitter based on nonlinear multimode interference coupler

International Nuclear Information System (INIS)

Tajaldini, Mehdi; Jafri, Mohd Zubir Mat

2015-01-01

We propose an ultra-compact multimode interference (MMI) power splitter based on nonlinear effects from simulations using nonlinear modal propagation analysis (NMPA) cooperation with finite difference Method (FDM) to access free choice of splitting ratio. Conventional multimode interference power splitter could only obtain a few discrete ratios. The power splitting ratio may be adjusted continuously while the input set power is varying by a tunable laser. In fact, using an ultra- compact MMI with a simple structure that is launched by a tunable nonlinear input fulfills the problem of arbitrary-ratio in integrated photonics circuits. Silicon on insulator (SOI) is used as the offered material due to the high contrast refractive index and Centro symmetric properties. The high-resolution images at the end of the multimode waveguide in the simulated power splitter have a high power balance, whereas access to a free choice of splitting ratio is not possible under the linear regime in the proposed length range except changes in the dimension for any ratio. The compact dimensions and ideal performance of the device are established according to optimized parameters. The proposed regime can be extended to the design of M×N arbitrary power splitters ratio for programmable logic devices in all optical digital signal processing. The results of this study indicate that nonlinear modal propagation analysis solves the miniaturization problem for all-optical devices based on MMI couplers to achieve multiple functions in a compact planar integrated circuit and also overcomes the limitations of previously proposed methods for nonlinear MMI

Arbitrary-ratio power splitter based on nonlinear multimode interference coupler

Energy Technology Data Exchange (ETDEWEB)

Tajaldini, Mehdi [School of Physics, Universiti Sains Malaysia, 11800 Pulau Pinang (Malaysia); Young Researchers and Elite Club, Baft Branch, Islamic Azad University, Baft (Iran, Islamic Republic of); Jafri, Mohd Zubir Mat [School of Physics, Universiti Sains Malaysia, 11800 Pulau Pinang (Malaysia)

2015-04-24

We propose an ultra-compact multimode interference (MMI) power splitter based on nonlinear effects from simulations using nonlinear modal propagation analysis (NMPA) cooperation with finite difference Method (FDM) to access free choice of splitting ratio. Conventional multimode interference power splitter could only obtain a few discrete ratios. The power splitting ratio may be adjusted continuously while the input set power is varying by a tunable laser. In fact, using an ultra- compact MMI with a simple structure that is launched by a tunable nonlinear input fulfills the problem of arbitrary-ratio in integrated photonics circuits. Silicon on insulator (SOI) is used as the offered material due to the high contrast refractive index and Centro symmetric properties. The high-resolution images at the end of the multimode waveguide in the simulated power splitter have a high power balance, whereas access to a free choice of splitting ratio is not possible under the linear regime in the proposed length range except changes in the dimension for any ratio. The compact dimensions and ideal performance of the device are established according to optimized parameters. The proposed regime can be extended to the design of M×N arbitrary power splitters ratio for programmable logic devices in all optical digital signal processing. The results of this study indicate that nonlinear modal propagation analysis solves the miniaturization problem for all-optical devices based on MMI couplers to achieve multiple functions in a compact planar integrated circuit and also overcomes the limitations of previously proposed methods for nonlinear MMI.

Image Registration Algorithm Based on Parallax Constraint and Clustering Analysis

Science.gov (United States)

Wang, Zhe; Dong, Min; Mu, Xiaomin; Wang, Song

2018-01-01

To resolve the problem of slow computation speed and low matching accuracy in image registration, a new image registration algorithm based on parallax constraint and clustering analysis is proposed. Firstly, Harris corner detection algorithm is used to extract the feature points of two images. Secondly, use Normalized Cross Correlation (NCC) function to perform the approximate matching of feature points, and the initial feature pair is obtained. Then, according to the parallax constraint condition, the initial feature pair is preprocessed by K-means clustering algorithm, which is used to remove the feature point pairs with obvious errors in the approximate matching process. Finally, adopt Random Sample Consensus (RANSAC) algorithm to optimize the feature points to obtain the final feature point matching result, and the fast and accurate image registration is realized. The experimental results show that the image registration algorithm proposed in this paper can improve the accuracy of the image matching while ensuring the real-time performance of the algorithm.

[Non-rigid medical image registration based on mutual information and thin-plate spline].

Science.gov (United States)

Cao, Guo-gang; Luo, Li-min

2009-01-01

To get precise and complete details, the contrast in different images is needed in medical diagnosis and computer assisted treatment. The image registration is the basis of contrast, but the regular rigid registration does not satisfy the clinic requirements. A non-rigid medical image registration method based on mutual information and thin-plate spline was present. Firstly, registering two images globally based on mutual information; secondly, dividing reference image and global-registered image into blocks and registering them; then getting the thin-plate spline transformation according to the shift of blocks' center; finally, applying the transformation to the global-registered image. The results show that the method is more precise than the global rigid registration based on mutual information and it reduces the complexity of getting control points and satisfy the clinic requirements better by getting control points of the thin-plate transformation automatically.

Inverse consistent non-rigid image registration based on robust point set matching

Science.gov (United States)

2014-01-01

Background Robust point matching (RPM) has been extensively used in non-rigid registration of images to robustly register two sets of image points. However, except for the location at control points, RPM cannot estimate the consistent correspondence between two images because RPM is a unidirectional image matching approach. Therefore, it is an important issue to make an improvement in image registration based on RPM. Methods In our work, a consistent image registration approach based on the point sets matching is proposed to incorporate the property of inverse consistency and improve registration accuracy. Instead of only estimating the forward transformation between the source point sets and the target point sets in state-of-the-art RPM algorithms, the forward and backward transformations between two point sets are estimated concurrently in our algorithm. The inverse consistency constraints are introduced to the cost function of RPM and the fuzzy correspondences between two point sets are estimated based on both the forward and backward transformations simultaneously. A modified consistent landmark thin-plate spline registration is discussed in detail to find the forward and backward transformations during the optimization of RPM. The similarity of image content is also incorporated into point matching in order to improve image matching. Results Synthetic data sets, medical images are employed to demonstrate and validate the performance of our approach. The inverse consistent errors of our algorithm are smaller than RPM. Especially, the topology of transformations is preserved well for our algorithm for the large deformation between point sets. Moreover, the distance errors of our algorithm are similar to that of RPM, and they maintain a downward trend as whole, which demonstrates the convergence of our algorithm. The registration errors for image registrations are evaluated also. Again, our algorithm achieves the lower registration errors in same iteration number

Accelerated gradient-based free form deformable registration for online adaptive radiotherapy

International Nuclear Information System (INIS)

Yu, Gang; Yang, Guanyu; Shu, Huazhong; Li, Baosheng; Liang, Yueqiang; Yin, Yong; Li, Dengwang

2015-01-01

The registration of planning fan-beam computed tomography (FBCT) and daily cone-beam CT (CBCT) is a crucial step in adaptive radiation therapy. The current intensity-based registration algorithms, such as Demons, may fail when they are used to register FBCT and CBCT, because the CT numbers in CBCT cannot exactly correspond to the electron densities. In this paper, we investigated the effects of CBCT intensity inaccuracy on the registration accuracy and developed an accurate gradient-based free form deformation algorithm (GFFD). GFFD distinguishes itself from other free form deformable registration algorithms by (a) measuring the similarity using the 3D gradient vector fields to avoid the effect of inconsistent intensities between the two modalities; (b) accommodating image sampling anisotropy using the local polynomial approximation-intersection of confidence intervals (LPA-ICI) algorithm to ensure a smooth and continuous displacement field; and (c) introducing a ‘bi-directional’ force along with an adaptive force strength adjustment to accelerate the convergence process. It is expected that such a strategy can decrease the effect of the inconsistent intensities between the two modalities, thus improving the registration accuracy and robustness. Moreover, for clinical application, the algorithm was implemented by graphics processing units (GPU) through OpenCL framework. The registration time of the GFFD algorithm for each set of CT data ranges from 8 to 13 s. The applications of on-line adaptive image-guided radiation therapy, including auto-propagation of contours, aperture-optimization and dose volume histogram (DVH) in the course of radiation therapy were also studied by in-house-developed software. (paper)

Improving alignment in Tract-based spatial statistics: evaluation and optimization of image registration.

Science.gov (United States)

de Groot, Marius; Vernooij, Meike W; Klein, Stefan; Ikram, M Arfan; Vos, Frans M; Smith, Stephen M; Niessen, Wiro J; Andersson, Jesper L R

2013-08-01

Anatomical alignment in neuroimaging studies is of such importance that considerable effort is put into improving the registration used to establish spatial correspondence. Tract-based spatial statistics (TBSS) is a popular method for comparing diffusion characteristics across subjects. TBSS establishes spatial correspondence using a combination of nonlinear registration and a "skeleton projection" that may break topological consistency of the transformed brain images. We therefore investigated feasibility of replacing the two-stage registration-projection procedure in TBSS with a single, regularized, high-dimensional registration. To optimize registration parameters and to evaluate registration performance in diffusion MRI, we designed an evaluation framework that uses native space probabilistic tractography for 23 white matter tracts, and quantifies tract similarity across subjects in standard space. We optimized parameters for two registration algorithms on two diffusion datasets of different quality. We investigated reproducibility of the evaluation framework, and of the optimized registration algorithms. Next, we compared registration performance of the regularized registration methods and TBSS. Finally, feasibility and effect of incorporating the improved registration in TBSS were evaluated in an example study. The evaluation framework was highly reproducible for both algorithms (R(2) 0.993; 0.931). The optimal registration parameters depended on the quality of the dataset in a graded and predictable manner. At optimal parameters, both algorithms outperformed the registration of TBSS, showing feasibility of adopting such approaches in TBSS. This was further confirmed in the example experiment. Copyright © 2013 Elsevier Inc. All rights reserved.

Entropy-Based Block Processing for Satellite Image Registration

Directory of Open Access Journals (Sweden)

Ikhyun Lee

2012-11-01

Full Text Available Image registration is an important task in many computer vision applications such as fusion systems, 3D shape recovery and earth observation. Particularly, registering satellite images is challenging and time-consuming due to limited resources and large image size. In such scenario, state-of-the-art image registration methods such as scale-invariant feature transform (SIFT may not be suitable due to high processing time. In this paper, we propose an algorithm based on block processing via entropy to register satellite images. The performance of the proposed method is evaluated using different real images. The comparative analysis shows that it not only reduces the processing time but also enhances the accuracy.

The Pose Estimation of Mobile Robot Based on Improved Point Cloud Registration

Directory of Open Access Journals (Sweden)

Yanzi Miao

2016-03-01

Full Text Available Due to GPS restrictions, an inertial sensor is usually used to estimate the location of indoor mobile robots. However, it is difficult to achieve high-accuracy localization and control by inertial sensors alone. In this paper, a new method is proposed to estimate an indoor mobile robot pose with six degrees of freedom based on an improved 3D-Normal Distributions Transform algorithm (3D-NDT. First, point cloud data are captured by a Kinect sensor and segmented according to the distance to the robot. After the segmentation, the input point cloud data are processed by the Approximate Voxel Grid Filter algorithm in different sized voxel grids. Second, the initial registration and precise registration are performed respectively according to the distance to the sensor. The most distant point cloud data use the 3D-Normal Distributions Transform algorithm (3D-NDT with large-sized voxel grids for initial registration, based on the transformation matrix from the odometry method. The closest point cloud data use the 3D-NDT algorithm with small-sized voxel grids for precise registration. After the registrations above, a final transformation matrix is obtained and coordinated. Based on this transformation matrix, the pose estimation problem of the indoor mobile robot is solved. Test results show that this method can obtain accurate robot pose estimation and has better robustness.

Contour Propagation Using Feature-Based Deformable Registration for Lung Cancer

Directory of Open Access Journals (Sweden)

Yuhan Yang

2013-01-01

Full Text Available Accurate target delineation of CT image is a critical step in radiotherapy treatment planning. This paper describes a novel strategy for automatic contour propagation, based on deformable registration, for CT images of lung cancer. The proposed strategy starts with a manual-delineated contour in one slice of a 3D CT image. By means of feature-based deformable registration, the initial contour in other slices of the image can be propagated automatically, and then refined by active contour approach. Three algorithms are employed in the strategy: the Speeded-Up Robust Features (SURF, Thin-Plate Spline (TPS, and an adapted active contour (Snake, used to refine and modify the initial contours. Five pulmonary cancer cases with about 400 slices and 1000 contours have been used to verify the proposed strategy. Experiments demonstrate that the proposed strategy can improve the segmentation performance in the pulmonary CT images. Jaccard similarity (JS mean is about 0.88 and the maximum of Hausdorff distance (HD is about 90%. In addition, delineation time has been considerably reduced. The proposed feature-based deformable registration method in the automatic contour propagation improves the delineation efficiency significantly.

CO-REGISTRATION AIRBORNE LIDAR POINT CLOUD DATA AND SYNCHRONOUS DIGITAL IMAGE REGISTRATION BASED ON COMBINED ADJUSTMENT

Directory of Open Access Journals (Sweden)

Z. H. Yang

2016-06-01

Full Text Available Aim at the problem of co-registration airborne laser point cloud data with the synchronous digital image, this paper proposed a registration method based on combined adjustment. By integrating tie point, point cloud data with elevation constraint pseudo observations, using the principle of least-squares adjustment to solve the corrections of exterior orientation elements of each image, high-precision registration results can be obtained. In order to ensure the reliability of the tie point, and the effectiveness of pseudo observations, this paper proposed a point cloud data constrain SIFT matching and optimizing method, can ensure that the tie points are located on flat terrain area. Experiments with the airborne laser point cloud data and its synchronous digital image, there are about 43 pixels error in image space using the original POS data. If only considering the bore-sight of POS system, there are still 1.3 pixels error in image space. The proposed method regards the corrections of the exterior orientation elements of each image as unknowns and the errors are reduced to 0.15 pixels.

Registration of SPECT, PET and/or X-ray CT images in patients with lung cancer

International Nuclear Information System (INIS)

Uemura, K.; Toyama, H.; Miyamoto, T.; Yoshikawa, K.; Mori, Y.

2002-01-01

Aim: In order to evaluate the therapeutic gain of heavy ion therapy performed on patients with lung cancer, the regional pulmonary functions and the amount of radio tracer accumulation to the tumor, we are investigated by using the region of interest based on anatomical information obtained from X-ray CT. There are many registration techniques for brain images, but not so much for the other organ images that we have studied registration of chest SPECT, PET and/or X-ray CT images. Materials and Methods: Perfusion, ventilation and blood pool images with Tc 99m labeled radiopharmaceuticals and SPECT, tumor images with 11 C-methionine and PET and X-ray CT scans were performed on several patients with lung cancer before and after heavy ion therapy. The registrations of SPECT-CT, PET-CT and CT-CT were performed by using AMIR (Automatic Multimodality Image Registration), which was developed by Babak et al. for registration of brain images. In a case of SPECT-CT registration, each of the three functional images was registered to the X-ray CT image, and the accuracy of each registration was compared. In the studies of PET-CT registration, the transmission images and X-ray CT images were registered at first, because the 11 C-methionine PET images bear little resemblance to the underlying anatomical images. Next, the emission images were realigned by using the same registration parameters. The X-ray CT images obtained from a single subject at the different time were registered to the first X-ray CT images, respectively. Results: In the SPECT-CT registration, the blood pool-CT registration is the best among three SPECT images in visual inspection by radiologists. In the PET-CT registration, the Transmission-CT registrations got good results. Therefore, Emission-CT registrations also got good results. In the CT-CT registration, the X-ray CT images obtained from a single subject at the different time were superimposed well each other except for lower lobe. As the results, it was

Highly Sensitive Liquid Core Temperature Sensor Based on Multimode Interference Effects

Directory of Open Access Journals (Sweden)

Miguel A. Fuentes-Fuentes

2015-10-01

Full Text Available A novel fiber optic temperature sensor based on a liquid-core multimode interference device is demonstrated. The advantage of such structure is that the thermo-optic coefficient (TOC of the liquid is at least one order of magnitude larger than that of silica and this, combined with the fact that the TOC of silica and the liquid have opposite signs, provides a liquid-core multimode fiber (MMF highly sensitive to temperature. Since the refractive index of the liquid can be easily modified, this allows us to control the modal properties of the liquid-core MMF at will and the sensor sensitivity can be easily tuned by selecting the refractive index of the liquid in the core of the device. The maximum sensitivity measured in our experiments is 20 nm/°C in the low-temperature regime up to 60 °C. To the best of our knowledge, to date, this is the largest sensitivity reported for fiber-based MMI temperature sensors.

Hierarchical Spatial Concept Formation Based on Multimodal Information for Human Support Robots.

Science.gov (United States)

Hagiwara, Yoshinobu; Inoue, Masakazu; Kobayashi, Hiroyoshi; Taniguchi, Tadahiro

2018-01-01

In this paper, we propose a hierarchical spatial concept formation method based on the Bayesian generative model with multimodal information e.g., vision, position and word information. Since humans have the ability to select an appropriate level of abstraction according to the situation and describe their position linguistically, e.g., "I am in my home" and "I am in front of the table," a hierarchical structure of spatial concepts is necessary in order for human support robots to communicate smoothly with users. The proposed method enables a robot to form hierarchical spatial concepts by categorizing multimodal information using hierarchical multimodal latent Dirichlet allocation (hMLDA). Object recognition results using convolutional neural network (CNN), hierarchical k-means clustering result of self-position estimated by Monte Carlo localization (MCL), and a set of location names are used, respectively, as features in vision, position, and word information. Experiments in forming hierarchical spatial concepts and evaluating how the proposed method can predict unobserved location names and position categories are performed using a robot in the real world. Results verify that, relative to comparable baseline methods, the proposed method enables a robot to predict location names and position categories closer to predictions made by humans. As an application example of the proposed method in a home environment, a demonstration in which a human support robot moves to an instructed place based on human speech instructions is achieved based on the formed hierarchical spatial concept.

Hierarchical Spatial Concept Formation Based on Multimodal Information for Human Support Robots

Directory of Open Access Journals (Sweden)

Yoshinobu Hagiwara

2018-03-01

Full Text Available In this paper, we propose a hierarchical spatial concept formation method based on the Bayesian generative model with multimodal information e.g., vision, position and word information. Since humans have the ability to select an appropriate level of abstraction according to the situation and describe their position linguistically, e.g., “I am in my home” and “I am in front of the table,” a hierarchical structure of spatial concepts is necessary in order for human support robots to communicate smoothly with users. The proposed method enables a robot to form hierarchical spatial concepts by categorizing multimodal information using hierarchical multimodal latent Dirichlet allocation (hMLDA. Object recognition results using convolutional neural network (CNN, hierarchical k-means clustering result of self-position estimated by Monte Carlo localization (MCL, and a set of location names are used, respectively, as features in vision, position, and word information. Experiments in forming hierarchical spatial concepts and evaluating how the proposed method can predict unobserved location names and position categories are performed using a robot in the real world. Results verify that, relative to comparable baseline methods, the proposed method enables a robot to predict location names and position categories closer to predictions made by humans. As an application example of the proposed method in a home environment, a demonstration in which a human support robot moves to an instructed place based on human speech instructions is achieved based on the formed hierarchical spatial concept.

Comparing registration methods for mapping brain change using tensor-based morphometry.

Science.gov (United States)

Yanovsky, Igor; Leow, Alex D; Lee, Suh; Osher, Stanley J; Thompson, Paul M

2009-10-01

Measures of brain changes can be computed from sequential MRI scans, providing valuable information on disease progression for neuroscientific studies and clinical trials. Tensor-based morphometry (TBM) creates maps of these brain changes, visualizing the 3D profile and rates of tissue growth or atrophy. In this paper, we examine the power of different nonrigid registration models to detect changes in TBM, and their stability when no real changes are present. Specifically, we investigate an asymmetric version of a recently proposed Unbiased registration method, using mutual information as the matching criterion. We compare matching functionals (sum of squared differences and mutual information), as well as large-deformation registration schemes (viscous fluid and inverse-consistent linear elastic registration methods versus Symmetric and Asymmetric Unbiased registration) for detecting changes in serial MRI scans of 10 elderly normal subjects and 10 patients with Alzheimer's Disease scanned at 2-week and 1-year intervals. We also analyzed registration results when matching images corrupted with artificial noise. We demonstrated that the unbiased methods, both symmetric and asymmetric, have higher reproducibility. The unbiased methods were also less likely to detect changes in the absence of any real physiological change. Moreover, they measured biological deformations more accurately by penalizing bias in the corresponding statistical maps.

Performance evaluation of 2D image registration algorithms with the numeric image registration and comparison platform

International Nuclear Information System (INIS)

Gerganov, G.; Kuvandjiev, V.; Dimitrova, I.; Mitev, K.; Kawrakow, I.

2012-01-01

The objective of this work is to present the capabilities of the NUMERICS web platform for evaluation of the performance of image registration algorithms. The NUMERICS platform is a web accessible tool which provides access to dedicated numerical algorithms for registration and comparison of medical images (http://numerics.phys.uni-sofia.bg). The platform allows comparison of noisy medical images by means of different types of image comparison algorithms, which are based on statistical tests for outliers. The platform also allows 2D image registration with different techniques like Elastic Thin-Plate Spline registration, registration based on rigid transformations, affine transformations, as well as non-rigid image registration based on Mobius transformations. In this work we demonstrate how the platform can be used as a tool for evaluation of the quality of the image registration process. We demonstrate performance evaluation of a deformable image registration technique based on Mobius transformations. The transformations are applied with appropriate cost functions like: Mutual information, Correlation coefficient, Sum of Squared Differences. The accent is on the results provided by the platform to the user and their interpretation in the context of the performance evaluation of 2D image registration. The NUMERICS image registration and image comparison platform provides detailed statistical information about submitted image registration jobs and can be used to perform quantitative evaluation of the performance of different image registration techniques. (authors)

A three-dimensional correlation method for registration of medical images in radiology

Energy Technology Data Exchange (ETDEWEB)

Georgiou, Michalakis; Sfakianakis, George N [Department of Radiology, University of Miami, Jackson Memorial Hospital, Miami, FL 33136 (United States); Nagel, Joachim H [Institute of Biomedical Engineering, University of Stuttgart, Stuttgart 70174 (Germany)

1999-12-31

The availability of methods to register multi-modality images in order to `fuse` them to correlate their information is increasingly becoming an important requirement for various diagnostic and therapeutic procedures. A variety of image registration methods have been developed but they remain limited to specific clinical applications. Assuming rigid body transformation, two images can be registered if their differences are calculated in terms of translation, rotation and scaling. This paper describes the development and testing of a new correlation based approach for three-dimensional image registration. First, the scaling factors introduced by the imaging devices are calculated and compensated for. Then, the two images become translation invariant by computing their three-dimensional Fourier magnitude spectra. Subsequently, spherical coordinate transformation is performed and then the three-dimensional rotation is computed using a novice approach referred to as {sup p}olar Shells{sup .} The method of polar shells maps the three angles of rotation into one rotation and two translations of a two-dimensional function and then proceeds to calculate them using appropriate transformations based on the Fourier invariance properties. A basic assumption in the method is that the three-dimensional rotation is constrained to one large and two relatively small angles. This assumption is generally satisfied in normal clinical settings. The new three-dimensional image registration method was tested with simulations using computer generated phantom data as well as actual clinical data. Performance analysis and accuracy evaluation of the method using computer simulations yielded errors in the sub-pixel range. (authors) 6 refs., 3 figs.

A three-dimensional correlation method for registration of medical images in radiology

International Nuclear Information System (INIS)

Georgiou, Michalakis; Sfakianakis, George N.; Nagel, Joachim H.

1998-01-01

The availability of methods to register multi-modality images in order to 'fuse' them to correlate their information is increasingly becoming an important requirement for various diagnostic and therapeutic procedures. A variety of image registration methods have been developed but they remain limited to specific clinical applications. Assuming rigid body transformation, two images can be registered if their differences are calculated in terms of translation, rotation and scaling. This paper describes the development and testing of a new correlation based approach for three-dimensional image registration. First, the scaling factors introduced by the imaging devices are calculated and compensated for. Then, the two images become translation invariant by computing their three-dimensional Fourier magnitude spectra. Subsequently, spherical coordinate transformation is performed and then the three-dimensional rotation is computed using a novice approach referred to as p olar Shells . The method of polar shells maps the three angles of rotation into one rotation and two translations of a two-dimensional function and then proceeds to calculate them using appropriate transformations based on the Fourier invariance properties. A basic assumption in the method is that the three-dimensional rotation is constrained to one large and two relatively small angles. This assumption is generally satisfied in normal clinical settings. The new three-dimensional image registration method was tested with simulations using computer generated phantom data as well as actual clinical data. Performance analysis and accuracy evaluation of the method using computer simulations yielded errors in the sub-pixel range. (authors)

A hybrid biomechanical intensity based deformable image registration of lung 4DCT

International Nuclear Information System (INIS)

Samavati, Navid; Velec, Michael; Brock, Kristy

2015-01-01

Deformable image registration (DIR) has been extensively studied over the past two decades due to its essential role in many image-guided interventions (IGI). IGI demands a highly accurate registration that maintains its accuracy across the entire region of interest. This work evaluates the improvement in accuracy and consistency by refining the results of Morfeus, a biomechanical model-based DIR algorithm.A hybrid DIR algorithm is proposed based on, a biomechanical model–based DIR algorithm and a refinement step based on a B-spline intensity-based algorithm. Inhale and exhale reconstructions of four-dimensional computed tomography (4DCT) lung images from 31 patients were initially registered using the biomechanical DIR by modeling contact surface between the lungs and the chest cavity. The resulting deformations were then refined using the intensity-based algorithm to reduce any residual uncertainties. Important parameters in the intensity-based algorithm, including grid spacing, number of pyramids, and regularization coefficient, were optimized on 10 randomly-chosen patients (out of 31). Target registration error (TRE) was calculated by measuring the Euclidean distance of common anatomical points on both images after registration. For each patient a minimum of 30 points/lung were used.Grid spacing of 8 mm, 5 levels of grid pyramids, and regularization coefficient of 3.0 were found to provide optimal results on 10 randomly chosen patients. Overall the entire patient population (n = 31), the hybrid method resulted in mean ± SD (90th%) TRE of 1.5 ± 1.4 (2.9) mm compared to 3.1 ± 1.9 (5.6) using biomechanical DIR and 2.6 ± 2.5 (6.1) using intensity-based DIR alone.The proposed hybrid biomechanical modeling intensity based algorithm is a promising DIR technique which could be used in various IGI procedures. The current investigation shows the efficacy of this approach for the registration of 4DCT images of the lungs with average accuracy of 1.5�

Study on Huizhou architecture of point cloud registration based on optimized ICP algorithm

Science.gov (United States)

Zhang, Runmei; Wu, Yulu; Zhang, Guangbin; Zhou, Wei; Tao, Yuqian

2018-03-01

In view of the current point cloud registration software has high hardware requirements, heavy workload and moltiple interactive definition, the source of software with better processing effect is not open, a two--step registration method based on normal vector distribution feature and coarse feature based iterative closest point (ICP) algorithm is proposed in this paper. This method combines fast point feature histogram (FPFH) algorithm, define the adjacency region of point cloud and the calculation model of the distribution of normal vectors, setting up the local coordinate system for each key point, and obtaining the transformation matrix to finish rough registration, the rough registration results of two stations are accurately registered by using the ICP algorithm. Experimental results show that, compared with the traditional ICP algorithm, the method used in this paper has obvious time and precision advantages for large amount of point clouds.

Medical Image Registration by means of a Bio-Inspired Optimization Strategy

Directory of Open Access Journals (Sweden)

Hariton Costin

2012-07-01

Full Text Available Medical imaging mainly treats and processes missing, ambiguous, complementary, redundant and distorted data. Biomedical image registration is the process of geometric overlaying or alignment of two or more 2D/3D images of the same scene, taken at different time slots, from different angles, and/or by different acquisition systems. In medical practice, it is becoming increasingly important in diagnosis, treatment planning, functional studies, computer-guided therapies, and in biomedical research. Technically, image registration implies a complex optimization of different parameters, performed at local or/and global levels. Local optimization methods frequently fail because functions of the involved metrics with respect to transformation parameters are generally nonconvex and irregular. Therefore, global methods are often required, at least at the beginning of the procedure. In this paper, a new evolutionary and bio-inspired approach -- bacterial foraging optimization -- is adapted for single-slice to 3-D PET and CT multimodal image registration. Preliminary results of optimizing the normalized mutual information similarity metric validated the efficacy of the proposed method by using a freely available medical image database.

Computer-aided psychotherapy based on multimodal elicitation, estimation and regulation of emotion.

Science.gov (United States)

Cosić, Krešimir; Popović, Siniša; Horvat, Marko; Kukolja, Davor; Dropuljić, Branimir; Kovač, Bernard; Jakovljević, Miro

2013-09-01

Contemporary psychiatry is looking at affective sciences to understand human behavior, cognition and the mind in health and disease. Since it has been recognized that emotions have a pivotal role for the human mind, an ever increasing number of laboratories and research centers are interested in affective sciences, affective neuroscience, affective psychology and affective psychopathology. Therefore, this paper presents multidisciplinary research results of Laboratory for Interactive Simulation System at Faculty of Electrical Engineering and Computing, University of Zagreb in the stress resilience. Patient's distortion in emotional processing of multimodal input stimuli is predominantly consequence of his/her cognitive deficit which is result of their individual mental health disorders. These emotional distortions in patient's multimodal physiological, facial, acoustic, and linguistic features related to presented stimulation can be used as indicator of patient's mental illness. Real-time processing and analysis of patient's multimodal response related to annotated input stimuli is based on appropriate machine learning methods from computer science. Comprehensive longitudinal multimodal analysis of patient's emotion, mood, feelings, attention, motivation, decision-making, and working memory in synchronization with multimodal stimuli provides extremely valuable big database for data mining, machine learning and machine reasoning. Presented multimedia stimuli sequence includes personalized images, movies and sounds, as well as semantically congruent narratives. Simultaneously, with stimuli presentation patient provides subjective emotional ratings of presented stimuli in terms of subjective units of discomfort/distress, discrete emotions, or valence and arousal. These subjective emotional ratings of input stimuli and corresponding physiological, speech, and facial output features provides enough information for evaluation of patient's cognitive appraisal deficit

Multimodal pain management after arthroscopic surgery

DEFF Research Database (Denmark)

Rasmussen, Sten

Multimodal Pain Management after Arthroscopic Surgery By Sten Rasmussen, M.D. The thesis is based on four randomized controlled trials. The main hypothesis was that multimodal pain treatment provides faster recovery after arthroscopic surgery. NSAID was tested against placebo after knee arthroscopy...

A robo-pigeon based on an innovative multi-mode telestimulation system.

Science.gov (United States)

Yang, Junqing; Huai, Ruituo; Wang, Hui; Lv, Changzhi; Su, Xuecheng

2015-01-01

In this paper, we describe a new multi-mode telestimulation system for brain-microstimulation for the navigation of a robo-pigeon, a new type of bio-robot based on Brain-Computer Interface (BCI) techniques. The multi-mode telestimulation system overcomes neuron adaptation that was a key shortcoming of the previous single-mode stimulation by the use of non-steady TTL biphasic pulses accomplished by randomly alternating pulse modes. To improve efficiency, a new behavior model ("virtual fear") is proposed and applied to the robo-pigeon. Unlike the previous "virtual reward" model, the "virtual fear" behavior model does not require special training. The performance and effectiveness of the system to alleviate the adaptation of neurons was verified by a robo-pigeon navigation test, simultaneously confirming the practicality of the "virtual fear" behavioral model.

A Multi-Modality CMOS Sensor Array for Cell-Based Assay and Drug Screening.

Science.gov (United States)

Chi, Taiyun; Park, Jong Seok; Butts, Jessica C; Hookway, Tracy A; Su, Amy; Zhu, Chengjie; Styczynski, Mark P; McDevitt, Todd C; Wang, Hua

2015-12-01

In this paper, we present a fully integrated multi-modality CMOS cellular sensor array with four sensing modalities to characterize different cell physiological responses, including extracellular voltage recording, cellular impedance mapping, optical detection with shadow imaging and bioluminescence sensing, and thermal monitoring. The sensor array consists of nine parallel pixel groups and nine corresponding signal conditioning blocks. Each pixel group comprises one temperature sensor and 16 tri-modality sensor pixels, while each tri-modality sensor pixel can be independently configured for extracellular voltage recording, cellular impedance measurement (voltage excitation/current sensing), and optical detection. This sensor array supports multi-modality cellular sensing at the pixel level, which enables holistic cell characterization and joint-modality physiological monitoring on the same cellular sample with a pixel resolution of 80 μm × 100 μm. Comprehensive biological experiments with different living cell samples demonstrate the functionality and benefit of the proposed multi-modality sensing in cell-based assay and drug screening.

Histopathology in 3D: From three-dimensional reconstruction to multi-stain and multi-modal analysis

Directory of Open Access Journals (Sweden)

Derek Magee

2015-01-01

Full Text Available Light microscopy applied to the domain of histopathology has traditionally been a two-dimensional imaging modality. Several authors, including the authors of this work, have extended the use of digital microscopy to three dimensions by stacking digital images of serial sections using image-based registration. In this paper, we give an overview of our approach, and of extensions to the approach to register multi-modal data sets such as sets of interleaved histopathology sections with different stains, and sets of histopathology images to radiology volumes with very different appearance. Our approach involves transforming dissimilar images into a multi-channel representation derived from co-occurrence statistics between roughly aligned images.

Practical multimodal care for cancer cachexia.

Science.gov (United States)

Maddocks, Matthew; Hopkinson, Jane; Conibear, John; Reeves, Annie; Shaw, Clare; Fearon, Ken C H

2016-12-01

Cancer cachexia is common and reduces function, treatment tolerability and quality of life. Given its multifaceted pathophysiology a multimodal approach to cachexia management is advocated for, but can be difficult to realise in practice. We use a case-based approach to highlight practical approaches to the multimodal management of cachexia for patients across the cancer trajectory. Four cases with lung cancer spanning surgical resection, radical chemoradiotherapy, palliative chemotherapy and no anticancer treatment are presented. We propose multimodal care approaches that incorporate nutritional support, exercise, and anti-inflammatory agents, on a background of personalized oncology care and family-centred education. Collectively, the cases reveal that multimodal care is part of everyone's remit, often focuses on supported self-management, and demands buy-in from the patient and their family. Once operationalized, multimodal care approaches can be tested pragmatically, including alongside emerging pharmacological cachexia treatments. We demonstrate that multimodal care for cancer cachexia can be achieved using simple treatments and without a dedicated team of specialists. The sharing of advice between health professionals can help build collective confidence and expertise, moving towards a position in which every team member feels they can contribute towards multimodal care.

Time-Of-Flight Camera, Optical Tracker and Computed Tomography in Pairwise Data Registration.

Directory of Open Access Journals (Sweden)

Bartlomiej Pycinski

Full Text Available A growing number of medical applications, including minimal invasive surgery, depends on multi-modal or multi-sensors data processing. Fast and accurate 3D scene analysis, comprising data registration, seems to be crucial for the development of computer aided diagnosis and therapy. The advancement of surface tracking system based on optical trackers already plays an important role in surgical procedures planning. However, new modalities, like the time-of-flight (ToF sensors, widely explored in non-medical fields are powerful and have the potential to become a part of computer aided surgery set-up. Connection of different acquisition systems promises to provide a valuable support for operating room procedures. Therefore, the detailed analysis of the accuracy of such multi-sensors positioning systems is needed.We present the system combining pre-operative CT series with intra-operative ToF-sensor and optical tracker point clouds. The methodology contains: optical sensor set-up and the ToF-camera calibration procedures, data pre-processing algorithms, and registration technique. The data pre-processing yields a surface, in case of CT, and point clouds for ToF-sensor and marker-driven optical tracker representation of an object of interest. An applied registration technique is based on Iterative Closest Point algorithm.The experiments validate the registration of each pair of modalities/sensors involving phantoms of four various human organs in terms of Hausdorff distance and mean absolute distance metrics. The best surface alignment was obtained for CT and optical tracker combination, whereas the worst for experiments involving ToF-camera.The obtained accuracies encourage to further develop the multi-sensors systems. The presented substantive discussion concerning the system limitations and possible improvements mainly related to the depth information produced by the ToF-sensor is useful for computer aided surgery developers.

A Rigid Image Registration Based on the Nonsubsampled Contourlet Transform and Genetic Algorithms

Directory of Open Access Journals (Sweden)

Nasreddine Taleb

2010-09-01

Full Text Available Image registration is a fundamental task used in image processing to match two or more images taken at different times, from different sensors or from different viewpoints. The objective is to find in a huge search space of geometric transformations, an acceptable accurate solution in a reasonable time to provide better registered images. Exhaustive search is computationally expensive and the computational cost increases exponentially with the number of transformation parameters and the size of the data set. In this work, we present an efficient image registration algorithm that uses genetic algorithms within a multi-resolution framework based on the Non-Subsampled Contourlet Transform (NSCT. An adaptable genetic algorithm for registration is adopted in order to minimize the search space. This approach is used within a hybrid scheme applying the two techniques fitness sharing and elitism. Two NSCT based methods are proposed for registration. A comparative study is established between these methods and a wavelet based one. Because the NSCT is a shift-invariant multidirectional transform, the second method is adopted for its search speeding up property. Simulation results clearly show that both proposed techniques are really promising methods for image registration compared to the wavelet approach, while the second technique has led to the best performance results of all. Moreover, to demonstrate the effectiveness of these methods, these registration techniques have been successfully applied to register SPOT, IKONOS and Synthetic Aperture Radar (SAR images. The algorithm has been shown to work perfectly well for multi-temporal satellite images as well, even in the presence of noise.

A rigid image registration based on the nonsubsampled contourlet transform and genetic algorithms.

Science.gov (United States)

Meskine, Fatiha; Chikr El Mezouar, Miloud; Taleb, Nasreddine

2010-01-01

Image registration is a fundamental task used in image processing to match two or more images taken at different times, from different sensors or from different viewpoints. The objective is to find in a huge search space of geometric transformations, an acceptable accurate solution in a reasonable time to provide better registered images. Exhaustive search is computationally expensive and the computational cost increases exponentially with the number of transformation parameters and the size of the data set. In this work, we present an efficient image registration algorithm that uses genetic algorithms within a multi-resolution framework based on the Non-Subsampled Contourlet Transform (NSCT). An adaptable genetic algorithm for registration is adopted in order to minimize the search space. This approach is used within a hybrid scheme applying the two techniques fitness sharing and elitism. Two NSCT based methods are proposed for registration. A comparative study is established between these methods and a wavelet based one. Because the NSCT is a shift-invariant multidirectional transform, the second method is adopted for its search speeding up property. Simulation results clearly show that both proposed techniques are really promising methods for image registration compared to the wavelet approach, while the second technique has led to the best performance results of all. Moreover, to demonstrate the effectiveness of these methods, these registration techniques have been successfully applied to register SPOT, IKONOS and Synthetic Aperture Radar (SAR) images. The algorithm has been shown to work perfectly well for multi-temporal satellite images as well, even in the presence of noise.

Visual tracking for multi-modality computer-assisted image guidance

Science.gov (United States)

Basafa, Ehsan; Foroughi, Pezhman; Hossbach, Martin; Bhanushali, Jasmine; Stolka, Philipp

2017-03-01

With optical cameras, many interventional navigation tasks previously relying on EM, optical, or mechanical guidance can be performed robustly, quickly, and conveniently. We developed a family of novel guidance systems based on wide-spectrum cameras and vision algorithms for real-time tracking of interventional instruments and multi-modality markers. These navigation systems support the localization of anatomical targets, support placement of imaging probe and instruments, and provide fusion imaging. The unique architecture - low-cost, miniature, in-hand stereo vision cameras fitted directly to imaging probes - allows for an intuitive workflow that fits a wide variety of specialties such as anesthesiology, interventional radiology, interventional oncology, emergency medicine, urology, and others, many of which see increasing pressure to utilize medical imaging and especially ultrasound, but have yet to develop the requisite skills for reliable success. We developed a modular system, consisting of hardware (the Optical Head containing the mini cameras) and software (components for visual instrument tracking with or without specialized visual features, fully automated marker segmentation from a variety of 3D imaging modalities, visual observation of meshes of widely separated markers, instant automatic registration, and target tracking and guidance on real-time multi-modality fusion views). From these components, we implemented a family of distinct clinical and pre-clinical systems (for combinations of ultrasound, CT, CBCT, and MRI), most of which have international regulatory clearance for clinical use. We present technical and clinical results on phantoms, ex- and in-vivo animals, and patients.

Learning in data-limited multimodal scenarios: Scandent decision forests and tree-based features.

Science.gov (United States)

Hor, Soheil; Moradi, Mehdi

2016-12-01

Incomplete and inconsistent datasets often pose difficulties in multimodal studies. We introduce the concept of scandent decision trees to tackle these difficulties. Scandent trees are decision trees that optimally mimic the partitioning of the data determined by another decision tree, and crucially, use only a subset of the feature set. We show how scandent trees can be used to enhance the performance of decision forests trained on a small number of multimodal samples when we have access to larger datasets with vastly incomplete feature sets. Additionally, we introduce the concept of tree-based feature transforms in the decision forest paradigm. When combined with scandent trees, the tree-based feature transforms enable us to train a classifier on a rich multimodal dataset, and use it to classify samples with only a subset of features of the training data. Using this methodology, we build a model trained on MRI and PET images of the ADNI dataset, and then test it on cases with only MRI data. We show that this is significantly more effective in staging of cognitive impairments compared to a similar decision forest model trained and tested on MRI only, or one that uses other kinds of feature transform applied to the MRI data. Copyright © 2016. Published by Elsevier B.V.

Performance Evaluation and Optimal Management of Distance-Based Registration Using a Semi-Markov Process

Directory of Open Access Journals (Sweden)

Jae Joon Suh

2017-01-01

Full Text Available We consider the distance-based registration (DBR which is a kind of dynamic location registration scheme in a mobile communication network. In the DBR, the location of a mobile station (MS is updated when it enters a base station more than or equal to a specified distance away from the base station where the location registration for the MS was done last. In this study, we first investigate the existing performance-evaluation methods on the DBR with implicit registration (DBIR presented to improve the performance of the DBR and point out some problems of the evaluation methods. We propose a new performance-evaluation method for the DBIR scheme using a semi-Markov process (SMP which can resolve the controversial issues of the existing methods. The numerical results obtained with the proposed SMP model are compared with those from previous models. It is shown that the SMP model should be considered to get an accurate performance of the DBIR scheme.

Line-Based Registration of Panoramic Images and LiDAR Point Clouds for Mobile Mapping

Directory of Open Access Journals (Sweden)

Tingting Cui

2016-12-01

Full Text Available For multi-sensor integrated systems, such as the mobile mapping system (MMS, data fusion at sensor-level, i.e., the 2D-3D registration between an optical camera and LiDAR, is a prerequisite for higher level fusion and further applications. This paper proposes a line-based registration method for panoramic images and a LiDAR point cloud collected by a MMS. We first introduce the system configuration and specification, including the coordinate systems of the MMS, the 3D LiDAR scanners, and the two panoramic camera models. We then establish the line-based transformation model for the panoramic camera. Finally, the proposed registration method is evaluated for two types of camera models by visual inspection and quantitative comparison. The results demonstrate that the line-based registration method can significantly improve the alignment of the panoramic image and the LiDAR datasets under either the ideal spherical or the rigorous panoramic camera model, with the latter being more reliable.

MINERVA - a multi-modal radiation treatment planning system

Energy Technology Data Exchange (ETDEWEB)

Wemple, C.A. E-mail: cew@enel.gov; Wessol, D.E.; Nigg, D.W.; Cogliati, J.J.; Milvich, M.L.; Frederickson, C.; Perkins, M.; Harkin, G.J

2004-11-01

Researchers at the Idaho National Engineering and Environmental Laboratory and Montana State University have undertaken development of MINERVA, a patient-centric, multi-modal, radiation treatment planning system. This system can be used for planning and analyzing several radiotherapy modalities, either singly or combined, using common modality independent image and geometry construction and dose reporting and guiding. It employs an integrated, lightweight plugin architecture to accommodate multi-modal treatment planning using standard interface components. The MINERVA design also facilitates the future integration of improved planning technologies. The code is being developed with the Java Virtual Machine for interoperability. A full computation path has been established for molecular targeted radiotherapy treatment planning, with the associated transport plugin developed by researchers at the Lawrence Livermore National Laboratory. Development of the neutron transport plugin module is proceeding rapidly, with completion expected later this year. Future development efforts will include development of deformable registration methods, improved segmentation methods for patient model definition, and three-dimensional visualization of the patient images, geometry, and dose data. Transport and source plugins will be created for additional treatment modalities, including brachytherapy, external beam proton radiotherapy, and the EGSnrc/BEAMnrc codes for external beam photon and electron radiotherapy.

Precise localization of dysfunctional areas in vertebro-basilar infarction by FDG- and 0-15-H2O-PET using standardized image analysis and image registration to 3-D MR

International Nuclear Information System (INIS)

Juengling, F.D.; Moser, E.; Nitzsche, E.U.; Kassubek, J.

1999-01-01

The advantages of standardized multimodal image analysis are demonstrated in a case of symptomatic tremor after basilar thrombosis. Functionally and structurally lesioned areas were mapped in Talairach space using 3-D MRI, cerebral FDG-PET and O-15-H 2 O-PET. Structural lesions were found in the left midbrain, thalamus, putamen and cerebellar areas. Voxel-based statistics in comparison to a normal data base revealed hypometabolism in the left thalamus, left red nucleus, left cerebellar hemisphere including dentate nucleus and in the left interior olivary nucleus. The O-15-H 2 O-PET investigation revealed metabolic uncoupling along the rubroolivocerebellar loop. Given the delicate anatomy of the structures involved, image registration and standardized image analysis techniques are essential for a synoptic multimodality analysis of morphological and functional pathology and should generally be used for cerebral PET investigations. (orig.) [de

Learning-Based Approaches to Deformable Image Registration

NARCIS (Netherlands)

Münzing, SEA

2014-01-01

Accurate registration of images is an important and often crucial step in many areas of image processing and analysis, yet it is only used in a small percentage of possible applications. Automated registration methods are not considered to be sufficiently robust to handle complex deformations and

Image Registration for PET/CT and CT Images with Particle Swarm Optimization

International Nuclear Information System (INIS)

Lee, Hak Jae; Kim, Yong Kwon; Lee, Ki Sung; Choi, Jong Hak; Kim, Chang Kyun; Moon, Guk Hyun; Joo, Sung Kwan; Kim, Kyeong Min; Cheon, Gi Jeong

2009-01-01

Image registration is a fundamental task in image processing used to match two or more images. It gives new information to the radiologists by matching images from different modalities. The objective of this study is to develop 2D image registration algorithm for PET/CT and CT images acquired by different systems at different times. We matched two CT images first (one from standalone CT and the other from PET/CT) that contain affluent anatomical information. Then, we geometrically transformed PET image according to the results of transformation parameters calculated by the previous step. We have used Affine transform to match the target and reference images. For the similarity measure, mutual information was explored. Use of particle swarm algorithm optimized the performance by finding the best matched parameter set within a reasonable amount of time. The results show good agreements of the images between PET/CT and CT. We expect the proposed algorithm can be used not only for PET/CT and CT image registration but also for different multi-modality imaging systems such as SPECT/CT, MRI/PET and so on.

Tensor-based morphometry with stationary velocity field diffeomorphic registration: application to ADNI.

Science.gov (United States)

Bossa, Matias; Zacur, Ernesto; Olmos, Salvador

2010-07-01

Tensor-based morphometry (TBM) is an analysis technique where anatomical information is characterized by means of the spatial transformations mapping a customized template with the observed images. Therefore, accurate inter-subject non-rigid registration is an essential prerequisite for both template estimation and image warping. Subsequent statistical analysis on the spatial transformations is performed to highlight voxel-wise differences. Most of previous TBM studies did not explore the influence of the registration parameters, such as the parameters defining the deformation and the regularization models. In this work performance evaluation of TBM using stationary velocity field (SVF) diffeomorphic registration was performed in a subset of subjects from Alzheimer's Disease Neuroimaging Initiative (ADNI) study. A wide range of values of the registration parameters that define the transformation smoothness and the balance between image matching and regularization were explored in the evaluation. The proposed methodology provided brain atrophy maps with very detailed anatomical resolution and with a high significance level compared with results recently published on the same data set using a non-linear elastic registration method. Copyright (c) 2010 Elsevier Inc. All rights reserved.

COMPARISON OF VOLUMETRIC REGISTRATION ALGORITHMS FOR TENSOR-BASED MORPHOMETRY.

Science.gov (United States)

Villalon, Julio; Joshi, Anand A; Toga, Arthur W; Thompson, Paul M

2011-01-01

Nonlinear registration of brain MRI scans is often used to quantify morphological differences associated with disease or genetic factors. Recently, surface-guided fully 3D volumetric registrations have been developed that combine intensity-guided volume registrations with cortical surface constraints. In this paper, we compare one such algorithm to two popular high-dimensional volumetric registration methods: large-deformation viscous fluid registration, formulated in a Riemannian framework, and the diffeomorphic "Demons" algorithm. We performed an objective morphometric comparison, by using a large MRI dataset from 340 young adult twin subjects to examine 3D patterns of correlations in anatomical volumes. Surface-constrained volume registration gave greater effect sizes for detecting morphometric associations near the cortex, while the other two approaches gave greater effects sizes subcortically. These findings suggest novel ways to combine the advantages of multiple methods in the future.

Mode-Selective Wavelength Conversion Based on Four-Wave Mixing in a Multimode Silicon Waveguide

DEFF Research Database (Denmark)

Ding, Yunhong; Xu, Jing; Ou, Haiyan

2013-01-01

We report all-optical mode-selective wavelength conversion based on four-wave mixing in a multimode Si waveguide. A two-mode division multiplexing circuit using tapered directional coupler based (de)multiplexers is used for the application. Experimental results show clear eye-diagrams and moderate...

A multimodal parallel architecture: A cognitive framework for multimodal interactions.

Science.gov (United States)

Cohn, Neil

2016-01-01

Human communication is naturally multimodal, and substantial focus has examined the semantic correspondences in speech-gesture and text-image relationships. However, visual narratives, like those in comics, provide an interesting challenge to multimodal communication because the words and/or images can guide the overall meaning, and both modalities can appear in complicated "grammatical" sequences: sentences use a syntactic structure and sequential images use a narrative structure. These dual structures create complexity beyond those typically addressed by theories of multimodality where only a single form uses combinatorial structure, and also poses challenges for models of the linguistic system that focus on single modalities. This paper outlines a broad theoretical framework for multimodal interactions by expanding on Jackendoff's (2002) parallel architecture for language. Multimodal interactions are characterized in terms of their component cognitive structures: whether a particular modality (verbal, bodily, visual) is present, whether it uses a grammatical structure (syntax, narrative), and whether it "dominates" the semantics of the overall expression. Altogether, this approach integrates multimodal interactions into an existing framework of language and cognition, and characterizes interactions between varying complexity in the verbal, bodily, and graphic domains. The resulting theoretical model presents an expanded consideration of the boundaries of the "linguistic" system and its involvement in multimodal interactions, with a framework that can benefit research on corpus analyses, experimentation, and the educational benefits of multimodality. Copyright © 2015.

AUTOMATED FEATURE BASED TLS DATA REGISTRATION FOR 3D BUILDING MODELING

OpenAIRE

K. Kitamura; N. Kochi; S. Kaneko

2012-01-01

In this paper we present a novel method for the registration of point cloud data obtained using terrestrial laser scanner (TLS). The final goal of our investigation is the automated reconstruction of CAD drawings and the 3D modeling of objects surveyed by TLS. Because objects are scanned from multiple positions, individual point cloud need to be registered to the same coordinate system. We propose in this paper an automated feature based registration procedure. Our proposed method does not re...

The skill of surface registration in CT-based navigation system for total hip arthroplasty

International Nuclear Information System (INIS)

Hananouchi, T.; Sugano, N.; Nishii, T.; Miki, H.; Sakai, T.; Yoshikawa, H.; Iwana, D.; Yamamura, M.; Nakamura, N.

2007-01-01

Surface registration of the CT-based navigation system, which is a matching between computational and real spatial spaces, is a key step to guarantee the accuracy of navigation. However, it has not been well described how the accuracy is affected by the registration skill of surgeon. Here, we reported the difference of the registration error between eight surgeons with the experience of navigation and six apprentice surgeons. A cadaveric pelvic model with an acetabular cup was made to measure the skill and learning curve of registration. After surface registration, two cup angles (inclination and anteversion) were recorded in the navigation system and the variance of these cup angles in ten trials were compared between the experienced surgeons and apprentices. In addition, we investigated whether the accuracy of registration by the apprentices was improved by visual information on how to take the surface points. The results showed that there was statistically significant difference in the accuracy of registration between the two groups. The accuracy of the second ten trials after getting the visual information showed great improvements. (orig.)

Applying Connectivist Principles and the Task-Based Approach to the Design of a Multimodal Didactic Unit

Directory of Open Access Journals (Sweden)

Yeraldine Aldana Gutiérrez

2012-12-01

Full Text Available This article describes the pedagogical intervention developed in a public school as part of the research “Exploring Communications Practices through Facebook as a Mediatic Device”, framed within the computer mediated communications field. Twelve ninth graders’ communications practices were explored and addressed by means of multimodal technological resources and tasks based on the connectivist learning view. As a result, a didactic unit was designed in the form of the digital book Diverface. This one in turn displayed information through different media channels and semiotic elements to support its multimodal features. Teachers and students might thus need to reconstruct an alternative multimodal literacy so that they can produce and interpret texts of the same nature in online environments.

FPFH-based graph matching for 3D point cloud registration

Science.gov (United States)

Zhao, Jiapeng; Li, Chen; Tian, Lihua; Zhu, Jihua

2018-04-01

Correspondence detection is a vital step in point cloud registration and it can help getting a reliable initial alignment. In this paper, we put forward an advanced point feature-based graph matching algorithm to solve the initial alignment problem of rigid 3D point cloud registration with partial overlap. Specifically, Fast Point Feature Histograms are used to determine the initial possible correspondences firstly. Next, a new objective function is provided to make the graph matching more suitable for partially overlapping point cloud. The objective function is optimized by the simulated annealing algorithm for final group of correct correspondences. Finally, we present a novel set partitioning method which can transform the NP-hard optimization problem into a O(n3)-solvable one. Experiments on the Stanford and UWA public data sets indicates that our method can obtain better result in terms of both accuracy and time cost compared with other point cloud registration methods.

Vision 20/20: Simultaneous CT-MRI — Next chapter of multimodality imaging

International Nuclear Information System (INIS)

Wang, Ge; Xi, Yan; Gjesteby, Lars; Getzin, Matthew; Yang, Qingsong; Cong, Wenxiang; Kalra, Mannudeep; Murugan, Venkatesh; Vannier, Michael

2015-01-01

Multimodality imaging systems such as positron emission tomography-computed tomography (PET-CT) and MRI-PET are widely available, but a simultaneous CT-MRI instrument has not been developed. Synergies between independent modalities, e.g., CT, MRI, and PET/SPECT can be realized with image registration, but such postprocessing suffers from registration errors that can be avoided with synchronized data acquisition. The clinical potential of simultaneous CT-MRI is significant, especially in cardiovascular and oncologic applications where studies of the vulnerable plaque, response to cancer therapy, and kinetic and dynamic mechanisms of targeted agents are limited by current imaging technologies. The rationale, feasibility, and realization of simultaneous CT-MRI are described in this perspective paper. The enabling technologies include interior tomography, unique gantry designs, open magnet and RF sequences, and source and detector adaptation. Based on the experience with PET-CT, PET-MRI, and MRI-LINAC instrumentation where hardware innovation and performance optimization were instrumental to construct commercial systems, the authors provide top-level concepts for simultaneous CT-MRI to meet clinical requirements and new challenges. Simultaneous CT-MRI fills a major gap of modality coupling and represents a key step toward the so-called “omnitomography” defined as the integration of all relevant imaging modalities for systems biology and precision medicine

Vision 20/20: Simultaneous CT-MRI — Next chapter of multimodality imaging

Energy Technology Data Exchange (ETDEWEB)

Wang, Ge, E-mail: wangg6@rpi.edu; Xi, Yan; Gjesteby, Lars; Getzin, Matthew; Yang, Qingsong; Cong, Wenxiang [Biomedical Imaging Center/Cluster, Rensselaer Polytechnic Institute, Troy, New York 12180 (United States); Kalra, Mannudeep; Murugan, Venkatesh [Department of Imaging, Massachusetts General Hospital, 55 Fruit Street, Boston, Massachusetts 02114 (United States); Vannier, Michael [Department of Radiology, University of Chicago, Chicago, Illinois 60637 (United States)

2015-10-15

Multimodality imaging systems such as positron emission tomography-computed tomography (PET-CT) and MRI-PET are widely available, but a simultaneous CT-MRI instrument has not been developed. Synergies between independent modalities, e.g., CT, MRI, and PET/SPECT can be realized with image registration, but such postprocessing suffers from registration errors that can be avoided with synchronized data acquisition. The clinical potential of simultaneous CT-MRI is significant, especially in cardiovascular and oncologic applications where studies of the vulnerable plaque, response to cancer therapy, and kinetic and dynamic mechanisms of targeted agents are limited by current imaging technologies. The rationale, feasibility, and realization of simultaneous CT-MRI are described in this perspective paper. The enabling technologies include interior tomography, unique gantry designs, open magnet and RF sequences, and source and detector adaptation. Based on the experience with PET-CT, PET-MRI, and MRI-LINAC instrumentation where hardware innovation and performance optimization were instrumental to construct commercial systems, the authors provide top-level concepts for simultaneous CT-MRI to meet clinical requirements and new challenges. Simultaneous CT-MRI fills a major gap of modality coupling and represents a key step toward the so-called “omnitomography” defined as the integration of all relevant imaging modalities for systems biology and precision medicine.

TH-CD-206-09: Learning-Based MRI-CT Prostate Registration Using Spare Patch-Deformation Dictionary

International Nuclear Information System (INIS)

Yang, X; Jani, A; Rossi, P; Mao, H; Curran, W; Liu, T

2016-01-01

Purpose: To enable MRI-guided prostate radiotherapy, MRI-CT deformable registration is required to map the MRI-defined tumor and key organ contours onto the CT images. Due to the intrinsic differences in grey-level intensity characteristics between MRI and CT images, the integration of MRI into CT-based radiotherapy is very challenging. We are developing a learning-based registration approach to address this technical challenge. Methods: We propose to estimate the deformation between MRI and CT images in a patch-wise fashion by using the sparse representation technique. Specifically, we assume that two image patches should follow the same deformation if their patch-wise appearance patterns are similar. We first extract a set of key points in the new CT image. Then, for each key point, we adaptively construct a coupled dictionary from the training MRI-CT images, where each coupled element includes both appearance and deformation of the same image patch. After calculating the sparse coefficients in representing the patch appearance of each key point based on the constructed dictionary, we can predict the deformation for this point by applying the same sparse coefficients to the respective deformations in the dictionary. Results: This registration technique was validated with 10 prostate-cancer patients’ data and its performance was compared with the commonly used free-form-deformation-based registration. Several landmarks in both images were identified to evaluate the accuracy of our approach. Overall, the averaged target registration error of the intensity-based registration and the proposed method was 3.8±0.4 mm and 1.9±0.3 mm, respectively. Conclusion: We have developed a novel prostate MR-CT registration approach based on patch-deformation dictionary, demonstrated its clinical feasibility, and validated its accuracy. This technique will either reduce or compensate for the effect of patient-specific treatment variation measured during the course of

TH-CD-206-09: Learning-Based MRI-CT Prostate Registration Using Spare Patch-Deformation Dictionary

Energy Technology Data Exchange (ETDEWEB)

Yang, X; Jani, A; Rossi, P; Mao, H; Curran, W; Liu, T [Emory University, Atlanta, GA (United States)

2016-06-15

Purpose: To enable MRI-guided prostate radiotherapy, MRI-CT deformable registration is required to map the MRI-defined tumor and key organ contours onto the CT images. Due to the intrinsic differences in grey-level intensity characteristics between MRI and CT images, the integration of MRI into CT-based radiotherapy is very challenging. We are developing a learning-based registration approach to address this technical challenge. Methods: We propose to estimate the deformation between MRI and CT images in a patch-wise fashion by using the sparse representation technique. Specifically, we assume that two image patches should follow the same deformation if their patch-wise appearance patterns are similar. We first extract a set of key points in the new CT image. Then, for each key point, we adaptively construct a coupled dictionary from the training MRI-CT images, where each coupled element includes both appearance and deformation of the same image patch. After calculating the sparse coefficients in representing the patch appearance of each key point based on the constructed dictionary, we can predict the deformation for this point by applying the same sparse coefficients to the respective deformations in the dictionary. Results: This registration technique was validated with 10 prostate-cancer patients’ data and its performance was compared with the commonly used free-form-deformation-based registration. Several landmarks in both images were identified to evaluate the accuracy of our approach. Overall, the averaged target registration error of the intensity-based registration and the proposed method was 3.8±0.4 mm and 1.9±0.3 mm, respectively. Conclusion: We have developed a novel prostate MR-CT registration approach based on patch-deformation dictionary, demonstrated its clinical feasibility, and validated its accuracy. This technique will either reduce or compensate for the effect of patient-specific treatment variation measured during the course of

Acoustic multimode interference and self-imaging phenomena realized in multimodal phononic crystal waveguides

International Nuclear Information System (INIS)

Zou, Qiushun; Yu, Tianbao; Liu, Jiangtao; Wang, Tongbiao; Liao, Qinghua; Liu, Nianhua

2015-01-01

We report an acoustic multimode interference effect and self-imaging phenomena in an acoustic multimode waveguide system which consists of M parallel phononic crystal waveguides (M-PnCWs). Results show that the self-imaging principle remains applicable for acoustic waveguides just as it does for optical multimode waveguides. To achieve the dispersions and replicas of the input acoustic waves produced along the propagation direction, we performed the finite element method on M-PnCWs, which support M guided modes within the target frequency range. The simulation results show that single images (including direct and mirrored images) and N-fold images (N is an integer) are identified along the propagation direction with asymmetric and symmetric incidence discussed separately. The simulated positions of the replicas agree well with the calculated values that are theoretically decided by self-imaging conditions based on the guided mode propagation analysis. Moreover, the potential applications based on this self-imaging effect for acoustic wavelength de-multiplexing and beam splitting in the acoustic field are also presented. (paper)

Researchers and teachers learning together and from each other using video-based multimodal analysis

DEFF Research Database (Denmark)

Davidsen, Jacob; Vanderlinde, Ruben

2014-01-01

integrated touch-screens into their teaching and learning. This paper examines the methodological usefulness of video-based multimodal analysis. Through reflection on the research project, we discuss how, by using video-based multimodal analysis, researchers and teachers can study children’s touch......This paper discusses a year-long technology integration project, during which teachers and researchers joined forces to explore children’s collaborative activities through the use of touch-screens. In the research project, discussed in this paper, 16 touch-screens were integrated into teaching...... and learning activities in two separate classrooms; the learning and collaborative processes were captured by using video, collecting over 150 hours of footage. By using digital research technologies and a longitudinal design, the authors of the research project studied how teachers and children gradually...

Multimodal Biometric System Based on the Recognition of Face and Both Irises

Directory of Open Access Journals (Sweden)

Yeong Gon Kim

2012-09-01

Full Text Available The performance of unimodal biometric systems (based on a single modality such as face or fingerprint has to contend with various problems, such as illumination variation, skin condition and environmental conditions, and device variations. Therefore, multimodal biometric systems have been used to overcome the limitations of unimodal biometrics and provide high accuracy recognition. In this paper, we propose a new multimodal biometric system based on score level fusion of face and both irises' recognition. Our study has the following novel features. First, the device proposed acquires images of the face and both irises simultaneously. The proposed device consists of a face camera, two iris cameras, near-infrared illuminators and cold mirrors. Second, fast and accurate iris detection is based on two circular edge detections, which are accomplished in the iris image on the basis of the size of the iris detected in the face image. Third, the combined accuracy is enhanced by combining each score for the face and both irises using a support vector machine. The experimental results show that the equal error rate for the proposed method is 0.131%, which is lower than that of face or iris recognition and other fusion methods.

Image registration assessment in radiotherapy image guidance based on control chart monitoring.

Science.gov (United States)

Xia, Wenyao; Breen, Stephen L

2018-04-01

Image guidance with cone beam computed tomography in radiotherapy can guarantee the precision and accuracy of patient positioning prior to treatment delivery. During the image guidance process, operators need to take great effort to evaluate the image guidance quality before correcting a patient's position. This work proposes an image registration assessment method based on control chart monitoring to reduce the effort taken by the operator. According to the control chart plotted by daily registration scores of each patient, the proposed method can quickly detect both alignment errors and image quality inconsistency. Therefore, the proposed method can provide a clear guideline for the operators to identify unacceptable image quality and unacceptable image registration with minimal effort. Experimental results demonstrate that by using control charts from a clinical database of 10 patients undergoing prostate radiotherapy, the proposed method can quickly identify out-of-control signals and find special cause of out-of-control registration events.

Neural network-based feature point descriptors for registration of optical and SAR images

Science.gov (United States)

Abulkhanov, Dmitry; Konovalenko, Ivan; Nikolaev, Dmitry; Savchik, Alexey; Shvets, Evgeny; Sidorchuk, Dmitry

2018-04-01

Registration of images of different nature is an important technique used in image fusion, change detection, efficient information representation and other problems of computer vision. Solving this task using feature-based approaches is usually more complex than registration of several optical images because traditional feature descriptors (SIFT, SURF, etc.) perform poorly when images have different nature. In this paper we consider the problem of registration of SAR and optical images. We train neural network to build feature point descriptors and use RANSAC algorithm to align found matches. Experimental results are presented that confirm the method's effectiveness.

MamMoeT : An intelligent agent-based communication support platform for multimodal transport

NARCIS (Netherlands)

Dullaert, Wout; Neutens, Tijs; Vanden Berghe, Greet; Vermeulen, Tijs; Vernimmen, Bert; Witlox, Frank

In this paper, an intelligent agent-based communication support platform for multimodal transport is developed. The rationale for doing so is found in the potential of such a system to increase cost efficiency, service and safety for different transport-related actors. Although, at present several

High-Precision Registration of Point Clouds Based on Sphere Feature Constraints

Directory of Open Access Journals (Sweden)

Junhui Huang

2016-12-01

Full Text Available Point cloud registration is a key process in multi-view 3D measurements. Its precision affects the measurement precision directly. However, in the case of the point clouds with non-overlapping areas or curvature invariant surface, it is difficult to achieve a high precision. A high precision registration method based on sphere feature constraint is presented to overcome the difficulty in the paper. Some known sphere features with constraints are used to construct virtual overlapping areas. The virtual overlapping areas provide more accurate corresponding point pairs and reduce the influence of noise. Then the transformation parameters between the registered point clouds are solved by an optimization method with weight function. In that case, the impact of large noise in point clouds can be reduced and a high precision registration is achieved. Simulation and experiments validate the proposed method.

Feature-Based Retinal Image Registration Using D-Saddle Feature

Directory of Open Access Journals (Sweden)

Roziana Ramli

2017-01-01

Full Text Available Retinal image registration is important to assist diagnosis and monitor retinal diseases, such as diabetic retinopathy and glaucoma. However, registering retinal images for various registration applications requires the detection and distribution of feature points on the low-quality region that consists of vessels of varying contrast and sizes. A recent feature detector known as Saddle detects feature points on vessels that are poorly distributed and densely positioned on strong contrast vessels. Therefore, we propose a multiresolution difference of Gaussian pyramid with Saddle detector (D-Saddle to detect feature points on the low-quality region that consists of vessels with varying contrast and sizes. D-Saddle is tested on Fundus Image Registration (FIRE Dataset that consists of 134 retinal image pairs. Experimental results show that D-Saddle successfully registered 43% of retinal image pairs with average registration accuracy of 2.329 pixels while a lower success rate is observed in other four state-of-the-art retinal image registration methods GDB-ICP (28%, Harris-PIIFD (4%, H-M (16%, and Saddle (16%. Furthermore, the registration accuracy of D-Saddle has the weakest correlation (Spearman with the intensity uniformity metric among all methods. Finally, the paired t-test shows that D-Saddle significantly improved the overall registration accuracy of the original Saddle.

“Abstractive description” of land registration system based on the theory of “public confidence”

Directory of Open Access Journals (Sweden)

Nasrini Tabatabai Hesari

2014-10-01

Full Text Available the system of land registration is protective formalism that is formed based on the theory of “public confidence”. This theory presumes that what reflected by the land registration offices is based on the legal fact. This theory, which provides legal stability and security in transactions, is manifested in three guiding principles including “mirror principle”, “curtain principle” and “insurance principle”, and offers an “abstractive description” to a land registration system. This character has different effects on diverse legal systems and can be studied for both positive and negative systems.

Deep assessment: an exploratory study of game-based, multimodal learning in Epidemic

Directory of Open Access Journals (Sweden)

Jennifer Jenson

2016-03-01

Full Text Available n this study, we examine what and how intermediate age students learned from playing in a health-focused game-based digital learning environment, Epidemic. Epidemic is a playful interactive environment designed to deliver factual knowledge, invite critical understanding, and encourage effective self-care practices in dealing with viral contagious diseases, using a social networking interface to integrate both serious games and game-like multimodal design projects. Epidemic invites a playful approach to its deadly serious core concern – communicable disease – in order to see what happens when students are encouraged to critically approach information from multiple or contradictory perspectives. To identify what participants learned while interacting within Epidemic, we deployed two instructional and assessment models, noting the differences each instructional approach could potentially make, and what approach to assessment might help us evaluate game-based learning. We found that each approach provided importantly different perspectives on what and how students learned, and on the very meaning of student success. Recognizing that traditional assessment tools based in print-cultural literacy may prove increasingly ill-suited for assessing emergent multimodal literacies in game-based learning environments, this study seeks to contribute to a growing body of work on the development of novel assessments for learning.

Population based ranking of frameless CT-MRI registration methods

Energy Technology Data Exchange (ETDEWEB)

Opposits, Gabor; Kis, Sandor A.; Tron, Lajos; Emri, Miklos [Debrecen Univ. (Hungary). Dept. of Nuclear Medicine; Berenyi, Ervin [Debrecen Univ. (Hungary). Dept. of Biomedical Laboratory and Imaging Science; Takacs, Endre [Rotating Gamma Ltd., Debrecen (Hungary); Dobai, Jozsef G.; Bognar, Laszlo [Debrecen Univ., Medical Center (Hungary). Dept. of Neurosurgery; Szuecs, Bernadett [ScanoMed Ltd., Debrecen (Hungary)

2015-07-01

Clinical practice often requires simultaneous information obtained by two different imaging modalities. Registration algorithms are commonly used for this purpose. Automated procedures are very helpful in cases when the same kind of registration has to be performed on images of a high number of subjects. Radiotherapists would prefer to use the best automated method to assist therapy planning, however there are not accepted procedures for ranking the different registration algorithms. We were interested in developing a method to measure the population level performance of CT-MRI registration algorithms by a parameter of values in the [0,1] interval. Pairs of CT and MRI images were collected from 1051 subjects. Results of an automated registration were corrected manually until a radiologist and a neurosurgeon expert both accepted the result as good. This way 1051 registered MRI images were produced by the same pair of experts to be used as gold standards for the evaluation of the performance of other registration algorithms. Pearson correlation coefficient, mutual information, normalized mutual information, Kullback-Leibler divergence, L{sub 1} norm and square L{sub 2} norm (dis)similarity measures were tested for sensitivity to indicate the extent of (dis)similarity of a pair of individual mismatched images. The square Hellinger distance proved suitable to grade the performance of registration algorithms at population level providing the developers with a valuable tool to rank algorithms. The developed procedure provides an objective method to find the registration algorithm performing the best on the population level out of newly constructed or available preselected ones.

Multimode-singlemode-multimode fiber sensor for alcohol sensing application

Science.gov (United States)

Rofi'ah, Iftihatur; Hatta, A. M.; Sekartedjo, Sekartedjo

2016-11-01

Alcohol is volatile and flammable liquid which is soluble substances both on polar and non polar substances that has been used in some industrial sectors. Alcohol detection method now widely used one of them is the optical fiber sensor. In this paper used fiber optic sensor based on Multimode-Single-mode-Multimode (MSM) to detect alcohol solution at a concentration range of 0-3%. The working principle of sensor utilizes the modal interference between the core modes and the cladding modes, thus make the sensor sensitive to environmental changes. The result showed that characteristic of the sensor not affect the length of the single-mode fiber (SMF). We obtain that the sensor with a length of 5 mm of single-mode can sensing the alcohol with a sensitivity of 0.107 dB/v%.

Learning-based deformable image registration for infant MR images in the first year of life.

Science.gov (United States)

Hu, Shunbo; Wei, Lifang; Gao, Yaozong; Guo, Yanrong; Wu, Guorong; Shen, Dinggang

2017-01-01

Many brain development studies have been devoted to investigate dynamic structural and functional changes in the first year of life. To quantitatively measure brain development in such a dynamic period, accurate image registration for different infant subjects with possible large age gap is of high demand. Although many state-of-the-art image registration methods have been proposed for young and elderly brain images, very few registration methods work for infant brain images acquired in the first year of life, because of (a) large anatomical changes due to fast brain development and (b) dynamic appearance changes due to white-matter myelination. To address these two difficulties, we propose a learning-based registration method to not only align the anatomical structures but also alleviate the appearance differences between two arbitrary infant MR images (with large age gap) by leveraging the regression forest to predict both the initial displacement vector and appearance changes. Specifically, in the training stage, two regression models are trained separately, with (a) one model learning the relationship between local image appearance (of one development phase) and its displacement toward the template (of another development phase) and (b) another model learning the local appearance changes between the two brain development phases. Then, in the testing stage, to register a new infant image to the template, we first predict both its voxel-wise displacement and appearance changes by the two learned regression models. Since such initializations can alleviate significant appearance and shape differences between new infant image and the template, it is easy to just use a conventional registration method to refine the remaining registration. We apply our proposed registration method to align 24 infant subjects at five different time points (i.e., 2-week-old, 3-month-old, 6-month-old, 9-month-old, and 12-month-old), and achieve more accurate and robust registration

Impact of Computed Tomography Image Quality on Image-Guided Radiation Therapy Based on Soft Tissue Registration

International Nuclear Information System (INIS)

Morrow, Natalya V.; Lawton, Colleen A.; Qi, X. Sharon; Li, X. Allen

2012-01-01

Purpose: In image-guided radiation therapy (IGRT), different computed tomography (CT) modalities with varying image quality are being used to correct for interfractional variations in patient set-up and anatomy changes, thereby reducing clinical target volume to the planning target volume (CTV-to-PTV) margins. We explore how CT image quality affects patient repositioning and CTV-to-PTV margins in soft tissue registration-based IGRT for prostate cancer patients. Methods and Materials: Four CT-based IGRT modalities used for prostate RT were considered in this study: MV fan beam CT (MVFBCT) (Tomotherapy), MV cone beam CT (MVCBCT) (MVision; Siemens), kV fan beam CT (kVFBCT) (CTVision, Siemens), and kV cone beam CT (kVCBCT) (Synergy; Elekta). Daily shifts were determined by manual registration to achieve the best soft tissue agreement. Effect of image quality on patient repositioning was determined by statistical analysis of daily shifts for 136 patients (34 per modality). Inter- and intraobserver variability of soft tissue registration was evaluated based on the registration of a representative scan for each CT modality with its corresponding planning scan. Results: Superior image quality with the kVFBCT resulted in reduced uncertainty in soft tissue registration during IGRT compared with other image modalities for IGRT. The largest interobserver variations of soft tissue registration were 1.1 mm, 2.5 mm, 2.6 mm, and 3.2 mm for kVFBCT, kVCBCT, MVFBCT, and MVCBCT, respectively. Conclusions: Image quality adversely affects the reproducibility of soft tissue-based registration for IGRT and necessitates a careful consideration of residual uncertainties in determining different CTV-to-PTV margins for IGRT using different image modalities.

Impact of Computed Tomography Image Quality on Image-Guided Radiation Therapy Based on Soft Tissue Registration

Energy Technology Data Exchange (ETDEWEB)

Morrow, Natalya V.; Lawton, Colleen A. [Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin (United States); Qi, X. Sharon [Department of Radiation Oncology, University of Colorado Denver, Denver, Colorado (United States); Li, X. Allen, E-mail: ali@mcw.edu [Department of Radiation Oncology, Medical College of Wisconsin, Milwaukee, Wisconsin (United States)

2012-04-01

Purpose: In image-guided radiation therapy (IGRT), different computed tomography (CT) modalities with varying image quality are being used to correct for interfractional variations in patient set-up and anatomy changes, thereby reducing clinical target volume to the planning target volume (CTV-to-PTV) margins. We explore how CT image quality affects patient repositioning and CTV-to-PTV margins in soft tissue registration-based IGRT for prostate cancer patients. Methods and Materials: Four CT-based IGRT modalities used for prostate RT were considered in this study: MV fan beam CT (MVFBCT) (Tomotherapy), MV cone beam CT (MVCBCT) (MVision; Siemens), kV fan beam CT (kVFBCT) (CTVision, Siemens), and kV cone beam CT (kVCBCT) (Synergy; Elekta). Daily shifts were determined by manual registration to achieve the best soft tissue agreement. Effect of image quality on patient repositioning was determined by statistical analysis of daily shifts for 136 patients (34 per modality). Inter- and intraobserver variability of soft tissue registration was evaluated based on the registration of a representative scan for each CT modality with its corresponding planning scan. Results: Superior image quality with the kVFBCT resulted in reduced uncertainty in soft tissue registration during IGRT compared with other image modalities for IGRT. The largest interobserver variations of soft tissue registration were 1.1 mm, 2.5 mm, 2.6 mm, and 3.2 mm for kVFBCT, kVCBCT, MVFBCT, and MVCBCT, respectively. Conclusions: Image quality adversely affects the reproducibility of soft tissue-based registration for IGRT and necessitates a careful consideration of residual uncertainties in determining different CTV-to-PTV margins for IGRT using different image modalities.

Image registration with uncertainty analysis

Science.gov (United States)

Simonson, Katherine M [Cedar Crest, NM

2011-03-22

In an image registration method, edges are detected in a first image and a second image. A percentage of edge pixels in a subset of the second image that are also edges in the first image shifted by a translation is calculated. A best registration point is calculated based on a maximum percentage of edges matched. In a predefined search region, all registration points other than the best registration point are identified that are not significantly worse than the best registration point according to a predetermined statistical criterion.

Multimodal Image-Based Virtual Reality Presurgical Simulation and Evaluation for Trigeminal Neuralgia and Hemifacial Spasm.

Science.gov (United States)

Yao, Shujing; Zhang, Jiashu; Zhao, Yining; Hou, Yuanzheng; Xu, Xinghua; Zhang, Zhizhong; Kikinis, Ron; Chen, Xiaolei

2018-05-01

To address the feasibility and predictive value of multimodal image-based virtual reality in detecting and assessing features of neurovascular confliction (NVC), particularly regarding the detection of offending vessels, degree of compression exerted on the nerve root, in patients who underwent microvascular decompression for nonlesional trigeminal neuralgia and hemifacial spasm (HFS). This prospective study includes 42 consecutive patients who underwent microvascular decompression for classic primary trigeminal neuralgia or HFS. All patients underwent preoperative 1.5-T magnetic resonance imaging (MRI) with T2-weighted three-dimensional (3D) sampling perfection with application-optimized contrasts by using different flip angle evolutions, 3D time-of-flight magnetic resonance angiography, and 3D T1-weighted gadolinium-enhanced sequences in combination, whereas 2 patients underwent extra experimental preoperative 7.0-T MRI scans with the same imaging protocol. Multimodal MRIs were then coregistered with open-source software 3D Slicer, followed by 3D image reconstruction to generate virtual reality (VR) images for detection of possible NVC in the cerebellopontine angle. Evaluations were performed by 2 reviewers and compared with the intraoperative findings. For detection of NVC, multimodal image-based VR sensitivity was 97.6% (40/41) and specificity was 100% (1/1). Compared with the intraoperative findings, the κ coefficients for predicting the offending vessel and the degree of compression were >0.75 (P < 0.001). The 7.0-T scans have a clearer view of vessels in the cerebellopontine angle, which may have significant impact on detection of small-caliber offending vessels with relatively slow flow speed in cases of HFS. Multimodal image-based VR using 3D sampling perfection with application-optimized contrasts by using different flip angle evolutions in combination with 3D time-of-flight magnetic resonance angiography sequences proved to be reliable in detecting NVC

Mode-selective wavelength conversion based on four-wave mixing in a multimode silicon waveguide

DEFF Research Database (Denmark)

Ding, Yunhong; Xu, Jing; Ou, Haiyan

2014-01-01

to phase mismatch. A two-mode division multiplexing circuit with tapered directional coupler based (de)multiplexers and a multimode waveguide is designed and fabricated for this application. Experimental results show clear eye-diagrams and moderate power penalties for the wavelength conversion of both...

Multi-Modality Medical Image Fusion Based on Wavelet Analysis and Quality Evaluation

Institute of Scientific and Technical Information of China (English)

无

2001-01-01

Multi-modality medical image fusion has more and more important applications in medical image analysisand understanding. In this paper, we develop and apply a multi-resolution method based on wavelet pyramid to fusemedical images from different modalities such as PET-MRI and CT-MRI. In particular, we evaluate the different fusionresults when applying different selection rules and obtain optimum combination of fusion parameters.

Registration of vehicle based panoramic image and LiDAR point cloud

Science.gov (United States)

Chen, Changjun; Cao, Liang; Xie, Hong; Zhuo, Xiangyu

2013-10-01

Higher quality surface information would be got when data from optical images and LiDAR were integrated, owing to the fact that optical images and LiDAR point cloud have unique characteristics that make them preferable in many applications. While most previous works focus on registration of pinhole perspective cameras to 2D or 3D LiDAR data. In this paper, a method for the registration of vehicle based panoramic image and LiDAR point cloud is proposed. Using the translation among panoramic image, single CCD image, laser scanner and Position and Orientation System (POS) along with the GPS/IMU data, precise co-registration between the panoramic image and the LiDAR point cloud in the world system is achieved. Results are presented under a real world data set collected by a new developed Mobile Mapping System (MMS) integrated with a high resolution panoramic camera, two laser scanners and a POS.

SU-E-J-95: Towards Optimum Boundary Conditions for Biomechanical Model Based Deformable Registration Using Intensity Based Image Matching for Prostate Correlative Pathology.

Science.gov (United States)

Samavati, N; McGrath, D M; Lee, J; van der Kwast, T; Jewett, M; Mã Nard, C; Pluim, J P W; Brock, K K

2012-06-01

Deformable registration of histology to MRI is an essential tool to validate in vivo prostate cancer imaging. However, direct registration of histology to in vivo MR is prone to error due to geometric differences between the tissue sections and the in vivo imaging planes. To increase the accuracy of registration, an ex vivo high resolution MRI is acquired to compensate for the direct registration difficulties. A novel intensity-based deformable registration algorithm based on local variation in image intensities is proposed to register the histology to ex vivo MRI of prostatectomy specimens. Four sets of ex vivo MR and whole mount pathology images from four patients were used to investigate and validate the technique. In addition, 9 synthetically deformed ex vivo MR images were used. The standard deviation in local windows within the images was calculated to generate intermediate images based on both MR and histology. The intermediate images were registered using the Drop package (Munich, Germany). To further increase the accuracy, a final refinement of the registration was performed using Drop with a finer B-spline rid. The registration parameters were tuned to achieve a visually acceptable registration. Magnitude of Differences (MOD) and Angular Error (AE) were used to validate the synthetic data, and the Target Registration Error (TRE) of manually indicated landmarks was used for the clinical data. MOD of 0.6mm and AE of 8.3 degrees showed the efficacy of using intermediate images, compared to 0.8mm and 10.0 degrees achieved with Drop without the intermediate images. The average mean±std TRE among the four patients was 1.0±0.6 mm using the proposed method compared to 1.6±1.1 mm using Elastix (Utrecht, The Netherlands). An intensity-based deformable registration algorithm which uses intermediate images was evaluated on prostatectomy specimens and synthetically deformed clinical data, indicating improvement in overall accuracy and robustness. OICR, Terry Fox

Multimodal warnings to enhance risk communication and safety

NARCIS (Netherlands)

Haas, E.C.; Erp, J.B.F. van

2014-01-01

Multimodal warnings incorporate audio and/or skin-based (tactile) cues to supplement or replace visual cues in environments where the user’s visual perception is busy, impaired, or nonexistent. This paper describes characteristics of audio, tactile, and multimodal warning displays and their role in

Registration performance on EUV masks using high-resolution registration metrology

Science.gov (United States)

Steinert, Steffen; Solowan, Hans-Michael; Park, Jinback; Han, Hakseung; Beyer, Dirk; Scherübl, Thomas

2016-10-01

Next-generation lithography based on EUV continues to move forward to high-volume manufacturing. Given the technical challenges and the throughput concerns a hybrid approach with 193 nm immersion lithography is expected, at least in the initial state. Due to the increasing complexity at smaller nodes a multitude of different masks, both DUV (193 nm) and EUV (13.5 nm) reticles, will then be required in the lithography process-flow. The individual registration of each mask and the resulting overlay error are of crucial importance in order to ensure proper functionality of the chips. While registration and overlay metrology on DUV masks has been the standard for decades, this has yet to be demonstrated on EUV masks. Past generations of mask registration tools were not necessarily limited in their tool stability, but in their resolution capabilities. The scope of this work is an image placement investigation of high-end EUV masks together with a registration and resolution performance qualification. For this we employ a new generation registration metrology system embedded in a production environment for full-spec EUV masks. This paper presents excellent registration performance not only on standard overlay markers but also on more sophisticated e-beam calibration patterns.

Automatic registration using implicit shape representations: applications in intraoperative 3D rotational angiography to preoperative CTA registration

International Nuclear Information System (INIS)

Subramanian, Navneeth; Pichon, Eric; Solomon, Stephen B.

2009-01-01

A solution for automatic registration of 3D rotational angiography (XA) to CT/MR of the liver. Targeted for use in treatment planning of liver interventions. A shape-based approach to registration is proposed that does not require specification of landmarks nor is it prone to local minima like purely intensity-based registration methods. Through the use of vessel characteristics, accurate registration is possible even in the presence of deformations induced by catheters and respiratory motion. Registration was performed on eight pairs of multiphase CT angiography and 3D rotational digital angiography datasets. Quantitative validation of the registration accuracy using vessel landmarks was performed on these datasets. The validation study showed that the method has a registration error of 9.41±4.13 mm. In addition, the computation time is well below 60 s making it attractive for clinical application. A new method for fully automatic 3DXA to CT/MR image registration was developed and found to be efficient and accurate using clinically realistic datasets. (orig.)

Facilitating Multiple Intelligences Through Multimodal Learning Analytics

Directory of Open Access Journals (Sweden)

Ayesha PERVEEN

2018-01-01

Full Text Available This paper develops a theoretical framework for employing learning analytics in online education to trace multiple learning variations of online students by considering their potential of being multiple intelligences based on Howard Gardner’s 1983 theory of multiple intelligences. The study first emphasizes the need to facilitate students as multiple intelligences by online education systems and then suggests a framework of the advanced form of learning analytics i.e., multimodal learning analytics for tracing and facilitating multiple intelligences while they are engaged in online ubiquitous learning. As multimodal learning analytics is still an evolving area, it poses many challenges for technologists, educationists as well as organizational managers. Learning analytics make machines meet humans, therefore, the educationists with an expertise in learning theories can help technologists devise latest technological methods for multimodal learning analytics and organizational managers can implement them for the improvement of online education. Therefore, a careful instructional design based on a deep understanding of students’ learning abilities, is required to develop teaching plans and technological possibilities for monitoring students’ learning paths. This is how learning analytics can help design an adaptive instructional design based on a quick analysis of the data gathered. Based on that analysis, the academicians can critically reflect upon the quick or delayed implementation of the existing instructional design based on students’ cognitive abilities or even about the single or double loop learning design. The researcher concludes that the online education is multimodal in nature, has the capacity to endorse multiliteracies and, therefore, multiple intelligences can be tracked and facilitated through multimodal learning analytics in an online mode. However, online teachers’ training both in technological implementations and

2.5D Multi-View Gait Recognition Based on Point Cloud Registration

Science.gov (United States)

Tang, Jin; Luo, Jian; Tjahjadi, Tardi; Gao, Yan

2014-01-01

This paper presents a method for modeling a 2.5-dimensional (2.5D) human body and extracting the gait features for identifying the human subject. To achieve view-invariant gait recognition, a multi-view synthesizing method based on point cloud registration (MVSM) to generate multi-view training galleries is proposed. The concept of a density and curvature-based Color Gait Curvature Image is introduced to map 2.5D data onto a 2D space to enable data dimension reduction by discrete cosine transform and 2D principle component analysis. Gait recognition is achieved via a 2.5D view-invariant gait recognition method based on point cloud registration. Experimental results on the in-house database captured by a Microsoft Kinect camera show a significant performance gain when using MVSM. PMID:24686727

Registration of 3-dimensional facial photographs for clinical use.

Science.gov (United States)

Maal, Thomas J J; van Loon, Bram; Plooij, Joanneke M; Rangel, Frits; Ettema, Anke M; Borstlap, Wilfred A; Bergé, Stefaan J

2010-10-01

To objectively evaluate treatment outcomes in oral and maxillofacial surgery, pre- and post-treatment 3-dimensional (3D) photographs of the patient's face can be registered. For clinical use, it is of great importance that this registration process is accurate (photographs were captured at 3 different times: baseline (T(0)), after 1 minute (T(1)), and 3 weeks later (T(2)). Furthermore, a 3D photograph of the volunteer laughing (T(L)) was acquired to investigate the effect of facial expression. Two different registration methods were used to register the photographs acquired at all different times: surface-based registration and reference-based registration. Within the surface-based registration, 2 different software packages (Maxilim [Medicim NV, Mechelen, Belgium] and 3dMD Patient [3dMD LLC, Atlanta, GA]) were used to register the 3D photographs acquired at the various times. The surface-based registration process was repeated with the preprocessed photographs. Reference-based registration (Maxilim) was performed twice by 2 observers investigating the inter- and intraobserver error. The mean registration errors are small for the 3D photographs at rest (0.39 mm for T(0)-T(1) and 0.52 mm for T(0)-T(2)). The mean registration error increased to 1.2 mm for the registration between the 3D photographs acquired at T(0) and T(L). The mean registration error for the reference-based method was 1.0 mm for T(0)-T(1), 1.1 mm for T(0)-T(2), and 1.5 mm for T(0) and T(L). The mean registration errors for the preprocessed photographs were even smaller (0.30 mm for T(0)-T(1), 0.42 mm for T(0)-T(2), and 1.2 mm for T(0) and T(L)). Furthermore, a strong correlation between the results of both software packages used for surface-based registration was found. The intra- and interobserver error for the reference-based registration method was found to be 1.2 and 1.0 mm, respectively. Surface-based registration is an accurate method to compare 3D photographs of the same individual at

Analysis of Point Based Image Registration Errors With Applications in Single Molecule Microscopy.

Science.gov (United States)

Cohen, E A K; Ober, R J

2013-12-15

We present an asymptotic treatment of errors involved in point-based image registration where control point (CP) localization is subject to heteroscedastic noise; a suitable model for image registration in fluorescence microscopy. Assuming an affine transform, CPs are used to solve a multivariate regression problem. With measurement errors existing for both sets of CPs this is an errors-in-variable problem and linear least squares is inappropriate; the correct method being generalized least squares. To allow for point dependent errors the equivalence of a generalized maximum likelihood and heteroscedastic generalized least squares model is achieved allowing previously published asymptotic results to be extended to image registration. For a particularly useful model of heteroscedastic noise where covariance matrices are scalar multiples of a known matrix (including the case where covariance matrices are multiples of the identity) we provide closed form solutions to estimators and derive their distribution. We consider the target registration error (TRE) and define a new measure called the localization registration error (LRE) believed to be useful, especially in microscopy registration experiments. Assuming Gaussianity of the CP localization errors, it is shown that the asymptotic distribution for the TRE and LRE are themselves Gaussian and the parameterized distributions are derived. Results are successfully applied to registration in single molecule microscopy to derive the key dependence of the TRE and LRE variance on the number of CPs and their associated photon counts. Simulations show asymptotic results are robust for low CP numbers and non-Gaussianity. The method presented here is shown to outperform GLS on real imaging data.

MOBILTEL - Mobile Multimodal Telecommunications dialogue system based on VoIP telephony

Directory of Open Access Journals (Sweden)

Anton Čižmár

2009-10-01

Full Text Available In this paper the project MobilTel ispresented. The communication itself is becoming amultimodal interactive process. The MobilTel projectprovides research and development activities inmultimodal interfaces area. The result is a functionalarchitecture for mobile multimodal telecommunicationsystem running on handheld device. The MobilTelcommunicator is a multimodal Slovak speech andgraphical interface with integrated VoIP client. Theother possible modalities are pen – touch screeninteraction, keyboard, and display on which theinformation is more user friendly presented (icons,emoticons, etc., and provides hyperlink and scrollingmenu availability.We describe the method of interaction between mobileterminal (PDA and MobilTel multimodal PCcommunicator over a VoIP WLAN connection basedon SIP protocol. We also present the graphicalexamples of services that enable users to obtaininformation about weather or information about trainconnection between two train stations.

Landmark Optimization Using Local Curvature for Point-Based Nonlinear Rodent Brain Image Registration

Directory of Open Access Journals (Sweden)

Yutong Liu

2012-01-01

Full Text Available Purpose. To develop a technique to automate landmark selection for point-based interpolating transformations for nonlinear medical image registration. Materials and Methods. Interpolating transformations were calculated from homologous point landmarks on the source (image to be transformed and target (reference image. Point landmarks are placed at regular intervals on contours of anatomical features, and their positions are optimized along the contour surface by a function composed of curvature similarity and displacements of the homologous landmarks. The method was evaluated in two cases (=5 each. In one, MRI was registered to histological sections; in the second, geometric distortions in EPI MRI were corrected. Normalized mutual information and target registration error were calculated to compare the registration accuracy of the automatically and manually generated landmarks. Results. Statistical analyses demonstrated significant improvement (<0.05 in registration accuracy by landmark optimization in most data sets and trends towards improvement (<0.1 in others as compared to manual landmark selection.

Compact Spectrometer based on a silicon multimode waveguide

DEFF Research Database (Denmark)

Piels, Molly; Zibar, Darko

2017-01-01

A multimode waveguide spectrometer with 4 GHz resolution, 250 GHz usable range, and a 1.6 mm × 2.1 mm footprint is demonstrated. The operating range is greatly extended by including distinct mode-exciting elements on chip....

THE IMAGE REGISTRATION OF FOURIER-MELLIN BASED ON THE COMBINATION OF PROJECTION AND GRADIENT PREPROCESSING

Directory of Open Access Journals (Sweden)

D. Gao

2017-09-01

Full Text Available Image registration is one of the most important applications in the field of image processing. The method of Fourier Merlin transform, which has the advantages of high precision and good robustness to change in light and shade, partial blocking, noise influence and so on, is widely used. However, not only this method can’t obtain the unique mutual power pulse function for non-parallel image pairs, even part of image pairs also can’t get the mutual power function pulse. In this paper, an image registration method based on Fourier-Mellin transformation in the view of projection-gradient preprocessing is proposed. According to the projection conformational equation, the method calculates the matrix of image projection transformation to correct the tilt image; then, gradient preprocessing and Fourier-Mellin transformation are performed on the corrected image to obtain the registration parameters. Eventually, the experiment results show that the method makes the image registration of Fourier-Mellin transformation not only applicable to the registration of the parallel image pairs, but also to the registration of non-parallel image pairs. What’s more, the better registration effect can be obtained

Registration-based Reconstruction of Four-dimensional Cone Beam Computed Tomography

DEFF Research Database (Denmark)

Christoffersen, Christian; Hansen, David Christoffer; Poulsen, Per Rugaard

2013-01-01

We present a new method for reconstruction of four-dimensional (4D) cone beam computed tomography from an undersampled set of X-ray projections. The novelty of the proposed method lies in utilizing optical flow based registration to facilitate that each temporal phase is reconstructed from the full...

Scan-based volume animation driven by locally adaptive articulated registrations.

Science.gov (United States)

Rhee, Taehyun; Lewis, J P; Neumann, Ulrich; Nayak, Krishna S

2011-03-01

This paper describes a complete system to create anatomically accurate example-based volume deformation and animation of articulated body regions, starting from multiple in vivo volume scans of a specific individual. In order to solve the correspondence problem across volume scans, a template volume is registered to each sample. The wide range of pose variations is first approximated by volume blend deformation (VBD), providing proper initialization of the articulated subject in different poses. A novel registration method is presented to efficiently reduce the computation cost while avoiding strong local minima inherent in complex articulated body volume registration. The algorithm highly constrains the degrees of freedom and search space involved in the nonlinear optimization, using hierarchical volume structures and locally constrained deformation based on the biharmonic clamped spline. Our registration step establishes a correspondence across scans, allowing a data-driven deformation approach in the volume domain. The results provide an occlusion-free person-specific 3D human body model, asymptotically accurate inner tissue deformations, and realistic volume animation of articulated movements driven by standard joint control estimated from the actual skeleton. Our approach also addresses the practical issues arising in using scans from living subjects. The robustness of our algorithms is tested by their applications on the hand, probably the most complex articulated region in the body, and the knee, a frequent subject area for medical imaging due to injuries. © 2011 IEEE

Registration of T2-weighted and diffusion-weighted MR images of the prostate: comparison between manual and landmark-based methods

Science.gov (United States)

Peng, Yahui; Jiang, Yulei; Soylu, Fatma N.; Tomek, Mark; Sensakovic, William; Oto, Aytekin

2012-02-01

Quantitative analysis of multi-parametric magnetic resonance (MR) images of the prostate, including T2-weighted (T2w) and diffusion-weighted (DW) images, requires accurate image registration. We compared two registration methods between T2w and DW images. We collected pre-operative MR images of 124 prostate cancer patients (68 patients scanned with a GE scanner and 56 with Philips scanners). A landmark-based rigid registration was done based on six prostate landmarks in both T2w and DW images identified by a radiologist. Independently, a researcher manually registered the same images. A radiologist visually evaluated the registration results by using a 5-point ordinal scale of 1 (worst) to 5 (best). The Wilcoxon signed-rank test was used to determine whether the radiologist's ratings of the results of the two registration methods were significantly different. Results demonstrated that both methods were accurate: the average ratings were 4.2, 3.3, and 3.8 for GE, Philips, and all images, respectively, for the landmark-based method; and 4.6, 3.7, and 4.2, respectively, for the manual method. The manual registration results were more accurate than the landmark-based registration results (p < 0.0001 for GE, Philips, and all images). Therefore, the manual method produces more accurate registration between T2w and DW images than the landmark-based method.

AUTOMATIC GLOBAL REGISTRATION BETWEEN AIRBORNE LIDAR DATA AND REMOTE SENSING IMAGE BASED ON STRAIGHT LINE FEATURES

Directory of Open Access Journals (Sweden)

Z. Q. Liu

2018-04-01

Full Text Available An automatic global registration approach for point clouds and remote sensing image based on straight line features is proposed which is insensitive to rotational and scale transformation. First, the building ridge lines and contour lines in point clouds are automatically detected as registration primitives by integrating region growth and topology identification. Second, the collinear condition equation is selected as registration transformation function which is based on rotation matrix described by unit quaternion. The similarity measure is established according to the distance between the corresponding straight line features from point clouds and the image in the same reference coordinate system. Finally, an iterative Hough transform is adopted to simultaneously estimate the parameters and obtain correspondence between registration primitives. Experimental results prove the proposed method is valid and the spectral information is useful for the following classification processing.

Multimodality and Ambient Intelligence

NARCIS (Netherlands)

Nijholt, Antinus; Verhaegh, W.; Aarts, E.; Korst, J.

2004-01-01

In this chapter we discuss multimodal interface technology. We present eexamples of multimodal interfaces and show problems and opportunities. Fusion of modalities is discussed and some roadmap discussions on research in multimodality are summarized. This chapter also discusses future developments

Optimal Face-Iris Multimodal Fusion Scheme

Directory of Open Access Journals (Sweden)

Omid Sharifi

2016-06-01

Full Text Available Multimodal biometric systems are considered a way to minimize the limitations raised by single traits. This paper proposes new schemes based on score level, feature level and decision level fusion to efficiently fuse face and iris modalities. Log-Gabor transformation is applied as the feature extraction method on face and iris modalities. At each level of fusion, different schemes are proposed to improve the recognition performance and, finally, a combination of schemes at different fusion levels constructs an optimized and robust scheme. In this study, CASIA Iris Distance database is used to examine the robustness of all unimodal and multimodal schemes. In addition, Backtracking Search Algorithm (BSA, a novel population-based iterative evolutionary algorithm, is applied to improve the recognition accuracy of schemes by reducing the number of features and selecting the optimized weights for feature level and score level fusion, respectively. Experimental results on verification rates demonstrate a significant improvement of proposed fusion schemes over unimodal and multimodal fusion methods.

Multimodal segmentation of optic disc and cup from stereo fundus and SD-OCT images

Science.gov (United States)

Miri, Mohammad Saleh; Lee, Kyungmoo; Niemeijer, Meindert; Abràmoff, Michael D.; Kwon, Young H.; Garvin, Mona K.

2013-03-01

Glaucoma is one of the major causes of blindness worldwide. One important structural parameter for the diagnosis and management of glaucoma is the cup-to-disc ratio (CDR), which tends to become larger as glaucoma progresses. While approaches exist for segmenting the optic disc and cup within fundus photographs, and more recently, within spectral-domain optical coherence tomography (SD-OCT) volumes, no approaches have been reported for the simultaneous segmentation of these structures within both modalities combined. In this work, a multimodal pixel-classification approach for the segmentation of the optic disc and cup within fundus photographs and SD-OCT volumes is presented. In particular, after segmentation of other important structures (such as the retinal layers and retinal blood vessels) and fundus-to-SD-OCT image registration, features are extracted from both modalities and a k-nearest-neighbor classification approach is used to classify each pixel as cup, rim, or background. The approach is evaluated on 70 multimodal image pairs from 35 subjects in a leave-10%-out fashion (by subject). A significant improvement in classification accuracy is obtained using the multimodal approach over that obtained from the corresponding unimodal approach (97.8% versus 95.2%; p < 0:05; paired t-test).

One registration multi-atlas-based pseudo-CT generation for attenuation correction in PET/MRI

DEFF Research Database (Denmark)

Arabi, H.; Zaidi, H.

2016-01-01

Purpose: The outcome of a detailed assessment of various strategies for atlas-based whole-body bone segmentation from magnetic resonance imaging (MRI) was exploited to select the optimal parameters and setting, with the aim of proposing a novel one-registration multi-atlas (ORMA) pseudo-CT genera......Purpose: The outcome of a detailed assessment of various strategies for atlas-based whole-body bone segmentation from magnetic resonance imaging (MRI) was exploited to select the optimal parameters and setting, with the aim of proposing a novel one-registration multi-atlas (ORMA) pseudo...... regarding the accuracy of extracted bone using the different techniques demonstrated the superiority of the VWW atlas fusion algorithm achieving a Dice similarity measure of 0.82 ± 0.04 compared to arithmetic averaging atlas fusion (0.60 ± 0.02), which uses conventional direct registration. Application...

Improving Intensity-Based Lung CT Registration Accuracy Utilizing Vascular Information

Directory of Open Access Journals (Sweden)

Kunlin Cao

2012-01-01

Full Text Available Accurate pulmonary image registration is a challenging problem when the lungs have a deformation with large distance. In this work, we present a nonrigid volumetric registration algorithm to track lung motion between a pair of intrasubject CT images acquired at different inflation levels and introduce a new vesselness similarity cost that improves intensity-only registration. Volumetric CT datasets from six human subjects were used in this study. The performance of four intensity-only registration algorithms was compared with and without adding the vesselness similarity cost function. Matching accuracy was evaluated using landmarks, vessel tree, and fissure planes. The Jacobian determinant of the transformation was used to reveal the deformation pattern of local parenchymal tissue. The average matching error for intensity-only registration methods was on the order of 1 mm at landmarks and 1.5 mm on fissure planes. After adding the vesselness preserving cost function, the landmark and fissure positioning errors decreased approximately by 25% and 30%, respectively. The vesselness cost function effectively helped improve the registration accuracy in regions near thoracic cage and near the diaphragm for all the intensity-only registration algorithms tested and also helped produce more consistent and more reliable patterns of regional tissue deformation.

Interactive, multi-modality image registrations for combined MRI/MRSI-planned HDR prostate brachytherapy

Directory of Open Access Journals (Sweden)

Galen Reed

2011-03-01

Full Text Available Purpose: This study presents the steps and criteria involved in the series of image registrations used clinically during the planning and dose delivery of focal high dose-rate (HDR brachytherapy of the prostate. Material and methods: Three imaging modalities – Magnetic Resonance Imaging (MRI, Magnetic Resonance Spectroscopic Imaging (MRSI, and Computed Tomography (CT – were used at different steps during the process. MRSI is used for identification of dominant intraprosatic lesions (DIL. A series of rigid and nonrigid transformations were applied to the data to correct for endorectal-coil-induced deformations and for alignment with the planning CT. Mutual information was calculated as a morphing metric. An inverse planning optimization algorithm was applied to boost dose to the DIL while providing protection to the urethra, penile bulb, rectum, and bladder. Six prostate cancer patients were treated using this protocol. Results: The morphing algorithm successfully modeled the probe-induced prostatic distortion. Mutual information calculated between the morphed images and images acquired without the endorectal probe showed a significant (p = 0.0071 increase to that calculated between the unmorphed images and images acquired without the endorectal probe. Both mutual information and visual inspection serve as effective diagnostics of image morphing. The entire procedure adds less than thirty minutes to the treatment planning. Conclusion: This work demonstrates the utility of image transformations and registrations to HDR brachytherapy of prostate cancer.

Facial Emotion Recognition Using Context Based Multimodal Approach

Directory of Open Access Journals (Sweden)

Priya Metri

2011-12-01

Full Text Available Emotions play a crucial role in person to person interaction. In recent years, there has been a growing interest in improving all aspects of interaction between humans and computers. The ability to understand human emotions is desirable for the computer in several applications especially by observing facial expressions. This paper explores a ways of human-computer interaction that enable the computer to be more aware of the user’s emotional expressions we present a approach for the emotion recognition from a facial expression, hand and body posture. Our model uses multimodal emotion recognition system in which we use two different models for facial expression recognition and for hand and body posture recognition and then combining the result of both classifiers using a third classifier which give the resulting emotion . Multimodal system gives more accurate result than a signal or bimodal system

MO-F-BRA-04: Voxel-Based Statistical Analysis of Deformable Image Registration Error via a Finite Element Method.

Science.gov (United States)

Li, S; Lu, M; Kim, J; Glide-Hurst, C; Chetty, I; Zhong, H

2012-06-01

Purpose Clinical implementation of adaptive treatment planning is limited by the lack of quantitative tools to assess deformable image registration errors (R-ERR). The purpose of this study was to develop a method, using finite element modeling (FEM), to estimate registration errors based on mechanical changes resulting from them. Methods An experimental platform to quantify the correlation between registration errors and their mechanical consequences was developed as follows: diaphragm deformation was simulated on the CT images in patients with lung cancer using a finite element method (FEM). The simulated displacement vector fields (F-DVF) were used to warp each CT image to generate a FEM image. B-Spline based (Elastix) registrations were performed from reference to FEM images to generate a registration DVF (R-DVF). The F- DVF was subtracted from R-DVF. The magnitude of the difference vector was defined as the registration error, which is a consequence of mechanically unbalanced energy (UE), computed using 'in-house-developed' FEM software. A nonlinear regression model was used based on imaging voxel data and the analysis considered clustered voxel data within images. Results A regression model analysis showed that UE was significantly correlated with registration error, DVF and the product of registration error and DVF respectively with R̂2=0.73 (R=0.854). The association was verified independently using 40 tracked landmarks. A linear function between the means of UE values and R- DVF*R-ERR has been established. The mean registration error (N=8) was 0.9 mm. 85.4% of voxels fit this model within one standard deviation. Conclusions An encouraging relationship between UE and registration error has been found. These experimental results suggest the feasibility of UE as a valuable tool for evaluating registration errors, thus supporting 4D and adaptive radiotherapy. The research was supported by NIH/NCI R01CA140341. © 2012 American Association of Physicists in

Sensitivity study of voxel-based PET image comparison to image registration algorithms

Energy Technology Data Exchange (ETDEWEB)

Yip, Stephen, E-mail: syip@lroc.harvard.edu; Chen, Aileen B.; Berbeco, Ross [Department of Radiation Oncology, Brigham and Women’s Hospital, Dana-Farber Cancer Institute and Harvard Medical School, Boston, Massachusetts 02115 (United States); Aerts, Hugo J. W. L. [Department of Radiation Oncology, Brigham and Women’s Hospital, Dana-Farber Cancer Institute and Harvard Medical School, Boston, Massachusetts 02115 and Department of Radiology, Brigham and Women’s Hospital and Harvard Medical School, Boston, Massachusetts 02115 (United States)

2014-11-01

Purpose: Accurate deformable registration is essential for voxel-based comparison of sequential positron emission tomography (PET) images for proper adaptation of treatment plan and treatment response assessment. The comparison may be sensitive to the method of deformable registration as the optimal algorithm is unknown. This study investigated the impact of registration algorithm choice on therapy response evaluation. Methods: Sixteen patients with 20 lung tumors underwent a pre- and post-treatment computed tomography (CT) and 4D FDG-PET scans before and after chemoradiotherapy. All CT images were coregistered using a rigid and ten deformable registration algorithms. The resulting transformations were then applied to the respective PET images. Moreover, the tumor region defined by a physician on the registered PET images was classified into progressor, stable-disease, and responder subvolumes. Particularly, voxels with standardized uptake value (SUV) decreases >30% were classified as responder, while voxels with SUV increases >30% were progressor. All other voxels were considered stable-disease. The agreement of the subvolumes resulting from difference registration algorithms was assessed by Dice similarity index (DSI). Coefficient of variation (CV) was computed to assess variability of DSI between individual tumors. Root mean square difference (RMS{sub rigid}) of the rigidly registered CT images was used to measure the degree of tumor deformation. RMS{sub rigid} and DSI were correlated by Spearman correlation coefficient (R) to investigate the effect of tumor deformation on DSI. Results: Median DSI{sub rigid} was found to be 72%, 66%, and 80%, for progressor, stable-disease, and responder, respectively. Median DSI{sub deformable} was 63%–84%, 65%–81%, and 82%–89%. Variability of DSI was substantial and similar for both rigid and deformable algorithms with CV > 10% for all subvolumes. Tumor deformation had moderate to significant impact on DSI for progressor

Image Registration-Based Bolt Loosening Detection of Steel Joints

Science.gov (United States)

2018-01-01

Self-loosening of bolts caused by repetitive loads and vibrations is one of the common defects that can weaken the structural integrity of bolted steel joints in civil structures. Many existing approaches for detecting loosening bolts are based on physical sensors and, hence, require extensive sensor deployment, which limit their abilities to cost-effectively detect loosened bolts in a large number of steel joints. Recently, computer vision-based structural health monitoring (SHM) technologies have demonstrated great potential for damage detection due to the benefits of being low cost, easy to deploy, and contactless. In this study, we propose a vision-based non-contact bolt loosening detection method that uses a consumer-grade digital camera. Two images of the monitored steel joint are first collected during different inspection periods and then aligned through two image registration processes. If the bolt experiences rotation between inspections, it will introduce differential features in the registration errors, serving as a good indicator for bolt loosening detection. The performance and robustness of this approach have been validated through a series of experimental investigations using three laboratory setups including a gusset plate on a cross frame, a column flange, and a girder web. The bolt loosening detection results are presented for easy interpretation such that informed decisions can be made about the detected loosened bolts. PMID:29597264

Image Registration-Based Bolt Loosening Detection of Steel Joints.

Science.gov (United States)

Kong, Xiangxiong; Li, Jian

2018-03-28

Self-loosening of bolts caused by repetitive loads and vibrations is one of the common defects that can weaken the structural integrity of bolted steel joints in civil structures. Many existing approaches for detecting loosening bolts are based on physical sensors and, hence, require extensive sensor deployment, which limit their abilities to cost-effectively detect loosened bolts in a large number of steel joints. Recently, computer vision-based structural health monitoring (SHM) technologies have demonstrated great potential for damage detection due to the benefits of being low cost, easy to deploy, and contactless. In this study, we propose a vision-based non-contact bolt loosening detection method that uses a consumer-grade digital camera. Two images of the monitored steel joint are first collected during different inspection periods and then aligned through two image registration processes. If the bolt experiences rotation between inspections, it will introduce differential features in the registration errors, serving as a good indicator for bolt loosening detection. The performance and robustness of this approach have been validated through a series of experimental investigations using three laboratory setups including a gusset plate on a cross frame, a column flange, and a girder web. The bolt loosening detection results are presented for easy interpretation such that informed decisions can be made about the detected loosened bolts.

Registration of Space Objects

Science.gov (United States)

Schmidt-Tedd, Bernhard

2017-07-01

Space objects are subject to registration in order to allocate "jurisdiction and control" over those objects in the sovereign-free environment of outer space. This approach is similar to the registration of ships in view of the high sea and for aircrafts with respect to the international airspace. Registration is one of the basic principles of space law, starting with UN General Assembly Resolution 1721 B (XVI) of December 20, 1961, followed by Resolution 1962 (XVIII) of December 13, 1963, then formulated in Article VIII of the Outer Space Treaty of 1967 and as specified in the Registration Convention of 1975. Registration of space objects can be seen today as a principle of customary international law, relevant for each spacefaring state. Registration is divided into a national and an international level. The State Party establishes a national registry for its space objects, and those registrations have to be communicated via diplomatic channel to the UN Register of space objects. This UN Register is handled by the UN Office for Outer Space Affairs (UNOOSA) and is an open source of information for space objects worldwide. Registration is linked to the so-called launching state of the relevant space object. There might be more than one launching state for the specific launch event, but only one state actor can register a specific space object. The state of registry gains "jurisdiction and control" over the space object and therefore no double registration is permissible. Based on the established UN Space Law, registration practice was subject to some adaptions due to technical developments and legal challenges. After the privatization of the major international satellite organizations, a number of non-registrations had to be faced. The state actors reacted with the UN Registration Practice Resolution of 2007 as elaborated in the Legal Subcommittee of UNCOPUOS, the Committee for the Peaceful Use of Outer Space. In this context an UNOOSA Registration Information

Registration of Aerial Image with Airborne LiDAR Data Based on Plücker Line

Directory of Open Access Journals (Sweden)

SHENG Qinghong

2015-07-01

Full Text Available Registration of aerial image with airborne LiDAR data is a key to feature extraction. A registration model based on Plücker line is proposed. The relative position and attitude relationship between the conjugate lines in LiDAR and image is determined based on Plücker linear equation, which describes line transformation in space, then coplanarity condition equation is established. Finally, coordinate transformation between image point and corresponding LiDAR point is achieved by the spiral movement of Plücker lines in the image. The registration model of Plücker linear coplanarity condition equation is simple, and jointly describes the rotation and translation to avoid coupling error between them, so the accuracy is approved. This research provides technical support for high-quality earth spatial information acquisition.

Evaluation of GMI and PMI diffeomorphic-based demons algorithms for aligning PET and CT Images.

Science.gov (United States)

Yang, Juan; Wang, Hongjun; Zhang, You; Yin, Yong

2015-07-08

Fusion of anatomic information in computed tomography (CT) and functional information in 18F-FDG positron emission tomography (PET) is crucial for accurate differentiation of tumor from benign masses, designing radiotherapy treatment plan and staging of cancer. Although current PET and CT images can be acquired from combined 18F-FDG PET/CT scanner, the two acquisitions are scanned separately and take a long time, which may induce potential positional errors in global and local caused by respiratory motion or organ peristalsis. So registration (alignment) of whole-body PET and CT images is a prerequisite for their meaningful fusion. The purpose of this study was to assess the performance of two multimodal registration algorithms for aligning PET and CT images. The proposed gradient of mutual information (GMI)-based demons algorithm, which incorporated the GMI between two images as an external force to facilitate the alignment, was compared with the point-wise mutual information (PMI) diffeomorphic-based demons algorithm whose external force was modified by replacing the image intensity difference in diffeomorphic demons algorithm with the PMI to make it appropriate for multimodal image registration. Eight patients with esophageal cancer(s) were enrolled in this IRB-approved study. Whole-body PET and CT images were acquired from a combined 18F-FDG PET/CT scanner for each patient. The modified Hausdorff distance (d(MH)) was used to evaluate the registration accuracy of the two algorithms. Of all patients, the mean values and standard deviations (SDs) of d(MH) were 6.65 (± 1.90) voxels and 6.01 (± 1.90) after the GMI-based demons and the PMI diffeomorphic-based demons registration algorithms respectively. Preliminary results on oncological patients showed that the respiratory motion and organ peristalsis in PET/CT esophageal images could not be neglected, although a combined 18F-FDG PET/CT scanner was used for image acquisition. The PMI diffeomorphic-based demons

Cycling in multimodal transport behaviours: Exploring modality styles in the Danish population

DEFF Research Database (Denmark)

Olafsson, Anton Stahl; Nielsen, Thomas Alexander Sick; Carstensen, Trine Agervig

2016-01-01

and small towns. Thus, the way in which travel modes relate to the urban environment and variations in modality styles must serve as the starting point for policies aiming to fulfil the potential of multimodal transport behaviour and promote cycling. (C) 2016 Elsevier Ltd. All rights reserved....... explores how cycling forms part of multimodal transport behaviour based on survey data on transport modes and travel purposes and the weekly frequency of out-of-home activities and travel mode use in a representative sample of adult Danes (n = 1957). The following five distinct multimodal travel segments...... or 'modality styles' are identified: 'education transport'; 'public-based transport'; 'limited transport'; 'bicycle-based transport'; and 'car-based transport'. Travel behaviour is predominantly multimodal with few unimodal car-drivers being identified. Substantial cycling takes place in all modality styles...

Facial expression recognition in the wild based on multimodal texture features

Science.gov (United States)

Sun, Bo; Li, Liandong; Zhou, Guoyan; He, Jun

2016-11-01

Facial expression recognition in the wild is a very challenging task. We describe our work in static and continuous facial expression recognition in the wild. We evaluate the recognition results of gray deep features and color deep features, and explore the fusion of multimodal texture features. For the continuous facial expression recognition, we design two temporal-spatial dense scale-invariant feature transform (SIFT) features and combine multimodal features to recognize expression from image sequences. For the static facial expression recognition based on video frames, we extract dense SIFT and some deep convolutional neural network (CNN) features, including our proposed CNN architecture. We train linear support vector machine and partial least squares classifiers for those kinds of features on the static facial expression in the wild (SFEW) and acted facial expression in the wild (AFEW) dataset, and we propose a fusion network to combine all the extracted features at decision level. The final achievement we gained is 56.32% on the SFEW testing set and 50.67% on the AFEW validation set, which are much better than the baseline recognition rates of 35.96% and 36.08%.

Registration of Urban Aerial Image and LiDAR Based on Line Vectors

Directory of Open Access Journals (Sweden)

Qinghong Sheng

2017-09-01

Full Text Available In a traditional registration of a single aerial image with airborne light detection and ranging (LiDAR data using linear features that regard line direction as a control or linear features as constraints in the solution, lacking the constraint of linear position leads to the error propagation of the adjustment model. To solve this problem, this paper presents a line vector-based registration mode (LVR in which image rays and LiDAR lines are expressed by a line vector that integrates the line direction and the line position. A registration equation of line vector is set up by coplanar imaging rays and corresponding control lines. Three types of datasets consisting of synthetic, theInternational Society for Photogrammetry and Remote Sensing (ISPRS test project, and real aerial data are used. A group of progressive experiments is undertaken to evaluate the robustness of the LVR. Experimental results demonstrate that the integrated line direction and the line position contributes a great deal to the theoretical and real accuracies of the unknowns, as well as the stability of the adjustment model. This paper provides a new suggestion that, for a single image and LiDAR data, registration in urban areas can be accomplished by accommodating rich line features.

GPU-based streaming architectures for fast cone-beam CT image reconstruction and demons deformable registration

International Nuclear Information System (INIS)

Sharp, G C; Kandasamy, N; Singh, H; Folkert, M

2007-01-01

This paper shows how to significantly accelerate cone-beam CT reconstruction and 3D deformable image registration using the stream-processing model. We describe data-parallel designs for the Feldkamp, Davis and Kress (FDK) reconstruction algorithm, and the demons deformable registration algorithm, suitable for use on a commodity graphics processing unit. The streaming versions of these algorithms are implemented using the Brook programming environment and executed on an NVidia 8800 GPU. Performance results using CT data of a preserved swine lung indicate that the GPU-based implementations of the FDK and demons algorithms achieve a substantial speedup-up to 80 times for FDK and 70 times for demons when compared to an optimized reference implementation on a 2.8 GHz Intel processor. In addition, the accuracy of the GPU-based implementations was found to be excellent. Compared with CPU-based implementations, the RMS differences were less than 0.1 Hounsfield unit for reconstruction and less than 0.1 mm for deformable registration

Design and fabrication of multimode interference couplers based on digital micro-mirror system

Science.gov (United States)

Wu, Sumei; He, Xingdao; Shen, Chenbo

2008-03-01

Multimode interference (MMI) couplers, based on the self-imaging effect (SIE), are accepted popularly in integrated optics. According to the importance of MMI devices, in this paper, we present a novel method to design and fabricate MMI couplers. A technology of maskless lithography to make MMI couplers based on a smart digital micro-mirror device (DMD) system is proposed. A 1×4 MMI device is designed as an example, which shows the present method is efficient and cost-effective.

All-optical flip-flop operation based on asymmetric active-multimode interferometer bi-stable laser diodes

DEFF Research Database (Denmark)

Jiang, H.; Chaen, Y.; Hagio, T.

2011-01-01

We demonstrate fast and low energy all optical flip-flop devices based on asymmetric active-multimode interferometer using high-mesa waveguide structure. The implemented devices showed high speed alloptical flip-flop operation with 25ps long pulses. The rising and falling times of the output sign...

Multimodality

DEFF Research Database (Denmark)

Buhl, Mie

2010-01-01

In this paper, I address an ongoing discussion in Danish E-learning research about how to take advantage of the fact that digital media facilitate other communication forms than text, so-called ‘multimodal' communication, which should not be confused with the term ‘multimedia'. While multimedia...... on their teaching and learning situations. The choices they make involve e-learning resources like videos, social platforms and mobile devices, not just as digital artefacts we interact with, but the entire practice of using digital media. In a life-long learning perspective, multimodality is potentially very...

Reading Multimodal Texts for Learning – a Model for Cultivating Multimodal Literacy

Directory of Open Access Journals (Sweden)

Kristina Danielsson

2016-08-01

Full Text Available The re-conceptualisation of texts over the last 20 years, as well as the development of a multimodal understanding of communication and representation of knowledge, has profound consequences for the reading and understanding of multimodal texts, not least in educational contexts. However, if teachers and students are given tools to “unwrap” multimodal texts, they can develop a deeper understanding of texts, information structures, and the textual organisation of knowledge. This article presents a model for working with multimodal texts in education with the intention to highlight mutual multimodal text analysis in relation to the subject content. Examples are taken from a Singaporean science textbook as well as a Chilean science textbook, in order to demonstrate that the framework is versatile and applicable across different cultural contexts. The model takes into account the following aspects of texts: the general structure, how different semiotic resources operate, the ways in which different resources are combined (including coherence, the use of figurative language, and explicit/implicit values. Since learning operates on different dimensions – such as social and affective dimensions besides the cognitive ones – our inclusion of figurative language and values as components for textual analysis is a contribution to multimodal text analysis for learning.

Multimode-singlemode-multimode optical fiber sensor coated with novolac resin for detecting liquid phase alcohol

Science.gov (United States)

Marfu'ah, Amalia, Niza Rosyda; Hatta, Agus Muhamad; Pratama, Detak Yan

2018-04-01

Alcohol sensor based on multimode-singlemode-multimode (MSM) optical fiber with novolac resin as the external medium is proposed and demonstrated experimentally. Novolac resin swells when it is exposed by the alcohol. This effect causes a change in the polymer density leading to the refractive index's variation. The transmission light of the sensor depends on the refractive index of external medium. Based on the results, alcohol sensor based on MSM optical fiber structure using novolac resin has a higher sensitivity compared to the sensor without using novolac resin in the mixture of alcohol and distilled water. Alcohol sensor based on MSM optical fiber structure using novolac resin in the mixture of alcohol and distilled water with a singlemode fiber length of 5 mm has a sensitivity of 0.028972 dBm per % V/V, and in the mixture of alcohol and sugar solution of 10% w/w has a sensitivity of 0.005005 dBm per % V/V.

Video genre classification using multimodal features

Science.gov (United States)

Jin, Sung Ho; Bae, Tae Meon; Choo, Jin Ho; Ro, Yong Man

2003-12-01

We propose a video genre classification method using multimodal features. The proposed method is applied for the preprocessing of automatic video summarization or the retrieval and classification of broadcasting video contents. Through a statistical analysis of low-level and middle-level audio-visual features in video, the proposed method can achieve good performance in classifying several broadcasting genres such as cartoon, drama, music video, news, and sports. In this paper, we adopt MPEG-7 audio-visual descriptors as multimodal features of video contents and evaluate the performance of the classification by feeding the features into a decision tree-based classifier which is trained by CART. The experimental results show that the proposed method can recognize several broadcasting video genres with a high accuracy and the classification performance with multimodal features is superior to the one with unimodal features in the genre classification.

Object recognition through a multi-mode fiber

Science.gov (United States)

Takagi, Ryosuke; Horisaki, Ryoichi; Tanida, Jun

2017-04-01

We present a method of recognizing an object through a multi-mode fiber. A number of speckle patterns transmitted through a multi-mode fiber are provided to a classifier based on machine learning. We experimentally demonstrated binary classification of face and non-face targets based on the method. The measurement process of the experimental setup was random and nonlinear because a multi-mode fiber is a typical strongly scattering medium and any reference light was not used in our setup. Comparisons between three supervised learning methods, support vector machine, adaptive boosting, and neural network, are also provided. All of those learning methods achieved high accuracy rates at about 90% for the classification. The approach presented here can realize a compact and smart optical sensor. It is practically useful for medical applications, such as endoscopy. Also our study indicated a promising utilization of artificial intelligence, which has rapidly progressed, for reducing optical and computational costs in optical sensing systems.

Quantifying Quality Aspects of Multimodal Interactive Systems

CERN Document Server

Kühnel, Christine

2012-01-01

This book systematically addresses the quantification of quality aspects of multimodal interactive systems. The conceptual structure is based on a schematic view on human-computer interaction where the user interacts with the system and perceives it via input and output interfaces. Thus, aspects of multimodal interaction are analyzed first, followed by a discussion of the evaluation of output and input and concluding with a view on the evaluation of a complete system.

Pattern recognition of neurotransmitters using multimode sensing.

Science.gov (United States)

Stefan-van Staden, Raluca-Ioana; Moldoveanu, Iuliana; van Staden, Jacobus Frederick

2014-05-30

Pattern recognition is essential in chemical analysis of biological fluids. Reliable and sensitive methods for neurotransmitters analysis are needed. Therefore, we developed for pattern recognition of neurotransmitters: dopamine, epinephrine, norepinephrine a method based on multimode sensing. Multimode sensing was performed using microsensors based on diamond paste modified with 5,10,15,20-tetraphenyl-21H,23H-porphyrine, hemin and protoporphyrin IX in stochastic and differential pulse voltammetry modes. Optimized working conditions: phosphate buffer solution of pH 3.01 and KCl 0.1mol/L (as electrolyte support), were determined using cyclic voltammetry and used in all measurements. The lowest limits of quantification were: 10(-10)mol/L for dopamine and epinephrine, and 10(-11)mol/L for norepinephrine. The multimode microsensors were selective over ascorbic and uric acids and the method facilitated reliable assay of neurotransmitters in urine samples, and therefore, the pattern recognition showed high reliability (RSDneurotransmitters on biological fluids at a lower determination level than chromatographic methods. The sampling of the biological fluids referees only to the buffering (1:1, v/v) with a phosphate buffer pH 3.01, while for chromatographic methods the sampling is laborious. Accordingly with the statistic evaluation of the results at 99.00% confidence level, both modes can be used for pattern recognition and quantification of neurotransmitters with high reliability. The best multimode microsensor was the one based on diamond paste modified with protoporphyrin IX. Copyright © 2014 Elsevier B.V. All rights reserved.

Solid Mesh Registration for Radiotherapy Treatment Planning

DEFF Research Database (Denmark)

Noe, Karsten Østergaard; Sørensen, Thomas Sangild

2010-01-01

We present an algorithm for solid organ registration of pre-segmented data represented as tetrahedral meshes. Registration of the organ surface is driven by force terms based on a distance field representation of the source and reference shapes. Registration of internal morphology is achieved usi...

Workplace-based assessment for vocational registration of international medical graduates.

Science.gov (United States)

Lillis, Steven; Van Dyk, Valencia

2014-01-01

Medical regulatory authorities need efficient and effective methods of ensuring the competence of immigrating international medical graduates (IMGs). Not all IMGs who apply for specialist vocational registration will have directly comparable qualifications to those usually accepted. As general licensure examinations are inappropriate for these doctors, workplace-based assessment (WBA) techniques would appear to provide a solution. However, there is little published data on such outcomes. All cases of WBA (n = 81) used for vocational registration of IMGs in New Zealand between 2008 and 2013 were collated and analyzed. The successful completion rate of IMGs through the pathway was 87%. The majority (64%) undertook the year of supervised practice and the final assessment in a provincial center. For those unsuccessful in the pathway, inadequate clinical knowledge was the most common deficit found, followed by poor clinical reasoning. A WBA approach for assessing readiness of IMGs for vocational registration is feasible. The constructivist theoretical perspective of WBA has particular advantages in assessing the standard of practice for experienced practitioners working in narrow scopes than traditional methods of assessment. The majority of IMGs undertook both the clinical year and the assessment in provincial hospitals, thus providing a workforce for underserved areas. © 2014 The Alliance for Continuing Education in the Health Professions, the Society for Academic Continuing Medical Education, and the Council on Continuing Medical Education, Association for Hospital Medical Education.

Validation of a multimodal travel simulator with travel information provision

NARCIS (Netherlands)

Chorus, C.G.; Molin, E.J.E.; Arentze, T.A.; Hoogendoorn, S.P.; Timmermans, H.J.P.; Wee, van G.P.

2007-01-01

This paper presents a computer based travel simulator for collecting data concerning the use of next-generation ATIS and their effects on traveler decision making in a multimodal travel environment. The tool distinguishes itself by presenting a completely abstract multimodal transport network, where

Efficient nonrigid registration using ranked order statistics

DEFF Research Database (Denmark)

Tennakoon, Ruwan B.; Bab-Hadiashar, Alireza; de Bruijne, Marleen

2013-01-01

of research. In this paper we propose a fast and accurate non-rigid registration method for intra-modality volumetric images. Our approach exploits the information provided by an order statistics based segmentation method, to find the important regions for registration and use an appropriate sampling scheme......Non-rigid image registration techniques are widely used in medical imaging applications. Due to high computational complexities of these techniques, finding appropriate registration method to both reduce the computation burden and increase the registration accuracy has become an intense area...... to target those areas and reduce the registration computation time. A unique advantage of the proposed method is its ability to identify the point of diminishing returns and stop the registration process. Our experiments on registration of real lung CT images, with expert annotated landmarks, show...

Automated, non-linear registration between 3-dimensional brain map and medical head image

International Nuclear Information System (INIS)

Mizuta, Shinobu; Urayama, Shin-ichi; Zoroofi, R.A.; Uyama, Chikao

1998-01-01

In this paper, we propose an automated, non-linear registration method between 3-dimensional medical head image and brain map in order to efficiently extract the regions of interest. In our method, input 3-dimensional image is registered into a reference image extracted from a brain map. The problems to be solved are automated, non-linear image matching procedure, and cost function which represents the similarity between two images. Non-linear matching is carried out by dividing the input image into connected partial regions, transforming the partial regions preserving connectivity among the adjacent images, evaluating the image similarity between the transformed regions of the input image and the correspondent regions of the reference image, and iteratively searching the optimal transformation of the partial regions. In order to measure the voxelwise similarity of multi-modal images, a cost function is introduced, which is based on the mutual information. Some experiments using MR images presented the effectiveness of the proposed method. (author)

Critical Analysis of Multimodal Discourse

DEFF Research Database (Denmark)

van Leeuwen, Theo

2013-01-01

This is an encyclopaedia article which defines the fields of critical discourse analysis and multimodality studies, argues that within critical discourse analysis more attention should be paid to multimodality, and within multimodality to critical analysis, and ends reviewing a few examples of re...

Multimodal integration of anatomy and physiology classes: How instructors utilize multimodal teaching in their classrooms

Science.gov (United States)

McGraw, Gerald M., Jr.

Multimodality is the theory of communication as it applies to social and educational semiotics (making meaning through the use of multiple signs and symbols). The term multimodality describes a communication methodology that includes multiple textual, aural, and visual applications (modes) that are woven together to create what is referred to as an artifact. Multimodal teaching methodology attempts to create a deeper meaning to course content by activating the higher cognitive areas of the student's brain, creating a more sustained retention of the information (Murray, 2009). The introduction of multimodality educational methodologies as a means to more optimally engage students has been documented within educational literature. However, studies analyzing the distribution and penetration into basic sciences, more specifically anatomy and physiology, have not been forthcoming. This study used a quantitative survey design to determine the degree to which instructors integrated multimodality teaching practices into their course curricula. The instrument used for the study was designed by the researcher based on evidence found in the literature and sent to members of three associations/societies for anatomy and physiology instructors: the Human Anatomy and Physiology Society; the iTeach Anatomy & Physiology Collaborate; and the American Physiology Society. Respondents totaled 182 instructor members of two- and four-year, private and public higher learning colleges collected from the three organizations collectively with over 13,500 members in over 925 higher learning institutions nationwide. The study concluded that the expansion of multimodal methodologies into anatomy and physiology classrooms is at the beginning of the process and that there is ample opportunity for expansion. Instructors continue to use lecture as their primary means of interaction with students. Email is still the major form of out-of-class communication for full-time instructors. Instructors with

Normalized mutual information based PET-MR registration using K-Means clustering and shading correction

NARCIS (Netherlands)

Knops, Z.F.; Maintz, J.B.A.; Viergever, M.A.; Pluim, J.P.W.; Gee, J.C.; Maintz, J.B.A.; Vannier, M.W.

2003-01-01

A method for the efficient re-binning and shading based correction of intensity distributions of the images prior to normalized mutual information based registration is presented. Our intensity distribution re-binning method is based on the K-means clustering algorithm as opposed to the generally

MIDA - Optimizing control room performance through multi-modal design

International Nuclear Information System (INIS)

Ronan, A. M.

2006-01-01

Multi-modal interfaces can support the integration of humans with information processing systems and computational devices to maximize the unique qualities that comprise a complex system. In a dynamic environment, such as a nuclear power plant control room, multi-modal interfaces, if designed correctly, can provide complementary interaction between the human operator and the system which can improve overall performance while reducing human error. Developing such interfaces can be difficult for a designer without explicit knowledge of Human Factors Engineering principles. The Multi-modal Interface Design Advisor (MIDA) was developed as a support tool for system designers and developers. It provides design recommendations based upon a combination of Human Factors principles, a knowledge base of historical research, and current interface technologies. MIDA's primary objective is to optimize available multi-modal technologies within a human computer interface in order to balance operator workload with efficient operator performance. The purpose of this paper is to demonstrate MIDA and illustrate its value as a design evaluation tool within the nuclear power industry. (authors)

Multimodal Hyper-connectivity Networks for MCI Classification.

Science.gov (United States)

Li, Yang; Gao, Xinqiang; Jie, Biao; Yap, Pew-Thian; Kim, Min-Jeong; Wee, Chong-Yaw; Shen, Dinggang

2017-09-01

Hyper-connectivity network is a network where every edge is connected to more than two nodes, and can be naturally denoted using a hyper-graph. Hyper-connectivity brain network, either based on structural or functional interactions among the brain regions, has been used for brain disease diagnosis. However, the conventional hyper-connectivity network is constructed solely based on single modality data, ignoring potential complementary information conveyed by other modalities. The integration of complementary information from multiple modalities has been shown to provide a more comprehensive representation about the brain disruptions. In this paper, a novel multimodal hyper-network modelling method was proposed for improving the diagnostic accuracy of mild cognitive impairment (MCI). Specifically, we first constructed a multimodal hyper-connectivity network by simultaneously considering information from diffusion tensor imaging and resting-state functional magnetic resonance imaging data. We then extracted different types of network features from the hyper-connectivity network, and further exploited a manifold regularized multi-task feature selection method to jointly select the most discriminative features. Our proposed multimodal hyper-connectivity network demonstrated a better MCI classification performance than the conventional single modality based hyper-connectivity networks.

Multimodal Desktop Interaction: The Face –Object-Gesture–Voice Example

DEFF Research Database (Denmark)

Vidakis, Nikolas; Vlasopoulos, Anastasios; Kounalakis, Tsampikos

2013-01-01

This paper presents a natural user interface system based on multimodal human computer interaction, which operates as an intermediate module between the user and the operating system. The aim of this work is to demonstrate a multimodal system which gives users the ability to interact with desktop...

Multi-feature-based plaque characterization in ex vivo MRI trained by registration to 3D histology

International Nuclear Information System (INIS)

Van Engelen, Arna; Niessen, Wiro J; Klein, Stefan; De Bruijne, Marleen; Groen, Harald C; Wentzel, Jolanda J; Verhagen, Hence JM; Lugt, Aad van der

2012-01-01

We present a new method for automated characterization of atherosclerotic plaque composition in ex vivo MRI. It uses MRI intensities as well as four other types of features: smoothed, gradient magnitude and Laplacian images at several scales, and the distances to the lumen and outer vessel wall. The ground truth for fibrous, necrotic and calcified tissue was provided by histology and μCT in 12 carotid plaque specimens. Semi-automatic registration of a 3D stack of histological slices and μCT images to MRI allowed for 3D rotations and in-plane deformations of histology. By basing voxelwise classification on different combinations of features, we evaluated their relative importance. To establish whether training by 3D registration yields different results than training by 2D registration, we determined plaque composition using (1) a 2D slice-based registration approach for three manually selected MRI and histology slices per specimen, and (2) an approach that uses only the three corresponding MRI slices from the 3D-registered volumes. Voxelwise classification accuracy was best when all features were used (73.3 ± 6.3%) and was significantly better than when only original intensities and distance features were used (Friedman, p < 0.05). Although 2D registration or selection of three slices from the 3D set slightly decreased accuracy, these differences were non-significant. (paper)

A multimodal image sensor system for identifying water stress in grapevines

Science.gov (United States)

Zhao, Yong; Zhang, Qin; Li, Minzan; Shao, Yongni; Zhou, Jianfeng; Sun, Hong

2012-11-01

Water stress is one of the most common limitations of fruit growth. Water is the most limiting resource for crop growth. In grapevines, as well as in other fruit crops, fruit quality benefits from a certain level of water deficit which facilitates to balance vegetative and reproductive growth and the flow of carbohydrates to reproductive structures. A multi-modal sensor system was designed to measure the reflectance signature of grape plant surfaces and identify different water stress levels in this paper. The multi-modal sensor system was equipped with one 3CCD camera (three channels in R, G, and IR). The multi-modal sensor can capture and analyze grape canopy from its reflectance features, and identify the different water stress levels. This research aims at solving the aforementioned problems. The core technology of this multi-modal sensor system could further be used as a decision support system that combines multi-modal sensory data to improve plant stress detection and identify the causes of stress. The images were taken by multi-modal sensor which could output images in spectral bands of near-infrared, green and red channel. Based on the analysis of the acquired images, color features based on color space and reflectance features based on image process method were calculated. The results showed that these parameters had the potential as water stress indicators. More experiments and analysis are needed to validate the conclusion.

Acting rehearsal in collaborative multimodal mixed reality environments

OpenAIRE

Steptoe, William; Normand, Jean-Marie; Oyekoya, Oyewole; Pece, Fabrizio; Giannopoulos, Elias; Tecchia, Franco; Steed, Anthony; Weyrich, Tim; Kautz, Jan; Slater, Mel

2012-01-01

This paper presents the use of our multimodal mixed reality telecommunication system to support remote acting rehearsal. The rehearsals involved two actors, located in London and Barcelona, and a director in another location in London. This triadic audiovisual telecommunication was performed in a spatial and multimodal collaborative mixed reality environment based on the 'destination-visitor' paradigm, which we define and put into use. We detail our heterogeneous system architecture, which sp...

MINERVA: A multi-modality plug-in-based radiation therapy treatment planning system

International Nuclear Information System (INIS)

Wemple, C. A.; Wessol, D. E.; Nigg, D. W.; Cogliati, J. J.; Milvich, M.; Fredrickson, C. M.; Perkins, M.; Harkin, G. J.; Hartmann-Siantar, C. L.; Lehmann, J.; Flickinger, T.; Pletcher, D.; Yuan, A.; DeNardo, G. L.

2005-01-01

Researchers at the INEEL, MSU, LLNL and UCD have undertaken development of MINERVA, a patient-centric, multi-modal, radiation treatment planning system, which can be used for planning and analysing several radiotherapy modalities, either singly or combined, using common treatment planning tools. It employs an integrated, lightweight plug-in architecture to accommodate multi-modal treatment planning using standard interface components. The design also facilitates the future integration of improved planning technologies. The code is being developed with the Java programming language for inter-operability. The MINERVA design includes the image processing, model definition and data analysis modules with a central module to coordinate communication and data transfer. Dose calculation is performed by source and transport plug-in modules, which communicate either directly through the database or through MINERVA's openly published, extensible markup language (XML)-based application programmer's interface (API). All internal data are managed by a database management system and can be exported to other applications or new installations through the API data formats. A full computation path has been established for molecular-targeted radiotherapy treatment planning, with additional treatment modalities presently under development. (authors)

The Multimodal Possibilities of Online Instructions

DEFF Research Database (Denmark)

Kampf, Constance

2006-01-01

The WWW simplifies the process of delivering online instructions through multimodal channels because of the ease of use for voice, video, pictures, and text modes of communication built into it.  Given that instructions are being produced in multimodal format for the WWW, how do multi-modal analy......The WWW simplifies the process of delivering online instructions through multimodal channels because of the ease of use for voice, video, pictures, and text modes of communication built into it.  Given that instructions are being produced in multimodal format for the WWW, how do multi...

Multimodal Speaker Diarization.

Science.gov (United States)

Noulas, A; Englebienne, G; Krose, B J A

2012-01-01

We present a novel probabilistic framework that fuses information coming from the audio and video modality to perform speaker diarization. The proposed framework is a Dynamic Bayesian Network (DBN) that is an extension of a factorial Hidden Markov Model (fHMM) and models the people appearing in an audiovisual recording as multimodal entities that generate observations in the audio stream, the video stream, and the joint audiovisual space. The framework is very robust to different contexts, makes no assumptions about the location of the recording equipment, and does not require labeled training data as it acquires the model parameters using the Expectation Maximization (EM) algorithm. We apply the proposed model to two meeting videos and a news broadcast video, all of which come from publicly available data sets. The results acquired in speaker diarization are in favor of the proposed multimodal framework, which outperforms the single modality analysis results and improves over the state-of-the-art audio-based speaker diarization.

Advanced Contrast Agents for Multimodal Biomedical Imaging Based on Nanotechnology.

Science.gov (United States)

Calle, Daniel; Ballesteros, Paloma; Cerdán, Sebastián

2018-01-01

Clinical imaging modalities have reached a prominent role in medical diagnosis and patient management in the last decades. Different image methodologies as Positron Emission Tomography, Single Photon Emission Tomography, X-Rays, or Magnetic Resonance Imaging are in continuous evolution to satisfy the increasing demands of current medical diagnosis. Progress in these methodologies has been favored by the parallel development of increasingly more powerful contrast agents. These are molecules that enhance the intrinsic contrast of the images in the tissues where they accumulate, revealing noninvasively the presence of characteristic molecular targets or differential physiopathological microenvironments. The contrast agent field is currently moving to improve the performance of these molecules by incorporating the advantages that modern nanotechnology offers. These include, mainly, the possibilities to combine imaging and therapeutic capabilities over the same theranostic platform or improve the targeting efficiency in vivo by molecular engineering of the nanostructures. In this review, we provide an introduction to multimodal imaging methods in biomedicine, the sub-nanometric imaging agents previously used and the development of advanced multimodal and theranostic imaging agents based in nanotechnology. We conclude providing some illustrative examples from our own laboratories, including recent progress in theranostic formulations of magnetoliposomes containing ω-3 poly-unsaturated fatty acids to treat inflammatory diseases, or the use of stealth liposomes engineered with a pH-sensitive nanovalve to release their cargo specifically in the acidic extracellular pH microenvironment of tumors.

Control over structure-specific flexibility improves anatomical accuracy for point-based deformable registration in bladder cancer radiotherapy.

Science.gov (United States)

Wognum, S; Bondar, L; Zolnay, A G; Chai, X; Hulshof, M C C M; Hoogeman, M S; Bel, A

2013-02-01

Future developments in image guided adaptive radiotherapy (IGART) for bladder cancer require accurate deformable image registration techniques for the precise assessment of tumor and bladder motion and deformation that occur as a result of large bladder volume changes during the course of radiotherapy treatment. The aim was to employ an extended version of a point-based deformable registration algorithm that allows control over tissue-specific flexibility in combination with the authors' unique patient dataset, in order to overcome two major challenges of bladder cancer registration, i.e., the difficulty in accounting for the difference in flexibility between the bladder wall and tumor and the lack of visible anatomical landmarks for validation. The registration algorithm used in the current study is an extension of the symmetric-thin plate splines-robust point matching (S-TPS-RPM) algorithm, a symmetric feature-based registration method. The S-TPS-RPM algorithm has been previously extended to allow control over the degree of flexibility of different structures via a weight parameter. The extended weighted S-TPS-RPM algorithm was tested and validated on CT data (planning- and four to five repeat-CTs) of five urinary bladder cancer patients who received lipiodol injections before radiotherapy. The performance of the weighted S-TPS-RPM method, applied to bladder and tumor structures simultaneously, was compared with a previous version of the S-TPS-RPM algorithm applied to bladder wall structure alone and with a simultaneous nonweighted S-TPS-RPM registration of the bladder and tumor structures. Performance was assessed in terms of anatomical and geometric accuracy. The anatomical accuracy was calculated as the residual distance error (RDE) of the lipiodol markers and the geometric accuracy was determined by the surface distance, surface coverage, and inverse consistency errors. Optimal parameter values for the flexibility and bladder weight parameters were determined

Control over structure-specific flexibility improves anatomical accuracy for point-based deformable registration in bladder cancer radiotherapy

International Nuclear Information System (INIS)

Wognum, S.; Chai, X.; Hulshof, M. C. C. M.; Bel, A.; Bondar, L.; Zolnay, A. G.; Hoogeman, M. S.

2013-01-01

Purpose: Future developments in image guided adaptive radiotherapy (IGART) for bladder cancer require accurate deformable image registration techniques for the precise assessment of tumor and bladder motion and deformation that occur as a result of large bladder volume changes during the course of radiotherapy treatment. The aim was to employ an extended version of a point-based deformable registration algorithm that allows control over tissue-specific flexibility in combination with the authors’ unique patient dataset, in order to overcome two major challenges of bladder cancer registration, i.e., the difficulty in accounting for the difference in flexibility between the bladder wall and tumor and the lack of visible anatomical landmarks for validation. Methods: The registration algorithm used in the current study is an extension of the symmetric-thin plate splines-robust point matching (S-TPS-RPM) algorithm, a symmetric feature-based registration method. The S-TPS-RPM algorithm has been previously extended to allow control over the degree of flexibility of different structures via a weight parameter. The extended weighted S-TPS-RPM algorithm was tested and validated on CT data (planning- and four to five repeat-CTs) of five urinary bladder cancer patients who received lipiodol injections before radiotherapy. The performance of the weighted S-TPS-RPM method, applied to bladder and tumor structures simultaneously, was compared with a previous version of the S-TPS-RPM algorithm applied to bladder wall structure alone and with a simultaneous nonweighted S-TPS-RPM registration of the bladder and tumor structures. Performance was assessed in terms of anatomical and geometric accuracy. The anatomical accuracy was calculated as the residual distance error (RDE) of the lipiodol markers and the geometric accuracy was determined by the surface distance, surface coverage, and inverse consistency errors. Optimal parameter values for the flexibility and bladder weight

Control over structure-specific flexibility improves anatomical accuracy for point-based deformable registration in bladder cancer radiotherapy

Energy Technology Data Exchange (ETDEWEB)

Wognum, S.; Chai, X.; Hulshof, M. C. C. M.; Bel, A. [Department of Radiotherapy, Academic Medical Center, Meiberdreef 9, 1105 AZ Amsterdam (Netherlands); Bondar, L.; Zolnay, A. G.; Hoogeman, M. S. [Department of Radiation Oncology, Daniel den Hoed Cancer Center, Erasmus Medical Center, Groene Hilledijk 301, 3075 EA Rotterdam (Netherlands)

2013-02-15

Purpose: Future developments in image guided adaptive radiotherapy (IGART) for bladder cancer require accurate deformable image registration techniques for the precise assessment of tumor and bladder motion and deformation that occur as a result of large bladder volume changes during the course of radiotherapy treatment. The aim was to employ an extended version of a point-based deformable registration algorithm that allows control over tissue-specific flexibility in combination with the authors' unique patient dataset, in order to overcome two major challenges of bladder cancer registration, i.e., the difficulty in accounting for the difference in flexibility between the bladder wall and tumor and the lack of visible anatomical landmarks for validation. Methods: The registration algorithm used in the current study is an extension of the symmetric-thin plate splines-robust point matching (S-TPS-RPM) algorithm, a symmetric feature-based registration method. The S-TPS-RPM algorithm has been previously extended to allow control over the degree of flexibility of different structures via a weight parameter. The extended weighted S-TPS-RPM algorithm was tested and validated on CT data (planning- and four to five repeat-CTs) of five urinary bladder cancer patients who received lipiodol injections before radiotherapy. The performance of the weighted S-TPS-RPM method, applied to bladder and tumor structures simultaneously, was compared with a previous version of the S-TPS-RPM algorithm applied to bladder wall structure alone and with a simultaneous nonweighted S-TPS-RPM registration of the bladder and tumor structures. Performance was assessed in terms of anatomical and geometric accuracy. The anatomical accuracy was calculated as the residual distance error (RDE) of the lipiodol markers and the geometric accuracy was determined by the surface distance, surface coverage, and inverse consistency errors. Optimal parameter values for the flexibility and bladder weight

Ultracompact photonic crystal polarization beam splitter based on multimode interference.

Science.gov (United States)

Lu, Ming-Feng; Liao, Shan-Mei; Huang, Yang-Tung

2010-02-01

We propose a theoretical design for a compact photonic crystal (PC) polarization beam splitter (PBS) based on the multimode interference (MMI) effect. The size of a conventional MMI device designed by the self-imaging principle is not compact enough; therefore, we design a compact PC PBS based on the difference of the interference effect between TE and TM modes. Within the MMI coupler, the dependence of interference of modes on propagation distance is weak for a TE wave and strong for a TM wave; as a result, the length of the MMI section can be only seven lattice constants. Simulation results show that the insertion losses are 0.32 and 0.89 dB, and the extinction ratios are 14.4 and 17.5 dB for Port 1 (TE mode) and Port 2 (TM mode), respectively.

Spherical Demons: Fast Surface Registration

Science.gov (United States)

Yeo, B.T. Thomas; Sabuncu, Mert; Vercauteren, Tom; Ayache, Nicholas; Fischl, Bruce; Golland, Polina

2009-01-01

We present the fast Spherical Demons algorithm for registering two spherical images. By exploiting spherical vector spline interpolation theory, we show that a large class of regularizers for the modified demons objective function can be efficiently implemented on the sphere using convolution. Based on the one parameter subgroups of diffeomorphisms, the resulting registration is diffeomorphic and fast – registration of two cortical mesh models with more than 100k nodes takes less than 5 minutes, comparable to the fastest surface registration algorithms. Moreover, the accuracy of our method compares favorably to the popular FreeSurfer registration algorithm. We validate the technique in two different settings: (1) parcellation in a set of in-vivo cortical surfaces and (2) Brodmann area localization in ex-vivo cortical surfaces. PMID:18979813

SU-F-J-96: Comparison of Frame-Based and Mutual Information Registration Techniques for CT and MR Image Sets

Energy Technology Data Exchange (ETDEWEB)

Popple, R; Bredel, M; Brezovich, I; Dobelbower, M; Fisher, W; Fiveash, J; Guthrie, B; Riley, K; Wu, X [The University of Alabama at Birmingham, Birmingham, AL (United States)

2016-06-15

Purpose: To compare the accuracy of CT-MR registration using a mutual information method with registration using a frame-based localizer box. Methods: Ten patients having the Leksell head frame and scanned with a modality specific localizer box were imported into the treatment planning system. The fiducial rods of the localizer box were contoured on both the MR and CT scans. The skull was contoured on the CT images. The MR and CT images were registered by two methods. The frame-based method used the transformation that minimized the mean square distance of the centroids of the contours of the fiducial rods from a mathematical model of the localizer. The mutual information method used automated image registration tools in the TPS and was restricted to a volume-of-interest defined by the skull contours with a 5 mm margin. For each case, the two registrations were adjusted by two evaluation teams, each comprised of an experienced radiation oncologist and neurosurgeon, to optimize alignment in the region of the brainstem. The teams were blinded to the registration method. Results: The mean adjustment was 0.4 mm (range 0 to 2 mm) and 0.2 mm (range 0 to 1 mm) for the frame and mutual information methods, respectively. The median difference between the frame and mutual information registrations was 0.3 mm, but was not statistically significant using the Wilcoxon signed rank test (p=0.37). Conclusion: The difference between frame and mutual information registration techniques was neither statistically significant nor, for most applications, clinically important. These results suggest that mutual information is equivalent to frame-based image registration for radiosurgery. Work is ongoing to add additional evaluators and to assess the differences between evaluators.

Non-Rigid Contour-Based Registration of Cell Nuclei in 2-D Live Cell Microscopy Images Using a Dynamic Elasticity Model.

Science.gov (United States)

Sorokin, Dmitry V; Peterlik, Igor; Tektonidis, Marco; Rohr, Karl; Matula, Pavel

2018-01-01

The analysis of the pure motion of subnuclear structures without influence of the cell nucleus motion and deformation is essential in live cell imaging. In this paper, we propose a 2-D contour-based image registration approach for compensation of nucleus motion and deformation in fluorescence microscopy time-lapse sequences. The proposed approach extends our previous approach, which uses a static elasticity model to register cell images. Compared with that scheme, the new approach employs a dynamic elasticity model for the forward simulation of nucleus motion and deformation based on the motion of its contours. The contour matching process is embedded as a constraint into the system of equations describing the elastic behavior of the nucleus. This results in better performance in terms of the registration accuracy. Our approach was successfully applied to real live cell microscopy image sequences of different types of cells including image data that was specifically designed and acquired for evaluation of cell image registration methods. An experimental comparison with the existing contour-based registration methods and an intensity-based registration method has been performed. We also studied the dependence of the results on the choice of method parameters.

SU-E-J-224: Multimodality Segmentation of Head and Neck Tumors

International Nuclear Information System (INIS)

Aristophanous, M; Yang, J; Beadle, B

2014-01-01

Purpose: Develop an algorithm that is able to automatically segment tumor volume in Head and Neck cancer by integrating information from CT, PET and MR imaging simultaneously. Methods: Twenty three patients that were recruited under an adaptive radiotherapy protocol had MR, CT and PET/CT scans within 2 months prior to start of radiotherapy. The patients had unresectable disease and were treated either with chemoradiotherapy or radiation therapy alone. Using the Velocity software, the PET/CT and MR (T1 weighted+contrast) scans were registered to the planning CT using deformable and rigid registration respectively. The PET and MR images were then resampled according to the registration to match the planning CT. The resampled images, together with the planning CT, were fed into a multi-channel segmentation algorithm, which is based on Gaussian mixture models and solved with the expectation-maximization algorithm and Markov random fields. A rectangular region of interest (ROI) was manually placed to identify the tumor area and facilitate the segmentation process. The auto-segmented tumor contours were compared with the gross tumor volume (GTV) manually defined by the physician. The volume difference and Dice similarity coefficient (DSC) between the manual and autosegmented GTV contours were calculated as the quantitative evaluation metrics. Results: The multimodality segmentation algorithm was applied to all 23 patients. The volumes of the auto-segmented GTV ranged from 18.4cc to 32.8cc. The average (range) volume difference between the manual and auto-segmented GTV was −42% (−32.8%–63.8%). The average DSC value was 0.62, ranging from 0.39 to 0.78. Conclusion: An algorithm for the automated definition of tumor volume using multiple imaging modalities simultaneously was successfully developed and implemented for Head and Neck cancer. This development along with more accurate registration algorithms can aid physicians in the efforts to interpret the multitude of

SU-E-J-224: Multimodality Segmentation of Head and Neck Tumors

Energy Technology Data Exchange (ETDEWEB)

Aristophanous, M; Yang, J; Beadle, B [UT MD Anderson Cancer Center, Houston, TX (United States)

2014-06-01

Purpose: Develop an algorithm that is able to automatically segment tumor volume in Head and Neck cancer by integrating information from CT, PET and MR imaging simultaneously. Methods: Twenty three patients that were recruited under an adaptive radiotherapy protocol had MR, CT and PET/CT scans within 2 months prior to start of radiotherapy. The patients had unresectable disease and were treated either with chemoradiotherapy or radiation therapy alone. Using the Velocity software, the PET/CT and MR (T1 weighted+contrast) scans were registered to the planning CT using deformable and rigid registration respectively. The PET and MR images were then resampled according to the registration to match the planning CT. The resampled images, together with the planning CT, were fed into a multi-channel segmentation algorithm, which is based on Gaussian mixture models and solved with the expectation-maximization algorithm and Markov random fields. A rectangular region of interest (ROI) was manually placed to identify the tumor area and facilitate the segmentation process. The auto-segmented tumor contours were compared with the gross tumor volume (GTV) manually defined by the physician. The volume difference and Dice similarity coefficient (DSC) between the manual and autosegmented GTV contours were calculated as the quantitative evaluation metrics. Results: The multimodality segmentation algorithm was applied to all 23 patients. The volumes of the auto-segmented GTV ranged from 18.4cc to 32.8cc. The average (range) volume difference between the manual and auto-segmented GTV was −42% (−32.8%–63.8%). The average DSC value was 0.62, ranging from 0.39 to 0.78. Conclusion: An algorithm for the automated definition of tumor volume using multiple imaging modalities simultaneously was successfully developed and implemented for Head and Neck cancer. This development along with more accurate registration algorithms can aid physicians in the efforts to interpret the multitude of

Polyaffine parametrization of image registration based on geodesic flows

DEFF Research Database (Denmark)

Hansen, Michael Sass; Thorup, Signe Strann; Warfield, Simon K.

2012-01-01

Image registration based on geodesic flows has gained much popularity in recent years. We describe a novel parametrization of the velocity field in a stationary flow equation. We show that the method offers both precision, flexibility, and simplicity of evaluation. With our representation, which ...... of geodesic shooting for computational anatomy. We avoid to do warp field convolution by interpolation in a dense field, we can easily calculate warp derivatives in a reference frame of choice, and we can consequently avoid interpolation in the image space altogether....

Multimodality imaging techniques.

Science.gov (United States)

Martí-Bonmatí, Luis; Sopena, Ramón; Bartumeus, Paula; Sopena, Pablo

2010-01-01

In multimodality imaging, the need to combine morphofunctional information can be approached by either acquiring images at different times (asynchronous), and fused them through digital image manipulation techniques or simultaneously acquiring images (synchronous) and merging them automatically. The asynchronous post-processing solution presents various constraints, mainly conditioned by the different positioning of the patient in the two scans acquired at different times in separated machines. The best solution to achieve consistency in time and space is obtained by the synchronous image acquisition. There are many multimodal technologies in molecular imaging. In this review we will focus on those multimodality image techniques more commonly used in the field of diagnostic imaging (SPECT-CT, PET-CT) and new developments (as PET-MR). The technological innovations and development of new tracers and smart probes are the main key points that will condition multimodality image and diagnostic imaging professionals' future. Although SPECT-CT and PET-CT are standard in most clinical scenarios, MR imaging has some advantages, providing excellent soft-tissue contrast and multidimensional functional, structural and morphological information. The next frontier is to develop efficient detectors and electronics systems capable of detecting two modality signals at the same time. Not only PET-MR but also MR-US or optic-PET will be introduced in clinical scenarios. Even more, MR diffusion-weighted, pharmacokinetic imaging, spectroscopy or functional BOLD imaging will merge with PET tracers to further increase molecular imaging as a relevant medical discipline. Multimodality imaging techniques will play a leading role in relevant clinical applications. The development of new diagnostic imaging research areas, mainly in the field of oncology, cardiology and neuropsychiatry, will impact the way medicine is performed today. Both clinical and experimental multimodality studies, in

A Multimodal Discourse Analysis of Advertisements-Based on Visual Grammar

Directory of Open Access Journals (Sweden)

Fang Guo

2017-03-01

Full Text Available In addition to words, the symbols, colors, sculptures, photographs, music, etc. are also frequently employed by participants to express themselves in communication. Advertising is closely related to sounds, colors, picture animations and other symbols. This paper aims to present how semiotics acts effectively to realize the real business purpose to reflect the unique significance of the multimodal discourse analysis. Based on Visual Grammar, this paper analyzes the 2014 Brazil World Cup advertisements from the perspective of representational meaning, interactive meaning and compositional meaning, this research means to prove that different modes within an advertisement depend on each other and have an interdependent relationship. And these relationships have different roles in different contexts.

Strategy development management of Multimodal Transport Network

Directory of Open Access Journals (Sweden)

Nesterova Natalia S.

2016-01-01

Full Text Available The article gives a brief overview of works on the development of transport infrastructure for multimodal transportation and integration of Russian transport system into the international transport corridors. The technology for control of the strategy, that changes shape and capacity of Multi-modal Transport Network (MTN, is considered as part of the methodology for designing and development of MTN. This technology allows to carry out strategic and operational management of the strategy implementation based on the use of the balanced scorecard.

A Multimodal Search Engine for Medical Imaging Studies.

Science.gov (United States)

Pinho, Eduardo; Godinho, Tiago; Valente, Frederico; Costa, Carlos

2017-02-01

The use of digital medical imaging systems in healthcare institutions has increased significantly, and the large amounts of data in these systems have led to the conception of powerful support tools: recent studies on content-based image retrieval (CBIR) and multimodal information retrieval in the field hold great potential in decision support, as well as for addressing multiple challenges in healthcare systems, such as computer-aided diagnosis (CAD). However, the subject is still under heavy research, and very few solutions have become part of Picture Archiving and Communication Systems (PACS) in hospitals and clinics. This paper proposes an extensible platform for multimodal medical image retrieval, integrated in an open-source PACS software with profile-based CBIR capabilities. In this article, we detail a technical approach to the problem by describing its main architecture and each sub-component, as well as the available web interfaces and the multimodal query techniques applied. Finally, we assess our implementation of the engine with computational performance benchmarks.

Multimodality with Eye tracking and Haptics: A New Horizon for Serious Games?

Directory of Open Access Journals (Sweden)

Shujie Deng

2014-10-01

Full Text Available The goal of this review is to illustrate the emerging use of multimodal virtual reality that can benefit learning-based games. The review begins with an introduction to multimodal virtual reality in serious games and we provide a brief discussion of why cognitive processes involved in learning and training are enhanced under immersive virtual environments. We initially outline studies that have used eye tracking and haptic feedback independently in serious games, and then review some innovative applications that have already combined eye tracking and haptic devices in order to provide applicable multimodal frameworks for learning-based games. Finally, some general conclusions are identified and clarified in order to advance current understanding in multimodal serious game production as well as exploring possible areas for new applications.

Interactive initialization of 2D/3D rigid registration

Energy Technology Data Exchange (ETDEWEB)

Gong, Ren Hui; Güler, Özgür [The Sheikh Zayed Institute for Pediatric Surgical Innovation, Children' s National Medical Center, Washington, DC 20010 (United States); Kürklüoglu, Mustafa [Department of Cardiac Surgery, Children' s National Medical Center, Washington, DC 20010 (United States); Lovejoy, John [Department of Orthopaedic Surgery and Sports Medicine, Children' s National Medical Center, Washington, DC 20010 (United States); Yaniv, Ziv, E-mail: ZYaniv@childrensnational.org [The Sheikh Zayed Institute for Pediatric Surgical Innovation, Children' s National Medical Center, Washington, DC 20010 and Departments of Pediatrics and Radiology, George Washington University, Washington, DC 20037 (United States)

2013-12-15

Purpose: Registration is one of the key technical components in an image-guided navigation system. A large number of 2D/3D registration algorithms have been previously proposed, but have not been able to transition into clinical practice. The authors identify the primary reason for the lack of adoption with the prerequisite for a sufficiently accurate initial transformation, mean target registration error of about 10 mm or less. In this paper, the authors present two interactive initialization approaches that provide the desired accuracy for x-ray/MR and x-ray/CT registration in the operating room setting. Methods: The authors have developed two interactive registration methods based on visual alignment of a preoperative image, MR, or CT to intraoperative x-rays. In the first approach, the operator uses a gesture based interface to align a volume rendering of the preoperative image to multiple x-rays. The second approach uses a tracked tool available as part of a navigation system. Preoperatively, a virtual replica of the tool is positioned next to the anatomical structures visible in the volumetric data. Intraoperatively, the physical tool is positioned in a similar manner and subsequently used to align a volume rendering to the x-ray images using an augmented reality (AR) approach. Both methods were assessed using three publicly available reference data sets for 2D/3D registration evaluation. Results: In the authors' experiments, the authors show that for x-ray/MR registration, the gesture based method resulted in a mean target registration error (mTRE) of 9.3 ± 5.0 mm with an average interaction time of 146.3 ± 73.0 s, and the AR-based method had mTREs of 7.2 ± 3.2 mm with interaction times of 44 ± 32 s. For x-ray/CT registration, the gesture based method resulted in a mTRE of 7.4 ± 5.0 mm with an average interaction time of 132.1 ± 66.4 s, and the AR-based method had mTREs of 8.3 ± 5.0 mm with interaction times of 58 ± 52 s. Conclusions: Based on

Interactive initialization of 2D/3D rigid registration

International Nuclear Information System (INIS)

Gong, Ren Hui; Güler, Özgür; Kürklüoglu, Mustafa; Lovejoy, John; Yaniv, Ziv

2013-01-01

Purpose: Registration is one of the key technical components in an image-guided navigation system. A large number of 2D/3D registration algorithms have been previously proposed, but have not been able to transition into clinical practice. The authors identify the primary reason for the lack of adoption with the prerequisite for a sufficiently accurate initial transformation, mean target registration error of about 10 mm or less. In this paper, the authors present two interactive initialization approaches that provide the desired accuracy for x-ray/MR and x-ray/CT registration in the operating room setting. Methods: The authors have developed two interactive registration methods based on visual alignment of a preoperative image, MR, or CT to intraoperative x-rays. In the first approach, the operator uses a gesture based interface to align a volume rendering of the preoperative image to multiple x-rays. The second approach uses a tracked tool available as part of a navigation system. Preoperatively, a virtual replica of the tool is positioned next to the anatomical structures visible in the volumetric data. Intraoperatively, the physical tool is positioned in a similar manner and subsequently used to align a volume rendering to the x-ray images using an augmented reality (AR) approach. Both methods were assessed using three publicly available reference data sets for 2D/3D registration evaluation. Results: In the authors' experiments, the authors show that for x-ray/MR registration, the gesture based method resulted in a mean target registration error (mTRE) of 9.3 ± 5.0 mm with an average interaction time of 146.3 ± 73.0 s, and the AR-based method had mTREs of 7.2 ± 3.2 mm with interaction times of 44 ± 32 s. For x-ray/CT registration, the gesture based method resulted in a mTRE of 7.4 ± 5.0 mm with an average interaction time of 132.1 ± 66.4 s, and the AR-based method had mTREs of 8.3 ± 5.0 mm with interaction times of 58 ± 52 s. Conclusions: Based on the

Projection-slice theorem based 2D-3D registration

Science.gov (United States)

van der Bom, M. J.; Pluim, J. P. W.; Homan, R.; Timmer, J.; Bartels, L. W.

2007-03-01

In X-ray guided procedures, the surgeon or interventionalist is dependent on his or her knowledge of the patient's specific anatomy and the projection images acquired during the procedure by a rotational X-ray source. Unfortunately, these X-ray projections fail to give information on the patient's anatomy in the dimension along the projection axis. It would be very profitable to provide the surgeon or interventionalist with a 3D insight of the patient's anatomy that is directly linked to the X-ray images acquired during the procedure. In this paper we present a new robust 2D-3D registration method based on the Projection-Slice Theorem. This theorem gives us a relation between the pre-operative 3D data set and the interventional projection images. Registration is performed by minimizing a translation invariant similarity measure that is applied to the Fourier transforms of the images. The method was tested by performing multiple exhaustive searches on phantom data of the Circle of Willis and on a post-mortem human skull. Validation was performed visually by comparing the test projections to the ones that corresponded to the minimal value of the similarity measure. The Projection-Slice Theorem Based method was shown to be very effective and robust, and provides capture ranges up to 62 degrees. Experiments have shown that the method is capable of retrieving similar results when translations are applied to the projection images.

Real-time CT-video registration for continuous endoscopic guidance

Science.gov (United States)

Merritt, Scott A.; Rai, Lav; Higgins, William E.

2006-03-01

Previous research has shown that CT-image-based guidance could be useful for the bronchoscopic assessment of lung cancer. This research drew upon the registration of bronchoscopic video images to CT-based endoluminal renderings of the airway tree. The proposed methods either were restricted to discrete single-frame registration, which took several seconds to complete, or required non-real-time buffering and processing of video sequences. We have devised a fast 2D/3D image registration method that performs single-frame CT-Video registration in under 1/15th of a second. This allows the method to be used for real-time registration at full video frame rates without significantly altering the physician's behavior. The method achieves its speed through a gradient-based optimization method that allows most of the computation to be performed off-line. During live registration, the optimization iteratively steps toward the locally optimal viewpoint at which a CT-based endoluminal view is most similar to a current bronchoscopic video frame. After an initial registration to begin the process (generally done in the trachea for bronchoscopy), subsequent registrations are performed in real-time on each incoming video frame. As each new bronchoscopic video frame becomes available, the current optimization is initialized using the previous frame's optimization result, allowing continuous guidance to proceed without manual re-initialization. Tests were performed using both synthetic and pre-recorded bronchoscopic video. The results show that the method is robust to initialization errors, that registration accuracy is high, and that continuous registration can proceed on real-time video at >15 frames per sec. with minimal user-intervention.

Improving alignment in Tract-based spatial statistics: evaluation and optimization of image registration

NARCIS (Netherlands)

de Groot, Marius; Vernooij, Meike W.; Klein, Stefan; Ikram, M. Arfan; Vos, Frans M.; Smith, Stephen M.; Niessen, Wiro J.; Andersson, Jesper L. R.

2013-01-01

Anatomical alignment in neuroimaging studies is of such importance that considerable effort is put into improving the registration used to establish spatial correspondence. Tract-based spatial statistics (TBSS) is a popular method for comparing diffusion characteristics across subjects. TBSS

Improving alignment in Tract-based spatial statistics : Evaluation and optimization of image registration

NARCIS (Netherlands)

De Groot, M.; Vernooij, M.W.; Klein, S.; Arfan Ikram, M.; Vos, F.M.; Smith, S.M.; Niessen, W.J.; Andersson, J.L.R.

2013-01-01

Anatomical alignment in neuroimaging studies is of such importance that considerable effort is put into improving the registration used to establish spatial correspondence. Tract-based spatial statistics (TBSS) is a popular method for comparing diffusion characteristics across subjects. TBSS

A semi-automated 2D/3D marker-based registration algorithm modelling prostate shrinkage during radiotherapy for prostate cancer

International Nuclear Information System (INIS)

Budiharto, Tom; Slagmolen, Pieter; Hermans, Jeroen; Maes, Frederik; Verstraete, Jan; Heuvel, Frank Van den; Depuydt, Tom; Oyen, Raymond; Haustermans, Karin

2009-01-01

Background and purpose: Currently, most available patient alignment tools based on implanted markers use manual marker matching and rigid registration transformations to measure the needed translational shifts. To quantify the particular effect of prostate gland shrinkage, implanted gold markers were tracked during a course of radiotherapy including an isotropic scaling factor to model prostate shrinkage. Materials and methods: Eight patients with prostate cancer had gold markers implanted transrectally and seven were treated with (neo) adjuvant androgen deprivation therapy. After patient alignment to skin tattoos, orthogonal electronic portal images (EPIs) were taken. A semi-automated 2D/3D marker-based registration was performed to calculate the necessary couch shifts. The registration consists of a rigid transformation combined with an isotropic scaling to model prostate shrinkage. Results: The inclusion of an isotropic shrinkage model in the registration algorithm cancelled the corresponding increase in registration error. The mean scaling factor was 0.89 ± 0.09. For all but two patients, a decrease of the isotropic scaling factor during treatment was observed. However, there was almost no difference in the translation offset between the manual matching of the EPIs to the digitally reconstructed radiographs and the semi-automated 2D/3D registration. A decrease in the intermarker distance was found correlating with prostate shrinkage rather than with random marker migration. Conclusions: Inclusion of shrinkage in the registration process reduces registration errors during a course of radiotherapy. Nevertheless, this did not lead to a clinically significant change in the proposed table translations when compared to translations obtained with manual marker matching without a scaling correction

Evaluation of GMI and PMI diffeomorphic‐based demons algorithms for aligning PET and CT Images

Science.gov (United States)

Yang, Juan; Zhang, You; Yin, Yong

2015-01-01

Fusion of anatomic information in computed tomography (CT) and functional information in F18‐FDG positron emission tomography (PET) is crucial for accurate differentiation of tumor from benign masses, designing radiotherapy treatment plan and staging of cancer. Although current PET and CT images can be acquired from combined F18‐FDG PET/CT scanner, the two acquisitions are scanned separately and take a long time, which may induce potential positional errors in global and local caused by respiratory motion or organ peristalsis. So registration (alignment) of whole‐body PET and CT images is a prerequisite for their meaningful fusion. The purpose of this study was to assess the performance of two multimodal registration algorithms for aligning PET and CT images. The proposed gradient of mutual information (GMI)‐based demons algorithm, which incorporated the GMI between two images as an external force to facilitate the alignment, was compared with the point‐wise mutual information (PMI) diffeomorphic‐based demons algorithm whose external force was modified by replacing the image intensity difference in diffeomorphic demons algorithm with the PMI to make it appropriate for multimodal image registration. Eight patients with esophageal cancer(s) were enrolled in this IRB‐approved study. Whole‐body PET and CT images were acquired from a combined F18‐FDG PET/CT scanner for each patient. The modified Hausdorff distance (dMH) was used to evaluate the registration accuracy of the two algorithms. Of all patients, the mean values and standard deviations (SDs) of dMH were 6.65 (± 1.90) voxels and 6.01 (± 1.90) after the GMI‐based demons and the PMI diffeomorphic‐based demons registration algorithms respectively. Preliminary results on oncological patients showed that the respiratory motion and organ peristalsis in PET/CT esophageal images could not be neglected, although a combined F18‐FDG PET/CT scanner was used for image acquisition. The PMI

A robust cloud registration method based on redundant data reduction using backpropagation neural network and shift window

Science.gov (United States)

Xin, Meiting; Li, Bing; Yan, Xiao; Chen, Lei; Wei, Xiang

2018-02-01

A robust coarse-to-fine registration method based on the backpropagation (BP) neural network and shift window technology is proposed in this study. Specifically, there are three steps: coarse alignment between the model data and measured data, data simplification based on the BP neural network and point reservation in the contour region of point clouds, and fine registration with the reweighted iterative closest point algorithm. In the process of rough alignment, the initial rotation matrix and the translation vector between the two datasets are obtained. After performing subsequent simplification operations, the number of points can be reduced greatly. Therefore, the time and space complexity of the accurate registration can be significantly reduced. The experimental results show that the proposed method improves the computational efficiency without loss of accuracy.

Multimodal aspects of CSR communication related to gender empowerment and environmental protection

DEFF Research Database (Denmark)

Maier, Carmen Daniela

Purpose – This paper explores how the multimodal persuasive strategies of CSR communication related to Coca-Cola’s “5 by 20” succeed to highlight the company’s continuous commitment to gender empowerment and environmental protection. Launched in 2010, “5 by 20” is a program designed to empower 5...... as the usual textual focus is extended to a multimodal one. Shedding light on how the multimodal interplay contributes to communicate corporate commitment to gender empowerment and environmental protection, this model can also be employed in order to explore multimodally other areas of CSR communication....... a multimodal analysis model through which it is possible to map and explain the multimodal persuasive strategies employed by the company in their CSR communication. The paper is focused on the analysis of the video series that can be accessed at: http://www.coca-colacompany.com/stories/5by20. Based on a social...

A Registration Method Based on Contour Point Cloud for 3D Whole-Body PET and CT Images

Directory of Open Access Journals (Sweden)

Zhiying Song

2017-01-01

Full Text Available The PET and CT fusion image, combining the anatomical and functional information, has important clinical meaning. An effective registration of PET and CT images is the basis of image fusion. This paper presents a multithread registration method based on contour point cloud for 3D whole-body PET and CT images. Firstly, a geometric feature-based segmentation (GFS method and a dynamic threshold denoising (DTD method are creatively proposed to preprocess CT and PET images, respectively. Next, a new automated trunk slices extraction method is presented for extracting feature point clouds. Finally, the multithread Iterative Closet Point is adopted to drive an affine transform. We compare our method with a multiresolution registration method based on Mattes Mutual Information on 13 pairs (246~286 slices per pair of 3D whole-body PET and CT data. Experimental results demonstrate the registration effectiveness of our method with lower negative normalization correlation (NC = −0.933 on feature images and less Euclidean distance error (ED = 2.826 on landmark points, outperforming the source data (NC = −0.496, ED = 25.847 and the compared method (NC = −0.614, ED = 16.085. Moreover, our method is about ten times faster than the compared one.

Registration of FA and T1-weighted MRI data of healthy human brain based on template matching and normalized cross-correlation.

Science.gov (United States)

Malinsky, Milos; Peter, Roman; Hodneland, Erlend; Lundervold, Astri J; Lundervold, Arvid; Jan, Jiri

2013-08-01

In this work, we propose a new approach for three-dimensional registration of MR fractional anisotropy images with T1-weighted anatomy images of human brain. From the clinical point of view, this accurate coregistration allows precise detection of nerve fibers that is essential in neuroscience. A template matching algorithm combined with normalized cross-correlation was used for this registration task. To show the suitability of the proposed method, it was compared with the normalized mutual information-based B-spline registration provided by the Elastix software library, considered a reference method. We also propose a general framework for the evaluation of robustness and reliability of both registration methods. Both registration methods were tested by four evaluation criteria on a dataset consisting of 74 healthy subjects. The template matching algorithm has shown more reliable results than the reference method in registration of the MR fractional anisotropy and T1 anatomy image data. Significant differences were observed in the regions splenium of corpus callosum and genu of corpus callosum, considered very important areas of brain connectivity. We demonstrate that, in this registration task, the currently used mutual information-based parametric registration can be replaced by more accurate local template matching utilizing the normalized cross-correlation similarity measure.

Multimodal Dialogue Management - State of the art

NARCIS (Netherlands)

Bui Huu Trung, B.H.T.

This report is about the state of the art in dialogue management. We first introduce an overview of a multimodal dialogue system and its components. Second, four main approaches to dialogue management are described (finite-state and frame-based, information-state based and probabilistic, plan-based,

Multimodal fluorescence imaging spectroscopy

NARCIS (Netherlands)

Stopel, Martijn H W; Blum, Christian; Subramaniam, Vinod; Engelborghs, Yves; Visser, Anthonie J.W.G.

2014-01-01

Multimodal fluorescence imaging is a versatile method that has a wide application range from biological studies to materials science. Typical observables in multimodal fluorescence imaging are intensity, lifetime, excitation, and emission spectra which are recorded at chosen locations at the sample.

A Matlab user interface for the statistically assisted fluid registration algorithm and tensor-based morphometry

Science.gov (United States)

Yepes-Calderon, Fernando; Brun, Caroline; Sant, Nishita; Thompson, Paul; Lepore, Natasha

2015-01-01

Tensor-Based Morphometry (TBM) is an increasingly popular method for group analysis of brain MRI data. The main steps in the analysis consist of a nonlinear registration to align each individual scan to a common space, and a subsequent statistical analysis to determine morphometric differences, or difference in fiber structure between groups. Recently, we implemented the Statistically-Assisted Fluid Registration Algorithm or SAFIRA,1 which is designed for tracking morphometric differences among populations. To this end, SAFIRA allows the inclusion of statistical priors extracted from the populations being studied as regularizers in the registration. This flexibility and degree of sophistication limit the tool to expert use, even more so considering that SAFIRA was initially implemented in command line mode. Here, we introduce a new, intuitive, easy to use, Matlab-based graphical user interface for SAFIRA's multivariate TBM. The interface also generates different choices for the TBM statistics, including both the traditional univariate statistics on the Jacobian matrix, and comparison of the full deformation tensors.2 This software will be freely disseminated to the neuroimaging research community.

The plant virus microscope image registration method based on mismatches removing.

Science.gov (United States)

Wei, Lifang; Zhou, Shucheng; Dong, Heng; Mao, Qianzhuo; Lin, Jiaxiang; Chen, Riqing

2016-01-01

The electron microscopy is one of the major means to observe the virus. The view of virus microscope images is limited by making specimen and the size of the camera's view field. To solve this problem, the virus sample is produced into multi-slice for information fusion and image registration techniques are applied to obtain large field and whole sections. Image registration techniques have been developed in the past decades for increasing the camera's field of view. Nevertheless, these approaches typically work in batch mode and rely on motorized microscopes. Alternatively, the methods are conceived just to provide visually pleasant registration for high overlap ratio image sequence. This work presents a method for virus microscope image registration acquired with detailed visual information and subpixel accuracy, even when overlap ratio of image sequence is 10% or less. The method proposed focus on the correspondence set and interimage transformation. A mismatch removal strategy is proposed by the spatial consistency and the components of keypoint to enrich the correspondence set. And the translation model parameter as well as tonal inhomogeneities is corrected by the hierarchical estimation and model select. In the experiments performed, we tested different registration approaches and virus images, confirming that the translation model is not always stationary, despite the fact that the images of the sample come from the same sequence. The mismatch removal strategy makes building registration of virus microscope images at subpixel accuracy easier and optional parameters for building registration according to the hierarchical estimation and model select strategies make the proposed method high precision and reliable for low overlap ratio image sequence. Copyright © 2015 Elsevier Ltd. All rights reserved.

SU-C-207B-06: Comparison of Registration Methods for Modeling Pathologic Response of Esophageal Cancer to Chemoradiation Therapy

Energy Technology Data Exchange (ETDEWEB)

Riyahi, S; Choi, W; Bhooshan, N; Tan, S; Zhang, H; Lu, W [University of Maryland School of Medicine, Baltimore, MD (United States)

2016-06-15

Purpose: To compare linear and deformable registration methods for evaluation of tumor response to Chemoradiation therapy (CRT) in patients with esophageal cancer. Methods: Linear and multi-resolution BSpline deformable registration were performed on Pre-Post-CRT CT/PET images of 20 patients with esophageal cancer. For both registration methods, we registered CT using Mean Square Error (MSE) metric, however to register PET we used transformation obtained using Mutual Information (MI) from the same CT due to being multi-modality. Similarity of Warped-CT/PET was quantitatively evaluated using Normalized Mutual Information and plausibility of DF was assessed using inverse consistency Error. To evaluate tumor response four groups of tumor features were examined: (1) Conventional PET/CT e.g. SUV, diameter (2) Clinical parameters e.g. TNM stage, histology (3)spatial-temporal PET features that describe intensity, texture and geometry of tumor (4)all features combined. Dominant features were identified using 10-fold cross-validation and Support Vector Machine (SVM) was deployed for tumor response prediction while the accuracy was evaluated by ROC Area Under Curve (AUC). Results: Average and standard deviation of Normalized mutual information for deformable registration using MSE was 0.2±0.054 and for linear registration was 0.1±0.026, showing higher NMI for deformable registration. Likewise for MI metric, deformable registration had 0.13±0.035 comparing to linear counterpart with 0.12±0.037. Inverse consistency error for deformable registration for MSE metric was 4.65±2.49 and for linear was 1.32±2.3 showing smaller value for linear registration. The same conclusion was obtained for MI in terms of inverse consistency error. AUC for both linear and deformable registration was 1 showing no absolute difference in terms of response evaluation. Conclusion: Deformable registration showed better NMI comparing to linear registration, however inverse consistency of

Wavelet based Image Registration Technique for Matching Dental x-rays

OpenAIRE

P. Ramprasad; H. C. Nagaraj; M. K. Parasuram

2008-01-01

Image registration plays an important role in the diagnosis of dental pathologies such as dental caries, alveolar bone loss and periapical lesions etc. This paper presents a new wavelet based algorithm for registering noisy and poor contrast dental x-rays. Proposed algorithm has two stages. First stage is a preprocessing stage, removes the noise from the x-ray images. Gaussian filter has been used. Second stage is a geometric transformation stage. Proposed work uses two l...

Hyperspectral tomography based on multi-mode absorption spectroscopy (MUMAS)

Science.gov (United States)

Dai, Jinghang; O'Hagan, Seamus; Liu, Hecong; Cai, Weiwei; Ewart, Paul

2017-10-01

This paper demonstrates a hyperspectral tomographic technique that can recover the temperature and concentration field of gas flows based on multi-mode absorption spectroscopy (MUMAS). This method relies on the recently proposed concept of nonlinear tomography, which can take full advantage of the nonlinear dependency of MUMAS signals on temperature and enables 2D spatial resolution of MUMAS which is naturally a line-of-sight technique. The principles of MUMAS and nonlinear tomography, as well as the mathematical formulation of the inversion problem, are introduced. Proof-of-concept numerical demonstrations are presented using representative flame phantoms and assuming typical laser parameters. The results show that faithful reconstruction of temperature distribution is achievable when a signal-to-noise ratio of 20 is assumed. This method can potentially be extended to simultaneously reconstructing distributions of temperature and the concentration of multiple flame species.

Multimodality in organization studies

DEFF Research Database (Denmark)

Van Leeuwen, Theo

2017-01-01

This afterword reviews the chapters in this volume and reflects on the synergies between organization and management studies and multimodality studies that emerge from the volume. These include the combination of strong sociological theorizing and detailed multimodal analysis, a focus on material...

Study on multimodal transport route under low carbon background

Science.gov (United States)

Liu, Lele; Liu, Jie

2018-06-01

Low-carbon environmental protection is the focus of attention around the world, scientists are constantly researching on production of carbon emissions and living carbon emissions. However, there is little literature about multimodal transportation based on carbon emission at home and abroad. Firstly, this paper introduces the theory of multimodal transportation, the multimodal transport models that didn't consider carbon emissions and consider carbon emissions are analyzed. On this basis, a multi-objective programming 0-1 programming model with minimum total transportation cost and minimum total carbon emission is proposed. The idea of weight is applied to Ideal point method for solving problem, multi-objective programming is transformed into a single objective function. The optimal solution of carbon emission to transportation cost under different weights is determined by a single objective function with variable weights. Based on the model and algorithm, an example is given and the results are analyzed.

A digital 3D atlas of the marmoset brain based on multi-modal MRI.

Science.gov (United States)

Liu, Cirong; Ye, Frank Q; Yen, Cecil Chern-Chyi; Newman, John D; Glen, Daniel; Leopold, David A; Silva, Afonso C

2018-04-01

The common marmoset (Callithrix jacchus) is a New-World monkey of growing interest in neuroscience. Magnetic resonance imaging (MRI) is an essential tool to unveil the anatomical and functional organization of the marmoset brain. To facilitate identification of regions of interest, it is desirable to register MR images to an atlas of the brain. However, currently available atlases of the marmoset brain are mainly based on 2D histological data, which are difficult to apply to 3D imaging techniques. Here, we constructed a 3D digital atlas based on high-resolution ex-vivo MRI images, including magnetization transfer ratio (a T1-like contrast), T2w images, and multi-shell diffusion MRI. Based on the multi-modal MRI images, we manually delineated 54 cortical areas and 16 subcortical regions on one hemisphere of the brain (the core version). The 54 cortical areas were merged into 13 larger cortical regions according to their locations to yield a coarse version of the atlas, and also parcellated into 106 sub-regions using a connectivity-based parcellation method to produce a refined atlas. Finally, we compared the new atlas set with existing histology atlases and demonstrated its applications in connectome studies, and in resting state and stimulus-based fMRI. The atlas set has been integrated into the widely-distributed neuroimaging data analysis software AFNI and SUMA, providing a readily usable multi-modal template space with multi-level anatomical labels (including labels from the Paxinos atlas) that can facilitate various neuroimaging studies of marmosets. Published by Elsevier Inc.

Accuracy of radiotherapy dose calculations based on cone-beam CT: comparison of deformable registration and image correction based methods

Science.gov (United States)

Marchant, T. E.; Joshi, K. D.; Moore, C. J.

2018-03-01

Radiotherapy dose calculations based on cone-beam CT (CBCT) images can be inaccurate due to unreliable Hounsfield units (HU) in the CBCT. Deformable image registration of planning CT images to CBCT, and direct correction of CBCT image values are two methods proposed to allow heterogeneity corrected dose calculations based on CBCT. In this paper we compare the accuracy and robustness of these two approaches. CBCT images for 44 patients were used including pelvis, lung and head & neck sites. CBCT HU were corrected using a ‘shading correction’ algorithm and via deformable registration of planning CT to CBCT using either Elastix or Niftyreg. Radiotherapy dose distributions were re-calculated with heterogeneity correction based on the corrected CBCT and several relevant dose metrics for target and OAR volumes were calculated. Accuracy of CBCT based dose metrics was determined using an ‘override ratio’ method where the ratio of the dose metric to that calculated on a bulk-density assigned version of the same image is assumed to be constant for each patient, allowing comparison to the patient’s planning CT as a gold standard. Similar performance is achieved by shading corrected CBCT and both deformable registration algorithms, with mean and standard deviation of dose metric error less than 1% for all sites studied. For lung images, use of deformed CT leads to slightly larger standard deviation of dose metric error than shading corrected CBCT with more dose metric errors greater than 2% observed (7% versus 1%).

A Fuzzy-Based Fusion Method of Multimodal Sensor-Based Measurements for the Quantitative Evaluation of Eye Fatigue on 3D Displays

Directory of Open Access Journals (Sweden)

Jae Won Bang

2015-05-01

Full Text Available With the rapid increase of 3-dimensional (3D content, considerable research related to the 3D human factor has been undertaken for quantitatively evaluating visual discomfort, including eye fatigue and dizziness, caused by viewing 3D content. Various modalities such as electroencephalograms (EEGs, biomedical signals, and eye responses have been investigated. However, the majority of the previous research has analyzed each modality separately to measure user eye fatigue. This cannot guarantee the credibility of the resulting eye fatigue evaluations. Therefore, we propose a new method for quantitatively evaluating eye fatigue related to 3D content by combining multimodal measurements. This research is novel for the following four reasons: first, for the evaluation of eye fatigue with high credibility on 3D displays, a fuzzy-based fusion method (FBFM is proposed based on the multimodalities of EEG signals, eye blinking rate (BR, facial temperature (FT, and subjective evaluation (SE; second, to measure a more accurate variation of eye fatigue (before and after watching a 3D display, we obtain the quality scores of EEG signals, eye BR, FT and SE; third, for combining the values of the four modalities we obtain the optimal weights of the EEG signals BR, FT and SE using a fuzzy system based on quality scores; fourth, the quantitative level of the variation of eye fatigue is finally obtained using the weighted sum of the values measured by the four modalities. Experimental results confirm that the effectiveness of the proposed FBFM is greater than other conventional multimodal measurements. Moreover, the credibility of the variations of the eye fatigue using the FBFM before and after watching the 3D display is proven using a t-test and descriptive statistical analysis using effect size.

Sulcal set optimization for cortical surface registration.

Science.gov (United States)

Joshi, Anand A; Pantazis, Dimitrios; Li, Quanzheng; Damasio, Hanna; Shattuck, David W; Toga, Arthur W; Leahy, Richard M

2010-04-15

Flat mapping based cortical surface registration constrained by manually traced sulcal curves has been widely used for inter subject comparisons of neuroanatomical data. Even for an experienced neuroanatomist, manual sulcal tracing can be quite time consuming, with the cost increasing with the number of sulcal curves used for registration. We present a method for estimation of an optimal subset of size N(C) from N possible candidate sulcal curves that minimizes a mean squared error metric over all combinations of N(C) curves. The resulting procedure allows us to estimate a subset with a reduced number of curves to be traced as part of the registration procedure leading to optimal use of manual labeling effort for registration. To minimize the error metric we analyze the correlation structure of the errors in the sulcal curves by modeling them as a multivariate Gaussian distribution. For a given subset of sulci used as constraints in surface registration, the proposed model estimates registration error based on the correlation structure of the sulcal errors. The optimal subset of constraint curves consists of the N(C) sulci that jointly minimize the estimated error variance for the subset of unconstrained curves conditioned on the N(C) constraint curves. The optimal subsets of sulci are presented and the estimated and actual registration errors for these subsets are computed. Copyright 2009 Elsevier Inc. All rights reserved.

Multi-mode energy management strategy for fuel cell electric vehicles based on driving pattern identification using learning vector quantization neural network algorithm

Science.gov (United States)

Song, Ke; Li, Feiqiang; Hu, Xiao; He, Lin; Niu, Wenxu; Lu, Sihao; Zhang, Tong

2018-06-01

The development of fuel cell electric vehicles can to a certain extent alleviate worldwide energy and environmental issues. While a single energy management strategy cannot meet the complex road conditions of an actual vehicle, this article proposes a multi-mode energy management strategy for electric vehicles with a fuel cell range extender based on driving condition recognition technology, which contains a patterns recognizer and a multi-mode energy management controller. This paper introduces a learning vector quantization (LVQ) neural network to design the driving patterns recognizer according to a vehicle's driving information. This multi-mode strategy can automatically switch to the genetic algorithm optimized thermostat strategy under specific driving conditions in the light of the differences in condition recognition results. Simulation experiments were carried out based on the model's validity verification using a dynamometer test bench. Simulation results show that the proposed strategy can obtain better economic performance than the single-mode thermostat strategy under dynamic driving conditions.

Fast fluid registration of medical images

DEFF Research Database (Denmark)

Bro-Nielsen, Morten; Gramkow, Claus

1996-01-01

This paper offers a new fast algorithm for non-rigid viscous fluid registration of medical images that is at least an order of magnitude faster than the previous method by (Christensen et al., 1994). The core algorithm in the fluid registration method is based on a linear elastic deformation...

Experimental Study on Bioluminescence Tomography with Multimodality Fusion

Directory of Open Access Journals (Sweden)

Yujie Lv

2007-01-01

Full Text Available To verify the influence of a priori information on the nonuniqueness problem of bioluminescence tomography (BLT, the multimodality imaging fusion based BLT experiment is performed by multiview noncontact detection mode, which incorporates the anatomical information obtained by the microCT scanner and the background optical properties based on diffuse reflectance measurements. In the reconstruction procedure, the utilization of adaptive finite element methods (FEMs and a priori permissible source region refines the reconstructed results and improves numerical robustness and efficiency. The comparison between the absence and employment of a priori information shows that multimodality imaging fusion is essential to quantitative BLT reconstruction.

Automatic markerless registration of point clouds with semantic-keypoint-based 4-points congruent sets

Science.gov (United States)

Ge, Xuming

2017-08-01

The coarse registration of point clouds from urban building scenes has become a key topic in applications of terrestrial laser scanning technology. Sampling-based algorithms in the random sample consensus (RANSAC) model have emerged as mainstream solutions to address coarse registration problems. In this paper, we propose a novel combined solution to automatically align two markerless point clouds from building scenes. Firstly, the method segments non-ground points from ground points. Secondly, the proposed method detects feature points from each cross section and then obtains semantic keypoints by connecting feature points with specific rules. Finally, the detected semantic keypoints from two point clouds act as inputs to a modified 4PCS algorithm. Examples are presented and the results compared with those of K-4PCS to demonstrate the main contributions of the proposed method, which are the extension of the original 4PCS to handle heavy datasets and the use of semantic keypoints to improve K-4PCS in relation to registration accuracy and computational efficiency.

[Affine transformation-based automatic registration for peripheral digital subtraction angiography (DSA)].

Science.gov (United States)

Kong, Gang; Dai, Dao-Qing; Zou, Lu-Min

2008-07-01

In order to remove the artifacts of peripheral digital subtraction angiography (DSA), an affine transformation-based automatic image registration algorithm is introduced here. The whole process is described as follows: First, rectangle feature templates are constructed with their centers of the extracted Harris corners in the mask, and motion vectors of the central feature points are estimated using template matching technology with the similarity measure of maximum histogram energy. And then the optimal parameters of the affine transformation are calculated with the matrix singular value decomposition (SVD) method. Finally, bilinear intensity interpolation is taken to the mask according to the specific affine transformation. More than 30 peripheral DSA registrations are performed with the presented algorithm, and as the result, moving artifacts of the images are removed with sub-pixel precision, and the time consumption is less enough to satisfy the clinical requirements. Experimental results show the efficiency and robustness of the algorithm.

Multimode Adaptable Microwave Radar Sensor Based on Leaky-Wave Antennas

Czech Academy of Sciences Publication Activity Database

Hudec, P.; Pánek, Petr; Jeník, V.

2017-01-01

Roč. 65, č. 9 (2017), s. 3464-3473 ISSN 0018-9480 Institutional support: RVO:67985882 Keywords : adaptable sensor * low-range radar * multimode sensor Subject RIV: JA - Electronics ; Optoelectronics, Electrical Engineering OBOR OECD: Electrical and electronic engineering Impact factor: 2.897, year: 2016

Image-based RSA: Roentgen stereophotogrammetric analysis based on 2D-3D image registration.

Science.gov (United States)

de Bruin, P W; Kaptein, B L; Stoel, B C; Reiber, J H C; Rozing, P M; Valstar, E R

2008-01-01

Image-based Roentgen stereophotogrammetric analysis (IBRSA) integrates 2D-3D image registration and conventional RSA. Instead of radiopaque RSA bone markers, IBRSA uses 3D CT data, from which digitally reconstructed radiographs (DRRs) are generated. Using 2D-3D image registration, the 3D pose of the CT is iteratively adjusted such that the generated DRRs resemble the 2D RSA images as closely as possible, according to an image matching metric. Effectively, by registering all 2D follow-up moments to the same 3D CT, the CT volume functions as common ground. In two experiments, using RSA and using a micromanipulator as gold standard, IBRSA has been validated on cadaveric and sawbone scapula radiographs, and good matching results have been achieved. The accuracy was: |mu |RSA but higher than in vivo standard RSA. Because IBRSA does not require radiopaque markers, it adds functionality to the RSA method by opening new directions and possibilities for research, such as dynamic analyses using fluoroscopy on subjects without markers and computer navigation applications.

Automated diagnosis of prostate cancer in multi-parametric MRI based on multimodal convolutional neural networks

Science.gov (United States)

Le, Minh Hung; Chen, Jingyu; Wang, Liang; Wang, Zhiwei; Liu, Wenyu; (Tim Cheng, Kwang-Ting; Yang, Xin

2017-08-01

Automated methods for prostate cancer (PCa) diagnosis in multi-parametric magnetic resonance imaging (MP-MRIs) are critical for alleviating requirements for interpretation of radiographs while helping to improve diagnostic accuracy (Artan et al 2010 IEEE Trans. Image Process. 19 2444-55, Litjens et al 2014 IEEE Trans. Med. Imaging 33 1083-92, Liu et al 2013 SPIE Medical Imaging (International Society for Optics and Photonics) p 86701G, Moradi et al 2012 J. Magn. Reson. Imaging 35 1403-13, Niaf et al 2014 IEEE Trans. Image Process. 23 979-91, Niaf et al 2012 Phys. Med. Biol. 57 3833, Peng et al 2013a SPIE Medical Imaging (International Society for Optics and Photonics) p 86701H, Peng et al 2013b Radiology 267 787-96, Wang et al 2014 BioMed. Res. Int. 2014). This paper presents an automated method based on multimodal convolutional neural networks (CNNs) for two PCa diagnostic tasks: (1) distinguishing between cancerous and noncancerous tissues and (2) distinguishing between clinically significant (CS) and indolent PCa. Specifically, our multimodal CNNs effectively fuse apparent diffusion coefficients (ADCs) and T2-weighted MP-MRI images (T2WIs). To effectively fuse ADCs and T2WIs we design a new similarity loss function to enforce consistent features being extracted from both ADCs and T2WIs. The similarity loss is combined with the conventional classification loss functions and integrated into the back-propagation procedure of CNN training. The similarity loss enables better fusion results than existing methods as the feature learning processes of both modalities are mutually guided, jointly facilitating CNN to ‘see’ the true visual patterns of PCa. The classification results of multimodal CNNs are further combined with the results based on handcrafted features using a support vector machine classifier. To achieve a satisfactory accuracy for clinical use, we comprehensively investigate three critical factors which could greatly affect the performance of our

Registration-Based Range-Dependence Compensation for Bistatic STAP Radars

Directory of Open Access Journals (Sweden)

Lapierre Fabian D

2005-01-01

Full Text Available We address the problem of detecting slow-moving targets using space-time adaptive processing (STAP radar. Determining the optimum weights at each range requires data snapshots at neighboring ranges. However, in virtually all configurations, snapshot statistics are range dependent, meaning that snapshots are nonstationary with respect to range. This results in poor performance. In this paper, we propose a new compensation method based on registration of clutter ridges and designed to work on a single realization of the stochastic snapshot at each range. The method has been successfully tested on simulated, stochastic snapshots. An evaluation of performance is presented.

4D-CT Lung registration using anatomy-based multi-level multi-resolution optical flow analysis and thin-plate splines.

Science.gov (United States)

Min, Yugang; Neylon, John; Shah, Amish; Meeks, Sanford; Lee, Percy; Kupelian, Patrick; Santhanam, Anand P

2014-09-01

The accuracy of 4D-CT registration is limited by inconsistent Hounsfield unit (HU) values in the 4D-CT data from one respiratory phase to another and lower image contrast for lung substructures. This paper presents an optical flow and thin-plate spline (TPS)-based 4D-CT registration method to account for these limitations. The use of unified HU values on multiple anatomy levels (e.g., the lung contour, blood vessels, and parenchyma) accounts for registration errors by inconsistent landmark HU value. While 3D multi-resolution optical flow analysis registers each anatomical level, TPS is employed for propagating the results from one anatomical level to another ultimately leading to the 4D-CT registration. 4D-CT registration was validated using target registration error (TRE), inverse consistency error (ICE) metrics, and a statistical image comparison using Gamma criteria of 1 % intensity difference in 2 mm(3) window range. Validation results showed that the proposed method was able to register CT lung datasets with TRE and ICE values <3 mm. In addition, the average number of voxel that failed the Gamma criteria was <3 %, which supports the clinical applicability of the propose registration mechanism. The proposed 4D-CT registration computes the volumetric lung deformations within clinically viable accuracy.

Face-iris multimodal biometric scheme based on feature level fusion

Science.gov (United States)

Huo, Guang; Liu, Yuanning; Zhu, Xiaodong; Dong, Hongxing; He, Fei

2015-11-01

Unlike score level fusion, feature level fusion demands all the features extracted from unimodal traits with high distinguishability, as well as homogeneity and compatibility, which is difficult to achieve. Therefore, most multimodal biometric research focuses on score level fusion, whereas few investigate feature level fusion. We propose a face-iris recognition method based on feature level fusion. We build a special two-dimensional-Gabor filter bank to extract local texture features from face and iris images, and then transform them by histogram statistics into an energy-orientation variance histogram feature with lower dimensions and higher distinguishability. Finally, through a fusion-recognition strategy based on principal components analysis and support vector machine (FRSPS), feature level fusion and one-to-n identification are accomplished. The experimental results demonstrate that this method can not only effectively extract face and iris features but also provide higher recognition accuracy. Compared with some state-of-the-art fusion methods, the proposed method has a significant performance advantage.

Multiscale climate emulator of multimodal wave spectra: MUSCLE-spectra

Science.gov (United States)

Rueda, Ana; Hegermiller, Christie A.; Antolinez, Jose A. A.; Camus, Paula; Vitousek, Sean; Ruggiero, Peter; Barnard, Patrick L.; Erikson, Li H.; Tomás, Antonio; Mendez, Fernando J.

2017-02-01

Characterization of multimodal directional wave spectra is important for many offshore and coastal applications, such as marine forecasting, coastal hazard assessment, and design of offshore wave energy farms and coastal structures. However, the multivariate and multiscale nature of wave climate variability makes this complex problem tractable using computationally expensive numerical models. So far, the skill of statistical-downscaling model-based parametric (unimodal) wave conditions is limited in large ocean basins such as the Pacific. The recent availability of long-term directional spectral data from buoys and wave hindcast models allows for development of stochastic models that include multimodal sea-state parameters. This work introduces a statistical downscaling framework based on weather types to predict multimodal wave spectra (e.g., significant wave height, mean wave period, and mean wave direction from different storm systems, including sea and swells) from large-scale atmospheric pressure fields. For each weather type, variables of interest are modeled using the categorical distribution for the sea-state type, the Generalized Extreme Value (GEV) distribution for wave height and wave period, a multivariate Gaussian copula for the interdependence between variables, and a Markov chain model for the chronology of daily weather types. We apply the model to the southern California coast, where local seas and swells from both the Northern and Southern Hemispheres contribute to the multimodal wave spectrum. This work allows attribution of particular extreme multimodal wave events to specific atmospheric conditions, expanding knowledge of time-dependent, climate-driven offshore and coastal sea-state conditions that have a significant influence on local nearshore processes, coastal morphology, and flood hazards.

Biomechanical modeling constrained surface-based image registration for prostate MR guided TRUS biopsy

NARCIS (Netherlands)

Ven, W.J.M. van de; Hu, Y.; Barentsz, J.O.; Karssemeijer, N.; Barratt, D.; Huisman, H.J.

2015-01-01

Adding magnetic resonance (MR)-derived information to standard transrectal ultrasound (TRUS) images for guiding prostate biopsy is of substantial clinical interest. A tumor visible on MR images can be projected on ultrasound (US) by using MR-US registration. A common approach is to use surface-based

Fast and accurate registration of cranial CT images with A-mode ultrasound.

Science.gov (United States)

Fieten, Lorenz; Schmieder, Kirsten; Engelhardt, Martin; Pasalic, Lamija; Radermacher, Klaus; Heger, Stefan

2009-05-01

Within the CRANIO project, a navigation module based on preoperative computed tomography (CT) data was developed for Computer and Robot Assisted Neurosurgery. The approach followed for non-invasive user-interactive registration of cranial CT images with the physical operating space consists of surface-based registration following pre-registration based on anatomical landmarks. Surface-based registration relies on bone surface points digitized transcutaneously by means of an optically tracked A-mode ultrasound (US) probe. As probe alignment and thus bone surface point digitization may be time-consuming, we investigated how to obtain high registration accuracy despite inaccurate pre-registration and a limited number of digitized bone surface points. Furthermore, we aimed at efficient man-machine-interaction during the probe alignment process. Finally, we addressed the problem of registration plausibility estimation in our approach. We modified the Iterative Closest Point (ICP) algorithm, presented by Besl and McKay and frequently used for surface-based registration, such that it can escape from local minima of the cost function to be iteratively minimized. The random-based ICP (R-ICP) we developed is less influenced by the quality of the pre-registration as it can escape from local minima close to the starting point for iterative optimization in the 6D domain of rigid transformations. The R-ICP is also better suited to approximate the global minimum as it can escape from local minima in the vicinity of the global minimum, too. Furthermore, we developed both CT-less and CT-based probe alignment tools along with appropriate man-machine strategies for a more time-efficient palpation process. To improve registration reliability, we developed a simple plausibility test based on data readily available after registration. In a cadaver study, where we evaluated the R-ICP algorithm, the probe alignment tools, and the plausibility test, the R-ICP algorithm consistently

Registration accuracy and quality of real-life images.

Directory of Open Access Journals (Sweden)

Wei-Yen Hsu

Full Text Available BACKGROUND: A common registration problem for the application of consumer device is to align all the acquired image sequences into a complete scene. Image alignment requires a registration algorithm that will compensate as much as possible for geometric variability among images. However, images captured views from a real scene usually produce different distortions. Some are derived from the optic characteristics of image sensors, and others are caused by the specific scenes and objects. METHODOLOGY/PRINCIPAL FINDINGS: An image registration algorithm considering the perspective projection is proposed for the application of consumer devices in this study. It exploits a multiresolution wavelet-based method to extract significant features. An analytic differential approach is then proposed to achieve fast convergence of point matching. Finally, the registration accuracy is further refined to obtain subpixel precision by a feature-based modified Levenberg-Marquardt method. Due to its feature-based and nonlinear characteristic, it converges considerably faster than most other methods. In addition, vignette compensation and color difference adjustment are also performed to further improve the quality of registration results. CONCLUSIONS/SIGNIFICANCE: The performance of the proposed method is evaluated by testing the synthetic and real images acquired by a hand-held digital still camera and in comparison with two registration techniques in terms of the squared sum of intensity differences (SSD and correlation coefficient (CC. The results indicate that the proposed method is promising in registration accuracy and quality, which are statistically significantly better than other two approaches.

A Learning Algorithm for Multimodal Grammar Inference.

Science.gov (United States)

D'Ulizia, A; Ferri, F; Grifoni, P

2011-12-01

The high costs of development and maintenance of multimodal grammars in integrating and understanding input in multimodal interfaces lead to the investigation of novel algorithmic solutions in automating grammar generation and in updating processes. Many algorithms for context-free grammar inference have been developed in the natural language processing literature. An extension of these algorithms toward the inference of multimodal grammars is necessary for multimodal input processing. In this paper, we propose a novel grammar inference mechanism that allows us to learn a multimodal grammar from its positive samples of multimodal sentences. The algorithm first generates the multimodal grammar that is able to parse the positive samples of sentences and, afterward, makes use of two learning operators and the minimum description length metrics in improving the grammar description and in avoiding the over-generalization problem. The experimental results highlight the acceptable performances of the algorithm proposed in this paper since it has a very high probability of parsing valid sentences.

Predicting standard-dose PET image from low-dose PET and multimodal MR images using mapping-based sparse representation

International Nuclear Information System (INIS)

Wang, Yan; Zhou, Jiliu; Zhang, Pei; An, Le; Ma, Guangkai; Kang, Jiayin; Shi, Feng; Shen, Dinggang; Wu, Xi; Lalush, David S; Lin, Weili

2016-01-01

Positron emission tomography (PET) has been widely used in clinical diagnosis for diseases and disorders. To obtain high-quality PET images requires a standard-dose radionuclide (tracer) injection into the human body, which inevitably increases risk of radiation exposure. One possible solution to this problem is to predict the standard-dose PET image from its low-dose counterpart and its corresponding multimodal magnetic resonance (MR) images. Inspired by the success of patch-based sparse representation (SR) in super-resolution image reconstruction, we propose a mapping-based SR (m-SR) framework for standard-dose PET image prediction. Compared with the conventional patch-based SR, our method uses a mapping strategy to ensure that the sparse coefficients, estimated from the multimodal MR images and low-dose PET image, can be applied directly to the prediction of standard-dose PET image. As the mapping between multimodal MR images (or low-dose PET image) and standard-dose PET images can be particularly complex, one step of mapping is often insufficient. To this end, an incremental refinement framework is therefore proposed. Specifically, the predicted standard-dose PET image is further mapped to the target standard-dose PET image, and then the SR is performed again to predict a new standard-dose PET image. This procedure can be repeated for prediction refinement of the iterations. Also, a patch selection based dictionary construction method is further used to speed up the prediction process. The proposed method is validated on a human brain dataset. The experimental results show that our method can outperform benchmark methods in both qualitative and quantitative measures. (paper)

Deformable image registration using convolutional neural networks

NARCIS (Netherlands)

Eppenhof, Koen A.J.; Lafarge, Maxime W.; Moeskops, Pim; Veta, Mitko; Pluim, Josien P.W.

2018-01-01

Deformable image registration can be time-consuming and often needs extensive parameterization to perform well on a specific application. We present a step towards a registration framework based on a three-dimensional convolutional neural network. The network directly learns transformations between

Toward multimodal signal detection of adverse drug reactions.

Science.gov (United States)

Harpaz, Rave; DuMouchel, William; Schuemie, Martijn; Bodenreider, Olivier; Friedman, Carol; Horvitz, Eric; Ripple, Anna; Sorbello, Alfred; White, Ryen W; Winnenburg, Rainer; Shah, Nigam H

2017-12-01

Improving mechanisms to detect adverse drug reactions (ADRs) is key to strengthening post-marketing drug safety surveillance. Signal detection is presently unimodal, relying on a single information source. Multimodal signal detection is based on jointly analyzing multiple information sources. Building on, and expanding the work done in prior studies, the aim of the article is to further research on multimodal signal detection, explore its potential benefits, and propose methods for its construction and evaluation. Four data sources are investigated; FDA's adverse event reporting system, insurance claims, the MEDLINE citation database, and the logs of major Web search engines. Published methods are used to generate and combine signals from each data source. Two distinct reference benchmarks corresponding to well-established and recently labeled ADRs respectively are used to evaluate the performance of multimodal signal detection in terms of area under the ROC curve (AUC) and lead-time-to-detection, with the latter relative to labeling revision dates. Limited to our reference benchmarks, multimodal signal detection provides AUC improvements ranging from 0.04 to 0.09 based on a widely used evaluation benchmark, and a comparative added lead-time of 7-22 months relative to labeling revision dates from a time-indexed benchmark. The results support the notion that utilizing and jointly analyzing multiple data sources may lead to improved signal detection. Given certain data and benchmark limitations, the early stage of development, and the complexity of ADRs, it is currently not possible to make definitive statements about the ultimate utility of the concept. Continued development of multimodal signal detection requires a deeper understanding the data sources used, additional benchmarks, and further research on methods to generate and synthesize signals. Copyright © 2017 Elsevier Inc. All rights reserved.

Registration Service

CERN Multimedia

GS Department

2010-01-01

Following a reorganization in Building 55, please note that the Registration Service is now organised as follows :  Ground floor: access cards (76903). 1st floor : registration of external firms’ personnel (76611 / 76622); car access stickers (76633); biometric registration (79710). Opening hours: 07-30 to 16-00 non-stop. GS-SEM Group General Infrastructure Services Department

Joint sparse representation for robust multimodal biometrics recognition.

Science.gov (United States)

Shekhar, Sumit; Patel, Vishal M; Nasrabadi, Nasser M; Chellappa, Rama

2014-01-01

Traditional biometric recognition systems rely on a single biometric signature for authentication. While the advantage of using multiple sources of information for establishing the identity has been widely recognized, computational models for multimodal biometrics recognition have only recently received attention. We propose a multimodal sparse representation method, which represents the test data by a sparse linear combination of training data, while constraining the observations from different modalities of the test subject to share their sparse representations. Thus, we simultaneously take into account correlations as well as coupling information among biometric modalities. A multimodal quality measure is also proposed to weigh each modality as it gets fused. Furthermore, we also kernelize the algorithm to handle nonlinearity in data. The optimization problem is solved using an efficient alternative direction method. Various experiments show that the proposed method compares favorably with competing fusion-based methods.

TU-C-BRD-01: Image Guided SBRT I: Multi-Modality 4D Imaging

International Nuclear Information System (INIS)

Cai, J; Mageras, G; Pan, T

2014-01-01

Motion management is one of the critical technical challenges for radiation therapy. 4D imaging has been rapidly adopted as essential tool to assess organ motion associated with respiratory breathing. A variety of 4D imaging techniques have been developed and are currently under development based on different imaging modalities such as CT, MRI, PET, and CBCT. Each modality provides specific and complementary information about organ and tumor respiratory motion. Effective use of each different technique or combined use of different techniques can introduce a comprehensive management of tumor motion. Specifically, these techniques have afforded tremendous opportunities to better define and delineate tumor volumes, more accurately perform patient positioning, and effectively apply highly conformal therapy techniques such as IMRT and SBRT. Successful implementation requires good understanding of not only each technique, including unique features, limitations, artifacts, imaging acquisition and process, but also how to systematically apply the information obtained from different imaging modalities using proper tools such as deformable image registration. Furthermore, it is important to understand the differences in the effects of breathing variation between different imaging modalities. A comprehensive motion management strategy using multi-modality 4D imaging has shown promise in improving patient care, but at the same time faces significant challenges. This session will focuses on the current status and advances in imaging respiration-induced organ motion with different imaging modalities: 4D-CT, 4D-MRI, 4D-PET, and 4D-CBCT/DTS. Learning Objectives: Understand the need and role of multimodality 4D imaging in radiation therapy. Understand the underlying physics behind each 4D imaging technique. Recognize the advantages and limitations of each 4D imaging technique

An Efficient Multimodal 2D + 3D Feature-based Approach to Automatic Facial Expression Recognition

KAUST Repository

Li, Huibin

2015-07-29

We present a fully automatic multimodal 2D + 3D feature-based facial expression recognition approach and demonstrate its performance on the BU-3DFE database. Our approach combines multi-order gradient-based local texture and shape descriptors in order to achieve efficiency and robustness. First, a large set of fiducial facial landmarks of 2D face images along with their 3D face scans are localized using a novel algorithm namely incremental Parallel Cascade of Linear Regression (iPar-CLR). Then, a novel Histogram of Second Order Gradients (HSOG) based local image descriptor in conjunction with the widely used first-order gradient based SIFT descriptor are used to describe the local texture around each 2D landmark. Similarly, the local geometry around each 3D landmark is described by two novel local shape descriptors constructed using the first-order and the second-order surface differential geometry quantities, i.e., Histogram of mesh Gradients (meshHOG) and Histogram of mesh Shape index (curvature quantization, meshHOS). Finally, the Support Vector Machine (SVM) based recognition results of all 2D and 3D descriptors are fused at both feature-level and score-level to further improve the accuracy. Comprehensive experimental results demonstrate that there exist impressive complementary characteristics between the 2D and 3D descriptors. We use the BU-3DFE benchmark to compare our approach to the state-of-the-art ones. Our multimodal feature-based approach outperforms the others by achieving an average recognition accuracy of 86.32%. Moreover, a good generalization ability is shown on the Bosphorus database.

An Efficient Multimodal 2D + 3D Feature-based Approach to Automatic Facial Expression Recognition

KAUST Repository

Li, Huibin; Ding, Huaxiong; Huang, Di; Wang, Yunhong; Zhao, Xi; Morvan, Jean-Marie; Chen, Liming

2015-01-01

We present a fully automatic multimodal 2D + 3D feature-based facial expression recognition approach and demonstrate its performance on the BU-3DFE database. Our approach combines multi-order gradient-based local texture and shape descriptors in order to achieve efficiency and robustness. First, a large set of fiducial facial landmarks of 2D face images along with their 3D face scans are localized using a novel algorithm namely incremental Parallel Cascade of Linear Regression (iPar-CLR). Then, a novel Histogram of Second Order Gradients (HSOG) based local image descriptor in conjunction with the widely used first-order gradient based SIFT descriptor are used to describe the local texture around each 2D landmark. Similarly, the local geometry around each 3D landmark is described by two novel local shape descriptors constructed using the first-order and the second-order surface differential geometry quantities, i.e., Histogram of mesh Gradients (meshHOG) and Histogram of mesh Shape index (curvature quantization, meshHOS). Finally, the Support Vector Machine (SVM) based recognition results of all 2D and 3D descriptors are fused at both feature-level and score-level to further improve the accuracy. Comprehensive experimental results demonstrate that there exist impressive complementary characteristics between the 2D and 3D descriptors. We use the BU-3DFE benchmark to compare our approach to the state-of-the-art ones. Our multimodal feature-based approach outperforms the others by achieving an average recognition accuracy of 86.32%. Moreover, a good generalization ability is shown on the Bosphorus database.

Parallel Processing and Bio-inspired Computing for Biomedical Image Registration

Directory of Open Access Journals (Sweden)

Silviu Ioan Bejinariu

2014-07-01

Full Text Available Image Registration (IR is an optimization problem computing optimal parameters of a geometric transform used to overlay one or more source images to a given model by maximizing a similarity measure. In this paper the use of bio-inspired optimization algorithms in image registration is analyzed. Results obtained by means of three different algorithms are compared: Bacterial Foraging Optimization Algorithm (BFOA, Genetic Algorithm (GA and Clonal Selection Algorithm (CSA. Depending on the images type, the registration may be: area based, which is slow but more precise, and features based, which is faster. In this paper a feature based approach based on the Scale Invariant Feature Transform (SIFT is proposed. Finally, results obtained using sequential and parallel implementations on multi-core systems for area based and features based image registration are compared.

A block matching-based registration algorithm for localization of locally advanced lung tumors

Energy Technology Data Exchange (ETDEWEB)

Robertson, Scott P.; Weiss, Elisabeth; Hugo, Geoffrey D., E-mail: gdhugo@vcu.edu [Department of Radiation Oncology, Virginia Commonwealth University, Richmond, Virginia, 23298 (United States)

2014-04-15

Purpose: To implement and evaluate a block matching-based registration (BMR) algorithm for locally advanced lung tumor localization during image-guided radiotherapy. Methods: Small (1 cm{sup 3}), nonoverlapping image subvolumes (“blocks”) were automatically identified on the planning image to cover the tumor surface using a measure of the local intensity gradient. Blocks were independently and automatically registered to the on-treatment image using a rigid transform. To improve speed and robustness, registrations were performed iteratively from coarse to fine image resolution. At each resolution, all block displacements having a near-maximum similarity score were stored. From this list, a single displacement vector for each block was iteratively selected which maximized the consistency of displacement vectors across immediately neighboring blocks. These selected displacements were regularized using a median filter before proceeding to registrations at finer image resolutions. After evaluating all image resolutions, the global rigid transform of the on-treatment image was computed using a Procrustes analysis, providing the couch shift for patient setup correction. This algorithm was evaluated for 18 locally advanced lung cancer patients, each with 4–7 weekly on-treatment computed tomography scans having physician-delineated gross tumor volumes. Volume overlap (VO) and border displacement errors (BDE) were calculated relative to the nominal physician-identified targets to establish residual error after registration. Results: Implementation of multiresolution registration improved block matching accuracy by 39% compared to registration using only the full resolution images. By also considering multiple potential displacements per block, initial errors were reduced by 65%. Using the final implementation of the BMR algorithm, VO was significantly improved from 77% ± 21% (range: 0%–100%) in the initial bony alignment to 91% ± 8% (range: 56%–100%;p < 0

A block matching-based registration algorithm for localization of locally advanced lung tumors

International Nuclear Information System (INIS)

Robertson, Scott P.; Weiss, Elisabeth; Hugo, Geoffrey D.

2014-01-01

Purpose: To implement and evaluate a block matching-based registration (BMR) algorithm for locally advanced lung tumor localization during image-guided radiotherapy. Methods: Small (1 cm 3 ), nonoverlapping image subvolumes (“blocks”) were automatically identified on the planning image to cover the tumor surface using a measure of the local intensity gradient. Blocks were independently and automatically registered to the on-treatment image using a rigid transform. To improve speed and robustness, registrations were performed iteratively from coarse to fine image resolution. At each resolution, all block displacements having a near-maximum similarity score were stored. From this list, a single displacement vector for each block was iteratively selected which maximized the consistency of displacement vectors across immediately neighboring blocks. These selected displacements were regularized using a median filter before proceeding to registrations at finer image resolutions. After evaluating all image resolutions, the global rigid transform of the on-treatment image was computed using a Procrustes analysis, providing the couch shift for patient setup correction. This algorithm was evaluated for 18 locally advanced lung cancer patients, each with 4–7 weekly on-treatment computed tomography scans having physician-delineated gross tumor volumes. Volume overlap (VO) and border displacement errors (BDE) were calculated relative to the nominal physician-identified targets to establish residual error after registration. Results: Implementation of multiresolution registration improved block matching accuracy by 39% compared to registration using only the full resolution images. By also considering multiple potential displacements per block, initial errors were reduced by 65%. Using the final implementation of the BMR algorithm, VO was significantly improved from 77% ± 21% (range: 0%–100%) in the initial bony alignment to 91% ± 8% (range: 56%–100%;p < 0.001). Left

Optimization Model for Web Based Multimodal Interactive Simulations.

Science.gov (United States)

Halic, Tansel; Ahn, Woojin; De, Suvranu

2015-07-15

This paper presents a technique for optimizing the performance of web based multimodal interactive simulations. For such applications where visual quality and the performance of simulations directly influence user experience, overloading of hardware resources may result in unsatisfactory reduction in the quality of the simulation and user satisfaction. However, optimization of simulation performance on individual hardware platforms is not practical. Hence, we present a mixed integer programming model to optimize the performance of graphical rendering and simulation performance while satisfying application specific constraints. Our approach includes three distinct phases: identification, optimization and update . In the identification phase, the computing and rendering capabilities of the client device are evaluated using an exploratory proxy code. This data is utilized in conjunction with user specified design requirements in the optimization phase to ensure best possible computational resource allocation. The optimum solution is used for rendering (e.g. texture size, canvas resolution) and simulation parameters (e.g. simulation domain) in the update phase. Test results are presented on multiple hardware platforms with diverse computing and graphics capabilities to demonstrate the effectiveness of our approach.

One registration multi-atlas-based pseudo-CT generation for attenuation correction in PET/MRI

International Nuclear Information System (INIS)

Arabi, Hossein; Zaidi, Habib

2016-01-01

The outcome of a detailed assessment of various strategies for atlas-based whole-body bone segmentation from magnetic resonance imaging (MRI) was exploited to select the optimal parameters and setting, with the aim of proposing a novel one-registration multi-atlas (ORMA) pseudo-CT generation approach. The proposed approach consists of only one online registration between the target and reference images, regardless of the number of atlas images (N), while for the remaining atlas images, the pre-computed transformation matrices to the reference image are used to align them to the target image. The performance characteristics of the proposed method were evaluated and compared with conventional atlas-based attenuation map generation strategies (direct registration of the entire atlas images followed by voxel-wise weighting (VWW) and arithmetic averaging atlas fusion). To this end, four different positron emission tomography (PET) attenuation maps were generated via arithmetic averaging and VWW scheme using both direct registration and ORMA approaches as well as the 3-class attenuation map obtained from the Philips Ingenuity TF PET/MRI scanner commonly used in the clinical setting. The evaluation was performed based on the accuracy of extracted whole-body bones by the different attenuation maps and by quantitative analysis of resulting PET images compared to CT-based attenuation-corrected PET images serving as reference. The comparison of validation metrics regarding the accuracy of extracted bone using the different techniques demonstrated the superiority of the VWW atlas fusion algorithm achieving a Dice similarity measure of 0.82 ± 0.04 compared to arithmetic averaging atlas fusion (0.60 ± 0.02), which uses conventional direct registration. Application of the ORMA approach modestly compromised the accuracy, yielding a Dice similarity measure of 0.76 ± 0.05 for ORMA-VWW and 0.55 ± 0.03 for ORMA-averaging. The results of quantitative PET analysis followed the same

One registration multi-atlas-based pseudo-CT generation for attenuation correction in PET/MRI

Energy Technology Data Exchange (ETDEWEB)

Arabi, Hossein [Geneva University Hospital, Division of Nuclear Medicine and Molecular Imaging, Geneva 4 (Switzerland); Zaidi, Habib [Geneva University Hospital, Division of Nuclear Medicine and Molecular Imaging, Geneva 4 (Switzerland); Geneva University, Geneva Neuroscience Center, Geneva (Switzerland); University of Groningen, University Medical Center Groningen, Department of Nuclear Medicine and Molecular Imaging, Groningen (Netherlands); University of Southern Denmark, Department of Nuclear Medicine, Odense (Denmark)

2016-10-15

The outcome of a detailed assessment of various strategies for atlas-based whole-body bone segmentation from magnetic resonance imaging (MRI) was exploited to select the optimal parameters and setting, with the aim of proposing a novel one-registration multi-atlas (ORMA) pseudo-CT generation approach. The proposed approach consists of only one online registration between the target and reference images, regardless of the number of atlas images (N), while for the remaining atlas images, the pre-computed transformation matrices to the reference image are used to align them to the target image. The performance characteristics of the proposed method were evaluated and compared with conventional atlas-based attenuation map generation strategies (direct registration of the entire atlas images followed by voxel-wise weighting (VWW) and arithmetic averaging atlas fusion). To this end, four different positron emission tomography (PET) attenuation maps were generated via arithmetic averaging and VWW scheme using both direct registration and ORMA approaches as well as the 3-class attenuation map obtained from the Philips Ingenuity TF PET/MRI scanner commonly used in the clinical setting. The evaluation was performed based on the accuracy of extracted whole-body bones by the different attenuation maps and by quantitative analysis of resulting PET images compared to CT-based attenuation-corrected PET images serving as reference. The comparison of validation metrics regarding the accuracy of extracted bone using the different techniques demonstrated the superiority of the VWW atlas fusion algorithm achieving a Dice similarity measure of 0.82 ± 0.04 compared to arithmetic averaging atlas fusion (0.60 ± 0.02), which uses conventional direct registration. Application of the ORMA approach modestly compromised the accuracy, yielding a Dice similarity measure of 0.76 ± 0.05 for ORMA-VWW and 0.55 ± 0.03 for ORMA-averaging. The results of quantitative PET analysis followed the same

Passively Q-switched dual-wavelength thulium-doped fiber laser based on a multimode interference filter and a semiconductor saturable absorber

Science.gov (United States)

Wang, M.; Huang, Y. J.; Ruan, S. C.

2018-04-01

In this paper, we have demonstrated a theta cavity passively Q-switched dual-wavelength fiber laser based on a multimode interference filter and a semiconductor saturable absorber. Relying on the properties of the fiber theta cavity, the laser can operate unidirectionally without an optical isolator. A semiconductor saturable absorber played the role of passive Q-switch while a section of single-mode-multimode-single-mode fiber structure served as an multimode interference filter and was used for selecting the lasing wavelengths. By suitably manipulating the polarization controller, stable dual-wavelength Q-switched operation was obtained at ~1946.8 nm and ~1983.8 nm with maximum output power and minimum pulse duration of ~47 mW and ~762.5 ns, respectively. The pulse repetition rate can be tuned from ~20.2 kHz to ~79.7 kHz by increasing the pump power from ~2.12 W to ~5.4 W.

Multimodal physiotherapy treatment based on a biobehavioral approach for patients with chronic cervico-craniofacial pain: a prospective case series.

Science.gov (United States)

Marcos-Martín, Fernando; González-Ferrero, Luis; Martín-Alcocer, Noelia; Paris-Alemany, Alba; La Touche, Roy

2018-01-17

The purpose of this prospective case series was to observe and describe changes in patients with chronic cervico-craniofacial pain of muscular origin treated with multimodal physiotherapy based on a biobehavioral approach. Nine patients diagnosed with chronic myofascial temporomandibular disorder and neck pain were treated with 6 sessions over the course of 2 weeks including: (1) orthopedic manual physiotherapy (joint mobilizations, neurodynamic mobilization, and dynamic soft tissue mobilizations); (2) therapeutic exercises (motor control and muscular endurance exercises); and (3) patient education. The outcome measures of craniofacial (CF-PDI) and neck disability (NDI), kinesiophobia (TSK-11) and catastrophizing (PCS), and range of cervical and mandibular motion (ROM) and posture were collected at baseline, and at 2 and 14 weeks post-baseline. Compared to baseline, statistically significant (p posture were observed following a multimodal physiotherapy treatment based on a biobehavioral approach.

Behaviors study of image registration algorithms in image guided radiation therapy

International Nuclear Information System (INIS)

Zou Lian; Hou Qing

2008-01-01

Objective: Study the behaviors of image registration algorithms, and analyze the elements which influence the performance of image registrations. Methods: Pre-known corresponding coordinates were appointed for reference image and moving image, and then the influence of region of interest (ROI) selection, transformation function initial parameters and coupled parameter spaces on registration results were studied with a software platform developed in home. Results: Region of interest selection had a manifest influence on registration performance. An improperly chosen ROI resulted in a bad registration. Transformation function initial parameters selection based on pre-known information could improve the accuracy of image registration. Coupled parameter spaces would enhance the dependence of image registration algorithm on ROI selection. Conclusions: It is necessary for clinic IGRT to obtain a ROI selection strategy (depending on specific commercial software) correlated to tumor sites. Three suggestions for image registration technique developers are automatic selection of the initial parameters of transformation function based on pre-known information, developing specific image registration algorithm for specific image feature, and assembling real-time image registration algorithms according to tumor sites selected by software user. (authors)

Water-stable NaLuF4-based upconversion nanophosphors with long-term validity for multimodal lymphatic imaging.

Science.gov (United States)

Zhou, Jing; Zhu, Xingjun; Chen, Min; Sun, Yun; Li, Fuyou

2012-09-01

Multimodal imaging is rapidly becoming an important tool for biomedical applications because it can compensate for the deficiencies of individual imaging modalities. Herein, multifunctional NaLuF(4)-based upconversion nanoparticles (Lu-UCNPs) were synthesized though a facile one-step microemulsion method under ambient condition. The doping of lanthanide ions (Gd(3+), Yb(3+) and Er(3+)/Tm(3+)) endows the Lu-UCNPs with high T(1)-enhancement, bright upconversion luminescence (UCL) emissions, and excellent X-ray absorption coefficient. Moreover, the as-prepared Lu-UCNPs are stable in water for more than six months, due to the protection of sodium glutamate and diethylene triamine pentacetate acid (DTPA) coordinating ligands on the surface. Lu-UCNPs have been successfully applied to the trimodal CT/MR/UCL lymphatic imaging on the modal of small animals. It is worth noting that Lu-UCNPs could be used for imaging even after preserving for over six months. In vitro transmission electron microscope (TEM), methyl thiazolyl tetrazolium (MTT) assay and histological analysis demonstrated that Lu-UCNPs exhibited low toxicity on living systems. Therefore, Lu-UCNPs could be multimodal agents for CT/MR/UCL imaging, and the concept can be served as a platform technology for the next-generation of probes for multimodal imaging. Copyright © 2012 Elsevier Ltd. All rights reserved.

Multimodal Aspects of Corporate Social Responsibility Communication

Directory of Open Access Journals (Sweden)

Carmen Daniela Maier

2014-12-01

Full Text Available This article addresses how the multimodal persuasive strategies of corporate social responsibility communication can highlight a company’s commitment to gender empowerment and environmental protection while advertising simultaneously its products. Drawing on an interdisciplinary methodological framework related to CSR communication, multimodal discourse analysis and gender theory, the article proposes a multimodal analysis model through which it is possible to map and explain the multimodal persuasive strategies employed by Coca-Cola company in their community-related films. By examining the semiotic modes’ interconnectivity and functional differentiation, this analytical endeavour expands the existing research work as the usual textual focus is extended to a multimodal one.

Multimodal sequence learning.

Science.gov (United States)

Kemény, Ferenc; Meier, Beat

2016-02-01

While sequence learning research models complex phenomena, previous studies have mostly focused on unimodal sequences. The goal of the current experiment is to put implicit sequence learning into a multimodal context: to test whether it can operate across different modalities. We used the Task Sequence Learning paradigm to test whether sequence learning varies across modalities, and whether participants are able to learn multimodal sequences. Our results show that implicit sequence learning is very similar regardless of the source modality. However, the presence of correlated task and response sequences was required for learning to take place. The experiment provides new evidence for implicit sequence learning of abstract conceptual representations. In general, the results suggest that correlated sequences are necessary for implicit sequence learning to occur. Moreover, they show that elements from different modalities can be automatically integrated into one unitary multimodal sequence. Copyright © 2015 Elsevier B.V. All rights reserved.

Automated registration of freehand B-mode ultrasound and magnetic resonance imaging of the carotid arteries based on geometric features

DEFF Research Database (Denmark)

Carvalho, Diego D. B.; Arias Lorza, Andres Mauricio; Niessen, Wiro J.

2017-01-01

and segmentations by minimizing a weighted sum of the Euclidean distance between centerlines and the dissimilarity between segmentations. The method was evaluated in 28 carotid arteries from eight patients and six healthy volunteers. First, the automated US lumen segmentation method was validated and optimized......-and-point-based method, a registration using only the centerlines and a registration using manual US lumen segmentations. Registration accuracy was measured in terms of the mean surface distance between manual US segmentations and the registered MRI segmentations. The average mean surface distance was 0.78 ± 0.34 mm...

Spline-based image-to-volume registration for three-dimensional electron microscopy

International Nuclear Information System (INIS)

Jonic, S.; Sorzano, C.O.S.; Thevenaz, P.; El-Bez, C.; De Carlo, S.; Unser, M.

2005-01-01

This paper presents an algorithm based on a continuous framework for a posteriori angular and translational assignment in three-dimensional electron microscopy (3DEM) of single particles. Our algorithm can be used advantageously to refine the assignment of standard quantized-parameter methods by registering the images to a reference 3D particle model. We achieve the registration by employing a gradient-based iterative minimization of a least-squares measure of dissimilarity between an image and a projection of the volume in the Fourier transform (FT) domain. We compute the FT of the projection using the central-slice theorem (CST). To compute the gradient accurately, we take advantage of a cubic B-spline model of the data in the frequency domain. To improve the robustness of the algorithm, we weight the cost function in the FT domain and apply a 'mixed' strategy for the assignment based on the minimum value of the cost function at registration for several different initializations. We validate our algorithm in a fully controlled simulation environment. We show that the mixed strategy improves the assignment accuracy; on our data, the quality of the angular and translational assignment was better than 2 voxel (i.e., 6.54 A). We also test the performance of our algorithm on real EM data. We conclude that our algorithm outperforms a standard projection-matching refinement in terms of both consistency of 3D reconstructions and speed

21 CFR 710.6 - Notification of registrant; cosmetic product establishment registration number.

Science.gov (United States)

2010-04-01

... 21 Food and Drugs 7 2010-04-01 2010-04-01 false Notification of registrant; cosmetic product... OF HEALTH AND HUMAN SERVICES (CONTINUED) COSMETICS VOLUNTARY REGISTRATION OF COSMETIC PRODUCT ESTABLISHMENTS § 710.6 Notification of registrant; cosmetic product establishment registration number. The...

Registration of Laser Scanning Point Clouds: A Review

Science.gov (United States)

Cheng, Liang; Chen, Song; Xu, Hao; Wu, Yang; Li, Manchun

2018-01-01

The integration of multi-platform, multi-angle, and multi-temporal LiDAR data has become important for geospatial data applications. This paper presents a comprehensive review of LiDAR data registration in the fields of photogrammetry and remote sensing. At present, a coarse-to-fine registration strategy is commonly used for LiDAR point clouds registration. The coarse registration method is first used to achieve a good initial position, based on which registration is then refined utilizing the fine registration method. According to the coarse-to-fine framework, this paper reviews current registration methods and their methodologies, and identifies important differences between them. The lack of standard data and unified evaluation systems is identified as a factor limiting objective comparison of different methods. The paper also describes the most commonly-used point cloud registration error analysis methods. Finally, avenues for future work on LiDAR data registration in terms of applications, data, and technology are discussed. In particular, there is a need to address registration of multi-angle and multi-scale data from various newly available types of LiDAR hardware, which will play an important role in diverse applications such as forest resource surveys, urban energy use, cultural heritage protection, and unmanned vehicles.

75 FR 79320 - Security-Based Swap Data Repository Registration, Duties, and Core Principles

Science.gov (United States)

2010-12-20

... SECURITIES AND EXCHANGE COMMISSION 17 CFR Parts 240 and 249 [Release No. 34-63347; File No. S7-35-10] RIN 3235-AK79 Security-Based Swap Data Repository Registration, Duties, and Core Principles Correction In proposed rule document 2010-29719 beginning on page 77306 in the issue of December 10, 2010...

EEG Classification for Hybrid Brain-Computer Interface Using a Tensor Based Multiclass Multimodal Analysis Scheme.

Science.gov (United States)

Ji, Hongfei; Li, Jie; Lu, Rongrong; Gu, Rong; Cao, Lei; Gong, Xiaoliang

2016-01-01

Electroencephalogram- (EEG-) based brain-computer interface (BCI) systems usually utilize one type of changes in the dynamics of brain oscillations for control, such as event-related desynchronization/synchronization (ERD/ERS), steady state visual evoked potential (SSVEP), and P300 evoked potentials. There is a recent trend to detect more than one of these signals in one system to create a hybrid BCI. However, in this case, EEG data were always divided into groups and analyzed by the separate processing procedures. As a result, the interactive effects were ignored when different types of BCI tasks were executed simultaneously. In this work, we propose an improved tensor based multiclass multimodal scheme especially for hybrid BCI, in which EEG signals are denoted as multiway tensors, a nonredundant rank-one tensor decomposition model is proposed to obtain nonredundant tensor components, a weighted fisher criterion is designed to select multimodal discriminative patterns without ignoring the interactive effects, and support vector machine (SVM) is extended to multiclass classification. Experiment results suggest that the proposed scheme can not only identify the different changes in the dynamics of brain oscillations induced by different types of tasks but also capture the interactive effects of simultaneous tasks properly. Therefore, it has great potential use for hybrid BCI.

SU-F-BRF-10: Deformable MRI to CT Validation Employing Same Day Planning MRI for Surrogate Analysis

Energy Technology Data Exchange (ETDEWEB)

Padgett, K; Stoyanova, R; Johnson, P; Dogan, N; Pollack, A [University of Miami School of Medicine, Miami, FL (United States); Piper, J; Javorek, A [MIM Software, Inc., Beachwood, OH (United States)

2014-06-15

Purpose: To compare rigid and deformable registrations of the prostate in the multi-modality setting (diagnostic-MRI to planning-CT) by utilizing a planning-MRI as a surrogate. The surrogate allows for the direct quantitative analysis which can be difficult in the multi-modality domain where intensity mapping differs. Methods: For ten subjects, T2 fast-spin-echo images were acquired at two different time points, the first several weeks prior to planning (diagnostic-MRI) and the second on the same day in which the planning CT was collected (planning-MRI). Significant effort in patient positioning and bowel/bladder preparation was undertaken to minimize distortion of the prostate in all datasets. The diagnostic-MRI was deformed to the planning-CT utilizing a commercially available deformable registration algorithm synthesized from local registrations. The deformed MRI was then rigidly aligned to the planning MRI which was used as the surrogate for the planning-CT. Agreement between the two MRI datasets was scored using intensity based metrics including Pearson correlation and normalized mutual information, NMI. A local analysis was performed by looking only within the prostate, proximal seminal vesicles, penile bulb and combined areas. A similar method was used to assess a rigid registration between the diagnostic-MRI and planning-CT. Results: Utilizing the NMI, the deformable registrations were superior to the rigid registrations in 9 of 10 cases demonstrating a 15.94% improvement (p-value < 0.001) within the combined area. The Pearson correlation showed similar results with the deformable registration superior in the same number of cases and demonstrating a 6.97% improvement (p-value <0.011). Conclusion: Validating deformable multi-modality registrations using spatial intensity based metrics is difficult due to the inherent differences in intensity mapping. This population provides an ideal testing ground for MRI to CT deformable registrations by obviating the need

Multimodal Processes Rescheduling

DEFF Research Database (Denmark)

Bocewicz, Grzegorz; Banaszak, Zbigniew A.; Nielsen, Peter

2013-01-01

Cyclic scheduling problems concerning multimodal processes are usually observed in FMSs producing multi-type parts where the Automated Guided Vehicles System (AGVS) plays a role of a material handling system. Schedulability analysis of concurrently flowing cyclic processes (SCCP) exe-cuted in the......Cyclic scheduling problems concerning multimodal processes are usually observed in FMSs producing multi-type parts where the Automated Guided Vehicles System (AGVS) plays a role of a material handling system. Schedulability analysis of concurrently flowing cyclic processes (SCCP) exe...

Evaluation of accuracy of B-spline transformation-based deformable image registration with different parameter settings for thoracic images

International Nuclear Information System (INIS)

Kanai, Takayuki; Kadoya, Noriyuki; Ito, Kengo

2014-01-01

Deformable image registration (DIR) is fundamental technique for adaptive radiotherapy and image-guided radiotherapy. However, further improvement of DIR is still needed. We evaluated the accuracy of B-spline transformation-based DIR implemented in elastix. This registration package is largely based on the Insight Segmentation and Registration Toolkit (ITK), and several new functions were implemented to achieve high DIR accuracy. The purpose of this study was to clarify whether new functions implemented in elastix are useful for improving DIR accuracy. Thoracic 4D computed tomography images of ten patients with esophageal or lung cancer were studied. Datasets for these patients were provided by DIR-lab (dir-lab.com) and included a coordinate list of anatomical landmarks that had been manually identified. DIR between peak-inhale and peak-exhale images was performed with four types of parameter settings. The first one represents original ITK (Parameter 1). The second employs the new function of elastix (Parameter 2), and the third was created to verify whether new functions improve DIR accuracy while keeping computational time (Parameter 3). The last one partially employs a new function (Parameter 4). Registration errors for these parameter settings were calculated using the manually determined landmark pairs. 3D registration errors with standard deviation over all cases were 1.78 (1.57), 1.28 (1.10), 1.44 (1.09) and 1.36 (1.35) mm for Parameter 1, 2, 3 and 4, respectively, indicating that the new functions are useful for improving DIR accuracy, even while maintaining the computational time, and this B-spline-based DIR could be used clinically to achieve high-accuracy adaptive radiotherapy. (author)

Language proficiency and nursing registration.

Science.gov (United States)

Müller, Amanda

2016-02-01

This discussion paper focuses on English proficiency standards for nursing registration in Australia, how Australia has dealt with the issue of language proficiency, and the factors which have led to the establishment of the current language standards. Also, this paper will provide a comparison of the two language tests that are currently accepted in Australia (OET and IELTS), including the appropriateness of these tests and the minimum standards used. The paper will also examine the use of educational background as an indicator of language proficiency. Finally, communication-based complaints in the post-registration environment will be explored, and some discussion will be provided about why pre-registration measures might have failed to prevent such problematic situations from occurring. Copyright © 2015 Elsevier Ltd. All rights reserved.

A novel multimodal chromatography based single step purification process for efficient manufacturing of an E. coli based biotherapeutic protein product.

Science.gov (United States)

Bhambure, Rahul; Gupta, Darpan; Rathore, Anurag S

2013-11-01

Methionine oxidized, reduced and fMet forms of a native recombinant protein product are often the critical product variants which are associated with proteins expressed as bacterial inclusion bodies in E. coli. Such product variants differ from native protein in their structural and functional aspects, and may lead to loss of biological activity and immunogenic response in patients. This investigation focuses on evaluation of multimodal chromatography for selective removal of these product variants using recombinant human granulocyte colony stimulating factor (GCSF) as the model protein. Unique selectivity in separation of closely related product variants was obtained using combined pH and salt based elution gradients in hydrophobic charge induction chromatography. Simultaneous removal of process related impurities was also achieved in flow-through leading to single step purification process for the GCSF. Results indicate that the product recovery of up to 90.0% can be obtained with purity levels of greater than 99.0%. Binding the target protein at pHproduct variants using the combined pH and salt based elution gradient and removal of the host cell impurities in flow-through are the key novel features of the developed multimodal chromatographic purification step. Copyright © 2013 Elsevier B.V. All rights reserved.

16 CFR 1130.8 - Requirements for Web site registration or alternative e-mail registration.

Science.gov (United States)

2010-01-01

... registration. (a) Link to registration page. The manufacturer's Web site, or other Web site established for the... web page that goes directly to “Product Registration.” (b) Purpose statement. The registration page... registration page. The Web site registration page shall request only the consumer's name, address, telephone...

Position tracking of moving liver lesion based on real-time registration between 2D ultrasound and 3D preoperative images

International Nuclear Information System (INIS)

Weon, Chijun; Hyun Nam, Woo; Lee, Duhgoon; Ra, Jong Beom; Lee, Jae Young

2015-01-01

Purpose: Registration between 2D ultrasound (US) and 3D preoperative magnetic resonance (MR) (or computed tomography, CT) images has been studied recently for US-guided intervention. However, the existing techniques have some limits, either in the registration speed or the performance. The purpose of this work is to develop a real-time and fully automatic registration system between two intermodal images of the liver, and subsequently an indirect lesion positioning/tracking algorithm based on the registration result, for image-guided interventions. Methods: The proposed position tracking system consists of three stages. In the preoperative stage, the authors acquire several 3D preoperative MR (or CT) images at different respiratory phases. Based on the transformations obtained from nonrigid registration of the acquired 3D images, they then generate a 4D preoperative image along the respiratory phase. In the intraoperative preparatory stage, they properly attach a 3D US transducer to the patient’s body and fix its pose using a holding mechanism. They then acquire a couple of respiratory-controlled 3D US images. Via the rigid registration of these US images to the 3D preoperative images in the 4D image, the pose information of the fixed-pose 3D US transducer is determined with respect to the preoperative image coordinates. As feature(s) to use for the rigid registration, they may choose either internal liver vessels or the inferior vena cava. Since the latter is especially useful in patients with a diffuse liver disease, the authors newly propose using it. In the intraoperative real-time stage, they acquire 2D US images in real-time from the fixed-pose transducer. For each US image, they select candidates for its corresponding 2D preoperative slice from the 4D preoperative MR (or CT) image, based on the predetermined pose information of the transducer. The correct corresponding image is then found among those candidates via real-time 2D registration based on a

Multimodal Resources in Transnational Adoption

DEFF Research Database (Denmark)

Raudaskoski, Pirkko Liisa

The paper discusses an empirical analysis which highlights the multimodal nature of identity construction. A documentary on transnational adoption provides real life incidents as research material. The incidents involve (or from them emerge) various kinds of multimodal resources and participants...

Validation for 2D/3D registration II: The comparison of intensity- and gradient-based merit functions using a new gold standard data set

International Nuclear Information System (INIS)

Gendrin, Christelle; Markelj, Primoz; Pawiro, Supriyanto Ardjo; Spoerk, Jakob; Bloch, Christoph; Weber, Christoph; Figl, Michael; Bergmann, Helmar; Birkfellner, Wolfgang; Likar, Bostjan; Pernus, Franjo

2011-01-01

Purpose: A new gold standard data set for validation of 2D/3D registration based on a porcine cadaver head with attached fiducial markers was presented in the first part of this article. The advantage of this new phantom is the large amount of soft tissue, which simulates realistic conditions for registration. This article tests the performance of intensity- and gradient-based algorithms for 2D/3D registration using the new phantom data set. Methods: Intensity-based methods with four merit functions, namely, cross correlation, rank correlation, correlation ratio, and mutual information (MI), and two gradient-based algorithms, the backprojection gradient-based (BGB) registration method and the reconstruction gradient-based (RGB) registration method, were compared. Four volumes consisting of CBCT with two fields of view, 64 slice multidetector CT, and magnetic resonance-T1 weighted images were registered to a pair of kV x-ray images and a pair of MV images. A standardized evaluation methodology was employed. Targets were evenly spread over the volumes and 250 starting positions of the 3D volumes with initial displacements of up to 25 mm from the gold standard position were calculated. After the registration, the displacement from the gold standard was retrieved and the root mean square (RMS), mean, and standard deviation mean target registration errors (mTREs) over 250 registrations were derived. Additionally, the following merit properties were computed: Accuracy, capture range, number of minima, risk of nonconvergence, and distinctiveness of optimum for better comparison of the robustness of each merit. Results: Among the merit functions used for the intensity-based method, MI reached the best accuracy with an RMS mTRE down to 1.30 mm. Furthermore, it was the only merit function that could accurately register the CT to the kV x rays with the presence of tissue deformation. As for the gradient-based methods, BGB and RGB methods achieved subvoxel accuracy (RMS m

Expert Talks: Understanding civil registration and vital statistics ...

International Development Research Centre (IDRC) Digital Library (Canada)

2017-09-13

Sep 13, 2017 ... What are CRVS systems and why do they matter? ... Cambodia cleared civil registration backlogs by providing free, time-bound registration. ... IDRC supports results-based research that has real impacts on the ground and ...

The dynamics of multimodal integration: The averaging diffusion model.

Science.gov (United States)

Turner, Brandon M; Gao, Juan; Koenig, Scott; Palfy, Dylan; L McClelland, James

2017-12-01

We combine extant theories of evidence accumulation and multi-modal integration to develop an integrated framework for modeling multimodal integration as a process that unfolds in real time. Many studies have formulated sensory processing as a dynamic process where noisy samples of evidence are accumulated until a decision is made. However, these studies are often limited to a single sensory modality. Studies of multimodal stimulus integration have focused on how best to combine different sources of information to elicit a judgment. These studies are often limited to a single time point, typically after the integration process has occurred. We address these limitations by combining the two approaches. Experimentally, we present data that allow us to study the time course of evidence accumulation within each of the visual and auditory domains as well as in a bimodal condition. Theoretically, we develop a new Averaging Diffusion Model in which the decision variable is the mean rather than the sum of evidence samples and use it as a base for comparing three alternative models of multimodal integration, allowing us to assess the optimality of this integration. The outcome reveals rich individual differences in multimodal integration: while some subjects' data are consistent with adaptive optimal integration, reweighting sources of evidence as their relative reliability changes during evidence integration, others exhibit patterns inconsistent with optimality.

Survey of Non-Rigid Registration Tools in Medicine.

Science.gov (United States)

Keszei, András P; Berkels, Benjamin; Deserno, Thomas M

2017-02-01

We catalogue available software solutions for non-rigid image registration to support scientists in selecting suitable tools for specific medical registration purposes. Registration tools were identified using non-systematic search in Pubmed, Web of Science, IEEE Xplore® Digital Library, Google Scholar, and through references in identified sources (n = 22). Exclusions are due to unavailability or inappropriateness. The remaining (n = 18) tools were classified by (i) access and technology, (ii) interfaces and application, (iii) living community, (iv) supported file formats, and (v) types of registration methodologies emphasizing the similarity measures implemented. Out of the 18 tools, (i) 12 are open source, 8 are released under a permissive free license, which imposes the least restrictions on the use and further development of the tool, 8 provide graphical processing unit (GPU) support; (ii) 7 are built on software platforms, 5 were developed for brain image registration; (iii) 6 are under active development but only 3 have had their last update in 2015 or 2016; (iv) 16 support the Analyze format, while 7 file formats can be read with only one of the tools; and (v) 6 provide multiple registration methods and 6 provide landmark-based registration methods. Based on open source, licensing, GPU support, active community, several file formats, algorithms, and similarity measures, the tools Elastics and Plastimatch are chosen for the platform ITK and without platform requirements, respectively. Researchers in medical image analysis already have a large choice of registration tools freely available. However, the most recently published algorithms may not be included in the tools, yet.

3-D brain image registration using optimal morphological processing

International Nuclear Information System (INIS)

Loncaric, S.; Dhawan, A.P.

1994-01-01

The three-dimensional (3-D) registration of Magnetic Resonance (MR) and Positron Emission Tomographic (PET) images of the brain is important for analysis of the human brain and its diseases. A procedure for optimization of (3-D) morphological structuring elements, based on a genetic algorithm, is presented in the paper. The registration of the MR and PET images is done by means of a registration procedure in two major phases. In the first phase, the Iterative Principal Axis Transform (IPAR) is used for initial registration. In the second phase, the optimal shape description method based on the Morphological Signature Transform (MST) is used for final registration. The morphological processing is used to improve the accuracy of the basic IPAR method. The brain ventricle is used as a landmark for MST registration. A near-optimal structuring element obtained by means of a genetic algorithm is used in MST to describe the shape of the ventricle. The method has been tested on the set of brain images demonstrating the feasibility of approach. (author). 11 refs., 3 figs

[Accurate 3D free-form registration between fan-beam CT and cone-beam CT].

Science.gov (United States)

Liang, Yueqiang; Xu, Hongbing; Li, Baosheng; Li, Hongsheng; Yang, Fujun

2012-06-01

Because the X-ray scatters, the CT numbers in cone-beam CT cannot exactly correspond to the electron densities. This, therefore, results in registration error when the intensity-based registration algorithm is used to register planning fan-beam CT and cone-beam CT. In order to reduce the registration error, we have developed an accurate gradient-based registration algorithm. The gradient-based deformable registration problem is described as a minimization of energy functional. Through the calculus of variations and Gauss-Seidel finite difference method, we derived the iterative formula of the deformable registration. The algorithm was implemented by GPU through OpenCL framework, with which the registration time was greatly reduced. Our experimental results showed that the proposed gradient-based registration algorithm could register more accurately the clinical cone-beam CT and fan-beam CT images compared with the intensity-based algorithm. The GPU-accelerated algorithm meets the real-time requirement in the online adaptive radiotherapy.

Multimodal Diversity of Postmodernist Fiction Text

Directory of Open Access Journals (Sweden)

U. I. Tykha

2016-12-01

Full Text Available The article is devoted to the analysis of structural and functional manifestations of multimodal diversity in postmodernist fiction texts. Multimodality is defined as the coexistence of more than one semiotic mode within a certain context. Multimodal texts feature a diversity of semiotic modes in the communication and development of their narrative. Such experimental texts subvert conventional patterns by introducing various semiotic resources – verbal or non-verbal.

Experiments in Multimodal Information Presentation

NARCIS (Netherlands)

van Hooijdonk, Charlotte; Bosma, W.E.; Krahmer, Emiel; Maes, Alfons; Theune, Mariet; van den Bosch, Antal; Bouma, Gosse

In this chapter we describe three experiments investigating multimodal information presentation in the context of a medical QA system. In Experiment 1, we wanted to know how non-experts design (multimodal) answers to medical questions, distinguishing between what questions and how questions. In

Modeling multimodal human-computer interaction

NARCIS (Netherlands)

Obrenovic, Z.; Starcevic, D.

2004-01-01

Incorporating the well-known Unified Modeling Language into a generic modeling framework makes research on multimodal human-computer interaction accessible to a wide range off software engineers. Multimodal interaction is part of everyday human discourse: We speak, move, gesture, and shift our gaze

Multimodal nonlinear microscope based on a compact fiber-format laser source

Science.gov (United States)

Crisafi, Francesco; Kumar, Vikas; Perri, Antonio; Marangoni, Marco; Cerullo, Giulio; Polli, Dario

2018-01-01

We present a multimodal non-linear optical (NLO) laser-scanning microscope, based on a compact fiber-format excitation laser and integrating coherent anti-Stokes Raman scattering (CARS), stimulated Raman scattering (SRS) and two-photon-excitation fluorescence (TPEF) on a single platform. We demonstrate its capabilities in simultaneously acquiring CARS and SRS images of a blend of 6-μm poly(methyl methacrylate) beads and 3-μm polystyrene beads. We then apply it to visualize cell walls and chloroplast of an unprocessed fresh leaf of Elodea aquatic plant via SRS and TPEF modalities, respectively. The presented NLO microscope, developed in house using off-the-shelf components, offers full accessibility to the optical path and ensures its easy re-configurability and flexibility.

Registration-based segmentation with articulated model from multipostural magnetic resonance images for hand bone motion animation.

Science.gov (United States)

Chen, Hsin-Chen; Jou, I-Ming; Wang, Chien-Kuo; Su, Fong-Chin; Sun, Yung-Nien

2010-06-01

The quantitative measurements of hand bones, including volume, surface, orientation, and position are essential in investigating hand kinematics. Moreover, within the measurement stage, bone segmentation is the most important step due to its certain influences on measuring accuracy. Since hand bones are small and tubular in shape, magnetic resonance (MR) imaging is prone to artifacts such as nonuniform intensity and fuzzy boundaries. Thus, greater detail is required for improving segmentation accuracy. The authors then propose using a novel registration-based method on an articulated hand model to segment hand bones from multipostural MR images. The proposed method consists of the model construction and registration-based segmentation stages. Given a reference postural image, the first stage requires construction of a drivable reference model characterized by hand bone shapes, intensity patterns, and articulated joint mechanism. By applying the reference model to the second stage, the authors initially design a model-based registration pursuant to intensity distribution similarity, MR bone intensity properties, and constraints of model geometry to align the reference model to target bone regions of the given postural image. The authors then refine the resulting surface to improve the superimposition between the registered reference model and target bone boundaries. For each subject, given a reference postural image, the proposed method can automatically segment all hand bones from all other postural images. Compared to the ground truth from two experts, the resulting surface image had an average margin of error within 1 mm (mm) only. In addition, the proposed method showed good agreement on the overlap of bone segmentations by dice similarity coefficient and also demonstrated better segmentation results than conventional methods. The proposed registration-based segmentation method can successfully overcome drawbacks caused by inherent artifacts in MR images and

Groupwise registration of MR brain images with tumors

Science.gov (United States)

Tang, Zhenyu; Wu, Yihong; Fan, Yong

2017-09-01

A novel groupwise image registration framework is developed for registering MR brain images with tumors. Our method iteratively estimates a normal-appearance counterpart for each tumor image to be registered and constructs a directed graph (digraph) of normal-appearance images to guide the groupwise image registration. Particularly, our method maps each tumor image to its normal appearance counterpart by identifying and inpainting brain tumor regions with intensity information estimated using a low-rank plus sparse matrix decomposition based image representation technique. The estimated normal-appearance images are groupwisely registered to a group center image guided by a digraph of images so that the total length of ‘image registration paths’ to be the minimum, and then the original tumor images are warped to the group center image using the resulting deformation fields. We have evaluated our method based on both simulated and real MR brain tumor images. The registration results were evaluated with overlap measures of corresponding brain regions and average entropy of image intensity information, and Wilcoxon signed rank tests were adopted to compare different methods with respect to their regional overlap measures. Compared with a groupwise image registration method that is applied to normal-appearance images estimated using the traditional low-rank plus sparse matrix decomposition based image inpainting, our method achieved higher image registration accuracy with statistical significance (p  =  7.02  ×  10-9).

Applications of Elpasolites as a Multimode Radiation Sensor

Science.gov (United States)

Guckes, Amber

This study consists of both computational and experimental investigations. The computational results enabled detector design selections and confirmed experimental results. The experimental results determined that the CLYC scintillation detector can be applied as a functional and field-deployable multimode radiation sensor. The computational study utilized MCNP6 code to investigate the response of CLYC to various incident radiations and to determine the feasibility of its application as a handheld multimode sensor and as a single-scintillator collimated directional detection system. These simulations include: • Characterization of the response of the CLYC scintillator to gamma-rays and neutrons; • Study of the isotopic enrichment of 7Li versus 6Li in the CLYC for optimal detection of both thermal neutrons and fast neutrons; • Analysis of collimator designs to determine the optimal collimator for the single CLYC sensor directional detection system to assay gamma rays and neutrons; Simulations of a handheld CLYC multimode sensor and a single CLYC scintillator collimated directional detection system with the optimized collimator to determine the feasibility of detecting nuclear materials that could be encountered during field operations. These nuclear materials include depleted uranium, natural uranium, low-enriched uranium, highly-enriched uranium, reactor-grade plutonium, and weapons-grade plutonium. The experimental study includes the design, construction, and testing of both a handheld CLYC multimode sensor and a single CLYC scintillator collimated directional detection system. Both were designed in the Inventor CAD software and based on results of the computational study to optimize its performance. The handheld CLYC multimode sensor is modular, scalable, low?power, and optimized for high count rates. Commercial?off?the?shelf components were used where possible in order to optimize size, increase robustness, and minimize cost. The handheld CLYC multimode

MIDA: A Multimodal Imaging-Based Detailed Anatomical Model of the Human Head and Neck.

Directory of Open Access Journals (Sweden)

Maria Ida Iacono

Full Text Available Computational modeling and simulations are increasingly being used to complement experimental testing for analysis of safety and efficacy of medical devices. Multiple voxel- and surface-based whole- and partial-body models have been proposed in the literature, typically with spatial resolution in the range of 1-2 mm and with 10-50 different tissue types resolved. We have developed a multimodal imaging-based detailed anatomical model of the human head and neck, named "MIDA". The model was obtained by integrating three different magnetic resonance imaging (MRI modalities, the parameters of which were tailored to enhance the signals of specific tissues: i structural T1- and T2-weighted MRIs; a specific heavily T2-weighted MRI slab with high nerve contrast optimized to enhance the structures of the ear and eye; ii magnetic resonance angiography (MRA data to image the vasculature, and iii diffusion tensor imaging (DTI to obtain information on anisotropy and fiber orientation. The unique multimodal high-resolution approach allowed resolving 153 structures, including several distinct muscles, bones and skull layers, arteries and veins, nerves, as well as salivary glands. The model offers also a detailed characterization of eyes, ears, and deep brain structures. A special automatic atlas-based segmentation procedure was adopted to include a detailed map of the nuclei of the thalamus and midbrain into the head model. The suitability of the model to simulations involving different numerical methods, discretization approaches, as well as DTI-based tensorial electrical conductivity, was examined in a case-study, in which the electric field was generated by transcranial alternating current stimulation. The voxel- and the surface-based versions of the models are freely available to the scientific community.

Efficient Variational Approaches for Deformable Registration of Images

Directory of Open Access Journals (Sweden)

Mehmet Ali Akinlar

2012-01-01

Full Text Available Dirichlet, anisotropic, and Huber regularization terms are presented for efficient registration of deformable images. Image registration, an ill-posed optimization problem, is solved using a gradient-descent-based method and some fundamental theorems in calculus of variations. Euler-Lagrange equations with homogeneous Neumann boundary conditions are obtained. These equations are discretized by multigrid and finite difference numerical techniques. The method is applied to the registration of brain MR images of size 65×65. Computational results indicate that the presented method is quite fast and efficient in the registration of deformable medical images.

Registration-based Bone Morphometry for Shape Analysis of the Bones of the Human Wrist

Science.gov (United States)

Joshi, Anand A.; Leahy, Richard M.; Badawi, Ramsey D.; Chaudhari, Abhijit J.

2015-01-01

We present a method that quantifies point-wise changes in surface morphology of the bones of the human wrist. The proposed method, referred to as Registration-based Bone Morphometry (RBM), consists of two steps: an atlas selection step and an atlas warping step. The atlas for individual wrist bones was selected based on the shortest l2 distance to the ensemble of wrist bones from a database of a healthy population of subjects. The selected atlas was then warped to the corresponding bones of individuals in the population using a non-linear registration method based on regularized l2 distance minimization. The displacement field thus calculated showed local differences in bone shape that then were used for the analysis of group differences. Our results indicate that RBM has potential to provide a standardized approach to shape analysis of bones of the human wrist. We demonstrate the performance of RBM for examining group differences in wrist bone shapes based on sex and between those of the right and left wrists in healthy individuals. We also present data to show the application of RBM for tracking bone erosion status in rheumatoid arthritis. PMID:26353369

Locally orderless registration

DEFF Research Database (Denmark)

Darkner, Sune; Sporring, Jon

2013-01-01

This paper presents a unifying approach for calculating a wide range of popular, but seemingly very different, similarity measures. Our domain is the registration of n-dimensional images sampled on a regular grid, and our approach is well suited for gradient-based optimization algorithms. Our app...

Multimodality image registration. A special development in medical imaging has been strongly influenced by a small but highly qualified software think-tank

International Nuclear Information System (INIS)

Diemling, M.

2007-01-01

The importance of image fusion and registration in the field of medical diagnostics will be shown. After some details and background of image registration, as well as the history of nuclear medicine imaging - given by the example of HERMES Medical Solutions of Stockholm, Sweden - the reader finds seven cases illustrating the clinical importance of this method. These cases were collected from various fields of applications of medical imaging, they are carefully documented and illustrated. (orig.)

Accurate and robust brain image alignment using boundary-based registration.

Science.gov (United States)

Greve, Douglas N; Fischl, Bruce

2009-10-15

The fine spatial scales of the structures in the human brain represent an enormous challenge to the successful integration of information from different images for both within- and between-subject analysis. While many algorithms to register image pairs from the same subject exist, visual inspection shows that their accuracy and robustness to be suspect, particularly when there are strong intensity gradients and/or only part of the brain is imaged. This paper introduces a new algorithm called Boundary-Based Registration, or BBR. The novelty of BBR is that it treats the two images very differently. The reference image must be of sufficient resolution and quality to extract surfaces that separate tissue types. The input image is then aligned to the reference by maximizing the intensity gradient across tissue boundaries. Several lower quality images can be aligned through their alignment with the reference. Visual inspection and fMRI results show that BBR is more accurate than correlation ratio or normalized mutual information and is considerably more robust to even strong intensity inhomogeneities. BBR also excels at aligning partial-brain images to whole-brain images, a domain in which existing registration algorithms frequently fail. Even in the limit of registering a single slice, we show the BBR results to be robust and accurate.

Multimodal designs for learning in contexts of diversity

Directory of Open Access Journals (Sweden)

Arlene Archer

2014-12-01

Full Text Available This paper aims to identify multimodal designs for learning in diverse and developing contexts, where access to resources remains vastly unequal. Using case studies from South African education, the paper explores ways of surfacing the range of students’ resources which are often not noticed or valued in formal educational settings. The studies showcased here demonstrate how ethnographic and textually-based approaches can be combined. Opening up the semiotic space of the classroom through multimodal designs for learning is important for finding innovative ways of addressing access, diversity, and past inequalities. This is of relevance not only to South Africa, but a range of global contexts. The paper argues that multimodal designs for learning can involve interrogating the relation between ‘tradition’ and ‘modernity’; harnessing students’ creative practices as resources for pedagogy; developing metalanguages for critical reflection; creating less regulated pedagogical spaces in order to enable useful teaching and learning practices.

Avoiding Stair-Step Artifacts in Image Registration for GOES-R Navigation and Registration Assessment

Science.gov (United States)

Grycewicz, Thomas J.; Tan, Bin; Isaacson, Peter J.; De Luccia, Frank J.; Dellomo, John

2016-01-01

In developing software for independent verification and validation (IVV) of the Image Navigation and Registration (INR) capability for the Geostationary Operational Environmental Satellite R Series (GOES-R) Advanced Baseline Imager (ABI), we have encountered an image registration artifact which limits the accuracy of image offset estimation at the subpixel scale using image correlation. Where the two images to be registered have the same pixel size, subpixel image registration preferentially selects registration values where the image pixel boundaries are close to lined up. Because of the shape of a curve plotting input displacement to estimated offset, we call this a stair-step artifact. When one image is at a higher resolution than the other, the stair-step artifact is minimized by correlating at the higher resolution. For validating ABI image navigation, GOES-R images are correlated with Landsat-based ground truth maps. To create the ground truth map, the Landsat image is first transformed to the perspective seen from the GOES-R satellite, and then is scaled to an appropriate pixel size. Minimizing processing time motivates choosing the map pixels to be the same size as the GOES-R pixels. At this pixel size image processing of the shift estimate is efficient, but the stair-step artifact is present. If the map pixel is very small, stair-step is not a problem, but image correlation is computation-intensive. This paper describes simulation-based selection of the scale for truth maps for registering GOES-R ABI images.

The Stability of Multi-modal Traffic Network

International Nuclear Information System (INIS)

Han Linghui; Sun Huijun; Zhu Chengjuan; Jia Bin; Wu Jianjun

2013-01-01

There is an explicit and implicit assumption in multimodal traffic equilibrium models, that is, if the equilibrium exists, then it will also occur. The assumption is very idealized; in fact, it may be shown that the quite contrary could happen, because in multimodal traffic network, especially in mixed traffic conditions the interaction among traffic modes is asymmetric and the asymmetric interaction may result in the instability of traffic system. In this paper, to study the stability of multimodal traffic system, we respectively present the travel cost function in mixed traffic conditions and in traffic network with dedicated bus lanes. Based on a day-to-day dynamical model, we study the evolution of daily route choice of travelers in multimodal traffic network using 10000 random initial values for different cases. From the results of simulation, it can be concluded that the asymmetric interaction between the cars and buses in mixed traffic conditions can lead the traffic system to instability when traffic demand is larger. We also study the effect of travelers' perception error on the stability of multimodal traffic network. Although the larger perception error can alleviate the effect of interaction between cars and buses and improve the stability of traffic system in mixed traffic conditions, the traffic system also become instable when the traffic demand is larger than a number. For all cases simulated in this study, with the same parameters, traffic system with dedicated bus lane has better stability for traffic demand than that in mixed traffic conditions. We also find that the network with dedicated bus lane has higher portion of travelers by bus than it of mixed traffic network. So it can be concluded that building dedicated bus lane can improve the stability of traffic system and attract more travelers to choose bus reducing the traffic congestion. (general)

Quantifying brain development in early childhood using segmentation and registration

Science.gov (United States)

Aljabar, P.; Bhatia, K. K.; Murgasova, M.; Hajnal, J. V.; Boardman, J. P.; Srinivasan, L.; Rutherford, M. A.; Dyet, L. E.; Edwards, A. D.; Rueckert, D.

2007-03-01

In this work we obtain estimates of tissue growth using longitudinal data comprising MR brain images of 25 preterm children scanned at one and two years. The growth estimates are obtained using segmentation and registration based methods. The segmentation approach used an expectation maximisation (EM) method to classify tissue types and the registration approach used tensor based morphometry (TBM) applied to a free form deformation (FFD) model. The two methods show very good agreement indicating that the registration and segmentation approaches can be used interchangeably. The advantage of the registration based method, however, is that it can provide more local estimates of tissue growth. This is the first longitudinal study of growth in early childhood, previous longitudinal studies have focused on later periods during childhood.

Mass preserving image registration

DEFF Research Database (Denmark)

Gorbunova, Vladlena; Sporring, Jon; Lo, Pechin Chien Pau

2010-01-01

The paper presents results the mass preserving image registration method in the Evaluation of Methods for Pulmonary Image Registration 2010 (EMPIRE10) Challenge. The mass preserving image registration algorithm was applied to the 20 image pairs. Registration was evaluated using four different...

Multimodal exemplification: The expansion of meaning in electronic ...

African Journals Online (AJOL)

Functional Multimodal Discourse Analysis (SF-MDA) and argues for improving their exemplifica-tion multimodally. Multimodal devices, if well coordinated, can help optimize e-dictionary exam-ples in informativity, diversity, dynamicity and ...

Semiconductor laser using multimode interference principle

Science.gov (United States)

Gong, Zisu; Yin, Rui; Ji, Wei; Wu, Chonghao

2018-01-01

Multimode interference (MMI) structure is introduced in semiconductor laser used in optical communication system to realize higher power and better temperature tolerance. Using beam propagation method (BPM), Multimode interference laser diode (MMI-LD) is designed and fabricated in InGaAsP/InP based material. As a comparison, conventional semiconductor laser using straight single-mode waveguide is also fabricated in the same wafer. With a low injection current (about 230 mA), the output power of the implemented MMI-LD is up to 2.296 mW which is about four times higher than the output power of the conventional semiconductor laser. The implemented MMI-LD exhibits stable output operating at the wavelength of 1.52 μm and better temperature tolerance when the temperature varies from 283.15 K to 293.15 K.

Gaussian Process Interpolation for Uncertainty Estimation in Image Registration

Science.gov (United States)

Wachinger, Christian; Golland, Polina; Reuter, Martin; Wells, William

2014-01-01

Intensity-based image registration requires resampling images on a common grid to evaluate the similarity function. The uncertainty of interpolation varies across the image, depending on the location of resampled points relative to the base grid. We propose to perform Bayesian inference with Gaussian processes, where the covariance matrix of the Gaussian process posterior distribution estimates the uncertainty in interpolation. The Gaussian process replaces a single image with a distribution over images that we integrate into a generative model for registration. Marginalization over resampled images leads to a new similarity measure that includes the uncertainty of the interpolation. We demonstrate that our approach increases the registration accuracy and propose an efficient approximation scheme that enables seamless integration with existing registration methods. PMID:25333127

Evaluation of accuracy of B-spline transformation-based deformable image registration with different parameter settings for thoracic images.

Science.gov (United States)

Kanai, Takayuki; Kadoya, Noriyuki; Ito, Kengo; Onozato, Yusuke; Cho, Sang Yong; Kishi, Kazuma; Dobashi, Suguru; Umezawa, Rei; Matsushita, Haruo; Takeda, Ken; Jingu, Keiichi

2014-11-01

Deformable image registration (DIR) is fundamental technique for adaptive radiotherapy and image-guided radiotherapy. However, further improvement of DIR is still needed. We evaluated the accuracy of B-spline transformation-based DIR implemented in elastix. This registration package is largely based on the Insight Segmentation and Registration Toolkit (ITK), and several new functions were implemented to achieve high DIR accuracy. The purpose of this study was to clarify whether new functions implemented in elastix are useful for improving DIR accuracy. Thoracic 4D computed tomography images of ten patients with esophageal or lung cancer were studied. Datasets for these patients were provided by DIR-lab (dir-lab.com) and included a coordinate list of anatomical landmarks that had been manually identified. DIR between peak-inhale and peak-exhale images was performed with four types of parameter settings. The first one represents original ITK (Parameter 1). The second employs the new function of elastix (Parameter 2), and the third was created to verify whether new functions improve DIR accuracy while keeping computational time (Parameter 3). The last one partially employs a new function (Parameter 4). Registration errors for these parameter settings were calculated using the manually determined landmark pairs. 3D registration errors with standard deviation over all cases were 1.78 (1.57), 1.28 (1.10), 1.44 (1.09) and 1.36 (1.35) mm for Parameter 1, 2, 3 and 4, respectively, indicating that the new functions are useful for improving DIR accuracy, even while maintaining the computational time, and this B-spline-based DIR could be used clinically to achieve high-accuracy adaptive radiotherapy. © The Author 2014. Published by Oxford University Press on behalf of The Japan Radiation Research Society and Japanese Society for Radiation Oncology.

Skull registration for prone patient position using tracked ultrasound

Science.gov (United States)

Underwood, Grace; Ungi, Tamas; Baum, Zachary; Lasso, Andras; Kronreif, Gernot; Fichtinger, Gabor

2017-03-01

PURPOSE: Tracked navigation has become prevalent in neurosurgery. Problems with registration of a patient and a preoperative image arise when the patient is in a prone position. Surfaces accessible to optical tracking on the back of the head are unreliable for registration. We investigated the accuracy of surface-based registration using points accessible through tracked ultrasound. Using ultrasound allows access to bone surfaces that are not available through optical tracking. Tracked ultrasound could eliminate the need to work (i) under the table for registration and (ii) adjust the tracker between surgery and registration. In addition, tracked ultrasound could provide a non-invasive method in comparison to an alternative method of registration involving screw implantation. METHODS: A phantom study was performed to test the feasibility of tracked ultrasound for registration. An initial registration was performed to partially align the pre-operative computer tomography data and skull phantom. The initial registration was performed by an anatomical landmark registration. Surface points accessible by tracked ultrasound were collected and used to perform an Iterative Closest Point Algorithm. RESULTS: When the surface registration was compared to a ground truth landmark registration, the average TRE was found to be 1.6+/-0.1mm and the average distance of points off the skull surface was 0.6+/-0.1mm. CONCLUSION: The use of tracked ultrasound is feasible for registration of patients in prone position and eliminates the need to perform registration under the table. The translational component of error found was minimal. Therefore, the amount of TRE in registration is due to a rotational component of error.

Mode converter based on an inverse taper for multimode silicon nanophotonic integrated circuits.

Science.gov (United States)

Dai, Daoxin; Mao, Mao

2015-11-02

An inverse taper on silicon is proposed and designed to realize an efficient mode converter available for the connection between multimode silicon nanophotonic integrated circuits and few-mode fibers. The present mode converter has a silicon-on-insulator inverse taper buried in a 3 × 3μm(2) SiN strip waveguide to deal with not only for the fundamental mode but also for the higher-order modes. The designed inverse taper enables the conversion between the six modes (i.e., TE(11), TE(21), TE(31), TE(41), TM(11), TM(12)) in a 1.4 × 0.22μm(2) multimode SOI waveguide and the six modes (like the LP(01), LP(11a), LP(11b) modes in a few-mode fiber) in a 3 × 3μm(2) SiN strip waveguide. The conversion efficiency for any desired mode is higher than 95.6% while any undesired mode excitation ratio is lower than 0.5%. This is helpful to make multimode silicon nanophotonic integrated circuits (e.g., the on-chip mode (de)multiplexers developed well) available to work together with few-mode fibers in the future.

On developing B-spline registration algorithms for multi-core processors

International Nuclear Information System (INIS)

Shackleford, J A; Kandasamy, N; Sharp, G C

2010-01-01

Spline-based deformable registration methods are quite popular within the medical-imaging community due to their flexibility and robustness. However, they require a large amount of computing time to obtain adequate results. This paper makes two contributions towards accelerating B-spline-based registration. First, we propose a grid-alignment scheme and associated data structures that greatly reduce the complexity of the registration algorithm. Based on this grid-alignment scheme, we then develop highly data parallel designs for B-spline registration within the stream-processing model, suitable for implementation on multi-core processors such as graphics processing units (GPUs). Particular attention is focused on an optimal method for performing analytic gradient computations in a data parallel fashion. CPU and GPU versions are validated for execution time and registration quality. Performance results on large images show that our GPU algorithm achieves a speedup of 15 times over the single-threaded CPU implementation whereas our multi-core CPU algorithm achieves a speedup of 8 times over the single-threaded implementation. The CPU and GPU versions achieve near-identical registration quality in terms of RMS differences between the generated vector fields.

Implementing a Web-Based Registration and Administration System for Credit-by-Examination: Graduate Education Course Test Case.

Science.gov (United States)

Wang, Lih-Ching Chen

2002-01-01

Discusses the problems and successes encountered in implementing a Web-based registration and administration system for credit-by-examination in a required graduate course, detailing the ways in which this system improves upon its paper-based predecessor. (EV)

21 CFR 1301.36 - Suspension or revocation of registration; suspension of registration pending final order...

Science.gov (United States)

2010-04-01

... 21 Food and Drugs 9 2010-04-01 2010-04-01 false Suspension or revocation of registration; suspension of registration pending final order; extension of registration pending final order. 1301.36... registration pending final order; extension of registration pending final order. (a) For any registration...

21 CFR 710.8 - Misbranding by reference to registration or to registration number.

Science.gov (United States)

2010-04-01

... HUMAN SERVICES (CONTINUED) COSMETICS VOLUNTARY REGISTRATION OF COSMETIC PRODUCT ESTABLISHMENTS § 710.8 Misbranding by reference to registration or to registration number. Registration of a cosmetic product... products by the Food and Drug Administration. Any representation in labeling or advertising that creates an...

Design of 1x2 wavelength demultiplexer based on multimode interference

Czech Academy of Sciences Publication Activity Database

Prajzler, Václav; Nekvindová, P.; Varga, Marián; Kromka, Alexander; Remeš, Zdeněk

2014-01-01

Roč. 16, 11-12 (2014), s. 1226-1231 ISSN 1454-4164 R&D Projects: GA ČR(CZ) GA14-05053S Institutional support: RVO:68378271 Keywords : demultiplexer * multimode interference * nanocrystalline diamond Subject RIV: BH - Optics, Masers, Lasers Impact factor: 0.429, year: 2014

Estimation of lung motion fields in 4D CT data by variational non-linear intensity-based registration: A comparison and evaluation study

International Nuclear Information System (INIS)

Werner, René; Schmidt-Richberg, Alexander; Handels, Heinz; Ehrhardt, Jan

2014-01-01

Accurate and robust estimation of motion fields in respiration-correlated CT (4D CT) images, usually performed by non-linear registration of the temporal CT frames, is a precondition for the analysis of patient-specific breathing dynamics and subsequent image-supported diagnostics and treatment planning. In this work, we present a comprehensive comparison and evaluation study of non-linear registration variants applied to the task of lung motion estimation in thoracic 4D CT data. In contrast to existing multi-institutional comparison studies (e.g. MIDRAS and EMPIRE10), we focus on the specific but common class of variational intensity-based non-parametric registration and analyze the impact of the different main building blocks of the underlying optimization problem: the distance measure to be minimized, the regularization approach and the transformation space considered during optimization. In total, 90 different combinations of building block instances are compared. Evaluated on proprietary and publicly accessible 4D CT images, landmark-based registration errors (TRE) between 1.14 and 1.20 mm for the most accurate registration variants demonstrate competitive performance of the applied general registration framework compared to other state-of-the-art approaches for lung CT registration. Although some specific trends can be observed, effects of interchanging individual instances of the building blocks on the TRE are in general rather small (no single outstanding registration variant existing); the same level of accuracy is, however, associated with significantly different degrees of motion field smoothness and computational demands. Consequently, the building block combination of choice will depend on application-specific requirements on motion field characteristics. (paper)

Image registration based on virtual frame sequence analysis

Energy Technology Data Exchange (ETDEWEB)

Chen, H.; Ng, W.S. [Nanyang Technological University, Computer Integrated Medical Intervention Laboratory, School of Mechanical and Aerospace Engineering, Singapore (Singapore); Shi, D. (Nanyang Technological University, School of Computer Engineering, Singapore, Singpore); Wee, S.B. [Tan Tock Seng Hospital, Department of General Surgery, Singapore (Singapore)

2007-08-15

This paper is to propose a new framework for medical image registration with large nonrigid deformations, which still remains one of the biggest challenges for image fusion and further analysis in many medical applications. Registration problem is formulated as to recover a deformation process with the known initial state and final state. To deal with large nonlinear deformations, virtual frames are proposed to be inserted to model the deformation process. A time parameter is introduced and the deformation between consecutive frames is described with a linear affine transformation. Experiments are conducted with simple geometric deformation as well as complex deformations presented in MRI and ultrasound images. All the deformations are characterized with nonlinearity. The positive results demonstrated the effectiveness of this algorithm. The framework proposed in this paper is feasible to register medical images with large nonlinear deformations and is especially useful for sequential images. (orig.)

Numerical methods for image registration

CERN Document Server

Modersitzki, Jan

2003-01-01

Based on the author's lecture notes and research, this well-illustrated and comprehensive text is one of the first to provide an introduction to image registration with particular emphasis on numerical methods in medical imaging. Ideal for researchers in industry and academia, it is also a suitable study guide for graduate mathematicians, computer scientists, engineers, medical physicists, and radiologists.Image registration is utilised whenever information obtained from different viewpoints needs to be combined or compared and unwanted distortion needs to be eliminated. For example, CCTV imag

The First 500 Registrations to the Research Registry®: Advancing Registration of Under-registered Study Types

Directory of Open Access Journals (Sweden)

Riaz Agha

2016-09-01

Full Text Available The Declaration of Helsinki 2013 encourages the registration of all research studies involving human participants. However, emphasis has been placed on prospective clinical trials, and it is estimated that only 10% of observational studies are registered. In response, Research Registry® was launched in February 2015; a retrospectively curated registry that is free and easy to use. Research Registry® enables prospective or retrospective registration of studies, including those study types that cannot be registered on existing registries. In this study, we describe the first 500 registrations on Research Registry®.Since the launch of Research Registry® in February 2015, data of registrations have been collected, including type of studies registered, country of origin and data curation activity. Inappropriate registrations, such as duplicates, were identified by the data curation process. These were removed from the database or modified as required. A quality score was assigned for each registration, based on Bradford-Hill’s criteria on what research studies should convey. Changes in quality scores over time were assessed. 500 studies were registered on Research Registry® from February 2015 to October 2015, with a total of 1.7 million patients enrolled. The most common study types were retrospective cohort studies (37.2%, case series (14.8% and first-in-man case reports (10.4%. Registrations were received from 57 different countries; the most submissions were received from Turkey, followed by China and the United Kingdom. Retrospective data curation identified 80 studies that were initially registered as the incorrect study type, and were subsequently correct. The Kruskal-Wallis test identified a significant improvement in quality scores for registrations from February 2015 to October 2015 (p < 0.0001.Since its conception in February 2015, Research Registry® has established itself as a new registry that is free, easy to use and enables the

The First 500 Registrations to the Research Registry®: Advancing Registration of Under-Registered Study Types.

Science.gov (United States)

Agha, Riaz; Fowler, Alexander J; Limb, Christopher; Al Omran, Yasser; Sagoo, Harkiran; Koshy, Kiron; Jafree, Daniyal J; Anwar, Mohammed Omer; McCullogh, Peter; Orgill, Dennis Paul

2016-01-01

The Declaration of Helsinki 2013 encourages the registration of all research studies involving human participants. However, emphasis has been placed on prospective clinical trials, and it is estimated that only 10% of observational studies are registered. In response, Research Registry ® was launched in February 2015; a retrospectively curated registry that is free and easy to use. Research Registry ® enables prospective or retrospective registration of studies, including those study types that cannot be registered on existing registries. In this study, we describe the first 500 registrations on Research Registry ® . Since the launch of Research Registry ® in February 2015, data of registrations have been collected, including type of studies registered, country of origin, and data curation activity. Inappropriate registrations, such as duplicates, were identified by the data curation process. These were removed from the database or modified as required. A quality score was assigned for each registration, based on Sir Austin Bradford Hill's criteria on what research studies should convey. Changes in quality scores over time were assessed. A total of 500 studies were registered on Research Registry ® from February 2015 to October 2015, with a total of 1.7 million patients enrolled. The most common study types were retrospective cohort studies (37.2%), case series (14.8%), and first-in-man case reports (10.4%). Registrations were received from 57 different countries; the most submissions were received from Turkey, followed by China and the United Kingdom. Retrospective data curation identified 80 studies that were initially registered as the incorrect study type, and were subsequently correct. The Kruskal-Wallis test identified a significant improvement in quality scores for registrations from February 2015 to October 2015 ( p  < 0.0001). Since its conception in February 2015, Research Registry ® has established itself as a new registry that is free, easy to

EXPLICITATION AND ADDITION TECHNIQUES IN AUDIOVISUAL TRANSLATION: A MULTIMODAL APPROACH OF ENGLISHINDONESIAN SUBTITLES

Directory of Open Access Journals (Sweden)

Ichwan Suyudi

2017-12-01

Full Text Available In audiovisual translation, the multimodality of the audiovisual text is both a challenge and a resource for subtitlers. This paper illustrates how multi-modes provide information that helps subtitlers to gain a better understanding of meaning-making practices that will influence them to make a decision-making in translating a certain verbal text. Subtitlers may explicit, add, and condense the texts based on the multi-modes as seen on the visual frames. Subtitlers have to consider the distribution and integration of the meanings of multi-modes in order to create comprehensive equivalence between the source and target texts. Excerpts of visual frames in this paper are taken from English films Forrest Gump (drama, 1996, and James Bond (thriller, 2010.

M3BA: A Mobile, Modular, Multimodal Biosignal Acquisition Architecture for Miniaturized EEG-NIRS-Based Hybrid BCI and Monitoring.

Science.gov (United States)

von Luhmann, Alexander; Wabnitz, Heidrun; Sander, Tilmann; Muller, Klaus-Robert

2017-06-01

For the further development of the fields of telemedicine, neurotechnology, and brain-computer interfaces, advances in hybrid multimodal signal acquisition and processing technology are invaluable. Currently, there are no commonly available hybrid devices combining bioelectrical and biooptical neurophysiological measurements [here electroencephalography (EEG) and functional near-infrared spectroscopy (NIRS)]. Our objective was to design such an instrument in a miniaturized, customizable, and wireless form. We present here the design and evaluation of a mobile, modular, multimodal biosignal acquisition architecture (M3BA) based on a high-performance analog front-end optimized for biopotential acquisition, a microcontroller, and our openNIRS technology. The designed M3BA modules are very small configurable high-precision and low-noise modules (EEG input referred noise @ 500 SPS 1.39 μV pp , NIRS noise equivalent power NEP 750 nm = 5.92 pW pp , and NEP 850 nm = 4.77 pW pp ) with full input linearity, Bluetooth, 3-D accelerometer, and low power consumption. They support flexible user-specified biopotential reference setups and wireless body area/sensor network scenarios. Performance characterization and in-vivo experiments confirmed functionality and quality of the designed architecture. Telemedicine and assistive neurotechnology scenarios will increasingly include wearable multimodal sensors in the future. The M3BA architecture can significantly facilitate future designs for research in these and other fields that rely on customized mobile hybrid biosignal modal biosignal acquisition architecture (M3BA), multimodal, near-infrared spectroscopy (NIRS), wireless body area network (WBAN), wireless body sensor network (WBSN).

Gestures and multimodal input

OpenAIRE

Keates, Simeon; Robinson, Peter

1999-01-01

For users with motion impairments, the standard keyboard and mouse arrangement for computer access often presents problems. Other approaches have to be adopted to overcome this. In this paper, we will describe the development of a prototype multimodal input system based on two gestural input channels. Results from extensive user trials of this system are presented. These trials showed that the physical and cognitive loads on the user can quickly become excessive and detrimental to the interac...

A new robust markerless method for automatic image-to-patient registration in image-guided neurosurgery system.

Science.gov (United States)

Liu, Yinlong; Song, Zhijian; Wang, Manning

2017-12-01

Compared with the traditional point-based registration in the image-guided neurosurgery system, surface-based registration is preferable because it does not use fiducial markers before image scanning and does not require image acquisition dedicated for navigation purposes. However, most existing surface-based registration methods must include a manual step for coarse registration, which increases the registration time and elicits some inconvenience and uncertainty. A new automatic surface-based registration method is proposed, which applies 3D surface feature description and matching algorithm to obtain point correspondences for coarse registration and uses the iterative closest point (ICP) algorithm in the last step to obtain an image-to-patient registration. Both phantom and clinical data were used to execute automatic registrations and target registration error (TRE) calculated to verify the practicality and robustness of the proposed method. In phantom experiments, the registration accuracy was stable across different downsampling resolutions (18-26 mm) and different support radii (2-6 mm). In clinical experiments, the mean TREs of two patients by registering full head surfaces were 1.30 mm and 1.85 mm. This study introduced a new robust automatic surface-based registration method based on 3D feature matching. The method achieved sufficient registration accuracy with different real-world surface regions in phantom and clinical experiments.

Drug-related webpages classification based on multi-modal local decision fusion

Science.gov (United States)

Hu, Ruiguang; Su, Xiaojing; Liu, Yanxin

2018-03-01

In this paper, multi-modal local decision fusion is used for drug-related webpages classification. First, meaningful text are extracted through HTML parsing, and effective images are chosen by the FOCARSS algorithm. Second, six SVM classifiers are trained for six kinds of drug-taking instruments, which are represented by PHOG. One SVM classifier is trained for the cannabis, which is represented by the mid-feature of BOW model. For each instance in a webpage, seven SVMs give seven labels for its image, and other seven labels are given by searching the names of drug-taking instruments and cannabis in its related text. Concatenating seven labels of image and seven labels of text, the representation of those instances in webpages are generated. Last, Multi-Instance Learning is used to classify those drugrelated webpages. Experimental results demonstrate that the classification accuracy of multi-instance learning with multi-modal local decision fusion is much higher than those of single-modal classification.

Surface membrane based bladder registration for evaluation of accumulated dose during brachytherapy in cervical cancer

DEFF Research Database (Denmark)

Noe, Karsten Østergaard; Tanderup, Kari; Sørensen, Thomas Sangild

2011-01-01

of the fixed surface. Optional landmark based matches can be included in the suggested iterative solver. The technique is demonstrated for bladder registration in brachytherapy treatment evaluation of cervical cancer. It holds promise to better estimate the accumulated but unintentional dose delivered...

Background suppression of infrared small target image based on inter-frame registration

Science.gov (United States)