WorldWideScience

Sample records for reliable assessment instruments

  1. A Reliability and Validity of an Instrument to Evaluate the School-Based Assessment System: A Pilot Study

    Science.gov (United States)

    Ghazali, Nor Hasnida Md

    2016-01-01

    A valid, reliable and practical instrument is needed to evaluate the implementation of the school-based assessment (SBA) system. The aim of this study is to develop and assess the validity and reliability of an instrument to measure the perception of teachers towards the SBA implementation in schools. The instrument is developed based on a…

  2. Automatic creation of Markov models for reliability assessment of safety instrumented systems

    International Nuclear Information System (INIS)

    Guo Haitao; Yang Xianhui

    2008-01-01

    After the release of new international functional safety standards like IEC 61508, people care more for the safety and availability of safety instrumented systems. Markov analysis is a powerful and flexible technique to assess the reliability measurements of safety instrumented systems, but it is fallible and time-consuming to create Markov models manually. This paper presents a new technique to automatically create Markov models for reliability assessment of safety instrumented systems. Many safety related factors, such as failure modes, self-diagnostic, restorations, common cause and voting, are included in Markov models. A framework is generated first based on voting, failure modes and self-diagnostic. Then, repairs and common-cause failures are incorporated into the framework to build a complete Markov model. Eventual simplification of Markov models can be done by state merging. Examples given in this paper show how explosively the size of Markov model increases as the system becomes a little more complicated as well as the advancement of automatic creation of Markov models

  3. Reliability assessment of a peer evaluation instrument in a team-based learning course

    Directory of Open Access Journals (Sweden)

    Wahawisan J

    2016-03-01

    Full Text Available Objective: To evaluate the reliability of a peer evaluation instrument in a longitudinal team-based learning setting. Methods: Student pharmacists were instructed to evaluate the contributions of their peers. Evaluations were analyzed for the variance of the scores by identifying low, medium, and high scores. Agreement between performance ratings within each group of students was assessed via intra-class correlation coefficient (ICC. Results: We found little variation in the standard deviation (SD based on the score means among the high, medium, and low scores within each group. The lack of variation in SD of results between groups suggests that the peer evaluation instrument produces precise results. The ICC showed strong concordance among raters. Conclusions: Findings suggest that our student peer evaluation instrument provides a reliable method for peer assessment in team-based learning settings.

  4. Satisfactory reliability among nursing students using the instrument PVC ASSESS to evaluate management of peripheral venous catheters.

    Science.gov (United States)

    Ahlqvist, Margary; Berglund, Britta; Nordström, Gun; Klang, Birgitta; Johansson, Eva

    2014-01-01

    Nursing students should be given opportunities to participate in clinical audits during their education. However, audit tools are seldom tested for reliability among nursing students. The aim of this study was to present reliability among nursing students using the instrument PVC assess to assess management of peripheral venous catheters (PVCs) and PVC-related signs of thrombophlebitis. PVC assess was used to assess 67 inserted PVCs in 60 patients at ten wards at a university hospital. One group of nursing students (n=4) assessed PVCs at the bedside (inter-rater reliability) and photographs of these PVCs were taken. Another group of students (n=3) assessed the PVCs in the photographs after 4 weeks (test-retest reliability). To determine reliability, proportion of agreement [P(A)] and Cohen's kappa coefficient (κ) were calculated. For bedside assessment of PVCs, P(A) ranged from good to excellent (0.80-1.0) in 55% of the 26 PVC assess items that were tested. P(A) was poor (satisfactory reliability among nursing students. However, students need training in how to use the instrument before assessing PVCs.

  5. Validity and reliability of a dental operator posture assessment instrument (PAI).

    Science.gov (United States)

    Branson, Bonnie G; Williams, Karen B; Bray, Kimberly Krust; Mcllnay, Sandy L; Dickey, Diana

    2002-01-01

    Basic operating posture is considered an important occupational health issue for oral health care clinicians. It is generally agreed that the physical posture of the operator, while providing care, should be such that all muscles are in a relaxed, well-balanced, and neutral position. Postures outside of this neutral position are likely to cause musculoskeletal discomfort. To date, the range of the neutral operator position has not been well-defined; nor have any specific instruments been identified that can quantitatively or semi-quantitatively assess dental operator posture. This paper reports on the development of an instrument that can be used to semi-quantitatively evaluate postural components. During the first phase of the study, an expert panel defined the basic parameters for acceptable, compromised, and harmful operator postures and established face validity of a posture assessment instrument (PAI). During the second phase, the PAI was tested for reliability using generalizability theory. Four raters tested the instrument for reliability. Overall, total PAI scores were similar amongst three of the raters, with the fourth rater's scores being slightly greater than the other three. The main effect of the rater on individual postural components was moderate, indicating that rater variance contributed to 11.9% of total variance. The PAI measures posture as it occurs and will have numerous applications when evaluating operator performance in the dental and dental hygiene education setting. Also, the PAI will prove useful when examining the effects of operator posture and musculoskeletal disorders.

  6. Validity and reliability of the Turkish version of the pressure ulcer prevention knowledge assessment instrument.

    Science.gov (United States)

    Tulek, Zeliha; Polat, Cansu; Ozkan, Ilknur; Theofanidis, Dimitris; Togrol, Rifat Erdem

    2016-11-01

    Sound knowledge of pressure ulcers is important to enable good prevention. There are limited instruments assessing pressure ulcer knowledge. The Pressure Ulcer Prevention Knowledge Assessment Instrument is among the scales of which psychometric properties have been studied rigorously and reflects the latest evidence. This study aimed to evaluate the validity and reliability of the Turkish version of the Pressure Ulcer Prevention Knowledge Assessment Instrument (PUPKAI-T), an instrument that assesses knowledge of pressure ulcer prevention by using multiple-choice questions. Linguistic validity was verified through front-to-back translation. Psychometric properties of the instrument were studied on a sample of 150 nurses working in a tertiary hospital in Istanbul, Turkey. The content validity index of the translated instrument was 0.94, intra-class correlation coefficients were between 0.37 and 0.80, item difficulty indices were between 0.21 and 0.88, discrimination indices were 0.20-0.78, and the Kuder Richardson for the internal consistency was 0.803. The PUPKAI-T was found to be a valid and reliable tool to evaluate nurses' knowledge on pressure ulcer prevention. The PUPKAI-T may be a useful tool for determining educational needs of nurses on pressure ulcer prevention. Copyright © 2016 Tissue Viability Society. Published by Elsevier Ltd. All rights reserved.

  7. Diagnosing paratonia in the demented elderly: reliability and validity of the Paratonia Assessment Instrument (PAI).

    Science.gov (United States)

    Hobbelen, Johannes S M; Koopmans, Raymond T C M; Verhey, Frans R J; Habraken, Kitty M; de Bie, Rob A

    2008-08-01

    Paratonia is one of the associated movement disorders characteristic of dementia. The aim of this study was to develop an assessment tool (the Paratonia Assessment Instrument, PAI), based on the new consensus definition of paratonia. An additional aim was to investigate the reliability and validity of the PAI. A three-phase cross-sectional survey was conducted. In the first two phases, the PAI was developed and validated. In the third phase, the inter-observer reliability and feasibility of the instrument was tested. The original PAI consisted of five criteria that all needed to be met in order to make the diagnosis. On the basis of a qualitative analysis, one criterion was reformulated and another was removed. Following this, inter-observer reliability between the two assessors resulted in an improvement of Cohen's kappa from 0.532 in the initial phase to 0.677 in the second phase. This improvement was substantiated in the third phase by two independent assessors with Cohen's kappa ranging from 0.625 to 1. The PAI is a reliable and valid assessment tool for diagnosing paratonia in elderly people with dementia that can be applied easily in daily practice.

  8. Diagnosing paratonia in the demented elderly : reliability and validity of the Paratonia Assessment Instrument (PAI)

    NARCIS (Netherlands)

    Hobbelen, Hans; Koopmans, Raymond T C M; Verhey, Frans R J; Habraken, Kitty M; de Bie, Rob A.

    BACKGROUND: Paratonia is one of the associated movement disorders characteristic of dementia. The aim of this study was to develop an assessment tool (the Paratonia Assessment Instrument, PAI), based on the new consensus definition of paratonia. An additional aim was to investigate the reliability

  9. Diagnosing paratonia in the demented elderly: reliability and validity of the Paratonia Assessment Instrument (PAI).

    NARCIS (Netherlands)

    Hobbelen, J.S.; Koopmans, R.T.C.M.; Verhey, F.R.J.; Habraken, K.M.; Bie, R.A. de

    2008-01-01

    BACKGROUND: Paratonia is one of the associated movement disorders characteristic of dementia. The aim of this study was to develop an assessment tool (the Paratonia Assessment Instrument, PAI), based on the new consensus definition of paratonia. An additional aim was to investigate the reliability

  10. Translation, cross-cultural adaptation to Brazilian- Portuguese and reliability analysis of the instrument Rapid Entire Body Assessment-REBA

    Directory of Open Access Journals (Sweden)

    Andressa M. Lamarão

    2014-06-01

    Full Text Available Background: Observational instruments, such as the Rapid Entire Body Assessment, quickly assess biomechanical risks present in the workplace. However, in order to use these instruments, it is necessary to conduct the translational/cross-cultural adaptation of the instrument and test its measurement properties. Objectives: To perform the translation and the cross-cultural adaptation to Brazilian-Portuguese and test the reliability of the REBA instrument. Method: The procedures of translation and cross-cultural adaptation to Brazilian-Portuguese were conducted following proposed guidelines that involved translation, synthesis of translations, back translation, committee review and testing of the pre-final version. In addition, reliability and the intra- and inter-rater percent agreement were obtained with the Linear Weighted Kappa Coefficient that was associated with the 95% Confidence Interval and the cross tabulation 2×2. Results : The procedures for translation and adaptation were adequate and the necessary adjustments were conducted on the instrument. The intra- and inter-rater reliability showed values of 0.104 to 0.504, respectively, ranging from very poor to moderate. The percentage agreement values ranged from 5.66% to 69.81%. The percentage agreement was closer to 100% at the item 'upper arm' (69.81% for the Intra-rater 1 and at the items 'legs' and 'upper arm' for the Intra-rater 2 (62.26%. Conclusions: The processes of translation and cross-cultural adaptation were conducted on the REBA instrument and the Brazilian version of the instrument was obtained. However, despite the reliability of the tests used to correct the translated and adapted version, the reliability values are unacceptable according to the guidelines standard, indicating that the reliability must be re-evaluated. Therefore, caution in the interpretation of the biomechanical risks measured by this instrument should be taken.

  11. Translation, cross-cultural adaptation to Brazilian- Portuguese and reliability analysis of the instrument Rapid Entire Body Assessment-REBA

    Science.gov (United States)

    Lamarão, Andressa M.; Costa, Lucíola C. M.; Comper, Maria L. C.; Padula, Rosimeire S.

    2014-01-01

    Background: Observational instruments, such as the Rapid Entire Body Assessment, quickly assess biomechanical risks present in the workplace. However, in order to use these instruments, it is necessary to conduct the translational/cross-cultural adaptation of the instrument and test its measurement properties. Objectives: To perform the translation and the cross-cultural adaptation to Brazilian-Portuguese and test the reliability of the REBA instrument. Method: The procedures of translation and cross-cultural adaptation to Brazilian-Portuguese were conducted following proposed guidelines that involved translation, synthesis of translations, back translation, committee review and testing of the pre-final version. In addition, reliability and the intra- and inter-rater percent agreement were obtained with the Linear Weighted Kappa Coefficient that was associated with the 95% Confidence Interval and the cross tabulation 2×2. Results : The procedures for translation and adaptation were adequate and the necessary adjustments were conducted on the instrument. The intra- and inter-rater reliability showed values of 0.104 to 0.504, respectively, ranging from very poor to moderate. The percentage agreement values ranged from 5.66% to 69.81%. The percentage agreement was closer to 100% at the item 'upper arm' (69.81%) for the Intra-rater 1 and at the items 'legs' and 'upper arm' for the Intra-rater 2 (62.26%). Conclusions: The processes of translation and cross-cultural adaptation were conducted on the REBA instrument and the Brazilian version of the instrument was obtained. However, despite the reliability of the tests used to correct the translated and adapted version, the reliability values are unacceptable according to the guidelines standard, indicating that the reliability must be re-evaluated. Therefore, caution in the interpretation of the biomechanical risks measured by this instrument should be taken. PMID:25003273

  12. The contribution of instrumentation and control software to system reliability

    International Nuclear Information System (INIS)

    Fryer, M.O.

    1984-01-01

    Advanced instrumentation and control systems are usually implemented using computers that monitor the instrumentation and issue commands to control elements. The control commands are based on instrument readings and software control logic. The reliability of the total system will be affected by the software design. When comparing software designs, an evaluation of how each design can contribute to the reliability of the system is desirable. Unfortunately, the science of reliability assessment of combined hardware and software systems is in its infancy. Reliability assessment of combined hardware/software systems is often based on over-simplified assumptions about software behavior. A new method of reliability assessment of combined software/hardware systems is presented. The method is based on a procedure called fault tree analysis which determines how component failures can contribute to system failure. Fault tree analysis is a well developed method for reliability assessment of hardware systems and produces quantitative estimates of failure probability based on component failure rates. It is shown how software control logic can be mapped into a fault tree that depicts both software and hardware contributions to system failure. The new method is important because it provides a way for quantitatively evaluating the reliability contribution of software designs. In many applications, this can help guide designers in producing safer and more reliable systems. An application to the nuclear power research industry is discussed

  13. Instrument reliability for high-level nuclear-waste-repository applications

    International Nuclear Information System (INIS)

    Rogue, F.; Binnall, E.P.; Armantrout, G.A.

    1983-01-01

    Reliable instrumentation will be needed to evaluate the characteristics of proposed high-level nuclear-wasted-repository sites and to monitor the performance of selected sites during the operational period and into repository closure. A study has been done to assess the reliability of instruments used in Department of Energy (DOE) waste repository related experiments and in other similar geological applications. The study included experiences with geotechnical, hydrological, geochemical, environmental, and radiological instrumentation and associated data acquisition equipment. Though this paper includes some findings on the reliability of instruments in each of these categories, the emphasis is on experiences with geotechnical instrumentation in hostile repository-type environments. We review the failure modes, rates, and mechanisms, along with manufacturers modifications and design changes to enhance and improve instrument performance; and include recommendations on areas where further improvements are needed

  14. Face validity and reliability of a pictorial instrument for assessing fundamental movement skill perceived competence in young children.

    Science.gov (United States)

    Barnett, Lisa M; Ridgers, Nicola D; Zask, Avigdor; Salmon, Jo

    2015-01-01

    To determine reliability and face validity of an instrument to assess young children's perceived fundamental movement skill competence. Validation and reliability study. A pictorial instrument based on the Test Gross Motor Development-2 assessed perceived locomotor (six skills) and object control (six skills) competence using the format and item structure from the physical competence subscale of the Pictorial Scale of Perceived Competence and Acceptance for Young Children. Sample 1 completed object control items in May (n=32) and locomotor items in October 2012 (n=23) at two time points seven days apart. Children were asked at the end of the test-retest their understanding of what was happening in each picture to determine face validity. Sample 2 (n=58) completed 12 items in November 2012 on a single occasion to test internal reliability only. Sample 1 children were aged 5-7 years (M=6.0, SD=0.8) at object control assessment and 5-8 years at locomotor assessment (M=6.5, SD=0.9). Sample 2 children were aged 6-8 years (M=7.2, SD=0.73). Intra-class correlations assessed in Sample 1 children were excellent for object control (intra-class correlation=0.78), locomotor (intra-class correlation=0.82) and all 12 skills (intra-class correlations=0.83). Face validity was acceptable. Internal consistency was adequate in both samples for each subscale and all 12 skills (alpha range 0.60-0.81). This study has provided preliminary evidence for instrument reliability and face validity. This enables future alignment between the measurement of perceived and actual fundamental movement skill competence in young children. Crown Copyright © 2014. Published by Elsevier Ltd. All rights reserved.

  15. Reliability of Instruments Measuring At-Risk and Problem Gambling Among Young Individuals

    DEFF Research Database (Denmark)

    Edgren, Robert; Castrén, Sari; Mäkelä, Marjukka

    2016-01-01

    This review aims to clarify which instruments measuring at-risk and problem gambling (ARPG) among youth are reliable and valid in light of reported estimates of internal consistency, classification accuracy, and psychometric properties. A systematic search was conducted in PubMed, Medline, and Psyc......Info covering the years 2009–2015. In total, 50 original research articles fulfilled the inclusion criteria: target age under 29 years, using an instrument designed for youth, and reporting a reliability estimate. Articles were evaluated with the revised Quality Assessment of Diagnostic Accuracy Studies tool....... Reliability estimates were reported for five ARPG instruments. Most studies (66%) evaluated the South Oaks Gambling Screen Revised for Adolescents. The Gambling Addictive Behavior Scale for Adolescents was the only novel instrument. In general, the evaluation of instrument reliability was superficial. Despite...

  16. Validity and reliability of an instrument for assessing case analyses in bioengineering ethics education.

    Science.gov (United States)

    Goldin, Ilya M; Pinkus, Rosa Lynn; Ashley, Kevin

    2015-06-01

    Assessment in ethics education faces a challenge. From the perspectives of teachers, students, and third-party evaluators like the Accreditation Board for Engineering and Technology and the National Institutes of Health, assessment of student performance is essential. Because of the complexity of ethical case analysis, however, it is difficult to formulate assessment criteria, and to recognize when students fulfill them. Improvement in students' moral reasoning skills can serve as the focus of assessment. In previous work, Rosa Lynn Pinkus and Claire Gloeckner developed a novel instrument for assessing moral reasoning skills in bioengineering ethics. In this paper, we compare that approach to existing assessment techniques, and evaluate its validity and reliability. We find that it is sensitive to knowledge gain and that independent coders agree on how to apply it.

  17. Validity evidence and reliability of a simulated patient feedback instrument.

    Science.gov (United States)

    Schlegel, Claudia; Woermann, Ulrich; Rethans, Jan-Joost; van der Vleuten, Cees

    2012-01-27

    In the training of healthcare professionals, one of the advantages of communication training with simulated patients (SPs) is the SP's ability to provide direct feedback to students after a simulated clinical encounter. The quality of SP feedback must be monitored, especially because it is well known that feedback can have a profound effect on student performance. Due to the current lack of valid and reliable instruments to assess the quality of SP feedback, our study examined the validity and reliability of one potential instrument, the 'modified Quality of Simulated Patient Feedback Form' (mQSF). Content validity of the mQSF was assessed by inviting experts in the area of simulated clinical encounters to rate the importance of the mQSF items. Moreover, generalizability theory was used to examine the reliability of the mQSF. Our data came from videotapes of clinical encounters between six simulated patients and six students and the ensuing feedback from the SPs to the students. Ten faculty members judged the SP feedback according to the items on the mQSF. Three weeks later, this procedure was repeated with the same faculty members and recordings. All but two items of the mQSF received importance ratings of > 2.5 on a four-point rating scale. A generalizability coefficient of 0.77 was established with two judges observing one encounter. The findings for content validity and reliability with two judges suggest that the mQSF is a valid and reliable instrument to assess the quality of feedback provided by simulated patients.

  18. Reliability of visual and instrumental color matching.

    Science.gov (United States)

    Igiel, Christopher; Lehmann, Karl Martin; Ghinea, Razvan; Weyhrauch, Michael; Hangx, Ysbrand; Scheller, Herbert; Paravina, Rade D

    2017-09-01

    The aim of this investigation was to evaluate intra-rater and inter-rater reliability of visual and instrumental shade matching. Forty individuals with normal color perception participated in this study. The right maxillary central incisor of a teaching model was prepared and restored with 10 feldspathic all-ceramic crowns of different shades. A shade matching session consisted of the observer (rater) visually selecting the best match by using VITA classical A1-D4 (VC) and VITA Toothguide 3D Master (3D) shade guides and the VITA Easyshade Advance intraoral spectrophotometer (ES) to obtain both VC and 3D matches. Three shade matching sessions were held with 4 to 6 weeks between sessions. Intra-rater reliability was assessed based on the percentage of agreement for the three sessions for the same observer, whereas the inter-rater reliability was calculated as mean percentage of agreement between different observers. The Fleiss' Kappa statistical analysis was used to evaluate visual inter-rater reliability. The mean intra-rater reliability for the visual shade selection was 64(11) for VC and 48(10) for 3D. The corresponding ES values were 96(4) for both VC and 3D. The percentages of observers who matched the same shade with VC and 3D were 55(10) and 43(12), respectively, while corresponding ES values were 88(8) for VC and 92(4) for 3D. The results for visual shade matching exhibited a high to moderate level of inconsistency for both intra-rater and inter-rater comparisons. The VITA Easyshade Advance intraoral spectrophotometer exhibited significantly better reliability compared with visual shade selection. This study evaluates the ability of observers to consistently match the same shade visually and with a dental spectrophotometer in different sessions. The intra-rater and inter-rater reliability (agreement of repeated shade matching) of visual and instrumental tooth color matching strongly suggest the use of color matching instruments as a supplementary tool in

  19. Development of assessment instruments to measure critical thinking skills

    Science.gov (United States)

    Sumarni, W.; Supardi, K. I.; Widiarti, N.

    2018-04-01

    Assessment instruments that is commonly used in the school generally have not been orientated on critical thinking skills. The purpose of this research is to develop assessment instruments to measure critical thinking skills, to test validity, reliability, and practicality. This type of research is Research and Development. There are two stages on the preface step, which are field study and literacy study. On the development steps, there some parts, which are 1) instrument construction, 2) expert validity, 3) limited scale tryout and 4) narrow scale try-out. The developed assessment instrument are analysis essay and problem solving. Instruments were declared valid, reliable and practical.

  20. Instrument for assessing mobile technology acceptability in diabetes self-management: a validation and reliability study.

    Science.gov (United States)

    Frandes, Mirela; Deiac, Anca V; Timar, Bogdan; Lungeanu, Diana

    2017-01-01

    Nowadays, mobile technologies are part of everyday life, but the lack of instruments to assess their acceptability for the management of chronic diseases makes their actual adoption for this purpose slow. The objective of this study was to develop a survey instrument for assessing patients' attitude toward and intention to use mobile technology for diabetes mellitus (DM) self-management, as well as to identify sociodemographic characteristics and quality of life factors that affect them. We first conducted the documentation and instrument design phases, which were subsequently followed by the pilot study and instrument validation. Afterward, the instrument was administered 103 patients (median age: 37 years; range: 18-65 years) diagnosed with type 1 or type 2 DM, who accepted to participate in the study. The reliability and construct validity were assessed by computing Cronbach's alpha and using factor analysis, respectively. The instrument included statements about the actual use of electronic devices for DM management, interaction between patient and physician, attitude toward using mobile technology, and quality of life evaluation. Cronbach's alpha was 0.9 for attitude toward using mobile technology and 0.97 for attitude toward using mobile device applications for DM self-management. Younger patients (Spearman's ρ =-0.429; P higher education level (Kendall's τ =0.51; P mobile assistive applications for DM control. Moreover, patients with a higher quality of life presented a significantly more positive attitude toward using modern technology (Spearman's ρ =0.466; P mobile technology for DM self-management. Additionally, we found that even if most of the patients showed positive attitude toward mobile applications, only a moderate level of intention to indeed use them was observed. Moreover, the study indicated that barriers were truthfulness and easiness to use.

  1. Antisocial personality disorder and psychopathy in women: a literature review on the reliability and validity of assessment instruments.

    Science.gov (United States)

    Dolan, Mairead; Völlm, Birgit

    2009-01-01

    Crime rates are low in women compared to men. The two disorders most commonly associated with offending behaviour, antisocial personality disorder (ASPD) and psychopathy, are also less prevalent in female samples. However, developments in forensic psychiatry have often ignored gender, and the utility of constructs such as psychopathy and their assessment instruments in female samples remains unclear. This article presents a review of studies looking at rates of ASPD and psychopathy and on the reliability and validity of assessment instruments of these disorders in women. Gender differences in symptom patterns will be considered. The literature seems to suggest that DSM-IV criteria for ASPD may lead to an underestimation of the prevalence of the disorder in women due to the requirement of childhood conduct disorder symptoms. The Psychopathy Checklist-Revised (PCL-R) is a valid and reliable instrument to identify psychopathy in women but there are gender differences in the factor structure and item loadings on this measure. Research to date seems to suggest a three-factor model may be most strongly supported in females. Preliminary evidence suggests the PCL-R may have some value in predicting future offending while the PCL:SV may be useful in predicting institutional violence. Clinical implications are discussed.

  2. Development of the Human Factors Skills for Healthcare Instrument: a valid and reliable tool for assessing interprofessional learning across healthcare practice settings.

    Science.gov (United States)

    Reedy, Gabriel B; Lavelle, Mary; Simpson, Thomas; Anderson, Janet E

    2017-10-01

    A central feature of clinical simulation training is human factors skills, providing staff with the social and cognitive skills to cope with demanding clinical situations. Although these skills are critical to safe patient care, assessing their learning is challenging. This study aimed to develop, pilot and evaluate a valid and reliable structured instrument to assess human factors skills, which can be used pre- and post-simulation training, and is relevant across a range of healthcare professions. Through consultation with a multi-professional expert group, we developed and piloted a 39-item survey with 272 healthcare professionals attending training courses across two large simulation centres in London, one specialising in acute care and one in mental health, both serving healthcare professionals working across acute and community settings. Following psychometric evaluation, the final 12-item instrument was evaluated with a second sample of 711 trainees. Exploratory factor analysis revealed a 12-item, one-factor solution with good internal consistency (α=0.92). The instrument had discriminant validity, with newly qualified trainees scoring significantly lower than experienced trainees ( t (98)=4.88, pSkills for Healthcare Instrument provides a reliable and valid method of assessing trainees' human factors skills self-efficacy across acute and mental health settings. This instrument has the potential to improve the assessment and evaluation of human factors skills learning in both uniprofessional and interprofessional clinical simulation training.

  3. Reliability and Validity of Three Instruments (DSM-IV, CPGI, and PPGM) in the Assessment of Problem Gambling in South Korea.

    Science.gov (United States)

    Back, Ki-Joon; Williams, Robert J; Lee, Choong-Ki

    2015-09-01

    Most research on the assessment, epidemiology, and treatment of problem gambling has occurred in Western jurisdictions. This potentially limits the cross-cultural validity of problem gambling assessment instruments as well as etiological models of problem gambling. The primary objective of the present research was to investigate the reliability and validity of three problem gambling assessment instruments within a South Korean context. A total of 4,330 South Korean adults participated in a comprehensive assessment of their gambling behavior that included the administration of the DSM-IV criteria for pathological gambling (NODS), the Canadian Problem Gambling Index (CPGI), and the Problem and Pathological Gambling Measure (PPGM). Cronbach alpha showed that all three instruments had good internal consistency. Concurrent validity was established by the significant associations observed between scores on the instruments and measures of gambling involvement (number of gambling formats engaged in; frequency of gambling; and gambling expenditure). Most importantly, kappa statistics showed that all instruments have satisfactory classification accuracy against clinical assessment of problem gambling conducted by South Korean clinicians (NODS κ = .66; PPGM κ = .62; CPGI κ = .51). These results confirm that Western-derived operationalizations of problem gambling have applicability in a South Korean setting.

  4. Operational and reliability experience with reactor instrumentation

    International Nuclear Information System (INIS)

    Dixon, F.; Gow, R.S.

    1978-01-01

    In the last 15 years the CEGB has experienced progressive plant development, integration and changes in operating regime through nine nuclear (gas-cooled reactor) power stations with corresponding instrumentation advances leading towards more refined centralized control. Operation and reliability experience with reactor instrumentation is reported in this paper with reference to the progressive changes related to the early magnox, late magnox and AGR periods. Data on instrumentation reliability in terms of reactor forced outages are presented and show that the instrumentation contributions to loss of generating plant availability are small. Reactor safety circuits, neutron flux and temperature measurements, gas analysis and vibration monitoring are discussed. In reviewing the reactor instrumentation the emphasis is on reporting recent experience, particularly on AGR equipment, but overall performance and changes to magnox equipment are included so that some appreciation can be obtained of instrumentation requirements with respect to plant lifetimes. (author)

  5. The Children's Play Therapy Instrument (CPTI). Description, development, and reliability studies.

    Science.gov (United States)

    Kernberg, P F; Chazan, S E; Normandin, L

    1998-01-01

    The Children's Play Therapy Instrument (CPTI), its development, and reliability studies are described. The CPTI is a new instrument to examine a child's play activity in individual psychotherapy. Three independent raters used the CPTI to rate eight videotaped play therapy vignettes. Results were compared with the authors' consensual scores from a preliminary study. Generally good to excellent levels of interrater reliability were obtained for the independent raters on intraclass correlation coefficients for ordinal categories of the CPTI. Likewise, kappa levels were acceptable to excellent for nominal categories of the scale. The CPTI holds promise to become a reliable measure of play activity in child psychotherapy. Further research is needed to assess discriminant validity of the CPTI for use as a diagnostic tool and as a measure of process and outcome.

  6. Validity and reliability of a pictorial instrument for assessing perceived motor competence in Portuguese children.

    Science.gov (United States)

    Lopes, V P; Barnett, L M; Saraiva, L; Gonçalves, C; Bowe, S J; Abbott, G; Rodrigues, L P

    2016-09-01

    It is important to assess young children's perceived Fundamental Movement Skill (FMS) competence in order to examine the role of perceived FMS competence in motivation toward physical activity. Children's perceptions of motor competence may vary according to the culture/country of origin; therefore, it is also important to measure perceptions in different cultural contexts. The purpose was to assess the face validity, internal consistency, test-retest reliability and construct validity of the 12 FMS items in the Pictorial Scale for Perceived Movement Skill Competence for Young Children (PMSC) in a Portuguese sample. Two hundred one Portuguese children (girls, n = 112), 5 to 10 years of age (7.6 ± 1.4), participated. All children completed the PMSC once. Ordinal alpha assessed internal consistency. A random subsamples (n = 47) were reassessed one week later to determine test-retest reliability with Bland-Altman method. Children were asked questions after the second administration to determine face validity. Construct validity was assessed on the whole sample with a Bayesian Structural Equation Modelling (BSEM) approach. The hypothesized theoretical model used the 12 items and two hypothesized factors: object control and locomotor skills. The majority of children correctly identified the skills and could understand most of the pictures. Test-retest reliability analysis was good, with an agreement ration between 0.99 and 1.02. Ordinal alpha values ranged from acceptable (object control 0.73, locomotor 0.68) to good (all FMS 0.81). The hypothesized BSEM model had an adequate fit. The PMSC can be used to investigate perceptions of children's FMS competence. This instrument can also be satisfactorily used among Portuguese children. © 2016 John Wiley & Sons Ltd.

  7. Rater reliability and concurrent validity of the Keyboard Personal Computer Style instrument (K-PeCS).

    Science.gov (United States)

    Baker, Nancy A; Cook, James R; Redfern, Mark S

    2009-01-01

    This paper describes the inter-rater and intra-rater reliability, and the concurrent validity of an observational instrument, the Keyboard Personal Computer Style instrument (K-PeCS), which assesses stereotypical postures and movements associated with computer keyboard use. Three trained raters independently rated the video clips of 45 computer keyboard users to ascertain inter-rater reliability, and then re-rated a sub-sample of 15 video clips to ascertain intra-rater reliability. Concurrent validity was assessed by comparing the ratings obtained using the K-PeCS to scores developed from a 3D motion analysis system. The overall K-PeCS had excellent reliability [inter-rater: intra-class correlation coefficients (ICC)=.90; intra-rater: ICC=.92]. Most individual items on the K-PeCS had from good to excellent reliability, although six items fell below ICC=.75. Those K-PeCS items that were assessed for concurrent validity compared favorably to the motion analysis data for all but two items. These results suggest that most items on the K-PeCS can be used to reliably document computer keyboarding style.

  8. The Nutrition Literacy Assessment Instrument is a Valid and Reliable Measure of Nutrition Literacy in Adults with Chronic Disease.

    Science.gov (United States)

    Gibbs, Heather D; Ellerbeck, Edward F; Gajewski, Byron; Zhang, Chuanwu; Sullivan, Debra K

    2018-03-01

    To test the reliability and validity of the Nutrition Literacy Assessment Instrument (NLit) in adult primary care and identify the relationship between nutrition literacy and diet quality. This instrument validation study included a cross-sectional sample participating in up to 2 visits 1 month apart. A total of 429 adults with nutrition-related chronic disease were recruited from clinics and a patient registry affiliated with a Midwestern university medical center. Nutrition literacy was measured by the NLit, which was composed of 6 subscales: nutrition and health, energy sources in food, food label and numeracy, household food measurement, food groups, and consumer skills. Diet quality was measured by Healthy Eating Index-2010 with nutrient data from Diet History Questionnaire II surveys. The researchers measured factor validity and reliability by using binary confirmatory factor analysis; test-retest reliability was measured by Pearson r and the intraclass correlation coefficient, and relationships between nutrition literacy and diet quality were analyzed by linear regression. The NLit demonstrated substantial factor validity and reliability (0.97; confidence interval, 0.96-0.98) and test-retest reliability (0.88; confidence interval, 0.85-0.90). Nutrition literacy was the most significant predictor of diet quality (β = .17; multivariate coefficient = 0.10; P measuring nutrition literacy in adult primary care patients. Copyright © 2017 Society for Nutrition Education and Behavior. Published by Elsevier Inc. All rights reserved.

  9. The validity and reliability of the Socioeconomic Status Instrument for assessing prostate cancer patients.

    Science.gov (United States)

    Cyrus-David, Mfon

    2010-08-01

    Because of the lack of consistency in the associations of the socioeconomic status (SES) of prostate cancer (PC) patients from diverse racial and ethnic backgrounds with PC health outcomes, I created the Socioeconomic Status Instrument (SESI) from the Demographic and Health Access components of the Behavioral Risk Factor Surveillance System 2004 Questionnaires and the socioeconomic indices of the subjects' residential counties to better assess the SES of PC patients. The SESI was tested on 220 consecutive subjects with pathologically confirmed PC at the Veterans Affairs Medical Center in Houston, TX. A team that included an epidemiologist, a validation statistician/health services research scientist, and PC survivors assessed the content validity of the SESI. The construct validity of the SESI was assessed with factor analysis by extracting and analyzing 5 principal components based on the subjects' individual responses on the assessment: county socioeconomic characteristics, individual socioeconomic characteristics, financial distress, increased domestic burden with limited earnings, and affluence. The internal consistency reliability of the SESI was assessed with Cronbach's alpha coefficients. Based on the reviews of the SESI, all of the initial 10 items were retained. The correlations between individual responses on the SESI were similar to the results of previous studies. The 5 principal components that I assessed accounted for 71.5% of the variance. Factor loadings ranged from 0.66 to 0.98 and communalities ranged from 0.55 to 0.94. County socioeconomic characteristics accounted for 22.6% of the variance, whereas individual socioeconomic characteristics accounted for 14.6% of the variance. The overall Cronbach's alpha coefficient was 0.78. The SESI is valid and reliable. Accurate measurements of the SES of PC patients would provide better guidance for future research and care deliveries.

  10. The Children's Play Therapy Instrument (CPTI): Description, Development, and Reliability Studies

    Science.gov (United States)

    Kernberg, Paulina F.; Chazan, Saralea E.; Normandin, Lina

    1998-01-01

    The Children's Play Therapy Instrument (CPTI), its development, and reliability studies are described. The CPTI is a new instrument to examine a child's play activity in individual psychotherapy. Three independent raters used the CPTI to rate eight videotaped play therapy vignettes. Results were compared with the authors' consensual scores from a preliminary study. Generally good to excellent levels of interrater reliability were obtained for the independent raters on intraclass correlation coefficients for ordinal categories of the CPTI. Likewise, kappa levels were acceptable to excellent for nominal categories of the scale. The CPTI holds promise to become a reliable measure of play activity in child psychotherapy. Further research is needed to assess discriminant validity of the CPTI for use as a diagnostic tool and as a measure of process and outcome.(The Journal of Psychotherapy Practice and Research 1998; 7:196–207) PMID:9631341

  11. The Assumption of a Reliable Instrument and Other Pitfalls to Avoid When Considering the Reliability of Data

    Science.gov (United States)

    Nimon, Kim; Zientek, Linda Reichwein; Henson, Robin K.

    2012-01-01

    The purpose of this article is to help researchers avoid common pitfalls associated with reliability including incorrectly assuming that (a) measurement error always attenuates observed score correlations, (b) different sources of measurement error originate from the same source, and (c) reliability is a function of instrumentation. To accomplish our purpose, we first describe what reliability is and why researchers should care about it with focus on its impact on effect sizes. Second, we review how reliability is assessed with comment on the consequences of cumulative measurement error. Third, we consider how researchers can use reliability generalization as a prescriptive method when designing their research studies to form hypotheses about whether or not reliability estimates will be acceptable given their sample and testing conditions. Finally, we discuss options that researchers may consider when faced with analyzing unreliable data. PMID:22518107

  12. Process Skill Assessment Instrument: Innovation to measure student’s learning result holistically

    Science.gov (United States)

    Azizah, K. N.; Ibrahim, M.; Widodo, W.

    2018-01-01

    Science process skills (SPS) are very important skills for students. However, the fact that SPS is not being main concern in the primary school learning is undeniable. This research aimed to develop a valid, practical, and effective assessment instrument to measure student’s SPS. Assessment instruments comprise of worksheet and test. This development research used one group pre-test post-test design. Data were obtained with validation, observation, and test method to investigate validity, practicality, and the effectivenss of the instruments. Results showed that the validity of assessment instruments is very valid, the reliability is categorized as reliable, student SPS activities have a high percentage, and there is significant improvement on student’s SPS score. It can be concluded that assessment instruments of SPS are valid, practical, and effective to be used to measure student’s SPS result.

  13. An Attitude Scale on Individual Instrument and Individual Instrument Course: Validity-Reliability Research

    Science.gov (United States)

    Kuçukosmanoglu, Hayrettin Onur

    2015-01-01

    The main purpose of this study is to develop a scale to determine students' attitude levels on individual instruments and individual instrument courses in instrument training, which is an important dimension of music education, and to conduct a validity-reliability research of the scale that has been developed. The scale consists of 16 items. The…

  14. Reliability Estimation for Digital Instrument/Control System

    Energy Technology Data Exchange (ETDEWEB)

    Yang, Yaguang; Sydnor, Russell [U.S. Nuclear Regulatory Commission, Washington, D.C. (United States)

    2011-08-15

    Digital instrumentation and controls (DI and C) systems are widely adopted in various industries because of their flexibility and ability to implement various functions that can be used to automatically monitor, analyze, and control complicated systems. It is anticipated that the DI and C will replace the traditional analog instrumentation and controls (AI and C) systems in all future nuclear reactor designs. There is an increasing interest for reliability and risk analyses for safety critical DI and C systems in regulatory organizations, such as The United States Nuclear Regulatory Commission. Developing reliability models and reliability estimation methods for digital reactor control and protection systems will involve every part of the DI and C system, such as sensors, signal conditioning and processing components, transmission lines and digital communication systems, D/A and A/D converters, computer system, signal processing software, control and protection software, power supply system, and actuators. Some of these components are hardware, such as sensors and actuators, their failure mechanisms are well understood, and the traditional reliability model and estimation methods can be directly applied. But many of these components are firmware which has software embedded in the hardware, and software needs special consideration because its failure mechanism is unique, and the reliability estimation method for a software system will be different from the ones used for hardware systems. In this paper, we will propose a reliability estimation method for the entire DI and C system reliability using a recently developed software reliability estimation method and a traditional hardware reliability estimation method.

  15. Reliability Estimation for Digital Instrument/Control System

    International Nuclear Information System (INIS)

    Yang, Yaguang; Sydnor, Russell

    2011-01-01

    Digital instrumentation and controls (DI and C) systems are widely adopted in various industries because of their flexibility and ability to implement various functions that can be used to automatically monitor, analyze, and control complicated systems. It is anticipated that the DI and C will replace the traditional analog instrumentation and controls (AI and C) systems in all future nuclear reactor designs. There is an increasing interest for reliability and risk analyses for safety critical DI and C systems in regulatory organizations, such as The United States Nuclear Regulatory Commission. Developing reliability models and reliability estimation methods for digital reactor control and protection systems will involve every part of the DI and C system, such as sensors, signal conditioning and processing components, transmission lines and digital communication systems, D/A and A/D converters, computer system, signal processing software, control and protection software, power supply system, and actuators. Some of these components are hardware, such as sensors and actuators, their failure mechanisms are well understood, and the traditional reliability model and estimation methods can be directly applied. But many of these components are firmware which has software embedded in the hardware, and software needs special consideration because its failure mechanism is unique, and the reliability estimation method for a software system will be different from the ones used for hardware systems. In this paper, we will propose a reliability estimation method for the entire DI and C system reliability using a recently developed software reliability estimation method and a traditional hardware reliability estimation method

  16. Validity and Reliability of Turkish Male Breast Self-Examination Instrument.

    Science.gov (United States)

    Erkin, Özüm; Göl, İlknur

    2018-04-01

    This study aims to measure the validity and reliability of Turkish male breast self-examination (MBSE) instrument. The methodological study was performed in 2016 at Ege University, Faculty of Nursing, İzmir, Turkey. The MBSE includes ten steps. For validity studies, face validity, content validity, and construct validity (exploratory factor analysis) were done. For reliability study, Kuder Richardson was calculated. The content validity index was found to be 0.94. Kendall W coefficient was 0.80 (p=0.551). The total variance explained by the two factors was found to be 63.24%. Kuder Richardson 21 was done for reliability study and found to be 0.97 for the instrument. The final instrument included 10 steps and two stages. The Turkish version of MBSE is a valid and reliable instrument for early diagnose. The MBSE can be used in Turkish speaking countries and cultures with two stages and 10 steps.

  17. Validity and Reliability of the Clinical Competency Evaluation Instrument for Use among Physiotherapy Students: Pilot study.

    Science.gov (United States)

    Muhamad, Zailani; Ramli, Ayiesah; Amat, Salleh

    2015-05-01

    The aim of this study was to determine the content validity, internal consistency, test-retest reliability and inter-rater reliability of the Clinical Competency Evaluation Instrument (CCEVI) in assessing the clinical performance of physiotherapy students. This study was carried out between June and September 2013 at University Kebangsaan Malaysia (UKM), Kuala Lumpur, Malaysia. A panel of 10 experts were identified to establish content validity by evaluating and rating each of the items used in the CCEVI with regards to their relevance in measuring students' clinical competency. A total of 50 UKM undergraduate physiotherapy students were assessed throughout their clinical placement to determine the construct validity of these items. The instrument's reliability was determined through a cross-sectional study involving a clinical performance assessment of 14 final-year undergraduate physiotherapy students. The content validity index of the entire CCEVI was 0.91, while the proportion of agreement on the content validity indices ranged from 0.83-1.00. The CCEVI construct validity was established with factor loading of ≥0.6, while internal consistency (Cronbach's alpha) overall was 0.97. Test-retest reliability of the CCEVI was confirmed with a Pearson's correlation range of 0.91-0.97 and an intraclass coefficient correlation range of 0.95-0.98. Inter-rater reliability of the CCEVI domains ranged from 0.59 to 0.97 on initial and subsequent assessments. This pilot study confirmed the content validity of the CCEVI. It showed high internal consistency, thereby providing evidence that the CCEVI has moderate to excellent inter-rater reliability. However, additional refinement in the wording of the CCEVI items, particularly in the domains of safety and documentation, is recommended to further improve the validity and reliability of the instrument.

  18. Assessment of Work Performance (AWP)--development of an instrument.

    Science.gov (United States)

    Sandqvist, Jan L; Törnquist, Kristina B; Henriksson, Chris M

    2006-01-01

    Adequate work assessments are a matter of importance both for individuals and society [5,29,31,38,40,46,52]. However, there is a lack of adequate and reliable instruments for use in work rehabilitation [14,15,20,21,31,44]. The purpose of this study was to develop and evaluate an observation instrument for assessing work performance, the AWP (Assessment of Work Performance). The purpose of the 14-item instrument is to assess the individual's observable working skills in three different areas: motor skills, process skills, and communication and interaction skills. This article describes the development and results of preliminary testing of the AWP. The testing indicates a satisfactory face validity and utility for the AWP and supports further research and testing of the instrument.

  19. Translation, Validation, and Reliability of the Dutch Late-Life Function and Disability Instrument Computer Adaptive Test.

    Science.gov (United States)

    Arensman, Remco M; Pisters, Martijn F; de Man-van Ginkel, Janneke M; Schuurmans, Marieke J; Jette, Alan M; de Bie, Rob A

    2016-09-01

    Adequate and user-friendly instruments for assessing physical function and disability in older adults are vital for estimating and predicting health care needs in clinical practice. The Late-Life Function and Disability Instrument Computer Adaptive Test (LLFDI-CAT) is a promising instrument for assessing physical function and disability in gerontology research and clinical practice. The aims of this study were: (1) to translate the LLFDI-CAT to the Dutch language and (2) to investigate its validity and reliability in a sample of older adults who spoke Dutch and dwelled in the community. For the assessment of validity of the LLFDI-CAT, a cross-sectional design was used. To assess reliability, measurement of the LLFDI-CAT was repeated in the same sample. The item bank of the LLFDI-CAT was translated with a forward-backward procedure. A sample of 54 older adults completed the LLFDI-CAT, World Health Organization Disability Assessment Schedule 2.0, RAND 36-Item Short-Form Health Survey physical functioning scale (10 items), and 10-Meter Walk Test. The LLFDI-CAT was repeated in 2 to 8 days (mean=4.5 days). Pearson's r and the intraclass correlation coefficient (ICC) (2,1) were calculated to assess validity, group-level reliability, and participant-level reliability. A correlation of .74 for the LLFDI-CAT function scale and the RAND 36-Item Short-Form Health Survey physical functioning scale (10 items) was found. The correlations of the LLFDI-CAT disability scale with the World Health Organization Disability Assessment Schedule 2.0 and the 10-Meter Walk Test were -.57 and -.53, respectively. The ICC (2,1) of the LLFDI-CAT function scale was .84, with a group-level reliability score of .85. The ICC (2,1) of the LLFDI-CAT disability scale was .76, with a group-level reliability score of .81. The high percentage of women in the study and the exclusion of older adults with recent joint replacement or hospitalization limit the generalizability of the results. The Dutch LLFDI

  20. Intrajudge and Interjudge Reliability of the Stuttering Severity Instrument-Fourth Edition.

    Science.gov (United States)

    Davidow, Jason H; Scott, Kathleen A

    2017-11-08

    The Stuttering Severity Instrument (SSI) is a tool used to measure the severity of stuttering. Previous versions of the instrument have known limitations (e.g., Lewis, 1995). The present study examined the intra- and interjudge reliability of the newest version, the Stuttering Severity Instrument-Fourth Edition (SSI-4) (Riley, 2009). Twelve judges who were trained on the SSI-4 protocol participated. Judges collected SSI-4 data while viewing 4 videos of adults who stutter at Time 1 and 4 weeks later at Time 2. Data were analyzed for intra- and interjudge reliability of the SSI-4 subscores (for Frequency, Duration, and Physical Concomitants), total score, and final severity rating. Intra- and interjudge reliability across the subscores and total score concurred with the manual's reported reliability when reliability was calculated using the methods described in the manual. New calculations of judge agreement produced different values from those in the manual-for the 3 subscores, total score, and final severity rating-and provided data absent from the manual. Clinicians and researchers who use the SSI-4 should carefully consider the limitations of the instrument. Investigation into the multitasking demands of the instrument may provide information on whether separating the collection of data for specific variables will improve intra- and interjudge reliability of those variables.

  1. Reliability assessment using Bayesian networks. Case study on quantative reliability estimation of a software-based motor protection relay

    International Nuclear Information System (INIS)

    Helminen, A.; Pulkkinen, U.

    2003-06-01

    In this report a quantitative reliability assessment of motor protection relay SPAM 150 C has been carried out. The assessment focuses to the methodological analysis of the quantitative reliability assessment using the software-based motor protection relay as a case study. The assessment method is based on Bayesian networks and tries to take the full advantage of the previous work done in a project called Programmable Automation System Safety Integrity assessment (PASSI). From the results and experiences achieved during the work it is justified to claim that the assessment method presented in the work enables a flexible use of qualitative and quantitative elements of reliability related evidence in a single reliability assessment. At the same time the assessment method is a concurrent way of reasoning one's beliefs and references about the reliability of the system. Full advantage of the assessment method is taken when using the method as a way to cultivate the information related to the reliability of software-based systems. The method can also be used as a communicational instrument in a licensing process of software-based systems. (orig.)

  2. Reliability and validity of a nutrition and physical activity environmental self-assessment for child care

    Directory of Open Access Journals (Sweden)

    Ammerman Alice S

    2007-07-01

    Full Text Available Abstract Background Few assessment instruments have examined the nutrition and physical activity environments in child care, and none are self-administered. Given the emerging focus on child care settings as a target for intervention, a valid and reliable measure of the nutrition and physical activity environment is needed. Methods To measure inter-rater reliability, 59 child care center directors and 109 staff completed the self-assessment concurrently, but independently. Three weeks later, a repeat self-assessment was completed by a sub-sample of 38 directors to assess test-retest reliability. To assess criterion validity, a researcher-administered environmental assessment was conducted at 69 centers and was compared to a self-assessment completed by the director. A weighted kappa test statistic and percent agreement were calculated to assess agreement for each question on the self-assessment. Results For inter-rater reliability, kappa statistics ranged from 0.20 to 1.00 across all questions. Test-retest reliability of the self-assessment yielded kappa statistics that ranged from 0.07 to 1.00. The inter-quartile kappa statistic ranges for inter-rater and test-retest reliability were 0.45 to 0.63 and 0.27 to 0.45, respectively. When percent agreement was calculated, questions ranged from 52.6% to 100% for inter-rater reliability and 34.3% to 100% for test-retest reliability. Kappa statistics for validity ranged from -0.01 to 0.79, with an inter-quartile range of 0.08 to 0.34. Percent agreement for validity ranged from 12.9% to 93.7%. Conclusion This study provides estimates of criterion validity, inter-rater reliability and test-retest reliability for an environmental nutrition and physical activity self-assessment instrument for child care. Results indicate that the self-assessment is a stable and reasonably accurate instrument for use with child care interventions. We therefore recommend the Nutrition and Physical Activity Self-Assessment for

  3. The reliability of three psoriasis assessment tools: Psoriasis area and severity index, body surface area and physician global assessment.

    Science.gov (United States)

    Bożek, Agnieszka; Reich, Adam

    2017-08-01

    A wide variety of psoriasis assessment tools have been proposed to evaluate the severity of psoriasis in clinical trials and daily practice. The most frequently used clinical instrument is the psoriasis area and severity index (PASI); however, none of the currently published severity scores used for psoriasis meets all the validation criteria required for an ideal score. The aim of this study was to compare and assess the reliability of 3 commonly used assessment instruments for psoriasis severity: the psoriasis area and severity index (PASI), body surface area (BSA) and physician global assessment (PGA). On the scoring day, 10 trained dermatologists evaluated 9 adult patients with plaque-type psoriasis using the PASI, BSA and PGA. All the subjects were assessed twice by each physician. Correlations between the assessments were analyzed using the Pearson correlation coefficient. Intra-class correlation coefficient (ICC) was calculated to analyze intra-rater reliability, and the coefficient of variation (CV) was used to assess inter-rater variability. Significant correlations were observed among the 3 scales in both assessments. In all 3 scales the ICCs were > 0.75, indicating high intra-rater reliability. The highest ICC was for the BSA (0.96) and the lowest one for the PGA (0.87). The CV for the PGA and PASI were 29.3 and 36.9, respectively, indicating moderate inter-rater variability. The CV for the BSA was 57.1, indicating high inter-rater variability. Comparing the PASI, PGA and BSA, it was shown that the PGA had the highest inter-rater reliability, whereas the BSA had the highest intra-rater reliability. The PASI showed intermediate values in terms of interand intra-rater reliability. None of the 3 assessment instruments showed a significant advantage over the other. A reliable assessment of psoriasis severity requires the use of several independent evaluations simultaneously.

  4. The Development and Evaluation of a Novel Instrument Assessing Residents' Discharge Summaries.

    Science.gov (United States)

    Hommos, Musab S; Kuperman, Ethan F; Kamath, Aparna; Kreiter, Clarence D

    2017-04-01

    To develop and determine the reliability of a novel measurement instrument assessing the quality of residents' discharge summaries. In 2014, the authors created a discharge summary evaluation instrument based on consensus recommendations from national regulatory bodies and input from primary care providers at their institution. After a brief pilot, they used the instrument to evaluate discharge summaries written by first-year internal medicine residents (n = 24) at a single U.S. teaching hospital during the 2013-2014 academic year. They conducted a generalizability study to determine the reliability of the instrument and a series of decision studies to determine the number of discharge summaries and raters needed to achieve a reliable evaluation score. The generalizability study demonstrated that 37% of the variance reflected residents' ability to generate an adequate discharge summary (true score variance). The decision studies estimated that the mean score from six discharge summary reviews completed by a unique rater for each review would yield a reliability coefficient of 0.75. Because of high interrater reliability, multiple raters per discharge summary would not significantly enhance the reliability of the mean rating. This evaluation instrument reliably measured residents' performance writing discharge summaries. A single rating of six discharge summaries can achieve a reliable mean evaluation score. Using this instrument is feasible even for programs with a limited number of inpatient encounters and a small pool of faculty preceptors.

  5. Development and testing of an assessment instrument for the formative peer review of significant event analyses.

    Science.gov (United States)

    McKay, J; Murphy, D J; Bowie, P; Schmuck, M-L; Lough, M; Eva, K W

    2007-04-01

    To establish the content validity and specific aspects of reliability for an assessment instrument designed to provide formative feedback to general practitioners (GPs) on the quality of their written analysis of a significant event. Content validity was quantified by application of a content validity index. Reliability testing involved a nested design, with 5 cells, each containing 4 assessors, rating 20 unique significant event analysis (SEA) reports (10 each from experienced GPs and GPs in training) using the assessment instrument. The variance attributable to each identified variable in the study was established by analysis of variance. Generalisability theory was then used to investigate the instrument's ability to discriminate among SEA reports. Content validity was demonstrated with at least 8 of 10 experts endorsing all 10 items of the assessment instrument. The overall G coefficient for the instrument was moderate to good (G>0.70), indicating that the instrument can provide consistent information on the standard achieved by the SEA report. There was moderate inter-rater reliability (G>0.60) when four raters were used to judge the quality of the SEA. This study provides the first steps towards validating an instrument that can provide educational feedback to GPs on their analysis of significant events. The key area identified to improve instrument reliability is variation among peer assessors in their assessment of SEA reports. Further validity and reliability testing should be carried out to provide GPs, their appraisers and contractual bodies with a validated feedback instrument on this aspect of the general practice quality agenda.

  6. Validity and reliability of the Mastication Observation and Evaluation (MOE) instrument.

    Science.gov (United States)

    Remijn, Lianne; Speyer, Renée; Groen, Brenda E; van Limbeek, Jacques; Nijhuis-van der Sanden, Maria W G

    2014-07-01

    The Mastication Observation and Evaluation (MOE) instrument was developed to allow objective assessment of a child's mastication process. It contains 14 items and was developed over three Delphi rounds. The present study concerns the further development of the MOE using the COSMIN (Consensus based Standard for the Selection of Measurement Instruments) and investigated the instrument's internal consistency, inter-observer reliability, construct validity and floor and ceiling effects. Consumption of three bites of bread and biscuit was evaluated using the MOE. Data of 59 healthy children (6-48 mths) and 38 children (bread) and 37 children (biscuit) with cerebral palsy (24-72 mths) were used. Four items were excluded before analysis due to zero variance. Principal Components Analysis showed one factor with 8 items. Internal consistency was >0.70 (Chronbach's alpha) for both food consistencies and for both groups of children. Inter-observer reliability varied from 0.51 to 0.98 (weighted Gwet's agreement coefficient). The total MOE scores for both groups showed normal distribution for the population. There were no floor or ceiling effects. The revised MOE now contains 8 items that (a) have a consistent concept for mastication and can be scored on a 4-point scale with sufficient reliability and (b) are sensitive to stages of chewing development in young children. The removed items are retained as part of a criterion referenced list within the MOE. Copyright © 2014 Elsevier Ltd. All rights reserved.

  7. Developing of Individual Instrument Performance Anxiety Scale: ValidityReliability Study

    Directory of Open Access Journals (Sweden)

    Esra DALKIRAN

    2016-07-01

    Full Text Available In this study, it is intended to develop a scale unique to our culture, concerning individual instrument performance anxiety of the students who are getting instrument training in the Department of Music Education. In the study, the descriptive research model is used and qualitative research techniques are utilized. The study population consists of the students attending the 23 universities which has Music Education Department. The sample of the study consists of 438 girls and 312 boys, totally 750 students who are studying in the Department of Music Education of randomly selected 10 universities. As a result of the explanatory and confirmatory factor analyses that were performed, a onedimensional structure consisting of 14 items was obtained. Also, t-scores and the coefficient scores of total item correlation concerning the distinguishing power of the items, the difference in the scores of the set of lower and upper 27% was calculated, and it was observed that the items are distinguishing as a result of both analyses. Of the scale, Cronbach's alpha coefficient of internal consistency was calculated as .94, and test-retest reliability coefficient was calculated as .93. As a result, a valid and reliable assessment and evaluation instrument that measures the exam performance anxiety of the students studying in the Department of Music Education, has been developed.

  8. The modified patient enablement instrument: a Portuguese cross-cultural adaptation, validity and reliability study.

    Science.gov (United States)

    Remelhe, Mafalda; Teixeira, Pedro M; Lopes, Irene; Silva, Luís; Correia de Sousa, Jaime

    2017-01-12

    Enabling patients with asthma to obtain the knowledge, confidence and skills they need in order to assume a major role in the management of their disease is cost effective. It should be an integral part of any plan for long-term control of asthma. The modified Patient Enablement Instrument (mPEI) is an easily administered questionnaire that was adapted in the United Kingdom to measure patient enablement in asthma, but its applicability in Portugal is not known. Validity and reliability of questionnaires should be tested before use in settings different from those of the original version. The purpose of this study was to test the applicability of the mPEI to Portuguese asthma patients after translation and cross-cultural adaptation, and to verify the structural validity, internal consistency and reproducibility of the instrument. The mPEI was translated to Portuguese and back translated to English. Its content validity was assessed by a debriefing interview with 10 asthma patients. The translated instrument was then administered to a random sample of 142 patients with persistent asthma. Structural validity and internal consistency were assessed. For reproducibility analysis, 86 patients completed the instrument again 7 days later. Item-scale correlations and exploratory factor analysis were used to assess structural validity. Cronbach's alpha was used to test internal consistency, and the intra-class correlation coefficient was used for the analysis of reproducibility. All items of the Portuguese version of the mPEI were found to be equivalent to the original English version. There were strong item-scale correlations that confirmed construct validity, with a one component structure and good internal consistency (Cronbach's alpha >0.8) as well as high test-retest reliability (ICC=0.85). The mPEI showed sound psychometric properties for the evaluation of enablement in patients with asthma making it a reliable instrument for use in research and clinical practice in

  9. WATCH: Warwick Assessment insTrument for Clinical teacHing: Development and testing.

    Science.gov (United States)

    Haider, Sonia Ijaz; Johnson, Neil; Thistlethwaite, Jill Elizabeth; Fagan, Gay; Bari, Muhammad Furqan

    2015-03-01

    Medical education and teaching skills are core competencies included in the generic curriculum for specialty training. To support the development of these skills, there is need for a validated instrument. This study aims to develop and test an instrument to measure the attributes of specialty trainees as effective teachers. The study was conducted in two phases. In first phase, the content of the instrument was generated from the literature and tested using the Delphi technique. In second phase, the instrument was field tested for validity and reliability using factor analysis and generalizability study. Feasibility was calculated by the time taken to complete the instrument. Acceptability and educational impact were determined by qualitative analysis of written feedback. Attributes of specialty trainees were assessed by clinical supervisors, peers, and students. The Delphi study produced consensus on 15 statements which formed the basis of the instrument. In field study, a total of 415 instruments were completed. Factor analysis demonstrated a three-factor solution ('learning-teaching milieu', 'teaching skills', and 'learner-orientated'). A generalizability coefficient was 0.92. Mean time to complete the instrument was five minutes. Feedback indicated that it was an acceptable and useful method of assessment. This new instrument provides valid, reliable, feasible, and acceptable assessment of clinical teaching.

  10. Test - retest reliability of two instruments for measuring public attitudes towards persons with mental illness

    Directory of Open Access Journals (Sweden)

    Leufstadius Christel

    2011-01-01

    Full Text Available Abstract Background Research has identified stigmatization as a major threat to successful treatment of individuals with mental illness. As a consequence several anti-stigma campaigns have been carried out. The results have been discouraging and the field suffers from lack of evidence about interventions that work. There are few reports on psychometric data for instruments used to assess stigma, which thus complicates research efforts. The aim of the present study was to investigate test-retest reliability of the Swedish versions of the questionnaires: FABI and "Changing Minds" and to examine the internal consistency of the two instruments. Method Two instruments, fear and behavioural intentions (FABI and "Changing Minds", used in earlier studies on public attitudes towards persons with mental illness were translated into Swedish and completed by 51 nursing students on two occasions, with an interval of three weeks. Test-retest reliability was calculated by using weighted kappa coefficient and internal consistency using the Cronbach's alpha coefficient. Results Both instruments attain at best moderate test-retest reliability. For the Changing Minds questionnaire almost one fifth (17.9% of the items present poor test-retest reliability and the alpha coefficient for the subscales ranges between 0.19 - 0.46. All of the items in the FABI reach a fair or a moderate agreement between the test and retest, and the questionnaire displays a high internal consistency, alpha 0.80. Conclusions There is a need for development of psychometrically tested instruments within this field of research.

  11. Development and validation of an instrument to assess perceived social influence on health behaviors

    Science.gov (United States)

    HOLT, CHERYL L.; CLARK, EDDIE M.; ROTH, DAVID L.; CROWTHER, MARTHA; KOHLER, CONNIE; FOUAD, MONA; FOUSHEE, RUSTY; LEE, PATRICIA A.; SOUTHWARD, PENNY L.

    2012-01-01

    Assessment of social influence on health behavior is often approached through a situational context. The current study adapted an existing, theory-based instrument from another content domain to assess Perceived Social Influence on Health Behavior (PSI-HB) among African Americans, using an individual difference approach. The adapted instrument was found to have high internal reliability (α = .81–.84) and acceptable testretest reliability (r = .68–.85). A measurement model revealed a three-factor structure and supported the theoretical underpinnings. Scores were predictive of health behaviors, particularly among women. Future research using the new instrument may have applied value assessing social influence in the context of health interventions. PMID:20522506

  12. A systematic review of instruments to assess organizational readiness for knowledge translation in health care.

    Directory of Open Access Journals (Sweden)

    Marie-Pierre Gagnon

    Full Text Available The translation of research into practices has been incomplete. Organizational readiness for change (ORC is a potential facilitator of effective knowledge translation (KT. However we know little about the best way to assess ORC. Therefore, we sought to systematically review ORC measurement instruments.We searched for published studies in bibliographic databases (Pubmed, Embase, CINAHL, PsychINFO, Web of Science, etc. up to November 1st, 2012. We included publications that developed ORC measures and/or empirically assessed ORC using an instrument at the organizational level in the health care context. We excluded articles if they did not refer specifically to ORC, did not concern the health care domain or were limited to individual-level change readiness. We focused on identifying the psychometric properties of instruments that were developed to assess readiness in an organization prior to implementing KT interventions in health care. We used the Standards for Educational and Psychological Testing to assess the psychometric properties of identified ORC measurement instruments.We found 26 eligible instruments described in 39 publications. According to the Standards for Educational and Psychological Testing, 18 (69% of a total of 26 measurement instruments presented both validity and reliability criteria. The Texas Christian University -ORC (TCU-ORC scale reported the highest instrument validity with a score of 4 out of 4. Only one instrument, namely the Modified Texas Christian University - Director version (TCU-ORC-D, reported a reliability score of 2 out of 3. No information was provided regarding the reliability and validity of five (19% instruments.Our findings indicate that there are few valid and reliable ORC measurement instruments that could be applied to KT in the health care sector. The TCU-ORC instrument presents the best evidence in terms of validity testing. Future studies using this instrument could provide more knowledge on its

  13. [Development and reliability evaluation of an instrument to measure health-related quality of life in independent elderly].

    Science.gov (United States)

    Lima, Maria José Barbosa de; Portela, Margareth Crisóstomo

    2010-08-01

    This study presents an instrument, the health-related quality of life (HRQOL) profile for independent elderly, to measure the health-related quality of life of the functionally independent elderly assisted in the outpatient setting, based on the adaptation of four validated scales: Short-Form Health Survey (SF-36), Duke-UNC Health Profile (DUHP), Sickness Impact Profile (SIP), and Nottingham Health Profile (NHP). The study also evaluates the instrument's reliability based on its use by two different observers with a 15-day interval. The instrument includes five dimensions (health perception, symptoms, physical function, psychological function, and social function) and 45 items. Reliability evaluation of the QUASI instrument was based on interviews with 142 elderly outpatients in the city of Rio de Janeiro, Brazil. Prevalence-adjusted kappa statistic was used to assess all 45 items. Correlation was also calculated between overall scores and scores on individual dimensions. In the reliability evaluation, 39 of the 45 items showed prevalence-adjusted kappa greater than 0.60.

  14. DEVELOPMENT OF PERFORMANCE ASSESSMENT INSTRUMENT FOR NURSES BASED ON WEB IN INPATIENT UNIT

    Directory of Open Access Journals (Sweden)

    Aprilia Nuryanti

    2017-06-01

    Full Text Available Introduction: Performance assessment instrument will be problematic when it is not representative in describing the competency because it is not obvious indicators and inappropriate performance standard to nursing’s task. The purpose of this study is to develop nurses’ performance assessment instrument based on the web from multi sources assessment inpatient unit at SMC Hospital. Methods: This study had two phases. The first phase was an explanatory overview of the performance assessment system using questionnaires completed by 53 respondents of nurses, selected by purposive sampling. Instrument development based on FGD with six decision makers in the hospital. Validity was tested by Pearson Product Moment Correlation and reliability of instrument’s was tested by alpha Cronbach. The second phase was socialization and instrument test to observe the quality of instrument using a questionnaire by 47 respondents and recommendations made by 8 participants of FGD. The samples were selected by purposive sampling technique. Performance assessment system was moderate at 58.49%. All questions which aimed to measure the performance of nurses were valid and reliable. The quality of nurses’ performance assessment instruments based on the web was a good category, which was functionality: 81.60; reliability: 78.16; efficiency: 80.85; usability: 81.70 and portability: 81.70. Results: The result was a web-based assessment format, scoring with Likert scale, resource assessment by the direct supervisor which was a multisource evaluator, the development of performance graph, and confidentiality of data on the database server. Discussion: Recommendations for hospital is to make policy based on the final value of the performance assessment by the supervisor which was multisource feedback and it needs a global writing on a form of performance assessment result.

  15. Measuring pediatric hematology-oncology fellows' skills in humanism and professionalism: A novel assessment instrument.

    Science.gov (United States)

    Kesselheim, Jennifer C; Agrawal, Anurag K; Bhatia, Nita; Cronin, Angel; Jubran, Rima; Kent, Paul; Kersun, Leslie; Rao, Amulya Nageswara; Rose, Melissa; Savelli, Stephanie; Sharma, Mukta; Shereck, Evan; Twist, Clare J; Wang, Michael

    2017-05-01

    Educators in pediatric hematology-oncology lack rigorously developed instruments to assess fellows' skills in humanism and professionalism. We developed a novel 15-item self-assessment instrument to address this gap in fellowship training. Fellows (N = 122) were asked to assess their skills in five domains: balancing competing demands of fellowship, caring for the dying patient, confronting depression and burnout, responding to challenging relationships with patients, and practicing humanistic medicine. An expert focus group predefined threshold scores on the instrument that could be used as a cutoff to identify fellows who need support. Reliability and feasibility were assessed and concurrent validity was measured using three established instruments: Maslach Burnout Inventory (MBI), Flourishing Scale (FS), and Jefferson Scale of Physician Empathy (JSPE). For 90 participating fellows (74%), the self-assessment proved feasible to administer and had high internal consistency reliability (Cronbach's α = 0.81). It was moderately correlated with the FS and MBI (Pearson's r = 0.41 and 0.4, respectively) and weakly correlated with the JSPE (Pearson's r = 0.15). Twenty-eight fellows (31%) were identified as needing support. The self-assessment had a sensitivity of 50% (95% confidence interval [CI]: 31-69) and a specificity of 77% (95% CI: 65-87) for identifying fellows who scored poorly on at least one of the three established scales. We developed a novel assessment instrument for use in pediatric fellowship training. The new scale proved feasible and demonstrated internal consistency reliability. Its moderate correlation with other established instruments shows that the novel assessment instrument provides unique, nonredundant information as compared to existing scales. © 2016 Wiley Periodicals, Inc.

  16. Functional leg length discrepancy between theories and reliable instrumental assessment: a study about newly invented NPoS system.

    Science.gov (United States)

    Mahmoud, Asmaa; Abundo, Paolo; Basile, Luisanna; Albensi, Caterina; Marasco, Morena; Bellizzi, Letizia; Galasso, Franco; Foti, Calogero

    2017-01-01

    In spite the instinct social&financial impact of Leg Length Discrepancy (LLD), controversial and conflicting results still exist regarding a reliable assessment/correction method. For proper management it's essential to discriminate between anatomical&functional Leg Length Discrepancy (FLLD). With the newly invented NPoS (New Postural Solution), under the umbrella of the collaboration of PRM Department, Tor Vergata University with Baro Postural Instruments srl, positive results were observed in both measuring& compensating the hemi-pelvic antero-medial rotation in FLLD through personalized bilateral heel raise using two NPoS components: Foot Image System (FIS) and Postural Optimizer System (POS). This led our research interest to test the validity of NPoS as a preliminary step before evaluating its implementations in postural disorders. After clinical evaluation, 4 subjects with FLLD have been assessed by NPoS. Over a period of 2 months, every subject was evaluated 12 times by two different operators, 48 measurements in total, results have been verified in correlation to BTS GaitLab results. Intra-Operator&inter-operator variability analysis showed statistically insignificant differences, while inter-method variability between NPoS and BTS parameters expressed a linear correlation. Results suggest a significant validity of NPoS in assessment&correction of FLLD, with high degree of reproducibility with minimal operator dependency. This can be considered a base for promising clinical implications of NPoS as a reliable cost effective postural assessment/corrective tool. V.

  17. Reliability and Validity of the Turkish Version of the Job Performance Scale Instrument.

    Science.gov (United States)

    Harmanci Seren, Arzu Kader; Tuna, Rujnan; Eskin Bacaksiz, Feride

    2018-02-01

    Objective measurement of the job performance of nursing staff using valid and reliable instruments is important in the evaluation of healthcare quality. A current, valid, and reliable instrument that specifically measures the performance of nurses is required for this purpose. The aim of this study was to determine the validity and reliability of the Turkish version of the Job Performance Instrument. This study used a methodological design and a sample of 240 nurses working at different units in four hospitals in Istanbul, Turkey. A descriptive data form, the Job Performance Scale, and the Employee Performance Scale were used to collect data. Data were analyzed using IBM SPSS Statistics Version 21.0 and LISREL Version 8.51. On the basis of the data analysis, the instrument was revised. Some items were deleted, and subscales were combined. The Turkish version of the Job Performance Instrument was determined to be valid and reliable to measure the performance of nurses. The instrument is suitable for evaluating current nursing roles.

  18. Using Google Earth to conduct a neighborhood audit: reliability of a virtual audit instrument.

    Science.gov (United States)

    Clarke, Philippa; Ailshire, Jennifer; Melendez, Robert; Bader, Michael; Morenoff, Jeffrey

    2010-11-01

    Over the last two decades, the impact of community characteristics on the physical and mental health of residents has emerged as an important frontier of research in population health and health disparities. However, the development and evaluation of measures to capture community characteristics is still at a relatively early stage. The purpose of this work was to assess the reliability of a neighborhood audit instrument administered in the city of Chicago using Google Street View by comparing these "virtual" data to those obtained from an identical instrument administered "in-person". We find that a virtual audit instrument can provide reliable indicators of recreational facilities, the local food environment, and general land use. However, caution should be exercised when trying to gather more finely detailed observations. Using the internet to conduct a neighborhood audit has the potential to significantly reduce the costs of collecting data objectively and unobtrusively. Copyright © 2010 Elsevier Ltd. All rights reserved.

  19. Portfolio assessment during medical internships: How to obtain a reliable and feasible assessment procedure?

    Science.gov (United States)

    Michels, Nele R M; Driessen, Erik W; Muijtjens, Arno M M; Van Gaal, Luc F; Bossaert, Leo L; De Winter, Benedicte Y

    2009-12-01

    A portfolio is used to mentor and assess students' clinical performance at the workplace. However, students and raters often perceive the portfolio as a time-consuming instrument. In this study, we investigated whether assessment during medical internship by a portfolio can combine reliability and feasibility. The domain-oriented reliability of 61 double-rated portfolios was measured, using a generalisability analysis with portfolio tasks and raters as sources of variation in measuring the performance of a student. We obtained reliability (Phi coefficient) of 0.87 with this internship portfolio containing 15 double-rated tasks. The generalisability analysis showed that an acceptable level of reliability (Phi = 0.80) was maintained when the amount of portfolio tasks was decreased to 13 or 9 using one and two raters, respectively. Our study shows that a portfolio can be a reliable method for the assessment of workplace learning. The possibility of reducing the amount of tasks or raters while maintaining a sufficient level of reliability suggests an increase in feasibility of portfolio use for both students and raters.

  20. Can Reliability of Multiple Component Measuring Instruments Depend on Response Option Presentation Mode?

    Science.gov (United States)

    Menold, Natalja; Raykov, Tenko

    2016-01-01

    This article examines the possible dependency of composite reliability on presentation format of the elements of a multi-item measuring instrument. Using empirical data and a recent method for interval estimation of group differences in reliability, we demonstrate that the reliability of an instrument need not be the same when polarity of the…

  1. A Systematic Review of Instruments to Assess Organizational Readiness for Knowledge Translation in Health Care

    Science.gov (United States)

    Gagnon, Marie-Pierre; Attieh, Randa; Ghandour, El Kebir; Légaré, France; Ouimet, Mathieu; Estabrooks, Carole A.; Grimshaw, Jeremy

    2014-01-01

    Background The translation of research into practices has been incomplete. Organizational readiness for change (ORC) is a potential facilitator of effective knowledge translation (KT). However we know little about the best way to assess ORC. Therefore, we sought to systematically review ORC measurement instruments. Methods We searched for published studies in bibliographic databases (Pubmed, Embase, CINAHL, PsychINFO, Web of Science, etc.) up to November 1st, 2012. We included publications that developed ORC measures and/or empirically assessed ORC using an instrument at the organizational level in the health care context. We excluded articles if they did not refer specifically to ORC, did not concern the health care domain or were limited to individual-level change readiness. We focused on identifying the psychometric properties of instruments that were developed to assess readiness in an organization prior to implementing KT interventions in health care. We used the Standards for Educational and Psychological Testing to assess the psychometric properties of identified ORC measurement instruments. Findings We found 26 eligible instruments described in 39 publications. According to the Standards for Educational and Psychological Testing, 18 (69%) of a total of 26 measurement instruments presented both validity and reliability criteria. The Texas Christian University –ORC (TCU-ORC) scale reported the highest instrument validity with a score of 4 out of 4. Only one instrument, namely the Modified Texas Christian University – Director version (TCU-ORC-D), reported a reliability score of 2 out of 3. No information was provided regarding the reliability and validity of five (19%) instruments. Conclusion Our findings indicate that there are few valid and reliable ORC measurement instruments that could be applied to KT in the health care sector. The TCU-ORC instrument presents the best evidence in terms of validity testing. Future studies using this instrument could

  2. Mixed-realism simulation of adverse event disclosure: an educational methodology and assessment instrument.

    Science.gov (United States)

    Matos, Francisco M; Raemer, Daniel B

    2013-04-01

    Physicians have an ethical duty to disclose adverse events to patients or families. Various strategies have been reported for teaching disclosure, but no instruments have been shown to be reliable for assessing them.The aims of this study were to report a structured method for teaching adverse event disclosure using mixed-realism simulation, develop and begin to validate an instrument for assessing performance, and describe the disclosure practice of anesthesiology trainees. Forty-two anesthesiology trainees participated in a 2-part exercise with mixed-realism simulation. The first part took place using a mannequin patient in a simulated operating room where trainees became enmeshed in a clinical episode that led to an adverse event and the second part in a simulated postoperative care unit where the learner is asked to disclose to a standardized patient who systematically moves through epochs of grief response. Two raters scored subjects using an assessment instrument we developed that combines a 4-element behaviorally anchored rating scale (BARS) and a 5-stage objective rating scale. The performance scores for elements within the BARS and the 5-stage instrument showed excellent interrater reliability (Cohen's κ = 0.7), appropriate range (mean range for BARS, 4.20-4.47; mean range for 5-stage instrument, 3.73-4.46), and high internal consistency (P realism simulation that engages learners in an adverse event and allows them to practice disclosure to a structured range of patient responses. We have developed a reliable 2-part instrument with strong psychometric properties for assessing disclosure performance.

  3. Reliability and Validity of Survey Instruments to Measure Work-Related Fatigue in the Emergency Medical Services Setting: A Systematic Review.

    Science.gov (United States)

    Patterson, P Daniel; Weaver, Matthew D; Fabio, Anthony; Teasley, Ellen M; Renn, Megan L; Curtis, Brett R; Matthews, Margaret E; Kroemer, Andrew J; Xun, Xiaoshuang; Bizhanova, Zhadyra; Weiss, Patricia M; Sequeira, Denisse J; Coppler, Patrick J; Lang, Eddy S; Higgins, J Stephen

    2018-02-15

    This study sought to systematically search the literature to identify reliable and valid survey instruments for fatigue measurement in the Emergency Medical Services (EMS) occupational setting. A systematic review study design was used and searched six databases, including one website. The research question guiding the search was developed a priori and registered with the PROSPERO database of systematic reviews: "Are there reliable and valid instruments for measuring fatigue among EMS personnel?" (2016:CRD42016040097). The primary outcome of interest was criterion-related validity. Important outcomes of interest included reliability (e.g., internal consistency), and indicators of sensitivity and specificity. Members of the research team independently screened records from the databases. Full-text articles were evaluated by adapting the Bolster and Rourke system for categorizing findings of systematic reviews, and the rated data abstracted from the body of literature as favorable, unfavorable, mixed/inconclusive, or no impact. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) methodology was used to evaluate the quality of evidence. The search strategy yielded 1,257 unique records. Thirty-four unique experimental and non-experimental studies were determined relevant following full-text review. Nineteen studies reported on the reliability and/or validity of ten different fatigue survey instruments. Eighteen different studies evaluated the reliability and/or validity of four different sleepiness survey instruments. None of the retained studies reported sensitivity or specificity. Evidence quality was rated as very low across all outcomes. In this systematic review, limited evidence of the reliability and validity of 14 different survey instruments to assess the fatigue and/or sleepiness status of EMS personnel and related shift worker groups was identified.

  4. Product analysis and initial reliability testing of the total mesorectal excision-quality assessment instrument.

    Science.gov (United States)

    Simunovic, Marko R; DeNardi, Franco G; Coates, Angela J; Szalay, David A; Eva, Kevin W

    2014-07-01

    Product analysis of rectal cancer resection specimens before specimen fixation may provide an immediate and relevant evaluation of surgical performance. We tested the interrater reliability (IRR) of a product analysis tool called the Total Mesorectal Excision-Quality Assessment Instrument (TME-QA). Participants included two gold standard raters, five pathology assistants, and eight pathologists. Domains of the TME-QA reflect total mesorectal excision principles including: (1) completeness of mesorectal margin; (2) completeness of mesorectum; (3) coning of distal mesorectum; (4) physical defects; and (5) overall specimen quality. Specimens were scored independently. We used the generalizability theory to assess the tool's internal consistency and IRR. There were 39 specimens and 120 ratings. Mean overall specimen quality scores for the gold standard raters, pathologists, and assistants were 4.43, 4.43, and 4.50, respectively (p > 0.85). IRR for the first nine items was 0.68 for the full sample, 0.62 for assistants alone, 0.63 for pathologists alone, and 0.74 for gold standard raters alone. IRR for the item overall specimen quality was 0.67 for the full sample, 0.45 for assistants, 0.80 for pathologists, and 0.86 for gold standard raters. IRR increased for all groups when scores were averaged across two raters. Assessment of surgical specimens using the TME-QA may provide rapid and relevant feedback to surgeons about their technical performance. Our results show good internal consistency and IRR when the TME-QA is used by pathologists. However, for pathology assistants, multiple ratings with the averaging of scores may be needed.

  5. The Toronto outcome measure for craniofacial prosthetics: reliability and validity of a condition-specific quality-of-life instrument.

    Science.gov (United States)

    Anderson, James D; Johnston, Dennis A; Haugh, Gil S; Kiat-Amnuay, Sudarat; Gettleman, Lawrence

    2013-01-01

    The purpose of this study was to refine the Toronto Outcome Measure for Craniofacial Prosthetics (TOMCP), present evidence for its reliability and validity, and use the instrument to explore differences in quality of life between prostheses made with chlorinated polyethylene (CPE) (experimental) and silicone (control). As part of a multicenter prospective controlled randomized double-blind single-crossover clinical trial of the two materials, the TOMCP was administered at the start and end of two 4-month study arms, during which 42 patients wore prostheses made from one material then the other. Reliability was assessed at the crossover. To determine validity of the TOMCP, the Linear Analogue Self-Assessment (LASA-12) and the Short-Form 8 (SF-8) were also administered with the TOMCP. The TOMCP was reduced by removing items that were unreliable, had poorly distributed answers, showed increased internal consistency after their removal, or were too highly correlated with more than one other item. The tests of reliability and validity were then repeated. Finally, the reduced instrument was used to test for differences in quality of life between prostheses made of the two materials. The item reduction tactics pared the 52-item instrument down to 27 items. The correlations of both TOMCP versions with the LASA-12 and the SF-8 were found to be statistically significant, providing evidence of the validity of the TOMCP. The instrument revealed significantly better quality of life with silicone rather than CPE prostheses. Both versions of the TOMCP were found to be reliable and valid. The instrument was able to show differences in quality of life between two materials.

  6. Intelligence Assessment Instruments in Adult Prison Populations: A Systematic Review.

    Science.gov (United States)

    van Esch, A Y M; Denzel, A D; Scherder, E J A; Masthoff, E D M

    2017-10-01

    Detection of intellectual disability (ID) in the penitentiary system is important for the following reasons: (a) to provide assistance to people with ID in understanding their legal rights and court proceedings; (b) to facilitate rehabilitation programs tailored to ID patients, which improves the enhancement of their quality of life and reduces their risk of reoffending; and (c) to provide a reliable estimate of the risk of offence recidivism. It requires a short assessment instrument that provides a reliable estimation of a person's intellectual functioning at the earliest possible stage of this process. The aim of this systematic review is (a) to provide an overview of recent short assessment instruments that provide a full-scale IQ score in adult prison populations and (b) to achieve a quality measurement of the validation studies regarding these instruments to determine which tests are most feasible in this target population. The Preferred Reporting Items for Systematic reviews and Meta-Analyses Statement is used to ensure reliability. The Satz-Mögel, an item-reduction short form of the Wechsler Adult Intelligence Scale, shows the highest correlation with the golden standard and is described to be most reliable. Nevertheless, when it comes to applicability in prison populations, the shorter and less verbal Quick Test can be preferred over others. Without affecting these conclusions, major limitations emerge from the present systematic review, which give rise to several important recommendations for further research.

  7. Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology.

    Science.gov (United States)

    Cicchetti, Domenic V.

    1994-01-01

    In the context of developing assessment instruments in psychology, issues of standardization, norming procedures, and test reliability and validity are discussed. Criteria, guidelines, and rules of thumb are provided to help the clinician with instrument selection for a given psychological assessment. (SLD)

  8. Development of the evaluation instrument use CIPP on the implementation of project assessment topic optik

    Science.gov (United States)

    Asfaroh, Jati Aurum; Rosana, Dadan; Supahar

    2017-08-01

    This research aims to develop an evaluation instrument models CIPP valid and reliable as well as determine the feasibility and practicality of an evaluation instrument models CIPP. An evaluation instrument models CIPP to evaluate the implementation of the project assessment topic optik to measure problem-solving skills of junior high school class VIII in the Yogyakarta region. This research is a model of development that uses 4-D. Subject of product trials are students in class VIII SMP N 1 Galur and SMP N 1 Sleman. Data collection techniques in this research using non-test techniques include interviews, questionnaires and observations. Validity in this research was analyzed using V'Aikens. Reliability analyzed using ICC. This research uses 7 raters are derived from two lecturers expert (expert judgment), two practitioners (science teacher) and three colleagues. The results of this research is the evaluation's instrument model of CIPP is used to evaluate the implementation of the implementation of the project assessment instruments. The validity result of evaluation instrument have V'Aikens values between 0.86 to 1, which means a valid and 0.836 reliability values into categories so well that it has been worth used as an evaluation instrument.

  9. The evaluation of equipment and Instrumentation Reliability Factors on Power Reactor

    International Nuclear Information System (INIS)

    Supriatna, Piping; Karlina, Itjeu; Widagdo, Suharyo; Santosa, Kussigit; Darlis; Sudiyono, Bambang; Yuniyanta, Sasongko; Sudarmin

    1999-01-01

    Equipment and instrumentation reliability on type power reactor control room was determined by its pattern and design. the principle of ergonomy applied on equipment and instrumentation layout in this ABWR type reactor are geometric pattern appropriate with economic body motion, average anthropometry data of operator especially operator hand-reach, range of vision, angle of vision, lighting, color arrangement and harmony as will as operator case in operating the equipment system. Limitation criteria of the parameter mentioned above are based on EPRI NP-3659, NURG 0700, and NUREG/CR-3331 documents. Besides that, the (working) physical environment parameter factor of the control room must be designed in order to fulfil the standard criteria of ergonomic condition based on NUREG-0800. The reliability evaluation of equipment and instrumentation system also occurs observed from man machine interaction side which happen between operator and equipment and instrumentation in the ABWR type power reactor control room. From the MMI analysis can be known the working failure possibility which is caused by the operator. The evaluation result of equipment and instrumentation reliability on ABWR type power reactor control room showed that the design of this ABWR control room is good and fulfils the ergonomy standard criteria have been determined

  10. Clinical reliability and validity of elbow functional assessment in rheumatoid arthritis.

    NARCIS (Netherlands)

    Boer, Y.A. de; Ende, C.H.M. van den; Eygendaal, D.; Jolie, I.M.M.; Hazes, J.M.W.; Rozing, P.M.

    1999-01-01

    OBJECTIVES: (1) To investigate the measurement characteristics of the Hospital for Special Surgery (HSS) and Mayo Clinic elbow assessment instruments, utilizing methodological criteria including feasibility, reliability, validity, and discriminative ability; and (2) to develop an efficient and

  11. Instruments to assess self-care among healthy children: A systematic review of measurement properties.

    Science.gov (United States)

    Urpí-Fernández, Ana-María; Zabaleta-Del-Olmo, Edurne; Montes-Hidalgo, Javier; Tomás-Sábado, Joaquín; Roldán-Merino, Juan-Francisco; Lluch-Canut, María-Teresa

    2017-12-01

    To identify, critically appraise and summarize the measurement properties of instruments to assess self-care in healthy children. Assessing self-care is a proper consideration for nursing practice and nursing research. No systematic review summarizes instruments of measurement validated in healthy children. Psychometric review in accordance with the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) panel. MEDLINE, CINAHL, PsycINFO, Web of Science and Open Grey were searched from their inception to December 2016. Validation studies with a healthy child population were included. Search was not restricted by language. Two reviewers independently assessed the methodological quality of included studies using the COSMIN checklist. Eleven studies were included in the review assessing the measurement properties of ten instruments. There was a maximum of two studies per instrument. None of the studies evaluated the properties of test-retest reliability, measurement error, criterion validity and responsiveness. Internal consistency and structural validity were rated as "excellent" or "good" in four studies. Four studies were rated as "excellent" in content validity. Cross-cultural validity was rated as "poor" in the two studies (three instruments) which cultural adaptation was carried out. The evidence available does not allow firm conclusions about the instruments identified in terms of reliability and validity. Future research should focus on generate evidence about a wider range of measurement properties of these instruments using a rigorous methodology, as well as instrument testing on different countries and child population. © 2017 John Wiley & Sons Ltd.

  12. Validity and reliability of a Malay version of the Lawton instrumental activities of daily living scale among the Malay speaking elderly in Malaysia.

    Science.gov (United States)

    Kadar, Masne; Ibrahim, Suhaili; Razaob, Nor Afifi; Chai, Siaw Chui; Harun, Dzalani

    2018-02-01

    The Lawton Instrumental Activities of Daily Living Scale is a tool often used to assess independence among elderly at home. Its suitability to be used with the elderly population in Malaysia has not been validated. This current study aimed to assess the validity and reliability of the Lawton Instrumental Activities of Daily Living Scale - Malay Version to Malay speaking elderly in Malaysia. This study was divided into three phases: (1) translation and linguistic validity involving both forward and backward translations; (2) establishment of face validity and content validity; and (3) establishment of reliability involving inter-rater, test-retest and internal consistency analyses. Data used for these analyses were obtained by interviewing 65 elderly respondents. Percentages of Content Validity Index for 4 criteria were from 88.89 to 100.0. The Cronbach α coefficient for internal consistency was 0.838. Intra-class Correlation Coefficient of inter-rater reliability and test-retest reliability was 0.957 and 0.950 respectively. The result shows that the Lawton Instrumental Activities of Daily Living Scale - Malay Version has excellent reliability and validity for use with the Malay speaking elderly people in Malaysia. This scale could be used by professionals to assess functional ability of elderly who live independently in community. © 2018 Occupational Therapy Australia.

  13. Analysis of the Reliability and Validity of a Mentor's Assessment for Principal Internships

    Science.gov (United States)

    Koonce, Glenn L.; Kelly, Michael D.

    2014-01-01

    In this study, researchers analyzed the reliability and validity of the mentor's assessment for principal internships at a university in the Southeast region of the United States. The results of the study yielded how trustworthy and dependable the instrument is and the effectiveness of the instrument in the current principal preparation program.…

  14. Development of a Standardized Kalamazoo Communication Skills Assessment Tool for Radiologists: Validation, Multisource Reliability, and Lessons Learned.

    Science.gov (United States)

    Brown, Stephen D; Rider, Elizabeth A; Jamieson, Katherine; Meyer, Elaine C; Callahan, Michael J; DeBenedectis, Carolynn M; Bixby, Sarah D; Walters, Michele; Forman, Sara F; Varrin, Pamela H; Forbes, Peter; Roussin, Christopher J

    2017-08-01

    The purpose of this study was to develop and test a standardized communication skills assessment instrument for radiology. The Delphi method was used to validate the Kalamazoo Communication Skills Assessment instrument for radiology by revising and achieving consensus on the 43 items of the preexisting instrument among an interdisciplinary team of experts consisting of five radiologists and four nonradiologists (two men, seven women). Reviewers assessed the applicability of the instrument to evaluation of conversations between radiology trainees and trained actors portraying concerned parents in enactments about bad news, radiation risks, and diagnostic errors that were video recorded during a communication workshop. Interrater reliability was assessed by use of the revised instrument to rate a series of enactments between trainees and actors video recorded in a hospital-based simulator center. Eight raters evaluated each of seven different video-recorded interactions between physicians and parent-actors. The final instrument contained 43 items. After three review rounds, 42 of 43 (98%) items had an average rating of relevant or very relevant for bad news conversations. All items were rated as relevant or very relevant for conversations about error disclosure and radiation risk. Reliability and rater agreement measures were moderate. The intraclass correlation coefficient range was 0.07-0.58; mean, 0.30; SD, 0.13; and median, 0.30. The range of weighted kappa values was 0.03-0.47; mean, 0.23; SD, 0.12; and median, 0.22. Ratings varied significantly among conversations (χ 2 6 = 1186; p communication skills assessment instrument is highly relevant for radiology, having moderate interrater reliability. These findings have important implications for assessing the relational competencies of radiology trainees.

  15. Reliability and validity of instruments measuring job satisfaction - a systematic review

    NARCIS (Netherlands)

    van Saane, N.; Sluiter, J. K.; Verbeek, J. H. A. M.; Frings-Dresen, M. H. W.

    2003-01-01

    Background Although job satisfaction research has been carried out for decades, no recent overview of job satisfaction instruments and their quality is available. Aim The aim of this systematic review is to select job satisfaction instruments of adequate reliability and validity for use as

  16. Reliability of the Q Force; a mobile instrument for measuring isometric quadriceps muscle strength.

    Science.gov (United States)

    Douma, K W; Regterschot, G R H; Krijnen, W P; Slager, G E C; van der Schans, C P; Zijlstra, W

    2016-01-01

    The ability to generate muscle strength is a pre-requisite for all human movement. Decreased quadriceps muscle strength is frequently observed in older adults and is associated with a decreased performance and activity limitations. To quantify the quadriceps muscle strength and to monitor changes over time, instruments and procedures with a sufficient reliability are needed. The Q Force is an innovative mobile muscle strength measurement instrument suitable to measure in various degrees of extension. Measurements between 110 and 130° extension present the highest values and the most significant increase after training. The objective of this study is to determine the test-retest reliability of muscle strength measurements by the Q Force in older adults in 110° extension. Forty-one healthy older adults, 13 males and 28 females were included in the study. Mean (SD) age was 81.9 (4.89) years. Isometric muscle strength of the Quadriceps muscle was assessed with the Q Force at 110° of knee extension. Participants were measured at two sessions with a three to eight day interval between sessions. To determine relative reliability, the intraclass correlation coefficient (ICC) was calculated. To determine absolute reliability, Bland and Altman Limits of Agreement (LOA) were calculated and t-tests were performed. Relative reliability of the Q Force is good to excellent as all ICC coefficients are higher than 0.75. Generally a large 95 % LOA, reflecting only moderate absolute reliability, is found as exemplified for the peak torque left leg of -18.6 N to 33.8 N and the right leg of -9.2 N to 26.4 N was between 15.7 and 23.6 Newton representing 25.2 % to 39.9 % of the size of the mean. Small systematic differences in mean were found between measurement session 1 and 2. The present study shows that the Q Force has excellent relative test-retest reliability, but limited absolute test-retest reliability. Since the Q Force is relatively cheap and mobile it is suitable for

  17. A questionnaire for assessing breastfeeding intentions and practices in Nigeria: validity, reliability and translation.

    Science.gov (United States)

    Emmanuel, Andy; Clow, Sheila E

    2017-06-07

    Validating a questionnaire/instrument (whether developed or adapted) before proceeding to the field for data collection is important. This article presents the modification of an Irish questionnaire for a Nigerian setting. The validation process and reliability testing of this questionnaire (which was used in assessing previous breastfeeding practices and breastfeeding intentions of pregnant women in English and Hausa languages) were also presented. Five experts in the field of breastfeeding and infant feeding voluntarily and independently evaluated the instrument. The experts evaluated the various items of the questionnaire based on relevance, clarity, simplicity and ambiguity on a Likert scale of 4. The analysis was performed to determine the content validity index (CVI).Two language experts performed the translation and back-translation. Ten pregnant women completed questionnaires which were evaluated for internal consistency. Two other pregnant women completed the questionnaire twice at an interval of two weeks to test the reliability. SPSS version 21 was used to calculate the coefficient of reliability. The content validity index was high (0.94 for relevance, clarity and ambiguity and 0.96 for simplicity). The analysis suggested that four of the seventy one items should be removed. Cronbach's Alpha was 0.81, while the reliability coefficient was 0.76. The emerged validated questionnaire was translated from English to Hausa, then, back-translated into English and compared for accuracy. The final instrument is reliable and valid for data collection on breastfeeding in Nigeria among English and Hausa speakers. Therefore, the instrument is recommended for use in assessing breastfeeding intention and practices in Nigeria.

  18. The Construct Validity and Reliability of an Assessment Tool for Competency in Cochlear Implant Surgery

    Directory of Open Access Journals (Sweden)

    Patorn Piromchai

    2014-01-01

    Full Text Available Introduction. We introduce a rating tool that objectively evaluates the skills of surgical trainees performing cochlear implant surgery. Methods. Seven residents and seven experts performed cochlear implant surgery sessions from mastoidectomy to cochleostomy on a standardized virtual reality temporal bone. A total of twenty-eight assessment videos were recorded and two consultant otolaryngologists evaluated the performance of each participant using these videos. Results. Interrater reliability was calculated using the intraclass correlation coefficient for both the global and checklist components of the assessment instrument. The overall agreement was high. The construct validity of this instrument was strongly supported by the significantly higher scores in the expert group for both components. Conclusion. Our results indicate that the proposed assessment tool for cochlear implant surgery is reliable, accurate, and easy to use. This instrument can thus be used to provide objective feedback on overall and task-specific competency in cochlear implantation.

  19. Reliability and validity evidence of the Assessment of Language Use in Social Contexts for Adults (ALUSCA).

    Science.gov (United States)

    Valente, Ana Rita S; Hall, Andreia; Alvelos, Helena; Leahy, Margaret; Jesus, Luis M T

    2018-04-12

    The appropriate use of language in context depends on the speaker's pragmatic language competencies. A coding system was used to develop a specific and adult-focused self-administered questionnaire to adults who stutter and adults who do not stutter, The Assessment of Language Use in Social Contexts for Adults, with three categories: precursors, basic exchanges, and extended literal/non-literal discourse. This paper presents the content validity, item analysis, reliability coefficients and evidences of construct validity of the instrument. Content validity analysis was based on a two-stage process: first, 11 pragmatic questionnaires were assessed to identify items that probe each pragmatic competency and to create the first version of the instrument; second, items were assessed qualitatively by an expert panel composed by adults who stutter and controls, and quantitatively and qualitatively by an expert panel composed by clinicians. A pilot study was conducted with five adults who stutter and five controls to analyse items and calculate reliability. Construct validity evidences were obtained using the hypothesized relationships method and factor analysis with 28 adults who stutter and 28 controls. Concerning content validity, the questionnaires assessed up to 13 pragmatic competencies. Qualitative and quantitative analysis revealed ambiguities in items construction. Disagreement between experts was solved through item modification. The pilot study showed that the instrument presented internal consistency and temporal stability. Significant differences between adults who stutter and controls and different response profiles revealed the instrument's underlying construct. The instrument is reliable and presented evidences of construct validity.

  20. Low-Budget Instrumentation of a Conventional Leg Press to Measure Reliable Isometric-Strength Capacity.

    Science.gov (United States)

    Baur, Heiner; Groppa, Alessia Severina; Limacher, Regula; Radlinger, Lorenz

    2016-02-02

    Maximum strength and rate of force development (RFD) are 2 important strength characteristics for everyday tasks and athletic performance. Measurements of both parameters must be reliable. Expensive isokinetic devices with isometric modes are often used. The possibility of cost-effective measurements in a practical setting would facilitate quality control. The purpose of this study was to assess the reliability of measurements of maximum isometric strength (Fmax) and RFD on a conventional leg press. Sixteen subjects (23 ± 2 y, 1.68 ± 0.05 m, 59 ± 5 kg) were tested twice within 1 session. After warm-up, subjects performed 2 times 5 trials eliciting maximum voluntary isometric contractions on an instrumented leg press (1- and 2-legged randomized). Fmax (N) and RFD (N/s) were extracted from force-time curves. Reliability was determined for Fmax and RFD by calculating the intraclass correlation coefficient (ICC), the test-retest variability (TRV), and the bias and limits of agreement. Reliability measures revealed good to excellent ICCs of .80-.93. TRV showed mean differences between measurement sessions of 0.4-6.9%. The systematic error was low compared with the absolute mean values (Fmax 5-6%, RFD 1-4%). The implementation of a force transducer into a conventional leg press provides a viable procedure to assess Fmax and RFD. Both performance parameters can be assessed with good to excellent reliability allowing quality control of interventions.

  1. Validity of instruments to assess students' travel and pedestrian safety.

    Science.gov (United States)

    Mendoza, Jason A; Watson, Kathy; Baranowski, Tom; Nicklas, Theresa A; Uscanga, Doris K; Hanfling, Marcus J

    2010-05-18

    Safe Routes to School (SRTS) programs are designed to make walking and bicycling to school safe and accessible for children. Despite their growing popularity, few validated measures exist for assessing important outcomes such as type of student transport or pedestrian safety behaviors. This research validated the SRTS school travel survey and a pedestrian safety behavior checklist. Fourth grade students completed a brief written survey on how they got to school that day with set responses. Test-retest reliability was obtained 3-4 hours apart. Convergent validity of the SRTS travel survey was assessed by comparison to parents' report. For the measure of pedestrian safety behavior, 10 research assistants observed 29 students at a school intersection for completion of 8 selected pedestrian safety behaviors. Reliability was determined in two ways: correlations between the research assistants' ratings to that of the Principal Investigator (PI) and intraclass correlations (ICC) across research assistant ratings. The SRTS travel survey had high test-retest reliability (kappa = 0.97, n = 96, p < 0.001) and convergent validity (kappa = 0.87, n = 81, p < 0.001). The pedestrian safety behavior checklist had moderate reliability across research assistants' ratings (ICC = 0.48) and moderate correlation with the PI (r = 0.55, p = < 0.01). When two raters simultaneously used the instrument, the ICC increased to 0.65. Overall percent agreement (91%), sensitivity (85%) and specificity (83%) were acceptable. These validated instruments can be used to assess SRTS programs. The pedestrian safety behavior checklist may benefit from further formative work.

  2. Quantitative assessment of probability of failing safely for the safety instrumented system using reliability block diagram method

    International Nuclear Information System (INIS)

    Jin, Jianghong; Pang, Lei; Zhao, Shoutang; Hu, Bin

    2015-01-01

    Highlights: • Models of PFS for SIS were established by using the reliability block diagram. • The more accurate calculation of PFS for SIS can be acquired by using SL. • Degraded operation of complex SIS does not affect the availability of SIS. • The safe undetected failure is the largest contribution to the PFS of SIS. - Abstract: The spurious trip of safety instrumented system (SIS) brings great economic losses to production. How to ensure the safety instrumented system is reliable and available has been put on the schedule. But the existing models on spurious trip rate (STR) or probability of failing safely (PFS) are too simplified and not accurate, in-depth studies of availability to obtain more accurate PFS for SIS are required. Based on the analysis of factors that influence the PFS for the SIS, using reliability block diagram method (RBD), the quantitative study of PFS for the SIS is carried out, and gives some application examples. The results show that, the common cause failure will increase the PFS; degraded operation does not affect the availability of the SIS; if the equipment was tested and repaired one by one, the unavailability of the SIS can be ignored; the corresponding occurrence time of independent safe undetected failure should be the system lifecycle (SL) rather than the proof test interval and the independent safe undetected failure is the largest contribution to the PFS for the SIS

  3. The psychometric properties of a shortened corporate entrepreneurship assessment instrument

    Directory of Open Access Journals (Sweden)

    Renier Steyn

    2017-08-01

    Aim: The aim of this research was to evaluate the psychometric properties of a measure of entrepreneurial climate. Entrepreneurial climate was measured using a shortened version of the Hornsby, Kuratko and Zahra (2002 instrument, called the Corporate Entrepreneurship Assessment Instrument (CEAI. Making information on the psychometric properties of the instrument available directly relates to its utility. Setting: The setting was medium to large South African companies. A random sample of employees was drawn from 53 selected companies across South Africa, with 60 respondents per company (N = 3 180. Methods: A cross-sectional survey design was used. Several instruments were administered, including the shortened version of the CEAI. Cronbach’s alpha was used to test for reliability and several methods were used to test for validity. Correlation analysis was used to test for concurrent validity, convergent validity and divergent validity. Principle component factor analysis was used to test for factorial validity and a t-test to test for known-group validity. Results: The results showed that the reliability for the total score of the shortened version of the CEAI was acceptable at 0.758. The results also showed some evidence of concurrent validity, as well as homogeneity among the items. With regard to factorial validity, all items loaded in accordance with the subscales of the instrument. The measure was able to distinguish, as expected, between government organisations and private business entities, suggesting known-group validity. Convergent validity and divergent validity were also assessed. Interesting to note was that entrepreneurship climate correlates more with general employee attitude (e.g. employee engagement; R= 0.420, p < 0.001 and organisational commitment, R = 0.331, p < 0.001 than with self-reported innovation (R = 0.277, p < 0.001 and R = 0.267, p < 0.001. Contribution: This paper not only provided information on the reliability

  4. Blinded evaluation of interrater reliability of an operative competency assessment tool for direct laryngoscopy and rigid bronchoscopy.

    Science.gov (United States)

    Ishman, Stacey L; Benke, James R; Johnson, Kaalan Erik; Zur, Karen B; Jacobs, Ian N; Thorne, Marc C; Brown, David J; Lin, Sandra Y; Bhatti, Nasir; Deutsch, Ellen S

    2012-10-01

    OBJECTIVES To confirm interrater reliability using blinded evaluation of a skills-assessment instrument to assess the surgical performance of resident and fellow trainees performing pediatric direct laryngoscopy and rigid bronchoscopy in simulated models. DESIGN Prospective, paired, blinded observational validation study. SUBJECTS Paired observers from multiple institutions simultaneously evaluated residents and fellows who were performing surgery in an animal laboratory or using high-fidelity manikins. The evaluators had no previous affiliation with the residents and fellows and did not know their year of training. INTERVENTIONS One- and 2-page versions of an objective structured assessment of technical skills (OSATS) assessment instrument composed of global and a task-specific surgical items were used to evaluate surgical performance. RESULTS Fifty-two evaluations were completed by 17 attending evaluators. The instrument agreement for the 2-page assessment was 71.4% when measured as a binary variable (ie, competent vs not competent) (κ = 0.38; P = .08). Evaluation as a continuous variable revealed a 42.9% percentage agreement (κ = 0.18; P = .14). The intraclass correlation was 0.53, considered substantial/good interrater reliability (69% reliable). For the 1-page instrument, agreement was 77.4% when measured as a binary variable (κ = 0.53, P = .0015). Agreement when evaluated as a continuous measure was 71.0% (κ = 0.54, P formative feedback on operational competency.

  5. Adaptation and testing of psychosocial assessment instruments for cross-cultural use: an example from the Thailand Burma border.

    Science.gov (United States)

    Haroz, Emily E; Bass, Judith K; Lee, Catherine; Murray, Laura K; Robinson, Courtland; Bolton, Paul

    2014-01-01

    The purpose of this study was to develop valid and reliable instruments to assess priority psychosocial problems and functioning among adult survivors of systematic violence from Burma living in Thailand. The process involved four steps: 1) instrument drafting and piloting; 2) reliability and validity testing; 3) instrument revision; and 4) retesting revised instrument. A total of N = 158 interviews were completed. Overall subscales showed good internal consistency (0.73-0.92) and satisfactory combined test-retest/inter rater reliability (0.63-0.84). Criterion validity, was not demonstrated for any scale. The alcohol and functioning scales underperformed and were revised (step 3) and retested (step 4). Upon retesting, the function scale showed good internal consistency reliability (0.91-0.92), and the alcohol scale showed acceptable internal consistency (0.79) and strong test-retest/inter-rater reliability (0.86-0.89). This paper describes the importance and process of adaptation and testing, illustrated by the experiences and results for selected instruments in this population.

  6. Assessment of Equipment Capability to Perform Reliably under Severe Accident Conditions

    International Nuclear Information System (INIS)

    2017-07-01

    The experience from the last 40 years has shown that severe accidents can subject electrical and instrumentation and control (I&C) equipment to environmental conditions exceeding the equipment’s original design basis assumptions. Severe accident conditions can then cause rapid degradation or damage to various degrees up to complete failure of such equipment. This publication provides the technical basis to consider when assessing the capability of electrical and I&C equipment to perform reliably during a severe accident. It provides examples of calculation tools to determine the environmental parameters as well as examples and methods that Member States can apply to assess equipment reliability.

  7. Investigating the Validity and Reliability of the Vanderbilt Assessment of Leadership in Education

    Science.gov (United States)

    Porter, Andrew C.; Polikoff, Morgan S.; Goldring, Ellen B.; Murphy, Joseph; Elliott, Stephen N.; May, Henry

    2010-01-01

    The Vanderbilt Assessment of Leadership in Education (VAL-ED) is a multirater assessment of principals' learning-centered leadership. The instrument was developed based on the Standards for Educational and Psychological Testing. In this article, we report on the validity and reliability evidence for the VAL-ED accumulated in a national field…

  8. OSS reliability measurement and assessment

    CERN Document Server

    Yamada, Shigeru

    2016-01-01

    This book analyses quantitative open source software (OSS) reliability assessment and its applications, focusing on three major topic areas: the Fundamentals of OSS Quality/Reliability Measurement and Assessment; the Practical Applications of OSS Reliability Modelling; and Recent Developments in OSS Reliability Modelling. Offering an ideal reference guide for graduate students and researchers in reliability for open source software (OSS) and modelling, the book introduces several methods of reliability assessment for OSS including component-oriented reliability analysis based on analytic hierarchy process (AHP), analytic network process (ANP), and non-homogeneous Poisson process (NHPP) models, the stochastic differential equation models and hazard rate models. These measurement and management technologies are essential to producing and maintaining quality/reliable systems using OSS.

  9. Development of Assessment Instrument of Critical Thinking in Physics at Senior High School

    Science.gov (United States)

    Sugiarti, T.; Kaniawati, I.; Aviyanti, L.

    2017-02-01

    The result of preliminary study shows that the assessment of physics in school did not train students’ critical thinking skill. The assessment instrument just measured low cognitive aspects. Supposedly, critical thinking skill is trained in the assessment activity. The study aims to determine the characteristics and the quality of critical thinking skill instrument. It employs descriptive-qualitative method with research and development as the research design. The research participants are 35 students involved in the limited trial and 188 students in the wider trial from three public senior high school in Ciamis which in high level school. The data was collected through expert validation, tests and interviews. The results indicate that the characteristics of the assessment instrument of critical thinking skill is open-ended. The instrument fulfills some indicators namely analyzing argument, deduction, induction, and display information in the form of scenario, text, graphic and table. In addition, the data processing through V4 Anates program shows that the instrument reliability achieves 0.67 with high interpretation of 0.67 and the validity is 0.47 with enough interpretation. Thus, the assessment instrument of critical thinking skill in the form of open-ended essay meets the criteria of quality test, so it can use as instrument of assessment critical thinking skill.

  10. RICE: A Reliable and Efficient Remote Instrumentation Collaboration Environment

    Directory of Open Access Journals (Sweden)

    Prasad Calyam

    2008-01-01

    Full Text Available Remote access of scientific instruments over the Internet (i.e., remote instrumentation demand high-resolution (2D and 3D video image transfers with simultaneous real-time mouse and keyboard controls. Consequently, user quality of experience (QoE is highly sensitive to network bottlenecks. Further, improper user control while reacting to impaired video caused due to network bottlenecks could result in physical damages to the expensive instrument equipment. Hence, it is vital to understand the interplay between (a user keyboard/mouse actions toward the instrument, and (b corresponding network reactions for transfer of instrument video images toward the user. In this paper, we first present an analytical model for characterizing user and network interplay during remote instrumentation sessions in terms of demand and supply interplay principles of traditional economics. Next, we describe the trends of the model parameters using subjective and objective measurements obtained from QoE experiments. Thereafter, we describe our Remote Instrumentation Collaboration Environment (RICE software that leverages our experiences from the user and network interplay studies, and has functionalities that facilitate reliable and efficient remote instrumentation such as (a network health awareness to detect network bottleneck periods, and (b collaboration tools for multiple participants to interact during research and training sessions.

  11. Evaluation of the McMahon Competence Assessment Instrument for Use with Midwifery Students During a Simulated Shoulder Dystocia.

    Science.gov (United States)

    McMahon, Erin; Jevitt, Cecilia; Aronson, Barbara

    2018-03-01

    Intrapartum emergencies occur infrequently but require a prompt and competent response from the midwife to prevent morbidity and mortality of the woman, fetus, and newborn. Simulation provides the opportunity for student midwives to develop competence in a safe environment. The purpose of this study was to determine the inter-rater reliability of the McMahon Competence Assessment Instrument (MCAI) for use with student midwives during a simulated shoulder dystocia scenario. A pilot study using a nonprobability convenience sample was used to evaluate the MCAI. Content validity indices were calculated for the individual items and the overall instrument using data from a panel of expert reviewers. Fourteen student midwives consented to be video recorded while participating in a simulated shoulder dystocia scenario. Three faculty raters used the MCAI to evaluate the student performance. These quantitative data were used to determine the inter-rater reliability of the MCAI. The intraclass correlation coefficient (ICC) was used to assess the inter-rater reliability of MCAI scores between 2 or more raters. The ICC was 0.86 (95% confidence interval, 0.60-0.96). Fleiss's kappa was calculated to determine the inter-rater reliability for individual items. Twenty-three of the 42 items corresponded to excellent strength of agreement. This study demonstrates a method to determine the inter-rater reliability of a competence assessment instrument to be used with student midwives. Data produced by this study were used to revise and improve the instrument. Additional research will further document the inter-rater reliability and can be used to determine changes in student competence. Valid and reliable methods of assessment will encourage the use of simulation to efficiently develop the competence of student midwives. © 2018 by the American College of Nurse-Midwives.

  12. Assessment of Patient-Reported Outcome Instruments to Assess Chronic Low Back Pain.

    Science.gov (United States)

    Ramasamy, Abhilasha; Martin, Mona L; Blum, Steven I; Liedgens, Hiltrud; Argoff, Charles; Freynhagen, Rainer; Wallace, Mark; McCarrier, Kelly P; Bushnell, Donald M; Hatley, Noël V; Patrick, Donald L

    2017-06-01

     To identify patient-reported outcome (PRO) instruments that assess chronic low back pain (cLBP) symptoms (specifically pain qualities) and/or impacts for potential use in cLBP clinical trials to demonstrate treatment benefit and support labeling claims.  Literature review of existing PRO measures.  Publications detailing existing PRO measures for cLBP were identified, reviewed, and summarized. As recommended by the US Food & Drug Administration (FDA) PRO development guidance, standard measurement characteristics were reviewed, including development history, psychometric properties (validity and reliability), ability to detect change, and interpretation of observed changes.  Thirteen instruments were selected and reviewed: Low Back Pain Bothersomeness Scale, Neuropathic Pain Symptom Inventory, PainDETECT, Pain Quality Assessment Scale Revised, Revised Short Form McGill Pain Questionnaire, Low Back Pain Impact Questionnaire, Oswestry Disability Index, Pain Disability Index, Roland-Morris Disability Questionnaire, Brief Pain Inventory and Brief Pain Inventory Short Form, Musculoskeletal Outcomes Data Evaluation and Management System Spine Module, Orebro Musculoskeletal Pain Questionnaire, and the West Haven-Yale Multidimensional Pain Inventory Interference Scale. The instruments varied in the aspects of pain and/or impacts that they assessed, and none of the instruments fulfilled all criteria for use in clinical trials to support labeling claims based on recommendations outlined in the FDA PRO guidance.  There is an unmet need for a validated PRO instrument to evaluate cLBP-related symptoms and impacts for use in clinical trials. © 2017 American Academy of Pain Medicine. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com

  13. Development and Psychometric Evaluation of an Instrument to Assess Cross-Cultural Competence of Healthcare Professionals (CCCHP)

    Science.gov (United States)

    Bernhard, Gerda; Knibbe, Ronald A.; von Wolff, Alessa; Dingoyan, Demet; Schulz, Holger; Mösko, Mike

    2015-01-01

    Background Cultural competence of healthcare professionals (HCPs) is recognized as a strategy to reduce cultural disparities in healthcare. However, standardised, valid and reliable instruments to assess HCPs’ cultural competence are notably lacking. The present study aims to 1) identify the core components of cultural competence from a healthcare perspective, 2) to develop a self-report instrument to assess cultural competence of HCPs and 3) to evaluate the psychometric properties of the new instrument. Methods The conceptual model and initial item pool, which were applied to the cross-cultural competence instrument for the healthcare profession (CCCHP), were derived from an expert survey (n = 23), interviews with HCPs (n = 12), and a broad narrative review on assessment instruments and conceptual models of cultural competence. The item pool was reduced systematically, which resulted in a 59-item instrument. A sample of 336 psychologists, in advanced psychotherapeutic training, and 409 medical students participated, in order to evaluate the construct validity and reliability of the CCCHP. Results Construct validity was supported by principal component analysis, which led to a 32-item six-component solution with 50% of the total variance explained. The different dimensions of HCPs’ cultural competence are: Cross-Cultural Motivation/Curiosity, Cross-Cultural Attitudes, Cross-Cultural Skills, Cross-Cultural Knowledge/Awareness and Cross-Cultural Emotions/Empathy. For the total instrument, the internal consistency reliability was .87 and the dimension’s Cronbach’s α ranged from .54 to .84. The discriminating power of the CCCHP was indicated by statistically significant mean differences in CCCHP subscale scores between predefined groups. Conclusions The 32-item CCCHP exhibits acceptable psychometric properties, particularly content and construct validity to examine HCPs’ cultural competence. The CCCHP with its five dimensions offers a comprehensive

  14. Development and Psychometric Evaluation of an Instrument to Assess Cross-Cultural Competence of Healthcare Professionals (CCCHP).

    Science.gov (United States)

    Bernhard, Gerda; Knibbe, Ronald A; von Wolff, Alessa; Dingoyan, Demet; Schulz, Holger; Mösko, Mike

    2015-01-01

    Cultural competence of healthcare professionals (HCPs) is recognized as a strategy to reduce cultural disparities in healthcare. However, standardised, valid and reliable instruments to assess HCPs' cultural competence are notably lacking. The present study aims to 1) identify the core components of cultural competence from a healthcare perspective, 2) to develop a self-report instrument to assess cultural competence of HCPs and 3) to evaluate the psychometric properties of the new instrument. The conceptual model and initial item pool, which were applied to the cross-cultural competence instrument for the healthcare profession (CCCHP), were derived from an expert survey (n = 23), interviews with HCPs (n = 12), and a broad narrative review on assessment instruments and conceptual models of cultural competence. The item pool was reduced systematically, which resulted in a 59-item instrument. A sample of 336 psychologists, in advanced psychotherapeutic training, and 409 medical students participated, in order to evaluate the construct validity and reliability of the CCCHP. Construct validity was supported by principal component analysis, which led to a 32-item six-component solution with 50% of the total variance explained. The different dimensions of HCPs' cultural competence are: Cross-Cultural Motivation/Curiosity, Cross-Cultural Attitudes, Cross-Cultural Skills, Cross-Cultural Knowledge/Awareness and Cross-Cultural Emotions/Empathy. For the total instrument, the internal consistency reliability was .87 and the dimension's Cronbach's α ranged from .54 to .84. The discriminating power of the CCCHP was indicated by statistically significant mean differences in CCCHP subscale scores between predefined groups. The 32-item CCCHP exhibits acceptable psychometric properties, particularly content and construct validity to examine HCPs' cultural competence. The CCCHP with its five dimensions offers a comprehensive assessment of HCPs' cultural competence, and has the

  15. A Systematic Review of Statistical Methods Used to Test for Reliability of Medical Instruments Measuring Continuous Variables

    Directory of Open Access Journals (Sweden)

    Rafdzah Zaki

    2013-06-01

    Full Text Available   Objective(s: Reliability measures precision or the extent to which test results can be replicated. This is the first ever systematic review to identify statistical methods used to measure reliability of equipment measuring continuous variables. This studyalso aims to highlight the inappropriate statistical method used in the reliability analysis and its implication in the medical practice.   Materials and Methods: In 2010, five electronic databases were searched between 2007 and 2009 to look for reliability studies. A total of 5,795 titles were initially identified. Only 282 titles were potentially related, and finally 42 fitted the inclusion criteria. Results: The Intra-class Correlation Coefficient (ICC is the most popular method with 25 (60% studies having used this method followed by the comparing means (8 or 19%. Out of 25 studies using the ICC, only 7 (28% reported the confidence intervals and types of ICC used. Most studies (71% also tested the agreement of instruments. Conclusion: This study finds that the Intra-class Correlation Coefficient is the most popular method used to assess the reliability of medical instruments measuring continuous outcomes. There are also inappropriate applications and interpretations of statistical methods in some studies. It is important for medical researchers to be aware of this issue, and be able to correctly perform analysis in reliability studies.

  16. Reliability and Validity Study of the Attitude towards Mathematics Instruments Short Form

    Directory of Open Access Journals (Sweden)

    Güney HACIÖMEROĞLU

    2017-05-01

    Full Text Available Purpose of this study was to investigate the reliability and validity of the Turkish form of the Attitude Towards Mathematics Instrument Short Form developed by Lim and Chapman (2013. In this study, data gathered from 310 elementary students were utilized for Exploratory and Confirmatory Factor Analysis to determine the structure of factor loading. The factor loading among the sub-scales were different from original. Confirmatory Factor analysis revealed that the model was acceptable. There were three sub-scales, value, self-confidence, enjoyment and motivation. Cronbach’s alpha coefficient for the overall instrument was calculated as .84, respectively. The adapted instrument includes three sub-scales: value (α=.91, self-confidence (α=.86, enjoyment and motivation (α=.82. Turkish adaptation of the questionnaire is valid and reliable and appropriate to use in Turkish culture.

  17. Reliability of risk assessment measures used in sexually violent predator proceedings.

    Science.gov (United States)

    Miller, Cailey S; Kimonis, Eva R; Otto, Randy K; Kline, Suzonne M; Wasserman, Adam L

    2012-12-01

    The field interrater reliability of three assessment tools frequently used by mental health professionals when evaluating sex offenders' risk for reoffending--the Psychopathy Checklist-Revised (PCL-R), the Minnesota Sex Offender Screening Tool-Revised (MnSOST-R) and the Static-99-was examined within the context of sexually violent predator program proceedings. Rater agreement was highest for the Static--99 (intraclass correlation coefficient [ICC₁] = .78) and lowest for the PCL-R (ICC₁ = .60; MnSOST-R ICC₁ = .74), although all instruments demonstrated lower field reliability than that reported in their test manuals. Findings raise concerns about the reliability of risk assessment tools that are used to inform judgments of risk in high-stake sexually violent predator proceedings. Implications for future research and suggestions for improving evaluator training to increase accuracy when informing legal decision making are discussed.

  18. DEVELOPING OF INDIVIDUAL INSTRUMENT PERFORMANCE ANXIETY SCALE: VALIDITY - RELIABILITY STUDY

    Directory of Open Access Journals (Sweden)

    Esra DALKIRAN

    2014-07-01

    Full Text Available The purpose of this study is to develop a scale unique to our culture, concerning  individual instrument performance anxiety of the students  who are getting instrument training  in the Department of Music Education. In the study, the descriptive research model is used and qualitative research techniques are utilized. The study population consists of the students attending the 23 universities which has Music Education Department. The sample of the study consists of 438 girls and 312 boys, totally 750 students  who are studying in the Department of Music Education of randomly selected 10 universities. As a result of the explanatory and confirmatory factor analyses that were performed, a one-dimensional structure consisting of 14 items was obtained. Also, t-scores and  the coefficient scores of total item correlation concerning the distinguishing power of the items, the difference in the scores of the set of lower and upper 27% was calculated, and it was observed that the items are distinguishing as a result of both analyses. Of the scale, Cronbach's alpha coefficient of internal consistency was calculated as .94, and test-retest reliability coefficient was calculated as .93. As a result, a valid and reliable assessment and evaluation instrument that measures the exam performance anxiety of the students studying in the Department of Music Education, has been developed.Extended AbstractsIntroductionAnxiety is a universal phenomenon which people experience once or a few times during lives. It was accepted as concern for the future or as an unpleasant emotional experience regarding probable hitches of the events (Di Tomasso & Gosch, 2002.In general, the occasions on which negative feelings are experienced cause anxiety to arise (Baltaş and Baltaş, 2000. People also feel anxious in dangerous situations. Anxiety may lead a person to be creative, while it may have hindering characteristics. Anxiety is that an individual considers him

  19. An Exploratory Factor Analysis and Reliability Analysis of the Student Online Learning Readiness (SOLR) Instrument

    Science.gov (United States)

    Yu, Taeho; Richardson, Jennifer C.

    2015-01-01

    The purpose of this study was to develop an effective instrument to measure student readiness in online learning with reliable predictors of online learning success factors such as learning outcomes and learner satisfaction. The validity and reliability of the Student Online Learning Readiness (SOLR) instrument were tested using exploratory factor…

  20. Instruments used in the assessment of expectation toward a spine surgery: an integrative review

    Directory of Open Access Journals (Sweden)

    Eliane Nepomuceno

    Full Text Available Abstract OBJECTIVE To identify and describe the instruments used to assess patients' expectations toward spine surgery. METHOD An integrative review was carried out in the databases PubMed, CINAHL, LILACS and PsycINFO. RESULTS A total of 4,402 publications were identified, of which 25 met the selection criteria. Of the studies selected, only three used tools that had confirmed validity and reliability to be applied; in five studies, clinical scores were used, and were modified for the assessment of patients' expectations, and in 17 studies the researchers developed scales without an adequate description of the method used for their development and validation. CONCLUSION The assessment of patients' expectations has been methodologically conducted in different ways. Until the completion of this integrative review, only two valid and reliable instruments had been used in three of the selected studies.

  1. Development and reliability testing of the Nordic Housing Enabler – an instrument for accessibility assessment of the physical housing

    DEFF Research Database (Denmark)

    Helle, Tina

    and adapted according to accessibility norms and guidelines for housing design in Sweden, Denmark, Iceland and Finland. This iterative process involved occupational therapists, architects, building engineers and professional translators, resulting in the Nordic Housing Enabler. For reliability testing...... serious deficits when it comes to accessibility. This study addresses development of a content valid cross-Nordic version of the Housing Enabler and investigation of inter-rater reliability, when used in occupational therapy practice. The instrument was translated from the original Swedish version......, the sample strategy and data collection procedures were the same in all countries. In total, twenty voluntary occupational therapists collected data from 106 cases by means of the Nordic Housing Enabler. Inter-rater reliability was calculated by means of percentage agreement and kappa statistics. Overall...

  2. Performance assessment instrument to assess the senior high students' psychomotor for the salt hydrolysis material

    Science.gov (United States)

    Nahadi, Firman, Harry; Yulina, Erlis

    2016-02-01

    The purposes of this study were to develop a performance assessment instrument for assessing the competence of psychomotor high school students on salt hydrolysis concepts. The design used in this study was the Research & Development which consists of three phases: development, testing and application of instruments. Subjects in this study were high school students in class XI science, which amounts to 93 students. In the development phase, seven validators validated 17 tasks instrument. In the test phase, we divided 19 students into three-part different times to conduct performance test in salt hydrolysis lab work and observed by six raters. The first, the second, and the third groups recpectively consist of five, six, and eight students. In the application phase, two raters observed the performance of 74 students in the salt hydrolysis lab work in several times. The results showed that 16 of 17 tasks of performance assessment instrument developed can be stated to be valid with CVR value of 1,00 and 0,714. While, the rest was not valid with CVR value was 0.429, below the critical value (0.622). In the test phase, reliability value of instrument obtained were 0,951 for the five-student group, 0,806 for the six-student group and 0,743 for the eight-student group. From the interviews, teachers strongly agree with the performance instrument developed. They stated that the instrument was feasible to use for maximum number of students were six in a single observation.

  3. Reliability and Validity of the Greek Migraine Disability Assessment (MIDAS) Questionnaire.

    Science.gov (United States)

    Oikonomidi, Theodora; Vikelis, Michail; Artemiadis, Artemios; Chrousos, George P; Darviri, Christina

    2018-03-01

    The Migraine Disability Assessment (MIDAS) Questionnaire is a reliable and valid instrument for migraine-related disability. Such a tool is needed to quantify migraine-related disability in the Greek population. This validation study aims to assess the test-retest reliability, internal consistency, item discriminant and convergent validity of the Greek translation of the MIDAS. Adults diagnosed with migraine completed the MIDAS Questionnaire on two occasions 3 weeks apart to assess reliability, and completed the RAND-36 to assess validity. Participants (n = 152) had a median MIDAS score of 24 and mostly severe disability (58% were grade IV). The test-retest reliability analysis (N = 59) revealed excellent reliability for the total score. Internal consistency was α = 0.71 for initial and α = 0.82 for retest completion. For item discriminant validity, the correlations between each question and the total score were significant, with high correlations for questions 2-5 (range 0.67 ≤ r ≤ 0.79; p MIDAS score tended to have better wellbeing. Psychometric properties are comparable with those of other published validation studies of the MIDAS and the original. Findings on question 1 show that missing work/school days may be closely related with increased affect issues. The Greek version of the MIDAS Questionnaire has good reliability and validity. This study allowed for cross-cultural comparability of research findings.

  4. Reliability and validity of the German version of the Utrecht Questionnaire for Outcome Assessment in Aesthetic Rhinoplasty (D-OAR).

    Science.gov (United States)

    Spiekermann, Christoph; Rudack, Claudia; Stenner, Markus

    2017-11-01

    The outcome of aesthetic rhinoplasty is determined by the patient's subjective satisfaction with the nasal appearance which is difficult to assess. The Utrecht Questionnaire for Outcome Assessment in Aesthetic Rhinoplasty (OAR) is a brief and reliable instrument to assess the influence of the subjective nasal appearance on quality of life in patients undergoing aesthetic rhinoplasty. Preoperative application of this questionnaire reveals important aspects and possible disturbances of the body image which could be negative predictors concerning the result. On the other hand, it represents an appropriate tool to assess the postoperative outcome. The aim of this study was to determine the validity, reliability and responsiveness of the adapted German version of the OAR (D-OAR). The adaption of the OAR to German language was performed by a forward and backward translation process. Patients undergoing rhinoplasty were asked to complete the D-OAR preoperatively, 1, 3 and 12 months after procedure and healthy volunteers without any nasal complaints served as controls to test validity, reliability and responsiveness. An excellent internal consistency, a good test-retest reliability and good inter-item and item-total correlations demonstrated a good reliability of the D-OAR. The convincing validity of the adapted version was proven by an excellent discriminant and a sufficient content validity. Significant differences between pre- and postoperative D-OAR scores revealed a good responsiveness of the instrument. Hence, with a sufficient validity, reliability and sensitivity to changes, the D-OAR is a short and helpful instrument to assess the subjective perception of the nasal appearance in German patients.

  5. Inter-rater reliability of the German version of the Nurses' Global Assessment of Suicide Risk scale.

    Science.gov (United States)

    Kozel, Bernd; Grieser, Manuela; Abderhalden, Christoph; Cutcliffe, John R

    2016-10-01

    In comparison to the general population, the suicide rates of psychiatric inpatient populations in Germany and Switzerland are very high. An important preventive contribution to the lowering of the suicide rates in mental health care is to ensure that the risk of suicide of psychiatric inpatients is assessed as accurately as possible. While risk-assessment instruments can serve an important function in determining such risk, very few have been translated to German. Therefore, in the present study, we reported on the German version of Nurses' Global Assessment of Suicide Risk (NGASR) scale. After translating the original instrument into German and pretesting the German version, we tested the inter-rater reliability of the instrument. Twelve video case studies were evaluated by 13 raters with the NGASR scale in a 'laboratory' trial. In each case, the observer's agreement was calculated for the single items, the overall scale, the risk levels, and the sum scores. The statistical data analysis was conducted with kappa and AC1 statistics for dichotomous (items, scale) scales. A high-to-very high observers' agreement (AC1: 0.62-1.00, kappa: 0.00-1.00) was determined for 16 items of the German version of the NGASR scale. We conclude that the German version of the NGASR scale is a reliable instrument for evaluating risk factors for suicide. A reliable application in the clinical practise appears to be enhanced by training in the use of the instrument and the right implementation instructions. © 2016 Australian College of Mental Health Nurses Inc.

  6. A Turkish version of myocardial infarction dimensional assessment scale (TR-MIDAS): reliability-validity assesment.

    Science.gov (United States)

    Uysal, Hilal; Ozcan, Şeyda

    2011-06-01

    Many new measuring devices have been developed so that broader psychometric measurements in the coronary artery disease, disease-specific health status measurements, and identification of the broader quality of life can be performed in the recent years. The study was intended to determine whether, and to what extent, MIDAS is a valid and reliable measurement to the patients suffering from myocardial infarction for the first time in Turkey. The research was conducted with the patients hospitalized and treated with myocardial infarction in the cardiology departments of 2 hospitals in Istanbul, Turkey, between 2007 and 2008. Psychometric evaluations of TR-MIDAS were used for validity studies; language validity, content validity, construct validity were examined. For reliability studies; the tool's internal consistency reliability, Cronbach's alpha reliability coefficient, and test-retest reliability were completed. The instrument's content validity index was determined to be "0.95". Principal component analysis revealed six factors with an eigenvalue >1.5. Cronbach's alpha was found to be 0.89 for total scale which was an acceptable value. The total's test-retest reliability was 0.51 (p<0.01). Data obtained at the end of the study supports that Turkish Myocardial Infarction Dimensional Assessment Scale is a valid and reliable instrument as a disease-specific scale to assess the patients' quality of life suffering from myocardial infarction in Turkey. Copyright © 2010 European Society of Cardiology. Published by Elsevier B.V. All rights reserved.

  7. Frame-of-Reference Training: Establishing Reliable Assessment of Teaching Effectiveness.

    Science.gov (United States)

    Newman, Lori R; Brodsky, Dara; Jones, Richard N; Schwartzstein, Richard M; Atkins, Katharyn Meredith; Roberts, David H

    2016-01-01

    Frame-of-reference (FOR) training has been used successfully to teach faculty how to produce accurate and reliable workplace-based ratings when assessing a performance. We engaged 21 Harvard Medical School faculty members in our pilot and implementation studies to determine the effectiveness of using FOR training to assess health professionals' teaching performances. All faculty were novices at rating their peers' teaching effectiveness. Before FOR training, we asked participants to evaluate a recorded lecture using a criterion-based peer assessment of medical lecturing instrument. At the start of training, we discussed the instrument and emphasized its precise behavioral standards. During training, participants practiced rating lectures and received immediate feedback on how well they categorized and scored performances as compared with expert-derived scores of the same lectures. At the conclusion of the training, we asked participants to rate a post-training recorded lecture to determine agreement with the experts' scores. Participants and experts had greater rating agreement for the post-training lecture compared with the pretraining lecture. Through this investigation, we determined that FOR training is a feasible method to teach faculty how to accurately and reliably assess medical lectures. Medical school instructors and continuing education presenters should have the opportunity to be observed and receive feedback from trained peer observers. Our results show that it is possible to use FOR rater training to teach peer observers how to accurately rate medical lectures. The process is time efficient and offers the prospect for assessment and feedback beyond traditional learner evaluation of instruction.

  8. Using the Hemophilia Joint Health Score for assessment of children: Reliability of the Spanish version.

    Science.gov (United States)

    R, Cuesta-Barriuso; A, Torres-Ortuño; S, Pérez-Alenda; J, Carrasco Juan; F, Querol; J, Nieto-Munuera; Ja, López-Pina

    2018-02-27

    Numerous measuring instruments for the evaluation of hemophilic arthropathy have been developed. One of the most used systems is the Hemophilia Joint Health Score (HJHS) given its sensitivity to clinical changes appearing in the joints because of recurrent hemarthrosis. Assessing the interrater reliability, using the Spanish version of the HJHS (version 2.1) in children with hemophilia. Reliability study to assess the interrater reliability of the Spanish version of HJHS. A sample of 36 children aged 7-13 years diagnosed with hemophilia A or B was used. Two physiotherapists performed physical assessments with the Spanish version of the HJHS. Descriptive statistics (range, mean, standard deviation) and the analysis of interrater reliability were calculated. The interrater reliability was heterogeneous since the Kappa coefficient range (ĸ), although significant (p reliability of the Spanish population version of the HJHS is high. This scale should be used generically in evaluating musculoskeletal pediatric patients with hemophilia.

  9. Development of an instrument to assess readiness to recover in anorexia nervosa.

    Science.gov (United States)

    Rieger, E; Touyz, S; Schotte, D; Beumont, P; Russell, J; Clarke, S; Kohn, M; Griffiths, R

    2000-12-01

    The degree to which patients with anorexia nervosa demonstrate readiness to recover from their illness has received scant theoretical or empirical attention. Investigating the prevalence and degree of amotivation for recovery in anorexia nervosa, its role in outcome, and the effectiveness of interventions designed to enhance readiness to recover necessitates the existence of a reliable and valid measure of motivational issues relevant to the disorder. The present study aimed to develop and evaluate an instrument for assessing readiness to recover in anorexia nervosa, namely, the Anorexia Nervosa Stages of Change Questionnaire (ANSOCQ), a 23-item self-report questionnaire based on Prochaska and DiClemente's stages of change model. Seventy-one inpatients with anorexia nervosa participated in the study. On several occasions during their admission, participants completed the ANSOCQ as well as questionnaires assessing readiness to recover, anorexic symptomatology, general distress, and social desirability. The ANSOCQ demonstrated good internal consistency (.90) and 1-week test-retest reliability (.89). Various aspects of validity were also supported, such as significant relationships with other instruments assessing readiness to recover and the prediction of weight gain during different periods of treatment. The results suggest that the ANSOCQ is a psychometrically sound instrument that may prove useful in investigating the role of readiness to recover in anorexia nervosa. Copyright 2000 John Wiley & Sons, Inc.

  10. Assessing Social Networks in Patients with Psychotic Disorders: A Systematic Review of Instruments.

    Science.gov (United States)

    Siette, Joyce; Gulea, Claudia; Priebe, Stefan

    2015-01-01

    Evidence suggests that social networks of patients with psychotic disorders influence symptoms, quality of life and treatment outcomes. It is therefore important to assess social networks for which appropriate and preferably established instruments should be used. To identify instruments assessing social networks in studies of patients with psychotic disorders and explore their properties. A systematic search of electronic databases was conducted to identify studies that used a measure of social networks in patients with psychotic disorders. Eight instruments were identified, all of which had been developed before 1991. They have been used in 65 studies (total N of patients = 8,522). They assess one or more aspects of social networks such as their size, structure, dimensionality and quality. Most instruments have various shortcomings, including questionable inter-rater and test-retest reliability. The assessment of social networks in patients with psychotic disorders is characterized by a variety of approaches which may reflect the complexity of the construct. Further research on social networks in patients with psychotic disorders would benefit from advanced and more precise instruments using comparable definitions of and timescales for social networks across studies.

  11. Psychometric properties of assessment instruments for autism spectrum disorder: a systematic review of Brazilian studies

    Directory of Open Access Journals (Sweden)

    Bárbara Backes

    2014-07-01

    Full Text Available Objective To systematically review the scientific literature on the psychometric properties of international instruments for the assessment of autism spectrum disorder (ASD in the Brazilian population. Methods A search of bibliographic references was conducted in six electronic databases: PsycINFO, PubMed, IndexPsi, Lilacs, Capes (theses and dissertations and SciELO. The studies were selected by two independent researchers. Results The procedure identified 11 studies of the Brazilian population that encompassed six ASD assessment tools. Given the information provided, the adaptation of the M-CHAT, a screening instrument, was the best conducted. All steps of the adaptation process were described and the changes made to the final version of the instrument were presented, which was not addressed in other studies. In terms of reliability, all of the instruments that assessed internal consistency showed adequate values. In addition, the ADI-R and the CARS adaptations also satisfactorily contemplated inter-rater reliability and test-retest indices, respectively. Finally, all studies aiming to validate instruments showed evidence of validity and sensitivity, and specificity values above 0.90 were observed in the ASQ, ADI-R and ABC. Conclusion Considering both the psychometric aspects and the copyright information, the screening instrument that currently appears to be best indicated for clinical and research use is the M-CHAT. It was also noticed that there are still no specific ASD diagnostic tools available for use in Brazil. This lack of diagnostic instruments consists in a critical situation for the improvement of clinical practice and the development of research in this area.

  12. Assessing Academic Advising Outcomes Using Social Cognitive Theory: A Validity and Reliability Study

    Science.gov (United States)

    Erlich, Richard J.; Russ-Eft, Darlene F.

    2012-01-01

    The validity and reliability of three instruments, the "Counselor Rubric for Gauging Student Understanding of Academic Planning," micro-analytic questions, and the "Student Survey for Understanding Academic Planning," all based on social cognitive theory, were tested as means to assess self-efficacy and self-regulated learning in college academic…

  13. Metacognition Assessment Interview: Instrument description and factor structure

    Directory of Open Access Journals (Sweden)

    Giovanni Pellecchia

    2015-12-01

    Full Text Available Objective: Metacognition is a multi-component psychological construct, characterised by the ability to identify and describe one’s own mental states and those of others. Evidence has been found for an association between deficits in metacognitive abilities and poor social functioning, low quality of life, psychopathology, and symptoms in Personality Disorders (PDs. However, to date, there are few psychometrically validated instruments available for assessing the different components of metacognition. A semi-structured interview, the Metacognition Assessment Interview (MAI, has been developed to evaluate different domains of metacognition. In the present study, we investigated the psychometric properties of the MAI in an outpatient clinical sample. Method: The MAI was administered to a clinical population of 306 outpatients attending a private clinical centre. Exploratory factor analysis, confirmatory factor analysis and correlation with instruments assessing alexithymia and interpersonal problems were carried out to examine the dimensionality and validity of the MAI. Result: Explorative and confirmatory factor analyses revealed a good fit for both a two-factor model and a four-factor model of metacognition. The two-factor model yielded two main dimensions, which we named: Self domain, defined as self-reflection, and Other domain, defined as critical distancing from one’s own mental state and that of others. The four-factor solution is composed of four sub-domains: monitoring, integration, differentiation and decentration. Moreover, the MAI showed good convergent validity, with significant correlations with both alexithymia and interpersonal problems. Conclusions: These results confirm that the MAI is a reliable instrument for measuring metacognition and its different sub-domains. In particular, the MAI represents a useful and flexible instrument for the assessment of metacognition impairments in different psychopathologies and it can provide

  14. Comparing the Psychometric Properties of Two Physical Activity Self-Efficacy Instruments in Urban, Adolescent Girls: Validity, Measurement Invariance, and Reliability

    Science.gov (United States)

    Voskuil, Vicki R.; Pierce, Steven J.; Robbins, Lorraine B.

    2017-01-01

    Aims: This study compared the psychometric properties of two self-efficacy instruments related to physical activity. Factorial validity, cross-group and longitudinal invariance, and composite reliability were examined. Methods: Secondary analysis was conducted on data from a group randomized controlled trial investigating the effect of a 17-week intervention on increasing moderate to vigorous physical activity among 5th–8th grade girls (N = 1,012). Participants completed a 6-item Physical Activity Self-Efficacy Scale (PASE) and a 7-item Self-Efficacy for Exercise Behaviors Scale (SEEB) at baseline and post-intervention. Confirmatory factor analyses for intervention and control groups were conducted with Mplus Version 7.4 using robust weighted least squares estimation. Model fit was evaluated with the chi-square index, comparative fit index, and root mean square error of approximation. Composite reliability for latent factors with ordinal indicators was computed from Mplus output using SAS 9.3. Results: Mean age of the girls was 12.2 years (SD = 0.96). One-third of the girls were obese. Girls represented a diverse sample with over 50% indicating black race and an additional 19% identifying as mixed or other race. Both instruments demonstrated configural invariance for simultaneous analysis of cross-group and longitudinal invariance based on alternative fit indices. However, simultaneous metric invariance was not met for the PASE or the SEEB instruments. Partial metric invariance for the simultaneous analysis was achieved for the PASE with one factor loading identified as non-invariant. Partial metric invariance was not met for the SEEB. Longitudinal scalar invariance was achieved for both instruments in the control group but not the intervention group. Composite reliability for the PASE ranged from 0.772 to 0.842. Reliability for the SEEB ranged from 0.719 to 0.800 indicating higher reliability for the PASE. Reliability was more stable over time in the control

  15. Validity of instruments to assess students' travel and pedestrian safety

    Directory of Open Access Journals (Sweden)

    Baranowski Tom

    2010-05-01

    Full Text Available Abstract Background Safe Routes to School (SRTS programs are designed to make walking and bicycling to school safe and accessible for children. Despite their growing popularity, few validated measures exist for assessing important outcomes such as type of student transport or pedestrian safety behaviors. This research validated the SRTS school travel survey and a pedestrian safety behavior checklist. Methods Fourth grade students completed a brief written survey on how they got to school that day with set responses. Test-retest reliability was obtained 3-4 hours apart. Convergent validity of the SRTS travel survey was assessed by comparison to parents' report. For the measure of pedestrian safety behavior, 10 research assistants observed 29 students at a school intersection for completion of 8 selected pedestrian safety behaviors. Reliability was determined in two ways: correlations between the research assistants' ratings to that of the Principal Investigator (PI and intraclass correlations (ICC across research assistant ratings. Results The SRTS travel survey had high test-retest reliability (κ = 0.97, n = 96, p Conclusions These validated instruments can be used to assess SRTS programs. The pedestrian safety behavior checklist may benefit from further formative work.

  16. [Development of an Instrument to Assess the Quality of Childbirth Care from the Mother's Perspective].

    Science.gov (United States)

    Jeong, Geum Hee; Kim, Hyun Kyoung; Kim, Young Hee; Kim, Sun Hee; Lee, Sun Hee; Kim, Kyung Won

    2018-02-01

    This study aimed to develop an instrument to assess the quality of childbirth care from the perspective of a mother after delivery. The instrument was developed from a literature review, interviews, and item validation. Thirty-eight items were compiled for the instrument. The data for validity and reliability testing were collected using a questionnaire survey conducted on 270 women who had undergone normal vaginal delivery in Korea and analyzed with descriptive statistics, exploratory factor analysis, and reliability coefficients. The exploratory factor analysis reduced the number of items in the instrument to 28 items that were factored into four subscales: family-centered care, personal care, emotional empowerment, and information provision. With respect to convergence validation, there was positive correlation between this instrument and birth satisfaction scale (r=.34, pinstrument could be used as a measure of the quality of nursing care for women who have a normal vaginal delivery. © 2018 Korean Society of Nursing Science.

  17. Design and validation of a three-instrument toolkit for the assessment of competence in electrocardiogram rhythm recognition.

    Science.gov (United States)

    Hernández-Padilla, José M; Granero-Molina, José; Márquez-Hernández, Verónica V; Suthers, Fiona; López-Entrambasaguas, Olga M; Fernández-Sola, Cayetano

    2017-06-01

    Rapid and accurate interpretation of cardiac arrhythmias by nurses has been linked with safe practice and positive patient outcomes. Although training in electrocardiogram rhythm recognition is part of most undergraduate nursing programmes, research continues to suggest that nurses and nursing students lack competence in recognising cardiac rhythms. In order to promote patient safety, nursing educators must develop valid and reliable assessment tools that allow the rigorous assessment of this competence before nursing students are allowed to practise without supervision. The aim of this study was to develop and psychometrically evaluate a toolkit to holistically assess competence in electrocardiogram rhythm recognition. Following a convenience sampling technique, 293 nursing students from a nursing faculty in a Spanish university were recruited for the study. The following three instruments were developed and psychometrically tested: an electrocardiogram knowledge assessment tool (ECG-KAT), an electrocardiogram skills assessment tool (ECG-SAT) and an electrocardiogram self-efficacy assessment tool (ECG-SES). Reliability and validity (content, criterion and construct) of these tools were meticulously examined. A high Cronbach's alpha coefficient demonstrated the excellent reliability of the instruments (ECG-KAT=0.89; ECG-SAT=0.93; ECG-SES=0.98). An excellent context validity index (scales' average content validity index>0.94) and very good criterion validity were evidenced for all the tools. Regarding construct validity, principal component analysis revealed that all items comprising the instruments contributed to measure knowledge, skills or self-efficacy in electrocardiogram rhythm recognition. Moreover, known-groups analysis showed the tools' ability to detect expected differences in competence between groups with different training experiences. The three-instrument toolkit developed showed excellent psychometric properties for measuring competence in

  18. Quality of leadership in multidisciplinary cancer tumor boards: development and evaluation of a leadership assessment instrument (ATLAS).

    Science.gov (United States)

    Jalil, Rozh; Soukup, Tayana; Akhter, Waseem; Sevdalis, Nick; Green, James S A

    2018-03-03

    High-quality leadership and chairing skills are vital for good performance in multidisciplinary tumor boards (MTBs), but no instruments currently exist for assessing and improving these skills. To construct and validate a robust instrument for assessment of MTB leading and chairing skills. We developed an observational MTB leadership assessment instrument (ATLAS). ATLAS includes 12 domains that assess the leadership and chairing skills of the MTB chairperson. ATLAS has gone through a rigorous process of refinement and content validation prior to use to assess the MTB lead by two urological surgeons (blinded to each other) in 7 real-live (n = 286 cases) and 10 video-recorded (n = 131 cases) MTBs. ATLAS domains were analyzed via descriptive statistics. Instrument content was evaluated for validity using the content validation index (CVI). Intraclass correlation coefficients (ICCs) were used to assess inter-observer reliability. Instrument refining resulted in ATLAS including the following 12 domains: time management, communication, encouraging contribution, ability to summarize, ensuring all patients have treatment plan, case prioritization, keeping meeting focused, facilitate discussion, conflict management, leadership, creating good working atmosphere, and recruitment for clinical trials. CVI was acceptable and inter-rater agreement adequate to high for all domains. Agreement was somewhat higher in real-time MTBs compared to video ratings. Concurrent validation evidence was derived via positive and significant correlations between ATLAS and an established validated brief MTB leadership assessment scale. ATLAS is an observational assessment instrument that can be reliably used for assessing leadership and chairing skills in cancer MTBs (both live and video-recorded). The ability to assess and feedback on team leader performance provides the ground for promotion of good practice and continuing professional development of tumor board leaders.

  19. The Nurses Self-Concept Instrument (NSCI): assessment of psychometric properties for Australian domestic and international student nurses.

    Science.gov (United States)

    Angel, Elizabeth; Craven, Rhonda; Denson, Nida

    2012-07-01

    Professional self-concept is a critical driver of job satisfaction. In Australia, as international nursing enrolments rise, nursing is increasingly characterised by a professional body of international nurses who may differ from domestic Australian nurses in their nursing self-concept. At present, no psychometrically sound instrument for assessing nursing self-concept for Australian domestic and international nursing students is available. The purpose of this study was to: (1) develop an instrument (the Nurses' Self-Concept Instrument (NSCI)) to measure the professional self-concept of domestic and international nursing students in Australia, and (2) test the psychometric properties of this newly developed instrument. A literature review was conducted to generate the initial dimension and item pools to measure nurses' professional self-concept (NSCI). Two stakeholders examined the content and face validity of dimensions and items. Analysis was performed on data collected from 253 undergraduate nursing students in a large public university in Sydney, Australia, and consisted of domestic (n=218) and international (n=35) nursing students. Internal reliability was assessed using Cronbach's Alpha. Confirmatory factor analysis (CFA) was used to assess the construct validity of the NSCI. The resulting NSCI consisted of 14 items across four self-concept domains: care, leadership, staff relations, and knowledge. The CFA supported the hypothesised factor structure of the self-concept model. All reliabilities were acceptable for both domestic and international students (ranging from r=.78 to .93). The NSCI was shown to be a valid and reliable tool for assessing Australian domestic and international student nurses' professional self-concept. This instrument may also enable those responsible for recruitment of students into nursing courses to assess students' professional self-concept and implement appropriate strategies to foster the growth of lifelong career development

  20. Development of a quality-assessment tool for experimental bruxism studies: reliability and validity.

    Science.gov (United States)

    Dawson, Andreas; Raphael, Karen G; Glaros, Alan; Axelsson, Susanna; Arima, Taro; Ernberg, Malin; Farella, Mauro; Lobbezoo, Frank; Manfredini, Daniele; Michelotti, Ambra; Svensson, Peter; List, Thomas

    2013-01-01

    To combine empirical evidence and expert opinion in a formal consensus method in order to develop a quality-assessment tool for experimental bruxism studies in systematic reviews. Tool development comprised five steps: (1) preliminary decisions, (2) item generation, (3) face-validity assessment, (4) reliability and discriminitive validity assessment, and (5) instrument refinement. The kappa value and phi-coefficient were calculated to assess inter-observer reliability and discriminative ability, respectively. Following preliminary decisions and a literature review, a list of 52 items to be considered for inclusion in the tool was compiled. Eleven experts were invited to join a Delphi panel and 10 accepted. Four Delphi rounds reduced the preliminary tool-Quality-Assessment Tool for Experimental Bruxism Studies (Qu-ATEBS)- to 8 items: study aim, study sample, control condition or group, study design, experimental bruxism task, statistics, interpretation of results, and conflict of interest statement. Consensus among the Delphi panelists yielded good face validity. Inter-observer reliability was acceptable (k = 0.77). Discriminative validity was excellent (phi coefficient 1.0; P reviews of experimental bruxism studies, exhibits face validity, excellent discriminative validity, and acceptable inter-observer reliability. Development of quality assessment tools for many other topics in the orofacial pain literature is needed and may follow the described procedure.

  1. Development Instruments Through Confirmatory Factor Analysis (CFA in Appropriate Intensity Assessment

    Directory of Open Access Journals (Sweden)

    Ari Saptono

    2017-06-01

    Full Text Available The research aims to develop the valid and reliable measurement instruments of entrepreneurship intention in vocational secondary school students. Multi stage random sampling was used as the technique to determine sample (300 respondents. The research method used research and development with confirmatory factor analysis (CFA. Result of confirmatory factor analysis (CFA at the second order with robust maximum likelihood method shows that valid and reliable instrument with the acquisition value of loading factor is more than 0.5 (> 0,5 and a significance value of t is more than 1,96 (> 1,96. Reliability test results shows that the value of the combined construct reliability (CR of 0.97and a variance value extract (VE to 0.52 is greater than the limit of acceptance CR ≥ 0.70 and VE ≥ 0.50. The conclusion of the measurement instruments of entrepreneurship intention with three dimensions and 31 items met the standards of validity and reliability in accordance with the instrument development process.

  2. Reliability assessments in qualitative health promotion research.

    Science.gov (United States)

    Cook, Kay E

    2012-03-01

    This article contributes to the debate about the use of reliability assessments in qualitative research in general, and health promotion research in particular. In this article, I examine the use of reliability assessments in qualitative health promotion research in response to health promotion researchers' commonly held misconception that reliability assessments improve the rigor of qualitative research. All qualitative articles published in the journal Health Promotion International from 2003 to 2009 employing reliability assessments were examined. In total, 31.3% (20/64) articles employed some form of reliability assessment. The use of reliability assessments increased over the study period, ranging from qualitative articles decreased. The articles were then classified into four types of reliability assessments, including the verification of thematic codes, the use of inter-rater reliability statistics, congruence in team coding and congruence in coding across sites. The merits of each type were discussed, with the subsequent discussion focusing on the deductive nature of reliable thematic coding, the limited depth of immediately verifiable data and the usefulness of such studies to health promotion and the advancement of the qualitative paradigm.

  3. Approach to reliability assessment

    International Nuclear Information System (INIS)

    Green, A.E.; Bourne, A.J.

    1975-01-01

    Experience has shown that reliability assessments can play an important role in the early design and subsequent operation of technological systems where reliability is at a premium. The approaches to and techniques for such assessments, which have been outlined in the paper, have been successfully applied in variety of applications ranging from individual equipments to large and complex systems. The general approach involves the logical and systematic establishment of the purpose, performance requirements and reliability criteria of systems. This is followed by an appraisal of likely system achievment based on the understanding of different types of variational behavior. A fundamental reliability model emerges from the correlation between the appropriate Q and H functions for performance requirement and achievement. This model may cover the complete spectrum of performance behavior in all the system dimensions

  4. Reliability of instruments in a cooperative, multisite study: employment intervention demonstration program.

    Science.gov (United States)

    Salyers, M P; McHugo, G J; Cook, J A; Razzano, L A; Drake, R E; Mueser, K T

    2001-09-01

    Reliability of well-known instruments was examined in 202 people with severe mental illness participating in a multisite vocational study. We examined interrater reliability of the Positive and Negative Syndrome Scale (PANSS) and the internal consistency and test-retest reliability of the PANSS, the Rosenberg Self-Esteem Scale, the Medical Outcomes Study Short Form-36 (SF-36), and the Quality of Life Interview. Most scales had good levels of reliability, with intraclass correlation coefficients (ICCs) and coefficient alphas above .70. However, the SF-36 scales were generally less stable over time, particularly Social Functioning (ICC = .55). Test-retest reliability was lower among less educated respondents and among ethnic minorities. We recommend close monitoring of psychometric issues in future multisite studies.

  5. Assessment of Perceived Stress Related to Migration and Acculturation in Patients with Psychiatric Disorders (MIGSTR10)-Development, Reliability, and Dimensionality of a Brief Instrument.

    Science.gov (United States)

    Müller, Matthias J; Zink, Sabrina; Koch, Eckhardt

    2017-09-01

    Assessment of stressors related to migration and acculturation in patients with psychiatric disorder and migration background could help improve culturally sensitive concepts of psychiatry and psychotherapy for diagnosis and treatment. The present overview delineates development and psychometric properties of an instrument (MIGSTR10) for assessment of stressors related to migration and acculturation, particularly for application in patients with psychiatric disorders. Ten migration-related stressors were derived from a qualitative content analysis of case histories of patients with psychiatric disorder and migration background and put into a suitable interview and questionnaire format (MIGSTR10; 10 questions, answer format: categorical yes/no, and dimensional 0-10) for self-assessment and observer ratings in several languages. Reliability (interrater agreement, internal consistency) and dimensionality (multi-dimensional scaling, MDS) were investigated in n = 235 patients with migration background and n = 612 indigenous German patients. Interrater agreement (ICC) for MIGSTR10 single items and sum scores (categorical and dimensional) was sufficiently high (≥.58); internal consistency (Cronbach's α) reached medium to high values (.56-.73). MDS revealed a two-dimensional solution with two item clusters (A: communication, migration history, forced marriage, homesickness, discrimination, other stressors; B: family conflicts, loss of status, feelings of shame, guilt feelings). The MIGSTR10 is a rationally developed, straightforward 10-item screening instrument with satisfactory psychometric properties for the assessment of individual and specific stressors related to migration and acculturation.

  6. INSTRUMENTS OF HIGH RISK SEXUAL BEHAVIOR ASSESSMENT: A SYSTEMATIC REVIEW.

    Science.gov (United States)

    Mirzaei, Mojtaba; Ahmadi, Khodabakhsh; Saadat, Seyed-Hassan; Ramezani, Mohammad Arash

    2016-02-01

    Sexual behavior is a complex activity affecting all aspects of human's life. Risky sexual behaviors impose negative outcomes on family, relationships and health. Unsafe sex is the second most leading cause of disability adjusted life years worldwide. Valid and reliable tools for assessment of risky sexual behaviors are necessary for implementing preventive measures. we searched Medline and the Cochrane Library of Systematic Reviews, with the keywords of "risky sexual behavior assessment", "sexual risk assessment", "high risk sexual behavior", "sexual risk taking". By reviewing references of the articles, some complementary studies were added. Assessment can be performed by questionnaire or non-questionnaire instruments. Questionnaires vary depending on their target population, evaluation of risky sexual behavior as a whole or focusing on an associated risk factor. In order to avoid usual biases in self reports, objective biomarker assessment of unprotected sex are employed. These markers include prostate specific antigen, chromosome Y DNA and Seminogelin. Risky sexual behavior can be assessed by various subjective and objective methods. While self-reports are more feasible, objective methods offer a higher degree of reliability. Further studies for finding more feasible methods of using biomarkers are recommended.

  7. The Cognitive Telephone Screening Instrument (COGTEL: A Brief, Reliable, and Valid Tool for Capturing Interindividual Differences in Cognitive Functioning in Epidemiological and Aging Studies

    Directory of Open Access Journals (Sweden)

    Andreas Ihle

    2017-10-01

    Full Text Available Aims: The present study set out to evaluate the psychometric properties of the Cognitive Telephone Screening Instrument (COGTEL in 2 different samples of older adults. Methods: We assessed COGTEL in 116 older adults, with retest after 7 days to evaluate the test-retest reliability. Moreover, we assessed COGTEL in 868 older adults to evaluate convergent validity to the Mini-Mental State Examination (MMSE. Results: Test-retest reliability of the COGTEL total score was good at 0.85 (p < 0.001. Latent variable analyses revealed that COGTEL and MMSE correlated by 0.93 (p < 0.001, indicating convergent validity of the COGTEL. Conclusion: The present analyses suggest COGTEL as a brief, reliable, and valid instrument for capturing interindividual differences in cognitive functioning in epidemiological and aging studies, with the advantage of covering more cognitive domains than traditional screening tools such as the MMSE, as well as differentiating between individual performance levels, in healthy older adults.

  8. Reliability and Validity of Modified Service Quality Instrument (SERVQUAL) in Patients' Motivation to Adhere to Insulin Therapy.

    Science.gov (United States)

    Jakupovic, Vedran; Solakovic, Suajb; Celebic, Nedim; Kulovic, Dzenan

    2018-03-01

    Diabetes is progressive condition which requires various ways of treatment. Adequate therapy prescribed in the right time helps patient to postpone development of complications. Adherence to complicated therapy is challenge for both patients and HCPs and is subject of research in many disciplines. Improvement in communication between HCP and patients is very important in patient's adherence to therapy. Aim of this research was to explore validity and reliability of modified SERVQUAL instrument in attempt to explore ways of motivating diabetic patient to accept prescribed insulin therapy. We used modified SERVQUAL questionnaire as instrument in the research. It was necessary to check validity and reliability of the new modified instrument. Results show that modified Servqual instrument has excellent reliability (α=0.908), so we could say that it measures precisely Expectations, Perceptions and Motivation at patients. Factor analysis (EFA method) with Varimax rotation extracted 4 factors which together explain 52.902% variance of the results on this subscale. Bifactorial solution could be seen on Scree-plot diagram (break at second factor). Results in this research show that modified Servqual instrument which is created in order to measure expectations and perceptions of the patients is valid and reliable. Reliability and validity are proven indeed in additional dimension which was created originally for this research - motivation to accept insulin therapy.

  9. Reliability and Validity of Modified Service Quality Instrument (SERVQUAL) in Patients’ Motivation to Adhere to Insulin Therapy

    Science.gov (United States)

    Jakupovic, Vedran; Solakovic, Suajb; Celebic, Nedim; Kulovic, Dzenan

    2018-01-01

    Introduction: Diabetes is progressive condition which requires various ways of treatment. Adequate therapy prescribed in the right time helps patient to postpone development of complications. Adherence to complicated therapy is challenge for both patients and HCPs and is subject of research in many disciplines. Improvement in communication between HCP and patients is very important in patient’s adherence to therapy. Aim: Aim of this research was to explore validity and reliability of modified SERVQUAL instrument in attempt to explore ways of motivating diabetic patient to accept prescribed insulin therapy. Material and Methods: We used modified SERVQUAL questionnaire as instrument in the research. It was necessary to check validity and reliability of the new modified instrument. Results: Results show that modified Servqual instrument has excellent reliability (α=0.908), so we could say that it measures precisely Expectations, Perceptions and Motivation at patients. Factor analysis (EFA method) with Varimax rotation extracted 4 factors which together explain 52.902% variance of the results on this subscale. Bifactorial solution could be seen on Scree-plot diagram (break at second factor). Conclusion: Results in this research show that modified Servqual instrument which is created in order to measure expectations and perceptions of the patients is valid and reliable. Reliability and validity are proven indeed in additional dimension which was created originally for this research - motivation to accept insulin therapy. PMID:29670478

  10. Reliability Assessment Of Wind Turbines

    DEFF Research Database (Denmark)

    Sørensen, John Dalsgaard

    2014-01-01

    Reduction of cost of energy for wind turbines are very important in order to make wind energy competitive compared to other energy sources. Therefore the turbine components should be designed to have sufficient reliability but also not be too costly (and safe). This paper presents models...... for uncertainty modeling and reliability assessment of especially the structural components such as tower, blades, substructure and foundation. But since the function of a wind turbine is highly dependent on many electrical and mechanical components as well as a control system also reliability aspects...... of these components are discussed and it is described how there reliability influences the reliability of the structural components. Two illustrative examples are presented considering uncertainty modeling, reliability assessment and calibration of partial safety factors for structural wind turbine components exposed...

  11. Systems analysis programs for hands-on integrated reliability evaluations (SAPHIRE) Version 5.0. Fault tree, event tree, and piping ampersand instrumentation diagram (FEP) editors reference manual: Volume 7

    International Nuclear Information System (INIS)

    McKay, M.K.; Skinner, N.L.; Wood, S.T.

    1994-07-01

    The Systems Analysis Programs for Hands-on Integrated Reliability Evaluations (SAPHIRE) refers to a set of several microcomputer programs that were developed to create and analyze probabilistic risk assessments (PRAs), primarily for nuclear power plants. The Fault Tree, Event Tree, and Piping and Instrumentation Diagram (FEP) editors allow the user to graphically build and edit fault trees, and event trees, and piping and instrumentation diagrams (P and IDs). The software is designed to enable the independent use of the graphical-based editors found in the Integrated Reliability and Risk Assessment System (IRRAS). FEP is comprised of three separate editors (Fault Tree, Event Tree, and Piping and Instrumentation Diagram) and a utility module. This reference manual provides a screen-by-screen guide of the entire FEP System

  12. The reliability and validity of Cognitive Abilities Screening Instrument C-2.0 in Screening elderly people in Chengdu

    Institute of Scientific and Technical Information of China (English)

    Luo Zhuming; Lu Rong

    2000-01-01

    Background: Cognitive Abilities Screening Instrument C-2.0 (CASI C-2.0) is founded on Chinese culture apply to person with little or no formal education.Objective: In order to assess the reliability and validity of Cognitive Abilities Screening Instrument C-2.0 (CASI C-2.0), we surveyed 807 persons aged from 55 to 100 years and with little or no formal education in the town and country of Chengdu city. Methods: Phase I: 807 persona were assessed by different doctors with CASI C-2.0 and Mini mental State Examination (MMSE). Phase Ⅱ: The positive persons of phase I were assessed by trained field physicians who did not know the CASI C-2.0 and MMSE scores. The physicians assessment included a detail history, a systematic physical examination and a selected group of psychometric tests to ascertain the clinical diagnoses of dementia、 Alzheimer′s disease and vascular dementia utilizing the ICD-10 and NINCDS-ADRAD、 DSM-Ⅳ criteria respectively. Phase Ⅲ: 30 persons were retest with CASI C-2.0 randomly after 3--4 weeks. Results: Among the 807 persons, 55 cases of dementia were identified, including 50 cases of probable Alzheimer′s disease and 5 cases of vascular dementia. Discussion: On reliability, the test-retest reliability coefficient of CASI C-2.0 is 0.97,p<0.001, the person′s correlation coefficient between nine items scores and total scores are all above 0.66, the Cronback a is 0.9056, the split-half reliability coefficient is 0.8355. On validity, using ICD-10 criteria as ”Gold Criteria" and 50 as a cut-offscore, the sensitivity、 specificity、 accuracy of CASI C-2.0 are 94.5%、 89.5%、 89.8% respectively, kappa is 0.5123. Comparing with MMSE, CAS1 C-2.0 has a better specificity, x2 test, p<0.01; a same sensitivity; and a lower refusal answer ratio, x2 test, p<0.005. Conclusion: We consider that CASI C-2.0 has excellent reliability and validity. It is worthy of disseminating and applying for both clinical and epidemiological surveys.

  13. Fitness to plead: Development and validation of a standardised assessment instrument.

    Directory of Open Access Journals (Sweden)

    Penelope Brown

    Full Text Available The ability of an individual to participate in courtroom proceedings is assessed by clinicians using legal 'fitness to plead' criteria. Findings of 'unfitness' are so rare that there is considerable professional unease concerning the utility of the current subjective assessment process. As a result, mentally disordered defendants may be subjected unfairly to criminal trials. The Law Commission in England and Wales has proposed legal reform, as well as the utilisation of a defined psychiatric instrument to assist in fitness to plead assessments. Similar legal reforms are occurring in other jurisdictions. Our objective was to produce and validate a standardised assessment instrument of fitness to plead employing a filmed vignette of criminal proceedings. The instrument was developed in consultation with legal and clinical professionals, and was refined using standard item reduction methods in two initial rounds of testing (n = 212. The factorial structure, test-retest reliability and convergent validity of the resultant instrument were assessed in a further round (n = 160. As a result of this iterative process a 25-item scale was produced, with an underlying two-factor structure representing the foundational and decision-making abilities underpinning fitness to plead. The sub-scales demonstrate good internal consistency (factor 1: 0·76; factor 2: 0·65 and test-retest stability (0·7 as well as excellent convergent validity with scores of intelligence, executive function and mentalising abilities (p≤0·01 in all domains. Overall the standardised Fitness to Plead Assessment instrument has good psychometric properties. It has the potential to ensure that the significant numbers of mentally ill and cognitively impaired individuals who face trial are objectively assessed, and the courtroom process critically informed.

  14. Reliability assessment of PARR-1 renovated I and C system startup channel-b

    Energy Technology Data Exchange (ETDEWEB)

    Qamar, M.A.; Habib, M.A.; Iqbal, M.

    1991-11-20

    The report covers the reliability assessment of the startup channel-B of the PARR-1 renovated instrumentation and Control. For startup channel-A, a separate report (DNSRP-10(7)/88) was prepared in 1988. As the modules are of different origins in both the channels, a separate calculation was necessary. In the report the overall unreliability of the startup channel has been calculated by considering 1 out of 2 operation mode. In the assessment, the reliability calculations for the channel has been performed using similar assumptions as used in the authors' report on startup channel-A. For the analysis the Component Count Method (CCM) was applied. Comparison of results for channel-A and channel-B were made.

  15. Development and validation of an instrument to evaluate science teachers' assessment beliefs and practices

    Science.gov (United States)

    Genc, Evrim

    The primary purpose of this study was to develop a valid and reliable instrument to examine science teachers' assessment beliefs and practices in science classrooms. The present study also investigated the relationship between teachers' beliefs and practices in terms of assessment issues in science, their perceptions of the factors that influenced their assessment practices and their feelings towards high-stakes testing. The participants of the study were 408 science teachers teaching at middle and high school levels in the State of Florida. Data were collected through two modes of administration of the instrument as a paper-and-pencil and a web-based form. The response rate for paper-and-pencil administration was estimated as 68% whereas the response for the web administration was found to be 27%. Results from the various dimensions of validity and reliability analyses revealed that the 24 item-four-factor belief and practice measures were psychometrically sound and conceptually anchored measures of science teachers' assessment beliefs and self-reported practices. Reliability estimates for the belief measure ranged from .83 to .91 whereas alpha values for the practice measure ranged from .56 to .90. Results from the multigroup analysis supported that the instrument has the same theoretical structure across both administration groups. Therefore, future researchers may use either a paper-and-pencil or web-based format of the instrument. This study underscored a discrepancy between what teachers believe and how they act in classroom settings. It was emphasized that certain factors were mediating the dynamics between the belief and the practice. The majority of teachers reported that instruction time, class size, professional development activities, availability of school funding, and state testing mandates impact their assessment routines. Teachers reported that both the preparation process and the results of the test created unbelievable tension both on students and

  16. An instrument measuring prospective mathematics teacher self-regulated learning: validity and reliability

    Science.gov (United States)

    Nugroho, A. A.; Juniati, D.; Siswono, T. Y. E.

    2018-03-01

    Self Regulated Learning (SRL) is an individual's ability to achieve academic goals by controlling behavior, motivate yourself and use cognitive in learning, so it is important for a teacher especially teachers of mathematics related to the ability of management, design, implementation of learning and evaluation of learning outcomes. The purpose of the research is to develop an instrument to describe the SRL of a prospective mathematics teacher. Data were collected by (1) the study of the theory of SRL produced the indicator SRL used to design the questionnaire SRL; (2) analysis of the questionnaire SRL obtained from several References; and (3) development stage of the SRL questionnaire through validity test of content and empirical validation. The study involved 2 content experts in mathematics, 1 linguist, and 92 prospective mathematics teachers. The results of the research on content validity test based on Indonesian expert and 2 content experts indicate that the content can assess the indicator of the SRL and feasible to be used, in the test of legibility of two prospective mathematics teacher concluded that the instrument has a language that can be understood by the prospective teacher of mathematics and on empirical validation involving 92 prospective mathematics teacher generate data that of 65 statements there are 3 invalid statements. Reliability calculation shows high category that values 0,93. The conclusion is the SRL instrument developed for the prospective mathematics teacher.

  17. Interactive reliability assessment using an integrated reliability data bank

    International Nuclear Information System (INIS)

    Allan, R.N.; Whitehead, A.M.

    1986-01-01

    The logical structure, techniques and practical application of a computer-aided technique based on a microcomputer using floppy disc Random Access Files is described. This interactive computational technique is efficient if the reliability prediction program is coupled directly to a relevant source of data to create an integrated reliability assessment/reliability data bank system. (DG)

  18. Instruments to assess integrated care

    DEFF Research Database (Denmark)

    Lyngsø, Anne Marie; Godtfredsen, Nina Skavlan; Høst, Dorte

    2014-01-01

    INTRODUCTION: Although several measurement instruments have been developed to measure the level of integrated health care delivery, no standardised, validated instrument exists covering all aspects of integrated care. The purpose of this review is to identify the instruments concerning how to mea...... was prevalent. It is uncertain whether development of a single 'all-inclusive' model for assessing integrated care is desirable. We emphasise the continuing need for validated instruments embedded in theoretical contexts.......INTRODUCTION: Although several measurement instruments have been developed to measure the level of integrated health care delivery, no standardised, validated instrument exists covering all aspects of integrated care. The purpose of this review is to identify the instruments concerning how...... to measure the level of integration across health-care sectors and to assess and evaluate the organisational elements within the instruments identified. METHODS: An extensive, systematic literature review in PubMed, CINAHL, PsycINFO, Cochrane Library, Web of Science for the years 1980-2011. Selected...

  19. Assessing patient-centred communication in teaching: a systematic review of instruments.

    Science.gov (United States)

    Brouwers, Marianne; Rasenberg, Ellemieke; van Weel, Chris; Laan, Roland; van Weel-Baumgarten, Evelyn

    2017-11-01

    Patient-centred communication is a key component of patient centredness in medical care. Therefore, adequate education in and assessment of patient-centred communication skills are necessary. In general, feedback on communication skills is most effective when it is provided directly and is systematic. This calls for adequate measurement instruments. The aim of this study was to provide a systematic review of existing instruments that measure patient centredness in doctor-patient communication and can be used to provide direct feedback. A systematic review was conducted using an extensive validated search strategy for measurement instruments in PubMed, EMBASE, PsycINFO and CINAHL. The databases were searched from their inception to 1 July 2016. Articles describing the development or evaluation of the measurement properties of instruments that measure patient centredness (by applying three or more of the six dimensions of a published definition of patient centredness) in doctor-patient communication and that can be used for the provision of direct feedback were included. The methodological quality of measurement properties was evaluated using the COSMIN checklist. Thirteen articles describing 14 instruments measuring patient centredness in doctor-patient communication were identified. These studies cover a wide range of settings and patient populations, and vary in the dimensions of patient centredness applied and in methodological quality on aspects of reliability and validity. This review gives a comprehensive overview of all instruments available for the measurement of patient centredness in doctor-patient communication that can be used for the provision of direct feedback and are described in the literature. Despite the widely felt need for valid and reliable instruments for the measurement of patient-centred communication, most of the instruments currently available have not been thoroughly investigated. Therefore, we recommend further research into and

  20. A dynamic simulation model for assessing the overall impact of incentive policies on power system reliability, costs and environment

    International Nuclear Information System (INIS)

    Ibanez-Lopez, A.S.; Martinez-Val, J.M.; Moratilla-Soria, B.Y.

    2017-01-01

    The liberalization of power markets has entailed dramatic changes in power system planning worldwide. The inception of new alternative technologies, smart grids and distributed generation and storage is expected to make system planning even more challenging. Government policies still play a major role in the evolution of a country's power generation mix, even in those countries with liberalized markets. This paper presents a System Dynamics model aimed at assessing the overall technical, economic and environmental impact of renewable energy incentives and capacity payment policies. The model has been used to simulate Spain's power industry in order to assess the impact of electric power policies with the goal of getting insights regarding how to achieve an optimum power generation mix. The main conclusions of the present paper are (i) the necessity of specific regulatory actions in Spain in order to keep adequate reliability levels, avoid price spikes and boom and bust investment cycles as well as to deploy specific technologies, (ii) the fact that capacity payments are a better instrument for keeping adequate reserve margins and avoiding power price spikes than renewable energy incentives and (iii) the evidence that both instruments entail additional system costs over the base case scenario. - Highlights: • A System Dynamics model of Spain's power generation mix is proposed. • The overall policy impact on system costs, environment and reliability is assessed. • Current policies are not enough to keep adequate reliability levels. • Capacity payments are an adequate instrument for guaranteeing system reliability. • RES incentives do not solve reliability issues and entail greater system costs.

  1. Developing a theory-based instrument to assess the impact of continuing professional development activities on clinical practice: a study protocol

    Directory of Open Access Journals (Sweden)

    Rousseau Michel

    2011-03-01

    Full Text Available Abstract Background Continuing professional development (CPD is one of the principal means by which health professionals (i.e. primary care physicians and specialists maintain, improve, and broaden the knowledge and skills required for optimal patient care and safety. However, the lack of a widely accepted instrument to assess the impact of CPD activities on clinical practice thwarts researchers' comparisons of the effectiveness of CPD activities. Using an integrated model for the study of healthcare professionals' behaviour, our objective is to develop a theory-based, valid, reliable global instrument to assess the impact of accredited CPD activities on clinical practice. Methods Phase 1: We will analyze the instruments identified in a systematic review of factors influencing health professionals' behaviours using criteria that reflect the literature on measurement development and CPD decision makers' priorities. The outcome of this phase will be an inventory of instruments based on social cognitive theories. Phase 2: Working from this inventory, the most relevant instruments and their related items for assessing the concepts listed in the integrated model will be selected. Through an e-Delphi process, we will verify whether these instruments are acceptable, what aspects need revision, and whether important items are missing and should be added. The outcome of this phase will be a new global instrument integrating the most relevant tools to fit our integrated model of healthcare professionals' behaviour. Phase 3: Two data collections are planned: (1 a test-retest of the new instrument, including item analysis, to assess its reliability and (2 a study using the instrument before and after CPD activities with a randomly selected control group to explore the instrument's mere-measurement effect. Phase 4: We will conduct individual interviews and focus groups with key stakeholders to identify anticipated barriers and enablers for implementing the

  2. How to interpret safety critical failures in risk and reliability assessments

    International Nuclear Information System (INIS)

    Selvik, Jon Tømmerås; Signoret, Jean-Pierre

    2017-01-01

    Management of safety systems often receives high attention due to the potential for industrial accidents. In risk and reliability literature concerning such systems, and particularly concerning safety-instrumented systems, one frequently comes across the term ‘safety critical failure’. It is a term associated with the term ‘critical failure’, and it is often deduced that a safety critical failure refers to a failure occurring in a safety critical system. Although this is correct in some situations, it is not matching with for example the mathematical definition given in ISO/TR 12489:2013 on reliability modeling, where a clear distinction is made between ‘safe failures’ and ‘dangerous failures’. In this article, we show that different interpretations of the term ‘safety critical failure’ exist, and there is room for misinterpretations and misunderstandings regarding risk and reliability assessments where failure information linked to safety systems are used, and which could influence decision-making. The article gives some examples from the oil and gas industry, showing different possible interpretations of the term. In particular we discuss the link between criticality and failure. The article points in general to the importance of adequate risk communication when using the term, and gives some clarification on interpretation in risk and reliability assessments.

  3. Validation of the translation of an instrument to measure reliability of written information on treatment choices: a study on attention deficit/hyperactivity disorder (ADHD).

    Science.gov (United States)

    Montoya, A; Llopis, N; Gilaberte, I

    2011-12-01

    DISCERN is an instrument designed to help patients assess the reliability of written information on treatment choices. Originally created in English, there is no validated Spanish version of this instrument. This study seeks to validate the Spanish translation of the DISCERN instrument used as a primary measure on a multicenter study aimed to assess the reliability of web-based information on treatment choices for attention deficit/hyperactivity disorder (ADHD). We used a modified version of a method for validating translated instruments in which the original source-language version is formally compared with the back-translated source-language version. Each item was ranked in terms of comparability of language, similarity of interpretability, and degree of understandability. Responses used Likert scales ranging from 1 to 7, where 1 indicates the best interpretability, language and understandability, and 7 indicates the worst. Assessments were performed by 20 raters fluent in the source language. The Spanish translation of DISCERN, based on ratings of comparability, interpretability and degree of understandability (mean score (SD): 1.8 (1.1), 1.4 (0.9) and 1.6 (1.1), respectively), was considered extremely comparable. All items received a score of less than three, therefore no further revision of the translation was needed. The validation process showed that the quality of DISCERN translation was high, validating the comparable language of the tool translated on assessing written information on treatment choices for ADHD.

  4. The MacArthur Competence Assessment Tool-Fitness to Plead: a preliminary evaluation of a research instrument for assessing fitness to plead in England and Wales.

    Science.gov (United States)

    Akinkunmi, Akintunde A

    2002-01-01

    This study concerns the preliminary evaluation of an instrument, the MacArthur Competence Assessment Tool-Fitness to Plead (MacCAT-FP), for assessing competence to stand trial, according to the legal requirements in England and Wales. The purposes of the study were to ascertain whether the instrument can be administered practically to groups of prisoners, both those with mental illness and those without; to examine its internal consistency and interrater reliability; to determine whether it can distinguish between fit and unfit individuals, as judged by expert forensic psychiatrists; and to discover whether it is sensitive to changes over time. The instrument was administered to two groups of remanded prisoners: those transferred to psychiatric units for treatment and those without mental illness. In addition to the MacCAT-FP, scales measuring symptom severity and IQ were administered to all participants. Results suggest that the instrument is practical, with good internal consistency and interrater reliability. The correlation of scores with psychiatrists' opinions as to fitness was 0.77. Scores of unfit patients were significantly different from those of fit individuals. The instrument was able to detect significant differences in scores over time. It is suggested that once the instrument has been further validated for research purposes, it may be developed for clinical application.

  5. Reliability and maintainability assessment factors for reliable fault-tolerant systems

    Science.gov (United States)

    Bavuso, S. J.

    1984-01-01

    A long term goal of the NASA Langley Research Center is the development of a reliability assessment methodology of sufficient power to enable the credible comparison of the stochastic attributes of one ultrareliable system design against others. This methodology, developed over a 10 year period, is a combined analytic and simulative technique. An analytic component is the Computer Aided Reliability Estimation capability, third generation, or simply CARE III. A simulative component is the Gate Logic Software Simulator capability, or GLOSS. The numerous factors that potentially have a degrading effect on system reliability and the ways in which these factors that are peculiar to highly reliable fault tolerant systems are accounted for in credible reliability assessments. Also presented are the modeling difficulties that result from their inclusion and the ways in which CARE III and GLOSS mitigate the intractability of the heretofore unworkable mathematics.

  6. Validity and reliability of wii fit balance board for the assessment of balance of healthy young adults and the elderly.

    Science.gov (United States)

    Chang, Wen-Dien; Chang, Wan-Yi; Lee, Chia-Lun; Feng, Chi-Yen

    2013-10-01

    [Purpose] Balance is an integral part of human ability. The smart balance master system (SBM) is a balance test instrument with good reliability and validity, but it is expensive. Therefore, we modified a Wii Fit balance board, which is a convenient balance assessment tool, and analyzed its reliability and validity. [Subjects and Methods] We recruited 20 healthy young adults and 20 elderly people, and administered 3 balance tests. The correlation coefficient and intraclass correlation of both instruments were analyzed. [Results] There were no statistically significant differences in the 3 tests between the Wii Fit balance board and the SBM. The Wii Fit balance board had a good intraclass correlation (0.86-0.99) for the elderly people and positive correlations (r = 0.58-0.86) with the SBM. [Conclusions] The Wii Fit balance board is a balance assessment tool with good reliability and high validity for elderly people, and we recommend it as an alternative tool for assessing balance ability.

  7. A Database Management Assessment Instrument

    Science.gov (United States)

    Landry, Jeffrey P.; Pardue, J. Harold; Daigle, Roy; Longenecker, Herbert E., Jr.

    2013-01-01

    This paper describes an instrument designed for assessing learning outcomes in data management. In addition to assessment of student learning and ABET outcomes, we have also found the instrument to be effective for determining database placement of incoming information systems (IS) graduate students. Each of these three uses is discussed in this…

  8. Instruments assessing attitudes toward or capability regarding self-management of osteoarthritis: a systematic review of measurement properties.

    Science.gov (United States)

    Eyles, J P; Hunter, D J; Meneses, S R F; Collins, N J; Dobson, F; Lucas, B R; Mills, K

    2017-08-01

    To make a recommendation on the "best" instrument to assess attitudes toward and/or capabilities regarding self-management of osteoarthritis (OA) based on available measurement property evidence. Electronic searches were performed in MEDLINE, EMBASE, CINAHL and PsychINFO (inception to 27 December 2016). Two reviewers independently rated measurement properties using the Consensus-based Standards for the selection of Health Measurement Instruments (COSMIN) 4-point scale. Best evidence synthesis was determined by considering COSMIN ratings for measurement property results and the level of evidence available for each measurement property of each instrument. Eight studies out of 5653 publications met the inclusion criteria, with eight instruments identified for evaluation: Multidimensional Health Locus of Control (MHLC), Perceived Behavioural Control (PBC), Patient Activation Measure (PAM), Educational Needs Assessment (ENAT), Stages of Change Questionnaire in Osteoarthritis (SCQOA), Effective Consumer Scale (EC-17) and Perceived Efficacy in Patient-Physician Interactions five item (PEPPI-5) and ten item scales. Measurement properties assessed for these instruments included internal consistency (k = 8), structural validity (k = 8), test-retest reliability (k = 2), measurement error (k = 1), hypothesis testing (k = 3) and cross-cultural validity (k = 3). No information was available for content validity, responsiveness or minimal important change (MIC)/minimal important difference (MID). The Dutch PEPPI-5 demonstrated the best measurement property evidence; strong evidence for internal consistency and structural validity but limited evidence for reliability and construct validity. Although PEPPI-5 was identified as having the best measurement properties, overall there is a poor level of evidence currently available concerning measurement properties of instruments to assess attitudes toward and/or capabilities regarding osteoarthritis self-management. Further

  9. Reliability and Validity of the Korean Cancer Pain Assessment Tool (KCPAT)

    Science.gov (United States)

    Kim, Jeong A; Lee, Juneyoung; Park, Jeanno; Lee, Myung Ah; Yeom, Chang Hwan; Jang, Se Kwon; Yoon, Duck Mi; Kim, Jun Suk

    2005-01-01

    The Korean Cancer Pain Assessment Tool (KCPAT), which was developed in 2003, consists of questions concerning the location of pain, the nature of pain, the present pain intensity, the symptoms associated with the pain, and psychosocial/spiritual pain assessments. This study was carried out to evaluate the reliability and validity of the KCPAT. A stratified, proportional-quota, clustered, systematic sampling procedure was used. The study population (903 cancer patients) was 1% of the target population (90,252 cancer patients). A total of 314 (34.8%) questionnaires were collected. The results showed that the average pain score (5 point on Likert scale) according to the cancer type and the at-present average pain score (VAS, 0-10) were correlated (r=0.56, p<0.0001), and showed moderate agreement (kappa=0.364). The mean satisfaction score was 3.8 (1-5). The average time to complete the questionnaire was 8.9 min. In conclusion, the KCPAT is a reliable and valid instrument for assessing cancer pain in Koreans. PMID:16224166

  10. Design and validation of the INICIARE instrument, for the assessment of dependency level in acutely ill hospitalised patients.

    Science.gov (United States)

    Morales-Asencio, José Miguel; Porcel-Gálvez, Ana María; Oliveros-Valenzuela, Rosa; Rodríguez-Gómez, Susana; Sánchez-Extremera, Lucrecia; Serrano-López, Francisco Andrés; Aranda-Gallardo, Marta; Canca-Sánchez, José Carlos; Barrientos-Trigo, Sergio

    2015-03-01

    The aim of this study was to establish the validity and reliability of an instrument (Inventario del NIvel de Cuidados mediante IndicAdores de clasificación de Resultados de Enfermería) used to assess the dependency level in acutely hospitalised patients. This instrument is novel, and it is based on the Nursing Outcomes Classification. Multiple existing instruments for needs assessment have been poorly validated and based predominately on interventions. Standardised Nursing Languages offer an ideal framework to develop nursing sensitive instruments. A cross-sectional validation study in two acute care hospitals in Spain. This study was implemented in two phases. First, the research team developed the instrument to be validated. In the second phase, the validation process was performed by experts, and the data analysis was conducted to establish the psychometric properties of the instrument. Seven hundred and sixty-one patient ratings performed by nurses were collected during the course of the research study. Data analysis yielded a Cronbach's alpha of 0·91. An exploratory factorial analysis identified three factors (Physiological, Instrumental and Cognitive-behavioural), which explained 74% of the variance. Inventario del NIvel de Cuidados mediante IndicAdores de clasificación de Resultados de Enfermería was demonstrated to be a valid and reliable instrument based on its use in acutely hospitalised patients to assess the level of dependency. Inventario del NIvel de Cuidados mediante IndicAdores de clasificación de Resultados de Enfermería can be used as an assessment tool in hospitalised patients during the nursing process throughout the entire hospitalisation period. It contributes information to support decisions on nursing diagnoses, interventions and outcomes. It also enables data codification in large databases. © 2014 John Wiley & Sons Ltd.

  11. An approach based on defense-in-depth and diversity (3D) for the reliability assessment of digital instrument and control systems of nuclear power plants

    Energy Technology Data Exchange (ETDEWEB)

    Silva, Paulo Adriano da; Saldanha, Pedro L.C., E-mail: pasilva@cnen.gov.b, E-mail: Saldanha@cnen.gov.b [Comissao Nacional de Energia Nuclear (CNEN), Rio de Janeiro, RJ (Brazil). Coord. Geral de Reatores Nucleares; Melo, Paulo F. Frutuoso e, E-mail: frutuoso@nuclear.ufrj.b [Universidade Federal do Rio de Janeiro (PEN/COPPE/UFRJ), RJ (Brazil). Coordenacao dos Programas de Pos-Graduacao em Engenharia. Programa de Engenharia Nuclear; Araujo, Ademir L. de [Associacao Brasileira de Ensino Universitario (UNIABEU), Angra dos Reis, RJ (Brazil)

    2011-07-01

    The adoption of instrumentation and control (I and C) digital technology has been slower in nuclear power plants. The reason has been unfruitful efforts to obtain evidence in order to prove that I and C systems can be used in nuclear safety systems, for example, the Reactor Protection System (RPS), ensuring the proper operation of all its functions. This technology offers a potential improvement for safety and reliability. However, there still no consensus about the model to be adopted for digital systems software to be used in reliability studies. This paper presents the 3D methodology approach to assess digital I and C reliability. It is based on the study of operational events occurring in NPPs. It is easy to identify, in general, the level of I and C system reliability, showing its key vulnerabilities, enabling to trace regulatory actions to minimize or avoid them. This approach makes it possible to identify the main types of digital I and C system failure, with the potential for common cause failures as well as evaluating the dominant failure modes. The MAFIC-D software was developed to assist the implementation of the relationships between the reliability criteria, the analysis of relationships and data collection. The results obtained through this tool proved to be satisfactory and complete the process of regulatory decision-making from licensing I and C digital of NPPs and call still be used to monitor the performance of I and C digital post-licensing during the lifetime of the system, providing the basis for the elaboration of checklists of regulatory inspections. (author)

  12. An approach based on defense-in-depth and diversity (3D) for the reliability assessment of digital instrument and control systems of nuclear power plants

    International Nuclear Information System (INIS)

    Silva, Paulo Adriano da; Saldanha, Pedro L.C.

    2011-01-01

    The adoption of instrumentation and control (I and C) digital technology has been slower in nuclear power plants. The reason has been unfruitful efforts to obtain evidence in order to prove that I and C systems can be used in nuclear safety systems, for example, the Reactor Protection System (RPS), ensuring the proper operation of all its functions. This technology offers a potential improvement for safety and reliability. However, there still no consensus about the model to be adopted for digital systems software to be used in reliability studies. This paper presents the 3D methodology approach to assess digital I and C reliability. It is based on the study of operational events occurring in NPPs. It is easy to identify, in general, the level of I and C system reliability, showing its key vulnerabilities, enabling to trace regulatory actions to minimize or avoid them. This approach makes it possible to identify the main types of digital I and C system failure, with the potential for common cause failures as well as evaluating the dominant failure modes. The MAFIC-D software was developed to assist the implementation of the relationships between the reliability criteria, the analysis of relationships and data collection. The results obtained through this tool proved to be satisfactory and complete the process of regulatory decision-making from licensing I and C digital of NPPs and call still be used to monitor the performance of I and C digital post-licensing during the lifetime of the system, providing the basis for the elaboration of checklists of regulatory inspections. (author)

  13. Human reliability assessment and probabilistic risk assessment

    International Nuclear Information System (INIS)

    Embrey, D.E.; Lucas, D.A.

    1989-01-01

    Human reliability assessment (HRA) is used within Probabilistic Risk Assessment (PRA) to identify the human errors (both omission and commission) which have a significant effect on the overall safety of the system and to quantify the probability of their occurrence. There exist a variey of HRA techniques and the selection of an appropriate one is often difficult. This paper reviews a number of available HRA techniques and discusses their strengths and weaknesses. The techniques reviewed include: decompositional methods, time-reliability curves and systematic expert judgement techniques. (orig.)

  14. Developing an assessment of fire-setting to guide treatment in secure settings: the St Andrew's Fire and Arson Risk Instrument (SAFARI).

    Science.gov (United States)

    Long, Clive G; Banyard, Ellen; Fulton, Barbara; Hollin, Clive R

    2014-09-01

    Arson and fire-setting are highly prevalent among patients in secure psychiatric settings but there is an absence of valid and reliable assessment instruments and no evidence of a significant approach to intervention. To develop a semi-structured interview assessment specifically for fire-setting to augment structured assessments of risk and need. The extant literature was used to frame interview questions relating to the antecedents, behaviour and consequences necessary to formulate a functional analysis. Questions also covered readiness to change, fire-setting self-efficacy, the probability of future fire-setting, barriers to change, and understanding of fire-setting behaviour. The assessment concludes with indications for assessment and a treatment action plan. The inventory was piloted with a sample of women in secure care and was assessed for comprehensibility, reliability and validity. Staff rated the St Andrews Fire and Risk Instrument (SAFARI) as acceptable to patients and easy to administer. SAFARI was found to be comprehensible by over 95% of the general population, to have good acceptance, high internal reliability, substantial test-retest reliability and validity. SAFARI helps to provide a clear explanation of fire-setting in terms of the complex interplay of antecedents and consequences and facilitates the design of an individually tailored treatment programme in sympathy with a cognitive-behavioural approach. Further studies are needed to verify the reliability and validity of SAFARI with male populations and across settings.

  15. Development and validation of an instrument to assess the prescribing readiness of medical students in Malaysia.

    Science.gov (United States)

    Lai, Pauline Siew Mei; Sim, Si Mui; Chua, Siew Siang; Tan, Choo Hock; Ng, Chirk Jenn; Achike, Francis Ifejika; Teng, Cheong Lieng

    2015-09-21

    Prescribing incompetence is an important factor that contributes to prescribing error, and this is often due to inadequate training during medical schools. We therefore aimed to develop and validate an instrument to assess the prescribing readiness of medical students (PROMS) in Malaysia. The PROMS comprised of 26 items with four domains: undergraduate learning opportunities; hands-on clinical skills practice; information gathering behaviour; and factors affecting the learning of prescribing skills. The first three domains were adapted from an existing questionnaire, while items from the last domain were formulated based on findings from a nominal group discussion. Face and content validity was determined by an expert panel, pilot tested in a class of final year (Year 5) medical students, and assessed using the Flesch reading ease. To assess the reliability of the PROMS, the internal consistency and test-retest (at baseline and 2 weeks later) were assessed using the Wilcoxon Signed Ranks test and Spearman's rho. The discriminative validity of the PROMS was assessed using the Mann-Whitney U-test (to assess if the PROMS could discriminate between final year medical students from a public and a private university). A total of 119 medical students were recruited. Flesch reading ease was 46.9, indicating that the instrument was suitable for use in participants undergoing tertiary education. The overall Cronbach alpha value of the PROMS was 0.695, which was satisfactory. Test-retest showed no difference for 25/26 items, indicating that our instrument was reliable. Responses from the public and private university final year medical students were significantly different in 10/26 items, indicating that the PROMS was able to discriminate between these two groups. Medical students from the private university reported fewer learning opportunities and hands-on practice compared to those from the public university. On the other hand, medical students from the private university

  16. Reliability of Circumplex Axes

    Directory of Open Access Journals (Sweden)

    Micha Strack

    2013-06-01

    Full Text Available We present a confirmatory factor analysis (CFA procedure for computing the reliability of circumplex axes. The tau-equivalent CFA variance decomposition model estimates five variance components: general factor, axes, scale-specificity, block-specificity, and item-specificity. Only the axes variance component is used for reliability estimation. We apply the model to six circumplex types and 13 instruments assessing interpersonal and motivational constructs—Interpersonal Adjective List (IAL, Interpersonal Adjective Scales (revised; IAS-R, Inventory of Interpersonal Problems (IIP, Impact Messages Inventory (IMI, Circumplex Scales of Interpersonal Values (CSIV, Support Action Scale Circumplex (SAS-C, Interaction Problems With Animals (IPI-A, Team Role Circle (TRC, Competing Values Leadership Instrument (CV-LI, Love Styles, Organizational Culture Assessment Instrument (OCAI, Customer Orientation Circle (COC, and System for Multi-Level Observation of Groups (behavioral adjectives; SYMLOG—in 17 German-speaking samples (29 subsamples, grouped by self-report, other report, and metaperception assessments. The general factor accounted for a proportion ranging from 1% to 48% of the item variance, the axes component for 2% to 30%; and scale specificity for 1% to 28%, respectively. Reliability estimates varied considerably from .13 to .92. An application of the Nunnally and Bernstein formula proposed by Markey, Markey, and Tinsley overestimated axes reliabilities in cases of large-scale specificities but otherwise works effectively. Contemporary circumplex evaluations such as Tracey’s RANDALL are sensitive to the ratio of the axes and scale-specificity components. In contrast, the proposed model isolates both components.

  17. Development, reliability, and validity of the Posttraumatic Stress Disorder Interview for Vietnamese refugees: a diagnostic instrument for Vietnamese refugees.

    Science.gov (United States)

    Dao, Tam K; Poritz, Julia M P; Moody, Rachel P; Szeto, Kim

    2012-08-01

    The Posttraumatic Stress Disorder Interview for Vietnamese Refugees (PTSD-IVR) was created specifically to assess for the presence of current and lifetime history of premigration, migration, encampment, and postmigration traumas in Vietnamese refugees. The purpose of the present study was to describe the development of and investigate the interrater and test-retest reliability of the PTSD-IVR and its validity in relation to the diagnoses obtained from the Longitudinal, Expert, and All Data (LEAD; Spitzer, 1983) standard. Clinicians conducted the diagnosis process with 127 Vietnamese refugees using the LEAD standard and the PTSD-IVR. Assessment of the reliability and validity of the PTSD-IVR yielded good to excellent AUC (area under the receiver operating characteristic curve; .86, .87) and κ values (.66, .74) indicating the reliability of the PTSD-IVR and the agreement between the LEAD procedure and the PTSD-IVR. The results of the present study suggest that the PTSD-IVR performs successfully as a diagnostic instrument specifically created for Vietnamese refugees in their native language. Copyright © 2012 International Society for Traumatic Stress Studies.

  18. Satisfaction and adherence in patients with iron overload receiving iron chelation therapy as assessed by a newly developed patient instrument.

    Science.gov (United States)

    Rofail, Diana; Abetz, Linda; Viala, Muriel; Gait, Claire; Baladi, Jean-Francois; Payne, Krista

    2009-01-01

    This study assesses satisfaction with iron chelation therapy (ICT) based on a reliable and valid instrument, and explores the relationship between satisfaction and adherence to ICT. Patients in the USA and UK completed a new "Satisfaction with ICT" (SICT) instrument consisting of 28 items, three pertaining to adherence. Simple and multivariate regression analyses assessed the relationship between satisfaction with different aspects of ICT and adherence. First assessments of the SICT instrument indicate its validity and reliability. Recommended thresholds for internal consistency, convergent validity, discriminant validity, and floor and ceiling effects were met. A number of variables were identified in the simple linear regression analyses as significant predictors of "never thinking about stopping ICT," a proxy for adherence. These significant variables were entered into the multivariate model to assess the combined factor effects, explaining 42% of the total variance of "never thinking about stopping ICT." A significant and positive relationship was demonstrated between "never thinking about stopping ICT" and age (P = 0.04), Perceived Effectiveness of ICT (P = 0.003), low Burden of ICT (P = 0.002), and low Side Effects of ICT (P = 0.01). The SICT is a reliable and valid instrument which will be useful in ICT clinical trials. Furthermore, the administration of ICT by slow subcutaneous infusion negatively impacts on satisfaction with ICT which was shown to be a determinant of adherence. This points to the need for new more convenient and less burdensome oral iron chelators to increase adherence, and ultimately to improve patient outcomes.

  19. A comparative study on the HW reliability assessment methods for digital I and C equipment

    Energy Technology Data Exchange (ETDEWEB)

    Jung, Hoan Sung; Sung, T. Y.; Eom, H. S.; Park, J. K.; Kang, H. G.; Lee, G. Y. [Korea Atomic Energy Research Institute, Taejeon (Korea); Kim, M. C. [Korea Advanced Institute of Science and Technology, Taejeon (Korea); Jun, S. T. [KHNP, Taejeon (Korea)

    2002-03-01

    It is necessary to predict or to evaluate the reliability of electronic equipment for the probabilistic safety analysis of digital instrument and control equipment. But most databases for the reliability prediction have no data for the up-to-date equipment and the failure modes are not classified. The prediction results for the specific component show different values according to the methods and databases. For boards and systems each method shows different values than others also. This study is for reliability prediction of PDC system for Wolsong NPP1 as a digital I and C equipment. Various reliability prediction methods and failure databases are used in calculation of the reliability to compare the effects of sensitivity and accuracy of each model and database. Many considerations for the reliability assessment of digital systems are derived with the results of this study. 14 refs., 19 figs., 15 tabs. (Author)

  20. Review of Software Reliability Assessment Methodologies for Digital I and C Software of Nuclear Power Plants

    Energy Technology Data Exchange (ETDEWEB)

    Cho, Jae Hyun; Lee, Seung Jun; Jung, Won Dea [KAERI, Daejeon (Korea, Republic of)

    2014-08-15

    Digital instrumentation and control (I and C) systems are increasingly being applied to current nuclear power plants (NPPs) due to its advantages; zero drift, advanced data calculation capacity, and design flexibility. Accordingly, safety issues of software that is main part of the digital I and C system have been raised. As with hardware components, the software failure in NPPs could lead to a large disaster, therefore failure rate test and reliability assessment of software should be properly performed, and after that adopted in NPPs. However, the reliability assessment of the software is quite different with that of hardware, owing to the nature difference between software and hardware. The one of the most different thing is that the software failures arising from design faults as 'error crystal', whereas the hardware failures are caused by deficiencies in design, production, and maintenance. For this reason, software reliability assessment has been focused on the optimal release time considering the economy. However, the safety goal and public acceptance of the NPPs is so distinctive with other industries that the software in NPPs is dependent on reliability quantitative value rather than economy. The safety goal of NPPs compared to other industries is exceptionally high, so conventional methodologies on software reliability assessment already used in other industries could not adjust to safety goal of NPPs. Thus, the new reliability assessment methodology of the software of digital I and C on NPPs need to be developed. In this paper, existing software reliability assessment methodologies are reviewed to obtain the pros and cons of them, and then to assess the usefulness of each method to software of NPPs.

  1. Review of Software Reliability Assessment Methodologies for Digital I and C Software of Nuclear Power Plants

    International Nuclear Information System (INIS)

    Cho, Jae Hyun; Lee, Seung Jun; Jung, Won Dea

    2014-01-01

    Digital instrumentation and control (I and C) systems are increasingly being applied to current nuclear power plants (NPPs) due to its advantages; zero drift, advanced data calculation capacity, and design flexibility. Accordingly, safety issues of software that is main part of the digital I and C system have been raised. As with hardware components, the software failure in NPPs could lead to a large disaster, therefore failure rate test and reliability assessment of software should be properly performed, and after that adopted in NPPs. However, the reliability assessment of the software is quite different with that of hardware, owing to the nature difference between software and hardware. The one of the most different thing is that the software failures arising from design faults as 'error crystal', whereas the hardware failures are caused by deficiencies in design, production, and maintenance. For this reason, software reliability assessment has been focused on the optimal release time considering the economy. However, the safety goal and public acceptance of the NPPs is so distinctive with other industries that the software in NPPs is dependent on reliability quantitative value rather than economy. The safety goal of NPPs compared to other industries is exceptionally high, so conventional methodologies on software reliability assessment already used in other industries could not adjust to safety goal of NPPs. Thus, the new reliability assessment methodology of the software of digital I and C on NPPs need to be developed. In this paper, existing software reliability assessment methodologies are reviewed to obtain the pros and cons of them, and then to assess the usefulness of each method to software of NPPs

  2. Assessing the Learning Environment for Medical Students: An Evaluation of a Novel Survey Instrument in Four Medical Schools.

    Science.gov (United States)

    Pololi, Linda H; Evans, Arthur T; Nickell, Leslie; Reboli, Annette C; Coplit, Lisa D; Stuber, Margaret L; Vasiliou, Vasilia; Civian, Janet T; Brennan, Robert T

    2017-06-01

    A practical, reliable, and valid instrument is needed to measure the impact of the learning environment on medical students' well-being and educational experience and to meet medical school accreditation requirements. From 2012 to 2015, medical students were surveyed at the end of their first, second, and third year of studies at four medical schools. The survey assessed students' perceptions of the following nine dimensions of the school culture: vitality, self-efficacy, institutional support, relationships/inclusion, values alignment, ethical/moral distress, work-life integration, gender equity, and ethnic minority equity. The internal reliability of each of the nine dimensions was measured. Construct validity was evaluated by assessing relationships predicted by our conceptual model and prior research. Assessment was made of whether the measurements were sensitive to differences over time and across institutions. Six hundred and eighty-six students completed the survey (49 % women; 9 % underrepresented minorities), with a response rate of 89 % (range over the student cohorts 72-100 %). Internal consistency of each dimension was high (Cronbach's α 0.71-0.86). The instrument was able to detect significant differences in the learning environment across institutions and over time. Construct validity was supported by demonstrating several relationships predicted by our conceptual model. The C-Change Medical Student Survey is a practical, reliable, and valid instrument for assessing the learning environment of medical students. Because it is sensitive to changes over time and differences across institution, results could potentially be used to facilitate and monitor improvements in the learning environment of medical students.

  3. [Caregiving consequences in mental disorders--definitions and instruments of assessment].

    Science.gov (United States)

    Ciałkowska-Kuźmińska, Magdalena; Kiejna, Andrzej

    2010-01-01

    Severe mental illnesses have far-reaching consequences for both patients and their relatives. This paper reviews literature on the measures of caregiving consequences. Authors provide a condensed knowledge and research results in the area of caregiving consequences, especially both subjective and objective caregivers' burden. The consequences of care apply to carers' social and leisure activities, financial status, health condition. The burden of care has three fundamental causes: the reorganisation of mental health services, a social isolation of patients and their families and the lack of systemic support for caregivers. The problem of caregiving consequences has been investigated in several studies. In order to identify factors, which have impact on caregiver distress, a variety instruments have been developed. This paper focuses on questionnaires useful for the systematic assessment of both objective and subjective burden: Involvement Evaluation Questionnaire (IEQ), Perceived Family Burden Scale (PFBS), Zarit Caregiver Burden Scale (ZCBS), Experience of Caregiving Inventory (ECI), Family Problems Questionnaire (FPQ). The mentioned instruments proved to be a reliable instrument for measuring caregiver consequences in mental healthcare.

  4. Vitiligo impact scale: An instrument to assess the psychosocial burden of vitiligo

    Directory of Open Access Journals (Sweden)

    Gaurang S Krishna

    2013-01-01

    Full Text Available Background : Vitiligo is a disease that significantly impairs quality of life. Previous studies have shown that vitiligo has an impact that may not correlate with the size and extent of depigmentation, indicating a need for an independent measure of the psychosocial burden. Aims : To develop a rating scale to assess the psychosocial impact of vitiligo. Methods : The study was undertaken in three broad phases: item generation, pre- and pilot testing, and test administration. Items were generated largely from a qualitative study using semi-structured interviews of patients. Face and content validity were assessed through pre- and pilot testing in 80 patients and the final version was administered to 100 patients who also received the Dermatology Life Quality Index (DLQI and the Skindex-16. Each patient also underwent a physician global assessment (PGA of the impact of vitiligo. Test-retest reliability was assessed in 20 patients. Results: Of 72 items initially generated for the scale, 27 were retained in the final version. Subjects were able to comprehend the items and took about 5-7 min to complete the instrument. The scale was internally consistent (Cronbach′s α = 0.85. Scores on the scale correlated moderately well with the DLQI and the Skindex (Spearman rank correlation: 0.51 and 0.65, respectively. The scale was able to discriminate between patients having mild and those having moderate and severe impact as assessed by PGA. The test-retest reliability coefficient (Spearman rank correlation was 0.80. Conclusion: The Vitiligo Impact Scale appears to be a valid measure of the psychosocial impact of vitiligo and this instrument may be useful both in the clinic and in clinical trials.

  5. Incorporation of Markov reliability models for digital instrumentation and control systems into existing PRAs

    International Nuclear Information System (INIS)

    Bucci, P.; Mangan, L. A.; Kirschenbaum, J.; Mandelli, D.; Aldemir, T.; Arndt, S. A.

    2006-01-01

    Markov models have the ability to capture the statistical dependence between failure events that can arise in the presence of complex dynamic interactions between components of digital instrumentation and control systems. One obstacle to the use of such models in an existing probabilistic risk assessment (PRA) is that most of the currently available PRA software is based on the static event-tree/fault-tree methodology which often cannot represent such interactions. We present an approach to the integration of Markov reliability models into existing PRAs by describing the Markov model of a digital steam generator feedwater level control system, how dynamic event trees (DETs) can be generated from the model, and how the DETs can be incorporated into an existing PRA with the SAPHIRE software. (authors)

  6. Validation of the Family Meeting Behavioral Skills Checklist. An Instrument to Assess Fellows' Communication Skills.

    Science.gov (United States)

    Gustin, Jillian L; Way, David P; Wells-Di Gregorio, Sharla; McCallister, Jennifer W

    2016-08-01

    Fellows in pulmonary and critical care medicine are required to show competency in facilitating family meetings for critically ill patients. There are many assessment measures available for evaluating physician-patient communication (e.g., the SEGUE Framework [Set the stage, Elicit information, Give information, Understand the patient's perspective, End the encounter]) and some designed for family meetings. However, no validated measure exists that is specifically designed to assess communication skills during family meetings with surrogate decision makers in intensive care settings. We developed the Family Meeting Behavioral Skills Checklist (FMBSC) to measure advanced communication skills of fellows in family meetings of critically ill patients based on a literature review and consensus of an interdisciplinary group of communications experts. We evaluated the psychometric properties of the FMBSC. We digitally recorded 16 pulmonary/critical care fellows performing a simulated family meeting for a critically ill patient at the end of 1 year of fellowship training. Two clinical health psychologists evaluated each recording independently using the FMBSC Rating Scale and the SEGUE Framework. Judges recorded the number of skills performed using the checklist and employed a summary rating scale to judge the level of performance for each of nine subsets of skills. Each instrument was scored and converted to percentage scores. The FMBSC and SEGUE Framework items were summed and converted to percentage scores for each category and as a total for each instrument. The rating scale items on the FMBSC were also summed and converted to a percentage score. Four primary analyses were conducted to evaluate interjudge reliability, internal consistency, and concurrent validity. Interrater reliability was higher for the FMBSC (intraclass correlation [ICC2,2] = 0.57) than for the SEGUE instrument (ICC2,2 = 0.32) or the FMBSC Rating Scale (ICC2,2 = 0.23). The FMBSC

  7. Validity, reliability and support for implementation of independence-scaled procedural assessment in laparoscopic surgery.

    Science.gov (United States)

    Kramp, Kelvin H; van Det, Marc J; Veeger, Nic J G M; Pierie, Jean-Pierre E N

    2016-06-01

    There is no widely used method to evaluate procedure-specific laparoscopic skills. The first aim of this study was to develop a procedure-based assessment method. The second aim was to compare its validity, reliability and feasibility with currently available global rating scales (GRSs). An independence-scaled procedural assessment was created by linking the procedural key steps of the laparoscopic cholecystectomy to an independence scale. Subtitled and blinded videos of a novice, an intermediate and an almost competent trainee, were evaluated with GRSs (OSATS and GOALS) and the independence-scaled procedural assessment by seven surgeons, three senior trainees and six scrub nurses. Participants received a short introduction to the GRSs and independence-scaled procedural assessment before assessment. The validity was estimated with the Friedman and Wilcoxon test and the reliability with the intra-class correlation coefficient (ICC). A questionnaire was used to evaluate user opinion. Independence-scaled procedural assessment and GRS scores improved significantly with surgical experience (OSATS p = 0.001, GOALS p < 0.001, independence-scaled procedural assessment p < 0.001). The ICCs of the OSATS, GOALS and independence-scaled procedural assessment were 0.78, 0.74 and 0.84, respectively, among surgeons. The ICCs increased when the ratings of scrub nurses were added to those of the surgeons. The independence-scaled procedural assessment was not considered more of an administrative burden than the GRSs (p = 0.692). A procedural assessment created by combining procedural key steps to an independence scale is a valid, reliable and acceptable assessment instrument in surgery. In contrast to the GRSs, the reliability of the independence-scaled procedural assessment exceeded the threshold of 0.8, indicating that it can also be used for summative assessment. It furthermore seems that scrub nurses can assess the operative competence of surgical trainees.

  8. Reliability of Computerized Neurocognitive Tests for Concussion Assessment: A Meta-Analysis.

    Science.gov (United States)

    Farnsworth, James L; Dargo, Lucas; Ragan, Brian G; Kang, Minsoo

    2017-09-01

      Although widely used, computerized neurocognitive tests (CNTs) have been criticized because of low reliability and poor sensitivity. A systematic review was published summarizing the reliability of Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) scores; however, this was limited to a single CNT. Expansion of the previous review to include additional CNTs and a meta-analysis is needed. Therefore, our purpose was to analyze reliability data for CNTs using meta-analysis and examine moderating factors that may influence reliability.   A systematic literature search (key terms: reliability, computerized neurocognitive test, concussion) of electronic databases (MEDLINE, PubMed, Google Scholar, and SPORTDiscus) was conducted to identify relevant studies.   Studies were included if they met all of the following criteria: used a test-retest design, involved at least 1 CNT, provided sufficient statistical data to allow for effect-size calculation, and were published in English.   Two independent reviewers investigated each article to assess inclusion criteria. Eighteen studies involving 2674 participants were retained. Intraclass correlation coefficients were extracted to calculate effect sizes and determine overall reliability. The Fisher Z transformation adjusted for sampling error associated with averaging correlations. Moderator analyses were conducted to evaluate the effects of the length of the test-retest interval, intraclass correlation coefficient model selection, participant demographics, and study design on reliability. Heterogeneity was evaluated using the Cochran Q statistic.   The proportion of acceptable outcomes was greatest for the Axon Sports CogState Test (75%) and lowest for the ImPACT (25%). Moderator analyses indicated that the type of intraclass correlation coefficient model used significantly influenced effect-size estimates, accounting for 17% of the variation in reliability.   The Axon Sports CogState Test, which

  9. Assessing learning outcomes in middle-division classical mechanics: The Colorado Classical Mechanics and Math Methods Instrument

    Science.gov (United States)

    Caballero, Marcos D.; Doughty, Leanne; Turnbull, Anna M.; Pepper, Rachel E.; Pollock, Steven J.

    2017-06-01

    Reliable and validated assessments of introductory physics have been instrumental in driving curricular and pedagogical reforms that lead to improved student learning. As part of an effort to systematically improve our sophomore-level classical mechanics and math methods course (CM 1) at CU Boulder, we have developed a tool to assess student learning of CM 1 concepts in the upper division. The Colorado Classical Mechanics and Math Methods Instrument (CCMI) builds on faculty consensus learning goals and systematic observations of student difficulties. The result is a 9-question open-ended post test that probes student learning in the first half of a two-semester classical mechanics and math methods sequence. In this paper, we describe the design and development of this instrument, its validation, and measurements made in classes at CU Boulder and elsewhere.

  10. Reliability of the Q Force; a mobile instrument for measuring isometric quadriceps muscle strength

    OpenAIRE

    Schans, van der, C.P.; Zijlstra, W.; Regterschot, G.R.H.; Krijnen, W.P.; Douma, K.W.; Slager, G.E.C.

    2016-01-01

    BACKGROUND: The ability to generate muscle strength is a pre-requisite for all human movement. Decreased quadriceps muscle strength is frequently observed in older adults and is associated with a decreased performance and activity limitations. To quantify the quadriceps muscle strength and to monitor changes over time, instruments and procedures with a sufficient reliability are needed. The Q Force is an innovative mobile muscle strength measurement instrument suitable to measure in various d...

  11. Dynamic reliability assessment and prediction for repairable systems with interval-censored data

    International Nuclear Information System (INIS)

    Peng, Yizhen; Wang, Yu; Zi, YanYang; Tsui, Kwok-Leung; Zhang, Chuhua

    2017-01-01

    The ‘Test, Analyze and Fix’ process is widely applied to improve the reliability of a repairable system. In this process, dynamic reliability assessment for the system has been paid a great deal of attention. Due to instrument malfunctions, staff omissions and imperfect inspection strategies, field reliability data are often subject to interval censoring, making dynamic reliability assessment become a difficult task. Most traditional methods assume this kind of data as multiple normal distributed variables or the missing mechanism as missing at random, which may cause a large bias in parameter estimation. This paper proposes a novel method to evaluate and predict the dynamic reliability of a repairable system subject to interval-censored problem. First, a multiple imputation strategy based on the assumption that the reliability growth trend follows a nonhomogeneous Poisson process is developed to derive the distributions of missing data. Second, a new order statistic model that can transfer the dependent variables into independent variables is developed to simplify the imputation procedure. The unknown parameters of the model are iteratively inferred by the Monte Carlo expectation maximization (MCEM) algorithm. Finally, to verify the effectiveness of the proposed method, a simulation and a real case study for gas pipeline compressor system are implemented. - Highlights: • A new multiple imputation strategy was developed to derive the PDF of missing data. • A new order statistic model was developed to simplify the imputation procedure. • The parameters of the order statistic model were iteratively inferred by MCEM. • A real cases study was conducted to verify the effectiveness of the proposed method.

  12. Validation of an instrument to assess evidence-based practice knowledge, attitudes, access, and confidence in the dental environment.

    Science.gov (United States)

    Hendricson, William D; Rugh, John D; Hatch, John P; Stark, Debra L; Deahl, Thomas; Wallmann, Elizabeth R

    2011-02-01

    This article reports the validation of an assessment instrument designed to measure the outcomes of training in evidence-based practice (EBP) in the context of dentistry. Four EBP dimensions are measured by this instrument: 1) understanding of EBP concepts, 2) attitudes about EBP, 3) evidence-accessing methods, and 4) confidence in critical appraisal. The instrument-the Knowledge, Attitudes, Access, and Confidence Evaluation (KACE)-has four scales, with a total of thirty-five items: EBP knowledge (ten items), EBP attitudes (ten), accessing evidence (nine), and confidence (six). Four elements of validity were assessed: consistency of items within the KACE scales (extent to which items within a scale measure the same dimension), discrimination (capacity to detect differences between individuals with different training or experience), responsiveness (capacity to detect the effects of education on trainees), and test-retest reliability. Internal consistency of scales was assessed by analyzing responses of second-year dental students, dental residents, and dental faculty members using Cronbach coefficient alpha, a statistical measure of reliability. Discriminative validity was assessed by comparing KACE scores for the three groups. Responsiveness was assessed by comparing pre- and post-training responses for dental students and residents. To measure test-retest reliability, the full KACE was completed twice by a class of freshman dental students seventeen days apart, and the knowledge scale was completed twice by sixteen faculty members fourteen days apart. Item-to-scale consistency ranged from 0.21 to 0.78 for knowledge, 0.57 to 0.83 for attitude, 0.70 to 0.84 for accessing evidence, and 0.87 to 0.94 for confidence. For discrimination, ANOVA and post hoc testing by the Tukey-Kramer method revealed significant score differences among students, residents, and faculty members consistent with education and experience levels. For responsiveness to training, dental students

  13. Development of the Quality of Australian Nursing Documentation in Aged Care (QANDAC) instrument to assess paper-based and electronic resident records.

    Science.gov (United States)

    Wang, Ning; Björvell, Catrin; Hailey, David; Yu, Ping

    2014-12-01

    To develop an Australian nursing documentation in aged care (Quality of Australian Nursing Documentation in Aged Care (QANDAC)) instrument to measure the quality of paper-based and electronic resident records. The instrument was based on the nursing process model and on three attributes of documentation quality identified in a systematic review. The development process involved five phases following approaches to designing criterion-referenced measures. The face and content validities and the inter-rater reliability of the instrument were estimated using a focus group approach and consensus model. The instrument contains 34 questions in three sections: completion of nursing history and assessment, description of care process and meeting the requirements of data entry. Estimates of the validity and inter-rater reliability of the instrument gave satisfactory results. The QANDAC instrument may be a useful audit tool for quality improvement and research in aged care documentation. © 2013 ACOTA.

  14. Development of a video-simulation instrument for assessing cognition in older adults.

    Science.gov (United States)

    Ip, Edward H; Barnard, Ryan; Marshall, Sarah A; Lu, Lingyi; Sink, Kaycee; Wilson, Valerie; Chamberlain, Dana; Rapp, Stephen R

    2017-12-06

    Commonly used methods to assess cognition, such as direct observation, self-report, or neuropsychological testing, have significant limitations. Therefore, a novel tablet computer-based video simulation was created with the goal of being valid, reliable, and easy to administer. The design and implementation of the SIMBAC (Simulation-Based Assessment of Cognition) instrument is described in detail, as well as informatics "lessons learned" during development. The software emulates 5 common instrumental activities of daily living (IADLs) and scores participants' performance. The modules were chosen by a panel of geriatricians based on relevance to daily functioning and ability to be modeled electronically, and included facial recognition, pairing faces with the correct names, filling a pillbox, using an automated teller machine (ATM), and automatic renewal of a prescription using a telephone. Software development included three phases 1) a period of initial design and testing (alpha version), 2) pilot study with 10 cognitively normal and 10 cognitively impaired adults over the age of 60 (beta version), and 3) larger validation study with 162 older adults of mixed cognitive status (release version). Results of the pilot study are discussed in the context of refining the instrument; full results of the validation study are reported in a separate article. In both studies, SIMBAC reliably differentiated controls from persons with cognitive impairment, and performance was highly correlated with Mini Mental Status Examination (MMSE) score. Several informatics challenges emerged during software development, which are broadly relevant to the design and use of electronic assessment tools. Solutions to these issues, such as protection of subject privacy and safeguarding against data loss, are discussed in depth. Collection of fine-grained data (highly detailed information such as time spent reading directions and the number of taps on screen) is also considered. SIMBAC provides

  15. Measurement properties of instruments that assess participation in young people with autism spectrum disorder: a systematic review.

    Science.gov (United States)

    Lami, Francesca; Egberts, Kristine; Ure, Alexandra; Conroy, Rowena; Williams, Katrina

    2018-03-01

    To systematically review the measurement properties of instruments assessing participation in young people with autism spectrum disorder (ASD). A search was performed in MEDLINE, PsycINFO, and PubMed combining three constructs ('ASD', 'test of participation', 'measurement properties'). Results were restricted to articles including people aged 6 to 29 years. The 2539 identified articles were independently screened by two reviewers. For the included articles, data were extracted using standard forms and their risk of bias was assessed. Nine studies (8 cross-sectional) met the inclusion criteria, providing information on seven different instruments. The total sample included 634 participants, with sex available for 600 (males=494; females=106) and age available for 570, with mean age for these participants 140.58 months (SD=9.11; range=36-624). Included instruments were the school function assessment, vocational index, children's assessment of participation and enjoyment/preferences for activities of children, experience sampling method, Pediatric Evaluation of Disability Inventory, Computer Adaptive Test, adolescent and young adult activity card sort, and Patient-Reported Outcomes Measurement Information System parent-proxy peer relationships. Seven studies assessed reliability and validity; good properties were reported for half of the instruments considered. Most studies (n=6) had high risk of bias. Overall the quality of the evidence for each tool was limited. Validation of these instruments, or others that comprehensively assess participation, is needed. Future studies should follow recommended methodological standards. Seven instruments have been used to assess participation in young people with autism. One instrument, with excellent measurement properties in one study, does not comprehensively assess participation. Studies of three instruments that incorporate a more comprehensive assessment of participation have methodological limitations. Overall, limited

  16. The Child Dental Control Assessment (CDCA) in youth: reliability, validity and cross-cultural differences.

    Science.gov (United States)

    Coolidge, T; Heima, M; Heaton, L J; Nakai, Y; Höskuldsson, O; Smith, T A; Weinstein, P; Milgrom, P

    2005-03-01

    The Child Dental Control Assessment (CDCA) measures children's preferred control strategies in the dental situation. Three studies are reported, assessing aspects of this instrument in youths from the USA, Japan and Australia. In particular, measurements were made as to the reliability and validity of this instrument in this age group in the three cultures, as well as comparing some results across cultures. These studies used a questionnaire design. Questionnaires (including the CDCA and other measures) were given to youths aged 11-15 in the three cultures. In one culture, youths received the questionnaire twice, to compute test-retest reliability. The measure's reliability and validity were similar to those of other measures. The CDCA behaves similarly to the Revised Iowa Dental Control Index (R-IDCI). Youths in all three cultures showed similar responses, although the Japanese were less likely to endorse items. Internal reliability of the scale ranged from 0.74 to 0.85. Test- retest reliability was 0.74. Participants in the High Desire/Low Predicted classification on the R-IDCI scored higher on the CDCA (t (73) = 2.9, p < .01). In the Japanese and Australian samples the correlation between CDCA and dental fear was 0.29-0.33 (p < .001). The Australian and USA samples scored significantly higher than the Japanese sample (overall F(2,1544) = 383.98, p < .001, followed by Tukey's HSD, p < .001). These results provide evidence for the reliability and validity of the CDCA in youth. It appears to measure the discrepancy between Desired and Predicted Control identified in the Revised Iowa Dental Control Index (R-IDCI). Responses of the youth in all three cultures were similar, indicating common dental control preferences for individuals of this age. However, consistent with cultural values, Japanese youth were less likely to endorse the control strategies. These results underline the need to develop culturally-specific, as well as situationally-specific control measures.

  17. Probabilistic assessment of pressure vessel and piping reliability

    International Nuclear Information System (INIS)

    Sundararajan, C.

    1986-01-01

    The paper presents a critical review of the state-of-the-art in probabilistic assessment of pressure vessel and piping reliability. First the differences in assessing the reliability directly from historical failure data and indirectly by a probabilistic analysis of the failure phenomenon are discussed and the advantages and disadvantages are pointed out. The rest of the paper deals with the latter approach of reliability assessment. Methods of probabilistic reliability assessment are described and major projects where these methods are applied for pressure vessel and piping problems are discussed. An extensive list of references is provided at the end of the paper

  18. Instrumented static and dynamic balance assessment after stroke using Wii Balance Boards: reliability and association with clinical tests.

    Science.gov (United States)

    Bower, Kelly J; McGinley, Jennifer L; Miller, Kimberly J; Clark, Ross A

    2014-01-01

    The Wii Balance Board (WBB) is a globally accessible device that shows promise as a clinically useful balance assessment tool. Although the WBB has been found to be comparable to a laboratory-grade force platform for obtaining centre of pressure data, it has not been comprehensively studied in clinical populations. The aim of this study was to investigate the measurement properties of tests utilising the WBB in people after stroke. Thirty individuals who were more than three months post-stroke and able to stand unsupported were recruited from a single outpatient rehabilitation facility. Participants performed standardised assessments incorporating the WBB and customised software (static stance with eyes open and closed, static weight-bearing asymmetry, dynamic mediolateral weight shifting and dynamic sit-to-stand) in addition to commonly employed clinical tests (10 Metre Walk Test, Timed Up and Go, Step Test and Functional Reach) on two testing occasions one week apart. Test-retest reliability and construct validity of the WBB tests were investigated. All WBB-based outcomes were found to be highly reliable between testing occasions (ICC  = 0.82 to 0.98). Correlations were poor to moderate between WBB variables and clinical tests, with the strongest associations observed between task-related activities, such as WBB mediolateral weight shifting and the Step Test. The WBB, used with customised software, is a reliable and potentially useful tool for the assessment of balance and weight-bearing asymmetry following stroke. Future research is recommended to further investigate validity and responsiveness.

  19. The patient assessment questionnaire: a new instrument for evaluating the interpersonal skills of vocational dental practitioners.

    Science.gov (United States)

    Hurst, Y K; Prescott-Clements, L E; Rennie, J S

    2004-10-23

    This paper describes a pilot study aimed at evaluating a new instrument, the patient assessment questionnaire (PAQ), which uses patient ratings for the assessment of communication skills and professionalism in vocational practitioners (VDPs). The PAQ was developed as part of an assessment system designed to address all round competence. Acohort of 99 VDPs took part in the study. Questionnaires were distributed to consecutive patients in the general dental service at two time points in the training year. Data from the pilot study was analysed to determine whether the PAQ fulfilled the criteria for robust assessment. Results provide evidence of high levels of reliability, validity and feasibility of the PAQ instrument. All indications to date suggest that the PAQ will prove to be a valuable assessment tool. It is currently being evaluated as part of the system used to assess the all round competence of dental graduates undertaking vocational training in Scotland.

  20. Cross-cultural adaptation and validation of a Brazilian version of an instrument to assess impairments related to oral functioning of people with Down syndrome

    Directory of Open Access Journals (Sweden)

    Bonanato Karina

    2013-01-01

    Full Text Available Abstract Background An instrument was developed in Canada to assess impairments related to oral functioning of individuals with four years of age or older with Down syndrome (DS. The present study attempted to carry out the cross-cultural adaptation and validation of the instrument for the Brazilian Portuguese language and to test its reliability and validity. Findings After translation and cross-cultural adaptation, the instrument was tested on caregivers of people with DS. Clinical examination for malocclusion was carried out in people with DS by two calibrated examiners. Inter and Intra examiner agreement was assessed by Intraclass Correlation Coefficient (ICC and ranged from 0.92 to 0.97 respectively. Total of 157 people with DS and their caregivers were able to compose the sample. They were selected from eight institutions for people with DS in five cities of southeastern Brazil. The mean age of people with DS was 20.7 [±13.1] and for caregivers was 53.1 [±13.7]. The mean instrument score was 18.6 [±9.0]. Internal reliability ranged from 0.49 to 0.80 and external reliability ranged from 0.78 to 0.88. Construct validity was verified by significant correlations identified between malocclusion and the total instrument; and caregivers’ educational level and the instrument (p Conclusions Initial validity tests indicated that the instrument related to the oral health for people with DS may be a valid instrument to this segment of the population in Brazil.

  1. Under sodium reliability tests on core components and in-core instrumentation

    International Nuclear Information System (INIS)

    Ruppert, E.; Stehle, H.; Vinzens, K.

    1977-01-01

    A sodium test facility for fast breeder core components (AKB), built by INTERATOM at Bensberg, has been operating since 1971 to test fuel dummies and blanket elements as well as absorber elements under simulated normal and extreme reactor conditions. Individual full-scale fuel or blanket elements and arrays of seven elements, modelling a section of the SNR-300 reactor core, have been tested under a wide range of sodium mass flow and isothermal test conditions up to 925K as well as under cyclic changed temperature transients. Besides endurance testing of the core components a special sodium and high-temperature instrumentation is provided to investigate thermohydraulic and vibrational behaviour of the test objects. During all test periods the main subassembly characteristics could be reproduced and the reliability of the instrumentation could be proven. (orig.) [de

  2. Assessing learning outcomes in middle-division classical mechanics: The Colorado Classical Mechanics and Math Methods Instrument

    Directory of Open Access Journals (Sweden)

    Marcos D. Caballero

    2017-04-01

    Full Text Available Reliable and validated assessments of introductory physics have been instrumental in driving curricular and pedagogical reforms that lead to improved student learning. As part of an effort to systematically improve our sophomore-level classical mechanics and math methods course (CM 1 at CU Boulder, we have developed a tool to assess student learning of CM 1 concepts in the upper division. The Colorado Classical Mechanics and Math Methods Instrument (CCMI builds on faculty consensus learning goals and systematic observations of student difficulties. The result is a 9-question open-ended post test that probes student learning in the first half of a two-semester classical mechanics and math methods sequence. In this paper, we describe the design and development of this instrument, its validation, and measurements made in classes at CU Boulder and elsewhere.

  3. Reliability of patient-reported outcome instruments in US adults with hemophilia: the Pain, Functional Impairment and Quality of life (P-FiQ study

    Directory of Open Access Journals (Sweden)

    Kempton CL

    2017-09-01

    Full Text Available Christine L Kempton,1 Michael Wang,2 Michael Recht,3 Anne Neff,4 Amy D Shapiro,5 Amit Soni,6 Roshni Kulkarni,7 Tyler W Buckner,2 Katharine Batt,8 Neeraj N Iyer,9 David L Cooper9 1Departments of Pediatrics and Hematology and Medical Oncology, Emory University School of Medicine, Atlanta, GA, USA; 2Hemophilia and Thrombosis Center, University of Colorado School of Medicine, Aurora, CO, USA; 3The Hemophilia Center, Oregon Health & Science University, Portland, OR, USA; 4Hematology and Medical Oncology, Cleveland Clinic, Cleveland, OH, USA; 5Indiana Hemophilia & Thrombosis Center, Indianapolis, IN, USA; 6Center for Inherited Blood Disorders and CHOC Children’s Hospital/UC Irvine, Orange, CA, USA; 7MSU Center for Bleeding and Clotting Disorders, Michigan State University, East Lansing, MI, USA; 8Hematology and Oncology, Wake Forest School of Medicine, Winston-Salem, NC, USA; 9Clinical, Medical and Regulatory Affairs, Novo Nordisk Inc., Plainsboro, NJ, USA Background: Hemophilia is marked by frequent joint bleeding, resulting in pain and functional impairment.Objective: This study aimed to assess the reliability of five patient-reported outcome (PRO instruments in people with hemophilia (PWH in a non-bleeding state.Methods: Adult male PWH of any severity and inhibitor status, with a history of joint pain or bleeding, completed a pain history and five PRO instruments (EQ-5D-5L, Brief Pain Inventory v2 [BPI], International Physical Activity Questionnaire [IPAQ], Short Form 36 Health Survey v2 [SF-36v2], and Hemophilia Activities List [HAL] during their routine comprehensive care visit. Patients were approached to complete the PRO instruments again at the end of their visit while in a similar non-bleeding state. Concordance of individual questionnaire items and correlation between domain scores were assessed using intra-class correlation coefficient (ICC.Results: Participants completing the retest (n=164 had a median age of 33.9 years. Median time for

  4. Development of a quality instrument for assessing the spontaneous reports of ADR/ADE using Delphi method in China.

    Science.gov (United States)

    Chen, Lixun; Jiang, Ling; Shen, Aizong; Wei, Wei

    2016-09-01

    The frequently low quality of submitted spontaneous reports is of an increasing concern; to our knowledge, no validated instrument exists for assessing case reports' quality comprehensively enough. This work was conducted to develop such a quality instrument for assessing the spontaneous reports of adverse drug reaction (ADR)/adverse drug event (ADE) in China. Initial evaluation indicators were generated using systematic and literature data analysis. Final indicators and their weights were identified using Delphi method. The final quality instrument was developed by adopting the synthetic scoring method. A consensus was reached after four rounds of Delphi survey. The developed quality instrument consisted of 6 first-rank indicators, 18 second-rank indicators, and 115 third-rank indicators, and each rank indicator has been weighted. It evaluates the quality of spontaneous reports of ADR/ADE comprehensively and quantitatively on six parameters: authenticity, duplication, regulatory, completeness, vigilance level, and reporting time frame. The developed instrument was tested with good reliability and validity, which can be used to comprehensively and quantitatively assess the submitted spontaneous reports of ADR/ADE in China.

  5. Assessing health status and quality-of-life instruments: attributes and review criteria.

    Science.gov (United States)

    Aaronson, Neil; Alonso, Jordi; Burnam, Audrey; Lohr, Kathleen N; Patrick, Donald L; Perrin, Edward; Stein, Ruth E

    2002-05-01

    The field of health status and quality of life (QoL) measurement - as a formal discipline with a cohesive theoretical framework, accepted methods, and diverse applications--has been evolving for the better part of 30 years. To identify health status and QoL instruments and review them against rigorous criteria as a precursor to creating an instrument library for later dissemination, the Medical Outcomes Trust in 1994 created an independently functioning Scientific Advisory Committee (SAC). In the mid-1990s, the SAC defined a set of attributes and criteria to carry out instrument assessments; 5 years later, it updated and revised these materials to take account of the expanding theories and technologies upon which such instruments were being developed. This paper offers the SAC's current conceptualization of eight key attributes of health status and QoL instruments (i.e., conceptual and measurement model; reliability; validity; responsiveness; interpretability; respondent and administrative burden; alternate forms; and cultural and language adaptations) and the criteria by which instruments would be reviewed on each of those attributes. These are suggested guidelines for the field to consider and debate; as measurement techniques become both more familiar and more sophisticated, we expect that experts will wish to update and refine these criteria accordingly.

  6. Parts and Components Reliability Assessment: A Cost Effective Approach

    Science.gov (United States)

    Lee, Lydia

    2009-01-01

    System reliability assessment is a methodology which incorporates reliability analyses performed at parts and components level such as Reliability Prediction, Failure Modes and Effects Analysis (FMEA) and Fault Tree Analysis (FTA) to assess risks, perform design tradeoffs, and therefore, to ensure effective productivity and/or mission success. The system reliability is used to optimize the product design to accommodate today?s mandated budget, manpower, and schedule constraints. Stand ard based reliability assessment is an effective approach consisting of reliability predictions together with other reliability analyses for electronic, electrical, and electro-mechanical (EEE) complex parts and components of large systems based on failure rate estimates published by the United States (U.S.) military or commercial standards and handbooks. Many of these standards are globally accepted and recognized. The reliability assessment is especially useful during the initial stages when the system design is still in the development and hard failure data is not yet available or manufacturers are not contractually obliged by their customers to publish the reliability estimates/predictions for their parts and components. This paper presents a methodology to assess system reliability using parts and components reliability estimates to ensure effective productivity and/or mission success in an efficient manner, low cost, and tight schedule.

  7. Instrument development and evaluation for patient-related outcomes assessments

    Directory of Open Access Journals (Sweden)

    Farnik M

    2012-03-01

    Full Text Available Małgorzata Farnik, Władysław PierzchałaDepartment of Pneumonology, Silesian University of Medicine, Katowice, PolandAbstract: Patient-related outcomes measures could provide important information for the current state of the art in medical care and even have an impact on macrodecisions in the health care system. Patient-related outcomes were initially defined as subjective health indicators that allow disability and illness to be assessed, based on patient, caregiver, or physician self-reports. As illness involves psychological and behavioral complex processes of care, a multidisciplinary approach in measuring patient-reported outcomes should be recommended, such as quality of life questionnaires. Patient-related outcomes measures should correspond to specific clinical situations and bring opportunities to improve quality of care. Objective measurements enable quantitative data to be collected and analyzed. Depending on the aim of the research, investigators can use existing methods or develop new tools. This publication presents a methodology for developing patient-related outcomes measures, based on a multistage procedure. The proper definition of specific study objectives and the methodology of instrument development are crucial for successfully transferring the study concept. The model of instrument development is the process of starting from the preliminary phase and includes questionnaire design and scaling, pilot testing (cognitive debriefing, revision of the preliminary version, evaluation of the new tool, and implementation. Validation of the new instrument includes reliability, reproducibility, internal consistency, and responsiveness. The process of designing the new tool should involve a panel of experts, including clinicians, psychologists (preliminary phase, and statisticians (scale development and scoring, and patients (cognitive debriefing. Implementation of a new tool should be followed by evaluation study – assessment of

  8. New Approaches to Reliability Assessment

    DEFF Research Database (Denmark)

    Ma, Ke; Wang, Huai; Blaabjerg, Frede

    2016-01-01

    of energy. New approaches for reliability assessment are being taken in the design phase of power electronics systems based on the physics-of-failure in components. In this approach, many new methods, such as multidisciplinary simulation tools, strength testing of components, translation of mission profiles......, and statistical analysis, are involved to enable better prediction and design of reliability for products. This article gives an overview of the new design flow in the reliability engineering of power electronics from the system-level point of view and discusses some of the emerging needs for the technology...

  9. Instrumented static and dynamic balance assessment after stroke using Wii Balance Boards: reliability and association with clinical tests.

    Directory of Open Access Journals (Sweden)

    Kelly J Bower

    Full Text Available The Wii Balance Board (WBB is a globally accessible device that shows promise as a clinically useful balance assessment tool. Although the WBB has been found to be comparable to a laboratory-grade force platform for obtaining centre of pressure data, it has not been comprehensively studied in clinical populations. The aim of this study was to investigate the measurement properties of tests utilising the WBB in people after stroke.Thirty individuals who were more than three months post-stroke and able to stand unsupported were recruited from a single outpatient rehabilitation facility. Participants performed standardised assessments incorporating the WBB and customised software (static stance with eyes open and closed, static weight-bearing asymmetry, dynamic mediolateral weight shifting and dynamic sit-to-stand in addition to commonly employed clinical tests (10 Metre Walk Test, Timed Up and Go, Step Test and Functional Reach on two testing occasions one week apart. Test-retest reliability and construct validity of the WBB tests were investigated.All WBB-based outcomes were found to be highly reliable between testing occasions (ICC  = 0.82 to 0.98. Correlations were poor to moderate between WBB variables and clinical tests, with the strongest associations observed between task-related activities, such as WBB mediolateral weight shifting and the Step Test.The WBB, used with customised software, is a reliable and potentially useful tool for the assessment of balance and weight-bearing asymmetry following stroke. Future research is recommended to further investigate validity and responsiveness.

  10. Validity and reliability of a brief self-reported questionnaire assessing fruit and vegetable consumption among pregnant women

    Directory of Open Access Journals (Sweden)

    Lydi-Anne Vézina-Im

    2016-09-01

    Full Text Available Abstract Background Short instruments measuring frequency of specific foods, such as fruit and vegetable (FV, are increasingly used in interventions. The objective of the study was to verify the validity and test-retest reliability of such an instrument among pregnant women. Methods Pregnant women from the region of Quebec City, Quebec, Canada, were recruited through e-mails sent to female students and employees of the local university from October 2014 to April 2015. To assess the validity of the fruit and vegetable questionnaire (FVQ developed by Godin et al. (Can J Public Health 99: 494-498, 2008, pregnant women were asked in a first mailing to complete the FVQ assessing FV intake over the past 7 days and a 3-day estimated food record. A subsample (n = 33 also gave a fasting blood sample and completed a validated semi-quantitative FFQ administered by a trained registered dietitian during a visit at the research center. FV intakes for all instruments were calculated in terms of servings of FV based on Canada’s Food Guide definition of a serving of fruit or vegetable. In order to assess its test-retest reliability, respondents were asked to complete the FVQ 14 days later in a second mailing. Results Forty-eight pregnant women from all three trimesters completed the questionnaires in the first mailing. FV intake assessed using the FVQ was correlated to FV consumption measured using the food record (r = 0.34, p = 0.0180 and the FFQ (r = 0.61, p = 0.0002. Results were similar when controlling for energy intake and the experience of nausea in the past month. Only β-cryptoxanthin was significantly correlated to FV intake assessed by the FFQ when adjusted for the presence of nausea (r = 0.35, p = 0.0471. Data on the test-retest reliability was available for 44 women and the intra-class coefficient for the FVQ was 0.72 at a mean 28-day interval. Conclusions The FVQ has acceptable validity and test-retest reliability

  11. Validity and reliability of a brief self-reported questionnaire assessing fruit and vegetable consumption among pregnant women.

    Science.gov (United States)

    Vézina-Im, Lydi-Anne; Godin, Gaston; Couillard, Charles; Perron, Julie; Lemieux, Simone; Robitaille, Julie

    2016-09-15

    Short instruments measuring frequency of specific foods, such as fruit and vegetable (FV), are increasingly used in interventions. The objective of the study was to verify the validity and test-retest reliability of such an instrument among pregnant women. Pregnant women from the region of Quebec City, Quebec, Canada, were recruited through e-mails sent to female students and employees of the local university from October 2014 to April 2015. To assess the validity of the fruit and vegetable questionnaire (FVQ) developed by Godin et al. (Can J Public Health 99: 494-498, 2008), pregnant women were asked in a first mailing to complete the FVQ assessing FV intake over the past 7 days and a 3-day estimated food record. A subsample (n = 33) also gave a fasting blood sample and completed a validated semi-quantitative FFQ administered by a trained registered dietitian during a visit at the research center. FV intakes for all instruments were calculated in terms of servings of FV based on Canada's Food Guide definition of a serving of fruit or vegetable. In order to assess its test-retest reliability, respondents were asked to complete the FVQ 14 days later in a second mailing. Forty-eight pregnant women from all three trimesters completed the questionnaires in the first mailing. FV intake assessed using the FVQ was correlated to FV consumption measured using the food record (r = 0.34, p = 0.0180) and the FFQ (r = 0.61, p = 0.0002). Results were similar when controlling for energy intake and the experience of nausea in the past month. Only β-cryptoxanthin was significantly correlated to FV intake assessed by the FFQ when adjusted for the presence of nausea (r = 0.35, p = 0.0471). Data on the test-retest reliability was available for 44 women and the intra-class coefficient for the FVQ was 0.72 at a mean 28-day interval. The FVQ has acceptable validity and test-retest reliability values, but seems to underestimate FV servings in pregnant women

  12. Towards an Operational Definition of Effective Co-Teaching: Instrument Development, Validity, and Reliability

    Science.gov (United States)

    La Monte, Michelle Evonne

    2012-01-01

    This study focused on developing a valid and reliable instrument that can not only identify successful co-teaching, but also the professional development needs of co-teachers and their administrators in public schools. Two general questions about the quality of co-teaching were addressed in this study: (a) How well did descriptors within each of…

  13. Structural reliability assessment capability in NESSUS

    Science.gov (United States)

    Millwater, H.; Wu, Y.-T.

    1992-07-01

    The principal capabilities of NESSUS (Numerical Evaluation of Stochastic Structures Under Stress), an advanced computer code developed for probabilistic structural response analysis, are reviewed, and its structural reliability assessed. The code combines flexible structural modeling tools with advanced probabilistic algorithms in order to compute probabilistic structural response and resistance, component reliability and risk, and system reliability and risk. An illustrative numerical example is presented.

  14. Failure Modes Taxonomy for Reliability Assessment of Digital Instrumentation and Control Systems for Probabilistic Risk Analysis - Failure modes taxonomy for reliability assessment of digital I and C systems for PRA

    International Nuclear Information System (INIS)

    Amri, A.; Blundell, N.; ); Authen, S.; Betancourt, L.; Coyne, K.; Halverson, D.; Li, M.; Taylor, G.; Bjoerkman, K.; Brinkman, H.; Postma, W.; Bruneliere, H.; Chirila, M.; Gheorge, R.; Chu, L.; Yue, M.; Delache, J.; Georgescu, G.; Deleuze, G.; Quatrain, R.; Thuy, N.; Holmberg, J.-E.; Kim, M.C.; Kondo, K.; Mancini, F.; Piljugin, E.; Stiller, J.; Sedlak, J.; Smidts, C.; Sopira, V.

    2015-01-01

    Digital protection and control systems appear as upgrades in older nuclear power plants (NPP), and are commonplace in new NPPs. To assess the risk of NPP operation and to determine the risk impact of digital systems, there is a need to quantitatively assess the reliability of the digital systems in a justifiable manner. Due to the many unique attributes of digital systems (e.g., functions are implemented by software, units of the system interact in a communication network, faults can be identified and handled online), a number of modelling and data collection challenges exist, and international consensus on the reliability modelling has not yet been reached. The objective of the task group called DIGREL has been to develop a taxonomy of failure modes of digital components for the purposes of probabilistic risk analysis (PRA). An activity focused on the development of a common taxonomy of failure modes is seen as an important step towards standardised digital instrumentation and control (I and C) reliability assessment techniques for PRA. Needs from PRA has guided the work, meaning, e.g., that the I and C system and its failures are studied from the point of view of their functional significance point of view. The taxonomy will be the basis of future modelling and quantification efforts. It will also help to define a structure for data collection and to review PRA studies. The proposed failure modes taxonomy has been developed by first collecting examples of taxonomies provided by the task group organisations. This material showed some variety in the handling of I and C hardware failure modes, depending on the context where the failure modes have been defined. Regarding the software part of I and C, failure modes defined in NPP PRAs have been simple - typically a software CCF failing identical processing units. The DIGREL task group has defined a new failure modes taxonomy based on a hierarchical definition of five levels of abstraction: 1. system level (complete

  15. The quality of instruments to assess the process of shared decision making: A systematic review

    Science.gov (United States)

    Bomhof-Roordink, Hanna; Smith, Ian P.; Scholl, Isabelle; Stiggelbout, Anne M.; Pieterse, Arwen H.

    2018-01-01

    Objective To inventory instruments assessing the process of shared decision making and appraise their measurement quality, taking into account the methodological quality of their validation studies. Methods In a systematic review we searched seven databases (PubMed, Embase, Emcare, Cochrane, PsycINFO, Web of Science, Academic Search Premier) for studies investigating instruments measuring the process of shared decision making. Per identified instrument, we assessed the level of evidence separately for 10 measurement properties following a three-step procedure: 1) appraisal of the methodological quality using the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist, 2) appraisal of the psychometric quality of the measurement property using three possible quality scores, 3) best-evidence synthesis based on the number of studies, their methodological and psychometrical quality, and the direction and consistency of the results. The study protocol was registered at PROSPERO: CRD42015023397. Results We included 51 articles describing the development and/or evaluation of 40 shared decision-making process instruments: 16 patient questionnaires, 4 provider questionnaires, 18 coding schemes and 2 instruments measuring multiple perspectives. There is an overall lack of evidence for their measurement quality, either because validation is missing or methods are poor. The best-evidence synthesis indicated positive results for a major part of instruments for content validity (50%) and structural validity (53%) if these were evaluated, but negative results for a major part of instruments when inter-rater reliability (47%) and hypotheses testing (59%) were evaluated. Conclusions Due to the lack of evidence on measurement quality, the choice for the most appropriate instrument can best be based on the instrument’s content and characteristics such as the perspective that they assess. We recommend refinement and validation of

  16. Assessing reliability in energy supply systems

    International Nuclear Information System (INIS)

    McCarthy, Ryan W.; Ogden, Joan M.; Sperling, Daniel

    2007-01-01

    Reliability has always been a concern in the energy sector, but concerns are escalating as energy demand increases and the political stability of many energy supply regions becomes more questionable. But how does one define and measure reliability? We introduce a method to assess reliability in energy supply systems in terms of adequacy and security. It derives from reliability assessment frameworks developed for the electricity sector, which are extended to include qualitative considerations and to be applicable to new energy systems by incorporating decision-making processes based on expert opinion and multi-attribute utility theory. The method presented here is flexible and can be applied to any energy system. To illustrate its use, we apply the method to two hydrogen pathways: (1) centralized steam reforming of imported liquefied natural gas with pipeline distribution of hydrogen, and (2) on-site electrolysis of water using renewable electricity produced independently from the electricity grid

  17. Reliability of a novel thermal imaging system for temperature assessment of healthy feet.

    Science.gov (United States)

    Petrova, N L; Whittam, A; MacDonald, A; Ainarkar, S; Donaldson, A N; Bevans, J; Allen, J; Plassmann, P; Kluwe, B; Ring, F; Rogers, L; Simpson, R; Machin, G; Edmonds, M E

    2018-01-01

    Thermal imaging is a useful modality for identifying preulcerative lesions ("hot spots") in diabetic foot patients. Despite its recognised potential, at present, there is no readily available instrument for routine podiatric assessment of patients at risk. To address this need, a novel thermal imaging system was recently developed. This paper reports the reliability of this device for temperature assessment of healthy feet. Plantar skin foot temperatures were measured with the novel thermal imaging device (Diabetic Foot Ulcer Prevention System (DFUPS), constructed by Photometrix Imaging Ltd) and also with a hand-held infrared spot thermometer (Thermofocus® 01500A3, Tecnimed, Italy) after 20 min of barefoot resting with legs supported and extended in 105 subjects (52 males and 53 females; age range 18 to 69 years) as part of a multicentre clinical trial. The temperature differences between the right and left foot at five regions of interest (ROIs), including 1st and 4th toes, 1st, 3rd and 5th metatarsal heads were calculated. The intra-instrument agreement (three repeated measures) and the inter-instrument agreement (hand-held thermometer and thermal imaging device) were quantified using intra-class correlation coefficients (ICCs) and the 95% confidence intervals (CI). Both devices showed almost perfect agreement in replication by instrument. The intra-instrument ICCs for the thermal imaging device at all five ROIs ranged from 0.95 to 0.97 and the intra-instrument ICCs for the hand-held-thermometer ranged from 0.94 to 0.97. There was substantial to perfect inter-instrument agreement between the hand-held thermometer and the thermal imaging device and the ICCs at all five ROIs ranged between 0.94 and 0.97. This study reports the performance of a novel thermal imaging device in the assessment of foot temperatures in healthy volunteers in comparison with a hand-held infrared thermometer. The newly developed thermal imaging device showed very good agreement in

  18. Assessment of physical server reliability in multi cloud computing system

    Science.gov (United States)

    Kalyani, B. J. D.; Rao, Kolasani Ramchand H.

    2018-04-01

    Business organizations nowadays functioning with more than one cloud provider. By spreading cloud deployment across multiple service providers, it creates space for competitive prices that minimize the burden on enterprises spending budget. To assess the software reliability of multi cloud application layered software reliability assessment paradigm is considered with three levels of abstractions application layer, virtualization layer, and server layer. The reliability of each layer is assessed separately and is combined to get the reliability of multi-cloud computing application. In this paper, we focused on how to assess the reliability of server layer with required algorithms and explore the steps in the assessment of server reliability.

  19. Highly Reliable Power and Communication System for Essential Instruments under a Severe Accident of NPPs

    Energy Technology Data Exchange (ETDEWEB)

    Yoo, S. J.; Choi, B. H.; Jung, S. Y.; Rim, Chun T. [Korea Advanced Institute of Science and Technology, Daejeon (Korea, Republic of)

    2013-10-15

    In this paper, three survivable strategies to overcome the problems listed above are proposed for the essential instruments under the severe accident of NPPs. First, wire/wireless multi power systems are adopted to the essential instruments for continuous power supply. Second, wire/wireless communication systems are proposed for reliable transmission of measuring information among instruments and operators. Third, a physical protection system such as a harness and a heat isolation box is introduced to ensure operable conditions for the proposed systems. In this paper, a highly reliable strategy, which consists of wire/wireless multi power and communication systems and physical protection system is proposed to ensure the survival of the essential instruments under harsh external conditions. The wire/wireless multi power and communication systems are designed to transfer power and data in spite of the failure of conventional wired systems. The physical protection system provides operable environments to the instruments. Therefore, the proposed system can be considered as a candidate of practical and urgent remedy for NPPs under the severe accident. After the Fukushima nuclear accident, survivability of essential instruments has been emphasized for immediate and accurate response. The essential instruments can measure environment conditions such as temperature, pressure, radioactivity and corium behavior inside nuclear power plants (NPPs) under a severe accident. Access to the inside of NPPs is restricted to human beings because of hazardous environment such as high radioactivity, high temperature and high pressure. Thus, monitoring the inside of NPPs is necessary for avoiding damage from the severe accident. Even though there were a number of instruments in Fukushima Daiichi NPP, they failed to obtain exact monitoring information. According to the details of the Fukushima nuclear accident, following problems can be counted as strong candidates of this instruments

  20. Highly Reliable Power and Communication System for Essential Instruments under a Severe Accident of NPPs

    International Nuclear Information System (INIS)

    Yoo, S. J.; Choi, B. H.; Jung, S. Y.; Rim, Chun T.

    2013-01-01

    In this paper, three survivable strategies to overcome the problems listed above are proposed for the essential instruments under the severe accident of NPPs. First, wire/wireless multi power systems are adopted to the essential instruments for continuous power supply. Second, wire/wireless communication systems are proposed for reliable transmission of measuring information among instruments and operators. Third, a physical protection system such as a harness and a heat isolation box is introduced to ensure operable conditions for the proposed systems. In this paper, a highly reliable strategy, which consists of wire/wireless multi power and communication systems and physical protection system is proposed to ensure the survival of the essential instruments under harsh external conditions. The wire/wireless multi power and communication systems are designed to transfer power and data in spite of the failure of conventional wired systems. The physical protection system provides operable environments to the instruments. Therefore, the proposed system can be considered as a candidate of practical and urgent remedy for NPPs under the severe accident. After the Fukushima nuclear accident, survivability of essential instruments has been emphasized for immediate and accurate response. The essential instruments can measure environment conditions such as temperature, pressure, radioactivity and corium behavior inside nuclear power plants (NPPs) under a severe accident. Access to the inside of NPPs is restricted to human beings because of hazardous environment such as high radioactivity, high temperature and high pressure. Thus, monitoring the inside of NPPs is necessary for avoiding damage from the severe accident. Even though there were a number of instruments in Fukushima Daiichi NPP, they failed to obtain exact monitoring information. According to the details of the Fukushima nuclear accident, following problems can be counted as strong candidates of this instruments

  1. Identifying the Barriers to Using Games and Simulations in Education: Creating a Valid and Reliable Survey Instrument

    Science.gov (United States)

    Justice, Lenora Jean

    2012-01-01

    The purpose of this study was to create a valid and reliable instrument to measure teacher perceived barriers to the adoption of games and simulations in instruction. Previous research, interviews with educators, a focus group, an expert review, and a think aloud protocol were used to design a survey instrument. After finalization, the survey was…

  2. Evaluating the effectiveness of impact assessment instruments

    DEFF Research Database (Denmark)

    Cashmore, Matthew; Richardson, Tim; Hilding-Ryedvik, Tuija

    2010-01-01

    to sharpen effectiveness evaluation theory for impact assessment instruments this article critically examines the neglected issue of their political constitution. Analytical examples are used to concretely explore the nature and significance of the politicisation of impact assessment. It is argued......The central role of impact assessment instruments globally in policy integration initiatives has been cemented in recent years. Associated with this trend, but also reflecting political emphasis on greater accountability in certain policy sectors and a renewed focus on economic competitiveness...... that raising awareness about the political character of impact assessment instruments, in itself, is a vital step in advancing effectiveness evaluation theory. Broader theoretical lessons on the framing of evaluation research are also drawn from the political analysis. We conclude that, at least within...

  3. Content validity of governing in Building Information Modelling (BIM) implementation assessment instrument

    Science.gov (United States)

    Hadzaman, N. A. H.; Takim, R.; Nawawi, A. H.; Mohamad Yusuwan, N.

    2018-04-01

    BIM governance assessment instrument is a process of analysing the importance in developing BIM governance solution to tackle the existing problems during team collaboration in BIM-based projects. Despite the deployment of integrative technologies in construction industry particularly BIM, it is still insufficient compare to other sectors. Several studies have been established the requirements of BIM implementation concerning all technical and non-technical BIM adoption issues. However, the data are regarded as inadequate to develop a BIM governance framework. Hence, the objective of the paper is to evaluate the content validity of the BIM governance instrument prior to the main data collection. Two methods were employed in the form of literature review and questionnaire survey. Based on the literature review, 273 items with six main constructs are suggested to be incorporated in the BIM governance instrument. The Content Validity Ratio (CVR) scores revealed that 202 out of 273 items are considered as the utmost critical by the content experts. The findings for Item Level Content Validity Index (I-CVI) and Modified Kappa Coefficient however revealed that 257 items in BIM governance instrument are appropriate and excellent. The instrument is highly reliable for future strategies and the development of BIM projects in Malaysia.

  4. Safety instrumented systems in the oil and gas industry : Concepts and methods for safety and reliability assessments in design and operation

    Energy Technology Data Exchange (ETDEWEB)

    Lundteigen, Mary Ann

    2009-07-01

    This thesis proposes new methods and gives new insight to safety and reliability assessments of safety instrumented systems (SISs). These systems play an important role in many industry sectors and are used to detect the onset of hazardous events and mitigate their consequences to humans, the environment, and material assets. The thesis focuses on SIS applications in the oil and gas industry. Here, the SIS must respond to hazardous events such as gas leakages, fires, and over pressurization. Because there are personnel onboard the oil and gas installations, the operations take place in a vulnerable marine environment, and substantial values are associated with the offshore facilities, the reliability of SIS is of great concern to the public, the authorities, and the plant owners. The objective of this project has been to identify some of the key factors that influence the SIS reliability, clarify their effects on reliability, and suggest means to improve the treatment of these factors in safety and reliability assessments in design and operation. The project builds on concepts, methods, and definitions in two key standards for SIS design, construction, and operation: IEC 61508 and IEC 61511. The main contributions from this project are: A product development model that integrates reliability, availability, maintainability, and safety (RAMS) requirements with product development. The contributions have been presented in ten articles, five published in international journals, two submitted for publication, and three presented at conferences and in conference proceedings. The contributions are also directed to the industry and the actors that are involved in SIS design, construction, and operation. Even if the oil and gas industry is the main focus area, the results may be relevant for other industry sectors as well. SIS manufacturers and SIS designers face a large number of requirements from authorities, oil companies, international standards, and so on. At the same

  5. First Results from the Test Of Astronomy STandards (TOAST) Assessment Instrument

    Science.gov (United States)

    Slater, Stephanie

    2009-01-01

    Considerable effort in the astronomy education research over the past several years has focused on developing assessment tools in the form of multiple-choice conceptual diagnostics and content knowledge surveys. This has been critically important in advancing astronomy as a sub-discipline of physics education research, allowing researchers to establish the initial knowledge state of students as well as to attempt to measure some of the impacts of innovative instructional interventions. Before now, few of the existing instruments were constructed upon a solid list of clearly articulated and widely agreed upon learning objectives. Moving beyond the 10-year old Astronomy Diagnostics Test, we have developed and validated a new assessment instrument that is tightly aligned to the consensus learning goals stated by the American Astronomical Society - Chair's Conference on ASTRO 101, the American Association of the Advancement of Science's Project 2061 Benchmarks, and the National Research Council's National Science Education Standards. Researchers from the Cognition in Astronomy, Physics and Earth sciences Research (CAPER) Team at the University of Wyoming's Science and Math Teaching Center (UWYO SMTC) designed a criterion-referenced assessment tool, called the Test Of Astronomy STandards (TOAST). Through iterative development, this multiple-choice instrument has a high degree of reliability and validity for instructors and researchers needing information on students’ initial knowledge state at the beginning of a course and can be used, in aggregate, to help measure the impact of course-length duration instructional strategies for undergraduate science survey courses with learning goals tightly aligned to the consensus goals of the astronomy education community.

  6. Reliability and Convergent Validity of the Algometer for Vestibular Pain Assessment in Women with Provoked Vestibulodynia.

    Science.gov (United States)

    Cyr, Marie-Pierre; Bourbonnais, Daniel; Pinard, Alexandra; Dubois, Olivia; Morin, Mélanie

    2016-07-01

    Women with provoked vestibulodynia (PVD) suffer pain at the entry of the vagina elicited by pressure as during vaginal penetration. To quantify vestibular pain, we developed a new instrument, an algometer. The aim of this study was to investigate the test-retest reliability of the algometer and evaluate its convergent validity for vestibular pain assessment in women with PVD. Twenty-six women with PVD participated in the study. Vestibular pain was assessed with the new algometer and the already known vulvalgesiometer during two different sessions 2 to 4 weeks apart. At each session, the pressure pain threshold (PPT) and pressure pain tolerance (PPTol) were measured twice at the 3, 6, and 9 o'clock sites of the vestibule in random order. The test-retest reliability (intra- and inter-session) of the algometer was calculated using the intraclass correlation coefficient (ICC) and standard error of measurement (SEM). Its convergent validity was evaluated by the correlation coefficients between PPTs and PPTols measured by the algometer and those measured with the vulvalgesiometer. Intra-session reliability at all three sites for PPTs and PPTols in both sessions was excellent (ICC = 0.859 to 0.988, P ≤ 0.002). Inter-session reliability was good to excellent (ICC = 0.683 to 0.922, SEM = 15.06 to 47.04 g, P ≤ 0.001). Significant correlations were found between the two tools for all sites for PPTs (r = 0.500 to 0.614, P ≤ 0.009) and PPTols (r = 0.809 to 0.842, P algometer is a reliable and valid instrument for measuring PPTs and PPTols in the vestibular area in women with PVD. This technology is promising for pinpointing treatment mechanisms and efficacy. © 2015 American Academy of Pain Medicine. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

  7. An instrument to assess subjective task value beliefs regarding the decision to pursue postgraduate training.

    Science.gov (United States)

    Hagemeier, Nicholas E; Murawski, Matthew M

    2014-02-12

    To develop and validate an instrument to assess subjective ratings of the perceived value of various postgraduate training paths followed using expectancy-value as a theoretical framework; and to explore differences in value beliefs across type of postgraduate training pursued and type of pharmacy training completed prior to postgraduate training. A survey instrument was developed to sample 4 theoretical domains of subjective task value: intrinsic value, attainment value, utility value, and perceived cost. Retrospective self-report methodology was employed to examine respondents' (N=1,148) subjective task value beliefs specific to their highest level of postgraduate training completed. Exploratory and confirmatory factor analytic techniques were used to evaluate and validate value belief constructs. Intrinsic, attainment, utility, cost, and financial value constructs resulted from exploratory factor analysis. Cross-validation resulted in a 26-item instrument that demonstrated good model fit. Differences in value beliefs were noted across type of postgraduate training pursued and pharmacy training characteristics. The Postgraduate Training Value Instrument demonstrated evidence of reliability and construct validity. The survey instrument can be used to assess value beliefs regarding multiple postgraduate training options in pharmacy and potentially inform targeted recruiting of individuals to those paths best matching their own value beliefs.

  8. The effect of Web-based Braden Scale training on the reliability and precision of Braden Scale pressure ulcer risk assessments.

    Science.gov (United States)

    Magnan, Morris A; Maklebust, Joann

    2008-01-01

    To evaluate the effect of Web-based Braden Scale training on the reliability and precision of pressure ulcer risk assessments made by registered nurses (RN) working in acute care settings. Pretest-posttest, 2-group, quasi-experimental design. Five hundred Braden Scale risk assessments were made on 102 acute care patients deemed to be at various levels of risk for pressure ulceration. Assessments were made by RNs working in acute care hospitals at 3 different medical centers where the Braden Scale was in regular daily use (2 medical centers) or new to the setting (1 medical center). The Braden Scale for Predicting Pressure Sore Risk was used to guide pressure ulcer risk assessments. A Web-based version of the Detroit Medical Center Braden Scale Computerized Training Module was used to teach nurses correct use of the Braden Scale and selection of risk-based pressure ulcer prevention interventions. In the aggregate, RN generated reliable Braden Scale pressure ulcer risk assessments 65% of the time after training. The effect of Web-based Braden Scale training on reliability and precision of assessments varied according to familiarity with the scale. With training, new users of the scale made reliable assessments 84% of the time and significantly improved precision of their assessments. The reliability and precision of Braden Scale risk assessments made by its regular users was unaffected by training. Technology-assisted Braden Scale training improved both reliability and precision of risk assessments made by new users of the scale, but had virtually no effect on the reliability or precision of risk assessments made by regular users of the instrument. Further research is needed to determine best approaches for improving reliability and precision of Braden Scale assessments made by its regular users.

  9. [Reliability and validity of the PAQ-A questionnaire to assess physical activity in Spanish adolescents].

    Science.gov (United States)

    Martínez-Gómez, David; Martínez-de-Haro, Vicente; Pozo, Tamara; Welk, Gregory J; Villagra, Ariel; Calle, Marisa E; Marcos, Ascensión; Veiga, Oscar L

    2009-01-01

    Questionnaires are feasible instruments to assess physical activity (PA) in large samples. The aim of the current study was to evaluate the reliability and validity of the PAQ-A questionnaire in Spanish adolescents using the measurement of PA by accelerometer as criterion. In a sample of 82 adolescents, aged 12 to 17 years, 1-week PAQ-A test-retest was administered. Reliability was analyzed by the Intraclass Correlation Coefficient (ICC) and the internal consistency by the Cronbach's alpha Coefficient. Two hundred thirty-two adolescents, aged 13-17 years, completed the PAQ-A and wore the ActiGraph GT1M accelerometer during 7-days. The PAQ-A was compared against total PA and moderate to vigorous PA (MVPA) obtained by the accelerometer. Test-retest reliability showed ICC = 0.71 for the final score of PAQ-A. Internal consistency was alpha = 0.65 in the first self-report, alpha = 0.67 in the retest in 82 adolescents sample, and alpha = 0.74 in the 232 adolescents sample. The PAQ-A was moderately correlated with total PA (rho = 0.39) and MVPA (rho= 0.34) assessed by the accelerometer. The PAQ-A obtained significantly moderate correlations in boys but not in girls against the accelerometer. The PAQ-A questionnaire shows an adequate reliability and a reasonable validity for assessing PA in Spanish adolescents.

  10. Development and Testing of Screen-Based and Psychometric Instruments for Assessing Resident Performance in an Operating Room Simulator

    Directory of Open Access Journals (Sweden)

    Richard R. McNeer

    2016-01-01

    Full Text Available Introduction. Medical simulators are used for assessing clinical skills and increasingly for testing hypotheses. We developed and tested an approach for assessing performance in anesthesia residents using screen-based simulation that ensures expert raters remain blinded to subject identity and experimental condition. Methods. Twenty anesthesia residents managed emergencies in an operating room simulator by logging actions through a custom graphical user interface. Two expert raters rated performance based on these entries using custom Global Rating Scale (GRS and Crisis Management Checklist (CMC instruments. Interrater reliability was measured by calculating intraclass correlation coefficients (ICC, and internal consistency of the instruments was assessed with Cronbach’s alpha. Agreement between GRS and CMC was measured using Spearman rank correlation (SRC. Results. Interrater agreement (GRS: ICC = 0.825, CMC: ICC = 0.878 and internal consistency (GRS: alpha = 0.838, CMC: alpha = 0.886 were good for both instruments. Subscale analysis indicated that several instrument items can be discarded. GRS and CMC scores were highly correlated (SRC = 0.948. Conclusions. In this pilot study, we demonstrated that screen-based simulation can allow blinded assessment of performance. GRS and CMC instruments demonstrated good rater agreement and internal consistency. We plan to further test construct validity of our instruments by measuring performance in our simulator as a function of training level.

  11. Korean round-robin result for new international program to assess the reliability of emerging nondestructive techniques

    Energy Technology Data Exchange (ETDEWEB)

    Kim, Kyung Cho; Kim, Jin Gyum; Kang, Sung Sik; Jhung, Myung Jo [Korea Institute of Nuclear Safety, Daejeon (Korea, Republic of)

    2017-04-15

    The Korea Institute of Nuclear Safety, as a representative organization of Korea, in February 2012 participated in an international Program to Assess the Reliability of Emerging Nondestructive Techniques initiated by the U.S. Nuclear Regulatory Commission. The goal of the Program to Assess the Reliability of Emerging Nondestructive Techniques is to investigate the performance of emerging and prospective novel nondestructive techniques to find flaws in nickel-alloy welds and base materials. In this article, Korean round-robin test results were evaluated with respect to the test blocks and various nondestructive examination techniques. The test blocks were prepared to simulate large-bore dissimilar metal welds, small-bore dissimilar metal welds, and bottom-mounted instrumentation penetration welds in nuclear power plants. Also, lessons learned from the Korean round-robin test were summarized and discussed.

  12. Systems reliability analysis: applications of the SPARCS System-Reliability Assessment Computer Program

    International Nuclear Information System (INIS)

    Locks, M.O.

    1978-01-01

    SPARCS-2 (Simulation Program for Assessing the Reliabilities of Complex Systems, Version 2) is a PL/1 computer program for assessing (establishing interval estimates for) the reliability and the MTBF of a large and complex s-coherent system of any modular configuration. The system can consist of a complex logical assembly of independently failing attribute (binomial-Bernoulli) and time-to-failure (Poisson-exponential) components, without regard to their placement. Alternatively, it can be a configuration of independently failing modules, where each module has either or both attribute and time-to-failure components. SPARCS-2 also has an improved super modularity feature. Modules with minimal-cut unreliabiliy calculations can be mixed with those having minimal-path reliability calculations. All output has been standardized to system reliability or probability of success, regardless of the form in which the input data is presented, and whatever the configuration of modules or elements within modules

  13. Health Service Quality Scale: Brazilian Portuguese translation, reliability and validity.

    Science.gov (United States)

    Rocha, Luiz Roberto Martins; Veiga, Daniela Francescato; e Oliveira, Paulo Rocha; Song, Elaine Horibe; Ferreira, Lydia Masako

    2013-01-17

    The Health Service Quality Scale is a multidimensional hierarchical scale that is based on interdisciplinary approach. This instrument was specifically created for measuring health service quality based on marketing and health care concepts. The aim of this study was to translate and culturally adapt the Health Service Quality Scale into Brazilian Portuguese and to assess the validity and reliability of the Brazilian Portuguese version of the instrument. We conducted a cross-sectional, observational study, with public health system patients in a Brazilian university hospital. Validity was assessed using Pearson's correlation coefficient to measure the strength of the association between the Brazilian Portuguese version of the instrument and the SERVQUAL scale. Internal consistency was evaluated using Cronbach's alpha coefficient; the intraclass (ICC) and Pearson's correlation coefficients were used for test-retest reliability. One hundred and sixteen consecutive postoperative patients completed the questionnaire. Pearson's correlation coefficient for validity was 0.20. Cronbach's alpha for the first and second administrations of the final version of the instrument were 0.982 and 0.986, respectively. For test-retest reliability, Pearson's correlation coefficient was 0.89 and ICC was 0.90. The culturally adapted, Brazilian Portuguese version of the Health Service Quality Scale is a valid and reliable instrument to measure health service quality.

  14. Good practices in normal childbirth: reliability analysis of an instrument by Cronbach's Alpha.

    Science.gov (United States)

    Gottems, Leila Bernarda Donato; Carvalho, Elisabete Mesquita Peres De; Guilhem, Dirce; Pires, Maria Raquel Gomes Maia

    2018-01-01

    to analyze the internal consistency of the evaluation instrument of the adherence to the good practices of childbirth and birth care in the professionals, through Cronbach's Alpha Coefficient for each of the dimensions and for the total instrument. this is a descriptive and cross-sectional study performed in obstetric centers of eleven public hospitals in the Federal District, with a questionnaire applied to 261 professionals who worked in the delivery care. The study was attended by 261 professionals, 42.5% (111) nurses and 57.5% (150) physicians. The reliability evaluation of the instrument by the Cronbach Alfa resulted in 0.53, 0.78 and 0.76 for dimensions 1, 2 and 3, after debugging that resulted in the exclusion of 11 items. the instrument obtained Cronbach's alpha of 0.80. There is a need for improvement in the items of dimension 1 that refer to attitudes, knowledge, and practices of the organization of the network of care to gestation, childbirth, and birth. However, it can be applied in the way it is used to evaluate practices based on scientific evidence of childbirth care.

  15. Determination of validity and reliability of performance assessments tasks developed for selected topics in high school chemistry

    Science.gov (United States)

    Zichittella, Gail Eberhardt

    The primary purpose of this study was to validate performance assessments, which can be used as teaching and assessment instruments in high school science classrooms. This study evaluated the classroom usability of these performance instruments and establishes the interrater reliability of the scoring rubrics when used by classroom teachers. The assessment instruments were designed to represent two levels of scientific inquiry. The high level of inquiry tasks are relatively unstructured in terms of student directions; the low inquiry tasks provided more structure for the student. The tasks cover two content topics studied in chemistry (scientific observation and density). Students from a variety of Western New York school districts who were enrolled in chemistry classes and other science courses were involved in completion of the tasks at the two levels of inquiry. The chemistry students completed the NYS Regents Examination in Chemistry. Their classroom teachers were interviewed and completed a questionnaire to aid in the establishment their epistemological view on the inclusion of inquiry based learning in the science classroom. Data showed that the performance assessment tasks were reliable, valid and helpful for obtaining a more complete picture of the students' scientific understanding. The teacher participants reported no difficulty with the usability of the task in the high school chemistry setting. Collected data gave no evidence of gender bias with reference to the performance tasks or the NYS Regents Chemistry Examination. Additionally, it was shown that the instructors' classroom practices do have an effect upon the students' achievement on the performance tasks and the NYS Regents examination. Data also showed that achievement on the performance tasks was influenced by the number of years of science instruction students had received.

  16. Development of the Test Of Astronomy STandards (TOAST) Assessment Instrument

    Science.gov (United States)

    Slater, Timothy F.; Slater, S. J.

    2008-05-01

    Considerable effort in the astronomy education research (AER) community over the past several years has focused on developing assessment tools in the form of multiple-choice conceptual diagnostics and content knowledge surveys. This has been critically important in advancing the AER discipline so that researchers could establish the initial knowledge state of students as well as to attempt measure some of the impacts of innovative instructional interventions. Unfortunately, few of the existing instruments were constructed upon a solid list of clearly articulated and widely agreed upon learning objectives. This was not done in oversight, but rather as a result of the relative youth of AER as a discipline. Now that several important science education reform documents exist and are generally accepted by the AER community, we are in a position to develop, validate, and disseminate a new assessment instrument which is tightly aligned to the consensus learning goals stated by the American Astronomical Society - Chair's Conference on ASTRO 101, the American Association of the Advancement of Science's Project 2061 Benchmarks, and the National Research Council's National Science Education Standards. In response, researchers from the Cognition in Astronomy, Physics and Earth sciences Research (CAPER) Team at the University of Wyoming's Science & Math Teaching Center (UWYO SMTC) have designed a criterion-referenced assessment tool, called the Test Of Astronomy STandards (TOAST). Through iterative development, this instrument has a high degree of reliability and validity for instructors and researchers needing information on students’ initial knowledge state at the beginning of a course and can be used, in aggregate, to help measure the impact of course-length duration instructional strategies for courses with learning goals tightly aligned to the consensus goals of our community.

  17. Preliminary investigation on reliability assessment of passive safety system

    International Nuclear Information System (INIS)

    Huang Changfan; Kuang Bo

    2012-01-01

    The reliability evaluation of passive safety system plays an important part in probabilistic safety assessment (PSA) of nuclear power plant applying passive safety design, which depends quantitatively on reliabilities of passive safety system. According to the object of reliability assessment of passive safety system, relevant parameters are identified. Then passive system behavior during accident scenarios are studied. A practical example of this method is given for the case of reliability assessment of AP1000 passive heat removal system in loss of normal feedwater accident. Key and design parameters of PRHRS are identified and functional failure criteria are established. Parameter combinations acquired by Latin hyper~ cube sampling (LHS) in possible parametric ranges are input and calculations of uncertainty propagation through RELAP5/MOD3 code are carried out. Based on the calculations, sensitivity assessment on PRHRS functional criteria and reliability evaluation of the system are presented, which might provide further PSA with PRHR system reliability. (authors)

  18. A systematic review of instruments for assessing parent satisfaction with family-centred care in neonatal intensive care units.

    Science.gov (United States)

    Dall'Oglio, Immacolata; Mascolo, Rachele; Gawronski, Orsola; Tiozzo, Emanuela; Portanova, Anna; Ragni, Angela; Alvaro, Rosaria; Rocco, Gennaro; Latour, Jos M

    2018-03-01

    This systematic review synthesised and described instruments measuring parent satisfaction with the increasing standard practice of family-centred care (FCC) in neonatal intensive care units. We evaluated 11 studies published from January 2006 to March 2016: two studies validated a parent satisfaction questionnaire, and nine developed or modified previous questionnaires to use as outcome measures in their local settings. Most instruments were not tested on reliability and validity. Only two validated instruments included all six of the FCC principles and could assess parent satisfaction with FCC in neonatal intensive care units and be considered as outcome indicators for further research. ©2017 Foundation Acta Paediatrica. Published by John Wiley & Sons Ltd.

  19. The Sound Access Parent Outcomes Instrument (SAPOI): Construction of a new instrument for children with severe multiple disabilities who use cochlear implants or hearing aids.

    Science.gov (United States)

    Hayward, Denyse V; Ritter, Kathryn; Mousavi, Amin; Vatanapour, Shabnam

    2016-01-01

    To report on the Phase 2 development of the Sound Access Parent Outcomes Instrument (SAPOI), a new instrument focused on formalizing outcomes that parents of children with severe multiple disabilities (SMD) who use amplification prioritize as important. Phase 2 of this project involved item selection and refinement of the SAPOI based on (a) Phase 1 study participant input, (b) clinical specialist feedback, and (c) test-retest instrument reliability. Phase 1 participant responses were utilized to construct a draft version of the SAPOI. Next, clinical specialists examined the instrument for content validity and utility and instrument reliability was examined through a test-retest process with parents of children with SMD. The draft SAPOI was constructed based on Phase 1 participant input. Clinical specialists supported content validity and utility of the instrument and the inclusion of 19 additional items across four categories, namely Child Affect, Child Interaction, Parent Well-being, and Child's Device Use. The SAPOI was completed twice at one-month intervals by parents of children with SMD to examine instrument reliability across the four categories (Child Affect, Child Interaction, Parent Well-being, and Child's Device Use). Instrument reliability was strong-to-excellent across all four sections. The SAPOI shows promise as a much-needed addition to the assessment battery currently used for children with SMD who use cochlear implants and hearing aids. It provides valuable information regarding outcomes resulting from access to sound in this population that currently used assessments do not identify.

  20. Development and evaluation of the OHCITIES instrument: assessing alcohol urban environments in the Heart Healthy Hoods project

    Science.gov (United States)

    Sureda, Xisca; Espelt, Albert; Villalbí, Joan R; Cebrecos, Alba; Baranda, Lucía; Pearce, Jamie; Franco, Manuel

    2017-01-01

    Objectives To describe the development and test–retest reliability of OHCITIES, an instrument characterising alcohol urban environment in terms of availability, promotion and signs of consumption. Design This study involved: (1) developing the conceptual framework for alcohol urban environment by means of literature reviewing and previous alcohol environment research experience; (2) pilot testing and redesigning the instrument; (3) instrument digitalisation; (4) instrument evaluation using test–retest reliability. Setting Data for testing the reliability of the instrument were collected in seven census sections in Madrid in 2016 by two observers. Primary and secondary outcome measures We computed per cent agreement and Cohen’s kappa coefficients to estimate inter-rater and test–retest reliability for alcohol outlet environment measures. We calculated interclass coefficients and their 95% CIs to provide a measure of inter-rater reliability for signs of alcohol consumption measures. Results We collected information on 92 on-premise and 24 off-premise alcohol outlets identified in the studied areas about availability, accessibility and promotion of alcohol. Most per cent-agreement values for alcohol measures in on-premise and off-premise alcohol outlets were greater than 80%, and inter-rater and test–retest reliability values were generally above 0.80. Observers identified 26 streets and 3 public squares with signs of alcohol consumption. Intraclass correlation coefficient between observers for any type of signs of alcohol consumption was 0.50 (95% CI −0.09 to 0.77). Few items promoting alcohol unrelated to alcohol outlets were found on public spaces. Conclusions The OHCITIES instrument is a reliable instrument to characterise alcohol urban environment. This instrument might be used to understand how alcohol environment associates with alcohol behaviours and its related health outcomes, and can help in the design and evaluation of policies to reduce the

  1. Reliability and comparison of acromion assessment techniques on x-ray and magnetic resonance imaging (reliability of acromion assessment techniques)

    International Nuclear Information System (INIS)

    Viskontas, D.G.; MacDermid, J.C.; Drosdowech, D.S.; Garvin, G.J.; Romano, W.M.; Faber, K.J.

    2005-01-01

    To determine the reliability and correlation of plain radiography and magnetic resonance imaging (MRI) in the assessment of acromion morphology. Materials and Methods: Acromion morphology was assessed using the lateral acromion angle (LAA) and the acromion-humeral interval (AHI). Thirty patients who had x-rays and MRI for impingement syndrome were included. Six blinded observers assessed the acromion morphology subjectively and objectively. Results: Neither acromion assessment technique demonstrated a positive correlation (kappa and intraclass coefficient 0.55) when measured objectively by experienced observers. Conclusion: The LAA and the AHI are both reliable acromion assessment techniques on X-ray and MRI when measured objectively and by experienced observers. (author)

  2. Evaluation of the FOCUS (Feedback on Counseling Using Simulation) instrument for assessment of client-centered nutrition counseling behaviors.

    Science.gov (United States)

    Henry, Beverly W; Smith, Thomas J

    2010-01-01

    To develop an instrument to assess client-centered counseling behaviors (skills) of student-counselors in a standardized patient (SP) exercise. Descriptive study of the accuracy and utility of a newly developed counseling evaluation instrument. Study participants included 11 female student-counselors at a Midwestern university-10 Caucasian, 1 African-American-for the simulated counseling sessions, in which the Feedback on Counseling Using Simulation (FOCUS) instrument was applied in 2 SP scenarios (cardiovascular disease and diabetes). FOCUS ratings of student-counselors by 4 SPs during 22 sessions were compared with ratings from a 3-member panel of experts who independently viewed the 22 videotaped sessions. Quantitative analysis of instrument validity included inter-rater reliability by computing generalizability coefficients, Pearson correlations, and Spearman rank-order correlations. FOCUS criteria encompassed relevant dimensions of nutrition counseling based in a client-centered perspective. The critical points of information gathering and counseling behaviors showed internal consistency overall and good inter-rater reliability with the cardiovascular disease scenario. For both scenarios, pooled ratings of 3 experts agreed with ratings carried out by SPs. Initial findings suggest that the FOCUS instrument with client-centered criteria may enhance evaluation of counseling skills in SP exercises, meriting further study with larger groups.

  3. The Child Behaviour Assessment Instrument: development and validation of a measure to screen for externalising child behavioural problems in community setting

    Directory of Open Access Journals (Sweden)

    Perera Hemamali

    2010-06-01

    Full Text Available Abstract Background In Sri Lanka, behavioural problems have grown to epidemic proportions accounting second highest category of mental health problems among children. Early identification of behavioural problems in children is an important pre-requisite of the implementation of interventions to prevent long term psychiatric outcomes. The objectives of the study were to develop and validate a screening instrument for use in the community setting to identify behavioural problems in children aged 4-6 years. Methods An initial 54 item questionnaire was developed following an extensive review of the literature. A three round Delphi process involving a panel of experts from six relevant fields was then undertaken to refine the nature and number of items and created the 15 item community screening instrument, Child Behaviour Assessment Instrument (CBAI. Validation study was conducted in the Medical Officer of Health area Kaduwela, Sri Lanka and a community sample of 332 children aged 4-6 years were recruited by two stage randomization process. The behaviour status of the participants was assessed by an interviewer using the CBAI and a clinical psychologist following clinical assessment concurrently. Criterion validity was appraised by assessing the sensitivity, specificity and predictive values at the optimum screen cut off value. Construct validity of the instrument was quantified by testing whether the data of validation study fits to a hypothetical model. Face and content validity of the CBAI were qualitatively assessed by a panel of experts. The reliability of the instrument was assessed by internal consistency analysis and test-retest methods in a 15% subset of the community sample. Results Using the Receiver Operating Characteristic analysis the CBAI score of >16 was identified as the cut off point that optimally differentiated children having behavioural problems, with a sensitivity of 0.88 (95% CI = 0.80-0.96 and specificity of 0.81 (95% CI = 0

  4. Reliability assessment based on subjective inferences

    International Nuclear Information System (INIS)

    Ma Zhibo; Zhu Jianshi; Xu Naixin

    2003-01-01

    The reliability information which comes from subjective analysis is often incomplete prior. This information can be generally assumed to exist in the form of either a stated prior mean of R (reliability) or a stated prior credibility interval on R. An efficient approach is developed to determine a complete beta prior distribution from the subjective information according to the principle of maximum entropy, and the the reliability of survival/failure product is assessed via Bayes theorem. Numerical examples are presented to illustrate the methods

  5. Development of Instruments to Assess Shame and Guilt in Adolescents: Empirical Differences Between the Constructs

    Directory of Open Access Journals (Sweden)

    Lorena Maria Laskoski

    2013-05-01

    Full Text Available Shame and guilt are terms easily mixed in everyday life and often used as synonyms. Although they are words of similar use, they represent theoretically different moral emotions. The aims of this study were to develop and validate instruments to assess shame and guilt and to test the empirical independence of the constructs. Five hundred and eighty high school students from three Brazilian states participated in this study, 55% of them female, with an average age of 16.0 years. The participants answered a questionnaire containing socio demographic questions and scales designed for this study in order to assess shame and guilt. The scales developed gave appropriate evidence of validity and reliability. A single factor was extracted to assess shame and two factors to assess guilt (recognition of mistake and regret. The constructs revealed to be correlated, but independent. It is considered that there is sufficient evidence to affirm that shame and guilt are emotions with distinctive peculiarities and can be measured using the instruments developed in this research.

  6. Development and reliability testing of a self-report instrument to measure the office layout as a correlate of occupational sitting

    Directory of Open Access Journals (Sweden)

    Duncan Mitch J

    2013-02-01

    Full Text Available Abstract Background Spatial configurations of office environments assessed by Space Syntax methodologies are related to employee movement patterns. These methods require analysis of floors plans which are not readily available in large population-based studies or otherwise unavailable. Therefore a self-report instrument to assess spatial configurations of office environments using four scales was developed. Methods The scales are: local connectivity (16 items, overall connectivity (11 items, visibility of co-workers (10 items, and proximity of co-workers (5 items. A panel cohort (N = 1154 completed an online survey, only data from individuals employed in office-based occupations (n = 307 were used to assess scale measurement properties. To assess test-retest reliability a separate sample of 37 office-based workers completed the survey on two occasions 7.7 (±3.2 days apart. Redundant scale items were eliminated using factor analysis; Chronbach’s α was used to evaluate internal consistency and test re-test reliability (retest-ICC. ANOVA was employed to examine differences between office types (Private, Shared, Open as a measure of construct validity. Generalized Linear Models were used to examine relationships between spatial configuration scales and the duration of and frequency of breaks in occupational sitting. Results The number of items on all scales were reduced, Chronbach’s α and ICCs indicated good scale internal consistency and test re-test reliability: local connectivity (5 items; α = 0.70; retest-ICC = 0.84, overall connectivity (6 items; α = 0.86; retest-ICC = 0.87, visibility of co-workers (4 items; α = 0.78; retest-ICC = 0.86, and proximity of co-workers (3 items; α = 0.85; retest-ICC = 0.70. Significant (p ≤ 0.001 differences, in theoretically expected directions, were observed for all scales between office types, except overall connectivity. Significant associations were

  7. Music@Home: A novel instrument to assess the home musical environment in the early years.

    Science.gov (United States)

    Politimou, Nina; Stewart, Lauren; Müllensiefen, Daniel; Franco, Fabia

    2018-01-01

    The majority of children under the age of 5 appear to show spontaneous enjoyment of singing, being exposed to music and interacting with musical instruments, but whether variations in engaging in such activities in the home could contribute to developmental outcomes is still largely unknown. Critically, researchers lack a comprehensive instrument with good psychometric properties to assess the home musical environment from infancy to the preschool years. To address this gap, this paper presents two studies that describe the development and validation of the Music@Home questionnaire, which comprises two versions: Infant and Preschool. In Study 1, an initial pool of items was generated and administered to a wide audience of parents (n = 287 for the Infant, n = 347 for the Preschool version). Exploratory factor analysis was used to identify different dimensions comprising the home musical environment of both infants and pre-schoolers, and to reduce the initial pool of items to a smaller number of meaningful items. In Study 2, convergent and divergent validity and internal and test-retest reliability of the new instrument were established, using data from a different sample of participants (n = 213 for the Infant, n = 213 for the Preschool version). The second study also investigated associations between the Music@Home and musical characteristics of the parents, such as their musical education and personal engagement with music. Overall, the Music@Home constitutes a novel, valid and reliable instrument that allows for the systematic assessment of distinct aspects of the home musical environment in families with children under the age of 5. Furthermore, the Infant and Preschool versions of the Music@Home present differential associations with musical characteristics of the parents opening a new area of inquiry into how musical exposure and interaction in the home may vary across different developmental stages.

  8. Music@Home: A novel instrument to assess the home musical environment in the early years

    Science.gov (United States)

    Stewart, Lauren; Müllensiefen, Daniel; Franco, Fabia

    2018-01-01

    The majority of children under the age of 5 appear to show spontaneous enjoyment of singing, being exposed to music and interacting with musical instruments, but whether variations in engaging in such activities in the home could contribute to developmental outcomes is still largely unknown. Critically, researchers lack a comprehensive instrument with good psychometric properties to assess the home musical environment from infancy to the preschool years. To address this gap, this paper presents two studies that describe the development and validation of the Music@Home questionnaire, which comprises two versions: Infant and Preschool. In Study 1, an initial pool of items was generated and administered to a wide audience of parents (n = 287 for the Infant, n = 347 for the Preschool version). Exploratory factor analysis was used to identify different dimensions comprising the home musical environment of both infants and pre-schoolers, and to reduce the initial pool of items to a smaller number of meaningful items. In Study 2, convergent and divergent validity and internal and test-retest reliability of the new instrument were established, using data from a different sample of participants (n = 213 for the Infant, n = 213 for the Preschool version). The second study also investigated associations between the Music@Home and musical characteristics of the parents, such as their musical education and personal engagement with music. Overall, the Music@Home constitutes a novel, valid and reliable instrument that allows for the systematic assessment of distinct aspects of the home musical environment in families with children under the age of 5. Furthermore, the Infant and Preschool versions of the Music@Home present differential associations with musical characteristics of the parents opening a new area of inquiry into how musical exposure and interaction in the home may vary across different developmental stages. PMID:29641607

  9. Sequential decision reliability concept and failure rate assessment

    International Nuclear Information System (INIS)

    Ciftcioglu, O.

    1990-11-01

    Conventionally, a reliability concept is considered together with both each basic unit and their integration in a complicated large scale system such as a nuclear power plant (NPP). Basically, as the plant's operational status is determined by the information obtained from various sensors, the plant's reliability and the risk assessment is closely related to the reliability of the sensory information and hence the sensor components. However, considering the relevant information-processing systems, e.g. fault detection processors, there exists a further question about the reliability of such systems, specifically the reliability of the systems' decision-based outcomes by means of which the further actions are performed. To this end, a general sequential decision reliability concept and the failure rate assessment methodology is introduced. The implications of the methodology are investigated and the importance of the decision reliability concept in system operation is demonstrated by means of sensory signals in real-time from the Borssele NPP in the Netherlands. (author). 21 refs.; 8 figs

  10. A systematic review of instruments that assess the implementation of hospital quality management systems.

    Science.gov (United States)

    Groene, Oliver; Botje, Daan; Suñol, Rosa; Lopez, Maria Andrée; Wagner, Cordula

    2013-10-01

    Health-care providers invest substantial resources to establish and implement hospital quality management systems. Nevertheless, few tools are available to assess implementation efforts and their effect on quality and safety outcomes. This review aims to (i) identify instruments to assess the implementation of hospital quality management systems, (ii) describe their measurement properties and (iii) assess the effects of quality management on quality improvement and quality of care outcomes. We performed a systematic literature search from 1990 to 2011 in PubMed, CINAHL, EMBASE, Cochrane Library and Web of Science. In addition, we used snowball strategies, screened the reference lists of eligible papers, reviewed grey literature and contacted experts in the field. and data extraction Two reviewers screened eligible papers based on pre-defined inclusion and exclusion criteria and all authors extracted data. Eligible papers are described in terms of general characteristics (settings, type and level of respondents, mode of data collection), methodological properties (sampling strategy, item derivation, conceptualization of quality management, assessment of reliability and validity, scoring) and application/implementation (accounting for context, organizational adaptations, sensitivity to change, deployment and effect size). Eighteen papers were deemed eligible for inclusion. While some common domains emerged in measurement conceptualization, substantial differences in scope persist. The instruments' measurement properties were insufficiently described and only few instruments assessed links between the implementation of quality management systems (QMS) and improvement strategies or outcomes. There is currently no well-established measure to assess the implementation and effectiveness of quality management systems. Future research should address this gap.

  11. Reliability Assessment of Concrete Bridges

    DEFF Research Database (Denmark)

    Thoft-Christensen, Palle; Middleton, C. R.

    This paper is partly based on research performed for the Highways Agency, London, UK under the project DPU/9/44 "Revision of Bridge Assessment Rules Based on Whole Life Performance: concrete bridges". It contains the details of a methodology which can be used to generate Whole Life (WL) reliability...... profiles. These WL reliability profiles may be used to establish revised rules for concrete bridges. This paper is to some extend based on Thoft-Christensen et. al. [1996], Thoft-Christensen [1996] et. al. and Thoft-Christensen [1996]....

  12. Reliability, validity, and significance of assessment of sense of contribution in the workplace.

    Science.gov (United States)

    Takaki, Jiro; Taniguchi, Toshiyo; Fujii, Yasuhito

    2014-01-29

    The purpose of this study was to assess the validity and reliability of the Sense of Contribution Scale (SCS), a newly developed, 7-item questionnaire used to measure sense of contribution in the workplace. Workers at 272 organizations answered questionnaires that included the SCS. Because of non-participation or missing data, the number of subjects included in the analyses for internal consistency and validity varied from 1,675 to 2,462 (response rates 54.6%-80.2%). Fifty-four workers were included in the analysis of test-retest reliability (response rate, 77.1%). The SCS showed high internal consistency (Cronbach's α coefficients in men and women were 0.85 and 0.86, respectively) and test-retest reliability (intraclass correlation coefficient = 0.91). Significant (p workplace bullying, and procedural and interactional justice. The SCS is a psychometrically satisfactory measure of sense of contribution in the workplace. The SCS provides a new and useful instrument to measure sense of contribution, which is independently associated with mental health in workers, for studies in organizational science, occupational health psychology and occupational medicine.

  13. Automated reliability assessment for spectroscopic redshift measurements

    Science.gov (United States)

    Jamal, S.; Le Brun, V.; Le Fèvre, O.; Vibert, D.; Schmitt, A.; Surace, C.; Copin, Y.; Garilli, B.; Moresco, M.; Pozzetti, L.

    2018-03-01

    Context. Future large-scale surveys, such as the ESA Euclid mission, will produce a large set of galaxy redshifts (≥106) that will require fully automated data-processing pipelines to analyze the data, extract crucial information and ensure that all requirements are met. A fundamental element in these pipelines is to associate to each galaxy redshift measurement a quality, or reliability, estimate. Aim. In this work, we introduce a new approach to automate the spectroscopic redshift reliability assessment based on machine learning (ML) and characteristics of the redshift probability density function. Methods: We propose to rephrase the spectroscopic redshift estimation into a Bayesian framework, in order to incorporate all sources of information and uncertainties related to the redshift estimation process and produce a redshift posterior probability density function (PDF). To automate the assessment of a reliability flag, we exploit key features in the redshift posterior PDF and machine learning algorithms. Results: As a working example, public data from the VIMOS VLT Deep Survey is exploited to present and test this new methodology. We first tried to reproduce the existing reliability flags using supervised classification in order to describe different types of redshift PDFs, but due to the subjective definition of these flags (classification accuracy 58%), we soon opted for a new homogeneous partitioning of the data into distinct clusters via unsupervised classification. After assessing the accuracy of the new clusters via resubstitution and test predictions (classification accuracy 98%), we projected unlabeled data from preliminary mock simulations for the Euclid space mission into this mapping to predict their redshift reliability labels. Conclusions: Through the development of a methodology in which a system can build its own experience to assess the quality of a parameter, we are able to set a preliminary basis of an automated reliability assessment for

  14. Operator reliability assessment system (OPERAS)

    International Nuclear Information System (INIS)

    Singh, A.; Spurgin, A.J.; Martin, T.; Welsch, J.; Hallam, J.W.

    1991-01-01

    OPERAS is a personal-computer (PC) based software to collect and process simulator data on control-room operators responses during requalification training scenarios. The data collection scheme is based upon approach developed earlier during the EPRI Operator Reliability Experiments project. The software allows automated data collection from simulator, thus minimizing simulator staff time and resources to collect, maintain and process data which can be useful in monitoring, assessing and enhancing the progress of crew reliability and effectiveness. The system is designed to provide the data and output information in the form of user-friendly charts, tables and figures for use by plant staff. OPERAS prototype software has been implemented at the Diablo Canyon (PWR) and Millstone (BWR) plants and is currently being used to collect operator response data. Data collected from similator include plant-state variables such as reactor pressure and temperature, malfunction, times at which annunciators are activated, operator actions and observations of crew behavior by training staff. The data and systematic analytical results provided by the OPERAS system can contribute to increase objectivity by the utility probabilistic risk analysis (PRA) and training staff in monitoring and assessing reliability of their crews

  15. Is the Scale for Measuring Motivational Interviewing Skills a valid and reliable instrument for measuring the primary care professionals motivational skills?: EVEM study protocol.

    Science.gov (United States)

    Pérula, Luis Á; Campiñez, Manuel; Bosch, Josep M; Barragán Brun, Nieves; Arboniés, Juan C; Bóveda Fontán, Julia; Martín Alvarez, Remedios; Prados, Jose A; Martín-Rioboó, Enrique; Massons, Josep; Criado, Margarita; Fernández, José Á; Parras, Juan M; Ruiz-Moral, Roger; Novo, Jesús M

    2012-11-22

    Lifestyle is one of the main determinants of people's health. It is essential to find the most effective prevention strategies to be used to encourage behavioral changes in their patients. Many theories are available that explain change or adherence to specific health behaviors in subjects. In this sense the named Motivational Interviewing has increasingly gained relevance. Few well-validated instruments are available for measuring doctors' communication skills, and more specifically the Motivational Interviewing. The hypothesis of this study is that the Scale for Measuring Motivational Interviewing Skills (EVEM questionnaire) is a valid and reliable instrument for measuring the primary care professionals skills to get behavior change in patients. To test the hypothesis we have designed a prospective, observational, multi-center study to validate a measuring instrument. - Thirty-two primary care centers in Spain. -Sampling and Size: a) face and consensual validity: A group composed of 15 experts in Motivational Interviewing. b) Assessment of the psychometric properties of the scale; 50 physician- patient encounters will be videoed; a total of 162 interviews will be conducted with six standardized patients, and another 200 interviews will be conducted with 50 real patients (n=362). Four physicians will be specially trained to assess 30 interviews randomly selected to test the scale reproducibility. -Measurements for to test the hypothesis: a) Face validity: development of a draft questionnaire based on a theoretical model, by using Delphi-type methodology with experts. b) Scale psychometric properties: intraobservers will evaluate video recorded interviews: content-scalability validity (Exploratory Factor Analysis), internal consistency (Cronbach alpha), intra-/inter-observer reliability (Kappa index, intraclass correlation coefficient, Bland & Altman methodology), generalizability, construct validity and sensitivity to change (Pearson product-moment correlation

  16. Is the Scale for Measuring Motivational Interviewing Skills a valid and reliable instrument for measuring the primary care professionals motivational skills?: EVEM study protocol

    Directory of Open Access Journals (Sweden)

    Pérula Luis Á

    2012-11-01

    Full Text Available Abstract Background Lifestyle is one of the main determinants of people’s health. It is essential to find the most effective prevention strategies to be used to encourage behavioral changes in their patients. Many theories are available that explain change or adherence to specific health behaviors in subjects. In this sense the named Motivational Interviewing has increasingly gained relevance. Few well-validated instruments are available for measuring doctors’ communication skills, and more specifically the Motivational Interviewing. Methods/Design The hypothesis of this study is that the Scale for Measuring Motivational Interviewing Skills (EVEM questionnaire is a valid and reliable instrument for measuring the primary care professionals skills to get behavior change in patients. To test the hypothesis we have designed a prospective, observational, multi-center study to validate a measuring instrument. –Scope: Thirty-two primary care centers in Spain. -Sampling and Size: a face and consensual validity: A group composed of 15 experts in Motivational Interviewing. b Assessment of the psychometric properties of the scale; 50 physician- patient encounters will be videoed; a total of 162 interviews will be conducted with six standardized patients, and another 200 interviews will be conducted with 50 real patients (n=362. Four physicians will be specially trained to assess 30 interviews randomly selected to test the scale reproducibility. -Measurements for to test the hypothesis: a Face validity: development of a draft questionnaire based on a theoretical model, by using Delphi-type methodology with experts. b Scale psychometric properties: intraobservers will evaluate video recorded interviews: content-scalability validity (Exploratory Factor Analysis, internal consistency (Cronbach alpha, intra-/inter-observer reliability (Kappa index, intraclass correlation coefficient, Bland & Altman methodology, generalizability, construct validity and

  17. Assessing Reliability and Validity of the "GroPromo" Audit Tool for Evaluation of Grocery Store Marketing and Promotional Environments

    Science.gov (United States)

    Kerr, Jacqueline; Sallis, James F.; Bromby, Erica; Glanz, Karen

    2012-01-01

    Objective: To evaluate reliability and validity of a new tool for assessing the placement and promotional environment in grocery stores. Methods: Trained observers used the "GroPromo" instrument in 40 stores to code the placement of 7 products in 9 locations within a store, along with other promotional characteristics. To test construct validity,…

  18. Health service quality scale: Brazilian Portuguese translation, reliability and validity

    Science.gov (United States)

    2013-01-01

    Background The Health Service Quality Scale is a multidimensional hierarchical scale that is based on interdisciplinary approach. This instrument was specifically created for measuring health service quality based on marketing and health care concepts. The aim of this study was to translate and culturally adapt the Health Service Quality Scale into Brazilian Portuguese and to assess the validity and reliability of the Brazilian Portuguese version of the instrument. Methods We conducted a cross-sectional, observational study, with public health system patients in a Brazilian university hospital. Validity was assessed using Pearson’s correlation coefficient to measure the strength of the association between the Brazilian Portuguese version of the instrument and the SERVQUAL scale. Internal consistency was evaluated using Cronbach’s alpha coefficient; the intraclass (ICC) and Pearson’s correlation coefficients were used for test-retest reliability. Results One hundred and sixteen consecutive postoperative patients completed the questionnaire. Pearson’s correlation coefficient for validity was 0.20. Cronbach's alpha for the first and second administrations of the final version of the instrument were 0.982 and 0.986, respectively. For test-retest reliability, Pearson’s correlation coefficient was 0.89 and ICC was 0.90. Conclusions The culturally adapted, Brazilian Portuguese version of the Health Service Quality Scale is a valid and reliable instrument to measure health service quality. PMID:23327598

  19. Different Reliability Assessment Approaches for Wave Energy Converters

    DEFF Research Database (Denmark)

    Ambühl, Simon; Kramer, Morten Mejlhede; Sørensen, John Dalsgaard

    2015-01-01

    Reliability assessments are of importance for wave energy converters (WECs) due to the fact that accessibility might be limited in case of failure and maintenance. These failure rates can be adapted by reliability considerations. There are two different approaches to how reliability can...

  20. The Effect of Instrument-Specific Rater Training on Interrater Reliability and Counseling Skills Performance Differentiation

    Science.gov (United States)

    Meacham, Paul Douglas, Jr.

    2013-01-01

    The purpose of this study was to explore the effect of instrument-specific rater training on interrater reliability (IRR) and counseling skills performance differentiation. Strong IRR is of primary concern to effective program evaluation (McCullough, Kuhn, Andrews, Valen, Hatch, & Osimo, 2003; Schanche, Nielsen, McCullough, Valen, &…

  1. A clinical assessment tool used for physiotherapy students--is it reliable?

    Science.gov (United States)

    Lewis, Lucy K; Stiller, Kathy; Hardy, Frances

    2008-01-01

    Educational institutions providing professional programs such as physiotherapy must provide high-quality student assessment procedures. To ensure that assessment is consistent, assessment tools should have an acceptable level of reliability. There is a paucity of research evaluating the reliability of clinical assessment tools used for physiotherapy students. This study evaluated the inter- and intrarater reliability of an assessment tool used for physiotherapy students during a clinical placement. Five clinical educators and one academic participated in the study. Each rater independently marked 22 student written assessments that had been completed by students after viewing a videotaped patient physiotherapy assessment. The raters repeated the marking process 7 weeks later, with the assessments provided in a randomised order. The interrater reliability (Intraclass Correlation Coefficient) for the total scores was 0.32, representing a poor level of reliability. A high level of intrarater reliability (percentage agreement) was found for the clinical educators, with a difference in section scores of one mark or less on 93.4% of occasions. Further research should be undertaken to reevaluate the reliability of this clinical assessment tool following training. The reliability of clinical assessment tools used in other areas of physiotherapy education should be formally measured rather than assumed.

  2. Toward automated assessment of health Web page quality using the DISCERN instrument.

    Science.gov (United States)

    Allam, Ahmed; Schulz, Peter J; Krauthammer, Michael

    2017-05-01

    As the Internet becomes the number one destination for obtaining health-related information, there is an increasing need to identify health Web pages that convey an accurate and current view of medical knowledge. In response, the research community has created multicriteria instruments for reliably assessing online medical information quality. One such instrument is DISCERN, which measures health Web page quality by assessing an array of features. In order to scale up use of the instrument, there is interest in automating the quality evaluation process by building machine learning (ML)-based DISCERN Web page classifiers. The paper addresses 2 key issues that are essential before constructing automated DISCERN classifiers: (1) generation of a robust DISCERN training corpus useful for training classification algorithms, and (2) assessment of the usefulness of the current DISCERN scoring schema as a metric for evaluating the performance of these algorithms. Using DISCERN, 272 Web pages discussing treatment options in breast cancer, arthritis, and depression were evaluated and rated by trained coders. First, different consensus models were compared to obtain a robust aggregated rating among the coders, suitable for a DISCERN ML training corpus. Second, a new DISCERN scoring criterion was proposed (features-based score) as an ML performance metric that is more reflective of the score distribution across different DISCERN quality criteria. First, we found that a probabilistic consensus model applied to the DISCERN instrument was robust against noise (random ratings) and superior to other approaches for building a training corpus. Second, we found that the established DISCERN scoring schema (overall score) is ill-suited to measure ML performance for automated classifiers. Use of a probabilistic consensus model is advantageous for building a training corpus for the DISCERN instrument, and use of a features-based score is an appropriate ML metric for automated DISCERN

  3. Assessing the quality of decision support technologies using the International Patient Decision Aid Standards instrument (IPDASi.

    Directory of Open Access Journals (Sweden)

    Glyn Elwyn

    Full Text Available To describe the development, validation and inter-rater reliability of an instrument to measure the quality of patient decision support technologies (decision aids.Scale development study, involving construct, item and scale development, validation and reliability testing.There has been increasing use of decision support technologies--adjuncts to the discussions clinicians have with patients about difficult decisions. A global interest in developing these interventions exists among both for-profit and not-for-profit organisations. It is therefore essential to have internationally accepted standards to assess the quality of their development, process, content, potential bias and method of field testing and evaluation.Scale development study, involving construct, item and scale development, validation and reliability testing.Twenty-five researcher-members of the International Patient Decision Aid Standards Collaboration worked together to develop the instrument (IPDASi. In the fourth Stage (reliability study, eight raters assessed thirty randomly selected decision support technologies.IPDASi measures quality in 10 dimensions, using 47 items, and provides an overall quality score (scaled from 0 to 100 for each intervention. Overall IPDASi scores ranged from 33 to 82 across the decision support technologies sampled (n = 30, enabling discrimination. The inter-rater intraclass correlation for the overall quality score was 0.80. Correlations of dimension scores with the overall score were all positive (0.31 to 0.68. Cronbach's alpha values for the 8 raters ranged from 0.72 to 0.93. Cronbach's alphas based on the dimension means ranged from 0.50 to 0.81, indicating that the dimensions, although well correlated, measure different aspects of decision support technology quality. A short version (19 items was also developed that had very similar mean scores to IPDASi and high correlation between short score and overall score 0.87 (CI 0.79 to 0.92.This work

  4. Introduction: The effectiveness of impact assessment instruments

    OpenAIRE

    Cashmore, M; Bond, A; Sadler, B

    2009-01-01

    The global application of impact assessment instruments to achieve a variety of policy integration goals (e.g. the mainstreaming of environmental, gender or economic efficiency concerns) continues to proliferate. These instruments represent important components of contemporary political governance and hence are an important locus for applied research. This special issue of Impact Assessment and Project Appraisal critically examines 'state-of-the-art' knowledge and understanding of the effecti...

  5. Organizational readiness to change assessment (ORCA: Development of an instrument based on the Promoting Action on Research in Health Services (PARIHS framework

    Directory of Open Access Journals (Sweden)

    Li Yu-Fang

    2009-07-01

    Full Text Available Abstract Background The Promoting Action on Research Implementation in Health Services, or PARIHS, framework is a theoretical framework widely promoted as a guide to implement evidence-based clinical practices. However, it has as yet no pool of validated measurement instruments that operationalize the constructs defined in the framework. The present article introduces an Organizational Readiness to Change Assessment instrument (ORCA, organized according to the core elements and sub-elements of the PARIHS framework, and reports on initial validation. Methods We conducted scale reliability and factor analyses on cross-sectional, secondary data from three quality improvement projects (n = 80 conducted in the Veterans Health Administration. In each project, identical 77-item ORCA instruments were administered to one or more staff from each facility involved in quality improvement projects. Items were organized into 19 subscales and three primary scales corresponding to the core elements of the PARIHS framework: (1 Strength and extent of evidence for the clinical practice changes represented by the QI program, assessed with four subscales, (2 Quality of the organizational context for the QI program, assessed with six subscales, and (3 Capacity for internal facilitation of the QI program, assessed with nine subscales. Results Cronbach's alpha for scale reliability were 0.74, 0.85 and 0.95 for the evidence, context and facilitation scales, respectively. The evidence scale and its three constituent subscales failed to meet the conventional threshold of 0.80 for reliability, and three individual items were eliminated from evidence subscales following reliability testing. In exploratory factor analysis, three factors were retained. Seven of the nine facilitation subscales loaded onto the first factor; five of the six context subscales loaded onto the second factor; and the three evidence subscales loaded on the third factor. Two subscales failed to load

  6. Organizational readiness to change assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework.

    Science.gov (United States)

    Helfrich, Christian D; Li, Yu-Fang; Sharp, Nancy D; Sales, Anne E

    2009-07-14

    The Promoting Action on Research Implementation in Health Services, or PARIHS, framework is a theoretical framework widely promoted as a guide to implement evidence-based clinical practices. However, it has as yet no pool of validated measurement instruments that operationalize the constructs defined in the framework. The present article introduces an Organizational Readiness to Change Assessment instrument (ORCA), organized according to the core elements and sub-elements of the PARIHS framework, and reports on initial validation. We conducted scale reliability and factor analyses on cross-sectional, secondary data from three quality improvement projects (n = 80) conducted in the Veterans Health Administration. In each project, identical 77-item ORCA instruments were administered to one or more staff from each facility involved in quality improvement projects. Items were organized into 19 subscales and three primary scales corresponding to the core elements of the PARIHS framework: (1) Strength and extent of evidence for the clinical practice changes represented by the QI program, assessed with four subscales, (2) Quality of the organizational context for the QI program, assessed with six subscales, and (3) Capacity for internal facilitation of the QI program, assessed with nine subscales. Cronbach's alpha for scale reliability were 0.74, 0.85 and 0.95 for the evidence, context and facilitation scales, respectively. The evidence scale and its three constituent subscales failed to meet the conventional threshold of 0.80 for reliability, and three individual items were eliminated from evidence subscales following reliability testing. In exploratory factor analysis, three factors were retained. Seven of the nine facilitation subscales loaded onto the first factor; five of the six context subscales loaded onto the second factor; and the three evidence subscales loaded on the third factor. Two subscales failed to load significantly on any factor. One measured resources

  7. An approach for assessing human decision reliability

    International Nuclear Information System (INIS)

    Pyy, P.

    2000-01-01

    This paper presents a method to study human reliability in decision situations related to nuclear power plant disturbances. Decisions often play a significant role in handling of emergency situations. The method may be applied to probabilistic safety assessments (PSAs) in cases where decision making is an important dimension of an accident sequence. Such situations are frequent e.g. in accident management. In this paper, a modelling approach for decision reliability studies is first proposed. Then, a case study with two decision situations with relatively different characteristics is presented. Qualitative and quantitative findings of the study are discussed. In very simple decision cases with time pressure, time reliability correlation proved out to be a feasible reliability modelling method. In all other decision situations, more advanced probabilistic decision models have to be used. Finally, decision probability assessment by using simulator run results and expert judgement is presented

  8. Generation reliability assessment in oligopoly power market using Monte Carlo simulation

    International Nuclear Information System (INIS)

    Haroonabadi, H.; Haghifam, M.R.

    2007-01-01

    This paper addressed issues regarding power generation reliability assessment (HLI) in deregulated power pool markets. Most HLI reliability evaluation methods are based on the loss of load (LOLE) approach which is among the most suitable indices to describe the level of generation reliability. LOLE refers to the time in which load is greater than the amount of available generation. While most reliability assessments deal only with power system constraints, this study considered HLI reliability assessment in an oligopoly power market using Monte Carlo simulation (MCS). It evaluated the sensitivity of the reliability index to different reserve margins and future margins. The reliability index was determined by intersecting the offer and demand curves of power plants and comparing them to other parameters. The paper described the fundamentals of an oligopoly power pool market and proposed an algorithm for HLI reliability assessment for such a market. The proposed method was assessed on the IEEE-Reliability Test System with satisfactory results. In all cases, generation reliability indices were evaluated with different reserve margins and various load levels. 19 refs., 7 figs., 1 appendix

  9. Developing an instrument for assessing students' concepts of the nature of technology

    Science.gov (United States)

    Liou, Pey-Yan

    2015-05-01

    Background:The nature of technology has been rarely discussed despite the fact that technology plays an essential role in modern society. It is important to discuss students' concepts of the nature of technology, and further to advance their technological literacy and adaptation to modern society. There is a need to assess high school students' concepts of the nature of technology. Purpose:This study aims to engage in discourse on students' concepts of the nature of technology based on a proposed theoretical framework. Moreover, another goal is to develop an instrument for measuring students' concepts of the nature of technology. Sample:Four hundred and fifty-five high school students' perceptions of technology were qualitatively analyzed. Furthermore, 530 students' responses to a newly developed questionnaire were quantitatively analyzed in the final test. Design and method:First, content analysis was utilized to discuss and categorize students' statements regarding technology and its related issues. The Student Concepts of the Nature of Technology Questionnaire was developed based on the proposed theoretical framework and was supported by the students' qualitative data. Finally, exploratory factor analysis and reliability analysis were applied to determine the structure of the items and the internal consistency of each scale. Results:Through a process of instrument development, the Student Concepts of the Nature of Technology Questionnaire was shown to be a valid and reliable tool for measuring students' concepts of the nature of technology. This newly developed questionnaire is composed of 29 items in six scales, namely 'technology as artifacts,' 'technology as an innovation change,' 'the current role of technology in society,' 'technology as a double-edged sword,' 'technology as a science-based form,' and 'history of technology.' Conclusions:The Student Concepts of the Nature of Technology Questionnaire has been confirmed as a reasonably valid and reliable

  10. Safety and reliability assessment

    International Nuclear Information System (INIS)

    1979-01-01

    This report contains the papers delivered at the course on safety and reliability assessment held at the CSIR Conference Centre, Scientia, Pretoria. The following topics were discussed: safety standards; licensing; biological effects of radiation; what is a PWR; safety principles in the design of a nuclear reactor; radio-release analysis; quality assurance; the staffing, organisation and training for a nuclear power plant project; event trees, fault trees and probability; Automatic Protective Systems; sources of failure-rate data; interpretation of failure data; synthesis and reliability; quantification of human error in man-machine systems; dispersion of noxious substances through the atmosphere; criticality aspects of enrichment and recovery plants; and risk and hazard analysis. Extensive examples are given as well as case studies

  11. Analysis and assessment of water treatment plant reliability

    Directory of Open Access Journals (Sweden)

    Szpak Dawid

    2017-03-01

    Full Text Available The subject of the publication is the analysis and assessment of the reliability of the surface water treatment plant (WTP. In the study the one parameter method of reliability assessment was used. Based on the flow sheet derived from the water company the reliability scheme of the analysed WTP was prepared. On the basis of the daily WTP work report the availability index Kg for the individual elements included in the WTP, was determined. Then, based on the developed reliability scheme showing the interrelationships between elements, the availability index Kg for the whole WTP was determined. The obtained value of the availability index Kg was compared with the criteria values.

  12. Development and evaluation of the OHCITIES instrument: assessing alcohol urban environments in the Heart Healthy Hoods project.

    Science.gov (United States)

    Sureda, Xisca; Espelt, Albert; Villalbí, Joan R; Cebrecos, Alba; Baranda, Lucía; Pearce, Jamie; Franco, Manuel

    2017-10-05

    To describe the development and test-retest reliability of OHCITIES, an instrument characterising alcohol urban environment in terms of availability, promotion and signs of consumption. This study involved: (1) developing the conceptual framework for alcohol urban environment by means of literature reviewing and previous alcohol environment research experience; (2) pilot testing and redesigning the instrument; (3) instrument digitalisation; (4) instrument evaluation using test-retest reliability. Data for testing the reliability of the instrument were collected in seven census sections in Madrid in 2016 by two observers. We computed per cent agreement and Cohen's kappa coefficients to estimate inter-rater and test-retest reliability for alcohol outlet environment measures. We calculated interclass coefficients and their 95% CIs to provide a measure of inter-rater reliability for signs of alcohol consumption measures. We collected information on 92 on-premise and 24 off-premise alcohol outlets identified in the studied areas about availability, accessibility and promotion of alcohol. Most per cent-agreement values for alcohol measures in on-premise and off-premise alcohol outlets were greater than 80%, and inter-rater and test-retest reliability values were generally above 0.80. Observers identified 26 streets and 3 public squares with signs of alcohol consumption. Intraclass correlation coefficient between observers for any type of signs of alcohol consumption was 0.50 (95% CI -0.09 to 0.77). Few items promoting alcohol unrelated to alcohol outlets were found on public spaces. The OHCITIES instrument is a reliable instrument to characterise alcohol urban environment. This instrument might be used to understand how alcohol environment associates with alcohol behaviours and its related health outcomes, and can help in the design and evaluation of policies to reduce the harm caused by alcohol. © Article author(s) (or their employer(s) unless otherwise stated in the

  13. Integrating software reliability concepts into risk and reliability modeling of digital instrumentation and control systems used in nuclear power plants

    International Nuclear Information System (INIS)

    Arndt, S. A.

    2006-01-01

    As software-based digital systems are becoming more and more common in all aspects of industrial process control, including the nuclear power industry, it is vital that the current state of the art in quality, reliability, and safety analysis be advanced to support the quantitative review of these systems. Several research groups throughout the world are working on the development and assessment of software-based digital system reliability methods and their applications in the nuclear power, aerospace, transportation, and defense industries. However, these groups are hampered by the fact that software experts and probabilistic safety assessment experts view reliability engineering very differently. This paper discusses the characteristics of a common vocabulary and modeling framework. (authors)

  14. The reasons for betel-quid chewing scale: assessment of factor structure, reliability, and validity.

    Science.gov (United States)

    Little, Melissa A; Pokhrel, Pallav; Murphy, Kelle L; Kawamoto, Crissy T; Suguitan, Gil S; Herzog, Thaddeus A

    2014-06-03

    Despite the fact that betel-quid is one of the most commonly used psychoactive substances worldwide and a major risk-factor for head-and-neck cancer incidence and mortality globally, currently no standardized instrument is available to assess the reasons why individuals chew betel-quid. A measure to assess reasons for chewing betel-quid could help researchers and clinicians develop prevention and treatment strategies. In the current study, we sought to develop and evaluate a self-report instrument for assessing the reasons for chewing betel quid which contributes toward the goal of developing effective interventions to reduce betel quid chewing in vulnerable populations. The current study assessed the factor structure, reliability and convergent validity of the Reasons for Betel-quid Chewing Scale (RBCS), a newly developed 10 item measure adapted from several existing "reasons for smoking" scales. The measure was administered to 351 adult betel-quid chewers in Guam. Confirmatory factor analysis of this measure revealed a three factor structure: reinforcement, social/cultural, and stimulation. Further tests revealed strong support for the internal consistency and convergent validity of this three factor measure. The goal of designing an intervention to reduce betel-quid chewing necessitates an understanding of why chewers chew; the current study makes considerable contributions towards that objective.

  15. Development and Psychometric Assessment of the Healthcare Provider Cultural Competence Instrument

    Directory of Open Access Journals (Sweden)

    Joshua L. Schwarz PhD

    2015-04-01

    Full Text Available This study presents the measurement properties of 5 scales used in the Healthcare Provider Cultural Competence Instrument (HPCCI. The HPCCI measures a health care provider’s cultural competence along 5 primary dimensions: (1 awareness/sensitivity, (2 behaviors, (3 patient-centered communication, (4 practice orientation, and (5 self-assessment. Exploratory factor analysis demonstrated that the 5 scales were distinct, and within each scale items loaded as expected. Reliability statistics indicated a high level of internal consistency within each scale. The results indicate that the HPCCI effectively measures the cultural competence of health care providers and can provide useful professional feedback for practitioners and organizations seeking to increase a practitioner’s cultural competence.

  16. [The CAD-S, an instrument for the assessment of adaptation to divorce - separation].

    Science.gov (United States)

    Yárnoz Yaben, Sagrario; Comino González, Priscila

    2010-02-01

    The CAD-S, an Instrument for the Assessment of Adaptation to Divorce - Separation. This paper presents an instrument for the evaluation of the adaptation to divorce-separation. The CAD-S is a questionnaire created in Spanish, made up of 20 items whose aim is the evaluation of the family's adaptation to divorce-separation, using one of the parents as informant. Data were collected in four different samples of divorced persons and their children from different autonomous comunities from the Spanish state. 223 parents and 160 children from divorced families took part in this study. Four factors emerged, accounting for 52.39 percent of the total variance: psychological and emotional difficulties, conflict with the ex-partner, disposition to co-parentality, and negative outcomes of separation for children. The results suggest that CAD-S appears to be a reliable and valid instrument, with high internal consistency (Cronbach's alpha) and adequate construct validity supported by its relations with measures of satisfaction with life (SWLS), in the case of parents, and conduct problems (CBCL) for children.

  17. New Contemporary Criterion-Referenced Assessment Instruments for Astronomy & Geology: TOAST & EGGS

    Science.gov (United States)

    Guffey, Sarah Katie; Slater, Stephanie J.; Slater, Timothy F.

    2015-08-01

    Considerable effort in the astronomy and Earth sciences education research over the past decade has focused on developing assessment tools in the form of multiple-choice conceptual diagnostics and content knowledge surveys. This has been critically important in advancing discipline-based education research allowing scholar to establish the initial, incoming knowledge state of students as well as to attempt to measure some of the impacts of innovative instructional interventions. Before now, few of the existing instruments were constructed upon a solid list of clearly articulated and widely agreed upon learning objectives. Whereas first-generation assessment tools, such as the Astronomy Diagnostics Test ADT2) were based primarily upon further identifying documented astronomy misconceptions, scholars from the CAPER Center for Astronomy & Physics Education Research team are creating contemporary instruments based instead by developing items using modern test construction techniques and tightly aligned to the consensus learning goals identified by the American Association of the Advancement of Science’s Project 2061 Benchmarks, and the National Research Council’s National Science Education Standards, and the National Research Council’s Frameworks for A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. These consensus learning goals are further enhanced guiding documents from the American Astronomical Society - Chair’s Conference on ASTRO 101 and the NSF-funded Earth Science Literacy Initiative. Two of the resulting criterion-referenced assessment tools widely used by researchers are the Test Of Astronomy STandards (TOAST) and the Exam of GeoloGy StandardS (EGGS). These easy-to-use and easy-to-score multiple-choice instruments have a high degree of reliability and validity for instructors and researchers needing information on students’ initial knowledge state at the beginning of a course and can be used, in aggregate, to

  18. EARLY READING ASSESSMENT INSTRUMENTS: ABILITIES AND PROCESSES INVOLVED

    Directory of Open Access Journals (Sweden)

    Ana Cláudia de Souza

    2017-04-01

    Full Text Available This study investigates the following early reading assessment instruments: “Bateria de Recepção e Produção da Linguagem Verbal” (SCLIAR-CABRAL, 2003a and “Teste de Competência de Leitura de Palavras e Pseudopalavras” (SEABRA; CAPOVILLA, 2010. The main research goal is to analyze in each one of these reading assessment instruments some of the multiple cognitive processes and basic low-level abilities involved in reading. In this sense, decoding, word recognition, lexical access, syntactic and textual processing, and comprehension are the cognitive processes taken into account. With regard to the basic reading abilities, accuracy and fluency (rhythm, prosody and speed are considered. The results indicate that each one of the analyzed reading assessment instruments assesses different aspects of the reading processes and abilities, mainly through off-line measures. ScliarCabral’s assessment battery allows the researcher or the teacher to evaluate the following processes: perception of the grapheme opposition in minimal pairs of words and in sentences, difficulties in sentence processing, skills in decoding the graphemic-phonemic relationship, and textual comprehension. In its turn, the reading assessment instrument proposed by Seabra e Capovilla allows one to evaluate student’s reading development level, by classifying the kind of processing as logographic, alphabetic or orthographic.

  19. A framework for assessing Health Economic Evaluation (HEE) quality appraisal instruments.

    Science.gov (United States)

    Langer, Astrid

    2012-08-16

    Health economic evaluations support the health care decision-making process by providing information on costs and consequences of health interventions. The quality of such studies is assessed by health economic evaluation (HEE) quality appraisal instruments. At present, there is no instrument for measuring and improving the quality of such HEE quality appraisal instruments. Therefore, the objectives of this study are to establish a framework for assessing the quality of HEE quality appraisal instruments to support and improve their quality, and to apply this framework to those HEE quality appraisal instruments which have been subject to more scrutiny than others, in order to test the framework and to demonstrate the shortcomings of existing HEE quality appraisal instruments. To develop the quality assessment framework for HEE quality appraisal instruments, the experiences of using appraisal tools for clinical guidelines are used. Based on a deductive iterative process, clinical guideline appraisal instruments identified through literature search are reviewed, consolidated, and adapted to produce the final quality assessment framework for HEE quality appraisal instruments. The final quality assessment framework for HEE quality appraisal instruments consists of 36 items organized within 7 dimensions, each of which captures a specific domain of quality. Applying the quality assessment framework to four existing HEE quality appraisal instruments, it is found that these four quality appraisal instruments are of variable quality. The framework described in this study should be regarded as a starting point for appraising the quality of HEE quality appraisal instruments. This framework can be used by HEE quality appraisal instrument producers to support and improve the quality and acceptance of existing and future HEE quality appraisal instruments. By applying this framework, users of HEE quality appraisal instruments can become aware of methodological deficiencies

  20. Reliability assessment of Wind turbines

    DEFF Research Database (Denmark)

    Sørensen, John Dalsgaard

    2015-01-01

    Wind turbines can be considered as structures that are in between civil engineering structures and machines since they consist of structural components and many electrical and machine components together with a control system. Further, a wind turbine is not a one-of-a-kind structure...... but manufactured in series production based on many component tests, some prototype tests and zeroseries wind turbines. These characteristics influence the reliability assessment where focus in this paper is on the structural components. Levelized Cost Of Energy is very important for wind energy, especially when...... comparing to other energy sources. Therefore much focus is on cost reductions and improved reliability both for offshore and onshore wind turbines. The wind turbine components should be designed to have sufficient reliability level with respect to both extreme and fatigue loads but also not be too costly...

  1. The Teamwork Assessment Scale: A Novel Instrument to Assess Quality of Undergraduate Medical Students' Teamwork Using the Example of Simulation-based Ward-Rounds.

    Science.gov (United States)

    Kiesewetter, Jan; Fischer, Martin R

    2015-01-01

    Simulation-based teamwork trainings are considered a powerful training method to advance teamwork, which becomes more relevant in medical education. The measurement of teamwork is of high importance and several instruments have been developed for various medical domains to meet this need. To our knowledge, no theoretically-based and easy-to-use measurement instrument has been published nor developed specifically for simulation-based teamwork trainings of medical students. Internist ward-rounds function as an important example of teamwork in medicine. The purpose of this study was to provide a validated, theoretically-based instrument that is easy-to-use. Furthermore, this study aimed to identify if and when rater scores relate to performance. Based on a theoretical framework for teamwork behaviour, items regarding four teamwork components (Team Coordination, Team Cooperation, Information Exchange, Team Adjustment Behaviours) were developed. In study one, three ward-round scenarios, simulated by 69 students, were videotaped and rated independently by four trained raters. The instrument was tested for the embedded psychometric properties and factorial structure. In study two, the instrument was tested for construct validity with an external criterion with a second set of 100 students and four raters. In study one, the factorial structure matched the theoretical components but was unable to separate Information Exchange and Team Cooperation. The preliminary version showed adequate psychometric properties (Cronbach's α=.75). In study two, the instrument showed physician rater scores were more reliable in measurement than those of student raters. Furthermore, a close correlation between the scale and clinical performance as an external criteria was shown (r=.64) and the sufficient psychometric properties were replicated (Cronbach's α=.78). The validation allows for use of the simulated teamwork assessment scale in undergraduate medical ward-round trainings to reliably

  2. The Teamwork Assessment Scale: A Novel Instrument to Assess Quality of Undergraduate Medical Students' Teamwork Using the Example of Simulation-based Ward-Rounds

    Directory of Open Access Journals (Sweden)

    Kiesewetter, Jan

    2015-05-01

    Full Text Available Background: Simulation-based teamwork trainings are considered a powerful training method to advance teamwork, which becomes more relevant in medical education. The measurement of teamwork is of high importance and several instruments have been developed for various medical domains to meet this need. To our knowledge, no theoretically-based and easy-to-use measurement instrument has been published nor developed specifically for simulation-based teamwork trainings of medical students. Internist ward-rounds function as an important example of teamwork in medicine.Purposes: The purpose of this study was to provide a validated, theoretically-based instrument that is easy-to-use. Furthermore, this study aimed to identify if and when rater scores relate to performance.Methods: Based on a theoretical framework for teamwork behaviour, items regarding four teamwork components ( were developed. In study one, three ward-round scenarios, simulated by 69 students, were videotaped and rated independently by four trained raters. The instrument was tested for the embedded psychometric properties and factorial structure. In study two, the instrument was tested for construct validity with an external criterion with a second set of 100 students and four raters. Results: In study one, the factorial structure matched the theoretical components but was unable to separate Information Exchange and Team Cooperation. The preliminary version showed adequate psychometric properties (Cronbach’s α=.75. In study two, the instrument showed physician rater scores were more reliable in measurement than those of student raters. Furthermore, a close correlation between the scale and clinical performance as an external criteria was shown (r=.64 and the sufficient psychometric properties were replicated (Cronbach’s α=.78.Conclusions: The validation allows for use of the simulated teamwork assessment scale in undergraduate medical ward-round trainings to reliably measure

  3. Reevaluating Interrater Reliability in Offender Risk Assessment

    NARCIS (Netherlands)

    van der Knaap, L.M.; Leenarts, L.E.W.; Born, M.P.; Oosterveld, P.

    2012-01-01

    Offender risk and needs assessment, one of the pillars of the risk-need-responsivity model of offender rehabilitation, usually depends on raters assessing offender risk and needs. The few available studies of interrater reliability in offender risk assessment are, however, limited in the

  4. Validity, reliability, and feasibility of clinical staging scales in dementia: a systematic review

    DEFF Research Database (Denmark)

    Rikkert, Marcel G M Olde; Tona, Klodiana Daphne; Janssen, Lieneke

    2011-01-01

    New staging systems of dementia require adaptation of disease management programs and adequate staging instruments. Therefore, we systematically reviewed the literature on validity and reliability of clinically applicable, multidomain, and dementia staging instruments. A total of 23 articles...... describing 12 staging instruments were identified (N = 6109 participants, age 65-87). Reliability was studied in most (91%) of the articles and was judged moderate to good. Approximately 78% of the articles evaluated concurrent validity, which was good to very good, while discriminant validity was assessed...... in only 25%. The scales can be applied in ±15 minutes. Clinical Dementia Rating (CDR), Global Deterioration scale (GDS), and Functional Assessment Staging (FAST) have been monitored on reliability and validity, and the CDR currently is the best-evidenced scale, also studied in international perspective...

  5. School Climate: Historical Review, Instrument Development, and School Assessment

    Science.gov (United States)

    Zullig, Keith J.; Koopman, Tommy M.; Patton, Jon M.; Ubbes, Valerie A.

    2010-01-01

    This study's purpose is to examine the existing school climate literature in an attempt to constitute its definition from a historical context and to create a valid and reliable student-reported school climate instrument. Five historically common school climate domains and five measurement tools were identified, combined, and previewed by the…

  6. The Pediatrics Milestones Assessment Pilot: Development of Workplace-Based Assessment Content, Instruments, and Processes.

    Science.gov (United States)

    Hicks, Patricia J; Margolis, Melissa; Poynter, Sue E; Chaffinch, Christa; Tenney-Soeiro, Rebecca; Turner, Teri L; Waggoner-Fountain, Linda; Lockridge, Robin; Clyman, Stephen G; Schwartz, Alan

    2016-05-01

    To report on the development of content and user feedback regarding the assessment process and utility of the workplace-based assessment instruments of the Pediatrics Milestones Assessment Pilot (PMAP). One multisource feedback instrument and two structured clinical observation instruments were developed and refined by experts in pediatrics and assessment to provide evidence for nine competencies based on the Pediatrics Milestones (PMs) and chosen to inform residency program faculty decisions about learners' readiness to serve as pediatric interns in the inpatient setting. During the 2012-2013 PMAP study, 18 U.S. pediatric residency programs enrolled interns and subinterns. Faculty, residents, nurses, and other observers used the instruments to assess learner performance through direct observation during a one-month rotation. At the end of the rotation, data were aggregated for each learner, milestone levels were assigned using a milestone classification form, and feedback was provided to learners. Learners and site leads were surveyed and/or interviewed about their experience as participants. Across the sites, 2,338 instruments assessing 239 learners were completed by 630 unique observers. Regarding end-of-rotation feedback, 93% of learners (128/137) agreed the assessments and feedback "helped me understand how those with whom I work perceive my performance," and 85% (117/137) agreed they were "useful for constructing future goals or identifying a developmental path." Site leads identified several benefits and challenges to the assessment process. PM-based instruments used in workplace-based assessment provide a meaningful and acceptable approach to collecting evidence of learner competency development. Learners valued feedback provided by PM-based assessment.

  7. Techniques, processes, and measures for software safety and reliability

    International Nuclear Information System (INIS)

    Sparkman, D.

    1992-01-01

    The purpose of this report is to provide a detailed survey of current recommended practices and measurement techniques for the development of reliable and safe software-based systems. This report is intended to assist the United States Nuclear Reaction Regulation (NRR) in determining the importance and maturity of the available techniques and in assessing the relevance of individual standards for application to instrumentation and control systems in nuclear power generating stations. Lawrence Livermore National Laboratory (LLNL) provides technical support for the Instrumentation and Control System Branch (ICSB) of NRRin advanced instrumentation and control systems, distributed digital systems, software reliability, and the application of verificafion and validafion for the development of software

  8. Validity and reliability of an instrumented leg-extension machine for measuring isometric muscle strength of the knee extensors.

    Science.gov (United States)

    Ruschel, Caroline; Haupenthal, Alessandro; Jacomel, Gabriel Fernandes; Fontana, Heiliane de Brito; Santos, Daniela Pacheco dos; Scoz, Robson Dias; Roesler, Helio

    2015-05-20

    Isometric muscle strength of knee extensors has been assessed for estimating performance, evaluating progress during physical training, and investigating the relationship between isometric and dynamic/functional performance. To assess the validity and reliability of an adapted leg-extension machine for measuring isometric knee extensor force. Validity (concurrent approach) and reliability (test and test-retest approach) study. University laboratory. 70 healthy men and women aged between 20 and 30 y (39 in the validity study and 31 in the reliability study). Intraclass correlation coefficient (ICC) values calculated for the maximum voluntary isometric torque of knee extensors at 30°, 60°, and 90°, measured with the prototype and with an isokinetic dynamometer (ICC2,1, validity study) and measured with the prototype in test and retest sessions, scheduled from 48 h to 72 h apart (ICC1,1, reliability study). In the validity analysis, the prototype showed good agreement for measurements at 30° (ICC2,1 = .75, SEM = 18.2 Nm) and excellent agreement for measurements at 60° (ICC2,1 = .93, SEM = 9.6 Nm) and at 90° (ICC2,1 = .94, SEM = 8.9 Nm). Regarding the reliability analysis, between-days' ICC1,1 were good to excellent, ranging from .88 to .93. Standard error of measurement and minimal detectable difference based on test-retest ranged from 11.7 Nm to 18.1 Nm and 32.5 Nm to 50.1 Nm, respectively, for the 3 analyzed knee angles. The analysis of validity and repeatability of the prototype for measuring isometric muscle strength has shown to be good or excellent, depending on the knee joint angle analyzed. The new instrument, which presents a relative low cost and easiness of transportation when compared with an isokinetic dynamometer, is valid and provides consistent data concerning isometric strength of knee extensors and, for this reason, can be used for practical, clinical, and research purposes.

  9. A quick aphasia battery for efficient, reliable, and multidimensional assessment of language function.

    Science.gov (United States)

    Wilson, Stephen M; Eriksson, Dana K; Schneck, Sarah M; Lucanie, Jillian M

    2018-01-01

    This paper describes a quick aphasia battery (QAB) that aims to provide a reliable and multidimensional assessment of language function in about a quarter of an hour, bridging the gap between comprehensive batteries that are time-consuming to administer, and rapid screening instruments that provide limited detail regarding individual profiles of deficits. The QAB is made up of eight subtests, each comprising sets of items that probe different language domains, vary in difficulty, and are scored with a graded system to maximize the informativeness of each item. From the eight subtests, eight summary measures are derived, which constitute a multidimensional profile of language function, quantifying strengths and weaknesses across core language domains. The QAB was administered to 28 individuals with acute stroke and aphasia, 25 individuals with acute stroke but no aphasia, 16 individuals with chronic post-stroke aphasia, and 14 healthy controls. The patients with chronic post-stroke aphasia were tested 3 times each and scored independently by 2 raters to establish test-retest and inter-rater reliability. The Western Aphasia Battery (WAB) was also administered to these patients to assess concurrent validity. We found that all QAB summary measures were sensitive to aphasic deficits in the two groups with aphasia. All measures showed good or excellent test-retest reliability (overall summary measure: intraclass correlation coefficient (ICC) = 0.98), and excellent inter-rater reliability (overall summary measure: ICC = 0.99). Sensitivity and specificity for diagnosis of aphasia (relative to clinical impression) were 0.91 and 0.95 respectively. All QAB measures were highly correlated with corresponding WAB measures where available. Individual patients showed distinct profiles of spared and impaired function across different language domains. In sum, the QAB efficiently and reliably characterized individual profiles of language deficits.

  10. Assessment of the reliability of ultrasonic inspection methods

    International Nuclear Information System (INIS)

    Haines, N.F.; Langston, D.B.; Green, A.J.; Wilson, R.

    1982-01-01

    The reliability of NDT techniques has remained an open question for many years. A reliable technique may be defined as one that, when rigorously applied by a number of inspection teams, consistently finds then correctly sizes all defects of concern. In this paper we report an assessment of the reliability of defect detection by manual ultrasonic methods applied to the inspection of thick section pressure vessel weldments. Initially we consider the available data relating to the inherent physical capabilities of ultrasonic techniques to detect cracks in weldment and then, independently, we assess the likely variability in team to team performance when several teams are asked to follow the same specified test procedure. The two aspects of 'capability' and 'variability' are brought together to provide quantitative estimates of the overall reliability of ultrasonic inspection of thick section pressure vessel weldments based on currently existing data. The final section of the paper considers current research programmes on reliability and presents a view on how these will help to further improve NDT reliability. (author)

  11. Development of an instrument to assess the health related quality of life of kidney stone formers.

    Science.gov (United States)

    Penniston, Kristina L; Nakada, Stephen Y

    2013-03-01

    Urolithiasis is associated with pain and other decreases in health related quality of life, yet there is no urolithiasis specific instrument to measure quality of life. Quality of life is an important end point in the management of urolithiasis. Therefore, we developed the Wisconsin StoneQOL, a disease specific instrument to assess the quality of life of patients with urolithiasis. Patients and urology providers identified important concepts related to quality of life of stone formers in groups and in individual cognitive interviews. Patients were recurrent stone formers including those with and those without current stones. A preliminary instrument was created, followed by patient feedback and item reduction. A 28-question instrument was ultimately developed which was tested for reliability as well as internal face, construct and discriminant validity in 248 stone formers. The internal consistency (for questions within domains) was high (mean Cronbach's α = 0.81). Correlation between domains was confirmed (Cronbach's α = 0.86). Discriminant validity was shown as stone formers with current stones scored lower than those who were stone-free. Among patients with active stones, those with symptoms scored lower on most questions and for the total score (p instrument that captures the unique symptoms and challenges associated with urolithiasis. As such, the Wisconsin StoneQOL is capable of assessing the health related quality of life of stone formers at various points along the disease continuum. Future assessment will establish minimal clinically important differences for use in individual patients. Copyright © 2013 American Urological Association Education and Research, Inc. Published by Elsevier Inc. All rights reserved.

  12. Reliability and validity of the transport and physical activity questionnaire (TPAQ) for assessing physical activity behaviour.

    Science.gov (United States)

    Adams, Emma J; Goad, Mary; Sahlqvist, Shannon; Bull, Fiona C; Cooper, Ashley R; Ogilvie, David

    2014-01-01

    No current validated survey instrument allows a comprehensive assessment of both physical activity and travel behaviours for use in interdisciplinary research on walking and cycling. This study reports on the test-retest reliability and validity of physical activity measures in the transport and physical activity questionnaire (TPAQ). The TPAQ assesses time spent in different domains of physical activity and using different modes of transport for five journey purposes. Test-retest reliability of eight physical activity summary variables was assessed using intra-class correlation coefficients (ICC) and Kappa scores for continuous and categorical variables respectively. In a separate study, the validity of three survey-reported physical activity summary variables was assessed by computing Spearman correlation coefficients using accelerometer-derived reference measures. The Bland-Altman technique was used to determine the absolute validity of survey-reported time spent in moderate-to-vigorous physical activity (MVPA). In the reliability study, ICC for time spent in different domains of physical activity ranged from fair to substantial for walking for transport (ICC = 0.59), cycling for transport (ICC = 0.61), walking for recreation (ICC = 0.48), cycling for recreation (ICC = 0.35), moderate leisure-time physical activity (ICC = 0.47), vigorous leisure-time physical activity (ICC = 0.63), and total physical activity (ICC = 0.56). The proportion of participants estimated to meet physical activity guidelines showed acceptable reliability (k = 0.60). In the validity study, comparison of survey-reported and accelerometer-derived time spent in physical activity showed strong agreement for vigorous physical activity (r = 0.72, ptravel behaviours and may be suitable for wider use. Its physical activity summary measures have comparable reliability and validity to those of similar existing questionnaires.

  13. A rating instrument for fear of hospitalisation.

    Science.gov (United States)

    Jankovic, Slobodan M; Antonijevic, Gordana V; Vasic, Ivana R; Zivkovic-Radojevic, Marija N; Mirkovic, Snjezana N; Nikolic, Bosko V; Opancina, Valentina D; Putnik, Srdjan S; Radoicic, Ljiljana R; Raspopovic, Katarina M; Stanojevic, Dragan R; Teofilov, Sladjana D; Tomasevic, Katarina V; Radonjic, Vesela

    2018-04-01

    To develop and validate a reliable instrument that can measure fear of hospitalisation experienced by outpatients. After having a diagnosis established, some patients experience sense of fear, unpleasantness and embarrassment due to the possibility to be admitted to a hospital. Currently, there is no available instrument for measuring fear of hospitalisation. Cross-sectional study for assessing reliability and validity of a questionnaire. The questionnaire with 17 items and answers according to the Likert scale was developed during two brainstorming sessions of the research team. Its reliability, validity and temporal stability were tested on the sample of 330 outpatients. The study was multicentric, involving patients from seven cities and three countries. Fear of hospitalisation scale showed satisfactory reliability, when rated both by the investigators (Cronbach's alpha .799) and by the patients themselves (Cronbach's alpha .760). It is temporally stable, and both divergent and convergent validity tests had good results. Factorial analysis revealed three domains: fear of being injured, trust to medical staff and fear of losing privacy or autonomy. This study developed new reliable and valid instrument for measuring fear of hospitalisation. Identification of patients with high level of fear of hospitalisation by this instrument should help clinicians to administer measures which may decrease fear and prevent avoidance of healthcare utilisation. © 2018 John Wiley & Sons Ltd.

  14. Measurement properties of instruments assessing permanent functional impairment of the spine: a systematic review protocol

    Science.gov (United States)

    Trask, Catherine M; Boden, Catherine; Bath, Brenna; Hendrick, Paul; Clay, Lynne; Zeng, Xiaoke; Milosavljevic, Stephan

    2018-01-01

    Introduction Permanent functional impairment (PFI) of the spine is a rating system used by compensation authorities, such as workers compensation boards, to establish an appropriate level of financial compensation for persistent loss of function. Determination of PFI of the spine is commonly based on the assessment of spinal movement combined with other measures of physical and functional impairments; however, the reliability and validity of the measurement instruments used for these evaluations have yet to be established. The aim of this study is to systematically review and synthesise the literature concerning measurement properties of the various and different instruments used for assessing PFI of the spine. Methods Three conceptual groups of terms (1) PFI, (2) spinal disorder and (3) measurement properties will be combined to search Medline, EMBASE, CINAHL, Web of Science, Scopus, PEDro, OTSeeker and Health and Safety Science Abstracts. We will examine peer-reviewed, full-text articles over the full available date range. Two reviewers will independently screen citations (title, abstract and full text) and perform data extraction. Included studies will be appraised as to their methodological quality using the COnsensus-based Standards for the selection of health Measurement INstruments criteria. Findings will be summarised and presented descriptively, with meta-analysis pursued as appropriate. Ethics and dissemination This review will summarise the current level of evidence of measurement properties of instruments used for assessing PFI of the spine. Findings of this review may be applicable to clinicians, policy-makers, workers’ compensation boards, other insurers and health and safety organisations. The findings will likely provide a foundation and direction for future research priorities for assessing spinal PFI. PROSPERO registration number CRD42017060390. PMID:29374671

  15. Reliability of the Motor Learning Strategy Rating Instrument for Children and Youth with Acquired Brain Injury

    Science.gov (United States)

    Kamath, Trishna; Pfeifer, Megan; Banerjee-Guenette, Priyanka; Hunter, Theresa; Ito, Julia; Salbach, Nancy M.; Wright, Virginia; Levac, Danielle

    2012-01-01

    Purpose: To evaluate reliability and feasibility of the Motor Learning Strategy Rating Instrument (MLSRI) in children with acquired brain injury (ABI). The MLSRI quantifies the extent to which motor learning strategies (MLS) are used within physiotherapy (PT) interventions. Methods: PT sessions conducted by ABI team physiotherapists with a…

  16. Instruments to assess stigmatization in dermatology.

    Science.gov (United States)

    Dimitrov, Dimitre; Szepietowski, Jacek C

    2017-11-03

    Stigmatization is the assignment of negative perceptions to an individual because of a perceived difference from the population at large. Skin conditions are frequently the reason of social rejection with a consequent negative influence on the personal and social life of patients. The aim of the current study was to review the available instruments that can be successfully utilized to measure the stigmatization level among dermatological patients. We performed our search on PubMed up to November 2016 and utilized combinations of key phrases containing such words as stigmatization, skin, dermatology, names of various skin conditions (psoriasis, vitiligo, acne, etc.), measurement. The search found a considerable number of articles - 548. After filtering them through a precise selection process, 58 articles remained. We concentrated only on the methodological aspects to assess stigmatization in various dermatoses. The review ascertained that there exist numerous instruments in the form of questionnaires. They were utilized in various researches in order to assess the stigmatization level in patients with skin problems. We divided them into two main groups: dermatology specific instruments (6 questionnaires) and dermatosis/disease specific ones (8 questionnaires). It is recommended to use dermatology-specific instruments to compare the stigmatization level in various skin conditions. They can be utilized as well as a first line tools to study the feeling of stigmatization in specific skin diseases; however, where it is possible, they should be supplemented with the disease-specific instrument for deeper analysis of both qualities of life and stigmatization.

  17. Quantitative reliability assessment for safety critical system software

    International Nuclear Information System (INIS)

    Chung, Dae Won; Kwon, Soon Man

    2005-01-01

    An essential issue in the replacement of the old analogue I and C to computer-based digital systems in nuclear power plants is the quantitative software reliability assessment. Software reliability models have been successfully applied to many industrial applications, but have the unfortunate drawback of requiring data from which one can formulate a model. Software which is developed for safety critical applications is frequently unable to produce such data for at least two reasons. First, the software is frequently one-of-a-kind, and second, it rarely fails. Safety critical software is normally expected to pass every unit test producing precious little failure data. The basic premise of the rare events approach is that well-tested software does not fail under normal routine and input signals, which means that failures must be triggered by unusual input data and computer states. The failure data found under the reasonable testing cases and testing time for these conditions should be considered for the quantitative reliability assessment. We will present the quantitative reliability assessment methodology of safety critical software for rare failure cases in this paper

  18. Development of a simple 12-item theory-based instrument to assess the impact of continuing professional development on clinical behavioral intentions.

    Directory of Open Access Journals (Sweden)

    France Légaré

    Full Text Available Decision-makers in organizations providing continuing professional development (CPD have identified the need for routine assessment of its impact on practice. We sought to develop a theory-based instrument for evaluating the impact of CPD activities on health professionals' clinical behavioral intentions.Our multipronged study had four phases. 1 We systematically reviewed the literature for instruments that used socio-cognitive theories to assess healthcare professionals' clinically-oriented behavioral intentions and/or behaviors; we extracted items relating to the theoretical constructs of an integrated model of healthcare professionals' behaviors and removed duplicates. 2 A committee of researchers and CPD decision-makers selected a pool of items relevant to CPD. 3 An international group of experts (n = 70 reached consensus on the most relevant items using electronic Delphi surveys. 4 We created a preliminary instrument with the items found most relevant and assessed its factorial validity, internal consistency and reliability (weighted kappa over a two-week period among 138 physicians attending a CPD activity. Out of 72 potentially relevant instruments, 47 were analyzed. Of the 1218 items extracted from these, 16% were discarded as improperly phrased and 70% discarded as duplicates. Mapping the remaining items onto the constructs of the integrated model of healthcare professionals' behaviors yielded a minimum of 18 and a maximum of 275 items per construct. The partnership committee retained 61 items covering all seven constructs. Two iterations of the Delphi process produced consensus on a provisional 40-item questionnaire. Exploratory factorial analysis following test-retest resulted in a 12-item questionnaire. Cronbach's coefficients for the constructs varied from 0.77 to 0.85.A 12-item theory-based instrument for assessing the impact of CPD activities on health professionals' clinical behavioral intentions showed adequate validity and

  19. The significance of motivation in periodontal treatment: validity and reliability of the motivation assessment scale among patients undergoing periodontal treatment.

    Science.gov (United States)

    Pac, A; Oruba, Z; Olszewska-Czyż, I; Chomyszyn-Gajewska, M

    2014-03-01

    The individual evaluation of patients' motivation should be introduced to the protocol of periodontal treatment, as it could impact positively on effective treatment planning and treatment outcomes. However, a standardised tool measuring the extent of periodontal patients' motivation has not yet been proposed in the literature. Thus, the objective of the present study was to determine the validity and reliability of the Zychlińscy motivation scale adjusted to the needs of periodontology. Cross sectional study. Department of Periodontology and Oral Medicine, Dental University Clinic, Jagiellonian University, Krakow, Poland. 199 adult periodontal patients, aged 20-78. 14-item questionnaire. The items were adopted from the original Zychlińscy motivation assessment scale. Validity and reliability of the proposed motivation assessment instrument. The assessed Cronbach's alpha of 0.79 indicates the scale is a reliable tool. Principal component analysis revealed a model with three factors, which explained half of the total variance. Those factors represented: the patient's attitude towards treatment and oral hygiene practice; previous experiences during treatment; and the influence of external conditions on the patient's attitude towards treatment. The proposed scale proved to be a reliable and accurate tool for the evaluation of periodontal patients' motivation.

  20. [Development and validation of an instrument for initial nursing assessment].

    Science.gov (United States)

    Fernández-Sola, Cayetano; Granero-Molina, José; Mollinedo-Mallea, Judith; de Gonzales, María Hilda Peredo; Aguilera-Manrique, Gabriel; Ponce, Mara Luna

    2012-12-01

    The objective of this study, conducted in Bolivia from April to July of 2008, is the design and validation of an initial nursing assessment instrument to be used in clinical and educational environments in Santa Cruz (Bolivia). Twelve Bolivian nurses participated; both document analysis as well as consensus techniques were used to determine the categories and criteria to be assessed. Categories included in the nursing assessment instrument are a physical assessment and the eleven Gordon's Functional Health Patterns. The nursing assessment instrument stands out as being concise, easy to complete and utilizing a nursing approach. It does not include items for advanced nursing assessment. However, it incorporates items regarding lifestyle and the patient's autonomy. The nursing assessment instrument contributes to improving the quality of clinical records, supports the nursing diagnosis and implementation of the nursing process, promotes the nurse's role and helps to standardize practice.

  1. Reliability of histologic assessment in patients with eosinophilic oesophagitis.

    Science.gov (United States)

    Warners, M J; Ambarus, C A; Bredenoord, A J; Verheij, J; Lauwers, G Y; Walsh, J C; Katzka, D A; Nelson, S; van Viegen, T; Furuta, G T; Gupta, S K; Stitt, L; Zou, G; Parker, C E; Shackelton, L M; D Haens, G R; Sandborn, W J; Dellon, E S; Feagan, B G; Collins, M H; Jairath, V; Pai, R K

    2018-04-01

    The validity of the eosinophilic oesophagitis (EoE) histologic scoring system (EoEHSS) has been demonstrated, but only preliminary reliability data exist. Formally assess the reliability of the EoEHSS and additional histologic features. Four expert gastrointestinal pathologists independently reviewed slides from adult patients with EoE (N = 45) twice, in random order, using standardised training materials and scoring conventions for the EoEHSS and additional histologic features agreed upon during a modified Delphi process. Intra- and inter-rater reliability for scoring the EoEHSS, a visual analogue scale (VAS) of overall histopathologic disease severity, and additional histologic features were assessed using intra-class correlation coefficients (ICCs). Almost perfect intra-rater reliability was observed for the composite EoEHSS scores and the VAS. Inter-rater reliability was also almost perfect for the composite EoEHSS scores and substantial for the VAS. Of the EoEHSS items, eosinophilic inflammation was associated with the highest ICC estimates and consistent with almost perfect intra- and inter-rater reliability. With the exception of dyskeratotic epithelial cells and surface epithelial alteration, ICC estimates for the remaining EoEHSS items were above the benchmarks for substantial intra-rater, and moderate inter-rater reliability. Estimation of peak eosinophil count and number of lamina propria eosinophils were associated with the highest ICC estimates among the exploratory items. The composite EoEHSS and most component items are associated with substantial reliability when assessed by central pathologists. Future studies should assess responsiveness of the score to change after a therapeutic intervention to facilitate its use in clinical trials. © 2018 John Wiley & Sons Ltd.

  2. The reliability of workplace-based assessment in postgraduate medical education and training: a national evaluation in general practice in the United Kingdom.

    Science.gov (United States)

    Murphy, Douglas J; Bruce, David A; Mercer, Stewart W; Eva, Kevin W

    2009-05-01

    To investigate the reliability and feasibility of six potential workplace-based assessment methods in general practice training: criterion audit, multi-source feedback from clinical and non-clinical colleagues, patient feedback (the CARE Measure), referral letters, significant event analysis, and video analysis of consultations. Performance of GP registrars (trainees) was evaluated with each tool to assess the reliabilities of the tools and feasibility, given raters and number of assessments needed. Participant experience of process determined by questionnaire. 171 GP registrars and their trainers, drawn from nine deaneries (representing all four countries in the UK), participated. The ability of each tool to differentiate between doctors (reliability) was assessed using generalisability theory. Decision studies were then conducted to determine the number of observations required to achieve an acceptably high reliability for "high-stakes assessment" using each instrument. Finally, descriptive statistics were used to summarise participants' ratings of their experience using these tools. Multi-source feedback from colleagues and patient feedback on consultations emerged as the two methods most likely to offer a reliable and feasible opinion of workplace performance. Reliability co-efficients of 0.8 were attainable with 41 CARE Measure patient questionnaires and six clinical and/or five non-clinical colleagues per doctor when assessed on two occasions. For the other four methods tested, 10 or more assessors were required per doctor in order to achieve a reliable assessment, making the feasibility of their use in high-stakes assessment extremely low. Participant feedback did not raise any major concerns regarding the acceptability, feasibility, or educational impact of the tools. The combination of patient and colleague views of doctors' performance, coupled with reliable competence measures, may offer a suitable evidence-base on which to monitor progress and

  3. Inter-rater reliability of measures to characterize the tobacco retail environment in Mexico

    Directory of Open Access Journals (Sweden)

    Marissa G Hall

    2015-11-01

    Full Text Available Objective. To evaluate the inter-rater reliability of a data collection instrument to assess the tobacco retail environ- ment in Mexico, after major marketing regulations were implemented. Materials and methods. In 2013, two data collectors independently evaluated 21 stores in two census tracts, through a data collection instrument that assessed the presence of price promotions, whether single cigarettes were sold, the number of visible advertisements, the pre- sence of signage prohibiting the sale of cigarettes to minors, and characteristics of cigarette pack displays. We evaluated the inter-rater reliability of the collected data, through the calculation of metrics such as intraclass correlation coefficient, percent agreement, Cohen’s kappa and Krippendorff’s alpha. Results. Most measures demonstrated substantial or perfect inter-rater reliability. Conclusions. Our results indicate the potential utility of the data collection instrument for future point-of-sale research.

  4. Transcultural adaptation to Spanish of the instrument "Effectiveness of Auditory Rehabilitation" for the assessment of quality of life in patients using hearing aids.

    Science.gov (United States)

    Cardemil, Felipe; Esquivel, Patricia; Aguayo, Lorena; Barría, Tamara; Fuente, Adrian; Carvajal, Rocío; Fromín, Rose; Villalobos, Iván; Yueh, Bevan

    2013-01-01

    It is becoming increasingly important to have reliable and valid questionnaires. This becomes especially important when evaluating hearing loss. the "Effectiveness of Auditory Rehabilitation" (EAR) questionnaire for the Spanish-speaking population. This instrument assesses quality of life and hearing aspects in patients using hearing aids. Cross-sectional validation study. A cultural adaptation through the use of English to Spanish translations and re-translations was carried out. The validity and reliability of the newly adapted instrument were evaluated. A total of 69 individuals (44 older adults and 25 younger adults) were examined. The pure-tone averages (PTA, 500, 1,000 and 2,000 Hz) were 47.3 dB HL and 47.1 dB HL for the left and right ears, respectively. The mean maximum speech discrimination in silence for monosyllables were 83.3% and 82.9% for the left and right ears, respectively. Internal consistency presented Cronbach alpha values of 0.85 and 0.77 for the internal and external dimensions, respectively. The intraclass correlation coefficients were 0.80 for the internal module and 0.85 for the external module. Construct validity reported a correlation coefficient of 0.71 at baseline and 0.76 at 3 months after the initial assessment for the internal module, and 0.62 at baseline and 0.74 at 3 months after the initial assessment for the external module. The size effects were 1.3 and 1.1 for the internal and external modules, respectively. The Spanish version of the EAR questionnaire seems to be a reliable and valid instrument. The evaluation of audiological aspects, as well as aspects relating to aesthetics and comfort are the main strengths of this instrument. Finally, the EAR scale is more sensitive to change than other scales. Copyright © 2013 Elsevier España, S.L. All rights reserved.

  5. Assessment of structural reliability of precast concrete buildings

    Directory of Open Access Journals (Sweden)

    Koyankin Alexandr

    2018-01-01

    Full Text Available Precast housing construction is currently being under rapid development, however, reliability of building structures made from precast reinforced concrete cannot be assessed rationally due to insufficient research data on that subject. In this regard, experimental and numerical studies were conducted to assess structural reliability of precast buildings as described in the given paper. Experimental studies of full-scale and model samples were conducted; numerical studies were held based on finite element models using “Lira” software. The objects under study included fragment of flooring of a building under construction, full-size fragment of flooring, full-scale models of precast cross-beams-to-columns joints and joints between hollow-core floor slabs and precast and cast-in-place cross-beams. Conducted research enabled to perform an objective assessment of structural reliability of precast buildings.

  6. Reevaluating Interrater Reliability in Offender Risk Assessment

    Science.gov (United States)

    van der Knaap, Leontien M.; Leenarts, Laura E. W.; Born, Marise Ph.; Oosterveld, Paul

    2012-01-01

    Offender risk and needs assessment, one of the pillars of the risk-need-responsivity model of offender rehabilitation, usually depends on raters assessing offender risk and needs. The few available studies of interrater reliability in offender risk assessment are, however, limited in the generalizability of their results. The present study…

  7. A new measurement of workload in Web application reliability assessment

    Directory of Open Access Journals (Sweden)

    CUI Xia

    2015-02-01

    Full Text Available Web application has been popular in various fields of social life.It becomes more and more important to study the reliability of Web application.In this paper the definition of Web application failure is firstly brought out,and then the definition of Web application reliability.By analyzing data in the IIS server logs and selecting corresponding usage and information delivery failure data,the paper study the feasibility of Web application reliability assessment from the perspective of Web software system based on IIS server logs.Because the usage for a Web site often has certain regularity,a new measurement of workload in Web application reliability assessment is raised.In this method,the unit is removed by weighted average technique;and the weights are assessed by setting objective function and optimization.Finally an experiment was raised for validation.The experiment result shows the assessment of Web application reliability base on the new workload is better.

  8. Results of a Demonstration Assessment of Passive System Reliability Utilizing the Reliability Method for Passive Systems (RMPS)

    Energy Technology Data Exchange (ETDEWEB)

    Bucknor, Matthew; Grabaskas, David; Brunett, Acacia; Grelle, Austin

    2015-04-26

    Advanced small modular reactor designs include many advantageous design features such as passively driven safety systems that are arguably more reliable and cost effective relative to conventional active systems. Despite their attractiveness, a reliability assessment of passive systems can be difficult using conventional reliability methods due to the nature of passive systems. Simple deviations in boundary conditions can induce functional failures in a passive system, and intermediate or unexpected operating modes can also occur. As part of an ongoing project, Argonne National Laboratory is investigating various methodologies to address passive system reliability. The Reliability Method for Passive Systems (RMPS), a systematic approach for examining reliability, is one technique chosen for this analysis. This methodology is combined with the Risk-Informed Safety Margin Characterization (RISMC) approach to assess the reliability of a passive system and the impact of its associated uncertainties. For this demonstration problem, an integrated plant model of an advanced small modular pool-type sodium fast reactor with a passive reactor cavity cooling system is subjected to a station blackout using RELAP5-3D. This paper discusses important aspects of the reliability assessment, including deployment of the methodology, the uncertainty identification and quantification process, and identification of key risk metrics.

  9. Sequential optimization and reliability assessment method for metal forming processes

    International Nuclear Information System (INIS)

    Sahai, Atul; Schramm, Uwe; Buranathiti, Thaweepat; Chen Wei; Cao Jian; Xia, Cedric Z.

    2004-01-01

    Uncertainty is inevitable in any design process. The uncertainty could be due to the variations in geometry of the part, material properties or due to the lack of knowledge about the phenomena being modeled itself. Deterministic design optimization does not take uncertainty into account and worst case scenario assumptions lead to vastly over conservative design. Probabilistic design, such as reliability-based design and robust design, offers tools for making robust and reliable decisions under the presence of uncertainty in the design process. Probabilistic design optimization often involves double-loop procedure for optimization and iterative probabilistic assessment. This results in high computational demand. The high computational demand can be reduced by replacing computationally intensive simulation models with less costly surrogate models and by employing Sequential Optimization and reliability assessment (SORA) method. The SORA method uses a single-loop strategy with a series of cycles of deterministic optimization and reliability assessment. The deterministic optimization and reliability assessment is decoupled in each cycle. This leads to quick improvement of design from one cycle to other and increase in computational efficiency. This paper demonstrates the effectiveness of Sequential Optimization and Reliability Assessment (SORA) method when applied to designing a sheet metal flanging process. Surrogate models are used as less costly approximations to the computationally expensive Finite Element simulations

  10. Elder abuse telephone screen reliability and validity.

    Science.gov (United States)

    Buri, Hilary M; Daly, Jeanette M; Jogerst, Gerald J

    2009-01-01

    (a) To identify reliable and valid questions that identify elder abuse, (b) to assess the reliability and validity of extant self-reported elder abuse screens in a high-risk elderly population, and (c) to describe difficulties of completing and interpreting screens in a high-need elderly population. All elders referred to research-trained social workers in a community service agency were asked to participate. Of the 70 elders asked, 49 participated, 44 completed the first questionnaire, and 32 completed the duplicate second questionnaire. A research assistant administered the telephone questionnaires. Twenty-nine (42%) persons were judged abused, 12 (17%) had abuse reported, and 4 (6%) had abuse substantiated. The elder abuse screen instruments were not found to be predictive of assessed abuse or as predictors of reported abuse; the measures tended toward being inversely predictive. Two questions regarding harm and taking of belongings were significantly different for the assessed abused group. In this small group of high-need community-dwelling elders, the screens were not effective in discriminating between abused and nonabused groups. Better instruments are needed to assess for elder abuse.

  11. Evaluation of physicians' professional performance: An iterative development and validation study of multisource feedback instruments

    Directory of Open Access Journals (Sweden)

    Overeem Karlijn

    2012-03-01

    Full Text Available Abstract Background There is a global need to assess physicians' professional performance in actual clinical practice. Valid and reliable instruments are necessary to support these efforts. This study focuses on the reliability and validity, the influences of some sociodemographic biasing factors, associations between self and other evaluations, and the number of evaluations needed for reliable assessment of a physician based on the three instruments used for the multisource assessment of physicians' professional performance in the Netherlands. Methods This observational validation study of three instruments underlying multisource feedback (MSF was set in 26 non-academic hospitals in the Netherlands. In total, 146 hospital-based physicians took part in the study. Each physician's professional performance was assessed by peers (physician colleagues, co-workers (including nurses, secretary assistants and other healthcare professionals and patients. Physicians also completed a self-evaluation. Ratings of 864 peers, 894 co-workers and 1960 patients on MSF were available. We used principal components analysis and methods of classical test theory to evaluate the factor structure, reliability and validity of instruments. We used Pearson's correlation coefficient and linear mixed models to address other objectives. Results The peer, co-worker and patient instruments respectively had six factors, three factors and one factor with high internal consistencies (Cronbach's alpha 0.95 - 0.96. It appeared that only 2 percent of variance in the mean ratings could be attributed to biasing factors. Self-ratings were not correlated with peer, co-worker or patient ratings. However, ratings of peers, co-workers and patients were correlated. Five peer evaluations, five co-worker evaluations and 11 patient evaluations are required to achieve reliable results (reliability coefficient ≥ 0.70. Conclusions The study demonstrated that the three MSF instruments produced

  12. An instrument assessing satisfaction with iron chelation therapy: Psychometric testing from an open-label clinical trial.

    Science.gov (United States)

    Rofail, Diana; Viala, Muriel; Gater, Adam; Abetz-Webb, Linda; Baladi, Jean-Francois; Cappellini, Maria Domenica

    2010-08-01

    The Satisfaction with Iron Chelation Therapy (SICT) instrument was developed based on a literature review, in-depth patient and clinician interviews, and cognitive debriefing interviews. An, open-label, single arm, multicenter trial evaluating the efficacy and safety of deferasirox in patients diagnosed with transfusion-dependent iron overload, provided an opportunity to assess the psychometric measurement properties of the instrument. Psychometric analyses were performed using data at baseline from 273 patients with a range of transfusion-dependent iron overload conditions who were participating in a multinational study. Responsiveness was further evaluated for all patients who also had subsequent satisfaction domain scores collected at week 4. Baseline SICT domain scores had acceptable floor and ceiling effects and internal consistency reliability (Cronbach's alpha: 0.75-0.85). Item discriminant and item convergent validity were both excellent although one item in each analysis did not meet the specified criterion. Small to moderate correlations were observed between SICT and Short Form 36 Health Survey (SF-36) domain scores. Patients with the highest levels of serum ferritin at baseline (>3100 ng/mL) were the least satisfied about the Perceived Effectiveness of ICT and vice versa. Satisfaction improved in all patients, although there were no clear differences observed between groups of patients defined according to changes in serum ferritin levels from baseline to week 4 (stable, improved, or worsened). The SICT domains are reliable and valid. Further testing using a more specific criterion (such as assessing patient global ratings of change in satisfaction domains that correspond to the SICT domains) could help to establish with greater confidence the responsiveness of the instrument.

  13. REFERQUAL: a pilot study of a new service quality assessment instrument in the GP exercise referral scheme setting

    Science.gov (United States)

    Cock, Don; Adams, Iain C; Ibbetson, Adrian B; Baugh, Phil

    2006-01-01

    Background The development of an instrument accurately assessing service quality in the GP Exercise Referral Scheme (ERS) industry could potentially inform scheme organisers of the factors that affect adherence rates leading to the implementation of strategic interventions aimed at reducing client drop-out. Methods A modified version of the SERVQUAL instrument was designed for use in the ERS setting and subsequently piloted amongst 27 ERS clients. Results Test re-test correlations were calculated via Pearson's 'r' or Spearman's 'rho', depending on whether the variables were Normally Distributed, to show a significant (mean r = 0.957, SD = 0.02, p < 0.05; mean rho = 0.934, SD = 0.03, p < 0.05) relationship between all items within the questionnaire. In addition, satisfactory internal consistency was demonstrated via Cronbach's 'α'. Furthermore, clients responded favourably towards the usability, wording and applicability of the instrument's items. Conclusion REFERQUAL is considered to represent promise as a suitable tool for future evaluation of service quality within the ERS community. Future research should further assess the validity and reliability of this instrument through the use of a confirmatory factor analysis to scrutinise the proposed dimensional structure. PMID:16725021

  14. Risk Assessment Stability: A Revalidation Study of the Arizona Risk/Needs Assessment Instrument

    Science.gov (United States)

    Schwalbe, Craig S.

    2009-01-01

    The actuarial method is the gold standard for risk assessment in child welfare, juvenile justice, and criminal justice. It produces risk classifications that are highly predictive and that may be robust to sampling error. This article reports a revalidation study of the Arizona Risk/Needs Assessment instrument, an actuarial instrument for juvenile…

  15. Reliability assessment of Port Harcourt 33/11kv Distribution System ...

    African Journals Online (AJOL)

    This makes reliability studies an important task besides all the other analyses required for assessing the system performance. The paper presents an analytical approach in the reliability assessment of the Port Harcourt 33/11kV power distribution system. The assessment was performed with the 2009 power outage data ...

  16. Reliability of sonographic assessment of tendinopathy in tennis elbow.

    Science.gov (United States)

    Poltawski, Leon; Ali, Syed; Jayaram, Vijay; Watson, Tim

    2012-01-01

    To assess the reliability and compute the minimum detectable change using sonographic scales to quantify the extent of pathology and hyperaemia in the common extensor tendon in people with tennis elbow. The lateral elbows of 19 people with tennis elbow were assessed sonographically twice, 1-2 weeks apart. Greyscale and power Doppler images were recorded for subsequent rating of abnormalities. Tendon thickening, hypoechogenicity, fibrillar disruption and calcification were each rated on four-point scales, and scores were summed to provide an overall rating of structural abnormality; hyperaemia was scored on a five point scale. Inter-rater reliability was established using the intraclass correlation coefficient (ICC) to compare scores assigned independently to the same set of images by a radiologist and a physiotherapist with training in musculoskeletal imaging. Test-retest reliability was assessed by comparing scores assigned by the physiotherapist to images recorded at the two sessions. The minimum detectable change (MDC) was calculated from the test-retest reliability data. ICC values for inter-rater reliability ranged from 0.35 (95% CI: 0.05, 0.60) for fibrillar disruption to 0.77 (0.55, 0.88) for overall greyscale score, and 0.89 (0.79, 0.95) for hyperaemia. Test-retest reliability ranged from 0.70 (0.48, 0.84) for tendon thickening to 0.82 (0.66, 0.90) for overall greyscale score and 0.86 (0.73, 0.93) for calcification. The MDC for the greyscale total score was 2.0/12 and for the hyperaemia score was 1.1/5. The sonographic scoring system used in this study may be used reliably to quantify tendon abnormalities and change over time. A relatively inexperienced imager can conduct the assessment and use the rating scales reliably.

  17. Using the Rasch measurement model to design a report writing assessment instrument.

    Science.gov (United States)

    Carlson, Wayne R

    2013-01-01

    This paper describes how the Rasch measurement model was used to develop an assessment instrument designed to measure student ability to write law enforcement incident and investigative reports. The ability to write reports is a requirement of all law enforcement recruits in the state of Michigan and is a part of the state's mandatory basic training curriculum, which is promulgated by the Michigan Commission on Law Enforcement Standards (MCOLES). Recently, MCOLES conducted research to modernize its training and testing in the area of report writing. A structured validation process was used, which included: a) an examination of the job tasks of a patrol officer, b) input from content experts, c) a review of the professional research, and d) the creation of an instrument to measure student competency. The Rasch model addressed several measurement principles that were central to construct validity, which were particularly useful for assessing student performances. Based on the results of the report writing validation project, the state established a legitimate connectivity between the report writing standard and the essential job functions of a patrol officer in Michigan. The project also produced an authentic instrument for measuring minimum levels of report writing competency, which generated results that are valid for inferences of student ability. Ultimately, the state of Michigan must ensure the safety of its citizens by licensing only those patrol officers who possess a minimum level of core competency. Maintaining the validity and reliability of both the training and testing processes can ensure that the system for producing such candidates functions as intended.

  18. Method for assessing reliability of a network considering probabilistic safety assessment

    International Nuclear Information System (INIS)

    Cepin, M.

    2005-01-01

    A method for assessment of reliability of the network is developed, which uses the features of the fault tree analysis. The method is developed in a way that the increase of the network under consideration does not require significant increase of the model. The method is applied to small examples of network consisting of a small number of nodes and a small number of their connections. The results give the network reliability. They identify equipment, which is to be carefully maintained in order that the network reliability is not reduced, and equipment, which is a candidate for redundancy, as this would improve network reliability significantly. (author)

  19. The Maastricht Clinical Teaching Questionnaire (MCTQ) as a valid and reliable instrument for the evaluation of clinical teachers.

    Science.gov (United States)

    Stalmeijer, Renée E; Dolmans, Diana H J M; Wolfhagen, Ineke H A P; Muijtjens, Arno M M; Scherpbier, Albert J J A

    2010-11-01

    Clinical teaching's importance in the medical curriculum has led to increased interest in its evaluation. Instruments for evaluating clinical teaching must be theory based, reliable, and valid. The Maastricht Clinical Teaching Questionnaire (MCTQ), based on the theoretical constructs of cognitive apprenticeship, elicits evaluations of individual clinical teachers' performance at the workplace. The authors investigated its construct validity and reliability, and they used the underlying factors to test a causal model representing effective clinical teaching. Between March 2007 and December 2008, the authors asked students who had completed clerkship rotations in different departments of two teaching hospitals to use the MCTQ to evaluate their clinical teachers. To establish construct validity, the authors performed a confirmatory factor analysis of the evaluation data, and they estimated reliability by calculating the generalizability coefficient and standard error measurement. Finally, to test a model of the factors, they fitted a structural linear model to the data. Confirmatory factor analysis yielded a five-factor model which fit the data well. Generalizability studies indicated that 7 to 10 student ratings can produce reliable ratings of individual teachers. The hypothesized structural linear model underlined the central roles played by modeling and coaching (mediated by articulation). The MCTQ is a valid and reliable evaluation instrument, thereby demonstrating the usefulness of the cognitive apprenticeship concept for clinical teaching during clerkships. Furthermore, a valuable model of clinical teaching emerged, highlighting modeling, coaching, and stimulating students' articulation and exploration as crucial to effective teaching at the clinical workplace.

  20. The Bereaved Parent Needs Assessment: a new instrument to assess the needs of parents whose children died in the pediatric intensive care unit*.

    Science.gov (United States)

    Meert, Kathleen L; Templin, Thomas N; Michelson, Kelly N; Morrison, Wynne E; Hackbarth, Richard; Custer, Joseph R; Schim, Stephanie M; Briller, Sherylyn H; Thurston, Celia S

    2012-11-01

    To evaluate the reliability and validity of the Bereaved Parent Needs Assessment, a new instrument to measure parents' needs and need fulfillment around the time of their child's death in the pediatric intensive care unit. We hypothesized that need fulfillment would be negatively related to complicated grief and positively related to quality of life during bereavement. Cross-sectional survey. Five U.S. children's hospital pediatric intensive care units. Parents (n = 121) bereaved in a pediatric intensive care unit 6 months earlier. Surveys included the 68-item Bereaved Parent Needs Assessment, the Inventory of Complicated Grief, and the abbreviated version of the World Health Organization Quality of Life questionnaire. Each Bereaved Parent Needs Assessment item described a potential need and was rated on two scales: 1) a 5-point rating of importance (1 = not at all important, 5 = very important) and 2) a 5-point rating of fulfillment (1 = not at all met, 5 = completely met). Three composite scales were computed: 1) total importance (percentage of all needs rated ≥4 for importance), 2) total fulfillment (percentage of all needs rated ≥4 for fulfillment), and 3) percent fulfillment (percentage of important needs that were fulfilled). Internal consistency reliability was assessed by Cronbach's α and Spearman-Brown-corrected split-half reliability. Generalized estimating equations were used to test predictions between composite scales and the Inventory of Complicated Grief and World Health Organization Quality of Life questionnaire. Two items had mean importance ratings 4. Reliability of composite scores ranged from 0.92 to 0.94. Total fulfillment was negatively correlated with Inventory of Complicated Grief (r = -.29; p Quality of Life questionnaire (r = .21; p education, and loss of an only child, percent fulfillment remained significantly correlated with Inventory of Complicated Grief but not with World Health Organization Quality of Life questionnaire. The

  1. Reliability assessment of complex electromechanical systems under epistemic uncertainty

    International Nuclear Information System (INIS)

    Mi, Jinhua; Li, Yan-Feng; Yang, Yuan-Jian; Peng, Weiwen; Huang, Hong-Zhong

    2016-01-01

    The appearance of macro-engineering and mega-project have led to the increasing complexity of modern electromechanical systems (EMSs). The complexity of the system structure and failure mechanism makes it more difficult for reliability assessment of these systems. Uncertainty, dynamic and nonlinearity characteristics always exist in engineering systems due to the complexity introduced by the changing environments, lack of data and random interference. This paper presents a comprehensive study on the reliability assessment of complex systems. In view of the dynamic characteristics within the system, it makes use of the advantages of the dynamic fault tree (DFT) for characterizing system behaviors. The lifetime of system units can be expressed as bounded closed intervals by incorporating field failures, test data and design expertize. Then the coefficient of variation (COV) method is employed to estimate the parameters of life distributions. An extended probability-box (P-Box) is proposed to convey the present of epistemic uncertainty induced by the incomplete information about the data. By mapping the DFT into an equivalent Bayesian network (BN), relevant reliability parameters and indexes have been calculated. Furthermore, the Monte Carlo (MC) simulation method is utilized to compute the DFT model with consideration of system replacement policy. The results show that this integrated approach is more flexible and effective for assessing the reliability of complex dynamic systems. - Highlights: • A comprehensive study on the reliability assessment of complex system is presented. • An extended probability-box is proposed to convey the present of epistemic uncertainty. • The dynamic fault tree model is built. • Bayesian network and Monte Carlo simulation methods are used. • The reliability assessment of a complex electromechanical system is performed.

  2. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project.

    Science.gov (United States)

    2003-02-01

    International interest in clinical practice guidelines has never been greater but many published guidelines do not meet the basic quality requirements. There have been renewed calls for validated criteria to assess the quality of guidelines. To develop and validate an international instrument for assessing the quality of the process and reporting of clinical practice guideline development. The instrument was developed through a multi-staged process of item generation, selection and scaling, field testing, and refinement procedures. 100 guidelines selected from 11 participating countries were evaluated independently by 194 appraisers with the instrument. Following refinement the instrument was further field tested on three guidelines per country by a new set of 70 appraisers. The final version of the instrument contained 23 items grouped into six quality domains with a 4 point Likert scale to score each item (scope and purpose, stakeholder involvement, rigour of development, clarity and presentation, applicability, editorial independence). 95% of appraisers found the instrument useful for assessing guidelines. Reliability was acceptable for most domains (Cronbach's alpha 0.64-0.88). Guidelines produced as part of an established guideline programme had significantly higher scores on editorial independence and, after the publication of a national policy, had significantly higher quality scores on rigour of development (pinternationally. The instrument is sensitive to differences in important aspects of guidelines and can be used consistently and easily by a wide range of professionals from different backgrounds. The adoption of common standards should improve the consistency and quality of the reporting of guideline development worldwide and provide a framework to encourage international comparison of clinical practice guidelines.

  3. Measurement properties of instruments assessing permanent functional impairment of the spine: a systematic review protocol.

    Science.gov (United States)

    Goes, Suelen Meira; Trask, Catherine M; Boden, Catherine; Bath, Brenna; Ribeiro, Daniel Cury; Hendrick, Paul; Clay, Lynne; Zeng, Xiaoke; Milosavljevic, Stephan

    2018-01-27

    Permanent functional impairment (PFI) of the spine is a rating system used by compensation authorities, such as workers compensation boards, to establish an appropriate level of financial compensation for persistent loss of function. Determination of PFI of the spine is commonly based on the assessment of spinal movement combined with other measures of physical and functional impairments; however, the reliability and validity of the measurement instruments used for these evaluations have yet to be established. The aim of this study is to systematically review and synthesise the literature concerning measurement properties of the various and different instruments used for assessing PFI of the spine. Three conceptual groups of terms (1) PFI, (2) spinal disorder and (3) measurement properties will be combined to search Medline, EMBASE, CINAHL, Web of Science, Scopus, PEDro, OTSeeker and Health and Safety Science Abstracts. We will examine peer-reviewed, full-text articles over the full available date range. Two reviewers will independently screen citations (title, abstract and full text) and perform data extraction. Included studies will be appraised as to their methodological quality using the COnsensus-based Standards for the selection of health Measurement INstruments criteria. Findings will be summarised and presented descriptively, with meta-analysis pursued as appropriate. This review will summarise the current level of evidence of measurement properties of instruments used for assessing PFI of the spine. Findings of this review may be applicable to clinicians, policy-makers, workers' compensation boards, other insurers and health and safety organisations. The findings will likely provide a foundation and direction for future research priorities for assessing spinal PFI. CRD42017060390. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise

  4. Assessing the reliability, predictive and construct validity of historical, clinical and risk management-20 (HCR-20) in Mexican psychiatric inpatients.

    Science.gov (United States)

    Sada, Andrea; Robles-García, Rebeca; Martínez-López, Nicolás; Hernández-Ramírez, Rafael; Tovilla-Zarate, Carlos-Alfonso; López-Munguía, Fernando; Suárez-Alvarez, Enrique; Ayala, Xochitl; Fresán, Ana

    2016-08-01

    Assessing dangerousness to gauge the likelihood of future violent behaviour has become an integral part of clinical mental health practice in forensic and non-forensic psychiatric settings, one of the most effective instruments for this being the Historical, Clinical and Risk Management-20 (HCR-20). To examine the HCR-20 factor structure in Mexican psychiatric inpatients and to obtain its predictive validity and reliability for use in this population. In total, 225 patients diagnosed with psychotic, affective or personality disorders were included. The HCR-20 was applied at hospital admission and violent behaviours were assessed during psychiatric hospitalization using the Overt Aggression Scale (OAS). Construct validity, predictive validity and internal consistency were determined. Violent behaviour remains more severe in patients classified in the high-risk group during hospitalization. Fifteen items displayed adequate communalities in the original designated domains of the HCR-20 and internal consistency of the instruments was high. The HCR-20 is a suitable instrument for predicting violence risk in Mexican psychiatric inpatients.

  5. Validation of a psychometric instrument to assess motivation in veterinary bachelor students.

    Science.gov (United States)

    Vandeweerd, Jean-Michel; Dugdale, Alexandra; Romainville, Marc

    2014-01-01

    There are indications that motivation correlates with better performance for those studying veterinary medicine. To assess objectively whether motivation profiles influence both veterinary students' attitudes towards educational interventions and their academic success and whether changes in curriculum can affect students' motivation, there is need for an instrument that can provide a valid measurement of the strength of motivation for the study of veterinary medicine. Our objectives were to design and validate a questionnaire that can be used as a psychometric scale to capture the motivation profiles of veterinary students. Question items were obtained from semi-structured interviews with students and from a review of the relevant literature. Each item was scored on a 5-point scale. The preliminary instrument was trialed on a cohort of 450 students. Responses were subjected to reliability and principal component analysis. A 14-item scale was designed, within which two factors explained 53.4% of the variance among the items. The scale had good face, content, and construct validities as well as a good internal consistency (Cronbach's alpha=.88).

  6. Development of an instrument for direct ozone production rate measurements: measurement reliability and current limitations

    Science.gov (United States)

    Sklaveniti, Sofia; Locoge, Nadine; Stevens, Philip S.; Wood, Ezra; Kundu, Shuvashish; Dusanter, Sébastien

    2018-02-01

    Ground-level ozone (O3) is an important pollutant that affects both global climate change and regional air quality, with the latter linked to detrimental effects on both human health and ecosystems. Ozone is not directly emitted in the atmosphere but is formed from chemical reactions involving volatile organic compounds (VOCs), nitrogen oxides (NOx = NO + NO2) and sunlight. The photochemical nature of ozone makes the implementation of reduction strategies challenging and a good understanding of its formation chemistry is fundamental in order to develop efficient strategies of ozone reduction from mitigation measures of primary VOCs and NOx emissions. An instrument for direct measurements of ozone production rates (OPRs) was developed and deployed in the field as part of the IRRONIC (Indiana Radical, Reactivity and Ozone Production Intercomparison) field campaign. The OPR instrument is based on the principle of the previously published MOPS instrument (Measurement of Ozone Production Sensor) but using a different sampling design made of quartz flow tubes and a different Ox (O3 and NO2) conversion-detection scheme composed of an O3-to-NO2 conversion unit and a cavity attenuated phase shift spectroscopy (CAPS) NO2 monitor. Tests performed in the laboratory and in the field, together with model simulations of the radical chemistry occurring inside the flow tubes, were used to assess (i) the reliability of the measurement principle and (ii) potential biases associated with OPR measurements. This publication reports the first field measurements made using this instrument to illustrate its performance. The results showed that a photo-enhanced loss of ozone inside the sampling flow tubes disturbs the measurements. This issue needs to be solved to be able to perform accurate ambient measurements of ozone production rates with the instrument described in this study. However, an attempt was made to investigate the OPR sensitivity to NOx by adding NO inside the instrument

  7. Student Precision and Reliability of the Team Sport Assessment in ...

    African Journals Online (AJOL)

    TSAP) and formative assessment of invasion sport. The specific objectives were to determine the degree of agreement among expert observers, inter-observer reliability (internal consistency), and intra observer reliability (temporal reliability).

  8. Reliability of the ECHOWS Tool for Assessment of Patient Interviewing Skills.

    Science.gov (United States)

    Boissonnault, Jill S; Evans, Kerrie; Tuttle, Neil; Hetzel, Scott J; Boissonnault, William G

    2016-04-01

    History taking is an important component of patient/client management. Assessment of student history-taking competency can be achieved via a standardized tool. The ECHOWS tool has been shown to be valid with modest intrarater reliability in a previous study but did not demonstrate sufficient power to definitively prove its stability. The purposes of this study were: (1) to assess the reliability of the ECHOWS tool for student assessment of patient interviewing skills and (2) to determine whether the tool discerns between novice and experienced skill levels. A reliability and construct validity assessment was conducted. Three faculty members from the United States and Australia scored videotaped histories from standardized patients taken by students and experienced clinicians from each of these countries. The tapes were scored twice, 3 to 6 weeks apart. Reliability was assessed using interclass correlation coefficients (ICCs) and repeated measures. Analysis of variance models assessed the ability of the tool to discern between novice and experienced skill levels. The ECHOWS tool showed excellent intrarater reliability (ICC [3,1]=.74-.89) and good interrater reliability (ICC [2,1]=.55) as a whole. The summary of performance (S) section showed poor interrater reliability (ICC [2,1]=.27). There was no statistical difference in performance on the tool between novice and experienced clinicians. A possible ceiling effect may occur when standardized patients are not coached to provide complex and obtuse responses to interviewer questions. Variation in familiarity with the ECHOWS tool and in use of the online training may have influenced scoring of the S section. The ECHOWS tool demonstrates excellent intrarater reliability and moderate interrater reliability. Sufficient training with the tool prior to student assessment is recommended. The S section must evolve in order to provide a more discerning measure of interviewing skills. © 2016 American Physical Therapy

  9. Assessing an organizational culture instrument based on the Competing Values Framework: Exploratory and confirmatory factor analyses

    Science.gov (United States)

    Helfrich, Christian D; Li, Yu-Fang; Mohr, David C; Meterko, Mark; Sales, Anne E

    2007-01-01

    Background The Competing Values Framework (CVF) has been widely used in health services research to assess organizational culture as a predictor of quality improvement implementation, employee and patient satisfaction, and team functioning, among other outcomes. CVF instruments generally are presented as well-validated with reliable aggregated subscales. However, only one study in the health sector has been conducted for the express purpose of validation, and that study population was limited to hospital managers from a single geographic locale. Methods We used exploratory and confirmatory factor analyses to examine the underlying structure of data from a CVF instrument. We analyzed cross-sectional data from a work environment survey conducted in the Veterans Health Administration (VHA). The study population comprised all staff in non-supervisory positions. The survey included 14 items adapted from a popular CVF instrument, which measures organizational culture according to four subscales: hierarchical, entrepreneurial, team, and rational. Results Data from 71,776 non-supervisory employees (approximate response rate 51%) from 168 VHA facilities were used in this analysis. Internal consistency of the subscales was moderate to strong (α = 0.68 to 0.85). However, the entrepreneurial, team, and rational subscales had higher correlations across subscales than within, indicating poor divergent properties. Exploratory factor analysis revealed two factors, comprising the ten items from the entrepreneurial, team, and rational subscales loading on the first factor, and two items from the hierarchical subscale loading on the second factor, along with one item from the rational subscale that cross-loaded on both factors. Results from confirmatory factor analysis suggested that the two-subscale solution provides a more parsimonious fit to the data as compared to the original four-subscale model. Conclusion This study suggests that there may be problems applying conventional

  10. Rapid assessment of infrastructure of primary health care facilities - a relevant instrument for health care systems management.

    Science.gov (United States)

    Scholz, Stefan; Ngoli, Baltazar; Flessa, Steffen

    2015-05-01

    Health care infrastructure constitutes a major component of the structural quality of a health system. Infrastructural deficiencies of health services are reported in literature and research. A number of instruments exist for the assessment of infrastructure. However, no easy-to-use instruments to assess health facility infrastructure in developing countries are available. Present tools are not applicable for a rapid assessment by health facility staff. Therefore, health information systems lack data on facility infrastructure. A rapid assessment tool for the infrastructure of primary health care facilities was developed by the authors and pilot-tested in Tanzania. The tool measures the quality of all infrastructural components comprehensively and with high standardization. Ratings use a 2-1-0 scheme which is frequently used in Tanzanian health care services. Infrastructural indicators and indices are obtained from the assessment and serve for reporting and tracing of interventions. The tool was pilot-tested in Tanga Region (Tanzania). The pilot test covered seven primary care facilities in the range between dispensary and district hospital. The assessment encompassed the facilities as entities as well as 42 facility buildings and 80 pieces of technical medical equipment. A full assessment of facility infrastructure was undertaken by health care professionals while the rapid assessment was performed by facility staff. Serious infrastructural deficiencies were revealed. The rapid assessment tool proved a reliable instrument of routine data collection by health facility staff. The authors recommend integrating the rapid assessment tool in the health information systems of developing countries. Health authorities in a decentralized health system are thus enabled to detect infrastructural deficiencies and trace the effects of interventions. The tool can lay the data foundation for district facility infrastructure management.

  11. Suprahyoid Muscle Complex: A Reliable Neural Assessment Tool For Dysphagia?

    DEFF Research Database (Denmark)

    Kothari, Mohit; Stubbs, Peter William; Pedersen, Asger Roer

    be a non-invasive reliable neural assessment tool for patients with dysphagia. Objective: To investigate the possibility of using the suprahyoid muscle complex (SMC) using surface electromyography (sEMG) to assess changes to neural pathways by determining the reliability of measurements in healthy...

  12. Assessment and Improving Methods of Reliability Indices in Bakhtar Regional Electricity Company

    Directory of Open Access Journals (Sweden)

    Saeed Shahrezaei

    2013-04-01

    Full Text Available Reliability of a system is the ability of a system to do prospected duties in future and the probability of desirable operation for doing predetermined duties. Power system elements failures data are the main data of reliability assessment in the network. Determining antiseptic parameters is the goal of reliability assessment by using system history data. These parameters help to recognize week points of the system. In other words, the goal of reliability assessment is operation improving and decreasing of the failures and power outages. This paper is developed to assess reliability indices of Bakhtar Regional Electricity Company up to 1393 and the improving methods and their effects on the reliability indices in this network. DIgSILENT Power Factory software is employed for simulation. Simulation results show the positive effect of improving methods in reliability indices of Bakhtar Regional Electricity Company.

  13. Human reliability analysis methods for probabilistic safety assessment

    International Nuclear Information System (INIS)

    Pyy, P.

    2000-11-01

    Human reliability analysis (HRA) of a probabilistic safety assessment (PSA) includes identifying human actions from safety point of view, modelling the most important of them in PSA models, and assessing their probabilities. As manifested by many incidents and studies, human actions may have both positive and negative effect on safety and economy. Human reliability analysis is one of the areas of probabilistic safety assessment (PSA) that has direct applications outside the nuclear industry. The thesis focuses upon developments in human reliability analysis methods and data. The aim is to support PSA by extending the applicability of HRA. The thesis consists of six publications and a summary. The summary includes general considerations and a discussion about human actions in the nuclear power plant (NPP) environment. A condensed discussion about the results of the attached publications is then given, including new development in methods and data. At the end of the summary part, the contribution of the publications to good practice in HRA is presented. In the publications, studies based on the collection of data on maintenance-related failures, simulator runs and expert judgement are presented in order to extend the human reliability analysis database. Furthermore, methodological frameworks are presented to perform a comprehensive HRA, including shutdown conditions, to study reliability of decision making, and to study the effects of wrong human actions. In the last publication, an interdisciplinary approach to analysing human decision making is presented. The publications also include practical applications of the presented methodological frameworks. (orig.)

  14. Design, validation, and reliability of survey to measure female athlete triad knowledge among coaches

    Directory of Open Access Journals (Sweden)

    Jillian E. Frideres

    2015-06-01

    Full Text Available The purpose of this study was to design and to test the validity and reliability of an instrument to evaluate coaches' knowledge about the female athlete triad syndrome and their confidence in this knowledge. The instrument collects information regarding: knowledge of the syndrome, components, prevention and intervention; confidence of the coaches in their answers; and coach's characteristics (gender, degree held, years of experience in coaching females, continuing education participation specific to the syndrome and its components, and sport coached. The process of designing the questionnaire and testing the validity and reliability of it was done in four phases: a design and development of the instrument, b content validity, c instrument reliability, and d concurrent validity. The results show that the instrument is suitable for measuring coaches' female athlete triad knowledge. The instrument can contribute to assessing the coaches' knowledge level in relation to this topic.

  15. Italian version of Nursing Students’ Perception of Instructor Caring (I-NSPIC: assessment of reliability and validity

    Directory of Open Access Journals (Sweden)

    C. Arrigoni

    2017-11-01

    Full Text Available Abstract Background Clinical experience is an essential component of nursing education since it provides students with the opportunity to construct and develop clinical competencies. Instructor caring is a pivotal facilitator at the forefront of clinical education, playing a key and complex educating role in clinical sectors. For these reasons the aims of this study was to assess the validity and reliability of the Italian version of NSPIC (I-NSPIC. Methods A validation multicentre study was conducted in three different Italian universities. A total of 333 nursing students were enrolled in the 2014/2015 academic year. Exploratory factor analysis (EFA with oblique rotation was performed to test the construct validity of I-NSPIC. The Cronbach’s alpha coefficient and test retest via Intraclass Correlation Coefficient (ICC analyses were done to assess the internal consistency and stability of the scale. A Spearman’s correlation with another scale (CLES-T was used to examine the concurrent validities. Results Four factors (control versus flexibility, supportive learning climate, confidence through caring, appreciation of life meaning and respectful sharing were identified in EFA. The Cronbach’s alpha value showed that I-NSPIC was a reliable instrument (α = 0.94 and the ICC coefficient was satisfactory. Conclusion The I-NSPIC is a valid instrument for assessing the perception of instructor caring in Italian nursing students. It may also prove helpful in promoting the caring ability of nursing students and in increasing the caring interactions in the relationship between instructor and nursing students. The knowledge emerged from this study provide important insight in developing effective training strategies in the clinical training of undergraduate nursing students.

  16. Reliability of assessment of adherence to an antimicrobial treatment guideline

    NARCIS (Netherlands)

    Mol, PGM; Gans, ROB; Panday, PVN; Degener, JE; Laseur, M; Haaijer-Ruskamp, FM

    Assessment procedures for adherence to a guideline must be reliable and credible. The aim of this study was to explore the reliability of assessment of adherence, taking account of the professional backgrounds of the observers. A secondary analysis explored the impact of case characteristics on

  17. Interformat reliability of digital psychiatric self-report questionnaires: a systematic review.

    Science.gov (United States)

    Alfonsson, Sven; Maathz, Pernilla; Hursti, Timo

    2014-12-03

    Research on Internet-based interventions typically use digital versions of pen and paper self-report symptom scales. However, adaptation into the digital format could affect the psychometric properties of established self-report scales. Several studies have investigated differences between digital and pen and paper versions of instruments, but no systematic review of the results has yet been done. This review aims to assess the interformat reliability of self-report symptom scales used in digital or online psychotherapy research. Three databases (MEDLINE, Embase, and PsycINFO) were systematically reviewed for studies investigating the reliability between digital and pen and paper versions of psychiatric symptom scales. From a total of 1504 publications, 33 were included in the review, and interformat reliability of 40 different symptom scales was assessed. Significant differences in mean total scores between formats were found in 10 of 62 analyses. These differences were found in just a few studies, which indicates that the results were due to study effects and sample effects rather than unreliable instruments. The interformat reliability ranged from r=.35 to r=.99; however, the majority of instruments showed a strong correlation between format scores. The quality of the included studies varied, and several studies had insufficient power to detect small differences between formats. When digital versions of self-report symptom scales are compared to pen and paper versions, most scales show high interformat reliability. This supports the reliability of results obtained in psychotherapy research on the Internet and the comparability of the results to traditional psychotherapy research. There are, however, some instruments that consistently show low interformat reliability, suggesting that these conclusions cannot be generalized to all questionnaires. Most studies had at least some methodological issues with insufficient statistical power being the most common issue

  18. Safety regulations concerning instrumentation and control systems for research reactors

    International Nuclear Information System (INIS)

    El-Shanshoury, A.I.

    2009-01-01

    A brief study on the safety and reliability issues related to instrumentation and control systems in nuclear reactor plants is performed. In response, technical and strategic issues are used to accomplish instrumentation and control systems safety. For technical issues there are ; systems aspects of digital I and C technology, software quality assurance, common-mode software, failure potential, safety and reliability assessment methods, and human factors and human machine interfaces. The strategic issues are the case-by-case licensing process and the adequacy of the technical infrastructure. The purpose of this work was to review the reliability of the safety systems related to these technical issues for research reactors

  19. Reliability and Probabilistic Risk Assessment - How They Play Together

    Science.gov (United States)

    Safie, Fayssal M.; Stutts, Richard G.; Zhaofeng, Huang

    2015-01-01

    PRA methodology is one of the probabilistic analysis methods that NASA brought from the nuclear industry to assess the risk of LOM, LOV and LOC for launch vehicles. PRA is a system scenario based risk assessment that uses a combination of fault trees, event trees, event sequence diagrams, and probability and statistical data to analyze the risk of a system, a process, or an activity. It is a process designed to answer three basic questions: What can go wrong? How likely is it? What is the severity of the degradation? Since 1986, NASA, along with industry partners, has conducted a number of PRA studies to predict the overall launch vehicles risks. Planning Research Corporation conducted the first of these studies in 1988. In 1995, Science Applications International Corporation (SAIC) conducted a comprehensive PRA study. In July 1996, NASA conducted a two-year study (October 1996 - September 1998) to develop a model that provided the overall Space Shuttle risk and estimates of risk changes due to proposed Space Shuttle upgrades. After the Columbia accident, NASA conducted a PRA on the Shuttle External Tank (ET) foam. This study was the most focused and extensive risk assessment that NASA has conducted in recent years. It used a dynamic, physics-based, integrated system analysis approach to understand the integrated system risk due to ET foam loss in flight. Most recently, a PRA for Ares I launch vehicle has been performed in support of the Constellation program. Reliability, on the other hand, addresses the loss of functions. In a broader sense, reliability engineering is a discipline that involves the application of engineering principles to the design and processing of products, both hardware and software, for meeting product reliability requirements or goals. It is a very broad design-support discipline. It has important interfaces with many other engineering disciplines. Reliability as a figure of merit (i.e. the metric) is the probability that an item will

  20. The control-and-instrumentation system of the IEA zero power reactor and its reliability calculation

    International Nuclear Information System (INIS)

    Peluso, M.A.V.

    1978-01-01

    The control-and instrumentation system for the Instituto de Energia Atomica Zero Power Reactor is described and the design criteria are presented and discussed. The reliability analysis for the reactor protection system was performed using the fault tree method. This was done using a computer code based on the Monte Carlo simulation. That code is an adaptation of the SAFTE-I, for the IBM 360/155 IEA Computer. (Author) [pt

  1. Inter-rater reliability of direct observations of the physical and psychosocial working conditions in eldercare

    DEFF Research Database (Denmark)

    Karstad, Kristina; Rugulies, Reiner; Skotte, Jørgen

    2018-01-01

    The aim of the study was to develop and evaluate the reliability of the "Danish observational study of eldercare work and musculoskeletal disorders" (DOSES) observation instrument to assess physical and psychosocial risk factors for musculoskeletal disorders (MSD) in eldercare work. During 1.5 ye...... is appropriate for assessing physical and psychosocial risk factors for MSD among eldercare workers.......The aim of the study was to develop and evaluate the reliability of the "Danish observational study of eldercare work and musculoskeletal disorders" (DOSES) observation instrument to assess physical and psychosocial risk factors for musculoskeletal disorders (MSD) in eldercare work. During 1...

  2. Statistical reliability assessment of software-based systems

    International Nuclear Information System (INIS)

    Korhonen, J.; Pulkkinen, U.; Haapanen, P.

    1997-01-01

    Plant vendors nowadays propose software-based systems even for the most critical safety functions. The reliability estimation of safety critical software-based systems is difficult since the conventional modeling techniques do not necessarily apply to the analysis of these systems, and the quantification seems to be impossible. Due to lack of operational experience and due to the nature of software faults, the conventional reliability estimation methods can not be applied. New methods are therefore needed for the safety assessment of software-based systems. In the research project Programmable automation systems in nuclear power plants (OHA), financed together by the Finnish Centre for Radiation and Nuclear Safety (STUK), the Ministry of Trade and Industry and the Technical Research Centre of Finland (VTT), various safety assessment methods and tools for software based systems are developed and evaluated. This volume in the OHA-report series deals with the statistical reliability assessment of software based systems on the basis of dynamic test results and qualitative evidence from the system design process. Other reports to be published later on in OHA-report series will handle the diversity requirements in safety critical software-based systems, generation of test data from operational profiles and handling of programmable automation in plant PSA-studies. (orig.) (25 refs.)

  3. Reliable and valid assessment of performance in thoracoscopy

    DEFF Research Database (Denmark)

    Konge, Lars; Lehnert, Per; Hansen, Henrik Jessen

    2012-01-01

    BACKGROUND: As we move toward competency-based education in medicine, we have lagged in developing competency-based evaluation methods. In the era of minimally invasive surgery, there is a need for a reliable and valid tool dedicated to measure competence in video-assisted thoracoscopic surgery....... The purpose of this study is to create such an assessment tool, and to explore its reliability and validity. METHODS: An expert group of physicians created an assessment tool consisting of 10 items rated on a five-point rating scale. The following factors were included: economy and confidence of movement...

  4. Development, initial reliability and validity testing of an observational tool for assessing technical skills of operating room nurses.

    Science.gov (United States)

    Sevdalis, Nick; Undre, Shabnam; Henry, Janet; Sydney, Elaine; Koutantji, Mary; Darzi, Ara; Vincent, Charles A

    2009-09-01

    The recent emergence of the Systems Approach to the safety and quality of surgical care has triggered individual and team skills training modules for surgeons and anaesthetists and relevant observational assessment tools have been developed. To develop an observational tool that captures operating room (OR) nurses' technical skill and can be used for assessment and training. The Imperial College Assessment of Technical Skills for Nurses (ICATS-N) assesses (i) gowning and gloving, (ii) setting up instrumentation, (iii) draping, and (iv) maintaining sterility. Three to five observable behaviours have been identified for each skill and are rated on 1-6 scales. Feasibility and aspects of reliability and validity were assessed in 20 simulation-based crisis management training modules for trainee nurses and doctors, carried out in a Simulated Operating Room. The tool was feasible to use in the context of simulation-based training. Satisfactory reliability (Cronbach alpha) was obtained across trainers' and trainees' scores (analysed jointly and separately). Moreover, trainer nurse's ratings of the four skills correlated positively, thus indicating adequate content validity. Trainer's and trainees' ratings did not correlate. Assessment of OR nurses' technical skill is becoming a training priority. The present evidence suggests that the ICATS-N could be considered for use as an assessment/training tool for junior OR nurses.

  5. Development of performance test instrument in the experiment of law of conservation mass using self and peer assessment’s technique

    Science.gov (United States)

    Siswaningsih, W.; Nahadi; Firmansyah, D. R.

    2018-05-01

    The purpose of this research is to develop the instrument of performance assessment of law of mass conservation using self and peer assessment technique that meet valid and reliable criteria. The instrument components consist of task and rubric. The method used is development and validation.Value of the instrument reliability obtained from twice observations that are at four and six students every group with three same observers. Cronbach alpha value for four and six students every group consecutively are 0.94 and 0.76, indicating that value shows that the instrument is reliable. Optimum amount of the students that can be observed are four students. The implementation of the instrument to limited group of students showed that All of the students give positive responses to the instrument used with the interpretation of questionnaire scores >90% that categorized as good.

  6. Screening for frailty in elderly subjects living at home: validation of the Modified Short Emergency Geriatric Assessment (SEGAm) instrument.

    Science.gov (United States)

    Oubaya, N; Mahmoudi, R; Jolly, D; Zulfiqar, A A; Quignard, E; Cunin, C; Nazeyrollas, P; Novella, J L; Dramé, M

    2014-01-01

    To validate the modified version of the Short Emergency Geriatric Assessment (SEGAm) frailty instrument in elderly people living at home. Longitudinal, prospective, multicentre study. Four departments (Ardennes, Marne, Meurthe-et-Moselle, Meuse) in two French Regions (Champagne-Ardenne and Lorraine). Subjects aged 65 years or more, living at home, who could read and understand French, with a degree of autonomy corresponding to groups 5, or 6 in the AGGIR autonomy evaluation scale. Assessment included demographic characteristics, comprehensive geriatric assessment, and the SEGAm instrument. Psychometric validation was used to study feasibility and acceptability, internal structure validity, reliability, and discriminant validity of the SEGAm instrument. Between July 1st 2012 and March 31st 2013, 167 patients were included in the study. Averaged age was 77±7 years, the majority were women (70.7%). Feasibility and acceptability of the SEGAm instrument were excellent: we observed no refusal to participate, no drop-out during administration, no missing items, no ceiling or floor effects, and the administration time was short (5.0±3.5 min). By factor analysis, the instrument proved to be unidimensional. It showed good internal consistency (Cronbach's alpha coefficient: 0.68) and good test-retest (intra-class correlation: 0.88) at 7 days interval. Discriminant validity showed a significant difference, mainly for nutritional status, fall risk, dependency, mood and depression risk, and comorbidities. Based on these psychometric properties, the SEGAm appears to be an easy-to-use instrument that is particularly suitable for use in the community to identify frail elderly people who could benefit from early targeted interventions.

  7. Reliability and accuracy of four dental shade-matching devices.

    Science.gov (United States)

    Kim-Pusateri, Seungyee; Brewer, Jane D; Davis, Elaine L; Wee, Alvin G

    2009-03-01

    There are several electronic shade-matching instruments available for clinical use, but the reliability and accuracy of these instruments have not been thoroughly investigated. The purpose of this in vitro study was to evaluate the reliability and accuracy of 4 dental shade-matching instruments in a standardized environment. Four shade-matching devices were tested: SpectroShade, ShadeVision, VITA Easyshade, and ShadeScan. Color measurements were made of 3 commercial shade guides (Vitapan Classical, Vitapan 3D-Master, and Chromascop). Shade tabs were placed in the middle of a gingival matrix (Shofu GUMY) with shade tabs of the same nominal shade from additional shade guides placed on both sides. Measurements were made of the central region of the shade tab positioned inside a black box. For the reliability assessment, each shade tab from each of the 3 shade guide types was measured 10 times. For the accuracy assessment, each shade tab from 10 guides of each of the 3 types evaluated was measured once. Differences in reliability and accuracy were evaluated using the Standard Normal z test (2 sided) (alpha=.05) with Bonferroni correction. Reliability of devices was as follows: ShadeVision, 99.0%; SpectroShade, 96.9%; VITA Easyshade, 96.4%; and ShadeScan, 87.4%. A significant difference in reliability was found between ShadeVision and ShadeScan (P=.008). All other comparisons showed similar reliability. Accuracy of devices was as follows: VITA Easyshade, 92.6%; ShadeVision, 84.8%; SpectroShade, 80.2%; and ShadeScan, 66.8%. Significant differences in accuracy were found between all device pairs (Preliability (over 96%), indicating predictable shade values from repeated measurements. However, there was more variability in accuracy among devices (67-93%), and differences in accuracy were seen with most device comparisons.

  8. Cultural validation of a new instrument to measure leprosy-related stigma: the SARI Stigma Scale

    NARCIS (Netherlands)

    Dadun,; Peters, Ruth M. H.; Van Brakel, Wim H.; Lusli, Mimi; Damayanti, Rita; Irwanto, A.; Bunders- Aelen, J.G.F.

    Background: There is a need for comprehensive, valid and reliable instruments to assess leprosy-related stigma. This paper presents the process of the cross-cultural validation of an instrument in Cirebon District, Indonesia initiated by the Stigma Assessment and Reduction of Impact (SARI) project.

  9. Life cycle reliability assessment of new products—A Bayesian model updating approach

    International Nuclear Information System (INIS)

    Peng, Weiwen; Huang, Hong-Zhong; Li, Yanfeng; Zuo, Ming J.; Xie, Min

    2013-01-01

    The rapidly increasing pace and continuously evolving reliability requirements of new products have made life cycle reliability assessment of new products an imperative yet difficult work. While much work has been done to separately estimate reliability of new products in specific stages, a gap exists in carrying out life cycle reliability assessment throughout all life cycle stages. We present a Bayesian model updating approach (BMUA) for life cycle reliability assessment of new products. Novel features of this approach are the development of Bayesian information toolkits by separately including “reliability improvement factor” and “information fusion factor”, which allow the integration of subjective information in a specific life cycle stage and the transition of integrated information between adjacent life cycle stages. They lead to the unique characteristics of the BMUA in which information generated throughout life cycle stages are integrated coherently. To illustrate the approach, an application to the life cycle reliability assessment of a newly developed Gantry Machining Center is shown

  10. The reliability and validity of the short version of the WHO Quality of Life Instrument in an Arab general population

    International Nuclear Information System (INIS)

    Ohaeri, Jude U; Awadallab, Abdel W

    2009-01-01

    There is rising interest in quality of life (QOL) research in Arabian countries. The aim of this study was to assess in a nationwide sample of Kuwaiti subjects the reliability and validity of the World Health Organization Quality of Life (WHOQOL-BREF), a shorter version of the widely used QOL assessment instrument that comprises 26 items in the domains of physical health, psychological health, social relationships, and the environment. A one-in-three systematic random proportionate sample of consenting Kuwaiti nationals attending large cooperative stores and municipal government offices in the six governorates completed the Arabic translation of the questionnaire. The indices assessed included test-retest reliability, internal consistency, item internal consistency (2C), item discriminant validity (IDV), known-groups and construct validity. There were 3303 participants (44.8% males, 55.2% females, mean age 35.4 years, range 16 to 87 years). The intra-class correlation for the test-retest statistic and the internal consistency values for the full questionnaire and the domains had a Cronbach's alpha > - 0.7. Of the 24 items that constitute the domains, 21 met the 2 C requirement of correlation > - 0.4 with the corresponding domain, while 16 met the IDV criterion of having a higher correlation with their corresponding domain than other domains. Domain scores discriminated significantly between well and sick groups. In the factor analysis, four strong factors emerged with the same construct as in the WHO report. The Arabic translation of the WHOQOL-BREF has impressive reliability and validity indices. The poor IDV findings are due to the multidimensional nature of the questionnaire. The highly significant validity indices should reassure researchers that the questionnaire represents the same constructs across cultures. Negatively worded items possibly need refinement. (author)

  11. Assessment of instruments in facilitating investment in off-grid renewable energy projects

    International Nuclear Information System (INIS)

    Shi, Xunpeng; Liu, Xiying; Yao, Lixia

    2016-01-01

    Renewable off-grid solution plays a critical role in supporting rural electrification. However, off-grid Renewable Energy (OGRE) project financing faces significant challenges due to limited financing access, low affordability of consumers, high transactions costs and etc. Various supporting instruments have been implemented to facilitate OGRE investment. This study assesses the effectiveness of those instruments with a framework consists of three dimensions: feasibility, sustainability and replicability. The weights of each dimension in the framework and the scores of each instrument are assessed by expert surveys based on the Delphi method. It is suggested that all the three dimensions should be taken into consideration while assessing the instruments, among which feasibility and sustainability are considered as the most important dimensions in the assessment framework. Furthermore, the top-5 most effective instruments in facilitating OGRE investment are local engagement in operation and maintenance, loan guarantee, start-up grant, end user financing, and concessional finance. Developing countries that need to increase electrification, such as most of the ASEAN member states, could use these top scored instruments despite of their limited amount of public finance. - Highlights: •Assess the effectiveness of instruments for promoting financing for OGRE projects. •A three-dimension assessment framework: feasibility, sustainability, replicability. •Use online surveys and the Delphi method to collect experts’ assessment. •The most effective instruments: local engagement, loan guarantee, and start-up grant.

  12. ASSESSING AND COMBINING RELIABILITY OF PROTEIN INTERACTION SOURCES

    Science.gov (United States)

    LEACH, SONIA; GABOW, AARON; HUNTER, LAWRENCE; GOLDBERG, DEBRA S.

    2008-01-01

    Integrating diverse sources of interaction information to create protein networks requires strategies sensitive to differences in accuracy and coverage of each source. Previous integration approaches calculate reliabilities of protein interaction information sources based on congruity to a designated ‘gold standard.’ In this paper, we provide a comparison of the two most popular existing approaches and propose a novel alternative for assessing reliabilities which does not require a gold standard. We identify a new method for combining the resultant reliabilities and compare it against an existing method. Further, we propose an extrinsic approach to evaluation of reliability estimates, considering their influence on the downstream tasks of inferring protein function and learning regulatory networks from expression data. Results using this evaluation method show 1) our method for reliability estimation is an attractive alternative to those requiring a gold standard and 2) the new method for combining reliabilities is less sensitive to noise in reliability assignments than the similar existing technique. PMID:17990508

  13. Resident Assessment Instrument/Minimum Data Set (RAI/MDS)

    Data.gov (United States)

    Department of Veterans Affairs — The Resident Assessment Instrument/Minimum Data Set (RAI/MDS) is a comprehensive assessment and care planning process used by the nursing home industry since 1990 as...

  14. Interrater reliability of videotaped observational gait-analysis assessments.

    Science.gov (United States)

    Eastlack, M E; Arvidson, J; Snyder-Mackler, L; Danoff, J V; McGarvey, C L

    1991-06-01

    The purpose of this study was to determine the interrater reliability of videotaped observational gait-analysis (VOGA) assessments. Fifty-four licensed physical therapists with varying amounts of clinical experience served as raters. Three patients with rheumatoid arthritis who demonstrated an abnormal gait pattern served as subjects for the videotape. The raters analyzed each patient's most severely involved knee during the four subphases of stance for the kinematic variables of knee flexion and genu valgum. Raters were asked to determine whether these variables were inadequate, normal, or excessive. The temporospatial variables analyzed throughout the entire gait cycle were cadence, step length, stride length, stance time, and step width. Generalized kappa coefficients ranged from .11 to .52. Intraclass correlation coefficients (2,1) and (3,1) were slightly higher. Our results indicate that physical therapists' VOGA assessments are only slightly to moderately reliable and that improved interrater reliability of the assessments of physical therapists utilizing this technique is needed. Our data suggest that there is a need for greater standardization of gait-analysis training.

  15. Reliability of an instrument to determine lower limb comfort in professional football

    Directory of Open Access Journals (Sweden)

    Michael Kinchington

    2010-06-01

    Full Text Available Michael Kinchington1, Kevin Ball1, Geraldine Naughton21School of Human Movement, Recreation and Performance, Victoria University, Melbourne, Australia; 2The Centre of Physical Activity Across the Lifespan (COPAAL, Australian Catholic University, Victoria, AustraliaAims and Objectives: This study extends previous work in the field of injury awareness using a novel lower limb comfort index (LLCI, which was developed to assess comfort in professional football. Participants rated comfort for designated anatomical segments of the lower limb utilizing a seven point Likert scale. The aims of the study were (i to assess the reliability of the LLCI in a competitive football environment (Australian Rules and Rugby League, and (ii to assess whether LLCI measurements were responsive to changes in lower limb comfort over time.Methods and Results: The reliability of the LLCI was observed in two professional football environments: Training Week (mean difference 0.1 point, intra-class correlation coefficient, ICC 0.99 for n = 41 participants; and Match Day (mean difference 0.2 points, ICC 0.97 for n = 22 players. Measurements of lower limb comfort were responsive to changes in comfort over time. Within-player differences were not significant for periods 0–8 hrs (P > 0.05 but, generally, significant for time periods 0–24 hrs (P < 0.05, and significant between 24–96 hrs (P < 0.01. The results indicate that the LLCI was reliable when tested for repeated measures and indicated how the index measures lower limb comfort changes over time.Conclusion: This study shows that the use of a lower limb comfort index, when used in a competitive football environment, is both reliable and responsive to change during both a training week and under match day conditions.Keywords: lower limb comfort, musculoskeletal, football, injury

  16. Using Instruments for Tactical Assessment in Physical Education and Extra-Curricular Sports

    Science.gov (United States)

    Arias-Estero, José; Castejón, Francisco

    2014-01-01

    The aim was to analyse the features concerning the applicational purpose and strategy of the most common instruments used to assess game tactics both in Physical Education (PE) and in extra-curricular sport contexts. The review focused on two instruments: the Game Performance Assessment Instrument (GPAI) and the Team Sport Assessment Procedure…

  17. [Psychometric properties of an instrument for assessing cyber-sex addiction].

    Science.gov (United States)

    Ballester Arnal, Rafael; Gil Llario, Ma Dolores; Gómez Martínez, Sandra; Gil Juliá, Beatriz

    2010-11-01

    Psychometric properties of an instrument for assessing cyber-sex addiction. Cyber-sex addiction is a «new pathology» whose prevalence has grown rapidly in recent years. Therefore, it is important to have validated assessment instruments. The aim of this study was the adaptation and validation of the Internet Sex Screening Test (ISST) for its use in a Spanish population. The instrument was administered to 1239 Spanish college students. The results yielded five components that account for 47.5% of the variance. Internal consistency was .88 and temporal stability was .84. Moreover, the instrument had adequate convergent and discriminant validity and was related to other behaviors such as the use of pornography, internet addiction, number of hours online and sexual frequency. Therefore, this tool is proposed as an appropriate measure to assess cyber-sex addiction.

  18. Ecological Assessment of Substance-abuse Experiences (EASE): findings from a new instrument development pilot study.

    Science.gov (United States)

    Matto, Holly C; Miller, Keith; Spera, Christopher

    2005-08-01

    A newly developed instrument that assesses a client's orientation to addiction or recovery communities using social context referents was pilot tested with a sample of 103 adults seeking treatment for substance abuse at outpatient and residential treatment facilities on the East Coast. Preliminary findings show promising subscale reliabilities, and suggest that drug- and recovery-related social identities are related to drug-use severity and drug-use concern; and drug-related attitudinal congruence between the treatment-seeker and family and treatment-seeker and other significant persons are related to intention to make behavioral changes in reducing substance abuse.

  19. Psychometric Evaluation of a New Instrument to Evaluate School Nurses' Perceptions on Concussion Care Management.

    Science.gov (United States)

    Schneider, Kathleen H

    2017-10-01

    The rate of concussions in adolescents has risen over the last decade, resulting in cognitive and emotional problems. Neurologists recommend cognitive and physical rest during the recovery period, followed by a transitional return-to-classroom protocol. The purpose of the study was to develop and test an instrument that explores the beliefs and roles of school nurses in concussion care management. The instrument was a cross-sectional descriptive survey based on the theory of planned behavior, using Qualtrics®. The psychometric properties of the instrument were assessed through exploratory factor analysis with orthogonal rotation. The reliability of the instrument was assessed for internal consistency reliability using Cronbach's α. Kaiser-Meyer-Olkin for sample adequacy was .8; Cronbach's α strong (.851).

  20. Validity and reliability of a new instrument for the evaluation of dental collaboration in disabled people

    Directory of Open Access Journals (Sweden)

    Scilla Sparabombe

    2013-10-01

    Full Text Available Background: nowadays, oral health in people with disabilities is an important topic. The phsychological and behavioural problems of these people, their difficulties with environmental adaptations and the absence of any traditional communication determine the compliance needed for treatment The aim of this work was to test the validity and reliability of an original questionnaire that could become an instrument assessing the individual features in people with mental retardation and other developmental disabilities at the time of dental treatment.Methods: it was created a questionnaire with standardised answers regarding four specific areas: neuropsychology, emotional-affect, autonomy and environmental resources. The questionnaire was completed by 63 patients from three different institutes (two rehabilitation institutes and an Institute of Dentistry for patients with special needs. To analyse the answers, each item was transformed into a numeric value. A value of 1 was displayed as the minimum while 4 represented full possession of the considered skills. A total of 17 variables were analysed with descriptive statistics and multivariate analysis. Internal consistency reliability was measured using Cronbach’s alpha. Furthermore, an analysis on convergent/discriminant validity was provided.Results: all variables were positively correlated. The most significant were “guidance”, “communication”, “sociability”, “view”, “hearing” and “feeding”. Items like “self-control”, “equanimity”, “problematic behaviour”, “extroversion” and “autonomy” offered vague and less significant information in identifying the patient’s collaboration level. Variables like “evaluation by the compiler about the patient’s collaboration”, “previous dental experiences” and “attendant” were confirmed. Cronbach’s alpha was 0.77 (standardized result, which meet the a priori criterion of 0.90≥alpha≥0.70.Conclusions

  1. Fast Reliability Assessing Method for Distribution Network with Distributed Renewable Energy Generation

    Science.gov (United States)

    Chen, Fan; Huang, Shaoxiong; Ding, Jinjin; Ding, Jinjin; Gao, Bo; Xie, Yuguang; Wang, Xiaoming

    2018-01-01

    This paper proposes a fast reliability assessing method for distribution grid with distributed renewable energy generation. First, the Weibull distribution and the Beta distribution are used to describe the probability distribution characteristics of wind speed and solar irradiance respectively, and the models of wind farm, solar park and local load are built for reliability assessment. Then based on power system production cost simulation probability discretization and linearization power flow, a optimal power flow objected with minimum cost of conventional power generation is to be resolved. Thus a reliability assessment for distribution grid is implemented fast and accurately. The Loss Of Load Probability (LOLP) and Expected Energy Not Supplied (EENS) are selected as the reliability index, a simulation for IEEE RBTS BUS6 system in MATLAB indicates that the fast reliability assessing method calculates the reliability index much faster with the accuracy ensured when compared with Monte Carlo method.

  2. Evaluation and feedback for effective clinical teaching in postgraduate medical education: validation of an assessment instrument incorporating the CanMEDS roles.

    Science.gov (United States)

    Fluit, Cornelia; Bolhuis, Sanneke; Grol, Richard; Ham, Marieke; Feskens, Remco; Laan, Roland; Wensing, Michel

    2012-01-01

    Providing clinical teachers in postgraduate medical education with feedback about their teaching skills is a powerful tool to improve clinical teaching. A systematic review showed that available instruments do not comprehensively cover all domains of clinical teaching. We developed and empirically test a comprehensive instrument for assessing clinical teachers in the setting of workplace learning and linked to the CanMEDS roles. In a Delphi study, the content validity of a preliminary instrument with 88 items was studied, leading to the construction of the EFFECT (evaluation and feedback for effective clinical teaching) instrument. The response process was explored in a pilot test and focus group research with 18 residents of 6 different disciplines. A confirmatory factor analyses (CFA) and reliability analyses were performed on 407 evaluations of 117 supervisors, collected in 3 medical disciplines (paediatrics, pulmonary diseases and surgery) of 6 departments in 4 different hospitals. CFA yielded an 11 factor model with a good to excellent fit and internal consistencies ranged from 0.740 to 0.940 per domain; 7 items could be deleted. The model of workplace learning showed to be a useful framework for developing EFFECT, which incorporates the CanMEDS competencies and proved to be valid and reliable.

  3. Manual dexterity in hereditary motor and sensory neuropathy type 1a: severity of limitations and feasibility and reliability of two assessment instruments

    NARCIS (Netherlands)

    Videler, Annemieke J.; Beelen, Anita; van Schaik, Ivo N.; de Visser, Marianne; Nollet, Frans

    2008-01-01

    OBJECTIVE: To assess the prevalence and significance of impaired manual dexterity in hereditary motor and sensory neuropathy type 1a (HMSN 1a), with the Sollerman hand function and the Functional Dexterity test, and compare the reliability and agreement of the tests. DESIGN: Descriptive

  4. Reliability data collection and use in risk and availability assessment

    International Nuclear Information System (INIS)

    Colombari, V.

    1989-01-01

    For EuReDatA it is a prevailing objective to initiate and support contact between experts, companies and institutions active in reliability engineering and research. Main topics of this 6th EuReDatA Conference are: Reliability data banks; incidents data banks; common cause data; source and propagation of uncertainties; computer aided risk analysis; reliability and incidents data acquisition and processing; human reliability; probabilistic safety and availability assessment; feedback of reliability into system design; data fusion; reliability modeling and techniques; structural and mechanical reliability; consequence modeling; software and electronic reliability; reliability tests. Some conference papers are separately indexed in the database. (HP)

  5. The Bahasa Melayu version of the Nursing Stress Scale among nurses: a reliability study in Malaysia.

    Science.gov (United States)

    Rosnawati, Muhamad Robat; Moe, Htay; Masilamani, Retneswari; Darus, A

    2010-10-01

    The Nursing Stress Scale (NSS) has been shown to be a valid and reliable instrument to assess occupational stressors among nurses. The NSS, which was previously used in the English version, was translated and back-translated into Bahasa Melayu. This study was conducted to assess the reliability of the Bahasa Melayu version of the NSS among nurses for future studies in this country. The reliability of the NSS was assessed after its readministration to 30 nurses with a 2-week interval. The Spearman coefficient was calculated to assess its stability. The internal consistency was measured through 4 measures: Cronbach's α, Spearman-Brown, Guttman split-half, and standardized item α coefficients. The total response rate was 70%. Test-retest reliability showed remarkable stability (Spearman's ρ exceeded .70). All 4 measures of internal consistency among items indicated a satisfactory level (coefficients in the range of .68 to .87). In conclusion, the Bahasa Melayu version of the NSS is a reliable and useful instrument for measuring the possible stressors at the workplace among nurses.

  6. Short version of the “instrument for assessment of stress in nursing students” in the Brazilian reality

    Directory of Open Access Journals (Sweden)

    Ana Lúcia Siqueira Costa

    2018-01-01

    Full Text Available ABSTRACT Goal: validate a short version of the Instrument for assessment of stress in nursing students in the Brazilian reality. Method: Methodological study conducted with 1047 nursing students from five Brazilian institutions, who answered the 30 items initially distributed in eight domains. Data were analyzed in the R Statistical Package and in the latent variable analysis, using exploratory and confirmatory factor analyses, Cronbach’s alpha and item-total correlation. Results: The short version of the instrument had 19 items distributed into four domains: Environment, Professional Training, Theoretical Activities and Performance of Practical Activities. The confirmatory analysis showed absolute and parsimony fit to the proposed model with satisfactory residual levels. Alpha values per factor ranged from 0.736 (Environment to 0.842 (Performance of Practical Activities. Conclusion: The short version of the instrument has construct validity and reliability for application to Brazilian nursing undergraduates at any stage of the course.

  7. Implementation outcome assessment instruments used in physical healthcare settings and their measurement properties: a systematic review protocol.

    Science.gov (United States)

    Khadjesari, Zarnie; Vitoratou, Silia; Sevdalis, Nick; Hull, Louise

    2017-10-08

    Over the past 10 years, research into methods that promote the uptake, implementation and sustainability of evidence-based interventions has gathered pace. However, implementation outcomes are defined in different ways and assessed by different measures; the extent to which these measures are valid and reliable is unknown. The aim of this systematic review is to identify and appraise studies that assess the measurement properties of quantitative implementation outcome instruments used in physical healthcare settings, to advance the use of precise and accurate measures. The following databases will be searched from inception to March 2017: MEDLINE, EMBASE, PsycINFO, CINAHL and the Cochrane Library. Grey literature will be sought via HMIC, OpenGrey, ProQuest for theses and Web of Science Conference Proceedings Citation Index-Science. Reference lists of included studies and relevant reviews will be hand searched. Three search strings will be combined to identify eligible studies: (1) implementation literature, (2) implementation outcomes and (3) measurement properties. Screening of titles, abstracts and full papers will be assessed for eligibility by two reviewers independently and any discrepancies resolved via consensus with the wider team. The methodological quality of the studies will be assessed using the COnsensus-based Standards for the selection of health Measurement INstruments checklist. A set of bespoke criteria to determine the quality of the instruments will be used, and the relationship between instrument usability and quality will be explored. Ethical approval is not necessary for systematic review protocols. Researchers and healthcare professionals can use the findings of this systematic review to guide the selection of implementation outcomes instruments, based on their psychometric quality, to assess the impact of their implementation efforts. The findings will also provide a useful guide for reviewers of papers and grants to determine the

  8. Implementation outcome assessment instruments used in physical healthcare settings and their measurement properties: a systematic review protocol

    Science.gov (United States)

    Vitoratou, Silia; Sevdalis, Nick; Hull, Louise

    2017-01-01

    Introduction Over the past 10 years, research into methods that promote the uptake, implementation and sustainability of evidence-based interventions has gathered pace. However, implementation outcomes are defined in different ways and assessed by different measures; the extent to which these measures are valid and reliable is unknown. The aim of this systematic review is to identify and appraise studies that assess the measurement properties of quantitative implementation outcome instruments used in physical healthcare settings, to advance the use of precise and accurate measures. Methods and analysis The following databases will be searched from inception to March 2017: MEDLINE, EMBASE, PsycINFO, CINAHL and the Cochrane Library. Grey literature will be sought via HMIC, OpenGrey, ProQuest for theses and Web of Science Conference Proceedings Citation Index-Science. Reference lists of included studies and relevant reviews will be hand searched. Three search strings will be combined to identify eligible studies: (1) implementation literature, (2) implementation outcomes and (3) measurement properties. Screening of titles, abstracts and full papers will be assessed for eligibility by two reviewers independently and any discrepancies resolved via consensus with the wider team. The methodological quality of the studies will be assessed using the COnsensus-based Standards for the selection of health Measurement INstruments checklist. A set of bespoke criteria to determine the quality of the instruments will be used, and the relationship between instrument usability and quality will be explored. Ethics and dissemination Ethical approval is not necessary for systematic review protocols. Researchers and healthcare professionals can use the findings of this systematic review to guide the selection of implementation outcomes instruments, based on their psychometric quality, to assess the impact of their implementation efforts. The findings will also provide a useful guide for

  9. Environmental education curriculum evaluation questionnaire: A reliability and validity study

    Science.gov (United States)

    Minner, Daphne Diane

    The intention of this research project was to bridge the gap between social science research and application to the environmental domain through the development of a theoretically derived instrument designed to give educators a template by which to evaluate environmental education curricula. The theoretical base for instrument development was provided by several developmental theories such as Piaget's theory of cognitive development, Developmental Systems Theory, Life-span Perspective, as well as curriculum research within the area of environmental education. This theoretical base fueled the generation of a list of components which were then translated into a questionnaire with specific questions relevant to the environmental education domain. The specific research question for this project is: Can a valid assessment instrument based largely on human development and education theory be developed that reliably discriminates high, moderate, and low quality in environmental education curricula? The types of analyses conducted to answer this question were interrater reliability (percent agreement, Cohen's Kappa coefficient, Pearson's Product-Moment correlation coefficient), test-retest reliability (percent agreement, correlation), and criterion-related validity (correlation). Face validity and content validity were also assessed through thorough reviews. Overall results indicate that 29% of the questions on the questionnaire demonstrated a high level of interrater reliability and 43% of the questions demonstrated a moderate level of interrater reliability. Seventy-one percent of the questions demonstrated a high test-retest reliability and 5% a moderate level. Fifty-five percent of the questions on the questionnaire were reliable (high or moderate) both across time and raters. Only eight questions (8%) did not show either interrater or test-retest reliability. The global overall rating of high, medium, or low quality was reliable across both coders and time, indicating

  10. Reliability and validity of the transport and physical activity questionnaire (TPAQ for assessing physical activity behaviour.

    Directory of Open Access Journals (Sweden)

    Emma J Adams

    Full Text Available No current validated survey instrument allows a comprehensive assessment of both physical activity and travel behaviours for use in interdisciplinary research on walking and cycling. This study reports on the test-retest reliability and validity of physical activity measures in the transport and physical activity questionnaire (TPAQ.The TPAQ assesses time spent in different domains of physical activity and using different modes of transport for five journey purposes. Test-retest reliability of eight physical activity summary variables was assessed using intra-class correlation coefficients (ICC and Kappa scores for continuous and categorical variables respectively. In a separate study, the validity of three survey-reported physical activity summary variables was assessed by computing Spearman correlation coefficients using accelerometer-derived reference measures. The Bland-Altman technique was used to determine the absolute validity of survey-reported time spent in moderate-to-vigorous physical activity (MVPA.In the reliability study, ICC for time spent in different domains of physical activity ranged from fair to substantial for walking for transport (ICC = 0.59, cycling for transport (ICC = 0.61, walking for recreation (ICC = 0.48, cycling for recreation (ICC = 0.35, moderate leisure-time physical activity (ICC = 0.47, vigorous leisure-time physical activity (ICC = 0.63, and total physical activity (ICC = 0.56. The proportion of participants estimated to meet physical activity guidelines showed acceptable reliability (k = 0.60. In the validity study, comparison of survey-reported and accelerometer-derived time spent in physical activity showed strong agreement for vigorous physical activity (r = 0.72, p<0.001, fair but non-significant agreement for moderate physical activity (r = 0.24, p = 0.09 and fair agreement for MVPA (r = 0.27, p = 0.05. Bland-Altman analysis showed a mean

  11. Reliability and Validity of the Transport and Physical Activity Questionnaire (TPAQ) for Assessing Physical Activity Behaviour

    Science.gov (United States)

    Adams, Emma J.; Goad, Mary; Sahlqvist, Shannon; Bull, Fiona C.; Cooper, Ashley R.; Ogilvie, David

    2014-01-01

    Background No current validated survey instrument allows a comprehensive assessment of both physical activity and travel behaviours for use in interdisciplinary research on walking and cycling. This study reports on the test-retest reliability and validity of physical activity measures in the transport and physical activity questionnaire (TPAQ). Methods The TPAQ assesses time spent in different domains of physical activity and using different modes of transport for five journey purposes. Test-retest reliability of eight physical activity summary variables was assessed using intra-class correlation coefficients (ICC) and Kappa scores for continuous and categorical variables respectively. In a separate study, the validity of three survey-reported physical activity summary variables was assessed by computing Spearman correlation coefficients using accelerometer-derived reference measures. The Bland-Altman technique was used to determine the absolute validity of survey-reported time spent in moderate-to-vigorous physical activity (MVPA). Results In the reliability study, ICC for time spent in different domains of physical activity ranged from fair to substantial for walking for transport (ICC = 0.59), cycling for transport (ICC = 0.61), walking for recreation (ICC = 0.48), cycling for recreation (ICC = 0.35), moderate leisure-time physical activity (ICC = 0.47), vigorous leisure-time physical activity (ICC = 0.63), and total physical activity (ICC = 0.56). The proportion of participants estimated to meet physical activity guidelines showed acceptable reliability (k = 0.60). In the validity study, comparison of survey-reported and accelerometer-derived time spent in physical activity showed strong agreement for vigorous physical activity (r = 0.72, pphysical activity (r = 0.24, p = 0.09) and fair agreement for MVPA (r = 0.27, p = 0.05). Bland-Altman analysis showed a mean overestimation of MVPA of 87.6 min/week (p

  12. Reliable and valid assessment of Lichtenstein hernia repair skills

    DEFF Research Database (Denmark)

    Carlsen, C G; Lindorff Larsen, Karen; Funch-Jensen, P

    2014-01-01

    PURPOSE: Lichtenstein hernia repair is a common surgical procedure and one of the first procedures performed by a surgical trainee. However, formal assessment tools developed for this procedure are few and sparsely validated. The aim of this study was to determine the reliability and validity...... of an assessment tool designed to measure surgical skills in Lichtenstein hernia repair. METHODS: Key issues were identified through a focus group interview. On this basis, an assessment tool with eight items was designed. Ten surgeons and surgical trainees were video recorded while performing Lichtenstein hernia...... a significant difference between the three groups which indicates construct validity, p skills can be assessed blindly by a single rater in a reliable and valid fashion with the new procedure-specific assessment tool. We recommend this tool for future assessment...

  13. RELIABILITY ASSESSMENT OF ENTROPY METHOD FOR SYSTEM CONSISTED OF IDENTICAL EXPONENTIAL UNITS

    Institute of Scientific and Technical Information of China (English)

    Sun Youchao; Shi Jun

    2004-01-01

    The reliability assessment of unit-system near two levels is the most important content in the reliability multi-level synthesis of complex systems. Introducing the information theory into system reliability assessment, using the addible characteristic of information quantity and the principle of equivalence of information quantity, an entropy method of data information conversion is presented for the system consisted of identical exponential units. The basic conversion formulae of entropy method of unit test data are derived based on the principle of information quantity equivalence. The general models of entropy method synthesis assessment for system reliability approximate lower limits are established according to the fundamental principle of the unit reliability assessment. The applications of the entropy method are discussed by way of practical examples. Compared with the traditional methods, the entropy method is found to be valid and practicable and the assessment results are very satisfactory.

  14. Development of a new instrument to assess AIDS-related attitudes among Spanish youngsters

    Directory of Open Access Journals (Sweden)

    José P. Espada

    2013-01-01

    Full Text Available This paper aims to describe the psychometric properties of a new brief multidimensional scale to measure HIV/AIDS related attitudes for adolescents, the HIV Attitudes Scale (HIV-AS. Exploratory analysis aimed to find the theory-driven structure and to develop an empirically derived factor structure for HIV-AS, which was tested with a confirmatory factor analysis. Factorial invariance was studied across gender and age, and multigroup structural equation approach was used in order to compare the groups. Reliability, convergent and discnmmant validity were examined. The analyses were based on a sample of 1216 Spanish adolescents. The instrument consists of 12 items distributed across four major factors: 1 Attitudes towards safe sex when there are obstacles, 2 Attitudes towards the HIV test, 3 Attitudes towards condom use, and 4 Attitudes towards people living with the AIDS. The HIV-AS shows good validity and internal reliability and equivalent four-factor structure across gender and the three age groups studied. It can be consider as a capable and parsimonious self-report scale for assessing major aspects of HIV/AIDS-related attitudes for adolescents.

  15. Methodology for reliability based condition assessment

    International Nuclear Information System (INIS)

    Mori, Y.; Ellingwood, B.

    1993-08-01

    Structures in nuclear power plants may be exposed to aggressive environmental effects that cause their strength to decrease over an extended period of service. A major concern in evaluating the continued service for such structures is to ensure that in their current condition they are able to withstand future extreme load events during the intended service life with a level of reliability sufficient for public safety. This report describes a methodology to facilitate quantitative assessments of current and future structural reliability and performance of structures in nuclear power plants. This methodology takes into account the nature of past and future loads, and randomness in strength and in degradation resulting from environmental factors. An adaptive Monte Carlo simulation procedure is used to evaluate time-dependent system reliability. The time-dependent reliability is sensitive to the time-varying load characteristics and to the choice of initial strength and strength degradation models but not to correlation in component strengths within a system. Inspection/maintenance strategies are identified that minimize the expected future costs of keeping the failure probability of a structure at or below an established target failure probability during its anticipated service period

  16. Adolescent Delinquency and Psychopathology: Is Maysi-2 a Valid Instrument to Assess Psychopathology Risk among Turkish Adolescents?

    Directory of Open Access Journals (Sweden)

    Emine Demirbaş-Çakır

    2015-10-01

    Full Text Available Introduction: In this study it was aimed to study validity and reliability of Turkish form of Massachusetts outh Screening Instrument-Second Version (MAYSI-2.Method and materials: Totally 387 adolescents, 333 boys and 54 girls participated to the study. Calculation of each subscale’s Cronbach’s alpha and inter subscales correlation coefficiency has been performed to assess reliability of Turkish form of MAYSI-2. Explanatory factor analysis and Turkish form of Youth Self Report (YSR have been used to assess validity of the scale.Results: Cronbach’s alpha coefficiency values were 0.84 for Alcohol/Drug Use, 0.78 for Anger/Irritability, 0.66 for Depression/Anxiety, 0.58 for Somatic Complaints, 0.84 for Suicidal Ideation, 0,15 for Thought Disturbance, 0.55 for Traumatic Experiences for boys and 0.66 for Traumatic Experiences for girls. Turkish form of MAYSI-2 consisted of seven factor structure as the original one in explanatory factor analysis. Statistically significant correlation coefficiencies which range between 0.27 and 0.69 were found in between all the subscales of both Turkish form of MAYSI-2 and Turkish YSR.Conclusion: Results of this study showed that Turkish form of MAYSI-2 is valid and reliable to perform risk assessment with regard to psychiatric disorders among Turkish youth population who entered to the juvenile justice system. 

  17. The knowledge, efficacy, and practices instrument for oral health providers: a validity study with dental students.

    Science.gov (United States)

    Behar-Horenstein, Linda S; Garvan, Cyndi W; Moore, Thomas E; Catalanotto, Frank A

    2013-08-01

    Valid and reliable instruments to measure and assess cultural competence for oral health care providers are scarce in the literature, and most published scales have been contested due to a lack of item analysis and internal estimates of reliability. The purposes of this study were, first, to develop a standardized instrument to measure dental students' knowledge of diversity, skills in culturally competent patient-centered communication, and use of culture-centered practices in patient care and, second, to provide preliminary validity support for this instrument. The initial instrument used in this study was a thirty-six-item Likert-scale survey entitled the Knowledge, Efficacy, and Practices Instrument for Oral Health Providers (KEPI-OHP). This instrument is an adaption of an initially thirty-three-item version of the Multicultural Awareness, Knowledge, and Skills Scale-Counselor Edition (MAKSS-CE), a scale that assesses factors related to social justice, cultural differences among clients, and cross-cultural client management. After the authors conducted cognitive and expert interviews, focus groups, pilot testing, and item analysis, their initial instrument was reduced to twenty-eight items. The KEPI-OHP was then distributed to 916 dental students (response rate=48.6 percent) across the United States to measure its reliability and assess its validity. Both exploratory and confirmatory factor analyses were conducted to test the scale's validity. The modification of the survey into a sensible instrument with a relatively clear factor structure using factor analysis resulted in twenty items. A scree test suggested three expressive factors, which were retained for rotation. Bentler's comparative fit and Bentler and Bonnett's non-normed indices were 0.95 and 0.92, respectively. A three-factor solution, including efficacy of assessment, knowledge of diversity, and culture-centered practice subscales, comprised of twenty-items was identified. The KEPI-OHP was found to

  18. A computational Bayesian approach to dependency assessment in system reliability

    International Nuclear Information System (INIS)

    Yontay, Petek; Pan, Rong

    2016-01-01

    Due to the increasing complexity of engineered products, it is of great importance to develop a tool to assess reliability dependencies among components and systems under the uncertainty of system reliability structure. In this paper, a Bayesian network approach is proposed for evaluating the conditional probability of failure within a complex system, using a multilevel system configuration. Coupling with Bayesian inference, the posterior distributions of these conditional probabilities can be estimated by combining failure information and expert opinions at both system and component levels. Three data scenarios are considered in this study, and they demonstrate that, with the quantification of the stochastic relationship of reliability within a system, the dependency structure in system reliability can be gradually revealed by the data collected at different system levels. - Highlights: • A Bayesian network representation of system reliability is presented. • Bayesian inference methods for assessing dependencies in system reliability are developed. • Complete and incomplete data scenarios are discussed. • The proposed approach is able to integrate reliability information from multiple sources at multiple levels of the system.

  19. Quality of prenatal care questionnaire: instrument development and testing.

    Science.gov (United States)

    Heaman, Maureen I; Sword, Wendy A; Akhtar-Danesh, Noori; Bradford, Amanda; Tough, Suzanne; Janssen, Patricia A; Young, David C; Kingston, Dawn A; Hutton, Eileen K; Helewa, Michael E

    2014-06-03

    Utilization indices exist to measure quantity of prenatal care, but currently there is no published instrument to assess quality of prenatal care. The purpose of this study was to develop and test a new instrument, the Quality of Prenatal Care Questionnaire (QPCQ). Data for this instrument development study were collected in five Canadian cities. Items for the QPCQ were generated through interviews with 40 pregnant women and 40 health care providers and a review of prenatal care guidelines, followed by assessment of content validity and rating of importance of items. The preliminary 100-item QPCQ was administered to 422 postpartum women to conduct item reduction using exploratory factor analysis. The final 46-item version of the QPCQ was then administered to another 422 postpartum women to establish its construct validity, and internal consistency and test-retest reliability. Exploratory factor analysis reduced the QPCQ to 46 items, factored into 6 subscales, which subsequently were validated by confirmatory factor analysis. Construct validity was also demonstrated using a hypothesis testing approach; there was a significant positive association between women's ratings of the quality of prenatal care and their satisfaction with care (r = 0.81). Convergent validity was demonstrated by a significant positive correlation (r = 0.63) between the "Support and Respect" subscale of the QPCQ and the "Respectfulness/Emotional Support" subscale of the Prenatal Interpersonal Processes of Care instrument. The overall QPCQ had acceptable internal consistency reliability (Cronbach's alpha = 0.96), as did each of the subscales. The test-retest reliability result (Intra-class correlation coefficient = 0.88) indicated stability of the instrument on repeat administration approximately one week later. Temporal stability testing confirmed that women's ratings of their quality of prenatal care did not change as a result of giving birth or between the early postpartum

  20. Quality of prenatal care questionnaire: instrument development and testing

    Science.gov (United States)

    2014-01-01

    Background Utilization indices exist to measure quantity of prenatal care, but currently there is no published instrument to assess quality of prenatal care. The purpose of this study was to develop and test a new instrument, the Quality of Prenatal Care Questionnaire (QPCQ). Methods Data for this instrument development study were collected in five Canadian cities. Items for the QPCQ were generated through interviews with 40 pregnant women and 40 health care providers and a review of prenatal care guidelines, followed by assessment of content validity and rating of importance of items. The preliminary 100-item QPCQ was administered to 422 postpartum women to conduct item reduction using exploratory factor analysis. The final 46-item version of the QPCQ was then administered to another 422 postpartum women to establish its construct validity, and internal consistency and test-retest reliability. Results Exploratory factor analysis reduced the QPCQ to 46 items, factored into 6 subscales, which subsequently were validated by confirmatory factor analysis. Construct validity was also demonstrated using a hypothesis testing approach; there was a significant positive association between women’s ratings of the quality of prenatal care and their satisfaction with care (r = 0.81). Convergent validity was demonstrated by a significant positive correlation (r = 0.63) between the “Support and Respect” subscale of the QPCQ and the “Respectfulness/Emotional Support” subscale of the Prenatal Interpersonal Processes of Care instrument. The overall QPCQ had acceptable internal consistency reliability (Cronbach’s alpha = 0.96), as did each of the subscales. The test-retest reliability result (Intra-class correlation coefficient = 0.88) indicated stability of the instrument on repeat administration approximately one week later. Temporal stability testing confirmed that women’s ratings of their quality of prenatal care did not change as a result of giving

  1. Reliable and valid assessment of Lichtenstein hernia repair skills.

    Science.gov (United States)

    Carlsen, C G; Lindorff-Larsen, K; Funch-Jensen, P; Lund, L; Charles, P; Konge, L

    2014-08-01

    Lichtenstein hernia repair is a common surgical procedure and one of the first procedures performed by a surgical trainee. However, formal assessment tools developed for this procedure are few and sparsely validated. The aim of this study was to determine the reliability and validity of an assessment tool designed to measure surgical skills in Lichtenstein hernia repair. Key issues were identified through a focus group interview. On this basis, an assessment tool with eight items was designed. Ten surgeons and surgical trainees were video recorded while performing Lichtenstein hernia repair, (four experts, three intermediates, and three novices). The videos were blindly and individually assessed by three raters (surgical consultants) using the assessment tool. Based on these assessments, validity and reliability were explored. The internal consistency of the items was high (Cronbach's alpha = 0.97). The inter-rater reliability was very good with an intra-class correlation coefficient (ICC) = 0.93. Generalizability analysis showed a coefficient above 0.8 even with one rater. The coefficient improved to 0.92 if three raters were used. One-way analysis of variance found a significant difference between the three groups which indicates construct validity, p fashion with the new procedure-specific assessment tool. We recommend this tool for future assessment of trainees performing Lichtenstein hernia repair to ensure that the objectives of competency-based surgical training are met.

  2. Reliability, precision, and measurement in the context of data from ability tests, surveys, and assessments

    International Nuclear Information System (INIS)

    Fisher, W P Jr; Elbaum, B; Coulter, A

    2010-01-01

    Reliability coefficients indicate the proportion of total variance attributable to differences among measures separated along a quantitative continuum by a testing, survey, or assessment instrument. Reliability is usually considered to be influenced by both the internal consistency of a data set and the number of items, though textbooks and research papers rarely evaluate the extent to which these factors independently affect the data in question. Probabilistic formulations of the requirements for unidimensional measurement separate consistency from error by modelling individual response processes instead of group-level variation. The utility of this separation is illustrated via analyses of small sets of simulated data, and of subsets of data from a 78-item survey of over 2,500 parents of children with disabilities. Measurement reliability ultimately concerns the structural invariance specified in models requiring sufficient statistics, parameter separation, unidimensionality, and other qualities that historically have made quantification simple, practical, and convenient for end users. The paper concludes with suggestions for a research program aimed at focusing measurement research more on the calibration and wide dissemination of tools applicable to individuals, and less on the statistical study of inter-variable relations in large data sets.

  3. Quality of Life among Persons with HIV/AIDS in Iran: Internal Reliability and Validity of an International Instrument and Associated Factors

    Directory of Open Access Journals (Sweden)

    Pedram Razavi

    2012-01-01

    Full Text Available The purpose of this cross-sectional study on 191 HIV/AIDS patient was to prepare the first Persian translation of complete WHOQOL-HIV instrument, evaluate its reliability and validity, and apply it to determine quality of life and its associated factors in Tehran, Iran. Student's t-test was used to compare quality of life between groups. Mean Cronbach’s α of facets in all six domains of instrument were more than 0.6 indicating good reliability. Item/total corrected correlations coefficients had a lower limit of more than 0.5 in all facets except for association between energy and fatigue facet and physical domain. Compared to younger participants, patients older than 35 years had significantly lower scores in overall quality of life (P = 0.003, social relationships (P = 0.021, and spirituality/religion/personal beliefs (P = 0.024. Unemployed patients had significantly lower scores in overall quality of life (P = 0.01, level of independence (P = 0.004, and environment (P = 0.001 compared to employed participants. This study demonstrated that the standard, complete WHOQOL-HIV 120 instrument translated into Farsi and evaluated among Iranian participants provides a reliable and valid basis for future research on quality of life for HIV and other patients in Iran.

  4. Automation is an Effective Way to Improve Quality of Verification (Calibration) of Measuring Instruments

    Science.gov (United States)

    Golobokov, M.; Danilevich, S.

    2018-04-01

    In order to assess calibration reliability and automate such assessment, procedures for data collection and simulation study of thermal imager calibration procedure have been elaborated. The existing calibration techniques do not always provide high reliability. A new method for analyzing the existing calibration techniques and developing new efficient ones has been suggested and tested. A type of software has been studied that allows generating instrument calibration reports automatically, monitoring their proper configuration, processing measurement results and assessing instrument validity. The use of such software allows reducing man-hours spent on finalization of calibration data 2 to 5 times and eliminating a whole set of typical operator errors.

  5. Assessing the Construct Validity and Internal Reliability of the Screening Tool Test Your Memory in Patients with Chronic Pain

    Science.gov (United States)

    Ojeda, B.; Salazar, A.; Dueñas, M.; Torres, L. M.; Mico, J. A.; Failde, I.

    2016-01-01

    Patients with chronic pain often complain about cognitive difficulties, and since these symptoms represent an additional source of suffering and distress, evaluating the cognitive status of these patients with valid and reliable tests should be an important part of their overall assessment. Although cognitive impairment is a critical characteristic of pain, there is no specific measure designed to detect these effects in this population. The objective was to analyze the psychometric properties of the “Test Your Memory” (TYM) test in patients with chronic pain of three different origins. A cross-sectional study was carried out on 72 subjects free of pain and 254 patients suffering from different types of chronic pain: neuropathic pain (104), musculoskeletal pain (99) and fibromyalgia (51). The construct validity of the TYM was assessed using the Mini-Mental State Examination (MMSE), Hospital Anxiety and Depression Scale (HADs), Index-9 from MOS-sleep, SF-12, and through the intensity (Visual Analogical Scale) and duration of pain. An exploratory factor analysis was also performed and internal reliability was assessed using Cronbach’s alpha. After adjusting for potential confounders the TYM could distinguish between pain and pain-free patients, and it was correlated with the: MMSE (0.89, pmental components (0.55, p valid and reliable screening instrument to assess cognitive function in chronic pain patients that will be of particular value in clinical situations. PMID:27119165

  6. Assessing the Construct Validity and Internal Reliability of the Screening Tool Test Your Memory in Patients with Chronic Pain.

    Science.gov (United States)

    Ojeda, B; Salazar, A; Dueñas, M; Torres, L M; Mico, J A; Failde, I

    2016-01-01

    Patients with chronic pain often complain about cognitive difficulties, and since these symptoms represent an additional source of suffering and distress, evaluating the cognitive status of these patients with valid and reliable tests should be an important part of their overall assessment. Although cognitive impairment is a critical characteristic of pain, there is no specific measure designed to detect these effects in this population. The objective was to analyze the psychometric properties of the "Test Your Memory" (TYM) test in patients with chronic pain of three different origins. A cross-sectional study was carried out on 72 subjects free of pain and 254 patients suffering from different types of chronic pain: neuropathic pain (104), musculoskeletal pain (99) and fibromyalgia (51). The construct validity of the TYM was assessed using the Mini-Mental State Examination (MMSE), Hospital Anxiety and Depression Scale (HADs), Index-9 from MOS-sleep, SF-12, and through the intensity (Visual Analogical Scale) and duration of pain. An exploratory factor analysis was also performed and internal reliability was assessed using Cronbach's alpha. After adjusting for potential confounders the TYM could distinguish between pain and pain-free patients, and it was correlated with the: MMSE (0.89, pmental components (0.55, p reliable screening instrument to assess cognitive function in chronic pain patients that will be of particular value in clinical situations.

  7. Concurrent validity and interrater reliability of a new smartphone application to assess 3D active cervical range of motion in patients with neck pain.

    Science.gov (United States)

    Stenneberg, Martijn S; Busstra, Harm; Eskes, Michel; van Trijffel, Emiel; Cattrysse, Erik; Scholten-Peeters, Gwendolijne G M; de Bie, Rob A

    2018-04-01

    There is a lack of valid, reliable, and feasible instruments for measuring planar active cervical range of motion (aCROM) and associated 3D coupling motions in patients with neck pain. Smartphones have advanced sensors and appear to be suitable for these measurements. To estimate the concurrent validity and interrater reliability of a new iPhone application for assessing planar aCROM and associated 3D coupling motions in patients with neck pain, using an electromagnetic tracking device as a reference test. Cross-sectional study. Two samples of neck pain patients were recruited; 30 patients for the validity study and 26 patients for the reliability study. Validity was estimated using intraclass correlation coefficients (ICCs), and by calculating 95% limits of agreement (LoA). To estimate interrater reliability, ICCs were calculated. Cervical 3D coupling motions were analyzed by calculating the cross-correlation coefficients and ratio between the main motions and coupled motions for both instruments. ICCs for concurrent validity and interrater reliability ranged from 0.90 to 0.99. The width of the 95% LoA ranged from about 5° for right lateral bending to 11° for total rotation. No significant differences were found between both devices for associated coupling motion analysis. The iPhone application appears to be a useful discriminative tool for the measurement of planar aCROM and associated coupling motions in patients with neck pain. It fulfills the need for a valid, reliable, and feasible instrument in clinical practice and research. Therapists and researchers should consider measurement error when interpreting scores. Copyright © 2017 Elsevier Ltd. All rights reserved.

  8. Reliability of a semi-quantitative method for dermal exposure assessment (DREAM)

    NARCIS (Netherlands)

    Wendel de Joode, B. van; Hemmen, J.J. van; Meijster, T.; Major, V.; London, L.; Kromhout, H.

    2005-01-01

    Valid and reliable semi-quantitative dermal exposure assessment methods for epidemiological research and for occupational hygiene practice, applicable for different chemical agents, are practically nonexistent. The aim of this study was to assess the reliability of a recently developed

  9. Assessment of the measurement properties of the post stroke motor function instruments available in Brazil: a systematic review.

    Science.gov (United States)

    Lima, Elaine; Teixeira-Salmela, Luci F; Simões, Luan; Guerra, Ana C C; Lemos, Andrea

    2016-03-15

    While there are several instruments in Brazil that measure motor function in patients after stroke, it is unknown whether the measurement properties of these instruments are appropriate. To identify the motor function instruments available in Brazil for patients after stroke. To assess the methodological quality of the studies and the results related to the measurement properties of these instruments. Two independent reviewers conducted searches on PubMed, LILACS, CINAHL, Web of Science, and Scopus. Studies that aimed to cross-culturally adapt an existing instrument or create a Brazilian instrument and test at least one measurement property related to motor function in patients after stroke were included. The methodological quality of these studies was checked by the COSMIN checklist with 4-point rating scale and the results of the measurement properties were analyzed by the criteria developed by Terwee et al. A total of 11 instruments were considered eligible, none of which were created in Brazil. The process of cross-cultural adaptation was inadequate in 10 out of 11 instruments due to the lack of back-translation or due to inappropriate target population. All of the instruments presented flaws in the measurement properties, especially reliability, internal consistency, and construct validity. The flaws observed in both cross-cultural adaptation process and testing measurement properties make the results inconclusive on the validity of the available instruments. Adequate procedures of cross-cultural adaptation and measurement properties of these instruments are strongly needed.

  10. Assessment of the measurement properties of the post stroke motor function instruments available in Brazil: a systematic review

    Directory of Open Access Journals (Sweden)

    Elaine Lima

    2016-01-01

    Full Text Available Background While there are several instruments in Brazil that measure motor function in patients after stroke, it is unknown whether the measurement properties of these instruments are appropriate. Objective To identify the motor function instruments available in Brazil for patients after stroke. To assess the methodological quality of the studies and the results related to the measurement properties of these instruments. Method Two independent reviewers conducted searches on PubMed, LILACS, CINAHL, Web of Science, and Scopus. Studies that aimed to cross-culturally adapt an existing instrument or create a Brazilian instrument and test at least one measurement property related to motor function in patients after stroke were included. The methodological quality of these studies was checked by the COSMIN checklist with 4-point rating scale and the results of the measurement properties were analyzed by the criteria developed by Terwee et al. Results A total of 11 instruments were considered eligible, none of which were created in Brazil. The process of cross-cultural adaptation was inadequate in 10 out of 11 instruments due to the lack of back-translation or due to inappropriate target population. All of the instruments presented flaws in the measurement properties, especially reliability, internal consistency, and construct validity. Conclusion The flaws observed in both cross-cultural adaptation process and testing measurement properties make the results inconclusive on the validity of the available instruments. Adequate procedures of cross-cultural adaptation and measurement properties of these instruments are strongly needed.

  11. Reliability and Validity of Survey Instruments to Measure Work-Related Fatigue in the Emergency Medical Services Setting: A Systematic Review

    Science.gov (United States)

    2018-01-11

    Background: This study sought to systematically search the literature to identify reliable and valid survey instruments for fatigue measurement in the Emergency Medical Services (EMS) occupational setting. Methods: A systematic review study design wa...

  12. Health literacy screening instruments for eHealth applications: a systematic review.

    Science.gov (United States)

    Collins, Sarah A; Currie, Leanne M; Bakken, Suzanne; Vawdrey, David K; Stone, Patricia W

    2012-06-01

    To systematically review current health literacy (HL) instruments for use in consumer-facing and mobile health information technology screening and evaluation tools. The databases, PubMed, OVID, Google Scholar, Cochrane Library and Science Citation Index, were searched for health literacy assessment instruments using the terms "health", "literacy", "computer-based," and "psychometrics". All instruments identified by this method were critically appraised according to their reported psychometric properties and clinical feasibility. Eleven different health literacy instruments were found. Screening questions, such as asking a patient about his/her need for assistance in navigating health information, were evaluated in seven different studies and are promising for use as a valid, reliable, and feasible computer-based approach to identify patients that struggle with low health literacy. However, there was a lack of consistency in the types of screening questions proposed. There is also a lack of information regarding the psychometric properties of computer-based health literacy instruments. Only English language health literacy assessment instruments were reviewed and analyzed. Current health literacy screening tools demonstrate varying benefits depending on the context of their use. In many cases, it seems that a single screening question may be a reliable, valid, and feasible means for establishing health literacy. A combination of screening questions that assess health literacy and technological literacy may enable tailoring eHealth applications to user needs. Further research should determine the best screening question(s) and the best synthesis of various instruments' content and methodologies for computer-based health literacy screening and assessment. Copyright © 2012 Elsevier Inc. All rights reserved.

  13. Assessment of modern methods of human factor reliability analysis in PSA studies

    International Nuclear Information System (INIS)

    Holy, J.

    2001-12-01

    The report is structured as follows: Classical terms and objects (Probabilistic safety assessment as a framework for human reliability assessment; Human failure within the PSA model; Basic types of operator failure modelled in a PSA study and analyzed by HRA methods; Qualitative analysis of human reliability; Quantitative analysis of human reliability used; Process of analysis of nuclear reactor operator reliability in a PSA study); New terms and objects (Analysis of dependences; Errors of omission; Errors of commission; Error forcing context); and Overview and brief assessment of human reliability analysis (Basic characteristics of the methods; Assets and drawbacks of the use of each of HRA method; History and prospects of the use of the methods). (P.A.)

  14. Reliability of inertial sensors in the assessment of patients with vestibular disorders: a feasibility study

    Directory of Open Access Journals (Sweden)

    Sathish K. Sankarpandi

    2017-02-01

    Full Text Available Abstract Background Vestibular disorders affect an individual’s stability, balance, and gait and predispose them to falls. Traditional laboratory-based semi-objective vestibular assessments are intrusive and cumbersome provide little information about their functional ability. Commercially available wearable inertial sensors allow us to make this real life assessments objective, with a detailed view of their functional abilities. Timed Up and Go (TUG and Postural Sway tests are commonly used tests for gait and balance assessments. Our aim was to assess the feasibility, test-retest reliability and ability to classify fall status in individuals with vestibular disorders using parameters derived from the commercially available wearable system (inertial sensors and the Mobility Lab Software, APDM, Inc.. Methods We recruited 27 individuals diagnosed either with unilateral or bilateral vestibular loss on vestibular function testing. Instrumented Timed Up and Go (iTUG and Postural Sway (iSway were administered three times during the first session and then repeated at a similar time the following week. To evaluate within and between sessions reliability of the parameters the Intra-Class Correlation coefficient (ICC was used. Subsequently, the ability of reliable parameters (ICC ≥ 0.8 to classify fallers from non-fallers was estimated. Results The iTUG test parameters showed good within and between sessions’ reliability with mean ICC (between-sessions values of 0.81 ± 0.17 and 0.69 ± 0.15, respectively. For the iSway test, the relative figures were; 0.76 ± 0.13 and 0.71 ± 0.14, respectively. A retrospective falls classification analysis with past 12 months falls history data yielded an accuracy of 66.70% with an area under the curve of 0.79. Mean Distance from centre of COP (mm of accelerometer’s trajectory (m/s2 from the iSway test was the only significant parameter to classify fallers from non-fallers. Conclusions Using

  15. Counselor Competence, Performance Assessment, and Program Evaluation: Using Psychometric Instruments

    Science.gov (United States)

    Tate, Kevin A.; Bloom, Margaret L.; Tassara, Marcel H.; Caperton, William

    2014-01-01

    Psychometric instruments have been underutilized by counselor educators in performance assessment and program evaluation efforts. As such, we conducted a review of the literature that revealed 41 instruments fit for such efforts. We described and critiqued these instruments along four dimensions--"Target Domain," "Format,"…

  16. Test-retest Reliability and Agreement of the Satisfaction with the Assistive Technology Services (SATS) Instrument in Two Nordic Countries

    DEFF Research Database (Denmark)

    Sund, Terje; Anttila, Heidi; Iwarsson, Susanne

    2014-01-01

    Purpose: The purpose of this study was to investigate test–retest reliability, agreement, internal consistency, and floor- and ceiling effects of the Danish and Finnish versions of the Satisfaction with the Assistive Technology Services (SATS) instrument among adult users of powered wheelchairs (...

  17. Intra-Rater, Inter-Rater and Test-Retest Reliability of an Instrumented Timed Up and Go (iTUG Test in Patients with Parkinson's Disease.

    Directory of Open Access Journals (Sweden)

    Rob C van Lummel

    Full Text Available The "Timed Up and Go" (TUG is a widely used measure of physical functioning in older people and in neurological populations, including Parkinson's Disease. When using an inertial sensor measurement system (instrumented TUG [iTUG], the individual components of the iTUG and the trunk kinematics can be measured separately, which may provide relevant additional information.The aim of this study was to determine intra-rater, inter-rater and test-retest reliability of the iTUG in patients with Parkinson's Disease.Twenty eight PD patients, aged 50 years or older, were included. For the iTUG the DynaPort Hybrid (McRoberts, The Hague, The Netherlands was worn at the lower back. The device measured acceleration and angular velocity in three directions at a rate of 100 samples/s. Patients performed the iTUG five times on two consecutive days. Repeated measurements by the same rater on the same day were used to calculate intra-rater reliability. Repeated measurements by different raters on the same day were used to calculate intra-rater and inter-rater reliability. Repeated measurements by the same rater on different days were used to calculate test-retest reliability.Nineteen ICC values (15% were ≥ 0.9 which is considered as excellent reliability. Sixty four ICC values (49% were ≥ 0.70 and < 0.90 which is considered as good reliability. Thirty one ICC values (24% were ≥ 0.50 and < 0.70, indicating moderate reliability. Sixteen ICC values (12% were ≥ 0.30 and < 0.50 indicating poor reliability. Two ICT values (2% were < 0.30 indicating very poor reliability.In conclusion, in patients with Parkinson's disease the intra-rater, inter-rater, and test-retest reliability of the individual components of the instrumented TUG (iTUG was excellent to good for total duration and for turning durations, and good to low for the sub durations and for the kinematics of the SiSt and StSi. The results of this fully automated analysis of instrumented TUG movements

  18. Uncertainty propagation and sensitivity analysis in system reliability assessment via unscented transformation

    International Nuclear Information System (INIS)

    Rocco Sanseverino, Claudio M.; Ramirez-Marquez, José Emmanuel

    2014-01-01

    The reliability of a system, notwithstanding it intended function, can be significantly affected by the uncertainty in the reliability estimate of the components that define the system. This paper implements the Unscented Transformation to quantify the effects of the uncertainty of component reliability through two approaches. The first approach is based on the concept of uncertainty propagation, which is the assessment of the effect that the variability of the component reliabilities produces on the variance of the system reliability. This assessment based on UT has been previously considered in the literature but only for system represented through series/parallel configuration. In this paper the assessment is extended to systems whose reliability cannot be represented through analytical expressions and require, for example, Monte Carlo Simulation. The second approach consists on the evaluation of the importance of components, i.e., the evaluation of the components that most contribute to the variance of the system reliability. An extension of the UT is proposed to evaluate the so called “main effects” of each component, as well to assess high order component interaction. Several examples with excellent results illustrate the proposed approach. - Highlights: • Simulation based approach for computing reliability estimates. • Computation of reliability variance via 2n+1 points. • Immediate computation of component importance. • Application to network systems

  19. Validity and reliability of instruments aimed at measuring Evidence-Based Practice in Physical Therapy: a systematic review of the literature.

    Science.gov (United States)

    Fernández-Domínguez, Juan Carlos; Sesé-Abad, Albert; Morales-Asencio, Jose Miguel; Oliva-Pascual-Vaca, Angel; Salinas-Bueno, Iosune; de Pedro-Gómez, Joan Ernest

    2014-12-01

    Our goal is to compile and analyse the characteristics - especially validity and reliability - of all the existing international tools that have been used to measure evidence-based clinical practice in physiotherapy. A systematic review conducted with data from exclusively quantitative-type studies synthesized in narrative format. An in-depth search of the literature was conducted in two phases: initial, structured, electronic search of databases and also journals with summarized evidence; followed by a residual-directed search in the bibliographical references of the main articles found in the primary search procedure. The studies included were assigned to members of the research team who acted as peer reviewers. Relevant information was extracted from each of the selected articles using a template that included the general characteristics of the instrument as well as an analysis of the quality of the validation processes carried out, by following the criteria of Terwee. Twenty-four instruments were found to comply with the review screening criteria; however, in all cases, they were found to be limited as regards the 'constructs' included. Besides, they can all be seen to be lacking as regards comprehensiveness associated to the validation process of the psychometric tests used. It seems that what constitutes a rigorously developed assessment instrument for EBP in physical therapy continues to be a challenge. © 2014 John Wiley & Sons, Ltd.

  20. [An instrument in Spanish to evaluate the performance of clinical teachers by students].

    Science.gov (United States)

    Bitran, Marcela; Mena, Beltrán; Riquelme, Arnoldo; Padilla, Oslando; Sánchez, Ignacio; Moreno, Rodrigo

    2010-06-01

    The modernization of clinical teaching has called for the creation of faculty development programs, and the design of suitable instruments to evaluate clinical teachers' performance. To report the development and validation of an instrument in Spanish designed to measure the students' perceptions of their clinical teachers' performance and to provide them with feedback to improve their teaching practices. In a process that included the active participation of authorities, professors in charge of courses and internships, clinical teachers, students and medical education experts, we developed a 30-item questionnaire called MEDUC30 to evaluate the performance of clinical teachers by their students. The internal validity was assessed by factor analysis of 5214 evaluations of 265 teachers, gathered from 2004 to 2007. The reliability was measured with the Cronbach's alpha coefficient and the generalizability coefficient (g). MEDUC30 had good content and construct validity. Its internal structure was compatible with four factors: patient-centered teaching, teaching skills, assessment skills and learning climate, and it proved to be consistent with the structure anticipated by the theory. The scores were highly reliable (Cronbach's alpha: 0.97); five evaluations per teacher were sufficient to reach a reliability coefficient (g) of 0.8. MEDUC30 is a valid, reliable and useful instrument to evaluate the performance of clinical teachers. To our knowledge, this is the first instrument in Spanish for which solid validity and reliability evidences have been reported. We hope that MEDUC30 will be used to improve medical education in Spanish-speaking medical schools, providing teachers a specific feedback upon which to improve their pedagogical practice, and authorities with valuable information for the assessment of their faculty.

  1. Direct concurrent comparison of multiple pediatric acute asthma scoring instruments.

    Science.gov (United States)

    Johnson, Michael D; Nkoy, Flory L; Sheng, Xiaoming; Greene, Tom; Stone, Bryan L; Garvin, Jennifer

    2017-09-01

    Appropriate delivery of Emergency Department (ED) treatment to children with acute asthma requires clinician assessment of acute asthma severity. Various clinical scoring instruments exist to standardize assessment of acute asthma severity in the ED, but their selection remains arbitrary due to few published direct comparisons of their properties. Our objective was to test the feasibility of directly comparing properties of multiple scoring instruments in a pediatric ED. Using a novel approach supported by a composite data collection form, clinicians categorized elements of five scoring instruments before and after initial treatment for 48 patients 2-18 years of age with acute asthma seen at the ED of a tertiary care pediatric hospital ED from August to December 2014. Scoring instruments were compared for inter-rater reliability between clinician types and their ability to predict hospitalization. Inter-rater reliability between clinician types was not different between instruments at any point and was lower (weighted kappa range 0.21-0.55) than values reported elsewhere. Predictive ability of most instruments for hospitalization was higher after treatment than before treatment (p < 0.05) and may vary between instruments after treatment (p = 0.054). We demonstrate the feasibility of comparing multiple clinical scoring instruments simultaneously in ED clinical practice. Scoring instruments had higher predictive ability for hospitalization after treatment than before treatment and may differ in their predictive ability after initial treatment. Definitive conclusions about the best instrument or meaningful comparison between instruments will require a study with a larger sample size.

  2. Developing an Instrument of Scientific Literacy Assessment on the Cycle Theme

    Science.gov (United States)

    Rusilowati, Ani; Kurniawati, Lina; Nugroho, Sunyoto E.; Widiyatmoko, Arif

    2016-01-01

    The purpose of this study is to develop scientific literacy evaluation instrument that tested its validity, reliability, and characteristics to measure the skill of student's scientific literacy used four scientific literacy, categories as follow:science as a body of knowledge (category A), science as a way of thinking (category B), science as a…

  3. The Trojan Lifetime Champions Health Survey: Development, Validity, and Reliability

    Science.gov (United States)

    Sorenson, Shawn C.; Romano, Russell; Scholefield, Robin M.; Schroeder, E. Todd; Azen, Stanley P.; Salem, George J.

    2015-01-01

    Context Self-report questionnaires are an important method of evaluating lifespan health, exercise, and health-related quality of life (HRQL) outcomes among elite, competitive athletes. Few instruments, however, have undergone formal characterization of their psychometric properties within this population. Objective To evaluate the validity and reliability of a novel health and exercise questionnaire, the Trojan Lifetime Champions (TLC) Health Survey. Design Descriptive laboratory study. Setting A large National Collegiate Athletic Association Division I university. Patients or Other Participants A total of 63 university alumni (age range, 24 to 84 years), including former varsity collegiate athletes and a control group of nonathletes. Intervention(s) Participants completed the TLC Health Survey twice at a mean interval of 23 days with randomization to the paper or electronic version of the instrument. Main Outcome Measure(s) Content validity, feasibility of administration, test-retest reliability, parallel-form reliability between paper and electronic forms, and estimates of systematic and typical error versus differences of clinical interest were assessed across a broad range of health, exercise, and HRQL measures. Results Correlation coefficients, including intraclass correlation coefficients (ICCs) for continuous variables and κ agreement statistics for ordinal variables, for test-retest reliability averaged 0.86, 0.90, 0.80, and 0.74 for HRQL, lifetime health, recent health, and exercise variables, respectively. Correlation coefficients, again ICCs and κ, for parallel-form reliability (ie, equivalence) between paper and electronic versions averaged 0.90, 0.85, 0.85, and 0.81 for HRQL, lifetime health, recent health, and exercise variables, respectively. Typical measurement error was less than the a priori thresholds of clinical interest, and we found minimal evidence of systematic test-retest error. We found strong evidence of content validity, convergent

  4. Scale for positive aspects of caregiving experience: development, reliability, and factor structure.

    Science.gov (United States)

    Kate, N; Grover, S; Kulhara, P; Nehra, R

    2012-06-01

    OBJECTIVE. To develop an instrument (Scale for Positive Aspects of Caregiving Experience [SPACE]) that evaluates positive caregiving experience and assess its psychometric properties. METHODS. Available scales which assess some aspects of positive caregiving experience were reviewed and a 50-item questionnaire with a 5-point rating was constructed. In all, 203 primary caregivers of patients with severe mental disorders were asked to complete the questionnaire. Internal consistency, test-retest reliability, cross-language reliability, split-half reliability, and face validity were evaluated. Principal component factor analysis was run to assess the factorial validity of the scale. RESULTS. The scale developed as part of the study was found to have good internal consistency, test-retest reliability, cross-language reliability, split-half reliability, and face validity. Principal component factor analysis yielded a 4-factor structure, which also had good test-retest reliability and cross-language reliability. There was a strong correlation between the 4 factors obtained. CONCLUSION. The SPACE developed as part of this study has good psychometric properties.

  5. Review of Reliability Assessment of Westinghouse SSPS Using SPC by WEC

    International Nuclear Information System (INIS)

    Kang, H. T.; Chung, H. Y.

    2007-01-01

    Westinghouse Electric Company (WEC) has accomplished the reliability assessment of Westinghouse Solid State Protection System (SSPS) in KORI no. 2, 3, 4, and YGN no. 1, 2. In their studies, it is reported that creating a cost-effective plan for improving the reliability of the SSPS and at KORI no. 2, 3 and 4, and YGN no. 1, 2 should be needed while reducing their maintenance cost. In this paper, we reviewed the reliability assessment of Westinghouse SSPS analyzed in two performance standards, availability, and the maintenance expense using Statistic Process Control (SPC). As a result, it is concluded all plants have several failures reported but no effect on the system's availability, and the maintenance expense analysis did not reduce the current maintenance expense by 30%. Therefore, overall review for the reliability assessment is that a new strategy for cost-effective plan and/or upgrade approach for improving the reliability of the aging Westinghouse SSPS should be needed

  6. Translation, reliability, and clinical utility of the Melbourne Assessment 2.

    Science.gov (United States)

    Gerber, Corinna N; Plebani, Anael; Labruyère, Rob

    2017-10-12

    The aims were to (i) provide a German translation of the Melbourne Assessment 2 (MA2), a quantitative test to measure unilateral upper limb function in children with neurological disabilities and (ii) to evaluate its reliability and aspects of clinical utility. After its translation into German and approval of the back translation by the original authors, the MA2 was performed and videotaped twice with 30 children with neuromotor disorders. For each participant, two raters scored the video of the first test for inter-rater reliability. To determine test-retest reliability, one rater additionally scored the video of the second test while the other rater repeated the scoring of the first video to evaluate intra-rater reliability. Time needed for rater training, test administration, and scoring was recorded. The four subscale scores showed excellent intra-, inter-rater, and test-retest reliability with intraclass correlation coefficients of 0.90-1.00 (95%-confidence intervals 0.78-1.00). Score items revealed substantial to almost perfect intra-rater reliability (weighted kappa k w  = 0.66-1.00) for the more affected side. Score item inter-rater and test-retest reliability of the same extremity were, with one exception, moderate to almost perfect (k w  = 0.42-0.97; k w  = 0.40-0.89). Furthermore, the MA2 was feasible and acceptable for patients and clinicians. The MA2 showed excellent subscale and moderate to almost perfect score item reliability. Implications for Rehabilitation There is a lack of high-quality studies about psychometric properties of upper limb measurement tools in the neuropediatric population. The Melbourne Assessment 2 is a promising tool for reliable measurement of unilateral upper limb movement quality in the neuropediatric population. The Melbourne Assessment 2 is acceptable and practicable to therapists and patients for routine use in clinical care.

  7. Complex method to calculate objective assessments of information systems protection to improve expert assessments reliability

    Science.gov (United States)

    Abdenov, A. Zh; Trushin, V. A.; Abdenova, G. A.

    2018-01-01

    The paper considers the questions of filling the relevant SIEM nodes based on calculations of objective assessments in order to improve the reliability of subjective expert assessments. The proposed methodology is necessary for the most accurate security risk assessment of information systems. This technique is also intended for the purpose of establishing real-time operational information protection in the enterprise information systems. Risk calculations are based on objective estimates of the adverse events implementation probabilities, predictions of the damage magnitude from information security violations. Calculations of objective assessments are necessary to increase the reliability of the proposed expert assessments.

  8. Adaptation of WHOQOL as health-related quality of life instrument to develop a vision-specific instrument.

    Directory of Open Access Journals (Sweden)

    Dandona Lalit

    2000-01-01

    Full Text Available The WHOQOL instrument was adapted as a health-related QOL instrument for a population-based epidemiologic study of eye diseases in southern India, the Andhra Pradesh Eye Disease Study (APEDS. A follow-up question was added to each item in WHOQOL to determine whether the decrease in QOL was due to any health reasons including eye-related reasons. Modifications in WHOQOL and translation in local language were done through the use of the focus groups including health professionals and people not related to health care. The modified instrument has 28 items across 6 domains of the WHOQOL and was translated into the local language, Telugu, using the pragmatic approach. It takes 10-20 minutes to be administered by a trained interviewer. Reliability was within acceptable range. This health-related QOL instrument is being used in the population-based study APEDS to develop a vision-specific QOL instrument which could potentially be used to assess the impact of visual impairment on QOL across different cultures and for use in evaluating eye-care interventions. This health-related QOL instrument could also be used to develop other disease-specific instruments as it allows assessment of the extent to which various aspects of QOL are affected by a variety of health problems.

  9. Assessment of the Reliability of Concrete Bridges

    DEFF Research Database (Denmark)

    Middleton, C. R.; Thoft-Christensen, Palle

    Although there has been a considerable amount of research into different aspects of concrete bridge reliability, it has still not been widely adopted in professional practice other than in the development and calibration of codes. This situation appears to be changing as there has been a signific......Although there has been a considerable amount of research into different aspects of concrete bridge reliability, it has still not been widely adopted in professional practice other than in the development and calibration of codes. This situation appears to be changing as there has been...... adopted to assist in achieving this goal. Rather than review the specific research on this subject this paper examines a number of key issues related to the practical application of reliability analysis to the assessment of concrete bridges....

  10. Assessing environmental features related to mental health: a reliability study of visual streetscape images.

    Science.gov (United States)

    Wu, Yu-Tzu; Nash, Paul; Barnes, Linda E; Minett, Thais; Matthews, Fiona E; Jones, Andy; Brayne, Carol

    2014-10-22

    An association between depressive symptoms and features of built environment has been reported in the literature. A remaining research challenge is the development of methods to efficiently capture pertinent environmental features in relevant study settings. Visual streetscape images have been used to replace traditional physical audits and directly observe the built environment of communities. The aim of this work is to examine the inter-method reliability of the two audit methods for assessing community environments with a specific focus on physical features related to mental health. Forty-eight postcodes in urban and rural areas of Cambridgeshire, England were randomly selected from an alphabetical list of streets hosted on a UK property website. The assessment was conducted in July and August 2012 by both physical and visual image audits based on the items in Residential Environment Assessment Tool (REAT), an observational instrument targeting the micro-scale environmental features related to mental health in UK postcodes. The assessor used the images of Google Street View and virtually "walked through" the streets to conduct the property and street level assessments. Gwet's AC1 coefficients and Bland-Altman plots were used to compare the concordance of two audits. The results of conducting the REAT by visual image audits generally correspond to direct observations. More variations were found in property level items regarding physical incivilities, with broad limits of agreement which importantly lead to most of the variation in the overall REAT score. Postcodes in urban areas had lower consistency between the two methods than rural areas. Google Street View has the potential to assess environmental features related to mental health with fair reliability and provide a less resource intense method of assessing community environments than physical audits.

  11. Reliability and Validity of the "Decision-Making Skills Instrument for Children"

    Science.gov (United States)

    Pekdogan, Serpil; Ulutas, Ilkay

    2016-01-01

    The purpose of this study is to develop a valid and reliable data collection tool to assess the decision-making skills of children at the age of 5 to 6. The study group is composed of 300 children attending independent pre-schools located in the central district of Amasya province and their parents. In the study, four-factor and 29-item…

  12. Test-retest reliability and cross validation of the functioning everyday with a wheelchair instrument.

    Science.gov (United States)

    Mills, Tamara L; Holm, Margo B; Schmeler, Mark

    2007-01-01

    The purpose of this study was to establish the test-retest reliability and content validity of an outcomes tool designed to measure the effectiveness of seating-mobility interventions on the functional performance of individuals who use wheelchairs or scooters as their primary seating-mobility device. The instrument, Functioning Everyday With a Wheelchair (FEW), is a questionnaire designed to measure perceived user function related to wheelchair/scooter use. Using consumer-generated items, FEW Beta Version 1.0 was developed and test-retest reliability was established. Cross-validation of FEW Beta Version 1.0 was then carried out with five samples of seating-mobility users to establish content validity. Based on the content validity study, FEW Version 2.0 was developed and administered to seating-mobility consumers to examine its test-retest reliability. FEW Beta Version 1.0 yielded an intraclass correlation coefficient (ICC) Model (3,k) of .92, p content validity results revealed that FEW Beta Version 1.0 captured 55% of seating-mobility goals reported by consumers across five samples. FEW Version 2.0 yielded ICC(3,k) = .86, p content validity of FEW Version 2.0 was confirmed. FEW Beta Version 1.0 and FEW Version 2.0 were highly stable in their measurement of participants' seating-mobility goals over a 1-week interval.

  13. Predicting child maltreatment: A meta-analysis of the predictive validity of risk assessment instruments.

    Science.gov (United States)

    van der Put, Claudia E; Assink, Mark; Boekhout van Solinge, Noëlle F

    2017-11-01

    Risk assessment is crucial in preventing child maltreatment since it can identify high-risk cases in need of child protection intervention. Despite widespread use of risk assessment instruments in child welfare, it is unknown how well these instruments predict maltreatment and what instrument characteristics are associated with higher levels of predictive validity. Therefore, a multilevel meta-analysis was conducted to examine the predictive accuracy of (characteristics of) risk assessment instruments. A literature search yielded 30 independent studies (N=87,329) examining the predictive validity of 27 different risk assessment instruments. From these studies, 67 effect sizes could be extracted. Overall, a medium significant effect was found (AUC=0.681), indicating a moderate predictive accuracy. Moderator analyses revealed that onset of maltreatment can be better predicted than recurrence of maltreatment, which is a promising finding for early detection and prevention of child maltreatment. In addition, actuarial instruments were found to outperform clinical instruments. To bring risk and needs assessment in child welfare to a higher level, actuarial instruments should be further developed and strengthened by distinguishing risk assessment from needs assessment and by integrating risk assessment with case management. Copyright © 2017 Elsevier Ltd. All rights reserved.

  14. Clinical assessment of scapular positioning in musicians: an intertester reliability study.

    Science.gov (United States)

    Struyf, Filip; Nijs, Jo; De Coninck, Kris; Giunta, Marco; Mottram, Sarah; Meeusen, Romain

    2009-01-01

    The reliability of the measurement of the distance between the posterior border of the acromion and the wall and the reliability of the modified lateral scapular slide test have not been studied. Overall, the reliability of the clinical tools used to assess scapular positioning has not been studied in musicians. To examine the intertester reliability of scapular observation and 2 clinical tests for the assessment of scapular positioning in musicians. Intertester reliability study. University research laboratory. Thirty healthy student musicians at a single university. Two assessors performed a standardized observation protocol, the measurement of the distance between the posterior border of the acromion and the wall, and the modified lateral scapular slide test. Each assessor was blinded to the other's findings. The intertester reliability coefficients (kappa) for the observation in relaxed position, during unloaded movement, and during loaded movement were 0.41, 0.63, and 0.36, respectively. The kappa values for the observation of tilting and winging at rest were 0.48 and 0.42, respectively; during unloaded movement, the kappa values were 0.52 and 0.78, respectively; and with a 1-kg load, the kappa values were 0.24 and 0.50, respectively. The intraclass correlation coefficient (ICC) of the measurement of the acromial distance was 0.72 in relaxed position and 0.75 with the participant actively retracting both shoulders. The ICCs for the modified lateral scapular slide test varied between 0.63 and 0.58. Our results demonstrated that the modified lateral scapular slide test was not a reliable tool to assess scapular positioning in these participants. Our data indicated that scapular observation in the relaxed position and during unloaded abduction in the frontal plane was a reliable assessment tool. The reliability of the measurement of the distance between the posterior border of the acromion and the wall in healthy musicians was moderate.

  15. Developing Instrumentation for Assessing Creativity in Engineering Design

    Science.gov (United States)

    Denson, Cameron D.; Buelin, Jennifer K.; Lammi, Matthew D.; D'Amico, Susan

    2015-01-01

    A perceived inability to assess creative attributes of students' work has often precluded creativity instruction in the classroom. The Consensual Assessment Technique (CAT) has shown promise in a variety of domains for its potential as a valid and reliable means of creativity assessment. Relying upon an operational definition of creativity and a…

  16. Reliable and Valid Assessment of Point-of-care Ultrasonography

    DEFF Research Database (Denmark)

    Todsen, Tobias; Tolsgaard, Martin Grønnebæk; Olsen, Beth Härstedt

    2015-01-01

    physicians' OSAUS scores with diagnostic accuracy. RESULTS: The generalizability coefficient was high (0.81) and a D-study demonstrated that 1 assessor and 5 cases would result in similar reliability. The construct validity of the OSAUS scale was supported by a significant difference in the mean scores......OBJECTIVE: To explore the reliability and validity of the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale for point-of-care ultrasonography (POC US) performance. BACKGROUND: POC US is increasingly used by clinicians and is an essential part of the management of acute surgical...... conditions. However, the quality of performance is highly operator-dependent. Therefore, reliable and valid assessment of trainees' ultrasonography competence is needed to ensure patient safety. METHODS: Twenty-four physicians, representing novices, intermediates, and experts in POC US, scanned 4 different...

  17. The Friedrich-Lively Instrument to Assess the Impact of Schizophrenia on Siblings (FLIISS): Part I--instrument construction.

    Science.gov (United States)

    Friedrich, Rose Marie; Lively, Sonja; Rubenstein, Linda; Buckwalter, Kathleen

    2002-01-01

    Siblings of persons with schizophrenia may provide primary or secondary care for their sibling and support to parents who are primary care givers. These siblings experience stress and the accompanying sequelae of decreased quality of life, grief, chronic illness, and symptoms of depression and anxiety. Comprehensive measures of sibling stress have not been published. Before interventions to help siblings can be created, health professionals need such an assessment. The Friedrich-Lively Instrument to Assess the Impact of Schizophrenia on Siblings (FLIISS) has been developed to meet the needs of both health workers and siblings. The instrument has a strong conceptual basis adapted from Pearlin's model of stress in caregiving and was preceded by pilot work that evaluated both quantitative and qualitative data. In this article, (Part I), the development of the instrument and its relationships to the conceptual model are described. Health professionals can use the instrument for collecting information that will increase their ability to identify sources of stress faced by siblings of persons with schizophrenia, and from this assessment they may develop interventions for this underserved population.

  18. Validation of the Organizational Culture Assessment Instrument

    Science.gov (United States)

    Heritage, Brody; Pollock, Clare; Roberts, Lynne

    2014-01-01

    Organizational culture is a commonly studied area in industrial/organizational psychology due to its important role in workplace behaviour, cognitions, and outcomes. Jung et al.'s [1] review of the psychometric properties of organizational culture measurement instruments noted many instruments have limited validation data despite frequent use in both theoretical and applied situations. The Organizational Culture Assessment Instrument (OCAI) has had conflicting data regarding its psychometric properties, particularly regarding its factor structure. Our study examined the factor structure and criterion validity of the OCAI using robust analysis methods on data gathered from 328 (females = 226, males = 102) Australian employees. Confirmatory factor analysis supported a four factor structure of the OCAI for both ideal and current organizational culture perspectives. Current organizational culture data demonstrated expected reciprocally-opposed relationships between three of the four OCAI factors and the outcome variable of job satisfaction but ideal culture data did not, thus indicating possible weak criterion validity when the OCAI is used to assess ideal culture. Based on the mixed evidence regarding the measure's properties, further examination of the factor structure and broad validity of the measure is encouraged. PMID:24667839

  19. Validation of the organizational culture assessment instrument.

    Directory of Open Access Journals (Sweden)

    Brody Heritage

    Full Text Available Organizational culture is a commonly studied area in industrial/organizational psychology due to its important role in workplace behaviour, cognitions, and outcomes. Jung et al.'s [1] review of the psychometric properties of organizational culture measurement instruments noted many instruments have limited validation data despite frequent use in both theoretical and applied situations. The Organizational Culture Assessment Instrument (OCAI has had conflicting data regarding its psychometric properties, particularly regarding its factor structure. Our study examined the factor structure and criterion validity of the OCAI using robust analysis methods on data gathered from 328 (females = 226, males = 102 Australian employees. Confirmatory factor analysis supported a four factor structure of the OCAI for both ideal and current organizational culture perspectives. Current organizational culture data demonstrated expected reciprocally-opposed relationships between three of the four OCAI factors and the outcome variable of job satisfaction but ideal culture data did not, thus indicating possible weak criterion validity when the OCAI is used to assess ideal culture. Based on the mixed evidence regarding the measure's properties, further examination of the factor structure and broad validity of the measure is encouraged.

  20. Psychometric evaluation of the Sibling Cancer Needs Instrument (SCNI): an instrument to assess the psychosocial unmet needs of young people who are siblings of cancer patients.

    Science.gov (United States)

    Patterson, P; McDonald, F E J; Butow, P; White, K J; Costa, D S J; Millar, B; Bell, M L; Wakefield, C E; Cohn, R J

    2014-03-01

    The current study sought to establish the psychometric properties of the revised Sibling Cancer Needs Instrument (SCNI) when completed by young people who have a brother or sister with cancer. The participants were 106 young people aged between 12 and 24 who had a living brother or sister diagnosed with any type or stage of cancer in the last 5 years. They were recruited from multiple settings. The initial step in determining the dimensional structure of the questionnaire was exploratory factor analysis and further assessment followed using Rasch analysis. Construct validity and test-retest reliability (n = 17) were also assessed. The final SCNI has 45 items and seven domains: information; practical assistance; "time out" and recreation; feelings; support (friends and other young people); understanding from my family; and sibling relationship. There was a reasonable spread of responses across the scale for every item. Rasch analysis results suggested that overall, respondents used the scale consistently. Support for construct validity was provided by the correlations between psychological distress and the SCNI domains. The internal consistency was good to excellent; Cronbach's alphas ranged from 0.78 to 0.94. The test-retest reliability of the overall measure is 0.88. The SCNI is the first measure of psychosocial unmet needs which has been developed for young people who have a brother or sister with cancer. The sound psychometric properties allow the instrument to be used with confidence. The measure will provide a substantial clinical benefit in highlighting the unmet needs of this population to assist with the prioritisation of targeted supportive care services and evaluating the impact of interventions targeted at siblings.

  1. Health Literacy Screening Instruments for eHealth Applications: A Systematic Review

    Science.gov (United States)

    Collins, Sarah A.; Currie, Leanne M.; Bakken, Suzanne; Vawdrey, David K.; Stone, Patricia W.

    2012-01-01

    Objective To systematically review current health literacy (HL) instruments for use in consumer-facing and mobile health information technology screening and evaluation tools. Design The databases, PubMed, OVID, Google Scholar, Cochrane Library and Science Citation Index, were searched for health literacy assessment instruments using the terms “health”, “literacy”, “computer-based,” and “psychometrics”. All instruments identified by this method were critically appraised according to their reported psychometric properties and clinical feasibility. Results Eleven different health literacy instruments were found. Screening questions, such as asking a patient about his/her need for assistance in navigating health information, were evaluated in 7 different studies and are promising for use as a valid, reliable, and feasible computer-based approach to identify patients that struggle with low health literacy. However, there was a lack of consistency in the types of screening questions proposed. There is also a lack of information regarding the psychometric properties of computer-based health literacy instruments. Limitations Only English language health literacy assessment instruments were reviewed and analyzed. Conclusions Current health literacy screening tools demonstrate varying benefits depending on the context of their use. In many cases, it seems that a single screening question may be a reliable, valid, and feasible means for establishing health literacy. A combination of screening questions that assess health literacy and technological literacy may enable tailoring eHealth applications to user needs. Further research should determine the best screening question(s) and the best synthesis of various instruments’ content and methodologies for computer-based health literacy screening and assessment. PMID:22521719

  2. The reliability and validity of the Turkish version of the Neuropsychiatric Inventory-Clinician.

    Science.gov (United States)

    Sahin Cankurtaran, Eylem; Danişman, Mustafa; Tutar, Hasan; Ulusoy Kaymak, Semra

    2015-01-01

    The Neuropsychiatric Inventory-Clinician (NPI-C) scale is one of the best-known scales for evaluating the behavioral and psychological symptoms of dementia. This study aimed to assess the reliability and validity of the Turkish version of the NPI-C scale in patients with Alzheimer disease (AD). The NPI-C scale was administered to 125 patients with AD. For reliability, both Cronbach's α and interrater reliability were analyzed. The Behavioral Pathology in Alzheimer's Disease (BEHAVE-AD) scale was applied for validity and, in addition, the Mini Mental State Examination (MMSE), Instrumental Activities of Daily Living (IADL) scale, and Disability Assessment of Dementia (DAD) scale were completed. The Turkish version of the NPI-C scale showed high internal consistency (Cronbach's α = 0.75) and mostly good interrater reliability. Assessments of validity showed that the NPI-C and corresponding BEHAVE-AD domains were found to be significantly correlated, between 0.925 and 0.195. Moreover, the correlations between NPI-C and MMSE were significant for all domains except the dysphoria, anxiety, and elation/euphoria domains. When we conducted a correlation analysis of NPI-C with IADL, all domains were statistically significantly correlated except aggression, anxiety, elation/euphoria, and dysphoria. The Turkish version of the NPI-C scale was found to be a reliable and valid instrument to assess neuropsychiatric symptoms in Turkish elderly subjects with AD.

  3. Measuring decision quality: psychometric evaluation of a new instrument for breast cancer surgery

    Directory of Open Access Journals (Sweden)

    Sepucha Karen R

    2012-06-01

    Full Text Available Abstract Background The purpose of this paper is to examine the acceptability, feasibility, reliability and validity of a new decision quality instrument that assesses the extent to which patients are informed and receive treatments that match their goals. Methods Cross-sectional mail survey of recent breast cancer survivors, providers and healthy controls and a retest survey of survivors. The decision quality instrument includes knowledge questions and a set of goals, and results in two scores: a breast cancer surgery knowledge score and a concordance score, which reflects the percentage of patients who received treatments that match their goals. Hypotheses related to acceptability, feasibility, discriminant validity, content validity, predictive validity and retest reliability of the survey instrument were examined. Results We had responses from 440 eligible patients, 88 providers and 35 healthy controls. The decision quality instrument was feasible to implement in this study, with low missing data. The knowledge score had good retest reliability (intraclass correlation coefficient = 0.70 and discriminated between providers and patients (mean difference 35%, p  Conclusions The decision quality instrument met the criteria of feasibility, reliability, discriminant and content validity in this sample. Additional research to examine performance of the instrument in prospective studies and more diverse populations is needed.

  4. Validity and Reliability of Persian Version of Johns Hopkins Fall Risk Assessment Tool among Aged People

    Directory of Open Access Journals (Sweden)

    hadi hojati

    2018-04-01

    Full Text Available Background & Aim: It is crucial to identify aged patients in risk of falls in clinical settings. Johns Hopkins Fall Risk Assessment Tool (JHFRAT is one of most applied international instrument to assess elderly patients for the risk of falls. The aim of this study was to evaluate reliability and internal consistency of the JHFRAT. Methods & Materials: In this cross-sectional study for validity assessment of the tool, WHO’s standard protocol was applied for translation-back translation of the tool. Face and content validity of the tool was confirmed by ten person of expert faculty members for its applicability in clinical setting. In this pilot study, the inclusion criteria were being 60 or more years old, hospitalized in the last 8 hours prior to assessment and in proper cognitive condition assessed by MMSE. Subjects of the study were (n=70 elderly patients who were newly hospitalized in Shahroud Emam Hossein Hospital. Data were analyzed using SPSS software- version 16. Internal consistency of the tool was calculated by Cronbach’s alpha. Results: According to the results of the study Persian version of JHFRAT was a valid tool for application on clinical setting. The Persian version of the tool had Cronbach’s alpha equal to 0/733. Conclusion: Based on the findings of the current study, it can be concluded that Persian version of the JHFRAT is a valid and reliable tool to be applied for assessment of elderly senior citizens on admission in any clinical settings.

  5. The Revised Child Anxiety and Depression Scale: A systematic review and reliability generalization meta-analysis.

    Science.gov (United States)

    Piqueras, Jose A; Martín-Vivar, María; Sandin, Bonifacio; San Luis, Concepción; Pineda, David

    2017-08-15

    Anxiety and depression are among the most common mental disorders during childhood and adolescence. Among the instruments for the brief screening assessment of symptoms of anxiety and depression, the Revised Child Anxiety and Depression Scale (RCADS) is one of the more widely used. Previous studies have demonstrated the reliability of the RCADS for different assessment settings and different versions. The aims of this study were to examine the mean reliability of the RCADS and the influence of the moderators on the RCADS reliability. We searched in EBSCO, PsycINFO, Google Scholar, Web of Science, and NCBI databases and other articles manually from lists of references of extracted articles. A total of 146 studies were included in our meta-analysis. The RCADS showed robust internal consistency reliability in different assessment settings, countries, and languages. We only found that reliability of the RCADS was significantly moderated by the version of RCADS. However, these differences in reliability between different versions of the RCADS were slight and can be due to the number of items. We did not examine factor structure, factorial invariance across gender, age, or country, and test-retest reliability of the RCADS. The RCADS is a reliable instrument for cross-cultural use, with the advantage of providing more information with a low number of items in the assessment of both anxiety and depression symptoms in children and adolescents. Copyright © 2017. Published by Elsevier B.V.

  6. Assessing the Impact of Imperfect Diagnosis on Service Reliability

    DEFF Research Database (Denmark)

    Grønbæk, Lars Jesper; Schwefel, Hans-Peter; Kjærgaard, Jens Kristian

    2010-01-01

    , representative diagnosis performance metrics have been defined and their closed-form solutions obtained for the Markov model. These equations enable model parameterization from traces of implemented diagnosis components. The diagnosis model has been integrated in a reliability model assessing the impact...... of the diagnosis functions for the studied reliability problem. In a simulation study we finally analyze trade-off properties of diagnosis heuristics from literature, map them to the analytic Markov model, and investigate its suitability for service reliability optimization....

  7. In vitro and in vivo evaluations of three computer-aided shade matching instruments.

    Science.gov (United States)

    Yuan, Kun; Sun, Xiang; Wang, Fu; Wang, Hui; Chen, Ji-hua

    2012-01-01

    This study evaluated the accuracy and reliability of three computer-aided shade matching instruments (Shadepilot, VITA Easyshade, and ShadeEye NCC) using both in vitro and in vivo models. The in vitro model included the measurement of five VITA Classical shade guides. The in vivo model utilized three instruments to measure the central region of the labial surface of maxillary right central incisors of 85 people. The accuracy and reliability of the three instruments in these two evaluating models were calculated. Significant differences were observed in the accuracy of instruments both in vitro and in vivo. No significant differences were found in the reliability of instruments between and within the in vitro and the in vivo groups. VITA Easyshade was significantly different in accuracy between in vitro and in vivo models, while no significant difference was found for the other two instruments. Shadepilot was the only instrument tested in the present study that showed high accuracy and reliability both in vitro and in vivo. Significant differences were observed in the L*a*b* values of the 85 natural teeth measured using three instruments in the in vivo assessment. The pair-agreement rates of shade matching among the three instruments ranged from 37.7% to 48.2%, and the incidence of identical shade results shared by all three instruments was 25.9%. As different L*a*b* values and shade matching results were reported for the same tooth, a combination of the evaluated shade matching instruments and visual shade confirmation is recommended for clinical use.

  8. Development of a radiology faculty appraisal instrument by using critical incident interviewing.

    Science.gov (United States)

    Collins, J; Albanese, M A; Thakor, S K; Propeck, P A; Scanlan, K A

    1997-12-01

    To develop a valid and reliable radiology faculty appraisal instrument based on scientific methods. Fifteen radiology residents participated in critical incident interviewing. During a 1-hour interview, a resident was asked to describe five incidents each of effective and ineffective faculty behavior. Two investigators independently listened to the tape-recorded interviews, and two different investigators sorted the incidents into broad categories. A faculty appraisal instrument was developed by listing similar incidents under broad categories. A five-point rating scale was applied to each item. Content validity was assessed by resident and faculty critique of the appraisal instrument. A total of 168 incidents of faculty behavior were generated. The frequency with which similar incidents were reported was recorded. The most common behaviors reported were related to staff expertise and teaching. Interjudge reliability was good, as determined by computing K indices of agreement (overall K = 0.59). There was good agreement regarding instrument content validity among residents but not among faculty. Residents supported the use of the new appraisal instrument, but further tests of validity and reliability and faculty acceptance of the instrument will determine its usefulness as a tool for monitoring faculty teaching performance and making decisions regarding faculty promotion.

  9. Selecting instruments for assessing psychological wellbeing in Afghan and Kurdish refugee groups

    Directory of Open Access Journals (Sweden)

    Sulaiman-Hill Cheryl MR

    2010-09-01

    Full Text Available Abstract Background Afghan and Iraqi refugees comprise nearly half of all those currently under United Nations protection. As many of them will eventually be resettled in countries outside the region of origin, their long term health and settlement concerns are of relevance to host societies, and will be a likely focus for future research. Since Australia and New Zealand have both accepted refugees for many years and have dedicated, but different settlement and immigration policies, a study comparing the resettlement of two different refugee groups in these countries was undertaken. The purpose of this article is to describe the instrument selection for this study assessing mental health and psychological well being with Afghan and Kurdish former refugees, in particular to address linguistic considerations and translated instrument availability. A summary of instruments previously used with refugee and migrant groups from the Middle East region is presented to assist other researchers, before describing the three instruments ultimately selected for the quantitative component of our study. Findings The Kessler-10 Psychological Distress Scale (K10, General Perceived Self-Efficacy Scale (GPSE, and Personal Well-Being Index (PWI all showed good reliability (Cronbach's alphas of 0.86, 0.89 and 0.83 respectively for combined language versions and ease of use even for pre-literate participants, with the sample of 193 refugees, although some concepts in the GPSE proved problematic for a small number of respondents. Farsi was the language of choice for the majority of Afghan participants, while most of the Kurds chose to complete English versions in addition to Farsi. No one used Arabic or Turkish translations. Participants settled less than ten years were more likely to complete questionnaires in Farsi. Descriptive summary statistics are presented for each instrument with results split by gender, refugee group and language version completed. Conclusion

  10. Reliability Assessment for Low-cost Unmanned Aerial Vehicles

    Science.gov (United States)

    Freeman, Paul Michael

    Existing low-cost unmanned aerospace systems are unreliable, and engineers must blend reliability analysis with fault-tolerant control in novel ways. This dissertation introduces the University of Minnesota unmanned aerial vehicle flight research platform, a comprehensive simulation and flight test facility for reliability and fault-tolerance research. An industry-standard reliability assessment technique, the failure modes and effects analysis, is performed for an unmanned aircraft. Particular attention is afforded to the control surface and servo-actuation subsystem. Maintaining effector health is essential for safe flight; failures may lead to loss of control incidents. Failure likelihood, severity, and risk are qualitatively assessed for several effector failure modes. Design changes are recommended to improve aircraft reliability based on this analysis. Most notably, the control surfaces are split, providing independent actuation and dual-redundancy. The simulation models for control surface aerodynamic effects are updated to reflect the split surfaces using a first-principles geometric analysis. The failure modes and effects analysis is extended by using a high-fidelity nonlinear aircraft simulation. A trim state discovery is performed to identify the achievable steady, wings-level flight envelope of the healthy and damaged vehicle. Tolerance of elevator actuator failures is studied using familiar tools from linear systems analysis. This analysis reveals significant inherent performance limitations for candidate adaptive/reconfigurable control algorithms used for the vehicle. Moreover, it demonstrates how these tools can be applied in a design feedback loop to make safety-critical unmanned systems more reliable. Control surface impairments that do occur must be quickly and accurately detected. This dissertation also considers fault detection and identification for an unmanned aerial vehicle using model-based and model-free approaches and applies those

  11. Attention to gender in communication skills assessment instruments in medical education: a review.

    Science.gov (United States)

    Dielissen, Patrick; Bottema, Ben; Verdonk, Petra; Lagro-Janssen, Toine

    2011-03-01

    Gender is increasingly regarded as an important factor in doctor-patient communication education. This review aims to assess if and how gender is addressed by current assessment instruments for communication skills in medical education. In 2009 at Radboud University Nijmegen Medical Centre, an online search was conducted in the bibliographic databases PubMed, PsycINFO and ERIC for references about communication skills assessment instruments designed to be completed by trained faculty staff and used in medical education. The search strategy used the following search terms: 'consultation skills'; 'doctor-patient communication'; 'physician-patient relations'; 'medical education'; 'instruments'; 'measurement', and 'assessment'. Papers published between January 1999 and June 2009 were included. The assessment instruments identified were analysed for gender-specific content. The search yielded 21 communication skills assessment instruments. Only two of the 17 checklists obtained explicitly considered gender as a communication-related issue. Only six of 21 manuals considered gender in any way and none gave specific details to explain which aspects of communication behaviour should be assessed with regard to gender. Very few communication assessment instruments in medical education focus on gender. Nevertheless, interest exists in using gender in communication skills assessment. The criteria for and purpose of assessing gender in communication skills in medical education are yet to be clarified. © Blackwell Publishing Ltd 2011.

  12. High level issues in reliability quantification of safety-critical software

    International Nuclear Information System (INIS)

    Kim, Man Cheol

    2012-01-01

    For the purpose of developing a consensus method for the reliability assessment of safety-critical digital instrumentation and control systems in nuclear power plants, several high level issues in reliability assessment of the safety-critical software based on Bayesian belief network modeling and statistical testing are discussed. Related to the Bayesian belief network modeling, the relation between the assessment approach and the sources of evidence, the relation between qualitative evidence and quantitative evidence, how to consider qualitative evidence, and the cause-consequence relation are discussed. Related to the statistical testing, the need of the consideration of context-specific software failure probabilities and the inability to perform a huge number of tests in the real world are discussed. The discussions in this paper are expected to provide a common basis for future discussions on the reliability assessment of safety-critical software. (author)

  13. Reliability and validity of the Repeatable Battery for the Assessment of Neuropsychological Status in community-dwelling elderly

    Science.gov (United States)

    Cheng, Yan; Wu, Wenyuan; Wang, Jiaqi; Feng, Wei; Li, Chunbo

    2011-01-01

    Introduction The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) is a widely used screening instrument in neuropsychological assessment and is a brief, individually administered measure. The present study aims to assess the reliability and validity of the Chinese version of the RBANS in community-dwelling elderly. Material and methods All subjects come from the community-dwelling elderly in Shanghai, China. They completed a questionnaire concerning demographic information, the mini-mental state examination (MMSE) and the Chinese version of the RBANS. To test for internal consistency, Cronbach's α was calculated for all six RBANS indices. Correlations between each of the RBANS and MMSE subtests were conducted to measure the concurrent validity. A confirmatory factor analysis (CFA) was conducted to test the construct validity. Results The final sample of participants included 236 community-dwelling elderly. The mean total score on the RBANS was 86.02 (±14.19). The RBANS total score showed strong internal consistency (r = 0.806), and the coefficient α value for each of the RBANS scales ranged from 0.142 to 0.727. The total RBANS score was highly correlated with that of the MMSE (r = 0.594, pvalidity in a community-dwelling elderly sample. It may be a useful screening instrument for conducting cognitive assessments in community-dwelling elderly. PMID:22291831

  14. Characteristics of protective instrumentation

    International Nuclear Information System (INIS)

    Reichart, G.

    1982-01-01

    Protective Instrumentation (PI) for Nuclear Power Plants (NPP) is a general term for an highly reliable instrumentation, which provides information for keeping the system within safe limits, for initation of countermeasures in the case of an incident or for mitigation of consequences of an accident. In German NPPs one can find a hierarchical structure of protective instrumentation, wherein the Reactor Protection System (RPS) has the highest priority. To meet the reliability requirements different design principles are used, like - redundancy - diversity - fail safe - decoupling. The presentation gives an overview about the different design principles and characterizes their reliability aspects. As an example for the technical realization the RPS of a German NPP is discussed in some detail. Furthermore some information about other type of PI is given and reliability aspects of the interaction of operating personell with these systems are mentioned. (orig.)

  15. A Bayesian belief nets based quantitative software reliability assessment for PSA: COTS case study

    International Nuclear Information System (INIS)

    Eom, H. S.; Sung, T. Y.; Jeong, H. S.; Park, J. H.; Kang, H. G.; Lee, K. Y.; Park, J. K

    2002-03-01

    Current reliability assessments of safety critical software embedded in the digital systems in nuclear power plants are based on the rule-based qualitative assessment methods. Then recently practical needs require the quantitative features of software reliability for Probabilistic Safety Assessment (PSA) that is one of important methods being used in assessing the whole safety of nuclear power plant. But conventional quantitative software reliability assessment methods are not enough to get the necessary results in assessing the safety critical software used in nuclear power plants. Thus, current reliability assessment methods for these digital systems exclude the software part or use arbitrary values for the software reliability in the assessment. This reports discusses a Bayesian Belief Nets (BBN) based quantification method that models current qualitative software assessment in formal way and produces quantitative results required for PSA. Commercial Off-The-Shelf (COTS) software dedication process that KAERI developed was applied to the discussed BBN based method for evaluating the plausibility of the proposed method in PSA

  16. The Malay version of the Early Childhood Oral Health Impact Scale (Malay-ECOHIS)--assessing validity and reliability.

    Science.gov (United States)

    Hashim, Azlina N; Yusof, Zamros Y M; Esa, Rashidah

    2015-11-25

    The Early Childhood Oral Health Impact Scale (ECOHIS) is used to assess oral impacts on the quality of life of preschool aged children and their families. The objective of this study was to perform a cross-cultural adaptation of the ECOHIS into Malay and assess its psychometric properties. The cross-cultural adaptation of ECOHIS into Malay comprised of translating the ECOHIS into the Malay language (Malay-ECOHIS) by experts followed by face validation of the Malay-ECOHIS by a group of mothers. The Malay-ECOHIS was back translated into English and this was compared with the original ECOHIS. Minor changes were made to the Malay-ECOHIS before it was finalised. The Malay-ECOHIS' psychometric properties were assessed in terms of construct, convergent and discriminant validity as well as internal and test-retest reliability based on two separate studies involving 127 parents of 4-6 year old preschool children followed by oral examinations of 860 preschool children from 25 kindergartens from two districts in Selangor state, Malaysia. Non-parametric statistics were used to assess the relationships between the Malay-ECOHIS and the subjective and clinical outcome measures. The Cronbach's alpha was 0.83 and the weighted Kappa was 0.95 (intraclass correlation = 0.94). The Malay-ECOHIS demonstrated significant associations with different subjective and normative measures, i.e. levels of oral health satisfaction, perceived oral health status, perceived oral health need, toothache experience, pattern of dental attendance, and caries status of preschool children. These significant associations supported its construct, convergent and discriminant validity as well as internal and test-retest reliability. This study showed that the Malay-ECOHIS is a valid and reliable instrument to assess the negative impacts of oral disorders/conditions on the quality of life of 4-6 year old preschool children and their families in Malaysia.

  17. Using Longitudinal Scales Assessment for Instrumental Music Students

    Science.gov (United States)

    Simon, Samuel H.

    2014-01-01

    In music education, current assessment trends emphasize student reflection, tracking progress over time, and formative as well as summative measures. This view of assessment requires instrumental music educators to modernize their approaches without interfering with methods that have proven to be successful. To this end, the Longitudinal Scales…

  18. A Valid and Reliable Tool to Assess Nursing Students` Clinical Performance

    OpenAIRE

    Mehrnoosh Pazargadi; Tahereh Ashktorab; Sharareh Khosravi; Hamid Alavi majd

    2013-01-01

    Background: The necessity of a valid and reliable assessment tool is one of the most repeated issues in nursing students` clinical evaluation. But it is believed that present tools are not mostly valid and can not assess students` performance properly.Objectives: This study was conducted to design a valid and reliable assessment tool for evaluating nursing students` performance in clinical education.Methods: In this methodological study considering nursing students` performance definition; th...

  19. Translation, cross-cultural adaptation and reliability of the German version of the migraine disability assessment (MIDAS) questionnaire.

    Science.gov (United States)

    Benz, Thomas; Lehmann, Susanne; Gantenbein, Andreas R; Sandor, Peter S; Stewart, Walter F; Elfering, Achim; Aeschlimann, André G; Angst, Felix

    2018-03-09

    The Migraine Disability Assessment (MIDAS) is a brief questionnaire and measures headache-related disability. This study aimed to translate and cross-culturally adapt the original English version of the MIDAS to German and to test its reliability. The standardized translation process followed international guidelines. The pre-final version was tested for clarity and comprehensibility by 34 headache sufferers. Test-retest reliability of the final version was quantified by 36 headache patients completing the MIDAS twice with an interval of 48 h. Reliability was determined by intraclass correlation coefficients and internal consistency by Cronbach's α. All steps of the translation process were followed, documented and approved by the developer of the MIDAS. The expert committee discussed in detail the complex phrasing of the questions that refer to one to another, especially exclusion of headache-days from one item to the next. The German version contains more active verb sentences and prefers the perfect to the imperfect tense. The MIDAS scales intraclass correlation coefficients ranged from 0.884 to 0.994 and was 0.991 (95% CI: 0.982-0.995) for the MIDAS total score. Cronbach's α for the MIDAS as a whole was 0.69 at test and 0.67 at retest. The translation process was challenged by the comprehensibility of the questionnaire. The German version of the MIDAS is a highly reliable instrument for assessing headache related disability with moderate internal consistency. Provided validity testing of the German MIDAS is successful, it can be recommended for use in clinical practice as well as in research.

  20. Project W-314 DST and DCRT instrument and control systems, initial assessment

    International Nuclear Information System (INIS)

    Acree, C.D.

    1996-01-01

    This report contains an assessment of the instrument and control systems in the Double Shell Tank Farms and the 244-A DCRT. The assessment report contains data from physical inspection activities and an overall engineering assessment of the instruments and control systems in use in the Double Shell Tanks

  1. The Stanmore Nursing Assessment of Psychological Status: Understanding the emotions of patients with spinal cord injury.

    Science.gov (United States)

    Smyth, Carol; Spada, Marcantonio M; Coultry-Keane, Katherine; Ikkos, George

    2016-09-01

    Research has shown that individuals who have sustained a spinal cord injury can experience strong and abrupt variations in their emotional state; however no instrument for nurses has been developed to assess these patients' psychological status. To develop a brief, reliable instrument to enable nurses to accurately assess, record and respond to spinal cord injury patients' psychological status. In Phase 1, semi-structured interviews were conducted with spinal cord injury patients (n = 10) and nurses (n = 10) which were audio recorded, transcribed and thematically analysed to develop the instrument. The instrument's content validity was then ensured via independent expert review. In Phase 2, the instrument was trialled on 80 spinal cord injury patients to determine inter-rater reliability, internal consistency and test-retest reliability. In Phase 1, four core themes (emotional impact, coping, relationships and assessment) were identified together with a number of related sub-themes. In Phase 2, the instrument was shown to have excellent inter-rater reliability, acceptable internal consistency and satisfactory test re-test reliability. Subsequently a rating sheet, user manual and prompt card were produced. The new instrument, the Stanmore Nursing Assessment of Psychological Status, was shown to be valid and reliable. It is anticipated that training nurses to use this instrument may help to enhance good emotional care of patients.

  2. Reliability and validity in a nutshell.

    Science.gov (United States)

    Bannigan, Katrina; Watson, Roger

    2009-12-01

    To explore and explain the different concepts of reliability and validity as they are related to measurement instruments in social science and health care. There are different concepts contained in the terms reliability and validity and these are often explained poorly and there is often confusion between them. To develop some clarity about reliability and validity a conceptual framework was built based on the existing literature. The concepts of reliability, validity and utility are explored and explained. Reliability contains the concepts of internal consistency and stability and equivalence. Validity contains the concepts of content, face, criterion, concurrent, predictive, construct, convergent (and divergent), factorial and discriminant. In addition, for clinical practice and research, it is essential to establish the utility of a measurement instrument. To use measurement instruments appropriately in clinical practice, the extent to which they are reliable, valid and usable must be established.

  3. Developing the Coach Analysis and Intervention System (CAIS): establishing validity and reliability of a computerised systematic observation instrument.

    Science.gov (United States)

    Cushion, Christopher; Harvey, Stephen; Muir, Bob; Nelson, Lee

    2012-01-01

    We outline the evolution of a computerised systematic observation tool and describe the process for establishing the validity and reliability of this new instrument. The Coach Analysis and Interventions System (CAIS) has 23 primary behaviours related to physical behaviour, feedback/reinforcement, instruction, verbal/non-verbal, questioning and management. The instrument also analyses secondary coach behaviour related to performance states, recipient, timing, content and questioning/silence. The CAIS is a multi-dimensional and multi-level mechanism able to provide detailed and contextualised data about specific coaching behaviours occurring in complex and nuanced coaching interventions and environments that can be applied to both practice sessions and competition.

  4. Quality of Work: Validation of a New Instrument in Three Languages.

    Science.gov (United States)

    Steffgen, Georges; Kohl, Diane; Reese, Gerhard; Happ, Christian; Sischka, Philipp

    2015-11-26

    A new instrument to measure quality of work was developed in three languages (German, French and Luxembourgish) and validated in a study of employees working in Luxembourg. A representative sample (n = 1529) was taken and exploratory factor analysis revealed a six-factor solution for the 21-item instrument (satisfaction and respect, mobbing, mental strain at work, cooperation, communication and feedback, and appraisal). Reliability analysis showed satisfying reliability for all six factors and the total questionnaire. In order to examine the construct validity of the new instrument, regression analyses were conducted to test whether the instrument predicted work characteristics’ influence on three components of well-being -- burnout, psychological stress and maladaptive coping behaviors. The present validation offers a trilingual inventory for measuring quality of work that may be used, for example, as an assessment tool or for testing the effectiveness of interventions.

  5. A heuristic-based approach for reliability importance assessment of energy producers

    International Nuclear Information System (INIS)

    Akhavein, A.; Fotuhi Firuzabad, M.

    2011-01-01

    Reliability of energy supply is one of the most important issues of service quality. On one hand, customers usually have different expectations for service reliability and price. On the other hand, providing different level of reliability at load points is a challenge for system operators. In order to take reasonable decisions and obviate reliability implementation difficulties, market players need to know impacts of their assets on system and load-point reliabilities. One tool to specify reliability impacts of assets is the criticality or reliability importance measure by which system components can be ranked based on their effect on reliability. Conventional methods for determination of reliability importance are essentially on the basis of risk sensitivity analysis and hence, impose prohibitive calculation burden in large power systems. An approach is proposed in this paper to determine reliability importance of energy producers from perspective of consumers or distribution companies in a composite generation and transmission system. In the presented method, while avoiding immense computational burden, the energy producers are ranked based on their rating, unavailability and impact on power flows in the lines connecting to the considered load points. Study results on the IEEE reliability test system show successful application of the proposed method. - Research highlights: → Required reliability level at load points is a concern in modern power systems. → It is important to assess reliability importance of energy producers or generators. → Generators can be ranked based on their impacts on power flow to a selected area. → Ranking of generators is an efficient tool to assess their reliability importance.

  6. Reliability of Lactation Assessment Tools Applied to Overweight and Obese Women.

    Science.gov (United States)

    Chapman, Donna J; Doughty, Katherine; Mullin, Elizabeth M; Pérez-Escamilla, Rafael

    2016-05-01

    The interrater reliability of lactation assessment tools has not been evaluated in overweight/obese women. This study aimed to compare the interrater reliability of 4 lactation assessment tools in this population. A convenience sample of 45 women (body mass index > 27.0) was videotaped while breastfeeding (twice daily on days 2, 4, and 7 postpartum). Three International Board Certified Lactation Consultants independently rated each videotaped session using 4 tools (Infant Breastfeeding Assessment Tool [IBFAT], modified LATCH [mLATCH], modified Via Christi [mVC], and Riordan's Tool [RT]). For each day and tool, we evaluated interrater reliability with 1-way repeated-measures analyses of variance, intraclass correlation coefficients (ICCs), and percentage absolute agreement between raters. Analyses of variance showed significant differences between raters' scores on day 2 (all scales) and day 7 (RT). Intraclass correlation coefficient values reflected good (mLATCH) to excellent reliability (IBFAT, mVC, and RT) on days 2 and 7. All day 4 ICCs reflected good reliability. The ICC for mLATCH was significantly lower than all others on day 2 and was significantly lower than IBFAT (day 7). Percentage absolute interrater agreement for scale components ranged from 31% (day 2: observable swallowing, RT) to 92% (day 7: IBFAT, fixing; and mVC, latch time). Swallowing scores on all scales had the lowest levels of interrater agreement (31%-64%). We demonstrated differences in the interrater reliability of 4 lactation assessment tools when applied to overweight/obese women, with the lowest values observed on day 4. Swallowing assessment was particularly unreliable. Researchers and clinicians using these scales should be aware of the differences in their psychometric behavior. © The Author(s) 2015.

  7. Validation of a method for assessing resident physicians' quality improvement proposals.

    Science.gov (United States)

    Leenstra, James L; Beckman, Thomas J; Reed, Darcy A; Mundell, William C; Thomas, Kris G; Krajicek, Bryan J; Cha, Stephen S; Kolars, Joseph C; McDonald, Furman S

    2007-09-01

    Residency programs involve trainees in quality improvement (QI) projects to evaluate competency in systems-based practice and practice-based learning and improvement. Valid approaches to assess QI proposals are lacking. We developed an instrument for assessing resident QI proposals--the Quality Improvement Proposal Assessment Tool (QIPAT-7)-and determined its validity and reliability. QIPAT-7 content was initially obtained from a national panel of QI experts. Through an iterative process, the instrument was refined, pilot-tested, and revised. Seven raters used the instrument to assess 45 resident QI proposals. Principal factor analysis was used to explore the dimensionality of instrument scores. Cronbach's alpha and intraclass correlations were calculated to determine internal consistency and interrater reliability, respectively. QIPAT-7 items comprised a single factor (eigenvalue = 3.4) suggesting a single assessment dimension. Interrater reliability for each item (range 0.79 to 0.93) and internal consistency reliability among the items (Cronbach's alpha = 0.87) were high. This method for assessing resident physician QI proposals is supported by content and internal structure validity evidence. QIPAT-7 is a useful tool for assessing resident QI proposals. Future research should determine the reliability of QIPAT-7 scores in other residency and fellowship training programs. Correlations should also be made between assessment scores and criteria for QI proposal success such as implementation of QI proposals, resident scholarly productivity, and improved patient outcomes.

  8. The Italian version of cognitive function instrument (CFI): reliability and validity in a cohort of healthy elderly.

    Science.gov (United States)

    Chipi, Elena; Frattini, Giulia; Eusebi, Paolo; Mollica, Anita; D'Andrea, Katia; Russo, Mirella; Bernardelli, Alice; Montanucci, Chiara; Luchetti, Elisa; Calabresi, Paolo; Parnetti, Lucilla

    2018-01-01

    The Alzheimer's disease Cooperative Study (ADCS)-Cognitive Function Instrument (CFI) is a 14-item questionnaire administered to the subject and the referent, aimed at detecting early changes in cognitive and functional abilities in individuals without clinical impairment. It is used for monitoring annual variations in cognitive functioning in prevention trials. The aim of the present study was to validate the Italian version of the CFI. A consecutive series of 257 functionally independent subjects was recruited among relatives of patients or as volunteers. They were administered CFI and global cognition measurements: Mini-Mental Status Examination (MMSE) and Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). The reliability and criterion validity were comparable to the original in both self- and partner-report. Similarly to what reported in the original version, we found a corrected item-total correlation ranging between 0.38 and 0.54 in self-report and between 0.33 and 0.64 in partner-report. Cronbach's α was 0.77 (95% CI 0.72-0.83) in self-report and 0.78 (95% CI 0.73-0.84) in partner-report. Total partner- and self-report scores were significantly correlated (rS = 0.31, p reliability and validity of the Italian version of CFI. In order to definitely propose the use of CFI for tracking longitudinal changes of cognitive and functional abilities in subjects without clinical impairment, data from the follow-up of this cohort are needed.

  9. The Development and Validation of a Generic Instrument, QoDoS, for Assessing the Quality of Decision Making.

    Science.gov (United States)

    Donelan, Ronan; Walker, Stuart; Salek, Sam

    2016-01-01

    The impact of decision-making during the development and the regulatory review of medicines greatly influences the delivery of new medicinal products. Currently, there is no generic instrument that can be used to assess the quality of decision-making. This study describes the development of the Quality of Decision-Making Orientation Scheme QoDoS(©) instrument for appraising the quality of decision-making. Semi-structured interviews about decision-making were carried out with 29 senior decision makers from the pharmaceutical industry (10), regulatory authorities (9) and contract research organizations (10). The interviews offered a qualified understanding of the subjective decision-making approach, influences, behaviors and other factors that impact such processes for individuals and organizations involved in the delivery of new medicines. Thematic analysis of the transcribed interviews was carried out using NVivo8® software. Content validity was carried out using qualitative and quantitative data by an expert panel, which led to the developmental version of the QoDoS. Further psychometric evaluations were performed, including factor analysis, item reduction, reliability testing and construct validation. The thematic analysis of the interviews yielded a 94-item initial version of the QoDoS(©) with a 5-point Likert scale. The instrument was tested for content validity using a panel of experts for language clarity, completeness, relevance and scaling, resulting in a favorable agreement by panel members with an intra-class correlation coefficient value of 0.89 (95% confidence interval = 0.56, 0.99). A 76-item QoDoS(©) (version 2) emerged from content validation. Factor analysis produced a 47-item measure with four domains. The 47-item QoDoS(©) (version 3) showed high internal consistency (n = 120, Cronbach's alpha = 0.89), high reproducibility (n = 20, intra-class correlation = 0.77) and a mean completion time of 10 min. Reliability testing and construct

  10. Validation and reliability of the Turkish Utian Quality-of-Life Scale in postmenopausal women.

    Science.gov (United States)

    Abay, Halime; Kaplan, Sena

    2016-04-01

    There are a limited number of menopause-specific quality-of-life scales for the Turkish population. This study was conducted to evaluate the validity and reliability of the Turkish Utian Quality-of-Life Scale in postmenopausal women. The study group was comprised of 250 postmenopausal women who applied to a training and research hospital's menopause clinic in Turkey. A survey form and the Turkish Utian quality-of-Life Scale were used to collect data, and the Turkish version of Short Form-36 was used to evaluate reliability with an equivalent form. Language-validity, content-validity, and construct-validity methods were used to assess the validity of the scale, and Cronbach's α coefficient calculation and the equivalent-form reliability methods were used to assess the reliability of the scale. The Turkish Utian Quality-of-Life Scale was determined to be a valid and reliable instrument for measuring the quality of life of postmenopausal women. Confirmatory factor analysis demonstrates that the instrument fits well with 23 items and a four-factor model. The Cronbach's α coefficient for the quality-of-life domains were as follows: 0.88 overall, 0.79 health, 0.78 emotional, 0.76 sexual, and 0.75 occupational. Reliability of the instrument was confirmed through significant correlations between scores on the Turkish version of the Utian Quality-of-Life Scale and the Turkish version of the Short Form-36 (r = 0.745, P measuring quality of life during menopause.

  11. Development and validation of an instrument to assess future orientation and resilience in adolescence.

    Science.gov (United States)

    Di Maggio, Ilaria; Ginevra, Maria Cristina; Nota, Laura; Soresi, Salvatore

    2016-08-01

    The study is aimed at providing the development and initial validation of the Design My Future (DMF), which may be administered in career counseling and research activities to assess adolescents' future orientation and resilience. Two studies with two independent samples of Italian adolescents were conducted to examine psychometric requisites of DMF. Specifically, in the first study, after developing items and examined the content validity, the factorial structure, reliability and discriminant validity of the DMF were tested. In the second study, the measurement invariance across gender, conducing a sequence of nested CFA models, was evaluated. Results showed good psychometric support for the instrument with Italian adolescents. Copyright © 2016 The Foundation for Professionals in Services for Adolescents. Published by Elsevier Ltd. All rights reserved.

  12. the research quality plus (rq+) assessment instrument

    International Development Research Centre (IDRC) Digital Library (Canada)

    Thomas Schwandt

    THE RESEARCH QUALITY PLUS (RQ+) ASSESSMENT INSTRUMENT ... consistent way to allow for further meta-analysis about research quality over time. ... Addresses complex and integrative problems, requiring systems-based approaches ..... benefits or financial costs for participants that might not be appropriate in the ...

  13. Development, validation, and utility of an instrument to assess core competencies in the Leadership Education in Neurodevelopmental and Related Disabilities (LEND) program.

    Science.gov (United States)

    Leff, Stephen S; Baum, Katherine T; Bevans, Katherine B; Blum, Nathan J

    2015-02-01

    To describe the development and psychometric evaluation of the Core Competency Measure (CCM), an instrument designed to assess professional competencies as defined by the Maternal Child Health Bureau (MCHB) and targeted by Leadership Education in Neurodevelopmental and Related Disabilities (LEND) programs. The CCM is a 44-item self-report measure comprised of six subscales to assess clinical, interdisciplinary, family-centered/cultural, community, research, and advocacy/policy competencies. The CCM was developed in an iterative fashion through participatory action research, and then nine cohorts of LEND trainees (N = 144) from 14 different disciplines completed the CCM during the first week of the training program. A 6-factor confirmatory factor analysis model was fit to data from the 44 original items. After three items were removed, the model adequately fit the data (comparative fit indices = .93, root mean error of approximation = .06) with all factor loadings exceeding .55. The measure was determined to be quite reliable as adequate internal consistency and test-retest reliability were found for each subscale. The instrument's construct validity was supported by expected differences in self-rated competencies among fellows representing various disciplines, and the convergent validity was supported by the pattern of inter-correlations between subscale scores. The CCM appears to be a reliable and valid measure of MCHB core competencies for our sample of LEND trainees. It provides an assessment of key training areas addressed by the LEND program. Although the measure was developed within only one LEND Program, with additional research it has the potential to serve as a standardized tool to evaluate the strengths and limitations of MCHB training, both within and between programs.

  14. Peer-Assessment Debriefing Instrument (PADI): Assessing Faculty Effectiveness in Simulation Education.

    Science.gov (United States)

    Saylor, Jennifer L; Wainwright, Susan F; Herge, E A; Pohlig, Ryan T

    2016-01-01

    Evaluating the debriefing skill of faculty in simulations is essential but often challenging. The Peer Assessment Debriefing Instrument (PADI) is a self and peer assessment designed to assess a debriefers' effectiveness and is grounded in current scientific debriefing literature and peer review methodology. This pilot study tested the PADI at three institutions during 10 different healthcare debriefing sessions. The PADI is useful to faculty and administrators of simulation centers by demonstrating ongoing quality improvement and as a guide to train new debriefers.

  15. A Structured Review of Generic and Specific Instruments for Measuring the Subjectively Assessed Quality of Life of Seniors

    Directory of Open Access Journals (Sweden)

    Monika Kacmarova

    2015-12-01

    Full Text Available The objective of the study is to offer a review of the instruments designed for measuring the subjectively assessed quality of life of seniors. At present it is possible to notice an increase of interest in the issue of the quality of life of specific groups of population; in addition, there is a large number of tools for its measuring. The aim of the present study is to provide a systematic review of generic and specific instruments for measuring quality of life of seniors which have been published in peer-reviewed journals and whose psychometric parameters have been verified. The search procedure formed a part of a larger retrieval search in which we analyzed 4829 abstracts in EBSCO and ProQuest Central full-text databases. We found 831 instruments which claimed to be measuring quality of life and were verified their reliability or validity. We identified 3 groups of instruments suitable for use in the senior age-group: generic methodologies applicable to adults in general, 7 generic tools and 9 specific tools designed exclusively for the senior age. The paper presents the measures designed for seniors who were analyzed and compared with regard to their psychometric parameters, purpose and theoretical framework utilized for their construction. In conclusion the authors of the study provide recommendations for the use of the selected methodologies for measuring the subjectively assessed quality of life of seniors.

  16. On the use of NDT Data for Reliability-Based Assessment of Existing Timber Structures

    DEFF Research Database (Denmark)

    Sousa, Hélder S.; Sørensen, John Dalsgaard; Kirkegaard, Poul Henning

    2013-01-01

    The objective of this paper is to address the possibilities of using non-destructive testing (NDT) data for updating information and obtaining adequate characterization of the reliability level of existing timber structures and, also, for assessing the evolution in time of performance...... of these structures when exposed to deterioration. By improving the knowledge upon the mechanical properties of timber, better and more substantiated decisions after a reliability safety assessment are aimed at. Bayesian methods are used to update the mechanical properties of timber and reliability assessment......, and information of NDT is also used to calibrate these models. The proposed approach is used for reliability assessment of different structural timber systems. Reliability of the structural system is assessed regarding the failure consequences of individual elements defined as key elements which were determined...

  17. The spiritual distress assessment tool: an instrument to assess spiritual distress in hospitalised elderly persons

    Directory of Open Access Journals (Sweden)

    Martin Estelle

    2010-12-01

    Full Text Available Abstract Background Although spirituality is usually considered a positive resource for coping with illness, spiritual distress may have a negative influence on health outcomes. Tools are needed to identify spiritual distress in clinical practice and subsequently address identified needs. This study describes the first steps in the development of a clinically acceptable instrument to assess spiritual distress in hospitalized elderly patients. Methods A three-step process was used to develop the Spiritual Distress Assessment Tool (SDAT: 1 Conceptualisation by a multidisciplinary group of a model (Spiritual Needs Model to define the different dimensions characterizing a patient's spirituality and their corresponding needs; 2 Operationalisation of the Spiritual Needs Model within geriatric hospital care leading to a set of questions (SDAT investigating needs related to each of the defined dimensions; 3 Qualitative assessment of the instrument's acceptability and face validity in hospital chaplains. Results Four dimensions of spirituality (Meaning, Transcendence, Values, and Psychosocial Identity and their corresponding needs were defined. A formalised assessment procedure to both identify and subsequently score unmet spiritual needs and spiritual distress was developed. Face validity and acceptability in clinical practice were confirmed by chaplains involved in the focus groups. Conclusions The SDAT appears to be a clinically acceptable instrument to assess spiritual distress in elderly hospitalised persons. Studies are ongoing to investigate the psychometric properties of the instrument and to assess its potential to serve as a basis for integrating the spiritual dimension in the patient's plan of care.

  18. Is the Frontal Assessment Battery reliable in ALS patients?

    NARCIS (Netherlands)

    Raaphorst, J.; Beeldman, E.; Jaeger, B.; Schmand, B.A.; Berg, L.H. van den; Weikamp, J.G.; Schelhaas, H.J.; Visser, M. de; Haan, R.J. de

    2013-01-01

    The assessment of frontal functions in ALS patients is important because of the overlap with the behavioural variant of frontotemporal dementia (bvFTD). We investigated the applicability and reliability of the Frontal Assessment Battery (FAB) within a cohort of predominantly prevalent ALS patients.

  19. [The "Mini-ICF-Rating for Mental Disorders (Mini-ICF-P)". A short instrument for the assessment of disabilities in mental disorders].

    Science.gov (United States)

    Linden, M; Baron, S

    2005-06-01

    Supplementary to the description of diseases at symptom level, the International Classification of Functioning, Disability and Health (ICF), edited by the WHO, for the first time enables a systematic description also at the level of disabilities and impairments. The Mini-ICF-Rating for Mental Disorders (Mini-ICF-P) is a short observer rating instrument for the assessment of disabilities, especially with regard to occupational functioning. The Mini-ICF-P was first evaluated empirically in 125 patients of a Department of Behavioural Medicine and Psychosomatics. Parallel-test reliability was r = 0.59. Correlates were found with cognitive and motivational variables and duration of sick leave from work. In summary, the Mini-ICF-P is a quick and practicable instrument.

  20. An approach for assessing ALWR passive safety system reliability

    International Nuclear Information System (INIS)

    Hake, T.M.

    1991-01-01

    Many of the advanced light water reactor (ALWR) concepts proposed for the next generation of nuclear power plants rely on passive rather than active systems to perform safety functions. Despite the reduced redundancy of the passive systems as compared to active systems in current plants, the assertion is that the overall safety of the plant is enhanced due to the much higher expected reliability of the passive systems. In order to investigate this assertion, a study is being conducted at Sandia National Laboratories to evaluate the reliability of ALWR passive safety features in the context of probabilistic risk assessment (PRA). The purpose of this paper is to provide a brief overview of the approach to this study. The quantification of passive system reliability is not as straightforward as for active systems, due to the lack of operating experience, and to the greater uncertainty in the governing physical phenomena. Thus, the adequacy of current methods for evaluating system reliability must be assessed, and alternatives proposed if necessary. For this study, the Westinghouse Advanced Passive 600 MWe reactor (AP600) was chosen as the advanced reactor for analysis, because of the availability of AP600 design information. This study compares the reliability of AP600 emergency cooling system with that of corresponding systems in a current generation reactor

  1. Development and Reliability Evaluation of the Movement Rating Instrument for Virtual Reality Video Game Play.

    Science.gov (United States)

    Levac, Danielle; Nawrotek, Joanna; Deschenes, Emilie; Giguere, Tia; Serafin, Julie; Bilodeau, Martin; Sveistrup, Heidi

    2016-06-01

    Virtual reality active video games are increasingly popular physical therapy interventions for children with cerebral palsy. However, physical therapists require educational resources to support decision making about game selection to match individual patient goals. Quantifying the movements elicited during virtual reality active video game play can inform individualized game selection in pediatric rehabilitation. The objectives of this study were to develop and evaluate the feasibility and reliability of the Movement Rating Instrument for Virtual Reality Game Play (MRI-VRGP). Item generation occurred through an iterative process of literature review and sample videotape viewing. The MRI-VRGP includes 25 items quantifying upper extremity, lower extremity, and total body movements. A total of 176 videotaped 90-second game play sessions involving 7 typically developing children and 4 children with cerebral palsy were rated by 3 raters trained in MRI-VRGP use. Children played 8 games on 2 virtual reality and active video game systems. Intraclass correlation coefficients (ICCs) determined intra-rater and interrater reliability. Excellent intrarater reliability was evidenced by ICCs of >0.75 for 17 of the 25 items across the 3 raters. Interrater reliability estimates were less precise. Excellent interrater reliability was achieved for far reach upper extremity movements (ICC=0.92 [for right and ICC=0.90 for left) and for squat (ICC=0.80) and jump items (ICC=0.99), with 9 items achieving ICCs of >0.70, 12 items achieving ICCs of between 0.40 and 0.70, and 4 items achieving poor reliability (close-reach upper extremity-ICC=0.14 for right and ICC=0.07 for left) and single-leg stance (ICC=0.55 for right and ICC=0.27 for left). Poor video quality, differing item interpretations between raters, and difficulty quantifying the high-speed movements involved in game play affected reliability. With item definition clarification and further psychometric property evaluation, the MRI

  2. Towards Developing a Quantitative Literacy/Reasoning Assessment Instrument

    Directory of Open Access Journals (Sweden)

    Eric Gaze

    2014-07-01

    Full Text Available This article reports on the development and implementation of a non-proprietary assessment instrument for Quantitative Literacy/Reasoning. This instrument was based on prior work by Bowdoin College, Colby-Sawyer College, and Wellesley College and was piloted in 2012 and 2013. This article presents a discussion of its development as well as the results of the pilot implementation. This work was supported by a TUES Type 1 grant from the National Science Foundation.

  3. Development and Validation of a Personality Assessment Instrument for Traditional Korean Medicine: Sasang Personality Questionnaire

    Directory of Open Access Journals (Sweden)

    Han Chae

    2012-01-01

    Full Text Available Objective. Sasang typology is a traditional Korean medicine based on the biopsychosocial perspectives of Neo-Confucianism and utilizes medical herbs and acupuncture for type-specific treatment. This study was designed to develop and validate the Sasang Personality Questionnaire (SPQ for future use in the assessment of personality based on Sasang typology. Design and Methods. We selected questionnaire items using internal consistency analysis and examined construct validity with explorative factor analysis using 245 healthy participants. Test-retest reliability as well as convergent validity were examined. Results. The 14-item SPQ showed acceptable internal consistency (Cronbach’s alpha=.817 and test-retest reliability (=.837. Three extracted subscales, SPQ-behavior, SPQ-emotionality, and SPQ-cognition, were found, explaining 55.77% of the total variance. The SPQ significantly correlated with Temperament and Character Inventory novelty seeking (=.462, harm avoidance (=−.390, and NEO Personality Inventory extraversion (=.629. The SPQ score of the So-Eum (24.43±4.93, Tae-Eum (27.33±5.88, and So-Yang (30.90±5.23 types were significantly different from each other (<.01. Conclusion. Current results demonstrated the reliability and validity of the SPQ and its subscales that can be utilized as an objective instrument for conducting personalized medicine research incorporating the biopsychosocial perspective.

  4. Remaining life prediction of I and C cables for reliability assessment of NPP systems

    International Nuclear Information System (INIS)

    Santhosh, T.V.; Ghosh, A.K.; Fernandes, B.G.

    2012-01-01

    Highlights: ► A framework for time dependent reliability prediction of I and C cables for use in PSA of NPP has been developed using stress–strength interference theory. ► The proposed methodology has been illustrated with the accelerated thermal aging data on a typical XLPE cable. ► The behavior of insulation resistance when the degradation process is linear or exponential has also been modeled. ► The reliability index or probability of failure obtained from this approach can be used in system reliability evaluation to account for cable aging for PSA of NPP. - Abstract: Instrumentation and control (I and C) cables are one of the most important components in nuclear power plants (NPPs) because they provide power to safety-related equipment and also to transmit signals to and from various controllers to perform safety operations. I and C cables in NPP are subjected to a variety of aging and degradation stressors that can produce immediate degradation or aging-related mechanisms causing the degradation of cable components over time. Although, there exits several life estimation techniques, currently there is no any standard methodology or an approach toward estimating the time dependent reliability of I and C cables that can be directly used in probabilistic safety assessment (PSA) applications. Hence, the objective of this study is to develop an approach to estimate and confirm the continued acceptable margin in cable insulation life over time subjected to aging. This paper presents a framework based on the structural reliability theory to quantify the life time of I and C cable subjecting to thermal aging. Since cross-linked polyethylene (XLPE) cables are extensively being used in Indian NPPs, the remaining life time evaluations have been carried out for a typical XLPE cable. However, the methodology can be extended to other cables such as polyvinyl chloride (PVC), ethylene propylene rubber (EPR), etc.

  5. Cross-sectional validation of the Aging Perceptions Questionnaire: a multidimensional instrument for assessing self-perceptions of aging.

    LENUS (Irish Health Repository)

    Barker, Maja

    2007-01-01

    BACKGROUND: Self-perceptions of aging have been implicated as independent predictors of functional disability and mortality in older adults. In spite of this, research on self-perceptions of aging is limited. One reason for this is the absence of adequate measures. Specifically, there is a need to develop a measure that is theoretically-derived, has good psychometric properties, and is multidimensional in nature. The present research seeks to address this need by adopting the Self-Regulation Model as a framework and using it to develop a comprehensive, multi-dimensional instrument for assessing self-perceptions of aging. This study describes the validation of this newly-developed instrument, the Aging Perceptions Questionnaire (APQ). METHODS: Participants were 2,033 randomly selected community-dwelling older (+65 yrs) Irish adults who completed the APQ alongside measures of physical and psychological health. The APQ assesses self-perceptions of aging along eight distinct domains or subscales; seven of these examine views about own aging, these are: timeline chronic, timeline cyclical, consequences positive, consequences negative, control positive, control negative, and emotional representations; the eighth domain is the identity domain and this examines the experience of health-related changes. RESULTS: Mokken scale analysis showed that the majority of items within the views about aging subscales were strongly scalable. Confirmatory factor analysis also indicated that the model provided a good fit for the data. Overall, subscales had good internal reliabilities. Hierarchical linear regression was conducted to investigate the independent contribution of APQ subscales to physical and psychological health and in doing so determine the construct validity of the APQ. Results showed that self-perceptions of aging were independently related to physical and psychological health. Mediation testing also supported a role for self-perceptions of aging as partial mediators in

  6. Reliability-based assessment of polyethylene pipe creep lifetime

    International Nuclear Information System (INIS)

    Khelif, Rabia; Chateauneuf, Alaa; Chaoui, Kamel

    2007-01-01

    Lifetime management of underground pipelines is mandatory for safe hydrocarbon transmission and distribution systems. The use of high-density polyethylene tubes subjected to internal pressure, external loading and environmental variations requires a reliability study in order to define the service limits and the optimal operating conditions. In service, the time-dependent phenomena, especially creep, take place during the pipe lifetime, leading to significant strength reduction. In this work, the reliability-based assessment of pipe lifetime models is carried out, in order to propose a probabilistic methodology for lifetime model selection and to determine the pipe safety levels as well as the most important parameters for pipeline reliability. This study is enhanced by parametric analysis on pipe configuration, gas pressure and operating temperature

  7. Reliability-based assessment of polyethylene pipe creep lifetime

    Energy Technology Data Exchange (ETDEWEB)

    Khelif, Rabia [LaMI-UBP and IFMA, Campus de Clermont-Fd, Les Cezeaux, BP 265, 63175 Aubiere Cedex (France); LR3MI, Departement de Genie Mecanique, Universite Badji Mokhtar, BP 12, Annaba 23000 (Algeria)], E-mail: rabia.khelif@ifma.fr; Chateauneuf, Alaa [LGC-University Blaise Pascal, Campus des Cezeaux, BP 206, 63174 Aubiere Cedex (France)], E-mail: alaa.chateauneuf@polytech.univ-bpclermont.fr; Chaoui, Kamel [LR3MI, Departement de Genie Mecanique, Universite Badji Mokhtar, BP 12, Annaba 23000 (Algeria)], E-mail: chaoui@univ-annaba.org

    2007-12-15

    Lifetime management of underground pipelines is mandatory for safe hydrocarbon transmission and distribution systems. The use of high-density polyethylene tubes subjected to internal pressure, external loading and environmental variations requires a reliability study in order to define the service limits and the optimal operating conditions. In service, the time-dependent phenomena, especially creep, take place during the pipe lifetime, leading to significant strength reduction. In this work, the reliability-based assessment of pipe lifetime models is carried out, in order to propose a probabilistic methodology for lifetime model selection and to determine the pipe safety levels as well as the most important parameters for pipeline reliability. This study is enhanced by parametric analysis on pipe configuration, gas pressure and operating temperature.

  8. Developing a new instrument to assess the impact of cancer in young adult survivors of childhood cancer.

    Science.gov (United States)

    Zebrack, Brad

    2009-09-01

    Thirty years of psychosocial oncology research have detailed issues having significant impact in both pediatric and adult populations; yet, few studies have captured the subtle and unique ways in which cancer impacts, disrupts and in some instances promotes the growth and development of adolescents and young adults with a cancer history. This paper reports the initiation of an effort to assess the impact of cancer in this young survivor population through the development of a new Impact of Cancer (IOC) instrument. 64 young adults aged 18-39 years and treated for a pediatric malignancy participated in face-to-face interviews and responded to questions prompting them to describe the impact of cancer on their physical, psychological, social and spiritual/existential well-being. Intent of analysis was to organize data into meaningful sub-categories from which to develop a set of candidate survey items that assess a range of problems, issues and changes that long-term survivors ascribe to their cancer experience. A total of 82 candidate survey items represented content across 11 topical domains including Body, Health and Body image, Treatment and Health Care, Having Children, Identity, Talking and Thinking About Cancer, Meaning of Cancer, Memory and Thinking, Finances, Family and Relationships, Socializing, and Life Goals. Assessing the instrument's psychometric properties in a large representative group of young cancer survivors is the next step for further development of such a measure. Once established, a valid and reliable Impact of Cancer instrument has the potential for identifying salient survivorship issues in a clinical setting.

  9. Assessing hospitals' clinical risk management: Development of a monitoring instrument

    Directory of Open Access Journals (Sweden)

    Pfeiffer Yvonne

    2010-12-01

    Full Text Available Abstract Background Clinical risk management (CRM plays a crucial role in enabling hospitals to identify, contain, and manage risks related to patient safety. So far, no instruments are available to measure and monitor the level of implementation of CRM. Therefore, our objective was to develop an instrument for assessing CRM in hospitals. Methods The instrument was developed based on a literature review, which identified key elements of CRM. These elements were then discussed with a panel of patient safety experts. A theoretical model was used to describe the level to which CRM elements have been implemented within the organization. Interviews with CRM practitioners and a pilot evaluation were conducted to revise the instrument. The first nationwide application of the instrument (138 participating Swiss hospitals was complemented by in-depth interviews with 25 CRM practitioners in selected hospitals, for validation purposes. Results The monitoring instrument consists of 28 main questions organized in three sections: 1 Implementation and organizational integration of CRM, 2 Strategic objectives and operational implementation of CRM at hospital level, and 3 Overview of CRM in different services. The instrument is available in four languages (English, German, French, and Italian. It allows hospitals to gather comprehensive and systematic data on their CRM practice and to identify areas for further improvement. Conclusions We have developed an instrument for assessing development stages of CRM in hospitals that should be feasible for a continuous monitoring of developments in this important area of patient safety.

  10. Assessing hospitals' clinical risk management: Development of a monitoring instrument.

    Science.gov (United States)

    Briner, Matthias; Kessler, Oliver; Pfeiffer, Yvonne; Wehner, Theo; Manser, Tanja

    2010-12-13

    Clinical risk management (CRM) plays a crucial role in enabling hospitals to identify, contain, and manage risks related to patient safety. So far, no instruments are available to measure and monitor the level of implementation of CRM. Therefore, our objective was to develop an instrument for assessing CRM in hospitals. The instrument was developed based on a literature review, which identified key elements of CRM. These elements were then discussed with a panel of patient safety experts. A theoretical model was used to describe the level to which CRM elements have been implemented within the organization. Interviews with CRM practitioners and a pilot evaluation were conducted to revise the instrument. The first nationwide application of the instrument (138 participating Swiss hospitals) was complemented by in-depth interviews with 25 CRM practitioners in selected hospitals, for validation purposes. The monitoring instrument consists of 28 main questions organized in three sections: 1) Implementation and organizational integration of CRM, 2) Strategic objectives and operational implementation of CRM at hospital level, and 3) Overview of CRM in different services. The instrument is available in four languages (English, German, French, and Italian). It allows hospitals to gather comprehensive and systematic data on their CRM practice and to identify areas for further improvement. We have developed an instrument for assessing development stages of CRM in hospitals that should be feasible for a continuous monitoring of developments in this important area of patient safety.

  11. Reliability and Validity of Selected PROMIS Measures in People with Rheumatoid Arthritis.

    Directory of Open Access Journals (Sweden)

    Susan J Bartlett

    Full Text Available To evaluate the reliability and validity of 11 PROMIS measures to assess symptoms and impacts identified as important by people with rheumatoid arthritis (RA.Consecutive patients (N = 177 in an observational study completed PROMIS computer adapted tests (CATs and a short form (SF assessing pain, fatigue, physical function, mood, sleep, and participation. We assessed test-test reliability and internal consistency using correlation and Cronbach's alpha. We assessed convergent validity by examining Pearson correlations between PROMIS measures and existing measures of similar domains and known groups validity by comparing scores across disease activity levels using ANOVA.Participants were mostly female (82% and white (83% with mean (SD age of 56 (13 years; 24% had ≤ high school, 29% had RA ≤ 5 years with 13% ≤ 2 years, and 22% were disabled. PROMIS Physical Function, Pain Interference and Fatigue instruments correlated moderately to strongly (rho's ≥ 0.68 with corresponding PROs. Test-retest reliability ranged from .725-.883, and Cronbach's alpha from .906-.991. A dose-response relationship with disease activity was evident in Physical Function with similar trends in other scales except Anger.These data provide preliminary evidence of reliability and construct validity of PROMIS CATs to assess RA symptoms and impacts, and feasibility of use in clinical care. PROMIS instruments captured the experiences of RA patients across the broad continuum of RA symptoms and function, especially at low disease activity levels. Future research is needed to evaluate performance in relevant subgroups, assess responsiveness and identify clinically meaningful changes.

  12. Feasibility and reliability of frailty assessment in the critically ill: a systematic review.

    Science.gov (United States)

    Pugh, Richard J; Ellison, Amy; Pye, Kate; Subbe, Christian P; Thorpe, Chris M; Lone, Nazir I; Clegg, Andrew

    2018-02-26

    For healthcare systems, an ageing population poses challenges in the delivery of equitable and effective care. Frailty assessment has the potential to improve care in the intensive care setting, but applying assessment tools in critical illness may be problematic. The aim of this systematic review was to evaluate evidence for the feasibility and reliability of frailty assessment in critical care. Our primary search was conducted in Medline, Medline In-process, EMBASE, CINAHL, PsycINFO, AMED, Cochrane Database of Systematic Reviews, and Web of Science (January 2001 to October 2017). We included observational studies reporting data on feasibility and reliability of frailty assessment in the critical care setting in patients 16 years and older. Feasibility was assessed in terms of timing of evaluation, the background, training and expertise required for assessors, and reliance upon proxy input. Reliability was assessed in terms of inter-rater reliability. Data from 11 study publications are included, representing 8 study cohorts and 7761 patients. Proxy involvement in frailty assessment ranged from 58 to 100%. Feasibility data were not well-reported overall, but the exclusion rate due to lack of proxy availability ranged from 0 to 45%, the highest rate observed where family involvement was mandatory and the assessment tool relatively complex (frailty index, FI). Conventional elements of frailty phenotype (FP) assessment required modification prior to use in two studies. Clinical staff tended to use a simple judgement-based tool, the clinical frailty scale (CFS). Inter-rater reliability was reported in one study using the CFS and although a good level of agreement was observed between clinician assessments, this was a small and single-centre study. Though of unproven reliability in the critically ill, CFS was the tool used most widely by critical care clinical staff. Conventional FP assessment required modification for general application in critical care, and an FI

  13. Mammography image assessment; validity and reliability of current scheme

    International Nuclear Information System (INIS)

    Hill, C.; Robinson, L.

    2015-01-01

    Mammographers currently score their own images according to criteria set out by Regional Quality Assurance. The criteria used are based on the ‘Perfect, Good, Moderate, Inadequate’ (PGMI) marking criteria established by the National Health Service Breast Screening Programme (NHSBSP) in their Quality Assurance Guidelines of 2006 1 . This document discusses the validity and reliability of the current mammography image assessment scheme. Commencing with a critical review of the literature this document sets out to highlight problems with the national approach to the use of marking schemes. The findings suggest that ‘PGMI’ scheme is flawed in terms of reliability and validity and is not universally applied across the UK. There also appear to be differences in schemes used by trainees and qualified mammographers. Initial recommendations are to be made in collaboration with colleagues within the National Health Service Breast Screening Programme (NHSBSP), Higher Education Centres, College of Radiographers and the Royal College of Radiologists in order to identify a mammography image appraisal scheme that is fit for purpose. - Highlights: • Currently no robust evidence based marking tools in use for the assessment of images in mammography. • Is current system valid, reliable and robust? • How can the current image assessment tool be improved? • Should students and qualified mammographers use the same tool? • What marking criteria are available for image assessment?

  14. Quality of Work: Validation of a New Instrument in Three Languages

    Directory of Open Access Journals (Sweden)

    Georges Steffgen

    2015-11-01

    Full Text Available Introduction and objective: A new instrument to measure quality of work was developed in three languages (German, French and Luxembourgish and validated in a study of employees working in Luxembourg. Methods and results: A representative sample (n = 1529 was taken and exploratory factor analysis revealed a six-factor solution for the 21-item instrument (satisfaction and respect, mobbing, mental strain at work, cooperation, communication and feedback, and appraisal. Reliability analysis showed satisfying reliability for all six factors and the total questionnaire. In order to examine the construct validity of the new instrument, regression analyses were conducted to test whether the instrument predicted work characteristics’ influence on three components of well-being—burnout, psychological stress and maladaptive coping behaviors. Conclusion: The present validation offers a trilingual inventory for measuring quality of work that may be used, for example, as an assessment tool or for testing the effectiveness of interventions.

  15. Quality of Work: Validation of a New Instrument in Three Languages

    Science.gov (United States)

    Steffgen, Georges; Kohl, Diane; Reese, Gerhard; Happ, Christian; Sischka, Philipp

    2015-01-01

    Introduction and objective: A new instrument to measure quality of work was developed in three languages (German, French and Luxembourgish) and validated in a study of employees working in Luxembourg. Methods and results: A representative sample (n = 1529) was taken and exploratory factor analysis revealed a six-factor solution for the 21-item instrument (satisfaction and respect, mobbing, mental strain at work, cooperation, communication and feedback, and appraisal). Reliability analysis showed satisfying reliability for all six factors and the total questionnaire. In order to examine the construct validity of the new instrument, regression analyses were conducted to test whether the instrument predicted work characteristics’ influence on three components of well-being—burnout, psychological stress and maladaptive coping behaviors. Conclusion: The present validation offers a trilingual inventory for measuring quality of work that may be used, for example, as an assessment tool or for testing the effectiveness of interventions. PMID:26703634

  16. Development of performance assessment instrument based contextual learning for measuring students laboratory skills

    Science.gov (United States)

    Susilaningsih, E.; Khotimah, K.; Nurhayati, S.

    2018-04-01

    The assessment of laboratory skill in general hasn’t specific guideline in assessment, while the individual assessment of students during a performance and skill in performing laboratory is still not been observed and measured properly. Alternative assessment that can be used to measure student laboratory skill is use performance assessment. The purpose of this study was to determine whether the performance assessment instrument that the result of research can be used to assess basic skills student laboratory. This research was conducted by the Research and Development. The result of the data analysis performance assessment instruments developed feasible to implement and validation result 62.5 with very good categories for observation sheets laboratory skills and all of the components with the very good category. The procedure is the preliminary stages of research and development stages. Preliminary stages are divided in two, namely the field studies and literature studies. The development stages are divided into several parts, namely 1) development of the type instrument, 2) validation by an expert, 3) a limited scale trial, 4) large-scale trials and 5) implementation of the product. The instrument included in the category of effective because 26 from 29 students have very high laboratory skill and high laboratory skill. The research of performance assessment instrument is standard and can be used to assess basic skill student laboratory.

  17. Inter-rater reliability of an observation-based ergonomics assessment checklist for office workers.

    Science.gov (United States)

    Pereira, Michelle Jessica; Straker, Leon Melville; Comans, Tracy Anne; Johnston, Venerina

    2016-12-01

    To establish the inter-rater reliability of an observation-based ergonomics assessment checklist for computer workers. A 37-item (38-item if a laptop was part of the workstation) comprehensive observational ergonomics assessment checklist comparable to government guidelines and up to date with empirical evidence was developed. Two trained practitioners assessed full-time office workers performing their usual computer-based work and evaluated the suitability of workstations used. Practitioners assessed each participant consecutively. The order of assessors was randomised, and the second assessor was blinded to the findings of the first. Unadjusted kappa coefficients between the raters were obtained for the overall checklist and subsections that were formed from question-items relevant to specific workstation equipment. Twenty-seven office workers were recruited. The inter-rater reliability between two trained practitioners achieved moderate to good reliability for all except one checklist component. This checklist has mostly moderate to good reliability between two trained practitioners. Practitioner Summary: This reliable ergonomics assessment checklist for computer workers was designed using accessible government guidelines and supplemented with up-to-date evidence. Employers in Queensland (Australia) can fulfil legislative requirements by using this reliable checklist to identify and subsequently address potential risk factors for work-related injury to provide a safe working environment.

  18. Training less-experienced faculty improves reliability of skills assessment in cardiac surgery.

    Science.gov (United States)

    Lou, Xiaoying; Lee, Richard; Feins, Richard H; Enter, Daniel; Hicks, George L; Verrier, Edward D; Fann, James I

    2014-12-01

    Previous work has demonstrated high inter-rater reliability in the objective assessment of simulated anastomoses among experienced educators. We evaluated the inter-rater reliability of less-experienced educators and the impact of focused training with a video-embedded coronary anastomosis assessment tool. Nine less-experienced cardiothoracic surgery faculty members from different institutions evaluated 2 videos of simulated coronary anastomoses (1 by a medical student and 1 by a resident) at the Thoracic Surgery Directors Association Boot Camp. They then underwent a 30-minute training session using an assessment tool with embedded videos to anchor rating scores for 10 components of coronary artery anastomosis. Afterward, they evaluated 2 videos of a different student and resident performing the task. Components were scored on a 1 to 5 Likert scale, yielding an average composite score. Inter-rater reliabilities of component and composite scores were assessed using intraclass correlation coefficients (ICCs) and overall pass/fail ratings with kappa. All components of the assessment tool exhibited improvement in reliability, with 4 (bite, needle holder use, needle angles, and hand mechanics) improving the most from poor (ICC range, 0.09-0.48) to strong (ICC range, 0.80-0.90) agreement. After training, inter-rater reliabilities for composite scores improved from moderate (ICC, 0.76) to strong (ICC, 0.90) agreement, and for overall pass/fail ratings, from poor (kappa = 0.20) to moderate (kappa = 0.78) agreement. Focused, video-based anchor training facilitates greater inter-rater reliability in the objective assessment of simulated coronary anastomoses. Among raters with less teaching experience, such training may be needed before objective evaluation of technical skills. Published by Elsevier Inc.

  19. Financial and testamentary capacity evaluations: procedures and assessment instruments underneath a functional approach.

    Science.gov (United States)

    Sousa, Liliana B; Simões, Mário R; Firmino, Horácio; Peisah, Carmelle

    2014-02-01

    Mental health professionals are frequently involved in mental capacity determinations. However, there is a lack of specific measures and well-defined procedures for these evaluations. The main purpose of this paper is to provide a review of financial and testamentary capacity evaluation procedures, including not only the traditional neuropsychological and functional assessment but also the more recently developed forensic assessment instruments (FAIs), which have been developed to provide a specialized answer to legal systems regarding civil competencies. Here the main guidelines, papers, and other references are reviewed in order to achieve a complete and comprehensive selection of instruments used in the assessment of financial and testamentary capacity. Although some specific measures for financial abilities have been developed recently, the same is not true for testamentary capacity. Here are presented several instruments or methodologies for assessing financial and testamentary capacity, including neuropsychological assessment, functional assessment scales, performance based functional assessment instruments, and specific FAIs. FAIs are the only specific instruments intended to provide a specific and direct answer to the assessment of financial capacity based on legal systems. Considering the need to move from a diagnostic to a functional approach in financial and testamentary capacity evaluations, it is essential to consider both general functional examination as well as cognitive functioning.

  20. RELIABILITY AND VALIDITY OF SUBJECTIVE ASSESSMENT OF LUMBAR LORDOSIS IN CONVENTIONAL RADIOGRAPHY.

    Science.gov (United States)

    Ruhinda, E; Byanyima, R K; Mugerwa, H

    2014-10-01

    Reliability and validity studies of different lumbar curvature analysis and measurement techniques have been documented however there is limited literature on the reliability and validity of subjective visual analysis. Radiological assessment of lumbar lordotic curve aids in early diagnosis of conditions even before neurologic changes set in. To ascertain the level of reliability and validity of subjective assessment of lumbar lordosis in conventional radiography. A blinded, repeated-measures diagnostic test was carried out on lumbar spine x-ray radiographs. Radiology Department at Joint Clinical Research Centre (JCRC), Mengo-Kampala-Uganda. Seventy (70) lateral lumbar x-ray films were used for this study and were obtained from the archive of JCRC radiology department at Butikiro house, Mengo-Kampala. Poor observer agreement, both inter- and intra-observer, with kappa values of 0.16 was found. Inter-observer agreement was poorer than intra-observer agreement. Kappa values significantly rose when the lumbar lordosis was clustered into four categories without grading each abnormality. The results confirm that subjective assessment of lumbar lordosis has low reliability and validity. Film quality has limited influence on the observer reliability. This study further shows that fewer scale categories of lordosis abnormalities produce better observer reliability.

  1. Accuracy and reliability of peer assessment of athletic training psychomotor laboratory skills.

    Science.gov (United States)

    Marty, Melissa C; Henning, Jolene M; Willse, John T

    2010-01-01

    Peer assessment is defined as students judging the level or quality of a fellow student's understanding. No researchers have yet demonstrated the accuracy or reliability of peer assessment in athletic training education. To determine the accuracy and reliability of peer assessment of athletic training students' psychomotor skills. Cross-sectional study. Entry-level master's athletic training education program. First-year (n  =  5) and second-year (n  =  8) students. Participants evaluated 10 videos of a peer performing 3 psychomotor skills (middle deltoid manual muscle test, Faber test, and Slocum drawer test) on 2 separate occasions using a valid assessment tool. Accuracy of each peer-assessment score was examined through percentage correct scores. We used a generalizability study to determine how reliable athletic training students were in assessing a peer performing the aforementioned skills. Decision studies using generalizability theory demonstrated how the peer-assessment scores were affected by the number of participants and number of occasions. Participants had a high percentage of correct scores: 96.84% for the middle deltoid manual muscle test, 94.83% for the Faber test, and 97.13% for the Slocum drawer test. They were not able to reliably assess a peer performing any of the psychomotor skills on only 1 occasion. However, the φ increased (exceeding the 0.70 minimal standard) when 2 participants assessed the skill on 3 occasions (φ  =  0.79) for the Faber test, with 1 participant on 2 occasions (φ  =  0.76) for the Slocum drawer test, and with 3 participants on 2 occasions for the middle deltoid manual muscle test (φ  =  0.72). Although students did not detect all errors, they assessed their peers with an average of 96% accuracy. Having only 1 student assess a peer performing certain psychomotor skills was less reliable than having more than 1 student assess those skills on more than 1 occasion. Peer assessment of psychomotor skills

  2. Reliability of the Arabic Smartphone Addiction Scale and Smartphone Addiction Scale-Short Version in Two Different Moroccan Samples.

    Science.gov (United States)

    Sfendla, Anis; Laita, Meriame; Nejjar, Basma; Souirti, Zouhayr; Touhami, Ahami Ahmed Omar; Senhaji, Meftaha

    2018-05-01

    The extensive accessibility to smartphones in the last decade raises the concerns of addictive behavior patterns toward these technologies worldwide and in developing countries, and Arabic ones in particular. In an area of stigmatized behavior such as Internet and smartphone addiction, the hypothesis extends to whether there is a reliable instrument that can assess smartphone addiction. To our knowledge, no scale in Arabic language is available to assess maladaptive behavior associated with smartphone use. This study aims to assess the factorial validity and internal reliability of the Arabic Smartphone Addiction Scale (SAS) and Smartphone Addiction Scale-Short Version (SAS-SV) in a Moroccan surveyed population. Participants (N = 440 and N = 310) completed an online survey, including SAS, SAS-SV, and questions about sociodemographic status. Factor analysis results showed six factors with factor loading ranging from 0.25 to 0.99 for SAS. Reliability, based on Cronbach's alpha, was excellent (α = 0.94) for this instrument. The SAS-SV showed one factor (unidimensional construct), and internal reliability was in the good range with an alpha coefficient of (α = 0.87). The prevalence of excessive users was 55.8 percent with highest symptom prevalence reported for tolerance and preoccupation. This study proved factor validity of the Arabic SAS and SAS-SV instruments and confirmed their internal reliability.

  3. Advanced Reactor PSA Methodologies for System Reliability Analysis and Source Term Assessment

    Energy Technology Data Exchange (ETDEWEB)

    Grabaskas, D.; Brunett, A.; Passerini, S.; Grelle, A.; Bucknor, M.

    2017-06-26

    Beginning in 2015, a project was initiated to update and modernize the probabilistic safety assessment (PSA) of the GE-Hitachi PRISM sodium fast reactor. This project is a collaboration between GE-Hitachi and Argonne National Laboratory (Argonne), and funded in part by the U.S. Department of Energy. Specifically, the role of Argonne is to assess the reliability of passive safety systems, complete a mechanistic source term calculation, and provide component reliability estimates. The assessment of passive system reliability focused on the performance of the Reactor Vessel Auxiliary Cooling System (RVACS) and the inherent reactivity feedback mechanisms of the metal fuel core. The mechanistic source term assessment attempted to provide a sequence specific source term evaluation to quantify offsite consequences. Lastly, the reliability assessment focused on components specific to the sodium fast reactor, including electromagnetic pumps, intermediate heat exchangers, the steam generator, and sodium valves and piping.

  4. Practice of value-based distribution reliability assessment (VBDRA) at Scarborough Public Utilities

    International Nuclear Information System (INIS)

    Chen, R-L.

    1995-01-01

    The development of value-based distribution reliability assessment (VBDRA) at Scarborough Public Utilities was described. Load point reliability indices, customer interruption costs (CIC), continuity and service reliability, accuracy of CIC, and the aspects of application of VBDRA were addressed. The application of VBDRA to a long-term rebuild plan for 4.16 kV distribution system was described. The importance of a cost-benefit analysis for implementation of VBDRA was emphasized. In the case of the Scarborough Public Utilities Commission the enhanced feeder reliability assessment was found to influence the allocation of funding to where it provided the most value to customers. 14 refs., 3 tabs., 3 figs

  5. Assessment of the reliability and consistency of the "malnutrition inflammation score" (MIS) in Mexican adults with chronic kidney disease for diagnosis of protein-energy wasting syndrome (PEW).

    Science.gov (United States)

    González-Ortiz, Ailema Janeth; Arce-Santander, Celene Viridiana; Vega-Vega, Olynka; Correa-Rotter, Ricardo; Espinosa-Cuevas, María de Los Angeles

    2014-10-04

    The protein-energy wasting syndrome (PEW) is a condition of malnutrition, inflammation, anorexia and wasting of body reserves resulting from inflammatory and non-inflammatory conditions in patients with chronic kidney disease (CKD).One way of assessing PEW, extensively described in the literature, is using the Malnutrition Inflammation Score (MIS). To assess the reliability and consistency of MIS for diagnosis of PEW in Mexican adults with CKD on hemodialysis (HD). Study of diagnostic tests. A sample of 45 adults with CKD on HD were analyzed during the period June-July 2014.The instrument was applied on 2 occasions; the test-retest reliability was calculated using the Intraclass Correlation Coefficient (ICC); the internal consistency of the questionnaire was analyzed using Cronbach's αcoefficient. A weighted Kappa test was used to estimate the validity of the instrument; the result was subsequently compared with the Bilbrey nutritional index (BNI). The reliability of the questionnaires, evaluated in the patient sample, was ICC=0.829.The agreement between MIS observations was considered adequate, k= 0.585 (p <0.001); when comparing it with BNI, a value of k = 0.114 was obtained (p <0.001).In order to estimate the tendency, a correlation test was performed. The r² correlation coefficient was 0.488 (P <0.001). MIS has adequate reliability and validity for diagnosing PEW in the population with chronic kidney disease on HD. Copyright AULA MEDICA EDICIONES 2014. Published by AULA MEDICA. All rights reserved.

  6. Valid and Reliable Science Content Assessments for Science Teachers

    Science.gov (United States)

    Tretter, Thomas R.; Brown, Sherri L.; Bush, William S.; Saderholm, Jon C.; Holmes, Vicki-Lynn

    2013-01-01

    Science teachers' content knowledge is an important influence on student learning, highlighting an ongoing need for programs, and assessments of those programs, designed to support teacher learning of science. Valid and reliable assessments of teacher science knowledge are needed for direct measurement of this crucial variable. This paper…

  7. Lower limb spasticity assessment using an inertial sensor: a reliability study

    International Nuclear Information System (INIS)

    Sterpi, I; Colombo, R; Caroli, A; Meazza, E; Maggioni, G; Pistarini, C

    2013-01-01

    Spasticity is a common motor impairment in patients with neurological disorders that can prevent functional recovery after rehabilitation. In the clinical setting, its assessment is carried out using standardized clinical scales. The aim of this study was to verify the applicability of inertial sensors for an objective measurement of quadriceps spasticity and evaluate its test–retest and inter-rater reliability during the implementation of the Wartenberg pendulum test. Ten healthy subjects and 11 patients in vegetative state with severe brain damage were enrolled in this study. Subjects were evaluated three times on three consecutive days. The test–retest reliability of measurement was assessed in the first two days. The third day was devoted to inter-rater reliability assessment. In addition, the lower limb muscle tone was bilaterally evaluated at the knee joint by the modified Ashworth scale. The factorial ANOVA analysis showed that the implemented method allowed us to discriminate between healthy and pathological conditions. The fairly low SEM and high ICC values obtained for the pendulum parameters indicated a good test–retest and inter-rater reliability of measurement. This study shows that an inertial sensor can be reliably used to characterize leg kinematics during the Wartenberg pendulum test and provide quantitative evaluation of quadriceps spasticity. (paper)

  8. Assessing the potential of economic instruments for managing drought risk at river basin scale

    Science.gov (United States)

    Pulido-Velazquez, M.; Lopez-Nicolas, A.; Macian-Sorribes, H.

    2015-12-01

    Economic instruments work as incentives to adapt individual decisions to collectively agreed goals. Different types of economic instruments have been applied to manage water resources, such as water-related taxes and charges (water pricing, environmental taxes, etc.), subsidies, markets or voluntary agreements. Hydroeconomic models (HEM) provide useful insight on optimal strategies for coping with droughts by simultaneously analysing engineering, hydrology and economics of water resources management. We use HEMs for evaluating the potential of economic instruments on managing drought risk at river basin scale, considering three criteria for assessing drought risk: reliability, resilience and vulnerability. HEMs allow to calculate water scarcity costs as the economic losses due to water deliveries below the target demands, which can be used as a vulnerability descriptor of drought risk. Two generic hydroeconomic DSS tools, SIMGAMS and OPTIGAMS ( both programmed in GAMS) have been developed to evaluate water scarcity cost at river basin scale based on simulation and optimization approaches. The simulation tool SIMGAMS allocates water according to the system priorities and operating rules, and evaluate the scarcity costs using economic demand functions. The optimization tool allocates water resources for maximizing net benefits (minimizing total water scarcity plus operating cost of water use). SIMGAS allows to simulate incentive water pricing policies based on water availability in the system (scarcity pricing), while OPTIGAMS is used to simulate the effect of ideal water markets by economic optimization. These tools have been applied to the Jucar river system (Spain), highly regulated and with high share of water use for crop irrigation (greater than 80%), where water scarcity, irregular hydrology and groundwater overdraft cause droughts to have significant economic, social and environmental consequences. An econometric model was first used to explain the variation

  9. The Therapeutic Environment Screening Survey for Nursing Homes (TESS-NH): an observational instrument for assessing the physical environment of institutional settings for persons with dementia.

    Science.gov (United States)

    Sloane, Philip D; Mitchell, C Madeline; Weisman, Gerald; Zimmerman, Sheryl; Foley, Kristie M Long; Lynn, Mary; Calkins, Margaret; Lawton, M Powell; Teresi, Jeanne; Grant, Leslie; Lindeman, David; Montgomery, Rhonda

    2002-03-01

    To develop an observational instrument that describes the ability of physical environments of institutional settings to address therapeutic goals for persons with dementia. A National Institute on Aging workgroup identified and subsequently revised items that evaluated exit control, maintenance, cleanliness, safety, orientation/cueing, privacy, unit autonomy, outdoor access, lighting, noise, visual/tactile stimulation, space/seating, and familiarity/homelikeness. The final instrument contains 84 discrete items and one global rating. A summary scale, the Special Care Unit Environmental Quality Scale (SCUEQS), consists of 18 items. Lighting items were validated using portable light meters. Concurrent criterion validation compared SCUEQS scores with the Professional Environmental Assessment Protocol (PEAP). Interrater kappa statistics for 74% of items were above.60. For another 10% of items, kappas could not be calculated due to empty cells, but interrater agreement was above 80%. The SCUEQS demonstrated an interrater reliability of.93, a test--retest reliability of.88, and an internal consistency of.81--.83. Light meter ratings correlated significantly with the Therapeutic Environment Screening Survey for Nursing Homes (TESS-NH) lighting items (r =.29--.38, p =.01--.04), and the SCUEQS correlated significantly with global PEAP ratings (r =.52, p TESS-NH efficiently assesses discrete elements of the physical environment and has strong reliability and validity. The SCUEQS provides a quantitative measure of environmental quality in institutional settings.

  10. Reliability analysis and assessment of structural systems

    International Nuclear Information System (INIS)

    Yao, J.T.P.; Anderson, C.A.

    1977-01-01

    The study of structural reliability deals with the probability of having satisfactory performance of the structure under consideration within any specific time period. To pursue this study, it is necessary to apply available knowledge and methodology in structural analysis (including dynamics) and design, behavior of materials and structures, experimental mechanics, and the theory of probability and statistics. In addition, various severe loading phenomena such as strong motion earthquakes and wind storms are important considerations. For three decades now, much work has been done on reliability analysis of structures, and during this past decade, certain so-called 'Level I' reliability-based design codes have been proposed and are in various stages of implementation. These contributions will be critically reviewed and summarized in this paper. Because of the undesirable consequences resulting from the failure of nuclear structures, it is important and desirable to consider the structural reliability in the analysis and design of these structures. Moreover, after these nuclear structures are constructed, it is desirable for engineers to be able to assess the structural reliability periodically as well as immediately following the occurrence of severe loading conditions such as a strong-motion earthquake. During this past decade, increasing use has been made of techniques of system identification in structural engineering. On the basis of non-destructive test results, various methods have been developed to obtain an adequate mathematical model (such as the equations of motion with more realistic parameters) to represent the structural system

  11. [Assessment of municipal management of oral health in primary care: data collection instrument accuracy].

    Science.gov (United States)

    Pires, Diego Anselmi; Colussi, Claudia Flemming; Calvo, Maria Cristina Marino

    2014-11-01

    This validation study seeks to check the accuracy of an evaluation model. In an evaluation, it is necessary to validate the precision and reliability of the data collection instrument. In this study, the Management Assessment of Oral Health in Primary Care in Santa Catarina was used as a benchmark to calculate the indicators. Its model analyzes primary data, collected via an electronic form, and secondary data, available in the Unified Health System (SUS) database. For this study, the form was applied in the cities of Santa Catarina's Coal Region at two different moments to check its reproducibility, followed by a discussion over the answers with the researcher. The results obtained were analyzed and debated in a consensus workshop with specialists in the field, detecting inaccuracies relating to the concept, the source used and the profile of the respondents themselves. The gross agreement rate in the two data collections was 87%, and the inaccuracies amounted to 36% of the answers. Preferential source suggestions, question modifications and guidelines for the correct filling out of the form were some of the proposed changes, improving the original matrix and the data collection instrument.

  12. Reliability assessment of nuclear structural systems

    International Nuclear Information System (INIS)

    Reich, M.; Hwang, H.

    1983-01-01

    Reliability assessment of nuclear structural systems has been receiving more emphasis over the last few years. This paper deals with the recent progress made by the Structural Analysis Division of Brookhaven National Laboratory (BNL), in the development of a probability-based reliability analysis methodology for safety evaluation of reactor containments and other seismic category I structures. An important feature of this methodology is the incorporation of finite element analysis and random vibration theory. By utilizing this method, it is possible to evaluate the safety of nuclear structures under various static and dynamic loads in terms of limit state probability. Progress in other related areas, such as the establishment of probabilistic characteristics for various loads and structural resistance, are also described. Results of an application of the methodology to a realistic reinforced concrete containment subjected to dead and live loads, accidental internal pressures and earthquake ground accelerations are presented

  13. Impact of Alzheimer's Disease on Caregiver Questionnaire: internal consistency, convergent validity, and test-retest reliability of a new measure for assessing caregiver burden.

    Science.gov (United States)

    Cole, Jason C; Ito, Diane; Chen, Yaozhu J; Cheng, Rebecca; Bolognese, Jennifer; Li-McLeod, Josephine

    2014-09-04

    There is a lack of validated instruments to measure the level of burden of Alzheimer's disease (AD) on caregivers. The Impact of Alzheimer's Disease on Caregiver Questionnaire (IADCQ) is a 12-item instrument with a seven-day recall period that measures AD caregiver's burden across emotional, physical, social, financial, sleep, and time aspects. Primary objectives of this study were to evaluate psychometric properties of IADCQ administered on the Web and to determine most appropriate scoring algorithm. A national sample of 200 unpaid AD caregivers participated in this study by completing the Web-based version of IADCQ and Short Form-12 Health Survey Version 2 (SF-12v2™). The SF-12v2 was used to measure convergent validity of IADCQ scores and to provide an understanding of the overall health-related quality of life of sampled AD caregivers. The IADCQ survey was also completed four weeks later by a randomly selected subgroup of 50 participants to assess test-retest reliability. Confirmatory factor analysis (CFA) was implemented to test the dimensionality of the IADCQ items. Classical item-level and scale-level psychometric analyses were conducted to estimate psychometric characteristics of the instrument. Test-retest reliability was performed to evaluate the instrument's stability and consistency over time. Virtually none (2%) of the respondents had either floor or ceiling effects, indicating the IADCQ covers an ideal range of burden. A single-factor model obtained appropriate goodness of fit and provided evidence that a simple sum score of the 12 items of IADCQ can be used to measure AD caregiver's burden. Scales-level reliability was supported with a coefficient alpha of 0.93 and an intra-class correlation coefficient (for test-retest reliability) of 0.68 (95% CI: 0.50-0.80). Low-moderate negative correlations were observed between the IADCQ and scales of the SF-12v2. The study findings suggest the IADCQ has appropriate psychometric characteristics as a

  14. Operational human performance reliability assessment (OHPRA)

    International Nuclear Information System (INIS)

    Haas, P.M.; Swanson, P.J.; Connelly, E.M.

    1993-01-01

    Operational Human Performance Reliability Assessment (OHPRA) is an approach for assessing human performance that is being developed in response to demands from modern process industries for practical and effective tools to assess and improve human performance, and therefore overall system performance and safety. The single most distinguishing feature of the approach is that is defines human performance in open-quotes operationalclose quotes terms. OHPRA is focused not on generation of human error probabilities, but on practical analysis of human performance to aid management in (1) identifying open-quotes fixableclose quotes problems and (2) providing input on the importance and nature of potential improvements. Development of the model in progress uses a unique approach for eliciting expert strategies for assessing performance. A PC-based model incorporating this expertise is planned. A preliminary version of the approach has already been used successfully to identify practical human performance problems in reactor and chemical process plant operations

  15. Reliability of the Alzheimer's disease assessment scale (ADAS-Cog) in longitudinal studies.

    Science.gov (United States)

    Khan, Anzalee; Yavorsky, Christian; DiClemente, Guillermo; Opler, Mark; Liechti, Stacy; Rothman, Brian; Jovic, Sofija

    2013-11-01

    Considering the scarcity of longitudinal assessments of reliability, there is need for a more precise understanding of cognitive decline in Alzheimer's Disease (AD). The primary goal was to assess longitudinal changes in inter-rater reliability, test retest reliability and internal consistency of scores of the ADAS-Cog. 2,618 AD subjects were enrolled in seven randomized, double-blind, placebo-controlled, multicenter-trials from 1986 to 2009. Reliability, internal-consistency and cross-sectional analysis of ADAS-Cog and MMSE across seven visits were examined. Intra-class correlation (ICC) for ADAS-Cog was moderate to high supporting their reliability. Absolute Agreement ICCs 0.392 (Visit-7) to 0.806 (Visit-2) showed a progressive decrease in correlations across time. Item analysis revealed a decrease in item correlations, with the lowest correlations for Visit 7 for Commands (ICC=0.148), Comprehension (ICC=0.092), Spoken Language (ICC=0.044). Suitable assessment of AD treatments is maintained through accurate measurement of clinically significant outcomes. Targeted rater education ADAS-Cog items over-time can improve ability to administer and score the scale.

  16. Developing evaluation instrument based on CIPP models on the implementation of portfolio assessment

    Science.gov (United States)

    Kurnia, Feni; Rosana, Dadan; Supahar

    2017-08-01

    This study aimed to develop an evaluation instrument constructed by CIPP model on the implementation of portfolio assessment in science learning. This study used research and development (R & D) method; adapting 4-D by the development of non-test instrument, and the evaluation instrument constructed by CIPP model. CIPP is the abbreviation of Context, Input, Process, and Product. The techniques of data collection were interviews, questionnaires, and observations. Data collection instruments were: 1) the interview guidelines for the analysis of the problems and the needs, 2) questionnaire to see level of accomplishment of portfolio assessment instrument, and 3) observation sheets for teacher and student to dig up responses to the portfolio assessment instrument. The data obtained was quantitative data obtained from several validators. The validators consist of two lecturers as the evaluation experts, two practitioners (science teachers), and three colleagues. This paper shows the results of content validity obtained from the validators and the analysis result of the data obtained by using Aikens' V formula. The results of this study shows that the evaluation instrument based on CIPP models is proper to evaluate the implementation of portfolio assessment instruments. Based on the experts' judgments, practitioners, and colleagues, the Aikens' V coefficient was between 0.86-1,00 which means that it is valid and can be used in the limited trial and operational field trial.

  17. Reliability and Validity of Athletes Disability Index Questionnaire.

    Science.gov (United States)

    Noormohammadpour, Pardis; Hosseini Khezri, Alireza; Farahbakhsh, Farzin; Mansournia, Mohammad Ali; Smuck, Matthew; Kordi, Ramin

    2018-03-01

    The purpose of this study was to evaluate validity and reliability of a new proposed questionnaire for assessment of functional disability in athletes with low back pain (LBP). Validity and reliability study. Elite athletes participating in different fields of sports. Participants were 165 male and female athletes (between 12 and 50 years old) with LBP. Athlete Disability Index (ADI) Questionnaire which is developed by the authors for assessing LBP-related disability in athletes, Oswestry Disability Index (ODI), and the Roland-Morris Disability Questionnaire (RDQ). Self-reported responses were collected regarding LBP-related disability through ADI, ODI, and RDQ. The test-retest reliability was strong, and intraclass correlation value ranged between 0.74 and 0.94. The Cronbach alpha coefficient value of 0.91 (P visual analog scale was r = 0.626 (P disability levels were mild in the large majority of subjects (91.5% and 86.0%, respectively). Alternatively, disability assessments by the ADI did not cluster at the mild level and ranged more broadly from mild to very high. The ADI is a reliable and valid instrument for assessing disability in athletes with LBP. Compared with the available LBP disability questionnaires used in the general population, ADI can more precisely stratify the disability levels of athletes due to LBP.

  18. Test-retest reliability of the Military Pre-training Questionnaire.

    Science.gov (United States)

    Robinson, M; Stokes, K; Bilzon, J; Standage, M; Brown, P; Thompson, D

    2010-09-01

    Musculoskeletal injuries are a significant cause of morbidity during military training. A brief, inexpensive and user-friendly tool that demonstrates reliability and validity is warranted to effectively monitor the relationship between multiple predictor variables and injury incidence in military populations. To examine the test-retest reliability of the Military Pre-training Questionnaire (MPQ), designed specifically to assess risk factors for injury among military trainees across five domains (physical activity, injury history, diet, alcohol and smoking). Analyses were based on a convenience sample of 58 male British Army trainees. Kappa (kappa), weighted kappa (kappa(w)) and intraclass correlation coefficients (ICC) were used to evaluate the 2-week test-retest reliability of the MPQ. For index measures constituting the assessment of a given construct, internal consistency was assessed by Cronbach's alpha (alpha) coefficients. Reliability of individual items ranged from poor to almost perfect (kappa range = 0.45-0.86; kappa(w) range = 0.11-0.91; ICC range = 0.34-0.86) with most items demonstrating moderate reliability. Overall scores related to physical activity, diet, alcohol and smoking constructs were reliable between both administrations (ICC = 0.63-0.85). Support for the internal consistency of the incorporated alcohol (alpha = 0.78) and cigarette (alpha = 0.75) scales was also provided. The MPQ is a reliable self-report instrument for assessing multiple injury-related risk factors during initial military training. Further assessment of the psychometric properties of the MPQ (e.g. different types of validity) with military populations/samples will support its interpretation and use in future surveillance and epidemiological studies.

  19. We need more replication research - A case for test-retest reliability.

    Science.gov (United States)

    Leppink, Jimmie; Pérez-Fuster, Patricia

    2017-06-01

    Following debates in psychology on the importance of replication research, we have also started to see pleas for a more prominent role for replication research in medical education. To enable replication research, it is of paramount importance to carefully study the reliability of the instruments we use. Cronbach's alpha has been the most widely used estimator of reliability in the field of medical education, notably as some kind of quality label of test or questionnaire scores based on multiple items or of the reliability of assessment across exam stations. However, as this narrative review outlines, Cronbach's alpha or alternative reliability statistics may complement but not replace psychometric methods such as factor analysis. Moreover, multiple-item measurements should be preferred above single-item measurements, and when using single-item measurements, coefficients as Cronbach's alpha should not be interpreted as indicators of the reliability of a single item when that item is administered after fundamentally different activities, such as learning tasks that differ in content. Finally, if we want to follow up on recent pleas for more replication research, we have to start studying the test-retest reliability of the instruments we use.

  20. Construct validity and reliability of automated body reaction test ...

    African Journals Online (AJOL)

    Automated Body Reaction Test (ABRT) is a new device for skills and physical assessment instrument to measure ability on react, move quickly and accurately in accordance with stimulus. A total of 474 subjects aged 7-17 years old were randomly selected for the construct validity (n=330) and reliability (n=144). The ABRT ...

  1. Assessing medical professionalism: A systematic review of instruments and their measurement properties

    Science.gov (United States)

    Li, Honghe; Liu, Yang; Wen, Deliang

    2017-01-01

    Background Over the last three decades, various instruments were developed and employed to assess medical professionalism, but their measurement properties have yet to be fully evaluated. This study aimed to systematically evaluate these instruments’ measurement properties and the methodological quality of their related studies within a universally acceptable standardized framework and then provide corresponding recommendations. Methods A systematic search of the electronic databases PubMed, Web of Science, and PsycINFO was conducted to collect studies published from 1990–2015. After screening titles, abstracts, and full texts for eligibility, the articles included in this study were classified according to their respective instrument’s usage. A two-phase assessment was conducted: 1) methodological quality was assessed by following the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist; and 2) the quality of measurement properties was assessed according to Terwee’s criteria. Results were integrated using best-evidence synthesis to look for recommendable instruments. Results After screening 2,959 records, 74 instruments from 80 existing studies were included. The overall methodological quality of these studies was unsatisfactory, with reasons including but not limited to unknown missing data, inadequate sample sizes, and vague hypotheses. Content validity, cross-cultural validity, and criterion validity were either unreported or negative ratings in most studies. Based on best-evidence synthesis, three instruments were recommended: Hisar’s instrument for nursing students, Nurse Practitioners’ Roles and Competencies Scale, and Perceived Faculty Competency Inventory. Conclusion Although instruments measuring medical professionalism are diverse, only a limited number of studies were methodologically sound. Future studies should give priority to systematically improving the performance of existing

  2. Independent Orbiter Assessment (IOA): Assessment of instrumental subsystem FMEA/CIL

    Science.gov (United States)

    Gardner, J. R.; Addis, A. W.

    1988-01-01

    The McDonnell Douglas Astronautics Company (MDAC) was selected in June 1986 to perform an Independent Orbiter Assessment (IOA) of the Failure Modes and Effects Analysis (FMEA) and Critical Items List (CIL). The IOA effort first completed an analysis of the Instrumentation hardware, generating draft failure modes and potential critical items. To preserve independence, this analysis was accomplished without reliance upon the results contained within the NASA FMEA/CIL documentation. The IOA results were then compared to the NASA FMEA/CIL baseline. A resolution of each discrepancy from the comparison is provided through additional analysis as required. The results of that comparison for the Orbiter Instrumentation hardware are documented. The IOA product for Instrumentation analysis consisted of 107 failure mode worksheets that resulted in 22 critical items being identified. Comparison was made to the Pre 51-L NASA baseline with 14 Post 51-L FMEAs added, which consists of 96 FMEAs and 18 CIL items. This comparison produced agreement on all but 25 FMEAs which caused differences in 5 CIL items.

  3. Human reliability. Is probabilistic human reliability assessment possible?

    International Nuclear Information System (INIS)

    Mosneron Dupin, F.

    1996-01-01

    The possibility of carrying out Probabilistic Human Reliability Assessments (PHRA) is often doubted. Basing ourselves on the experience Electricite de France (EDF) has acquired in Probabilistic Safety Assessments for nuclear power plants, we show why the uncertainty of PHRA is very high. We then specify the limits of generic data and models for PHRA: very important factors are often poorly taken into account. To account for them, you need to have proper understanding of the actual context in which operators work. This demands surveys on the field (power plant and simulator) all of which must be carried out with behaviours science skills. The idea of estimating the probabilities of operator failure must not be abandoned, but probabilities must be given less importance, for they are only approximate indications. The qualitative aspects of PHRA should be given greater value (analysis process and qualitative insights). That is why the description (illustrated by case histories) of the main mechanisms of human behaviour, and of their manifestations in the nuclear power plant context (in terms of habits, attitudes, and informal methods and organization in particular) should be an important part of PHRA handbooks. These handbooks should also insist more on methods for gathering information on the actual context of the work of operators. Under these conditions, the PHRA should be possible and even desirable as a process for systematic analysis and assessment of human intervention. (author). 24 refs, 2 figs, 1 tab

  4. Human reliability. Is probabilistic human reliability assessment possible?

    Energy Technology Data Exchange (ETDEWEB)

    Mosneron Dupin, F

    1997-12-31

    The possibility of carrying out Probabilistic Human Reliability Assessments (PHRA) is often doubted. Basing ourselves on the experience Electricite de France (EDF) has acquired in Probabilistic Safety Assessments for nuclear power plants, we show why the uncertainty of PHRA is very high. We then specify the limits of generic data and models for PHRA: very important factors are often poorly taken into account. To account for them, you need to have proper understanding of the actual context in which operators work. This demands surveys on the field (power plant and simulator) all of which must be carried out with behaviours science skills. The idea of estimating the probabilities of operator failure must not be abandoned, but probabilities must be given less importance, for they are only approximate indications. The qualitative aspects of PHRA should be given greater value (analysis process and qualitative insights). That is why the description (illustrated by case histories) of the main mechanisms of human behaviour, and of their manifestations in the nuclear power plant context (in terms of habits, attitudes, and informal methods and organization in particular) should be an important part of PHRA handbooks. These handbooks should also insist more on methods for gathering information on the actual context of the work of operators. Under these conditions, the PHRA should be possible and even desirable as a process for systematic analysis and assessment of human intervention. (author). 24 refs, 2 figs, 1 tab.

  5. Design and reliability of a didactic inphographic rubric assessment

    OpenAIRE

    Guzmán-cedillo, Yunuen Ixchel

    2017-01-01

    The objective of this study is to describe design, validity process and reliability of a rubric assessment to evaluate didactic infographics quality. Participants were fifteen judges who participate in different moments of elaboration rubric process; it was made in three process phases: design, settings and reliability determination. Content validity was obtained by percentage agreement between 3 judges by component of the rubric; likewise a Krippendorff’s alpha were applied (a = .710) in pi...

  6. Self-administered Gerocognitive Examination (SAGE): a brief cognitive assessment Instrument for mild cognitive impairment (MCI) and early dementia.

    Science.gov (United States)

    Scharre, Douglas W; Chang, Shu-Ing; Murden, Robert A; Lamb, James; Beversdorf, David Q; Kataki, Maria; Nagaraja, Haikady N; Bornstein, Robert A

    2010-01-01

    To develop a self-administered cognitive assessment instrument to facilitate the screening of mild cognitive impairment (MCI) and early dementia and determine its association with gold standard clinical assessments including neuropsychologic evaluation. Adults aged above 59 years with sufficient vision and English literacy were recruited from geriatric and memory disorder clinics, educational talks, independent living facilities, senior centers, and memory screens. After Self-administered Gerocognitive Examination (SAGE) screening, subjects were randomly selected to complete a clinical evaluation, neurologic examination, neuropsychologic battery, functional assessment, and mini-mental state examination (MMSE). Subjects were identified as dementia, MCI, or normal based on standard clinical criteria and neuropsychologic testing. Two hundred fifty-four participants took the SAGE screen and 63 subjects completed the extensive evaluation (21 normal, 21 MCI, and 21 dementia subjects). Spearman rank correlation between SAGE and neuropsychologic battery was 0.84 (0.76 for MMSE). SAGE receiver operating characteristics on the basis of clinical diagnosis showed 95% specificity (90% for MMSE) and 79% sensitivity (71% for MMSE) in detecting those with cognitive impairment from normal subjects. This study suggests that SAGE is a reliable instrument for detecting cognitive impairment and compares favorably with the MMSE. The self-administered feature may promote cognitive testing by busy clinicians prompting earlier diagnosis and treatment.

  7. Measurement Properties of Instruments for Measuring of Lymphedema: Systematic Review.

    Science.gov (United States)

    Hidding, Janine T; Viehoff, Peter B; Beurskens, Carien H G; van Laarhoven, Hanneke W M; Nijhuis-van der Sanden, Maria W G; van der Wees, Philip J

    2016-12-01

    Lymphedema is a common complication of cancer treatment, resulting in swelling and subjective symptoms. Reliable and valid measurement of this side effect of medical treatment is important. The purpose of this study was to provide best evidence regarding which measurement instruments are most appropriate in measuring lymphedema in its different stages. The PubMed and Web of Science databases were used, and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. Clinical studies on measurement instruments assessing lymphedema were reviewed using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) scoring instrument for quality assessment. Data on reliability, concurrent validity, convergent validity, sensitivity, specificity, applicability, and costs were extracted. Pooled data showed good intrarater intraclass correlation coefficients (ICCs) (.89) for bioimpedance spectroscopy (BIS) in the lower extremities and high intrarater and interrater ICCs for water volumetry, tape measurement, and perometry (.98-.99) in the upper extremities. In the upper extremities, the standard error of measurement was 3.6% (σ=0.7%) for water volumetry, 5.6% (σ=2.1%) for perometry, and 6.6% (σ=2.6%) for tape measurement. Sensitivity of tape measurement in the upper extremities, using different cutoff points, varied from 0.73 to 0.90, and specificity values varied from 0.72 to 0.78. No uniform definition of lymphedema was available, and a gold standard as a reference test was lacking. Items concerning risk of bias were study design, patient selection, description of lymphedema, blinding of test outcomes, and number of included participants. Measurement instruments with evidence for good reliability and validity were BIS, water volumetry, tape measurement, and perometry, where BIS can detect alterations in extracellular fluid in stage 1 lymphedema and the other measurement instruments can detect alterations in volume

  8. Virtual reality as a metric for the assessment of laparoscopic psychomotor skills. Learning curves and reliability measures.

    Science.gov (United States)

    Gallagher, A G; Satava, R M

    2002-12-01

    The objective assessment of the psychomotor skills of surgeons is now a priority; however, this is a difficult task because of measurement difficulties associated with the assessment of surgery in vivo. In this study, virtual reality (VR) was used to overcome these problems. Twelve experienced (>50 minimal-access procedures), 12 inexperienced laparoscopic surgeons (Virtual Reality (MIST VR). Experienced laparoscopic surgeons performed the tasks significantly (p < 0.01) faster, with less error, more economy in the movement of instruments and the use of diathermy, and with greater consistency in performance. The standardized coefficient alpha for performance measures ranged from a = 0.89 to 0.98, showing high internal measurement consistency. Test-retest reliability ranged from r = 0.96 to r = 0.5. VR is a useful tool for evaluating the psychomotor skills needed to perform laparoscopic surgery.

  9. 75 FR 66038 - Planning Resource Adequacy Assessment Reliability Standard

    Science.gov (United States)

    2010-10-27

    ..., providing a common framework for resource adequacy analysis, assessment, and documentation) effectively and...://www.nerc.com/commondocs.php?cd=2 . D. Proposed Effective Date 24. Proposed regional Reliability...

  10. Assessment of teacher competence using video portfolios: reliability, construct validity and consequential validity

    NARCIS (Netherlands)

    Admiraal, W.; Hoeksma, M.; van de Kamp, M.-T.; van Duin, G.

    2011-01-01

    The richness and complexity of video portfolios endanger both the reliability and validity of the assessment of teacher competencies. In a post-graduate teacher education program, the assessment of video portfolios was evaluated for its reliability, construct validity, and consequential validity.

  11. Air pollution assessment in two Moroccan cities using instrumental neutron activation analysis on bio-accumulators

    International Nuclear Information System (INIS)

    Embarch, K.; Moutea, Z.; Bounakhla, M.; Cherkaoui, R.; Chouak, A.; Lferde, M.; Gaudry, A.

    2008-01-01

    Full text: Biomonitoring is an appropriate tool for the air pollution assessment studies. In this work, lichens and barks have been used as bio-accumulators in several sites in two Moroccan cities (Rabat and Mohammadia). The specific ability of absorbing and accumulating heavy metals and toxic element from the air, their longevity and resistance to the environmental stresses, make those bioindicators suitable for this kind of studies. The Instrumental Neutron Activation Analysis (INAA) is universally accepted as one of the most reliable analytical tools for trace and ultra-trace elements determination. Its use in trace elements atmospheric pollution related studies has been and is still extensive as can be demonstrated by several specific works and detailed reviews. In this work, a preliminary investigation employing lichens, barks and instrumental neutron activation analysis (INAA) was carried out to evaluate the trace elements distribution in six different areas of Rabat and Mohammadia cities characterised by the presence of many industries and heavy traffic. Samples were irradiated with thermal neutrons in a nuclear reactor and the induced activity was counted using high-resolution Germanium-Lithium detectors. More than 30 elements were determined using two modes : short irradiation (1 minute) and long irradiation (17 hours). Accuracy and quality control were assessed using the reference standard material IAEA-336. This was less than 1% for major and about 5 to 10% for traces. [fr

  12. Development of an instrument to measure self-efficacy in caregivers of people with advanced cancer.

    Science.gov (United States)

    Ugalde, Anna; Krishnasamy, Meinir; Schofield, Penelope

    2013-06-01

    Informal caregivers of people with advanced cancer experience many negative impacts as a result of their role. There is a lack of suitable measures specifically designed to assess their experience. This study aimed to develop a new measure to assess self-efficacy in caregivers of people with advanced cancer. The development and testing of the new measure consisted of four separate, sequential phases: generation of issues, development of issues into items, pilot testing and field testing. In the generation of issues, 17 caregivers were interviewed to generate data. These data were analysed to generate codes, which were then systematically developed into items to construct the instrument. The instrument was pilot tested with 14 health professionals and five caregivers. It was then administered to a large sample for field testing to establish the psychometric properties, with established measures including the Brief Cope and the Family Appraisals for Caregiving Questionnaire for Palliative Care. Ninety-four caregivers completed the questionnaire booklet to establish the factor structure, reliability and validity. The factor analysis resulted in a 21-item, four-factor instrument, with the subscales being termed Resilience, Self-Maintenance, Emotional Connectivity and Instrumental Caregiving. The test-retest reliability and internal consistency were both excellent, ranging from 0.73 to 0.85 and 0.81 to 0.94, respectively. Six convergent and divergent hypotheses were made, and five were supported. This study has developed a new instrument to assess self-efficacy in caregivers of people with advanced cancer. The result is a four-factor, 21-item instrument with demonstrated reliability and validity. Copyright © 2012 John Wiley & Sons, Ltd.

  13. Reliability of scored patient generated subjective global assessment ...

    African Journals Online (AJOL)

    Objective: Establish the reliability of the scored Patient Generated-Subjective Global Assessment (PG-SGA) in determining nutritional status among Antiretroviral Therapy (ART) naive HIV-infected adults. Methods: A descriptive, cross sectional study among outpatient medical clinics, in The AIDS Support Organization ...

  14. Assessing Changes in the Reliability of the U.S. Electric Power System

    Energy Technology Data Exchange (ETDEWEB)

    Larsen, Peter H. [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Stanford Univ., CA (United States). Dept. of Physics; LaCommare, Kristina H. [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Eto, Joseph H. [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); Sweeney, James L. [Stanford Univ., CA (United States)

    2015-08-01

    Over the past 15 years, the most well-publicized efforts to assess trends in U.S. electric power system reliability have focused only on a subset of all power interruption events (see, for example, Amin 2008 and Campbell 2012)—namely, only the very largest events, which trigger immediate emergency reporting to federal agencies and industry regulators. Anecdotally, these events are thought by many to represent no more than 10% of the power interruptions experienced annually by electricity consumers. Moreover, a review of these emergency reports has identified shortcomings in relying on these data as reliable sources for assessing trends, even with the reliability events they report (Fisher et al. 2012). Recent work has begun to address these limitations by examining trends in reliability data collected annually by electricity distribution companies (Eto et al. 2012). In principle, all power interruptions experienced by electricity customers, regardless of size, are recorded by the distribution utility. Moreover, distribution utilities have a long history of recording this information, often in response to mandates from state public utility commissions (Eto et al. 2006). Thus, studies that rely on reliability data collected by distribution utilities can, in principle, provide a more complete basis upon which to assess trends or changes in reliability over time.

  15. [Instrument for the assessment of middle-aged and older adults' physical activity: design, eliability and application of the German-PAQ-50+].

    Science.gov (United States)

    Huy, Christina; Schneider, Sven

    2008-06-01

    Existing physical activity questionnaires have focused either on young and middle-aged adults or on the elderly. They have mainly assessed only a portion of possible physical activities or contained nation-specific sports. As there is no gold standard for a questionnaire-based assessment of physical activity in the over-50 population, recommendations for such a questionnaire relating to German-speaking countries were developed. This work included a systematic literature research, a survey of experts, and the design of a questionnaire based on validated measuring instruments. Finally, to test its reliability and application in the field, the complete questionnaire, including a retest, was applied by telephone interview (n = 57). The test-retest-correlation was r = 0.60 for the total time of physical activity and r = 0.52 for total energy expenditure. The researchers determined that the instrument is comprehensive in its coverage of all relevant domains of physical activity for the over-50 population; it is economically feasible and showed good acceptance.

  16. Reliability of the Hazelbaker Assessment Tool for Lingual Frenulum Function

    Directory of Open Access Journals (Sweden)

    James Jennifer P

    2006-03-01

    Full Text Available Abstract Background About 3% of infants are born with a tongue-tie which may lead to breastfeeding problems such as ineffective latch, painful attachment or poor weight gain. The Hazelbaker Assessment Tool for Lingual Frenulum Function (HATLFF has been developed to give a quantitative assessment of the tongue-tie and recommendation about frenotomy (release of the frenulum. The aim of this study was to assess the inter-rater reliability of the HATLFF. Methods Fifty-eight infants referred to the Breastfeeding Education and Support Services (BESS at The Royal Women's Hospital for assessment of tongue-tie and 25 control infants were assessed by two clinicians independently. Results The Appearance items received kappas between about 0.4 to 0.6, which represents "moderate" reliability. The first three Function items (lateralization, lift and extension of tongue had kappa values over 0.65 which indicates "substantial" agreement. The four Function items relating to infant sucking (spread, cupping, peristalsis and snapback received low kappa values with insignificant p values. There was 96% agreement between the two assessors on the recommendation for frenotomy (kappa 0.92, excellent agreement. The study found that the Function Score can be more simply assessed using only the first three function items (ie not scoring the sucking items, with a cut-off of ≤4 for recommendation of frenotomy. Conclusion We found that the HATLFF has a high reliability in a study of infants with tongue-tie and control infants

  17. Reliability assessment of power pole infrastructure incorporating deterioration and network maintenance

    International Nuclear Information System (INIS)

    Ryan, Paraic C.; Stewart, Mark G.; Spencer, Nathan; Li, Yue

    2014-01-01

    There is considerable investment in timber utility poles worldwide, and there is a need to examine the structural reliability and probability based management optimisation of these power distribution infrastructure elements. The work presented in this paper builds on the existing studies in this area through assessment of both treated and untreated timber power poles, with the effects of deterioration and network maintenance incorporated in the analysis. This more realistic assessment approach, with deterioration and maintenance considered, was achieved using event-based Monte Carlo simulation. The output from the probabilistic model is used to illustrate the importance of considering network maintenance in the time-dependent structural reliability assessment of timber power poles. Under wind load, treated and untreated poles designed and maintained in accordance with existing Australian standards were found to have similar failure rates. However, untreated pole networks required approximately twice as many maintenance based pole replacements to sustain the same level of reliability. The effect of four different network maintenance strategies on infrastructure performance was also investigated herein. This assessment highlighted the fact that slight alterations to network maintenance practices can lead to significant changes in performance of timber power pole networks. - Highlights: • A time-dependent structural reliability model was developed for timber power poles. • Deterioration and network maintenance were incorporated into this event based model. • Network maintenance had a significant impact on power pole wind vulnerability. • Treated and untreated poles designed to Australian standards had similar reliability. • Minor alterations to maintenance strategies had large effects on network performance

  18. Measuring stakeholder participation in evaluation: an empirical validation of the Participatory Evaluation Measurement Instrument (PEMI).

    Science.gov (United States)

    Daigneault, Pierre-Marc; Jacob, Steve; Tremblay, Joël

    2012-08-01

    Stakeholder participation is an important trend in the field of program evaluation. Although a few measurement instruments have been proposed, they either have not been empirically validated or do not cover the full content of the concept. This study consists of a first empirical validation of a measurement instrument that fully covers the content of participation, namely the Participatory Evaluation Measurement Instrument (PEMI). It specifically examines (1) the intercoder reliability of scores derived by two research assistants on published evaluation cases; (2) the convergence between the scores of coders and those of key respondents (i.e., authors); and (3) the convergence between the authors' scores on the PEMI and the Evaluation Involvement Scale (EIS). A purposive sample of 40 cases drawn from the evaluation literature was used to assess reliability. One author per case in this sample was then invited to participate in a survey; 25 fully usable questionnaires were received. Stakeholder participation was measured on nominal and ordinal scales. Cohen's κ, the intraclass correlation coefficient, and Spearman's ρ were used to assess reliability and convergence. Reliability results ranged from fair to excellent. Convergence between coders' and authors' scores ranged from poor to good. Scores derived from the PEMI and the EIS were moderately associated. Evidence from this study is strong in the case of intercoder reliability and ranges from weak to strong in the case of convergent validation. Globally, this suggests that the PEMI can produce scores that are both reliable and valid.

  19. From text to codings: intercoder reliability assessment in qualitative content analysis.

    Science.gov (United States)

    Burla, Laila; Knierim, Birte; Barth, Jurgen; Liewald, Katharina; Duetz, Margreet; Abel, Thomas

    2008-01-01

    High intercoder reliability (ICR) is required in qualitative content analysis for assuring quality when more than one coder is involved in data analysis. The literature is short of standardized procedures for ICR procedures in qualitative content analysis. To illustrate how ICR assessment can be used to improve codings in qualitative content analysis. Key steps of the procedure are presented, drawing on data from a qualitative study on patients' perspectives on low back pain. First, a coding scheme was developed using a comprehensive inductive and deductive approach. Second, 10 transcripts were coded independently by two researchers, and ICR was calculated. A resulting kappa value of .67 can be regarded as satisfactory to solid. Moreover, varying agreement rates helped to identify problems in the coding scheme. Low agreement rates, for instance, indicated that respective codes were defined too broadly and would need clarification. In a third step, the results of the analysis were used to improve the coding scheme, leading to consistent and high-quality results. The quantitative approach of ICR assessment is a viable instrument for quality assurance in qualitative content analysis. Kappa values and close inspection of agreement rates help to estimate and increase quality of codings. This approach facilitates good practice in coding and enhances credibility of analysis, especially when large samples are interviewed, different coders are involved, and quantitative results are presented.

  20. Measuring the Quality of Service in the Financial Area of a Public University: Development and Validation of the Instrument

    Directory of Open Access Journals (Sweden)

    Víctor Manuel Alcantar Enríquez

    2014-12-01

    Full Text Available This research, consisting of a descriptive study with a non-experimental design, involved an analysis to determine the validity and reliability of an instrument composed of seventeen items aimed at assessing the quality of service in the financial area of a public university by means of four variables: tangibility, reliability, responsiveness and empathy. The methodological strategy included the design of the measuring instrument; verification of the validity of content and construct; and analysis of internal consistency by means of Cronbach’s alpha. The instrument was applied to 152 users of the service, attaining a reliability coefficient of 0.943. The results show that with respect to specific concepts, the questions were clear; nevertheless we found it necessary to relocate items and rename variables, which resulted in a valid and reliable instrument for measuring the quality of service in the context under study.