Assessing communication skills in dietetic consultations: the development of the reliable and valid DIET-COMMS tool.

Science.gov (United States)

Whitehead, K A; Langley-Evans, S C; Tischler, V A; Swift, J A

2014-04-01

There is an increasing emphasis on the development of communication skills for dietitians but few evidence-based assessment tools available. The present study aimed to develop a dietetic-specific, short, reliable and valid assessment tool for measuring communication skills in patient consultations: DIET-COMMS. A literature review and feedback from 15 qualified dietitians were used to establish face and content validity during the development of DIET-COMMS. In total, 113 dietetic students and qualified dietitians were video-recorded undertaking mock consultations, assessed using DIET-COMMS by the lead author, and used to establish intra-rater reliability, as well as construct and predictive validity. Twenty recorded consultations were reassessed by nine qualified dietitians to assess inter-rater reliability: eight of these assessors were interviewed to determine user evaluation. Significant improvements in DIET-COMMS scores were achieved as students and qualified staff progressed through their training and gained experience, demonstrating construct validity, and also by qualified staff attending a training course, indicating predictive validity (P skills in practice was questioned. DIET-COMMS is a short, user-friendly, reliable and valid tool for measuring communication skills in patient consultations with both pre- and post-registration dietitians. Additional work is required to develop a training package for assessors and to identify how DIET-COMMS assessment can acceptably be incorporated into practice. © 2013 The British Dietetic Association Ltd.

Validation of the self-assessment teamwork tool (SATT) in a cohort of nursing and medical students.

Science.gov (United States)

Roper, Lucinda; Shulruf, Boaz; Jorm, Christine; Currie, Jane; Gordon, Christopher J

2018-02-09

Poor teamwork has been implicated in medical error and teamwork training has been shown to improve patient care. Simulation is an effective educational method for teamwork training. Post-simulation reflection aims to promote learning and we have previously developed a self-assessment teamwork tool (SATT) for health students to measure teamwork performance. This study aimed to evaluate the psychometric properties of a revised self-assessment teamwork tool. The tool was tested in 257 medical and nursing students after their participation in one of several mass casualty simulations. Using exploratory and confirmatory factor analysis, the revised self-assessment teamwork tool was shown to have strong construct validity, high reliability, and the construct demonstrated invariance across groups (Medicine & Nursing). The modified SATT was shown to be a reliable and valid student self-assessment tool. The SATT is a quick and practical method of guiding students' reflection on important teamwork skills.

Validation of screening tools to assess appetite among geriatric patients.

Science.gov (United States)

Hanisah, R; Suzana, S; Lee, F S

2012-07-01

Poor appetite is one of the main contributing factors of poor nutritional status among elderly individuals. Recognizing the importance of assessment of appetite, a cross sectional study was conducted to determine the validity of appetite screening tools namely, the Council on Nutrition Appetite questionnaire (CNAQ) and the simplified nutritional appetite questionnaire (SNAQ) against the appetite, hunger and sensory perception questionnaire (AHSPQ), measures of nutritional status and food intake among geriatric patients at the main general hospital in Malaysia. Nutritional status was assessed using the subjective global assessment (SGA) while food intake was measured using the dietary history questionnaire (DHQ). Anthropometric parameters included weight, height, body mass index (BMI), calf circumference (CC) and mid upper arm circumference (MUAC). A total of 145 subjects aged 60 to 86 years (68.3 ± 5.8 years) with 31.7% men and 68.3% women were recruited from outpatients (35 subjects) and inpatients (110 subjects) of Kuala Lumpur Hospital of Malaysia. As assessed by SGA, most subjects were classified as mild to moderately malnourished (50.4%), followed by normal (38.6%) and severely malnourished (11.0%). A total of 79.3% and 57.2% subjects were classified as having poor appetite according to CNAQ and SNAQ, respectively. CNAQ (80.9%) had a higher sensitivity than SNAQ (69.7%) when validated against nutritional status as assessed using SGA. However, the specificity of SNAQ (62.5%) was higher than CNAQ (23.2%). Positive predictive value for CNAQ and SNAQ were 62.6% and 74.7%, respectively. Cronbach's alpha for CNAQ and SNAQ were 0.546 and 0.578, respectively. History of weight loss over the past one year (Adjusted odds ratio 2.49) (p risk factors for poor appetite among subjects. In conclusion, malnutrition and poor appetite were prevalent among the geriatric outpatients and inpatients. SNAQ was more reliable and valid as an appetite screening tool among this special

Development and psychometric validation of a scale to assess information needs in cardiac rehabilitation: the INCR Tool.

Science.gov (United States)

Ghisi, Gabriela Lima de Melo; Grace, Sherry L; Thomas, Scott; Evans, Michael F; Oh, Paul

2013-06-01

To develop and psychometrically validate a tool to assess information needs in cardiac rehabilitation (CR) patients. After a literature search, 60 information items divided into 11 areas of needs were identified. To establish content validity, they were reviewed by an expert panel (N=10). Refined items were pilot-tested in 34 patients on a 5-point Likert-scale from 1 "really not helpful" to 5 "very important". A final version was generated and psychometrically tested in 203 CR patients. Test-retest reliability was assessed via the intraclass correlation coefficient (ICC), the internal consistency using Cronbach's alpha, and criterion validity was assessed with regard to patient's education and duration in CR. Five items were excluded after ICC analysis as well as one area of needs. All 10 areas were considered internally consistent (Cronbach's alpha>0.7). Criterion validity was supported by significant differences in mean scores by educational level (pinformation need. The INCR Tool was demonstrated to have good reliability and validity. This is an appropriate tool for application in clinical and research settings, assessing patients' needs during CR and as part of education programming. Copyright © 2013 Elsevier Ireland Ltd. All rights reserved.

Development, Validation, and Verification of a Self-Assessment Tool to Estimate Agnibala (Digestive Strength).

Science.gov (United States)

Singh, Aparna; Singh, Girish; Patwardhan, Kishor; Gehlot, Sangeeta

2017-01-01

According to Ayurveda, the traditional system of healthcare of Indian origin, Agni is the factor responsible for digestion and metabolism. Four functional states (Agnibala) of Agni have been recognized: regular, irregular, intense, and weak. The objective of the present study was to develop and validate a self-assessment tool to estimate Agnibala The developed tool was evaluated for its reliability and validity by administering it to 300 healthy volunteers of either gender belonging to 18 to 40-year age group. Besides confirming the statistical validity and reliability, the practical utility of the newly developed tool was also evaluated by recording serum lipid parameters of all the volunteers. The results show that the lipid parameters vary significantly according to the status of Agni The tool, therefore, may be used to screen normal population to look for possible susceptibility to certain health conditions. © The Author(s) 2016.

Validation of the tool assessment of clinical education (AssCE): A study using Delphi method and clinical experts.

Science.gov (United States)

Löfmark, Anna; Mårtensson, Gunilla

2017-03-01

The aim of the present study was to establish the validity of the tool Assessment of Clinical Education (AssCE). The tool is widely used in Sweden and some Nordic countries for assessing nursing students' performance in clinical education. It is important that the tools in use be subjected to regular audit and critical reviews. The validation process, performed in two stages, was concluded with a high level of congruence. In the first stage, Delphi technique was used to elaborate the AssCE tool using a group of 35 clinical nurse lecturers. After three rounds, we reached consensus. In the second stage, a group of 46 clinical nurse lecturers representing 12 universities in Sweden and Norway audited the revised version of the AssCE in relation to learning outcomes from the last clinical course at their respective institutions. Validation of the revised AssCE was established with high congruence between the factors in the AssCE and examined learning outcomes. The revised AssCE tool seems to meet its objective to be a validated assessment tool for use in clinical nursing education. Copyright © 2016 Elsevier Ltd. All rights reserved.

Reliability and validity of a tool to assess airway management skills in anesthesia trainees

Directory of Open Access Journals (Sweden)

Aliya Ahmed

2016-01-01

Conclusion: The tool designed to assess bag-mask ventilation and tracheal intubation skills in anesthesia trainees demonstrated excellent inter-rater reliability, fair test-retest reliability, and good construct validity. The authors recommend its use for formative and summative assessment of junior anesthesia trainees.

A Turkish Version of the Critical-Care Pain Observation Tool: Reliability and Validity Assessment.

Science.gov (United States)

Aktaş, Yeşim Yaman; Karabulut, Neziha

2017-08-01

The study aim was to evaluate the validity and reliability of the Critical-Care Pain Observation Tool in critically ill patients. A repeated measures design was used for the study. A convenience sample of 66 patients who had undergone open-heart surgery in the cardiovascular surgery intensive care unit in Ordu, Turkey, was recruited for the study. The patients were evaluated by using the Critical-Care Pain Observation Tool at rest, during a nociceptive procedure (suctioning), and 20 minutes after the procedure while they were conscious and intubated after surgery. The Turkish version of the Critical-Care Pain Observation Tool has shown statistically acceptable levels of validity and reliability. Inter-rater reliability was supported by moderate-to-high-weighted κ coefficients (weighted κ coefficient = 0.55 to 1.00). For concurrent validity, significant associations were found between the scores on the Critical-Care Pain Observation Tool and the Behavioral Pain Scale scores. Discriminant validity was also supported by higher scores during suctioning (a nociceptive procedure) versus non-nociceptive procedures. The internal consistency of the Critical-Care Pain Observation Tool was 0.72 during a nociceptive procedure and 0.71 during a non-nociceptive procedure. The validity and reliability of the Turkish version of the Critical-Care Pain Observation Tool was determined to be acceptable for pain assessment in critical care, especially for patients who cannot communicate verbally. Copyright © 2016 American Society of PeriAnesthesia Nurses. Published by Elsevier Inc. All rights reserved.

Validation of an early childhood caries risk assessment tool in a low-income Hispanic population.

Science.gov (United States)

Custodio-Lumsden, Christie L; Wolf, Randi L; Contento, Isobel R; Basch, Charles E; Zybert, Patricia A; Koch, Pamela A; Edelstein, Burton L

2016-03-01

There is a recognized need for valid risk assessment tools for use by both dental and nondental personnel to identify young children at risk for, or with, precavitated stages of early childhood caries (i.e., early stage decalcifications or white spot lesions).The aim of this study is to establish concurrent criterion validity of "MySmileBuddy" (MSB), a novel technology-assisted ECC risk assessment and behavioral intervention tool against four measures of ECC activity: semi-quantitative assays of salivary mutans streptococci levels, visible quantity of dental plaque, visual evidence of enamel decalcifications, and cavitation status (none, ECC, severe ECC). One hundred eight children 2-6 years of age presenting to a pediatric dental clinic were recruited from a predominantly Spanish-speaking, low-income, urban population. All children received a comprehensive oral examination and saliva culture for assessment of ECC indicators. Their caregivers completed the iPad-based MSB assessment in its entirety (15-20 minutes). MSB calculated both diet and comprehensive ECC risk scores. Associations between all variables were determined using ordinal logistic regression. MSB diet risk scores were significantly positively associated with salivary mutans (P valid risk assessment tool for identifying children with early precursors of cavitations but does not add value in identifying children with extant lesions. © 2015 American Association of Public Health Dentistry.

Validity and applicability of a video-based animated tool to assess mobility in elderly Latin American populations.

Science.gov (United States)

Guerra, Ricardo Oliveira; Oliveira, Bruna Silva; Alvarado, Beatriz Eugenia; Curcio, Carmen Lucia; Rejeski, W Jack; Marsh, Anthony P; Ip, Edward H; Barnard, Ryan T; Guralnik, Jack M; Zunzunegui, Maria Victoria

2014-10-01

To assess the reliability and the validity of Portuguese- and Spanish-translated versions of the video-based short-form Mobility Assessment Tool in assessing self-reported mobility, and to provide evidence for the applicability of these videos in elderly Latin American populations as a complement to physical performance measures. The sample consisted of 300 elderly participants (150 from Brazil, 150 from Colombia) recruited at neighborhood social centers. Mobility was assessed with the Mobility Assessment Tool, and compared with the Short Physical Performance Battery score and self-reported functional limitations. Reliability was calculated using intraclass correlation coefficients. Multiple linear regression analyses were used to assess associations among mobility assessment tools and health, and sociodemographic variables. A significant gradient of increasing Mobility Assessment Tool score with better physical function was observed for both self-reported and objective measures, and in each city. Associations between self-reported mobility and health were strong, and significant. Mobility Assessment Tool scores were lower in women at both sites. Intraclass correlation coefficients of the Mobility Assessment Tool were 0.94 (95% confidence interval 0.90-0.97) in Brazil and 0.81 (95% confidence interval 0.66-0.91) in Colombia. Mobility Assessment Tool scores were lower in Manizales than in Natal after adjustment by Short Physical Performance Battery, self-rated health and sex. These results provide evidence for high reliability and good validity of the Mobility Assessment Tool in its Spanish and Portuguese versions used in Latin American populations. In addition, the Mobility Assessment Tool can detect mobility differences related to environmental features that cannot be captured by objective performance measures. © 2013 Japan Geriatrics Society.

Assessing teamwork performance in obstetrics: A systematic search and review of validated tools.

Science.gov (United States)

Fransen, Annemarie F; de Boer, Liza; Kienhorst, Dieneke; Truijens, Sophie E; van Runnard Heimel, Pieter J; Oei, S Guid

2017-09-01

Teamwork performance is an essential component for the clinical efficiency of multi-professional teams in obstetric care. As patient safety is related to teamwork performance, it has become an important learning goal in simulation-based education. In order to improve teamwork performance, reliable assessment tools are required. These can be used to provide feedback during training courses, or to compare learning effects between different types of training courses. The aim of the current study is to (1) identify the available assessment tools to evaluate obstetric teamwork performance in a simulated environment, and (2) evaluate their psychometric properties in order to identify the most valuable tool(s) to use. We performed a systematic search in PubMed, MEDLINE, and EMBASE to identify articles describing assessment tools for the evaluation of obstetric teamwork performance in a simulated environment. In order to evaluate the quality of the identified assessment tools the standards and grading rules have been applied as recommended by the Accreditation Council for Graduate Medical Education (ACGME) Committee on Educational Outcomes. The included studies were also assessed according to the Oxford Centre for Evidence Based Medicine (OCEBM) levels of evidence. This search resulted in the inclusion of five articles describing the following six tools: Clinical Teamwork Scale, Human Factors Rating Scale, Global Rating Scale, Assessment of Obstetric Team Performance, Global Assessment of Obstetric Team Performance, and the Teamwork Measurement Tool. Based on the ACGME guidelines we assigned a Class 3, level C of evidence, to all tools. Regarding the OCEBM levels of evidence, a level 3b was assigned to two studies and a level 4 to four studies. The Clinical Teamwork Scale demonstrated the most comprehensive validation, and the Teamwork Measurement Tool demonstrated promising results, however it is recommended to further investigate its reliability. Copyright © 2017

Cross-cultural validity of standardized motor development screening and assessment tools: a systematic review.

Science.gov (United States)

Mendonça, Bianca; Sargent, Barbara; Fetters, Linda

2016-12-01

To investigate whether standardized motor development screening and assessment tools that are used to evaluate motor abilities of children aged 0 to 2 years are valid in cultures other than those in which the normative sample was established. This was a systematic review in which six databases were searched. Studies were selected based on inclusion/exclusion criteria and appraised for evidence level and quality. Study variables were extracted. Twenty-three studies representing six motor development screening and assessment tools in 16 cultural contexts met the inclusion criteria: Alberta Infant Motor Scale (n=7), Ages and Stages Questionnaire, 3rd edition (n=2), Bayley Scales of Infant and Toddler Development, 3rd edition (n=8), Denver Developmental Screening Test, 2nd edition (n=4), Harris Infant Neuromotor Test (n=1), and Peabody Developmental Motor Scales, 2nd edition (n=1). Thirteen studies found significant differences between the cultural context and normative sample. Two studies established reliability and/or validity of standardized motor development assessments in high-risk infants from different cultural contexts. Five studies established new population norms. Eight studies described the cross-cultural adaptation of a standardized motor development assessment. Standardized motor development assessments have limited validity in cultures other than that in which the normative sample was established. Their use can result in under- or over-referral for services. © 2016 Mac Keith Press.

Validity Evidence for the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT).

Science.gov (United States)

Breimer, Gerben E; Haji, Faizal A; Cinalli, Giuseppe; Hoving, Eelco W; Drake, James M

2017-02-01

Growing demand for transparent and standardized methods for evaluating surgical competence prompted the construction of the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). To provide validity evidence of the NEVAT by reporting on the tool's internal structure and its relationship with surgical expertise during simulation-based training. The NEVAT was used to assess performance of trainees and faculty at an international neuroendoscopy workshop. All participants performed an endoscopic third ventriculostomy (ETV) on a synthetic simulator. Participants were simultaneously scored by 2 raters using the NEVAT procedural checklist and global rating scale (GRS). Evidence of internal structure was collected by calculating interrater reliability and internal consistency of raters' scores. Evidence of relationships with other variables was collected by comparing the ETV performance of experts, experienced trainees, and novices using Jonckheere's test (evidence of construct validity). Thirteen experts, 11 experienced trainees, and 10 novices participated. The interrater reliability by the intraclass correlation coefficient for the checklist and GRS was 0.82 and 0.94, respectively. Internal consistency (Cronbach's α) for the checklist and the GRS was 0.74 and 0.97, respectively. Median scores with interquartile range on the checklist and GRS for novices, experienced trainees, and experts were 0.69 (0.58-0.86), 0.85 (0.63-0.89), and 0.85 (0.81-0.91) and 3.1 (2.5-3.8), 3.7 (2.2-4.3) and 4.6 (4.4-4.9), respectively. Jonckheere's test showed that the median checklist and GRS score increased with performer expertise ( P = .04 and .002, respectively). This study provides validity evidence for the NEVAT to support its use as a standardized method of evaluating neuroendoscopic competence during simulation-based training. Copyright © 2016 by the Congress of Neurological Surgeons

Designing and Assessing the Validity and Reliability of the Hospital Readiness Assessment Tools to Conducting Quality Improvement Program

Directory of Open Access Journals (Sweden)

Kamal Gholipoor

2016-09-01

Full Text Available Background and objectives : Identifying the readiness of hospital and its strengths and weaknesses can be useful in developing appropriate planning and situation analyses and management to getting effective in clinical audit programs. The aim of this study was to design and assess the validity of the Hospital Readiness Assessment Tools to conduct quality improvement and clinical audit programs. Material and Methods: In this study, based on the results of a systematic review of literature, an initial questionnaire with 77 items was designed. Questionnaire content validity was reviewed by experts in the field of hospital management and quality improvement in Tabriz University of Medical Sciences. For this purpose, 20 questionnaires were sent to experts. Finally, 15 participants returned completed questionnaire. Questionnaire validity was reviewed and confirmed based on Content Validity Index and Content Validity Ratio. Questionnaire reliability was confirmed based on Cronbach's alpha index (α = 0.96 in a pilot study by participation of 30 hospital managers. Results: The results showed that the final questionnaire contains 54 questions as nine category as: data and information (9 items, teamwork (12 questions, resources (5 questions, patient and education (5, intervention design and implementation (5 questions, clinical audit management (4 questions, human resources (6 questions, evidence and standard (4 items and evaluation and feedback (4 items. The final questionnaire content validity index was 0.91 and final questionnaire Cronbach's alpha coefficient was 0.96. Conclusion: Considering the relative good validity and reliability of the designed tool in this study, it appears that the questionnaire can be used to identify and assess the readiness of hospitals for quality improvement and clinical audit program implementation

Development and validation of the Learning Disabilities Needs Assessment Tool (LDNAT), a HoNOS-based needs assessment tool for use with people with intellectual disability.

Science.gov (United States)

Painter, J; Trevithick, L; Hastings, R P; Ingham, B; Roy, A

2016-12-01

In meeting the needs of individuals with intellectual disabilities (ID) who access health services, a brief, holistic assessment of need is useful. This study outlines the development and testing of the Learning Disabilities Needs Assessment Tool (LDNAT), a tool intended for this purpose. An existing mental health (MH) tool was extended by a multidisciplinary group of ID practitioners. Additional scales were drafted to capture needs across six ID treatment domains that the group identified. LDNAT ratings were analysed for the following: item redundancy, relevance, construct validity and internal consistency (n = 1692); test-retest reliability (n = 27); and concurrent validity (n = 160). All LDNAT scales were deemed clinically relevant with little redundancy apparent. Principal component analysis indicated three components (developmental needs, challenging behaviour, MH and well-being). Internal consistency was good (Cronbach alpha 0.80). Individual item test-retest reliability was substantial-near perfect for 20 scales and slight-fair for three scales. Overall reliability was near perfect (intra-class correlation = 0.91). There were significant associations with five of six condition-specific measures, i.e. the Waisman Activities of Daily Living Scale (general ability/disability), Threshold Assessment Grid (risk), Behaviour Problems Inventory for Individuals with Intellectual Disabilities-Short Form (challenging behaviour) Social Communication Questionnaire (autism) and a bespoke physical health questionnaire. Additionally, the statistically significant correlations between these tools and the LDNAT components made sense clinically. There were no statistically significant correlations with the Psychiatric Assessment Schedules for Adults with Developmental Disabilities (a measure of MH symptoms in people with ID). The LDNAT had clinically utility when rating the needs of people with ID prior to condition-specific assessment(s). Analyses of internal

Development and validation of a new assessment tool for suturing skills in medical students.

Science.gov (United States)

Sundhagen, Henriette Pisani; Almeland, Stian Kreken; Hansson, Emma

2018-01-01

In recent years, emphasis has been put on that medical student should demonstrate pre-practice/pre-registration core procedural skills to ensure patient safety. Nonetheless, the formal teaching and training of basic suturing skills to medical students have received relatively little attention and there is no standard for what should be tested and how. The aim of this study was to develop and validate, using scientific methods, a tool for assessment of medical students' suturing skills, measuring both micro- and macrosurgical qualities. A tool was constructed and content, construct, concurrent validity, and inter-rater, inter-item, inter-test reliability were tested. Three groups were included: students with no training in suturing skills, students who have had training, plastic surgery. The results show promising reliability and validity when assessing novice medical students' suturing skills. Further studies are needed on implementation of the instrument. Moreover, how the instrument can be used to give formative feedback, evaluate if a required standard is met and for curriculum development needs further investigation.Level of Evidence: Not ratable.

Reliability and Validity of the Korean Cancer Pain Assessment Tool (KCPAT)

Science.gov (United States)

Kim, Jeong A; Lee, Juneyoung; Park, Jeanno; Lee, Myung Ah; Yeom, Chang Hwan; Jang, Se Kwon; Yoon, Duck Mi; Kim, Jun Suk

2005-01-01

The Korean Cancer Pain Assessment Tool (KCPAT), which was developed in 2003, consists of questions concerning the location of pain, the nature of pain, the present pain intensity, the symptoms associated with the pain, and psychosocial/spiritual pain assessments. This study was carried out to evaluate the reliability and validity of the KCPAT. A stratified, proportional-quota, clustered, systematic sampling procedure was used. The study population (903 cancer patients) was 1% of the target population (90,252 cancer patients). A total of 314 (34.8%) questionnaires were collected. The results showed that the average pain score (5 point on Likert scale) according to the cancer type and the at-present average pain score (VAS, 0-10) were correlated (r=0.56, p<0.0001), and showed moderate agreement (kappa=0.364). The mean satisfaction score was 3.8 (1-5). The average time to complete the questionnaire was 8.9 min. In conclusion, the KCPAT is a reliable and valid instrument for assessing cancer pain in Koreans. PMID:16224166

Validation of an Albuminuria Self-assessment Tool in the Multi-Ethnic Study of Atherosclerosis.

Science.gov (United States)

Tanner, Rikki M; Woodward, Mark; Peralta, Carmen; Warnock, David G; Gutiérrez, Orlando; Shimbo, Daichi; Kramer, Holly; Katz, Ronit; Muntner, Paul

2015-11-05

We previously developed an 8-item self-assessment tool to identify individuals with a high probability of having albuminuria. This tool was developed and externally validated among non-Hispanic Whites and non-Hispanic Blacks. We sought to validate it in a multi-ethnic cohort that also included Hispanics and Chinese Americans. This is a cross-sectional study. Data were collected using standardized questionnaires and spot urine samples at a baseline examination in 2000-2002. The 8 items in the self-assessment tool include age, race, gender, current cigarette smoking, history of diabetes, hypertension, or stroke, and self-rated health. Of 6,814 community-dwelling adults aged 45-84 years participating in the Multi-Ethnic Study of Atherosclerosis (MESA), 6,542 were included in the primary analysis. Albuminuria was defined as urine albumin-to-creatinine ratio ≥ 30 mg/g at baseline. Among non-Hispanic Whites, non-Hispanic Blacks, Hispanics, and Chinese Americans, the prevalence of albuminuria was 6.0%, 11.3%, 11.6%, and 10.8%, respectively. The c-statistic for discriminating participants with and without albuminuria was .731 (95% CI: .692, .771), .728 (95% CI: .687, .761), .747 (95% CI: .709, .784), and .761 (95% CI: .699, .814) for non-Hispanic Whites, non-Hispanic Blacks, Hispanics, and Chinese Americans, respectively. The self-assessment tool over-estimated the probability of albuminuria for non-Hispanic Whites and Blacks, but was well-calibrated for Hispanics and Chinese Americans. The albuminuria self-assessment tool maintained good test characteristics in this large multi-ethnic cohort, suggesting it may be helpful for increasing awareness of albuminuria in an ethnically diverse population.

Doloplus-2, a valid tool for behavioural pain assessment?

Directory of Open Access Journals (Sweden)

Loge Jon H

2007-12-01

Full Text Available Abstract Background The Doloplus-2 is used for behavioural pain assessment in cognitively impaired patients. Little data exists on the psychometric properties of the Doloplus-2. Our objectives were to test the criterion validity and inter-rater reliability of the Doloplus-2, and to explore a design for validations of behavioural pain assessment tools. Methods Fifty-one nursing home patients and 22 patients admitted to a geriatric hospital ward were included. All were cognitively impaired and unable to self-report pain. Each patient was examined by an expert in pain evaluation and treatment, who rated the pain on a numerical rating scale. The ratings were based on information from the medical record, reports from nurses and patients (if possible about pain during the past 24 hours, and a clinical examination. These ratings were used as pain criterion. The Doloplus-2 was administered by the attending nurse. Regression analyses were used to estimate the ability of the Doloplus-2 to explain the expert's ratings. The inter-rater reliability of the Doloplus-2 was evaluated in 16 patients by comparing the ratings of two nurses administrating the Doloplus-2. Results There was no association between the Doloplus-2 and the expert's pain ratings (R2 = 0.02. There was an association (R2 = 0.54 between the expert's ratings and the Doloplus-2 scores in a subgroup of 16 patients assessed by a geriatric expert nurse (the most experienced Doloplus-2 administrator. The inter-rater reliability between the Doloplus-2 administrators assessed by the intra-class coefficient was 0.77. The pain expert's ratings were compared with ratings of two independent geriatricians in a sub sample of 15, and were found satisfactory (intra-class correlation 0.74. Conclusion It was challenging to conduct such a study in patients with cognitive impairment and the study has several limitations. The results do not support the validity of the Doloplus-2 in its present version and they

The Development and Validation of the Contextualized Assessment Tool for Risk and Protection Management (CAT-RPM)

Science.gov (United States)

Bower, Julie M.; Carroll, Annemaree; Ashman, Adrian

2014-01-01

Traditional resources for ascertaining risk and protection for disengaged youth are often unsuitable, due to the stamina and skill required to complete them. Many of these tools assess risk without considering participants' potential for personal growth. The present study outlines the development and initial validation of a tool titled the…

Validating Appetite Assessment Tools among Patients Receiving Hemodialysis

Science.gov (United States)

Molfino, Alessio; Kaysen, George A.; Chertow, Glenn M.; Doyle, Julie; Delgado, Cynthia; Dwyer, Tjien; Laviano, Alessandro; Fanelli, Filippo Rossi; Johansen, Kirsten L.

2016-01-01

Objective To test the performance of appetite assessment tools among patients receiving hemodialysis. Design Cross-sectional. Setting Seven dialysis facilities in Northern California. Subjects 221 patients receiving hemodialysis. Intervention We assessed five appetite assessment tools [self-assessment of appetite, subjective assessment of appetite, visual analogue scale (VAS), Functional Assessment of Anorexia/Cachexia Therapy (FAACT) score and the Anorexia Questionnaire (AQ)]. Main outcome measures Reported food intake, normalized protein catabolic rate (nPCR), and change in body weight were used as criterion measures, and we assessed associations among the appetite tools and biomarkers associated with nutrition and inflammation. Patients were asked to report their appetite and the percentage of food eaten (from 0% to 100%) during the last meal compared to usual intake. Results Fifty-eight (26%) patients reported food intake ≤50% (defined as poor appetite). The prevalence of anorexia was 12% by self-assessment of appetite, 6% by subjective assessment of appetite, 24% by VAS, 17% by FAACT score, and 12% by AQ. All tools were significantly associated with food intake ≤50% (pappetite. The FAACT score and the VAS had the strongest association with food intake ≤50% (c-statistic 0.80 and 0.76). Patients with food intake ≤50% reported weight loss more frequently than patients without low intake (36% vs 22%) and weight gain less frequently (19% vs 35%; p=0.03). nPCR was lower among anorexic patients based on the VAS (1.1 ± 0.3 vs 1.2 ± 0.3, p=0.03). Ln IL-6 correlated inversely with food intake (p=0.03), but neither IL-6 nor CRP correlated with any of the appetite tools. Furthermore, only the self-assessment of appetite was significantly associated with serum albumin (p=0.02), prealbumin (p=0.02) and adiponectin concentrations (p=0.03). Conclusions Alternative appetite assessment tools yielded widely different estimates of the prevalence of anorexia in

Development of the Music Therapy Assessment Tool for Advanced Huntington's Disease: A Pilot Validation Study.

Science.gov (United States)

O'Kelly, Julian; Bodak, Rebeka

2016-01-01

Case studies of people with Huntington's disease (HD) report that music therapy provides a range of benefits that may improve quality of life; however, no robust music therapy assessment tools exist for this population. Develop and conduct preliminary psychometric testing of a music therapy assessment tool for patients with advanced HD. First, we established content and face validity of the Music Therapy Assessment Tool for Advanced HD (MATA-HD) through focus groups and field testing. Second, we examined psychometric properties of the resulting MATA-HD in terms of its construct validity, internal consistency, and inter-rater and intra-rater reliability over 10 group music therapy sessions with 19 patients. The resulting MATA-HD included a total of 15 items across six subscales (Arousal/Attention, Physical Presentation, Communication, Musical, Cognition, and Psychological/Behavioral). We found good construct validity (r ≥ 0.7) for Mood, Communication Level, Communication Effectiveness, Choice, Social Behavior, Arousal, and Attention items. Cronbach's α of 0.825 indicated good internal consistency across 11 items with a common focus of engagement in therapy. The inter-rater reliability (IRR) Intra-Class Coefficient (ICC) scores averaged 0.65, and a mean intra-rater ICC reliability of 0.68 was obtained. Further training and retesting provided a mean of IRR ICC of 0.7. Preliminary data indicate that the MATA-HD is a promising tool for measuring patient responses to music therapy interventions across psychological, physical, social, and communication domains of functioning in patients with advanced HD. © the American Music Therapy Association 2016. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Development, Evaluation, and Validation of Environmental Assessment Tools to Evaluate the College Nutrition Environment

Science.gov (United States)

Freedman, Marjorie R.

2010-01-01

Objective: To develop, evaluate, and validate 2 nutrition environment assessment tools (surveys), for specific use in combating overweight on college/university campuses. Participants and Methods: Invitations to complete surveys were e-mailed to food service and health center directors at 47 universities, Winter 2008. Overall response rate was…

Validation of the spiritual distress assessment tool in older hospitalized patients

Directory of Open Access Journals (Sweden)

Monod Stefanie

2012-03-01

Full Text Available Abstract Background The Spiritual Distress Assessment Tool (SDAT is a 5-item instrument developed to assess unmet spiritual needs in hospitalized elderly patients and to determine the presence of spiritual distress. The objective of this study was to investigate the SDAT psychometric properties. Methods This cross-sectional study was performed in a Geriatric Rehabilitation Unit. Patients (N = 203, aged 65 years and over with Mini Mental State Exam score ≥ 20, were consecutively enrolled over a 6-month period. Data on health, functional, cognitive, affective and spiritual status were collected upon admission. Interviews using the SDAT (score from 0 to 15, higher scores indicating higher distress were conducted by a trained chaplain. Factor analysis, measures of internal consistency (inter-item and item-to-total correlations, Cronbach α, and reliability (intra-rater and inter-rater were performed. Criterion-related validity was assessed using the Functional Assessment of Chronic Illness Therapy-Spiritual well-being (FACIT-Sp and the question "Are you at peace?" as criterion-standard. Concurrent and predictive validity were assessed using the Geriatric Depression Scale (GDS, occurrence of a family meeting, hospital length of stay (LOS and destination at discharge. Results SDAT scores ranged from 1 to 11 (mean 5.6 ± 2.4. Overall, 65.0% (132/203 of the patients reported some spiritual distress on SDAT total score and 22.2% (45/203 reported at least one severe unmet spiritual need. A two-factor solution explained 60% of the variance. Inter-item correlations ranged from 0.11 to 0.41 (eight out of ten with P Conclusions SDAT has acceptable psychometrics properties and appears to be a valid and reliable instrument to assess spiritual distress in elderly hospitalized patients.

Assessing children’s competence to consent in research by a standardized tool: a validity study

Science.gov (United States)

2012-01-01

Background Currently over 50% of drugs prescribed to children have not been evaluated properly for use in their age group. One key reason why children have been excluded from clinical trials is that they are not considered able to exercise meaningful autonomy over the decision to participate. Dutch law states that competence to consent can be presumed present at the age of 12 and above; however, in pediatric practice children’s competence is not that clearly presented and the transition from assent to active consent is gradual. A gold standard for competence assessment in children does not exist. In this article we describe a study protocol on the development of a standardized tool for assessing competence to consent in research in children and adolescents. Methods/design In this study we modified the MacCAT-CR, the best evaluated competence assessment tool for adults, for use in children and adolescents. We will administer the tool prospectively to a cohort of pediatric patients from 6 to18 years during the selection stages of ongoing clinical trials. The outcomes of the MacCAT-CR interviews will be compared to a reference standard, established by the judgments of clinical investigators, and an expert panel consisting of child psychiatrists, child psychologists and medical ethicists. The reliability, criterion-related validity and reproducibility of the tool will be determined. As MacCAT-CR is a multi-item scale consisting of 13 items, power was justified at 130–190 subjects, providing a minimum of 10–15 observations per item. MacCAT-CR outcomes will be correlated with age, life experience, IQ, ethnicity, socio-economic status and competence judgment of the parent(s). It is anticipated that 160 participants will be recruited over 2 years to complete enrollment. Discussion A validity study on an assessment tool of competence to consent is strongly needed in research practice, particularly in the child and adolescent population. In this study we will establish

Assessing children’s competence to consent in research by a standardized tool: a validity study

Directory of Open Access Journals (Sweden)

Hein Irma M

2012-09-01

Full Text Available Abstract Background Currently over 50% of drugs prescribed to children have not been evaluated properly for use in their age group. One key reason why children have been excluded from clinical trials is that they are not considered able to exercise meaningful autonomy over the decision to participate. Dutch law states that competence to consent can be presumed present at the age of 12 and above; however, in pediatric practice children’s competence is not that clearly presented and the transition from assent to active consent is gradual. A gold standard for competence assessment in children does not exist. In this article we describe a study protocol on the development of a standardized tool for assessing competence to consent in research in children and adolescents. Methods/design In this study we modified the MacCAT-CR, the best evaluated competence assessment tool for adults, for use in children and adolescents. We will administer the tool prospectively to a cohort of pediatric patients from 6 to18 years during the selection stages of ongoing clinical trials. The outcomes of the MacCAT-CR interviews will be compared to a reference standard, established by the judgments of clinical investigators, and an expert panel consisting of child psychiatrists, child psychologists and medical ethicists. The reliability, criterion-related validity and reproducibility of the tool will be determined. As MacCAT-CR is a multi-item scale consisting of 13 items, power was justified at 130–190 subjects, providing a minimum of 10–15 observations per item. MacCAT-CR outcomes will be correlated with age, life experience, IQ, ethnicity, socio-economic status and competence judgment of the parent(s. It is anticipated that 160 participants will be recruited over 2 years to complete enrollment. Discussion A validity study on an assessment tool of competence to consent is strongly needed in research practice, particularly in the child and adolescent population. In

Reliability and criterion-related validity testing (construct) of the Endotracheal Suction Assessment Tool (ESAT©).

Science.gov (United States)

Davies, Kylie; Bulsara, Max K; Ramelet, Anne-Sylvie; Monterosso, Leanne

2018-05-01

To establish criterion-related construct validity and test-retest reliability for the Endotracheal Suction Assessment Tool© (ESAT©). Endotracheal tube suction performed in children can significantly affect clinical stability. Previously identified clinical indicators for endotracheal tube suction were used as criteria when designing the ESAT©. Content validity was reported previously. The final stages of psychometric testing are presented. Observational testing was used to measure construct validity and determine whether the ESAT© could guide "inexperienced" paediatric intensive care nurses' decision-making regarding endotracheal tube suction. Test-retest reliability of the ESAT© was performed at two time points. The researchers and paediatric intensive care nurse "experts" developed 10 hypothetical clinical scenarios with predetermined endotracheal tube suction outcomes. "Experienced" (n = 12) and "inexperienced" (n = 14) paediatric intensive care nurses were presented with the scenarios and the ESAT© guiding decision-making about whether to perform endotracheal tube suction for each scenario. Outcomes were compared with those predetermined by the "experts" (n = 9). Test-retest reliability of the ESAT© was measured at two consecutive time points (4 weeks apart) with "experienced" and "inexperienced" paediatric intensive care nurses using the same scenarios and tool to guide decision-making. No differences were observed between endotracheal tube suction decisions made by "experts" (n = 9), "inexperienced" (n = 14) and "experienced" (n = 12) nurses confirming the tool's construct validity. No differences were observed between groups for endotracheal tube suction decisions at T1 and T2. Criterion-related construct validity and test-retest reliability of the ESAT© were demonstrated. Further testing is recommended to confirm reliability in the clinical setting with the "inexperienced" nurse to guide decision-making related to endotracheal tube

Reliable and valid assessment of Lichtenstein hernia repair skills

DEFF Research Database (Denmark)

Carlsen, C G; Lindorff Larsen, Karen; Funch-Jensen, P

2014-01-01

PURPOSE: Lichtenstein hernia repair is a common surgical procedure and one of the first procedures performed by a surgical trainee. However, formal assessment tools developed for this procedure are few and sparsely validated. The aim of this study was to determine the reliability and validity...... of an assessment tool designed to measure surgical skills in Lichtenstein hernia repair. METHODS: Key issues were identified through a focus group interview. On this basis, an assessment tool with eight items was designed. Ten surgeons and surgical trainees were video recorded while performing Lichtenstein hernia...... a significant difference between the three groups which indicates construct validity, p skills can be assessed blindly by a single rater in a reliable and valid fashion with the new procedure-specific assessment tool. We recommend this tool for future assessment...

Assessing peristomal skin changes in ostomy patients: validation of the Ostomy Skin Tool.

Science.gov (United States)

Jemec, G B; Martins, L; Claessens, I; Ayello, E A; Hansen, A S; Poulsen, L H; Sibbald, R G

2011-02-01

Peristomal skin problems are common and are treated by a variety of health professionals. Clear and consistent communication among these professionals is therefore particularly important. The Ostomy Skin Tool (OST) is a new assessment instrument for the extent and severity of peristomal skin conditions. Formal tests of reliability and validity are necessary for its use in clinical practice, research, and education. To estimate inter- and intra nurse assessment variability of the OST and validity by comparison to a 'gold standard' (GS) defined by an expert panel. Thirty photographs of peristomal skin were presented twice to 20 ostomy care nurses--10 from Denmark (DK) and 10 from Spain (ES)--to determine intra- and inter nurse assessment variability. The same photographs were presented to an international group of experts (dermatologist and ostomy care nurses), to establish a GS for comparison and validation of the results. A high intra-nurse assessment agreement, κ=0·84, was found with no differences in the intra-nurse assessments from the two groups of nurses (DK and ES). The inter-nurse assessment agreement was 'moderate to good', κ=0·54, with the agreement between the experts higher, κ=0·70. A high correlation between the scores from the nurses and the GS were seen in the lower part of the two scales [Discoloration, Erosion, Tissue overgrowth (DET) score<7)]. The study supported the validity of the OST. It is suggested that a categorical scale can be used to illustrate the severity of the DET scores. © 2011 The Authors. BJD © 2011 British Association of Dermatologists.

The Bristol Radiology Report Assessment Tool (BRRAT): Developing a workplace-based assessment tool for radiology reporting skills

International Nuclear Information System (INIS)

Wallis, A.; Edey, A.; Prothero, D.; McCoubrie, P.

2013-01-01

Aim: To review the development of a workplace-based assessment tool to assess the quality of written radiology reports and assess its reliability, feasibility, and validity. Materials and methods: A comprehensive literature review and rigorous Delphi study enabled the development of the Bristol Radiology Report Assessment Tool (BRRAT), which consists of 19 questions and a global assessment score. Three assessors applied the assessment tool to 240 radiology reports provided by 24 radiology trainees. Results: The reliability coefficient for the 19 questions was 0.79 and the equivalent coefficient for the global assessment scores was 0.67. Generalizability coefficients demonstrate that higher numbers of assessors and assessments are needed to reach acceptable levels of reliability for summative assessments due to assessor subjectivity. Conclusion: The study methodology gives good validity and strong foundation in best-practice. The assessment tool developed for radiology reporting is reliable and most suited to formative assessments

Development, initial reliability and validity testing of an observational tool for assessing technical skills of operating room nurses.

Science.gov (United States)

Sevdalis, Nick; Undre, Shabnam; Henry, Janet; Sydney, Elaine; Koutantji, Mary; Darzi, Ara; Vincent, Charles A

2009-09-01

The recent emergence of the Systems Approach to the safety and quality of surgical care has triggered individual and team skills training modules for surgeons and anaesthetists and relevant observational assessment tools have been developed. To develop an observational tool that captures operating room (OR) nurses' technical skill and can be used for assessment and training. The Imperial College Assessment of Technical Skills for Nurses (ICATS-N) assesses (i) gowning and gloving, (ii) setting up instrumentation, (iii) draping, and (iv) maintaining sterility. Three to five observable behaviours have been identified for each skill and are rated on 1-6 scales. Feasibility and aspects of reliability and validity were assessed in 20 simulation-based crisis management training modules for trainee nurses and doctors, carried out in a Simulated Operating Room. The tool was feasible to use in the context of simulation-based training. Satisfactory reliability (Cronbach alpha) was obtained across trainers' and trainees' scores (analysed jointly and separately). Moreover, trainer nurse's ratings of the four skills correlated positively, thus indicating adequate content validity. Trainer's and trainees' ratings did not correlate. Assessment of OR nurses' technical skill is becoming a training priority. The present evidence suggests that the ICATS-N could be considered for use as an assessment/training tool for junior OR nurses.

Breastfeeding assessment tools

International Nuclear Information System (INIS)

Bizouerne, Cécile; Kerac, Marko; Macgrath, Marie

2014-01-01

Full text: Breastfeeding plays a major role in reducing the global burden of child mortality and under-nutrition. Whilst many programmes aim to support breastfeeding and prevent feeding problems occurring, interventions are also needed once they have developed. In this situation, accurate assessment of a problem is critical to inform prognosis and enables tailored, appropriate treatment. The presentation will present a review, which aims to identify breastfeeding assessment tools/checklists for use in assessing malnourished infants in poor resource settings. The literature review identified 24 breastfeeding assessment tools, and 41 validation studies. Evidence underpinning most of the tools was mainly low quality, and conducted in high-income countries and hospital settings. The presentation will describe the main findings of the literature review and propose recommendations for improving existing tools in order to appropriately assess malnourished infants and enable early, appropriate intervention and treatment of malnutrition. (author)

The Bristol Radiology Report Assessment Tool (BRRAT): developing a workplace-based assessment tool for radiology reporting skills.

Science.gov (United States)

Wallis, A; Edey, A; Prothero, D; McCoubrie, P

2013-11-01

To review the development of a workplace-based assessment tool to assess the quality of written radiology reports and assess its reliability, feasibility, and validity. A comprehensive literature review and rigorous Delphi study enabled the development of the Bristol Radiology Report Assessment Tool (BRRAT), which consists of 19 questions and a global assessment score. Three assessors applied the assessment tool to 240 radiology reports provided by 24 radiology trainees. The reliability coefficient for the 19 questions was 0.79 and the equivalent coefficient for the global assessment scores was 0.67. Generalizability coefficients demonstrate that higher numbers of assessors and assessments are needed to reach acceptable levels of reliability for summative assessments due to assessor subjectivity. The study methodology gives good validity and strong foundation in best-practice. The assessment tool developed for radiology reporting is reliable and most suited to formative assessments. Copyright © 2013 The Royal College of Radiologists. Published by Elsevier Ltd. All rights reserved.

Validation of a tool for assessing the quality of pharmaceutical services

Directory of Open Access Journals (Sweden)

Cosendey Marly Aparecida E.

2003-01-01

Full Text Available This paper presents the validation process for a tool assessing basic pharmaceutical services through an analysis of the implementation of a Basic Pharmaceuticals Distribution Program by the Brazilian Federal government. The process began with the drafting of a theoretical model, based on a state-of-the-art review and allowing the selection of various conceptual dimensions and respective criteria that best represented the construct. The second step involved weighting indicators for the construction of quality scores. Three models were tested for ranking implementation levels, and seven simulations were conducted, determining the score most closely reflecting the selected indicators in two different matrices. The objective was to select the most coherent and consistent version between implementation levels and expected outcomes, while simultaneously enhancing validity of chosen criteria. Testing of the various models and the results obtained showed that augmenting the validity of the study was possible without altering data. This endeavor is justified in understanding the scope and limitations of these measurements and of the choices involved in issues concerning their weighting and interpretation.

Assessing trauma and mental health in refugee children and youth: a systematic review of validated screening and measurement tools.

Science.gov (United States)

Gadeberg, A K; Montgomery, E; Frederiksen, H W; Norredam, M

2017-06-01

: It is estimated that children below 18 years constitute 50% of the refugee population worldwide, which is the highest figure in a decade. Due to conflicts like the Syrian crises, children are continuously exposed to traumatic events. Trauma exposure can cause mental health problems that may in turn increase the risk of morbidity and mortality. Tools such as questionnaires and interview guides are being used extensively, despite the fact that only a few have been tested and their validity confirmed in refugee children and youth. : Our aim was to provide a systematic review of the validated screening and measurement tools available for assessment of trauma and mental health among refugee children and youth. : We systematically searched the databases PubMed, PsycINFO and PILOTS. The search yielded 913 articles and 97 were retained for further investigation. In accordance with the PRISMA guidelines two authors performed the eligibility assessment. The full text of 23 articles was assessed and 9 met the eligibility criteria. Results : Only nine studies had validated trauma and mental health tools in refugee children and youth populations. A serious lack of validated tools for refugee children below the age of 6 was identified. : There is a lack of validated trauma and mental health tools, especially for refugees below the age of 6. Detection and treatment of mental health issues among refugee children and youth should be a priority both within the scientific community and in practice in order to reduce morbidity and mortality. © The Author 2017. Published by Oxford University Press on behalf of the European Public Health Association. All rights reserved.

Assessing physiotherapists' communication skills for promoting patient autonomy for self-management: reliability and validity of the communication evaluation in rehabilitation tool.

Science.gov (United States)

Murray, Aileen; Hall, Amanda; Williams, Geoffrey C; McDonough, Suzanne M; Ntoumanis, Nikos; Taylor, Ian; Jackson, Ben; Copsey, Bethan; Hurley, Deirdre A; Matthews, James

2018-02-27

To assess the inter-rater reliability and concurrent validity of the Communication Evaluation in Rehabilitation Tool, which aims to externally assess physiotherapists competency in using Self-Determination Theory-based communication strategies in practice. Audio recordings of initial consultations between 24 physiotherapists and 24 patients with chronic low back pain in four hospitals in Ireland were obtained as part of a larger randomised controlled trial. Three raters, all of whom had Ph.Ds in psychology and expertise in motivation and physical activity, independently listened to the 24 audio recordings and completed the 18-item Communication Evaluation in Rehabilitation Tool. Inter-rater reliability between all three raters was assessed using intraclass correlation coefficients. Concurrent validity was assessed using Pearson's r correlations with a reference standard, the Health Care Climate Questionnaire. The total score for the Communication Evaluation in Rehabilitation Tool is an average of all 18 items. Total scores demonstrated good inter-rater reliability (Intraclass Correlation Coefficient (ICC) = 0.8) and concurrent validity with the Health Care Climate Questionnaire total score (range: r = 0.7-0.88). Item-level scores of the Communication Evaluation in Rehabilitation Tool identified five items that need improvement. Results provide preliminary evidence to support future use and testing of the Communication Evaluation in Rehabilitation Tool. Implications for Rehabilitation Promoting patient autonomy is a learned skill and while interventions exist to train clinicians in these skills there are no tools to assess how well clinicians use these skills when interacting with a patient. The lack of robust assessment has severe implications regarding both the fidelity of clinician training packages and resulting outcomes for promoting patient autonomy. This study has developed a novel measurement tool Communication Evaluation in Rehabilitation Tool and a

Validation of a scenario-based assessment of critical thinking using an externally validated tool.

Science.gov (United States)

Buur, Jennifer L; Schmidt, Peggy; Smylie, Dean; Irizarry, Kris; Crocker, Carlos; Tyler, John; Barr, Margaret

2012-01-01

With medical education transitioning from knowledge-based curricula to competency-based curricula, critical thinking skills have emerged as a major competency. While there are validated external instruments for assessing critical thinking, many educators have created their own custom assessments of critical thinking. However, the face validity of these assessments has not been challenged. The purpose of this study was to compare results from a custom assessment of critical thinking with the results from a validated external instrument of critical thinking. Students from the College of Veterinary Medicine at Western University of Health Sciences were administered a custom assessment of critical thinking (ACT) examination and the externally validated instrument, California Critical Thinking Skills Test (CCTST), in the spring of 2011. Total scores and sub-scores from each exam were analyzed for significant correlations using Pearson correlation coefficients. Significant correlations between ACT Blooms 2 and deductive reasoning and total ACT score and deductive reasoning were demonstrated with correlation coefficients of 0.24 and 0.22, respectively. No other statistically significant correlations were found. The lack of significant correlation between the two examinations illustrates the need in medical education to externally validate internal custom assessments. Ultimately, the development and validation of custom assessments of non-knowledge-based competencies will produce higher quality medical professionals.

[Spanish validation of the MacArthur Competence Assessment Tool for Treatment interview to assess patients competence to consent treatment].

Science.gov (United States)

Alvarez Marrodán, Ignacio; Baón Pérez, Beatriz; Navío Acosta, Mercedes; López-Antón, Raul; Lobo Escolar, Elena; Ventura Faci, Tirso

2014-09-09

To validate the MacArthur Competence Assessment Tool for Treatment (MacCAT-T) Spanish version, which assesses the mental capacity of patients to consent treatment, by examining 4 areas (Understanding, Appreciation, Reasoning and Expressing a choice). 160 subjects (80 Internal Medicine inpatients, 40 Psychiatric inpatients and 40 healthy controls). MacCAT-T, Mini-Mental Status Examination (MMSE). Feasibility study, reliability and validity calculations (against to gold standard of clinical expert). Mean duration of the MacCAT-T interview was 18min. Inter-rater reliability: Intraclass correlation coefficient for Understanding=0.98, Appreciation=0.97, Reasoning=0.98, Expressing a choice=0.91. Internal consistency (Cronbach's alpha): Understanding=0.87, for Appreciation=0.76, for Reasoning=0.86. Patients considered to be incapable (gold standard) scored lower in all the MacCAT-T areas. Poor performance on the MacCAT-T was related to cognitive impairment assessed by MMSE. Spanish version of the MacCAT-T is feasible, reliable, and valid for assessing the capacity of patients to consent treatment. Copyright © 2013 Elsevier España, S.L. All rights reserved.

Improving Escalation of Care: Development and Validation of the Quality of Information Transfer Tool.

Science.gov (United States)

Johnston, Maximilian J; Arora, Sonal; Pucher, Philip H; Reissis, Yannis; Hull, Louise; Huddy, Jeremy R; King, Dominic; Darzi, Ara

2016-03-01

To develop and provide validity and feasibility evidence for the QUality of Information Transfer (QUIT) tool. Prompt escalation of care in the setting of patient deterioration can prevent further harm. Escalation and information transfer skills are not currently measured in surgery. This study comprised 3 phases: the development (phase 1), validation (phase 2), and feasibility analysis (phase 3) of the QUIT tool. Phase 1 involved identification of core skills needed for successful escalation of care through literature review and 33 semistructured interviews with stakeholders. Phase 2 involved the generation of validity evidence for the tool using a simulated setting. Thirty surgeons assessed a deteriorating postoperative patient in a simulated ward and escalated their care to a senior colleague. The face and content validity were assessed using a survey. Construct and concurrent validity of the tool were determined by comparing performance scores using the QUIT tool with those measured using the Situation-Background-Assessment-Recommendation (SBAR) tool. Phase 3 was conducted using direct observation of escalation scenarios on surgical wards in 2 hospitals. A 7-category assessment tool was developed from phase 1 consisting of 24 items. Twenty-one of 24 items had excellent content validity (content validity index >0.8). All 7 categories and 18 of 24 (P validity. The correlation between the QUIT and SBAR tools used was strong indicating concurrent validity (r = 0.694, P information transfer skills than nurses when faced with a deteriorating patient. A validated tool to assess information transfer for deteriorating surgical patients was developed and tested using simulation and real-time clinical scenarios. It may improve the quality and safety of patient care on the surgical ward.

External validation of a measurement tool to assess systematic reviews (AMSTAR.

Directory of Open Access Journals (Sweden)

Beverley J Shea

Full Text Available BACKGROUND: Thousands of systematic reviews have been conducted in all areas of health care. However, the methodological quality of these reviews is variable and should routinely be appraised. AMSTAR is a measurement tool to assess systematic reviews. METHODOLOGY: AMSTAR was used to appraise 42 reviews focusing on therapies to treat gastro-esophageal reflux disease, peptic ulcer disease, and other acid-related diseases. Two assessors applied the AMSTAR to each review. Two other assessors, plus a clinician and/or methodologist applied a global assessment to each review independently. CONCLUSIONS: The sample of 42 reviews covered a wide range of methodological quality. The overall scores on AMSTAR ranged from 0 to 10 (out of a maximum of 11 with a mean of 4.6 (95% CI: 3.7 to 5.6 and median 4.0 (range 2.0 to 6.0. The inter-observer agreement of the individual items ranged from moderate to almost perfect agreement. Nine items scored a kappa of >0.75 (95% CI: 0.55 to 0.96. The reliability of the total AMSTAR score was excellent: kappa 0.84 (95% CI: 0.67 to 1.00 and Pearson's R 0.96 (95% CI: 0.92 to 0.98. The overall scores for the global assessment ranged from 2 to 7 (out of a maximum score of 7 with a mean of 4.43 (95% CI: 3.6 to 5.3 and median 4.0 (range 2.25 to 5.75. The agreement was lower with a kappa of 0.63 (95% CI: 0.40 to 0.88. Construct validity was shown by AMSTAR convergence with the results of the global assessment: Pearson's R 0.72 (95% CI: 0.53 to 0.84. For the AMSTAR total score, the limits of agreement were -0.19+/-1.38. This translates to a minimum detectable difference between reviews of 0.64 'AMSTAR points'. Further validation of AMSTAR is needed to assess its validity, reliability and perceived utility by appraisers and end users of reviews across a broader range of systematic reviews.

The FORGE AHEAD clinical readiness consultation tool: a validated tool to assess clinical readiness for chronic disease care mobilization in Canada's First Nations.

Science.gov (United States)

Hayward, Mariam Naqshbandi; Mequanint, Selam; Paquette-Warren, Jann; Bailie, Ross; Chirila, Alexandra; Dyck, Roland; Green, Michael; Hanley, Anthony; Tompkins, Jordan; Harris, Stewart

2017-03-23

Given the astounding rates of diabetes and related complications, and the barriers to providing care present in Indigenous communities in Canada, intervention strategies that take into account contextual factors such as readiness to mobilize are needed to maximize improvements and increase the likelihood of success and sustainment. As part of the national FORGE AHEAD Program, we sought to develop, test and validate a clinical readiness consultation tool aimed at assessing the readiness of clinical teams working on-reserve in First Nations communities to participate in quality improvement (QI) to enhance diabetes care in Canada. A literature review was conducted to identify existing readiness tools. The ABCD - SAT was adapted using a consensus approach that emphasized a community-based participatory approach and prioritized the knowledge and wisdom held by community members. The tool was piloted with a group of 16 people from 7 provinces and 11 partnering communities to assess language use, clarity, relevance, format, and ease of completion using examples. Internal reliability analysis and convergence validity were conducted with data from 53 clinical team members from 11 First Nations communities (3-5 per community) who have participated in the FORGE AHEAD program. The 27-page Clinical Readiness Consultation Tool (CRCT) consists of five main components, 21 sub-components, and 74 items that are aligned with the Expanded Chronic Care Model. Five-point Likert scale feedback from the pilot ranged from 3.25 to 4.5. Length of the tool was reported as a drawback but respondents noted that all the items were needed to provide a comprehensive picture of the healthcare system. Results for internal consistency showed that all sub-components except for two were within acceptable ranges (0.77-0.93). The Team Structure and Function sub-component scale had a moderately significant positive correlation with the validated Team Climate Inventory, r = 0.45, p < 0.05. The

Clinical assessment of dysphagia in neurodegeneration (CADN): development, validity and reliability of a bedside tool for dysphagia assessment.

Science.gov (United States)

Vogel, Adam P; Rommel, Natalie; Sauer, Carina; Horger, Marius; Krumm, Patrick; Himmelbach, Marc; Synofzik, Matthis

2017-06-01

Screening assessments for dysphagia are essential in neurodegenerative disease. Yet there are no purpose-built tools to quantify swallowing deficits at bedside or in clinical trials. A quantifiable, brief, easy to administer assessment that measures the impact of dysphagia and predicts the presence or absence of aspiration is needed. The Clinical Assessment of Dysphagia in Neurodegeneration (CADN) was designed by a multidisciplinary team (neurology, neuropsychology, speech pathology) validated against strict methodological criteria in two neurodegenerative diseases, Parkinson's disease (PD) and degenerative ataxia (DA). CADN comprises two parts, an anamnesis (part one) and consumption (part two). Two-thirds of patients were assessed using reference tests, the SWAL-QOL symptoms subscale (part one) and videofluoroscopic assessment of swallowing (part two). CADN has 11 items and can be administered and scored in an average of 7 min. Test-retest reliability was established using correlation and Bland-Altman plots. 125 patients with a neurodegenerative disease were recruited; 60 PD and 65 DA. Validity was established using ROC graphs and correlations. CADN has sensitivity of 79 and 84% and specificity 71 and 69% for parts one and two, respectively. Significant correlations with disease severity were also observed (p dysphagia symptomatology and risk of aspiration. The CADN is a reliable, valid, brief, quantifiable, and easily deployed assessment of swallowing in neurodegenerative disease. It is thus ideally suited for both clinical bedside assessment and future multicentre clinical trials in neurodegenerative disease.

Development and validation of a parenting assessment tool for Chinese parents.

Science.gov (United States)

Cheng, L; Dai, Y; Zhu, Z; Xie, X; Chen, L

2012-07-01

The objective of this study was to develop a parenting assessment tool (PAT) for parents of children aged 3 to 6 years to assess parenting attitudes and skills within the Chinese socio-cultural context. Focus group discussions were used to gather information for composing the PAT test items. Factor analysis was performed to build and refine the factor structure of PAT. Ten cities included in a larger programme, the Early Childhood Development Promotion Project carried out by the Capital Institute of Pediatrics in China, were chosen as the study area. Six hundred and twelve parents and their single children were chosen from those 10 cities by convenience sampling for the large-scale investigation of PAT, and the results were used to test and validate the psychometric properties of PAT. The internal consistency of total PAT scores was 0.87, and those of the eight subscales ranged from 0.53 to 0.79. Test-retest reliability over 2 weeks was also significant for the total scores and all subscales (range: 0.68-0.89). For the factor structure, an eight-factor solution accounting for 65.54% of the variance was most consistently fit. Concurrent validity was supported by significant positive correlations between total PAT scores and children's language ability, sociability and adaptive ability as assessed by the Developmental Diagnostic Scale of Children Aged 0-6 Years. The newly devised PAT test is a reliable instrument to assess the parenting skills and attitudes of parents of young children in urban China. © 2011 Blackwell Publishing Ltd.

Measuring couple relationship quality in a rural African population: Validation of a Couple Functionality Assessment Tool in Malawi.

Directory of Open Access Journals (Sweden)

Allison Ruark

Full Text Available Available data suggest that individual and family well-being are linked to the quality of women's and men's couple relationships, but few tools exist to assess couple relationship functioning in low- and middle-income countries. In response to this gap, Catholic Relief Services has developed a Couple Functionality Assessment Tool (CFAT to capture valid and reliable data on various domains of relationship quality. This tool is designed to be used by interventions which aim to improve couple and family well-being as a means of measuring the effectiveness of these interventions, particularly related to couple relationship quality. We carried out a validation study of the CFAT among 401 married and cohabiting adults (203 women and 198 men in rural Chikhwawa District, Malawi. Using psychometric scales, the CFAT addressed six domains of couple relationship quality (intimacy, partner support, sexual satisfaction, gender roles, decision-making, and communication and conflict management, and included questions on intimate partner violence. We used exploratory factor analysis to assess scale performance of each domain and produce a shortened Relationship Quality Index (RQI composed of items from five relationship quality domains. This article reports the performance of the RQI. Internal reliability and validity of the RQI were found to be good. Regression analyses examined the relationship of the RQI to outcomes important to health and development: intra-household cooperation, positive health behaviors, intimate partner violence, and gender-equitable norms. We found many significant correlations between RQI scores and these couple- and family-level development issues. There is a need to further validate the tool with use in other populations as well as to continue to explore whether the observed linkages between couple functionality and development outcomes are causal relationships.

Reliable and valid assessment of Lichtenstein hernia repair skills.

Science.gov (United States)

Carlsen, C G; Lindorff-Larsen, K; Funch-Jensen, P; Lund, L; Charles, P; Konge, L

2014-08-01

Lichtenstein hernia repair is a common surgical procedure and one of the first procedures performed by a surgical trainee. However, formal assessment tools developed for this procedure are few and sparsely validated. The aim of this study was to determine the reliability and validity of an assessment tool designed to measure surgical skills in Lichtenstein hernia repair. Key issues were identified through a focus group interview. On this basis, an assessment tool with eight items was designed. Ten surgeons and surgical trainees were video recorded while performing Lichtenstein hernia repair, (four experts, three intermediates, and three novices). The videos were blindly and individually assessed by three raters (surgical consultants) using the assessment tool. Based on these assessments, validity and reliability were explored. The internal consistency of the items was high (Cronbach's alpha = 0.97). The inter-rater reliability was very good with an intra-class correlation coefficient (ICC) = 0.93. Generalizability analysis showed a coefficient above 0.8 even with one rater. The coefficient improved to 0.92 if three raters were used. One-way analysis of variance found a significant difference between the three groups which indicates construct validity, p fashion with the new procedure-specific assessment tool. We recommend this tool for future assessment of trainees performing Lichtenstein hernia repair to ensure that the objectives of competency-based surgical training are met.

Detecting acute distress and risk of future psychological morbidity in critically ill patients: validation of the intensive care psychological assessment tool.

Science.gov (United States)

Wade, Dorothy M; Hankins, Matthew; Smyth, Deborah A; Rhone, Elijah E; Mythen, Michael G; Howell, David C J; Weinman, John A

2014-09-24

The psychological impact of critical illness on a patient can be severe, and frequently results in acute distress as well as psychological morbidity after leaving hospital. A UK guideline states that patients should be assessed in critical care units, both for acute distress and risk of future psychological morbidity; but no suitable method for carrying out this assessment exists. The Intensive care psychological assessment tool (IPAT) was developed as a simple, quick screening tool to be used routinely to detect acute distress, and the risk of future psychological morbidity, in critical care units. A validation study of IPAT was conducted in the critical care unit of a London hospital. Once un-sedated, orientated and alert, critical care patients were assessed with the IPAT and validated tools for distress, to determine the IPAT's concurrent validity. Fifty six patients took IPAT again to establish test-retest reliability. Finally, patients completed posttraumatic stress disorder (PTSD), depression and anxiety questionnaires at three months, to determine predictive validity of the IPAT. One hundred and sixty six patients completed the IPAT, and 106 completed follow-up questionnaires at 3 months. Scale analysis showed IPAT was a reliable 10-item measure of critical care-related psychological distress. Test-retest reliability was good (r =0.8). There was good concurrent validity with measures of anxiety and depression (r =0.7, P psychological morbidity was good (r =0.4, P psychological morbidity (AUC =0.7). The IPAT was found to have good reliability and validity. Sensitivity and specificity analysis suggest the IPAT could provide a way of allowing staff to assess psychological distress among critical care patients after further replication and validation. Further work is also needed to determine its utility in predicting future psychological morbidity.

Development and validation of surgical training tool: cystectomy assessment and surgical evaluation (CASE) for robot-assisted radical cystectomy for men.

Science.gov (United States)

Hussein, Ahmed A; Sexton, Kevin J; May, Paul R; Meng, Maxwell V; Hosseini, Abolfazl; Eun, Daniel D; Daneshmand, Siamak; Bochner, Bernard H; Peabody, James O; Abaza, Ronney; Skinner, Eila C; Hautmann, Richard E; Guru, Khurshid A

2018-04-13

We aimed to develop a structured scoring tool: cystectomy assessment and surgical evaluation (CASE) that objectively measures and quantifies performance during robot-assisted radical cystectomy (RARC) for men. A multinational 10-surgeon expert panel collaborated towards development and validation of CASE. The critical steps of RARC in men were deconstructed into nine key domains, each assessed by five anchors. Content validation was done utilizing the Delphi methodology. Each anchor was assessed in terms of context, score concordance, and clarity. The content validity index (CVI) was calculated for each aspect. A CVI ≥ 0.75 represented consensus, and this statement was removed from the next round. This process was repeated until consensus was achieved for all statements. CASE was used to assess de-identified videos of RARC to determine reliability and construct validity. Linearly weighted percent agreement was used to assess inter-rater reliability (IRR). A logit model for odds ratio (OR) was used to assess construct validation. The expert panel reached consensus on CASE after four rounds. The final eight domains of the CASE included: pelvic lymph node dissection, development of the peri-ureteral space, lateral pelvic space, anterior rectal space, control of the vascular pedicle, anterior vesical space, control of the dorsal venous complex, and apical dissection. IRR > 0.6 was achieved for all eight domains. Experts outperformed trainees across all domains. We developed and validated a reliable structured, procedure-specific tool for objective evaluation of surgical performance during RARC. CASE may help differentiate novice from expert performances.

Validating the CORE-10 as a mental health screening tool for prisoners

OpenAIRE

Lewis, Gwen

2016-01-01

Background: Few mental health screening tools have been validated with prisoners and existing tools, do not assess severity of need in line with contemporary stepped care service models. \\ud \\ud Aims: The current research aims to assess the CORE-10’s psychometric reliability, validity and predictive accuracy as a screening tool for common (primary care) and severe (secondary care) mental health problems in prisoners. \\ud \\ud Method: Cross –sectional study of 150 prisoners. All participants co...

A web-based team-oriented medical error communication assessment tool: development, preliminary reliability, validity, and user ratings.

Science.gov (United States)

Kim, Sara; Brock, Doug; Prouty, Carolyn D; Odegard, Peggy Soule; Shannon, Sarah E; Robins, Lynne; Boggs, Jim G; Clark, Fiona J; Gallagher, Thomas

2011-01-01

Multiple-choice exams are not well suited for assessing communication skills. Standardized patient assessments are costly and patient and peer assessments are often biased. Web-based assessment using video content offers the possibility of reliable, valid, and cost-efficient means for measuring complex communication skills, including interprofessional communication. We report development of the Web-based Team-Oriented Medical Error Communication Assessment Tool, which uses videotaped cases for assessing skills in error disclosure and team communication. Steps in development included (a) defining communication behaviors, (b) creating scenarios, (c) developing scripts, (d) filming video with professional actors, and (e) writing assessment questions targeting team communication during planning and error disclosure. Using valid data from 78 participants in the intervention group, coefficient alpha estimates of internal consistency were calculated based on the Likert-scale questions and ranged from α=.79 to α=.89 for each set of 7 Likert-type discussion/planning items and from α=.70 to α=.86 for each set of 8 Likert-type disclosure items. The preliminary test-retest Pearson correlation based on the scores of the intervention group was r=.59 for discussion/planning and r=.25 for error disclosure sections, respectively. Content validity was established through reliance on empirically driven published principles of effective disclosure as well as integration of expert views across all aspects of the development process. In addition, data from 122 medicine and surgical physicians and nurses showed high ratings for video quality (4.3 of 5.0), acting (4.3), and case content (4.5). Web assessment of communication skills appears promising. Physicians and nurses across specialties respond favorably to the tool.

Validation and Evaluation of Two Observational Pain Assessment Tools in a Trauma and Neurosurgical Intensive Care Unit

Directory of Open Access Journals (Sweden)

Jane Topolovec-Vranic

2013-01-01

Full Text Available BACKGROUND: Studies have demonstrated that patients in the intensive care unit experience high levels of pain. While many of these patients are nonverbal at some point during their stay, there are few valid tools available to assess pain in this group.

The Construct Validity and Reliability of an Assessment Tool for Competency in Cochlear Implant Surgery

Directory of Open Access Journals (Sweden)

Patorn Piromchai

2014-01-01

Full Text Available Introduction. We introduce a rating tool that objectively evaluates the skills of surgical trainees performing cochlear implant surgery. Methods. Seven residents and seven experts performed cochlear implant surgery sessions from mastoidectomy to cochleostomy on a standardized virtual reality temporal bone. A total of twenty-eight assessment videos were recorded and two consultant otolaryngologists evaluated the performance of each participant using these videos. Results. Interrater reliability was calculated using the intraclass correlation coefficient for both the global and checklist components of the assessment instrument. The overall agreement was high. The construct validity of this instrument was strongly supported by the significantly higher scores in the expert group for both components. Conclusion. Our results indicate that the proposed assessment tool for cochlear implant surgery is reliable, accurate, and easy to use. This instrument can thus be used to provide objective feedback on overall and task-specific competency in cochlear implantation.

Validity evidence for the Fundamentals of Laparoscopic Surgery (FLS) program as an assessment tool: a systematic review.

Science.gov (United States)

Zendejas, Benjamin; Ruparel, Raaj K; Cook, David A

2016-02-01

The Fundamentals of Laparoscopic Surgery (FLS) program uses five simulation stations (peg transfer, precision cutting, loop ligation, and suturing with extracorporeal and intracorporeal knot tying) to teach and assess laparoscopic surgery skills. We sought to summarize evidence regarding the validity of scores from the FLS assessment. We systematically searched for studies evaluating the FLS as an assessment tool (last search update February 26, 2013). We classified validity evidence using the currently standard validity framework (content, response process, internal structure, relations with other variables, and consequences). From a pool of 11,628 studies, we identified 23 studies reporting validity evidence for FLS scores. Studies involved residents (n = 19), practicing physicians (n = 17), and medical students (n = 8), in specialties of general (n = 17), gynecologic (n = 4), urologic (n = 1), and veterinary (n = 1) surgery. Evidence was most common in the form of relations with other variables (n = 22, most often expert-novice differences). Only three studies reported internal structure evidence (inter-rater or inter-station reliability), two studies reported content evidence (i.e., derivation of assessment elements), and three studies reported consequences evidence (definition of pass/fail thresholds). Evidence nearly always supported the validity of FLS total scores. However, the loop ligation task lacks discriminatory ability. Validity evidence confirms expected relations with other variables and acceptable inter-rater reliability, but other validity evidence is sparse. Given the high-stakes use of this assessment (required for board eligibility), we suggest that more validity evidence is required, especially to support its content (selection of tasks and scoring rubric) and the consequences (favorable and unfavorable impact) of assessment.

Assessing Reliability and Validity of the "GroPromo" Audit Tool for Evaluation of Grocery Store Marketing and Promotional Environments

Science.gov (United States)

Kerr, Jacqueline; Sallis, James F.; Bromby, Erica; Glanz, Karen

2012-01-01

Objective: To evaluate reliability and validity of a new tool for assessing the placement and promotional environment in grocery stores. Methods: Trained observers used the "GroPromo" instrument in 40 stores to code the placement of 7 products in 9 locations within a store, along with other promotional characteristics. To test construct validity,…

Construct and criterion validity testing of the Non-Technical Skills for Surgeons (NOTSS) behaviour assessment tool using videos of simulated operations.

Science.gov (United States)

Yule, S; Gupta, A; Gazarian, D; Geraghty, A; Smink, D S; Beard, J; Sundt, T; Youngson, G; McIlhenny, C; Paterson-Brown, S

2018-05-01

Surgeons' non-technical skills are an important part of surgical performance and surgical education. The most widely adopted assessment tool is the Non-Technical Skills for Surgeons (NOTSS) behaviour rating system. Psychometric analysis of this tool to date has focused on inter-rater reliability and feasibility rather than validation. NOTSS assessments were collected from two groups of consultant/attending surgeons in the UK and USA, who rated behaviours of the lead surgeon during a video-based simulated crisis scenario after either online or classroom instruction. The process of validation consisted of assessing construct validity, scale reliability and concurrent criterion validity, and undertaking a sensitivity analysis. Central to this was confirmatory factor analysis to evaluate the structure of the NOTSS taxonomy. Some 255 consultant surgeons participated in the study. The four-category NOTSS model was found to have robust construct validity evidence, and a superior fit compared with alternative models. Logistic regression and sensitivity analysis revealed that, after adjusting for technical skills, for every 1-point increase in NOTSS score of the lead surgeon, the odds of having a higher versus lower patient safety score was 2·29 times. The same pattern of results was obtained for a broad mix of surgical specialties (UK) as well as a single discipline (cardiothoracic, USA). The NOTSS tool can be applied in research and education settings to measure non-technical skills in a valid and efficient manner. © 2018 BJS Society Ltd Published by John Wiley & Sons Ltd.

Developing and Validating a Tool to Assess Ethical Decision-Making Ability of Nursing Students, Using Rubrics

Science.gov (United States)

Indhraratana, Apinya; Kaemkate, Wannee

2012-01-01

The aim of this paper is to develop a reliable and valid tool to assess ethical decision-making ability of nursing students using rubrics. A proposed ethical decision making process, from reviewing related literature was used as a framework for developing the rubrics. Participants included purposive sample of 86 nursing students from the Royal…

Issues in developing valid assessments of speech pathology students' performance in the workplace.

Science.gov (United States)

McAllister, Sue; Lincoln, Michelle; Ferguson, Alison; McAllister, Lindy

2010-01-01

Workplace-based learning is a critical component of professional preparation in speech pathology. A validated assessment of this learning is seen to be 'the gold standard', but it is difficult to develop because of design and validation issues. These issues include the role and nature of judgement in assessment, challenges in measuring quality, and the relationship between assessment and learning. Valid assessment of workplace-based performance needs to capture the development of competence over time and account for both occupation specific and generic competencies. This paper reviews important conceptual issues in the design of valid and reliable workplace-based assessments of competence including assessment content, process, impact on learning, measurement issues, and validation strategies. It then goes on to share what has been learned about quality assessment and validation of a workplace-based performance assessment using competency-based ratings. The outcomes of a four-year national development and validation of an assessment tool are described. A literature review of issues in conceptualizing, designing, and validating workplace-based assessments was conducted. Key factors to consider in the design of a new tool were identified and built into the cycle of design, trialling, and data analysis in the validation stages of the development process. This paper provides an accessible overview of factors to consider in the design and validation of workplace-based assessment tools. It presents strategies used in the development and national validation of a tool COMPASS, used in an every speech pathology programme in Australia, New Zealand, and Singapore. The paper also describes Rasch analysis, a model-based statistical approach which is useful for establishing validity and reliability of assessment tools. Through careful attention to conceptual and design issues in the development and trialling of workplace-based assessments, it has been possible to develop the

Montreal Cognitive Assessment and Mini-Mental State Examination are both valid cognitive tools in stroke.

Science.gov (United States)

Cumming, T B; Churilov, L; Linden, T; Bernhardt, J

2013-08-01

To determine the validity of the Montreal Cognitive Assessment (MoCA) and the Mini-Mental State Examination (MMSE) as screening tools for cognitive impairment after stroke. Cognitive assessments were administered over 2 sessions (1 week apart) at 3 months post-stroke. Scores on the MoCA and MMSE were evaluated against a diagnosis of cognitive impairment derived from a comprehensive neuropsychological battery (the criterion standard). Sixty patients participated in the study [mean age 72.1 years (SD = 13.9), mean education 10.5 years (SD = 3.9), median acute NIHSS score 5 (IQR 3-7)]. The MoCA yielded lower scores (median = 21, IQR = 17-24; mean = 20.0, SD = 5.4) than the MMSE (median = 26, IQR = 22-27; mean = 24.2, SD = 4.5). MMSE data were more skewed towards ceiling than MoCA data (skewness = -1.09 vs -0.73). Area under the receiver operator curve was higher for MoCA than for MMSE (0.87 vs 0.84), although this difference was not significant (χ(2) = 0.48, P = 0.49). At their optimal cut-offs, the MoCA had better sensitivity than the MMSE (0.92 vs 0.82) but poorer specificity (0.67 vs 0.76). The MoCA is a valid screening tool for post-stroke cognitive impairment; it is more sensitive but less specific than the MMSE. Contrary to the prevailing view, the MMSE also exhibited acceptable validity in this setting. © 2013 John Wiley & Sons A/S.

Reliable and valid assessment of performance in thoracoscopy

DEFF Research Database (Denmark)

Konge, Lars; Lehnert, Per; Hansen, Henrik Jessen

2012-01-01

BACKGROUND: As we move toward competency-based education in medicine, we have lagged in developing competency-based evaluation methods. In the era of minimally invasive surgery, there is a need for a reliable and valid tool dedicated to measure competence in video-assisted thoracoscopic surgery....... The purpose of this study is to create such an assessment tool, and to explore its reliability and validity. METHODS: An expert group of physicians created an assessment tool consisting of 10 items rated on a five-point rating scale. The following factors were included: economy and confidence of movement...

Technical skills assessment toolbox: a review using the unitary framework of validity.

Science.gov (United States)

Ghaderi, Iman; Manji, Farouq; Park, Yoon Soo; Juul, Dorthea; Ott, Michael; Harris, Ilene; Farrell, Timothy M

2015-02-01

The purpose of this study was to create a technical skills assessment toolbox for 35 basic and advanced skills/procedures that comprise the American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) surgical skills curriculum and to provide a critical appraisal of the included tools, using contemporary framework of validity. Competency-based training has become the predominant model in surgical education and assessment of performance is an essential component. Assessment methods must produce valid results to accurately determine the level of competency. A search was performed, using PubMed and Google Scholar, to identify tools that have been developed for assessment of the targeted technical skills. A total of 23 assessment tools for the 35 ACS/APDS skills modules were identified. Some tools, such as Operative Performance Rating System (OSATS) and Objective Structured Assessment of Technical Skill (OPRS), have been tested for more than 1 procedure. Therefore, 30 modules had at least 1 assessment tool, with some common surgical procedures being addressed by several tools. Five modules had none. Only 3 studies used Messick's framework to design their validity studies. The remaining studies used an outdated framework on the basis of "types of validity." When analyzed using the contemporary framework, few of these studies demonstrated validity for content, internal structure, and relationship to other variables. This study provides an assessment toolbox for common surgical skills/procedures. Our review shows that few authors have used the contemporary unitary concept of validity for development of their assessment tools. As we progress toward competency-based training, future studies should provide evidence for various sources of validity using the contemporary framework.

Spanish validation of the Person-centered Care Assessment Tool (P-CAT).

Science.gov (United States)

Martínez, Teresa; Suárez-Álvarez, Javier; Yanguas, Javier; Muñiz, José

2016-01-01

Person-centered Care (PCC) is an innovative approach which seeks to improve the quality of care services given to the care-dependent elderly. At present there are no Spanish language instruments for the evaluation of PCC delivered by elderly care services. The aim of this work is the adaptation and validation of the Person-centered Care Assessment Tool (P-CAT) for a Spanish population. The P-CAT was translated and adapted into Spanish, then given to a sample of 1339 front-line care professionals from 56 residential elderly care homes. The reliability and validity of the P-CAT were analyzed, within the frameworks of Classical Test Theory and Item Response Theory models. The Spanish P-CAT demonstrated good reliability, with an alpha coefficient of .88 and a test-retest reliability coefficient of .79. The P-CAT information function indicates that the test measures with good precision for the majority of levels of the measured variables (θ values between -2 and +1). The factorial structure of the test is essentially one-dimensional and the item discrimination indices are high, with values between .26 and .61. In terms of predictive validity, the correlations which stand out are between the P-CAT and organizational climate (r = .689), and the burnout factors; personal accomplishment (r = .382), and emotional exhaustion (r = - .510). The Spanish version of the P-CAT demonstrates good psychometric properties for its use in the evaluation of elderly care homes both professionally and in research.

Assessing Households Preparedness for Earthquakes: An Exploratory Study in the Development of a Valid and Reliable Persian-version Tool.

Science.gov (United States)

Ardalan, Ali; Sohrabizadeh, Sanaz

2016-02-25

Iran is placed among countries suffering from the highest number of earthquake casualties. Household preparedness, as one component of risk reduction efforts, is often supported in quake-prone areas. In Iran, lack of a valid and reliable household preparedness tool was reported by previous disaster studies. This study is aimed to fill this gap by developing a valid and reliable tool for assessing household preparedness in the event of an earthquake.  This survey was conducted through three phases including literature review and focus group discussions with the participation of eight key informants, validity measurements and reliability measurements. Field investigation was completed with the participation of 450 households within three provinces of Iran. Content validity, construct validity, the use of factor analysis; internal consistency using Cronbach's alpha coefficient, and test-retest reliability were carried out to develop the tool.  Based on the CVIs, ranging from 0.80 to 0.100, and exploratory factor analysis with factor loading of more than 0.5, all items were valid. The amount of Cronbach's alpha (0.7) and test-retest examination by Spearman correlations indicated that the scale was also reliable. The final instrument consisted of six categories and 18 questions including actions at the time of earthquakes, nonstructural safety, structural safety, hazard map, communications, drill, and safety skills.  Using a Persian-version tool that is adjusted to the socio-cultural determinants and native language may result in more trustful information on earthquake preparedness. It is suggested that disaster managers and researchers apply this tool in their future household preparedness projects. Further research is needed to make effective policies and plans for transforming preparedness knowledge into behavior.

Development and validation of a fine-motor assessment tool for use with young children in a Chinese population.

Science.gov (United States)

Siu, Andrew M H; Lai, Cynthia Y Y; Chiu, Amy S M; Yip, Calvin C K

2011-01-01

Most of the fine-motor assessment tools used in Hong Kong have been designed in Western countries, so there is a need to develop a standardized assessment which is relevant to the culture and daily living tasks of the local (that is, Chinese) population. This study aimed to (1) develop a fine-motor assessment tool (the Hong Kong Preschool Fine-Motor Developmental Assessment [HK-PFMDA]) for use with young children in a Chinese population and (2) examine the HK-PFMDA's psychometric properties. The HK-PFMDA was developed by a group of occupational therapists specializing in the area of developmental disabilities in Hong Kong. A panel of 21 experts reviewed the content validity of the instrument. Rasch item analysis was used to examine the model fit of items against the rating scale model, and to explore the dimensionality of the test. Intra- and interrater reliability, convergent validity, and criterion-related validity were examined. The participants included 783 children without disabilities, 45 with autistic spectrum disorder, and 35 with developmental delay. The Rasch analysis suggested that the 87-item HK-PFMDA had a unidimensional structure, as the items explained most (91.6%) of the variance. The HK-PFMDA demonstrated excellent intra- (ICC = .99) and interrater reliability (ICC = .99), and internal consistency (α ranging from .83 to .92). In terms of validity, the HK-PFMDA had significant positive correlations with both age and the convergent measures of the Peabody Developmental Motor Scales (PDMS-2). A set of normative data for local children aged from birth to 6 years was established. The HK-PFMDA has shown excellent psychometric properties and is suitable for clinical application by occupational therapists in the assessment of fine-motor skills development of young children in Chinese populations. Copyright © 2010 Elsevier Ltd. All rights reserved.

Teamwork Assessment Tools in Obstetric Emergencies: A Systematic Review.

Science.gov (United States)

Onwochei, Desire N; Halpern, Stephen; Balki, Mrinalini

2017-06-01

Team-based training and simulation can improve patient safety, by improving communication, decision making, and performance of team members. Currently, there is no general consensus on whether or not a specific assessment tool is better adapted to evaluate teamwork in obstetric emergencies. The purpose of this qualitative systematic review was to find the tools available to assess team effectiveness in obstetric emergencies. We searched Embase, Medline, PubMed, Web of Science, PsycINFO, CINAHL, and Google Scholar for prospective studies that evaluated nontechnical skills in multidisciplinary teams involving obstetric emergencies. The search included studies from 1944 until January 11, 2016. Data on reliability and validity measures were collected and used for interpretation. A descriptive analysis was performed on the data. Thirteen studies were included in the final qualitative synthesis. All the studies assessed teams in the context of obstetric simulation scenarios, but only six included anesthetists in the simulations. One study evaluated their teamwork tool using just validity measures, five using just reliability measures, and one used both. The most reliable tools identified were the Clinical Teamwork Scale, the Global Assessment of Obstetric Team Performance, and the Global Rating Scale of performance. However, they were still lacking in terms of quality and validity. More work needs to be conducted to establish the validity of teamwork tools for nontechnical skills, and the development of an ideal tool is warranted. Further studies are required to assess how outcomes, such as performance and patient safety, are influenced when using these tools.

International cross-cultural validation study of the Canadian haemophilia outcomes: kids' life assessment tool.

Science.gov (United States)

McCusker, P J; Fischer, K; Holzhauer, S; Meunier, S; Altisent, C; Grainger, J D; Blanchette, V S; Burke, T A; Wakefield, C; Young, N L

2015-05-01

Health-related quality of life (HRQoL) assessment is recognized as an important outcome in the evaluation of different therapeutic regimens for persons with haemophilia. The Canadian Haemophilia Outcomes-Kids' Life Assessment Tool (CHO-KLAT) is a disease-specific measure of HRQoL for 4 to 18-year-old boys with haemophilia. The purpose of this study was to extend this disease-specific, child-centric, outcome measure for use in international clinical trials. We adapted the North American English CHO-KLAT version for use in five countries: France, Germany, the Netherlands, Spain and the United Kingdom (UK). The process included four stages: (i) translation; (ii) cognitive debriefing; (iii) validity assessment relative to the PedsQL (generic) and the Haemo-QoL (disease-specific) and (iv) assessment of inter and intra-rater reliability. Cognitive debriefing was performed in 57 boys (mean age 11.4 years), validation was performed in 144 boys (mean age 11.0 years) and reliability was assessed for a subgroup of 64 boys (mean age 12.0 years). Parents also participated. The mean scores reported by the boys were high: CHO-KLAT 77.0 (SD = 11.2); PedsQL 83.8 (SD = 11.9) and Haemo-QoL 79.6 (SD = 11.5). Correlations between the CHO-KLAT and PedsQL ranged from 0.63 in Germany to 0.39 in the Netherlands and Spain. Test-retest reliability (concordance) for child self-report was 0.67. Child-parent concordance was slightly lower at 0.57. The CHO-KLAT has been fully culturally adapted and validated for use in five different languages and cultures (in England, the Netherlands, France, Germany and Spain) where treatment is readily available either on demand or as prophylaxis. © 2014 John Wiley & Sons Ltd.

Validation of the educational needs assessment tool as a generic instrument for rheumatic diseases in seven European countries.

Science.gov (United States)

Ndosi, Mwidimi; Bremander, Ann; Hamnes, Bente; Horton, Mike; Kukkurainen, Marja Leena; Machado, Pedro; Marques, Andrea; Meesters, Jorit; Stamm, Tanja A; Tennant, Alan; de la Torre-Aboki, Jenny; Vliet Vlieland, Theodora P M; Zangi, Heidi A; Hill, Jackie

2014-12-01

To validate the educational needs assessment tool (ENAT) as a generic tool for assessing the educational needs of patients with rheumatic diseases in European Countries. A convenience sample of patients from seven European countries was included comprising the following diagnostic groups: ankylosing spondylitis, psoriatic arthritis, systemic sclerosis, systemic lupus erythematosus, osteoarthritis (OA) and fibromyalgia syndrome. Translated versions of the ENAT were completed through surveys in each country. Rasch analysis was used to assess the construct validity of the adapted ENATs including differential item functioning by culture (cross-cultural DIF). Initially, the data from each country and diagnostic group were fitted to the Rasch model separately, and then the pooled data from each diagnostic group. The sample comprised 3015 patients; the majority, 1996 (66.2%), were women. Patient characteristics (stratified by diagnostic group) were comparable across countries except the educational background, which was variable. In most occasions, the 39-item ENAT deviated significantly from the Rasch model expectations (item-trait interaction χ(2) pseven domains and analysing them as 'testlets'), fit to the model was satisfied (item-trait interaction χ(2) p>0.18) in all pooled disease group datasets except OA (χ(2)=99.91; p=0.002). The internal consistency in each group was high (Person Separation Index above 0.90). There was no significant DIF by person characteristics. Cross-cultural DIF was found in some items, which required adjustments. Subsequently, interval-level scales were calibrated to enable transformation of ENAT scores when required. The adapted ENAT is a valid tool with high internal consistency providing accurate estimation of the educational needs of people with rheumatic diseases. Cross-cultural comparison of educational needs is now possible. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a

Assessing readiness to work in primary health care: the content validity of a self-check tool for physiotherapists and other health professionals.

Science.gov (United States)

Stewart, Jenny; Haswell, Kate

2013-03-01

The New Zealand Primary Health Care Strategy has emphasised the importance of well-coordinated service teams in managing complex chronic conditions. There is international evidence that physiotherapists can contribute effectively to the prevention and management of these conditions. However, there are few examples of physiotherapists in New Zealand (NZ) engaging in primary health care (PHC). It has been recognised that professional development is necessary to optimise physiotherapists' participation in PHC. The aim of this study was to both design a self-check tool that physiotherapists could use as an initial step in preparing to work in PHC and to assess the content validity of the tool. A literature review informed the development of the self-check tool. The tool was reviewed by members of the Physiotherapy New Zealand PHC working party to establish content validity. The tool was found to have excellent content validity with an overall score of 0.937, exceeding the acceptable index of 0.8. Item validity was excellent or acceptable for all except two items, which were subsequently modified in the final tool. This investigation provides initial support for the tool's potential use by physiotherapists as a means of determining their readiness to work in PHC. It could have application beyond individual professional development to the wider context of team and organisational development. Additionally, with minor modifications the tool could have broader application to other professional groups.

Comparison of seven fall risk assessment tools in community-dwelling Korean older women.

Science.gov (United States)

Kim, Taekyoung; Xiong, Shuping

2017-03-01

This study aimed to compare seven widely used fall risk assessment tools in terms of validity and practicality, and to provide a guideline for choosing appropriate fall risk assessment tools for elderly Koreans. Sixty community-dwelling Korean older women (30 fallers and 30 matched non-fallers) were evaluated. Performance measures of all tools were compared between the faller and non-faller groups through two sample t-tests. Receiver Operating Characteristic curves were generated with odds ratios for discriminant analysis. Results showed that four tools had significant discriminative power, and the shortened version of Falls Efficacy Scale (SFES) showed excellent discriminant validity, followed by Berg Balance Scale (BBS) with acceptable discriminant validity. The Mini Balance Evaluation System Test and Timed Up and Go, however, had limited discriminant validities. In terms of practicality, SFES was also excellent. These findings suggest that SFES is the most suitable tool for assessing the fall risks of community-dwelling Korean older women, followed by BBS. Practitioner Summary: There is no general guideline on which fall risk assessment tools are suitable for community-dwelling Korean older women. This study compared seven widely used assessment tools in terms of validity and practicality. Results suggested that the short Falls Efficacy Scale is the most suitable tool, followed by Berg Balance Scale.

Validation of the TRUST tool in a Greek perioperative setting.

Science.gov (United States)

Chatzea, Vasiliki-Eirini; Sifaki-Pistolla, Dimitra; Dey, Nilanjan; Melidoniotis, Evangelos

2017-06-01

The aim of this study was to translate, culturally adapt and validate the TRUST questionnaire in a Greek perioperative setting. The TRUST questionnaire assesses the relationship between trust and performance. The study assessed the levels of trust and performance in the surgery and anaesthesiology department during a very stressful period for Greece (economic crisis) and offered a user friendly and robust assessment tool. The study concludes that the Greek version of the TRUST questionnaire is a reliable and valid instrument for measuring team performance among Greek perioperative teams. Copyright the Association for Perioperative Practice.

Evaluation of the validity of osteoporosis and fracture risk assessment tools (IOF One Minute Test, SCORE, and FRAX) in postmenopausal Palestinian women.

Science.gov (United States)

Kharroubi, Akram; Saba, Elias; Ghannam, Ibrahim; Darwish, Hisham

2017-12-01

The need for simple self-assessment tools is necessary to predict women at high risk for developing osteoporosis. In this study, tools like the IOF One Minute Test, Fracture Risk Assessment Tool (FRAX), and Simple Calculated Osteoporosis Risk Estimation (SCORE) were found to be valid for Palestinian women. The threshold for predicting women at risk for each tool was estimated. The purpose of this study is to evaluate the validity of the updated IOF (International Osteoporosis Foundation) One Minute Osteoporosis Risk Assessment Test, FRAX, SCORE as well as age alone to detect the risk of developing osteoporosis in postmenopausal Palestinian women. Three hundred eighty-two women 45 years and older were recruited including 131 women with osteoporosis and 251 controls following bone mineral density (BMD) measurement, 287 completed questionnaires of the different risk assessment tools. Receiver operating characteristic (ROC) curves were evaluated for each tool using bone BMD as the gold standard for osteoporosis. The area under the ROC curve (AUC) was the highest for FRAX calculated with BMD for predicting hip fractures (0.897) followed by FRAX for major fractures (0.826) with cut-off values ˃1.5 and ˃7.8%, respectively. The IOF One Minute Test AUC (0.629) was the lowest compared to other tested tools but with sufficient accuracy for predicting the risk of developing osteoporosis with a cut-off value ˃4 total yes questions out of 18. SCORE test and age alone were also as good predictors of risk for developing osteoporosis. According to the ROC curve for age, women ≥64 years had a higher risk of developing osteoporosis. Higher percentage of women with low BMD (T-score ≤-1.5) or osteoporosis (T-score ≤-2.5) was found among women who were not exposed to the sun, who had menopause before the age of 45 years, or had lower body mass index (BMI) compared to controls. Women who often fall had lower BMI and approximately 27% of the recruited postmenopausal

Iatrogenic Opioid Withdrawal in Critically Ill Patients: A Review of Assessment Tools and Management.

Science.gov (United States)

Chiu, Ada W; Contreras, Sofia; Mehta, Sangeeta; Korman, Jennifer; Perreault, Marc M; Williamson, David R; Burry, Lisa D

2017-12-01

To (1) provide an overview of the epidemiology, clinical presentation, and risk factors of iatrogenic opioid withdrawal in critically ill patients and (2) conduct a literature review of assessment and management of iatrogenic opioid withdrawal in critically ill patients. We searched MEDLINE (1946-June 2017), EMBASE (1974-June 2017), and CINAHL (1982-June 2017) with the terms opioid withdrawal, opioid, opiate, critical care, critically ill, assessment tool, scale, taper, weaning, and management. Reference list of identified literature was searched for additional references as well as www.clinicaltrials.gov . We restricted articles to those in English and dealing with humans. We identified 2 validated pediatric critically ill opioid withdrawal assessment tools: (1) Withdrawal Assessment Tool-Version 1 (WAT-1) and (2) Sophia Observation Withdrawal Symptoms Scale (SOS). Neither tool differentiated between opioid and benzodiazepine withdrawal. WAT-1 was evaluated in critically ill adults but not found to be valid. No other adult tool was identified. For management, we identified 5 randomized controlled trials, 2 prospective studies, and 2 systematic reviews. Most studies were small and only 2 studies utilized a validated assessment tool. Enteral methadone, α-2 agonists, and protocolized weaning were studied. We identified 2 validated assessment tools for pediatric intensive care unit patients; no valid tool for adults. Management strategies tested in small trials included methadone, α-2 agonists, and protocolized sedation/weaning. We challenge researchers to create validated tools assessing specifically for opioid withdrawal in critically ill children and adults to direct management.

Creation of complexity assessment tool for patients receiving home care

Directory of Open Access Journals (Sweden)

Maria Leopoldina de Castro Villas Bôas

2016-06-01

Full Text Available Abstract OBJECTIVE To create and validate a complexity assessment tool for patients receiving home care from a public health service. METHOD A diagnostic accuracy study, with estimates for the tool's validity and reliability. Measurements of sensitivity and specificity were considered when producing validity estimates. The resulting tool was used for testing. Assessment by a specialized team of home care professionals was used as the gold standard. In the tool's reliability study, the authors used the Kappa statistic. The tool's sensitivity and specificity were analyzed using various cut-off points. RESULTS On the best cut-off point-21-with the gold standard, a sensitivity of 75.5% was obtained, with the limits of confidence interval (95% at 68.3% and 82.8% and specificity of 53.2%, with the limits of confidence interval (95% at 43.8% and 62.7%. CONCLUSION The tool presented evidence of validity and reliability, possibly helping in service organization at patient admission, care type change, or support during the creation of care plans.

Validation of Alternative In Vitro Methods to Animal Testing: Concepts, Challenges, Processes and Tools.

Science.gov (United States)

Griesinger, Claudius; Desprez, Bertrand; Coecke, Sandra; Casey, Warren; Zuang, Valérie

This chapter explores the concepts, processes, tools and challenges relating to the validation of alternative methods for toxicity and safety testing. In general terms, validation is the process of assessing the appropriateness and usefulness of a tool for its intended purpose. Validation is routinely used in various contexts in science, technology, the manufacturing and services sectors. It serves to assess the fitness-for-purpose of devices, systems, software up to entire methodologies. In the area of toxicity testing, validation plays an indispensable role: "alternative approaches" are increasingly replacing animal models as predictive tools and it needs to be demonstrated that these novel methods are fit for purpose. Alternative approaches include in vitro test methods, non-testing approaches such as predictive computer models up to entire testing and assessment strategies composed of method suites, data sources and decision-aiding tools. Data generated with alternative approaches are ultimately used for decision-making on public health and the protection of the environment. It is therefore essential that the underlying methods and methodologies are thoroughly characterised, assessed and transparently documented through validation studies involving impartial actors. Importantly, validation serves as a filter to ensure that only test methods able to produce data that help to address legislative requirements (e.g. EU's REACH legislation) are accepted as official testing tools and, owing to the globalisation of markets, recognised on international level (e.g. through inclusion in OECD test guidelines). Since validation creates a credible and transparent evidence base on test methods, it provides a quality stamp, supporting companies developing and marketing alternative methods and creating considerable business opportunities. Validation of alternative methods is conducted through scientific studies assessing two key hypotheses, reliability and relevance of the

Development and Validation of a Self-Assessment Tool for Albuminuria: Results From the Reasons for Geographic and Racial Differences in Stroke (REGARDS) Study

Science.gov (United States)

Muntner, Paul; Woodward, Mark; Carson, April P; Judd, Suzanne E; Levitan, Emily B; Mann, Devin; McClellan, William; Warnock, David G

2011-01-01

Background The prevalence of albuminuria in the general population is high, but awareness of it is low. Therefore, we sought to develop and validate a self-assessment tool that allows individuals to estimate their probability of having albuminuria. Study Design Cross-sectional study Setting & Participants The population-based REasons for Geographic And Racial Differences in Stroke (REGARDS) study for model development and the National Health and Nutrition Examination Survey 1999-2004 (NHANES 1999-2004) for model validation. US adults ≥ 45 years of age in the REGARDS study (n=19,697) and NHANES 1999-2004 (n=7,168) [nijsje 1]Factor Candidate items for the self-assessment tool were collected using a combination of interviewer- and self-administered questionnaires. Outcome Albuminuria was defined as a urinary albumin to urinary creatinine ratio ≥ 30 mg/g in spot samples. Results Eight items were included in the self-assessment tool (age, race, gender, current smoking, self-rated health, and self-reported history of diabetes, hypertension, and stroke). These items provided a c-statistic of 0.709 (95% CI, 0.699 – 0.720) and a good model fit (Hosmer-Lemeshow chi-square p-value = 0.49). In the external validation data set, the c-statistic for discriminating individuals with and without albuminuria using the self-assessment tool was 0.714. Using a threshold of ≥ 10% probability of albuminuria from the self-assessment tool, 36% of US adults ≥ 45 years of age in NHANES 1999-2004 would test positive and be recommended screening. The sensitivity, specificity, and positive and negative predictive values for albuminuria associated with a probability ≥ 10% were 66%, 68%, 23% and 93%, respectively. Limitations Repeat urine samples were not available to assess the persistency of albuminuria. Conclusions Eight self-report items provide good discrimination for the probability of having albuminuria. This tool may encourage individuals with a high probability to request

Development of Reliable and Validated Tools to Evaluate Technical Resuscitation Skills in a Pediatric Simulation Setting: Resuscitation and Emergency Simulation Checklist for Assessment in Pediatrics.

Science.gov (United States)

Faudeux, Camille; Tran, Antoine; Dupont, Audrey; Desmontils, Jonathan; Montaudié, Isabelle; Bréaud, Jean; Braun, Marc; Fournier, Jean-Paul; Bérard, Etienne; Berlengi, Noémie; Schweitzer, Cyril; Haas, Hervé; Caci, Hervé; Gatin, Amélie; Giovannini-Chami, Lisa

2017-09-01

To develop a reliable and validated tool to evaluate technical resuscitation skills in a pediatric simulation setting. Four Resuscitation and Emergency Simulation Checklist for Assessment in Pediatrics (RESCAPE) evaluation tools were created, following international guidelines: intraosseous needle insertion, bag mask ventilation, endotracheal intubation, and cardiac massage. We applied a modified Delphi methodology evaluation to binary rating items. Reliability was assessed comparing the ratings of 2 observers (1 in real time and 1 after a video-recorded review). The tools were assessed for content, construct, and criterion validity, and for sensitivity to change. Inter-rater reliability, evaluated with Cohen kappa coefficients, was perfect or near-perfect (>0.8) for 92.5% of items and each Cronbach alpha coefficient was ≥0.91. Principal component analyses showed that all 4 tools were unidimensional. Significant increases in median scores with increasing levels of medical expertise were demonstrated for RESCAPE-intraosseous needle insertion (P = .0002), RESCAPE-bag mask ventilation (P = .0002), RESCAPE-endotracheal intubation (P = .0001), and RESCAPE-cardiac massage (P = .0037). Significantly increased median scores over time were also demonstrated during a simulation-based educational program. RESCAPE tools are reliable and validated tools for the evaluation of technical resuscitation skills in pediatric settings during simulation-based educational programs. They might also be used for medical practice performance evaluations. Copyright © 2017 Elsevier Inc. All rights reserved.

Validity of tools used for surveying physicians about their interactions with pharmaceutical company: a systematic review.

Science.gov (United States)

Lotfi, Tamara; Morsi, Rami Z; Zmeter, Nada; Godah, Mohammad W; Alkhaled, Lina; Kahale, Lara A; Nass, Hala; Brax, Hneine; Fadlallah, Racha; Akl, Elie A

2015-11-25

There is evidence that physicians' prescription behavior is negatively affected by the extent of their interactions with pharmaceutical companies. In order to develop and implement policies and interventions for better management of interactions, we need to understand physicians' perspectives on this issue. Surveys addressing physicians' interactions with pharmaceutical companies need to use validated tools to ensure the validity of their findings. To assess the validity of tools used in surveys of physicians about the extent and nature of their interactions with pharmaceutical companies, and about their knowledge, beliefs and attitudes towards such interactions; and to identify those tools that have been formally validated. We developed a search strategy with the assistance of a medical librarian. We electronically searched MEDLINE and EMBASE databases in September 2015. Teams of two reviewers conducted data selection and data abstraction. They identified eligible studies in one table and then abstracted the relevant data from the studies with validated tools in another table. Tables were piloted and standardized. We identified one validated questionnaire out of the 11 assessing the nature and extent of the interaction, and three validated questionnaires out of the 47 assessing knowledge, beliefs and attitudes of physicians toward the interaction. None of these validated questionnaires were used in more than one survey. The available supporting evidence of the issue of physicians' interaction with pharmaceutical company is of low quality. There is a need for research to develop and validate tools to survey physicians about their interactions with pharmaceutical companies.

The letter knowledge assessment tool.

Science.gov (United States)

Pedro, Cassandra; Lousada, Marisa; Pereira, Rita; Hall, Andreia; Jesus, Luis M T

2017-10-10

There is a need to develop letter knowledge assessment tools to characterise the letter knowledge in Portuguese pre-schoolers and to compare it with pre-schoolers from other countries, but there are no tools for this purpose in Portugal. The aim of this paper is to describe the development and validation procedures of the Prova de Avaliação de Competências de Pré-Literacia (PACPL), which assesses letter knowledge. This study includes data that has been gathered in two phases: pilot and main study. In the pilot study, an expert panel of six speech and language pathologists analysed the instrument. Children (n = 216) aged 5;0-7;11 participated in the main study that reports data related to the psychometric characteristics of the PACPL. Content validity, internal consistency, reliability and contributing factors to performance were examined statistically. A modified Bland-Altman method revealed good agreement amongst evaluators. The main study showed that the PACPL has a very good internal consistency and high inter-rater (96.2% of agreement and a Cohen's k value of 0.92) and intra-rater (95.6% of agreement and a Cohen's k value of 0.91) agreement. Construct validity of the PCAPL was also assured (Cronbach's α of 0.982). Significant differences were found between age groups with children increasing their letter knowledge with age. In addition, they were better at identifying than at producing both letter names and letter sounds. The PACPL is a valid and reliable instrument to assess letter knowledge in Portuguese children.

A systematic review and meta-analysis of the criterion validity of nutrition assessment tools for diagnosing protein-energy malnutrition in the older community setting (the MACRo study).

Science.gov (United States)

Marshall, Skye; Craven, Dana; Kelly, Jaimon; Isenring, Elizabeth

2017-10-12

Malnutrition is a significant barrier to healthy and independent ageing in older adults who live in their own homes, and accurate diagnosis is a key step in managing the condition. However, there has not been sufficient systematic review or pooling of existing data regarding malnutrition diagnosis in the geriatric community setting. The current paper was conducted as part of the MACRo (Malnutrition in the Ageing Community Review) Study and seeks to determine the criterion (concurrent and predictive) validity and reliability of nutrition assessment tools in making a diagnosis of protein-energy malnutrition in the general older adult community. A systematic literature review was undertaken using six electronic databases in September 2016. Studies in any language were included which measured malnutrition via a nutrition assessment tool in adults ≥65 years living in their own homes. Data relating to the predictive validity of tools were analysed via meta-analyses. GRADE was used to evaluate the body of evidence. There were 6412 records identified, of which 104 potentially eligible records were screened via full text. Eight papers were included; two which evaluated the concurrent validity of the Mini Nutritional Assessment (MNA) and Subjective Global Assessment (SGA) and six which evaluated the predictive validity of the MNA. The quality of the body of evidence for the concurrent validity of both the MNA and SGA was very low. The quality of the body of evidence for the predictive validity of the MNA in detecting risk of death was moderate (RR: 1.92 [95% CI: 1.55-2.39]; P < 0.00001; n = 2013 participants; n = 4 studies; I 2 : 0%). The quality of the body of evidence for the predictive validity of the MNA in detecting risk of poor physical function was very low (SMD: 1.02 [95%CI: 0.24-1.80]; P = 0.01; n = 4046 participants; n = 3 studies; I 2 :89%). Due to the small number of studies identified and no evaluation of the predictive validity of tools other than

Children's Physical Activity While Gardening: Development of a Valid and Reliable Direct Observation Tool.

Science.gov (United States)

Myers, Beth M; Wells, Nancy M

2015-04-01

Gardens are a promising intervention to promote physical activity (PA) and foster health. However, because of the unique characteristics of gardening, no extant tool can capture PA, postures, and motions that take place in a garden. The Physical Activity Research and Assessment tool for Garden Observation (PARAGON) was developed to assess children's PA levels, tasks, postures, and motions, associations, and interactions while gardening. PARAGON uses momentary time sampling in which a trained observer watches a focal child for 15 seconds and then records behavior for 15 seconds. Sixty-five children (38 girls, 27 boys) at 4 elementary schools in New York State were observed over 8 days. During the observation, children simultaneously wore Actigraph GT3X+ accelerometers. The overall interrater reliability was 88% agreement, and Ebel was .97. Percent agreement values for activity level (93%), garden tasks (93%), motions (80%), associations (95%), and interactions (91%) also met acceptable criteria. Validity was established by previously validated PA codes and by expected convergent validity with accelerometry. PARAGON is a valid and reliable observation tool for assessing children's PA in the context of gardening.

Development of the music therapy assessment Tool for advanced Huntington’s disease: A pilot validation study

DEFF Research Database (Denmark)

O'Kelly, Julian; Bodak, Rebeka

2016-01-01

Background: Case studies of people with Huntington's disease (HD) report that music therapy provides a range of benefits that may improve quality of life; however, no robust music therapy assessment tools exist for this population. Objective: Develop and conduct preliminary psychometric testing...... of its construct validity, internal consistency, and inter-rater and intra-rater reliability over 10 group music therapy sessions with 19 patients. Results: The resulting MATA-HD included a total of 15 items across six subscales (Arousal/Attention, Physical Presentation, Communication, Musical, Cognition......, and Psychological/Behavioral). We found good construct validity (r ≥ 0.7) for Mood, Communication Level, Communication Effectiveness, Choice, Social Behavior, Arousal, and Attention items. Cronbach's α of 0.825 indicated good internal consistency across 11 items with a common focus of engagement in therapy...

Validity evidence as a key marker of quality of technical skill assessment in OTL-HNS.

Science.gov (United States)

Labbé, Mathilde; Young, Meredith; Nguyen, Lily H P

2018-01-13

Quality monitoring of assessment practices should be a priority in all residency programs. Validity evidence is one of the main hallmarks of assessment quality and should be collected to support the interpretation and use of assessment data. Our objective was to identify, synthesize, and present the validity evidence reported supporting different technical skill assessment tools in otolaryngology-head and neck surgery (OTL-HNS). We performed a secondary analysis of data generated through a systematic review of all published tools for assessing technical skills in OTL-HNS (n = 16). For each tool, we coded validity evidence according to the five types of evidence described by the American Educational Research Association's interpretation of Messick's validity framework. Descriptive statistical analyses were conducted. All 16 tools included in our analysis were supported by internal structure and relationship to variables validity evidence. Eleven articles presented evidence supporting content. Response process was discussed only in one article, and no study reported on evidence exploring consequences. We present the validity evidence reported for 16 rater-based tools that could be used for work-based assessment of OTL-HNS residents in the operating room. The articles included in our review were consistently deficient in evidence for response process and consequences. Rater-based assessment tools that support high-stakes decisions that impact the learner and programs should include several sources of validity evidence. Thus, use of any assessment should be done with careful consideration of the context-specific validity evidence supporting score interpretation, and we encourage deliberate continual assessment quality-monitoring. NA. Laryngoscope, 2018. © 2018 The American Laryngological, Rhinological and Otological Society, Inc.

Assessing functional mobility in survivors of lower-extremity sarcoma: reliability and validity of a new assessment tool.

Science.gov (United States)

Marchese, Victoria G; Rai, Shesh N; Carlson, Claire A; Hinds, Pamela S; Spearing, Elena M; Zhang, Lijun; Callaway, Lulie; Neel, Michael D; Rao, Bhaskar N; Ginsberg, Jill P

2007-08-01

Reliability and validity of a new tool, Functional Mobility Assessment (FMA), were examined in patients with lower-extremity sarcoma. FMA requires the patients to physically perform the functional mobility measures, unlike patient self-report or clinician administered measures. A sample of 114 subjects participated, 20 healthy volunteers and 94 patients with lower-extremity sarcoma after amputation, limb-sparing, or rotationplasty surgery. Reliability of the FMA was examined by three raters testing 20 healthy volunteers and 23 subjects with lower-extremity sarcoma. Concurrent validity was examined using data from 94 subjects with lower-extremity sarcoma who completed the FMA, Musculoskeletal Tumor Society (MSTS), Short-Form 36 (SF-36v2), and Toronto Extremity Salvage Scale (TESS) scores. Construct validity was measured by the ability of the FMA to discriminate between subjects with and without functional mobility deficits. FMA demonstrated excellent reliability (ICC [2,1] >or=0.97). Moderate correlations were found between FMA and SF-36v2 (r = 0.60, P < 0.01), FMA and MSTS (r = 0.68, P < 0.01), and FMA and TESS (r = 0.62, P < 0.01). The patients with lower-extremity sarcoma scored lower on the FMA as compared to healthy controls (P < 0.01). The FMA is a reliable and valid functional outcome measure for patients with lower-extremity sarcoma. This study supports the ability of the FMA to discriminate between patients with varying functional abilities and supports the need to include measures of objective functional mobility in examination of patients with lower-extremity sarcoma.

Validation of vocational assessment tool for persons with substance use disorders

Directory of Open Access Journals (Sweden)

Lakshmanan Sethuraman

2016-01-01

Full Text Available Background: Work-related problems are a serious concern among persons with substance use but due to lack of a standardized tool to measure it; these problems are neither systematically assessed nor appropriately addressed. Most existing measures of work performance cater to the needs of the workplace rather than focusing on the workers' perception of the difficulties at work. Aim: To develop a standardized instrument to measure work-related problems in persons with substance use disorders. Methods: Qualitative data obtained from interviews with substance users were used to develop a scale. The refined list of the scale was circulated among an expert panel for content validation. The modified scale was administered to 150 cases, and 50 cases completed the scale twice at the interval of 2 weeks for test–retest reliability. Results: Items with a test–retest reliability kappa coefficient of 0.4 or greater were retained and subjected to factor analysis. The final 45-item scale has a five-factor structure. The value of Cronbach's alpha of the final version of the scale was 0.91. Conclusions: This self-report questionnaire, which can be completed in 10 min, may help us in making a baseline assessment of the work-related impairment among persons with substance use and the impact of substance use on work.

Validation of an organizational communication climate assessment toolkit.

Science.gov (United States)

Wynia, Matthew K; Johnson, Megan; McCoy, Thomas P; Griffin, Leah Passmore; Osborn, Chandra Y

2010-01-01

Effective communication is critical to providing quality health care and can be affected by a number of modifiable organizational factors. The authors performed a prospective multisite validation study of an organizational communication climate assessment tool in 13 geographically and ethnically diverse health care organizations. Communication climate was measured across 9 discrete domains. Patient and staff surveys with matched items in each domain were developed using a national consensus process, which then underwent psychometric field testing and assessment of domain coherence. The authors found meaningful within-site and between-site performance score variability in all domains. In multivariable models, most communication domains were significant predictors of patient-reported quality of care and trust. The authors conclude that these assessment tools provide a valid empirical assessment of organizational communication climate in 9 domains. Assessment results may be useful to track organizational performance, to benchmark, and to inform tailored quality improvement interventions.

Validating Imaginary Worlds? The AdViSHE Assessment Tool

Directory of Open Access Journals (Sweden)

Paul C Langley

2017-01-01

Full Text Available The publication in April 2016 of the Assessment of the Validation Status of Health-Economic Decision Models (AdViSHE checklist for decision models raises a number of issues that the health technology assessment literature has yet to address. The principal issue being the role of decision models in generating claims that are evaluable and replicable. Unfortunately, this is not addressed in this new checklist which is intended to address the perceived need for a tradeoff between confidence in a decision model and the need to allocate resources by developers and payers to validating the model. Irrespective of the degree of confidence a developer or payers may have in the sufficiency of the model in representing ‘reality’ unless the model has generated evaluable claims and evidence for those claims in target treating populations, the model fails the standards of normal science. Apart from the absence of a commitment in the AdViSHE checklist to the modeling of claims that are evaluable and replicable, the validation check list makes no allowance for a product pricing strategy that may commits a manufacturer to regular and substantial annual or semi-annual product price increases. Indeed, product pricing assumptions are conspicuous by their absence. The commentary argues that failure to accommodate anticipated pricing behavior renders lifetime cost-per-QALY models and the application of willingness-to-pay thresholds meaningless.   Type: Commentary

Validation of a 4-item Negative Symptom Assessment (NSA-4): a short, practical clinical tool for the assessment of negative symptoms in schizophrenia.

Science.gov (United States)

Alphs, Larry; Morlock, Robert; Coon, Cheryl; Cazorla, Pilar; Szegedi, Armin; Panagides, John

2011-06-01

The 16-item Negative Symptom Assessment (NSA-16) scale is a validated tool for evaluating negative symptoms of schizophrenia. The psychometric properties and predictive power of a four-item version (NSA-4) were compared with the NSA-16. Baseline data from 561 patients with predominant negative symptoms of schizophrenia who participated in two identically designed clinical trials were evaluated. Ordered logistic regression analysis of ratings using NSA-4 and NSA-16 were compared with ratings using several other standard tools to determine predictive validity and construct validity. Internal consistency and test--retest reliability were also analyzed. NSA-16 and NSA-4 scores were both predictive of scores on the NSA global rating (odds ratio = 0.83-0.86) and the Clinical Global Impressions--Severity scale (odds ratio = 0.91-0.93). NSA-16 and NSA-4 showed high correlation with each other (Pearson r = 0.85), similar high correlation with other measures of negative symptoms (demonstrating convergent validity), and lesser correlations with measures of other forms of psychopathology (demonstrating divergent validity). NSA-16 and NSA-4 both showed acceptable internal consistency (Cronbach α, 0.85 and 0.64, respectively) and test--retest reliability (intraclass correlation coefficient, 0.87 and 0.82). This study demonstrates that NSA-4 offers accuracy comparable to the NSA-16 in rating negative symptoms in patients with schizophrenia. Copyright © 2011 John Wiley & Sons, Ltd.

Assessing patient-caregiver communication in cancer--a psychometric validation of the Cancer Communication Assessment Tool (CCAT-PF) in a German sample.

Science.gov (United States)

Haun, Markus W; Sklenarova, Halina; Winkler, Eva C; Huber, Johannes; Thomas, Michael; Siminoff, Laura A; Woll, Michael; Brechtel, Anette; Herzog, Wolfgang; Hartmann, Mechthild

2014-09-01

The recently introduced Cancer Communication Assessment Tool (CCAT-PF) measures congruence in patient-caregiver communication and was initially validated in lung cancer patients. Contributing to a greater proportion of the variance in the conflict scores, primary caregivers were hypothesized to experience greater stress. For a detailed understanding of conflicting communication patterns of cancer-affected families, our study aimed for psychometric validation of the CCAT-PF in a sample covering heterogeneous tumor entities. Subsequent to a cross-sectional survey of 189 pairs of cancer patients (31 % gastrointestinal, 34 % lung, and 35 % urological) and their caregivers' exploratory factor analysis with principal component condensation and varimax rotation was conducted (response rate, 74.2 %). Reliability and construct validity were assessed calculating Cronbach's α and Pearson correlation coefficients for CCAT-P and CCAT-F scales and related constructs, respectively. Cancer-related communication according to the CCAT-PF can be subdivided into four factors including the scales Disclosure, Limitation of treatment, Family involvement in treatment decisions, and Continuing treatment. Reliability ranged from α = .51-.68. The Disclosure scale, describing poor cancer-related communication of the patient, was correlated with patient's distress (QSC-R10: r = .30, p reliable stand-alone instrument for identifying conflicting communication in patient-caregiver-dyads at risk.

Screening for Psychosocial Risk in Dutch Families of a Child With Cancer: Reliability, Validity, and Usability of the Psychosocial Assessment Tool

NARCIS (Netherlands)

Sint Nicolaas, Simone M.; Schepers, Sasja A.; Hoogerbrugge, Peter M.; Caron, Huib N.; Kaspers, Gertjan J. L.; van den Heuvel-Eibrink, Marry M.; Grootenhuis, Martha A.; Verhaak, Chris M.

2016-01-01

The Psychosocial Assessment Tool (PAT) was developed to screen for psychosocial risk in families of a child diagnosed with cancer. The current study is the first describing the cross-cultural adaptation, reliability, validity, and usability of the PAT in an European country (Dutch translation).   A

Validation of a measurement tool to assess awareness of breast cancer.

Science.gov (United States)

Linsell, Louise; Forbes, Lindsay J L; Burgess, Caroline; Kapari, Marcia; Thurnham, Angela; Ramirez, Amanda J

2010-05-01

Until now, there has been no universally accepted and validated measure of breast cancer awareness. This study aimed to validate the new Breast Cancer Awareness Measure (BCAM) which assesses, using a self-complete questionnaire, knowledge of breast cancer symptoms and age-related risk, and frequency of breast checking. We measured the psychometric properties of the BCAM in 1035 women attending the NHS Breast Screening Programme: acceptability was assessed using a feedback questionnaire (n=292); sensitivity to change after an intervention promoting breast cancer awareness (n=576), and test-retest reliability (n=167). We also assessed readability, and construct validity using the 'known-groups' method. The readability of the BCAM was high. Over 90% of women found it acceptable. The BCAM was sensitive to change: there was an increase in the proportion of women obtaining the full score for breast cancer awareness one month after receiving the intervention promoting breast cancer awareness; this was greater among those who received a more intensive version (less intensive version (booklet): 9.3%, 95% confidence interval (CI): 4.5-14.1%; more intensive version (interaction with health professional plus booklet): 30%, 95% CI: 23.4-36.6%). Test-retest reliability of the BCAM was moderate to good for most items. Cancer experts had higher levels of cancer awareness than non-medical academics (50% versus 6%, p=0.001), indicating good construct validity. The BCAM is a valid and robust measure of breast cancer awareness suitable for use in surveys of breast cancer awareness in the general population and to evaluate the impact of awareness-raising interventions. Copyright (c) 2010 Elsevier Ltd. All rights reserved.

Content Validity of a Tool Measuring Medication Errors.

Science.gov (United States)

Tabassum, Nishat; Allana, Saleema; Saeed, Tanveer; Dias, Jacqueline Maria

2015-08-01

The objective of this study was to determine content and face validity of a tool measuring medication errors among nursing students in baccalaureate nursing education. Data was collected from the Aga Khan University School of Nursing and Midwifery (AKUSoNaM), Karachi, from March to August 2014. The tool was developed utilizing literature and the expertise of the team members, expert in different areas. The developed tool was then sent to five experts from all over Karachi for ensuring the content validity of the tool, which was measured on relevance and clarity of the questions. The Scale Content Validity Index (S-CVI) for clarity and relevance of the questions was found to be 0.94 and 0.98, respectively. The tool measuring medication errors has an excellent content validity. This tool should be used for future studies on medication errors, with different study populations such as medical students, doctors, and nurses.

The risk of bias in systematic reviews tool showed fair reliability and good construct validity.

Science.gov (United States)

Bühn, Stefanie; Mathes, Tim; Prengel, Peggy; Wegewitz, Uta; Ostermann, Thomas; Robens, Sibylle; Pieper, Dawid

2017-11-01

There is a movement from generic quality checklists toward a more domain-based approach in critical appraisal tools. This study aimed to report on a first experience with the newly developed risk of bias in systematic reviews (ROBIS) tool and compare it with A Measurement Tool to Assess Systematic Reviews (AMSTAR), that is, the most common used tool to assess methodological quality of systematic reviews while assessing validity, reliability, and applicability. Validation study with four reviewers based on 16 systematic reviews in the field of occupational health. Interrater reliability (IRR) of all four raters was highest for domain 2 (Fleiss' kappa κ = 0.56) and lowest for domain 4 (κ = 0.04). For ROBIS, median IRR was κ = 0.52 (range 0.13-0.88) for the experienced pair of raters compared to κ = 0.32 (range 0.12-0.76) for the less experienced pair of raters. The percentage of "yes" scores of each review of ROBIS ratings was strongly correlated with the AMSTAR ratings (r s  = 0.76; P = 0.01). ROBIS has fair reliability and good construct validity to assess the risk of bias in systematic reviews. More validation studies are needed to investigate reliability and applicability, in particular. Copyright © 2017 Elsevier Inc. All rights reserved.

The validity of a professional competence tool for physiotherapy students in simulation-based clinical education: a Rasch analysis.

Science.gov (United States)

Judd, Belinda K; Scanlan, Justin N; Alison, Jennifer A; Waters, Donna; Gordon, Christopher J

2016-08-05

Despite the recent widespread adoption of simulation in clinical education in physiotherapy, there is a lack of validated tools for assessment in this setting. The Assessment of Physiotherapy Practice (APP) is a comprehensive tool used in clinical placement settings in Australia to measure professional competence of physiotherapy students. The aim of the study was to evaluate the validity of the APP for student assessment in simulation settings. A total of 1260 APPs were collected, 971 from students in simulation and 289 from students in clinical placements. Rasch analysis was used to examine the construct validity of the APP tool in three different simulation assessment formats: longitudinal assessment over 1 week of simulation; longitudinal assessment over 2 weeks; and a short-form (25 min) assessment of a single simulation scenario. Comparison with APPs from 5 week clinical placements in hospital and clinic-based settings were also conducted. The APP demonstrated acceptable fit to the expectations of the Rasch model for the 1 and 2 week clinical simulations, exhibiting unidimensional properties that were able to distinguish different levels of student performance. For the short-form simulation, nine of the 20 items recorded greater than 25 % of scores as 'not-assessed' by clinical educators which impacted on the suitability of the APP tool in this simulation format. The APP was a valid assessment tool when used in longitudinal simulation formats. A revised APP may be required for assessment in short-form simulation scenarios.

Operator competence in fetoscopic laser surgery for twin-twin transfusion syndrome: validation of a procedure-specific evaluation tool.

Science.gov (United States)

Peeters, S H P; Akkermans, J; Bustraan, J; Middeldorp, J M; Lopriore, E; Devlieger, R; Lewi, L; Deprest, J; Oepkes, D

2016-03-01

Fetoscopic laser surgery for twin-twin transfusion syndrome is a procedure for which no objective tools exist to assess technical skills. To ensure that future fetal surgeons reach competence prior to performing the procedure unsupervised, we developed a performance assessment tool. The aim of this study was to validate this assessment tool for reliability and construct validity. We made use of a procedure-specific evaluation instrument containing all essential steps of the fetoscopic laser procedure, which was previously created using Delphi methodology. Eleven experts and 13 novices from three fetal medicine centers performed the procedure on the same simulator. Two independent observers assessed each surgery using the instrument (maximum score: 52). Interobserver reliability was assessed using Spearman correlation. We compared the performance of novices and experts to assess construct validity. The interobserver reliability was high (Rs  = 0.974, P performed by experts and in 9/13 (69%) procedures performed by novices (P = 0.005). Multivariable analysis showed that the checklist score, independent of age and gender, predicted competence. The procedure-specific assessment tool for fetoscopic laser surgery shows good interobserver reliability and discriminates experts from novices. This instrument may therefore be a useful tool in the training curriculum for fetal surgeons. Further intervention studies with reassessment before and after training may increase the construct validity of the tool. Copyright © 2015 ISUOG. Published by John Wiley & Sons Ltd. Copyright © 2015 ISUOG. Published by John Wiley & Sons Ltd.

Teamwork assessment in internal medicine: a systematic review of validity evidence and outcomes.

Science.gov (United States)

Havyer, Rachel D A; Wingo, Majken T; Comfere, Nneka I; Nelson, Darlene R; Halvorsen, Andrew J; McDonald, Furman S; Reed, Darcy A

2014-06-01

Valid teamwork assessment is imperative to determine physician competency and optimize patient outcomes. We systematically reviewed published instruments assessing teamwork in undergraduate, graduate, and continuing medical education in general internal medicine and all medical subspecialties. We searched MEDLINE, MEDLINE In-process, CINAHL and PsycINFO from January 1979 through October 2012, references of included articles, and abstracts from four professional meetings. Two content experts were queried for additional studies. Included studies described quantitative tools measuring teamwork among medical students, residents, fellows, and practicing physicians on single or multi-professional (interprofessional) teams. Instrument validity and study quality were extracted using established frameworks with existing validity evidence. Two authors independently abstracted 30 % of articles and agreement was calculated. Of 12,922 citations, 178 articles describing 73 unique teamwork assessment tools met inclusion criteria. Interrater agreement was intraclass correlation coefficient 0.73 (95 % CI 0.63-0.81). Studies involved practicing physicians (142, 80 %), residents/fellows (70, 39 %), and medical students (11, 6 %). The majority (152, 85 %) assessed interprofessional teams. Studies were conducted in inpatient (77, 43 %), outpatient (42, 24 %), simulation (37, 21 %), and classroom (13, 7 %) settings. Validity evidence for the 73 tools included content (54, 74 %), internal structure (51, 70 %), relationships to other variables (25, 34 %), and response process (12, 16 %). Attitudes and opinions were the most frequently assessed outcomes. Relationships between teamwork scores and patient outcomes were directly examined for 13 (18 %) of tools. Scores from the Safety Attitudes Questionnaire and Team Climate Inventory have substantial validity evidence and have been associated with improved patient outcomes. Review is limited to quantitative assessments of teamwork in internal

A tool for assessing the quality of nursing handovers: a validation study.

Science.gov (United States)

Ferrara, Paolo; Terzoni, Stefano; Davì, Salvatore; Bisesti, Alberto; Destrebecq, Anne

2017-08-10

Handover, in particular between two shifts, is a crucial aspect of nursing for patient safety, aimed at ensuring continuity of care. During this process, several factors can affect quality of care and cause errors. This study aimed to assess quality of handovers, by validating the Handoff CEX-Italian scale. The scale was translated from English into Italian and the content validity index was calculated and internal consistency assessed. The scale was used in several units of the San Paolo Teaching Hospital in Milan, Italy. A total of 48 reports were assessed (192 evaluations). The median score was 6, interquartile range (IQR) [5;7] and was not influenced by specific (p=0.21) or overall working experience (p=0.13). The domains showing the lowest median values (median=6, IQR [4;8]) were context, communication, and organisation. Night to morning handovers obtained the lowest scores. CVI-S was 0.96, Cronbach's alpha was 0.79. The Handoff CEX-Italian scale is valid and reliable and it can be used to assess the quality of nurse handovers.

A Rapid Assessment Tool for affirming good practice in midwifery education programming.

Science.gov (United States)

Fullerton, Judith T; Johnson, Peter; Lobe, Erika; Myint, Khine Haymar; Aung, Nan Nan; Moe, Thida; Linn, Nay Aung

2016-03-01

to design a criterion-referenced assessment tool that could be used globally in a rapid assessment of good practices and bottlenecks in midwifery education programs. a standard tool development process was followed, to generate standards and reference criteria; followed by external review and field testing to document psychometric properties. review of standards and scoring criteria were conducted by stakeholders around the globe. Field testing of the tool was conducted in Myanmar. eleven of Myanmar׳s 22 midwifery education programs participated in the assessment. the clinimetric tool was demonstrated to have content validity and high inter-rater reliability in use. a globally validated tool, and accompanying user guide and handbook are now available for conducting rapid assessments of compliance with good practice criteria in midwifery education programming. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.

Content validation of an interprofessional learning video peer assessment tool.

Science.gov (United States)

Nisbet, Gillian; Jorm, Christine; Roberts, Chris; Gordon, Christopher J; Chen, Timothy F

2017-12-16

Large scale models of interprofessional learning (IPL) where outcomes are assessed are rare within health professional curricula. To date, there is sparse research describing robust assessment strategies to support such activities. We describe the development of an IPL assessment task based on peer rating of a student generated video evidencing collaborative interprofessional practice. We provide content validation evidence of an assessment rubric in the context of large scale IPL. Two established approaches to scale development in an educational setting were combined. A literature review was undertaken to develop a conceptual model of the relevant domains and issues pertaining to assessment of student generated videos within IPL. Starting with a prototype rubric developed from the literature, a series of staff and student workshops were undertaken to integrate expert opinion and user perspectives. Participants assessed five-minute videos produced in a prior pilot IPL activity. Outcomes from each workshop informed the next version of the rubric until agreement was reached on anchoring statements and criteria. At this point the rubric was declared fit to be used in the upcoming mandatory large scale IPL activity. The assessment rubric consisted of four domains: patient issues, interprofessional negotiation; interprofessional management plan in action; and effective use of video medium to engage audience. The first three domains reflected topic content relevant to the underlying construct of interprofessional collaborative practice. The fourth domain was consistent with the broader video assessment literature calling for greater emphasis on creativity in education. We have provided evidence for the content validity of a video-based peer assessment task portraying interprofessional collaborative practice in the context of large-scale IPL activities for healthcare professional students. Further research is needed to establish the reliability of such a scale.

Development of a Multi-Domain Assessment Tool for Quality Improvement Projects.

Science.gov (United States)

Rosenbluth, Glenn; Burman, Natalie J; Ranji, Sumant R; Boscardin, Christy K

2017-08-01

Improving the quality of health care and education has become a mandate at all levels within the medical profession. While several published quality improvement (QI) assessment tools exist, all have limitations in addressing the range of QI projects undertaken by learners in undergraduate medical education, graduate medical education, and continuing medical education. We developed and validated a tool to assess QI projects with learner engagement across the educational continuum. After reviewing existing tools, we interviewed local faculty who taught QI to understand how learners were engaged and what these faculty wanted in an ideal assessment tool. We then developed a list of competencies associated with QI, established items linked to these competencies, revised the items using an iterative process, and collected validity evidence for the tool. The resulting Multi-Domain Assessment of Quality Improvement Projects (MAQIP) rating tool contains 9 items, with criteria that may be completely fulfilled, partially fulfilled, or not fulfilled. Interrater reliability was 0.77. Untrained local faculty were able to use the tool with minimal guidance. The MAQIP is a 9-item, user-friendly tool that can be used to assess QI projects at various stages and to provide formative and summative feedback to learners at all levels.

Validity of portfolio assessment: which qualities determine ratings?

NARCIS (Netherlands)

Driessen, E.W.; Overeem, K.; Tartwijk, J. van; Vleuten, C.P.M. van der; Muijtjens, A.M.M.

2006-01-01

The portfolio is becoming increasingly accepted as a valuable tool for learning and assessment. The validity of portfolio assessment, however, may suffer from bias due to irrelevant qualities, such as lay-out and writing style. We examined the possible effects of such qualities in a portfolio

Validating GIS tool to assess eelgrass potential recovery in the Limfjorden (Denmark)

DEFF Research Database (Denmark)

Canal-Vergés, Paula; Petersen, Jens Kjerulf; Rasmussen, Erik K.

2016-01-01

Eelgrass is a key indicator for the water quality in Europe (WFD, European Union, 2000). However, although water quality has been improved in most Danish water bodies, the eelgrass population does not seem to be recovering. In this study, we validate and further develop a GIS tool designed...... by Flindt et al. (2016), to evaluate the potential of eelgrass reestablishment in Danish waters. The GIS tool was tested in two large broads of the Limfjorden, Løgstør and Lovns broad (Denmark), where two scenarios are run. The first scenario was set up including modelled data, whereas the second scenario...... in the period 2009-2013, in the two investigated broads. A combination of modelled and monitored data was found to be optimal to achieve accurate predictions for eelgrass development in the Limfjorden using this GIS tool. In order to implement the current model or to use this GIS tool in other locations...

Older adult mistreatment risk screening: contribution to the validation of a screening tool in a domestic setting.

Science.gov (United States)

Lindenbach, Jeannette M; Larocque, Sylvie; Lavoie, Anne-Marise; Garceau, Marie-Luce

2012-06-01

ABSTRACTThe hidden nature of older adult mistreatment renders its detection in the domestic setting particularly challenging. A validated screening instrument that can provide a systematic assessment of risk factors can facilitate this detection. One such instrument, the "expanded Indicators of Abuse" tool, has been previously validated in the Hebrew language in a hospital setting. The present study has contributed to the validation of the "e-IOA" in an English-speaking community setting in Ontario, Canada. It consisted of two phases: (a) a content validity review and adaptation of the instrument by experts throughout Ontario, and (b) an inter-rater reliability assessment by home visiting nurses. The adaptation, the "Mistreatment of Older Adult Risk Factors" tool, offers a comprehensive tool for screening in the home setting. This instrument is significant to professional practice as practitioners working with older adults will be better equipped to assess for risk of mistreatment.

The script concordance test in radiation oncology: validation study of a new tool to assess clinical reasoning

International Nuclear Information System (INIS)

Lambert, Carole; Gagnon, Robert; Nguyen, David; Charlin, Bernard

2009-01-01

The Script Concordance test (SCT) is a reliable and valid tool to evaluate clinical reasoning in complex situations where experts' opinions may be divided. Scores reflect the degree of concordance between the performance of examinees and that of a reference panel of experienced physicians. The purpose of this study is to demonstrate SCT's usefulness in radiation oncology. A 90 items radiation oncology SCT was administered to 155 participants. Three levels of experience were tested: medical students (n = 70), radiation oncology residents (n = 38) and radiation oncologists (n = 47). Statistical tests were performed to assess reliability and to document validity. After item optimization, the test comprised 30 cases and 70 questions. Cronbach alpha was 0.90. Mean scores were 51.62 (± 8.19) for students, 71.20 (± 9.45) for residents and 76.67 (± 6.14) for radiation oncologists. The difference between the three groups was statistically significant when compared by the Kruskall-Wallis test (p < 0.001). The SCT is reliable and useful to discriminate among participants according to their level of experience in radiation oncology. It appears as a useful tool to document the progression of reasoning during residency training

Validation of a novel venous duplex ultrasound objective structured assessment of technical skills for the assessment of venous reflux.

Science.gov (United States)

Jaffer, Usman; Normahani, Pasha; Lackenby, Kimberly; Aslam, Mohammed; Standfield, Nigel J

2015-01-01

Duplex ultrasound measurement of reflux time is central to the diagnosis of venous incompetence. We have developed an assessment tool for Duplex measurement of venous reflux for both simulator and patient-based training. A novel assessment tool, Venous Duplex Ultrasound Assessment of Technical Skills (V-DUOSATS), was developed. A modified DUOSATS was used for simulator training. Participants of varying skill level were invited to viewed an instructional video and were allowed ample time to familiarize with the Duplex equipment. Attempts made by the participants were recorded and independently assessed by 3 expert assessors and 5 novice assessors using the modified V-DUOSATS. "Global" assessment was also done by expert assessors on a 4-point Likert scale. Content, construct, and concurrent validities as well as reliability were evaluated. Content and construct validity as well as reliability were demonstrated. Receiver operator characteristic analysis-established cut points of 19/22 and 21/30 were most appropriate for simulator and patient-based assessment, respectively. We have validated a novel assessment tool for Duplex venous reflux measurement. Further work is required to establish transference validity of simulator training to improve skill in scanning patients. We have developed and validated V-DUOSATS for simulator training. Copyright © 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

The Shoulder Objective Practical Assessment Tool: Evaluation of a New Tool Assessing Residents Learning in Diagnostic Shoulder Arthroscopy.

Science.gov (United States)

Talbot, Christopher L; Holt, Edward M; Gooding, Benjamin W T; Tennent, Thomas D; Foden, Philip

2015-08-01

To design and validate an objective practical assessment tool for diagnostic shoulder arthroscopy that would provide residents with a method to evaluate their progression in this field of surgery and to identify specific learning needs. We designed and evaluated the shoulder Objective Practical Assessment Tool (OPAT). The shoulder OPAT was designed by us, and scoring domains were created using a Delphi process. The shoulder OPAT was trialed by members of the British Elbow & Shoulder Society Education Committee for internal consistency and ease of use before being offered to other trainers and residents. Inter-rater reliability and intrarater reliability were calculated. One hundred forty orthopaedic residents, of varying seniority, within 5 training regions in the United Kingdom, were questioned regarding the tool. A pilot study of 6 residents was undertaken. Internal consistency was 0.77 (standardized Cronbach α). Inter-rater reliability was 0.60, and intrarater reliability was 0.82. The Spearman correlation coefficient (r) between the global summary score for the shoulder OPAT and the current assessment tool used in postgraduate training for orthopaedic residents undertaking diagnostic shoulder arthroscopy equaled 0.74. Of the residents, 82% agreed or strongly agreed when asked if the shoulder OPAT would be a useful tool in monitoring progression and 72% agreed or strongly agreed with the introduction of the shoulder OPAT within the orthopaedic domain. This study shows that the shoulder OPAT fulfills several aspects of reliability and validity when tested. Despite the inter-rater reliability being 0.60, we believe that the shoulder OPAT has the potential to play a role alongside the current assessment tool in the training of orthopaedic residents. The shoulder OPAT can be used to assess residents during shoulder arthroscopy and has the potential for use in medical education, as well as arthroscopic skills training in the operating theater. Copyright © 2015

Implementation of a digital preparation validation tool in dental skills laboratory training.

Science.gov (United States)

Kozarovska, A; Larsson, C

2018-05-01

To describe the implementation of a digital tool for preparation validation and evaluate it as an aid in students' self-assessment. Students at the final semester of skills laboratory training were asked to use a digital preparation validation tool (PVT) when performing two different tasks; preparation of crowns for teeth 11 and 21. The students were divided into two groups. Group A self-assessed and scanned all three attempts at 21 ("prep-and-scan"). Group B self-assessed all attempts chose the best one and scanned it ("best-of-three"). The situation was reversed for 11. The students assessed five parameters of the preparation and marked them as approved (A) or failed (F). These marks were compared with the information from the PVT. The students also completed a questionnaire. Each question was rated from 1 to 5. Teachers' opinions were collected at staff meetings throughout the project. Most students in the "prep-and-scan" groups showed an increase in agreement between their self-assessment and the information from the PVT, whereas students in the "best-of-three" groups showed lower levels of agreement. All students rated the PVT positively. Most strongly agreed that the tool was helpful in developing skills (mean 4.15), easy to use (mean 4.23) and that it added benefits in comparison to existing assessment tools (mean 4.05). They did not however, fully agree that the tool is time efficient (mean 2.55), and they did not consider it a substitute for verbal teacher feedback. Teachers' feedback suggested advantages of the tool in the form of ease of use, visual aid and increasing interest and motivation during skills laboratory training however, they did not notice a reduction in need of verbal feedback. Within the limitations of the study, our conclusion is that a digital PVT may be a valuable adjunct to other assessment tools in skills laboratory training. © 2017 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Development and Validity Testing of the Worksite Health Index: An Assessment Tool to Help and Improve Korean Employees' Health-Related Outcome.

Science.gov (United States)

Yun, Young Ho; Sim, Jin Ah; Lim, Ye Jin; Lim, Cheol Il; Kang, Sung-Choon; Kang, Joon-Ho; Park, Jun Dong; Noh, Dong Young

2016-06-01

The objective of this study was to develop the Worksite Health Index (WHI) and validate its psychometric properties. The development of the WHI questionnaire included item generation, item construction, and field testing. To assess the instrument's reliability and validity, we recruited 30 different Korean worksites. We developed the WHI questionnaire of 136 items categorized into five domains, namely Governance and Infrastructure, Need Assessment and Planning, Health Prevention and Promotion Program, Occupational Safety, and Monitoring and Feedback. All WHI domains demonstrated a high reliability with good internal consistency. The total WHI scores differentiated worksite groups effectively according to firm size. Each domain was associated significantly with employees' health status, absence, and financial outcome. The WHI can assess comprehensive worksite health programs. This tool is publicly available for addressing the growing need for worksite health programs.

Validation of a dietary intake tool for african-american dialysis patients with low literacy.

Science.gov (United States)

Duffrin, Christopher; Carraway-Stage, Virginia G; Briley, Alexis; Christiano, Cynthia

2015-06-01

This study analysed the validity and reliability of a food frequency questionnaire designed for African-American patients with low literacy. This instrument was designed specifically to meet the need for a tool that was short, easy to understand, and met clinical reliability and validity standards. Assessing patient nutritional status and dietary intake is crucial to the care of patients in end stage kidney disease. The development of a quick and reliable nutritional assessment tool for patients with low literacy could increase nutritional counselling effectiveness and improve patient outcomes. The renal food frequency questionnaire (RFF) and a standard 24-hour recall were administered to a general population of African-American patients undergoing dialysis. Registered Dieticians and statistical analyses were used to validate the content and structural validity and reliability of the RFF to adequately measure dietary intake. The study sample consisted of 30 African-American patients who received dialysis treatment at a regional teaching hospital facility. The RFF was found to be a simple, easy to understand instrument with low reading complexity (grade level 4.4). Inter-rater reliability was found to be high (.81-1.00), and statistical analysis determined a high level of clinical validity. The RFF was found to be a valid dietary recall tool that is appropriate for patients with limited literacy. It was found to have acceptable reliability and validity when compared with a standard 24-hour recall and has potential for use as a dietary intake and monitoring tool in patients undergoing dialysis. © 2014 European Dialysis and Transplant Nurses Association/European Renal Care Association.

Predictive validity of the post-enrolment English language assessment tool for commencing undergraduate nursing students.

Science.gov (United States)

Glew, Paul J; Hillege, Sharon P; Salamonson, Yenna; Dixon, Kathleen; Good, Anthony; Lombardo, Lien

2015-12-01

Nursing students with English as an additional language (EAL) may underperform academically. The post-enrolment English language assessment (PELA) is used in literacy support, but its predictive validity in identifying those at risk of underperformance remains unknown. To validate a PELA, as a predictor of academic performance. Prospective survey design. The study was conducted at a university located in culturally and linguistically diverse areas of western Sydney, Australia. Commencing undergraduate nursing students who were Australian-born (n=1323, 49.6%) and born outside of Australia (n=1346, 50.4%) were recruited for this study. The 2669 (67% of 3957) participants provided consent and completed a first year nursing unit that focussed on developing literacy skills. Between 2010 and 2013, commencing students completed the PELA and English language acculturation scale (ELAS), a previously validated instrument. The grading levels of the PELA tool were: Level 1 (proficient), Level 2 (borderline), and Level 3 (poor, and requiring additional support). Participants with a PELA Level 2 or 3 were more likely to be: a) non-Australian-born (χ(2): 520.6, df: 2, pstudent (χ(2): 225.6, df: 2, pstudents who are at risk of academic underachievement. Crown Copyright © 2015. Published by Elsevier Ltd. All rights reserved.

The Myotonometer: Not a Valid Measurement Tool for Active Hamstring Musculotendinous Stiffness.

Science.gov (United States)

Pamukoff, Derek N; Bell, Sarah E; Ryan, Eric D; Blackburn, J Troy

2016-05-01

Hamstring musculotendinous stiffness (MTS) is associated with lower-extremity injury risk (ie, hamstring strain, anterior cruciate ligament injury) and is commonly assessed using the damped oscillatory technique. However, despite a preponderance of studies that measure MTS reliably in laboratory settings, there are no valid clinical measurement tools. A valid clinical measurement technique is needed to assess MTS and permit identification of individuals at heightened risk of injury and track rehabilitation progress. To determine the validity and reliability of the Myotonometer for measuring active hamstring MTS. Descriptive laboratory study. Laboratory. 33 healthy participants (15 men, age 21.33 ± 2.94 y, height 172.03 ± 16.36 cm, mass 74.21 ± 16.36 kg). Hamstring MTS was assessed using the damped oscillatory technique and the Myotonometer. Intraclass correlations were used to determine the intrasession, intersession, and interrater reliability of the Myotonometer. Criterion validity was assessed via Pearson product-moment correlation between MTS measures obtained from the Myotonometer and from the damped oscillatory technique. The Myotonometer demonstrated good intrasession (ICC3,1 = .807) and interrater reliability (ICC2,k = .830) and moderate intersession reliability (ICC2,k = .693). However, it did not provide a valid measurement of MTS compared with the damped oscillatory technique (r = .346, P = .061). The Myotonometer does not provide a valid measure of active hamstring MTS. Although the Myotonometer does not measure active MTS, it possesses good reliability and portability and could be used clinically to measure tissue compliance, muscle tone, or spasticity associated with multiple musculoskeletal disorders. Future research should focus on portable and clinically applicable tools to measure active hamstring MTS in efforts to prevent and monitor injuries.

Teamwork Assessment Tools in Modern Surgical Practice: A Systematic Review

Science.gov (United States)

Whittaker, George; Abboudi, Hamid; Khan, Muhammed Shamim; Dasgupta, Prokar; Ahmed, Kamran

2015-01-01

Introduction. Deficiencies in teamwork skills have been shown to contribute to the occurrence of adverse events during surgery. Consequently, several teamwork assessment tools have been developed to evaluate trainee nontechnical performance. This paper aims to provide an overview of these instruments and review the validity of each tool. Furthermore, the present paper aims to review the deficiencies surrounding training and propose several recommendations to address these issues. Methods. A systematic literature search was conducted to identify teamwork assessment tools using MEDLINE (1946 to August 2015), EMBASE (1974 to August 2015), and PsycINFO (1806 to August 2015) databases. Results. Eight assessment tools which encompass aspects of teamwork were identified. The Nontechnical Skills for Surgeons (NOTSS) assessment was found to possess the highest level of validity from a variety of sources; reliability and acceptability have also been established for this tool. Conclusions. Deficits in current surgical training pathways have prompted several recommendations to meet the evolving requirements of surgeons. Recommendations from the current paper include integration of teamwork training and assessment into medical school curricula, standardised formal training of assessors to ensure accurate evaluation of nontechnical skill acquisition, and integration of concurrent technical and nontechnical skills training throughout training. PMID:26425732

Endoscopy nurse-administered propofol sedation performance. Development of an assessment tool and a reliability testing model

DEFF Research Database (Denmark)

Jensen, Jeppe Thue; Konge, Lars; Møller, Ann

2014-01-01

of training and for future certification. The aim of this study was to develop an assessment tool for measuring competency in propofol sedation and to explore the reliability and validity of the tool. MATERIAL AND METHODS: The nurse-administered propofol assessment tool (NAPSAT) was developed in a Delphi...... and good construct validity. This makes NAPSAT fit for formative assessment and proficiency feedback; however, high stakes and summative assessment cannot be advised....

Validity and Reliability of Knowledge, Attitude and Behavior Assessment Tool Among Vulnerable Women Concerning Sexually Transmitted Diseases

Directory of Open Access Journals (Sweden)

Zahra Boroumandfar

2016-05-01

Full Text Available Objective: The study aimed to design and evaluate the content and face validity, and reliability of knowledge, attitude, and behavior questionnaire on preventive behaviors among vulnerable women concerning sexually transmitted diseases (STDs.Materials and methods: This cross-sectional study was carried out in two phases of an action research. In the first phase, to explain STDs preventive domains, 20 semi- structured interviews were conducted with the vulnerable women, residing at women prison and women referred to counseling centers. After analyzing content of interviews, three domains were identified: improve their knowledge, modify their attitude and change their behaviors. In the second phase, the questionnaire was designed and tested in a pilot study. Then, its content validity was evaluated. Face validity and reliability of the questionnaire were assessed by test re- test method and Cronbach alpha respectively.Results: Index of content validity in each three domain of the questionnaire (knowledge, attitude and behavior concerning STDs was obtained over 0.6. Overall content validity index was 0.86 in all three domains of the questionnaire. The Cronbach’s alpha as reliability of questionnaire was 0.80 for knowledge, 0.79 for attitude and 0.85 for behavior.Conclusion: The results showed that the designed questionnaire was a valid and reliable tool to measure knowledge, attitude and behavior of vulnerable women, predisposed to risk of STDs.

Mammography image assessment; validity and reliability of current scheme

International Nuclear Information System (INIS)

Hill, C.; Robinson, L.

2015-01-01

Mammographers currently score their own images according to criteria set out by Regional Quality Assurance. The criteria used are based on the ‘Perfect, Good, Moderate, Inadequate’ (PGMI) marking criteria established by the National Health Service Breast Screening Programme (NHSBSP) in their Quality Assurance Guidelines of 2006 1 . This document discusses the validity and reliability of the current mammography image assessment scheme. Commencing with a critical review of the literature this document sets out to highlight problems with the national approach to the use of marking schemes. The findings suggest that ‘PGMI’ scheme is flawed in terms of reliability and validity and is not universally applied across the UK. There also appear to be differences in schemes used by trainees and qualified mammographers. Initial recommendations are to be made in collaboration with colleagues within the National Health Service Breast Screening Programme (NHSBSP), Higher Education Centres, College of Radiographers and the Royal College of Radiologists in order to identify a mammography image appraisal scheme that is fit for purpose. - Highlights: • Currently no robust evidence based marking tools in use for the assessment of images in mammography. • Is current system valid, reliable and robust? • How can the current image assessment tool be improved? • Should students and qualified mammographers use the same tool? • What marking criteria are available for image assessment?

Cross-cultural validation of health literacy measurement tools in Italian oncology patients.

Science.gov (United States)

Zotti, Paola; Cocchi, Simone; Polesel, Jerry; Cipolat Mis, Chiara; Bragatto, Donato; Cavuto, Silvio; Conficconi, Alice; Costanzo, Carla; De Giorgi, Melissa; Drace, Christina A; Fiorini, Federica; Gangeri, Laura; Lisi, Andrea; Martino, Rosalba; Mosconi, Paola; Paradiso, Angelo; Ravaioli, Valentina; Truccolo, Ivana; De Paoli, Paolo

2017-06-19

The aim of this study was to assess the psychometric characteristics of four Health Literacy (HL) measurement tools, viz. Newest Vital Sign (NVS), Short Test of Functional Health Literacy in Adults (STOFHLA), Single Item Literacy Screener (SILS) and Single question on Self-rated Reading Ability (SrRA) among Italian oncology patients. The original version of the tools were translated from the English language into Italian using a standard forward-backward procedure and according to internationally recognized good practices. Their internal consistency (reliability) and validity (construct, convergent and discriminative) were tested in a sample of 245 consecutive cancer patients recruited from seven Italian health care centers. The internal consistency of the STOFHLA-I was Chronbach's α=0.96 and that of NVS-I was α=0.74. The STOFHLA-I, NVS-I, SILS-I and SrRA-I scores were in a good relative correlation and in all tools the discriminative known-group validity was confirmed. The reliability and validity values were similar to those obtained from other cultural context studies. The psychometric characteristics of the Italian version of NVS, STHOFLA, SILS and SrRA were found to be good, with satisfactory reliability and validity. This indicates that they could be used as a screening tool in Italian patients. Moreover, the use of the same cross-cultural tools, validated in different languages, is essential for implementing multicenter studies to measure and compare the functional HL levels across countries.

Assessing behavioural changes in ALS: cross-validation of ALS-specific measures.

Science.gov (United States)

Pinto-Grau, Marta; Costello, Emmet; O'Connor, Sarah; Elamin, Marwa; Burke, Tom; Heverin, Mark; Pender, Niall; Hardiman, Orla

2017-07-01

The Beaumont Behavioural Inventory (BBI) is a behavioural proxy report for the assessment of behavioural changes in ALS. This tool has been validated against the FrSBe, a non-ALS-specific behavioural assessment, and further comparison of the BBI against a disease-specific tool was considered. This study cross-validates the BBI against the ALS-FTD-Q. Sixty ALS patients, 8% also meeting criteria for FTD, were recruited. All patients were evaluated using the BBI and the ALS-FTD-Q, completed by a carer. Correlational analysis was performed to assess construct validity. Precision, sensitivity, specificity, and overall accuracy of the BBI when compared to the ALS-FTD-Q, were obtained. The mean score of the whole sample on the BBI was 11.45 ± 13.06. ALS-FTD patients scored significantly higher than non-demented ALS patients (31.6 ± 14.64, 9.62 ± 11.38; p ALS-FTD-Q was observed (r = 0.807, p ALS-FTD-Q. Good construct validity has been further confirmed when the BBI is compared to an ALS-specific tool. Furthermore, the BBI is a more comprehensive behavioural assessment for ALS, as it measures the whole behavioural spectrum in this condition.

Construction and validation of a tool to Assess the Use of Light Technologies at Intensive Care Units.

Science.gov (United States)

Marinho, Pabliane Matias Lordelo; Campos, Maria Pontes de Aguiar; Rodrigues, Eliana Ofélia Llapa; Gois, Cristiane Franca Lisboa; Barreto, Ikaro Daniel de Carvalho

2016-12-19

to construct and validate a tool to assess the use of light technologies by the nursing team at Intensive Care Units. methodological study in which the tool was elaborated by means of the psychometric method for construction based on the categorization of health technologies by Merhy and Franco, from the National Humanization Policy, using the Nursing Intervention Classification taxonomy to categorize the domains of the tool. Agreement Percentages and Content Validity Indices were used for the purpose of validation. The result of the application of the Interrater Agreement Percentage exceeded the recommended level of 80%, highlighting the relevance for the proposed theme in the assessment, with an agreement rate of 99%. the tool was validated with four domains (Bond, Autonomy, Welcoming and Management) and nineteen items that assess the use of light technologies at Intensive Care Units. construir e validar um instrumento para avaliação do uso de tecnologias leves, pela equipe de enfermagem, em Unidades de Terapia Intensiva. estudo metodológico no qual o instrumento foi elaborado utilizando o método psicométrico para construção com base na categorização das tecnologias em saúde de Merhy e Franco, da Política Nacional de Humanização, utilizando-se a taxonomia Nursing Intervention Classification para categorizar os domínios do instrumento. Utilizou-se o Percentual de Concordância e o Índice de Validade de Conteúdo (IVC) para validação. o resultado da aplicação do Percentual de Concordância entre os juízes foi superior ao recomendado de 80%, havendo destaque na avaliação da pertinência ao tema proposto, apresentando um percentual de concordância de 99%. o instrumento foi validado com quatro domínios (Vínculo, Autonomia, Acolhimento e Gestão) e dezenove itens que avaliam o uso das tecnologias leves em Unidade de Terapia Intensiva. construir y validar un instrumento para evaluación del uso de tecnologías leves, por el equipo de enfermer

A Student Assessment Tool for Standardized Patient Simulations (SAT-SPS): Psychometric analysis.

Science.gov (United States)

Castro-Yuste, Cristina; García-Cabanillas, María José; Rodríguez-Cornejo, María Jesús; Carnicer-Fuentes, Concepción; Paloma-Castro, Olga; Moreno-Corral, Luis Javier

2018-05-01

The evaluation of the level of clinical competence acquired by the student is a complex process that must meet various requirements to ensure its quality. The psychometric analysis of the data collected by the assessment tools used is a fundamental aspect to guarantee the student's competence level. To conduct a psychometric analysis of an instrument which assesses clinical competence in nursing students at simulation stations with standardized patients in OSCE-format tests. The construct of clinical competence was operationalized as a set of observable and measurable behaviors, measured by the newly-created Student Assessment Tool for Standardized Patient Simulations (SAT-SPS), which was comprised of 27 items. The categories assigned to the items were 'incorrect or not performed' (0), 'acceptable' (1), and 'correct' (2). 499 nursing students. Data were collected by two independent observers during the assessment of the students' performance at a four-station OSCE with standardized patients. Descriptive statistics were used to summarize the variables. The difficulty levels and floor and ceiling effects were determined for each item. Reliability was analyzed using internal consistency and inter-observer reliability. The validity analysis was performed considering face validity, content and construct validity (through exploratory factor analysis), and criterion validity. Internal reliability and inter-observer reliability were higher than 0.80. The construct validity analysis suggested a three-factor model accounting for 37.1% of the variance. These three factors were named 'Nursing process', 'Communication skills', and 'Safe practice'. A significant correlation was found between the scores obtained and the students' grades in general, as well as with the grades obtained in subjects with clinical content. The assessment tool has proven to be sufficiently reliable and valid for the assessment of the clinical competence of nursing students using standardized patients

A clinical assessment tool for ultrasound-guided axillary brachial plexus block.

LENUS (Irish Health Repository)

Sultan, S F

2012-05-01

Competency in anesthesia traditionally has been determined subjectively in practice. Optimal training in procedural skills requires valid and reliable forms of assessment. The objective was to examine a procedure-specific clinical assessment tool for ultrasound-guided axillary brachial plexus block for inter-rater reliability and construct validity in a clinical setting.

An Integrative Review of Pediatric Fall Risk Assessment Tools.

Science.gov (United States)

DiGerolamo, Kimberly; Davis, Katherine Finn

Patient fall prevention begins with accurate risk assessment. However, sustained improvements in prevention and quality of care include use of validated fall risk assessment tools (FRATs). The goal of FRATs is to identify patients at highest risk. Adult FRATs are often borrowed from to create tools for pediatric patients. Though factors associated with pediatric falls in the hospital setting are similar to those in adults, such as mobility, medication use, and cognitive impairment, adult FRATs and the factors associated with them do not adequately assess risk in children. Articles were limited to English language, ages 0-21years, and publish date 2006-2015. The search yielded 22 articles. Ten were excluded as the population was primarily adult or lacked discussion of a FRAT. Critical appraisal and findings were synthesized using the Johns Hopkins Nursing evidence appraisal system. Twelve articles relevant to fall prevention in the pediatric hospital setting that discussed fall risk assessment and use of a FRAT were reviewed. Comparison between and accuracy of FRATs is challenged when different classifications, definitions, risk stratification, and inclusion criteria are used. Though there are several pediatric FRATs published in the literature, none have been found to be reliable and valid across institutions and diverse populations. This integrative review highlights the importance of choosing a FRAT based on an institution's identified risk factors and validating the tool for one's own patient population as well as using the tool in conjunction with nursing clinical judgment to guide interventions. Copyright © 2017 Elsevier Inc. All rights reserved.

A Systematic Review of Tools Used to Assess Team Leadership in Health Care Action Teams.

Science.gov (United States)

Rosenman, Elizabeth D; Ilgen, Jonathan S; Shandro, Jamie R; Harper, Amy L; Fernandez, Rosemarie

2015-10-01

To summarize the characteristics of tools used to assess leadership in health care action (HCA) teams. HCA teams are interdisciplinary teams performing complex, critical tasks under high-pressure conditions. The authors conducted a systematic review of the PubMed/MEDLINE, CINAHL, ERIC, EMBASE, PsycINFO, and Web of Science databases, key journals, and review articles published through March 2012 for English-language articles that applied leadership assessment tools to HCA teams in all specialties. Pairs of reviewers assessed identified articles for inclusion and exclusion criteria and abstracted data on study characteristics, tool characteristics, and validity evidence. Of the 9,913 abstracts screened, 83 studies were included. They described 61 team leadership assessment tools. Forty-nine tools (80%) provided behaviors, skills, or characteristics to define leadership. Forty-four tools (72%) assessed leadership as one component of a larger assessment, 13 tools (21%) identified leadership as the primary focus of the assessment, and 4 (7%) assessed leadership style. Fifty-three studies (64%) assessed leadership at the team level; 29 (35%) did so at the individual level. Assessments of simulated (n = 55) and live (n = 30) patient care events were performed. Validity evidence included content validity (n = 75), internal structure (n = 61), relationship to other variables (n = 44), and response process (n = 15). Leadership assessment tools applied to HCA teams are heterogeneous in content and application. Comparisons between tools are limited by study variability. A systematic approach to team leadership tool development, evaluation, and implementation will strengthen understanding of this important competency.

Comparison of tools for assessing the methodological quality of primary and secondary studies in health technology assessment reports in Germany

Directory of Open Access Journals (Sweden)

Walter, Ulla

2010-01-01

Full Text Available Health care policy background: Findings from scientific studies form the basis for evidence-based health policy decisions. Scientific background: Quality assessments to evaluate the credibility of study results are an essential part of health technology assessment reports and systematic reviews. Quality assessment tools (QAT for assessing the study quality examine to what extent study results are systematically distorted by confounding or bias (internal validity. The tools can be divided into checklists, scales and component ratings. Research questions: What QAT are available to assess the quality of interventional studies or studies in the field of health economics, how do they differ from each other and what conclusions can be drawn from these results for quality assessments? Methods: A systematic search of relevant databases from 1988 onwards is done, supplemented by screening of the references, of the HTA reports of the German Agency for Health Technology Assessment (DAHTA and an internet search. The selection of relevant literature, the data extraction and the quality assessment are carried out by two independent reviewers. The substantive elements of the QAT are extracted using a modified criteria list consisting of items and domains specific to randomized trials, observational studies, diagnostic studies, systematic reviews and health economic studies. Based on the number of covered items and domains, more and less comprehensive QAT are distinguished. In order to exchange experiences regarding problems in the practical application of tools, a workshop is hosted. Results: A total of eight systematic methodological reviews is identified as well as 147 QAT: 15 for systematic reviews, 80 for randomized trials, 30 for observational studies, 17 for diagnostic studies and 22 for health economic studies. The tools vary considerably with regard to the content, the performance and quality of operationalisation. Some tools do not only include the

Comparison of tools for assessing the methodological quality of primary and secondary studies in health technology assessment reports in Germany.

Science.gov (United States)

Dreier, Maren; Borutta, Birgit; Stahmeyer, Jona; Krauth, Christian; Walter, Ulla

2010-06-14

HEALTH CARE POLICY BACKGROUND: Findings from scientific studies form the basis for evidence-based health policy decisions. Quality assessments to evaluate the credibility of study results are an essential part of health technology assessment reports and systematic reviews. Quality assessment tools (QAT) for assessing the study quality examine to what extent study results are systematically distorted by confounding or bias (internal validity). The tools can be divided into checklists, scales and component ratings. What QAT are available to assess the quality of interventional studies or studies in the field of health economics, how do they differ from each other and what conclusions can be drawn from these results for quality assessments? A systematic search of relevant databases from 1988 onwards is done, supplemented by screening of the references, of the HTA reports of the German Agency for Health Technology Assessment (DAHTA) and an internet search. The selection of relevant literature, the data extraction and the quality assessment are carried out by two independent reviewers. The substantive elements of the QAT are extracted using a modified criteria list consisting of items and domains specific to randomized trials, observational studies, diagnostic studies, systematic reviews and health economic studies. Based on the number of covered items and domains, more and less comprehensive QAT are distinguished. In order to exchange experiences regarding problems in the practical application of tools, a workshop is hosted. A total of eight systematic methodological reviews is identified as well as 147 QAT: 15 for systematic reviews, 80 for randomized trials, 30 for observational studies, 17 for diagnostic studies and 22 for health economic studies. The tools vary considerably with regard to the content, the performance and quality of operationalisation. Some tools do not only include the items of internal validity but also the items of quality of reporting and

Development and validation of the FRAGIRE tool for assessment an older person's risk for frailty.

Science.gov (United States)

Vernerey, Dewi; Anota, Amelie; Vandel, Pierre; Paget-Bailly, Sophie; Dion, Michele; Bailly, Vanessa; Bonin, Marie; Pozet, Astrid; Foubert, Audrey; Benetkiewicz, Magdalena; Manckoundia, Patrick; Bonnetain, Franck

2016-11-17

Frailty is highly prevalent in elderly people. While significant progress has been made to understand its pathogenesis process, few validated questionnaire exist to assess the multidimensional concept of frailty and to detect people frail or at risk to become frail. The objectives of this study were to construct and validate a new frailty-screening instrument named Frailty Groupe Iso-Ressource Evaluation (FRAGIRE) that accurately predicts the risk for frailty in older adults. A prospective multicenter recruitment of the elderly patients was undertaken in France. The subjects were classified into financially-helped group (FH, with financial assistance) and non-financially helped group (NFH, without any financial assistance), considering FH subjects are more frail than the NFH group and thus representing an acceptable surrogate population for frailty. Psychometric properties of the FRAGIRE grid were assessed including discrimination between the FH and NFH groups. Items reduction was made according to statistical analyses and experts' point of view. The association between items response and tests with "help requested status" was assessed in univariate and multivariate unconditional logistic regression analyses and a prognostic score to become frail was finally proposed for each subject. Between May 2013 and July 2013, 385 subjects were included: 338 (88%) in the FH group and 47 (12%) in the NFH group. The initial FRAGIRE grid included 65 items. After conducting the item selection, the final grid of the FRAGIRE was reduced to 19 items. The final grid showed fair discrimination ability to predict frailty (area under the curve (AUC) = 0.85) and good calibration (Hosmer-Lemeshow P-value = 0.580), reflecting a good agreement between the prediction by the final model and actual observation. The Cronbach's alpha for the developed tool scored as high as 0.69 (95% Confidence Interval: 0.64 to 0.74). The final prognostic score was excellent, with an AUC of 0

Developing the Polish Educational Needs Assessment Tool (Pol-ENAT) in rheumatoid arthritis and systemic sclerosis: a cross-cultural validation study using Rasch analysis.

Science.gov (United States)

Sierakowska, Matylda; Sierakowski, Stanisław; Sierakowska, Justyna; Horton, Mike; Ndosi, Mwidimi

2015-03-01

To undertake cross-cultural adaptation and validation of the educational needs assessment tool (ENAT) for use with people with rheumatoid arthritis (RA) and systemic sclerosis (SSc) in Poland. The study involved two main phases: (1) cross-cultural adaptation of the ENAT from English into Polish and (2) Cross-cultural validation of Polish Educational Needs Assessment Tool (Pol-ENAT). The first phase followed an established process of cross-cultural adaptation of self-report measures. The second phase involved completion of the Pol-ENAT by patients and subjecting the data to Rasch analysis to assess the construct validity, unidimensionality, internal consistency and cross-cultural invariance. An adequate conceptual equivalence was achieved following the adaptation process. The dataset for validation comprised a total of 278 patients, 237 (85.3 %) of which were female. In each disease group (145, RA and 133, SSc), the 7 domains of the Pol-ENAT were found to fit the Rasch model, X (2)(df) = 16.953(14), p = 0.259 and 8.132(14), p = 0.882 for RA and SSc, respectively. Internal consistency of the Pol-ENAT was high (patient separation index = 0.85 and 0.89 for SSc and RA, respectively), and unidimensionality was confirmed. Cross-cultural differential item functioning (DIF) was detected in some subscales, and DIF-adjusted conversion tables were calibrated to enable cross-cultural comparison of data between Poland and the UK. Using a standard process in cross-cultural adaptation, conceptual equivalence was achieved between the original (UK) ENAT and the adapted Pol-ENAT. Fit to the Rasch model, confirmed that the construct validity, unidimensionality and internal consistency of the ENAT have been preserved.

Patient Simulation: A Literary Synthesis of Assessment Tools in Anesthesiology

Directory of Open Access Journals (Sweden)

Alice A. Edler

2009-12-01

Full Text Available High-fidelity patient simulation (HFPS has been hypothesized as a modality for assessing competency of knowledge and skill in patient simulation, but uniform methods for HFPS performance assessment (PA have not yet been completely achieved. Anesthesiology as a field founded the HFPS discipline and also leads in its PA. This project reviews the types, quality, and designated purpose of HFPS PA tools in anesthesiology. We used the systematic review method and systematically reviewed anesthesiology literature referenced in PubMed to assess the quality and reliability of available PA tools in HFPS. Of 412 articles identified, 50 met our inclusion criteria. Seventy seven percent of studies have been published since 2000; more recent studies demonstrated higher quality. Investigators reported a variety of test construction and validation methods. The most commonly reported test construction methods included “modified Delphi Techniques” for item selection, reliability measurement using inter-rater agreement, and intra-class correlations between test items or subtests. Modern test theory, in particular generalizability theory, was used in nine (18% of studies. Test score validity has been addressed in multiple investigations and shown a significant improvement in reporting accuracy. However the assessment of predicative has been low across the majority of studies. Usability and practicality of testing occasions and tools was only anecdotally reported. To more completely comply with the gold standards for PA design, both shared experience of experts and recognition of test construction standards, including reliability and validity measurements, instrument piloting, rater training, and explicit identification of the purpose and proposed use of the assessment tool, are required.

Verification and Validation Strategy for LWRS Tools

Energy Technology Data Exchange (ETDEWEB)

Carl M. Stoots; Richard R. Schultz; Hans D. Gougar; Thomas K Larson; Michael Corradini; Laura Swiler; David Pointer; Jess Gehin

2012-09-01

One intension of the Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) program is to create advanced computational tools for safety assessment that enable more accurate representation of a nuclear power plant safety margin. These tools are to be used to study the unique issues posed by lifetime extension and relicensing of the existing operating fleet of nuclear power plants well beyond their first license extension period. The extent to which new computational models / codes such as RELAP-7 can be used for reactor licensing / relicensing activities depends mainly upon the thoroughness with which they have been verified and validated (V&V). This document outlines the LWRS program strategy by which RELAP-7 code V&V planning is to be accomplished. From the perspective of developing and applying thermal-hydraulic and reactivity-specific models to reactor systems, the US Nuclear Regulatory Commission (NRC) Regulatory Guide 1.203 gives key guidance to numeric model developers and those tasked with the validation of numeric models. By creating Regulatory Guide 1.203 the NRC defined a framework for development, assessment, and approval of transient and accident analysis methods. As a result, this methodology is very relevant and is recommended as the path forward for RELAP-7 V&V. However, the unique issues posed by lifetime extension will require considerations in addition to those addressed in Regulatory Guide 1.203. Some of these include prioritization of which plants / designs should be studied first, coupling modern supporting experiments to the stringent needs of new high fidelity models / codes, and scaling of aging effects.

The PRECIS-2 tool has good interrater reliability and modest discriminant validity.

Science.gov (United States)

Loudon, Kirsty; Zwarenstein, Merrick; Sullivan, Frank M; Donnan, Peter T; Gágyor, Ildikó; Hobbelen, Hans J S M; Althabe, Fernando; Krishnan, Jerry A; Treweek, Shaun

2017-08-01

PRagmatic Explanatory Continuum Indicator Summary (PRECIS)-2 is a tool that could improve design insight for trialists. Our aim was to validate the PRECIS-2 tool, unlike its predecessor, testing the discriminant validity and interrater reliability. Over 80 international trialists, methodologists, clinicians, and policymakers created PRECIS-2 helping to ensure face validity and content validity. The interrater reliability of PRECIS-2 was measured using 19 experienced trialists who used PRECIS-2 to score a diverse sample of 15 randomized controlled trial protocols. Discriminant validity was tested with two raters to independently determine if the trial protocols were more pragmatic or more explanatory, with scores from the 19 raters for the 15 trials as predictors of pragmatism. Interrater reliability was generally good, with seven of nine domains having an intraclass correlation coefficient over 0.65. Flexibility (adherence) and recruitment had wide confidence intervals, but raters found these difficult to rate and wanted more information. Each of the nine PRECIS-2 domains could be used to differentiate between trials taking more pragmatic or more explanatory approaches with better than chance discrimination for all domains. We have assessed the validity and reliability of PRECIS-2. An elaboration study and web site provide guidance to help future users of the tool which is continuing to be tested by trial teams, systematic reviewers, and funders. Copyright © 2017 Elsevier Inc. All rights reserved.

Reliability of the ECHOWS Tool for Assessment of Patient Interviewing Skills.

Science.gov (United States)

Boissonnault, Jill S; Evans, Kerrie; Tuttle, Neil; Hetzel, Scott J; Boissonnault, William G

2016-04-01

History taking is an important component of patient/client management. Assessment of student history-taking competency can be achieved via a standardized tool. The ECHOWS tool has been shown to be valid with modest intrarater reliability in a previous study but did not demonstrate sufficient power to definitively prove its stability. The purposes of this study were: (1) to assess the reliability of the ECHOWS tool for student assessment of patient interviewing skills and (2) to determine whether the tool discerns between novice and experienced skill levels. A reliability and construct validity assessment was conducted. Three faculty members from the United States and Australia scored videotaped histories from standardized patients taken by students and experienced clinicians from each of these countries. The tapes were scored twice, 3 to 6 weeks apart. Reliability was assessed using interclass correlation coefficients (ICCs) and repeated measures. Analysis of variance models assessed the ability of the tool to discern between novice and experienced skill levels. The ECHOWS tool showed excellent intrarater reliability (ICC [3,1]=.74-.89) and good interrater reliability (ICC [2,1]=.55) as a whole. The summary of performance (S) section showed poor interrater reliability (ICC [2,1]=.27). There was no statistical difference in performance on the tool between novice and experienced clinicians. A possible ceiling effect may occur when standardized patients are not coached to provide complex and obtuse responses to interviewer questions. Variation in familiarity with the ECHOWS tool and in use of the online training may have influenced scoring of the S section. The ECHOWS tool demonstrates excellent intrarater reliability and moderate interrater reliability. Sufficient training with the tool prior to student assessment is recommended. The S section must evolve in order to provide a more discerning measure of interviewing skills. © 2016 American Physical Therapy

Validation of a survey tool for use in cross-cultural studies

Directory of Open Access Journals (Sweden)

Costa FA

2008-09-01

Full Text Available There is a need for tools to measure the information patients need in order for healthcare professionals in general, and particularly pharmacists, to communicate effectively and play an active part in the way patients manage their medicines. Previous research has developed and validated constructs to measure patients’ desires for information and their perceptions of how useful their medicines are. It is important to develop these tools for use in different settings and countries so that best practice is shared and is based on the best available evidence. Objectives: this project sought to validate of a survey tool measuring the “Extent of Information Desired” (EID, the “Perceived Utility of Medicines” (PUM, and the “Anxiety about Illness” (AI that had been previously translated for use with Portuguese patients. Methods: The scales were validated in a patient sample of 596: construct validity was explored in Factor analysis (PCA and internal consistency analysed using Cronbach’s alpha. Criterion validity was explored correlating scores to the AI scale and patients’ perceived health status. Discriminatory power was assessed using ANOVA. Temporal stability was explored in a sub-sample of patients who responded at two time points, using a T-test to compare their mean scores. Results: Construct validity results indicated the need to remove 1 item from the Perceived Harm of Medicines (PHM and Perceived Benefit of Medicines (PBM for use in a Portuguese sample and the abandon of the tolerance scale. The internal consistency was high for the EID, PBM and AI scales (alpha>0.600 and acceptable for the PHM scale (alpha=0.536. All scales, except the EID, were consistent over time (p>0.05; p<0.01. All the scales tested showed good discriminatory power. The comparison of the AI scale with the SF-36 indicated good criterion validity (p<0.05.Conclusion: The translated tool was valid and reliable in Portuguese patients- excluding the Tolerance

Developing a tool for assessing competency in root cause analysis.

Science.gov (United States)

Gupta, Priyanka; Varkey, Prathibha

2009-01-01

Root cause analysis (RCA) is a tool for identifying the key cause(s) contributing to a sentinel event or near miss. Although training in RCA is gaining popularity in medical education, there is no published literature on valid or reliable methods for assessing competency in the same. A tool for assessing competency in RCA was pilot tested as part of an eight-station Objective Structured Clinical Examination that was conducted at the completion of a three-week quality improvement (QI) curriculum for the Mayo Clinic Preventive Medicine and Endocrinology fellowship programs. As part of the curriculum, fellows completed a QI project to enhance physician communication of the diagnosis and treatment plan at the end of a patient visit. They had a didactic session on RCA, followed by process mapping of the information flow at the project clinic, after which fellows conducted an actual RCA using the Ishikawa fishbone diagram. For the RCA competency assessment, fellows performed an RCA regarding a scenario describing an adverse medication event and provided possible solutions to prevent such errors in the future. All faculty strongly agreed or agreed that they were able to accurately assess competency in RCA using the tool. Interrater reliability for the global competency rating and checklist scoring were 0.96 and 0.85, respectively. Internal consistency (Cronbach's alpha) was 0.76. Six of eight of the fellows found the difficulty level of the test to be optimal. Assessment methods must accompany education programs to ensure that graduates are competent in QI methodologies and are able to apply them effectively in the workplace. The RCA assessment tool was found to be a valid, reliable, feasible, and acceptable method for assessing competency in RCA. Further research is needed to examine its predictive validity and generalizability.

Development of an assessment tool to measure students′ perceptions of respiratory care education programs: Item generation, item reduction, and preliminary validation

Directory of Open Access Journals (Sweden)

Ghazi Alotaibi

2013-01-01

Full Text Available Objectives: Students who perceived their learning environment positively are more likely to develop effective learning strategies, and adopt a deep learning approach. Currently, there is no validated instrument for measuring the educational environment of educational programs on respiratory care (RC. The aim of this study was to develop an instrument to measure students′ perception of the RC educational environment. Materials and Methods: Based on the literature review and an assessment of content validity by multiple focus groups of RC educationalists, potential items of the instrument relevant to RC educational environment construct were generated by the research group. The initial 71 item questionnaire was then field-tested on all students from the 3 RC programs in Saudi Arabia and was subjected to multi-trait scaling analysis. Cronbach′s alpha was used to assess internal consistency reliabilities. Results: Two hundred and twelve students (100% completed the survey. The initial instrument of 71 items was reduced to 65 across 5 scales. Convergent and discriminant validity assessment demonstrated that the majority of items correlated more highly with their intended scale than a competing one. Cronbach′s alpha exceeded the standard criterion of >0.70 in all scales except one. There was no floor or ceiling effect for scale or overall score. Conclusions: This instrument is the first assessment tool developed to measure the RC educational environment. There was evidence of its good feasibility, validity, and reliability. This first validation of the instrument supports its use by RC students to evaluate educational environment.

Initial Steps in Creating a Developmentally Valid Tool for Observing/Assessing Rope Jumping

Science.gov (United States)

Roberton, Mary Ann; Thompson, Gregory; Langendorfer, Stephen J.

2017-01-01

Background: Valid motor development sequences show the various behaviors that children display as they progress toward competence in specific motor skills. Teachers can use these sequences to observe informally or formally assess their students. While longitudinal study is ultimately required to validate developmental sequences, there are earlier,…

Development and validation of an observation tool for the assessment of nursing pain management practices in intensive care unit in a standardized clinical simulation setting.

Science.gov (United States)

Gosselin, Emilie; Bourgault, Patricia; Lavoie, Stephan; Coleman, Robin-Marie; Méziat-Burdin, Anne

2014-12-01

Pain management in the intensive care unit is often inadequate. There is no tool available to assess nursing pain management practices. The aim of this study was to develop and validate a measuring tool to assess nursing pain management in the intensive care unit during standardized clinical simulation. A literature review was performed to identify relevant components demonstrating optimal pain management in adult intensive care units and to integrate them in an observation tool. This tool was submitted to an expert panel and pretested. It was then used to assess pain management practice during 26 discrete standardized clinical simulation sessions with intensive care nurses. The Nursing Observation Tool for Pain Management (NOTPaM) contains 28 statements grouped into 8 categories, which are grouped into 4 dimensions: subjective assessment, objective assessment, interventions, and reassessment. The tool's internal consistency was calculated at a Cronbach's alpha of 0.436 for the whole tool; the alpha varies from 0.328 to 0.518 for each dimension. To evaluate the inter-rater reliability, intra-class correlation coefficient was used, which was calculated at 0.751 (p nurses' pain management in a standardized clinical simulation. The NOTPaM is the first tool created for this purpose. Copyright © 2014 American Society for Pain Management Nursing. Published by Elsevier Inc. All rights reserved.

Health impact assessment in planning: Development of the design for health HIA tools

International Nuclear Information System (INIS)

Forsyth, Ann; Slotterback, Carissa Schively; Krizek, Kevin J.

2010-01-01

How can planners more systematically incorporate health concerns into practical planning processes? This paper describes a suite of health impact assessment tools (HIAs) developed specifically for planning practice. Taking an evidence-based approach the tools are designed to fit into existing planning activities. The tools include: a short audit tool, the Preliminary Checklist; a structured participatory workshop, the Rapid HIA; an intermediate health impact assessment, the Threshold Analysis; and a set of Plan Review Checklists. This description provides a basis for future work including assessing tool validity, refining specific tools, and creating alternatives.

Pain Assessment in Critically İll Adult Patients: Validity and Reliability Research of the Turkish Version of the Critical-Care Pain Observation Tool

Directory of Open Access Journals (Sweden)

Onur Gündoğan

2016-12-01

Full Text Available Objective: Critical-Care Pain Observation Tool (CPOT and the Behavioral Pain Scale (BPS are behavioral pain assessment scales for unconscious intensive care unit (ICU patients. The aim is to determine the validation and reliability of the CPOT in Turkish in mechanically ventilated adult ICU patients. Material and Method: This prospective observational cohort study included 50 mechanically ventilated mixed ICU patients who were unable to report pain. CPOT and BPS was translated into Turkish and language validity was performed by ten intensive care specialists. Pain was assessed in the course of painless and painful routine care procedures using the CPOT and the BPS by a resident and an intensivist concomitantly. Tests reliability, interrater reliability, and validity of the CPOT and the BPS were evaluated. Results: The mean age was 57.4 years and the mean APACHE II score was 18.7. A total of 100 assessments were recorded from 50 patients using CPOT and BPS. Scores of CPOT and BPS during the painful procedures were both significantly higher than painless procedures. The agreement between CPOT and BPS during painful and painless stimuli was ranged as; sensitivity 66.7%-90.3%; specificity 89.7%-97.9%; kappa value 0.712-0.892. The agreement between resident and intensivist during painful and painless stimuli was ranged from 97% to 100% and the kappa value was between 0.904 and 1.0. Conclusion: The Turkish version of the CPOT showed good correlation with the BPS. Interrater reliability between resident and intensivist was good. The study showed that the Turkish version of BPS and CPOT are reliable and valid tools to assess pain in daily clinical practice for intubated and unconscious ICU patients who are mechanically ventilated.

Development and validation of a short version of the Partnership Self-Assessment Tool (PSAT among professionals in Dutch disease-management partnerships

Directory of Open Access Journals (Sweden)

Nieboer Anna P

2011-06-01

Full Text Available Abstract Background The extent to which partnership synergy is created within quality improvement programmes in the Netherlands is unknown. In this article, we describe the psychometric testing of the Partnership Self-Assessment Tool (PSAT among professionals in twenty-two disease-management partnerships participating in quality improvement projects focused on chronic care in the Netherlands. Our objectives are to validate the PSAT in the Netherlands and to reduce the number of items of the original PSAT while maintaining validity and reliability. Methods The Dutch version of the PSAT was tested in twenty-two disease-management partnerships with 218 professionals. We tested the instrument by means of structural equation modelling, and examined its validity and reliability. Results After eliminating 14 items, the confirmatory factor analyses revealed good indices of fit with the resulting 15-item PSAT-Short version (PSAT-S. Internal consistency as represented by Cronbach's alpha ranged from acceptable (0.75 for the 'efficiency' subscale to excellent for the 'leadership' subscale (0.87. Convergent validity was provided with high correlations of the partnership dimensions and partnership synergy (ranged from 0.512 to 0.609 and high correlations with chronic illness care (ranged from 0.447 to 0.329. Conclusion The psychometric properties and convergent validity of the PSAT-S were satisfactory rendering it a valid and reliable instrument for assessing partnership synergy and its dimensions of partnership functioning.

Design and validation of a three-instrument toolkit for the assessment of competence in electrocardiogram rhythm recognition.

Science.gov (United States)

Hernández-Padilla, José M; Granero-Molina, José; Márquez-Hernández, Verónica V; Suthers, Fiona; López-Entrambasaguas, Olga M; Fernández-Sola, Cayetano

2017-06-01

Rapid and accurate interpretation of cardiac arrhythmias by nurses has been linked with safe practice and positive patient outcomes. Although training in electrocardiogram rhythm recognition is part of most undergraduate nursing programmes, research continues to suggest that nurses and nursing students lack competence in recognising cardiac rhythms. In order to promote patient safety, nursing educators must develop valid and reliable assessment tools that allow the rigorous assessment of this competence before nursing students are allowed to practise without supervision. The aim of this study was to develop and psychometrically evaluate a toolkit to holistically assess competence in electrocardiogram rhythm recognition. Following a convenience sampling technique, 293 nursing students from a nursing faculty in a Spanish university were recruited for the study. The following three instruments were developed and psychometrically tested: an electrocardiogram knowledge assessment tool (ECG-KAT), an electrocardiogram skills assessment tool (ECG-SAT) and an electrocardiogram self-efficacy assessment tool (ECG-SES). Reliability and validity (content, criterion and construct) of these tools were meticulously examined. A high Cronbach's alpha coefficient demonstrated the excellent reliability of the instruments (ECG-KAT=0.89; ECG-SAT=0.93; ECG-SES=0.98). An excellent context validity index (scales' average content validity index>0.94) and very good criterion validity were evidenced for all the tools. Regarding construct validity, principal component analysis revealed that all items comprising the instruments contributed to measure knowledge, skills or self-efficacy in electrocardiogram rhythm recognition. Moreover, known-groups analysis showed the tools' ability to detect expected differences in competence between groups with different training experiences. The three-instrument toolkit developed showed excellent psychometric properties for measuring competence in

The Music Therapy Session Assessment Scale (MT-SAS): Validation of a new tool for music therapy process evaluation.

Science.gov (United States)

Raglio, Alfredo; Gnesi, Marco; Monti, Maria Cristina; Oasi, Osmano; Gianotti, Marta; Attardo, Lapo; Gontero, Giulia; Morotti, Lara; Boffelli, Sara; Imbriani, Chiara; Montomoli, Cristina; Imbriani, Marcello

2017-11-01

Music therapy (MT) interventions are aimed at creating and developing a relationship between patient and therapist. However, there is a lack of validated observational instruments to consistently evaluate the MT process. The purpose of this study was the validation of Music Therapy Session Assessment Scale (MT-SAS), designed to assess the relationship between therapist and patient during active MT sessions. Videotapes of a single 30-min session per patient were considered. A pilot study on the videotapes of 10 patients was carried out to help refine the items, define the scoring system and improve inter-rater reliability among the five raters. Then, a validation study on 100 patients with different clinical conditions was carried out. The Italian MT-SAS was used throughout the process, although we also provide an English translation. The final scale consisted of 7 binary items accounting for eye contact, countenance, and nonverbal and sound-music communication. In the pilot study, raters were found to share an acceptable level of agreement in their assessments. Explorative factorial analysis disclosed a single homogeneous factor including 6 items (thus supporting an ordinal total score), with only the item about eye contact being unrelated to the others. Moreover, the existence of 2 different archetypal profiles of attuned and disattuned behaviours was highlighted through multiple correspondence analysis. As suggested by the consistent results of 2 different analyses, MT-SAS is a reliable tool that globally evaluates sonorous-musical and nonverbal behaviours related to emotional attunement and empathetic relationship between patient and therapist during active MT sessions. Copyright © 2017 John Wiley & Sons, Ltd.

Diagnostic flexible pharyngo-laryngoscopy: development of a procedure specific assessment tool using a Delphi methodology.

Science.gov (United States)

Melchiors, Jacob; Henriksen, Mikael Johannes Vuokko; Dikkers, Frederik G; Gavilán, Javier; Noordzij, J Pieter; Fried, Marvin P; Novakovic, Daniel; Fagan, Johannes; Charabi, Birgitte W; Konge, Lars; von Buchwald, Christian

2018-05-01

Proper training and assessment of skill in flexible pharyngo-laryngoscopy are central in the education of otorhinolaryngologists. To facilitate an evidence-based approach to curriculum development in this field, a structured analysis of what constitutes flexible pharyngo-laryngoscopy is necessary. Our aim was to develop an assessment tool based on this analysis. We conducted an international Delphi study involving experts from twelve countries in five continents. Utilizing reiterative assessment, the panel defined the procedure and reached consensus (defined as 80% agreement) on the phrasing of an assessment tool. FIFTY PANELISTS COMPLETED THE DELPHI PROCESS. THE MEDIAN AGE OF THE PANELISTS WAS 44 YEARS (RANGE 33-64 YEARS). MEDIAN EXPERIENCE IN OTORHINOLARYNGOLOGY WAS 15 YEARS (RANGE 6-35 YEARS). TWENTY-FIVE WERE SPECIALIZED IN LARYNGOLOGY, 16 WERE HEAD AND NECK SURGEONS, AND NINE WERE GENERAL OTORHINOLARYNGOLOGISTS. AN ASSESSMENT TOOL WAS CREATED CONSISTING OF TWELVE DISTINCT ITEMS.: Conclusion The gathering of validity evidence for assessment of core procedural skills within Otorhinolaryngology is central to the development of a competence-based education. The use of an international Delphi panel allows for the creation of an assessment tool which is widely applicable and valid. This work allows for an informed approach to technical skills training for flexible pharyngo-laryngoscopy and as further validity evidence is gathered allows for a valid assessment of clinical performance within this important skillset.

Risk assessment tools to identify women with increased risk of osteoporotic fracture. Complexity or simplicity?

DEFF Research Database (Denmark)

Rubin, Katrine Hass; Friis-Holmberg, Teresa; Hermann, Anne Pernille

2013-01-01

A huge number of risk assessment tools have been developed. Far from all have been validated in external studies, more of them have absence of methodological and transparent evidence and few are integrated in national guidelines. Therefore, we performed a systematic review to provide an overview...... of existing valid and reliable risk assessment tools for prediction of osteoporotic fractures. Additionally, we aimed to determine if the performance each tool was sufficient for practical use and lastly to examine whether the complexity of the tools influenced their discriminative power. We searched Pub......Med, Embase and Cochrane databases for papers and evaluated these with respect to methodological quality using the QUADAS checklist. A total of 48 tools were identified, 20 had been externally validated, however only 6 tools had been tested more than once in a population-based setting with acceptable...

Cross-cultural validation of the educational needs assessment tool into Chinese for use in severe knee osteoarthritis

Directory of Open Access Journals (Sweden)

Zhao H

2018-05-01

Full Text Available Huiwen Zhao,1,* Zhe Dong,2,* Fei Xie,3,4 Guanxin Wang,2 Zhihua Wen,5 Lixia Zhang,5 Mwidimi Ndosi,6,7 Wen Luo1 1Joint Department, The 2nd Ward of Joint Surgery, Tianjin Hospital, Tianjin, China; 2Medical Examination Center, No 6 Hospital, Beijing, China; 3Nursing Experimental Teaching Center, Tianjin Hospital, Tianjin, China; 4College of Nursing, Tianjin University of Traditional Chinese Medicine, Tianjin, China; 5Joint Department, The 1st Ward of Joint Surgery, Tianjin Hospital, Tianjin, China; 6Department of Nursing, University of the West of England, Bristol, UK; 7Academic Rheumatology Unit, Bristol Teaching Hospitals, Bristol, UK *These authors contributed equally to this work Background: Patient education is an integral part of the management of osteoarthritis. The educational needs assessment tool (ENAT was developed in the UK to help direct needs-based patient education in rheumatic diseases. Aim: The aim of the study was to adapt and validate the ENAT into Chinese, for use in severe knee osteoarthritis (KOA. Methods: This cross-cultural validation study took two phases: 1 adaptation of the ENAT into Chinese (CENAT and 2 validation of the CENAT. The Construct validity was determined using factor analysis and criterion-related validity by comparing data from CENAT with data from different self-efficacy scales: patient–physician interactions scale (PEPPI-10, self-efficacy for rehabilitation outcome scale (SER, and the self-efficacy for exercise scale (SEE. Results: The sample comprised 196 patients, with mean age 63.6±8.7 years, disease duration was 11.5 years, and 57.1% were female. The CENAT was found to have high internal consistency. The CENAT had weak correlations with the Chinese versions of PEPPI r=0.40, SER r=0.40, and SEE r=0.39. There were no correlations with age r=-0.03 or disease duration r=-0.11. Conclusion: The ENAT translated well into Chinese and has evidence of validity in KOA. Future studies will further inform its

Validation of the Mobile Information Software Evaluation Tool (MISET) With Nursing Students.

Science.gov (United States)

Secco, M Loretta; Furlong, Karen E; Doyle, Glynda; Bailey, Judy

2016-07-01

This study evaluated the Mobile Information Software Evaluation Tool (MISET) with a sample of Canadian undergraduate nursing students (N = 240). Psychometric analyses determined how well the MISET assessed the extent that nursing students find mobile device-based information resources useful and supportive of learning in the clinical and classroom settings. The MISET has a valid three-factor structure with high explained variance (74.7%). Internal consistency reliabilities were high for the MISET total (.90) and three subscales: Usefulness/Helpfulness, Information Literacy Support, and Use of Evidence-Based Sources (.87 to .94). Construct validity evidence included significantly higher mean total MISET, Helpfulness/Usefulness, and Information Literacy Support scores for senior students and those with higher computer competence. The MISET is a promising tool to evaluate mobile information technologies and information literacy support; however, longitudinal assessment of changes in scores over time would determine scale sensitivity and responsiveness. [J Nurs Educ. 2016;55(7):385-390.]. Copyright 2016, SLACK Incorporated.

Exploring the Validity of a Second Language Intercultural Pragmatics Assessment Tool

Science.gov (United States)

Timpe-Laughlin, Veronika; Choi, Ikkyu

2017-01-01

Pragmatics has been a key component of language competence frameworks. While the majority of second/foreign language (L2) pragmatics tests have targeted productive skills, the assessment of receptive pragmatic skills remains a developing field. This study explores validation evidence for a test of receptive L2 pragmatic ability called the American…

From Clinical-Developmental Theory to Assessment: The Holistic Student Assessment Tool

Directory of Open Access Journals (Sweden)

Gil Noam

2012-12-01

Full Text Available A description and test of the Holistic Student Assessment Tool (HSA, an assessment tool to measure children’s and adolescents’ resiliencies in relation to externalizing and internalizing problem behaviors. The HSA is based on the authors’ research-based clinical-developmental Clover Leaf Model of resilience and psychopathology, and is one of the first attempts at closing the gap between risk and resilience approaches in developmental assessment. The HSA was tested in a cross-sectional sample of 423 children and adolescents.The results lend support to the HSA as a valid measure of children’s and adolescents’ resiliencies. Furthermore, the resilience scales mostly exhibited the theoretically expected convergent and divergent relationships with the psychopathology scales. In addition, we show how the resilience scales predict adolescents’ externalizing and internalizing symptoms. We contend that evidence-based intervention to address youth aggression needs to be based on sounddevelopmental assessment.

Work-related stress risk assessment in Italy: the validation study of health safety and executive indicator tool.

Science.gov (United States)

Rondinone, Bruna Maria; Persechino, Benedetta; Castaldi, Tiziana; Valenti, Antonio; Ferrante, Pierpaolo; Ronchetti, Matteo; Iavicoli, Sergio

2012-01-01

In compliance with the Italian occupational health and safety regulatory framework, as provided by the Lgs. Decree 81/2008, the "work-related stress" risk assessment should follow the same principles as other risk assessments, in accordance with the European Agreement of 8 October 2004; therefore, validated and scientifically proven methodological tools are needed to conduct an adequate work-related stress risk assessment. The UK's Health Safety and Executive (HSE) Indicator Tool (IT) is used for assessing the risk of work-related stress. The aim of this study is to test the factor structure of IT as a measure of work-related stress in a sample of Italian workers. Data collected from 65 Italian organizations (6378 workers) was used for a Confirmatory Factor Analysis (CFA) on the 35-item seven-factor model. The results showed acceptable fit to the data (CFI .90; TLI .89, RMSEA .045). A second CFA was done to test a 35-item six-factor model (CFI .89, TLI .87, RMSEA .047). Both models were tested after removing six items (factor loadings less than .50.), resulting in a 29-item model. Here again, there was an acceptable fit to the data (29-item seven-factor model: CFI .93, TLI .91, RMSEA .044; 29-item six-factor model: CFI .92, TLI .90, RMSEA .046). These findings show that the HSE model satisfactorily adapts to use in a sample of Italian workers. One of the most important innovations introduced in the assessment of work-related stress with the HSE IT is the global approach for identifying work-related stress risk factors, aimed at establishing the best strategy from the viewpoints of prevention officers and also of workers.

Pathology-based validation of FDG PET segmentation tools for volume assessment of lymph node metastases from head and neck cancer

Energy Technology Data Exchange (ETDEWEB)

Schinagl, Dominic A.X. [Radboud University Nijmegen Medical Centre, Department of Radiation Oncology, Nijmegen (Netherlands); Radboud University Nijmegen Medical Centre, Department of Radiation Oncology (874), P.O. Box 9101, Nijmegen (Netherlands); Span, Paul N.; Kaanders, Johannes H.A.M. [Radboud University Nijmegen Medical Centre, Department of Radiation Oncology, Nijmegen (Netherlands); Hoogen, Frank J.A. van den [Radboud University Nijmegen Medical Centre, Department of Otorhinolaryngology, Head and Neck Surgery, Nijmegen (Netherlands); Merkx, Matthias A.W. [Radboud University Nijmegen Medical Centre, Department of Oral and Maxillofacial Surgery, Nijmegen (Netherlands); Slootweg, Piet J. [Radboud University Nijmegen Medical Centre, Department of Pathology, Nijmegen (Netherlands); Oyen, Wim J.G. [Radboud University Nijmegen Medical Centre, Department of Nuclear Medicine, Nijmegen (Netherlands)

2013-12-15

FDG PET is increasingly incorporated into radiation treatment planning of head and neck cancer. However, there are only limited data on the accuracy of radiotherapy target volume delineation by FDG PET. The purpose of this study was to validate FDG PET segmentation tools for volume assessment of lymph node metastases from head and neck cancer against the pathological method as the standard. Twelve patients with head and neck cancer and 28 metastatic lymph nodes eligible for therapeutic neck dissection underwent preoperative FDG PET/CT. The metastatic lymph nodes were delineated on CT (Node{sub CT}) and ten PET segmentation tools were used to assess FDG PET-based nodal volumes: interpreting FDG PET visually (PET{sub VIS}), applying an isocontour at a standardized uptake value (SUV) of 2.5 (PET{sub SUV}), two segmentation tools with a fixed threshold of 40 % and 50 %, and two adaptive threshold based methods. The latter four tools were applied with the primary tumour as reference and also with the lymph node itself as reference. Nodal volumes were compared with the true volume as determined by pathological examination. Both Node{sub CT} and PET{sub VIS} showed good correlations with the pathological volume. PET segmentation tools using the metastatic node as reference all performed well but not better than PET{sub VIS}. The tools using the primary tumour as reference correlated poorly with pathology. PET{sub SUV} was unsatisfactory in 35 % of the patients due to merging of the contours of adjacent nodes. FDG PET accurately estimates metastatic lymph node volume, but beyond the detection of lymph node metastases (staging), it has no added value over CT alone for the delineation of routine radiotherapy target volumes. If FDG PET is used in radiotherapy planning, treatment adaptation or response assessment, we recommend an automated segmentation method for purposes of reproducibility and interinstitutional comparison. (orig.)

Development and content validation of performance assessments for endoscopic third ventriculostomy

NARCIS (Netherlands)

Breimer, Gerben E.; Haji, Faizal A.; Hoving, Eelco W; Drake, James M.

This study aims to develop and establish the content validity of multiple expert rating instruments to assess performance in endoscopic third ventriculostomy (ETV), collectively called the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). The important aspects of ETV were identified through

Monitoring dietary intake and physical activity electronically: feasibility, usability, and ecological validity of a mobile-based Ecological Momentary Assessment tool.

Science.gov (United States)

Spook, Jorinde Eline; Paulussen, Theo; Kok, Gerjo; Van Empelen, Pepijn

2013-09-24

Despite the growing body of research on complex lifestyle behaviors (eg, Dietary Intake [DI] and Physical Activity [PA]), monitoring of these behaviors has been hampered by a lack of suitable methods. A possible solution to this deficiency is mobile-based Ecological Momentary Assessment (mEMA), which enables researchers to collect data on participants' states in real-time by means of a smartphone application. However, feasibility, usability, and ecological validity need to be anticipated and managed in order to enhance the validity of mEMA. To examine the feasibility, usability, and ecological validity of a mEMA application (app) with regard to DI and PA among Dutch vocational education students. The students (n=30) participated in the mEMA study for seven consecutive days. They downloaded the mEMA app on their smartphone. Feasibility and usability of the mEMA app were evaluated by completing an online evaluation after seven days of participation. Ecological validity was measured by assessing the degree to which the content of the mEMA app approximated the real-world setting that was being examined, through several multiple-choice questions. Compliance rates, as registered by the mEMA app, declined 46% over a seven-day period, while self-reported compliance, as measured with an online evaluation questionnaire afterwards, indicated a smaller decrease in compliance (29%). The students evaluated the mEMA app as feasible and usable. Ecological validity analyses showed that all DI and almost all PA multiple-choice options were covered with the compound response categories. The mEMA app offers the opportunity to assess complex health behaviors (eg, DI and PA) in real-time settings, in which specifically routinized behaviors are involved. However, the mEMA app faced several challenges that needed to be overcome in order to improve its validity. Overall, the present study showed that the mEMA app is a usable and ecologically valid tool to measure DI and PA behaviors among

Physics validation of detector simulation tools for LHC

International Nuclear Information System (INIS)

Beringer, J.

2004-01-01

Extensive studies aimed at validating the physics processes built into the detector simulation tools Geant4 and Fluka are in progress within all Large Hardon Collider (LHC) experiments, within the collaborations developing these tools, and within the LHC Computing Grid (LCG) Simulation Physics Validation Project, which has become the primary forum for these activities. This work includes detailed comparisons with test beam data, as well as benchmark studies of simple geometries and materials with single incident particles of various energies for which experimental data is available. We give an overview of these validation activities with emphasis on the latest results

Validity of the Consensual Assessment Technique--Evidence with Three Groups of Judges and an Elementary School Student Sample

Science.gov (United States)

Long, Haiying

2012-01-01

As one of the most widely used creativity assessment tools, the Consensual Assessment Technique (CAT) has been praised as a valid tool to assess creativity. In Amabile's (1982) seminal work, the inter-rater reliability was defined as construct validity of the CAT. During the past three decades, researchers followed this definition and…

System of tactical assessment in Soccer (FUT-SAT: Development and preliminary validation

Directory of Open Access Journals (Sweden)

I.T. Costa

2011-01-01

Full Text Available The purpose of this paper was to report the development and preliminary validation of tactical assessment system in Soccer and highlight its advantages. The validation process followed five perspectives of the concept of validity that consider the value of heuristic methods and the importance of the description of behavior performed in playing situations. Thus, the process of validation was focused on four points: i acceptability and reasonableness of the test perceived by players; ii analysis of content of assessment tool through a panel of experts; iii potential of the assessment tool to discriminate the quality of the performance of players; iv observation reliability. The results displayed values higher than 0.63 for correlation between the evaluations of coaches and the system. It shows the potential of this system to distinguish the performances of players based on the evaluations of coaches. The players who performed the field test agreed with its physical demands and spatial and normative configurations. All experts endorsed the categories and variables of this system. The reliabilities showed values higher than 0.79 for intra and inter-observers. Therefore, it is possible to conclude that the system is valid and reliable for the assessment of the tactical behavior of soccer players.

The IDEA Assessment Tool: Assessing the Reporting, Diagnostic Reasoning, and Decision-Making Skills Demonstrated in Medical Students' Hospital Admission Notes.

Science.gov (United States)

Baker, Elizabeth A; Ledford, Cynthia H; Fogg, Louis; Way, David P; Park, Yoon Soo

2015-01-01

Construct: Clinical skills are used in the care of patients, including reporting, diagnostic reasoning, and decision-making skills. Written comprehensive new patient admission notes (H&Ps) are a ubiquitous part of student education but are underutilized in the assessment of clinical skills. The interpretive summary, differential diagnosis, explanation of reasoning, and alternatives (IDEA) assessment tool was developed to assess students' clinical skills using written comprehensive new patient admission notes. The validity evidence for assessment of clinical skills using clinical documentation following authentic patient encounters has not been well documented. Diagnostic justification tools and postencounter notes are described in the literature (1,2) but are based on standardized patient encounters. To our knowledge, the IDEA assessment tool is the first published tool that uses medical students' H&Ps to rate students' clinical skills. The IDEA assessment tool is a 15-item instrument that asks evaluators to rate students' reporting, diagnostic reasoning, and decision-making skills based on medical students' new patient admission notes. This study presents validity evidence in support of the IDEA assessment tool using Messick's unified framework, including content (theoretical framework), response process (interrater reliability), internal structure (factor analysis and internal-consistency reliability), and relationship to other variables. Validity evidence is based on results from four studies conducted between 2010 and 2013. First, the factor analysis (2010, n = 216) yielded a three-factor solution, measuring patient story, IDEA, and completeness, with reliabilities of .79, .88, and .79, respectively. Second, an initial interrater reliability study (2010) involving two raters demonstrated fair to moderate consensus (κ = .21-.56, ρ =.42-.79). Third, a second interrater reliability study (2011) with 22 trained raters also demonstrated fair to moderate agreement

Validation of a method for assessing resident physicians' quality improvement proposals.

Science.gov (United States)

Leenstra, James L; Beckman, Thomas J; Reed, Darcy A; Mundell, William C; Thomas, Kris G; Krajicek, Bryan J; Cha, Stephen S; Kolars, Joseph C; McDonald, Furman S

2007-09-01

Residency programs involve trainees in quality improvement (QI) projects to evaluate competency in systems-based practice and practice-based learning and improvement. Valid approaches to assess QI proposals are lacking. We developed an instrument for assessing resident QI proposals--the Quality Improvement Proposal Assessment Tool (QIPAT-7)-and determined its validity and reliability. QIPAT-7 content was initially obtained from a national panel of QI experts. Through an iterative process, the instrument was refined, pilot-tested, and revised. Seven raters used the instrument to assess 45 resident QI proposals. Principal factor analysis was used to explore the dimensionality of instrument scores. Cronbach's alpha and intraclass correlations were calculated to determine internal consistency and interrater reliability, respectively. QIPAT-7 items comprised a single factor (eigenvalue = 3.4) suggesting a single assessment dimension. Interrater reliability for each item (range 0.79 to 0.93) and internal consistency reliability among the items (Cronbach's alpha = 0.87) were high. This method for assessing resident physician QI proposals is supported by content and internal structure validity evidence. QIPAT-7 is a useful tool for assessing resident QI proposals. Future research should determine the reliability of QIPAT-7 scores in other residency and fellowship training programs. Correlations should also be made between assessment scores and criteria for QI proposal success such as implementation of QI proposals, resident scholarly productivity, and improved patient outcomes.

The SAFE-T assessment tool: derivation and validation of a web-based application for point-of-care evaluation of gastroenterology fellow performance in colonoscopy.

Science.gov (United States)

Kumar, Navin L; Kugener, Guillaume; Perencevich, Molly L; Saltzman, John R

2018-01-01

Attending assessment is a critical part of endoscopic education for gastroenterology fellows. The aim of this study was to develop and validate a concise assessment tool to evaluate real-time fellow performance in colonoscopy administered via a web-based application. The Skill Assessment in Fellow Endoscopy Training (SAFE-T) tool was derived as a novel 5-question evaluation tool that captures both summative and formative feedback adapted into a web-based application. A prospective study of 15 gastroenterology fellows (5 fellows each from years 1 to 3 of training) was performed using the SAFE-T tool. An independent reviewer evaluated a subset of these procedures and completed the SAFE-T tool and Mayo Colonoscopy Skills Assessment Tool (MCSAT) for reliability testing. Twenty-six faculty completed 350 SAFE-T evaluations of the 15 fellows in the study. The mean SAFE-T overall score (year 1, 2.00; year 2, 3.84; year 3, 4.28) differentiated each sequential fellow year of training (P case complexity score, with straightforward cases compared with average cases (4.07 vs 3.50, P cases compared with challenging cases (3.50 vs 3.08, P = .0134). In dual-observed procedures, the SAFE-T tool showed excellent inter-rater reliability with a kappa agreement statistic of 0.898 (P  0.90, P tool, a concise and web-based means of assessing real-time gastroenterology fellow performance in colonoscopy. Copyright © 2018 American Society for Gastrointestinal Endoscopy. Published by Elsevier Inc. All rights reserved.

MOVEMENT SKILL ASSESSMENT OF TYPICALLY DEVELOPING PRESCHOOL CHILDREN: A REVIEW OF SEVEN MOVEMENT SKILL ASSESSMENT TOOLS

Directory of Open Access Journals (Sweden)

Wouter Cools

2009-06-01

Full Text Available The importance of movement is often overlooked because it is such a natural part of human life. It is, however, crucial for a child's physical, cognitive and social development. In addition, experiences support learning and development of fundamental movement skills. The foundations of those skills are laid in early childhood and essential to encourage a physically active lifestyle. Fundamental movement skill performance can be examined with several assessment tools. The choice of a test will depend on the context in which the assessment is planned. This article compares seven assessment tools which are often referred to in European or international context. It discusses the tools' usefulness for the assessment of movement skill development in general population samples. After a brief description of each assessment tool the article focuses on contents, reliability, validity and normative data. A conclusion outline of strengths and weaknesses of all reviewed assessment tools focusing on their use in educational research settings is provided and stresses the importance of regular data collection of fundamental movement skill development among preschool children.

STOPP (Screening Tool of Older Person's Prescriptions) and START (Screening Tool to Alert doctors to Right Treatment). Consensus validation.

LENUS (Irish Health Repository)

Gallagher, P

2012-02-03

OBJECTIVE: Older people experience more concurrent illnesses, are prescribed more medications and suffer more adverse drug events than younger people. Many drugs predispose older people to adverse events such as falls and cognitive impairment, thus increasing morbidity and health resource utilization. At the same time, older people are often denied potentially beneficial, clinically indicated medications without a valid reason. We aimed to validate a new screening tool of older persons\\' prescriptions incorporating criteria for potentially inappropriate drugs called STOPP (Screening Tool of Older Persons\\' Prescriptions) and criteria for potentially appropriate, indicated drugs called START (Screening Tool to Alert doctors to Right, i.e. appropriate, indicated Treatment). METHODS: A Delphi consensus technique was used to establish the content validity of STOPP\\/START. An 18-member expert panel from academic centers in Ireland and the United Kingdom completed two rounds of the Delphi process by mail survey. Inter-rater reliability was assessed by determining the kappa-statistic for measure of agreement on 100 data-sets. RESULTS: STOPP is comprised of 65 clinically significant criteria for potentially inappropriate prescribing in older people. Each criterion is accompanied by a concise explanation as to why the prescribing practice is potentially inappropriate. START consists of 22 evidence-based prescribing indicators for commonly encountered diseases in older people. Inter-rater reliability is favorable with a kappa-coefficient of 0.75 for STOPP and 0.68 for START. CONCLUSION: STOPP\\/START is a valid, reliable and comprehensive screening tool that enables the prescribing physician to appraise an older patient\\'s prescription drugs in the context of his\\/her concurrent diagnoses.

A tool for assessing continuity of care across care levels: an extended psychometric validation of the CCAENA questionnaire

Directory of Open Access Journals (Sweden)

Marta Beatriz Aller

2013-12-01

Full Text Available Background: The CCAENA questionnaire was developed to assess care continuity across levels from the patients’ perspective. The aim is to provide additional evidence on the psychometric properties of the scales of this questionnaire. Methods: Cross-sectional study by means of a survey of a random sample of 1500 patients attended in primary and secondary care in three healthcare areas of the Catalan healthcare system. Data were collected in 2010 using the CCAENA questionnaire. To assess psychometric properties, an exploratory factor analysis was performed (construct validity and the item-rest correlations and Cronbach’s alpha were calculated (internal consistency. Spearman correlation coefficients were calculated (multidimensionality and the ability to discriminate between groups was tested. Results: The factor analysis resulted in 21 items grouped into three factors: patient-primary care provider relationship, patient-secondary care provider relationship and continuity across care levels. Cronbach’s alpha indicated good internal consistency (0.97, 0.93, 0.80 and the correlation coefficients indicated that dimensions can be interpreted as separated scales. Scales discriminated patients according to healthcare area, age and educational level. Conclusion: The CCAENA questionnaire has proved to be a valid and reliable tool for measuring patients’ perceptions of continuity. Providers and researchers could apply the questionnaire to identify areas for healthcare improvement.

Above, Beyond, and Over the Side rails: Evaluating the New Memorial Emergency Department Fall-Risk-Assessment Tool.

Science.gov (United States)

Scott, Robin A; Oman, Kathleen S; Flarity, Kathleen; Comer, Jennifer L

2018-03-06

Patient falls are a significant issue in hospitalized patients and financially costly to hospitals. The Joint Commission requires that patients be assessed for fall risk and interventions in place to mitigate the risk of falls. It is imperative to have a patient population/setting specific fall risk assessment tool to identify patients at risk for falling. The purpose of this study was to evaluate the reliability and validity of the 2013 Memorial ED Fall Risk Assessment tool (MEDFRAT) specifically designed for the ED population. A two-phase prospective design was used for this study. Phase one determined the interrater reliability of the MEDFRAT. Phase two assessed the validity of the MEDFRAT in an emergency department (ED) within a 600-bed academic/teaching institution; Level II Trauma Center with >100,000 annual patient visits. The Memorial ED Fall Risk Assessment Tool was validated in this ED setting. The tool demonstrated positive interrater reliability (k=0.701) and when implemented with a falls prevention strategy and staff education demonstrated a 48% decrease in ED fall rate (0.57 falls/1000 patient visits) post implementation during the study period. The MEDFRAT, an evidenced based ED-specific fall risk tool was implemented on the basis of the risk factors consistently identified in the literature: prior fall history, impaired mobility, altered mental status, altered elimination, and the use of sedative medication. The Memorial ED Fall Risk Assessment Tool demonstrated to be a valid tool for this hospital system. Copyright © 2018 Emergency Nurses Association. Published by Elsevier Inc. All rights reserved.

Health system context and implementation of evidence-based practices-development and validation of the Context Assessment for Community Health (COACH) tool for low- and middle-income settings.

Science.gov (United States)

Bergström, Anna; Skeen, Sarah; Duc, Duong M; Blandon, Elmer Zelaya; Estabrooks, Carole; Gustavsson, Petter; Hoa, Dinh Thi Phuong; Källestål, Carina; Målqvist, Mats; Nga, Nguyen Thu; Persson, Lars-Åke; Pervin, Jesmin; Peterson, Stefan; Rahman, Anisur; Selling, Katarina; Squires, Janet E; Tomlinson, Mark; Waiswa, Peter; Wallin, Lars

2015-08-15

The gap between what is known and what is practiced results in health service users not benefitting from advances in healthcare, and in unnecessary costs. A supportive context is considered a key element for successful implementation of evidence-based practices (EBP). There were no tools available for the systematic mapping of aspects of organizational context influencing the implementation of EBPs in low- and middle-income countries (LMICs). Thus, this project aimed to develop and psychometrically validate a tool for this purpose. The development of the Context Assessment for Community Health (COACH) tool was premised on the context dimension in the Promoting Action on Research Implementation in Health Services framework, and is a derivative product of the Alberta Context Tool. Its development was undertaken in Bangladesh, Vietnam, Uganda, South Africa and Nicaragua in six phases: (1) defining dimensions and draft tool development, (2) content validity amongst in-country expert panels, (3) content validity amongst international experts, (4) response process validity, (5) translation and (6) evaluation of psychometric properties amongst 690 health workers in the five countries. The tool was validated for use amongst physicians, nurse/midwives and community health workers. The six phases of development resulted in a good fit between the theoretical dimensions of the COACH tool and its psychometric properties. The tool has 49 items measuring eight aspects of context: Resources, Community engagement, Commitment to work, Informal payment, Leadership, Work culture, Monitoring services for action and Sources of knowledge. Aspects of organizational context that were identified as influencing the implementation of EBPs in high-income settings were also found to be relevant in LMICs. However, there were additional aspects of context of relevance in LMICs specifically Resources, Community engagement, Commitment to work and Informal payment. Use of the COACH tool will allow

Validating the JobFit system functional assessment method

Energy Technology Data Exchange (ETDEWEB)

Jenny Legge; Robin Burgess-Limerick

2007-05-15

Workplace injuries are costing the Australian coal mining industry and its communities $410 Million a year. This ACARP study aims to meet those demands by developing a safe, reliable and valid pre-employment functional assessment tool. All JobFit System Pre-Employment Functional Assessments (PEFAs) consist of a musculoskeletal screen, balance test, aerobic fitness test and job-specific postural tolerances and material handling tasks. The results of each component are compared to the applicant's job demands and an overall PEFA score between 1 and 4 is given with 1 being the better score. The reliability study and validity study were conducted concurrently. The reliability study examined test-retest, intra-tester and inter-tester reliability of the JobFit System Functional Assessment Method. Overall, good to excellent reliability was found, which was sufficient to be used for comparison with injury data for determining the validity of the assessment. The overall assessment score and material handling tasks had the greatest reliability. The validity study compared the assessment results of 336 records from a Queensland underground and open cut coal mine with their injury records. A predictive relationship was found between PEFA score and the risk of a back/trunk/shoulder injury from manual handling. An association was also found between PEFA score of 1 and increased length of employment. Lower aerobic fitness test results had an inverse relationship with injury rates. The study found that underground workers, regardless of PEFA score, were more likely to have an injury when compared to other departments. No relationship was found between age and risk of injury. These results confirm the validity of the JobFit System Functional Assessment method.

Validation of a novel duplex ultrasound objective structured assessment of technical skills (DUOSATS) for arterial stenosis detection.

Science.gov (United States)

Jaffer, U; Singh, P; Pandey, V A; Aslam, M; Standfield, N J

2014-01-01

Duplex ultrasound facilitates bedside diagnosis and hence timely patient care. Its uptake has been hampered by training and accreditation issues. We have developed an assessment tool for Duplex arterial stenosis measurement for both simulator and patient based training. A novel assessment tool: duplex ultrasound assessment of technical skills was developed. A modified duplex ultrasound assessment of technical skills was used for simulator training. Novice, intermediate experience and expert users of duplex ultrasound were invited to participate. Participants viewed an instructional video and were allowed ample time to familiarize with the equipment. Participants' attempts were recorded and independently assessed by four experts using the modified duplex ultrasound assessment of technical skills. 'Global' assessment was also done on a four point Likert scale. Content, construct and concurrent validity as well as reliability were evaluated. Content and construct validity as well as reliability were demonstrated. The simulator had good satisfaction rating from participants: median 4; range 3-5. Receiver operator characteristic analysis has established a cut point of 22/ 34 and 25/ 40 were most appropriate for simulator and patient based assessment respectively. We have validated a novel assessment tool for duplex arterial stenosis detection. Further work is underway to establish transference validity of simulator training to improved skill in scanning patients. We have developed and validated duplex ultrasound assessment of technical skills for simulator training.

Longitudinal Evaluation of Johns Hopkins Fall Risk Assessment Tool and Nurses' Experience.

Science.gov (United States)

Hur, Eun Young; Jin, Yinji; Jin, Taixian; Lee, Sun-Mi

The Johns Hopkins Fall Risk Assessment Tool (JHFRAT) is relatively new in Korea, and it has not been fully evaluated. This study revealed that the JHFRAT had good predictive validity throughout the hospitalization period. However, 2 items (fall history and elimination patterns) on the tool were not determinants of falls in this population. Interestingly, the nurses indicated those 2 items were the most difficult items to assess and needed further training to develop the assessment skills.

Development and validation of the FRAGIRE tool for assessment an older person’s risk for frailty

Directory of Open Access Journals (Sweden)

Dewi Vernerey

2016-11-01

Full Text Available Abstract Background Frailty is highly prevalent in elderly people. While significant progress has been made to understand its pathogenesis process, few validated questionnaire exist to assess the multidimensional concept of frailty and to detect people frail or at risk to become frail. The objectives of this study were to construct and validate a new frailty-screening instrument named Frailty Groupe Iso-Ressource Evaluation (FRAGIRE that accurately predicts the risk for frailty in older adults. Methods A prospective multicenter recruitment of the elderly patients was undertaken in France. The subjects were classified into financially-helped group (FH, with financial assistance and non-financially helped group (NFH, without any financial assistance, considering FH subjects are more frail than the NFH group and thus representing an acceptable surrogate population for frailty. Psychometric properties of the FRAGIRE grid were assessed including discrimination between the FH and NFH groups. Items reduction was made according to statistical analyses and experts’ point of view. The association between items response and tests with “help requested status” was assessed in univariate and multivariate unconditional logistic regression analyses and a prognostic score to become frail was finally proposed for each subject. Results Between May 2013 and July 2013, 385 subjects were included: 338 (88% in the FH group and 47 (12% in the NFH group. The initial FRAGIRE grid included 65 items. After conducting the item selection, the final grid of the FRAGIRE was reduced to 19 items. The final grid showed fair discrimination ability to predict frailty (area under the curve (AUC = 0.85 and good calibration (Hosmer-Lemeshow P-value = 0.580, reflecting a good agreement between the prediction by the final model and actual observation. The Cronbach's alpha for the developed tool scored as high as 0.69 (95% Confidence Interval: 0.64 to 0.74. The final

Multi-Trait Multi-Method Matrices for the Validation of Creativity and Critical Thinking Assessments for Secondary School Students in England and Greece

Directory of Open Access Journals (Sweden)

Ourania Maria Ventista

2017-08-01

Full Text Available The aim of this paper is the validation of measurement tools which assess critical thinking and creativity as general constructs instead of subject-specific skills. Specifically, this research examined whether there is convergent and discriminant (or divergent validity between measurement tools of creativity and critical thinking. For this purpose, the multi-trait and multi-method matrix suggested by Campbell and Fiske (1959 was used. This matrix presented the correlation of scores that students obtain in different assessments in order to reveal whether the assessments measure the same or different constructs. Specifically, the two methods used were written and oral exams, and the two traits measured were critical thinking and creativity. For the validation of the assessments, 30 secondary-school students in Greece and 21 in England completed the assessments. The sample in both countries provided similar results. The critical thinking tools demonstrated convergent validity when compared with each other and discriminant validity with the creativity assessments. Furthermore, creativity assessments which measure the same aspect of creativity demonstrated convergent validity. To conclude, this research provided indicators that critical thinking and creativity as general constructs can be measured in a valid way. However, since the sample was small, further investigation of the validation of the assessment tools with a bigger sample is recommended.

DarcyTools, Version 2.1. Verification and validation

International Nuclear Information System (INIS)

Svensson, Urban

2004-03-01

DarcyTools is a computer code for simulation of flow and transport in porous and/or fractured media. The fractured media in mind is a fractured rock and the porous media the soil cover on the top of the rock; it is hence groundwater flows, which is the class of flows in mind. A number of novel methods and features form the present version of DarcyTools. In the verification studies, these methods are evaluated by comparisons with analytical solutions for idealized situations. The five verification groups, thus reflect the main areas of recent developments. The present report will focus on the Verification and Validation of DarcyTools. Two accompanying reports cover other aspects: - Concepts, Methods, Equations and Demo Simulations. - User's Guide. The objective of this report is to compile all verification and validation studies that have been carried out so far. After some brief introductory sections, all cases will be reported in Appendix A (verification cases) and Appendix B (validation cases)

DarcyTools, Version 2.1. Verification and validation

Energy Technology Data Exchange (ETDEWEB)

Svensson, Urban [Computer-aided Fluid Engineering AB, Norrkoeping (Sweden)

2004-03-01

DarcyTools is a computer code for simulation of flow and transport in porous and/or fractured media. The fractured media in mind is a fractured rock and the porous media the soil cover on the top of the rock; it is hence groundwater flows, which is the class of flows in mind. A number of novel methods and features form the present version of DarcyTools. In the verification studies, these methods are evaluated by comparisons with analytical solutions for idealized situations. The five verification groups, thus reflect the main areas of recent developments. The present report will focus on the Verification and Validation of DarcyTools. Two accompanying reports cover other aspects: - Concepts, Methods, Equations and Demo Simulations. - User's Guide. The objective of this report is to compile all verification and validation studies that have been carried out so far. After some brief introductory sections, all cases will be reported in Appendix A (verification cases) and Appendix B (validation cases)

Dutch Risk Assessment tools

NARCIS (Netherlands)

Venema, A.

2015-01-01

The ‘Risico- Inventarisatie- en Evaluatie-instrumenten’ is the name for the Dutch risk assessment (RA) tools. A RA tool can be used to perform a risk assessment including an evaluation of the identified risks. These tools were among the first online risk assessment tools developed in Europe. The

Validity of the Special Needs Education Assessment Tool (SNEAT), a Newly Developed Scale for Children with Disabilities.

Science.gov (United States)

Kohara, Aiko; Han, ChangWan; Kwon, HaeJin; Kohzuki, Masahiro

2015-11-01

The improvement of the quality of life (QOL) of children with disabilities has been considered important. Therefore, the Special Needs Education Assessment Tool (SNEAT) was developed based on the concept of QOL to objectively evaluate the educational outcome of children with disabilities. SNEAT consists of 11 items in three domains: physical functioning, mental health, and social functioning. This study aimed to verify the reliability and construct validity of SNEAT using 93 children collected from the classes on independent activities of daily living for children with disabilities in Okinawa Prefecture between October and November 2014. Survey data were collected in a longitudinal prospective cohort study. The reliability of SNEAT was verified via the internal consistency method and the test-pretest method; both the coefficient of Cronbach's α and the intra-class correlation coefficient were over 0.7. The validity of SNEAT was also verified via one-way repeated-measures ANOVA and the latent growth curve model. The scores of all the items and domains and the total scores obtained from one-way repeated-measures ANOVA were the same as the predicted scores. SNEAT is valid based on its goodness-of-fit values obtained using the latent growth curve model, where the values of comparative fit index (0.983) and root mean square error of approximation (0.062) were within the goodness-of-fit range. These results indicate that SNEAT has high reliability and construct validity and may contribute to improve QOL of children with disabilities in the classes on independent activities of daily living for children with disabilities.

Validity of the growth model of the 'computerized visual perception assessment tool for Chinese characters structures'.

Science.gov (United States)

Wu, Huey-Min; Li, Cheng-Hsaun; Kuo, Bor-Chen; Yang, Yu-Mao; Lin, Chin-Kai; Wan, Wei-Hsiang

2017-08-01

Morphological awareness is the foundation for the important developmental skills involved with vocabulary, as well as understanding the meaning of words, orthographic knowledge, reading, and writing. Visual perception of space and radicals in two-dimensional positions of Chinese characters' morphology is very important in identifying Chinese characters. The important predictive variables of special and visual perception in Chinese characters identification were investigated in the growth model in this research. The assessment tool is the "Computerized Visual Perception Assessment Tool for Chinese Characters Structures" developed by this study. There are two constructs, basic stroke and character structure. In the basic stroke, there are three subtests of one, two, and more than three strokes. In the character structure, there are three subtests of single-component character, horizontal-compound character, and vertical-compound character. This study used purposive sampling. In the first year, 551 children 4-6 years old participated in the study and were monitored for one year. In the second year, 388 children remained in the study and the successful follow-up rate was 70.4%. This study used a two-wave cross-lagged panel design to validate the growth model of the basic stroke and the character structure. There was significant correlation of the basic stroke and the character structure at different time points. The abilities in the basic stroke and in the character structure steadily developed over time for preschool children. Children's knowledge of the basic stroke effectively predicted their knowledge of the basic stroke and the character structure. Copyright © 2017 Elsevier Ltd. All rights reserved.

Methodology and tools for independent verification and validation of computerized I and C systems important to safety

International Nuclear Information System (INIS)

Lindner, A.; Miedl, H.

1998-01-01

Modular software based I and C systems are state-of-the-art in industrial automation. For I and C systems important to safety in nuclear power plants, software based systems are also more and more applied. According to existing national and international guidelines and standards, the assessment of these systems calls for appropriate test methods and tools. By use of tools quality of the assessment process should be improved and expense should be limited. The paper outlines the structure of the independent verification and validation (V and V) process of the Teleperm XS system and the lessons learnt from this process. Furthermore, tools are discussed used for V and V of the Teleperm XS software. The recently developed tool VALIDATOR, dedicated to V and V of the plant specific I and C functions is described in more detail. We consider V and V of the basic software components and the system software to be required only once, but the C source codes of the plant specific functional diagrams have to be checked for each application separately. The VALIDATOR is designed to perform this task. It gives evidence of compliance of the automatically generated C source codes with the graphical design of the functional diagrams in reasonable time and with acceptable costs. The working method, performance and results of the VALIDATOR are shown by means of an actual example. (author)

The East London glaucoma prediction score: web-based validation of glaucoma risk screening tool

Science.gov (United States)

Stephen, Cook; Benjamin, Longo-Mbenza

2013-01-01

AIM It is difficult for Optometrists and General Practitioners to know which patients are at risk. The East London glaucoma prediction score (ELGPS) is a web based risk calculator that has been developed to determine Glaucoma risk at the time of screening. Multiple risk factors that are available in a low tech environment are assessed to provide a risk assessment. This is extremely useful in settings where access to specialist care is difficult. Use of the calculator is educational. It is a free web based service. Data capture is user specific. METHOD The scoring system is a web based questionnaire that captures and subsequently calculates the relative risk for the presence of Glaucoma at the time of screening. Three categories of patient are described: Unlikely to have Glaucoma; Glaucoma Suspect and Glaucoma. A case review methodology of patients with known diagnosis is employed to validate the calculator risk assessment. RESULTS Data from the patient records of 400 patients with an established diagnosis has been captured and used to validate the screening tool. The website reports that the calculated diagnosis correlates with the actual diagnosis 82% of the time. Biostatistics analysis showed: Sensitivity = 88%; Positive predictive value = 97%; Specificity = 75%. CONCLUSION Analysis of the first 400 patients validates the web based screening tool as being a good method of screening for the at risk population. The validation is ongoing. The web based format will allow a more widespread recruitment for different geographic, population and personnel variables. PMID:23550097

Judging the Neonatal Abstinence Syndrome Assessment Tools to Guide Future Tool Development: The use of Clinimetrics as Opposed to Psychometrics.

Science.gov (United States)

Westgate, Philip M; Gomez-Pomar, Enrique

2017-01-01

In the face of the current Neonatal Abstinence Syndrome (NAS) epidemic, there is considerable variability in the assessment and management of infants with NAS. In this manuscript, we particularly focus on NAS assessment, with special attention given to the popular Finnegan Neonatal Abstinence Score (FNAS). A major instigator of the problem of variable practices is that multiple modified versions of the FNAS exist and continue to be proposed, including shortened versions. Furthermore, the validity of such assessment tools has been questioned, and as a result, the need for better tools has been suggested. The ultimate purpose of this manuscript, therefore, is to increase researchers' and clinicians' understanding on how to judge the usefulness of NAS assessment tools in order to guide future tool development and to reduce variable practices. In short, we suggest that judgment of NAS assessment tools should be made on a clinimetrics viewpoint as opposed to psychometrically. We provide examples, address multiple issues that must be considered, and discuss future tool development. Furthermore, we urge researchers and clinicians to come together, utilizing their knowledge and experience, to assess the utility and practicality of existing assessment tools and to determine if one or more new or modified tools are needed with the goal of increased agreement on the assessment of NAS in practice.

Validation and inter-rater reliability of a three item falls risk screening tool

Directory of Open Access Journals (Sweden)

Catherine Maree Said

2017-11-01

Full Text Available Abstract Background Falls screening tools are routinely used in hospital settings and the psychometric properties of tools should be examined in the setting in which they are used. The aim of this study was to explore the concurrent and predictive validity of the Austin Health Falls Risk Screening Tool (AHFRST, compared with The Northern Hospital Modified St Thomas’s Risk Assessment Tool (TNH-STRATIFY, and the inter-rater reliability of the AHFRST. Methods A research physiotherapist used the AHFRST and TNH-STRATIFY to classify 130 participants admitted to Austin Health (five acute wards, n = 115 two subacute wards n = 15; median length of stay 6 days IQR 3–12 as ‘High’ or ‘Low’ falls risk. The AHFRST was also completed by nursing staff on patient admission. Falls data was collected from the hospital incident reporting system. Results Six falls occurred during the study period (fall rate of 4.6 falls per 1000 bed days. There was substantial agreement between the AHFRST and the TNH-STRATIFY (Kappa = 0.68, 95% CI 0.52–0.78. Both tools had poor predictive validity, with low specificity (AHFRST 46.0%, 95% CI 37.0–55.1; TNH-STRATIFY 34.7%, 95% CI 26.4–43.7 and positive predictive values (AHFRST 5.6%, 95% CI 1.6–13.8; TNH-STRATIFY 6.9%, 95% CI 2.6–14.4. The AHFRST showed moderate inter-rater reliability (Kappa = 0.54, 95% CI = 0.36–0.67, p < 0.001 although 18 patients did not have the AHFRST completed by nursing staff. Conclusions There was an acceptable level of agreement between the 3 item AHFRST classification of falls risk and the longer, 9 item TNH-STRATIFY classification. However, both tools demonstrated limited predictive validity in the Austin Health population. The results highlight the importance of evaluating the validity of falls screening tools, and the clinical utility of these tools should be reconsidered.

Development and validation of a tool to assess knowledge and attitudes towards generic medicines among students in Greece: The ATtitude TOwards GENerics (ATTOGEN questionnaire.

Directory of Open Access Journals (Sweden)

Philip J Domeyer

Full Text Available The use of generic medicines is a cost-effective policy, often dictated by fiscal restraints. To our knowledge, no fully validated tool exploring the students' knowledge and attitudes towards generic medicines exists. The aim of our study was to develop and validate a questionnaire exploring the knowledge and attitudes of M.Sc. in Health Care Management students and recent alumni's towards generic drugs in Greece.The development of the questionnaire was a result of literature review and pilot-testing of its preliminary versions to researchers and students. The final version of the questionnaire contains 18 items measuring the respondents' knowledge and attitude towards generic medicines on a 5-point Likert scale. Given the ordinal nature of the data, ordinal alpha and polychoric correlations were computed. The sample was randomly split into two halves. Exploratory factor analysis, performed in the first sample, was used for the creation of multi-item scales. Confirmatory factor analysis and Generalized Linear Latent and Mixed Model analysis (GLLAMM with the use of the rating scale model were used in the second sample to assess goodness of fit. An assessment of internal consistency reliability, test-retest reliability, and construct validity was also performed.Among 1402 persons contacted, 986 persons completed our questionnaire (response rate = 70.3%. Overall Cronbach's alpha was 0.871. The conjoint use of exploratory and confirmatory factor analysis resulted in a six-scale model, which seemed to fit the data well. Five of the six scales, namely trust, drug quality, state audit, fiscal impact and drug substitution were found to be valid and reliable, while the knowledge scale suffered only from low inter-scale correlations and a ceiling effect. However, the subsequent confirmatory factor and GLLAMM analyses indicated a good fit of the model to the data.The ATTOGEN instrument proved to be a reliable and valid tool, suitable for assessing students

Validating a tool to measure auxiliary nurse midwife and nurse motivation in rural Nepal.

Science.gov (United States)

Morrison, Joanna; Batura, Neha; Thapa, Rita; Basnyat, Regina; Skordis-Worrall, Jolene

2015-05-12

A global shortage of health workers in rural areas increases the salience of motivating and supporting existing health workers. Understandings of motivation may vary in different settings, and it is important to use measurement methods that are contextually appropriate. We identified a measurement tool, previously used in Kenya, and explored its validity and reliability to measure the motivation of auxiliary nurse midwives (ANM) and staff nurses (SN) in rural Nepal. Qualitative and quantitative methods were used to assess the content validity, the construct validity, the internal consistency and the reliability of the tool. We translated the tool into Nepali and it was administered to 137 ANMs and SNs in three districts. We collected qualitative data from 78 nursing personnel and district- and central-level stakeholders using interviews and focus group discussions. We calculated motivation scores for ANMs and SNs using the quantitative data and conducted statistical tests for validity and reliability. Motivation scores were compared with qualitative data. Descriptive exploratory analysis compared mean motivation scores by ANM and SN sociodemographic characteristics. The concept of self-efficacy was added to the tool before data collection. Motivation was revealed through conscientiousness. Teamwork and the exertion of extra effort were not adequately captured by the tool, but important in illustrating motivation. The statement on punctuality was problematic in quantitative analysis, and attendance was more expressive of motivation. The calculated motivation scores usually reflected ANM and SN interview data, with some variation in other stakeholder responses. The tool scored within acceptable limits in validity and reliability testing and was able to distinguish motivation of nursing personnel with different sociodemographic characteristics. We found that with minor modifications, the tool provided valid and internally consistent measures of motivation among ANMs

Ethical decision-making climate in the ICU: theoretical framework and validation of a self-assessment tool.

Science.gov (United States)

Van den Bulcke, Bo; Piers, Ruth; Jensen, Hanne Irene; Malmgren, Johan; Metaxa, Victoria; Reyners, Anna K; Darmon, Michael; Rusinova, Katerina; Talmor, Daniel; Meert, Anne-Pascale; Cancelliere, Laura; Zubek, Làszló; Maia, Paolo; Michalsen, Andrej; Decruyenaere, Johan; Kompanje, Erwin J O; Azoulay, Elie; Meganck, Reitske; Van de Sompel, Ariëlla; Vansteelandt, Stijn; Vlerick, Peter; Vanheule, Stijn; Benoit, Dominique D

2018-02-23

Literature depicts differences in ethical decision-making (EDM) between countries and intensive care units (ICU). To better conceptualise EDM climate in the ICU and to validate a tool to assess EDM climates. Using a modified Delphi method, we built a theoretical framework and a self-assessment instrument consisting of 35 statements. This Ethical Decision-Making Climate Questionnaire (EDMCQ) was developed to capture three EDM domains in healthcare: interdisciplinary collaboration and communication; leadership by physicians; and ethical environment. This instrument was subsequently validated among clinicians working in 68 adult ICUs in 13 European countries and the USA. Exploratory and confirmatory factor analysis was used to determine the structure of the EDM climate as perceived by clinicians. Measurement invariance was tested to make sure that variables used in the analysis were comparable constructs across different groups. Of 3610 nurses and 1137 physicians providing ICU bedside care, 2275 (63.1%) and 717 (62.9%) participated respectively. Statistical analyses revealed that a shortened 32-item version of the EDMCQ scale provides a factorial valid measurement of seven facets of the extent to which clinicians perceive an EDM climate: self-reflective and empowering leadership by physicians; practice and culture of open interdisciplinary reflection; culture of not avoiding end-of-life decisions; culture of mutual respect within the interdisciplinary team; active involvement of nurses in end-of-life care and decision-making; active decision-making by physicians; and practice and culture of ethical awareness. Measurement invariance of the EDMCQ across occupational groups was shown, reflecting that nurses and physicians interpret the EDMCQ items in a similar manner. The 32-item version of the EDMCQ might enrich the EDM climate measurement, clinicians' behaviour and the performance of healthcare organisations. This instrument offers opportunities to develop tailored ICU

An Extended Validity Argument for Assessing Feedback Culture.

Science.gov (United States)

Rougas, Steven; Clyne, Brian; Cianciolo, Anna T; Chan, Teresa M; Sherbino, Jonathan; Yarris, Lalena M

2015-01-01

NEGEA 2015 CONFERENCE ABSTRACT (EDITED): Measuring an Organization's Culture of Feedback: Can It Be Done? Steven Rougas and Brian Clyne. CONSTRUCT: This study sought to develop a construct for measuring formative feedback culture in an academic emergency medicine department. Four archetypes (Market, Adhocracy, Clan, Hierarchy) reflecting an organization's values with respect to focus (internal vs. external) and process (flexibility vs. stability and control) were used to characterize one department's receptiveness to formative feedback. The prevalence of residents' identification with certain archetypes served as an indicator of the department's organizational feedback culture. New regulations have forced academic institutions to implement wide-ranging changes to accommodate competency-based milestones and their assessment. These changes challenge residencies that use formative feedback from faculty as a major source of data for determining training advancement. Though various approaches have been taken to improve formative feedback to residents, there currently exists no tool to objectively measure the organizational culture that surrounds this process. Assessing organizational culture, commonly used in the business sector to represent organizational health, may help residency directors gauge their program's success in fostering formative feedback. The Organizational Culture Assessment Instrument (OCAI) is widely used, extensively validated, applicable to survey research, and theoretically based and may be modifiable to assess formative feedback culture in the emergency department. Using a modified Delphi technique and several iterations of focus groups amongst educators at one institution, four of the original six OCAI domains (which each contain 4 possible responses) were modified to create a 16-item Formative Feedback Culture Tool (FFCT) that was administered to 26 residents (response rate = 55%) at a single academic emergency medicine department. The mean

Is a sphygmomanometer a valid and reliable tool to measure the isometric strength of hip muscles? A systematic review.

Science.gov (United States)

Toohey, Liam Anthony; De Noronha, Marcos; Taylor, Carolyn; Thomas, James

2015-02-01

Muscle strength measurement is a key component of physiotherapists' assessment and is frequently used as an outcome measure. A sphygmomanometer is an instrument commonly used to measure blood pressure that can be potentially used as a tool to assess isometric muscle strength. To systematically review the evidence on the reliability and validity of a sphygmomanometer for measuring isometric strength of hip muscles. A literature search was conducted across four databases. Studies were eligible if they presented data on reliability and/or validity, used a sphygmomanometer to measure isometric muscle strength of the hip region, and were peer reviewed. The individual studies were evaluated for quality using a standardized critical appraisal tool. A total of 644 articles were screened for eligibility, with five articles chosen for inclusion. The use of a sphygmomanometer to objectively assess isometric muscle strength of the hip muscles appears to be reliable with intraclass correlation coefficient values ranging from 0.66 to 0.94 in elderly and young populations. No studies were identified that have assessed the validity of a sphygmomanometer. The sphygmomanometer appears to be reliable for assessment of isometric muscle strength around the hip joint, but further research is warranted to establish its validity.

Evaluating the utility of workplace-based assessment tools for speciality training.

Science.gov (United States)

Setna, Z; Jha, V; Boursicot, K A M; Roberts, T E

2010-12-01

Workplace assessment has been incorporated into speciality training in the UK following changes in the training and work patterns within the National Health Service (NHS). There are various types of assessment tools that have been adopted to assess the clinical competence of trainees. In obstetrics and gynaecology, these include mini-CEX, Objective Structured Assessment of Technical skills (OSATS) and case-based discussion (CbDs). This review provides a theoretical background of workplace assessment and the educational framework that may be adopted to evaluate their effectiveness. It summarises current evidence for the utility of these tools with regard to reliability, validity, acceptability, educational impact and cost. Copyright © 2010 Elsevier Ltd. All rights reserved.

Development of a self-assessment teamwork tool for use by medical and nursing students.

Science.gov (United States)

Gordon, Christopher J; Jorm, Christine; Shulruf, Boaz; Weller, Jennifer; Currie, Jane; Lim, Renee; Osomanski, Adam

2016-08-24

Teamwork training is an essential component of health professional student education. A valid and reliable teamwork self-assessment tool could assist students to identify desirable teamwork behaviours with the potential to promote learning about effective teamwork. The aim of this study was to develop and evaluate a self-assessment teamwork tool for health professional students for use in the context of emergency response to a mass casualty. The authors modified a previously published teamwork instrument designed for experienced critical care teams for use with medical and nursing students involved in mass casualty simulations. The 17-item questionnaire was administered to students immediately following the simulations. These scores were used to explore the psychometric properties of the tool, using Exploratory and Confirmatory Factor Analysis. 202 (128 medical and 74 nursing) students completed the self-assessment teamwork tool for students. Exploratory factor analysis revealed 2 factors (5 items - Teamwork coordination and communication; 4 items - Information sharing and support) and these were justified with confirmatory factor analysis. Internal consistency was 0.823 for Teamwork coordination and communication, and 0.812 for Information sharing and support. These data provide evidence to support the validity and reliability of the self-assessment teamwork tool for students This self-assessment tool could be of value to health professional students following team training activities to help them identify the attributes of effective teamwork.

Validation of the MEDFICTS dietary questionnaire: A clinical tool to assess adherence to American Heart Association dietary fat intake guidelines

Directory of Open Access Journals (Sweden)

Bindeman Jody

2003-06-01

Full Text Available Abstract Background Dietary assessment tools are often too long, difficult to quantify, expensive to process, and largely used for research purposes. A rapid and accurate assessment of dietary fat intake is critically important in clinical decision-making regarding dietary advice for coronary risk reduction. We assessed the validity of the MEDFICTS (MF questionnaire, a brief instrument developed to assess fat intake according to the American Heart Association (AHA dietary "steps". Methods We surveyed 164 active-duty US Army personnel without known coronary artery disease at their intake interview for a primary prevention cardiac intervention trial using the Block food frequency (FFQ and MF questionnaires. Both surveys were completed on the same intake visit and independently scored. Correlations between each tools' assessment of fat intake, the agreement in AHA step categorization of dietary quality with each tool, and the test characteristics of the MF using the FFQ as the gold standard were assessed. Results Subjects consumed a mean of 36.0 ± 13.0% of their total calories as fat, which included saturated fat consumption of 13.0 ± 0.4%. The majority of subjects (125/164; 76.2% had a high fat (worse than AHA Step 1 diet. There were significant correlations between the MF and the FFQ for the intake of total fat (r = 0.52, P 70 [high fat diet] was negligible (kappa statistic = 0.036. The MF was accurate at the extremes of fat intake, but could not reliably identify the 3 AHA dietary classifications. Alternative MF cutpoints of 50 (high fat diet were highly sensitive (96%, but had low specificity (46% for a high fat diet. ROC curve analysis identified that a MF score cutoff of 38 provided optimal sensitivity 75% and specificity 72%, and had modest agreement (kappa = 0.39, P Conclusions The MEDFICTS questionnaire is most suitable as a tool to identify high fat diets, rather than discriminate AHA Step 1 and Step 2 diets. Currently recommended

Assessing attachment in school-aged children: Do the School-Age Assessment of Attachment and Family Drawings work together as complementary tools?

Science.gov (United States)

Carr-Hopkins, Rebecca; De Burca, Calem; Aldridge, Felicity A

2017-07-01

Our goal was to identify an assessment package that could improve treatment planning for troubled children and their families. To assess the validity of our tools, we tested the relations among the School-Age Assessment of Attachment, the Family Drawing and children's risk status. We used the Dynamic-Maturational Model of Attachment and Adaptation to interpret the assessments in the hope of identifying a gradient of risk, and explore whether a new coding method improved the validity of Family Drawings and their utility as a tool to complement the School-Age Assessment of Attachment. The participants were 89 children, aged between 5 and 12 years; 32 children were involved with mental health services or child protection. Each child completed a School-Age Assessment of Attachment and a Family Drawing. Both assessments differentiated between clinical and normative referrals with moderate effect sizes when dichotomizing risk versus non-risk attachment. When the analysis incorporated a gradient of six attachment classifications, the effect sizes decreased, but specificity of risk increased. The School-Age Assessment of Attachment had greater validity for discriminating risk, and type of risk, than the Family Drawings. With a School-Age Assessment of Attachment and family history, the Family Drawing can provide information about distress that some children do not provide verbally. Integration of the two assessment tools alongside information about parental and family functioning appears to be the key to formulating children's problems.

Judging the Neonatal Abstinence Syndrome Assessment Tools to Guide Future Tool Development: The use of Clinimetrics as Opposed to Psychometrics

Directory of Open Access Journals (Sweden)

Philip M. Westgate

2017-09-01

Full Text Available In the face of the current Neonatal Abstinence Syndrome (NAS epidemic, there is considerable variability in the assessment and management of infants with NAS. In this manuscript, we particularly focus on NAS assessment, with special attention given to the popular Finnegan Neonatal Abstinence Score (FNAS. A major instigator of the problem of variable practices is that multiple modified versions of the FNAS exist and continue to be proposed, including shortened versions. Furthermore, the validity of such assessment tools has been questioned, and as a result, the need for better tools has been suggested. The ultimate purpose of this manuscript, therefore, is to increase researchers’ and clinicians’ understanding on how to judge the usefulness of NAS assessment tools in order to guide future tool development and to reduce variable practices. In short, we suggest that judgment of NAS assessment tools should be made on a clinimetrics viewpoint as opposed to psychometrically. We provide examples, address multiple issues that must be considered, and discuss future tool development. Furthermore, we urge researchers and clinicians to come together, utilizing their knowledge and experience, to assess the utility and practicality of existing assessment tools and to determine if one or more new or modified tools are needed with the goal of increased agreement on the assessment of NAS in practice.

Telephone-based screening tools for mild cognitive impairment and dementia in aging studies: a review of validated instruments

Directory of Open Access Journals (Sweden)

Teresa Costa Castanho

2014-02-01

Full Text Available The decline of cognitive function in old age is a great challenge for modern society. The simultaneous increase in dementia and other neurodegenerative diseases justifies a growing need for accurate and valid cognitive assessment instruments. Although in-person testing is considered the most effective and preferred administration mode of assessment, it can pose not only a research difficulty in reaching large and diverse population samples, but it may also limit the assessment and follow-up of individuals with either physical or health limitations or reduced motivation. Therefore, telephone-based cognitive screening instruments pose an alternative and attractive strategy to in-person assessments. In order to give a current view of the state of the art of telephone-based tools for cognitive assessment in aging, this review highlights some of the existing instruments with particular focus on data validation, cognitive domains assessed, administration time and instrument limitations and advantages. From the review of the literature, performed using the databases EBSCO, Science Direct and PubMed, it was possible to verify that while telephone-based tools are useful in research and clinical practice, providing a promising approach, the methodologies still need refinement in the validation steps, including comparison with either single instruments or neurocognitive test batteries, to improve specificity and sensitivity to validly detect subtle changes in cognition that may precede cognitive impairment.

Development of a golf-specific load monitoring tool: Content validity and feasibility.

Science.gov (United States)

Williams, Scott B; Gastin, Paul B; Saw, Anna E; Robertson, Sam

2018-05-01

Athletes often record details of their training and competitions, supported by information such as environmental conditions, travel, as well as how they felt. However, it is not known how prevalent these practices are in golfers, or how valuable this process is perceived. The purpose of this study was to develop a golf-specific load monitoring tool (GLMT), and establish the content validity and feasibility of this tool amongst high-level golfers. In the first phase of development, 21 experts were surveyed to determine the suitability of items for inclusion in the GLMT. Of the 36 items, 21 received >78% agreement, a requirement to establish content validity and for inclusion in the GLMT. Total duration was the preferred metric for golf-specific activities, whilst rating of perceived exertion (RPE) was preferred for measuring physical training. In the second phase, feasibility of the tool was assessed by surveying 13 high-level male golfers following 28-days of daily GLMT use. All items included in the GLMT were deemed feasible to record, with all players participating in the feasibility study providing high to very high ratings. Golfers responded that they would consider using a load monitoring tool of this nature long term, provided it can be completed in less than five minutes per day.

Development, Sensibility, and Validity of a Systemic Autoimmune Rheumatic Disease Case Ascertainment Tool.

Science.gov (United States)

Armstrong, Susan M; Wither, Joan E; Borowoy, Alan M; Landolt-Marticorena, Carolina; Davis, Aileen M; Johnson, Sindhu R

2017-01-01

Case ascertainment through self-report is a convenient but often inaccurate method to collect information. The purposes of this study were to develop, assess the sensibility, and validate a tool to identify cases of systemic autoimmune rheumatic diseases (SARD) in the outpatient setting. The SARD tool was administered to subjects sampled from specialty clinics. Determinants of sensibility - comprehensibility, feasibility, validity, and acceptability - were evaluated using a numeric rating scale from 1-7. Comprehensibility was evaluated using the Flesch Reading Ease and the Flesch-Kincaid Grade Level. Self-reported diagnoses were validated against medical records using Cohen's κ statistic. There were 141 participants [systemic lupus erythematosus (SLE), systemic sclerosis (SSc), rheumatoid arthritis, Sjögren syndrome (SS), inflammatory myositis (polymyositis/dermatomyositis; PM/DM), and controls] who completed the questionnaire. The Flesch Reading Ease score was 77.1 and the Flesch-Kincaid Grade Level was 4.4. Respondents endorsed (mean ± SD) comprehensibility (6.12 ± 0.92), feasibility (5.94 ± 0.81), validity (5.35 ± 1.10), and acceptability (3.10 ± 2.03). The SARD tool had a sensitivity of 0.91 (95% CI 0.88-0.94) and a specificity of 0.99 (95% CI 0.96-1.00). The agreement between the SARD tool and medical record was κ = 0.82 (95% CI 0.77-0.88). Subgroup analysis by SARD found κ coefficients for SLE to be κ = 0.88 (95% CI 0.79-0.97), SSc κ = 1.0 (95% CI 1.0-1.0), PM/DM κ = 0.72 (95% CI 0.49-0.95), and SS κ = 0.85 (95% CI 0.71-0.99). The screening questions had sensitivity ranging from 0.96 to 1.0 and specificity ranging from 0.88 to 1.0. This SARD case ascertainment tool has demonstrable sensibility and validity. The use of both screening and confirmatory questions confers added accuracy.

Measuring patient-provider communication skills in Rwanda: Selection, adaptation and assessment of psychometric properties of the Communication Assessment Tool.

Science.gov (United States)

Cubaka, Vincent Kalumire; Schriver, Michael; Vedsted, Peter; Makoul, Gregory; Kallestrup, Per

2018-04-23

To identify, adapt and validate a measure for providers' communication and interpersonal skills in Rwanda. After selection, translation and piloting of the measure, structural validity, test-retest reliability, and differential item functioning were assessed. Identification and adaptation: The 14-item Communication Assessment Tool (CAT) was selected and adapted. Content validation found all items highly relevant in the local context except two, which were retained upon understanding the reasoning applied by patients. Eleven providers and 291 patients were involved in the field-testing. Confirmatory factor analysis showed a good fit for the original one factor model. Test-retest reliability assessment revealed a mean quadratic weighted Kappa = 0.81 (range: 0.69-0.89, N = 57). The average proportion of excellent scores was 15.7% (SD: 24.7, range: 9.9-21.8%, N = 180). Differential item functioning was not observed except for item 1, which focuses on greetings, for age groups (p = 0.02, N = 180). The Kinyarwanda version of CAT (K-CAT) is a reliable and valid patient-reported measure of providers' communication and interpersonal skills. K-CAT was validated on nurses and its use on other types of providers may require further validation. K-CAT is expected to be a valuable feedback tool for providers in practice and in training. Copyright © 2018 Elsevier B.V. All rights reserved.

Validated assessment scales for the lower face.

Science.gov (United States)

Narins, Rhoda S; Carruthers, Jean; Flynn, Timothy C; Geister, Thorin L; Görtelmeyer, Roman; Hardas, Bhushan; Himmrich, Silvia; Jones, Derek; Kerscher, Martina; de Maio, Maurício; Mohrmann, Cornelia; Pooth, Rainer; Rzany, Berthold; Sattler, Gerhard; Buchner, Larry; Benter, Ursula; Breitscheidel, Lusine; Carruthers, Alastair

2012-02-01

Aging in the lower face leads to lines, wrinkles, depression of the corners of the mouth, and changes in lip volume and lip shape, with increased sagging of the skin of the jawline. Refined, easy-to-use, validated, objective standards assessing the severity of these changes are required in clinical research and practice. To establish the reliability of eight lower face scales assessing nasolabial folds, marionette lines, upper and lower lip fullness, lip wrinkles (at rest and dynamic), the oral commissure and jawline, aesthetic areas, and the lower face unit. Four 5-point rating scales were developed to objectively assess upper and lower lip wrinkles, oral commissures, and the jawline. Twelve experts rated identical lower face photographs of 50 subjects in two separate rating cycles using eight 5-point scales. Inter- and intrarater reliability of responses was assessed. Interrater reliability was substantial or almost perfect for all lower face scales, aesthetic areas, and the lower face unit. Intrarater reliability was high for all scales, areas and the lower face unit. Our rating scales are reliable tools for valid and reproducible assessment of the aging process in lower face areas. © 2012 by the American Society for Dermatologic Surgery, Inc. Published by Wiley Periodicals, Inc.

Tools for assessing fall risk in the elderly: a systematic review and meta-analysis.

Science.gov (United States)

Park, Seong-Hi

2018-01-01

The prevention of falls among the elderly is arguably one of the most important public health issues in today's aging society. The aim of this study was to assess which tools best predict the risk of falls in the elderly. Electronic searches were performed using Medline, EMBASE, the Cochrane Library, CINAHL, etc., using the following keywords: "fall risk assessment", "elderly fall screening", and "elderly mobility scale". The QUADAS-2 was applied to assess the internal validity of the diagnostic studies. Selected studies were meta-analyzed with MetaDisc 1.4. A total of 33 studies were eligible out of the 2,321 studies retrieved from selected databases. Twenty-six assessment tools for fall risk were used in the selected articles, and they tended to vary based on the setting. The fall risk assessment tools currently used for the elderly did not show sufficiently high predictive validity for differentiating high and low fall risks. The Berg Balance scale and Mobility Interaction Fall chart showed stable and high specificity, while the Downton Fall Risk Index, Hendrich II Fall Risk Model, St. Thomas's Risk Assessment Tool in Falling elderly inpatients, Timed Up and Go test, and Tinetti Balance scale showed the opposite results. We concluded that rather than a single measure, two assessment tools used together would better evaluate the characteristics of falls by the elderly that can occur due to a multitude of factors and maximize the advantages of each for predicting the occurrence of falls.

Clinical Case Vignettes: A Promising Tool to Assess Competence in the Management of Agitation.

Science.gov (United States)

Sowden, Gillian L; Vestal, Heather S; Stoklosa, Joseph B; Valcourt, Stephanie C; Peabody, John W; Keary, Christopher J; Nejad, Shamim H; Caminis, Argyro; Huffman, Jeff C

2017-06-01

While standardized patients (SPs) remain the gold standard for assessing clinical competence in a standardized setting, clinical case vignettes that allow free-text, open-ended written responses are more resource- and time-efficient assessment tools. It remains unknown, however, whether this is a valid method for assessing competence in the management of agitation. Twenty-six psychiatry residents partook in a randomized controlled study evaluating a simulation-based teaching intervention on the management of agitated patients. Competence in the management of agitation was assessed using three separate modalities: simulation with SPs, open-ended clinical vignettes, and self-report questionnaires. Performance on clinical vignettes correlated significantly with SP-based assessments (r = 0.59, p = 0.002); self-report questionnaires that assessed one's own ability to manage agitation did not correlate with SP-based assessments (r = -0.06, p = 0.77). Standardized clinical vignettes may be a simple, time-efficient, and valid tool for assessing residents' competence in the management of agitation.

Development and Validation of the Brief Folate-Specific Food Frequency Questionnaire for Young Women's Diet Assessment.

Science.gov (United States)

Głąbska, Dominika; Książek, Aneta; Guzek, Dominika

2017-12-14

The tools enabling brief assessment of folate intake may be of great value for public health purposes. The aim of the presented study was to design a brief folate-specific food frequency questionnaire for Central and Eastern European population of women, as well as to assess the validity and reproducibility of the designed Folate-Intake Calculation-Food Frequency Questionnaire (Fol-IC-FFQ) on a group of Polish women aged 20-30 years. Participants collected 3-day dietary records and completed the Fol-IC-FFQ twice (FFQ1: directly after the dietary record; and FFQ2: six weeks later). The analysis included an assessment of validity (comparison of the results of FFQ1 and 3-day dietary record) and of reproducibility (comparison of the results of FFQ1 and FFQ2). In assessment of validity, a Bland-Altman index of 5.3% was observed. In assessment of reproducibility, a Bland-Altman index of 2.7% was observed, the share of individuals classified into the same intake adequacy category was over 85%, the share of individuals classified into the same tertile was almost 75%, the weighted κ statistic indicated substantial agreement (0.67) and correlation was significant ( p = 0.0000; R = 0.7995). Assessment of the Fol-IC-FFQ revealed a satisfactory level of validity and very good level of reproducibility in the population of young Polish women. The Fol-IC-FFQ may be considered a valid tool for the assessment of folate intake in young Polish women and a promising tool for the assessment of folate intake in young women in Central and Eastern Europe.

The effectiveness of environment assessment tools to guide refurbishment of Australian residential aged care facilities: A systematic review.

Science.gov (United States)

Neylon, Samantha; Bulsara, Caroline; Hill, Anne-Marie

2017-06-01

To determine applicability of environment assessment tools in guiding minor refurbishments of Australian residential aged care facilities. Studies conducted in residential aged care settings using assessment tools which address the physical environment were eligible for inclusion in a systematic review. Given these studies are limited, tools which have not yet been utilised in research settings were also included. Tools were analysed using a critical appraisal screen. Forty-three publications met the inclusion criteria. Ten environment assessment tools were identified, of which four addressed all seven minor refurbishment domains of lighting, colour and contrast, sound, flooring, furniture, signage and way finding. Only one had undergone reliability and validity testing. There are four tools which may be suitable to use for minor refurbishment of Australian residential aged care facilities. Data on their reliability, validity and quality are limited. © 2017 AJA Inc.

Validation of an explanatory tool for data-fused displays for high-technology future aircraft

Science.gov (United States)

Fletcher, Georgina C. L.; Shanks, Craig R.; Selcon, Stephen J.

1996-05-01

As the number of sensor and data sources in the military cockpit increases, pilots will suffer high levels of workload which could result in reduced performance and the loss of situational awareness. A DRA research program has been investigating the use of data-fused displays in decision support and has developed and laboratory-tested an explanatory tool for displaying information in air combat scenarios. The tool has been designed to provide pictorial explanations of data that maintain situational awareness by involving the pilot in the hostile aircraft threat assessment task. This paper reports a study carried out to validate the success of the explanatory tool in a realistic flight simulation facility. Aircrew were asked to perform a threat assessment task, either with or without the explanatory tool providing information in the form of missile launch success zone envelopes, while concurrently flying a waypoint course within set flight parameters. The results showed that there was a significant improvement (p less than 0.01) in threat assessment accuracy of 30% when using the explanatory tool. This threat assessment performance advantage was achieved without a trade-off with flying task performance. Situational awareness measures showed no general differences between the explanatory and control conditions, but significant learning effects suggested that the explanatory tool makes the task initially more intuitive and hence less demanding on the pilots' attentional resources. The paper concludes that DRA's data-fused explanatory tool is successful at improving threat assessment accuracy in a realistic simulated flying environment, and briefly discusses the requirements for further research in the area.

Motivational Interviewing Skills in Health Care Encounters (MISHCE): Development and psychometric testing of an assessment tool.

Science.gov (United States)

Petrova, Tatjana; Kavookjian, Jan; Madson, Michael B; Dagley, John; Shannon, David; McDonough, Sharon K

2015-01-01

Motivational interviewing (MI) has demonstrated a significant impact as an intervention strategy for addiction management, change in lifestyle behaviors, and adherence to prescribed medication and other treatments. Key elements to studying MI include training in MI of professionals who will use it, assessment of skills acquisition in trainees, and the use of a validated skills assessment tool. The purpose of this research project was to develop a psychometrically valid and reliable tool that has been designed to assess MI skills competence in health care provider trainees. The goal was to develop an assessment tool that would evaluate the acquisition and use of specific MI skills and principles, as well as the quality of the patient-provider therapeutic alliance in brief health care encounters. To address this purpose, specific steps were followed, beginning with a literature review. This review contributed to the development of relevant conceptual and operational definitions, selecting a scaling technique and response format, and methods for analyzing validity and reliability. Internal consistency reliability was established on 88 video recorded interactions. The inter-rater and test-retest reliability were established using randomly selected 18 from the 88 interactions. The assessment tool Motivational Interviewing Skills for Health Care Encounters (MISHCE) and a manual for use of the tool were developed. Validity and reliability of MISHCE were examined. Face and content validity were supported with well-defined conceptual and operational definitions and feedback from an expert panel. Reliability was established through internal consistency, inter-rater reliability, and test-retest reliability. The overall internal consistency reliability (Cronbach's alpha) for all fifteen items was 0.75. MISHCE demonstrated good inter-rater reliability and good to excellent test-retest reliability. MISHCE assesses the health provider's level of knowledge and skills in brief

CROSS-CULTURAL ADAPTATION AND VALIDATION OF THE KOREAN VERSION OF THE CUMBERLAND ANKLE INSTABILITY TOOL.

Science.gov (United States)

Ko, Jupil; Rosen, Adam B; Brown, Cathleen N

2015-12-01

The Cumberland Ankle Instability Tool (CAIT) is a valid and reliable patient reported outcome used to assess the presence and severity of chronic ankle instability (CAI). The CAIT has been cross-culturally adapted into other languages for use in non-English speaking populations. However, there are no valid questionnaires to assess CAI in individuals who speak Korean. The purpose of this study was to translate, cross-culturally adapt, and validate the CAIT, for use in a Korean-speaking population with CAI. Cross-cultural reliability study. The CAIT was cross-culturally adapted into Korean according to accepted guidelines and renamed the Cumberland Ankle Instability Tool-Korean (CAIT-K). Twenty-three participants (12 males, 11 females) who were bilingual in English and Korean were recruited and completed the original and adapted versions to assess agreement between versions. An additional 168 national level Korean athletes (106 male, 62 females; age = 20.3 ± 1.1 yrs), who participated in ≥ 90 minutes of physical activity per week, completed the final version of the CAIT-K twice within 14 days. Their completed questionnaires were assessed for internal consistency, test-retest reliability, criterion validity, and construct validity. For bilingual participants, intra-class correlation coefficients (ICC2,1) between the CAIT and the CAIT-K for test-retest reliability were 0.95 (SEM=1.83) and 0.96 (SEM=1.50) in right and left limbs, respectively. The Cronbach's alpha coefficients were 0.92 and 0.90 for the CAIT-K in right and left limbs, respectively. For native Korean speakers, the CAIT-K had high internal consistency (Cronbach's α=0.89) and intra-class correlation coefficient (ICC2,1 = 0.94, SEM=1.72), correlation with the physical component score (rho=0.70, p = 0.001) of the Short-Form Health Survey (SF-36), and the Kaiser-Meyer-Olkin score was 0.87. The original CAIT was translated, cross-culturally adapted, and validated from English to Korean

Predictive validity of measurements of clinical competence using the team objective structured bedside assessment (TOSBA): assessing the clinical competence of final year medical students.

LENUS (Irish Health Repository)

Meagher, Frances M

2009-11-01

The importance of valid and reliable assessment of student competence and performance is gaining increased recognition. Provision of valid patient-based formative assessment is an increasing challenge for clinical teachers in a busy hospital setting. A formative assessment tool that reliably predicts performance in the summative setting would be of value to both students and teachers.

Measurement properties of tools used to assess suicidality in autistic and general population adults: A systematic review.

Science.gov (United States)

Cassidy, S A; Bradley, L; Bowen, E; Wigham, S; Rodgers, J

2018-05-05

Adults diagnosed with autism are at significantly increased risk of suicidal thoughts, suicidal behaviours and dying by suicide. However, it is unclear whether any validated tools are currently available to effectively assess suicidality in autistic adults in research and clinical practice. This is crucial for understanding and preventing premature death by suicide in this vulnerable group. This two stage systematic review therefore aimed to identify tools used to assess suicidality in autistic and general population adults, evaluate these tools for their appropriateness and measurement properties, and make recommendations for appropriate selection of suicidality assessment tools in research and clinical practice. Three databases were searched (PsycInfo, Medline and Web of Knowledge). Four frequently used suicidality assessment tools were identified, and subsequently rated for quality of the evidence in support of their measurement properties using the COSMIN checklist. Despite studies having explored suicidality in autistic adults, none had utilised a validated tool. Overall, there was lack of evidence in support of suicidality risk assessments successfully predicting future suicide attempts. We recommend adaptations to current suicidality assessment tools and priorities for future research, in order to better conceptualise suicidality and its measurement in autism. Copyright © 2018 The Authors. Published by Elsevier Ltd.. All rights reserved.

Improving Major Depressive Episode Assessment: A New Tool Developed by Formal Psychological Assessment.

Science.gov (United States)

Serra, Francesca; Spoto, Andrea; Ghisi, Marta; Vidotto, Giulio

2017-01-01

Aim: Major depressive episode (MDE) can manifest with different features. Discriminating between different types of MDEs is crucial for proper treatment. The aim of this study is to propose a new tool for MDE assessment in bipolar disorder (BD) or major depressive disorder (MDD) to overcome some limitations of current rating scales. The proposed tool investigates all of the clinical features of different MDEs and gives qualitative information, differentiating patients with the same score but different symptoms and psychopathology severity. To achieve this purpose authors used a new methodology called Formal Psychological Assessment (FPA). FPA allows creating relations between the items of an assessment tool, and the set of diagnostic criteria of a given clinical disorder. In the application at hand, given the capability to analyze all clinical features, FPA appears a useful way to highlight and differentiate between inhibited and agitated depressive symptoms. Method: The new tool contains 41 items constructed through 23 clinical criteria from the DSM-5 and literature symptoms. In line with FPA, starting from a set of items and a set of clinical criteria, a Boolean matrix was built assigning to each item its own set of clinical criteria. The participants include 265 in the control group and 38 patients with MDE (diagnosed with MDD or BD) who answered the QuEDS. After 1 month, 63 participants performed the test again and 113 took the Depression-Anxiety-Stress Scale to analyze convergent-divergent validity. Results: The scale showed adequate reliability and validity. A hierarchical confirmatory factor analysis highlighted the presence of three sub factors (affective, somatic, and cognitive) and one high-order factor (depression). Conclusions: The new tool is potentially able to inform clinicians about the patients' most likely diagnostic configuration. Indeed, the clinical state of a patient consists of the subset of items he/she answered affirmatively, along with his

Body Dysmorphic Disorder in aesthetic rhinoplasty: Validating a new screening tool.

Science.gov (United States)

Lekakis, Garyfalia; Picavet, Valerie A; Gabriëls, Loes; Grietens, Jente; Hellings, Peter W

2016-08-01

To validate a new screening tool for body dysmorphic disorder (BDD) in patients seeking aesthetic rhinoplasty. We performed a prospective instrument validation study in an academic rhinology clinic. The Body Dysmorphic Disorder Questionnaire-Aesthetic Surgery (BDDQ-AS) is a seven-item short questionnaire validated in 116 patients undergoing aesthetic rhinoplasty. Screening was positive if the patient acknowledged on the BDDQ-AS that he/she was concerned about their appearance (question 1 = yes) AND preoccupied with these concerns (question 2 = yes) AND that these concerns caused at least moderate distress or impairment in different domains of daily life (question 3 or 4 or 5 or 6 ≥ 3 or question 7 = yes). Construct validity was assessed by comparing the BDDQ-AS to the Sheehan Disability Scale and the Derriford Appearance Scale-59. To determine concurrent validity, the BDDQ-AS was compared to the Yale-Brown Obsessive Compulsive Scale Modified for BDD. Finally, the predictive value of the BDDQ-AS on satisfaction 12 months after rhinoplasty was evaluated using a visual analogue scale and the Rhinoplasty Outcome Evaluation. Reliability of the BDDQ-AS was adequate, with Cronbach alpha = .83 for rhinoplasty patients and .84 for controls. Sensitivity was 89.6% and specificity 81.4%. BDDQ-AS-positive patients (n = 55) were more impaired in daily life and experienced more appearance-related distress and dysfunction compared to BDDQ-AS-negative patients. Moreover, they had more severe BDD symptoms. Finally, BDDQ-AS-positive patients were less satisfied after surgery compared to BDDQ-AS-negative patients. We hereby validated a new screening tool for BDD in an aesthetic rhinoplasty population. 3b. Laryngoscope, 126:1739-1745, 2016. © 2016 The American Laryngological, Rhinological and Otological Society, Inc.

Validation of a checklist to assess ward round performance in internal medicine

DEFF Research Database (Denmark)

Nørgaard, Kirsten; Ringsted, Charlotte; Dolmans, Diana

2004-01-01

and construct validity of the task-specific checklist. METHODS: To determine content validity, a questionnaire was mailed to 295 internists. They were requested to give their opinion on the relevance of each item included on the checklist and to indicate the comprehensiveness of the checklist. To determine...... construct validity, an observer assessed 4 groups of doctors during performance of a complete ward round (n = 32). The nurse who accompanied the doctor on rounds made a global assessment of the performance. RESULTS: The response rate to the questionnaire was 80.7%. The respondents found that all 10 items......BACKGROUND: Ward rounds are an essential responsibility for doctors in hospital settings. Tools for guiding and assessing trainees' performance of ward rounds are needed. A checklist was developed for that purpose for use with trainees in internal medicine. OBJECTIVE: To assess the content...

Quantum mechanics concept assessment: Development and validation study

Directory of Open Access Journals (Sweden)

Homeyra R. Sadaghiani

2015-03-01

Full Text Available As part of an ongoing investigation of students’ learning in first semester upper-division quantum mechanics, we needed a high-quality conceptual assessment instrument for comparing outcomes of different curricular approaches. The process of developing such a tool started with converting a preliminary version of a 14-item open-ended quantum mechanics assessment tool (QMAT to a multiple-choice (MC format. Further question refinement, development of effective distractors, adding new questions, and robust statistical analysis has led to a 31-item quantum mechanics concept assessment (QMCA test. The QMCA is used as post-test only to assess students’ knowledge about five main topics of quantum measurement: the time-independent Schrödinger equation, wave functions and boundary conditions, time evolution, and probability density. During two years of testing and refinement, the QMCA has been given in alpha (N=61 and beta versions (N=263 to students in upper division quantum mechanics courses at 11 different institutions with an average post-test score of 54%. By allowing for comparisons of student learning across different populations and institutions, the QMCA provides instructors and researchers a more standard measure of effectiveness of different curricula or teaching strategies on student conceptual understanding of quantum mechanics. In this paper, we discuss the construction of effective distractors and the use of student interviews and expert feedback to revise and validate both questions and distractors. We include the results of common statistical tests of reliability and validity, which suggest the instrument is presently in a stable, usable, and promising form.

Development of a tool to support holistic generic assessment of clinical procedure skills.

Science.gov (United States)

McKinley, Robert K; Strand, Janice; Gray, Tracey; Schuwirth, Lambert; Alun-Jones, Tom; Miller, Helen

2008-06-01

The challenges of maintaining comprehensive banks of valid checklists make context-specific checklists for assessment of clinical procedural skills problematic. This paper reports the development of a tool which supports generic holistic assessment of clinical procedural skills. We carried out a literature review, focus groups and non-participant observation of assessments with interview of participants, participant evaluation of a pilot objective structured clinical examination (OSCE), a national modified Delphi study with prior definitions of consensus and an OSCE. Participants were volunteers from a large acute teaching trust, a teaching primary care trust and a national sample of National Health Service staff. Results In total, 86 students, trainees and staff took part in the focus groups, observation of assessments and pilot OSCE, 252 in the Delphi study and 46 candidates and 50 assessors in the final OSCE. We developed a prototype tool with 5 broad categories amongst which were distributed 38 component competencies. There was > 70% agreement (our prior definition of consensus) at the first round of the Delphi study for inclusion of all categories and themes and no consensus for inclusion of additional categories or themes. Generalisability was 0.76. An OSCE based on the instrument has a predicted reliability of 0.79 with 12 stations and 1 assessor per station or 10 stations and 2 assessors per station. This clinical procedural skills assessment tool enables reliable assessment and has content and face validity for the assessment of clinical procedural skills. We have designated it the Leicester Clinical Procedure Assessment Tool (LCAT).

Validity and inter-observer reliability of subjective hand-arm vibration assessments

NARCIS (Netherlands)

Coenen, P.; Formanoy, M.; Douwes, M.; Bosch, T.; Kraker, H. de

2014-01-01

Exposure to mechanical vibrations at work (e.g., due to handling powered tools) is a potential occupational risk as it may cause upper extremity complaints. However, reliable and valid assessment methods for vibration exposure at work are lacking. Measuring hand-arm vibration objectively is often

Construct Validity and Case Validity in Assessment

Science.gov (United States)

Teglasi, Hedwig; Nebbergall, Allison Joan; Newman, Daniel

2012-01-01

Clinical assessment relies on both "construct validity", which focuses on the accuracy of conclusions about a psychological phenomenon drawn from responses to a measure, and "case validity", which focuses on the synthesis of the full range of psychological phenomena pertaining to the concern or question at hand. Whereas construct validity is…

An Evaluation Framework and Instrument for Evaluating e-Assessment Tools

Science.gov (United States)

Singh, Upasana Gitanjali; de Villiers, Mary Ruth

2017-01-01

e-Assessment, in the form of tools and systems that deliver and administer multiple choice questions (MCQs), is used increasingly, raising the need for evaluation and validation of such systems. This research uses literature and a series of six empirical action research studies to develop an evaluation framework of categories and criteria called…

Validation of a global scale to assess the quality of interprofessional teamwork in mental health settings.

Science.gov (United States)

Tomizawa, Ryoko; Yamano, Mayumi; Osako, Mitue; Hirabayashi, Naotugu; Oshima, Nobuo; Sigeta, Masahiro; Reeves, Scott

2017-12-01

Few scales currently exist to assess the quality of interprofessional teamwork through team members' perceptions of working together in mental health settings. The purpose of this study was to revise and validate an interprofessional scale to assess the quality of teamwork in inpatient psychiatric units and to use it multi-nationally. A literature review was undertaken to identify evaluative teamwork tools and develop an additional 12 items to ensure a broad global focus. Focus group discussions considered adaptation to different care systems using subjective judgements from 11 participants in a pre-test of items. Data quality, construct validity, reproducibility, and internal consistency were investigated in the survey using an international comparative design. Exploratory factor analysis yielded five factors with 21 items: 'patient/community centred care', 'collaborative communication', 'interprofessional conflict', 'role clarification', and 'environment'. High overall internal consistency, reproducibility, adequate face validity, and reasonable construct validity were shown in the USA and Japan. The revised Collaborative Practice Assessment Tool (CPAT) is a valid measure to assess the quality of interprofessional teamwork in psychiatry and identifies the best strategies to improve team performance. Furthermore, the revised scale will generate more rigorous evidence for collaborative practice in psychiatry internationally.

Assessing peristomal skin changes in ostomy patients : validation of the Ostomy Skin Tool

NARCIS (Netherlands)

Jemec, G. B.; Martins, L.; Claessens, I.; Ayello, E. A.; Hansen, A. S.; Poulsen, L. H.; Sibbald, R. G.

P>Background Peristomal skin problems are common and are treated by a variety of health professionals. Clear and consistent communication among these professionals is therefore particularly important. The Ostomy Skin Tool (OST) is a new assessment instrument for the extent and severity of peristomal

Validation of three tools for identifying painful new osteoporotic vertebral fractures in older Chinese men: bone mineral density, Osteoporosis Self-Assessment Tool for Asians, and fracture risk assessment tool.

Science.gov (United States)

Lin, JiSheng; Yang, Yong; Fei, Qi; Zhang, XiaoDong; Ma, Zhao; Wang, Qi; Li, JinJun; Li, Dong; Meng, Qian; Wang, BingQiang

2016-01-01

This cross-sectional study compared three tools for predicting painful new osteoporotic vertebral fractures (PNOVFs) in older Chinese men: bone mineral density (BMD), the Osteoporosis Self-Assessment Tool for Asians (OSTA), and the World Health Organization fracture risk assessment tool (FRAX) (without BMD). Men aged ≥50 years were apportioned to a group for men with fractures who had undergone percutaneous vertebroplasty (n=111), or a control group of healthy men (n=385). Fractures were verified on X-ray and magnetic resonance imaging. BMD T-scores were determined by dual energy X-ray absorptiometry. Diagnosis of osteoporosis was determined by a BMD T-score of ≤2.5 standard deviations below the average for a young adult at peak bone density at the femoral neck, total hip, or L1-L4. Demographic and clinical risk factor data were self-reported through a questionnaire. BMD, OSTA, and FRAX scores were assessed for identifying PNOVFs via receiver-operating characteristic (ROC) curves. Optimal cutoff points, sensitivity, specificity, and areas under the ROC curves (AUCs) were determined. Between the men with fractures and the control group, there were significant differences in BMD T-scores (at femoral neck, total hip, and L1-L4), and OSTA and FRAX scores. In those with fractures, only 53.15% satisfied the criteria for osteoporosis. Compared to BMD or OSTA, the FRAX score had the best predictive value for PNOVFs: the AUC of the FRAX score (cutoff =2.9%) was 0.738, and the sensitivity and specificity were 82% and 62%, respectively. FRAX may be a valuable tool for identifying PNOVFs in older Chinese men.

Movement Skill Assessment of Typically Developing Preschool Children: A Review of Seven Movement Skill Assessment Tools

Science.gov (United States)

Cools, Wouter; Martelaer, Kristine De; Samaey, Christiane; Andries, Caroline

2009-01-01

The importance of movement is often overlooked because it is such a natural part of human life. It is, however, crucial for a child’s physical, cognitive and social development. In addition, experiences support learning and development of fundamental movement skills. The foundations of those skills are laid in early childhood and essential to encourage a physically active lifestyle. Fundamental movement skill performance can be examined with several assessment tools. The choice of a test will depend on the context in which the assessment is planned. This article compares seven assessment tools which are often referred to in European or international context. It discusses the tools’ usefulness for the assessment of movement skill development in general population samples. After a brief description of each assessment tool the article focuses on contents, reliability, validity and normative data. A conclusion outline of strengths and weaknesses of all reviewed assessment tools focusing on their use in educational research settings is provided and stresses the importance of regular data collection of fundamental movement skill development among preschool children. Key pointsThis review discusses seven movement skill assessment tool’s test content, reliability, validity and normative samples.The seven assessment tools all showed to be of great value. Strengths and weaknesses indicate that test choice will depend on specific purpose of test use.Further data collection should also include larger data samples of able bodied preschool children.Admitting PE specialists in assessment of fundamental movement skill performance among preschool children is recommended.The assessment tool’s normative data samples would benefit from frequent movement skill performance follow-up of today’s children. Abbreviations MOT 4-6: Motoriktest fur vier- bis sechsjährige Kinder, M-ABC: Movement Assessment Battery for Children, PDMS: Peabody Development Scales, KTK: K�

The spiritual distress assessment tool: an instrument to assess spiritual distress in hospitalised elderly persons

Directory of Open Access Journals (Sweden)

Martin Estelle

2010-12-01

Full Text Available Abstract Background Although spirituality is usually considered a positive resource for coping with illness, spiritual distress may have a negative influence on health outcomes. Tools are needed to identify spiritual distress in clinical practice and subsequently address identified needs. This study describes the first steps in the development of a clinically acceptable instrument to assess spiritual distress in hospitalized elderly patients. Methods A three-step process was used to develop the Spiritual Distress Assessment Tool (SDAT: 1 Conceptualisation by a multidisciplinary group of a model (Spiritual Needs Model to define the different dimensions characterizing a patient's spirituality and their corresponding needs; 2 Operationalisation of the Spiritual Needs Model within geriatric hospital care leading to a set of questions (SDAT investigating needs related to each of the defined dimensions; 3 Qualitative assessment of the instrument's acceptability and face validity in hospital chaplains. Results Four dimensions of spirituality (Meaning, Transcendence, Values, and Psychosocial Identity and their corresponding needs were defined. A formalised assessment procedure to both identify and subsequently score unmet spiritual needs and spiritual distress was developed. Face validity and acceptability in clinical practice were confirmed by chaplains involved in the focus groups. Conclusions The SDAT appears to be a clinically acceptable instrument to assess spiritual distress in elderly hospitalised persons. Studies are ongoing to investigate the psychometric properties of the instrument and to assess its potential to serve as a basis for integrating the spiritual dimension in the patient's plan of care.

Reliability and Validity of the Greek Migraine Disability Assessment (MIDAS) Questionnaire.

Science.gov (United States)

Oikonomidi, Theodora; Vikelis, Michail; Artemiadis, Artemios; Chrousos, George P; Darviri, Christina

2018-03-01

The Migraine Disability Assessment (MIDAS) Questionnaire is a reliable and valid instrument for migraine-related disability. Such a tool is needed to quantify migraine-related disability in the Greek population. This validation study aims to assess the test-retest reliability, internal consistency, item discriminant and convergent validity of the Greek translation of the MIDAS. Adults diagnosed with migraine completed the MIDAS Questionnaire on two occasions 3 weeks apart to assess reliability, and completed the RAND-36 to assess validity. Participants (n = 152) had a median MIDAS score of 24 and mostly severe disability (58% were grade IV). The test-retest reliability analysis (N = 59) revealed excellent reliability for the total score. Internal consistency was α = 0.71 for initial and α = 0.82 for retest completion. For item discriminant validity, the correlations between each question and the total score were significant, with high correlations for questions 2-5 (range 0.67 ≤ r ≤ 0.79; p MIDAS score tended to have better wellbeing. Psychometric properties are comparable with those of other published validation studies of the MIDAS and the original. Findings on question 1 show that missing work/school days may be closely related with increased affect issues. The Greek version of the MIDAS Questionnaire has good reliability and validity. This study allowed for cross-cultural comparability of research findings.

Evidence for validity within workplace assessment: the Longitudinal Evaluation of Performance (LEP).

Science.gov (United States)

Prescott-Clements, Linda; van der Vleuten, Cees P M; Schuwirth, Lambert W T; Hurst, Yvonne; Rennie, James S

2008-05-01

The drive towards valid and reliable assessment methods for health professions' training is becoming increasingly focused towards authentic models of workplace performance assessment. This study investigates the validity of such a method, longitudinal evaluation of performance (LEP), which has been implemented in the assessment of postgraduate dental trainees in Scotland. Although it is similar in format to the mini-CEX (mini clinical evaluation exercise) and other tools that use global ratings for assessing performance in the workplace, a number of differences exist in the way in which the LEP has been implemented. These include the use of a reference point for evaluators' judgement that represents the standard expected upon completion of the training, flexibility, a greater range of cases assessed and the use of frequency scores within feedback to identify trainees' progress over time. A range of qualitative and quantitative data were collected and analysed from 2 consecutive cohorts of trainees in Scotland (2002-03 and 2003-04). There is rich evidence supporting the validity, educational impact and feasibility of the LEP. In particular, a great deal of support was given by trainers for the use of a fixed reference point for judgements, despite initial concerns that this might be demotivating to trainees. Trainers were highly positive about this approach and considered it useful in identifying trainees' progress and helping to drive learning. The LEP has been successful in combining a strong formative approach to continuous assessment with the collection of evidence on performance within the workplace that (alongside other tools within an assessment system) can contribute towards a summative decision regarding competence.

A web-based rapid assessment tool for production publishing solutions

Science.gov (United States)

Sun, Tong

2010-02-01

Solution assessment is a critical first-step in understanding and measuring the business process efficiency enabled by an integrated solution package. However, assessing the effectiveness of any solution is usually a very expensive and timeconsuming task which involves lots of domain knowledge, collecting and understanding the specific customer operational context, defining validation scenarios and estimating the expected performance and operational cost. This paper presents an intelligent web-based tool that can rapidly assess any given solution package for production publishing workflows via a simulation engine and create a report for various estimated performance metrics (e.g. throughput, turnaround time, resource utilization) and operational cost. By integrating the digital publishing workflow ontology and an activity based costing model with a Petri-net based workflow simulation engine, this web-based tool allows users to quickly evaluate any potential digital publishing solutions side-by-side within their desired operational contexts, and provides a low-cost and rapid assessment for organizations before committing any purchase. This tool also benefits the solution providers to shorten the sales cycles, establishing a trustworthy customer relationship and supplement the professional assessment services with a proven quantitative simulation and estimation technology.

Evaluating trauma team performance in a Level I trauma center: Validation of the trauma team communication assessment (TTCA-24).

Science.gov (United States)

DeMoor, Stephanie; Abdel-Rehim, Shady; Olmsted, Richard; Myers, John G; Parker-Raley, Jessica

2017-07-01

Nontechnical skills (NTS), such as team communication, are well-recognized determinants of trauma team performance and good patient care. Measuring these competencies during trauma resuscitations is essential, yet few valid and reliable tools are available. We aimed to demonstrate that the Trauma Team Communication Assessment (TTCA-24) is a valid and reliable instrument that measures communication effectiveness during activations. Two tools with adequate psychometric strength (Trauma Nontechnical Skills Scale [T-NOTECHS], Team Emergency Assessment Measure [TEAM]) were identified during a systematic review of medical literature and compared with TTCA-24. Three coders used each tool to evaluate 35 stable and 35 unstable patient activations (defined according to Advanced Trauma Life Support criteria). Interrater reliability was calculated between coders using the intraclass correlation coefficient. Spearman rank correlation coefficient was used to establish concurrent validity between TTCA-24 and the other two validated tools. Coders achieved an intraclass correlation coefficient of 0.87 for stable patient activations and 0.78 for unstable activations scoring excellent on the interrater agreement guidelines. The median score for each assessment showed good team communication for all 70 videos (TEAM, 39.8 of 54; T-NOTECHS, 17.4 of 25; and TTCA-24, 87.4 of 96). A significant correlation between TTTC-24 and T-NOTECHS was revealed (p = 0.029), but no significant correlation between TTCA-24 and TEAM (p = 0.77). Team communication was rated slightly better across all assessments for stable versus unstable patient activations, but not statistically significant. TTCA-24 correlated with T-NOTECHS, an instrument measuring nontechnical skills for trauma teams, but not TEAM, a tool that assesses communication in generic emergency settings. TTCA-24 is a reliable and valid assessment that can be a useful adjunct when evaluating interpersonal and team communication during trauma

Validity and reliability of balance assessment software using the Nintendo Wii balance board: usability and validation.

Science.gov (United States)

Park, Dae-Sung; Lee, GyuChang

2014-06-10

A balance test provides important information such as the standard to judge an individual's functional recovery or make the prediction of falls. The development of a tool for a balance test that is inexpensive and widely available is needed, especially in clinical settings. The Wii Balance Board (WBB) is designed to test balance, but there is little software used in balance tests, and there are few studies on reliability and validity. Thus, we developed a balance assessment software using the Nintendo Wii Balance Board, investigated its reliability and validity, and compared it with a laboratory-grade force platform. Twenty healthy adults participated in our study. The participants participated in the test for inter-rater reliability, intra-rater reliability, and concurrent validity. The tests were performed with balance assessment software using the Nintendo Wii balance board and a laboratory-grade force platform. Data such as Center of Pressure (COP) path length and COP velocity were acquired from the assessment systems. The inter-rater reliability, the intra-rater reliability, and concurrent validity were analyzed by an intraclass correlation coefficient (ICC) value and a standard error of measurement (SEM). The inter-rater reliability (ICC: 0.89-0.79, SEM in path length: 7.14-1.90, SEM in velocity: 0.74-0.07), intra-rater reliability (ICC: 0.92-0.70, SEM in path length: 7.59-2.04, SEM in velocity: 0.80-0.07), and concurrent validity (ICC: 0.87-0.73, SEM in path length: 5.94-0.32, SEM in velocity: 0.62-0.08) were high in terms of COP path length and COP velocity. The balance assessment software incorporating the Nintendo Wii balance board was used in our study and was found to be a reliable assessment device. In clinical settings, the device can be remarkably inexpensive, portable, and convenient for the balance assessment.

Development of a self-assessment teamwork tool for use by medical and nursing students

OpenAIRE

Gordon, Christopher J.; Jorm, Christine; Shulruf, Boaz; Weller, Jennifer; Currie, Jane; Lim, Renee; Osomanski, Adam

2016-01-01

Background Teamwork training is an essential component of health professional student education. A valid and reliable teamwork self-assessment tool could assist students to identify desirable teamwork behaviours with the potential to promote learning about effective teamwork. The aim of this study was to develop and evaluate a self-assessment teamwork tool for health professional students for use in the context of emergency response to a mass casualty. Methods The authors modified a previousl...

Validating SPICES as a Screening Tool for Frailty Risks among Hospitalized Older Adults

Science.gov (United States)

Aronow, Harriet Udin; Borenstein, Jeff; Haus, Flora; Braunstein, Glenn D.; Bolton, Linda Burnes

2014-01-01

Older patients are vulnerable to adverse hospital events related to frailty. SPICES, a common screening protocol to identify risk factors in older patients, alerts nurses to initiate care plans to reduce the probability of patient harm. However, there is little published validating the association between SPICES and measures of frailty and adverse outcomes. This paper used data from a prospective cohort study on frailty among 174 older adult inpatients to validate SPICES. Almost all patients met one or more SPICES criteria. The sum of SPICES was significantly correlated with age and other well-validated assessments for vulnerability, comorbid conditions, and depression. Individuals meeting two or more SPICES criteria had a risk of adverse hospital events three times greater than individuals with either no or one criterion. Results suggest that as a screening tool used within 24 hours of admission, SPICES is both valid and predictive of adverse events. PMID:24876954

The Assessment, Development, Assurance Pharmacist's Tool (ADAPT) for ensuring quality implementation of health promotion programs.

Science.gov (United States)

Truong, Hoai-An; Taylor, Catherine R; DiPietro, Natalie A

2012-02-10

To develop and validate the Assessment, Development, Assurance Pharmacist's Tool (ADAPT), an instrument for pharmacists and student pharmacists to use in developing and implementing health promotion programs. The 36-item ADAPT instrument was developed using the framework of public health's 3 core functions (assessment, policy development, and assurance) and 10 essential services. The tool's content and usage was assessed and conducted through peer-review and initial validity testing processes. Over 20 faculty members, preceptors, and student pharmacists at 5 institutions involved in planning and implementing health promotion initiatives reviewed the instrument and conducted validity testing. The instrument took approximately 15 minutes to complete and the findings resulted in changes and improvements to elements of the programs evaluated. The ADAPT instrument fills a need to more effectively plan, develop, implement, and evaluate pharmacist-directed public health programs that are evidence-based, high-quality, and compliant with laws and regulations and facilitates documentation of pharmacists' contributions to public health.

Health Heritage© a web-based tool for the collection and assessment of family health history: initial user experience and analytic validity.

Science.gov (United States)

Cohn, W F; Ropka, M E; Pelletier, S L; Barrett, J R; Kinzie, M B; Harrison, M B; Liu, Z; Miesfeldt, S; Tucker, A L; Worrall, B B; Gibson, J; Mullins, I M; Elward, K S; Franko, J; Guterbock, T M; Knaus, W A

2010-01-01

A detailed family health history is currently the most potentially useful tool for diagnosis and risk assessment in clinical genetics. We developed and evaluated the usability and analytic validity of a patient-driven web-based family health history collection and analysis tool. Health Heritage(©) guides users through the collection of their family health history by relative, generates a pedigree, completes risk assessment, stratification, and recommendations for 89 conditions. We compared the performance of Health Heritage to that of Usual Care using a nonrandomized cohort trial of 109 volunteers. We contrasted the completeness and sensitivity of family health history collection and risk assessments derived from Health Heritage and Usual Care to those obtained by genetic counselors and genetic assessment teams. Nearly half (42%) of the Health Heritage participants reported discovery of health risks; 63% found the information easy to understand and 56% indicated it would change their health behavior. Health Heritage consistently outperformed Usual Care in the completeness and accuracy of family health history collection, identifying 60% of the elevated risk conditions specified by the genetic team versus 24% identified by Usual Care. Health Heritage also had greater sensitivity than Usual Care when comparing the identification of risks. These results suggest a strong role for automated family health history collection and risk assessment and underscore the potential of these data to serve as the foundation for comprehensive, cost-effective personalized genomic medicine. Copyright © 2010 S. Karger AG, Basel.

[Validation of the German version of Eating Assessment Tool for head and neck cancer patients].

Science.gov (United States)

Zaretsky, Eugen; Steinbach-Hundt, Silke; Pluschinski, Petra; Grethel, Isabel

2018-04-10

The assessment of subjective swallowing complaints constitutes an important element in a multidimensional, modern management of head and neck cancer patients suffering from dysphagia. For this purpose, an internationally recognized and validated 10-item questionnaire EAT-10 is used that was developed and validated by Belafski et al. in 2008. The purpose of the present study is the translation of EAT-10 into the German language and its validation for head and neck cancer patients. After the translation of EAT-10 into German according to the guidelines for the translation of foreign measuring instruments, a validation of gEAT-10 was carried out on the basis of the sample of 210 head and neck cancer patients. The reliability was determined by means of the internal consistency (Cronbach's Alpha) and item-total correlations (Spearman). The construct validity was verified by the uni- and multivariate analyses of the distribution of gEAT-10 total scores depending on gender, age, BMI, tumor stage and localization as well as type of the oncological therapy. The internal consistency amounted to α = .94, the item-total correlations varied between ρ = .59 and ρ = .85. No significant associations between gEAT-10 total scores and gender as well as age were identified in univariate calculations. Such associations were found for BMI, tumor stage and localization as well as type of the oncological therapy. However, only the tumor stage yielded a significant result in a regression. The gEAT-10 was shown to be a reliable and construct valid questionnaire for the assessment of subjective swallowing complaints in patients with head and neck cancer. © Georg Thieme Verlag KG Stuttgart · New York.

The web-buffet--development and validation of an online tool to measure food choice.

Science.gov (United States)

Bucher, Tamara; Keller, Carmen

2015-08-01

To date, no data exist on the agreement of food choice measured using an online tool with subsequent actual consumption. This needs to be shown before food choice, measured by means of an online tool, is used as a dependent variable to examine intake in the general population. A 'web-buffet' was developed to assess food choice. Choice was measured as planned meal composition from photographic material; respondents chose preferred foods and proportions for a main meal (out of a possible 144 combinations) online and the validity was assessed by comparison of a meal composed from a web-buffet with actual food intake 24-48 h later. Furthermore, correlations of food preferences, energy needs and health interest with meals chosen from the web-buffet were analysed. Students: n 106 (Study I), n 32 (Study II). Meals chosen from the web-buffet (mean = 2998 kJ, SD = 471 kJ) agreed with actual consumption (rs = 0.63, P choice in the web-buffet agrees sufficiently well with actual intake to measure food choice as a dependent variable in online surveys. However, we found an average underestimation of subsequent consumption. High correlations of preferences with chosen amounts and an inverse association of health interest with total energy further indicate the validity of the tool. Applications in behavioural nutrition research are discussed.

A content validity approach to creating an end-user computer skill assessment tool

Directory of Open Access Journals (Sweden)

Shirley Gibbs

Full Text Available Practical assessment instruments are commonly used in the workplace and educational environments to assess a person\\'s level of digital literacy and end-user computer skill. However, it is often difficult to find statistical evidence of the actual validity of instruments being used. To ensure that the correct factors are being assessed for a particular purpose it is necessary to undertake some type of psychometric testing, and the first step is to study the content relevance of the measure. The purpose of this paper is to report on the rigorous judgment-quantification process using panels of experts in order to establish inter-rater reliability and agreement in the development of end-user instruments developed to measure workplace skills using spreadsheet and word-processing applications.

Reliability and validity of a novel tool to comprehensively assess food and beverage marketing in recreational sport settings.

Science.gov (United States)

Prowse, Rachel J L; Naylor, Patti-Jean; Olstad, Dana Lee; Carson, Valerie; Mâsse, Louise C; Storey, Kate; Kirk, Sara F L; Raine, Kim D

2018-05-31

Current methods for evaluating food marketing to children often study a single marketing channel or approach. As the World Health Organization urges the removal of unhealthy food marketing in children's settings, methods that comprehensively explore the exposure and power of food marketing within a setting from multiple marketing channels and approaches are needed. The purpose of this study was to test the inter-rater reliability and the validity of a novel settings-based food marketing audit tool. The Food and beverage Marketing Assessment Tool for Settings (FoodMATS) was developed and its psychometric properties evaluated in five public recreation and sport facilities (sites) and subsequently used in 51 sites across Canada for a cross-sectional analysis of food marketing. Raters recorded the count of food marketing occasions, presence of child-targeted and sports-related marketing techniques, and the physical size of marketing occasions. Marketing occasions were classified by healthfulness. Inter-rater reliability was tested using Cohen's kappa (κ) and intra-class correlations (ICC). FoodMATS scores for each site were calculated using an algorithm that represented the theoretical impact of the marketing environment on food preferences, purchases, and consumption. Higher FoodMATS scores represented sites with higher exposure to, and more powerful (unhealthy, child-targeted, sports-related, large) food marketing. Validity of the scoring algorithm was tested through (1) Pearson's correlations between FoodMATS scores and facility sponsorship dollars, and (2) sequential multiple regression for predicting "Least Healthy" food sales from FoodMATS scores. Inter-rater reliability was very good to excellent (κ = 0.88-1.00, p marketing in recreation facilities, the FoodMATS provides a novel means to comprehensively track changes in food marketing environments that can assist in developing and monitoring the impact of policies and interventions.

Validation and assessment of uncertainty of chemical tests as a tool for the reliability analysis of wastewater IPEN

International Nuclear Information System (INIS)

Silva, Renan A.; Martins, Elaine A.J.; Furusawa, Helio A.

2011-01-01

The validation of analytical methods has become an indispensable tool for the analysis in chemical laboratories, including being required for such accreditation. However, even if a laboratory using validated methods of analysis there is the possibility that these methods generate results discrepant with reality by making necessary the addition of a quantitative attribute (a value) which indicates the degree of certainty the extent or the analytical method used. This measure assigned to the result of measurement is called measurement uncertainty. We estimate this uncertainty with a level of confidence both direction, an analytical result has limited significance if not carried out proper assessment of its uncertainty. One of the activities of this work was to elaborate a program to help the validation and evaluation of uncertainty in chemical analysis. The program was developed with Visual Basic programming language and method of evaluation of uncertainty introduced the following concepts based on the GUM (Guide to the Expression of Uncertainty in Measurement). This evaluation program uncertainty measurement will be applied to chemical analysis in support of the characterization of the Nuclear Fuel Cycle developed by IPEN and the study of organic substances in wastewater associated with professional activities of the Institute. In the first case, primarily for the determination of total uranium and the second case for substances that were generated by human activities and that are contained in resolution 357/2005. As strategy for development of this work was considered the PDCA cycle to improve the efficiency of each step and minimize errors while performing the experimental part. The program should be validated to meet requirements of standards such as, for example, the standard ISO/IEC 17025. The application, it is projected to use in other analytical procedures of both the Nuclear Fuel Cycle and in the control program and chemical waste management of IPEN

Validation and assessment of uncertainty of chemical tests as a tool for the reliability analysis of wastewater IPEN

Energy Technology Data Exchange (ETDEWEB)

Silva, Renan A.; Martins, Elaine A.J.; Furusawa, Helio A., E-mail: elaine@ipen.br, E-mail: helioaf@ipen.br [Instituto de Pesquisas Energeticas e Nucleares (IPEN/CNEN-SP), Sao Paulo, SP (Brazil)

2011-07-01

The validation of analytical methods has become an indispensable tool for the analysis in chemical laboratories, including being required for such accreditation. However, even if a laboratory using validated methods of analysis there is the possibility that these methods generate results discrepant with reality by making necessary the addition of a quantitative attribute (a value) which indicates the degree of certainty the extent or the analytical method used. This measure assigned to the result of measurement is called measurement uncertainty. We estimate this uncertainty with a level of confidence both direction, an analytical result has limited significance if not carried out proper assessment of its uncertainty. One of the activities of this work was to elaborate a program to help the validation and evaluation of uncertainty in chemical analysis. The program was developed with Visual Basic programming language and method of evaluation of uncertainty introduced the following concepts based on the GUM (Guide to the Expression of Uncertainty in Measurement). This evaluation program uncertainty measurement will be applied to chemical analysis in support of the characterization of the Nuclear Fuel Cycle developed by IPEN and the study of organic substances in wastewater associated with professional activities of the Institute. In the first case, primarily for the determination of total uranium and the second case for substances that were generated by human activities and that are contained in resolution 357/2005. As strategy for development of this work was considered the PDCA cycle to improve the efficiency of each step and minimize errors while performing the experimental part. The program should be validated to meet requirements of standards such as, for example, the standard ISO/IEC 17025. The application, it is projected to use in other analytical procedures of both the Nuclear Fuel Cycle and in the control program and chemical waste management of IPEN

Methodology for validating technical tools to assess customer Demand Response: Application to a commercial customer

International Nuclear Information System (INIS)

Alcazar-Ortega, Manuel; Escriva-Escriva, Guillermo; Segura-Heras, Isidoro

2011-01-01

The authors present a methodology, which is demonstrated with some applications to the commercial sector, in order to validate a Demand Response (DR) evaluation method previously developed and applied to a wide range of industrial and commercial segments, whose flexibility was evaluated by modeling. DR is playing a more and more important role in the framework of electricity systems management for the effective integration of other distributed energy resources. Consequently, customers must identify what they are using the energy for in order to use their flexible loads for management purposes. Modeling tools are used to predict the impact of flexibility on the behavior of customers, but this result needs to be validated since both customers and grid operators have to be confident in these flexibility predictions. An easy-to-use two-steps method to achieve this goal is presented in this paper.

Validity of the Nintendo Wii® balance board for the assessment of standing balance in Parkinson's disease.

Science.gov (United States)

Holmes, Jeffrey D; Jenkins, Mary E; Johnson, Andrew M; Hunt, Michael A; Clark, Ross A

2013-04-01

Impaired postural stability places individuals with Parkinson's at an increased risk for falls. Given the high incidence of fall-related injuries within this population, ongoing assessment of postural stability is important. To evaluate the validity of the Nintendo Wii(®) balance board as a measurement tool for the assessment of postural stability in individuals with Parkinson's. Twenty individuals with Parkinson's participated. Subjects completed testing on two balance tasks with eyes open and closed on a Wii(®) balance board and biomechanical force platform. Bland-Altman plots and a two-way, random-effects, single measure intraclass correlation coefficient model were used to assess concurrent validity of centre-of-pressure data. Concurrent validity was demonstrated to be excellent across balance tasks (intraclass correlation coefficients = 0.96, 0.98, 0.92, 0.94). This study suggests that the Wii(®) balance board is a valid tool for the quantification of postural stability among individuals with Parkinson's.

Using qualitative methods to improve questionnaires for Spanish speakers: assessing face validity of a food behavior checklist.

Science.gov (United States)

Banna, Jinan C; Vera Becerra, Luz E; Kaiser, Lucia L; Townsend, Marilyn S

2010-01-01

Development of outcome measures relevant to health nutrition behaviors requires a rigorous process of testing and revision. Whereas researchers often report performance of quantitative data collection to assess questionnaire validity and reliability, qualitative testing procedures are often overlooked. This report outlines a procedure for assessing face validity of a Spanish-language dietary assessment tool. Reviewing the literature produced no rigorously validated Spanish-language food behavior assessment tools for the US Department of Agriculture's food assistance and education programs. In response to this need, this study evaluated the face validity of a Spanish-language food behavior checklist adapted from a 16-item English version of a food behavior checklist shown to be valid and reliable for limited-resource English speakers. The English version was translated using rigorous methods involving initial translation by one party and creation of five possible versions. Photos were modified based on client input and new photos were taken as necessary. A sample of low-income, Spanish-speaking women completed cognitive interviews (n=20). Spanish translation experts (n=7) fluent in both languages and familiar with both cultures made minor modifications but essentially approved client preferences. The resulting checklist generated a readability score of 93, indicating low reading difficulty. The Spanish-language checklist has adequate face validity in the target population and is ready for further validation using convergent measures. At the conclusion of testing, this instrument may be used to evaluate nutrition education interventions in California. These qualitative procedures provide a framework for designing evaluation tools for low-literate audiences participating in the US Department of Agriculture food assistance and education programs. Copyright 2010 American Dietetic Association. Published by Elsevier Inc. All rights reserved.

Application of parameters space analysis tools for empirical model validation

Energy Technology Data Exchange (ETDEWEB)

Paloma del Barrio, E. [LEPT-ENSAM UMR 8508, Talence (France); Guyon, G. [Electricite de France, Moret-sur-Loing (France)

2004-01-01

A new methodology for empirical model validation has been proposed in the framework of the Task 22 (Building Energy Analysis Tools) of the International Energy Agency. It involves two main steps: checking model validity and diagnosis. Both steps, as well as the underlying methods, have been presented in the first part of the paper. In this part, they are applied for testing modelling hypothesis in the framework of the thermal analysis of an actual building. Sensitivity analysis tools have been first used to identify the parts of the model that can be really tested on the available data. A preliminary diagnosis is then supplied by principal components analysis. Useful information for model behaviour improvement has been finally obtained by optimisation techniques. This example of application shows how model parameters space analysis is a powerful tool for empirical validation. In particular, diagnosis possibilities are largely increased in comparison with residuals analysis techniques. (author)

Evaluating teaching methods: validation of an evaluation tool for hydrodissection and phacoemulsification portions of cataract surgery.

Science.gov (United States)

Smith, Ronald J; McCannel, Colin A; Gordon, Lynn K; Hollander, David A; Giaconi, JoAnn A; Stelzner, Sadiqa K; Devgan, Uday; Bartlett, John; Mondino, Bartly J

2014-09-01

To develop and assess the validity of an evaluation tool to assess quantitatively the hydrodissection and phacoemulsification portions of cataract surgery performed by residents. Case series. Jules Stein Eye Institute, Olive View-UCLA Medical Center, and Veterans Administration Medical Center, Los Angeles, California, USA. The UCLA ophthalmology faculty members were surveyed and the literature was reviewed to develop a grading tool consisting of 15 questions to evaluate surgical technique, including questions from the Global Rating Assessment of Skills in Intraocular Surgery and from the International Council of Ophthalmology's Ophthalmology Surgical Competency Assessment Rubric. Video clips of the hydrodissection and phacoemulsification portions of cataract surgery performed by 1 postgraduate year 2 (PGY2) resident, 1 PGY3 resident, 2 PGY4 residents, and an advanced surgeon were independently graded in a masked fashion by an 8-member faculty panel. Eleven of the 15 questions had a significant association with surgical experience level (Pinstrument handling, flow of operation, and nucleus rotation. Nucleus cracking also had low variability. Less directly visible tasks, especially 3-dimensional tasks, had wider interobserver variability. Surgical performance can be validly measured using an evaluation tool. Improved videography and studies to identify the best questions for evaluating each step of cataract surgery may help ophthalmic educators more precisely measure training outcomes for improving teaching interventions. No author has a financial or proprietary interest in any material or method mentioned. Published by Elsevier Inc.

Am I getting an accurate picture: a tool to assess clinical handover in remote settings?

Directory of Open Access Journals (Sweden)

Malcolm Moore

2017-11-01

Full Text Available Abstract Background Good clinical handover is critical to safe medical care. Little research has investigated handover in rural settings. In a remote setting where nurses and medical students give telephone handover to an aeromedical retrieval service, we developed a tool by which the receiving clinician might assess the handover; and investigated factors impacting on the reliability and validity of that assessment. Methods Researchers consulted with clinicians to develop an assessment tool, based on the ISBAR handover framework, combining validity evidence and the existing literature. The tool was applied ‘live’ by receiving clinicians and from recorded handovers by academic assessors. The tool’s performance was analysed using generalisability theory. Receiving clinicians and assessors provided feedback. Results Reliability for assessing a call was good (G = 0.73 with 4 assessments. The scale had a single factor structure with good internal consistency (Cronbach’s alpha = 0.8. The group mean for the global score for nurses and students was 2.30 (SD 0.85 out of a maximum 3.0, with no difference between these sub-groups. Conclusions We have developed and evaluated a tool to assess high-stakes handover in a remote setting. It showed good reliability and was easy for working clinicians to use. Further investigation and use is warranted beyond this setting.

Reliability and validity assessment of gastrointestinal dystemperaments questionnaire: a novel scale in Persian traditional medicine

Science.gov (United States)

Hoseinzadeh, Hamidreza; Taghipour, Ali; Yousefi, Mahdi

2018-01-01

Background Development of a questionnaire based on the resources of Persian traditional medicine seems necessary. One of the problems faced by practitioners of traditional medicine is the different opinions regarding the diagnosis of general temperament or temperament of member. One of the reasons is the lack of validity tools, and it has led to difficulties in training the student of traditional medicine and the treatment of patients. The differences in the detection methods, have given rise to several treatment methods. Objective The present study aimed to develop a questionnaire and standard software for diagnosis of gastrointestinal dystemperaments. Methods The present research is a tool developing study which included 8 stages of developing the items, determining the statements based on items, assessing the face validity, assessing the content validity, assessing the reliability, rating the items, developing a software for calculation of the total score of the questionnaire named GDS v.1.1, and evaluating the concurrent validity using statistical tests including Cronbach’s alpha coefficient, Cohen’s kappa coefficient. Results Based on the results, 112 notes including 62 symptoms were extracted from resources, and 58 items were obtained from in-person interview sessions with a panel of experts. A statement was selected for each item and, after merging a number of statements, a total of 49 statements were finally obtained. By calculating the score of statement impact and determining the content validity, respectively, 6 and 10 other items were removed from the list of statements. Standardized Cronbach’s alpha for this questionnaire was obtained 0.795 and its concurrent validity was equal to 0.8. Conclusion A quantitative tool was developed for diagnosis and examination of gastrointestinal dystemperaments. The developed questionnaire is adequately reliable and valid for this purpose. In addition, the software can be used for clinical diagnosis. PMID

Validity and reliability of the Turkish version of the pressure ulcer prevention knowledge assessment instrument.

Science.gov (United States)

Tulek, Zeliha; Polat, Cansu; Ozkan, Ilknur; Theofanidis, Dimitris; Togrol, Rifat Erdem

2016-11-01

Sound knowledge of pressure ulcers is important to enable good prevention. There are limited instruments assessing pressure ulcer knowledge. The Pressure Ulcer Prevention Knowledge Assessment Instrument is among the scales of which psychometric properties have been studied rigorously and reflects the latest evidence. This study aimed to evaluate the validity and reliability of the Turkish version of the Pressure Ulcer Prevention Knowledge Assessment Instrument (PUPKAI-T), an instrument that assesses knowledge of pressure ulcer prevention by using multiple-choice questions. Linguistic validity was verified through front-to-back translation. Psychometric properties of the instrument were studied on a sample of 150 nurses working in a tertiary hospital in Istanbul, Turkey. The content validity index of the translated instrument was 0.94, intra-class correlation coefficients were between 0.37 and 0.80, item difficulty indices were between 0.21 and 0.88, discrimination indices were 0.20-0.78, and the Kuder Richardson for the internal consistency was 0.803. The PUPKAI-T was found to be a valid and reliable tool to evaluate nurses' knowledge on pressure ulcer prevention. The PUPKAI-T may be a useful tool for determining educational needs of nurses on pressure ulcer prevention. Copyright © 2016 Tissue Viability Society. Published by Elsevier Ltd. All rights reserved.

Validation of the National Aeronautics and Space Administration Task Load Index as a tool to evaluate the learning curve for endoscopy training.

Science.gov (United States)

Mohamed, Rachid; Raman, Maitreyi; Anderson, John; McLaughlin, Kevin; Rostom, Alaa; Coderre, Sylvain

2014-03-01

Although workplace workload assessments exist in different fields, an endoscopy-specific workload assessment tool is lacking. To validate such a workload tool and use it to map the progression of novice trainees in gastroenterology in performing their first endoscopies. The National Aeronautics and Space Administration Task Load Index (NASA-TLX) workload assessment tool was completed by eight novice trainees in gastroenterology and 10 practicing gastroenterologists⁄surgeons. An exploratory factor analysis was performed to construct a streamlined endoscopy-specific task load index, which was subsequently validated. The 'Endoscopy Task Load Index' was used to monitor progression of trainee exertion and self-assessed performance over their first 40 procedures. From the factor analysis of the NASA-TLX, two principal components emerged: a measure of exertion and a measure of self-efficacy. These items became the components of the newly validated Endoscopy Task Load Index. There was a steady decline in self-perceived exertion over the training period, which was more rapid for gastroscopy than colonoscopy. The self-efficacy scores for gastroscopy rapidly increased over the first few procedures, reaching a plateau after this period of time. For colonoscopy, there was a progressive increase in reported self-efficacy over the first three quartiles of procedures, followed by a drop in self-efficacy scores over the final quartile. The present study validated an Endoscopy Task Load Index that can be completed in <1 min. Practical implications of such a tool in endoscopy education include identifying periods of higher perceived exertion among novice endoscopists, facilitating appropriate levels of guidance from trainers.

Validation and Development of a Modified Breast Graded Prognostic Assessment As a Tool for Survival in Patients With Breast Cancer and Brain Metastases.

Science.gov (United States)

Subbiah, Ishwaria M; Lei, Xiudong; Weinberg, Jeffrey S; Sulman, Erik P; Chavez-MacGregor, Mariana; Tripathy, Debu; Gupta, Rohan; Varma, Ankur; Chouhan, Jay; Guevarra, Richard P; Valero, Vicente; Gilbert, Mark R; Gonzalez-Angulo, Ana M

2015-07-10

Several indices have been developed to predict overall survival (OS) in patients with breast cancer with brain metastases, including the breast graded prognostic assessment (breast-GPA), comprising age, tumor subtype, and Karnofsky performance score. However, number of brain metastases-a highly relevant clinical variable-is less often incorporated into the final model. We sought to validate the existing breast-GPA in an independent larger cohort and refine it integrating number of brain metastases. Data were retrospectively gathered from a prospectively maintained institutional database. Patients with newly diagnosed brain metastases from 1996 to 2013 were identified. After validating the breast-GPA, multivariable Cox regression and recursive partitioning analysis led to the development of the modified breast-GPA. The performances of the breast-GPA and modified breast-GPA were compared using the concordance index. In our cohort of 1,552 patients, the breast-GPA was validated as a prognostic tool for OS (P three v ≤ three), both were independent predictors of OS. We therefore developed the modified breast-GPA integrating a fourth clinical parameter. Recursive partitioning analysis reinforced the prognostic significance of these four factors. Concordance indices were 0.78 (95% CI, 0.77 to 0.80) and 0.84 (95% CI, 0.83 to 0.85) for the breast-GPA and modified breast-GPA, respectively (P formative part of the clinician's discussion of prognosis and direction of care and as a potential patient selection tool for clinical trials. © 2015 by American Society of Clinical Oncology.

Development and assessment of the Alberta Context Tool.

Science.gov (United States)

Estabrooks, Carole A; Squires, Janet E; Cummings, Greta G; Birdsell, Judy M; Norton, Peter G

2009-12-15

The context of healthcare organizations such as hospitals is increasingly accepted as having the potential to influence the use of new knowledge. However, the mechanisms by which the organizational context influences evidence-based practices are not well understood. Current measures of organizational context lack a theory-informed approach, lack construct clarity and generally have modest psychometric properties. This paper presents the development and initial psychometric validation of the Alberta Context Tool (ACT), an eight dimension measure of organizational context for healthcare settings. Three principles guided the development of the ACT: substantive theory, brevity, and modifiability. The Promoting Action on Research Implementation in Health Services (PARiHS) framework and related literature were used to guide selection of items in the ACT. The ACT was required to be brief enough to be tolerated in busy and resource stretched work settings and to assess concepts of organizational context that were potentially modifiable. The English version of the ACT was completed by 764 nurses (752 valid responses) working in seven Canadian pediatric care hospitals as part of its initial validation. Cronbach's alpha, exploratory factor analysis, analysis of variance, and tests of association were used to assess instrument reliability and validity. Factor analysis indicated a 13-factor solution (accounting for 59.26% of the variance in 'organizational context'). The composition of the factors was similar to those originally conceptualized. Cronbach's alpha for the 13 factors ranged from .54 to .91 with 4 factors performing below the commonly accepted alpha cut off of .70. Bivariate associations between instrumental research utilization levels (which the ACT was developed to predict) and the ACT's 13 factors were statistically significant at the 5% level for 12 of the 13 factors. Each factor also showed a trend of increasing mean score ranging from the lowest level to the

Development and assessment of the Alberta Context Tool

Directory of Open Access Journals (Sweden)

Birdsell Judy M

2009-12-01

Full Text Available Abstract Background The context of healthcare organizations such as hospitals is increasingly accepted as having the potential to influence the use of new knowledge. However, the mechanisms by which the organizational context influences evidence-based practices are not well understood. Current measures of organizational context lack a theory-informed approach, lack construct clarity and generally have modest psychometric properties. This paper presents the development and initial psychometric validation of the Alberta Context Tool (ACT, an eight dimension measure of organizational context for healthcare settings. Methods Three principles guided the development of the ACT: substantive theory, brevity, and modifiability. The Promoting Action on Research Implementation in Health Services (PARiHS framework and related literature were used to guide selection of items in the ACT. The ACT was required to be brief enough to be tolerated in busy and resource stretched work settings and to assess concepts of organizational context that were potentially modifiable. The English version of the ACT was completed by 764 nurses (752 valid responses working in seven Canadian pediatric care hospitals as part of its initial validation. Cronbach's alpha, exploratory factor analysis, analysis of variance, and tests of association were used to assess instrument reliability and validity. Results Factor analysis indicated a 13-factor solution (accounting for 59.26% of the variance in 'organizational context'. The composition of the factors was similar to those originally conceptualized. Cronbach's alpha for the 13 factors ranged from .54 to .91 with 4 factors performing below the commonly accepted alpha cut off of .70. Bivariate associations between instrumental research utilization levels (which the ACT was developed to predict and the ACT's 13 factors were statistically significant at the 5% level for 12 of the 13 factors. Each factor also showed a trend of

The Hamburg-Hannover Agitation Scale (H2A): Development and validation of a self-assessment tool for symptoms of agitation.

Science.gov (United States)

Jung, Stefanie; Wollmer, M Axel; Kruger, Tillmann H C

2015-10-01

Agitation has long been underestimated as a symptom occurring across psychiatric disorders. While several instruments exist for highly specific clinical target groups (e.g., dementia, traumatic brain injury), no tool captures agitation in a broader range of psychiatric patients. The Hamburg-Hannover Agitation Scale (H2A) has been designed to satisfy this demand. This study concentrated on the development and validation of the scale in a psychiatric and a healthy control sample. The H2A was developed, tested in an expert sample, and revised. The German version was validated in a study involving two clinical institutions. Patients (n = 180) completed the H2A and several other questionnaires in order to test for congruent and discriminant validity. Healthy subjects (n = 685) completed the H2A only. The H2A was translated into English. The H2A showed very satisfying quality criteria (reliability, selectivity, item difficulty) and regression analysis demonstrated the H2A's ability to distinguish between subjects with a psychiatric diagnosis and healthy subjects with or without psychiatric record. Factor analysis revealed a three-factorial structure representing a physiological/somatic, a mental and a mixed ('psychophysiological') dimension of agitation. Although validation showed promising quality criteria and predictive value of the H2A, calibration tests with bigger and more balanced sample sizes are necessary. Agitation has become more clinically relevant as a symptom occurring in various affective disorders, yet its assessment is limited. The H2A was developed in order to meet this need. Validation of the H2A revealed very satisfactory item and scale quality criteria promoting its utility. Copyright © 2015. Published by Elsevier Ltd.

An Approach for the Development and Implementation of an Assessment Tool for Interprofessional Education Learning Activities

Directory of Open Access Journals (Sweden)

Lisa Salvati

2017-10-01

Full Text Available The Accreditation Council for Pharmacy Education Standards 2016 state that colleges of pharmacy must assess student achievement and readiness to contribute as a member of an interprofessional collaborative patient care team. There are a limited number of assessment tools available to achieve this part of the Standards. The purpose of this Case Study Report is to describe the process that one college of pharmacy took to develop an interprofessional education (IPE assessment tool to be used for their longitudinal assessment approach for IPE in the didactic portion of the curriculum. Strategies for the development of an assessment tool are provided through three themes: continuous refinement, collaboration and streamlining. Next steps for the implementation of the assessment tool, as well as evaluating its validity and reliability, are discussed.   Type: Case Study

Developing an Assessment (Tool) for Touch Screen Devices.

Science.gov (United States)

Danial-Saad, Alexandra; Chiari, Lorenzo

2017-01-01

Touch screen devices have become prevalent in our lives. Assistive technology experts working with people with disabilities face difficulty in understanding and assessing the problems experienced by individuals with disabilities in operating touch screen devices. This paper presents the processes of collecting and creating the required knowledge needed for assessing the user's skills for operating various touch screen devices, in order to develop an application for assessing the user's abilities and limitations. A six step procedure was used to collect and validate the required knowledge for the assessment from a multidisciplinary team. To determine the agreement levels between the experts, content validity was calculated. To test correlation between the experts from the different disciplines, a comparison was made between the discipline groups and their choice of specific skills/measurements. The final number of domains and skills/measurements was 15 domains and 50 skills/measurements. The result of Cronbach's α test for the final assessment questionnaire (50 skills/measurements) was 0.94, which indicates a high degree of reliability. The results of Kruskal-Wallis test showed the lack of any significant difference between agreements of the clinicians and the technicians groups, but significant differences were found between the educators and the clinicians groups. Each of the skills appearing in the final questionnaire was illustrated in a flowchart in preparation for developing the assessment (tool) for using touch screen devices.

What Parents Really Think about Their Feeding Practices and Behaviors: Lessons Learned from the Development of a Parental Feeding Assessment Tool

Directory of Open Access Journals (Sweden)

Stephanie L. Sitnick

2014-06-01

Full Text Available Interest in the role that parenting assumes in child obesity has increased the need for valid and reliable screening tools that are specific for populations targeted by programming efforts. While low-income families comprise a large audience for Cooperative Extension obesity prevention programs, valid and reliable self-administered parenting assessments for this population are lacking. Development of such tools requires understanding low-income parents’ interpretations of questions related to their parenting. The current paper reports on interviews conducted with low-income parents (N = 44 of 3- to 5-year-old children during the development of a tool to assess parenting in the context of feeding. Interviews revealed areas of potential discrepancy between parents’ and researchers’ interpretations of items that may affect parents’ responses and subsequent measurement validity when used in Cooperative Extension community intervention setting. Three themes emerged that may interfere with valid and reliable assessments of constructs: fear of being labeled a “harsh parent,” response bias due to previous knowledge, and discrepancy in interpretation of the intended construct. Results highlight complexities of constructing parent-report assessments of parenting for low-income audiences, and potential hazards of using research-focused tools with high respondent burden. Guidelines for educators assessing parents’ feeding behaviors are presented.

Design and validation of a questionnaire to assess organizational culture in French hospital wards.

Science.gov (United States)

Saillour-Glénisson, F; Domecq, S; Kret, M; Sibe, M; Dumond, J P; Michel, P

2016-09-17

Although many organizational culture questionnaires have been developed, there is a lack of any validated multidimensional questionnaire assessing organizational culture at hospital ward level and adapted to health care context. Facing the lack of an appropriate tool, a multidisciplinary team designed and validated a dimensional organizational culture questionnaire for healthcare settings to be administered at ward level. A database of organizational culture items and themes was created after extensive literature review. Items were regrouped into dimensions and subdimensions (classification validated by experts). Pre-test and face validation was conducted with 15 health care professionals. In a stratified cluster random sample of hospitals, the psychometric validation was conducted in three phases on a sample of 859 healthcare professionals from 36 multidisciplinary medicine services: 1) the exploratory phase included a description of responses' saturation levels, factor and correlations analyses and an internal consistency analysis (Cronbach's alpha coefficient); 2) confirmatory phase used the Structural Equation Modeling (SEM); 3) reproducibility was studied by a test-retest. The overall response rate was 80 %; the completion average was 97 %. The metrological results were: a global Cronbach's alpha coefficient of 0.93, higher than 0.70 for 12 sub-dimensions; all Dillon-Goldstein's rho coefficients higher than 0.70; an excellent quality of external model with a Goodness of Fitness (GoF) criterion of 0.99. Seventy percent of the items had a reproducibility ranging from moderate (Intra-Class Coefficient between 50 and 70 % for 25 items) to good (ICC higher than 70 % for 33 items). COMEt (Contexte Organisationnel et Managérial en Etablissement de Santé) questionnaire is a validated multidimensional organizational culture questionnaire made of 6 dimensions, 21 sub-dimensions and 83 items. It is the first dimensional organizational culture questionnaire

Feasibility, reliability, and validity of a smartphone based application for the assessment of cognitive function in the elderly.

Science.gov (United States)

Brouillette, Robert M; Foil, Heather; Fontenot, Stephanie; Correro, Anthony; Allen, Ray; Martin, Corby K; Bruce-Keller, Annadora J; Keller, Jeffrey N

2013-01-01

While considerable knowledge has been gained through the use of established cognitive and motor assessment tools, there is a considerable interest and need for the development of a battery of reliable and validated assessment tools that provide real-time and remote analysis of cognitive and motor function in the elderly. Smartphones appear to be an obvious choice for the development of these "next-generation" assessment tools for geriatric research, although to date no studies have reported on the use of smartphone-based applications for the study of cognition in the elderly. The primary focus of the current study was to assess the feasibility, reliability, and validity of a smartphone-based application for the assessment of cognitive function in the elderly. A total of 57 non-demented elderly individuals were administered a newly developed smartphone application-based Color-Shape Test (CST) in order to determine its utility in measuring cognitive processing speed in the elderly. Validity of this novel cognitive task was assessed by correlating performance on the CST with scores on widely accepted assessments of cognitive function. Scores on the CST were significantly correlated with global cognition (Mini-Mental State Exam: r = 0.515, psmartphone-based application for the purpose of assessing cognitive function in the elderly. The importance of these findings for the establishment of smartphone-based assessment batteries of cognitive and motor function in the elderly is discussed.

Development of a music therapy assessment tool for patients in low awareness states.

Science.gov (United States)

Magee, Wendy L

2007-01-01

People in low awareness states following profound brain injury typically demonstrate subtle changes in functional behaviors which challenge the sensitivity of measurement tools. Failure to identify and measure changes in functioning can lead to misdiagnosis and withdrawal of treatment with this population. Thus, the development of tools which are sensitive to responsiveness is of central concern. As the auditory modality has been found to be particularly sensitive in identifying responses indicating awareness, a convincing case can be made for music therapy as a treatment medium. However, little has been recommended about protocols for intervention or tools for measuring patient responses within the music therapy setting. This paper presents the rationale for an assessment tool specifically designed to measure responses in the music therapy setting with patients who are diagnosed as minimally conscious or in a vegetative state. Developed over fourteen years as part of interdisciplinary assessment and treatment, the music therapy assessment tool for low awareness states (MATLAS) contains fourteen items which rate behavioral responses across a number of domains. The tool can provide important information for interdisciplinary assessment and treatment particularly in the auditory and communication domains. Recommendations are made for testing its reliability and validity through research.

Development and validation of the mindfulness-based interventions - teaching assessment criteria (MBI:TAC).

Science.gov (United States)

Crane, Rebecca S; Eames, Catrin; Kuyken, Willem; Hastings, Richard P; Williams, J Mark G; Bartley, Trish; Evans, Alison; Silverton, Sara; Soulsby, Judith G; Surawy, Christina

2013-12-01

The assessment of intervention integrity is essential in psychotherapeutic intervention outcome research and psychotherapist training. There has been little attention given to it in mindfulness-based interventions research, training programs, and practice. To address this, the Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC) was developed. This article describes the MBI:TAC and its development and presents initial data on reliability and validity. Sixteen assessors from three centers evaluated teaching integrity of 43 teachers using the MBI:TAC. Internal consistency (α = .94) and interrater reliability (overall intraclass correlation coefficient = .81; range = .60-.81) were high. Face and content validity were established through the MBI:TAC development process. Data on construct validity were acceptable. Initial data indicate that the MBI:TAC is a reliable and valid tool. It can be used in Mindfulness-Based Stress Reduction/Mindfulness-Based Cognitive Therapy outcome evaluation research, training and pragmatic practice settings, and in research to assess the impact of teaching integrity on participant outcome.

Critical thinking skills in midwifery practice: Development of a self-assessment tool for students.

Science.gov (United States)

Carter, Amanda G; Creedy, Debra K; Sidebotham, Mary

2017-07-01

Develop and test a tool designed for use by pre-registration midwifery students to self-appraise their critical thinking in practice. A descriptive cohort design was used. All students (n=164) enrolled in a three-year Bachelor of Midwifery program in Queensland, Australia. The staged model for tool development involved item generation, mapping draft items to critical thinking concepts and expert review to test content validity, pilot testing of the tool to a convenience sample of students, and psychometric testing. Students (n=126, 76.8% response rate) provided demographic details, completed the new tool, and five questions from the Motivated Strategies for Learning Questionnaire (MSLQ) via an online platform or paper version. A high content validity index score of 0.97 was achieved through expert review. Construct validity via factor analysis revealed four factors: seeks information, reflects on practice, facilitates shared decision making, and evaluates practice. The mean total score for the tool was 124.98 (SD=12.58). Total and subscale scores correlated significantly. The scale achieved good internal reliability with a Cronbach's alpha coefficient of 0.92. Concurrent validity with the MSLQ subscale was 0.35 (pcritical thinking in practice. Critical thinking skills are vital for safe and effective midwifery practice. Students' assessment of their critical thinking development throughout their pre-registration programme makes these skills explicit, and could guide teaching innovation to address identified deficits. The availability of a reliable and valid tool assists research into the development of critical thinking in education and practice. Crown Copyright © 2017. Published by Elsevier Ltd. All rights reserved.

Fish habitat simulation models and integrated assessment tools

International Nuclear Information System (INIS)

Harby, A.; Alfredsen, K.

1999-01-01

Because of human development water use increases in importance, and this worldwide trend is leading to an increasing number of user conflicts with a strong need for assessment tools to measure the impacts both on the ecosystem and the different users and user groups. The quantitative tools must allow a comparison of alternatives, different user groups, etc., and the tools must be integrated while impact assessments includes different disciplines. Fish species, especially young ones, are indicators of the environmental state of a riverine system and monitoring them is a way to follow environmental changes. The direct and indirect impacts on the ecosystem itself are measured, and impacts on user groups is not included. Fish habitat simulation models are concentrated on, and methods and examples are considered from Norway. Some ideas on integrated modelling tools for impact assessment studies are included. One dimensional hydraulic models are rapidly calibrated and do not require any expert knowledge in hydraulics. Two and three dimensional models require a bit more skilled users, especially if the topography is very heterogeneous. The advantages of using two and three dimensional models include: they do not need any calibration, just validation; they are predictive; and they can be more cost effective than traditional habitat hydraulic models when combined with modern data acquisition systems and tailored in a multi-disciplinary study. Suitable modelling model choice should be based on available data and possible data acquisition, available manpower, computer, and software resources, and needed output and accuracy in the output. 58 refs

The Dutch motor skills assessment as tool for talent development in table tennis: a reproducibility and validity study.

Science.gov (United States)

Faber, Irene R; Nijhuis-Van Der Sanden, Maria W G; Elferink-Gemser, Marije T; Oosterveld, Frits G J

2015-01-01

A motor skills assessment could be helpful in talent development by estimating essential perceptuo-motor skills of young players, which are considered requisite to develop excellent technical and tactical qualities. The Netherlands Table Tennis Association uses a motor skills assessment in their talent development programme consisting of eight items measuring perceptuo-motor skills specific to table tennis under varying conditions. This study aimed to investigate this assessment regarding its reproducibility, internal consistency, underlying dimensions and concurrent validity in 113 young table tennis players (6-10 years). Intraclass correlation coefficients of six test items met the criteria of 0.7 with coefficients of variation between 3% and 8%. Cronbach's alpha valued 0.853 for internal consistency. The principal components analysis distinguished two conceptually meaningful factors: "ball control" and "gross motor function." Concurrent validity analyses demonstrated moderate associations between the motor skills assessment's results and national ranking; boys r = -0.53 (P motor skills assessment seems to be a reproducible, objective part of a talent development programme, more longitudinal studies are required to investigate its predictive validity.

Validation of a motivation survey tool for pharmacy students: Exploring a link to professional identity development.

Science.gov (United States)

Mylrea, Martina F; Sen Gupta, Tarun; Glass, Beverley D

2017-09-01

Self-determination theory (SDT), which describes a continuum of motivation regulators, is proposed as an appropriate framework to study pharmacy student motivation. The aim was to develop a Pharmacy Motivation Scale (Pharm-S) to determine motivation regulators in undergraduate students and explore a possible link to professional identity development. The Pharm-S was adapted from the SDT-based, Sports Motivation Scale (SMS-II), and administered to undergraduate students in an Australian pharmacy course. Convergent validity was assessed by conducting a correlation analysis between the Pharm-S and MacLeod Clark Professional Identity Scale (MCPIS-9). Face, content and construct validity were established for the Pharm-S through the analysis of 327 survey responses. Factor analysis extracted four of the six theoretical subscales as proposed by SDT (variance explained: 65.7%). Support for the SDT structure was confirmed by high factor loadings in each of the subscales and acceptable reliability coefficients. Subscale correlations revealed a simplex pattern, supporting the presence of a motivation continuum, as described by SDT. A moderate positive correlation (0.64) between Pharm-S responses and the validated professional identity instrument, MCPIS-9, indicated a possible link between levels of motivation and professional identity. and conclusions: Content and structural validity and internal consistency of the Pharm-S confirmed the reliability of the Pharm-S as a valid tool to assess motivational regulators. Pharm-S and the MCPIS-9 were positively correlated, lending support to a link between motivation and professional identity. This suggests a potential role for the Pharm-S as a valid tool to measure pharmacy student professional identity development. Crown Copyright © 2017. Published by Elsevier Inc. All rights reserved.

Validity and Reproducibility of the Iodine Dietary Intake Questionnaire Assessment Conducted for Young Polish Women

Science.gov (United States)

Malowaniec, Ewa

2017-01-01

The aim of this study was to analyse a designed brief iodine dietary intake questionnaire based on a food frequency assessment (IOdine Dietary INtake Evaluation-Food Frequency Questionnaire—IODINE-FFQ), including the assessment of validity and reproducibility in a group of 90 Polish women aged 20–35 years. Participants collected 3-day dietary records and filled in the IODINE-FFQ twice (FFQ1—directly after the dietary record and FFQ2—6 weeks later). The analysis included an assessment of validity (comparison with the results of the 3-day dietary record) and of reproducibility (comparison of the results obtained twice—FFQ1 and FFQ2). In the analysis of validity, a Bland-Altman index of 5.5% and 4.4% was recorded, respectively for FFQ1 and FFQ2. In the analysis of reproducibility it was 6.7%, but the share of individuals correctly classified into tertiles was over 70% (weighted κ of 0.675). It was stated, that assessment of IODINE-FFQ revealed a satisfactory level of validity and reproducibility in the analysis of Bland-Alman plot. The IODINE-FFQ may be indicated as a tool for the assessment of iodine intake in the young women in Poland, however further studies should be considered in order to obtain the practical tool for public health specialists. Due to the lack of validated iodine-specific food frequency questionnaires for countries of Eastern Europe, the IODINE-FFQ may be adjusted for courtiers other than Poland including iodine-fortified products. PMID:28661461

A Prospective Validation Study of a Rainbow Model of Integrated Care Measurement Tool in Singapore.

Science.gov (United States)

Nurjono, Milawaty; Valentijn, Pim P; Bautista, Mary Ann C; Wei, Lim Yee; Vrijhoef, Hubertus Johannes Maria

2016-04-08

The conceptual ambiguity of the integrated care concept precludes a full understanding of what constitutes a well-integrated health system, posing a significant challenge in measuring the level of integrated care. Most available measures have been developed from a disease-specific perspective and only measure certain aspects of integrated care. Based on the Rainbow Model of Integrated Care, which provides a detailed description of the complex concept of integrated care, a measurement tool has been developed to assess integrated care within a care system as a whole gathered from healthcare providers' and managerial perspectives. This paper describes the methodology of a study seeking to validate the Rainbow Model of Integrated Care measurement tool within and across the Singapore Regional Health System. The Singapore Regional Health System is a recent national strategy developed to provide a better-integrated health system to deliver seamless and person-focused care to patients through a network of providers within a specified geographical region. The validation process includes the assessment of the content of the measure and its psychometric properties. If the measure is deemed to be valid, the study will provide the first opportunity to measure integrated care within Singapore Regional Health System with the results allowing insights in making recommendations for improving the Regional Health System and supporting international comparison.

A new assessment tool for patients with multiple sclerosis from Spanish-speaking countries: validation of the Brief International Cognitive Assessment for MS (BICAMS) in Argentina.

Science.gov (United States)

Vanotti, Sandra; Smerbeck, Audrey; Benedict, Ralph H B; Caceres, Fernando

2016-10-01

The Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) is an international assessment tool for monitoring cognitive function in multiple sclerosis (MS) patients. BICAMS comprises the Symbol Digit Modalities Test (SDMT), the California Verbal Learning Test - Second Edition (CVLT II) and the Brief Visuospatial Memory Test - Revised (BVMT-R). Our objective was to validate and assess the reliability of BICAMS as applied in Argentina and to obtain normative data in Spanish for this population. The sample composed of 50 MS patients and 100 healthy controls (HC). In order to test its reliability, BICAMS was re-administered in a subset of 25 patients. The sample's average age was 43.42 ± 10.17 years old, and average years of schooling were 14.86 ± 2.78. About 74% of the participants were women. The groups did not differ in age, years of schooling, or gender. The MS group performed significantly worse than the HC group across the three neuropsychological tests, yielding the following Cohen's d values: SDMT: .85; CVLT I: .87; and BVMT-R: .40. The mean raw scores for Argentina normative data were as follows: SDMT: 56.71 ± 10.85; CVLT I: 60.88 ± 10.46; and BVMT-R: 23.44 ± 5.84. Finally, test-retest reliability coefficients for each test were as follows: SDMT: r = .95; CVLT I: r = .87; and BVMT-R: r = .82. This BICAMS version is reliable and useful as a monitoring tool for identifying MS patients with cognitive impairment.

Development, validation and psychometric properties of a diagnostic/prognostic tool for breakthrough pain in mixed chronic-pain patients.

Science.gov (United States)

Samolsky Dekel, Boaz Gedaliahu; Remondini, Francesca; Gori, Alberto; Vasarri, Alessio; Di Nino, GianFranco; Melotti, Rita Maria

2016-02-01

Breakthrough pain (BTP) shows variable prevalence in different clinical contexts of cancer and non-cancer patients. BTP diagnostic tools with demonstrated reliability, validation and prognostic capability are lacking. We report the development, psychometric and validation properties of a diagnostic/prognostic tool, the IQ-BTP, for BTP recognition, its likelihood and clinical features among chronic-pain (CP) patients. n=120 consecutive mixed cancer/non-cancer CP in/outpatients. Development, psychometric analyses and formal validation included: Face/Content validity (by 'experts' opinion and assessing the relationship between the IQ-BTP classes and criteria derived from BTP operational-case-definition); Construct validity, by Principle Component Analysis (PCA); and the strength of Spearman correlation between IQ-BTP classes and the Brief Pain Inventory (BPI) items; Reliability, by Cronbach's alpha statistics. Associations with clinical/demographic moderators were assessed applying χ(2) analysis. Potential-BTP was found in 36.7% of patients (38.4% of non-cancer and 32.4% of cancer patients). Among these the likelihood for BTP diagnosis was 'high' in 25%, 'intermediate' in 41% and, 'low' 34% of patients. Analyses showed significant differences between IQ-BTP classes and between the latter BPI pain-item scores. Correlation between IQ-BTP classes and BPI items was moderate. PCA and scree test identified 3 components accounting for 62.3% of the variance. Cronbach's alpha was 0.71. The IQ-BTP showed satisfactory psychometric and validation properties. With adequate feasibility it enabled the allocating of cancer/non-cancer CP patients in three prognostic classes. Results are sufficient to warrant a subsequent impact study of the IQ-BTP as prognostic model and screening tool for BTP in both CP populations. Copyright © 2016 Elsevier B.V. All rights reserved.

Validation of equilibrium tools on the COMPASS tokamak

Energy Technology Data Exchange (ETDEWEB)

Urban, J., E-mail: urban@ipp.cas.cz [Institute of Plasma Physics ASCR, Za Slovankou 3, 182 00 Praha 8 (Czech Republic); Appel, L.C. [CCFE, Culham Science Centre, Abingdon, Oxfordshire (United Kingdom); Artaud, J.F. [CEA, IRFM, F-13108 Saint Paul Lez Durance (France); Faugeras, B. [Laboratoire J.A. Dieudonné, UMR 7351, Université de Nice Sophia-Antipolis, Parc Valrose, 06108 Nice Cedex 02 (France); Havlicek, J. [Institute of Plasma Physics ASCR, Za Slovankou 3, 182 00 Praha 8 (Czech Republic); Department of Surface and Plasma Science, Faculty of Mathematics and Physics, Charles University in Prague, V Holešovičkách 2, 180 00 Praha 8 (Czech Republic); Komm, M. [Institute of Plasma Physics ASCR, Za Slovankou 3, 182 00 Praha 8 (Czech Republic); Lupelli, I. [CCFE, Culham Science Centre, Abingdon, Oxfordshire (United Kingdom); Peterka, M. [Institute of Plasma Physics ASCR, Za Slovankou 3, 182 00 Praha 8 (Czech Republic); Department of Surface and Plasma Science, Faculty of Mathematics and Physics, Charles University in Prague, V Holešovičkách 2, 180 00 Praha 8 (Czech Republic)

2015-10-15

Highlights: • Three equilibrium codes—EFIT++, FREEBIE and VacTH—have been successfully set up and validated on COMPASS. • FREEBIE can predictively calculate the equilibrium and corresponding poloidal field coil currents. • EFIT++ can reconstruct equilibria generated by FREEBIE from synthetic, optionally noisy diagnostic data. • VacTH is a promising tool for real time plasma shape reconstruction. • Optimized parameters are estimated for EFIT++ and VacTH by a statistical analysis. - Abstract: Various MHD (magnetohydrodynamic) equilibrium tools, some of which being recently developed or considerably updated, are used on the COMPASS tokamak at IPP Prague. MHD equilibrium is a fundamental property of the tokamak plasma, whose knowledge is required for many diagnostics and modelling tools. Proper benchmarking and validation of equilibrium tools is thus key for interpreting and planning tokamak experiments. We present here benchmarks and comparisons to experimental data of the EFIT++ reconstruction code (Appel et al., 2006), the free-boundary equilibrium code FREEBIE (Artaud and Kim, 2012), and a rapid plasma boundary reconstruction code VacTH (Faugeras et al., 2014). We demonstrate that FREEBIE can calculate the equilibrium and corresponding poloidal field (PF) coils currents consistently with EFIT++ reconstructions from experimental data. Both EFIT++ and VacTH can reconstruct equilibria generated by FREEBIE from synthetic, optionally noisy diagnostic data. Hence, VacTH is suitable for real-time control. Optimum reconstruction parameters are estimated.

Validation of the MASK-rhinitis visual analogue scale on smartphone screens to assess allergic rhinitis control

NARCIS (Netherlands)

Caimmi, D.; Baiz, N.; Tanno, L. K.; Demoly, P.; Arnavielhe, S.; Murray, R.; Bedbrook, A.; Bergmann, K. C.; de Vries, G.; Fokkens, W. J.; Fonseca, J.; Haahtela, T.; Keil, T.; Kuna, P.; Mullol, J.; Papadopoulos, N.; Passalacqua, G.; Samolinski, B.; Tomazic, P. V.; Valiulis, A.; van Eerd, M.; Wickman, M.; Annesi-Maesano, I.; Bousquet, J.; Agache, I.; Angles, R.; Anto, J. M.; Asayag, E.; Bacci, E.; Bachert, C.; Baroni, I.; Barreto, B. A.; Bedolla-Barajas, M.; Bertorello, L.; Bewick, M.; Bieber, T.; Birov, S.; Bindslev-Jensen, C.; Blua, A.; Bochenska Marciniak, M.; Bogus-Buczynska, I.; Bosnic-Ancevich, S.; Bosse, I.; Bourret, R.; Bucca, C.; Buonaiuto, R.; Caiazza, D.; Caillot, D.; Caimmi, D. P.; Camargos, P.; Canfora, G.; Cardona, V.; Carriazo, A. M.; Cartier, C.; Castellano, G.; Chavannes, N. H.; Ciaravolo, M. M.; Cingi, C.; Ciceran, A.; Colas, L.; Colgan, E.; Coll, J.; Conforti, D.; Correira de Sousa, J.; Cortés-Grimaldo, R. M.; Corti, F.; Costa, E.; Courbis, A. L.; Cruz, A.; Custovic, A.; Dario, C.; da Silva, M.; Dauvilliers, Y.; de Blay, F.; Dedeu, T.; de Feo, G.; de Martino, B.; Di Capua, S.; Di Carluccio, N.; Dray, G.; Dubakiene, R.; Eller, E.; Emuzyte, R.; Espinoza-Contreras, J. M.; Estrada-Cardona, A.; Farrell, J.; Ferrero, J.; Fontaine, J. F.; Forti, S.; Gálvez-Romero, J. L.; Garcia Cruz, M. H.; García-Cobas, C. I.; Gemicioğlu, B.; Gerth van Wijck, R.; Guidacci, M.; Gómez-Vera, J.; Guldemond, N. A.; Gutter, Z.; Hajjam, J.; Hellings, P.; Hernández-Velázquez, L.; Illario, M.; Ivancevich, J. C.; Jares, E.; Joos, G.; Just, J.; Kalayci, O.; Kalyoncu, A. F.; Karjalainen, J.; Khaltaev, N.; Klimek, L.; Kull, I.; Kuna, T. P.; Kvedariene, V.; Kolek, V.; Krzych-Fałta, E.; Kupczyk, M.; Lacwik, P.; Larenas-Linnemann, D.; Laune, D.; Lauri, D.; Lavrut, J.; Lessa, M.; Levato, G.; Lewis, L.; Lieten, I.; Lipiec, A.; Louis, R.; Luna-Pech, J. A.; Magnan, A.; Malva, J.; Maspero, J. F.; Mayora, O.; Medina-Ávalos, M. A.; Melen, E.; Menditto, E.; Millot-Keurinck, J.; Moda, G.; Morais-Almeida, M.; Mösges, R.; Mota-Pinto, A.; Muraro, A.; Noguès, M.; Nalin, M.; Napoli, L.; Neffen, H.; O'Hehir, R.; Olivé Elias, M.; Onorato, G.; Palkonen, S.; Pépin, J. L.; Pereira, A. M.; Persico, M.; Pfaar, O.; Pozzi, A. C.; Prokopakis, E. P.; Raciborski, F.; Rizzo, J. A.; Robalo-Cordeiro, C.; Rodríguez-González, M.; Rolla, G.; Roller-Wirnsberger, R. E.; Romano, A.; Romano, M.; Salimäki, J.; Serpa, F. S.; Shamai, S.; Sierra, M.; Sova, M.; Sorlini, M.; Stellato, C.; Stelmach, R.; Strandberg, T.; Stroetman, V.; Stukas, R.; Szylling, A.; Tibaldi, V.; Todo-Bom, A.; Toppila-Salmi, S.; Tomazic, P.; Trama, U.; Triggiani, M.; Valero, A.; Valovirta, E.; Vasankari, T.; Vatrella, A.; Ventura, M. T.; Verissimo, M. T.; Viart, F.; Williams, S.; Wagenmann, M.; Wanscher, C.; Westman, M.; Young, I.; Yorgancioglu, A.; Zernotti, E.; Zurbierber, T.; Zurkuhlen, A.; de Oliviera, B.; Senn, A.

2017-01-01

Background: Visual Analogue Scale (VAS) is a validated tool to assess control in allergic rhinitis patients. Objective: The aim of this study was to validate the use of VAS in the MASK-rhinitis (MACVIA-ARIA Sentinel NetworK for allergic rhinitis) app (Allergy Diary) on smartphones screens to

Validation of the National Aeronautics and Space Administration Task Load Index as a tool to evaluate-the learning curve for endoscopy training

Science.gov (United States)

Mohamed, Rachid; Raman, Maitreyi; Anderson, John; McLaughlin, Kevin; Rostom, Alaa; Coderre, Sylvain

2014-01-01

BACKGROUND: Although workplace workload assessments exist in different fields, an endoscopy-specific workload assessment tool is lacking. OBJECTIVE: To validate such a workload tool and use it to map the progression of novice trainees in gastroenterology in performing their first endoscopies. METHODS: The National Aeronautics and Space Administration Task Load Index (NASA-TLX) workload assessment tool was completed by eight novice trainees in gastroenterology and 10 practicing gastroenterologists/surgeons. An exploratory factor analysis was performed to construct a streamlined endoscopy-specific task load index, which was subsequently validated. The ‘Endoscopy Task Load Index’ was used to monitor progression of trainee exertion and self-assessed performance over their first 40 procedures. RESULTS: From the factor analysis of the NASA-TLX, two principal components emerged: a measure of exertion and a measure of self-efficacy. These items became the components of the newly validated Endoscopy Task Load Index. There was a steady decline in self-perceived exertion over the training period, which was more rapid for gastroscopy than colonoscopy. The self-efficacy scores for gastroscopy rapidly increased over the first few procedures, reaching a plateau after this period of time. For colonoscopy, there was a progressive increase in reported self-efficacy over the first three quartiles of procedures, followed by a drop in self-efficacy scores over the final quartile. DISCUSSION: The present study validated an Endoscopy Task Load Index that can be completed in <1 min. Practical implications of such a tool in endoscopy education include identifying periods of higher perceived exertion among novice endoscopists, facilitating appropriate levels of guidance from trainers. PMID:24619638

Validity of Dietary Assessment in Athletes: A Systematic Review

Directory of Open Access Journals (Sweden)

Louise Capling

2017-12-01

Full Text Available Dietary assessment methods that are recognized as appropriate for the general population are usually applied in a similar manner to athletes, despite the knowledge that sport-specific factors can complicate assessment and impact accuracy in unique ways. As dietary assessment methods are used extensively within the field of sports nutrition, there is concern the validity of methodologies have not undergone more rigorous evaluation in this unique population sub-group. The purpose of this systematic review was to compare two or more methods of dietary assessment, including dietary intake measured against biomarkers or reference measures of energy expenditure, in athletes. Six electronic databases were searched for English-language, full-text articles published from January 1980 until June 2016. The search strategy combined the following keywords: diet, nutrition assessment, athlete, and validity; where the following outcomes are reported but not limited to: energy intake, macro and/or micronutrient intake, food intake, nutritional adequacy, diet quality, or nutritional status. Meta-analysis was performed on studies with sufficient methodological similarity, with between-group standardized mean differences (or effect size and 95% confidence intervals (CI being calculated. Of the 1624 studies identified, 18 were eligible for inclusion. Studies comparing self-reported energy intake (EI to energy expenditure assessed via doubly labelled water were grouped for comparison (n = 11 and demonstrated mean EI was under-estimated by 19% (−2793 ± 1134 kJ/day. Meta-analysis revealed a large pooled effect size of −1.006 (95% CI: −1.3 to −0.7; p < 0.001. The remaining studies (n = 7 compared a new dietary tool or instrument to a reference method(s (e.g., food record, 24-h dietary recall, biomarker as part of a validation study. This systematic review revealed there are limited robust studies evaluating dietary assessment methods in athletes. Existing

Facial expression: An under-utilised tool for the assessment of welfare in mammals.

Science.gov (United States)

Descovich, Kris A; Wathan, Jennifer; Leach, Matthew C; Buchanan-Smith, Hannah M; Flecknell, Paul; Farningham, David; Vick, Sarah-Jane

2017-01-01

Animal welfare is a key issue for industries that use or impact upon animals. The accurate identification of welfare states is particularly relevant to the field of bioscience, where the 3Rs framework encourages refinement of experimental procedures involving animal models. The assessment and improvement of welfare states in animals depends on reliable and valid measurement tools. Behavioral measures (activity, attention, posture and vocalization) are frequently used because they are immediate and non-invasive, however no single indicator can yield a complete picture of the internal state of an animal. Facial expressions are extensively studied in humans as a measure of psychological and emotional experiences but are infrequently used in animal studies, with the exception of emerging research on pain behavior. In this review, we discuss current evidence for facial representations of underlying affective states, and how communicative or functional expressions can be useful within welfare assessments. Validated tools for measuring facial movement are outlined, and the potential of expressions as honest signals is discussed, alongside other challenges and limitations to facial expression measurement within the context of animal welfare. We conclude that facial expression determination in animals is a useful but underutilized measure that complements existing tools in the assessment of welfare.

Validity and reliability of a short questionnaire for assessing the impact of cooking skills interventions.

Science.gov (United States)

Barton, K L; Wrieden, W L; Anderson, A S

2011-12-01

Food skills programmes are widely used as a means to improve confidence in food preparation, the use of basic food skills and food selections amongst low income communities. However, the impact of such interventions are rarely evaluated as a result of a lack of validated assessment tools appropriate for use within this target group. A two-page questionnaire utilising a closed-question format was designed based on key domains known to be influenced by cooking skills programmes. Content validity was assessed by a panel of public health experts and face validity by individuals, typical of those who may attend cooking skills classes. Internal and repeat reliability were assessed with groups of adults attending community-based classes. The feasibility of using the tool in community settings was also assessed. The draft questionnaire was amended as appropriate subsequent to content and face validity testing. Cronbach's alpha for confidence and knowledge sections was 0.86 and 0.84, respectively, indicating good internal consistency. Spearman correlation coefficients for repeat reliability testing between time 1 and time 2 for each item were in the range 0.46-0.91 (all significant at P cooking skills interventions that could be utilised in the development and evaluation of multicentre cooking skills interventions. © 2011 The Authors. Journal of Human Nutrition and Dietetics © 2011 The British Dietetic Association Ltd.

Validation of designing tools as part of nuclear pump development process

International Nuclear Information System (INIS)

Klemm, T.; Sehr, F.; Spenner, P.; Fritz, J.

2010-01-01

Nuclear pumps are characterized by high safety standards, operational reliability as well as long life cycles. For the design process it is of common use to have a down scaled model pump to qualify operating data and simulate exceptional operating conditions. In case of modifications of the pump design compared to existing reactor coolant pumps a model pump is required to develop methods and tools to design the full scale pump. In the presented case it has a geometry scale of 1:2 regarding the full scale pump size. The experimental data of the model pump is basis for validation of methods and tools which are applied in the designing process of the full scale pump. In this paper the selection of qualified tools and the validation process is demonstrated exemplarily on a cooling circuit. The aim is to predict the resulting flow rate. Tools are chosen for different components depending on the benefit to effort ratio. For elementary flow phenomena such as fluid flow in straight pipes or gaps analytic or empirical laws can be used. For more complex flow situations numerical methods are utilized. Main focus is set on the validation process of the applied numerical flow simulation. In this case not only integral data should be compared, it is also necessary to validate local flow structure of numerical flow simulation to avoid systematic errors in CFD Model generation. Due to complex design internal flow measurements are not possible. On that reason simple comparisons of similar flow test cases are used. Results of this study show, that the flow simulation data closely match measured integral pump and test case data. With this validation it is now possible to qualify CFD simulations as a design tool for the full scale pump in similar cooling circuit. (authors)

[Emotional Intelligence Index: a tool for the routine assessment of mental health promotion programs in schools].

Science.gov (United States)

Veltro, Franco; Ialenti, Valentina; Morales García, Manuel Alejandro; Gigantesco, Antonella

2016-01-01

After critical examination of several aspects relating to the evaluation of some dimensions of emotional intelligence through self-assessment tools, is described the procedure of construction and validation of an Index for its measurement, conceived only for the routine assessment of health promotion programs mental in schools that include among their objectives the improvement of emotional intelligence specifically "outcome-oriented". On the basis of the two most common international tools, are listed 27 items plus 6 of control, illustrated two Focus Group (FG) of students (face validity). The scale obtained by FG was administered to 300 students, and the results were submitted to factorial analysis (construct validity). It was also evaluated the internal consistency with Cronbach's Alpha and studied concurrent validity with the emotional quotient inventory, a scale of perceived self-efficacy and a stress test rating. From the analysis of FG all the original items were modified, deleted 4, and reduced the encoding system from 6 to 4 levels of Likert scale. Of the 23 items included in the analysis have emerged five factors (intra-psychic dimension, interpersonal, impulsivity, adaptive coping, sense of self-efficacy) for a total of 15 items. Very satisfactory were the results of the validation process of internal consistency (0.72) and the concurrent validity. The results are positive. It is obtained in fact the shortest routine assessment tool currently available in Italy which constitutes a real Index, for which compilation are required on average 3 minutes. Is emphasized the characteristic of an Index, and not of questionnaire or interview for clinical use, highlighting the only specific use for mental health promotion programs in schools.

Using the eating disorder examination in the assessment of bulimia and anorexia: issues of reliability and validity.

Science.gov (United States)

Guest, T

2000-01-01

The Eating Disorder Examination will be assessed according to its reliability and validity in the assessment of anorexia nervosa and bulimia nervosa. A thorough review of the literature was conducted to judge the reliability and validity of the Eating Disorder Examination and its subscales. The review shows that the EDE and its subscales have good interrater reliability and internal consistency reliability. Similarly, high levels of discriminant validity, construct validity, and treatment validity in the assessment of eating disorders were also found. A summary of each study concerning the various types of reliability and validity will be provided. The EDE is considered to be the "gold standard" by which to identify eating disorders, so this tool used in conjunction with other behavioral measures will be imperative for clinical social work practice.

Development and Validation of a Standardized Tool for Prioritization of Information Sources.

Science.gov (United States)

Akwar, Holy; Kloeze, Harold; Mukhi, Shamir

2016-01-01

To validate the utility and effectiveness of a standardized tool for prioritization of information sources for early detection of diseases. The tool was developed with input from diverse public health experts garnered through survey. Ten raters used the tool to evaluate ten information sources and reliability among raters was computed. The Proc mixed procedure with random effect statement and SAS Macros were used to compute multiple raters' Fleiss Kappa agreement and Kendall's Coefficient of Concordance. Ten disparate information sources evaluated obtained the following composite scores: ProMed 91%; WAHID 90%; Eurosurv 87%; MediSys 85%; SciDaily 84%; EurekAl 83%; CSHB 78%; GermTrax 75%; Google 74%; and CBC 70%. A Fleiss Kappa agreement of 50.7% was obtained for ten information sources and 72.5% for a sub-set of five sources rated, which is substantial agreement validating the utility and effectiveness of the tool. This study validated the utility and effectiveness of a standardized criteria tool developed to prioritize information sources. The new tool was used to identify five information sources suited for use by the KIWI system in the CEZD-IIR project to improve surveillance of infectious diseases. The tool can be generalized to situations when prioritization of numerous information sources is necessary.

Validity and reliability of the Brazilian Portuguese version of the BACS (Brief Assessment of Cognition in Schizophrenia

Directory of Open Access Journals (Sweden)

Geovany Eliberto Araújo

2015-04-01

Full Text Available OBJECTIVE: To assess the validity and reliability of the Brazilian Portuguese version of the Brief Assessment of Cognition in Schizophrenia by examining its temporal stability, internal consistency, and discriminant and convergent validity. METHODS: The Brief Assessment of Cognition in Schizophrenia was administered to 116 stable patients with schizophrenia and 58 matched control subjects. To assess concurrent validity, a subset of patients underwent a traditional neuropsychological assessment. RESULTS: The patients with schizophrenia performed significantly worse than the controls (p0.8. The internal consistency of the Brief Assessment of Cognition in Schizophrenia was high (Cronbach's α ϝ 0.874. CONCLUSION: The Brazilian Portuguese version of the Brief Assessment of Cognition in Schizophrenia exhibits good reliability and discriminant and concurrent validity and is a promising tool for easily assessing cognitive impairment in schizophrenia and for comparing the performance of Brazilian patients with that of patients from other countries.

Investigating the reliability and validity of the waterlow risk assessment scale: a literature review.

LENUS (Irish Health Repository)

Walsh, Breda

2012-02-01

The aim of this review was to examine health literature on the reliability and validity of the Waterlow pressure sore assessment scale. A systematic review of published studies relating to the topic was conducted and literature was examined for its relevancy to the topic under investigation. Findings suggest that despite the availability of over 40 assessment tools, the Waterlow assessment scale is the most frequently used by health care staff. Research suggests that the Waterlow Scale is an unreliable method of assessing individuals at risk of pressure sore development with all studies indicating a poor interrater reliability status. Its validity has also been criticized because of its high-sensitivity but low-specificity levels.

Investigating the reliability and validity of the waterlow risk assessment scale: a literature review.

LENUS (Irish Health Repository)

Walsh, Breda

2011-05-01

The aim of this review was to examine health literature on the reliability and validity of the Waterlow pressure sore assessment scale. A systematic review of published studies relating to the topic was conducted and literature was examined for its relevancy to the topic under investigation. Findings suggest that despite the availability of over 40 assessment tools, the Waterlow assessment scale is the most frequently used by health care staff. Research suggests that the Waterlow Scale is an unreliable method of assessing individuals at risk of pressure sore development with all studies indicating a poor interrater reliability status. Its validity has also been criticized because of its high-sensitivity but low-specificity levels.

The validity and reliability of the portuguese versions of three tools used to diagnose delirium in critically ill patients

Directory of Open Access Journals (Sweden)

Dimitri Gusmao-Flores

2011-01-01

Full Text Available OBJECTIVES: The objectives of this study are to compare the sensitivity and specificity of three diagnostic tools for delirium (the Intensive Care Delirium Screening Checklist, the Confusion Assessment Method for Intensive Care Units and the Confusion Assessment Method for Intensive Care Units Flowsheet in a mixed population of critically ill patients, and to validate the Brazilian Portuguese Confusion Assessment Method for Intensive Care Units. METHODS: The study was conducted in four intensive care units in Brazil. Patients were screened for delirium by a psychiatrist or neurologist using the Diagnostic and Statistical Manual of Mental Disorders. Patients were subsequently screened by an intensivist using Portuguese translations of the three tools. RESULTS: One hundred and nineteen patients were evaluated and 38.6% were diagnosed with delirium by the reference rater. The Confusion Assessment Method for Intensive Care Units had a sensitivity of 72.5% and a specificity of 96.2%; the Confusion Assessment Method for Intensive Care Units Flowsheet had a sensitivity of 72.5% and a specificity of 96.2%; the Intensive Care Delirium Screening Checklist had a sensitivity of 96.0% and a specificity of 72.4%. There was strong agreement between the Confusion Assessment Method for Intensive Care Units and the Confusion Assessment Method for Intensive Care Units Flowsheet (kappa coefficient = 0.96 CONCLUSION: All three instruments are effective diagnostic tools in critically ill intensive care unit patients. In addition, the Brazilian Portuguese version of the Confusion Assessment Method for Intensive Care Units is a valid and reliable instrument for the assessment of delirium among critically ill patients.

Validation of self assessment patient knowledge questionnaire for heart failure patients.

Science.gov (United States)

Lainscak, Mitja; Keber, Irena

2005-12-01

Several studies showed insufficient knowledge and poor compliance to non-pharmacological management in heart failure patients. Only a limited number of validated tools are available to assess their knowledge. The aim of the study was to test our 10-item Patient knowledge questionnaire. The Patient knowledge questionnaire was administered to 42 heart failure patients from Heart failure clinic and to 40 heart failure patients receiving usual care. Construct validity (Pearson correlation coefficient), internal consistency (Cronbach alpha), reproducibility (Wilcoxon signed rank test), and reliability (chi-square test and Student's t-test for independent samples) were assessed. Overall score of the Patient knowledge questionnaire had the strongest correlation to the question about regular weighing (r=0.69) and the weakest to the question about presence of heart disease (r=0.33). There was a strong correlation between question about fluid retention and questions assessing regular weighing, (r=0.86), weight of one litre of water (r=0.86), and salt restriction (r=0.57). The Cronbach alpha was 0.74 and could be improved by exclusion of questions about clear explanation (Chronbach alpha 0.75), importance of fruit, soup, and vegetables (Chronbach alpha 0.75), and self adjustment of diuretic (Chronbach alpha 0.81). During reproducibility testing 91% to 98% of questions were answered equally. Patients from Heart failure clinic scored significantly better than patients receiving usual care (7.9 (1.3) vs. 5.7 (2.2), p<0.001). Patient knowledge questionnaire is a valid and reliable tool to measure knowledge of heart failure patients.

Validity in assessment of prior learning

DEFF Research Database (Denmark)

Wahlgren, Bjarne; Aarkrog, Vibe

2015-01-01

, the article discusses the need for specific criteria for assessment. The reliability and validity of the assessment procedures depend on whether the competences are well-defined, and whether the teachers are adequately trained for the assessment procedures. Keywords: assessment, prior learning, adult...... education, vocational training, lifelong learning, validity...

The ELPAT living organ donor Psychosocial Assessment Tool (EPAT): from 'what' to 'how' of psychosocial screening - a pilot study.

Science.gov (United States)

Massey, Emma K; Timmerman, Lotte; Ismail, Sohal Y; Duerinckx, Nathalie; Lopes, Alice; Maple, Hannah; Mega, Inês; Papachristou, Christina; Dobbels, Fabienne

2018-01-01

Thorough psychosocial screening of donor candidates is required in order to minimize potential negative consequences and to strive for optimal safety within living donation programmes. We aimed to develop an evidence-based tool to standardize the psychosocial screening process. Key concepts of psychosocial screening were used to structure our tool: motivation and decision-making, personal resources, psychopathology, social resources, ethical and legal factors and information and risk processing. We (i) discussed how each item per concept could be measured, (ii) reviewed and rated available validated tools, (iii) where necessary developed new items, (iv) assessed content validity and (v) pilot-tested the new items. The resulting ELPAT living organ donor Psychosocial Assessment Tool (EPAT) consists of a selection of validated questionnaires (28 items in total), a semi-structured interview (43 questions) and a Red Flag Checklist. We outline optimal procedures and conditions for implementing this tool. The EPAT and user manual are available from the authors. Use of this tool will standardize the psychosocial screening procedure ensuring that no psychosocial issues are overlooked and ensure that comparable selection criteria are used and facilitate generation of comparable psychosocial data on living donor candidates. © 2017 Steunstichting ESOT.

Psychometric properties of the Peer Proficiency Assessment (PEPA): a tool for evaluation of undergraduate peer counselors' motivational interviewing fidelity.

Science.gov (United States)

Mastroleo, Nadine R; Mallett, Kimberly A; Turrisi, Rob; Ray, Anne E

2009-09-01

Despite the expanding use of undergraduate student peer counseling interventions aimed at reducing college student drinking, few programs evaluate peer counselors' competency to conduct these interventions. The present research describes the development and psychometric assessments of the Peer Proficiency Assessment (PEPA), a new tool for examining Motivational Interviewing adherence in undergraduate student peer delivered interventions. Twenty peer delivered sessions were evaluated by master and undergraduate student coders using a cross-validation design to examine peer based alcohol intervention sessions. Assessments revealed high inter-rater reliability between student and master coders and good correlations between previously established fidelity tools. Findings lend support for the use of the PEPA to examine peer counselor competency. The PEPA, training for use, inter-rater reliability information, construct and predictive validity, and tool usefulness are described.

Validity and reliability of wii fit balance board for the assessment of balance of healthy young adults and the elderly.

Science.gov (United States)

Chang, Wen-Dien; Chang, Wan-Yi; Lee, Chia-Lun; Feng, Chi-Yen

2013-10-01

[Purpose] Balance is an integral part of human ability. The smart balance master system (SBM) is a balance test instrument with good reliability and validity, but it is expensive. Therefore, we modified a Wii Fit balance board, which is a convenient balance assessment tool, and analyzed its reliability and validity. [Subjects and Methods] We recruited 20 healthy young adults and 20 elderly people, and administered 3 balance tests. The correlation coefficient and intraclass correlation of both instruments were analyzed. [Results] There were no statistically significant differences in the 3 tests between the Wii Fit balance board and the SBM. The Wii Fit balance board had a good intraclass correlation (0.86-0.99) for the elderly people and positive correlations (r = 0.58-0.86) with the SBM. [Conclusions] The Wii Fit balance board is a balance assessment tool with good reliability and high validity for elderly people, and we recommend it as an alternative tool for assessing balance ability.

An evaluation tool for Myofascial Adhesions in Patients after Breast Cancer (MAP-BC evaluation tool): Concurrent, face and content validity.

Science.gov (United States)

De Groef, An; Van Kampen, Marijke; Moortgat, Peter; Anthonissen, Mieke; Van den Kerckhove, Eric; Christiaens, Marie-Rose; Neven, Patrick; Geraerts, Inge; Devoogdt, Nele

2018-01-01

To investigate the concurrent, face and content validity of an evaluation tool for Myofascial Adhesions in Patients after Breast Cancer (MAP-BC evaluation tool). 1) Concurrent validity of the MAP-BC evaluation tool was investigated by exploring correlations (Spearman's rank Correlation Coefficient) between the subjective scores (0 -no adhesions to 3 -very strong adhesions) of the skin level using the MAP-BC evaluation tool and objective elasticity parameters (maximal skin extension and gross elasticity) generated by the Cutometer Dual MPA 580. Nine different examination points on and around the mastectomy scar were evaluated. 2) Face and content validity were explored by questioning therapists experienced with myofascial therapy in breast cancer patients about the comprehensibility and comprehensiveness of the MAP-BC evaluation tool. 1) Only three meaningful correlations were found on the mastectomy scar. For the most lateral examination point on the mastectomy scar a moderate negative correlation (-0.44, p = 0.01) with the maximal skin extension and a moderate positive correlation with the resistance versus ability of returning or 'gross elasticity' (0.42, p = 0.02) were found. For the middle point on the mastectomy scar an almost moderate positive correlation with gross elasticity was found as well (0.38, p = 0.04) 2) Content and face validity have been found to be good. Eighty-nine percent of the respondent found the instructions understandable and 98% found the scoring system obvious. Thirty-seven percent of the therapists suggested to add the possibility to evaluate additional anatomical locations in case of reconstructive and/or bilateral surgery. The MAP-BC evaluation tool for myofascial adhesions in breast cancer patients has good face and content validity. Evidence for good concurrent validity of the skin level was found only on the mastectomy scar itself.

Data Center IT Equipment Energy Assessment Tools: Current State of Commercial Tools, Proposal for a Future Set of Assessment Tools

Energy Technology Data Exchange (ETDEWEB)

Radhakrishnan, Ben D. [Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States); National Univ., San Diego, CA (United States). School of Engineering

2012-06-30

This research project, which was conducted during the Summer and Fall of 2011, investigated some commercially available assessment tools with a focus on IT equipment to see if such tools could round out the DC Pro tool suite. In this research, the assessment capabilities of the various tools were compiled to help make “non-biased” information available to the public. This research should not be considered to be exhaustive on all existing vendor tools although a number of vendors were contacted. Large IT equipment OEM’s like IBM and Dell provide their proprietary internal automated software which does not work on any other IT equipment. However, found two companies with products that showed promise in performing automated assessments for IT equipment from different OEM vendors. This report documents the research and provides a list of software products reviewed, contacts and websites, product details, discussions with specific companies, a set of recommendations, and next steps. As a result of this research, a simple 3-level approach to an IT assessment tool is proposed along with an example of an assessment using a simple IT equipment data collection tool (Level 1, spreadsheet). The tool has been reviewed with the Green Grid and LBNL staff. The initial feedback has been positive although further refinement to the tool will be necessary. Proposed next steps include a field trial of at least two vendors’ software in two different data centers with an objective to prove the concept, ascertain the extent of energy and computational assessment, ease of installation and opportunities for continuous improvement. Based on the discussions, field trials (or case studies) are proposed with two vendors – JouleX (expected to be completed in 2012) and Sentilla.

Validity of Montreal Cognitive Assessment in non-english speaking patients with Parkinson's disease.

Science.gov (United States)

Krishnan, Syam; Justus, Sunitha; Meluveettil, Radhamani; Menon, Ramshekhar N; Sarma, Sankara P; Kishore, Asha

2015-01-01

The Montreal Cognitive Assessment is a brief and easy screening tool for accurately testing cognitive dysfunction in Parkinson's disease. We tested its validity for use in non-English (Malayalam) speaking patients with Parkinson's disease. We developed a Malayalam (a south-Indian language) version of Montreal Cognitive Assessment and applied to 70 patients with Parkinson's disease and 60 age- and education-matched healthy controls. Metric properties were assessed, and the scores were compared with the performance in validated Malayalam versions of Mini Mental Status Examination and Addenbrooke's Cognitive Examination. The Montreal Cognitive Assessment-Malayalam showed good internal consistency and test-retest reliability and its scores correlated with Mini Mental Status Examination (patients: R = 0.70; P speaking Parkinson's disease patients for early screening and potential future interventions for cognitive dysfunction.

Food frequency questionnaire is a valid tool for the assessment of dietary habits of South Indian pregnant women.

Science.gov (United States)

Dwarkanath, Pratibha; Soares, Mario J; Thomas, Tinku; Vaz, Mario; Swaminathan, Sumathi; Kurpad, Anura V

2014-09-01

The food frequency questionnaire (FFQ) was validated against multiple 24-hour dietary recalls (24-HDRs) and for a few blood biomarkers in 154 pregnant women at the obstetrics and gynecology department of St John's Medical College Hospital, Bangalore, India. Absolute nutrient intakes from the FFQ correlated positively with the average 24-HDR during pregnancy. Energy-adjusted nutrients from the FFQ in all trimesters, except proteins, carbohydrate, folate intake, and vitamin B6 in the third trimester, correlated positively with average 24-HDR. Overestimation by the FFQ compared with the 24-HDR ranged from 9% to 41%. Vitamin B12 status in the first and second trimesters positively correlated with energy-adjusted and absolute vitamin B12 intakes from the FFQ. The Bland Altman plots showed a pattern such that a trend was seen toward underreporting of intakes through the FFQ, with increasing mean intakes by the 2 methods, considering 24-HDR as the reference tool. We conclude that the FFQ is a valid tool to measure dietary intakes during pregnancy. © 2012 APJPH.

Perioperative Respiratory Adverse Events in Pediatric Ambulatory Anesthesia: Development and Validation of a Risk Prediction Tool.

Science.gov (United States)

Subramanyam, Rajeev; Yeramaneni, Samrat; Hossain, Mohamed Monir; Anneken, Amy M; Varughese, Anna M

2016-05-01

Perioperative respiratory adverse events (PRAEs) are the most common cause of serious adverse events in children receiving anesthesia. Our primary aim of this study was to develop and validate a risk prediction tool for the occurrence of PRAE from the onset of anesthesia induction until discharge from the postanesthesia care unit in children younger than 18 years undergoing elective ambulatory anesthesia for surgery and radiology. The incidence of PRAE was studied. We analyzed data from 19,059 patients from our department's quality improvement database. The predictor variables were age, sex, ASA physical status, morbid obesity, preexisting pulmonary disorder, preexisting neurologic disorder, and location of ambulatory anesthesia (surgery or radiology). Composite PRAE was defined as the presence of any 1 of the following events: intraoperative bronchospasm, intraoperative laryngospasm, postoperative apnea, postoperative laryngospasm, postoperative bronchospasm, or postoperative prolonged oxygen requirement. Development and validation of the risk prediction tool for PRAE were performed using a split sampling technique to split the database into 2 independent cohorts based on the year when the patient received ambulatory anesthesia for surgery and radiology using logistic regression. A risk score was developed based on the regression coefficients from the validation tool. The performance of the risk prediction tool was assessed by using tests of discrimination and calibration. The overall incidence of composite PRAE was 2.8%. The derivation cohort included 8904 patients, and the validation cohort included 10,155 patients. The risk of PRAE was 3.9% in the development cohort and 1.8% in the validation cohort. Age ≤ 3 years (versus >3 years), ASA physical status II or III (versus ASA physical status I), morbid obesity, preexisting pulmonary disorder, and surgery (versus radiology) significantly predicted the occurrence of PRAE in a multivariable logistic regression

Experience Curves: A Tool for Energy Policy Assessment

Energy Technology Data Exchange (ETDEWEB)

Neij, Lena; Helby, Peter [Lund Univ. (Sweden). Environmental and Energy Systems Studies; Dannemand Andersen, Per; Morthorst, Poul Erik [Riso National Laboratory, Roskilde (Denmark); Durstewitz, Michael; Hoppe-Kilpper, Martin [Inst. fuer Solare Energieversorgungstechnik e.V., Kassel (DE); and others

2003-07-01

The objective of the project, Experience curves: a tool for energy policy assessment (EXTOOL), was to analyse the experience curve as a tool for the assessment of energy policy measures. This is of special interest, since the use of experience curves for the assessment of energy policy measures requires the development of the established experience curve methodology. This development raises several questions which have been addressed and analysed in this project. The analysis is based on case studies of wind power, an area with considerable experience in technology development, deployment and policy measures. Therefore, a case study based on wind power provides a good opportunity to study the usefulness of experience curves as a tool for the assessment of energy policy measures. However, the results are discussed in terms of using experience curves for the assessment of any energy technology. The project shows that experience curves can be used to assess the effect of combined policy measures in terms of cost reductions. Moreover, the result of the project show that experience curves could be used to analyse international 'learning systems', i.e. cost reductions brought about by the development of wind power and policy measures used in other countries. Nevertheless, the use of experience curves for the assessment of policy programmes has several limitations. First, the analysis and assessment of policy programmes cannot be achieved unless relevant experience curves based on good data can be developed. The authors are of the opinion that only studies that provide evidence of the validity, reliability and relevance of experience curves should be taken into account in policy making. Second, experience curves provide an aggregated picture of the situation and more detailed analysis of various sources of cost reduction, and cost reductions resulting from individual policy measures, requires additional data and analysis tools. Third, we do not recommend the use of

Additional Support for the Information Systems Analyst Exam as a Valid Program Assessment Tool

Science.gov (United States)

Carpenter, Donald A.; Snyder, Johnny; Slauson, Gayla Jo; Bridge, Morgan K.

2011-01-01

This paper presents a statistical analysis to support the notion that the Information Systems Analyst (ISA) exam can be used as a program assessment tool in addition to measuring student performance. It compares ISA exam scores earned by students in one particular Computer Information Systems program with scores earned by the same students on the…

A tool to assess sex-gender when selecting health research projects.

Science.gov (United States)

Tomás, Concepción; Yago, Teresa; Eguiluz, Mercedes; Samitier, M A Luisa; Oliveros, Teresa; Palacios, Gemma

2015-04-01

To validate the questionnaire "Gender Perspective in Health Research" (GPIHR) to assess the inclusion of gender perspective in research projects. Validation study in two stages. Feasibility was analysed in the first, and reliability, internal consistence and validity in the second. Aragón Institute of Health Science, Aragón, Spain. GPIHR was applied to 118 research projects funded in national and international competitive tenders from 2003 to 2012. Analysis of inter- and intra-observer reliability with Kappa index and internal consistency with Cronbach's alpha. Content validity analysed through literature review and construct validity with an exploratory factor analysis. Validated GPIHR has 10 questions: 3 in the introduction, 1 for objectives, 3 for methodology and 3 for research purpose. Average time of application was 13min Inter-observer reliability (Kappa) varied between 0.35 and 0.94 and intra-observer between 0.40 and 0.94. Theoretical construct is supported in the literature. Factor analysis identifies three levels of GP inclusion: "difference by sex", "gender sensitive" and "feminist research" with an internal consistency of 0.64, 0.87 and 0.81, respectively, which explain 74.78% of variance. GPIHR questionnaire is a valid tool to assess GP and useful for those researchers who would like to include GP in their projects. Copyright © 2014 Elsevier España, S.L.U. All rights reserved.

Validity of assessing child feeding with virtual reality.

Science.gov (United States)

Persky, Susan; Goldring, Megan R; Turner, Sara A; Cohen, Rachel W; Kistler, William D

2018-04-01

Assessment of parents' child feeding behavior is challenging, and there is need for additional methodological approaches. Virtual reality technology allows for the creation of behavioral measures, and its implementation overcomes several limitations of existing methods. This report evaluates the validity and usability of the Virtual Reality (VR) Buffet among a sample of 52 parents of children aged 3-7. Participants served a meal of pasta and apple juice in both a virtual setting and real-world setting (counterbalanced and separated by a distractor task). They then created another meal for their child, this time choosing from the full set of food options in the VR Buffet. Finally, participants completed a food estimation task followed by a questionnaire, which assessed their perceptions of the VR Buffet. Results revealed that the amount of virtual pasta served by parents correlated significantly with the amount of real pasta they served, r s  = 0.613, p < .0001, as did served amounts of virtual and real apple juice, r s  = 0.822, p < .0001. Furthermore, parents' perception of the calorie content of chosen foods was significantly correlated with observed calorie content (r s  = 0.438, p = .002), and parents agreed that they would feed the meal they created to their child (M = 4.43, SD = 0.82 on a 1-5 scale). The data presented here demonstrate that parent behavior in the VR Buffet is highly related to real-world behavior, and that the tool is well-rated by parents. Given the data presented and the potential benefits of the abundant behavioral data the VR Buffet can provide, we conclude that it is a valid and needed addition to the array of tools for assessing feeding behavior. Published by Elsevier Ltd.

Knowing Every Child: Validation of the Holistic Student Assessment (HSA) as a Measure of Social-Emotional Development.

Science.gov (United States)

Malti, Tina; Zuffianò, Antonio; Noam, Gil G

2018-04-01

Knowing every child's social-emotional development is important as it can support prevention and intervention approaches to meet the developmental needs and strengths of children. Here, we discuss the role of social-emotional assessment tools in planning, implementing, and evaluating preventative strategies to promote mental health in all children and adolescents. We, first, selectively review existing tools and identify current gaps in the measurement literature. Next, we introduce the Holistic Student Assessment (HSA), a tool that is based in our social-emotional developmental theory, The Clover Model, and designed to measure social-emotional development in children and adolescents. Using a sample of 5946 students (51% boys, M age  = 13.16 years), we provide evidence for the psychometric validity of the self-report version of the HSA. First, we document the theoretically expected 7-dimension factor structure in a calibration sub-sample (n = 984) and cross-validate its structure in a validation sub-sample (n = 4962). Next, we show measurement invariance across development, i.e., late childhood (9- to 11-year-olds), early adolescence (12- to 14-year-olds), and middle adolescence (15- to 18-year-olds), and evidence for the HSA's construct validity in each age group. The findings support the robustness of the factor structure and confirm its developmental sensitivity. Structural equation modeling validity analysis in a multiple-group framework indicates that the HSA is associated with mental health in expected directions across ages. Overall, these findings show the psychometric properties of the tool, and we discuss how social-emotional tools such as the HSA can guide future research and inform large-scale dissemination of preventive strategies.

ExEP yield modeling tool and validation test results

Science.gov (United States)

Morgan, Rhonda; Turmon, Michael; Delacroix, Christian; Savransky, Dmitry; Garrett, Daniel; Lowrance, Patrick; Liu, Xiang Cate; Nunez, Paul

2017-09-01

EXOSIMS is an open-source simulation tool for parametric modeling of the detection yield and characterization of exoplanets. EXOSIMS has been adopted by the Exoplanet Exploration Programs Standards Definition and Evaluation Team (ExSDET) as a common mechanism for comparison of exoplanet mission concept studies. To ensure trustworthiness of the tool, we developed a validation test plan that leverages the Python-language unit-test framework, utilizes integration tests for selected module interactions, and performs end-to-end crossvalidation with other yield tools. This paper presents the test methods and results, with the physics-based tests such as photometry and integration time calculation treated in detail and the functional tests treated summarily. The test case utilized a 4m unobscured telescope with an idealized coronagraph and an exoplanet population from the IPAC radial velocity (RV) exoplanet catalog. The known RV planets were set at quadrature to allow deterministic validation of the calculation of physical parameters, such as working angle, photon counts and integration time. The observing keepout region was tested by generating plots and movies of the targets and the keepout zone over a year. Although the keepout integration test required the interpretation of a user, the test revealed problems in the L2 halo orbit and the parameterization of keepout applied to some solar system bodies, which the development team was able to address. The validation testing of EXOSIMS was performed iteratively with the developers of EXOSIMS and resulted in a more robust, stable, and trustworthy tool that the exoplanet community can use to simulate exoplanet direct-detection missions from probe class, to WFIRST, up to large mission concepts such as HabEx and LUVOIR.

Diagnosing paratonia in the demented elderly: reliability and validity of the Paratonia Assessment Instrument (PAI).

Science.gov (United States)

Hobbelen, Johannes S M; Koopmans, Raymond T C M; Verhey, Frans R J; Habraken, Kitty M; de Bie, Rob A

2008-08-01

Paratonia is one of the associated movement disorders characteristic of dementia. The aim of this study was to develop an assessment tool (the Paratonia Assessment Instrument, PAI), based on the new consensus definition of paratonia. An additional aim was to investigate the reliability and validity of the PAI. A three-phase cross-sectional survey was conducted. In the first two phases, the PAI was developed and validated. In the third phase, the inter-observer reliability and feasibility of the instrument was tested. The original PAI consisted of five criteria that all needed to be met in order to make the diagnosis. On the basis of a qualitative analysis, one criterion was reformulated and another was removed. Following this, inter-observer reliability between the two assessors resulted in an improvement of Cohen's kappa from 0.532 in the initial phase to 0.677 in the second phase. This improvement was substantiated in the third phase by two independent assessors with Cohen's kappa ranging from 0.625 to 1. The PAI is a reliable and valid assessment tool for diagnosing paratonia in elderly people with dementia that can be applied easily in daily practice.

Development, use, and validation of the CFD tool FLACS for hydrogen safety studies

Energy Technology Data Exchange (ETDEWEB)

Middha, Prankul

2010-07-01

Computational Fluid Dynamics (CFD) calculations for gas explosion safety have been widely used for doing risk assessments within the oil and gas industry for more than a decade. Based on predicted consequences of a range of potential accident scenarios a risk level is predicted. The development of applications using hydrogen as a clean energy carrier has accelerated in recent years, and hydrogen may be used widely in future. Due to the very high reactivity of hydrogen, safe handling is critical. To be able to perform proper consequence modeling as a part of a risk assessment, it is essential to be able to model the physical processes well. CFD tools have the potential to model the relevant physics and predict well, but without proper user guidelines based on extensive validation work, very mixed prediction capability can be expected. This paper deals with the development and validation of the CFD tool FLACS for hydrogen safety applications. Significant validation work against several experiments has been carried out in order to increase the confidence of predictions of scenarios relevant to hydrogen safety. The validation studies have included dispersion, explosion and combined dispersion and explosion studies. A range of different dispersion experiments is simulated, including low momentum releases in a garage, sub-sonic jets in a garage with stratification effects and subsequent slow diffusion, low momentum and subsonic horizontal jets influenced by buoyancy, and free jets from high-pressure vessels. LH2 releases are also considered. Some of the simulations are performed as blind predictions. FLACS uses a utility program in order to model releases from high-pressure reservoirs. Work has been carried out in order to extend the models in the utility program in order to include real gas effects. Validation against explosion experiments in geometries ranging from smooth and obstructed pipes, refueling station, tunnel, vented vessels, jet-ignited lane, etc. have been

Pilot Use of a Novel Tool to Assess Neurosurgical Capacity in Uganda.

Science.gov (United States)

Ploss, Brittany; Abdelgadir, Jihad; Smith, Emily R; Fuller, Anthony; Nickenig Vissoci, Joao Ricardo; Muhindo, Alex; Galukande, Moses; Haglund, Michael M

2017-12-01

There is a significant burden of unmet surgical need in many low- and middle-income countries (>80% in parts of Africa). This need is even larger for specialties such as neurosurgery. Surgical capacity tools have been developed and used to assess needs and plan for resource allocation. This study piloted a new tool to assess neurosurgical capacity and describes its use. A surgical capacity tool was adapted to assess neurosurgical capacity. An expert panel of neurosurgeons and researchers reviewed the Surgeons OverSeas PIPES (personnel, infrastructure, procedures, equipment, and supplies) assessment and added additional items essential to perform common neurosurgery procedures. This tool was then piloted at 3 public hospitals in Uganda and each hospital was given a score of neurosurgical capacity. At 1 hospital, 3 respondents were asked to answer the survey to assess reliability. The hospital with the largest neurosurgery caseload and 5 neurosurgeons scored the highest on our survey, followed by a regional hospital with 1 practicing neurosurgeon. The third hospital, without a neurosurgeon, scored the lowest on the scale. At the hospital that completed the reliability assessment, scores were varied between respondents. NeuroPIPES survey scores were in keeping with the number of neurosurgeons and respective caseloads of each hospital. However, the variation in scores between respondents at the same hospital suggests that adaptations could be made to the tool that may improve reliability and validity. The methodology used to create NeuroPIPES may be successfully applied to a variety of other surgical subspecialties for similar assessments. Copyright © 2017 Elsevier Inc. All rights reserved.

Validation of a Tool Evaluating Educational Apps for Smart Education

Science.gov (United States)

Lee, Jeong-Sook; Kim, Sung-Wan

2015-01-01

The purpose of this study is to develop and validate an evaluation tool of educational apps for smart education. Based on literature reviews, a potential model for evaluating educational apps was suggested. An evaluation tool consisting of 57 survey items was delivered to 156 students in middle and high schools. An exploratory factor analysis was…

Reliability and validity of procedure-based assessments in otolaryngology training.

Science.gov (United States)

Awad, Zaid; Hayden, Lindsay; Robson, Andrew K; Muthuswamy, Keerthini; Tolley, Neil S

2015-06-01

To investigate the reliability and construct validity of procedure-based assessment (PBA) in assessing performance and progress in otolaryngology training. Retrospective database analysis using a national electronic database. We analyzed PBAs of otolaryngology trainees in North London from core trainees (CTs) to specialty trainees (STs). The tool contains six multi-item domains: consent, planning, preparation, exposure/closure, technique, and postoperative care, rated as "satisfactory" or "development required," in addition to an overall performance rating (pS) of 1 to 4. Individual domain score, overall calculated score (cS), and number of "development-required" items were calculated for each PBA. Receiver operating characteristic analysis helped determine sensitivity and specificity. There were 3,152 otolaryngology PBAs from 46 otolaryngology trainees analyzed. PBA reliability was high (Cronbach's α 0.899), and sensitivity approached 99%. cS correlated positively with pS and level in training (rs : +0.681 and +0.324, respectively). ST had higher cS and pS than CT (93% ± 0.6 and 3.2 ± 0.03 vs. 71% ± 3.1 and 2.3 ± 0.08, respectively; P reliable and valid for assessing otolaryngology trainees' performance and progress at all levels. It is highly sensitive in identifying competent trainees. The tool is used in a formative and feedback capacity. The technical domain is the best predictor and should be given close attention. NA. © 2014 The American Laryngological, Rhinological and Otological Society, Inc.

Development of risk assessment tool for foundry workers.

Science.gov (United States)

Mohan, G Madhan; Prasad, P S S; Mokkapati, Anil Kumar; Venkataraman, G

2008-01-01

Occupational ill-health and work-related disorders are predominant in manufacturing industries due to the inevitable presence of manual work even after several waves of industrial automation and technological advancements. Ergonomic risk factors and musculoskeletal disorders like low-back symptoms have been noted amongst foundry workers. The purpose of this study was to formulate and develop a Physical Effort Index to assess risk factor. The questionnaire tool applicable to foundry environment has been designed and validated. The data recorded through survey across the foundries has been subjected to regression analysis to correlate between proposed physical effort index and the standard Borg's Ratings of Perceived Exertion (RPE) scale. The physical efforts of sixty seven workers in various foundry shop floors were assessed subjectively. The 'Job factors' and 'Work environment' were the two major parameters considered in assessing the worker discomfort level at workplace. A relation between Borg's RPE scale and the above two parameters were arrived at, through regression analysis. The study demonstrates the prevalence of risk factors amongst foundry workers and the effectiveness of the proposed index in estimating the risk factor levels. RELEVANCE TO THE INDUSTRY: The proposed tool will assist foundry supervisors and managers to assess the risk factors and helps in better understanding of the workplace to avoid work-related disorders, ensuring better output.

Social Validation of the New England Center for Children-Core Skills Assessment

Science.gov (United States)

Dickson, Chata A.; MacDonald, Rebecca P. F.; Mansfield, Renee; Guilhardi, Paulo; Johnson, Cammarie; Ahearn, William H.

2014-01-01

We investigated the social validity of the NECC Core Skills Assessment (NECC-CSA) with parents and professionals as participants. The NECC-CSA is a measurement tool consisting of direct and indirect measures of skills important to all individuals with autism, across the lifespan. Participants (N = 245) were provided with a list of 66 skills, 47 of…

Criticism on Environmental Assessment Tools

NARCIS (Netherlands)

Abdalla, G.; Maas, G.J.; Huyghe, J.; Oostra, M.; Saji Baby, xx; Bogdan Zygmunt, xx

2011-01-01

Using environmental assessment tools to assess the sustainability of buildings, homes and mixed- use area is increasing. Environmental tools assign scores to projects using some sustainability (sub) aspects according to design and realization documents and evidences. Six European sustainable urban

Assessing the quality of websites providing information on multiple sclerosis: evaluating tools and comparing sites.

Science.gov (United States)

Harland, Juliet; Bath, Peter

2007-09-01

The quality of health information available on the Internet has proved difficult to assess objectively. The Internet's growing popularity as a source of health information, accompanied by the lack of regulation of websites, has resulted in research that has developed and tested tools to evaluate health website quality. However, only a few studies have tested the validity and reliability of these tools. There is a lack of consensus about appropriate indicators with which to operationalize the concept of quality health information. This study aimed to contribute to this research by testing the validity and reliability of existing tools, through their application to websites that provided information about multiple sclerosis. Furthermore, a specific tool for evaluating multiple sclerosis information was developed, contributing to the debate about suitable criteria for measuring the ;quality' of health information on the web.

Nursing Informatics Competencies: Psychometric Validation, Dissemination, and Maintenance of Self-Assessment Tool for Nurse Leaders.

Science.gov (United States)

Collins, Sarah

2016-01-01

Due to rapid advances in technology, HIT competencies for nursing leaders require frequent attention and updating from experts in the field to ensure relevance to nursing leaders' work. This workshop will target nursing informatics researchers and leaders to: 1) learn methods and findings from a study validating a Self-Assessment Scale for Nursing Informatics Competencies for Nurse Leaders, 2) generate awareness of the Self-Assessment scale, 3) discuss strategies for maintenance of competencies overtime and 4) identify strategies to engage nursing leaders in this pursuit.

The Learning Loss Scale as an Assessment Tool: An Empirical Examination of Convergent Validity with Performative Measures

Science.gov (United States)

Hooker, John; Denker, Katherine

2014-01-01

Higher education has placed an increasingly greater value on assessment. The Learning Loss Scale may be an appropriate tool to assess learning across disciplines. In this paper, we review the culture of assessment, conceptualizations of cognitive learning, the Learning Loss Scale, and a theoretical explanation, and then we test this measure to…

Validation of balance-quality assessment using a modified bathroom scale.

Science.gov (United States)

Hewson, D J; Duchêne, J; Hogrel, J-Y

2015-02-01

The balance quality tester (BQT), based on a standard electronic bathroom scale has been developed in order to assess balance quality. The BQT includes automatic detection of the person to be tested by means of an infrared detector and bluetooth communication capability for remote assessment when linked to a long-distance communication device such as a mobile phone. The BQT was compared to a standard force plate for validity and agreement. The two most widely reported parameters in balance literature, the area of the centre of pressure (COP) displacement and the velocity of the COP displacement, were compared for 12 subjects, each of whom was tested on ten occasions on each of the 2 days. No significant differences were observed between the BQT and the force plate for either of the two parameters. In addition a high level of agreement was observed between both devices. The BQT is a valid device for remote assessment of balance quality, and could provide a useful tool for long-term monitoring of people with balance problems, particularly during home monitoring.

Joint assessment in von Willebrand disease : Validation of the Haemophilia Joint Health score and Haemophilia Activities List

NARCIS (Netherlands)

van Galen, Karin P. M.; Timmer, Merel A.; de Kleijn, Piet; Fischer, Kathelijn; Foppen, Wouter; Schutgens, Roger E. G.; Eikenboom, Jeroen; Meijer, Karina; Cnossen, Marjon H.; Fijnvandraat, Karin; van der Bom, Johanna G.; Laros-van Gorkom, Britta A. P.; Leebeek, Frank W. G.; Mauser-Bunschoten, Eveline P.

Assessment of clinical outcome after joint bleeding is essential to identify joint damage and optimise treatment, to prevent disability. However, disease-specific tools to assess the musculoskeletal status in patients with von Willebrand disease (VWD) are lacking. We aimed to determine validity and

The role of quality tools in assessing reliability of the internet for health information.

Science.gov (United States)

Hanif, Faisal; Read, Janet C; Goodacre, John A; Chaudhry, Afzal; Gibbs, Paul

2009-12-01

The Internet has made it possible for patients and their families to access vast quantities of information that previously would have been difficult for anyone but a physician or librarian to obtain. Health information websites, however, are recognised to differ widely in quality and reliability of their content. This has led to the development of various codes of conduct or quality rating tools to assess the quality of health websites. However, the validity and reliability of these quality tools and their applicability to different health websites also varies. In principle, rating tools should be available to consumers, require a limited number of elements to be assessed, be assessable in all elements, be readable and be able to gauge the readability and consistency of information provided from a patient's view point. This article reviews the literature on the trends of the Internet use for health and analyses various codes of conduct/ethics or 'quality tools' available to monitor the quality of health websites from a patient perspective.

Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study.

Science.gov (United States)

Schnall, Rebecca; Cho, Hwayoung; Liu, Jianfang

2018-01-05

Mobile technology has become a ubiquitous technology and can be particularly useful in the delivery of health interventions. This technology can allow us to deliver interventions to scale, cover broad geographic areas, and deliver technologies in highly tailored ways based on the preferences or characteristics of users. The broad use of mobile technologies supports the need for usability assessments of these tools. Although there have been a number of usability assessment instruments developed, none have been validated for use with mobile technologies. The goal of this work was to validate the Health Information Technology Usability Evaluation Scale (Health-ITUES), a customizable usability assessment instrument in a sample of community-dwelling adults who were testing the use of a new mobile health (mHealth) technology. A sample of 92 community-dwelling adults living with HIV used a new mobile app for symptom self-management and completed the Health-ITUES to assess the usability of the app. They also completed the Post-Study System Usability Questionnaire (PSSUQ), a widely used and well-validated usability assessment tool. Correlations between these scales and each of the subscales were assessed. The subscales of the Health-ITUES showed high internal consistency reliability (Cronbach alpha=.85-.92). Each of the Health-ITUES subscales and the overall scale was moderately to strongly correlated with the PSSUQ scales (r=.46-.70), demonstrating the criterion validity of the Health-ITUES. The Health-ITUES has demonstrated reliability and validity for use in assessing the usability of mHealth technologies in community-dwelling adults living with a chronic illness. ©Rebecca Schnall, Hwayoung Cho, Jianfang Liu. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 05.01.2018.

Validation of a Dietary Screening Tool in a Middle-Aged Appalachian Population

Directory of Open Access Journals (Sweden)

Melissa Ventura Marra

2018-03-01

Full Text Available Proactive nutrition screening is an effective public health strategy for identifying and targeting individuals who could benefit from making dietary improvements for primary and secondary prevention of disease. The Dietary Screening Tool (DST was developed and validated to assess nutritional risk among rural older adults. The purpose of this study was to evaluate the utility and validity of the DST to identify nutritional risk in middle-aged adults. This cross-sectional study in middle-aged adults (45–64 year olds, n = 87 who reside in Appalachia, examined nutritional status using an online health survey, biochemical measures, anthropometry, and three representative 24-h dietary recalls. The Healthy Eating Index (HEI was calculated to describe overall diet quality. Adults identified by the DST with a nutrition risk had lower HEI scores (50 vs. 64, p < 0.001 and were much more likely to also be considered at dietary risk by the HEI (OR 11.6; 3.2–42.6 when compared to those not at risk. Those at risk had higher energy-adjusted total fat, saturated fat, and added sugar intakes and lower intakes of dietary fiber, and several micronutrients than those classified as not at risk by the DST. Similarly, the at-risk group had significantly lower serum levels of α-carotene, β-carotene, cryptoxanthin, lutein, and zeaxanthin but did not differ in retinol or methylmalonic acid compared with those not at risk. The DST is a valid tool to identify middle-aged adults with nutritional risk.

Assessment of juveniles testimonies’ validity

Directory of Open Access Journals (Sweden)

Dozortseva E.G.

2015-12-01

Full Text Available The article presents a review of the English language publications concerning the history and the current state of differential psychological assessment of validity of testimonies produced by child and adolescent victims of crimes. The topicality of the problem in Russia is high due to the tendency of Russian specialists to use methodical means and instruments developed abroad in this sphere for forensic assessments of witness testimony veracity. A system of Statement Validity Analysis (SVA by means of Criteria-Based Content Analysis (CBCA and Validity Checklist is described. The results of laboratory and field studies of validity of CBCA criteria on the basis of child and adult witnesses are discussed. The data display a good differentiating capacity of the method, however, a high level of error probability. The researchers recommend implementation of SVA in the criminal investigation process, but not in the forensic assessment. New perspective developments in the field of methods for differentiation of witness statements based on the real experience and fictional are noted. The conclusion is drawn that empirical studies and a special work for adaptation and development of new approaches should precede their implementation into Russian criminal investigation and forensic assessment practice

Conceptual assessment tool for advanced undergraduate electrodynamics

Science.gov (United States)

Baily, Charles; Ryan, Qing X.; Astolfi, Cecilia; Pollock, Steven J.

2017-12-01

As part of ongoing investigations into student learning in advanced undergraduate courses, we have developed a conceptual assessment tool for upper-division electrodynamics (E&M II): the Colorado UppeR-division ElectrodyNamics Test (CURrENT). This is a free response, postinstruction diagnostic with 6 multipart questions, an optional 3-question preinstruction test, and accompanying grading rubrics. The instrument's development was guided by faculty-consensus learning goals and research into common student difficulties. It can be used to gauge the effectiveness of transformed pedagogy, and to gain insights into student thinking in the covered topic areas. We present baseline data representing 500 students across 9 institutions, along with validity, reliability, and discrimination measures of the instrument and scoring rubric.

Cultural Orientations Framework (COF) Assessment Questionnaire in Cross-Cultural Coaching: A Cross-Validation with Wave Focus Styles

OpenAIRE

Rojon, C; McDowall, A

2010-01-01

This paper outlines a cross-validation of the Cultural Orientations Framework assessment questionnaire\\ud (COF, Rosinski, 2007; a new tool designed for cross-cultural coaching) with the Saville Consulting\\ud Wave Focus Styles questionnaire (Saville Consulting, 2006; an existing validated measure of\\ud occupational personality), using data from UK and German participants (N = 222). The convergent and\\ud divergent validity of the questionnaire was adequate. Contrary to previous findings which u...

Assessing Face Validity of a Food Behavior Checklist for Limited-resource Filipinos.

Science.gov (United States)

Banna, Jinan C; Buchthal, Opal Vanessa; Tauyan, Socorro

2015-10-01

Diet-related chronic health conditions are prevalent in the Filipino American community; however, there is a lack of rigorously validated nutrition education evaluation tools in Tagalog for use in this population. This study aimed to develop and evaluate the face validity of a Tagalog-language food behavior checklist (FBC). A multi-step method was used, involving translation of questionnaire text from English to Tagalog by a team of professionals, creation of accompanying color photographs, cognitive testing with the target population, final review by the team of professionals, and assessment of readability. Subjects for cognitive testing were men (n=6) and women (n=14) 18 years or older in Hawai'i who received or were eligible to receive Supplemental Nutrition Assistance Program (SNAP) benefits, self-identified as Filipino, and preferred Tagalog rather than English. Participants were recruited from churches, the Filipino Center, and other community sites. Cognitive interviews revealed several issues with text and photographs, such as preferences for specific terms, and images that did not adequately illustrate the text. Image changes were made to reflect items most commonly consumed. The team of professionals agreed with participant suggestions. Assessment of readability revealed a reading level appropriate for a low-literacy population of grade 5.9. The multi-step process, which allowed members of the target audience to reveal the appropriateness of the questionnaire, yielded a Tagalog-language FBC found to have adequate face validity. After further evaluation of validity and reliability, this tool may be used to evaluate behavior change resulting from the United States Department of Agriculture's (USDA) nutrition education programs.

Development and validation of a Meal Index of dietary Quality (Meal IQ) to assess the dietary quality of school lunches

DEFF Research Database (Denmark)

Sabinsky, Marianne; Toft, Ulla; Andersen, Klaus K.

2012-01-01

Objective School lunch programmes are one strategy to promote healthier dietary habits in children, but better evaluation tools for assessing the dietary quality of such programmes are needed. The aim of the present study was to develop and validate a simple index to assess the dietary quality...... of school lunches for children aged 7–13 years. Design A Meal Index of dietary Quality (Meal IQ) was developed to consist of seven components (nutrients and food groups) based on dietary issues for children aged 7–13 years, which were identified in a national dietary survey. The Meal IQ was validated......, higher contents of fibre, various vitamins and minerals, and more fruits, vegetables and fish. Conclusions The Meal IQ is a valid and useful evaluation tool for assessing the dietary quality of lunches provided by schools or brought to school from home....

Validation of Transition Readiness Assessment Questionaire in Turkish Adolescents with Diabetes

Directory of Open Access Journals (Sweden)

Evrim Kızıler

2018-02-01

Full Text Available Background: Today, more than 90% of adolescents with chronic conditions survive into adulthood and move from pediatric care to adult care for the management of their chronic illness. It is important to grant autonomy and ensure that adolescents/young adults are ready to use the adult health care system prior to the transfer of care. However, the lack of a transition readiness assessment tool that is validated, patient-centered, and appropriate to developmental differences in pediatric care is a major obstacle for the transition of care from pediatric services to adult services. Aims: This study examined the validity and reliability of the Turkish version of the Transition Readiness Assessment Questionnaire, which assesses the readiness for transition from pediatric to adult health care for adolescents/young adults with diabetes mellitus. Study Design: Methodological study. Methods: Participants were 109 adolescents/young adults with type 1 diabetes mellitus aged 14-21 years. After permission was obtained to adapt the Transition Readiness Assessment Questionnaire, the Turkish Transition Readiness Assessment Questionnaire and self-care scale were administered to participants through face-to-face interviews at two pediatric endocrinology clinics. Validity was evaluated by exploratory and confirmatory factor analysis and content-scope validity assessment; reliability was evaluated by item-total score correlation and continuity methods. Internal reliability was assessed by Cronbach’s alpha coefficient and criterion validity assessment. Results: The item analysis, exploratory factor analysis, and confirmatory factor analysis identified five basic dimensions, with high internal consistency (0.89-0.75. The ratio χ2/df and other conformity indices were a good fit to the data. The correlation coefficient in the analyses of test-retest scores was 0.86 for the total scale (p<0.05, and the Cronbach’s alpha coefficient was 0.88 for overall scale

The Computerized Perceptual Motor Skills Assessment: A new visual perceptual motor skills evaluation tool for children in early elementary grades.

Science.gov (United States)

Howe, Tsu-Hsin; Chen, Hao-Ling; Lee, Candy Chieh; Chen, Ying-Dar; Wang, Tien-Ni

2017-10-01

Visual perceptual motor skills have been proposed as underlying courses of handwriting difficulties. However, there is no evaluation tool currently available to assess these skills comprehensively and to serve as a sensitive measure. The purpose of this study was to validate the Computerized Perceptual Motor Skills Assessment (CPMSA), a newly developed evaluation tool for children in early elementary grades. Its test-retest reliability, concurrent validity, discriminant validity, and responsiveness were examined in 43 typically developing children and 26 children with handwriting difficulty. The CPMSA demonstrated excellent reliability across all subtests with intra-class correlation coefficients (ICCs)≥0.80. Significant moderate correlations between the domains of the CPMSA and corresponding gold standards including Beery VMI, the TVPS-3, and the eye-hand coordination subtest of the DTVP-2 demonstrated good concurrent validity. In addition, the CPMSA showed evidence of discriminant validity in samples of children with and without handwriting difficulty. This article provides evidence in support of the CPMSA. The CPMSA is a reliable, valid, and promising measure of visual perceptual motor skills for children in early elementary grades. Directions for future study and improvements to the assessment are discussed. Copyright © 2017. Published by Elsevier Ltd.

Pain Assessment in Impaired Cognition (PAIC: content validity of the Dutch version of a new and universal tool to measure pain in dementia

Directory of Open Access Journals (Sweden)

van Dalen-Kok AH

2017-12-01

Full Text Available Annelore H van Dalen-Kok,1 Wilco P Achterberg,1 Wieke E Rijkmans,1 Sara A Tukker-van Vuuren,1 Suzanne Delwel,2,3 Henrica CW de Vet,4 Frank Lobbezoo,2,5 Margot WM de Waal1 1Department of Public Health and Primary Care, Leiden University Medical Centre, Leiden, 2Department of Oral Kinesiology, Academic Centre for Dentistry Amsterdam (ACTA, University of Amsterdam and VU University Amsterdam, 3Department of Clinical Neuropsychology, Faculty of Behavioral and Movement Sciences, VU University, 4Department of Epidemiology and Biostatistics, The EMGO Institute for Health and Care Research, VU University Medical Center, 5MOVE Research Institute Amsterdam, VU University, Amsterdam, the Netherlands Objectives: Detection and measurement of pain in persons with dementia by using observational pain measurement tools is essential. However, the evidence for the psychometric properties of existing observational tools remains limited. Therefore, a new meta-tool has been developed: Pain Assessment in Impaired Cognition (PAIC, as a collaborative EU action. The aim is to describe the translation procedure and content validity of the Dutch version of the PAIC.Methods: Translation of the PAIC into Dutch followed the forward-backward approach of the Guidelines for Establishing Cultural Equivalence of Instruments. A questionnaire survey was administered to clinical nursing home experts (20 physicians and 20 nurses to determine whether the PAIC items are indicative of pain and whether items are specific for pain or for other disorders (anxiety disorder, delirium, dementia, or depression. To quantify content validity, mean scores per item were calculated.Results: Eleven items were indicative of pain, for example, “frowning,” “freezing,” and “groaning.” Fifteen items were considered to be pain-specific, for example, “frowning,” “curling up,” and “complaining.” There were discrepancies between the notion of pain characteristics according to nurses

Validation of an observation tool to assess physical activity-promoting physical education lessons in high schools: SOFIT.

Science.gov (United States)

Fairclough, Stuart J; Weaver, R Glenn; Johnson, Siobhan; Rawlinson, Jack

2018-05-01

SOFIT+ is an observation tool to measure teacher practices related to moderate-to-vigorous physical activity (MVPA) promotion during physical education (PE). The objective of the study was to examine the validity of SOFIT+ during high school PE lessons. This cross-sectional, observational study tested the construct validity of SOFIT+ in boys' and girls' high school PE lessons. Twenty-one PE lessons were video-recorded and retrospectively coded using SOFIT+. Students wore hip-mounted accelerometers during lessons as an objective measure of MVPA. Multinomial logistic regression was used to estimate the likelihood of students engaging in MVPA during different teacher practices represented by observed individual codes and a combined SOFIT+ index-score. Fourteen individual SOFIT+ variables demonstrated a statistically significant relationship with girls' and boys' MVPA. Observed lesson segments identified as high MVPA-promoting were related to an increased likelihood of girls engaging in 5-10 (OR=2.86 [95% CI 2.41-3.40]), 15-25 (OR=7.41 [95% CI 6.05-9.06]), and 30-40 (OR=22.70 [95% CI 16.97-30.37])s of MVPA. For boys, observed high-MVPA promoting segments were related to an increased likelihood of engaging in 5-10 (OR=1.71 [95% CI 1.45-2.01]), 15-25 (OR=2.69 [95% CI 2.31-3.13]) and 30-40 (OR=4.26 [95% CI 3.44-5.29])s of MVPA. Teacher practices during high school PE lessons are significantly related to students' participation in MVPA. SOFIT+ is a valid and reliable tool to examine relationships between PE teacher practices and student MVPA during PE. Copyright © 2017 Sports Medicine Australia. Published by Elsevier Ltd. All rights reserved.

cluML: A markup language for clustering and cluster validity assessment of microarray data.

Science.gov (United States)

Bolshakova, Nadia; Cunningham, Pádraig

2005-01-01

cluML is a new markup language for microarray data clustering and cluster validity assessment. The XML-based format has been designed to address some of the limitations observed in traditional formats, such as inability to store multiple clustering (including biclustering) and validation results within a dataset. cluML is an effective tool to support biomedical knowledge representation in gene expression data analysis. Although cluML was developed for DNA microarray analysis applications, it can be effectively used for the representation of clustering and for the validation of other biomedical and physical data that has no limitations.

Development and validation of a premature ejaculation diagnostic tool.

Science.gov (United States)

Symonds, Tara; Perelman, Michael A; Althof, Stanley; Giuliano, François; Martin, Mona; May, Kathryn; Abraham, Lucy; Crossland, Anna; Morris, Mark

2007-08-01

Diagnosis of premature ejaculation (PE) for clinical trial purposes has typically relied on intravaginal ejaculation latency time (IELT) for entry, but this parameter does not capture the multidimensional nature of PE. Therefore, the aim was to develop a brief, multidimensional, psychometrically validated instrument for diagnosing PE status. The questionnaire development involved three stages: (1) Five focus groups and six individual interviews were conducted to develop the content; (2) psychometric validation using three different groups of men; and (3) generation of a scoring system. For psychometric validation/scoring system development, data was collected from (1) men with PE based on clinician diagnosis, using DSM-IV-TR, who also had IELTs or =11 PE. The development and validation of this new PE diagnostic tool has resulted in a new, user-friendly, and brief self-report questionnaire for use in clinical trials to diagnose PE.

Assessment of teacher competence using video portfolios: reliability, construct validity and consequential validity

NARCIS (Netherlands)

Admiraal, W.; Hoeksma, M.; van de Kamp, M.-T.; van Duin, G.

2011-01-01

The richness and complexity of video portfolios endanger both the reliability and validity of the assessment of teacher competencies. In a post-graduate teacher education program, the assessment of video portfolios was evaluated for its reliability, construct validity, and consequential validity.

Development and validation of a screening instrument to assess the types and quality of foods served at home meals

Directory of Open Access Journals (Sweden)

Fulkerson Jayne A

2012-02-01

Full Text Available Abstract Background Although there is growing interest in assessing the home food environment, no easy-to-use, low cost tools exist to assess the foods served at home meals, making it difficult to assess the meal component of the food environment. The aim of this study was to develop and validate a user-friendly screener to assess the types of foods served at home meals. Methods Primary food preparing adults (n = 51 participated in a validation study in their own homes. Staff and participants independently completed a screener as participants cooked dinner. The screener assessed the types of foods offered, method(s of preparation, and use of added fats. Two scale scores were created: 1 to assess offerings of foods in five food groups (meat and other protein, milk, vegetables, fruit, grains, 2 to assess the relative healthfulness of foods based on types offered, preparation method, and added fats. Criterion validity was assessed comparing staff and participant reports of individual foods (kappa (k and scale scores (Spearman correlations. Results Criterion validity was high between participants' and staffs' record of whether major food categories (meat and other protein, bread and cereal, salad, vegetables, fruits, dessert were served (k = 0.79-1.0, moderate for reports of other starches (e.g., rice being served (k = 0.52, and high for the Five Food Group and Healthfulness scale scores (r = 0.75-0.85, p Conclusions This new meal screening tool has high validity and can be used to assess the types of foods served at home meals allowing a more comprehensive assessment of the home food environment.

Measuring the food service environment: development and implementation of assessment tools.

Science.gov (United States)

Minaker, Leia M; Raine, Kim D; Cash, Sean B

2009-01-01

The food environment is increasingly being implicated in the obesity epidemic, though few reported measures of it exist. In order to assess the impact of the food environment on food intake, valid measures must be developed and tested. The current study describes the development of a food service environment assessment tool and its implementation in a community setting. A descriptive study with mixed qualitative and quantitative methods at a large, North American university campus was undertaken. Measures were developed on the basis of a conceptual model of nutrition environments. Measures of community nutrition environment were the number, type and hours of operation of each food service outlet on campus. Measures of consumer nutrition environment were food availability, food affordability, food promotion and nutrition information availability. Seventy-five food service outlets within the geographic boundaries were assessed. Assessment tools could be implemented in a reasonable amount of time and showed good face and content validity. The food environments were described and measures were grouped so that food service outlet types could be compared in terms of purchasing convenience, cost/value, healthy food promotion and health. Food service outlet types that scored higher in purchasing convenience and cost/value tended to score lower in healthy food promotion and health. This study adds evidence that food service outlet types that are convenient to consumers and supply high value (in terms of calories per dollar) tend to be less health-promoting. Results from this study also suggest the possibility of characterizing the food environment according to the type of food service outlet observed.

When Assessment Data Are Words: Validity Evidence for Qualitative Educational Assessments.

Science.gov (United States)

Cook, David A; Kuper, Ayelet; Hatala, Rose; Ginsburg, Shiphra

2016-10-01

Quantitative scores fail to capture all important features of learner performance. This awareness has led to increased use of qualitative data when assessing health professionals. Yet the use of qualitative assessments is hampered by incomplete understanding of their role in forming judgments, and lack of consensus in how to appraise the rigor of judgments therein derived. The authors articulate the role of qualitative assessment as part of a comprehensive program of assessment, and translate the concept of validity to apply to judgments arising from qualitative assessments. They first identify standards for rigor in qualitative research, and then use two contemporary assessment validity frameworks to reorganize these standards for application to qualitative assessment.Standards for rigor in qualitative research include responsiveness, reflexivity, purposive sampling, thick description, triangulation, transparency, and transferability. These standards can be reframed using Messick's five sources of validity evidence (content, response process, internal structure, relationships with other variables, and consequences) and Kane's four inferences in validation (scoring, generalization, extrapolation, and implications). Evidence can be collected and evaluated for each evidence source or inference. The authors illustrate this approach using published research on learning portfolios.The authors advocate a "methods-neutral" approach to assessment, in which a clearly stated purpose determines the nature of and approach to data collection and analysis. Increased use of qualitative assessments will necessitate more rigorous judgments of the defensibility (validity) of inferences and decisions. Evidence should be strategically sought to inform a coherent validity argument.

Evaluation of Phosphorus Site Assessment Tools: Lessons from the USA.

Science.gov (United States)

Sharpley, Andrew; Kleinman, Peter; Baffaut, Claire; Beegle, Doug; Bolster, Carl; Collick, Amy; Easton, Zachary; Lory, John; Nelson, Nathan; Osmond, Deanna; Radcliffe, David; Veith, Tamie; Weld, Jennifer

2017-11-01

Critical source area identification through phosphorus (P) site assessment is a fundamental part of modern nutrient management planning in the United States, yet there has been only sparse testing of the many versions of the P Index that now exist. Each P site assessment tool was developed to be applicable across a range of field conditions found in a given geographic area, making evaluation extremely difficult. In general, evaluation with in-field monitoring data has been limited, focusing primarily on corroborating manure and fertilizer "source" factors. Thus, a multiregional effort (Chesapeake Bay, Heartland, and Southern States) was undertaken to evaluate P Indices using a combination of limited field data, as well as output from simulation models (i.e., Agricultural Policy Environmental eXtender, Annual P Loss Estimator, Soil and Water Assessment Tool [SWAT], and Texas Best Management Practice Evaluation Tool [TBET]) to compare against P Index ratings. These comparisons show promise for advancing the weighting and formulation of qualitative P Index components but require careful vetting of the simulation models. Differences among regional conclusions highlight model strengths and weaknesses. For example, the Southern States region found that, although models could simulate the effects of nutrient management on P runoff, they often more accurately predicted hydrology than total P loads. Furthermore, SWAT and TBET overpredicted particulate P and underpredicted dissolved P, resulting in correct total P predictions but for the wrong reasons. Experience in the United States supports expanded regional approaches to P site assessment, assuming closely coordinated efforts that engage science, policy, and implementation communities, but limited scientific validity exists for uniform national P site assessment tools at the present time. Copyright © by the American Society of Agronomy, Crop Science Society of America, and Soil Science Society of America, Inc.

Clinical validation of the C-VAT 2.0 assessment tool for gaming disorder: a sensitivity analysis of the proposed DSM-5 criteria and the clinical characteristics of young patients with ‘video game addiction’.

NARCIS (Netherlands)

Rooij, A.J. van; Schoenmakers, T.M.; Mheen, D. van de

2017-01-01

Aims: Clinicians struggle with the identification of video gaming problems. To address this issue, a clinical assessment tool (C-VAT 2.0) was developed and tested in a clinical setting. The instrument allows exploration of the validity of the DSM-5 proposal for ‘internet gaming disorder’. Method:

Feasibility, reliability, and validity of a smartphone based application for the assessment of cognitive function in the elderly.

Directory of Open Access Journals (Sweden)

Robert M Brouillette

Full Text Available While considerable knowledge has been gained through the use of established cognitive and motor assessment tools, there is a considerable interest and need for the development of a battery of reliable and validated assessment tools that provide real-time and remote analysis of cognitive and motor function in the elderly. Smartphones appear to be an obvious choice for the development of these "next-generation" assessment tools for geriatric research, although to date no studies have reported on the use of smartphone-based applications for the study of cognition in the elderly. The primary focus of the current study was to assess the feasibility, reliability, and validity of a smartphone-based application for the assessment of cognitive function in the elderly. A total of 57 non-demented elderly individuals were administered a newly developed smartphone application-based Color-Shape Test (CST in order to determine its utility in measuring cognitive processing speed in the elderly. Validity of this novel cognitive task was assessed by correlating performance on the CST with scores on widely accepted assessments of cognitive function. Scores on the CST were significantly correlated with global cognition (Mini-Mental State Exam: r = 0.515, p<0.0001 and multiple measures of processing speed and attention (Digit Span: r = 0.427, p<0.0001; Trail Making Test: r = -0.651, p<0.00001; Digit Symbol Test: r = 0.508, p<0.0001. The CST was not correlated with naming and verbal fluency tasks (Boston Naming Test, Vegetable/Animal Naming or memory tasks (Logical Memory Test. Test re-test reliability was observed to be significant (r = 0.726; p = 0.02. Together, these data are the first to demonstrate the feasibility, reliability, and validity of using a smartphone-based application for the purpose of assessing cognitive function in the elderly. The importance of these findings for the establishment of smartphone-based assessment batteries

Validation of Visual Caries Activity Assessment

DEFF Research Database (Denmark)

Guedes, R S; Piovesan, C; Ardenghi, T M

2014-01-01

We evaluated the predictive and construct validity of a caries activity assessment system associated with the International Caries Detection and Assessment System (ICDAS) in primary teeth. A total of 469 children were reexamined: participants of a caries survey performed 2 yr before (follow-up rate...... of 73.4%). At baseline, children (12-59 mo old) were examined with the ICDAS and a caries activity assessment system. The predictive validity was assessed by evaluating the risk of active caries lesion progression to more severe conditions in the follow-up, compared with inactive lesions. We also...... assessed if children with a higher number of active caries lesions were more likely to develop new lesions (construct validity). Noncavitated active caries lesions at occlusal surfaces presented higher risk of progression than inactive ones. Children with a higher number of active lesions and with higher...

Brief International Cognitive Assessment for MS (BICAMS): international standards for validation.

Science.gov (United States)

Benedict, Ralph H B; Amato, Maria Pia; Boringa, Jan; Brochet, Bruno; Foley, Fred; Fredrikson, Stan; Hamalainen, Paivi; Hartung, Hans; Krupp, Lauren; Penner, Iris; Reder, Anthony T; Langdon, Dawn

2012-07-16

An international expert consensus committee recently recommended a brief battery of tests for cognitive evaluation in multiple sclerosis. The Brief International Cognitive Assessment for MS (BICAMS) battery includes tests of mental processing speed and memory. Recognizing that resources for validation will vary internationally, the committee identified validation priorities, to facilitate international acceptance of BICAMS. Practical matters pertaining to implementation across different languages and countries were discussed. Five steps to achieve optimal psychometric validation were proposed. In Step 1, test stimuli should be standardized for the target culture or language under consideration. In Step 2, examiner instructions must be standardized and translated, including all information from manuals necessary for administration and interpretation. In Step 3, samples of at least 65 healthy persons should be studied for normalization, matched to patients on demographics such as age, gender and education. The objective of Step 4 is test-retest reliability, which can be investigated in a small sample of MS and/or healthy volunteers over 1-3 weeks. Finally, in Step 5, criterion validity should be established by comparing MS and healthy controls. At this time, preliminary studies are underway in a number of countries as we move forward with this international assessment tool for cognition in MS.

ADVISHE: A new tool to report validation of health-economic decision models

NARCIS (Netherlands)

Vemer, P.; Corro Ramos, I.; Van Voorn, G.; Al, M.J.; Feenstra, T.L.

2014-01-01

Background: Modelers and reimbursement decision makers could both profit from a more systematic reporting of the efforts to validate health-economic (HE) models. Objectives: Development of a tool to systematically report validation efforts of HE decision models and their outcomes. Methods: A gross

Computational assessment of hemodynamics-based diagnostic tools using a database of virtual subjects: Application to three case studies.

Science.gov (United States)

Willemet, Marie; Vennin, Samuel; Alastruey, Jordi

2016-12-08

Many physiological indexes and algorithms based on pulse wave analysis have been suggested in order to better assess cardiovascular function. Because these tools are often computed from in-vivo hemodynamic measurements, their validation is time-consuming, challenging, and biased by measurement errors. Recently, a new methodology has been suggested to assess theoretically these computed tools: a database of virtual subjects generated using numerical 1D-0D modeling of arterial hemodynamics. The generated set of simulations encloses a wide selection of healthy cases that could be encountered in a clinical study. We applied this new methodology to three different case studies that demonstrate the potential of our new tool, and illustrated each of them with a clinically relevant example: (i) we assessed the accuracy of indexes estimating pulse wave velocity; (ii) we validated and refined an algorithm that computes central blood pressure; and (iii) we investigated theoretical mechanisms behind the augmentation index. Our database of virtual subjects is a new tool to assist the clinician: it provides insight into the physical mechanisms underlying the correlations observed in clinical practice. Copyright © 2016 The Authors. Published by Elsevier Ltd.. All rights reserved.