Self-perceived assessment skill of prospective physics teachers

Science.gov (United States)

Efendi, R.; Rustaman, N. Y.; Kaniawati, I.

2018-05-01

Assessment skills are an important component of assessment practice, without adequate assessment skills it is unlikely that teacher assessment practices will produce desired student learning outcomes. This study was conducted to reveal self-perceived assessment skills of prospective physics teachers by using quantitative descriptive analysis, and involving 92 prospective physics teachers who were experiencing teaching practice in junior high school and final project related to assessment. Data was collected by using Self-Perceived Assessment Science Skills Questionnaire consisted of 29 items related seven assessment competencies was developed and used in the study. Internal consistency reliability coefficient for the total scale scores was 0.87 as measured by Cronbach’s alpha. Determination of self-perceived assessment science skills detected from prospective physics teachers was carried out in descriptive statistics, in the form of respondent average values. Research findings show that self-perceived assessment skills of prospective physics teachers was categorized as transition.

Crowd-sourced assessment of technical skills: an opportunity for improvement in the assessment of laparoscopic surgical skills.

Science.gov (United States)

Deal, Shanley B; Lendvay, Thomas S; Haque, Mohamad I; Brand, Timothy; Comstock, Bryan; Warren, Justin; Alseidi, Adnan

2016-02-01

Objective, unbiased assessment of surgical skills remains a challenge in surgical education. We sought to evaluate the feasibility and reliability of Crowd-Sourced Assessment of Technical Skills. Seven volunteer general surgery interns were given time for training and then testing, on laparoscopic peg transfer, precision cutting, and intracorporeal knot-tying. Six faculty experts (FEs) and 203 Amazon.com Mechanical Turk crowd workers (CWs) evaluated 21 deidentified video clips using the Global Objective Assessment of Laparoscopic Skills validated rating instrument. Within 19 hours and 15 minutes we received 662 eligible ratings from 203 CWs and 126 ratings from 6 FEs over 10 days. FE video ratings were of borderline internal consistency (Krippendorff's alpha = .55). FE ratings were highly correlated with CW ratings (Pearson's correlation coefficient = .78, P Assessment of Technical Skills as a reliable, basic tool to standardize the evaluation of technical skills in general surgery. Copyright © 2016 Elsevier Inc. All rights reserved.

Enhancing Student Communication Skills Through Arabic Language Competency and Simulated Patient Assessments.

Science.gov (United States)

Hasan, Sanah; Tarazi, Hamadeh M Khier; Halim Hilal, Dana Abdel

2017-05-01

Objective. To assess student communication and patient management skill with introduction of Arabic and use of simulated patient assessments to a communication and counseling course. Design. Five, 3-hour tutorials (clinical skill laboratory) were added to the course covering: listening and empathic responding, non-verbal communications, interviewing skills, assertiveness, counseling in special situations: conflict, anger, worry or rushed situations, and professional decision making. Arabic content was introduced to the course to enhance Arabic communications and competence among students. Simulated patient assessment was used to evaluate student skills. Students' feedback about course changes was evaluated. Assessment. The course now covers a wider content and Arabic language. Students' scores were similar in the assessment and other assessments within the course and between Arabic and English groups. Students favorably rated the changes in the course and provided constructive feedback on content usefulness and adequacy. Conclusion. Expanding the course to include Arabic language and content and simulated patient assessments enhanced student communication skills.

Crowd-sourced assessment of surgical skills in cricothyrotomy procedure.

Science.gov (United States)

Aghdasi, Nava; Bly, Randall; White, Lee W; Hannaford, Blake; Moe, Kris; Lendvay, Thomas S

2015-06-15

Objective assessment of surgical skills is resource intensive and requires valuable time of expert surgeons. The goal of this study was to assess the ability of a large group of laypersons using a crowd-sourcing tool to grade a surgical procedure (cricothyrotomy) performed on a simulator. The grading included an assessment of the entire procedure by completing an objective assessment of technical skills survey. Two groups of graders were recruited as follows: (1) Amazon Mechanical Turk users and (2) three expert surgeons from University of Washington Department of Otolaryngology. Graders were presented with a video of participants performing the procedure on the simulator and were asked to grade the video using the objective assessment of technical skills questions. Mechanical Turk users were paid $0.50 for each completed survey. It took 10 h to obtain all responses from 30 Mechanical Turk users for 26 training participants (26 videos/tasks), whereas it took 60 d for three expert surgeons to complete the same 26 tasks. The assessment of surgical performance by a group (n = 30) of laypersons matched the assessment by a group (n = 3) of expert surgeons with a good level of agreement determined by Cronbach alpha coefficient = 0.83. We found crowd sourcing was an efficient, accurate, and inexpensive method for skills assessment with a good level of agreement to experts' grading. Copyright © 2015 Elsevier Inc. All rights reserved.

A qualitative study of advanced nurse practitioners' use of physical assessment skills in the community: shifting skills across professional boundaries.

Science.gov (United States)

Raleigh, Mary; Allan, Helen

2017-07-01

To explore multiple perspectives on the use of physical assessment skills by advanced nurse practitioners in the UK. Physical assessment skills practices are embedded in advanced nursing practice roles in the UK. There is little evidence on how these skills are used by advanced nurse practitioners in the community. Case study. A qualitative interpretative single-embedded case study of 22 participants from South of England. A framework method analysed interview data collected by the researcher between March-August 2013. Participants included nurses, doctors, nurse educators and managers. Physical assessment skills education at universities is part of a policy shift to develop a flexible workforce in the UK. Shared physical assessment practices are less to do with role substitution and more about preparing practitioners with skills that are fit for purpose. Competence, capability and performance with physical assessment skills are an expectation of advanced nursing practice. These skills are used successfully by community advanced nurse practitioners to deliver a wide range of services in response to changing patient need. The introduction of physical assessment skills education to undergraduate professional preparation would create a firm foundation to develop these skills in postgraduate education. Physical assessment education prepares nurses with the clinical competencies to carry out healthcare reforms in the UK. Shared sets of clinical assessment competencies between disciplines have better outcomes for patients. Levels of assessment competence can depend on the professional attributes of individual practitioners. Unsupportive learning cultures can hinder professional development of advanced nursing practice. © 2016 John Wiley & Sons Ltd.

Self-Assessment of Video-Recorded Presentations: Does It Improve Skills?

Science.gov (United States)

Ritchie, Sian M.

2016-01-01

The opportunity to give oral presentations is important because it helps develop skills that are necessary for many aspects of professional careers. Competencies for oral presentations include appropriate content, design and organization of the presentation, and delivery skills. Self-assessment has been shown to help students develop…

The role of multimedia in surgical skills training and assessment.

Science.gov (United States)

Shariff, Umar; Seretis, Charalampos; Lee, Doreen; Balasubramanian, Saba P

2016-06-01

Multimedia is an educational resource that can be used to supplement surgical skills training. The aim of this review was to determine the role of multimedia in surgical training and assessment by performing a systematic review of the literature. A systematic review for published articles was conducted on the following databases: PubMed/MEDLINE (1992 to November 2014), SCOPUS (1992 to November 2014) and EMBASE (1992 to November 2014). For each study the educational content, study design, surgical skill assessed and outcomes were recorded. A standard data extraction form was created to ensure systematic retrieval of relevant information. 21 studies were included; 14 randomized controlled trials (RCTs) and 7 non-randomized controlled trials (Non-RCTs). Technical skills were assessed in 7 RCTs and 3 non-RCTs; cognitive skills were assessed in 9 RCTs and 4 non-RCTs. In controlled studies, multimedia was associated with significant improvement in technical skills (4 studies; 4 RCTs) and cognitive skills (7 studies; 6 RCTs). In two studies multimedia was inferior in comparison to conventional teaching. Evaluation of multimedia (9 studies) demonstrated strongly favourable results. This review suggests that multimedia effectively facilitates both technical and cognitive skills acquisition and is well accepted as an educational resource. Copyright © 2015 Royal College of Surgeons of Edinburgh (Scottish charity number SC005317) and Royal College of Surgeons in Ireland. Published by Elsevier Ltd. All rights reserved.

Basic Laparoscopic Skills Assessment Study: Validation and Standard Setting among Canadian Urology Trainees.

Science.gov (United States)

Lee, Jason Y; Andonian, Sero; Pace, Kenneth T; Grober, Ethan

2017-06-01

As urology training programs move to a competency based medical education model, iterative assessments with objective standards will be required. To develop a valid set of technical skills standards we initiated a national skills assessment study focusing initially on laparoscopic skills. Between February 2014 and March 2016 the basic laparoscopic skill of Canadian urology trainees and attending urologists was assessed using 4 standardized tasks from the AUA (American Urological Association) BLUS (Basic Laparoscopic Urological Surgery) curriculum, including peg transfer, pattern cutting, suturing and knot tying, and vascular clip applying. All performances were video recorded and assessed using 3 methods, including time and error based scoring, expert global rating scores and C-SATS (Crowd-Sourced Assessments of Technical Skill Global Rating Scale), a novel, crowd sourced assessment platform. Different methods of standard setting were used to develop pass-fail cut points. Six attending urologists and 99 trainees completed testing. Reported laparoscopic experience and training level correlated with performance (p standard setting methods to define pass-fail cut points for all 4 AUA BLUS tasks. The 4 AUA BLUS tasks demonstrated good construct validity evidence for use in assessing basic laparoscopic skill. Performance scores using the novel C-SATS platform correlated well with traditional time-consuming methods of assessment. Various standard setting methods were used to develop pass-fail cut points for educators to use when making formative and summative assessments of basic laparoscopic skill. Copyright © 2017 American Urological Association Education and Research, Inc. Published by Elsevier Inc. All rights reserved.

Assessment of Surgical Skills and Competency.

Science.gov (United States)

Bhatti, Nasir I

2017-10-01

Evaluation of surgical skills and competency are important aspects of the medical education process. Measurable and reproducible methods of assessment with objective feedback are essential components of surgical training. Objective Structured Assessment of Technical Skills (OSATS) is widely used across the medical specialties and otolaryngology-specific tools have been developed and validated for sinus and mastoid surgery. Although assessment of surgical skills can be time-consuming and requires human and financial resources, new evaluation methods and emerging technology may alleviate these barriers while also improving data collection practices. Copyright © 2017 Elsevier Inc. All rights reserved.

Assessing Trainee Surgeons’ Nontechnical Skills

DEFF Research Database (Denmark)

Spanager, Lene; Konge, Lars; Dieckmann, Peter

2015-01-01

BACKGROUND: Trainee surgeons would benefit from regular, formative assessments to ensure they learn the nontechnical aspects of surgical performance. Non-Technical Skills for Surgeons in Denmark (NOTSSdk) is a tool to assess surgeons' nontechnical skills (NTS) during an operation. The aims...... of this study were to explore which parts of NOTSSdk supervisors use to assess trainee surgeons' NTS, to determine the internal consistency reliability of NOTSSdk, and to estimate how many operations were needed to obtain reliable ratings of a trainee surgeon's NTS. METHODS: A total of 12 supervisors from 2...

Teacher Assessment of Practical Skills in A-Level Chemistry

Science.gov (United States)

Wood, R.; Ferguson, Carolyn M.

1975-01-01

Discusses a two-year assessment undertaken to evaluate the Nuffield A-Level chemistry course. Secondary teachers selected chemistry experiments for assessment purposes and assessed their students in manipulative skills, observational skills, interpretation skills, creative skills, and attitudes. (MLH)

Government services: sector skills assessment 2012

OpenAIRE

Parry, Jon; Ojo, Adegbola

2012-01-01

The aim of this report is to provide authoritative labour market intelligence (LMI) for the Government Services sector to inform the strategic decision making of national governments in the development of employment and skills policy. It is one of 15 UK Sector Skills Assessment (SSA) reports produced by Sector Skills Councils and the UK Commission for Employment and Skills.

Development of performance assessment instrument based contextual learning for measuring students laboratory skills

Science.gov (United States)

Susilaningsih, E.; Khotimah, K.; Nurhayati, S.

2018-04-01

The assessment of laboratory skill in general hasn’t specific guideline in assessment, while the individual assessment of students during a performance and skill in performing laboratory is still not been observed and measured properly. Alternative assessment that can be used to measure student laboratory skill is use performance assessment. The purpose of this study was to determine whether the performance assessment instrument that the result of research can be used to assess basic skills student laboratory. This research was conducted by the Research and Development. The result of the data analysis performance assessment instruments developed feasible to implement and validation result 62.5 with very good categories for observation sheets laboratory skills and all of the components with the very good category. The procedure is the preliminary stages of research and development stages. Preliminary stages are divided in two, namely the field studies and literature studies. The development stages are divided into several parts, namely 1) development of the type instrument, 2) validation by an expert, 3) a limited scale trial, 4) large-scale trials and 5) implementation of the product. The instrument included in the category of effective because 26 from 29 students have very high laboratory skill and high laboratory skill. The research of performance assessment instrument is standard and can be used to assess basic skill student laboratory.

Non-technical skills assessment for prelicensure nursing students: An integrative review.

Science.gov (United States)

Pires, Sara; Monteiro, Sara; Pereira, Anabela; Chaló, Daniela; Melo, Elsa; Rodrigues, Alexandre

2017-11-01

In nursing, non-technical skills are recognized as playing an important role to increase patient safety and successful clinical outcomes (Pearson and McLafferty, 2011). Non-technical skills are cognitive and social resource skills that complement technical skills and contribute to safe and efficient task performance (Flin et al., 2008). In order to effectively provide non-technical skills training, it is essential to have an instrument to measure these skills. An online search was conducted. Articles were selected if they referred to and/or described instruments assessing non-technical skills for nurses and/or prelicensure nursing students in educational, clinical and/or simulated settings with validation evidence (inclusion criteria). Of the 53 articles located, 26 met the inclusion criteria. Those referred to and/or described 16 instruments with validation evidence developed to assess non-technical skills in multidisciplinary teams including nurses. Although articles have shown 16 valid and reliable instruments, to our knowledge, no instrument has been published or developed and validated for the assessment of non-technical skills of only nurses in general, relevant for use in high-fidelity simulation-based training for prelicensure nursing students. Therefore, there is a need for the development of such an instrument. Copyright © 2017 Elsevier Ltd. All rights reserved.

Enabling performance skills: Assessment in engineering education

Science.gov (United States)

Ferrone, Jenny Kristina

Current reform in engineering education is part of a national trend emphasizing student learning as well as accountability in instruction. Assessing student performance to demonstrate accountability has become a necessity in academia. In newly adopted criterion proposed by the Accreditation Board for Engineering and Technology (ABET), undergraduates are expected to demonstrate proficiency in outcomes considered essential for graduating engineers. The case study was designed as a formative evaluation of freshman engineering students to assess the perceived effectiveness of performance skills in a design laboratory environment. The mixed methodology used both quantitative and qualitative approaches to assess students' performance skills and congruency among the respondents, based on individual, team, and faculty perceptions of team effectiveness in three ABET areas: Communications Skills. Design Skills, and Teamwork. The findings of the research were used to address future use of the assessment tool and process. The results of the study found statistically significant differences in perceptions of Teamwork Skills (p performance skills, such as teamwork, among freshman engineering students; (2) incorporate feedback into the learning process; (3) strengthen the assessment process with a follow-up plan that specifically targets performance skill deficiencies, and (4) integrate the assessment instrument and practice with ongoing curriculum development. The findings generated by this study provides engineering departments engaged in assessment activity, opportunity to reflect, refine, and develop their programs as it continues. It also extends research on ABET competencies of engineering students in an under-investigated topic of factors correlated with team processes, behavior, and student learning.

Attitudes to proposed assessment of pharmacy skills in Korean pharmacist licensure examination.

Science.gov (United States)

Kim, Joo Hee; Lee, Ju-Yeun; Lee, Young Sook; Yong, Chul-Soon; Han, Nayoung; Gwak, Hye Sun; Oh, Jungmi; Lee, Byung Koo; Lee, Sukhyang

2017-01-01

The survey aimed to obtain opinions about a proposed implementation of pharmacy skills assessment in Korean pharmacist licensure examination (KPLE). A 16-question survey was distributed electronically to 2,738 people including 570 pharmacy professors of 35 pharmacy schools, 550 preceptors from 865 practice sites and 1,618 students who graduated in 2015. The survey solicited responses concerning the adequacy of the current KPLE in assessing pharmacy knowledge/skills/attitudes, deficiencies of pharmacy skills testing in assessing the professional competencies necessary for pharmacists, plans for pharmacy skills tests in the current KPLE, and subject areas of pharmacy practice. A total of 466 surveys were returned. The current exam is not adequate for assessing skills and attitudes according to 42%-48% of respondents. Sixty percent felt that skills test is necessary to assess qualifications and professional competencies. Almost two-thirds of participants stated that testing should be implemented within 5 years. More than 60% agreed that candidates should be graduates and that written and skills test scores can be combined for pass-fail decisions. About 70% of respondents felt that the test should be less than 2 hours in duration. Over half of the respondents thought that the assessor should be a pharmacy faculty member with at least 5 years of clinical experience. Up to 70% stated that activities related to patient care were appropriate and practical for the scope of skills test. Pharmacy skills assessment was supported by the majority of respondents.

The IDEA Assessment Tool: Assessing the Reporting, Diagnostic Reasoning, and Decision-Making Skills Demonstrated in Medical Students' Hospital Admission Notes.

Science.gov (United States)

Baker, Elizabeth A; Ledford, Cynthia H; Fogg, Louis; Way, David P; Park, Yoon Soo

2015-01-01

Construct: Clinical skills are used in the care of patients, including reporting, diagnostic reasoning, and decision-making skills. Written comprehensive new patient admission notes (H&Ps) are a ubiquitous part of student education but are underutilized in the assessment of clinical skills. The interpretive summary, differential diagnosis, explanation of reasoning, and alternatives (IDEA) assessment tool was developed to assess students' clinical skills using written comprehensive new patient admission notes. The validity evidence for assessment of clinical skills using clinical documentation following authentic patient encounters has not been well documented. Diagnostic justification tools and postencounter notes are described in the literature (1,2) but are based on standardized patient encounters. To our knowledge, the IDEA assessment tool is the first published tool that uses medical students' H&Ps to rate students' clinical skills. The IDEA assessment tool is a 15-item instrument that asks evaluators to rate students' reporting, diagnostic reasoning, and decision-making skills based on medical students' new patient admission notes. This study presents validity evidence in support of the IDEA assessment tool using Messick's unified framework, including content (theoretical framework), response process (interrater reliability), internal structure (factor analysis and internal-consistency reliability), and relationship to other variables. Validity evidence is based on results from four studies conducted between 2010 and 2013. First, the factor analysis (2010, n = 216) yielded a three-factor solution, measuring patient story, IDEA, and completeness, with reliabilities of .79, .88, and .79, respectively. Second, an initial interrater reliability study (2010) involving two raters demonstrated fair to moderate consensus (κ = .21-.56, ρ =.42-.79). Third, a second interrater reliability study (2011) with 22 trained raters also demonstrated fair to moderate agreement

Teaching and Assessing Communication Skills in Medical Undergraduate Training.

Science.gov (United States)

Modi, Jyoti Nath; Anshu, -; Chhatwal, Jugesh; Gupta, Piyush; Singh, Tejinder

2016-06-08

Good communication skills are essential for an optimal doctor-patient relationship, and also contribute to improved health outcomes. Although the need for training in communication skills is stated as a requirement in the 1997 Graduate Medical Education Regulations of the Medical Council of India, formal training in these skills has been fragmentary and non-uniform in most Indian curricula. The Vision 2015 document of the Medical Council of India reaffirms the need to include training in communication skills in the MBBS curriculum. Training in communication skills needs approaches which are different from that of teaching other clinical subjects. It is also a challenge to ensure that students not only imbibe the nuances of communication and interpersonal skills, but adhere to them throughout their careers. This article addresses the possible ways of standardizing teaching and assessment of communication skills and integrating them into the existing curriculum.

Peer Assessment of Elementary Science Teaching Skills

Science.gov (United States)

Kilic, Gulsen Bagci; Cakan, Mehtap

2007-01-01

In this study, peer assessment was applied in assessing elementary science teaching skills. Preservice teachers taught a science topic as a team to their peers in an elementary science methods course. The peers participating in the science lesson assessed teacher-groups' elementary science teaching skills on an assessment form provided by the…

Ecosystem Model Skill Assessment. Yes We Can!

Science.gov (United States)

Olsen, Erik; Fay, Gavin; Gaichas, Sarah; Gamble, Robert; Lucey, Sean; Link, Jason S

2016-01-01

Accelerated changes to global ecosystems call for holistic and integrated analyses of past, present and future states under various pressures to adequately understand current and projected future system states. Ecosystem models can inform management of human activities in a complex and changing environment, but are these models reliable? Ensuring that models are reliable for addressing management questions requires evaluating their skill in representing real-world processes and dynamics. Skill has been evaluated for just a limited set of some biophysical models. A range of skill assessment methods have been reviewed but skill assessment of full marine ecosystem models has not yet been attempted. We assessed the skill of the Northeast U.S. (NEUS) Atlantis marine ecosystem model by comparing 10-year model forecasts with observed data. Model forecast performance was compared to that obtained from a 40-year hindcast. Multiple metrics (average absolute error, root mean squared error, modeling efficiency, and Spearman rank correlation), and a suite of time-series (species biomass, fisheries landings, and ecosystem indicators) were used to adequately measure model skill. Overall, the NEUS model performed above average and thus better than expected for the key species that had been the focus of the model tuning. Model forecast skill was comparable to the hindcast skill, showing that model performance does not degenerate in a 10-year forecast mode, an important characteristic for an end-to-end ecosystem model to be useful for strategic management purposes. We identify best-practice approaches for end-to-end ecosystem model skill assessment that would improve both operational use of other ecosystem models and future model development. We show that it is possible to not only assess the skill of a complicated marine ecosystem model, but that it is necessary do so to instill confidence in model results and encourage their use for strategic management. Our methods are applicable

Clinical Skills Assessment in the Twenty-First Century.

Science.gov (United States)

Elder, Andrew

2018-05-01

Clinical skills remain fundamental to the practice of medicine and form a core component of the professional identity of the physician. However, evidence exists to suggest that the practice of some clinical skills is declining, particularly in the United States. A decline in practice of any skill can lead to a decline in its teaching and assessment, with further decline in practice as a result. Consequently, assessment not only drives learning of clinical skills, but their practice. This article summarizes contemporary approaches to clinical skills assessment that, if more widely adopted, could support the maintenance and reinvigoration of bedside clinical skills. Crown Copyright © 2017. Published by Elsevier Inc. All rights reserved.

Teaching and Assessing Engineering Professional Skills

Directory of Open Access Journals (Sweden)

Ali M. Al-Bahi

2013-05-01

Full Text Available Engineering students are required to have, by the time of graduation, a set of professional skills related to teamwork, oral and written communications, impact of engineering solutions, life-long learning, and knowledge of contemporary issues. Teaching and assessment of these skills, as part of ABET accreditation, remains problematic. A systematic methodology to integrate these skills and their assessment in the curriculum is described. The method was recently applied in several engineering programs and proved to be efficient in generating data and evidences for evaluation and continuous improvement of these outcomes.

Nurses' Evaluation of Their Use and Mastery in Health Assessment Skills: Selected Iran's Hospitals.

Science.gov (United States)

Adib-Hajbaghery, Mohsen; Safa, Azade

2013-09-01

Health assessment skills are of the most important skills which nurses require. The more precise assessment, the better results would be obtained and the quality of patient care would be improved. However, in Iran, few studies have investigated nurses' assessment skills. This study was aimed to assessnurses' evaluation of the learned skills of health assessment and their use. This cross-sectional study was conducted on 200 nurses in Isfahan province hospitals. Data was collected by a questionnaire including demographic data and 120 health assessment skills. Nurses scored their frequency of using and proficiency in skills. Statistical analysis was conducted by ANOVA, Tukey test and independent sample T-tests. The highest level of using and proficiency in skills was related to taking history. Nurses received 87.25% of score in this field. The lowest level of application was in assessment of the urogenital system so that nurses received 16.37% of score in this area. Also the lowest proficiency was in assessment of the nervous system and nurses received 34.58% of score in this area. The level of nurses' proficiency in the health assessment skills was not satisfactory. Modifying the curriculum and cooperating of nurse managers and nursing schools can help to improve the situation.

Evaluating structured assessment of anaesthesiologists' non-technical skills

DEFF Research Database (Denmark)

Jepsen, R M H G; Dieckmann, P; Spanager, L

2016-01-01

BACKGROUND: Non-technical skills (NTS) are essential for safe and efficient anaesthesia. Assessment instruments with appropriate validity evidence can be used to ensure that anaesthesiologists possess the NTS necessary to deliver high-standard patient care. The aims were to collect validity...... evidence using a contemporary validity framework for the assessment instrument Anaesthesiologists' Non-Technical Skills in Denmark (ANTSdk) regarding response process and internal structure (including reliability), and to investigate the effect of rater training on these properties. METHODS: An explorative...... study was undertaken at the Danish Institute for Medical Simulation, Copenhagen, Denmark. In a 1-day session, using ANTSdk, a convenience sample of 19 anaesthesiologists rated trainee anaesthesiologists' NTS in nine video-recorded simulation scenarios before and after a 3-h training session. RESULTS...

Does Computer-Based Motor Skill Assessment Training Transfer to Live Assessing?

Science.gov (United States)

Kelly, Luke E.; Taliaferro, Andrea; Krause, Jennifer

2012-01-01

Developing competency in motor skill assessment has been identified as a critical need in physical educator preparation. We conducted this study to evaluate (a) the effectiveness of a web-based instructional program--Motor Skill Assessment Program (MSAP)--for developing assessment competency, and specifically (b) whether competency developed by…

[Use of physical assessment skills and education needs of advanced practice nurses and nurse specialists].

Science.gov (United States)

Shin, Hyunsook; Kim, Bog-Ja; Kang, Hee Sun

2009-10-01

The study was done to investigate physical assessment skills used by, and educational needs of, advanced practice nurses (APNs) and nurse specialists in Korea. A total of 123 APNs and nurse specialists working in five major hospitals in Seoul were surveyed from July 15 to August 20, 2007. Fourteen skills out of 126 items were reported as being performed on a regular basis by participants. The majority of these skills involved general observation. Forty-six skills were rarely used. Some participants showed a lack of confidence in certain assessment skills, such as in doing a rectal or pelvic exam, and the use of some assessment equipment. Over 90% of participants required in-depth education on health assessment provided by specialists or nursing professional organizations. More educational opportunities in physical assessment should be provided including education programs based on the nurses' skill levels and needs. This effort will help to increase confidence of APNs and nurse specialists in physical assessment skills, ultimately resulting in better nursing outcomes.

Training and Assessment of Hysteroscopic Skills: A Systematic Review.

Science.gov (United States)

Savran, Mona Meral; Sørensen, Stine Maya Dreier; Konge, Lars; Tolsgaard, Martin G; Bjerrum, Flemming

2016-01-01

The aim of this systematic review was to identify studies on hysteroscopic training and assessment. PubMed, Excerpta Medica, the Cochrane Library, and Web of Science were searched in January 2015. Manual screening of references and citation tracking were also performed. Studies on hysteroscopic educational interventions were selected without restrictions on study design, populations, language, or publication year. A qualitative data synthesis including the setting, study participants, training model, training characteristics, hysteroscopic skills, assessment parameters, and study outcomes was performed by 2 authors working independently. Effect sizes were calculated when possible. Overall, 2 raters independently evaluated sources of validity evidence supporting the outcomes of the hysteroscopy assessment tools. A total of 25 studies on hysteroscopy training were identified, of which 23 were performed in simulated settings. Overall, 10 studies used virtual-reality simulators and reported effect sizes for technical skills ranging from 0.31 to 2.65; 12 used inanimate models and reported effect sizes for technical skills ranging from 0.35 to 3.19. One study involved live animal models; 2 studies were performed in clinical settings. The validity evidence supporting the assessment tools used was low. Consensus between the 2 raters on the reported validity evidence was high (94%). This systematic review demonstrated large variations in the effect of different tools for hysteroscopy training. The validity evidence supporting the assessment of hysteroscopic skills was limited. Copyright © 2016 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

A contextual approach to social skills assessment in the peer group: who is the best judge?

Science.gov (United States)

Kwon, Kyongboon; Kim, Elizabeth Moorman; Sheridan, Susan M

2012-09-01

Using a contextual approach to social skills assessment in the peer group, this study examined the criterion-related validity of contextually relevant social skills and the incremental validity of peers and teachers as judges of children's social skills. Study participants included 342 (180 male and 162 female) students and their classroom teachers (N = 22) from rural communities. As expected, contextually relevant social skills were significantly related to a variety of social status indicators (i.e., likability, peer- and teacher-assessed popularity, reciprocated friendships, clique centrality) and positive school functioning (i.e., school liking and academic competence). Peer-assessed social skills, not teacher-assessed social skills, demonstrated consistent incremental validity in predicting various indicators of social status outcomes; peer- and teacher-assessed social skills alike showed incremental validity in predicting positive school functioning. The relation between contextually relevant social skills and study outcomes did not vary by child gender. Findings are discussed in terms of the significance of peers in the assessment of children's social skills in the peer group as well as the usefulness of a contextual approach to social skills assessment.

Attention to gender in communication skills assessment instruments in medical education: a review.

Science.gov (United States)

Dielissen, Patrick; Bottema, Ben; Verdonk, Petra; Lagro-Janssen, Toine

2011-03-01

Gender is increasingly regarded as an important factor in doctor-patient communication education. This review aims to assess if and how gender is addressed by current assessment instruments for communication skills in medical education. In 2009 at Radboud University Nijmegen Medical Centre, an online search was conducted in the bibliographic databases PubMed, PsycINFO and ERIC for references about communication skills assessment instruments designed to be completed by trained faculty staff and used in medical education. The search strategy used the following search terms: 'consultation skills'; 'doctor-patient communication'; 'physician-patient relations'; 'medical education'; 'instruments'; 'measurement', and 'assessment'. Papers published between January 1999 and June 2009 were included. The assessment instruments identified were analysed for gender-specific content. The search yielded 21 communication skills assessment instruments. Only two of the 17 checklists obtained explicitly considered gender as a communication-related issue. Only six of 21 manuals considered gender in any way and none gave specific details to explain which aspects of communication behaviour should be assessed with regard to gender. Very few communication assessment instruments in medical education focus on gender. Nevertheless, interest exists in using gender in communication skills assessment. The criteria for and purpose of assessing gender in communication skills in medical education are yet to be clarified. © Blackwell Publishing Ltd 2011.

Technical skills assessment toolbox: a review using the unitary framework of validity.

Science.gov (United States)

Ghaderi, Iman; Manji, Farouq; Park, Yoon Soo; Juul, Dorthea; Ott, Michael; Harris, Ilene; Farrell, Timothy M

2015-02-01

The purpose of this study was to create a technical skills assessment toolbox for 35 basic and advanced skills/procedures that comprise the American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) surgical skills curriculum and to provide a critical appraisal of the included tools, using contemporary framework of validity. Competency-based training has become the predominant model in surgical education and assessment of performance is an essential component. Assessment methods must produce valid results to accurately determine the level of competency. A search was performed, using PubMed and Google Scholar, to identify tools that have been developed for assessment of the targeted technical skills. A total of 23 assessment tools for the 35 ACS/APDS skills modules were identified. Some tools, such as Operative Performance Rating System (OSATS) and Objective Structured Assessment of Technical Skill (OPRS), have been tested for more than 1 procedure. Therefore, 30 modules had at least 1 assessment tool, with some common surgical procedures being addressed by several tools. Five modules had none. Only 3 studies used Messick's framework to design their validity studies. The remaining studies used an outdated framework on the basis of "types of validity." When analyzed using the contemporary framework, few of these studies demonstrated validity for content, internal structure, and relationship to other variables. This study provides an assessment toolbox for common surgical skills/procedures. Our review shows that few authors have used the contemporary unitary concept of validity for development of their assessment tools. As we progress toward competency-based training, future studies should provide evidence for various sources of validity using the contemporary framework.

Energy-Based Metrics for Arthroscopic Skills Assessment.

Science.gov (United States)

Poursartip, Behnaz; LeBel, Marie-Eve; McCracken, Laura C; Escoto, Abelardo; Patel, Rajni V; Naish, Michael D; Trejos, Ana Luisa

2017-08-05

Minimally invasive skills assessment methods are essential in developing efficient surgical simulators and implementing consistent skills evaluation. Although numerous methods have been investigated in the literature, there is still a need to further improve the accuracy of surgical skills assessment. Energy expenditure can be an indication of motor skills proficiency. The goals of this study are to develop objective metrics based on energy expenditure, normalize these metrics, and investigate classifying trainees using these metrics. To this end, different forms of energy consisting of mechanical energy and work were considered and their values were divided by the related value of an ideal performance to develop normalized metrics. These metrics were used as inputs for various machine learning algorithms including support vector machines (SVM) and neural networks (NNs) for classification. The accuracy of the combination of the normalized energy-based metrics with these classifiers was evaluated through a leave-one-subject-out cross-validation. The proposed method was validated using 26 subjects at two experience levels (novices and experts) in three arthroscopic tasks. The results showed that there are statistically significant differences between novices and experts for almost all of the normalized energy-based metrics. The accuracy of classification using SVM and NN methods was between 70% and 95% for the various tasks. The results show that the normalized energy-based metrics and their combination with SVM and NN classifiers are capable of providing accurate classification of trainees. The assessment method proposed in this study can enhance surgical training by providing appropriate feedback to trainees about their level of expertise and can be used in the evaluation of proficiency.

Accuracy of young male drivers’ self-assessments of driving skill

DEFF Research Database (Denmark)

Martinussen, Laila Marianne; Møller, Mette; Prato, Carlo Giacomo

2017-01-01

Accurate self-assessment of skill is important because it creates an appropriate level of confidence and hence behaviour. Inaccurate self-assessment of driving ability has been linked to reckless driving and accidents. Inaccurate self-assessment of driving skills may be a contributing factor...... to the over-representation of young male drivers in accident statistics. Most previous research on self-assessment of driving skills did not compare self-reported skills to objectively measured driving skills, so the aims of this study were: (1) to test the accuracy of young male drivers’ self......-assessments of specific driving skills by comparing them with performance in a driving simulator; (2) to test whether self-assessment accuracy varied with driving skill, driving experience and sensation-seeking propensity. We found that young male drivers’ self-assessments were inconsistent with their driving performance...

Development and Application of a Stepwise Assessment Process for Rational Redesign of Sequential Skills-Based Courses.

Science.gov (United States)

Gallimore, Casey E; Porter, Andrea L; Barnett, Susanne G

2016-10-25

Objective. To develop and apply a stepwise process to assess achievement of course learning objectives related to advanced pharmacy practice experiences (APPEs) preparedness and inform redesign of sequential skills-based courses. Design. Four steps comprised the assessment and redesign process: (1) identify skills critical for APPE preparedness; (2) utilize focus groups and course evaluations to determine student competence in skill performance; (3) apply course mapping to identify course deficits contributing to suboptimal skill performance; and (4) initiate course redesign to target exposed deficits. Assessment. Focus group participants perceived students were least prepared for skills within the Accreditation Council for Pharmacy Education's pre-APPE core domains of Identification and Assessment of Drug-related Problems and General Communication Abilities. Course mapping identified gaps in instruction, performance, and assessment of skills within aforementioned domains. Conclusions. A stepwise process that identified strengths and weaknesses of a course, was used to facilitate structured course redesign. Strengths of the process included input and corroboration from both preceptors and students. Limitations included feedback from a small number of pharmacy preceptors and increased workload on course coordinators.

Technical and Technological Skills Assessment in Laparoscopic Surgery

Science.gov (United States)

Chang, Avril; Vincent, Charles

2006-01-01

Objectives: Surgical appraisal and revalidation are key components of good surgical practice and training. Assessing technical skills in a structured manner is still not widely used. Laparoscopic surgery also requires the surgeon to be competent in technological aspects of the operation. Methods: Checklists for generic, specific technical, and technological skills for laparoscopic cholecystectomies were constructed. Two surgeons with >12 years postgraduate surgical experience assessed each operation blindly and independently on DVD. The technological skills were assessed in the operating room. Results: One hundred operations were analyzed. Eight trainees and 10 consultant surgeons were recruited. No adverse events occurred due to technical or technological skills. Mean interrater reliability was kappa=0.88, P=technical and technological skills between trainee and consultant surgeons were significant, Mann-Whitney P=technical and technological skills can be measured to assess performance of laparoscopic surgeons. This technical and technological assessment tool for laparoscopic surgery seems to have face, content, concurrent, and construct validities and could be modified and applied to any laparoscopic operation. The tool has the possibility of being used in surgical training and appraisal. We aim to modify and apply this tool to advanced laparoscopic operations. PMID:17212881

Metrics for Objective Assessment of Surgical Skills Workshop

National Research Council Canada - National Science Library

Satava, Richard

2001-01-01

On 9-10 July, 2001 the Metrics for Objective Assessment of Surgical Skills Workshop convened an international assemblage of subject matter experts in objective assessment of surgical technical skills...

Training and Assessment of Hysteroscopic Skills

DEFF Research Database (Denmark)

Savran, Mona Meral; Sørensen, Stine Maya Dreier; Konge, Lars

2016-01-01

OBJECTIVE: The aim of this systematic review was to identify studies on hysteroscopic training and assessment. DESIGN: PubMed, Excerpta Medica, the Cochrane Library, and Web of Science were searched in January 2015. Manual screening of references and citation tracking were also performed. Studies...... on hysteroscopic educational interventions were selected without restrictions on study design, populations, language, or publication year. A qualitative data synthesis including the setting, study participants, training model, training characteristics, hysteroscopic skills, assessment parameters, and study...... outcomes was performed by 2 authors working independently. Effect sizes were calculated when possible. Overall, 2 raters independently evaluated sources of validity evidence supporting the outcomes of the hysteroscopy assessment tools. RESULTS: A total of 25 studies on hysteroscopy training were identified...

Multi-informant assessment of transition-related skills and skill importance in adolescents with autism spectrum disorder.

Science.gov (United States)

Hume, Kara; Dykstra Steinbrenner, Jessica; Sideris, John; Smith, Leann; Kucharczyk, Suzanne; Szidon, Kate

2018-01-01

Adolescents with autism spectrum disorder have limited participation in the transition planning process, despite the link between active participation and an improvement in postsecondary education and employment outcomes. The Secondary School Success Checklist was designed to support transition planning for adolescents with autism spectrum disorder by incorporating their own assessments of strengths, skill deficits, and prioritization for instruction along with those of their parents and teachers across multiple skill domains. Findings from more than 500 adolescents with autism spectrum disorder across the United States indicate discrepancies between adolescent, teacher, and parent ratings of skills highlighting the importance of the inclusion of multiple perspectives in transition planning. Although ratings varied, agreement between adolescents with autism spectrum disorder, parents, and teachers across the highest and lowest rated skills suggests the need to broaden the focus on critical transition skills to include problem-solving, planning for life after high school, and self-advocacy.

Non-technical skills assessment in surgery.

Science.gov (United States)

Sharma, Bharat; Mishra, Amit; Aggarwal, Rajesh; Grantcharov, Teodor P

2011-09-01

Adverse events in surgery have highlighted the importance of non-technical skills, such as communication, decision-making, teamwork, situational awareness and leadership, to effective organizational performance. These skills carry particular importance to surgical oncology, as members of a multidisciplinary team must work cohesively to formulate effective patient care plans. Several non-technical skills evaluation tools have been developed for use in surgery, without adequate comparison and consensus on which should be standard for training. Eleven articles describing the use of three non-technical evaluation tools related to surgery: NOTSS (Non Technical Skills for Surgeons), NOTECHS (Non Technical Skills) and OTAS (Observational Teamwork Assessment for Surgery) were analyzed with respect to scale formulation, validity, reliability and feasibility. Furthermore, their use in training thus far and the future of non-technical rating scales in surgical curricula was discussed. Future work should focus on incorporating these assessment tools into training and into a real operating room setting to provide formative evaluations for surgical residents. Copyright © 2010 Elsevier Ltd. All rights reserved.

Using the Entrustable Professional Activities Framework in the Assessment of Procedural Skills.

Science.gov (United States)

Pugh, Debra; Cavalcanti, Rodrigo B; Halman, Samantha; Ma, Irene W Y; Mylopoulos, Maria; Shanks, David; Stroud, Lynfa

2017-04-01

The entrustable professional activity (EPA) framework has been identified as a useful approach to assessment in competency-based education. To apply an EPA framework for assessment, essential skills necessary for entrustment to occur must first be identified. Using an EPA framework, our study sought to (1) define the essential skills required for entrustment for 7 bedside procedures expected of graduates of Canadian internal medicine (IM) residency programs, and (2) develop rubrics for the assessment of these procedural skills. An initial list of essential skills was defined for each procedural EPA by focus groups of experts at 4 academic centers using the nominal group technique. These lists were subsequently vetted by representatives from all Canadian IM training programs through a web-based survey. Consensus (more than 80% agreement) about inclusion of each item was sought using a modified Delphi exercise. Qualitative survey data were analyzed using a framework approach to inform final assessment rubrics for each procedure. Initial lists of essential skills for procedural EPAs ranged from 10 to 24 items. A total of 111 experts completed the national survey. After 2 iterations, consensus was reached on all items. Following qualitative analysis, final rubrics were created, which included 6 to 10 items per procedure. These EPA-based assessment rubrics represent a national consensus by Canadian IM clinician educators. They provide a practical guide for the assessment of procedural skills in a competency-based education model, and a robust foundation for future research on their implementation and evaluation.

E-learning and nursing assessment skills and knowledge - An integrative review.

Science.gov (United States)

McDonald, Ewan W; Boulton, Jessica L; Davis, Jacqueline L

2018-07-01

This review examines the current evidence on the effectiveness of digital technologies or e-based learning for enhancing the skills and knowledge of nursing students in nursing assessment. This integrative review identifies themes emerging from e-learning and 'nursing assessment' literature. Literature reviews have been undertaken in relation to digital learning and nursing education, including clinical skills, clinical case studies and the nurse-educator role. Whilst perceptions of digital learning are well covered, a gap in knowledge persists for understanding the effectiveness of e-learning on nursing assessment skills and knowledge. This is important as comprehensive assessment skills and knowledge are a key competency for newly qualified nurses. The MEDLINE, CINAHL, Cochrane Library and ProQuest Nursing and Allied Health Source electronic databases were searched for the period 2006 to 2016. Hand searching in bibliographies was also undertaken. Selection criteria for this review included: FINDINGS: Twenty articles met the selection criteria for this review, and five major themes for e-based learning were identified (a) students become self-evaluators; (b) blend and scaffold learning; (c) measurement of clinical reasoning; (d) mobile technology and Facebook are effective; and (e) training and preparation is vital. Although e-based learning programs provide a flexible teaching method, evidence suggests e-based learning alone does not exceed face-to-face patient simulation. This is particularly the case where nursing assessment learning is not scaffolded. This review demonstrates that e-based learning and traditional teaching methods used in conjunction with each other create a superior learning style. Copyright © 2018 Elsevier Ltd. All rights reserved.

Parent assessment of medical student skills in ambulatory pediatrics

Directory of Open Access Journals (Sweden)

Erika Persson

2013-09-01

Full Text Available Background: Partnership with parents is a vital part of pediatric medical education, yet few studies have examined parent attitudes towards learners in pediatric settings. Methods: Questionnaires were used to determine parent and student assessment of professional and clinical skills (primary outcome and parent attitudes towards 3rd year medical students (secondary outcome at the University of Alberta. Chi Square, Kendall’s Tau and Kappa coefficients were calculated to compare parent and student responses in 8 areas: communication, respect, knowledge, listening, history taking, physical examination, supervision, and overall satisfaction. Results: Overall satisfaction with medical student involvement by parents was high: 56.7% of all parents ranked the encounter as ‘excellent’. Areas of lesser satisfaction included physician supervision of students. Compared to the parent assessment, students tended to underrate many of their skills, including communication, history taking and physical exam. There was no relationship between parent demographics and their attitude to rating any of the students’ skills. Conclusions: Parents were satisfied with medical student involvement in the care of their children. Areas identified for improvement included increased supervision of students in both history taking and physical examination. This is one of the largest studies examining parent attitudes towards pediatric students. The results may enhance undergraduate curriculum development and teaching in pediatric ambulatory clinics and strengthen the ongoing partnership between the community and teaching clinics.

Assessment of psychomotor skills acquisition during laparoscopic cholecystectomy courses.

Science.gov (United States)

Hance, Julian; Aggarwal, Rajesh; Moorthy, Krishna; Munz, Yaron; Undre, Shabnam; Darzi, Ara

2005-09-01

Standardized short courses in laparoscopic cholecystectomy aim to teach laparoscopic skills to surgical trainees, although end-of-course assessments of performance remain subjective. The current study aims to objectively assess psychomotor skills acquisition of trainees attending laparoscopic cholecystectomy courses. Thirty-seven junior surgical trainees had their laparoscopic skills assessed before and after attending 1 of 3 separate 2-day courses (A, B, and C), all with identical format. Assessments were comprised of a standardized simulated laparoscopic task, with performance measured using a valid electromagnetic hand-motion tracking device. Overall, trainees made significant improvements in path length (P=.006), number of movements (Ppsychomotor skills on courses. In addition to providing participants with an insight into their skills, these data can be used to demonstrate course efficacy.

Challenge Course Facilitator Technical Skills Assessment Tool Mark

Science.gov (United States)

Wagstaff, William Quinn

2007-01-01

A study was conducted to develop a technical skills assessment tool for the training and development of challenge course facilitators. Researchers accessed two professional on-line listserves to collect a sample size of twenty-seven currently used technical skills assessment tools. The assessment tools were critically analysed by three independent…

Validation of a global assessment of arthroscopic skills in a cadaveric knee model.

Science.gov (United States)

Slade Shantz, Jesse A; Leiter, Jeff R; Collins, John B; MacDonald, Peter B

2013-01-01

The purpose of this study was to determine whether a global assessment of arthroscopic skills was valid for blinded assessment of cadaveric diagnostic knee arthroscopy. A global skills assessment for arthroscopy was created using a published theory of the development of expertise. Faculty surgeons, fellows, and residents were consented and enrolled in this institutional review board-approved validation study. All participants were oriented to the equipment and procedures for diagnostic arthroscopy of the knee. After reviewing the anatomic structures to be visualized, participants were allowed 10 minutes to complete a diagnostic arthroscopy of the knee. The hands and arthroscopic view were recorded during this attempt. Resident participants completed a second filmed diagnostic arthroscopy 1 week after the initial attempt. Five blinded reviewers watched the synchronized videos and assessed arthroscopic skills with a procedure-specific checklist and the newly developed global skills assessment. The agreement between reviewers was determined by intraclass correlation coefficient. Internal consistency was determined with Cronbach's α. Test-retest reliability was measured by correlating repeated arthroscopies by residents. The ability of the global assessment to discriminate skill levels was determined with between-group Mann-Whitney U tests. The agreement between global assessment scores was strong (I.C.C. = 0.80, 95% C.I. 0.68-0.92). The internal consistency of evaluations was excellent (Cronbach's α = 0.97), and the test-retest reliability was strong (r = 0.52). The global assessment score was shown to be able to discriminate between skill levels by an analysis of variance indicating the difference in means among the various levels of training (P Assessment of Arthroscopic Skills is a useful adjunct to arthroscopic educators and learners and could be used for in-training evaluations. The Objective Assessment of Arthroscopic Skills is an instrument that can be

Brief Report: Suitability of the Social Skills Performance Assessment (SSPA) for the Assessment of Social Skills in Adults with Autism Spectrum Disorders

Science.gov (United States)

Verhoeven, E. W. M.; Smeekens, I.; Didden, R.

2013-01-01

The present study aims at examining whether the "Social Skills Performance Assessment" (SSPA; Patterson et al. in "Schizophr Res" 48(2-3):351-360, 2001) is a suitable performance-based measure to assess social skills in adults with autism spectrum disorders (ASD). For this purpose, social skills of individuals with ASD and…

How can teachers assess reading skills of generation z learners in German language class?

Science.gov (United States)

Dirga, R. N.; Wijayati, P. H.

2018-01-01

Generation Z is a digital native generation who has unique characteristics on the daily basis includes reading. In order to assess their reading skills correctly, teachers need to take these characteristics into consideration. This paper aims to describe the process in developing an instrument to assess reading skills of Generation Z learners in German language class. This study used developmental method. The developed instrument has not only good quality but also consists of texts that are suitable for German learners of Generation Z. This instrument can be used as an example in assessing German learners’ reading skills in the 21st century.

The Measurement Properties of the Assessing Math Concepts' Assessments of Primary Students' Number Sense Skills.

Science.gov (United States)

Martin, Christie; Lambert, Richard; Polly, Drew; Wang, Chuang; Pugalee, David

The purpose of this study was to examine the measurement properties of the Assessing Math Concepts AMC Anywhere Hiding and Ten Frame Assessments, formative assessments of primary students' number sense skills. Each assessment has two parts, where Part 1 is intended to be foundational skills for part two. Part 1 includes manipulatives whereas Part 2 does not. Student data from 228 kindergarten through second grade teachers with a total of 3,666 students was analyzed using Rasch scaling. Data analyses indicated that when the two assessments were examined separately the intended order of item difficulty was clear. When the parts of both assessments were analyzed together, the items in Part 2 were not consistently more difficult that the items in Part 1. This suggests an alternative sequence of tasks in that students may progress from working with a specific number with manipulatives then without manipulatives rather than working with a variety of numbers with manipulatives before moving onto assessments without manipulatives.

Assessment of the critical thinking skills of student radiographers

Energy Technology Data Exchange (ETDEWEB)

Castle, Alan [University of Portsmouth, Centre for Radiography Education, St George' s Building, Portsmouth PO1 2HY (United Kingdom)]. E-mail: alan.castle@port.ac.uk

2006-05-15

Purpose: Enabling students to develop critical thinking skills is one of the key aims of higher education and in preparing student radiographers for the future, there are increasing demands on educators to teach critical thinking skills to facilitate reflective, evidence-based practice and inter-professional working. The aim of the paper is to attempt to compare students' self-perception of their critical thinking skills to their actual written assessment performance. Methods: Students were asked to self-report how they thought the course had developed their critical thinking skills and the outcomes of this exercise were compared to the scores of previous assessments that required the demonstration of these skills. Results: The results suggest that whilst students report having developed critical thinking skills during the course, the results of their written assessments requiring the demonstration of these skills all had a mean score of less than 60% which indicates (in terms of the university's grade criteria guidelines) 'little attempt to use critical discussion in their work.' Discussion: Thirteen components of critical thinking are proposed, together with ways in which they could be incorporated into a radiographic curriculum. Conclusions: It is suggested that educators may need to review the constructive alignment of their curricula and re-assess their teaching and assessment strategies in order to effectively develop students' critical thinking skills.

Assessment of the critical thinking skills of student radiographers

International Nuclear Information System (INIS)

Castle, Alan

2006-01-01

Purpose: Enabling students to develop critical thinking skills is one of the key aims of higher education and in preparing student radiographers for the future, there are increasing demands on educators to teach critical thinking skills to facilitate reflective, evidence-based practice and inter-professional working. The aim of the paper is to attempt to compare students' self-perception of their critical thinking skills to their actual written assessment performance. Methods: Students were asked to self-report how they thought the course had developed their critical thinking skills and the outcomes of this exercise were compared to the scores of previous assessments that required the demonstration of these skills. Results: The results suggest that whilst students report having developed critical thinking skills during the course, the results of their written assessments requiring the demonstration of these skills all had a mean score of less than 60% which indicates (in terms of the university's grade criteria guidelines) 'little attempt to use critical discussion in their work.' Discussion: Thirteen components of critical thinking are proposed, together with ways in which they could be incorporated into a radiographic curriculum. Conclusions: It is suggested that educators may need to review the constructive alignment of their curricula and re-assess their teaching and assessment strategies in order to effectively develop students' critical thinking skills

Assessment of time management skills: psychometric properties of the Swedish version.

Science.gov (United States)

Janeslätt, Gunnel Kristina; Holmqvist, Kajsa Lidström; White, Suzanne; Holmefur, Marie

2018-05-01

Persons with impaired time management skills are often in need of occupational therapy. Valid and reliable instruments to assess time management and organizational skills are needed for the evaluation of intervention. The purpose of this study was to evaluate the psychometric properties of a Swedish version of the Assessment of Time Management Skills (ATMS-S) for persons with and without impaired time management skills. A total of 238 persons participated in the study, of whom 94 had self-reported impaired time management skills due to mental disorders such as schizophrenic spectrum or neurodevelopmental disorders such as attention deficit/hyperactivity disorder (ADHD), autism spectrum disorder (ASD) and mild intellectual disabilities, and 144 persons had no reported impaired time management skills. Rasch analysis was used to analyze data. Three subscales were detected: the time management subscale with 11 items, the organization & planning subscale with 11 items, and the subscale of regulation of emotions with 5 items, with excellent to acceptable psychometric properties. The conclusions were that: ATMS-S is a valid instrument for self-rating of time management, organization & planning and for the regulation of emotions. ATMS-S can be useful for persons with mental disorders including mild neurodevelopmental disorders.

The Development and Validation of a Golf Swing and Putt Skill Assessment for Children

Science.gov (United States)

Barnett, Lisa M.; Hardy, Louise L.; Brian, Ali S.; Robertson, Sam

2015-01-01

The aim was to describe development of a process-oriented instrument designed to assess the golf swing and putt stroke, and to assess the instrument’s discriminative validity in terms of age and reliability (intra-rater and re-test). A Delphi consultation (with golf industry professionals and researchers in movement skill assessment) was used to develop an assessment for each skill based on existing skill assessment protocols. Each skill had six components to be marked as present/absent. Individual scores were based on the number of performance components successfully demonstrated over two trials for each skill (potential score range 0 to 24). Children (n = 43) aged 6-10 years (M = 7.8 years, SD = 1.3) were assessed in both skills live in the field by one rater at Time 1(T1). A subset of children (n = 28) had consent for assessments to be videoed. Six weeks later 19 children were reassessed, five days apart (T2, T3). An ANOVA assessed discriminative validity i.e. whether skill competence at T1 differed by age (6 years, 7/8 years and 9/10 years). Intraclass correlations (ICC) assessed intra-rater reliability between the live and video assessment at T1 and test-retest reliability (between T2 and T3). Paired t-tests assessed any systematic differences between live and video assessments (T1) and between T2 and T3. Older children were more skilled (F (2, 40) = 11.18, p golf coaches and physical education teachers as part of systematic early player assessment and feedback. Key points Golf is becoming an increasingly popular sport among young children, however there is no standard protocol available to assess and identify skill deficits, mastery level, and talent identification in beginner young golf players. Process rather than product oriented outcomes better identify areas of skill deficit in young children. The proposed swing and putt instrument can reliably identify skill deficits in children of elementary school age who are new to golf and can be used by a range

The effectiveness of the Geritalk communication skills course: a real-time assessment of skill acquisition and deliberate practice.

Science.gov (United States)

Gelfman, Laura P; Lindenberger, Elizabeth; Fernandez, Helen; Goldberg, Gabrielle R; Lim, Betty B; Litrivis, Evgenia; O'Neill, Lynn; Smith, Cardinale B; Kelley, Amy S

2014-10-01

Communication skills are critical in Geriatrics and Palliative Medicine because these patients confront complex clinical scenarios. We evaluated the effectiveness of the Geritalk communication skills course by comparing pre- and post-course real-time assessment of the participants leading family meetings. We also evaluated the participants' sustained skills practice. We compare the participants' skill acquisition before and after Geritalk using a direct observation Family Meeting Communication Assessment Tool and assess their deliberate practice at follow-up. First-year Geriatrics or Palliative Medicine fellows at Mount Sinai Medical Center and the James J. Peters Bronx VA Medical Center participated in Geritalk. Pre- and post-course family meeting assessments were compared. An average net gain of 6.8 skills represented a greater than 20% improvement in use of applicable skills. At two month follow-up, most participants reported deliberate practice of fundamental and advanced skills. This intensive training and family meeting assessment offers evidence-based communication skills training. Published by Elsevier Inc.

Teamwork Skills Assessment for Cooperative Learning

Science.gov (United States)

Strom, Paris S.; Strom, Robert D.

2011-01-01

Teamwork skills are required at work, but teacher efforts in many countries to track achievement within this context have been hindered by lack of assessment tools and input from students. The Teamwork Skills Inventory relies on peer and self-evaluation to establish accountability, identify competencies, and detect learning needs. Twenty-five…

Assessing Communication Skills of Medical Students in Objective Structured Clinical Examinations (OSCE)--A Systematic Review of Rating Scales.

Science.gov (United States)

Cömert, Musa; Zill, Jördis Maria; Christalle, Eva; Dirmaier, Jörg; Härter, Martin; Scholl, Isabelle

2016-01-01

Teaching and assessment of communication skills have become essential in medical education. The Objective Structured Clinical Examination (OSCE) has been found as an appropriate means to assess communication skills within medical education. Studies have demonstrated the importance of a valid assessment of medical students' communication skills. Yet, the validity of the performance scores depends fundamentally on the quality of the rating scales used in an OSCE. Thus, this systematic review aimed at providing an overview of existing rating scales, describing their underlying definition of communication skills, determining the methodological quality of psychometric studies and the quality of psychometric properties of the identified rating scales. We conducted a systematic review to identify psychometrically tested rating scales, which have been applied in OSCE settings to assess communication skills of medical students. Our search strategy comprised three databases (EMBASE, PsycINFO, and PubMed), reference tracking and consultation of experts. We included studies that reported psychometric properties of communication skills assessment rating scales used in OSCEs by examiners only. The methodological quality of included studies was assessed using the COnsensus based Standards for the selection of health status Measurement INstruments (COSMIN) checklist. The quality of psychometric properties was evaluated using the quality criteria of Terwee and colleagues. Data of twelve studies reporting on eight rating scales on communication skills assessment in OSCEs were included. Five of eight rating scales were explicitly developed based on a specific definition of communication skills. The methodological quality of studies was mainly poor. The psychometric quality of the eight rating scales was mainly intermediate. Our results reveal that future psychometric evaluation studies focusing on improving the methodological quality are needed in order to yield psychometrically

Development of a Social Skills Assessment Screening Scale for Psychiatric Rehabilitation Settings: A Pilot Study.

Science.gov (United States)

Bhola, Poornima; Basavarajappa, Chethan; Guruprasad, Deepti; Hegde, Gayatri; Khanam, Fatema; Thirthalli, Jagadisha; Chaturvedi, Santosh K

2016-01-01

Deficits in social skills may present in a range of psychiatric disorders, particularly in the more serious and persistent conditions, and have an influence on functioning across various domains. This pilot study aimed at developing a brief measure, for structured evaluation and screening for social skills deficits, which can be easily integrated into routine clinical practice. The sample consisted of 380 inpatients and their accompanying caregivers, referred to Psychiatric Rehabilitation Services at a tertiary care government psychiatric hospital. The evaluation included an Inpatient intake Proforma and the 20-item Social Skills Assessment Screening Scale (SSASS). Disability was assessed using the Indian Disability Evaluation and Assessment Scale (IDEAS) for a subset of 94 inpatients. The analysis included means and standard deviations, frequency and percentages, Cronbach's alpha to assess internal consistency, t -tests to assess differences in social skills deficits between select subgroups, and correlation between SSASS and IDEAS scores. The results indicated the profile of social skills deficits assessed among the inpatients with varied psychiatric diagnoses. The "psychosis" group exhibited significantly higher deficits than the "mood disorder" group. Results indicated high internal consistency of the SSASS and adequate criterion validity demonstrated by correlations with select IDEAS domains. Modifications were made to the SSASS following the pilot study. The SSASS has potential value as a measure for screening and individualised intervention plans for social skills training in mental health and rehabilitation settings. The implications for future work on the psychometric properties and clinical applications are discussed.

Improving the accuracy of self-assessment of practical clinical skills using video feedback--the importance of including benchmarks.

Science.gov (United States)

Hawkins, S C; Osborne, A; Schofield, S J; Pournaras, D J; Chester, J F

2012-01-01

Isolated video recording has not been demonstrated to improve self-assessment accuracy. This study examines if the inclusion of a defined standard benchmark performance in association with video feedback of a student's own performance improves the accuracy of student self-assessment of clinical skills. Final year medical students were video recorded performing a standardised suturing task in a simulated environment. After the exercise, the students self-assessed their performance using global rating scales (GRSs). An identical self-assessment process was repeated following video review of their performance. Students were then shown a video-recorded 'benchmark performance', which was specifically developed for the study. This demonstrated the competency levels required to score full marks (30 points). A further self-assessment task was then completed. Students' scores were correlated against expert assessor scores. A total of 31 final year medical students participated. Student self-assessment scores before video feedback demonstrated moderate positive correlation with expert assessor scores (r = 0.48, p benchmark performance demonstration, self-assessment scores demonstrated a very strong positive correlation with expert scores (r = 0.83, p benchmark performance in combination with video feedback may significantly improve the accuracy of students' self-assessments.

Comparison of performance on process- and product-oriented assessments of fundamental motor skills across childhood.

Science.gov (United States)

Logan, Samuel W; Barnett, Lisa M; Goodway, Jacqueline D; Stodden, David F

2017-04-01

Process-oriented motor competence (MC) assessments evaluate how a movement is performed. Product-oriented assessments evaluate the outcome of a movement. Determining the concurrent validity of process and product assessments is important to address the predictive utility of motor competence for health. The current study aimed to: (1) compare process and product assessments of the standing long jump, hop and throw across age groups and (2) determine the capacity of process assessments to classify levels of MC. Participants included 170 children classified into three age groups: 4-5, 7-8 and 10-11 years old. Participants' skills were examined concurrently using three process assessments ((Test of Gross Motor Development-2nd edition [TGMD-2]), Get Skilled; Get Active, and developmental sequences) and one product measure (throw speed, jump and hop distance). Results indicate moderate to strong correlations between (1) process assessments across skills and age groups (r range = .37-70) and (2) process and product assessments across skills and age groups (r range = .26-.88). In general, sensitivity to detect advanced skill level is lowest for TGMD-2 and highest for developmental sequences for all three skills. The use of process and product assessments is suggested to comprehensively capture levels of MC in human movement.

The development and validation of a golf swing and putt skill assessment for children.

Science.gov (United States)

Barnett, Lisa M; Hardy, Louise L; Brian, Ali S; Robertson, Sam

2015-03-01

The aim was to describe development of a process-oriented instrument designed to assess the golf swing and putt stroke, and to assess the instrument's discriminative validity in terms of age and reliability (intra-rater and re-test). A Delphi consultation (with golf industry professionals and researchers in movement skill assessment) was used to develop an assessment for each skill based on existing skill assessment protocols. Each skill had six components to be marked as present/absent. Individual scores were based on the number of performance components successfully demonstrated over two trials for each skill (potential score range 0 to 24). Children (n = 43) aged 6-10 years (M = 7.8 years, SD = 1.3) were assessed in both skills live in the field by one rater at Time 1(T1). A subset of children (n = 28) had consent for assessments to be videoed. Six weeks later 19 children were reassessed, five days apart (T2, T3). An ANOVA assessed discriminative validity i.e. whether skill competence at T1 differed by age (6 years, 7/8 years and 9/10 years). Intraclass correlations (ICC) assessed intra-rater reliability between the live and video assessment at T1 and test-retest reliability (between T2 and T3). Paired t-tests assessed any systematic differences between live and video assessments (T1) and between T2 and T3. Older children were more skilled (F (2, 40) = 11.18, p < 0.001). The live assessment reflected the video assessment (ICC = 0.79, 95% CI 0.59, 0.90) and scores did not differ between live and video assessments. Test retest reliability was acceptable (ICC = 0.60, 95% CI 0.23, 0.82), although the mean score was slightly higher at retest. This instrument could be used reliably by golf coaches and physical education teachers as part of systematic early player assessment and feedback. Key pointsGolf is becoming an increasingly popular sport among young children, however there is no standard protocol available to assess and identify skill deficits, mastery level

A Clinic-Based Assessment for Evaluating Job-Related Social Skills in Adolescents and Adults with Autism.

Science.gov (United States)

Lerman, Dorothea C; White, Bridgette; Grob, Carolyn; Laudont, Courtney

2017-12-01

Many individuals with autism spectrum disorder (ASD) have difficulties obtaining and maintaining employment, yet little research has evaluated methods for assessing and improving critical vocational skills. In this study, we evaluated an assessment of job-related social skills for individuals with ASD by arranging conditions that simulated on-the-job experiences in a clinic setting. The experimenter contrived situations to assess a variety of social skills, including asking for help, asking for more materials, and responding to corrective feedback. A total of eight individuals, aged 16 to 32 years, participated. Results suggested that the assessment was useful for identifying specific social skills that could be targeted for intervention to increase success in the work environment. These findings add to the current literature by demonstrating an objective method for assessing a variety of job-related social skills under controlled, naturalistic conditions.

Facilitating the implementation of the American College of Surgeons/Association of Program Directors in Surgery phase III skills curriculum: training faculty in the assessment of team skills.

Science.gov (United States)

Hull, Louise; Arora, Sonal; Stefanidis, Dimitrios; Sevdalis, Nick

2015-11-01

Effective teamwork is critical to safety in the operating room; however, implementation of phase III of the American College of Surgeons (ACS) and Association of Program Directors in Surgery (APDS) Curriculum that focuses on team-based skills remains worryingly low. Training and assessing the complexities of teamwork is challenging. The objective of this study was to establish guidelines and recommendations for training faculty in assessing/debriefing team skills. A multistage survey-based consensus study was completed by 108 experts responsible for training and assessing surgical residents from the ACS Accredited Educational Institutes. Experts agreed that a program to teach faculty to assess team-based skills should include training in the recognition of teamwork skills, practice rating these skills, and training in the provision of feedback/debriefing. Agreement was reached that faculty responsible for conducting team-based skills assessment should be revalidated every 2 years and stringent proficiency criteria should be met. Faculty development is critical to ensure high-quality, standardized training and assessment. Training faculty to assess team-based skills has the potential to facilitate the effective implementation of phase III of the ACS and APDS Curriculum. Copyright © 2015 Elsevier Inc. All rights reserved.

Assessing Business Student Thinking Skills

Science.gov (United States)

Smith, Gerald F.

2014-01-01

The development of student thinking skills is a major goal of business education. As with other such goals, student outcomes assessment must be undertaken to measure goal achievement. Thinking is difficult to teach; it is also difficult to assess. The purpose of this article is to improve management educators' understanding of student thinking…

Multi-model global assessment of subseasonal prediction skill of atmospheric rivers

Science.gov (United States)

Deflorio, M. J.

2017-12-01

Atmospheric rivers (ARs) are global phenomena that are characterized by long, narrow plumes of water vapor transport. They are most often observed in the midlatitudes near climatologically active storm track regions. Because of their frequent association with floods, landslides, and other hydrological impacts on society, there is significant incentive at the intersection of academic research, water management, and policymaking to understand the skill with which state-of-the-art operational weather models can predict ARs weeks-to-months in advance. We use the newly assembled Subseasonal-to-Seasonal (S2S) database, which includes extensive hindcast records of eleven operational weather models, to assess global prediction skill of atmospheric rivers on S2S timescales. We develop a metric to assess AR skill that is suitable for S2S timescales by counting the total number of AR days which occur over each model and observational grid cell during a 2-week time window. This "2-week AR occurrence" metric is suitable for S2S prediction skill assessment because it does not consider discrete hourly or daily AR objects, but rather a smoothed representation of AR occurrence over a longer period of time. Our results indicate that several of the S2S models, especially the ECMWF model, show useful prediction skill in the 2-week forecast window, with significant interannual variation in some regions. We also present results from an experimental forecast of S2S AR prediction skill using the ECMWF and NCEP models.

A Perkins Challenge: Assessing Technical Skills in CTE

Science.gov (United States)

Stone, James R., III

2009-01-01

Federal law requires state to develop performance measures and data-collection systems for secondary and postsecondary technical-skill attainment. This poses many challenges, such as defining a technical skills, measurement and when to assess students. In this article, the author outlines various assessment models and looks at the challenges…

Collecting Validity Evidence for Simulation-Based Assessment of Point-of-Care Ultrasound Skills

DEFF Research Database (Denmark)

Jensen, Jesper Kørup; Dyre, Liv; Jørgensen, Mattis Enggaard

2017-01-01

OBJECTIVES: The aim of this study was to examine the validity of a simulator test designed to evaluate focused assessment with sonography for trauma (FAST) skills. METHODS: Participants included a group of ultrasound novices (n = 25) and ultrasound experts (n = 10). All participants had their FAST...... skills assessed using a virtual reality ultrasound simulator. Procedural performance on the 4 FAST windows was assessed by automated simulator metrics, which received a passing or failing score. The validity evidence for these simulator metrics was examined by a stepwise approach according...

Evidence Regarding Teaching and Assessment of Record-Keeping Skills in Training of Dental Students.

Science.gov (United States)

Amos, Kate J; Bearman, Margaret; Palermo, Claire

2015-10-01

The aim of this study was to assess the literature on teaching and assessing dental students' record-keeping skills prior to qualification to practice independently as a dentist. A systematic literature review was performed using Ovid MEDLINE and SCOPUS. Keywords used in the search included dental, record, audit, education, and assessment. Electronic search results were screened for publications that targeted undergraduate dental training, related to a record-keeping education intervention, and were published in English and available in full text. Six studies met the inclusion and exclusion criteria. Data extraction and quality assessment were performed, and research findings were compared across the included studies. These six articles addressed the techniques used to teach and assess record-keeping skills in a pre-qualification context. The techniques included supervisor audits, peer audits, lectures, tutorials, research assignments, case reports, record-keeping templates, and checklists of required record components. The use of record audit as part of teaching and evaluation dominated these articles; it was used as the assessment method in five of the six studies. All methods of record-keeping training in studies published to date were found effective in improving student record-keeping skills. However, there was insufficient evidence to determine whether certain methods were more effective than others.

The use of advanced physical assessment skills by cardiac nurses.

Science.gov (United States)

Edmunds, Linda; Ward, Susan; Barnes, Rhian

To establish what advanced physical assessment skills are being used by cardiac nurses after they undertook a clinical patient assessment module; and to explore the factors that influence their use of these skills. A longitudinal descriptive approach using convenience sampling was employed. Qualitative data was obtained from individual interviews, non-participant observation within the participants' clinical environment and self-reported activity logs. Five key themes emerged: use of advanced physical assessment skills varied; use and development of skills was linked to personal characteristics; use influenced by perceptions of role boundaries, permission and cooperation; use influenced by participants' perception of nursing and the development of their own nursing practice; and use influenced by the physical environment and the human support within it. Cardiac nurses selectively use physical assessment skills, predominately related to the cardiorespiratory systems. Organisational structure, professional relationships and the professionalism of the individual nurse appear to play a significant part in the use of physical assessment skills. Although the findings from this qualitative study cannot be generalized, they concur with findings from recent research into physical assessment skills used by a variety of UK nurses. The implications identified are: first, for those who provide the education, in terms of what should be taught and facilitated; and second, for organizations, in ensuring staff have assessment skills relevant to their role and that systems are in place to enable the development of a supportive and progressive culture that embraces modernization congruent with healthcare policy.

Employability Skills among Students and Employers’ Perceptions: An Assessment of Levels of Employability Skills Acquired by Business Students at Ishik University

Directory of Open Access Journals (Sweden)

Fayeq Ali Ali

2017-01-01

Full Text Available Skills are prerequisite for managers and employees success, especially for newly graduate students. This study is to evaluate the employability skills of business students at Ishik University and to assess how employability skills are perceived by potential employers. Three sets of employability skills have been used in this study such as basic academic skills, high-order thinking, and personal qualities. A questionnaire has been developed which included above dimensions and was distributed among students in the faculty of administrative sciences and economics at Ishik University, Kurdistan Region. Respondents’ opinions were assessed using a Likert scale analysis that shows divergent opinions between two extremes of levels of agreement and disagreement. Another technique of an open-ended questionnaire was used when conducting interviews with a few of the potential employers in some private sector companies. Study focuses on the common employability skills of business graduates by evaluating the faculty of administrative sciences and economics courses. Study find out that communication skills, team working skills, computer skills, and critical thinking were among the employability skills which are expected by potential employers. The paper concludes that business graduates have developed an adequate level of employability skill through their years of academic training at business department in Ishik University. Thus, the curriculum of business department at Ishik University is adequately developed to prompt the employability skills that are sought by potential employers that every business student should acquire to stand out in the aggressively competitive job market.

Assessment of structured physical examination skills training using a retro-pre-questionnaire.

Science.gov (United States)

Piryani, Rano Mal; Shankar, P Ravi; Piryani, Suneel; Thapa, Trilok Pati; Karki, Balmansingh; Khakurel, Mahesh Prasad; Bhandary, Shital

2013-01-01

The effectiveness of physical examination skills (PES) training is very rarely assessed using the "post-then-pre" approach. In this study, a retro-pre-questionnaire was used to study the effect of structured physical examination skills training (SPEST) imparted to second-year undergraduate medical students. KIST Medical College (KISTMC) affiliated to Tribhuvan University Nepal admitted its first batch of MBBS students in November 2008. The university curriculum recommends the involvement of Medicine and Surgery Departments in PES training, but the methods for teaching and assessment are not well defined. KISTMC has made training more structured and involved the Medicine, Surgery, Gynaecology and Obstetrics, Orthopaedics, ENT, Ophthalmology, Paediatrics, and Family Medicine Departments. SPEST includes the teaching/learning of basic PES for 210 minutes once a week for 28 weeks. Self-assessment is done by using a retro-pre-questionnaire at the end of the last session of training, and these data are analysed using SPSS. Out of 100 students, 98 participated in the objective structured clinical examination (OSCE); 82 completed the retro-pre-questionnaire. Forty-six skills representing various systems were selected for inclusion in the retro-pre-questionnaire from among the many skills taught in different departments. The average perceived skills score (maximum score, 46×4=184) before training was 15.9 and increased to 116.5 after training. The increase was statistically significant upon the application of a paired t-test. The students perceived that their level of skills improved after the training. The retro-pre- instrument seems to be useful for assessing the learners' self-reported changes in PES after training if a large number of skills need to be assessed. However, it should be noted that although a retro-pre-questionnaire may reveal valuable information, it is not a substitute for an objective measure or gold standard.

Comparison between young male drivers' self-assessed and objectively measured driving skills

DEFF Research Database (Denmark)

Martinussen, Laila Marianne; Møller, Mette; Prato, Carlo Giacomo

2017-01-01

Self-assessment of skills is a self-generated feedback process that contributes to confidence in one's skills. The higher one's self-assessed skills, the more likely one is to feel competent a particular domain thereby influencing the related behaviors. Drivers' self-assessed driving skills...... are not always accurate, which may cause serious problems such as underestimation of risk, reckless driving and accidents. Most previous research on self-assessment of driving skills did not compare self-reported skills to objectively measured driving skills, so the aim of this study was to test the accuracy...

Assessing Communication Skills of Medical Students in Objective Structured Clinical Examinations (OSCE) - A Systematic Review of Rating Scales

Science.gov (United States)

Cömert, Musa; Zill, Jördis Maria; Christalle, Eva; Dirmaier, Jörg; Härter, Martin; Scholl, Isabelle

2016-01-01

Background Teaching and assessment of communication skills have become essential in medical education. The Objective Structured Clinical Examination (OSCE) has been found as an appropriate means to assess communication skills within medical education. Studies have demonstrated the importance of a valid assessment of medical students’ communication skills. Yet, the validity of the performance scores depends fundamentally on the quality of the rating scales used in an OSCE. Thus, this systematic review aimed at providing an overview of existing rating scales, describing their underlying definition of communication skills, determining the methodological quality of psychometric studies and the quality of psychometric properties of the identified rating scales. Methods We conducted a systematic review to identify psychometrically tested rating scales, which have been applied in OSCE settings to assess communication skills of medical students. Our search strategy comprised three databases (EMBASE, PsycINFO, and PubMed), reference tracking and consultation of experts. We included studies that reported psychometric properties of communication skills assessment rating scales used in OSCEs by examiners only. The methodological quality of included studies was assessed using the COnsensus based Standards for the selection of health status Measurement INstruments (COSMIN) checklist. The quality of psychometric properties was evaluated using the quality criteria of Terwee and colleagues. Results Data of twelve studies reporting on eight rating scales on communication skills assessment in OSCEs were included. Five of eight rating scales were explicitly developed based on a specific definition of communication skills. The methodological quality of studies was mainly poor. The psychometric quality of the eight rating scales was mainly intermediate. Discussion Our results reveal that future psychometric evaluation studies focusing on improving the methodological quality are needed

Clinical skills assessment of procedural and advanced communication skills: performance expectations of residency program directors

Science.gov (United States)

Langenau, Erik E.; Zhang, Xiuyuan; Roberts, William L.; DeChamplain, Andre F.; Boulet, John R.

2012-01-01

Background High stakes medical licensing programs are planning to augment and adapt current examinations to be relevant for a two-decision point model for licensure: entry into supervised practice and entry into unsupervised practice. Therefore, identifying which skills should be assessed at each decision point is critical for informing examination development, and gathering input from residency program directors is important. Methods Using data from previously developed surveys and expert panels, a web-delivered survey was distributed to 3,443 residency program directors. For each of the 28 procedural and 18 advanced communication skills, program directors were asked which clinical skills should be assessed, by whom, when, and how. Descriptive statistics were collected, and Intraclass Correlations (ICC) were conducted to determine consistency across different specialties. Results Among 347 respondents, program directors reported that all advanced communication and some procedural tasks are important to assess. The following procedures were considered ‘important’ or ‘extremely important’ to assess: sterile technique (93.8%), advanced cardiovascular life support (ACLS) (91.1%), basic life support (BLS) (90.0%), interpretation of electrocardiogram (89.4%) and blood gas (88.7%). Program directors reported that most clinical skills should be assessed at the end of the first year of residency (or later) and not before graduation from medical school. A minority were considered important to assess prior to the start of residency training: demonstration of respectfulness (64%), sterile technique (67.2%), BLS (68.9%), ACLS (65.9%) and phlebotomy (63.5%). Discussion Results from this study support that assessing procedural skills such as cardiac resuscitation, sterile technique, and phlebotomy would be amenable to assessment at the end of medical school, but most procedural and advanced communications skills would be amenable to assessment at the end of the first

A Procedural Skills OSCE: Assessing Technical and Non-Technical Skills of Internal Medicine Residents

Science.gov (United States)

Pugh, Debra; Hamstra, Stanley J.; Wood, Timothy J.; Humphrey-Murto, Susan; Touchie, Claire; Yudkowsky, Rachel; Bordage, Georges

2015-01-01

Internists are required to perform a number of procedures that require mastery of technical and non-technical skills, however, formal assessment of these skills is often lacking. The purpose of this study was to develop, implement, and gather validity evidence for a procedural skills objective structured clinical examination (PS-OSCE) for internal…

Laparoscopic skills assessment: an additional modality for pediatric surgery fellowship selection.

Science.gov (United States)

Hazboun, Rajaie; Rodriguez, Samuel; Thirumoorthi, Arul; Baerg, Joanne; Moores, Donald; Tagge, Edward P

2017-12-01

The Pediatric Surgery fellow selection is a multi-layered process which has not included assessment of surgical dexterity. Data was collected prospectively as part of the 2016 Pediatric Surgery Match interview process. Applicants completed a questionnaire to document laparoscopic experience and fine motor skills activities. Actual laparoscopic skills were assessed using a simulator. Time to complete an intracorporeal knot was tabulated. An initial rank list was formulated based only on the ERAS application and interview scores. The rank list was re-formulated following the laparoscopic assessment. Un-paired T-test and regression were utilized to analyze the data. Forty applicants were interviewed with 18 matched (45%). The mean knot tying time was 201.31s for matched and 202.35s for unmatched applicants. Playing a musical instrument correlated with faster knot tying (p=0.03). No correlation was identified between knot tying time and either video game experience (p=0.4) or passing the FLS exam (p=0.78). Laparoscopic skills assessment lead to significant reordering of rank list (p=0.01). Laparoscopic skills performance significantly impacted ranking. Playing a musical instrument correlated with faster knot tying. No correlation was identified between laparoscopic performance and passing the FLS exam or other activities traditionally believed to improve technical ability. Prospective study. Level II. Copyright © 2017 Elsevier Inc. All rights reserved.

The Computerized Perceptual Motor Skills Assessment: A new visual perceptual motor skills evaluation tool for children in early elementary grades.

Science.gov (United States)

Howe, Tsu-Hsin; Chen, Hao-Ling; Lee, Candy Chieh; Chen, Ying-Dar; Wang, Tien-Ni

2017-10-01

Visual perceptual motor skills have been proposed as underlying courses of handwriting difficulties. However, there is no evaluation tool currently available to assess these skills comprehensively and to serve as a sensitive measure. The purpose of this study was to validate the Computerized Perceptual Motor Skills Assessment (CPMSA), a newly developed evaluation tool for children in early elementary grades. Its test-retest reliability, concurrent validity, discriminant validity, and responsiveness were examined in 43 typically developing children and 26 children with handwriting difficulty. The CPMSA demonstrated excellent reliability across all subtests with intra-class correlation coefficients (ICCs)≥0.80. Significant moderate correlations between the domains of the CPMSA and corresponding gold standards including Beery VMI, the TVPS-3, and the eye-hand coordination subtest of the DTVP-2 demonstrated good concurrent validity. In addition, the CPMSA showed evidence of discriminant validity in samples of children with and without handwriting difficulty. This article provides evidence in support of the CPMSA. The CPMSA is a reliable, valid, and promising measure of visual perceptual motor skills for children in early elementary grades. Directions for future study and improvements to the assessment are discussed. Copyright © 2017. Published by Elsevier Ltd.

Multi-Institutional Validation of an OSATS for the Assessment of Cystoscopic and Ureteroscopic Skills.

Science.gov (United States)

Argun, Omer Burak; Chrouser, Kristin; Chauhan, Sanket; Monga, Manoj; Knudsen, Bodo; Box, Geoffrey N; Lee, David I; Gettman, Matthew T; Poniatowski, Lauren H; Wang, Qi; Reihsen, Troy E; Sweet, Robert M

2015-10-01

We evaluated the internal and construct validity of an assessment tool for cystoscopic and ureteroscopic cognitive and psychomotor skills at a multi-institutional level. Subjects included a total of 30 urology residents at Ohio State University, Columbus, Ohio; Penn Presbyterian Medical Center, Philadelphia, Pennsylvania; and Mayo Clinic, Rochester, Minnesota. A single external blinded reviewer evaluated cognitive and psychomotor skills associated with cystoscopic and ureteroscopic surgery using high fidelity bench models. Exercises included navigation, basketing and relocation; holmium laser lithotripsy; and cystoscope assembly. Each resident received a total cognitive score, checklist score and global psychomotor skills score. Construct validity was assessed by calculating correlations between training year and performance scores (both cognitive and psychomotor). Internal validity was confirmed by calculating correlations between test components. The median total cognitive score was 91 (IQR 86.25, 97). For psychomotor performance residents had a median total checklist score of 7 (IQR 5, 8) and a median global psychomotor skills score of 21 (IQR 18, 24.5). Construct validity was supported by the positive and statistically significant correlations between training year and total cognitive score (r = 0.66, 95% CI 0.39-0.82, p = 0.01), checklist scores (r = 0.66, 95% CI 0.35-0.84, p = 0.32) and global psychomotor skills score (r = 0.76, 95% CI 0.55-0.88, p = 0.002). The internal validity of OSATS was supported since total cognitive and checklist scores correlated with the global psychomotor skills score. In this multi-institutional study we successfully demonstrated the construct and internal validity of an objective assessment of cystoscopic and ureteroscopic cognitive and technical skills, including laser lithotripsy. Copyright © 2015 American Urological Association Education and Research, Inc. Published by Elsevier Inc. All rights reserved.

Teaching Thinking Skills in Context-Based Learning: Teachers' Challenges and Assessment Knowledge

Science.gov (United States)

Avargil, Shirly; Herscovitz, Orit; Dori, Yehudit Judy

2012-04-01

For an educational reform to succeed, teachers need to adjust their perceptions to the reform's new curricula and strategies and cope with new content, as well as new teaching and assessment strategies. Developing students' scientific literacy through context-based chemistry and higher order thinking skills was the framework for establishing a new chemistry curriculum for Israeli high school students. As part of this endeavor, we developed the Taste of Chemistry module, which focuses on context-based chemistry, chemical understanding, and higher order thinking skills. Our research objectives were (a) to identify the challenges and difficulties chemistry teachers faced, as well as the advantages they found, while teaching and assessing the Taste of Chemistry module; and (b) to investigate how they coped with teaching and assessing thinking skills that include analyzing data from graphs and tables, transferring between multiple representations and, transferring between chemistry understanding levels. Research participants included eight teachers who taught the module. Research tools included interviews, classroom observations, teachers-designed students' assignments, and developers-designed students' assignments. We documented different challenges teachers had faced while teaching the module and found that the teachers developed different ways of coping with these challenges. Developing teachers' assessment knowledge (AK) was found to be the highest stage in teachers' professional growth, building on teachers' content knowledge (CK), pedagogy knowledge (PK), and pedagogical-content knowledge (PCK). We propose the use of assignments designed by teachers as an instrument for determining their professional growth.

Development and validation of trauma surgical skills metrics: Preliminary assessment of performance after training.

Science.gov (United States)

Shackelford, Stacy; Garofalo, Evan; Shalin, Valerie; Pugh, Kristy; Chen, Hegang; Pasley, Jason; Sarani, Babak; Henry, Sharon; Bowyer, Mark; Mackenzie, Colin F

2015-07-01

control by one third. Future applications include assessing specific skills in a larger surgeon cohort, assessing military surgical readiness, and quantifying skill degradation with time since training.

The consequences of using advanced physical assessment skills in medical and surgical nursing: A hermeneutic pragmatic study.

Science.gov (United States)

Zambas, Shelaine I; Smythe, Elizabeth A; Koziol-Mclain, Jane

2016-01-01

The aim of this study was to explore the consequences of the nurse's use of advanced assessment skills on medical and surgical wards. Appropriate, accurate, and timely assessment by nurses is the cornerstone of maintaining patient safety in hospitals. The inclusion of "advanced" physical assessment skills such as auscultation, palpation, and percussion is thought to better prepare nurses for complex patient presentations within a wide range of clinical situations. This qualitative study used a hermeneutic pragmatic approach. Unstructured interviews were conducted with five experienced medical and surgical nurses to obtain 13 detailed narratives of assessment practice. Narratives were analyzed using Van Manen's six-step approach to identify the consequences of the nurse's use of advanced assessment skills. The consequences of using advanced assessment skills include looking for more, challenging interpretations, and perseverance. The use of advanced assessment skills directs what the nurse looks for, what she sees, interpretation of the findings, and her response. It is the interpretation of what is seen, heard, or felt within the full context of the patient situation, which is the advanced skill. Advanced assessment skill is the means to an accurate interpretation of the clinical situation and contributes to appropriate diagnosis and medical management in complex patient situations. The nurse's use of advanced assessment skills enables her to contribute to diagnostic reasoning within the acute medical and surgical setting.

Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery.

Science.gov (United States)

Oropesa, Ignacio; Sánchez-González, Patricia; Lamata, Pablo; Chmarra, Magdalena K; Pagador, José B; Sánchez-Margallo, Juan A; Sánchez-Margallo, Francisco M; Gómez, Enrique J

2011-11-01

Training and assessment paradigms for laparoscopic surgical skills are evolving from traditional mentor-trainee tutorship towards structured, more objective and safer programs. Accreditation of surgeons requires reaching a consensus on metrics and tasks used to assess surgeons' psychomotor skills. Ongoing development of tracking systems and software solutions has allowed for the expansion of novel training and assessment means in laparoscopy. The current challenge is to adapt and include these systems within training programs, and to exploit their possibilities for evaluation purposes. This paper describes the state of the art in research on measuring and assessing psychomotor laparoscopic skills. It gives an overview on tracking systems as well as on metrics and advanced statistical and machine learning techniques employed for evaluation purposes. The later ones have a potential to be used as an aid in deciding on the surgical competence level, which is an important aspect when accreditation of the surgeons in particular, and patient safety in general, are considered. The prospective of these methods and tools make them complementary means for surgical assessment of motor skills, especially in the early stages of training. Successful examples such as the Fundamentals of Laparoscopic Surgery should help drive a paradigm change to structured curricula based on objective parameters. These may improve the accreditation of new surgeons, as well as optimize their already overloaded training schedules. Copyright © 2011 Elsevier Inc. All rights reserved.

Alternative Assessment--Can Real-World Skills Be Tested? Policy Briefs.

Science.gov (United States)

Simon, Karen; Gregg, Soleil

Many educators are shifting their teaching strategies and approaches to include more emphasis on critical thinking skills, the communication of ideas, a variety of approaches to content emphasizing varied student learning styles, and the need to draw explicit connections among topics for retention of learning. Real-world assessment measures, then,…

Examining construct validity of a new naturalistic observational assessment of hand skills for preschool- and school-age children.

Science.gov (United States)

Chien, Chi-Wen; Brown, Ted; McDonald, Rachael

2012-04-01

The Assessment of Children's Hand Skills is a new assessment that utilises a naturalistic observational method to capture children's real-life hand skill performance when engaged at various types of daily activities in everyday living contexts. The Assessment of Children's Hand Skills is designed for use with 2- to 12-year-old children with a range of disabilities or health conditions. The study aimed to investigate construct validity of the Assessment of Children's Hand Skills in Australian children. Rasch analysis was used to examine internal construct validity of the Assessment of Children's Hand Skills in a mixed sample of 53 children with disabilities (including autism spectrum disorder, developmental/genetic disorders and physical disabilities) and 85 typically developing children. External construct validity was examined by correlating with three questionnaires evaluating daily living skills and hand skills. Rasch goodness-of-fit analysis suggested that all 22 activity items and 19 of 20 hand skill items in the Assessment of Children's Hand Skills measured a single construct. The Assessment of Children's Hand Skills items were placed in a clinically meaningful hierarchy from easy to hard, and the difficulty range of the items also matched the majority of children with disabilities and typically developing preschool-aged children. Moderate to high correlations (0.59 ≤ Spearman's ρ coefficients ≤ 0.89, P assessments of daily living and fine motor skills. This study provided preliminary evidence supporting the construct validity of the Assessment of Children's Hand Skills for its clinical application in assessing children's real-life hand skill performance in Australian contexts. © 2012 The Authors Australian Occupational Therapy Journal © 2012 Occupational Therapy Australia.

Setting pass scores for clinical skills assessment.

Science.gov (United States)

Liu, Min; Liu, Keh-Min

2008-12-01

In a clinical skills assessment, the decision to pass or fail an examinee should be based on the test content or on the examinees' performance. The process of deciding a pass score is known as setting a standard of the examination. This requires a properly selected panel of expert judges and a suitable standard setting method, which best fits the purpose of the examination. Six standard setting methods that are often used in clinical skills assessment are described to provide an overview of the standard setting process.

Gender-fair assessment of young gifted students' scientific thinking skills

Science.gov (United States)

Dori, Y. J.; Zohar, A.; Fischer-Shachor, D.; Kohan-Mass, J.; Carmi, M.

2018-04-01

This paper describes an Israeli national-level research examining the extent to which admissions of elementary school students to the gifted programmes based on standardised tests are gender-fair. In the research, the gifted students consisted of 275 boys, 128 girls, and additional 80 girls who were admitted to the gifted programme through affirmative action (AA). To assess these young students' scientific thinking skills, also referred to as science practices, open-ended questions of case-based questionnaires were developed. The investigated scientific thinking skills were question posing, explanation, graphing, inquiry, and metacognition. Analysis of the students' responses revealed that gifted girls who entered the programmes through AA performed at the same level as the other gifted students. We found significant differences between the three research groups in question posing and graphing skills. We suggest increasing gender-fairness by revising the standard national testing system to include case-based narratives followed by open-ended questions that assess gifted students' scientific thinking skills. This may diminish the gender inequity expressed by the different number of girls and boys accepted to the gifted programmes. We show that open-ended tools for analysing students' scientific thinking might better serve both research and practice by identifying gifted girls and boys equally well.

A comparison of students' self-assessments with faculty evaluations of their communication skills.

Science.gov (United States)

Lundquist, Lisa M; Shogbon, Angela O; Momary, Kathryn M; Rogers, Hannah K

2013-05-13

To compare students' self-assessment of their communication skills with faculty members' formal evaluation of their skills in a therapeutics course. Over a 3-year period, faculty members evaluated second-year pharmacy students' communication skills as part of a requirement in a therapeutics course. Immediately following an individual oral assessment and again following a group oral assessment, students self-assessed their communication skills using the same rubric the faculty members had used. Students' self-assessments were then compared with faculty members' evaluation of students' communication skills. Four hundred one (97.3%) students consented to participate in this study. Faculty evaluation scores of students for both the individual and group oral assessments were significantly higher than students' self-assessment scores. Students' self-assessment scores of their communication skills increased from the individual to the group oral assessment. Students' self-assessments of communication skills were consistently lower than faculty members' evaluations. Greater use of oral assessments throughout the pharmacy curriculum may help to improve students' confidence in and self-assessment of their communication skills.

Discussion of skill improvement in marine ecosystem dynamic models based on parameter optimization and skill assessment

Science.gov (United States)

Shen, Chengcheng; Shi, Honghua; Liu, Yongzhi; Li, Fen; Ding, Dewen

2016-07-01

Marine ecosystem dynamic models (MEDMs) are important tools for the simulation and prediction of marine ecosystems. This article summarizes the methods and strategies used for the improvement and assessment of MEDM skill, and it attempts to establish a technical framework to inspire further ideas concerning MEDM skill improvement. The skill of MEDMs can be improved by parameter optimization (PO), which is an important step in model calibration. An efficient approach to solve the problem of PO constrained by MEDMs is the global treatment of both sensitivity analysis and PO. Model validation is an essential step following PO, which validates the efficiency of model calibration by analyzing and estimating the goodness-of-fit of the optimized model. Additionally, by focusing on the degree of impact of various factors on model skill, model uncertainty analysis can supply model users with a quantitative assessment of model confidence. Research on MEDMs is ongoing; however, improvement in model skill still lacks global treatments and its assessment is not integrated. Thus, the predictive performance of MEDMs is not strong and model uncertainties lack quantitative descriptions, limiting their application. Therefore, a large number of case studies concerning model skill should be performed to promote the development of a scientific and normative technical framework for the improvement of MEDM skill.

The primacy of vital signs--acute care nurses' and midwives' use of physical assessment skills: a cross sectional study.

Science.gov (United States)

Osborne, Sonya; Douglas, Clint; Reid, Carol; Jones, Lee; Gardner, Glenn

2015-05-01

Registered nurses and midwives play an essential role in detecting patients at risk of deterioration through ongoing assessment and action in response to changing health status. Yet, evidence suggests that clinical deterioration frequently goes unnoticed in hospitalised patients. While much attention has been paid to early warning and rapid response systems, little research has examined factors related to physical assessment skills. To determine a minimum data set of core skills used during nursing assessment of hospitalised patients and identify nurse and workplace predictors of the use of physical assessment to detect patient deterioration. The study used a single-centre, cross-sectional survey design. The study included 434 registered nurses and midwives (Grades 5-7) involved in clinical care of patients on acute care wards, including medicine, surgery, oncology, mental health and maternity service areas, at a 929-bed tertiary referral teaching hospital in Southeast Queensland, Australia. We conducted a hospital-wide survey of registered nurses and midwives using the 133-item Physical Assessment Skills Inventory and the 58-item Barriers to Registered Nurses' Use of Physical Assessment Scale. Median frequency for each physical assessment skill was calculated to determine core skills. To explore predictors of core skill utilisation, backward stepwise general linear modelling was conducted. Means and regression coefficients are reported with 95% confidence intervals. A p value skills used by most nurses every time they worked included assessment of temperature, oxygen saturation, blood pressure, breathing effort, skin, wound and mental status. Reliance on others and technology (F=35.77, pskill use. The increasing acuity of the acute care patient plausibly warrants more than vital signs assessment; however, our study confirms nurses' physical assessment core skill set is mainly comprised of vital signs. The focus on these endpoints of deterioration as dictated by

Comparing the self-assessed and examiner-assessed driving skills of Japanese driving school students

Directory of Open Access Journals (Sweden)

Hiroshi Nakai

2012-03-01

A sample of Japanese driving test candidates (n=2021 completed a self-assessment using a 5-point scale applied to 19 items. The candidates completed the assessment shortly after passing their practical driving test conducted at a driving school. Their performance was also assessed by an examiner who used the same scale. The comparison between self-assessment and examiner-assessment revealed that around 40% of Japanese driving school students made a realistic assessment of their skills. With regard to the gender differences, although males displayed higher levels of overconfidence than females did, the differences were not as large as earlier studies with questionnaires had suggested. Furthermore, the effect of age on the accuracy of novice drivers' skill assessment was found to be relatively small. Our findings, which are based on a comparison of subjective assessments of driving skills between examiners and novices, instead of a questionable method which relies on a comparison with a hypothetical average driver, suggest that the majority of candidates in fact do not overrate their own skills. These results were discussed from the viewpoint of the driver education system and compared to other European research using the same framework.

Assessment of communication, professionalism, and surgical skills in an objective structured performance-related examination (OSPRE): a psychometric study.

Science.gov (United States)

Ponton-Carss, Alicia; Hutchison, Carol; Violato, Claudio

2011-10-01

The purpose of this study was to investigate the reliability and validity of a performance assessment of communication, professionalism, and surgical skills competencies for surgery residents. Fourteen residents from the general surgery program of the University of Calgary were assessed in 7 surgical simulation stations that included communication and professionalism skills. The internal consistency reliability of the checklists and global rating scales combined was adequate for communication (α = .75-.92) and surgical skills (α = .86-.96), but not for professionalism (α = 0). There was evidence of validity as surgical skills performance improved as a function of postgraduate year level but not for the professionalism checklist. Surgical skills and communication correlated in the 2 stations assessed (r = .55 and .57; P communication skills. Further instrument development is required to assess professionalism in a structured examination context. Copyright © 2011 Elsevier Inc. All rights reserved.

Setting Pass Scores for Clinical Skills Assessment

Directory of Open Access Journals (Sweden)

Min Liu

2008-12-01

Full Text Available In a clinical skills assessment, the decision to pass or fail an examinee should be based on the test content or on the examinees' performance. The process of deciding a pass score is known as setting a standard of the examination. This requires a properly selected panel of expert judges and a suitable standard setting method, which best fits the purpose of the examination. Six standard setting methods that are often used in clinical skills assessment are described to provide an overview of the standard setting process.

How Effective Are Self- and Peer Assessment of Oral Presentation Skills Compared with Teachers' Assessments?

Science.gov (United States)

De Grez, Luc; Valcke, Martin; Roozen, Irene

2012-01-01

Assessment of oral presentation skills is an underexplored area. The study described here focuses on the agreement between professional assessment and self- and peer assessment of oral presentation skills and explores student perceptions about peer assessment. The study has the merit of paying attention to the inter-rater reliability of the…

Defining and assessing critical thinking skills for student radiographers

International Nuclear Information System (INIS)

Castle, Alan

2009-01-01

Developing critical thinking skills is a key aim of higher education and is important in preparing student radiographers for their future careers in clinical practice. The aim of this paper was to attempt to devise and assess six key components of critical thinking appropriate for radiographic practice. Each of the six components was divided into three dimensions and a Critical Thinking Skills Scoring Chart (CTSSC) devised to assess students' written performance against each dimension. Scores revealed that approximately 30% of students were rated as good and approximately 10% of students were rated as poor in each component, although there was some variability between different dimensions. It is suggested that educators need to encourage and support students to develop their critical thinking skills by reviewing their curriculum to clearly define specific skills and ensure that they are appropriately taught and assessed

Helpers' Self-Assessment Biases Before and after Helping Skills Training

Science.gov (United States)

Jaeken, Marine; Zech, Emmanuelle; Brison, Céline; Verhofstadt, Lesley L.; Van Broeck, Nady; Mikolajczak, Moïra

2017-01-01

Several studies have shown that therapists are generally biased concerning their performed helping skills, as compared to judges' ratings. As clients' ratings of therapists' performance are better predictors of psychotherapy effectiveness than judges' ratings, this study examined the validity and effectiveness of a helping skills training program at reducing novice helpers' self-enhancement biases concerning their helping skills, in comparison to their clients' ratings. Helping skills were assessed by three objective measures (a knowledge multiple choice test, a video test and a role play), as well as by a self- and peer-reported questionnaire. In addition, some performed helping skills' correlates (relationship quality, session quality, and helpers' therapeutic attitudes) were assessed both by helpers and their simulated helpees. Seventy-two sophomores in psychology participated to this study, 37 being assigned to a 12-h helping skills training program, and 35 to a control group. Helpers were expected to assess the aforementioned performed helping skills and correlates as being better than their helpees' assessments at pretest, thus revealing a self-enhancement bias. At posttest, we expected that trained helpers would objectively exhibit better helping skills than untrained helpers while beginning to underestimate their performance, thus indexing a self-diminishment bias. In contrast, we hypothesized that untrained helpers would continue to overestimate their performance. Our hypotheses were only partly confirmed but results reflected a skilled-unaware pattern among trainees. Trained helpers went either from a pretest overestimation to a posttest equivalence (performed helping skills and performed therapeutic attitudes), or from a pretest equivalence to a posttest underestimation (performed session quality and performed therapeutic relationship), as compared to helpees' ratings. Results showed that trained helpers improved on all helping skills objective measures

Helpers' Self-Assessment Biases Before and after Helping Skills Training.

Science.gov (United States)

Jaeken, Marine; Zech, Emmanuelle; Brison, Céline; Verhofstadt, Lesley L; Van Broeck, Nady; Mikolajczak, Moïra

2017-01-01

Several studies have shown that therapists are generally biased concerning their performed helping skills, as compared to judges' ratings. As clients' ratings of therapists' performance are better predictors of psychotherapy effectiveness than judges' ratings, this study examined the validity and effectiveness of a helping skills training program at reducing novice helpers' self-enhancement biases concerning their helping skills, in comparison to their clients' ratings. Helping skills were assessed by three objective measures (a knowledge multiple choice test, a video test and a role play), as well as by a self- and peer-reported questionnaire. In addition, some performed helping skills' correlates (relationship quality, session quality, and helpers' therapeutic attitudes) were assessed both by helpers and their simulated helpees. Seventy-two sophomores in psychology participated to this study, 37 being assigned to a 12-h helping skills training program, and 35 to a control group. Helpers were expected to assess the aforementioned performed helping skills and correlates as being better than their helpees' assessments at pretest, thus revealing a self-enhancement bias. At posttest, we expected that trained helpers would objectively exhibit better helping skills than untrained helpers while beginning to underestimate their performance, thus indexing a self-diminishment bias. In contrast, we hypothesized that untrained helpers would continue to overestimate their performance. Our hypotheses were only partly confirmed but results reflected a skilled-unaware pattern among trainees. Trained helpers went either from a pretest overestimation to a posttest equivalence (performed helping skills and performed therapeutic attitudes), or from a pretest equivalence to a posttest underestimation (performed session quality and performed therapeutic relationship), as compared to helpees' ratings. Results showed that trained helpers improved on all helping skills objective measures

Objective assessment of technical skills in surgery

OpenAIRE

Moorthy, Krishna; Munz, Yaron; Sarker, Sudip K; Darzi, Ara

2003-01-01

In the past few years, considerable developments have been made in the objective assessment of technical proficiency of surgeons. Technical skills should be assessed during training, and various methods have been developed for this purpose

Automated surgical skill assessment in RMIS training.

Science.gov (United States)

Zia, Aneeq; Essa, Irfan

2018-05-01

Manual feedback in basic robot-assisted minimally invasive surgery (RMIS) training can consume a significant amount of time from expert surgeons' schedule and is prone to subjectivity. In this paper, we explore the usage of different holistic features for automated skill assessment using only robot kinematic data and propose a weighted feature fusion technique for improving score prediction performance. Moreover, we also propose a method for generating 'task highlights' which can give surgeons a more directed feedback regarding which segments had the most effect on the final skill score. We perform our experiments on the publicly available JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS) and evaluate four different types of holistic features from robot kinematic data-sequential motion texture (SMT), discrete Fourier transform (DFT), discrete cosine transform (DCT) and approximate entropy (ApEn). The features are then used for skill classification and exact skill score prediction. Along with using these features individually, we also evaluate the performance using our proposed weighted combination technique. The task highlights are produced using DCT features. Our results demonstrate that these holistic features outperform all previous Hidden Markov Model (HMM)-based state-of-the-art methods for skill classification on the JIGSAWS dataset. Also, our proposed feature fusion strategy significantly improves performance for skill score predictions achieving up to 0.61 average spearman correlation coefficient. Moreover, we provide an analysis on how the proposed task highlights can relate to different surgical gestures within a task. Holistic features capturing global information from robot kinematic data can successfully be used for evaluating surgeon skill in basic surgical tasks on the da Vinci robot. Using the framework presented can potentially allow for real-time score feedback in RMIS training and help surgical trainees have more focused training.

Rubric Authoring Tool Supporting Cognitive Skills Assessment across an Institution

Science.gov (United States)

Simper, Natalie

2018-01-01

This paper explores a method to support instructors in assessing cognitive skills in their course, designed to enable aggregation of data across an institution. A rubric authoring tool, "BASICS" (Building Assessment Scaffolds for Intellectual Cognitive Skills) was built as part of the Queen's University Learning Outcomes Assessment (LOA)…

Using the Interpersonal Skills tool to assess interpersonal skills of internationally educated nurses.

Science.gov (United States)

Shen, Jay J; Xu, Yu; Staples, Shelley; Bolstad, Anne L

2014-07-01

To assess interpersonal skills of internationally educated nurses (IEN) while interacting with standardized patients. Participants included 52 IEN at two community hospitals in the southwestern region of the USA. Standardized patients were used to create patient-nurse encounter. Seventeen items in four domains ("skills in interviewing and collecting information"; "skills in counseling and delivering information"; "rapport"; and "personal manner") in an Interpersonal Skills (IPS) instrument were measured by a Likert scale 1-4 with 4 indicating the best performance. The average composite score per domain and scores of the 17 items were compared across the domains. On 10 of the 17 items, the nurses received scores under 3. Counseling with an average score of 2.10 and closure with an average score of 2.44 in domain 2, small talk with an average score of 2.06 in domain 3, and physical exam with average score of 2.21 in domain 4 were below 2.5. The average composite score of domain 1 was 3.54, significantly higher than those of domains 2-4 (2.77, 2.81, and 2.71, respectively). Age was moderately related to the average score per domain with every 10 year increase in age resulting in a 0.1 increase in the average score. Sex and country of origin showed mixed results. The interpersonal skills of IEN in three of the four domains need improvement. Well-designed educational programs may facilitate the improvement, especially in areas of small talk, counseling, closure, and physical exam. Future research should examine relationships between the IPS and demographics factors. © 2013 The Authors. Japan Journal of Nursing Science © 2013 Japan Academy of Nursing Science.

Medical and psychology students' self-assessed communication skills: A pilot study.

Science.gov (United States)

Tiuraniemi, Juhani; Läärä, Riitta; Kyrö, Tuuli; Lindeman, Sari

2011-05-01

The purpose of this study was to describe how psychology and medical students assess their own competency and skills before and after training, in which role-play was used to teach interpersonal and communication skills. Interpersonal and communication skills were assessed with a semi-structured questionnaire before and after the training. The students of both medicine and psychology estimated their skill levels to be higher after the course. The psychology students estimated their skills for communication, motivating interviewing, empathy and reflection, and change orientation to be better at the end of the course. Medical students estimated their communication skills, motivating interviewing skills, and change orientation skills to be better at the end of the course. Even a short period of training in interpersonal and communication skills can positively affect the self-assessed skills of the medical students. In the future, it would be worthwhile to pay attention to reflective teaching practices in the training of both medical and psychology students. The cognitive and emotional components of these practices help students to develop their own communication skills. Copyright © 2010 Elsevier Ireland Ltd. All rights reserved.

Adapting the helpful responses questionnaire to assess communication skills involved in delivering contingency management: preliminary psychometrics.

Science.gov (United States)

Hartzler, Bryan

2015-08-01

A paper/pencil instrument, adapted from Miller and colleagues' (1991) Helpful Responses Questionnaire (HRQ), was developed to assess clinician skill with core communicative aspects involved in delivering contingency management (CM). The instrument presents a single vignette consisting of six points of client dialogue to which respondents write 'what they would say next.' In the context of an implementation/effectiveness hybrid trial, 19 staff clinicians at an opiate treatment program completed serial training outcome assessments before, following, and three months after CM training. Assessments included this adaptation of the HRQ, a multiple-choice CM knowledge test, and a recorded standardized patient encounter scored for CM skillfulness. Study results reveal promising psychometric properties for the instrument, including strong scoring reliability, internal consistency, concurrent and predictive validity, test-retest reliability and sensitivity to training effects. These preliminary findings suggest the instrument is a viable, practical method to assess clinician skill in communicative aspects of CM delivery. Copyright © 2015 Elsevier Inc. All rights reserved.

A Field-Based Testing Protocol for Assessing Gross Motor Skills in Preschool Children: The Children's Activity and Movement in Preschool Study Motor Skills Protocol

Science.gov (United States)

Williams, Harriet G.; Pfeiffer, Karin A.; Dowda, Marsha; Jeter, Chevy; Jones, Shaverra; Pate, Russell R.

2009-01-01

The purpose of this study was to develop a valid and reliable tool for use in assessing motor skills in preschool children in field-based settings. The development of the Children's Activity and Movement in Preschool Study Motor Skills Protocol included evidence of its reliability and validity for use in field-based environments as part of large…

Tools for the direct observation and assessment of psychomotor skills in medical trainees: a systematic review.

Science.gov (United States)

Jelovsek, J Eric; Kow, Nathan; Diwadkar, Gouri B

2013-07-01

The Accreditation Council for Graduate Medical Education (ACGME) Milestone Project mandates programmes to assess the attainment of training outcomes, including the psychomotor (surgical or procedural) skills of medical trainees. The objectives of this study were to determine which tools exist to directly assess psychomotor skills in medical trainees on live patients and to identify the data indicating their psychometric and edumetric properties. An electronic search was conducted for papers published from January 1948 to May 2011 using the PubMed, Education Resource Information Center (ERIC), Cumulative Index to Nursing and Allied Health Literature (CINAHL) and Web of Science electronic databases and the review of references in article bibliographies. A study was included if it described a tool or instrument designed for the direct observation of psychomotor skills in patient care settings by supervisors. Studies were excluded if they referred to tools that assessed only clinical or non-technical skills, involved non-medical health professionals, or assessed skills performed on a simulator. Overall, 4114 citations were screened, 168 (4.1%) articles were reviewed for eligibility and 51 (1.2%) manuscripts were identified as meeting the study inclusion criteria. Three authors abstracted and reviewed studies using a standardised form for the presence of key psychometric and edumetric elements as per ACGME and American Psychological Association (APA) recommendations, and also assigned an overall grade based on the ACGME Committee on Educational Outcome Assessment grading system. A total of 30 tools were identified. Construct validity based on associations between scores and training level was identified in 24 tools, internal consistency in 14, test-retest reliability in five and inter-rater reliability in 20. The modification of attitudes, knowledge or skills was reported using five tools. The seven-item Global Rating Scale and the Procedure-Based Assessment received an

Zoo U: A Stealth Approach to Social Skills Assessment in Schools

Directory of Open Access Journals (Sweden)

Melissa E. DeRosier

2012-01-01

Full Text Available This paper describes the design and evaluation of Zoo U, a novel computer game to assess children’s social skills development. Zoo U is an innovative product that combines theory-driven content and customized game mechanics. The game-like play creates the opportunity for stealth assessment, in which dynamic evidence of social skills is collected in real time and players’ choices during gameplay provide the needed data. To ensure the development of an engaging and valid game, we utilized an iterative data-driven validation process in which the game was created, tested, revised based on student performance and feedback, and retested until game play was statistically matched to independent ratings of social skills. We first investigated whether the data collected through extensive logging of student actions provided information that could be used to improve the assessment. We found that detailed game logs of socially relevant player behavior combined with external measures of player social skills provided an efficient vector to incrementally improve the accuracy of the embedded assessments. Next, we investigated whether the game performance correlated with teachers’ assessments of students’ social skills competencies. An evaluation of the final game showed (a significant correlations between in-game social skills assessments and independently obtained standard psychological assessments of the same students and (b high levels of engagement and likeability for students. These findings support the use of the interactive and engaging computer game format for the stealth assessment of children’s social skills. The created innovative design methodologies should prove useful in the design and improvement of computer games in education.

A simple framework for assessing technical skills in a resident observed structured clinical examination (OSCE): vaginal laceration repair.

Science.gov (United States)

Winkel, Abigail Ford; Lerner, Veronica; Zabar, Sondra R; Szyld, Demian

2013-01-01

Educators of trainees in procedure-based specialties need focused assessment tools that are valid, objective, and assess technical skills in a realistic context. A framework for hybrid assessment using standardized patient scenarios and bench skills testing might facilitate evaluation of competency. Seven PGY-1 obstetrics and gynecology residents participated in a hybrid assessment that used observed structured clinical examination (OSCE) by a standardized patient who had sustained a vaginal laceration during vaginal delivery. The residents elicited a history and counseled the patient, and then completed a laceration repair on a pelvic model. The residents were rated on their performance in the scenario, which included issues of cultural competency, rapport-building, patient counseling. The technical skills were videotaped and rated using a modified global assessment form by 2 faculty members on a 3-point scale from "not done" to "partly done" to "well-done." Residents also completed a subjective assessment of the station. Mean technical performance of the residents on the technical skills was 55% "well-done," with a range of 20%-90%. The assessment identified 3 residents as below the mean, and 1 resident with areas of deficiency. Subjective assessment by the residents was that juggling the technical, cognitive, and affective components of the examination was challenging. Technical skills can be included in a case-based assessment using scenarios that address a range of cognitive and affective skills required of physicians. Results may help training programs assess individuals' abilities as well as identify program needs for curricular improvement. This framework might be useful in setting standards for competency and identifying poor performers. Copyright © 2012 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

The Role of Crowdsourcing in Assessing Surgical Skills.

Science.gov (United States)

Katz, Andrew J

2016-08-01

Assessing surgical skill is critical in improving patient care while reducing medical errors, length of stay, and readmission rates. Crowdsourcing provides 1 potential method for accurately assessing this; only recently has crowdsourcing been studied as a valid way to provide feedback to surgeons. The results of such studies are explored. A systematic literature search was performed on PubMed to identify studies that have attempted to validate crowdsourcing as a method for assessing surgical skill. Through a combination of abstract screening and full-length review, 9 studies that met the inclusion criteria were reviewed. Crowdsourcing has been validated as an important way to provide feedback for surgical skill. It has been demonstrated to be effective in both dry-lab and live surgery, for a variety of tasks and methods. However, more studies must be performed to ensure that crowdsourcing can provide quality feedback in a wider variety of scenarios.

Teaching and Assessing Teamwork Skills in Engineering and Computer Science

Directory of Open Access Journals (Sweden)

Robert W. Lingard

2010-02-01

Full Text Available To be successful in today's workplace, engineering and computer science students must possess high levels of teamwork skills. Unfortunately, most engineering programs provide little or no specific instruction in this area. This paper outlines an assessment-driven approach toward teaching teamwork skills. Working with the Industrial Advisory Board for the College, a set of performance criteria for teamwork was developed. This set of criteria was used to build an assessment instrument to measure the extent to which students are able to achieve the necessary skills. This set of criteria provides a clear basis for the development of an approach toward teaching teamwork skills. Furthermore, the results from the assessment can be used to adjust the teaching techniques to address the particular skills where students show some weaknesses. Although this effort is in the early stages, the approach seems promising and will be improved over time.

Video and accelerometer-based motion analysis for automated surgical skills assessment.

Science.gov (United States)

Zia, Aneeq; Sharma, Yachna; Bettadapura, Vinay; Sarin, Eric L; Essa, Irfan

2018-03-01

Basic surgical skills of suturing and knot tying are an essential part of medical training. Having an automated system for surgical skills assessment could help save experts time and improve training efficiency. There have been some recent attempts at automated surgical skills assessment using either video analysis or acceleration data. In this paper, we present a novel approach for automated assessment of OSATS-like surgical skills and provide an analysis of different features on multi-modal data (video and accelerometer data). We conduct a large study for basic surgical skill assessment on a dataset that contained video and accelerometer data for suturing and knot-tying tasks. We introduce "entropy-based" features-approximate entropy and cross-approximate entropy, which quantify the amount of predictability and regularity of fluctuations in time series data. The proposed features are compared to existing methods of Sequential Motion Texture, Discrete Cosine Transform and Discrete Fourier Transform, for surgical skills assessment. We report average performance of different features across all applicable OSATS-like criteria for suturing and knot-tying tasks. Our analysis shows that the proposed entropy-based features outperform previous state-of-the-art methods using video data, achieving average classification accuracies of 95.1 and 92.2% for suturing and knot tying, respectively. For accelerometer data, our method performs better for suturing achieving 86.8% average accuracy. We also show that fusion of video and acceleration features can improve overall performance for skill assessment. Automated surgical skills assessment can be achieved with high accuracy using the proposed entropy features. Such a system can significantly improve the efficiency of surgical training in medical schools and teaching hospitals.

Assessing Students' Technical Skill Attainment

Science.gov (United States)

Jorgensen, Haley

2010-01-01

The Wisconsin Technical College System (WTCS) is working to comply with the Carl D. Perkins Career and Technical Education Improvement Act of 2006 (Perkins) to ensure that its graduates have mastered the technical skills needed by business and industry. The legislation requires that each state identify and approve program assessment strategies…

A Comparison of Assessment Tools: Is Direct Observation an Improvement Over Objective Structured Clinical Examinations for Communications Skills Evaluation?

Science.gov (United States)

Goch, Abraham M; Karia, Raj; Taormina, David; Kalet, Adina; Zuckerman, Joseph; Egol, Kenneth A; Phillips, Donna

2018-04-01

Evaluation of resident physicians' communications skills is a challenging task and is increasingly accomplished with standardized examinations. There exists a need to identify the effective, efficient methods for assessment of communications skills. We compared objective structured clinical examination (OSCE) and direct observation as approaches for assessing resident communications skills. We conducted a retrospective cohort analysis of orthopaedic surgery resident physicians at a single tertiary care academic institution, using the Institute for Healthcare Communication "4 Es" model for effective communication. Data were collected between 2011 and 2015. A total of 28 residents, each with OSCE and complete direct observation assessment checklists, were included in the analysis. Residents were included if they had 1 OSCE assessment and 2 or more complete direct observation assessments. There were 28 of a possible 59 residents (47%) included. A total of 89% (25 of 28) of residents passed the communications skills OSCE; only 54% (15 of 28) of residents passed the direct observation communications assessment. There was a positive, moderate correlation between OSCE and direct observation scores overall ( r  = 0.415, P  = .028). There was no agreement between OSCE and direct observation in categorizing residents into passing and failing scores (κ = 0.205, P  = .16), after adjusting for chance agreement. Our results suggest that OSCE and direct observation tools provide different insights into resident communications skills (simulation of rare and challenging situations versus real-life daily encounters), and may provide useful perspectives on resident communications skills in different contexts.

Soft Skills Assessment: Theory Development and the Research Agenda

Science.gov (United States)

Gibb, Stephen

2014-01-01

Soft skills have become a subject of increasing interest in lifelong learning. Soft skills development is intended to enable and enhance personal development, participation in learning and success in employment. The assessment of soft skill is therefore widely practised, but there is little in the way of research or evidence on how well this…

The development of aerobic and skill assessment in soccer.

Science.gov (United States)

O'Reilly, John; Wong, Stephen H S

2012-12-01

Methods of assessing soccer players' performance have developed significantly in recent times. The fitness profiles and skill levels of a prospective elite soccer player is a valuable resource for coaches in the process of identifying talent. Traditional means to measure aerobic fitness have centred on the 'aerobic capacity' or '&OV0312;O(2max)' test (also known as the maximal oxygen consumption test) but, over time, this has been shown not to be a sensitive measure for specific aspects of soccer in a match situation. Therefore, numerous soccer-specific simulations have been designed to re-create exercise patterns similar to those experienced during a match. Some of these studies have yet to be validated, while others have been shown to result in a similar physiological load to that encountered during regular match play. Further developments have led to specifically designed intermittent sprint tests, which are used as a sensitive tool to accurately measure the fluctuations in players' ability both between and within soccer seasons. Testing procedures have also been developed that incorporate elements of both skill and physical ability. Soccer-specific field tests have been designed, incorporating skill and dynamic movements, and this opens up the possibility of teams testing the aerobic capacity of their elite players using soccer-specific movements. Valid studies assessing soccer-specific skills in an ecologically sound environment have been quite rare until recently. Some test protocols have been deemed largely irrelevant to soccer match play, while others have had limited impact on scientific literature. More recently, skill tests have been developed and shown to be valid and reliable methods of assessing soccer skill performance. Many new skill tests continue to be developed, and some have been shown to be highly reliable, but further study of these relatively novel concepts is required before a more solid recommendation can be made. Overall, while significant

Skill and independence weighting for multi-model assessments

International Nuclear Information System (INIS)

Sanderson, Benjamin M.; Wehner, Michael; Knutti, Reto

2017-01-01

We present a weighting strategy for use with the CMIP5 multi-model archive in the fourth National Climate Assessment, which considers both skill in the climatological performance of models over North America as well as the inter-dependency of models arising from common parameterizations or tuning practices. The method exploits information relating to the climatological mean state of a number of projection-relevant variables as well as metrics representing long-term statistics of weather extremes. The weights, once computed can be used to simply compute weighted means and significance information from an ensemble containing multiple initial condition members from potentially co-dependent models of varying skill. Two parameters in the algorithm determine the degree to which model climatological skill and model uniqueness are rewarded; these parameters are explored and final values are defended for the assessment. The influence of model weighting on projected temperature and precipitation changes is found to be moderate, partly due to a compensating effect between model skill and uniqueness. However, more aggressive skill weighting and weighting by targeted metrics is found to have a more significant effect on inferred ensemble confidence in future patterns of change for a given projection.

The feasibility of a multi-format Web-based assessment of physicians' communication skills.

Science.gov (United States)

Kim, Sara; Brock, Douglas M; Hess, Brian J; Holmboe, Eric S; Gallagher, Thomas H; Lipner, Rebecca S; Mazor, Kathleen M

2011-09-01

Little is known about the best approaches and format for measuring physicians' communication skills in an online environment. This study examines the reliability and validity of scores from two Web-based communication skill assessment formats. We created two online communication skill assessment formats: (a) MCQ (multiple-choice questions) consisting of video-based multiple-choice questions; (b) multi-format including video-based multiple-choice questions with rationales, Likert-type scales, and free text responses of what physicians would say to a patient. We randomized 100 general internists to each test format. Peer and patient ratings collected via the American Board of Internal Medicine (ABIM) served as validity sources. Seventy-seven internists completed the tests (MCQ: 38; multi-format: 39). The adjusted reliability was 0.74 for both formats. Excellent communicators, as based on their peer and patient ratings, performed slightly better on both tests than adequate communicators, though this difference was not statistically significant. Physicians in both groups rated test format innovative (4.2 out of 5.0). The acceptable reliability and participants' overall positive experiences point to the value of ongoing research into rigorous Web-based communication skills assessment. With efficient and reliable scoring, the Web offers an important way to measure and potentially enhance physicians' communication skills. Copyright © 2011 Elsevier Ireland Ltd. All rights reserved.

Developing Employability Skills: Peer Assessment in Higher Education

Science.gov (United States)

Cassidy, Simon

2006-01-01

Purpose: Reports examining graduate employment issues suggest that employers are concerned by the lack of employability skills exhibited by entry-level job applications. It is also suggested that employers consider it the responsibility of educational institutions to develop such skills. The current study seeks to identify peer assessment as a…

Defining and Assessing Team Skills of Business and Accountancy Students

Science.gov (United States)

Alghalith, Nabil; Blum, Michael; Medlock, Amanda; Weber, Sandy

2004-01-01

The objectives of the project are (1) to define the skills necessary for students to work effectively with others to achieve common goals, and (2) to develop an assessment instrument to measure student progress toward achieving these skills. The defined skill set will form a basis for common expectations related to team skills that will be shared…

Development and validation of a new assessment tool for suturing skills in medical students.

Science.gov (United States)

Sundhagen, Henriette Pisani; Almeland, Stian Kreken; Hansson, Emma

2018-01-01

In recent years, emphasis has been put on that medical student should demonstrate pre-practice/pre-registration core procedural skills to ensure patient safety. Nonetheless, the formal teaching and training of basic suturing skills to medical students have received relatively little attention and there is no standard for what should be tested and how. The aim of this study was to develop and validate, using scientific methods, a tool for assessment of medical students' suturing skills, measuring both micro- and macrosurgical qualities. A tool was constructed and content, construct, concurrent validity, and inter-rater, inter-item, inter-test reliability were tested. Three groups were included: students with no training in suturing skills, students who have had training, plastic surgery. The results show promising reliability and validity when assessing novice medical students' suturing skills. Further studies are needed on implementation of the instrument. Moreover, how the instrument can be used to give formative feedback, evaluate if a required standard is met and for curriculum development needs further investigation.Level of Evidence: Not ratable.

Accuracy and reliability of peer assessment of athletic training psychomotor laboratory skills.

Science.gov (United States)

Marty, Melissa C; Henning, Jolene M; Willse, John T

2010-01-01

Peer assessment is defined as students judging the level or quality of a fellow student's understanding. No researchers have yet demonstrated the accuracy or reliability of peer assessment in athletic training education. To determine the accuracy and reliability of peer assessment of athletic training students' psychomotor skills. Cross-sectional study. Entry-level master's athletic training education program. First-year (n  =  5) and second-year (n  =  8) students. Participants evaluated 10 videos of a peer performing 3 psychomotor skills (middle deltoid manual muscle test, Faber test, and Slocum drawer test) on 2 separate occasions using a valid assessment tool. Accuracy of each peer-assessment score was examined through percentage correct scores. We used a generalizability study to determine how reliable athletic training students were in assessing a peer performing the aforementioned skills. Decision studies using generalizability theory demonstrated how the peer-assessment scores were affected by the number of participants and number of occasions. Participants had a high percentage of correct scores: 96.84% for the middle deltoid manual muscle test, 94.83% for the Faber test, and 97.13% for the Slocum drawer test. They were not able to reliably assess a peer performing any of the psychomotor skills on only 1 occasion. However, the φ increased (exceeding the 0.70 minimal standard) when 2 participants assessed the skill on 3 occasions (φ  =  0.79) for the Faber test, with 1 participant on 2 occasions (φ  =  0.76) for the Slocum drawer test, and with 3 participants on 2 occasions for the middle deltoid manual muscle test (φ  =  0.72). Although students did not detect all errors, they assessed their peers with an average of 96% accuracy. Having only 1 student assess a peer performing certain psychomotor skills was less reliable than having more than 1 student assess those skills on more than 1 occasion. Peer assessment of psychomotor skills

How to Assess Creative Thinking Skill in Making Products of Liquid Pressure?

Science.gov (United States)

Chasanah, L.; Kaniawati, I.; Hernani, H.

2017-09-01

The primary skills that must be possessed in the 21st century curriculum are learning and innovation skills. One of the learning strategies that can train students to innovate and improve creative thinking skills is by applying Science, Technology, Engineering and Mathematics (STEM) in learning. Based on an interview to one of the science teachers that learning that aims to train learning and innovation skills has not been applied to learning in the classroom because there is not enough time, learning materials and assessment instruments used. This study aims to determine the results of the implementation of performance assessment of creative thinking skills on STEM-based learning in junior high school for the material of liquid pressure. This research uses descriptive method. Participants in this study were junior high school students 8th in Kudus area. The research instrument consists of observation sheet, performance assessment and documentation. The result showed that creative thinking skills performance assessment can assess student’s creativity in making products of STEM-based learning for junior high school.

A Qualitative Analysis of Narrative Preclerkship Assessment Data to Evaluate Teamwork Skills.

Science.gov (United States)

Dolan, Brigid M; O'Brien, Celia Laird; Cameron, Kenzie A; Green, Marianne M

2018-04-16

Construct: Students entering the health professions require competency in teamwork. Although many teamwork curricula and assessments exist, studies have not demonstrated robust longitudinal assessment of preclerkship students' teamwork skills and attitudes. Assessment portfolios may serve to fill this gap, but it is unknown how narrative comments within portfolios describe student teamwork behaviors. We performed a qualitative analysis of narrative data in 15 assessment portfolios. Student portfolios were randomly selected from 3 groups stratified by quantitative ratings of teamwork performance gathered from small-group and clinical preceptor assessment forms. Narrative data included peer and faculty feedback from these same forms. Data were coded for teamwork-related behaviors using a constant comparative approach combined with an identification of the valence of the coded statements as either "positive observation" or "suggestion for improvement." Eight codes related to teamwork emerged: attitude and demeanor, information facilitation, leadership, preparation and dependability, professionalism, team orientation, values team member contributions, and nonspecific teamwork comments. The frequency of codes and valence varied across the 3 performance groups, with students in the low-performing group receiving more suggestions for improvement across all teamwork codes. Narrative data from assessment portfolios included specific descriptions of teamwork behavior, with important contributions provided by both faculty and peers. A variety of teamwork domains were represented. Such feedback as collected in an assessment portfolio can be used for longitudinal assessment of preclerkship student teamwork skills and attitudes.

Does a Rater's Professional Background Influence Communication Skills Assessment?

Science.gov (United States)

Artemiou, Elpida; Hecker, Kent G; Adams, Cindy L; Coe, Jason B

2015-01-01

There is increasing pressure in veterinary education to teach and assess communication skills, with the Objective Structured Clinical Examination (OSCE) being the most common assessment method. Previous research reveals that raters are a large source of variance in OSCEs. This study focused on examining the effect of raters' professional background as a source of variance when assessing students' communication skills. Twenty-three raters were categorized according to their professional background: clinical sciences (n=11), basic sciences (n=4), clinical communication (n=5), or hospital administrator/clinical skills technicians (n=3). Raters from each professional background were assigned to the same station and assessed the same students during two four-station OSCEs. Students were in year 2 of their pre-clinical program. Repeated-measures ANOVA results showed that OSCE scores awarded by the rater groups differed significantly: (F(matched_station_1) [2,91]=6.97, p=.002), (F(matched_station_2) [3,90]=13.95, p=.001), (F(matched_station_3) [3,90]=8.76, p=.001), and ((Fmatched_station_4) [2,91]=30.60, p=.001). A significant time effect between the two OSCEs was calculated for matched stations 1, 2, and 4, indicating improved student performances. Raters with a clinical communication skills background assigned scores that were significantly lower compared to the other rater groups. Analysis of written feedback provided by the clinical sciences raters showed that they were influenced by the students' clinical knowledge of the case and that they did not rely solely on the communication checklist items. This study shows that it is important to consider rater background both in recruitment and training programs for communication skills' assessment.

Structured assessment of microsurgery skills in the clinical setting.

Science.gov (United States)

Chan, WoanYi; Niranjan, Niri; Ramakrishnan, Venkat

2010-08-01

Microsurgery is an essential component in plastic surgery training. Competence has become an important issue in current surgical practice and training. The complexity of microsurgery requires detailed assessment and feedback on skills components. This article proposes a method of Structured Assessment of Microsurgery Skills (SAMS) in a clinical setting. Three types of assessment (i.e., modified Global Rating Score, errors list and summative rating) were incorporated to develop the SAMS method. Clinical anastomoses were recorded on videos using a digital microscope system and were rated by three consultants independently and in a blinded fashion. Fifteen clinical cases of microvascular anastomoses performed by trainees and a consultant microsurgeon were assessed using SAMS. The consultant had consistently the highest scores. Construct validity was also demonstrated by improvement of SAMS scores of microsurgery trainees. The overall inter-rater reliability was strong (alpha=0.78). The SAMS method provides both formative and summative assessment of microsurgery skills. It is demonstrated to be a valid, reliable and feasible assessment tool of operating room performance to provide systematic and comprehensive feedback as part of the learning cycle. Copyright 2009 British Association of Plastic, Reconstructive and Aesthetic Surgeons. Published by Elsevier Ltd. All rights reserved.

Training self-assessment and task-selection skills to foster self-regulated learning: Do trained skills transfer across domains?

Science.gov (United States)

Raaijmakers, Steven F; Baars, Martine; Paas, Fred; van Merriënboer, Jeroen J G; van Gog, Tamara

2018-01-01

Students' ability to accurately self-assess their performance and select a suitable subsequent learning task in response is imperative for effective self-regulated learning. Video modeling examples have proven effective for training self-assessment and task-selection skills, and-importantly-such training fostered self-regulated learning outcomes. It is unclear, however, whether trained skills would transfer across domains. We investigated whether skills acquired from training with either a specific, algorithmic task-selection rule or a more general heuristic task-selection rule in biology would transfer to self-regulated learning in math. A manipulation check performed after the training confirmed that both algorithmic and heuristic training improved task-selection skills on the biology problems compared with the control condition. However, we found no evidence that students subsequently applied the acquired skills during self-regulated learning in math. Future research should investigate how to support transfer of task-selection skills across domains.

Approaching time is important for assessment of endoscopic surgical skills.

Science.gov (United States)

Tokunaga, Masakazu; Egi, Hiroyuki; Hattori, Minoru; Yoshimitsu, Masanori; Sumitani, Daisuke; Kawahara, Tomohiro; Okajima, Masazumi; Ohdan, Hideki

2012-05-01

This study aimed to verify whether the approaching time (the time taken to reach the target point from another point, a short distance apart, during point-to-point movement in endoscopic surgery), assessed using the Hiroshima University Endoscopic Surgical Assessment Device (HUESAD), could distinguish the skill level of surgeons. Expert surgeons (who had performed more than 50 endoscopic surgeries) and novice surgeons (who had no experience in performing endoscopic surgery) were tested using the HUESAD. The approaching time, total time, and intermediate time (total time--approaching time) were measured and analyzed using the trajectory of the tip of the instrument. The approaching time and total time were significantly shorter in the expert group than in the novice group (p time did not significantly differ between the groups (p > 0.05). The approaching time, which is a component of the total time, is very mportant in the measurement of the total time to assess endoscopic surgical skills. Further, the approaching time was useful for skill assessment by the HUESAD for evaluating the skill of surgeons performing endoscopic surgery.

Accuracy and Reliability of Peer Assessment of Athletic Training Psychomotor Laboratory Skills

Science.gov (United States)

Marty, Melissa C.; Henning, Jolene M.; Willse, John T.

2010-01-01

Abstract Context: Peer assessment is defined as students judging the level or quality of a fellow student's understanding. No researchers have yet demonstrated the accuracy or reliability of peer assessment in athletic training education. Objective: To determine the accuracy and reliability of peer assessment of athletic training students' psychomotor skills. Design: Cross-sectional study. Setting: Entry-level master's athletic training education program. Patients or Other Participants: First-year (n  =  5) and second-year (n  =  8) students. Main Outcome Measure(s): Participants evaluated 10 videos of a peer performing 3 psychomotor skills (middle deltoid manual muscle test, Faber test, and Slocum drawer test) on 2 separate occasions using a valid assessment tool. Accuracy of each peer-assessment score was examined through percentage correct scores. We used a generalizability study to determine how reliable athletic training students were in assessing a peer performing the aforementioned skills. Decision studies using generalizability theory demonstrated how the peer-assessment scores were affected by the number of participants and number of occasions. Results: Participants had a high percentage of correct scores: 96.84% for the middle deltoid manual muscle test, 94.83% for the Faber test, and 97.13% for the Slocum drawer test. They were not able to reliably assess a peer performing any of the psychomotor skills on only 1 occasion. However, the ϕ increased (exceeding the 0.70 minimal standard) when 2 participants assessed the skill on 3 occasions (ϕ  =  0.79) for the Faber test, with 1 participant on 2 occasions (ϕ  =  0.76) for the Slocum drawer test, and with 3 participants on 2 occasions for the middle deltoid manual muscle test (ϕ  =  0.72). Conclusions: Although students did not detect all errors, they assessed their peers with an average of 96% accuracy. Having only 1 student assess a peer performing certain psychomotor skills was

Educational Assessment Knowledge and Skills for Teachers

Science.gov (United States)

Brookhart, Susan M.

2011-01-01

The 1990 Standards for Teacher Competence in Educational Assessment of Students (AFT, NCME, & NEA, 1990) made a documentable contribution to the field. However, the Standards have become a bit dated, most notably in two ways: (1) the Standards do not consider current conceptions of formative assessment knowledge and skills, and (2) the Standards…

Assessment Training Effects on Student Assessment Skills and Task Performance in a Technology-Facilitated Peer Assessment

Science.gov (United States)

Liu, Xiongyi; Li, Lan

2014-01-01

This study examines the impact of an assessment training module on student assessment skills and task performance in a technology-facilitated peer assessment. Seventy-eight undergraduate students participated in the study. The participants completed an assessment training exercise, prior to engaging in peer-assessment activities. During the…

Assessment of medical communication skills by computer: assessment method and student experiences

NARCIS (Netherlands)

Hulsman, R. L.; Mollema, E. D.; Hoos, A. M.; de Haes, J. C. J. M.; Donnison-Speijer, J. D.

2004-01-01

BACKGROUND A computer-assisted assessment (CAA) program for communication skills designated ACT was developed using the objective structured video examination (OSVE) format. This method features assessment of cognitive scripts underlying communication behaviour, a broad range of communication

Interpersonal Skills in MBA Admissions: How Are They Conceptualized and Assessed?

Science.gov (United States)

Beenen, Gerard; Pichler, Shaun; Davoudpour, Shahin

2018-01-01

Employers and students concur that soft skills or interpersonal skills are critical to managerial success, yet we know little about how MBA program admissions professionals conceptualize and assess these skills in the context of global management education. Such practices have key implications for interpersonal skills curriculum and training in…

The Differential Effects of Task Complexity on Domain-Specific and Peer Assessment Skills

Science.gov (United States)

van Zundert, Marjo J.; Sluijsmans, Dominique M. A.; Konings, Karen D.; van Merrienboer, Jeroen J. G.

2012-01-01

In this study the relationship between domain-specific skills and peer assessment skills as a function of task complexity is investigated. We hypothesised that peer assessment skills were superposed on domain-specific skills and will therefore suffer more when higher cognitive load is induced by increased task complexity. In a mixed factorial…

In-game assessment and training of nonverbal cognitive skills using TagTiles

NARCIS (Netherlands)

Verhaegh, J.; Fontijn, W.F.J.; Aarts, E.H.L.; Resing, W.C.M.

2013-01-01

We present a field study with a game for children called 'Tap the little hedgehog', which is played on the TagTiles console, a tangible electronic interface. The game was developed to train and assess cognitive skills and includes tasks which, in isolation, exhibit a high correlation with a number

Observational assessment of fundamental movement skill proficiency in preschool children

OpenAIRE

佐々木, 玲子; 石沢, 順子

2014-01-01

Fundamental movement skill competency in children has been declining in recent years. Early childhood is a sensitive period for the development of fundamental movement skills ; the mastery of certain of these skills is a prerequisite for daily functioning and participation in later physical or sport-specific activities. Although quantitative methods have been developed for assessing movement development in children, it is also important to qualitatively evaluate such skills in developing chil...

Objective assessment of technical surgical skills

NARCIS (Netherlands)

van Hove, P. D.; Tuijthof, G. J. M.; Verdaasdonk, E. G. G.; Stassen, L. P. S.; Dankelman, J.

2010-01-01

Surgeons are increasingly being scrutinized for their performance and there is growing interest in objective assessment of technical skills. The purpose of this study was to review all evidence for these methods, in order to provide a guideline for use in clinical practice. A systematic search was

Reliable and valid assessment of Lichtenstein hernia repair skills

DEFF Research Database (Denmark)

Carlsen, C G; Lindorff Larsen, Karen; Funch-Jensen, P

2014-01-01

PURPOSE: Lichtenstein hernia repair is a common surgical procedure and one of the first procedures performed by a surgical trainee. However, formal assessment tools developed for this procedure are few and sparsely validated. The aim of this study was to determine the reliability and validity...... of an assessment tool designed to measure surgical skills in Lichtenstein hernia repair. METHODS: Key issues were identified through a focus group interview. On this basis, an assessment tool with eight items was designed. Ten surgeons and surgical trainees were video recorded while performing Lichtenstein hernia...... a significant difference between the three groups which indicates construct validity, p skills can be assessed blindly by a single rater in a reliable and valid fashion with the new procedure-specific assessment tool. We recommend this tool for future assessment...

Development of Authentic Assessment instruments for Critical Thinking skills in Global Warming with a Scientific Approach

Directory of Open Access Journals (Sweden)

R. Surya Damayanti

2017-12-01

Full Text Available This study aims to develop an authentic assessment instrument to measure critical thinking skills in global warming learning and to describe the suitability, easiness, and usefulness of the use instruments which are developed base on the teacher’s opinion.   The development design is carried out by Borg & Gall (2003 development model, which is conducted with seven stages: information gathering stage, planning stage, product development stage, product test stage, product revision stage, field trial stage, and final product. The test subjects are students and teachers in SMA Lampung Tengah by using purposive sampling technique.  Global warming learning using authentic assessment consists of a series of learning activities, including observing, discussing, exploring, associating and communicating.  The results show the authentic assessment techniques global warming to measure and cultivate critical thinking skills consisting of written tests, performance, portfolios, projects, and attitudes.  The developed assessment model meets content and constructs validity, and effectively improves students' critical thinking skills and has a high level of suitability, easiness, and usefulness well-being. The assessment techniques are used in global warming learning are performance assessment techniques, portfolios, projects, products, and attitude that together contribute to the improvement of critical thinking skills on 97,4% of global warming learning.

High educational impact of a national simulation-based urological curriculum including technical and non-technical skills.

Science.gov (United States)

de Vries, Anna H; Schout, Barbara M A; van Merriënboer, Jeroen J G; Pelger, Rob C M; Koldewijn, Evert L; Muijtjens, Arno M M; Wagner, Cordula

2017-02-01

Although simulation training is increasingly used to meet modern technology and patient safety demands, its successful integration within surgical curricula is still rare. The Dutch Urological Practical Skills (D-UPS) curriculum provides modular simulation-based training of technical and non-technical basic urological skills in the local hospital setting. This study aims to assess the educational impact of implementing the D-UPS curriculum in the Netherlands and to provide focus points for improvement of the D-UPS curriculum according to the participants. Educational impact was assessed by means of qualitative individual module-specific feedback and a quantitative cross-sectional survey among residents and supervisors. Twenty out of 26 Dutch teaching hospitals participated. The survey focussed on practical aspects, the D-UPS curriculum in general, and the impact of the D-UPS curriculum on the development of technical and non-technical skills. A considerable survey response of 95 % for residents and 76 % for supervisors was obtained. Modules were attended by junior and senior residents, supervised by a urologist, and peer teaching was used. Ninety percent of supervisors versus 67 % of residents judged the D-UPS curriculum as an important addition to current residency training (p = 0.007). Participants' aggregated general judgement of the modules showed a substantial percentage favorable score (M ± SE: 57 ± 4 %). The impact of training on, e.g., knowledge of materials/equipment and ability to anticipate on complications was high, especially for junior residents (77 ± 5 and 71 ± 7 %, respectively). Focus points for improvement of the D-UPS curriculum according to the participants include adaptation of the training level to residents' level of experience and focus on logistics. The simulation-based D-UPS curriculum has a high educational impact. Residents and supervisors consider the curriculum to be an important addition to current residency

Reliability and Validity of the Research Methods Skills Assessment

Science.gov (United States)

Smith, Tamarah; Smith, Samantha

2018-01-01

The Research Methods Skills Assessment (RMSA) was created to measure psychology majors' statistics knowledge and skills. The American Psychological Association's Guidelines for the Undergraduate Major in Psychology (APA, 2007, 2013) served as a framework for development. Results from a Rasch analysis with data from n = 330 undergraduates showed…

Assessing Thinking Skills in Astro 101: Do We Make an Impact?

Science.gov (United States)

Bruning, D.

2005-12-01

Most instructors agree that a major goal of "Astronomy 101" is to develop thinking skills in our students (Partridge and Greenstein, AER 2, 46, 2003). Much educational research in astronomy has initially concentrated on "best practices" for improving student learning (development of "think-pair-share", lecture tutorials, peer tutoring, etc.). Little has been done to date to assess our efforts to improve student thinking skills and students' desire to think more deeply about the cognitively rich ideas offered in the typical astronomy class. This study surveys several astronomy and physics courses to determine whether general analytical thinking skills increase because of the science course and whether students' attitudes toward cognition improve. Cacioppo, Petty and Kao's "Need for Cognition" scale is used for the latter assessment (J. Personality Assessment 48, 306, 1984). A shortened version of Whimbey and Lochhead's ASI skills instrument is used to assess analytical skills ("Problem Solving and Comprehension," 1986). Preliminary results suggest that students need for cognition does not change in general, although there may be a correlation between increasing need for cognition and improvement in grades through the semester. There is a suggestion that need for cognition is slightly predictive of course performance, but a greater correlation exists between the post-course survey and grades. Gains in general analytical skills have been seen in initial surveys, but correlations with course performance appear elusive.

Formative assessment of GP trainees' clinical skills.

Science.gov (United States)

Wiener-Ogilvie, Sharon; Begg, Drummond

2012-03-01

Clinical skill assessment (CSA) has been an integral part of the Royal College of General Practitioners' membership examination (MRCGP) since 2008. It is an expensive, high-stakes examination with first time pass rates ranging from 76.4 to 81.3. In this paper we describe the South East Scotland Deanery, NHS Education Scotland, pilot of a formative clinical skills assessment (fCSA) using the principles of formative assessment and OSCE. The purpose of the study was to assess the acceptability of the fCSA and to examine whether trainees, identified during the fCSA as 'at risk of failing the MRCGP CSA exam', are more likely to fail the MRCGP CSA exam later on in the year. Trainees were assessed in four clinical skills stations under exam conditions. After each station they were given verbal feedback and subsequently both trainee and their trainer received written feedback. We assessed the value of the exercise through written feedback from trainees and trainers. Each trainee's performance in fCSA was triangulated with trainer assessment to identify 'flagged trainees'. We compared flagged and non-flagged trainees' performance in MRCGP CSA. Both trainees and trainers highly rated the fCSA. Overall 97% of non-flagged trainees have passed the RCGP CSA exam by May of that year in comparison to 80% of flagged trainees who have passed the RCGP CSA (P = 0.005). Trainers and trainees rated the fCSA as excellent and useful. We were able to demonstrate that the fCSA can be used to identify those trainees likely to fail the RCGP CSA. Contrary to reservations about the potential to demoralise trainees, the fCSA was viewed as a useful and a positive experience by both trainees and trainers. In addition, we suggest that feedback from fCSA was useful in triggering appropriate educational interventions. Early intervention with trainees who are predicted to fail the CSA has the potential to reduce deaneries overall fail rate. Preventing one trainee failure could save over £30 000.

Development of assessment instruments to measure critical thinking skills

Science.gov (United States)

Sumarni, W.; Supardi, K. I.; Widiarti, N.

2018-04-01

Assessment instruments that is commonly used in the school generally have not been orientated on critical thinking skills. The purpose of this research is to develop assessment instruments to measure critical thinking skills, to test validity, reliability, and practicality. This type of research is Research and Development. There are two stages on the preface step, which are field study and literacy study. On the development steps, there some parts, which are 1) instrument construction, 2) expert validity, 3) limited scale tryout and 4) narrow scale try-out. The developed assessment instrument are analysis essay and problem solving. Instruments were declared valid, reliable and practical.

Are general surgeons able to accurately self-assess their level of technical skills?

Science.gov (United States)

Rizan, C; Ansell, J; Tilston, T W; Warren, N; Torkington, J

2015-11-01

Self-assessment is a way of improving technical capabilities without the need for trainer feedback. It can identify areas for improvement and promote professional medical development. The aim of this review was to identify whether self-assessment is an accurate form of technical skills appraisal in general surgery. The PubMed, MEDLINE(®), Embase(™) and Cochrane databases were searched for studies assessing the reliability of self-assessment of technical skills in general surgery. For each study, we recorded the skills assessed and the evaluation methods used. Common endpoints between studies were compared to provide recommendations based on the levels of evidence. Twelve studies met the inclusion criteria from 22,292 initial papers. There was no level 1 evidence published. All papers compared the correlation between self-appraisal versus an expert score but differed in the technical skills assessment and the evaluation tools used. The accuracy of self-assessment improved with increasing experience (level 2 recommendation), age (level 3 recommendation) and the use of video playback (level 3 recommendation). Accuracy was reduced by stressful learning environments (level 2 recommendation), lack of familiarity with assessment tools (level 3 recommendation) and in advanced surgical procedures (level 3 recommendation). Evidence exists to support the reliability of self-assessment of technical skills in general surgery. Several variables have been shown to affect the accuracy of self-assessment of technical skills. Future work should focus on evaluating the reliability of self-assessment during live operating procedures.

The assessment of mindfulness skills: the "what" and the "how".

Science.gov (United States)

Iani, Luca; Lauriola, Marco; Cafaro, Valentina

2017-10-06

The five facets mindfulness questionnaire-short form (FFMQ-SF) is a new, brief measure for the assessment of mindfulness skills in clinical and nonclinical samples. The construct validity of the FFMQ-SF has not been previously assessed in community samples. The present study investigated the factor structure of the Italian version of the FFMQ-SF. Structured equation modeling was used to test the fit of three alternative models in a sample of highly educated adults (n = 211). A hierarchical model with a single second-order factor loaded by observing, describing, and acting with awareness (i.e. the mindfulness "what" skills) performed slightly better than both a five-factor model with correlated factors and a hierarchical model with a general second-order factor. The FFMQ-SF scores were significantly higher than those reported in both Dutch depressed patients and Australian undergraduate students for all facets (but nonreactivity for the Australian sample). Data support the multifaceted nature of mindfulness skills. Because of its brevity and simplicity of use, the FFMQ-SF is a promising questionnaire in longitudinal and clinical research. This questionnaire can serve as a guideline to help clinicians assess and monitor mindfulness skills acquisition, strengthening, and generalization, and prioritize mindfulness skills that need immediate attention.

Assessing computer skills in Tanzanian medical students: an elective experience

Directory of Open Access Journals (Sweden)

Melvin Rob

2004-08-01

Full Text Available Abstract Background One estimate suggests that by 2010 more than 30% of a physician's time will be spent using information technology tools. The aim of this study is to assess the information and communication technologies (ICT skills of medical students in Tanzania. We also report a pilot intervention of peer mentoring training in ICT by medical students from the UK tutoring students in Tanzania. Methods Design: Cross sectional study and pilot intervention study. Participants: Fourth year medical students (n = 92 attending Muhimbili University College of Health Sciences, Dar es Salaam, Tanzania. Main outcome measures: Self-reported assessment of competence on ICT-related topics and ability to perform specific ICT tasks. Further information related to frequency of computer use (hours per week, years of computer use, reasons for use and access to computers. Skills at specific tasks were reassessed for 12 students following 4 to 6 hours of peer mentoring training. Results The highest levels of competence in generic ICT areas were for email, Internet and file management. For other skills such as word processing most respondents reported low levels of competence. The abilities to perform specific ICT skills were low – less than 60% of the participants were able to perform the core specific skills assessed. A period of approximately 5 hours of peer mentoring training produced an approximate doubling of competence scores for these skills. Conclusion Our study has found a low level of ability to use ICT facilities among medical students in a leading university in sub-Saharan Africa. A pilot scheme utilising UK elective students to tutor basic skills showed potential. Attention is required to develop interventions that can improve ICT skills, as well as computer access, in order to bridge the digital divide.

Self vs expert assessment of technical and non-technical skills in high fidelity simulation.

Science.gov (United States)

Arora, Sonal; Miskovic, Danilo; Hull, Louise; Moorthy, Krishna; Aggarwal, Rajesh; Johannsson, Helgi; Gautama, Sanjay; Kneebone, Roger; Sevdalis, Nick

2011-10-01

Accurate assessment is imperative for learning, feedback and progression. The aim of this study was to examine whether surgeons can accurately self-assess their technical and nontechnical skills compared with expert faculty members' assessments. Twenty-five surgeons performed a laparoscopic cholecystectomy (LC) in a simulated operating room. Technical and nontechnical performance was assessed by participants and faculty members using the validated Objective Structured Assessment of Technical Skills (OSATS) and the Non-Technical Skills for Surgeons scale (NOTSS). Assessment of technical performance correlated between self and faculty members' ratings for experienced (median score, 30.0 vs 31.0; ρ = .831; P = .001) and inexperienced (median score, 22.0 vs 28.0; ρ = .761; P = .003) surgeons. Assessment of nontechnical skills between self and faculty members did not correlate for experienced surgeons (median score, 8.0 vs 10.5; ρ = -.375; P = .229) or their more inexperienced counterparts (median score, 9.0 vs 7.0; ρ = -.018; P = .953). Surgeons can accurately self-assess their technical skills in virtual reality LC. Conversely, formal assessment with faculty members' input is required for nontechnical skills, for which surgeons lack insight into their behaviours. Copyright © 2011 Elsevier Inc. All rights reserved.

Analysis of Student Service-Learning Reflections for the Assessment of Transferable-Skills Development

Science.gov (United States)

Rizzo, D. M.; Dewoolkar, M.; Hayden, N.; Oka, L.; Pearce, A. R.

2010-12-01

The civil and environmental engineering (CEE) programs at the University of Vermont (UVM) incorporate systems thinking and a systems approach to sustainable engineering problem solving. A systems approach considers long-term social, environmental and economic factors within the context of the engineering problem solution and encompasses sustainable engineering solutions. Our goal is to prepare students to become leaders in their chosen field who can anticipate co-products associated with forecasted solutions. As a way of practicing the systems approach, we include service-learning projects in many of our undergraduate engineering courses, culminating with the senior capstone design course. We use a variety of formative and summative assessment methods to gage student understanding and attitudes including student surveys, focus groups, assessment of student projects, and student reflections. Student reflections from two courses -Modeling Environmental and Transportation Systems (31 juniors) and Senior Design Project (30 seniors) are compared. Of these, 25 students were common to both courses. The focus of the systems modeling service-learning project involved mentoring home-schooled children (11-14 yrs old) to solve problems of mobility, using the fun and inspiration of biomimicry. Students were required to invent innovative methods to move people or goods that improve associated constraints (i.e., minimize congestion, reduce pollution, increase safety), or reduce the need for transportation altogether. The capstone design project required a comprehensive engineering design involving two or more CEE sub-disciplines. Both service-learning projects were intended to enhance students’ academic learning experience, attain civic engagement and reinforce transferable skills (written and oral communication, teamwork, leadership and mentoring skills). The student course reflections were not guided; yet they provided valuable data to assess commonalities and differences in

Developing Enterprise Skills through Peer-Assessed Pitch Presentations

Science.gov (United States)

Faherty, Anna

2015-01-01

Purpose: The purpose of this paper is to investigate the impact of using summative peer assessment to develop enterprise skills within higher education. Design/methodology/approach: An empirical investigation analysing students own perceptions of the peer assessment process to evaluate its impact. Findings: Participating students indicate that…

Investigating Secondary School Students' Unmediated Peer Assessment Skills

Science.gov (United States)

Tsivitanidou, Olia E.; Zacharia, Zacharias C.; Hovardas, Tasos

2011-01-01

The purpose of this study was to investigate secondary school students' unmediated peer assessment skills. Specifically, 36 seventh graders, without receiving any kind of support, were anonymously assigned to reciprocally assess their peers' science web-portfolios. Additionally, students' attitudes towards and intentions about the use of…

Objective psychomotor skills assessment of experienced, junior, and novice laparoscopists with virtual reality.

Science.gov (United States)

Gallagher, A G; Richie, K; McClure, N; McGuigan, J

2001-11-01

Objective assessment of psychomotor skills in surgery is now a priority; however, this assessment is difficult to achieve because of measurement difficulties associated with the reliability and validity of assessing surgical skills in vivo and in the laboratory. In this study virtual reality (VR) was used to overcome these problems in the objective psychomotor assessment of senior, junior, and novice laparoscopists. Twelve experienced laparoscopic surgeons (performed >50 Minimal Access Surgery (MAS) procedures), 12 inexperienced laparoscopic surgeons (psychomotor skills for laparoscopic surgery.

Instructor and Dental Student Perceptions of Clinical Communication Skills via Structured Assessments.

Science.gov (United States)

McKenzie, Carly T

2016-05-01

The aim of this study was to use structured assessments to assess dental students' clinical communication skills exhibited during patient appointments. Fourth-year dental students (n=55) at the University of Alabama at Birmingham evaluated their own interpersonal skills in a clinical setting utilizing the Four Habits Coding Scheme. An instructor also assessed student-patient clinical communication. These assessments were used to identify perceived strengths and weaknesses in students' clinical communication. Both instructor assessments and student self-assessments pinpointed the following clinical communication skills as effective the most often: patient greeting, avoidance of jargon, and non-verbal behavior. There was also relative agreement between instructor assessments and student self-assessments regarding clinical communication skills that were rated as not effective most frequently: ensuring patient comprehension, identification of patient feelings, and exploration of barriers to treatment. These resulted pointed to strengths and weaknesses in the portion of the curriculum designed to prepare students for effective provider-patient communication. These results may suggest a need for the school's current behavioral science curriculum to better address discussion of potential treatment barriers and patient feelings as well as techniques to ensure patient comprehension.

Communication skills assessment in the final postgraduate years to established practice: a systematic review.

Science.gov (United States)

Gillis, Amy E; Morris, Marie C; Ridgway, Paul F

2015-01-01

Communication breakdown is a factor in the majority of all instances of medical error. Despite the importance, a relative paucity of time is invested in communication skills in postgraduate curricula. Our objective is to systematically review the literature to identify the current tools used to assess communication skills in postgraduate trainees in the latter 2 years of training and in established practice. Two reviewers independently reviewed the literature identifying communication skill assessment tools, for postgraduate trainees in the latter 2 years of training and in established practice following Preferred Reporting Items for Systematic Reviews and Meta-Analyses framework, and inclusion/exclusion criteria from January 1990 to 15 August 2014. PubMed/CINAHL/ERIC/EMBASE/PsycInfo/Psyc Articles/Cochrane. 222 articles were identified; after review, 34 articles fulfilled criteria for complete evaluation; the majority (26) had a high level of evidence scoring 3 or greater on the Best Evidence Medical Education guide. 22 articles used objective structured clinical examination/standardised patient (SP)-based formats in an assessment or training capacity. Evaluation tools included author-developed questionnaires and validated tools. Nineteen articles demonstrated an educational initiative. The reviewed literature is heterogeneous for objectives and measurement techniques for communication. Observed interactions, with patients or SPs, is the current favoured method of evaluation using author-developed questionnaires. The role of self-evaluation of skill level is questioned. The need for a validated assessment tool for communication skills is highlighted. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Using the Dynamic Model to Identify Stages of Teacher Skills in Assessment

Science.gov (United States)

Christoforidou, Margarita; Xirafidou, Elisavet

2014-01-01

The article presents the results of two cross-sectional studies that investigate teachers' skills in using various techniques of assessment in mathematics by taking into account the four phases of assessment. The five dimensions of the dynamic model are also taken into account in proposing a framework for measuring teacher skills in assessment.…

A qualitative study of advanced nurse practitioners’ use of physical assessment skills in the community: shifting skills across professional boundaries

OpenAIRE

Raleigh, Mary; Allan, Helen T.

2017-01-01

Aim\\ud To explore multiple perspectives on the use of physical assessment skills by Advanced Nurse Practitioners in the UK\\ud Background \\ud Physical assessment skills practices are embedded in advanced nursing practice roles in the UK. There is little evidence on how these skills are used by Advanced Nurse Practitioners' on the community.\\ud Design \\ud Case study\\ud Methodology and methods. \\ud A qualitative interpretative single-embedded case study of 22 participants from South of England. ...

Assessing Change in High School Student Information Literacy Using the Tool for Real-Time Assessment of Information Literacy Skills

Science.gov (United States)

Kovalik, Cindy L.; Yutzey, Susan D.; Piazza, Laura M.

2012-01-01

Change in high school student information literacy (IL) knowledge and skills, from freshman year to senior year in high school was the focus of this quasi-experimental research project. Researchers used a free information literacy skills assessment tool entitled TRAILS (Tool for Real-time Assessment of Information Literacy Skills) to measure…

Towards an Integrated Model for Developing Sustainable Assessment Skills

Science.gov (United States)

Fastre, Greet M. J.; van der Klink, Marcel R.; Sluijsmans, Dominique; van Merrienboer, Jeroen J. G.

2013-01-01

One of the goals of current education is to ensure that graduates can act as independent lifelong learners. Graduates need to be able to assess their own learning and interpret assessment results. The central question in this article is how to acquire sustainable assessment skills, enabling students to assess their performance and learning…

The Development and Validation of a Golf Swing and Putt Skill Assessment for Children

Directory of Open Access Journals (Sweden)

Lisa M. Barnett, Louise L. Hardy, Ali S. Brian

2015-03-01

Full Text Available The aim was to describe development of a process-oriented instrument designed to assess the golf swing and putt stroke, and to assess the instrument’s discriminative validity in terms of age and reliability (intra-rater and re-test. A Delphi consultation (with golf industry professionals and researchers in movement skill assessment was used to develop an assessment for each skill based on existing skill assessment protocols. Each skill had six components to be marked as present/absent. Individual scores were based on the number of performance components successfully demonstrated over two trials for each skill (potential score range 0 to 24. Children (n = 43 aged 6-10 years (M = 7.8 years, SD = 1.3 were assessed in both skills live in the field by one rater at Time 1(T1. A subset of children (n = 28 had consent for assessments to be videoed. Six weeks later 19 children were reassessed, five days apart (T2, T3. An ANOVA assessed discriminative validity i.e. whether skill competence at T1 differed by age (6 years, 7/8 years and 9/10 years. Intraclass correlations (ICC assessed intra-rater reliability between the live and video assessment at T1 and test-retest reliability (between T2 and T3. Paired t-tests assessed any systematic differences between live and video assessments (T1 and between T2 and T3. Older children were more skilled (F (2, 40 = 11.18, p < 0.001. The live assessment reflected the video assessment (ICC = 0.79, 95% CI 0.59, 0.90 and scores did not differ between live and video assessments. Test retest reliability was acceptable (ICC = 0.60, 95% CI 0.23, 0.82, although the mean score was slightly higher at retest. This instrument could be used reliably by golf coaches and physical education teachers as part of systematic early player assessment and feedback.

Nursing assessment of older people who are in hospital: exploring registered nurses' understanding of their assessment skills.

Science.gov (United States)

Penney, Wendy; Poulter, Nola; Cole, Clare; Wellard, Sally

2016-01-01

Nurses worldwide are expected to take a leading role in caring for older people. Considerable literature dedicated to the range and application of assessment skills used by nurses vary. There is limited knowledge of registered nurses' (RNs) views of their assessment of older adults. The aim of this project was to explore RNs current perceptions of nursing assessment, and the core skills they identified as necessary. A qualitative descriptive design study was conducted in three inpatient units in one regional hospital in Victoria. Date were collected through participant observation of RNs (n = 13) followed by 1:1 semi-structured interviews. Data were analysed thematically. This research has illuminated that an ill-defined repertoire of skills was used by RNs when assessing older persons. Skills identified appeared to be based on years of personal-professional experience. Differences were noted between the descriptions nurses gave and what was observed during interactions with older persons.

Impact of tailored feedback in assessment of communication skills for medical students.

Science.gov (United States)

Uhm, Seilin; Lee, Gui H; Jin, Jeong K; Bak, Yong I; Jeoung, Yeon O; Kim, Chan W

2015-01-01

Finding out the effective ways of teaching and assessing communication skills remain a challenging part of medication education. This study aims at exploring the usefulness and effectiveness of having additional feedback using qualitative analysis in assessment of communication skills in undergraduate medical training. We also determined the possibilities of using qualitative analysis in developing tailored strategies for improvement in communication skills training. This study was carried out on medical students (n=87) undergoing their final year clinical performance examination on communication skills using standardized patient by video-recording and transcribing their performances. Video-recordings of 26 students were randomly selected for qualitative analysis, and additional feedback was provided. We assessed the level of acceptance of communication skills scores between the study and nonstudy group and within the study group, before and after receiving feedback based on qualitative analysis. There was a statistically significant increase in the level of acceptance of feedback after delivering additional feedback using qualitative analysis, where the percentage of agreement with feedback increased from 15.4 to 80.8% (pskills assessment gives essential information for medical students to learn and self-reflect, which could potentially lead to improved communication skills. As evident from our study, feedback becomes more meaningful and effective with additional feedback using qualitative analysis.

Virtual reality-based assessment of basic laparoscopic skills using the Leap Motion controller.

Science.gov (United States)

Lahanas, Vasileios; Loukas, Constantinos; Georgiou, Konstantinos; Lababidi, Hani; Al-Jaroudi, Dania

2017-12-01

The majority of the current surgical simulators employ specialized sensory equipment for instrument tracking. The Leap Motion controller is a new device able to track linear objects with sub-millimeter accuracy. The aim of this study was to investigate the potential of a virtual reality (VR) simulator for assessment of basic laparoscopic skills, based on the low-cost Leap Motion controller. A simple interface was constructed to simulate the insertion point of the instruments into the abdominal cavity. The controller provided information about the position and orientation of the instruments. Custom tools were constructed to simulate the laparoscopic setup. Three basic VR tasks were developed: camera navigation (CN), instrument navigation (IN), and bimanual operation (BO). The experiments were carried out in two simulation centers: MPLSC (Athens, Greece) and CRESENT (Riyadh, Kingdom of Saudi Arabia). Two groups of surgeons (28 experts and 21 novices) participated in the study by performing the VR tasks. Skills assessment metrics included time, pathlength, and two task-specific errors. The face validity of the training scenarios was also investigated via a questionnaire completed by the participants. Expert surgeons significantly outperformed novices in all assessment metrics for IN and BO (p assessment of basic laparoscopic skills. The proposed system allowed the evaluation of dexterity of the hand movements. Future work will involve comparison studies with validated simulators and development of advanced training scenarios on current Leap Motion controller.

Assessment of HIV/AIDS and Life Skills Delivery in Primary Schools in Tanzania

Science.gov (United States)

Rushahu, Bernadetha Gabriel

2015-01-01

This study was conducted to assess the effectiveness of HIV/AIDS and the Life Skills Education delivery in primary schools in Tanzania. Specifically the study investigated pupils' views about the effect of HIV/AIDS and Life Skills teaching in primary schools in Tanzania; assessed pupils' knowledge related to HIV/AIDS and Life Skills education, and…

Assessing maths literacy skills in type 1 diabetic children and their ...

African Journals Online (AJOL)

Background: It is essential that children have adequate numeracy and literacy skills in order to manage their diabetes effectively. The objective was to undertake a pilot study to assess the level of numeracy skills in type 1 diabetic children and their caregivers, and to ascertain if there was a deficit in these skills.

Concrete Steps for Assessing the "Soft Skills" in an MBA Program

Science.gov (United States)

Ingols, Cynthia; Shapiro, Mary

2014-01-01

In 2006, our School of Management began the serious path of assessing both the "hard skills" (such as accounting, finance, and strategy) and the "soft skills" (such as leadership, team work, and ethics) of our MBA Program. The data generated from examining the "soft skills" that we want students to learn within our…

A Guttman scale for assessing condom use skills among college students.

Science.gov (United States)

Lindemann, Dana F; Brigham, Thomas A

2003-03-01

HIV risk-reduction interventions frequently teach correct condom use skills, yet few researchers actually measure those skills. Without measuring condom use skills, it is not possible to evaluate the effectiveness of condom use interventions, or determine what variables may be related to those skills. The Measure of Observed Condom Use Skills (MOCUS) was developed using standard Guttman scaling procedures, and administered to 106 female and 71 male undergraduates. The MOCUS was administered by the primary investigator to each participant individually. Guttman scale analyses demonstrated the MOCUS has acceptable reproducibility. These findings support future use of the MOCUS to assess condom use skills.

Assessment of Teacher Perceived Skill in Classroom Assessment Practices Using IRT Models

Science.gov (United States)

Koloi-Keaikitse, Setlhomo

2017-01-01

The purpose of this study was to assess teacher perceived skill in classroom assessment practices. Data were collected from a sample of (N = 691) teachers selected from government primary, junior secondary, and senior secondary schools in Botswana. Item response theory models were used to identify teacher response on items that measured their…

STUDENTS’ COMMUNICATION SKILLS ASSESSMENT BY EXTERNAL LECTURERS AND INDUSTRY REPRESENTATIVES

Directory of Open Access Journals (Sweden)

NOORFAZILA KAMAL

2016-11-01

Full Text Available Soft skills, especially communication skills, are important skills which each graduate must possess. Accordingly, several courses and approaches have been carried out in universities to train students in communication skills. The skills are normally evaluated by course lecturers. However, assessments by a third party from outside the university are valuable so that the students’ ahievements may be weighed against external evaluators’ point of views. In the Department of Electrical, Electronic and Systems Engineering (DEESE, Universiti Kebangsaan Malaysia (UKM, communication skills assessment by external lecturers and industry representatives are performed on Hari Poster JKEES, where students present their final year project poster. There are two categories of evaluation, namely project and communication skills. The project evaluation covers content, result and impact, while communication skills evaluation covers poster layout and design, and delivery. This study only analyse the students’ communication skills achievement. The participants of this study consists of 109 final year students from two batches, of which 51 students are from year 2014 and the other 58 students from year 2015. The results show that for the year 2014 students, the mean mark given by external lecturers in layout and design category is 6.7, while the mean mark from industry evaluators is 6.5. For the 2015 batch, the mean mark in the layout and design category is 6.3 from external lecturers and 5.9 from industry evaluators. In the delivery category, the mean marks for the 2014 batch are 7.1 and 6.6 from external lecturers and industry evaluators, espectively. Meanwhile, for the 2015 batch, the mean marks by external lecturers and industry evaluators are 6.3 and 5.8, respectively. The results show that both external lecturers and industry representatives judged DEESE students’ communication skills to be good.

Certified Basic Life Support Instructors Assess Cardiopulmonary Resuscitation Skills Poorly

DEFF Research Database (Denmark)

Hansen, Camilla; Rasmussen, Stinne E; Kristensen, Mette Amalie

2016-01-01

Introduction: High-quality cardiopulmonary resuscitation (CPR) improves survival from cardiac arrest. During basic life support (BLS) training, instructors assess CPR skills to enhance learning outcome. Emergency department staff and senior residents have been shown to assess chest compression...... quality poorly. Currently no studies have evaluated CPR assessment among certified BLS instructors. The aim of this study was to investigate certified BLS instructors’ assessment of chest compressions and rescue breathing.Methods: Data were collected at BLS courses for medical students at Aarhus...... of CPR skills may be beneficial to ensure high-quality learning outcome.Author Disclosures: C. Hansen: None. S.E. Rasmussen: None. M.A. Nebsbjerg: None. M. Stærk: None. B. Løfgren: None....

Cognitive skills assessment during robot-assisted surgery: separating the wheat from the chaff.

Science.gov (United States)

Guru, Khurshid A; Esfahani, Ehsan T; Raza, Syed J; Bhat, Rohit; Wang, Katy; Hammond, Yana; Wilding, Gregory; Peabody, James O; Chowriappa, Ashirwad J

2015-01-01

To investigate the utility of cognitive assessment during robot-assisted surgery (RAS) to define skills in terms of cognitive engagement, mental workload, and mental state; while objectively differentiating between novice and expert surgeons. In all, 10 surgeons with varying operative experience were assigned to beginner (BG), combined competent and proficient (CPG), and expert (EG) groups based on the Dreyfus model. The participants performed tasks for basic, intermediate and advanced skills on the da Vinci Surgical System. Participant performance was assessed using both tool-based and cognitive metrics. Tool-based metrics showed significant differences between the BG vs CPG and the BG vs EG, in basic skills. While performing intermediate skills, there were significant differences only on the instrument-to-instrument collisions between the BG vs CPG (2.0 vs 0.2, P = 0.028), and the BG vs EG (2.0 vs 0.1, P = 0.018). There were no significant differences between the CPG and EG for both basic and intermediate skills. However, using cognitive metrics, there were significant differences between all groups for the basic and intermediate skills. In advanced skills, there were no significant differences between the CPG and the EG except time (1116 vs 599.6 s), using tool-based metrics. However, cognitive metrics revealed significant differences between both groups. Cognitive assessment of surgeons may aid in defining levels of expertise performing complex surgical tasks once competence is achieved. Cognitive assessment may be used as an adjunct to the traditional methods for skill assessment during RAS. © 2014 The Authors. BJU International © 2014 BJU International.

Subjective and objective assessment of patients' compression therapy skills as a predicator of ulcer recurrence.

Science.gov (United States)

Mościcka, Paulina; Szewczyk, Maria T; Jawień, Arkadiusz; Cierzniakowska, Katarzyna; Cwajda-Białasik, Justyna

2016-07-01

To verify whether the subjectively and objectively assessed patient's skills in applying compression therapy constitute a predicting factor of venous ulcer recurrence. Systematic implementation of compression therapy by the patient is a core of prophylaxis for recurrent ulcers. Therefore, patient education constitutes a significant element of care. However, controversies remain if all individuals benefit equally from education. A retrospective analysis. The study included medical records of patients with venous ulcers (n = 351) treated between 2001 and 2011 at the Clinic for Chronic Wounds at Bydgoszcz Clinical Hospital. We compared two groups of patients, (1) with at least one episode of recurrent ulcer during the five-year observation period, and (2) without recurrences throughout the analysed period in terms of their theoretical skills and knowledge on compression therapy recorded at baseline and after one month. Very good self-assessment of a patient's compression therapy skills and weak assessment of these skills by a nurse proved significant risk factors for recurrence of the ulcers on univariate analysis. The significance of these variables as independent risk factors for recurrent ulcers has been also confirmed on multivariate analysis, which also took into account other clinical parameters. Building up proper compression therapy skills among the patients should be the key element of a properly construed nurse-based prophylactic program, as it is the most significant modifiable risk factor for recurrent ulcers. Although the development of compression skills is undeniably important, also other factors should be considered, e.g. surgical correction of superficial reflux. Instruction on compression therapy should be conducted by properly trained nursing personnel - the nurses should have received both content and psychological training. The compression therapy training should contain practical instruction with guided exercises and in-depth objective

Preliminary Analysis of Assessment Instrument Design to Reveal Science Generic Skill and Chemistry Literacy

Science.gov (United States)

Sumarni, Woro; Sudarmin; Supartono, Wiyanto

2016-01-01

The purpose of this research is to design assessment instrument to evaluate science generic skill (SGS) achievement and chemistry literacy in ethnoscience-integrated chemistry learning. The steps of tool designing refers to Plomp models including 1) Investigation Phase (Prelimenary Investigation); 2) Designing Phase (Design); 3)…

Assessing Gross Motor Skills of Kosovar Preschool Children

Science.gov (United States)

Shala, Merita

2009-01-01

In the light of the new developments in preschool education in Kosovo, this study attempts to carry out an assessment of the development of gross motor skills of preschool children attending institutional education. The emphasis is on creating a set of tests to measure the motor attainments of these children by conducting assessments of the…

Assessment of knowledge and skills in information literacy instruction for rehabilitation sciences students: a scoping review.

Science.gov (United States)

Boruff, Jill T; Harrison, Pamela

2018-01-01

This scoping review investigates how knowledge and skills are assessed in the information literacy (IL) instruction for students in physical therapy, occupational therapy, or speech-language pathology, regardless of whether the instruction was given by a librarian. The objectives were to discover what assessment measures were used, determine whether these assessment methods were tested for reliability and validity, and provide librarians with guidance on assessment methods to use in their instruction in evidence-based practice contexts. A scoping review methodology was used. A systematic search strategy was run in Ovid MEDLINE and adapted for CINAHL; EMBASE; Education Resources Information Center (ERIC) (EBSCO); Library and Information Science Abstracts (LISA); Library, Information Science & Technology Abstracts (LISTA); and Proquest Theses and Dissertations from 1990 to January 16, 2017. Forty articles were included for data extraction. Three major themes emerged: types of measures used, type and context of librarian involvement, and skills and outcomes described. Thirty-four measures of attitude and thirty-seven measures of performance were identified. Course products were the most commonly used type of performance measure. Librarians were involved in almost half the studies, most frequently as instructor, but also as author or assessor. Information literacy skills such as question formulation and database searching were described in studies that did not involve a librarian. Librarians involved in instructional assessment can use rubrics such as the Valid Assessment of Learning in Undergraduate Education (VALUE) when grading assignments to improve the measurement of knowledge and skills in course-integrated IL instruction. The Adapted Fresno Test could be modified to better suit the real-life application of IL knowledge and skills.

Assessment Work: Examining the Prevalence and Nature of Assessment Competencies and Skills in Student Affairs Job Postings

Science.gov (United States)

Hoffman, John L.; Bresciani, Marilee J.

2010-01-01

This mixed method study explored the assessment-related skills and job duties that student affairs administrators expect from new employees as reflected in 1,759 job openings posted in 2008, of which seven job postings were specialist positions in outcomes-based assessment. The skills and duties required of these seven positions were primarily…

Implementing assessments of robot-assisted technical skill in urological education: a systematic review and synthesis of the validity evidence.

Science.gov (United States)

Goldenberg, Mitchell G; Lee, Jason Y; Kwong, Jethro C C; Grantcharov, Teodor P; Costello, Anthony

2018-03-31

To systematically review and synthesise the validity evidence supporting intraoperative and simulation-based assessments of technical skill in urological robot-assisted surgery (RAS), and make evidence-based recommendations for the implementation of these assessments in urological training. A literature search of the Medline, PsycINFO and Embase databases was performed. Articles using technical skill and simulation-based assessments in RAS were abstracted. Only studies involving urology trainees or faculty were included in the final analysis. Multiple tools for the assessment of technical robotic skill have been published, with mixed sources of validity evidence to support their use. These evaluations have been used in both the ex vivo and in vivo settings. Performance evaluations range from global rating scales to psychometrics, and assessments are carried out through automation, expert analysts, and crowdsourcing. There have been rapid expansions in approaches to RAS technical skills assessment, both in simulated and clinical settings. Alternative approaches to assessment in RAS, such as crowdsourcing and psychometrics, remain under investigation. Evidence to support the use of these metrics in high-stakes decisions is likely insufficient at present. © 2018 The Authors BJU International © 2018 BJU International Published by John Wiley & Sons Ltd.

Retrospectively Assessed Early Motor and Current Pragmatic Language Skills in Autistic and Neurotypical Children.

Science.gov (United States)

Stevenson, Jennifer L; Lindley, Caitlin E; Murlo, Nicole

2017-08-01

Autistic individuals often struggle developmentally, even in areas that are not explicit diagnostic criteria, such as motor skills. This study explored the relation between early motor skills, assessed retrospectively, and current pragmatic language skills. Caregivers of neurotypical and autistic children, matched on gender and age, completed assessments of their child's early motor development and current language abilities. Early motor skills were correlated with later pragmatic language skills, and autistic children exhibited fewer motor skills than neurotypical children. In fact, motor skills were a better predictor of an autism spectrum diagnosis than were scores on a measure of current pragmatic language. These results highlight the important role of motor skills in autism spectrum disorders.

Assessment of validity of an OSATS for cystoscopic and ureteroscopic cognitive and psychomotor skills.

Science.gov (United States)

Kishore, Thekke Adiyat; Pedro, Renato N; Monga, Manoj; Sweet, Robert M

2008-12-01

The purpose of the study was to assess the construct validity of an Objective Structured Assessment of Technical Skills (OSATS) developed for cystoscopic and ureteroscopic cognitive and psychomotor skills. An OSATS was designed based on a 14-point comprehensive curriculum prepared by two experts that targeted both cognitive and psychomotor cystoscopic and ureteroscopic skills. Ten urology residents from a single institution with different levels of training were assessed on a series of stations that targeted these skills. Evaluation of cognitive skills was done via a written examination, and psychomotor skills assessment was done by experts using both subjective and objective metrics. Twelve of 15 cognitive tasks and 5 of 5 psychomotor tasks demonstrated construct validity with correlation coefficient (r) more than .75. All three of the cognitive tasks that failed to initially demonstrate validity did so on editorial revision and restructuring of the questions. Our cystoscopic and ureteroscopic OSATS showed excellent construct validity for our population of residents, and we have incorporated it into our urologic skills curriculum.

Construct validation of teacher portfolio assessment : Procedures for improving teacher competence assessment illustrated by teaching students research skills

NARCIS (Netherlands)

Schaaf, M.F. van der

2005-01-01

The study aims to design and test procedures for teacher portfolio assessments. What are suitable procedures to assess teachers' competencies in developing students' research skills? We first searched into the tasks teachers have in teaching students research skills and the competencies needed to

Teacher Compliance and Accuracy in State Assessment of Student Motor Skill Performance

Science.gov (United States)

Hall, Tina J.; Hicklin, Lori K.; French, Karen E.

2015-01-01

Purpose: The purpose of this study was to investigate teacher compliance with state mandated assessment protocols and teacher accuracy in assessing student motor skill performance. Method: Middle school teachers (N = 116) submitted eighth grade student motor skill performance data from 318 physical education classes to a trained monitoring…

Can Perceptuo-Motor Skills Assessment Outcomes in Young Table Tennis Players (7-11 years) Predict Future Competition Participation and Performance? An Observational Prospective Study.

Science.gov (United States)

Faber, Irene R; Elferink-Gemser, Marije T; Faber, Niels R; Oosterveld, Frits G J; Nijhuis-Van der Sanden, Maria W G

2016-01-01

Forecasting future performance in youth table tennis players based on current performance is complex due to, among other things, differences between youth players in growth, development, maturity, context and table tennis experience. Talent development programmes might benefit from an assessment of underlying perceptuo-motor skills for table tennis, which is hypothesized to determine the players' potential concerning the perceptuo-motor domain. The Dutch perceptuo-motor skills assessment intends to measure the perceptuo-motor potential for table tennis in youth players by assessing the underlying skills crucial for developing technical and tactical qualities. Untrained perceptuo-motor tasks are used as these are suggested to represent a player's future potential better than specific sport skills themselves as the latter depend on exposure to the sport itself. This study evaluated the value of the perceptuo-motor skills assessment for a talent developmental programme by evaluating its predictive validity for competition participation and performance in 48 young table tennis players (7-11 years). Players were tested on their perceptuo-motor skills once during a regional talent day, and the subsequent competition results were recorded half-yearly over a period of 2.5 years. Logistic regression analysis showed that test scores did not predict future competition participation (p >0.05). Yet, the Generalized Estimating Equations analysis, including the test items 'aiming at target', 'throwing a ball', and 'eye-hand coordination' in the best fitting model, revealed that the outcomes of the perceptuo-motor skills assessment were significant predictors for future competition results (R2 = 51%). Since the test age influences the perceptuo-motor skills assessment's outcome, another multivariable model was proposed including test age as a covariate (R2 = 53%). This evaluation demonstrates promising prospects for the perceptuo-motor skills assessment to be included in a talent

Can Perceptuo-Motor Skills Assessment Outcomes in Young Table Tennis Players (7-11 years Predict Future Competition Participation and Performance? An Observational Prospective Study.

Directory of Open Access Journals (Sweden)

Irene R Faber

Full Text Available Forecasting future performance in youth table tennis players based on current performance is complex due to, among other things, differences between youth players in growth, development, maturity, context and table tennis experience. Talent development programmes might benefit from an assessment of underlying perceptuo-motor skills for table tennis, which is hypothesized to determine the players' potential concerning the perceptuo-motor domain. The Dutch perceptuo-motor skills assessment intends to measure the perceptuo-motor potential for table tennis in youth players by assessing the underlying skills crucial for developing technical and tactical qualities. Untrained perceptuo-motor tasks are used as these are suggested to represent a player's future potential better than specific sport skills themselves as the latter depend on exposure to the sport itself. This study evaluated the value of the perceptuo-motor skills assessment for a talent developmental programme by evaluating its predictive validity for competition participation and performance in 48 young table tennis players (7-11 years. Players were tested on their perceptuo-motor skills once during a regional talent day, and the subsequent competition results were recorded half-yearly over a period of 2.5 years. Logistic regression analysis showed that test scores did not predict future competition participation (p >0.05. Yet, the Generalized Estimating Equations analysis, including the test items 'aiming at target', 'throwing a ball', and 'eye-hand coordination' in the best fitting model, revealed that the outcomes of the perceptuo-motor skills assessment were significant predictors for future competition results (R2 = 51%. Since the test age influences the perceptuo-motor skills assessment's outcome, another multivariable model was proposed including test age as a covariate (R2 = 53%. This evaluation demonstrates promising prospects for the perceptuo-motor skills assessment to be

Assessing study skills among university students: an Iranian survey.

Science.gov (United States)

Didarloo, Alireza; Khalkhali, Hamid Reza

2014-05-05

Numerous studies have revealed that study skills have a constructive role on the academic performance of students, in addition to educational quality, student' intelligence, and their affective characteristics. This study aims to examine study skills and the factors influencing them among the health sciences students of Urmia University of Medical Sciences in Iran. This was a cross-sectional study carried out from May to November 2013. A total of 340 Urmia health sciences students were selected using a simple sampling method. Data were collected using the Study Skills Assessment Questionnaire of Counseling Center of Houston University and analyzed with descriptive and analytical statistics. The mean and standard deviation of the students' study skills were 172.5±23.2, out of a total score of 240. Around 1.2% of the study skills were weak; 86.8%, moderate; and 12%, good. Among the study skills, the scores of time management, and memory and concentration were better than the others. Also, there was a significant positive correlation between study skills scores and the students' family housing status and academic level (Pstudy skills, these were not sufficient and far from good. Improving and promoting the study skills of university students require the designing and implementing of education programs for study strategies. Therefore, decision makers and planners in the educational areas of universities should consider the topic described above.

Employability Skills Assessment: Measuring Work Ethic for Research and Learning

Science.gov (United States)

Park, HwaChoon; Hill, Roger B.

2016-01-01

The Employability Skills Assessment (ESA) was developed by Hill (1995) to provide an alternative measure of work ethic needed for success in employment. This study tested goodness-of-fit for a model used to interpret ESA results. The model had three factors: interpersonal skills, initiative, and dependability. Confirmatory factor analysis results…

Aligning CASAS Competencies and Assessments to Basic Skills Content Standards. Second Edition

Science.gov (United States)

CASAS - Comprehensive Adult Student Assessment Systems (NJ1), 2009

2009-01-01

Since its inception, the Comprehensive Adult Student Assessment System (CASAS) has focused on teaching and assessing basic skills in contexts that are relevant and important to adult learners. CASAS has developed and continues to refine a highly formalized hierarchy of competencies, the application of basic skills that adults need to be fully…

Ability-versus skill-based assessment of emotional intelligence.

Science.gov (United States)

Bradberry, Travis R; Su, Lac D

2006-01-01

Emotional intelligence has received an intense amount of attention in leadership circles during the last decade and continuing debate exists concerning the best method for measuring this construct. This study analyzed leader emotional intelligence scores, measured via skill and ability methodologies, against leader job performance. Two hundred twelve employees from three organizations participated in this study. Scores on the Emotional Intelligence Appraisal, a skill-based assessment, were positively, though not significantly, correlated with scores on the MSCEIT, an ability-based assessment of emotional intelligence. Scores on the MSCEIT did not have a significant relationship with job performance in this study, whereas, scores on the Emotional Intelligence Appraisal had a strong link to leader job performance. The four subcomponents of the Emotional Intelligence Appraisal were examined against job performance. Relationship management was a stronger predictor of leader job performance than the other three subcomponents. Social awareness was the single emotional intelligence skill that did not have a significant link to leader job performance. Factor analyses yielded a two-component model of emotional intelligence encompassing personal and social competence, rather than confirmation of a four-part taxonomy.

Extended score interval in the assessment of basic surgical skills.

Science.gov (United States)

Acosta, Stefan; Sevonius, Dan; Beckman, Anders

2015-01-01

The Basic Surgical Skills course uses an assessment score interval of 0-3. An extended score interval, 1-6, was proposed by the Swedish steering committee of the course. The aim of this study was to analyze the trainee scores in the current 0-3 scored version compared to a proposed 1-6 scored version. Sixteen participants, seven females and nine males, were evaluated in the current and proposed assessment forms by instructors, observers, and learners themselves during the first and second day. In each assessment form, 17 tasks were assessed. The inter-rater reliability between the current and the proposed score sheets were evaluated with intraclass correlation (ICC) with 95% confidence intervals (CI). The distribution of scores for 'knot tying' at the last time point and 'bowel anastomosis side to side' given by the instructors in the current assessment form showed that the highest score was given in 31 and 62%, respectively. No ceiling effects were found in the proposed assessment form. The overall ICC between the current and proposed score sheets after assessment by the instructors increased from 0.38 (95% CI 0.77-0.78) on Day 1 to 0.83 (95% CI 0.51-0.94) on Day 2. A clear ceiling effect of scores was demonstrated in the current assessment form, questioning its validity. The proposed score sheet provides more accurate scores and seems to be a better feedback instrument for learning technical surgical skills in the Basic Surgical Skills course.

MOJECT: MOTION ANALYSIS TO SUPPORT ASSESSMENT OF SURGICAL SKILLS

NARCIS (Netherlands)

Uineken, Ruben; Groot Jebbink, Erik; Halfwerk, F.R.; Bulten, Anne; Knoben, Peter; Roux, Moritz; Wicik, Ola; Groenier, Marleen

2018-01-01

Assessment of surgical skills is usually performed through direct observation by experts. This is subjective, expensive and requires assessor training. Motion analysis can support objective and cost-effective assessment. The aim of the current study is to design a low-cost, unobtrusive system for

Staff and students' perceptions and experiences of teaching and assessment in Clinical Skills Laboratories: interview findings from a multiple case study.

Science.gov (United States)

Houghton, Catherine E; Casey, Dympna; Shaw, David; Murphy, Kathy

2012-08-01

The Clinical Skills Laboratory has become an essential structure in nurse education and several benefits of its use have been identified. However, the literature identifies the need to examine the transferability of skills learned there into the reality of practice. This research explored the role of the Clinical Skills Laboratory in preparing nursing students for the real world of practice. This paper focuses specifically on the perceptions of the teaching and assessment strategies employed there. Qualitative multiple case study design. Five case study sites. Interviewees (n=58) included academic staff, clinical staff and nursing students. Semi-structured interviews. The Clinical Skills Laboratory can provide a pathway to practice and its authenticity is significant. Teaching strategies need to incorporate communication as well as psychomotor skills. Including audio-visual recording into assessment strategies is beneficial. Effective relationships between education institutions and clinical settings are needed to enhance the transferability of the skills learned. The Clinical Skills Laboratory should provide an authentic learning environment, with the appropriate use of teaching strategies. It is crucial that effective links between educators and clinical staff are established and maintained. Copyright © 2011 Elsevier Ltd. All rights reserved.

Psychomotor skills assessment in practicing surgeons experienced in performing advanced laparoscopic procedures.

Science.gov (United States)

Gallagher, Anthony G; Smith, C Daniel; Bowers, Steven P; Seymour, Neal E; Pearson, Adam; McNatt, Steven; Hananel, David; Satava, Richard M

2003-09-01

Minimally invasive surgery (MIS) has introduced a new and unique set of psychomotor skills for a surgeon to acquire and master. Although assessment technologies have been proposed, precise and objective psychomotor skills assessment of surgeons performing laparoscopic procedures has not been detailed. Two hundred ten surgeons attending the 2001 annual meeting of the American College of Surgeons in New Orleans who reported having completed more than 50 laparoscopic procedures participated. Subjects were required to complete one box-trainer laparoscopic cutting task and a similar virtual reality task. These tasks were specifically designed to test only psychomotor and not cognitive skills. Both tasks were completed twice. Performance of tasks was assessed and analyzed. Demographic and laparoscopic experience data were also collected. Complete data were available on 195 surgeons. In this group, surgeons performed the box-trainer task better with their dominant hand (p psychomotor skills is now possible. Surgeons who had performed more than 50 laparoscopic procedures showed considerable variability in their performance on a simple laparoscopic and virtual reality task. Approximately 10% of surgeons tested performed the task significantly worse than the group's average performance. Studies such as this may form the methodology for establishing criteria levels and performance objectives in objective assessment of the technical skills component of determining surgical competence.

Assessing the Nontechnical Skills of Surgical Trainees: Views of the Theater Team.

Science.gov (United States)

Al-Jundi, Wissam; Wild, Jonathan; Ritchie, Judith; Daniels, Sarah; Robertson, Eleanor; Beard, Jonathan

2016-01-01

This study aims to explore the views of members of theater teams regarding the proposed introduction of a workplace-based assessment of nontechnical skills of surgeons (NOTSS) into the Intercollegiate Surgical Curriculum Programme in the United Kingdom. In addition, the previous training and familiarity of the members of the surgical theater team with the concept and assessment of NOTSS would be evaluated. A regional survey of members of theater teams (consultant surgeons, anesthetists, scrub nurses, and trainees) was performed at 1 teaching and 2 district general hospitals in South Yorkshire. There were 160 respondents corresponding to a response rate of 81%. The majority (77%) were not aware of the NOTSS assessment tool with only 9% of respondents reporting to have previously used the NOTSS tool and just 3% having received training in NOTSS assessment. Overall, 81% stated that assessing NOTSS was as important as assessing technical skills. Trainees attributed less importance to nontechnical skills than the other groups (p ≤ 0.016). Although opinion appears divided as to whether the presence of a consultant surgeon in theater could potentially make it difficult to assess a trainee's leadership skills and decision-making capabilities, overall 60% agree that the routine use of NOTSS assessment would enhance safety in the operating theater and 80% agree that the NOTSS tool should be introduced to assess the nontechnical skills of trainees in theater. However, a significantly lower proportion of trainees (45%) agreed on the latter compared with the other groups (p = 0.001). Our survey demonstrates acceptability among the theater team for the introduction of the NOTSS tool into the surgical curriculum. However, lack of familiarity highlights the importance of faculty training for assessors before such an introduction. Copyright © 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Fundamental Movement Skills Are More than Run, Throw and Catch: The Role of Stability Skills

Science.gov (United States)

Rudd, James R.; Barnett, Lisa M.; Butson, Michael L.; Farrow, Damian; Berry, Jason; Polman, Remco C. J.

2015-01-01

Introduction In motor development literature fundamental movement skills are divided into three constructs: locomotive, object control and stability skills. Most fundamental movement skills research has focused on children’s competency in locomotor and object control skills. The first aim of this study was to validate a test battery to assess the construct of stability skills, in children aged 6 to 10 (M age = 8.2, SD = 1.2). Secondly we assessed how the stability skills construct fitted into a model of fundamental movement skill. Method The Delphi method was used to select the stability skill battery. Confirmatory factor analysis (CFA) was used to assess if the skills loaded onto the same construct and a new model of FMS was developed using structural equation modelling. Results Three postural control tasks were selected (the log roll, rock and back support) because they had good face and content validity. These skills also demonstrated good predictive validity with gymnasts scoring significantly better than children without gymnastic training and children from a high SES school performing better than those from a mid and low SES schools and the mid SES children scored better than the low SES children (all p < .05). Inter rater reliability tests were excellent for all three skills (ICC = 0.81, 0.87, 0.87) as was test re-test reliability (ICC 0.87–0.95). CFA provided good construct validity, and structural equation modelling revealed stability skills to be an independent factor in an overall FMS model which included locomotor (r = .88), object control (r = .76) and stability skills (r = .81). Discussion This study provides a rationale for the inclusion of stability skills in FMS assessment. The stability skills could be used alongside other FMS assessment tools to provide a holistic assessment of children’s fundamental movement skills. PMID:26468644

Fundamental Movement Skills Are More than Run, Throw and Catch: The Role of Stability Skills.

Science.gov (United States)

Rudd, James R; Barnett, Lisa M; Butson, Michael L; Farrow, Damian; Berry, Jason; Polman, Remco C J

2015-01-01

In motor development literature fundamental movement skills are divided into three constructs: locomotive, object control and stability skills. Most fundamental movement skills research has focused on children's competency in locomotor and object control skills. The first aim of this study was to validate a test battery to assess the construct of stability skills, in children aged 6 to 10 (M age = 8.2, SD = 1.2). Secondly we assessed how the stability skills construct fitted into a model of fundamental movement skill. The Delphi method was used to select the stability skill battery. Confirmatory factor analysis (CFA) was used to assess if the skills loaded onto the same construct and a new model of FMS was developed using structural equation modelling. Three postural control tasks were selected (the log roll, rock and back support) because they had good face and content validity. These skills also demonstrated good predictive validity with gymnasts scoring significantly better than children without gymnastic training and children from a high SES school performing better than those from a mid and low SES schools and the mid SES children scored better than the low SES children (all p skills (ICC = 0.81, 0.87, 0.87) as was test re-test reliability (ICC 0.87-0.95). CFA provided good construct validity, and structural equation modelling revealed stability skills to be an independent factor in an overall FMS model which included locomotor (r = .88), object control (r = .76) and stability skills (r = .81). This study provides a rationale for the inclusion of stability skills in FMS assessment. The stability skills could be used alongside other FMS assessment tools to provide a holistic assessment of children's fundamental movement skills.

Fundamental Movement Skills Are More than Run, Throw and Catch: The Role of Stability Skills.

Directory of Open Access Journals (Sweden)

James R Rudd

Full Text Available In motor development literature fundamental movement skills are divided into three constructs: locomotive, object control and stability skills. Most fundamental movement skills research has focused on children's competency in locomotor and object control skills. The first aim of this study was to validate a test battery to assess the construct of stability skills, in children aged 6 to 10 (M age = 8.2, SD = 1.2. Secondly we assessed how the stability skills construct fitted into a model of fundamental movement skill.The Delphi method was used to select the stability skill battery. Confirmatory factor analysis (CFA was used to assess if the skills loaded onto the same construct and a new model of FMS was developed using structural equation modelling.Three postural control tasks were selected (the log roll, rock and back support because they had good face and content validity. These skills also demonstrated good predictive validity with gymnasts scoring significantly better than children without gymnastic training and children from a high SES school performing better than those from a mid and low SES schools and the mid SES children scored better than the low SES children (all p < .05. Inter rater reliability tests were excellent for all three skills (ICC = 0.81, 0.87, 0.87 as was test re-test reliability (ICC 0.87-0.95. CFA provided good construct validity, and structural equation modelling revealed stability skills to be an independent factor in an overall FMS model which included locomotor (r = .88, object control (r = .76 and stability skills (r = .81.This study provides a rationale for the inclusion of stability skills in FMS assessment. The stability skills could be used alongside other FMS assessment tools to provide a holistic assessment of children's fundamental movement skills.

The Effects of Performance-Based Assessment Criteria on Student Performance and Self-Assessment Skills

Science.gov (United States)

Fastre, Greet Mia Jos; van der Klink, Marcel R.; van Merrienboer, Jeroen J. G.

2010-01-01

This study investigated the effect of performance-based versus competence-based assessment criteria on task performance and self-assessment skills among 39 novice secondary vocational education students in the domain of nursing and care. In a performance-based assessment group students are provided with a preset list of performance-based…

Objective Structured Assessment of Technical Skills (OSATS) evaluation of hysteroscopy training: a prospective study.

Science.gov (United States)

Alici, Ferizan; Buerkle, Bernd; Tempfer, Clemens B

2014-07-01

To describe the performance curve of hysteroscopy-naïve probands repeatedly working through a surgery algorithm on a hysteroscopy trainer. We prospectively recruited medical students to a 30min demonstration session teaching a standardized surgery algorithm. Subjects subsequently performed three training courses immediately after training (T1) and after 24h (T2) and 48h (T3). Skills were recorded with a 20-item Objective Structured Assessment of Technical Skills (OSATS) at T1, T2, and T3. The presence of a sustained OSATS score improvement from T1 to T3 was the primary outcome. Performance time (PT) and self assessment (SA) were secondary outcomes. Statistics were performed using paired T-test and multiple linear regression analysis. 92 subjects were included. OSATS scores significantly improved over time from T1 to T2 (15.21±1.95 vs. 16.02±2.06, respectively; ptraining courses on a hysteroscopy trainer. Serial hysteroscopy trainings may be helpful for teaching hysteroscopy skills. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

Motivational Interviewing Skills in Health Care Encounters (MISHCE): Development and psychometric testing of an assessment tool.

Science.gov (United States)

Petrova, Tatjana; Kavookjian, Jan; Madson, Michael B; Dagley, John; Shannon, David; McDonough, Sharon K

2015-01-01

Motivational interviewing (MI) has demonstrated a significant impact as an intervention strategy for addiction management, change in lifestyle behaviors, and adherence to prescribed medication and other treatments. Key elements to studying MI include training in MI of professionals who will use it, assessment of skills acquisition in trainees, and the use of a validated skills assessment tool. The purpose of this research project was to develop a psychometrically valid and reliable tool that has been designed to assess MI skills competence in health care provider trainees. The goal was to develop an assessment tool that would evaluate the acquisition and use of specific MI skills and principles, as well as the quality of the patient-provider therapeutic alliance in brief health care encounters. To address this purpose, specific steps were followed, beginning with a literature review. This review contributed to the development of relevant conceptual and operational definitions, selecting a scaling technique and response format, and methods for analyzing validity and reliability. Internal consistency reliability was established on 88 video recorded interactions. The inter-rater and test-retest reliability were established using randomly selected 18 from the 88 interactions. The assessment tool Motivational Interviewing Skills for Health Care Encounters (MISHCE) and a manual for use of the tool were developed. Validity and reliability of MISHCE were examined. Face and content validity were supported with well-defined conceptual and operational definitions and feedback from an expert panel. Reliability was established through internal consistency, inter-rater reliability, and test-retest reliability. The overall internal consistency reliability (Cronbach's alpha) for all fifteen items was 0.75. MISHCE demonstrated good inter-rater reliability and good to excellent test-retest reliability. MISHCE assesses the health provider's level of knowledge and skills in brief

Implementing and Evaluating a National Certification Technical Skills Examination: The Colorectal Objective Structured Assessment of Technical Skill.

Science.gov (United States)

de Montbrun, Sandra; Roberts, Patricia L; Satterthwaite, Lisa; MacRae, Helen

2016-07-01

To implement the Colorectal Objective Structured Assessment of Technical skill (COSATS) into American Board of Colon and Rectal Surgery (ABCRS) certification and build evidence of validity for the interpretation of the scores of this high stakes assessment tool. Currently, technical skill assessment is not a formal component of board certification. With the technical demands of surgical specialties, documenting competence in technical skill at the time of certification with a valid tool is ideal. In September 2014, the COSATS was a mandatory component of ABCRS certification. Seventy candidates took the examination, with their performance evaluated by expert colorectal surgeons using a task-specific checklist, global rating scale, and overall performance scale. Passing scores were set and compared using 2 standard setting methodologies, using a compensatory and conjunctive model. Inter-rater reliability and the reliability of the pass/fail decision were calculated using Cronbach alpha and Subkoviak methodology, respectively. Overall COSATS scores and pass/fail status were compared with results on the ABCRS oral examination. The pass rate ranged from 85.7% to 90%. Inter-rater reliability (0.85) and reliability of the pass/fail decision (0.87 and 0.84) were high. A low positive correlation (r= 0.25) was seen between the COSATS and oral examination. All individuals who failed the COSATS passed the ABCRS oral examination. COSATS is the first technical skill examination used in national surgical board certification. This study suggests that the current certification process may be failing to identify individuals who have demonstrated technical deficiencies on this standardized assessment tool.

The assessment of surgical skills as a complement to the training method. Revision.

Science.gov (United States)

Sánchez-Fernández, J; Bachiller-Burgos, J; Serrano-Pascual, Á; Cózar-Olmo, J M; Díaz-Güemes Martín-Portugués, I; Pérez-Duarte, F J; Hernández-Hurtado, L; Álvarez-Ossorio, J L; Sánchez-Margallo, F M

2016-01-01

The acquisition and improvement of surgical skills constitute a fundamental element in the training of any practitioner. At present, however, the assessment of these skills is a scarcely developed area of research. The aim of this study was to analyse the peculiarities of the various assessment systems and establish the minimum criteria that a skills and knowledge assessment system should meet as a method for assessing surgical skills in urological surgery. Scientific literature review aimed at the various currently available assessment systems for skills and competencies (technical and nontechnical), with a special focus on the systematic reviews and prospective studies. After conducting the review, we found that the various assessment systems for surgical competence have, in our opinion, a number of shortcomings. There is a certain degree of subjectivity in the assessment of surgeons by the evaluators. The assessment of nontechnical competencies is not formally recorded. There is no description of a follow-up assessment or any basic parameters associated with healthcare quality. There is no registration of associated competencies associated with the various surgical techniques. There is also no ranking of these competencies and the specific peculiarities for their application. We believe that the development of a new assessment system for surgical competencies (technical and nontechnical) aimed at assessing urologists in the various surgical techniques is necessary. To this end, our team has worked on developing the Evaluation System for Surgical Competencies on Laparoscopy, which is based on the definition, ranking and assessment of competencies demonstrated by surgeons. Copyright © 2015 AEU. Publicado por Elsevier España, S.L.U. All rights reserved.

Assessing problem-solving skills in construction education with the virtual construction simulator

Science.gov (United States)

Castronovo, Fadi

The ability to solve complex problems is an essential skill that a construction and project manager must possess when entering the architectural, engineering, and construction industry. Such ability requires a mixture of problem-solving skills, ranging from lower to higher order thinking skills, composed of cognitive and metacognitive processes. These skills include the ability to develop and evaluate construction plans and manage the execution of such plans. However, in a typical construction program, introducing students to such complex problems can be a challenge, and most commonly the learner is presented with only part of a complex problem. To support this challenge, the traditional methodology of delivering design, engineering, and construction instruction has been going through a technological revolution, due to the rise of computer-based technology. For example, in construction classrooms, and other disciplines, simulations and educational games are being utilized to support the development of problem-solving skills. Previous engineering education research has illustrated the high potential that simulations and educational games have in engaging in lower and higher order thinking skills. Such research illustrated their capacity to support the development of problem-solving skills. This research presents evidence supporting the theory that educational simulation games can help with the learning and retention of transferable problem-solving skills, which are necessary to solve complex construction problems. The educational simulation game employed in this study is the Virtual Construction Simulator (VCS). The VCS is a game developed to provide students in an engaging learning activity that simulates the planning and managing phases of a construction project. Assessment of the third iteration of the VCS(3) game has shown pedagogical value in promoting students' motivation and a basic understanding of construction concepts. To further evaluate the benefits on

The effects of performance-based assessment criteria on student performance and self-assessment skills

NARCIS (Netherlands)

Fastré, Greet; Van der Klink, Marcel; Van Merriënboer, Jeroen

2010-01-01

Fastré, G. M. J., Van der Klink, M. R., & Van Merriënboer, J. J. G. (2010). The effects of performance-based assessment criteria on student performance and self-assessment skills. Advances in Health Science Education, 15(4), 517-532.

Innovative integrative bedside teaching model improves tutors' self-assessments of teaching skills and attitudes.

Science.gov (United States)

Gat, Itai; Pessach-Gelblum, Liat; Givati, Gili; Haim, Nadav; Paluch-Shimon, Shani; Unterman, Avraham; Bar-Shavit, Yochay; Grabler, Galit; Sagi, Doron; Achiron, Anat; Ziv, Amitai

2016-01-01

Patient bedside is the ideal setting for teaching physical examination, medical interviewing, and interpersonal skills. Herein we describe a novel model for bedside teaching (BST) practiced during tutor training workshop and its resulting effect on practitioners' self assessment of teaching skills and perceptions. One-day tutor training workshop included theoretical knowledge supplementation regarding tutors' roles as well as implementing practical tools for clinical education, mainly BST model. The model, which emphasizes simultaneous clinical and communication teaching in a stepwise approach, was practiced by consecutive simulations with a gradual escalation of difficulty and adjusted instruction approaches. Pre- and post-workshop-adjusted questionnaires using a Likert scale of 1 to 4 were completed by participants and compared. Analysis was based on 25 out of 48 participants who completed both questionnaires. Significantly improved teaching skills were demonstrated upon workshop completion (mean 3.3, SD 0.5) compared with pre-training (mean 2.6, SD 0.6; pteaching skills in this challenging environment.

Psychometric properties of the Global Operative Assessment of Laparoscopic Skills (GOALS) using item response theory.

Science.gov (United States)

Watanabe, Yusuke; Madani, Amin; Ito, Yoichi M; Bilgic, Elif; McKendy, Katherine M; Feldman, Liane S; Fried, Gerald M; Vassiliou, Melina C

2017-02-01

The extent to which each item assessed using the Global Operative Assessment of Laparoscopic Skills (GOALS) contributes to the total score remains unknown. The purpose of this study was to evaluate the level of difficulty and discriminative ability of each of the 5 GOALS items using item response theory (IRT). A total of 396 GOALS assessments for a variety of laparoscopic procedures over a 12-year time period were included. Threshold parameters of item difficulty and discrimination power were estimated for each item using IRT. The higher slope parameters seen with "bimanual dexterity" and "efficiency" are indicative of greater discriminative ability than "depth perception", "tissue handling", and "autonomy". IRT psychometric analysis indicates that the 5 GOALS items do not demonstrate uniform difficulty and discriminative power, suggesting that they should not be scored equally. "Bimanual dexterity" and "efficiency" seem to have stronger discrimination. Weighted scores based on these findings could improve the accuracy of assessing individual laparoscopic skills. Copyright © 2016 Elsevier Inc. All rights reserved.

Toolbox of assessment tools of technical skills in otolaryngology-head and neck surgery: A systematic review.

Science.gov (United States)

Labbé, Mathilde; Young, Meredith; Nguyen, Lily H P

2017-10-08

To support the development of programs of assessment of technical skills in the operating room (OR), we systematically reviewed the literature to identify assessment tools specific to otolaryngology-head and neck surgery (OTL-HNS) core procedures and summarized their characteristics. We systematically searched Embase, MEDLINE, PubMed, and Cochrane to identify and report on assessment tools that can be used to assess residents' technical surgical skills in the operating room for OTL-HNS core procedures. Of the 736 unique titles retrieved, 16 articles met inclusion criteria, covering 11 different procedures (in otology, rhinology, laryngology, head and neck, and general otolaryngology). The tools were composed of a task-specific checklist and/or global rating scale and were developed in the OR, on human cadavers, or in a simulation setting. Our study reports on published tools for assessing technical skills for OTL-HNS residents during core procedures conducted in the OR. These assessment tools could facilitate the provision of timely feedback to trainees including specific goals for improvement. However, the paucity of publications suggests little agreement on how to best perform work-based direct-observation assessment for core surgical procedures in OTL-HNS. The sparsity of tools specific to OTL-HNS may become a barrier to a fluid transition to competency-based medical education. Laryngoscope, 2017. © 2017 The American Laryngological, Rhinological and Otological Society, Inc.

Validity evidence as a key marker of quality of technical skill assessment in OTL-HNS.

Science.gov (United States)

Labbé, Mathilde; Young, Meredith; Nguyen, Lily H P

2018-01-13

Quality monitoring of assessment practices should be a priority in all residency programs. Validity evidence is one of the main hallmarks of assessment quality and should be collected to support the interpretation and use of assessment data. Our objective was to identify, synthesize, and present the validity evidence reported supporting different technical skill assessment tools in otolaryngology-head and neck surgery (OTL-HNS). We performed a secondary analysis of data generated through a systematic review of all published tools for assessing technical skills in OTL-HNS (n = 16). For each tool, we coded validity evidence according to the five types of evidence described by the American Educational Research Association's interpretation of Messick's validity framework. Descriptive statistical analyses were conducted. All 16 tools included in our analysis were supported by internal structure and relationship to variables validity evidence. Eleven articles presented evidence supporting content. Response process was discussed only in one article, and no study reported on evidence exploring consequences. We present the validity evidence reported for 16 rater-based tools that could be used for work-based assessment of OTL-HNS residents in the operating room. The articles included in our review were consistently deficient in evidence for response process and consequences. Rater-based assessment tools that support high-stakes decisions that impact the learner and programs should include several sources of validity evidence. Thus, use of any assessment should be done with careful consideration of the context-specific validity evidence supporting score interpretation, and we encourage deliberate continual assessment quality-monitoring. NA. Laryngoscope, 2018. © 2018 The American Laryngological, Rhinological and Otological Society, Inc.

Assessing family planning service-delivery skills in Kenya.

Science.gov (United States)

Valadez, J J; Transgrud, R; Mbugua, M; Smith, T

1997-06-01

This report demonstrates the use of Lot Quality Assurance Sampling (LQAS) to evaluate the technical competence of two cohorts of family planning service providers in Kenya trained with a new curriculum. One cohort had just finished training within two months of the study. The other cohort was the first group trained with the new curriculum about one year before the study. LQAS was adapted from industrial and other public health applications to assess both the individual competence of 30 service providers and the competence of each cohort. Results show that Cohorts One and Two did not differ markedly in the number of tasks needing improvement. However, both cohorts exhibited more tasks needing improvement in counseling skills as compared with physical examination skills or with all other skills. Care-givers who were not currently providing services accounted for most service-delivery problems. This result suggests that providers' use of their skills explains their ability to retain service-delivery skills learned in training to a greater degree than does the amount of time elapsed since they were trained. LQAS proved to be a rapid, easy-to-use empirical method for management decisionmaking for improvement of a family planning training curriculum and services.

Assessment of email communication skills of rheumatology fellows: a pilot study

Science.gov (United States)

Dhuper, Sonal; Siva, Chokkalingam; Fresen, John L; Petruc, Marius; Velázquez, Celso R

2010-01-01

Physician–patient email communication is gaining popularity. However, a formal assessment of physicians' email communication skills has not been described. We hypothesized that the email communication skills of rheumatology fellows can be measured in an objective structured clinical examination (OSCE) setting using a novel email content analysis instrument which has 18 items. During an OSCE, we asked 50 rheumatology fellows to respond to a simulated patient email. The content of the responses was assessed using our instrument. The majority of rheumatology fellows wrote appropriate responses scoring a mean (±SD) of 10.6 (±2.6) points (maximum score 18), with high inter-rater reliability (0.86). Most fellows were concise (74%) and courteous (68%) but not formal (22%). Ninety-two percent of fellows acknowledged that the patient's condition required urgent medical attention, but only 30% took active measures to contact the patient. No one encrypted their messages. The objective assessment of email communication skills is possible using simulated emails in an OSCE setting. The variable email communication scores and incidental patient safety gaps identified, suggest a need for further training and defined proficiency standards for physicians' email communication skills. PMID:20962134

Validation of Modified Soft Skills Assessment Instrument (MOSSAI) for Use in Nigeria

Science.gov (United States)

Aworanti, O. A.; Taiwo, M. B.; Iluobe, O. I.

2015-01-01

Currently, it has become an accepted norm nearly all over the globe to teach and assess soft skills. However, in Nigeria, it is an emerging area of interest that needs to be addressed squarely. In the light of the fore-going, this study validated a modified version of Measuring and Assessment Soft Skills (MASS) (an instrument developed and used by…

Rubric Assessment on Science and Creative Thinking Skills of Students

Science.gov (United States)

Ratnasusanti, H.; Ana, A.; Nurafiati, P.; Umusyaadah, L.

2018-02-01

The result of the monitoring and evaluation of the latest Indonesian curriculum (the 2013 curriculum) implementation at junior high school level year of 2014 showed that one of the difficult things that learners had in implementation 2013 curriculum is doing the result. The characteristic of applying the 2013 curriculum is to emphasize the modern pedagogic dimension of learning, which is using scientific approach, which requires learners to have highlevel thinking skills, one of which is creative thinking skills. The aims of this research is to implement performance assessment in measuring the creative thinking of junior high school students on subject Prakarya. The form of the main performance assessment is the task and assessment criteria. The experimental method that been used is the Quasi Experiment with Non-Equivalent Design Group Research. Population in this study is the students of VIII class of junior high school in Bandung, Indonesia which consists of six classes. And two classes are selected for the sample from that six classes and VIII A class were chosen, while VIII F class has been chosen as control class. The result of this research showed that the rubics of performance assessment can be measure or identify the creative thinking skill, its prove by the result of pre-test dan post-test are more dominant. In material of identification student’s creative thinking skills are reached an average 85 compare 79 with the control class. while in the presentation the experimental class got an average of 85 bigger than the control class which only reached 79.

Patients' assessment of professionalism and communication skills of medical graduates.

Science.gov (United States)

Abadel, Fatima T; Hattab, Abdulla S

2014-02-11

Professionalism and communication skills constitute important components of the integral formation of physicians which has repercussion on the quality of health care and medical education. The objective of this study was to assess medical graduates' professionalism and communication skills from the patients' perspective and to examine its association with patients' socio-demographic variables. This is a hospital based cross-sectional study. It involved 315 patients and 105 medical graduates selected by convenient sampling method. A modified and validated version of the American Board of Internal Medicine's (ABIM) Patient Assessment survey questionnaire was used for data collection through a face to face interview. Data processing and analysis were performed using the Statistical Package for Social Science (SPSS) 16.0. Mean, frequency distribution, and percentage of the variables were calculated. A non-parametric Kruskal Wallis test was applied to verify whether the patients' assessment was influenced by variables such as age, gender, education, at a level of significance, p ≤ 0.05. Female patients constituted 46% of the sample, whereas males constituted 54%. The mean age was 36 ± 16. Patients' scoring of the graduate's skills ranged from 3.29 to 3.83 with a mean of 3.64 on a five-point Likert scale. Items assessing the "patient involvement in decision-making" were assigned the minimum mean values, while items dealing with "establishing adequate communication with patient" assigned the maximum mean values. Patients, who were older than 45 years, gave higher scores than younger ones (p communication skills at a good level. Patients' age and educational level were significantly associated with the rating level.

Social Validation of the New England Center for Children-Core Skills Assessment

Science.gov (United States)

Dickson, Chata A.; MacDonald, Rebecca P. F.; Mansfield, Renee; Guilhardi, Paulo; Johnson, Cammarie; Ahearn, William H.

2014-01-01

We investigated the social validity of the NECC Core Skills Assessment (NECC-CSA) with parents and professionals as participants. The NECC-CSA is a measurement tool consisting of direct and indirect measures of skills important to all individuals with autism, across the lifespan. Participants (N = 245) were provided with a list of 66 skills, 47 of…

Self-assessment and goal-setting is associated with an improvement in interviewing skills.

Science.gov (United States)

Hanley, Kathleen; Zabar, Sondra; Charap, Joseph; Nicholson, Joseph; Disney, Lindsey; Kalet, Adina; Gillespie, Colleen

2014-01-01

Describe the relationship between medical students' self-assessment and goal-setting (SAGS) skills and development of interviewing skills during the first-year doctoring course. 157 first-year medical students completed three two-case standardized patient (SP) interviews. After each of the first two, students viewed videotapes of their interview, completed a SAGS worksheet, and reviewed a selected tape segment in a seminar. SAGS was categorized into good and poor quality and interviewing skills were rated by trained raters. SAGS improved over time (37% good week 1 vs. 61% good week 10). Baseline SAGS and interviewing skills were not associated. Initial SAGS quality was associated with change in interviewing skills - those with poor-quality SAGS demonstrated a decrease and those with good-quality SAGS demonstrated an increase in scores by 17 weeks (ANOVA F=4.16, p=0.024). For students whose SAGS skills were good at both week 1 and 10, interviewing skills declined in weeks 1-10 and then increased significantly at week 17. For those whose SAGS remained 'poor' in weeks 1-10, interviewing skills declined in weeks 10-17. In general, the quality of students' SAGS improved over time. Poor baseline SAGS skills and failure to improve were associated with a decrease in interviewing skills at 17 weeks. For students with better SAGS, interviewing skills increased at week 17. Improvement in SAGS skills was not associated with improved interviewing skills. Understanding structured self-assessment skills helps identify student characteristics that influence progressive mastery of communication skills and therefore may inform curriculum and remediation tailoring.

Can Perceptuo-Motor Skills Assessment Outcomes in Young Table Tennis Players (7–11 years) Predict Future Competition Participation and Performance? An Observational Prospective Study

Science.gov (United States)

2016-01-01

Forecasting future performance in youth table tennis players based on current performance is complex due to, among other things, differences between youth players in growth, development, maturity, context and table tennis experience. Talent development programmes might benefit from an assessment of underlying perceptuo-motor skills for table tennis, which is hypothesized to determine the players’ potential concerning the perceptuo-motor domain. The Dutch perceptuo-motor skills assessment intends to measure the perceptuo-motor potential for table tennis in youth players by assessing the underlying skills crucial for developing technical and tactical qualities. Untrained perceptuo-motor tasks are used as these are suggested to represent a player’s future potential better than specific sport skills themselves as the latter depend on exposure to the sport itself. This study evaluated the value of the perceptuo-motor skills assessment for a talent developmental programme by evaluating its predictive validity for competition participation and performance in 48 young table tennis players (7–11 years). Players were tested on their perceptuo-motor skills once during a regional talent day, and the subsequent competition results were recorded half-yearly over a period of 2.5 years. Logistic regression analysis showed that test scores did not predict future competition participation (p >0.05). Yet, the Generalized Estimating Equations analysis, including the test items ‘aiming at target’, ‘throwing a ball’, and ‘eye-hand coordination’ in the best fitting model, revealed that the outcomes of the perceptuo-motor skills assessment were significant predictors for future competition results (R2 = 51%). Since the test age influences the perceptuo-motor skills assessment’s outcome, another multivariable model was proposed including test age as a covariate (R2 = 53%). This evaluation demonstrates promising prospects for the perceptuo-motor skills assessment to be

Project Management: Essential Skill of Nurse Informaticists.

Science.gov (United States)

Sipes, Carolyn

2016-01-01

With the evolution of nursing informatics (NI), the list of skills has advanced from the original definition that included 21 competencies to 168 basic competencies identified in the TIGER-based Assessment of Nursing Informatics Competencies (TANIC) and 178 advanced skills in the Nursing Informatics Competency Assessment (NICA) L3/L4 developed by Chamberlain College of Nursing, Nursing Informatics Research Team (NIRT). Of these competencies, project management is one of the most important essentials identified since it impacts all areas of NI skills and provides an organizing framework for processes and projects including skills such as design, planning, implementation, follow-up and evaluation. Examples of job roles that specifically require project management skills as an essential part of the NI functions include management, administration, leadership, faculty, graduate level master's and doctorate practicum courses. But first, better understanding of the NI essential skills is vital before adequate education and training programs can be developed.

Goodness of Fit of Skills Assessment Approaches: Insights from Patterns of Real vs. Synthetic Data Sets

Science.gov (United States)

Beheshti, Behzad; Desmarais, Michel C.

2015-01-01

This study investigates the issue of the goodness of fit of different skills assessment models using both synthetic and real data. Synthetic data is generated from the different skills assessment models. The results show wide differences of performances between the skills assessment models over synthetic data sets. The set of relative performances…

The Promotion of Critical Thinking Skills in School-Based Assessment (SBA)

Science.gov (United States)

Kamarulzaman, Wirawani; Kamarulzaman, Wirawahida

2016-01-01

The new curriculum introduced in the Malaysian primary students; the Primary School Standard Curriculum (Kurikulum Standard Sekolah Rendah-KSSR) together with the school-based assessment (SBA) is a step taken by the Malaysian government to encourage thinking skills to students, specifically critical thinking skills. The study explores teachers'…

Middle School Teachers' Strategies for Including Overweight Students in Skill and Fitness Instruction

Science.gov (United States)

Rukavina, Paul B.; Doolittle, Sarah; Li, Weidong; Manson, Mara; Beale, Angela

2015-01-01

As part of a larger study, this paper describes teachers' perspectives and strategies on including overweight and obese students (OWS) in instruction related to motor skill/game play and fitness development in physical education. Using the Social Ecological Constraints framework, a qualitative multicase study was conducted using multiple in-depth…

The use of physical assessment skills by registered nurses in Australia: issues for nursing education.

Science.gov (United States)

Birks, Melanie; Cant, Robyn; James, Ainsley; Chung, Catherine; Davis, Jenny

2013-01-01

The purpose of pre-service nursing education programs is to prepare competent graduates who are able to function as safe, professional registered nurses. An extensive element of these programs is the teaching of physical assessment skills, with most programs educating students to perform over 120 such skills. Previous research from North America suggests that the majority of skills taught to nurses in their pre-service programs are not used in practice. As part of a larger study, an online survey was used to explore use of 121 physical assessment skills by Australian nurses. Recruitment occurred via mailed invitation to members of the Australian Nursing Federation. Data were extracted from 1220 completed questionnaires returned by nurses who were mostly employed in New South Wales, were female and experienced nurses. Respondents indicated that they used only 34% of skills routinely. Results reinforce evidence found in the literature that many of the skills taught to nurses are either not used at all (35.5%) or are used rarely (31%). These findings have implications for the teaching of physical assessment skills in pre-service nursing programs, and raise questions about the value of extensive skills teaching in the context of contemporary health care. Further research into barriers to the use of physical assessment skills in nursing and the need for comprehensive skills preparation for the generalist nurse is likely to offer some solutions to these questions.

Designing Standardized Patient Assessments to Measure SBIRT Skills

Science.gov (United States)

Wamsley, Maria A.; Julian, Katherine A.; O'Sullivan, Patricia; Satterfield, Jason M.; Satre, Derek D.; McCance-Katz, Elinore; Batki, Steven L.

2013-01-01

Objectives: Resident physicians report insufficient experience caring for patients with substance use disorders (SUDs). Resident training in Screening, Brief Intervention, and Referral to Treatment (SBIRT) has been recommended. We describe the development of a standardized patient (SP) assessment to measure SBIRT skills, resident perceptions of…

Reliable assessment of general surgeons' non-technical skills based on video-recordings of patient simulated scenarios

DEFF Research Database (Denmark)

Spanager, Lene; Beier-Holgersen, Randi; Dieckmann, Peter

2013-01-01

Nontechnical skills are essential for safe and efficient surgery. The aim of this study was to evaluate the reliability of an assessment tool for surgeons' nontechnical skills, Non-Technical Skills for Surgeons dk (NOTSSdk), and the effect of rater training.......Nontechnical skills are essential for safe and efficient surgery. The aim of this study was to evaluate the reliability of an assessment tool for surgeons' nontechnical skills, Non-Technical Skills for Surgeons dk (NOTSSdk), and the effect of rater training....

Reliability of the ECHOWS Tool for Assessment of Patient Interviewing Skills.

Science.gov (United States)

Boissonnault, Jill S; Evans, Kerrie; Tuttle, Neil; Hetzel, Scott J; Boissonnault, William G

2016-04-01

History taking is an important component of patient/client management. Assessment of student history-taking competency can be achieved via a standardized tool. The ECHOWS tool has been shown to be valid with modest intrarater reliability in a previous study but did not demonstrate sufficient power to definitively prove its stability. The purposes of this study were: (1) to assess the reliability of the ECHOWS tool for student assessment of patient interviewing skills and (2) to determine whether the tool discerns between novice and experienced skill levels. A reliability and construct validity assessment was conducted. Three faculty members from the United States and Australia scored videotaped histories from standardized patients taken by students and experienced clinicians from each of these countries. The tapes were scored twice, 3 to 6 weeks apart. Reliability was assessed using interclass correlation coefficients (ICCs) and repeated measures. Analysis of variance models assessed the ability of the tool to discern between novice and experienced skill levels. The ECHOWS tool showed excellent intrarater reliability (ICC [3,1]=.74-.89) and good interrater reliability (ICC [2,1]=.55) as a whole. The summary of performance (S) section showed poor interrater reliability (ICC [2,1]=.27). There was no statistical difference in performance on the tool between novice and experienced clinicians. A possible ceiling effect may occur when standardized patients are not coached to provide complex and obtuse responses to interviewer questions. Variation in familiarity with the ECHOWS tool and in use of the online training may have influenced scoring of the S section. The ECHOWS tool demonstrates excellent intrarater reliability and moderate interrater reliability. Sufficient training with the tool prior to student assessment is recommended. The S section must evolve in order to provide a more discerning measure of interviewing skills. © 2016 American Physical Therapy

Drawing students' attention to relevant assessment criteria: effects on self-assessment skills and performance

NARCIS (Netherlands)

Fastré, Greet; Van der Klink, Marcel; Sluijsmans, Dominique; Van Merriënboer, Jeroen

2012-01-01

Fastré, G. M. J., Van der Klink, M. R., Sluijsmans, D., & Van Merriënboer, J. J. G. (2012). Drawing students’ attention to relevant assessment criteria: effects on self-assessment skills and performance. Journal of Vocational Education & Training, 64(2), 185-198. doi:10.1080/13636820.2011.630537

Taxonomy of Trauma Leadership Skills: A Framework for Leadership Training and Assessment.

Science.gov (United States)

Leenstra, Nico F; Jung, Oliver C; Johnson, Addie; Wendt, Klaus W; Tulleken, Jaap E

2016-02-01

Good leadership is essential for optimal trauma team performance, and targeted training of leadership skills is necessary to achieve such leadership proficiency. To address the need for a taxonomy of leadership skills that specifies the skill components to be learned and the behaviors by which they can be assessed across the five phases of trauma care, the authors developed the Taxonomy of Trauma Leadership Skills (TTLS). Critical incident interviews were conducted with trauma team leaders and members from different specialties-emergency physicians, trauma surgeons, anesthesiologists, and emergency ward nurses-at three teaching hospitals in the Netherlands during January-June 2013. Data were iteratively analyzed for examples of excellent leadership skills at each phase of trauma care. Using the grounded theory approach, elements of excellent leadership skills were identified and classified. Elements and behavioral markers were sorted and categorized using multiple raters. In a two-round verification process in late 2013, the taxonomy was reviewed and rated by trauma team leaders and members from the multiple specialties for its coverage of essential items. Data were gathered from 28 interviews and 14 raters. The TTLS details 5 skill categories (information coordination, decision making, action coordination, communication management, and coaching and team development) and 37 skill elements. The skill elements are captured by 67 behavioral markers. The three-level taxonomy is presented according to five phases of trauma care. The TTLS provides a framework for teaching, learning, and assessing team leadership skills in trauma care and other complex, acute care situations.

Assessing Hourly Precipitation Forecast Skill with the Fractions Skill Score

Science.gov (United States)

Zhao, Bin; Zhang, Bo

2018-02-01

Statistical methods for category (yes/no) forecasts, such as the Threat Score, are typically used in the verification of precipitation forecasts. However, these standard methods are affected by the so-called "double-penalty" problem caused by slight displacements in either space or time with respect to the observations. Spatial techniques have recently been developed to help solve this problem. The fractions skill score (FSS), a neighborhood spatial verification method, directly compares the fractional coverage of events in windows surrounding the observations and forecasts. We applied the FSS to hourly precipitation verification by taking hourly forecast products from the GRAPES (Global/Regional Assimilation Prediction System) regional model and quantitative precipitation estimation products from the National Meteorological Information Center of China during July and August 2016, and investigated the difference between these results and those obtained with the traditional category score. We found that the model spin-up period affected the assessment of stability. Systematic errors had an insignificant role in the fraction Brier score and could be ignored. The dispersion of observations followed a diurnal cycle and the standard deviation of the forecast had a similar pattern to the reference maximum of the fraction Brier score. The coefficient of the forecasts and the observations is similar to the FSS; that is, the FSS may be a useful index that can be used to indicate correlation. Compared with the traditional skill score, the FSS has obvious advantages in distinguishing differences in precipitation time series, especially in the assessment of heavy rainfall.

Assessment of Robotic Console Skills (ARCS): construct validity of a novel global rating scale for technical skills in robotically assisted surgery.

Science.gov (United States)

Liu, May; Purohit, Shreya; Mazanetz, Joshua; Allen, Whitney; Kreaden, Usha S; Curet, Myriam

2018-01-01

Skill assessment during robotically assisted surgery remains challenging. While the popularity of the Global Evaluative Assessment of Robotics Skills (GEARS) has grown, its lack of discrimination between independent console skills limits its usefulness. The purpose of this study was to evaluate construct validity and interrater reliability of a novel assessment designed to overcome this limitation. We created the Assessment of Robotic Console Skills (ARCS), a global rating scale with six console skill domains. Fifteen volunteers who were console surgeons for 0 ("novice"), 1-100 ("intermediate"), or >100 ("experienced") robotically assisted procedures performed three standardized tasks. Three blinded raters scored the task videos using ARCS, with a 5-point Likert scale for each skill domain. Scores were analyzed for evidence of construct validity and interrater reliability. Group demographics were indistinguishable except for the number of robotically assisted procedures performed (p = 0.001). The mean scores of experienced subjects exceeded those of novices in dexterity (3.8 > 1.4, p  1.8, p  2.2, p  1.9, p = 0.001), and force sensitivity (4.3 > 2.6, p  1.4, p = 0.002), field of view (2.8 > 1.8, p = 0.021), instrument visualization (3.2 > 2.2, p = 0.045), manipulator workspace (3.1 > 1.9, p = 0.004), and force sensitivity (3.7 > 2.6, p = 0.033). The mean scores of experienced subjects exceeded those of intermediates in dexterity (3.8 > 2.8, p = 0.003), field of view (4.1 > 2.8, p  3.2, p = 0.044). Rater agreement in each domain demonstrated statistically significant concordance (p skills plateau faster than others. Therefore, ARCS may be more useful than GEARS to evaluate distinct console skills. Future studies will examine why some domains did not adequately differentiate between subjects and applications for intraoperative use.

Revisiting the psychometric properties of a revised Danish version of the McGill ingestive skills assessment

DEFF Research Database (Denmark)

Hansen, Tina; Kristiansen, Dorte Melgaard

2017-01-01

Background: During a longstanding validation process of the Danish version of the McGill Ingestive Skills Assessment (MISA2-DK) for measuring mealtime performance in dysphagic clients, extensive revisions have been undertaken. Therefore, this study aimed to determine the psychometric properties...... of this revised version. Methods: In a cross-sectional study, 328 adults referred to occupational therapy for swallowing evaluation were included. MISA2-DK with 36 items distributed into four subscales (positioning for meals, self-feeding skills, liquid ingestion, and solid ingestion) was administered...

Peer Assessment of Soft Skills and Hard Skills

Science.gov (United States)

Zhang, Aimao

2012-01-01

Both the information technology (IT) industry and the Accreditation Board for Engineering and Technology (ABET) demand soft-skill training in higher education and require IT graduates to demonstrate competence in interpersonal communication, teamwork, and conflict management. Group projects provide teamwork environment for soft-skill training, but…

Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator

DEFF Research Database (Denmark)

Larsen, C R; Grantcharov, Teodor; Aggarwal, R

2006-01-01

Safe realistic training and unbiased quantitative assessment of technical skills are required for laparoscopy. Virtual reality (VR) simulators may be useful tools for training and assessing basic and advanced surgical skills and procedures. This study aimed to investigate the construct validity...

Assessing competence in communication and interpersonal skills: the Kalamazoo II report.

Science.gov (United States)

Duffy, F Daniel; Gordon, Geoffrey H; Whelan, Gerald; Cole-Kelly, Kathy; Frankel, Richard; Buffone, Natalie; Lofton, Stephanie; Wallace, MaryAnne; Goode, Leslie; Langdon, Lynn

2004-06-01

Accreditation of residency programs and certification of physicians requires assessment of competence in communication and interpersonal skills. Residency and continuing medical education program directors seek ways to teach and evaluate these competencies. This report summarizes the methods and tools used by educators, evaluators, and researchers in the field of physician-patient communication as determined by the participants in the "Kalamazoo II" conference held in April 2002. Communication and interpersonal skills form an integrated competence with two distinct parts. Communication skills are the performance of specific tasks and behaviors such as obtaining a medical history, explaining a diagnosis and prognosis, giving therapeutic instructions, and counseling. Interpersonal skills are inherently relational and process oriented; they are the effect communication has on another person such as relieving anxiety or establishing a trusting relationship. This report reviews three methods for assessment of communication and interpersonal skills: (1) checklists of observed behaviors during interactions with real or simulated patients; (2) surveys of patients' experience in clinical interactions; and (3) examinations using oral, essay, or multiple-choice response questions. These methods are incorporated into educational programs to assess learning needs, create learning opportunities, or guide feedback for learning. The same assessment tools, when administered in a standardized way, rated by an evaluator other than the teacher, and using a predetermined passing score, become a summative evaluation. The report summarizes the experience of using these methods in a variety of educational and evaluation programs and presents an extensive bibliography of literature on the topic. Professional conversation between patients and doctors shapes diagnosis, initiates therapy, and establishes a caring relationship. The degree to which these activities are successful depends, in

Comparison of Parent Report and Direct Assessment of Child Skills in Toddlers.

Science.gov (United States)

Miller, Lauren E; Perkins, Kayla A; Dai, Yael G; Fein, Deborah A

2017-09-01

There are unique challenges associated with measuring development in early childhood. Two primary sources of information are used: parent report and direct assessment. Each approach has strengths and weaknesses, particularly when used to identify and diagnose developmental delays. The present study aimed to evaluate consistency between parent report and direct assessment of child skills in toddlers with and without Autism Spectrum Disorder (ASD) across receptive language, expressive language, and fine motor domains. 109 children were evaluated at an average age of two years; data on child skills were collected via parent report and direct assessment. Children were classified into three groups (i.e., ASD, Other Developmental Disorder, or Typical Development) based on DSM-IV-TR diagnosis. Mixed design ANOVAs, with data source as a within subjects factor and diagnostic group as a between subjects factor, were used to assess agreement. Chi square tests of agreement were then used to examine correspondence at the item level. Results suggested that parent report of language and fine motor skills did not significantly differ from direct assessment, and this finding held across diagnostic groups. Item level analyses revealed that, in most cases of significant disagreement, parents reported a skill as present, but it was not seen on direct testing. Results indicate that parents are generally reliable reporters of child language and fine motor abilities in toddlerhood, even when their children have developmental disorders such as ASD. However, the fullest picture may be obtained by using both parent report and direct assessment.

Patient Assessment Skills Pertinent to Practicing Pharmacists

Directory of Open Access Journals (Sweden)

Jennifer A. Wilson

2015-01-01

Full Text Available Objectives: To describe pharmacists’ opinions regarding which patient assessment skills are necessary to understand and/or perform to provide optimal patient care in pharmacy practice. Methods: An online questionnaire was distributed to pharmacists licensed in North Carolina (n=14,167, as identified by the Board of Pharmacy. The 80 patient assessment items in the questionnaire were derived from a course text book and faculty experience. Participants indicated whether they “need to understand and be able to perform”, “need to understand only”, or “not need to understand or be able to perform” each item in their current practice setting. Descriptive statistics were used to describe background demographics and perceived need for each item. Post-hoc chi-square analyses were performed to determine differences in need based on practice setting and Pharm.D. degree completion. Results: Of 1036 responses received, 770 were used in data analysis; incomplete questionnaires and non-practicing pharmacist responses were excluded. Fifty-nine percent of respondents held a Pharm.D. degree. Participants identified their practice site(s as inpatient (29%, outpatient (16%, community/retail (50%, long term care (5%, and other (8%. The top five patient assessment items respondents identified as important to understand and perform included automatic blood pressure measurement (63%, point of care testing (57%, manual blood pressure measurement (53%, heart rate measurement (52%, and peak flow meter use (47%. Post-hoc analyses showed a significant difference among those with a Pharm.D. versus those without for the response “need to understand and be able to perform” for 20 patient assessment items; a significant difference was also noted among practice settings for 29 items. Conclusions: The top items pharmacists identified they need to both understand and perform could be applied in various practice settings. The study results may guide which patient

Impact of teaching and assessment format on electrocardiogram interpretation skills.

Science.gov (United States)

Raupach, Tobias; Hanneforth, Nathalie; Anders, Sven; Pukrop, Tobias; Th J ten Cate, Olle; Harendza, Sigrid

2010-07-01

Interpretation of the electrocardiogram (ECG) is a core clinical skill that should be developed in undergraduate medical education. This study assessed whether small-group peer teaching is more effective than lectures in enhancing medical students' ECG interpretation skills. In addition, the impact of assessment format on study outcome was analysed. Two consecutive cohorts of Year 4 medical students (n=335) were randomised to receive either traditional ECG lectures or the same amount of small-group, near-peer teaching during a 6-week cardiorespiratory course. Before and after the course, written assessments of ECG interpretation skills were undertaken. Whereas this final assessment yielded a considerable amount of credit points for students in the first cohort, it was merely formative in nature for the second cohort. An unannounced retention test was applied 8 weeks after the end of the cardiovascular course. A significant advantage of near-peer teaching over lectures (effect size 0.33) was noted only in the second cohort, whereas, in the setting of a summative assessment, both teaching formats appeared to be equally effective. A summative instead of a formative assessment doubled the performance increase (Cohen's d 4.9 versus 2.4), mitigating any difference between teaching formats. Within the second cohort, the significant difference between the two teaching formats was maintained in the retention test (p=0.017). However, in both cohorts, a significant decrease in student performance was detected during the 8 weeks following the cardiovascular course. Assessment format appeared to be more powerful than choice of instructional method in enhancing student learning. The effect observed in the second cohort was masked by an overriding incentive generated by the summative assessment in the first cohort. This masking effect should be considered in studies assessing the effectiveness of different teaching methods.

Self-assessment and goal-setting is associated with an improvement in interviewing skills

Directory of Open Access Journals (Sweden)

Kathleen Hanley

2014-07-01

Full Text Available Purpose: Describe the relationship between medical students’ self-assessment and goal-setting (SAGS skills and development of interviewing skills during the first-year doctoring course. Method: 157 first-year medical students completed three two-case standardized patient (SP interviews. After each of the first two, students viewed videotapes of their interview, completed a SAGS worksheet, and reviewed a selected tape segment in a seminar. SAGS was categorized into good and poor quality and interviewing skills were rated by trained raters. Results: SAGS improved over time (37% good week 1 vs. 61% good week 10. Baseline SAGS and interviewing skills were not associated. Initial SAGS quality was associated with change in interviewing skills – those with poor-quality SAGS demonstrated a decrease and those with good-quality SAGS demonstrated an increase in scores by 17 weeks (ANOVA F=4.16, p=0.024. For students whose SAGS skills were good at both week 1 and 10, interviewing skills declined in weeks 1–10 and then increased significantly at week 17. For those whose SAGS remained ‘poor’ in weeks 1–10, interviewing skills declined in weeks 10–17. Conclusions: In general, the quality of students’ SAGS improved over time. Poor baseline SAGS skills and failure to improve were associated with a decrease in interviewing skills at 17 weeks. For students with better SAGS, interviewing skills increased at week 17. Improvement in SAGS skills was not associated with improved interviewing skills. Understanding structured self-assessment skills helps identify student characteristics that influence progressive mastery of communication skills and therefore may inform curriculum and remediation tailoring.

Self- and rater-assessed effectiveness of "thinking-aloud" and "regular" morning report to intensify young physicians' clinical skills.

Science.gov (United States)

Hsu, Hui-Chi; Lee, Fa-Yauh; Yang, Ying-Ying; Tsao, Yen-Po; Lee, Wen-Shin; Chuang, Chiao-Lin; Chang, Ching-Chih; Huang, Chia-Chang; Huang, Chin-Chou; Ho, Shung-Tai

2015-09-01

This study compared the effects of the "thinking aloud" (TA) morning report (MR), which is characterized by sequential and interactive case discussion by all participants, with "regular" MR for clinical skill training of young physicians. Between February 2011 and February 2014, young physicians [including postgraduate year-1 (PGY1) residents, interns, and clerks) from our hospital were sequentially enrolled and followed for 3 months. The self- and rater-assessed educational values of two MR models for building up clinical skills of young physicians were compared. The junior (intern and clerk) attendees had higher self-assessed educational values scores and reported post-training application frequency of skills trained by TA MR compared with the senior (PGY1 resident) attendees. Higher average and percentage of increased overall rater-assessed OSCE scores were noted among the regular MR senior attendees and TA MR junior attendees than in their corresponding control groups (regular MR junior attendees and TA MR senior attendees). Interestingly, regular MRs provided additional beneficial effects for establishing the "professionalism, consulting skills and organization efficiency" aspects of clinical skills of senior/junior attendees. Moreover, senior and junior attendees benefited the most by participating in seven sessions of regular MR and TA MR each month, respectively. TA MR effectively trains junior attendees in basic clinical skills, whereas regular MR enhances senior attendees' "work reports, professionalism, organizational efficiency, skills in dealing with controversial and professional issues." Undoubtedly, all elements of the two MR models should be integrated together to ensure patient safety and good discipline among young physicians. Copyright © 2015. Published by Elsevier Taiwan.

Including Performance Assessments in Accountability Systems: A Review of Scale-Up Efforts

Science.gov (United States)

Tung, Rosann

2010-01-01

The purpose of this literature and field review is to understand previous efforts at scaling up performance assessments for use across districts and states. Performance assessments benefit students and teachers by providing more opportunities for students to demonstrate their knowledge and complex skills, by providing teachers with better…

Development and validation of an instrument to assess and improve clinical consultation skills

OpenAIRE

Lefroy, JE; Gay, SP; Gibson, S; Williams, S; McKinley, RK

2011-01-01

Context: Development of medical students’ consultation skills with patients is at the core of the UK General Medical Council’s 'Tomorrow’s Doctors' guide (2009). Teaching and assessment of these skills must therefore be a core component of the medical undergraduate curriculum. The Calgary Cambridge guide to the medical interview and the Leicester Assessment Package (LAP) provide a foundation for teaching and assessment, but both have different strengths. Objective: To develop and validate a c...

Assessment of first-year veterinary students' communication skills using an objective structured clinical examination: the importance of context.

Science.gov (United States)

Hecker, Kent G; Adams, Cindy L; Coe, Jason B

2012-01-01

Communication skills are considered to be a core clinical skill in veterinary medicine and essential for practice success, including outcomes of care for patients and clients. While veterinary schools include communication skills training in their programs, there is minimal knowledge on how best to assess communication competence throughout the undergraduate program. The purpose of this study was to further our understanding of the reliability, utility, and suitability of a communication skills Objective Structured Clinical Examination (OSCE). Specifically we wanted to (1) identify the greatest source of variability (student, rater, station, and track) within a first-year, four station OSCE using exam scores and scores from videotape review by two trained raters, and (2) determine the effect of different stations on students' communication skills performance. Reliability of the scores from both the exam data and the two expert raters was 0.50 and 0.46 respectively, with the greatest amount of variance attributable to student by station. The percentage of variance due to raters in the exam data was 16.35%, whereas the percentage of variance for the two expert raters was 0%. These results have three important implications. First, the results reinforce the need for communication educators to emphasize that use of communication skills is moderated by the context of the clinical interaction. Second, by increasing rater training the amount of error in the scores due to raters can be reduced and inter-rater reliability increases. Third, the communication assessment method (in this case the OSCE checklist) should be built purposefully, taking into consideration the context of the case.

The Impact of Adaptive Complex Assessment on the HOT Skill Development of Students

Science.gov (United States)

Raiyn, Jamal; Tilchin, Oleg

2016-01-01

In this paper we propose a method for the adaptive complex assessment (ACA) of the higher-order thinking (HOT) skills needed by students for problem solving, and we examine the impact of the method on the development of HOT skills in a problem-based learning (PBL) environment. Complexity in the assessment is provided by initial, formative, and…

Cinemeducation: teaching family assessment skills using full-length movies.

Science.gov (United States)

Wilson, Astrid H; Blake, Barbara J; Taylor, Gloria A; Hannings, Glenda

2013-05-01

A thorough family assessment provides a foundation for the nursing process when working with families. Therefore, nurses, along with other health care providers must develop expertise in conducting family assessments to provide the best possible care within the community. This article describes an innovative educational strategy using movies to teach family assessment skills and puts forth recommendations for future research to provide evidence to support this teaching modality. © 2013 Wiley Periodicals, Inc.

Reliability and Validity of 3 Methods of Assessing Orthopedic Resident Skill in Shoulder Surgery.

Science.gov (United States)

Bernard, Johnathan A; Dattilo, Jonathan R; Srikumaran, Uma; Zikria, Bashir A; Jain, Amit; LaPorte, Dawn M

Traditional measures for evaluating resident surgical technical skills (e.g., case logs) assess operative volume but not level of surgical proficiency. Our goal was to compare the reliability and validity of 3 tools for measuring surgical skill among orthopedic residents when performing 3 open surgical approaches to the shoulder. A total of 23 residents at different stages of their surgical training were tested for technical skill pertaining to 3 shoulder surgical approaches using the following measures: Objective Structured Assessment of Technical Skills (OSATS) checklists, the Global Rating Scale (GRS), and a final pass/fail assessment determined by 3 upper extremity surgeons. Adverse events were recorded. The Cronbach α coefficient was used to assess reliability of the OSATS checklists and GRS scores. Interrater reliability was calculated with intraclass correlation coefficients. Correlations among OSATS checklist scores, GRS scores, and pass/fail assessment were calculated with Spearman ρ. Validity of OSATS checklists was determined using analysis of variance with postgraduate year (PGY) as a between-subjects factor. Significance was set at p shoulder approaches. Checklist scores showed superior interrater reliability compared with GRS and subjective pass/fail measurements. GRS scores were positively correlated across training years. The incidence of adverse events was significantly higher among PGY-1 and PGY-2 residents compared with more experienced residents. OSATS checklists are a valid and reliable assessment of technical skills across 3 surgical shoulder approaches. However, checklist scores do not measure quality of technique. Documenting adverse events is necessary to assess quality of technique and ultimate pass/fail status. Multiple methods of assessing surgical skill should be considered when evaluating orthopedic resident surgical performance. Copyright © 2016 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights

E-learning teaches attendings "how to" objectively assess pediatric urology trainees' surgery skills for orchiopexy.

Science.gov (United States)

Fernandez, Nicolas; Maizels, Max; Farhat, Walid; Smith, Edwin; Liu, Dennis; Chua, Michael; Bhanji, Yasin

2018-04-01

Established methods to train pediatric urology surgery by residency training programs require updating in response to administrative changes such as new, reduced trainee duty hours. Therefore, new objective methods must be developed to teach trainees. We approached this need by creating e-learning to teach attendings objective assessment of trainee skills using the Zwisch scale, an established assessment tool. The aim of this study was to identify whether or not e-learning is an appropriate platform for effective teaching of this assessment tool, by assessing inter-rater correlation of assessments made by the attendings after participation in the e-learning. Pediatric orchiopexy was used as the index case. An e-learning tool was created to teach attending surgeons objective assessment of trainees' surgical skills. First, e-learning content was created which showed the assessment method videotape of resident surgery done in the operating room. Next, attendings were enrolled to e-learn this method. Finally, the ability of enrollees to assess resident surgery skill performance was tested. Namely, test video was made showing a trainee performing inguinal orchiopexy. All enrollees viewed the same online videos. Assessments of surgical skills (Zwisch scale) were entered into an online survey. Data were analyzed by intercorrelation coefficient kappa analysis (strong correlation was ICC ≥ 0.7). A total of 11 attendings were enrolled. All accessed the online learning and then made assessments of surgical skills trainees showed on videotapes. The e-learning comprised three modules: 1. "Core concepts," in which users learned the assessment tool methods; 2. "Learn to assess," in which users learned how to assess by watching video clips, explaining the assessment method; and 3. "Test," in which users tested their skill at making assessments by watching video clips and then actively inputting their ratings of surgical and global skills as viewed in the video clips (Figure

Video observation of procedural skills for assessment of trabeculectomy performed by residents.

Science.gov (United States)

Hassanpour, Narges; Chen, Rebecca; Baikpour, Masoud; Moghimi, Sasan

2016-06-01

The efficacy and sufficiency of a healthcare system is directly related to the knowledge and skills of graduates working in the system. In this regard, many different assessment methods have been proposed to evaluate various skills of the learners. Video Observation of Procedural Skills (VOPS) is one newly-proposed method. In this study we aimed to compare the results of the VOPS method with the more commonly used Direct Observation of Procedural Skills (DOPS). In this prospective study conducted in 2012, all 10 ophthalmology residents of post graduate year 4 were selected for participation. Three months into training in the glaucoma ward, these residents performed trabeculectomy surgery on patients, and their procedural skills were assessed in real time by an expert via the DOPS method. All surgeries were also recorded and later evaluated via the VOPS method by an expert. Bland-Altman plot also was used to compare the two methods and calculating the mean and 95% limit of agreement. Residents have been done a mean of 14.9 ± 3.5 (range 10-20) independent trabeculectomy before the assessments. DOPS grade was positively associated with number of independent trabeculectomy during glaucoma rotation (β=0.227, p = 0.004). The intra-observer reproducibility of VOPS measurements was 0.847 (95% CI: 0.634, 0.961). The mean VOPS grade was significantly lower than the mean DOPS grade (8.4 vs. 8.9, p = 0.02). However, a good correlation was observed between the grades of VOPS and DOPS (r = 0.89, p = 0.001). Bland-Altman analysis demonstrated that all data points fell within the 95% limits of agreement (-1.46, 0.46). The present study showed that VOPS might be considered a feasible, valid, and reliable assessment method for procedural skills of medical students and residents that can be used as an alternative to the DOPS method. However, VOPS might underestimate DOPS in evaluating surgical skills of residents.

Students’ performance in the different clinical skills assessed in OSCE: what does it reveal?

Directory of Open Access Journals (Sweden)

Joong Hiong Sim

2015-02-01

Full Text Available Introduction: The purpose of this study was to compare students’ performance in the different clinical skills (CSs assessed in the objective structured clinical examination. Methods: Data for this study were obtained from final year medical students’ exit examination (n=185. Retrospective analysis of data was conducted using SPSS. Means for the six CSs assessed across the 16 stations were computed and compared. Results: Means for history taking, physical examination, communication skills, clinical reasoning skills (CRSs, procedural skills (PSs, and professionalism were 6.25±1.29, 6.39±1.36, 6.34±0.98, 5.86±0.99, 6.59±1.08, and 6.28±1.02, respectively. Repeated measures ANOVA showed there was a significant difference in the means of the six CSs assessed [F(2.980, 548.332=20.253, p<0.001]. Pairwise multiple comparisons revealed significant differences between the means of the eight pairs of CSs assessed, at p<0.05. Conclusions: CRSs appeared to be the weakest while PSs were the strongest, among the six CSs assessed. Students’ unsatisfactory performance in CRS needs to be addressed as CRS is one of the core competencies in medical education and a critical skill to be acquired by medical students before entering the workplace. Despite its challenges, students must learn the skills of clinical reasoning, while clinical teachers should facilitate the clinical reasoning process and guide students’ clinical reasoning development.

Haptic feedback can provide an objective assessment of arthroscopic skills.

Science.gov (United States)

Chami, George; Ward, James W; Phillips, Roger; Sherman, Kevin P

2008-04-01

The outcome of arthroscopic procedures is related to the surgeon's skills in arthroscopy. Currently, evaluation of such skills relies on direct observation by a surgeon trainer. This type of assessment, by its nature, is subjective and time-consuming. The aim of our study was to identify whether haptic information generated from arthroscopic tools could distinguish between skilled and less skilled surgeons. A standard arthroscopic probe was fitted with a force/torque sensor. The probe was used by five surgeons with different levels of experience in knee arthroscopy performing 11 different tasks in 10 standard knee arthroscopies. The force/torque data from the hand and tool interface were recorded and synchronized with a video recording of the procedure. The torque magnitude and patterns generated were analyzed and compared. A computerized system was used to analyze the force/torque signature based on general principles for quality of performance using such measures as economy in movement, time efficiency, and consistency in performance. The results showed a considerable correlation between three haptic parameters and the surgeon's experience, which could be used in an automated objective assessment system for arthroscopic surgery. Level II, diagnostic study. See the Guidelines for Authors for a complete description of levels of evidence.

OSCE vs. TEM: Different approaches to assess clinical skills of nursing students

Directory of Open Access Journals (Sweden)

Prasuna Jelly

2017-01-01

Full Text Available Introduction: Nurses are trained with specific clinical skills, and objective structured clinical examination (OSCE could be a better approach to assess clinical skills of nursing students. Materials and Methods: A comparative study was conducted by observational checklist regarding antenatal care and opinionnaire on the usefulness of OSCE and tradition evaluation method (TEM was used to assess the clinical skills and to get opinion. Results: The mean score of OSCE was more than TEM and the difference was statistically significant (P < 0.001. The opinion of students regarding the usefulness of OSCE was higher than TEM. Conclusions: The study concluded that implementing OSCE will overweigh the advantages of the TEM.

Identifying and assessing the factors affecting skill gap in digital marketing in communication industry companies

Directory of Open Access Journals (Sweden)

Fereshteh Ghotbifar

2017-03-01

Full Text Available As far as new communication channels are concerned, there have been extensive developments in communications and marketing in digital era. Today, therefore, companies try to take advantage of digital marketing channels to provide suitable services to customers to improve their satisfaction level. However, this study aimed to identify and assess factors affecting skill gap in digital marketing. This was descriptive correlation study. The population consisted of experts in communications industry to identify most important skill gaps in digital marketing and factors affecting them; also, managers and specialists of these companies were investigated to determine the role of identified factors in reducing skills gaps. Using localized questionnaire and interviewing with ten experts who were selected by Delphi snowball method, the skill gaps in marketing and factors affecting them were identified. Also, a researcher made questionnaire with 32 questions was distributed among 226 employees to investigate the identified factors role in reducing skills gap in digital marketing. The results showed that from four identified factors, the components including operational strategic factors and environmental factors had direct and positive impact on creating skill gap in digital marketing of studied companies. The environmental factors such as social and cultural conditions, religion, technology, and economy had more proactive impact on skills gap in digital marketing. Also, the results showed that among skills gaps in digital marketing of studied companies, the skills (Principles of Communication and (Predicting Future had the highest and lowest gaps, respectively.

Critical Emergency Medicine Procedural Skills: A Comparative Study of Methods for Teaching and Assessment.

Science.gov (United States)

Chapman, Dane M.; And Others

Three critical procedural skills in emergency medicine were evaluated using three assessment modalities--written, computer, and animal model. The effects of computer practice and previous procedure experience on skill competence were also examined in an experimental sequential assessment design. Subjects were six medical students, six residents,…

Assessing 21st Century Skills: A Guide to Evaluating Mastery and Authentic Learning

Science.gov (United States)

Greenstein, Laura

2012-01-01

The Common Core State Standards clearly define the skills students need for success in college and the 21st century workplace. The question is, how can you measure student mastery of skills like creativity, problem solving, and use of technology? Laura Greenstein demonstrates how teachers can teach and assess 21st century skills using authentic…

Teaching and assessment of communication skills in undergraduate dental education - a survey in German-speaking countries.

Science.gov (United States)

Rüttermann, S; Sobotta, A; Hahn, P; Kiessling, C; Härtl, A

2017-08-01

Teaching communication is perceived to be of importance in dental education. Several reports have been published worldwide in the educational literature describing modifications of the dental curriculum by implementing the teaching of communication skills. Surveys which evaluate the current state of training and assessment of communication skills in dental education in different countries exist already in some countries, but little information is available about German-speaking countries. In a cross-sectional study with the aim of a census, all 36 dental schools in Germany (30), Austria (3), and Switzerland (3) were surveyed. The present survey revealed that at 26 of the 34 dental schools (76%), communication skills training has been implemented. Training of communication skills mainly takes place between the 6th and the 9th semester. Ten schools were able to implement a partly longitudinal curriculum, while the other sites only offer stand-alone courses. Of the 34 dental schools, six assess communication skills in a summative way. Three of those schools also use formative assessments for their students. Another seven sites only use formative assessment. From the various formats of assessment, OSCE is mentioned most frequently. The necessity to train and assess communication skills has reached German-speaking dental schools. The present survey allows an overview of the training and assessment of communication skills in undergraduate dental education in German-speaking Europe. © 2016 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

Nursing physical assessment for patient safety in general wards: reaching consensus on core skills.

Science.gov (United States)

Douglas, Clint; Booker, Catriona; Fox, Robyn; Windsor, Carol; Osborne, Sonya; Gardner, Glenn

2016-07-01

To determine consensus across acute care specialty areas on core physical assessment skills necessary for early recognition of changes in patient status in general wards. Current approaches to physical assessment are inconsistent and have not evolved to meet increased patient and system demands. New models of nursing assessment are needed in general wards that ensure a proactive and patient safety approach. A modified Delphi study. Focus group interviews with 150 acute care registered nurses at a large tertiary referral hospital generated a framework of core skills that were developed into a web-based survey. We then sought consensus with a panel of 35 senior acute care registered nurses following a classical Delphi approach over three rounds. Consensus was predefined as at least 80% agreement for each skill across specialty areas. Content analysis of focus group transcripts identified 40 discrete core physical assessment skills. In the Delphi rounds, 16 of these were consensus validated as core skills and were conceptually aligned with the primary survey: (Airway) Assess airway patency; (Breathing) Measure respiratory rate, Evaluate work of breathing, Measure oxygen saturation; (Circulation) Palpate pulse rate and rhythm, Measure blood pressure by auscultation, Assess urine output; (Disability) Assess level of consciousness, Evaluate speech, Assess for pain; (Exposure) Measure body temperature, Inspect skin integrity, Inspect and palpate skin for signs of pressure injury, Observe any wounds, dressings, drains and invasive lines, Observe ability to transfer and mobilise, Assess bowel movements. Among a large and diverse group of experienced acute care registered nurses consensus was achieved on a structured core physical assessment to detect early changes in patient status. Although further research is needed to refine the model, clinical application should promote systematic assessment and clinical reasoning at the bedside. © 2016 John Wiley & Sons Ltd.

ASK Standards: Assessment, Skills, and Knowledge Content Standards for Student Affairs Practitioners and Scholars

Science.gov (United States)

ACPA College Student Educators International, 2011

2011-01-01

The Assessment Skills and Knowledge (ASK) standards seek to articulate the areas of content knowledge, skill and dispositions that student affairs professionals need in order to perform as practitioner-scholars to assess the degree to which students are mastering the learning and development outcomes the professionals intend. Consistent with…

Assessment of global motor performance and gross and fine motor skills of infants attending day care centers.

Science.gov (United States)

Souza, Carolina T; Santos, Denise C C; Tolocka, Rute E; Baltieri, Letícia; Gibim, Nathália C; Habechian, Fernanda A P

2010-01-01

To analyze the global motor performance and the gross and fine motor skills of infants attending two public child care centers full-time. This was a longitudinal study that included 30 infants assessed at 12 and 17 months of age with the Motor Scale of the Bayley Scales of Infant and Toddler Development, Third Edition (Bayley-III). This scale allows the analysis of global motor performance, fine and gross motor performance, and the discrepancy between them. The Wilcoxon test and Spearman's correlation coefficient were used. Most of the participants showed global motor performance within the normal range, but below the reference mean at 12 and 17 months, with 30% classified as having "suspected delays" in at least one of the assessments. Gross motor development was poorer than fine motor development at 12 and at 17 months of age, with great discrepancy between these two subtests in the second assessment. A clear individual variability was observed in fine motor skills, with weak linear correlation between the first and the second assessment of this subtest. A lower individual variability was found in the gross motor skills and global motor performance with positive moderate correlation between assessments. Considering both performance measurements obtained at 12 and 17 months of age, four infants were identified as having a "possible delay in motor development". The study showed the need for closer attention to the motor development of children who attend day care centers during the first 17 months of life, with special attention to gross motor skills (which are considered an integral part of the child's overall development) and to children with suspected delays in two consecutive assessments.

Skills in clinical communication: Are we correctly assessing them at undergraduate level?

Directory of Open Access Journals (Sweden)

Alberto Zamora Cervantes

2014-07-01

Full Text Available Communicating with the patient in clinical practice refers to the way in which the doctor and the patient interact both verbally and nonverbally, in order to achieve a shared understanding of problems and solutions. Traditional learning and assessment systems are overwhelmed when it comes to addressing the complex and multi-dimensional problems of professional practice. Problem Based Learning (PBL has been put forward as an alternative to the mere reproduction of knowledge and pre-established patterns, enabling students to develop their own learning strategies to overcome problems in their future professional practice. The challenge is to determine how to assess the acquisition of clinical communication skills. The authors have recommended a summative assessment of clinical communication skills based on the combination of different methods. It highlights the importance of feedback-based formative assessment. This raises the need to develop and validate assessment scales in clinical communication at an undergraduate level. Based on this work, the authors put forward a "fanned out" assessment in terms of clinical communication skills in Medicine degrees, with the use of different instruments in a "spiraled" manner, where the greater the contact with clinical practice in the various degree and integral courses, the greater difficulty experienced, with the participation of all the stakeholders involved (self, hetero and peer assessment without precluding the involvement of patients (real or simulated in the design of assessment instruments.

Using systematically observed clinical encounters (SOCEs to assess medical students’ skills in clinical settings

Directory of Open Access Journals (Sweden)

George R Bergus

2010-11-01

Full Text Available George R Bergus1–3, Jerold C Woodhead4, Clarence D Kreiter2,51Performance Based Assessment Program, Office of Student Affairs and Curriculum, 2Department of Family Medicine, 3Department of Psychiatry, 4Department of Pediatrics, 5Office of Consultation and Research in Medical Education, Roy J and Lucille A Carver College of Medicine, The University of Iowa, Iowa City, IA, USAIntroduction: The Objective Structured Clinical Examination (OSCE is widely used to assess the clinical performance of medical students. However, concerns related to cost, availability, and validity, have led educators to investigate alternatives to the OSCE. Some alternatives involve assessing students while they provide care to patients – the mini-CEX (mini-Clinical Evaluation Exercise and the Long Case are examples. We investigated the psychometrics of systematically observed clinical encounters (SOCEs, in which physicians are supplemented by lay trained observers, as a means of assessing the clinical performances of medical students.Methods: During the pediatrics clerkship at the University of Iowa, trained lay observers assessed the communication skills of third-year medical students using a communication checklist while the students interviewed and examined pediatric patients. Students then verbally presented their findings to faculty, who assessed students’ clinical skills using a standardized form. The reliability of the combined communication and clinical skills scores was calculated using generalizability theory.Results: Fifty-one medical students completed 199 observed patient encounters. The mean combined clinical and communication skills score (out of a maximum 45 points was 40.8 (standard deviation 3.3. The calculated reliability of the SOCE scores, using generalizability theory, from 10 observed patient encounters was 0.81. Students reported receiving helpful feedback from faculty after 97% of their observed clinical encounters.Conclusion: The SOCE can

The effect of video review of resident laparoscopic surgical skills measured by self- and external assessment.

Science.gov (United States)

Herrera-Almario, Gabriel E; Kirk, Katherine; Guerrero, Veronica T; Jeong, Kwonho; Kim, Sara; Hamad, Giselle G

2016-02-01

Video review of surgical skills is an educational modality that allows trainees to reflect on self-performance. The purpose of this study was to determine whether resident and attending assessments of a resident's laparoscopic performance differ and whether video review changes assessments. Third-year surgery residents were invited to participate. Elective laparoscopic procedures were video recorded. The Global Operative Assessment of Laparoscopic Skills evaluation was completed immediately after the procedure and again 7 to 10 days later by both resident and attending. Scores were compared using t tests. Nine residents participated and 76 video reviews were completed. Residents scored themselves significantly lower than the faculty scores both before and after video review. Resident scores did not change significantly after video review. Attending and resident self-assessment of laparoscopic skills differs and subsequent video review does not significantly affect Global Operative Assessment of Laparoscopic Skills scores. Further studies should evaluate the impact of video review combined with verbal feedback on skill acquisition and assessment. Copyright © 2016 Elsevier Inc. All rights reserved.

Project RAILS: Lessons Learned about Rubric Assessment of Information Literacy Skills

Science.gov (United States)

Belanger, Jackie; Zou, Ning; Mills, Jenny Rushing; Holmes, Claire; Oakleaf, Megan

2015-01-01

Rubric assessment of information literacy is an important tool for librarians seeking to show evidence of student learning. The authors, who collaborated on the Rubric Assessment of Informational Literacy Skills (RAILS) research project, draw from their shared experience to present practical recommendations for implementing rubric assessment in a…

Effectiveness and safety of wheelchair skills training program in improving the wheelchair skills capacity: a systematic review.

Science.gov (United States)

Tu, Chun-Jing; Liu, Lin; Wang, Wei; Du, He-Ping; Wang, Yu-Ming; Xu, Yan-Bing; Li, Ping

2017-12-01

To comprehensively assess the effectiveness and safety of wheelchair skills training program in improving wheelchair skills capacity. PubMed, OVID, EBSCO, ScienceDirect, Web of Science, CINAHL, Cochrane Library, Google Scholar, and China Knowledge Resource Integrated Database were searched up to March 2017. Controlled clinical trials that compared a wheelchair skills training program with a control group that received other interventions and used the wheelchair skills test scores to evaluate wheelchair skills capacity were included. Two authors independently screened articles, extracted data, and assessed the methodological quality using the Cochrane risk-of-bias tool in randomized controlled trial (RCT) and methodological index for non-randomized studies. The data results of wheelchair skills test scores were extracted. Data from 455 individuals in 10 RCTs and from 140 participants in seven non-randomized studies were included for meta-analysis using Stata version 12.0 (Stata Corporation, College Station, TX, USA). In the short term (immediately to one week) post-intervention, relative to a control group, manual wheelchair skills training could increase the total wheelchair skills test scores by 13.26% in RCTs (95% confidence interval (CI), 6.19%-20.34%; P skills training and the long-term (3-12 months) advantage of manual wheelchair skills training ( P = 0.755). The limited evidence suggests that wheelchair skills training program is beneficial in the short term, but its long-term effects remain unclear.

Assessing the surgical skills of trainees in the operating theatre: a prospective observational study of the methodology.

Science.gov (United States)

Beard, J D; Marriott, J; Purdie, H; Crossley, J

2011-01-01

To compare user satisfaction and acceptability, reliability and validity of three different methods of assessing the surgical skills of trainees by direct observation in the operating theatre across a range of different surgical specialties and index procedures. A 2-year prospective, observational study in the operating theatres of three teaching hospitals in Sheffield. The assessment methods were procedure-based assessment (PBA), Objective Structured Assessment of Technical Skills (OSATS) and Non-technical Skills for Surgeons (NOTSS). The specialties were obstetrics and gynaecology (O&G) and upper gastrointestinal, colorectal, cardiac, vascular and orthopaedic surgery. Two to four typical index procedures were selected from each specialty. Surgical trainees were directly observed performing typical index procedures and assessed using a combination of two of the three methods (OSATS or PBA and NOTSS for O&G, PBA and NOTSS for the other specialties) by the consultant clinical supervisor for the case and the anaesthetist and/or scrub nurse, as well as one or more independent assessors from the research team. Information on user satisfaction and acceptability of each assessment method from both assessor and trainee perspectives was obtained from structured questionnaires. The reliability of each method was measured using generalisability theory. Aspects of validity included the internal structure of each tool and correlation between tools, construct validity, predictive validity, interprocedural differences, the effect of assessor designation and the effect of assessment on performance. Of the 558 patients who were consented, a total of 437 (78%) cases were included in the study: 51 consultant clinical supervisors, 56 anaesthetists, 39 nurses, 2 surgical care practitioners and 4 independent assessors provided 1635 assessments on 85 trainees undertaking the 437 cases. A total of 749 PBAs, 695 NOTSS and 191 OSATSs were performed. Non-O&G clinical supervisors and

The development of a rubric for peer assessment of individual teamwork skills in undergraduate midwifery students.

Science.gov (United States)

Hastie, Carolyn; Fahy, Kathleen; Parratt, Jenny

2014-09-01

Poor teamwork is cited as one of the major root causes of adverse events in healthcare. Bullying, resulting in illness for staff, is an expression of poor teamwork skills. Despite this knowledge, poor teamwork persists in healthcare and teamwork skills are rarely the focus of teaching and assessment in undergraduate health courses. To develop and implement an assessment tool for use in facilitating midwifery students' learning of teamwork skills. This paper describes how the TeamUP rubric tool was developed. A review of the literature found no research reports on how to teach and assess health students' teamwork skills in standing teams. The literature, however, gives guidance about how university educators should evaluate individual students using peer assessment. The developmental processes of the rubric were grounded in the theoretical literature and feminist collaborative conversations. The rubric incorporates five domains of teamwork skills: Fostering a Team Climate; Project Planning; Facilitating Teams; Managing Conflict and Quality Individual Contribution. The process and outcomes of student and academic content validation are described. The TeamUP rubric is useful for articulating, teaching and assessing teamwork skills for health professional students. The TeamUP rubric is a robust, theoretically grounded model that defines and details effective teamwork skills and related behaviours. If these skills are mastered, we predict that graduates will be more effective in teams. Our assumption is that graduates, empowered by having these skills, are more likely to manage conflict effectively and less likely to engage in bullying behaviours. Crown Copyright © 2014. Published by Elsevier Ltd. All rights reserved.

The Impact of Diagnosing Skill Deficiencies and Assessment-Based Communication Training on Managerial Performance.

Science.gov (United States)

Papa, Michael J.; Graham, Elizabeth E.

1991-01-01

Evaluates an organizational diagnosis program that assesses managerial communication skills and provides the frame for follow-up training programs. Finds that managers participating in follow-up communication skills training performed significantly higher on interpersonal skills, problem-solving ability, and productivity over three long-term…

Development of a tool to support holistic generic assessment of clinical procedure skills.

Science.gov (United States)

McKinley, Robert K; Strand, Janice; Gray, Tracey; Schuwirth, Lambert; Alun-Jones, Tom; Miller, Helen

2008-06-01

The challenges of maintaining comprehensive banks of valid checklists make context-specific checklists for assessment of clinical procedural skills problematic. This paper reports the development of a tool which supports generic holistic assessment of clinical procedural skills. We carried out a literature review, focus groups and non-participant observation of assessments with interview of participants, participant evaluation of a pilot objective structured clinical examination (OSCE), a national modified Delphi study with prior definitions of consensus and an OSCE. Participants were volunteers from a large acute teaching trust, a teaching primary care trust and a national sample of National Health Service staff. Results In total, 86 students, trainees and staff took part in the focus groups, observation of assessments and pilot OSCE, 252 in the Delphi study and 46 candidates and 50 assessors in the final OSCE. We developed a prototype tool with 5 broad categories amongst which were distributed 38 component competencies. There was > 70% agreement (our prior definition of consensus) at the first round of the Delphi study for inclusion of all categories and themes and no consensus for inclusion of additional categories or themes. Generalisability was 0.76. An OSCE based on the instrument has a predicted reliability of 0.79 with 12 stations and 1 assessor per station or 10 stations and 2 assessors per station. This clinical procedural skills assessment tool enables reliable assessment and has content and face validity for the assessment of clinical procedural skills. We have designated it the Leicester Clinical Procedure Assessment Tool (LCAT).

Assessing soft skills components in science and technology programs within Malaysian Technical Universities

Directory of Open Access Journals (Sweden)

Kahirol Mohd Salleh

2017-06-01

Full Text Available The workforce is a social environment where particular skills are essential in order for workers to perform well, have a competitive edge and succeed in their careers. A soft skill is one of the skills needed in every type of workplace setting. Soft skills include communication skills, collaboration skills, entrepreneurship, and others. Workplace need workers who are competent not only with technical skills but who also have soft skills. There is lack of literature discussion on the perceptions of needed soft skills in the workplace. Therefore, this study aimed to close the gap in the literature concerning the perceptions of instructors and students in the area of soft skills especially in communication skills, collaboration skills, and entrepreneurship at the workplace. This study aims to decrease the divide in soft skills differences among instructors and students in higher learning institutions so that students have the opportunity to excel in their learning process. This study employs quantitative method specifically using survey as an instrument. The findings shows soft skills including communication, collaboration, and entrepreneurship skills are one of the aspects stressed by the students and instructors at higher learning institutions to ensure that graduates will be employable and competent in order to contribute to the nation development.

Students' performance in the different clinical skills assessed in OSCE: what does it reveal?

Science.gov (United States)

Sim, Joong Hiong; Abdul Aziz, Yang Faridah; Mansor, Azura; Vijayananthan, Anushya; Foong, Chan Choong; Vadivelu, Jamuna

2015-01-01

The purpose of this study was to compare students' performance in the different clinical skills (CSs) assessed in the objective structured clinical examination. Data for this study were obtained from final year medical students' exit examination (n=185). Retrospective analysis of data was conducted using SPSS. Means for the six CSs assessed across the 16 stations were computed and compared. Means for history taking, physical examination, communication skills, clinical reasoning skills (CRSs), procedural skills (PSs), and professionalism were 6.25±1.29, 6.39±1.36, 6.34±0.98, 5.86±0.99, 6.59±1.08, and 6.28±1.02, respectively. Repeated measures ANOVA showed there was a significant difference in the means of the six CSs assessed [F(2.980, 548.332)=20.253, pskill to be acquired by medical students before entering the workplace. Despite its challenges, students must learn the skills of clinical reasoning, while clinical teachers should facilitate the clinical reasoning process and guide students' clinical reasoning development.

Using a Structured Assessment Tool to Evaluate Nontechnical Skills of Nurse Anesthetists

DEFF Research Database (Denmark)

Lyk-Jensen, Helle Teglgaard; Dieckmann, Peter; Konge, Lars

2016-01-01

Nontechnical skills are critical for good anesthetic practice but are seldom addressed explicitly in clinical training. The purposes of this study were (1) to evaluate the reliability and validity of the observation-based assessment tool Nurse Anaesthetists' Non-Technical Skills system (N......-ANTS) and (2) to evaluate the effect of training nurse anesthetist supervisors in the use of N-ANTS. This system comprises a global rating score, 4 categories, and 15 elements to rate nurse anesthetists' nontechnical skills. A 1-day workshop was conducted for 22 nurse anesthetist supervisors to rate nurse...... nontechnical skills of nurse anesthetists....

Validation of the Family Meeting Behavioral Skills Checklist. An Instrument to Assess Fellows' Communication Skills.

Science.gov (United States)

Gustin, Jillian L; Way, David P; Wells-Di Gregorio, Sharla; McCallister, Jennifer W

2016-08-01

Fellows in pulmonary and critical care medicine are required to show competency in facilitating family meetings for critically ill patients. There are many assessment measures available for evaluating physician-patient communication (e.g., the SEGUE Framework [Set the stage, Elicit information, Give information, Understand the patient's perspective, End the encounter]) and some designed for family meetings. However, no validated measure exists that is specifically designed to assess communication skills during family meetings with surrogate decision makers in intensive care settings. We developed the Family Meeting Behavioral Skills Checklist (FMBSC) to measure advanced communication skills of fellows in family meetings of critically ill patients based on a literature review and consensus of an interdisciplinary group of communications experts. We evaluated the psychometric properties of the FMBSC. We digitally recorded 16 pulmonary/critical care fellows performing a simulated family meeting for a critically ill patient at the end of 1 year of fellowship training. Two clinical health psychologists evaluated each recording independently using the FMBSC Rating Scale and the SEGUE Framework. Judges recorded the number of skills performed using the checklist and employed a summary rating scale to judge the level of performance for each of nine subsets of skills. Each instrument was scored and converted to percentage scores. The FMBSC and SEGUE Framework items were summed and converted to percentage scores for each category and as a total for each instrument. The rating scale items on the FMBSC were also summed and converted to a percentage score. Four primary analyses were conducted to evaluate interjudge reliability, internal consistency, and concurrent validity. Interrater reliability was higher for the FMBSC (intraclass correlation [ICC2,2] = 0.57) than for the SEGUE instrument (ICC2,2 = 0.32) or the FMBSC Rating Scale (ICC2,2 = 0.23). The FMBSC

Assessing soft skills components in science and technology programs within Malaysian Technical Universities

OpenAIRE

Kahirol Mohd Salleh; Nor Lisa Sulaiman; Mimi Mohaffyza Mohamad; Lai Chee Sern

2017-01-01

The workforce is a social environment where particular skills are essential in order for workers to perform well, have a competitive edge and succeed in their careers. A soft skill is one of the skills needed in every type of workplace setting. Soft skills include communication skills, collaboration skills, entrepreneurship, and others. Workplace need workers who are competent not only with technical skills but who also have soft skills. There is lack of literature discussion on the ...

Assessing video games to improve driving skills: a literature review and observational study.

Science.gov (United States)

Sue, Damian; Ray, Pradeep; Talaei-Khoei, Amir; Jonnagaddala, Jitendra; Vichitvanichphong, Suchada

2014-08-07

For individuals, especially older adults, playing video games is a promising tool for improving their driving skills. The ease of use, wide availability, and interactivity of gaming consoles make them an attractive simulation tool. The objective of this study was to look at the feasibility and effects of installing video game consoles in the homes of individuals looking to improve their driving skills. A systematic literature review was conducted to assess the effect of playing video games on improving driving skills. An observatory study was performed to evaluate the feasibility of using an Xbox 360 Kinect console for improving driving skills. Twenty-nine articles, which discuss the implementation of video games in improving driving skills were found in literature. On our study, it was found the Xbox 360 with Kinect is capable of improving physical and mental activities. Xbox Video games were introduced to engage players in physical, visual and cognitive activities including endurance, postural sway, reaction time, eyesight, eye movement, attention and concentration, difficulties with orientation, and semantic fluency. However, manual dexterity, visuo-spatial perception and binocular vision could not be addressed by these games. It was observed that Xbox Kinect (by incorporating Kinect sensor facilities) combines physical, visual and cognitive engagement of players. These results were consistent with those from the literature review. From the research that has been carried out, we can conclude that video game consoles are a viable solution for improving user's physical and mental state. In future we propose to carry a thorough evaluation of the effects of video games on driving skills in elderly people.

Curriculum content and assessment of pre-clinical dental skills: A survey of undergraduate dental education in Europe.

Science.gov (United States)

Field, J; Stone, S; Orsini, C; Hussain, A; Vital, S; Crothers, A; Walmsley, D

2018-05-01

Since 1981, the qualifications for various healthcare professionals across the European Union have enjoyed mutual recognition in accordance with the EU Directive 81/1057/EEC. Whilst the directive includes dental practitioners, it is recognised that significant variation exists in curriculum structure, content and scope of practice across institutions. This article aimed to explore pan-European practice in relation to curriculum content, teaching and learning strategies and assessment of pre-clinical dental skills. A request to complete an online questionnaire, in English, was sent electronically to skills leads at all Association of Dental Education in Europe member schools. The questionnaire collected information in relation to institution and country, regulatory requirements to demonstrate safety, details of specific pre-clinical skills courses, learning materials and teaching staff. Forty-eight institutions, from 25 European countries responded. Seven countries (n=7, 28%) reported no requirement to demonstrate student operative safety prior to patient treatment. Several core and operative clinical skills are common to the majority of institutions. The most commonly taught core skills related directly to the clinical environment such as cross-infection control and hand washing. The least common were skills that indirectly related to patient care, such as communication skills and working as a team. There are clear differences within European pre-clinical dental education, and greater efforts are needed to demonstrate that all European students are fit to practice before they start treating patients. Learning outcomes, teaching activities and assessment activities of pre-clinical skills should be shared collaboratively to further standardise curricula. © 2017 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.

A Preliminary Investigation of General and Technique-specific Assessments for the Evaluation of Laparoscopic Technical Skills.

Science.gov (United States)

Vergis, Ashley; Steigerwald, Sarah

2017-10-07

Background  Both general and technique-specific assessments of technical skill have been validated in surgical education. The purpose of this study was to assess the correlation between the objective structured assessment of technical skills (OSATS) and the global operative assessment of laparoscopic skills (GOALS) rating scales using a high-fidelity porcine laparoscopic cholecystectomy model. Methods Post-graduate year-one general surgery and urology residents (n=14) performed a live laparoscopic porcine cholecystectomy. Trained surgeons rated their performance using OSATS and GOALS assessment scales. Results Pearson's correlation coefficient between OSATS and GOALS was 0.96 for overall scores. It ranged from 0.78 - 0.89 for domains that overlapped between the two scales. Conclusion There is a very high correlation between OSATS and GOALS. This implies that they likely measure similar constructs and that either may be used for summative-type assessments of trainee skill. However, further investigation is needed to determine if technique-specific assessments may provide more useful feedback in formative evaluation.

Toward an objective assessment of technical skills: a national survey of surgical program directors in Saudi Arabia.

Science.gov (United States)

Alkhayal, Abdullah; Aldhukair, Shahla; Alselaim, Nahar; Aldekhayel, Salah; Alhabdan, Sultan; Altaweel, Waleed; Magzoub, Mohi Elden; Zamakhshary, Mohammed

2012-01-01

After almost a decade of implementing competency-based programs in postgraduate training programs, the assessment of technical skills remains more subjective than objective. National data on the assessment of technical skills during surgical training are lacking. We conducted this study to document the assessment tools for technical skills currently used in different surgical specialties, their relationship with remediation, the recommended tools from the program directors' perspective, and program directors' attitudes toward the available objective tools to assess technical skills. This study was a cross-sectional survey of surgical program directors (PDs). The survey was initially developed using a focus group and was then sent to 116 PDs. The survey contains demographic information about the program, the objective assessment tools used, and the reason for not using assessment tools. The last section discusses the recommended tools to be used from the PDs' perspective and the PDs' attitude and motivation to apply these tools in each program. The associations between the responses to the assessment questions and remediation were statistically evaluated. Seventy-one (61%) participants responded. Of the respondents, 59% mentioned using only nonstandardized, subjective, direct observation for technical skills assessment. Sixty percent use only summative evaluation, whereas 15% perform only formative evaluations of their residents, and the remaining 22% conduct both summative and formative evaluations of their residents' technical skills. Operative portfolios are kept by 53% of programs. The percentage of programs with mechanisms for remediation is 29% (19 of 65). The survey showed that surgical training programs use different tools to assess surgical skills competency. Having a clear remediation mechanism was highly associated with reporting remediation, which reflects the capability to detect struggling residents. Surgical training leadership should invest more in

Teachers' Perceptions of a Fundamental Movement Skill (FMS) Assessment Battery in a School Setting

Science.gov (United States)

Lander, Natalie; Morgan, Philip J.; Salmon, Jo; Barnett, Lisa M.

2016-01-01

Fundamental movement skills (FMS) competence is low in adolescent girls. An assessment tool for teachers is needed to monitor FMS in this demographic. The present study explored whether the Canadian Agility and Movement Skill Assessment (CAMSA) is feasible for use by physical education (PE) teachers of Australian Year 7 girls in a school setting.…

Moving beyond Assessment to Improving Students' Critical Thinking Skills: A Model for Implementing Change

Science.gov (United States)

Haynes, Ada; Lisic, Elizabeth; Goltz, Michele; Stein, Barry; Harris, Kevin

2016-01-01

This research examines how the use of the CAT (Critical thinking Assessment Test) and involvement in CAT-Apps (CAT Applications within the discipline) training can serve as an important part of a faculty development model that assists faculty in the assessment of students' critical thinking skills and in the development of these skills within…

Simulation Technology for Skills Training and Competency Assessment in Medical Education

Science.gov (United States)

Obeso, Vivian T.; Issenberg, S. Barry

2007-01-01

Medical education during the past decade has witnessed a significant increase in the use of simulation technology for teaching and assessment. Contributing factors include: changes in health care delivery and academic environments that limit patient availability as educational opportunities; worldwide attention focused on the problem of medical errors and the need to improve patient safety; and the paradigm shift to outcomes-based education with its requirements for assessment and demonstration of competence. The use of simulators addresses many of these issues: they can be readily available at any time and can reproduce a wide variety of clinical conditions on demand. In lieu of the customary (and arguably unethical) system, whereby novices carry out the practice required to master various techniques—including invasive procedures—on real patients, simulation-based education allows trainees to hone their skills in a risk-free environment. Evaluators can also use simulators for reliable assessments of competence in multiple domains. For those readers less familiar with medical simulators, this article aims to provide a brief overview of these educational innovations and their uses; for decision makers in medical education, we hope to broaden awareness of the significant potential of these new technologies for improving physician training and assessment, with a resultant positive impact on patient safety and health care outcomes. PMID:18095044

Assessing Leadership and Problem-Solving Skills and Their Impacts in the Community.

Science.gov (United States)

Rohs, F. Richard; Langone, Christine A.

1993-01-01

A pretest-posttest control group design was used to assess the leadership and problem-solving skills of 281 participants and 110 controls in a statewide community leadership development program. Quantitative and qualitative data demonstrate that the program has been a catalyst to influence leadership and problem-solving skills for community…

Assessing neurosurgical non-technical skills: an exploratory study of a new behavioural marker system.

Science.gov (United States)

Michinov, Estelle; Jamet, Eric; Dodeler, Virginie; Haegelen, Claire; Jannin, Pierre

2014-10-01

The management of non-technical skills is a major factor affecting teamwork quality and patient safety. This article presents a behavioural marker system for assessing neurosurgical non-technical skills (BMS-NNTS). We tested the BMS during deep brain stimulation surgery. We developed the BMS in three stages. First, we drew up a provisional assessment tool based on the literature and observation tools developed for other surgical specialties. We then analysed videos made in an operating room (OR) during deep brain stimulation operations in order to ensure there were no significant omissions from the skills list. Finally, we used five videos of operations to identify the behavioural markers of non-technical skills in verbal communications. Analyses of more than six hours of observations revealed 3515 behaviours from which we determined the neurosurgeon's non-technical skills behaviour pattern. The neurosurgeon frequently engaged in explicit coordination, situation awareness and leadership behaviours. In addition, the neurosurgeon's behaviours differed according to the stage of the operation and the OR staff members with whom she was communicating. Our behavioural marker system provides a structured approach to assessing non-technical skills in the field of neurosurgery. It can also be transferred to other surgical specialties and used in surgeon training curricula. © 2014 John Wiley & Sons, Ltd.

Developing an Objective Structured Assessment of Technical Skills for Laparoscopic Suturing and Intracorporeal Knot Tying.

Science.gov (United States)

Chang, Olivia H; King, Louise P; Modest, Anna M; Hur, Hye-Chun

2016-01-01

To develop a teaching and assessment tool for laparoscopic suturing and intracorporeal knot tying. We designed an Objective Structured Assessment of Technical Skills (OSATS) tool that includes a procedure-specific checklist (PSC) and global rating scale (GRS) to assess laparoscopic suturing and intracorporeal knot-tying performance. Obstetrics and Gynecology residents at our institution were videotaped while performing a laparoscopic suturing and intracorporeal knot-tying task at a surgical simulation workshop. A total of 2 expert reviewers assessed resident performance using the OSATS tool during live performance and 1 month later using the videotaped recordings. OSATS scores were analyzed using the Wilcoxon rank-sum test. Data are presented as median scores (interquartile range [IQR]). Intrarater and interrater reliabilities were assessed using a Spearman correlation and are presented as an r correlation coefficient and p value. An r ≥ 0.8 was considered as a high correlation. After testing, we received feedback from residents and faculty to improve the OSATS tool as part of an iterative design process. In all, 14 of 21 residents (66.7%) completed the study, with 9 junior residents and 5 senior residents. Junior residents had a lower score on the PSC than senior residents did; however, this was not statistically significant (median = 6.0 [IQR: 4.0-10.0] and median = 13.0 [IQR: 10.0-13.0]; p = 0.09). There was excellent intrarater reliability with our OSATS tool (for PSC component, r = 0.88 for Rater 1 and 0.93 for Rater 2, both p assessment and teaching tool for laparoscopic suturing and intracorporeal knot-tying skills. Overall, good intrarater reliability was demonstrated, suggesting that this tool may be useful for longitudinal assessment of surgical skills. Copyright © 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Description of the Assessment of Basic Language and Learning Skills Revisited (ABLLS-R

Directory of Open Access Journals (Sweden)

Semenovich M.L.

2015-12-01

Full Text Available Diagnostics and assessment of the functional skills of children with disabilities and autism spectrum disorders are to be conducted to develop comprehensive remedial educational programmes. The described Methodology of the Assessment of Basic Language and Learning Skills — Revisited (ABLLS-R allows to simplify and make the diagnostics more efficient, to conduct a comprehensive examination of the child in different areas of development, detect the formed and deficit skills. The second and final part of the description of the methodology offers recommendations on the filling of the Table of the Results of Initial and Repeated Testing and on the choice of goals of correctional work with a child on the basis of performance of individual test scales. The pattern of the table filled after the initial and repeated testing is given. In drawing up of the programme of individual development the willingness of the child to the development of that skill should be considered. Regular practice of selected skills in various situations and the preventive measures against the regression of skills are also important. Conclusive part. Beginning in № 3 (48, 2015

Development and assessment of key skills in undergraduate students: An action-research experience

Directory of Open Access Journals (Sweden)

Ana Fernández-Santander

2012-03-01

Full Text Available Employers look for professionals able to work in a team, able to approach problems, with the capacity to analyze and resolve problems, under the constant renewal of knowledge and competencies. In this paper, a group of University teachers from different areas of knowledge presents an experience to introduce key employability skills in the higher education students’ curricula. This work has been developed under the action research scope. The first goal was to make an analysis of terms referred to key skills, generating an integrated denomination for each competency. The elaboration of general templates for key skills is proposed here as a useful tool that provides information about development, assessment and marking of each skill. Different types of rubrics and assessment templates, used during this experience, are presented. DOI: 10.18870/hlrc.v2i1.37

Assessing Writing Skills

Science.gov (United States)

Narayanan, M.

2013-12-01

This is an encore presentation of what was presented at the 2012 AGU International Conference. It was entitled: 'ASSESSING CORE COMPETENCIES.' The poster presentation, however, has been redesigned and reorganized with new, revised perspectives. The importance of ASSESSMENT principles has been emphasized. Catherine Palomba and Trudy Banta offer the following definition of assessment, adapted from one provided by Marchese in 1987. Assessment is the systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and development. (Palomba and Banta 1999). Educational institutions are committing substantial resources to the establishment of dedicated technology-based laboratories, so that they will be able to accommodate and fulfill students' desire to master certain of these specific skills. This type of technology-based instruction may raise some fundamental questions about the core competencies of the student learner. Some of the most important questions are : 1. Is the utilization of these fast high-powered computers and user-friendly software programs creating a totally non-challenging instructional environment for the student learner ? 2. Can technology itself all too easily overshadow the learning outcomes intended ? 3. Are the educational institutions simply training students how to use technology rather than educating them in the appropriate field ? 4. Are we still teaching content-driven courses and analysis oriented subject matter ? 5. Are these sophisticated modern era technologies contributing to a decline in the Critical Thinking Capabilities of the 21st century technology-savvy students ? The author tries to focus on technology as a tool and not on the technology itself. He further argues that students must demonstrate that they have the have the ability to think critically before they make an attempt to use technology in a chosen application-specific environment. The author further

Self-assessment of intercultural communication skills: a survey of physicians and medical students in Geneva, Switzerland.

Science.gov (United States)

Hudelson, Patricia; Perron, Noelle Junod; Perneger, Thomas

2011-09-01

Physicians working with multicultural populations need to know how to elicit the patient's understanding of the illness; determine the patient's sociocultural context and identify any issues that might affect care; communicate effectively across patient-provider social and cultural differences; and collaborate effectively with an interpreter. Skills self-assessment can contribute to identifying training needs and monitoring skills development in these areas. As part of a larger study exploring the knowledge, attitudes and practices of Geneva physicians and medical students regarding the care of immigrant patients, we asked respondents to self-rate their ability to perform a range of common yet challenging intercultural communication tasks. Overall, respondents rated themselves less competent at intercultural tasks than at basic medical skills and less competent at specific intercultural communication skills than at general intercultural skills. Qualified doctors (as opposed to students), those with greater interest in caring for immigrants, and those who rarely encountered difficulties with immigrants rated themselves significantly more competent for all clinical tasks. Having a higher percentage of immigrant patients and previous cultural competence training predicted greater self-rated intercultural communication skills. Our self-assessment results suggest that students and physicians should be provided with the opportunity to practice intercultural skills with immigrant patients as part of their cultural competence training. To strengthen the validity of self-assessment measures, they should ideally be combined with more objective methods to assess actual skills.

The objective assessment of experts' and novices' suturing skills using an image analysis program.

Science.gov (United States)

Frischknecht, Adam C; Kasten, Steven J; Hamstra, Stanley J; Perkins, Noel C; Gillespie, R Brent; Armstrong, Thomas J; Minter, Rebecca M

2013-02-01

To objectively assess suturing performance using an image analysis program and to provide validity evidence for this assessment method by comparing experts' and novices' performance. In 2009, the authors used an image analysis program to extract objective variables from digital images of suturing end products obtained during a previous study involving third-year medical students (novices) and surgical faculty and residents (experts). Variables included number of stitches, stitch length, total bite size, travel, stitch orientation, total bite-size-to-travel ratio, and symmetry across the incision ratio. The authors compared all variables between groups to detect significant differences and two variables (total bite-size-to-travel ratio and symmetry across the incision ratio) to ideal values. Five experts and 15 novices participated. Experts' and novices' performances differed significantly (P 0.8) for total bite size (P = .009, d = 1.5), travel (P = .045, d = 1.1), total bite-size-to-travel ratio (P algorithm can extract variables from digital images of a running suture and rapidly provide quantitative summative assessment feedback. The significant differences found between groups confirm that this system can discriminate between skill levels. This image analysis program represents a viable training tool for objectively assessing trainees' suturing, a foundational skill for many medical specialties.

Curricular Innovation in the Surgery Clerkship: Can Assessment Methods Influence Development of Critical Thinking and Clinical Skills?

Science.gov (United States)

McClintic, James A; Snyder, Clifford L; Brown, Kimberly M

2018-03-12

Although key clinical skills have been defined in the Core Entrustable Professional Activities, there is a need to improve medical school curricula with standardized training opportunities and assessments of these skills. Thus, we aimed to develop an innovative curriculum that emphasized critical thinking and clinical skills. We hypothesized that we would be able to observe measurable improvement on assessments of students' critical thinking and clinical skills after the implementation of the new curriculum. Prospective, Quasi-Experimental study with the use of historical controls. This study took place through the third-year surgical clerkship at the University of Texas Medical Branch at the Galveston, Houston, and Austin, Texas, locations. A total of 214 students taking the third-year surgical clerkship for the first time during the periods of interest were included. Although the students with traditional curriculum improved 9.5% on a short answer exam from preclerkship to postclerkship completion, the students with new curriculum improved by 40%. Students under the new curriculum performed significantly better on the Objective Structured Clinical Exam; however, their shelf scores were lower. Under this new curriculum and grading system, we demonstrated that students can be incentivized to improve critical thinking and clinical skills, but this needs to be balanced with knowledge-based incentives. Copyright © 2018 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

A Summary of an Assessment of Fourth and Sixth Grade Basic Skills.

Science.gov (United States)

CTB / McGraw-Hill, Monterey, CA.

A comprehensive assessment was made of the status of elementary education in Missouri in reading, mathematics, language, and study skills. The Comprehensive Tests of Basic Skills (CTBS) and the Short Form Test of Academic Aptitude (SFTAA) were administered to a sample of Missouri fourth and sixth graders. For each curricular area, Missouri…

Core skills assessment to improve mathematical competency

Science.gov (United States)

Carr, Michael; Bowe, Brian; Fhloinn, Eabhnat Ní

2013-12-01

Many engineering undergraduates begin third-level education with significant deficiencies in their core mathematical skills. Every year, in the Dublin Institute of Technology, a diagnostic test is given to incoming first-year students, consistently revealing problems in basic mathematics. It is difficult to motivate students to address these problems; instead, they struggle through their degree, carrying a serious handicap of poor core mathematical skills, as confirmed by exploratory testing of final year students. In order to improve these skills, a pilot project was set up in which a 'module' in core mathematics was developed. The course material was basic, but 90% or higher was required to pass. Students were allowed to repeat this module throughout the year by completing an automated examination on WebCT populated by a question bank. Subsequent to the success of this pilot with third-year mechanical engineering students, the project was extended to five different engineering programmes, across three different year-groups. Full results and analysis of this project are presented, including responses to interviews carried out with a selection of the students involved.

Development of a Brazilian Portuguese adapted version of the Gap-Kalamazoo communication skills assessment form.

Science.gov (United States)

Amaral, Anna Beatriz C N; Rider, Elizabeth A; Lajolo, Paula P; Tone, Luiz G; Pinto, Rogerio M C; Lajolo, Marisa P; Calhoun, Aaron W

2016-12-11

The goal of this study was to translate, adapt and validate the items of the Gap-Kalamazoo Communication Skills Assessment Form for use in the Brazilian cultural setting. The Gap-Kalamazoo Communication Skills Assessment Form was translated into Portuguese by two independent bilingual Brazilian translators and was reconciled by a third bilingual healthcare professional. The translated text was then assessed for content using a modified Delphi technique and adjusted as needed to assure content validity. A total of nine phrases in the completed tool were adjusted. The final tool was then used to assess videotaped simulations as a means of validation.  Response process was assessed using exploratory factor analysis and internal structure was assessed via Cronbach's Alpha (internal consistency) and Intraclass Correlation (test-retest reliability and inter-rater reliability). One hundred and four (104) videotaped communication skills simulations were assessed by 38 subjects (6 staff physicians, 4 faculty physicians, 8 resident physicians, 4 professional actors with experience in simulation, and 16 other allied healthcare professionals). Measures of Internal consistency (Cronbach's alpha = 0.818) and test-retest reliability (intra-class correlation coefficient = 0.942) were high.  Exploratory factor analysis confirmed the uni-dimensionality of the instrument. Our results support the validity and reliability of the Brazilian Gap-Kalamazoo Communication Skills Assessment Form when used among Brazilian medical residents.  The Brazilian version of Gap-Kalamazoo Communication Skills Assessment Form was found to be adequate both in the linguistic and technical aspects.  The use of this instrument in Brazilian medical education can enhance the assessment of physician-patient-team relationships on an ongoing basis.

Customisation of an instrument to assess anaesthesiologists' non-technical skills.

Science.gov (United States)

Jepsen, Rikke M H G; Spanager, Lene; Lyk-Jensen, Helle T; Dieckmann, Peter; Østergaard, Doris

2015-02-22

The objectives of the study were to identify Danish anaesthesiologists' non-technical skills and to customise the Scottish-developed Anaesthetists' Non-Technical Skills instrument for Danish anaesthesiologists. Six semi-structured group interviews were conducted with 31 operating room team members: anaes-thesiologists, nurse anaesthetists, surgeons, and scrub nurses. Interviews were transcribed verbatim and analysed using directed content analysis. Anaesthesiologists' non-technical skills were identified, coded, and sorted using the original instrument as a basis. The resulting prototype instrument was discussed with anaesthesiologists from 17 centres to ensure face validity. Interviews lasted 46-67 minutes. Identified examples of anaesthesiologists' good or poor non-technical skills fit the four categories in the original instrument: situation awareness; decision making; team working; and task management. Anaesthesiologists' leadership role in the operating room was emphasised: the original 'Task Management' category was named 'Leadership'. One new element, 'Demonstrating self-awareness' was added under the category 'Situation Awareness'. Compared with the original instrument, half of the behavioural markers were new, which reflected that being aware of and communicating one's own abilities to the team; working systematically; and speaking up to avoid adverse events were important skills. The Anaesthetists' Non-Technical Skills instrument was customised to a Danish setting using the identified non-technical skills for anaesthesiologists and the original instrument as basis. The customised instrument comprises four categories and 16 underpinning elements supported by multiple behavioural markers. Identifying non-technical skills through semi-structured group interviews and analysing them using direct content analysis proved a useful method for customising an assessment instrument to another setting.

Getting Skills Right: Assessing and Anticipating Changing Skill Needs

Science.gov (United States)

OECD Publishing, 2016

2016-01-01

Digitalisation, globalisation, demographic shifts and other changes in work organisation are constantly reshaping skill needs. This can lead to persistent skill shortages and mismatch which are costly for individuals, firms and society in terms of lost wages and lower productivity and growth. These costs can be reduced through better assessment…

Innovative integrative bedside teaching model improves tutors’ self-assessments of teaching skills and attitudes

Directory of Open Access Journals (Sweden)

Itai Gat

2016-02-01

Full Text Available Introduction: Patient bedside is the ideal setting for teaching physical examination, medical interviewing, and interpersonal skills. Herein we describe a novel model for bedside teaching (BST practiced during tutor training workshop and its resulting effect on practitioners’ self assessment of teaching skills and perceptions. Methods: One-day tutor training workshop included theoretical knowledge supplementation regarding tutors’ roles as well as implementing practical tools for clinical education, mainly BST model. The model, which emphasizes simultaneous clinical and communication teaching in a stepwise approach, was practiced by consecutive simulations with a gradual escalation of difficulty and adjusted instruction approaches. Pre- and post-workshop-adjusted questionnaires using a Likert scale of 1 to 4 were completed by participants and compared. Results: Analysis was based on 25 out of 48 participants who completed both questionnaires. Significantly improved teaching skills were demonstrated upon workshop completion (mean 3.3, SD 0.5 compared with pre-training (mean 2.6, SD 0.6; p<0.001 with significant increase in most examined parameters. Significantly improved tutor's roles internalization was demonstrated after training completion (mean 3.7, SD 0.3 compared with pre-workshop (mean 3.5 SD 0.5; p=0.002. Discussion: Successful BST involves combination of clinical and communication skills. BST model practiced during the workshop may contribute to improved teaching skills in this challenging environment.

Validity and reliability of a short questionnaire for assessing the impact of cooking skills interventions.

Science.gov (United States)

Barton, K L; Wrieden, W L; Anderson, A S

2011-12-01

Food skills programmes are widely used as a means to improve confidence in food preparation, the use of basic food skills and food selections amongst low income communities. However, the impact of such interventions are rarely evaluated as a result of a lack of validated assessment tools appropriate for use within this target group. A two-page questionnaire utilising a closed-question format was designed based on key domains known to be influenced by cooking skills programmes. Content validity was assessed by a panel of public health experts and face validity by individuals, typical of those who may attend cooking skills classes. Internal and repeat reliability were assessed with groups of adults attending community-based classes. The feasibility of using the tool in community settings was also assessed. The draft questionnaire was amended as appropriate subsequent to content and face validity testing. Cronbach's alpha for confidence and knowledge sections was 0.86 and 0.84, respectively, indicating good internal consistency. Spearman correlation coefficients for repeat reliability testing between time 1 and time 2 for each item were in the range 0.46-0.91 (all significant at P cooking skills interventions that could be utilised in the development and evaluation of multicentre cooking skills interventions. © 2011 The Authors. Journal of Human Nutrition and Dietetics © 2011 The British Dietetic Association Ltd.

Network Analysis of Conversation Data for Engineering Professional Skills Assessment. Research Report. ETS RR-17-59

Science.gov (United States)

Zhu, Mengxiao; Zhang, Mo

2017-01-01

In this paper, we examine the student group discussion processes in a scenario-based assessment of engineering professional skills called Engineering Professional Skills Assessment (EPSA). In the assessment, the students were evaluated through a discussion on a scenario related to an engineering problem with no clear-cut solution. We applied…

Guidelines for Assessing Job-Related Social Skills of Mildly Handicapped Students.

Science.gov (United States)

Bullis, Michael; Foss, Gilbert

1986-01-01

Effective assessment of job-related social skills of mildly handicapped workers should consider three components of social competence: context, cognition, and performance. A comprehensive profile of social competence is best obtained through use of several assessment tools, such as: rating scales; role play; problem-solving tests; applied…

Reliability and responsiveness of the Self-Efficacy in Assessing, Training and Spotting wheelchair skills (SEATS) outcome measure.

Science.gov (United States)

Rushton, Paula W; Smith, Emma M; Miller, William C; Kirby, R Lee; Daoust, Geneviève

2018-01-31

The aim of this study was to evaluate the internal consistency, test-retest reliability and responsiveness of the Self-Efficacy in Assessing, Training and Spotting manual wheelchair skills (SEATS-M) and Self-Efficacy in Assessing, Training and Spotting power wheelchair skills (SEATS-P). A 2-week test-retest design was used with a convenience sample of occupational and physical therapists who worked at a provincial rehabilitation centre (inpatient and outpatient services). Sixteen participants completed the SEATS-M and 18 participants completed the SEATS-P. For the SEATS-M assessment, training, spotting and documentation sections, Cronbach's alpha coefficients ranged from 0.90 to 0.97, the 2-week intraclass correlation coefficients (ICC 1,1 ) ranged from 0.81 to 0.95, the standard error of measurements (SEM) ranged from 5.06 to 8.70 and the smallest real differences (SRD) ranged from 6.24 to 8.18. For the SEATS-P assessment, training, spotting and documentation sections, Cronbach's alpha coefficients ranged from 0.83 to 0.92, the ICCs ranged from 0.72 to 0.86, the SEMs ranged from 4.54 to 8.91 and the SRDs ranged from 5.90 to 8.27. There is preliminary evidence that both the SEATS-M and the SEATS-P have high internal consistency, good test-retest reliability and support for responsiveness. These tools can be used in evaluating clinician self-efficacy with assessing, training, spotting and documenting wheelchair skills included on the Wheelchair Skills Test. Implications for Rehabilitation There is preliminary evidence that the SEATS-M and SEATS-P are reliable and responsive outcome measures that can be used to evaluate the self-efficacy of clinicians to administer the Wheelchair Skills Program. Measurement of clinicians' self-efficacy in this area of practice may enable an enhanced understanding of the areas in which clinicians lack self-efficacy, thereby informing the development of improved knowledge translation interventions.

The teaching of physical assessment skills in pre-registration nursing programmes in Australia: issues for nursing education.

Science.gov (United States)

Birks, Melanie; James, Ainsley; Chung, Catherine; Cant, Robyn; Davis, Jenny

2014-01-01

Health assessment is a fundamental aspect of the professional nursing role. The teaching of skills in physical assessment is therefore a large component of pre-registration nursing programmes. As the nursing curriculum becomes more crowded with what is deemed to be essential content, there is a need to rationalise what is taught in preparatory nursing programmes to ensure readiness for practice. The study outlined in this paper, as part of a larger project, explored the teaching of physical assessment skills in pre-registration nursing programmes across Australia. Fifty-three academics completed the 121 item online survey, indicating whether each skill was taught with practice, taught with no practice or not taught at all. The results suggest that only half the skills were being taught by more than 80% of the academics and 23 skills (19%) were taught by more than 90%. Of the 121 skills commonly taught--69 skills (57%) were taught with student practice and 29 (24%) were taught with no student practice. The results of this study raise questions about the teaching of physical assessment in pre-registration nursing programmes. The suggestion is not that skills that are used regularly or infrequently should be removed from the curriculum, rather, the authors propose that consideration be given to whether the teaching of skills that are never likely to be used is occurring at the expense of comprehensive mastery of core skills.

Self-assessment of intercultural communication skills: a survey of physicians and medical students in Geneva, Switzerland

Directory of Open Access Journals (Sweden)

Perneger Thomas

2011-09-01

Full Text Available Abstract Background Physicians working with multicultural populations need to know how to elicit the patient's understanding of the illness; determine the patient's sociocultural context and identify any issues that might affect care; communicate effectively across patient-provider social and cultural differences; and collaborate effectively with an interpreter. Skills self-assessment can contribute to identifying training needs and monitoring skills development in these areas. Methods As part of a larger study exploring the knowledge, attitudes and practices of Geneva physicians and medical students regarding the care of immigrant patients, we asked respondents to self-rate their ability to perform a range of common yet challenging intercultural communication tasks. Results Overall, respondents rated themselves less competent at intercultural tasks than at basic medical skills and less competent at specific intercultural communication skills than at general intercultural skills. Qualified doctors (as opposed to students, those with greater interest in caring for immigrants, and those who rarely encountered difficulties with immigrants rated themselves significantly more competent for all clinical tasks. Having a higher percentage of immigrant patients and previous cultural competence training predicted greater self-rated intercultural communication skills. Conclusion Our self-assessment results suggest that students and physicians should be provided with the opportunity to practice intercultural skills with immigrant patients as part of their cultural competence training. To strengthen the validity of self-assessment measures, they should ideally be combined with more objective methods to assess actual skills.

Associations between skill perceptions and young children's actual fundamental movement skills.

Science.gov (United States)

Liong, Grace H E; Ridgers, Nicola D; Barnett, Lisa M

2015-04-01

Given that children with low movement skill competence engage in less physical activity, it is important to understand how children's perceptions relate to actual movement competence. This study examined relationships between (i) children's self-perception and objective assessments of their movement skills (object control and locomotor) and (ii) parents' perceptions of the children's movement skills and objective assessment. Children's skill perceptions were assessed using the Pictorial Scale of Perceived Movement Skill Competence for Young Children. Parent perceptions of their child's skills were assessed using a modified version of this instrument. The Test of Gross Motor Development-2nd edition assessed children's skills objectively. Participants were 136 Australian children (51% boys; M=6.5 yr., SD=1.1) and 133 parents. Regression analyses (by sex) examined the relationship between perceptions and children's scores for actual skilled performance. Boys' perceptions were associated with their actual object control ability. Parents accurately perceived boys' object control ability and girls' locomotor ability, but not the reverse. This suggests interventions aiming to improve children's movement skills could target parents and be designed to teach parents how to recognize good and poor skill performance in their children.

Improving communication in cancer pain management nursing: a randomized controlled study assessing the efficacy of a communication skills training program.

Science.gov (United States)

Canivet, Delphine; Delvaux, Nicole; Gibon, Anne-Sophie; Brancart, Cyrielle; Slachmuylder, Jean-Louis; Razavi, Darius

2014-12-01

Effective communication is needed for optimal cancer pain management. This study assessed the efficacy of a general communication skills training program for oncology nurses on communication about pain management. A total of 115 nurses were randomly assigned to a training group (TG) or control group (CG). The assessment included the recording of interviews with a simulated cancer patient at baseline for both groups and after training (TG) or 3 months after baseline (CG). Two psychologists rated the content of interview transcripts to assess cancer pain management communication. Group-by-time effects were measured using a generalized estimating equation. Trained nurses asked the simulated patient more questions about emotions associated with pain (relative rate [RR] = 4.28, p = 0.049) and cognitions associated with pain treatment (RR = 3.23, p management (RR = 0.40, p = 0.006) compared with untrained nurses. The general communication skills training program improved only a few of the communication strategies needed for optimal cancer pain management in nursing. General communication skills training programs should be consolidated using specific modules focusing on communication skills related to cancer pain management.

Validation of a novel venous duplex ultrasound objective structured assessment of technical skills for the assessment of venous reflux.

Science.gov (United States)

Jaffer, Usman; Normahani, Pasha; Lackenby, Kimberly; Aslam, Mohammed; Standfield, Nigel J

2015-01-01

Duplex ultrasound measurement of reflux time is central to the diagnosis of venous incompetence. We have developed an assessment tool for Duplex measurement of venous reflux for both simulator and patient-based training. A novel assessment tool, Venous Duplex Ultrasound Assessment of Technical Skills (V-DUOSATS), was developed. A modified DUOSATS was used for simulator training. Participants of varying skill level were invited to viewed an instructional video and were allowed ample time to familiarize with the Duplex equipment. Attempts made by the participants were recorded and independently assessed by 3 expert assessors and 5 novice assessors using the modified V-DUOSATS. "Global" assessment was also done by expert assessors on a 4-point Likert scale. Content, construct, and concurrent validities as well as reliability were evaluated. Content and construct validity as well as reliability were demonstrated. Receiver operator characteristic analysis-established cut points of 19/22 and 21/30 were most appropriate for simulator and patient-based assessment, respectively. We have validated a novel assessment tool for Duplex venous reflux measurement. Further work is required to establish transference validity of simulator training to improve skill in scanning patients. We have developed and validated V-DUOSATS for simulator training. Copyright © 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Evaluating first-year nursing students' ability to self-assess psychomotor skills using videotape.

Science.gov (United States)

Watts, Wilda Ellen; Rush, Kathy; Wright, Marjorie

2009-01-01

Developing confidence in self-assessment is an important skill in becoming a self-regulated learner. This article describes the process undertaken by a group of educators of incorporating self-assessment in combination with psychomotor skill development with freshman students. Students were videotaped performing a wound-dressing change; the videotaping was immediately followed by a self-assessment of their performance using a faculty-generated checklist. Comparison of faculty and student ratings revealed the tendency for students to overrate their performance and identified discordance between students and faculty on several steps of the procedure. These evaluation findings are discussed and future directions explored.

Mathematical Skills and Motor Life Skills in Toddlers: Do Differences in Mathematical Skills Reflect Differences in Motor Skills?

Science.gov (United States)

Reikerås, Elin; Moser, Thomas; Tønnessen, Finn Egil

2017-01-01

This study examines possible relations between early mathematical skills and motor life skills in 450 toddlers aged two years and nine months. The study employs baseline data from the longitudinal Stavanger Project--The Learning Child. The children's mathematical skills and motor life skills were assessed by structured observation in the natural…

Self-confidence of medical students in performing clinical skills acquired during their surgical rotation. Assessing clinical skills education in Kuwait.

Science.gov (United States)

Karim, Jumanah A; Marwan, Yousef A; Dawas, Ahmed M; Akhtar, Saeed

2012-12-01

To assess the self-confidence of clinical years` medical students in performing clinical skills/procedures. A cross-sectional study was conducted in April 2011 at the Department of Surgery, Faculty of Medicine, Health Sciences Center, Kuwait University, Safat, Kuwait. A questionnaire was used to collect data from students who had completed their surgical rotation of their first clinical year. The students reported their level of self-confidence in performing specific skills/procedures related to that rotation. Data were presented using frequencies and percentages. A total score of confidence was calculated for each student. The Mann-Whitney and Kruskal-Wallis tests were used to assess the association between the students` sociodemographic characteristics and confidence score. Of the 122 students invited to participate in the study, only 15 (12.3%) declined to comply. Most students reported high confidence level (more than 75%) in performing 7 of the 13 history taking/physical examination skills, and 2 of the 39 diagnostic/treatment procedure skills. The highest confidence level was in performing abdominal examination, while the lowest level was in care of Jackson-Pratt drain site and emptying the drain bulb. The total confidence score was significantly higher among males (p=0.021), and students with higher monthly income (p=0.002). Medical students appeared to have poor self-confidence in performing clinical skills/procedures. Curriculum planners should explore potential reasons, and methods for the improvement of confidence level among medical students in performing skills/procedures they were expected to learn during their surgical rotation.

Feedback Simulation for Acupressure Training and Skill Assessment.

Science.gov (United States)

Noll, Eric; Romeiser, Jamie; Shodhan, Shivam; Madariaga, Maria Cecilia; Guo, Xiaojun; Rizwan, Sabeen; Al-Bizri, Ehab; Bennett-Guerrero, Elliott

2017-08-01

Previous acupressure studies have yielded varying results. This could be due to differences in the amount of pressure applied to the acupressure point (acupoint) by study personnel within a study as well as between studies. Standardizing the level of pressure applied at an acupoint could improve clinical care and future research. As part of an ongoing randomized clinical trial of postoperative acupressure, five trainees were asked to perform 2 minutes of acupressure and light touch sessions on a simulator. The applied weight was recorded every minute. Individual skill assessment was performed using cumulative sum analysis. Six pretraining and 20 posttraining measurements in each acupressure and light touch group were compared with an expert's simulation values. Before training (baseline), there was significant difference in applied weight (grams) between the expert [5705 (636)] and five trainees [2998 (798), P = 0.004]. Four of the five trainees crossed the lower decision limit assessing proficiency in the acupressure group, and all five trainees were successful in the light touch group. The trainees' average number of measurements needed to cross the lower decision limit (H0), that is, defining that an individual failure rate does not statistically differ from the acceptable failure rate, was 21.3 measurements for acupressure. After this feedback simulation, trainees' scores showed no significant difference (P > 0.05) when assessed against the expert. Feedback simulation for acupressure training and skill assessment, evaluated by cumulative sum analysis, may help in improving the standardization of acupressure therapy performed during clinical practice or research.

The Assessment of 21st Century Skills in Industrial and Organizational Psychology: Complex and Collaborative Problem Solving

OpenAIRE

Neubert, Jonas; Mainert, Jakob; Kretzschmar, André; Greiff, Samuel

2015-01-01

In the current paper, we highlight why and how industrial and organizational psychology can take advantage of research on 21st century skills and their assessment. We present vital theoretical perspectives, a suitable framework for assessment, and exemplary instruments with a focus on advances in the assessment of Human Capital. Specifically, Complex Problem Solving (CPS) and Collaborative Problem Solving (ColPS) are two transversal skills (i.e., skills that span multiple domains) that are...

Designing an Automated Assessment of Public Speaking Skills Using Multimodal Cues

Science.gov (United States)

Chen, Lei; Feng, Gary; Leong, Chee Wee; Joe, Jilliam; Kitchen, Christopher; Lee, Chong Min

2016-01-01

Traditional assessments of public speaking skills rely on human scoring. We report an initial study on the development of an automated scoring model for public speaking performances using multimodal technologies. Task design, rubric development, and human rating were conducted according to standards in educational assessment. An initial corpus of…

Objective structured assessment of nontechnical skills: Reliability of a global rating scale for the in-training assessment in the operating room.

Science.gov (United States)

Dedy, Nicolas J; Szasz, Peter; Louridas, Marisa; Bonrath, Esther M; Husslein, Heinrich; Grantcharov, Teodor P

2015-06-01

Nontechnical skills are critical for patient safety in the operating room (OR). As a result, regulatory bodies for accreditation and certification have mandated the integration of these competencies into postgraduate education. A generally accepted approach to the in-training assessment of nontechnical skills, however, is lacking. The goal of the present study was to develop an evidence-based and reliable tool for the in-training assessment of residents' nontechnical performance in the OR. The Objective Structured Assessment of Nontechnical Skills tool was designed as a 5-point global rating scale with descriptive anchors for each item, based on existing evidence-based frameworks of nontechnical skills, as well as resident training requirements. The tool was piloted on scripted videos and refined in an iterative process. The final version was used to rate residents' performance in recorded OR crisis simulations and during live observations in the OR. A total of 37 simulations and 10 live procedures were rated. Interrater agreement was good for total mean scores, both in simulation and in the real OR, with intraclass correlation coefficients >0.90 in all settings for average and single measures. Internal consistency of the scale was high (Cronbach's alpha = 0.80). The Objective Structured Assessment of Nontechnical Skills global rating scale was developed as an evidence-based tool for the in-training assessment of residents' nontechnical performance in the OR. Unique descriptive anchors allow for a criterion-referenced assessment of performance. Good reliability was demonstrated in different settings, supporting applications in research and education. Copyright © 2015 Elsevier Inc. All rights reserved.

Clinical skills temporal degradation assessment in undergraduate medical education.

Science.gov (United States)

Fisher, Joseph; Viscusi, Rebecca; Ratesic, Adam; Johnstone, Cameron; Kelley, Ross; Tegethoff, Angela M; Bates, Jessica; Situ-Lacasse, Elaine H; Adamas-Rappaport, William J; Amini, Richard

2018-01-01

Medical students' ability to learn clinical procedures and competently apply these skills is an essential component of medical education. Complex skills with limited opportunity for practice have been shown to degrade without continued refresher training. To our knowledge there is no evidence that objectively evaluates temporal degradation of clinical skills in undergraduate medical education. The purpose of this study was to evaluate temporal retention of clinical skills among third year medical students. This was a cross-sectional study conducted at four separate time intervals in the cadaver laboratory at a public medical school. Forty-five novice third year medical students were evaluated for retention of skills in the following three procedures: pigtail thoracostomy, femoral line placement, and endotracheal intubation. Prior to the start of third-year medical clerkships, medical students participated in a two-hour didactic session designed to teach clinically relevant materials including the procedures. Prior to the start of their respective surgery clerkships, students were asked to perform the same three procedures and were evaluated by trained emergency medicine and surgery faculty for retention rates, using three validated checklists. Students were then reassessed at six week intervals in four separate groups based on the start date of their respective surgical clerkships. We compared the evaluation results between students tested one week after training and those tested at three later dates for statistically significant differences in score distribution using a one-tailed Wilcoxon Mann-Whitney U-test for non-parametric rank-sum analysis. Retention rates were shown to have a statistically significant decline between six and 12 weeks for all three procedural skills. In the instruction of medical students, skill degradation should be considered when teaching complex technical skills. Based on the statistically significant decline in procedural skills noted

Information Literacy Follow-Through: Enhancing Preservice Teachers' Information Evaluation Skills through Formative Assessment

Science.gov (United States)

Seely, Sara Robertson; Fry, Sara Winstead; Ruppel, Margie

2011-01-01

An investigation into preservice teachers' information evaluation skills at a large university suggests that formative assessment can improve student achievement. Preservice teachers were asked to apply information evaluation skills in the areas of currency, relevancy, authority, accuracy, and purpose. The study used quantitative methods to assess…

The Dutch motor skills assessment as tool for talent development in table tennis : a reproducibility and validity study

NARCIS (Netherlands)

Faber, Irene R.; Nijhuis-Van der Sanden, Maria W. G.; Elferink-Gemser, Marije T.; Oosterveld, Frits G. J.

2015-01-01

A motor skills assessment could be helpful in talent development by estimating essential perceptuo-motor skills of young players, which are considered requisite to develop excellent technical and tactical qualities. The Netherlands Table Tennis Association uses a motor skills assessment in their

The Dutch motor skills assessment as tool for talent development in table tennis: a reproducibility and validity study

NARCIS (Netherlands)

Faber, I.R.; Nijhuis-Van der Sanden, M.W.G.; Elferink-Gemser, M.T.; Oosterveld, F.G.

2015-01-01

A motor skills assessment could be helpful in talent development by estimating essential perceptuo-motor skills of young players, which are considered requisite to develop excellent technical and tactical qualities. The Netherlands Table Tennis Association uses a motor skills assessment in their

The Dutch motor skills assessment as tool for talent development in table tennis: a reproducibility and validity study

NARCIS (Netherlands)

Faber, I.R.; Nijhuis-Van der Sanden, M.W.; Elferink-Gemser, M.T.; Oosterveld, F.G.J.

2014-01-01

Abstract A motor skills assessment could be helpful in talent development by estimating essential perceptuo-motor skills of young players, which are considered requisite to develop excellent technical and tactical qualities. The Netherlands Table Tennis Association uses a motor skills assessment in

"Reflection-Before-Practice" Improves Self-Assessment and End-Performance in Laparoscopic Surgical Skills Training.

Science.gov (United States)

Ganni, Sandeep; Botden, Sanne M B I; Schaap, Dennis P; Verhoeven, Bas H; Goossens, Richard H M; Jakimowicz, Jack J

To establish whether a systematized approach to self-assessment in a laparoscopic surgical skills course improves accordance between expert- and self-assessment. A systematic training course in self-assessment using Competency Assessment Tool was introduced into the normal course of evaluation within a Laparoscopic Surgical Skills training course for the test group (n = 30). Differences between these and a control group (n = 30) who did not receive the additional training were assessed. Catharina Hospital, Eindhoven, The Netherlands (n = 27), and GSL Medical College, Rajahmundry, India (n = 33). Sixty postgraduate year 2 and 3 surgical residents who attended the 2-day Laparoscopic Surgical Skills grade 1 level 1 curriculum were invited to participate. The test group (n = 30) showed better accordance between expert- and self-assessment (difference of 1.5, standard deviation [SD] = 0.2 versus 3.83, SD = 0.6, p = 0.009) as well as half the number (7 versus 14) of cases of overreporting. Furthermore, the test group also showed higher overall mean performance (mean = 38.1, SD = 0.7 versus mean = 31.8, SD = 1.0, p assessment can be viewed as responsible for this and can be seen as "reflection-before-practice" within the framework of reflective practice as defined by Donald Schon. Our results suggest that "reflection-before-practice" in implementing self-assessment is an important step in the development of surgical skills, yielding both better understanding of one's strengths and weaknesses and also improving overall performance. Copyright © 2017 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.

Validation of a novel duplex ultrasound objective structured assessment of technical skills (DUOSATS) for arterial stenosis detection.

Science.gov (United States)

Jaffer, U; Singh, P; Pandey, V A; Aslam, M; Standfield, N J

2014-01-01

Duplex ultrasound facilitates bedside diagnosis and hence timely patient care. Its uptake has been hampered by training and accreditation issues. We have developed an assessment tool for Duplex arterial stenosis measurement for both simulator and patient based training. A novel assessment tool: duplex ultrasound assessment of technical skills was developed. A modified duplex ultrasound assessment of technical skills was used for simulator training. Novice, intermediate experience and expert users of duplex ultrasound were invited to participate. Participants viewed an instructional video and were allowed ample time to familiarize with the equipment. Participants' attempts were recorded and independently assessed by four experts using the modified duplex ultrasound assessment of technical skills. 'Global' assessment was also done on a four point Likert scale. Content, construct and concurrent validity as well as reliability were evaluated. Content and construct validity as well as reliability were demonstrated. The simulator had good satisfaction rating from participants: median 4; range 3-5. Receiver operator characteristic analysis has established a cut point of 22/ 34 and 25/ 40 were most appropriate for simulator and patient based assessment respectively. We have validated a novel assessment tool for duplex arterial stenosis detection. Further work is underway to establish transference validity of simulator training to improved skill in scanning patients. We have developed and validated duplex ultrasound assessment of technical skills for simulator training.

Protocols for the Investigation of Information Processing in Human Assessment of Fundamental Movement Skills.

Science.gov (United States)

Ward, Brodie J; Thornton, Ashleigh; Lay, Brendan; Rosenberg, Michael

2017-01-01

Fundamental movement skill (FMS) assessment remains an important tool in classifying individuals' level of FMS proficiency. The collection of FMS performances for assessment and monitoring has remained unchanged over the last few decades, but new motion capture technologies offer opportunities to automate this process. To achieve this, a greater understanding of the human process of movement skill assessment is required. The authors present the rationale and protocols of a project in which they aim to investigate the visual search patterns and information extraction employed by human assessors during FMS assessment, as well as the implementation of the Kinect system for FMS capture.

A decade of imaging surgeons' brain function (part II): A systematic review of applications for technical and nontechnical skills assessment.

Science.gov (United States)

Modi, Hemel Narendra; Singh, Harsimrat; Yang, Guang-Zhong; Darzi, Ara; Leff, Daniel Richard

2017-11-01

Functional neuroimaging technologies enable assessment of operator brain function and can deepen our understanding of skills learning, ergonomic optima, and cognitive processes in surgeons. Although there has been a critical mass of data detailing surgeons' brain function, this literature has not been reviewed systematically. A systematic search of original neuroimaging studies assessing surgeons' brain function and published up until November 2016 was conducted using Medline, Embase, and PsycINFO databases. Twenty-seven studies fulfilled the inclusion criteria, including 3 feasibility studies, 14 studies exploring the neural correlates of technical skill acquisition, and the remainder investigating brain function in the context of intraoperative decision-making (n = 1), neurofeedback training (n = 1), robot-assisted technology (n = 5), and surgical teaching (n = 3). Early stages of learning open surgical tasks (knot-tying) are characterized by prefrontal cortical activation, which subsequently attenuates with deliberate practice. However, with complex laparoscopic skills (intracorporeal suturing), prefrontal cortical engagement requires substantial training, and attenuation occurs over a longer time course, after years of refinement. Neurofeedback and interventions that improve neural efficiency may enhance technical performance and skills learning. Imaging surgeons' brain function has identified neural signatures of expertise that might help inform objective assessment and selection processes. Interventions that improve neural efficiency may target skill-specific brain regions and augment surgical performance. Copyright © 2017 Elsevier Inc. All rights reserved.

Teaching and assessing consultation skills: an evaluation of a South ...

African Journals Online (AJOL)

Teaching and assessing consultation skills: an evaluation of a South African workshop ... of this work is unsupervised consultations with patients suffering from new or ... Their attitudes to the LAP and the forms of teaching that its use promotes ...

Process Skill Assessment Instrument: Innovation to measure student’s learning result holistically

Science.gov (United States)

Azizah, K. N.; Ibrahim, M.; Widodo, W.

2018-01-01

Science process skills (SPS) are very important skills for students. However, the fact that SPS is not being main concern in the primary school learning is undeniable. This research aimed to develop a valid, practical, and effective assessment instrument to measure student’s SPS. Assessment instruments comprise of worksheet and test. This development research used one group pre-test post-test design. Data were obtained with validation, observation, and test method to investigate validity, practicality, and the effectivenss of the instruments. Results showed that the validity of assessment instruments is very valid, the reliability is categorized as reliable, student SPS activities have a high percentage, and there is significant improvement on student’s SPS score. It can be concluded that assessment instruments of SPS are valid, practical, and effective to be used to measure student’s SPS result.

A Novel Clinical-Simulated Suture Education for Basic Surgical Skill: Suture on the Biological Tissue Fixed on Standardized Patient Evaluated with Objective Structured Assessment of Technical Skill (OSATS) Tools.

Science.gov (United States)

Shen, Zhanlong; Yang, Fan; Gao, Pengji; Zeng, Li; Jiang, Guanchao; Wang, Shan; Ye, Yingjiang; Zhu, Fengxue

2017-06-21

Clinical-simulated training has shown benefit in the education of medical students. However, the role of clinical simulation for surgical basic skill training such as suturing techniques remains unclear. Forty-two medical students were asked to perform specific suturing tasks at three stations with the different settings within four minutes (Station 1: Synthetic suture pad fixed on the bench, Station 2: Synthetic suture pad fixed on the standardized patient, Station 3: Pig skin fixed on the standardized patient); the OSATS (Objective Structured Assessment of Technical Skill) tool was used to evaluate the performance of students. A questionnaire was distributed to the students following the examination. Mean performance score of Station 3 was significant lower than that of Station 1 and 2 in the general performance including tissue handling, time, and motion. The suturing techniques of students at Station 2 and 3 were not as accurate as that at Station 1. Inappropriate tension was applied to the knot at Station 2 compared with Station 1 and 3. On the questionnaire, 93% of students considered clinical-simulated training of basic surgical skills was necessary and may increase their confidence in future clinical work as surgeons; 98% of students thought the assessment was more objective when OSATS tool was used for evaluation. Clinical simulation examination assessed with OSATS might throw a novel light on the education of basic surgical skills and may be worthy of wider adoption in the surgical education of medical students.

Toward an objective assessment of technical skills: a national survey of surgical program directors in Saudi Arabia

Directory of Open Access Journals (Sweden)

Alkhayal A

2012-10-01

Full Text Available Abdullah Alkhayal,1 Shahla Aldhukair,2 Nahar Alselaim,1 Salah Aldekhayel,1 Sultan Alhabdan,1 Waleed Altaweel,3 Mohi Elden Magzoub,4 Mohammed Zamakhshary1,21Department of Surgery, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia; 2Public Health Section, King Abdullah International Medical Research Center, Riyadh, Saudi Arabia; 3Urology Department, King Faisal Specialist Hospital and Research Center, Riyadh, Saudi Arabia; 4Department of Medical Education, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi ArabiaBackground: After almost a decade of implementing competency-based programs in postgraduate training programs, the assessment of technical skills remains more subjective than objective. National data on the assessment of technical skills during surgical training are lacking. We conducted this study to document the assessment tools for technical skills currently used in different surgical specialties, their relationship with remediation, the recommended tools from the program directors’ perspective, and program directors’ attitudes toward the available objective tools to assess technical skills.Methods: This study was a cross-sectional survey of surgical program directors (PDs. The survey was initially developed using a focus group and was then sent to 116 PDs. The survey contains demographic information about the program, the objective assessment tools used, and the reason for not using assessment tools. The last section discusses the recommended tools to be used from the PDs’ perspective and the PDs’ attitude and motivation to apply these tools in each program. The associations between the responses to the assessment questions and remediation were statistically evaluated.Results: Seventy-one (61% participants responded. Of the respondents, 59% mentioned using only nonstandardized, subjective, direct observation for technical skills assessment. Sixty percent use only summative

Large-scale Assessment Yields Evidence of Minimal Use of Reasoning Skills in Traditionally Taught Classes

Science.gov (United States)

Thacker, Beth

2017-01-01

Large-scale assessment data from Texas Tech University yielded evidence that most students taught traditionally in large lecture classes with online homework and predominantly multiple choice question exams, when asked to answer free-response (FR) questions, did not support their answers with logical arguments grounded in physics concepts. In addition to a lack of conceptual understanding, incorrect and partially correct answers lacked evidence of the ability to apply even lower level reasoning skills in order to solve a problem. Correct answers, however, did show evidence of at least lower level thinking skills as coded using a rubric based on Bloom's taxonomy. With the introduction of evidence-based instruction into the labs and recitations of the large courses and in a small, completely laboratory-based, hands-on course, the percentage of correct answers with correct explanations increased. The FR format, unlike other assessment formats, allowed assessment of both conceptual understanding and the application of thinking skills, clearly pointing out weaknesses not revealed by other assessment instruments, and providing data on skills beyond conceptual understanding for course and program assessment. Supported by National Institutes of Health (NIH) Challenge grant #1RC1GM090897-01.

Assessment and Teaching of Science Skills: Whole of Programme Perceptions of Graduating Students

Science.gov (United States)

Hodgson, Yvonne; Varsavsky, Cristina; Matthews, Kelly E.

2014-01-01

This study reports on science student perceptions of their skills (scientific knowledge, oral communication, scientific writing, quantitative skills, teamwork and ethical thinking) as they approach graduation. The focus is on which teaching activities and assessment tasks over the whole programme of study students thought utilised each of the six…

The Dutch motor skills assessment as tool for talent development in table tennis: a reproducibility and validity study.

Science.gov (United States)

Faber, Irene R; Nijhuis-Van Der Sanden, Maria W G; Elferink-Gemser, Marije T; Oosterveld, Frits G J

2015-01-01

A motor skills assessment could be helpful in talent development by estimating essential perceptuo-motor skills of young players, which are considered requisite to develop excellent technical and tactical qualities. The Netherlands Table Tennis Association uses a motor skills assessment in their talent development programme consisting of eight items measuring perceptuo-motor skills specific to table tennis under varying conditions. This study aimed to investigate this assessment regarding its reproducibility, internal consistency, underlying dimensions and concurrent validity in 113 young table tennis players (6-10 years). Intraclass correlation coefficients of six test items met the criteria of 0.7 with coefficients of variation between 3% and 8%. Cronbach's alpha valued 0.853 for internal consistency. The principal components analysis distinguished two conceptually meaningful factors: "ball control" and "gross motor function." Concurrent validity analyses demonstrated moderate associations between the motor skills assessment's results and national ranking; boys r = -0.53 (P motor skills assessment seems to be a reproducible, objective part of a talent development programme, more longitudinal studies are required to investigate its predictive validity.

The use of the "Objective Structured Assessment of Technical Skills" as an Assessment Tool Among Danish Vascular Surgeons in Training

DEFF Research Database (Denmark)

Lladó Grove, Gabriela; Langager Høgh, Annette; Nielsen, Judith

2015-01-01

OBJECTIVE: The concept of the Objective Structured Assessment of Technical Skills (OSATS) is to quantify surgical skills in an objective way and, thereby, produce an additional procedure-specific assessment tool. Since 2005, a 2-day practical course for upcoming specialist registrars in vascular...... surgery has been obligatory. The aim of this study is to describe the results from a tailored OSATS test as a tool for the evaluation of practical skills during an intensive training session in a simple simulator box for vascular anastomoses. METHOD: Between 2005 and 2013, we registered the OSATS scores......, or the experience with vascular anastomoses and outcomes. CONCLUSION: OSATS is a valuable tool for evaluating the advancement of technical skills during an intensive practical course in performing vascular anastomoses. (C) 2015 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights...

Instrument Motion Metrics for Laparoscopic Skills Assessment in Virtual Reality and Augmented Reality.

Science.gov (United States)

Fransson, Boel A; Chen, Chi-Ya; Noyes, Julie A; Ragle, Claude A

2016-11-01

To determine the construct and concurrent validity of instrument motion metrics for laparoscopic skills assessment in virtual reality and augmented reality simulators. Evaluation study. Veterinarian students (novice, n = 14) and veterinarians (experienced, n = 11) with no or variable laparoscopic experience. Participants' minimally invasive surgery (MIS) experience was determined by hospital records of MIS procedures performed in the Teaching Hospital. Basic laparoscopic skills were assessed by 5 tasks using a physical box trainer. Each participant completed 2 tasks for assessments in each type of simulator (virtual reality: bowel handling and cutting; augmented reality: object positioning and a pericardial window model). Motion metrics such as instrument path length, angle or drift, and economy of motion of each simulator were recorded. None of the motion metrics in a virtual reality simulator showed correlation with experience, or to the basic laparoscopic skills score. All metrics in augmented reality were significantly correlated with experience (time, instrument path, and economy of movement), except for the hand dominance metric. The basic laparoscopic skills score was correlated to all performance metrics in augmented reality. The augmented reality motion metrics differed between American College of Veterinary Surgeons diplomates and residents, whereas basic laparoscopic skills score and virtual reality metrics did not. Our results provide construct validity and concurrent validity for motion analysis metrics for an augmented reality system, whereas a virtual reality system was validated only for the time score. © Copyright 2016 by The American College of Veterinary Surgeons.

Assessment of Critical Business Skill Development by MBA Alumni

Science.gov (United States)

Glynn, Joseph G.; Wood, Gregory R.

2008-01-01

Six years of survey data were analyzed to assess, among other things, the degree to which an AACSB accredited graduate business program successfully developed student skills in a variety of areas deemed important for career success. The study illustrates a methodology institutions can use to respond to increasing demands for program evaluation and…

Development, initial reliability and validity testing of an observational tool for assessing technical skills of operating room nurses.

Science.gov (United States)

Sevdalis, Nick; Undre, Shabnam; Henry, Janet; Sydney, Elaine; Koutantji, Mary; Darzi, Ara; Vincent, Charles A

2009-09-01

The recent emergence of the Systems Approach to the safety and quality of surgical care has triggered individual and team skills training modules for surgeons and anaesthetists and relevant observational assessment tools have been developed. To develop an observational tool that captures operating room (OR) nurses' technical skill and can be used for assessment and training. The Imperial College Assessment of Technical Skills for Nurses (ICATS-N) assesses (i) gowning and gloving, (ii) setting up instrumentation, (iii) draping, and (iv) maintaining sterility. Three to five observable behaviours have been identified for each skill and are rated on 1-6 scales. Feasibility and aspects of reliability and validity were assessed in 20 simulation-based crisis management training modules for trainee nurses and doctors, carried out in a Simulated Operating Room. The tool was feasible to use in the context of simulation-based training. Satisfactory reliability (Cronbach alpha) was obtained across trainers' and trainees' scores (analysed jointly and separately). Moreover, trainer nurse's ratings of the four skills correlated positively, thus indicating adequate content validity. Trainer's and trainees' ratings did not correlate. Assessment of OR nurses' technical skill is becoming a training priority. The present evidence suggests that the ICATS-N could be considered for use as an assessment/training tool for junior OR nurses.

Further validation of the Danish version of the McGill Ingestive Skills Assessment (MISA-DK)

DEFF Research Database (Denmark)

Hansen, Tina

2014-01-01

Background/aims The McGill Ingestive Skill Assessment (MISA) for measuring dysphagic patients' functional performance during meals has been previously translated into Danish — the Danish McGill Ingestive Skill Assessment (MISA-DK) and this translated version validated. However, issues about......-DK was then tested using 102 videorecordings of geriatric patients' ingestive skill performance, and the data from the scale were examined using a second Rasch analysis. Results Initially, two of the six proposed subscales of the original MISA-DK failed to fit the Rasch model, and were removed. It was also necessary...

Self-assessment in laparoscopic surgical skills training: Is it reliable?

Science.gov (United States)

Ganni, Sandeep; Chmarra, Magdalena K; Goossens, Richard H M; Jakimowicz, Jack J

2017-06-01

The concept of self-assessment has been widely acclaimed for its role in the professional development cycle and self-regulation. In the field of medical education, self-assessment has been most used to evaluate the cognitive knowledge of students. The complexity of training and evaluation in laparoscopic surgery has previously acted as a barrier in determining the benefits self-assessment has to offer in comparison with other fields of medical education. Thirty-five surgical residents who attended the 2-day Laparoscopic Surgical Skills Grade 1 Level 1 curriculum were invited to participate from The Netherlands, India and Romania. The competency assessment tool (CAT) for laparoscopic cholecystectomy was used for self- and expert-assessment and the resulting distributions assessed. A comparison between the expert- and self-assessed aggregates of scores from the CAT agreed with previous studies. Uniquely to this study, the aggregates of individual sub-categories-'use of instruments'; 'tissue handling'; and errors 'within the component tasks' and the 'end product' from both self- and expert-assessments-were investigated. There was strong positive correlation (r s  > 0.5; p assessment in all categories with only the 'tissue handling' having a weaker correlation (r s  = 0.3; p = 0.04). The distribution of the mean of the differences between self-assessment and expert-assessment suggested no significant difference between the scores of experts and the residents in all categories except the 'end product' evaluation where the difference was significant (W = 119, p = 0.03). Self-assessment using the CAT form gives results that are consistently not different from expert-assessment when assessing one's proficiency in surgical skills. Areas where there was less agreement could be explained by variations in the level of training and understanding of the assessment criteria.

Condom-use skills checklist: a proxy for assessing condom-use knowledge and skills when direct observation is not possible.

Science.gov (United States)

Stanton, B; Deveaux, L; Lunn, S; Yu, S; Brathwaite, N; Li, X; Cottrell, L; Harris, C; Clemens, R; Marshall, S

2009-06-01

Because of the continued importance of correct condom-use in controlling the HIV epidemic and the limited availability of tools for assessing correct condom-use, methods for assessing condom-application skills, especially when direct observation is not feasible, are needed. Accordingly, in the context of a high-risk population (The Bahamas) for HIV, a 17-item scale--the Condom-use Skills Checklist (CUSC)--was developed for use among young adolescents and adults. The rationale and approach to developing the scale and some measures of internal consistency, construct validity, and criterion-related validity have been described. It is concluded that the scale offers a reasonable alternative to direct observation among older subjects and that further development may make it more useful among pre-adolescents.

Educational Needs Assessment Highlights Several Areas of Emphasis in Teaching Evidence-Based Medicine Skills to Physician Assistant Students.

Science.gov (United States)

Kuntz, Susan; Ali, Syed Haris; Hahn, Emily

2016-08-03

An assessment of educational needs is essential for curricular reform in medical education. Using the conceptual framework of needs assessment, this study aimed to determine which content should be emphasized in teaching evidence-based medicine (EBM) skills to physician assistant (PA) students. Key content areas were identified from the published literature and objectives for previous courses. A questionnaire-type needs assessment instrument was created and given to a graduating class of PA students (n = 21) at the University of North Dakota. The response format had two 5-option scales, one to assess current skill levels and the other to assess ideal skill levels. Means for each category were calculated, and a mean difference analysis was performed. An average mean difference of 0.5 was noted in 3 domains (information retrieval skills, writing skills, and overall gains), and a mean difference of 0.7 was noted in one domain (statistical skills). Items with a mean difference of ≥ 0.7 were identified for prioritization for curricular reform. Open-ended input from respondents substantiated the need for greater emphasis on these content areas. Several content areas related to EBM skills can be identified and prioritized through a systematically conducted educational needs assessment. This method can be used to identify discrepancies between the existing and ideal states of affairs in PA education.

Self-Assessment of Employability Skill Outcomes among Undergraduates and Alignment with Academic Ratings

Science.gov (United States)

Jackson, Denise

2014-01-01

Despite acknowledgement of the benefits of self-assessment in higher education, disparity between student and academic assessments, with associated trends in overrating and underrating, plagues its meaningful use, particularly as a tool for formal assessment. This study examines self-assessment of capabilities in certain employability skills in…

The Assessment of Comunication and Interaction Skills in Psychotic Patiens

Directory of Open Access Journals (Sweden)

Katayoun Khoushabi

2002-10-01

Full Text Available Objective: The main objective at study was to assess the communication and interaction of psychotic patient above age of 18 in 2000. Materials & Methods: The study sample consist of 80 psychotic patients (38 female and 42 male from Razi and Imam Hossein Hospital and Raad and Sina outpatient clinic, selected based on occupational therapist using the Assessment of Communication and Interaction Skills Questionnaire (ACIS. Statistical analyses of information have done by spearman coefficient correlation, Mann Whitney and Wilcoxon tests. Results: Spearman of correlation, mann-and wilcoxon Findings indicate: 1-The average grade of ACIS test in the group of schizophrenic patients were significantly lower than the group schizoaffective patients (P<0.00l. 2-The average of ACIS test in the group of inpatient was lower than the group of outpatient (p<0.00l. 3-No significant difference was found for the average grade of ACIS test in the group of male & female. Conclusion: The obtained results show the weakness of communication and interaction skills in schizophrenic patient and also inpatients. The obtained results of this investigation give useful information about the communication and interaction skill of patient to the occupational therapists but application of this instrument needs more studies.

Management of Radioactive Spills in Nuclear Medicine; Teaching and Assessing with Objectively Structured Assessment of Technical Skills

International Nuclear Information System (INIS)

Hussain, Riffat Parveen

2015-01-01

Routine work in nuclear medicine requires the careful elution of radioactivity and its subsequent, storage and handling. Though all effort is maintained to prevent any “spill” of this radioactivity, accidents are bound to happen. The response to this spill is a methodically worked out a plan that is written and adopted as a “standard operating procedure.” This protocol is taught to all involved in the area of working as a mock drill/apprenticeship model. No formal evaluation of learning is in place except for the mock drills. The objectively structured assessment of technical skills (OSATS) is a variation on the Objective Structured Clinical Examination, which is a form of workplace based assessment. The OSATS is cited in the Accreditation Council of Graduate Medical Education evaluation toolbox on the website as the most desirable evaluation tool for the patient care topics. It is the objective of this paper is to introduce the “OSATS” for teaching, and assessment of the learning, of the protocol for the management of radioactive spill. As a review of the literature on the subject failed to reveal any such teaching protocol/material/document for this important technical skill, we hope that it may act as a landmark for the development of teaching and assessment of other technical skills also

Promoting Assessment Efficacy through an Integrated System for Online Clinical Assessment of Practical Skills

Science.gov (United States)

Hay, Peter J.; Engstrom, Craig; Green, Anita; Friis, Peter; Dickens, Sue; Macdonald, Doune

2013-01-01

This paper presents evaluation outcomes from an externally funded research project involving the online clinical assessment of practical skills (eCAPS) using web-based video technologies within a university medical programme. eCAPS was implemented to trial this web-based approach for promoting the efficacy of "practical" skills…

Assessing Nurse Anaesthetists' Non-Technical Skills in the operating room.

Science.gov (United States)

Lyk-Jensen, H T; Jepsen, R M H G; Spanager, L; Dieckmann, P; Østergaard, D

2014-08-01

Incident reporting and fieldwork in operating rooms have shown that some of the errors that arise in anaesthesia relate to inadequate use of non-technical skills. To provide a tool for training and feedback on nurse anaesthetists' non-technical skills, this study aimed to adapt the Anaesthetists' Non-Technical Skills (ANTS) as a behavioural marker system for the formative assessment of nurse anaesthetists' non-technical skills in the operating room. A qualitative approach with focus group interviews was used to identify the non-technical skills of nurse anaesthetists in the operating room. The interview data were transcribed verbatim. Directed content analysis was used to code and sort data deductively into the ANTS categories: task management, team working, situation awareness and decision making. The prototype named Nurse Anaesthetists' Non-Technical Skills (N-ANTS) was presented and discussed in a group of subject matter experts to ensure face validity. The N-ANTS system consists of the same four categories as ANTS and 15 underlying elements. Three to five good and poor behavioural markers for each element were identified. The headings and definitions of the categories and elements were adjusted to encompass the behavioural markers in N-ANTS. The differences that emerged mainly reflected statements regarding the establishment of role, competence, and task delegation. A behavioural marker system, N-ANTS, for nurse anaesthetists was adapted from a behavioural marker system, ANTS, for anaesthesiologists. © 2014 The Acta Anaesthesiologica Scandinavica Foundation. Published by John Wiley & Sons Ltd.

Otologic Skills Training

DEFF Research Database (Denmark)

Wiet, Gregory J; Sørensen, Mads Sølvsten; Andersen, Steven Arild Wuyts

2017-01-01

This article presents a summary of the current simulation training for otologic skills. There is a wide variety of educational approaches, assessment tools, and simulators in use, including simple low-cost task trainers to complex computer-based virtual reality systems. A systematic approach...

Can the 'Assessment Drives Learning' effect be detected in clinical skills training? - Implications for curriculum design and resource planning

Science.gov (United States)

Buss, Beate; Krautter, Markus; Möltner, Andreas; Weyrich, Peter; Werner, Anne; Jünger, Jana; Nikendei, Christoph

2012-01-01

Purpose: The acquisition of clinical-technical skills is of particular importance for the doctors of tomorrow. Procedural skills are often trained for the first time in skills laboratories, which provide a sheltered learning environment. However, costs to implement and maintain skills laboratories are considerably high. Therefore, the purpose of the present study was to investigate students’ patterns of attendance of voluntary skills-lab training sessions and thereby answer the following question: Is it possible to measure an effect of the theoretical construct related to motivational psychology described in the literature – ‘Assessment drives learning’ – reflected in patterns of attendance at voluntary skills-lab training sessions? By answering this question, design recommendations for curriculum planning and resource management should be derived. Method: A retrospective, descriptive analysis of student skills-lab attendance related to voluntary basic and voluntary advanced skills-lab sessions was conducted. The attendance patterns of a total of 340 third-year medical students in different successive year groups from the Medical Faculty at the University of Heidelberg were assessed. Results: Students showed a preference for voluntary basic skills-lab training sessions, which were relevant to clinical skills assessment, especially at the beginning and at the end of the term. Voluntary advanced skills-lab training sessions without reference to clinical skills assessment were used especially at the beginning of the term, but declined towards the end of term. Conclusion: The results show a clear influence of assessments on students’ attendance at skills-lab training sessions. First recommendations for curriculum design and resource management will be described. Nevertheless, further prospective research studies will be necessary to gain a more comprehensive understanding of the motivational factors impacting students’ utilisation of voluntary skills

Assessing Teamwork in Undergraduate Education: A Measurement Tool to Evaluate Individual Teamwork Skills

Science.gov (United States)

Britton, Emily; Simper, Natalie; Leger, Andrew; Stephenson, Jenn

2017-01-01

Effective teamwork skills are essential for success in an increasingly team-based workplace. However, research suggests that there is often confusion concerning how teamwork is measured and assessed, making it difficult to develop these skills in undergraduate curricula. The goal of the present study was to develop a sustainable tool for assessing…

The Gain-Loss Model: A Probabilistic Skill Multimap Model for Assessing Learning Processes

Science.gov (United States)

Robusto, Egidio; Stefanutti, Luca; Anselmi, Pasquale

2010-01-01

Within the theoretical framework of knowledge space theory, a probabilistic skill multimap model for assessing learning processes is proposed. The learning process of a student is modeled as a function of the student's knowledge and of an educational intervention on the attainment of specific skills required to solve problems in a knowledge…

Tactical and operational response to major incidents: feasibility and reliability of skills assessment using novel virtual environments.

Science.gov (United States)

Cohen, Daniel; Sevdalis, Nick; Patel, Vishal; Taylor, Michael; Lee, Henry; Vokes, Mick; Heys, Mick; Taylor, David; Batrick, Nicola; Darzi, Ara

2013-07-01

To determine feasibility and reliability of skills assessment in a multi-agency, triple-site major incident response exercise carried out in a virtual world environment. Skills assessment was carried out across three scenarios. The pre-hospital scenario required paramedics to triage and treat casualties at the site of an explosion. Technical skills assessment forms were developed using training syllabus competencies and national guidelines identified by pre-hospital response experts. Non-technical skills were assessed using a seven-point scale previously developed for use by pre-hospital paramedics. The two in-hospital scenarios, focusing on a trauma team leader and a silver/clinical major incident co-ordinator, utilised the validated Trauma-NOTECHS scale to assess five domains of performance. Technical competencies were assessed using an ATLS-style competency scale for the trauma scenario. For the silver scenario, the assessment document was developed using competencies described from a similar role description in a real-life hospital major incident plan. The technical and non-technical performance of all participants was assessed live by two experts in each of the three scenarios and inter-assessor reliability was computed. Participants also self-assessed their performance using identical proformas immediately after the scenarios were completed. Self and expert assessments were correlated (assessment cross-validation). Twenty-three participants underwent all scenarios and assessments. Performance assessments were feasible for both experts as well as the participants. Non-technical performance was generally scored higher than technical performance. Very good inter-rater reliability was obtained between expert raters across all scenarios and both technical and non-technical aspects of performance (reliability range 0.59-0.90, Psassessment in technical skills across all three scenarios (correlation range 0.52-0.84, Psskills. This study establishes feasibility and

Assessing the Clinical Skills of Dental Students: A Review of the Literature

Science.gov (United States)

Taylor, Carly L.; Grey, Nick; Satterthwaite, Julian D.

2013-01-01

Education, from a student perspective, is largely driven by assessment. An effective assessment tool should be both valid and reliable, yet this is often not achieved. The aim of this literature review is to identify and appraise the evidence base for assessment tools used primarily in evaluating clinical skills of dental students. Methods:…

Assess Student Performance: Skills. Second Edition. Module D-4 of Category D--Instructional Evaluation. Professional Teacher Education Module Series.

Science.gov (United States)

Ohio State Univ., Columbus. National Center for Research in Vocational Education.

This module, one of a series of 127 performance-based teacher education learning packages focusing upon specific professional competencies of vocational education teachers, deals with assessing student performance of psychomotor skills. Included in the module are learning experiences that address the following topics: important considerations…

A Longitudinal Study Assessing the Microsoft Office Skills Course

Science.gov (United States)

Carpenter, Donald A.; McGinnis, Denise; Slauson, Gayla Jo; Snyder, Johnny

2013-01-01

This paper explains a four-year longitudinal study of the assessment process for a Microsoft Office skills course. It examines whether there is an increase in students' knowledge based on responses to pre- and post-surveys that asked students to evaluate how well they can do particular tasks. Classical classroom teaching methods were used in the…

A mentor-based portfolio program to evaluate pharmacy students' self-assessment skills.

Science.gov (United States)

Kalata, Lindsay R; Abate, Marie A

2013-05-13

Objective. To evaluate pharmacy students' self-assessment skills with an electronic portfolio program using mentor evaluators. Design. First-year (P1) and second-year (P2) pharmacy students used online portfolios that required self-assessments of specific graded class assignments. Using a rubric, faculty and alumni mentors evaluated students' self-assessments and provided feedback. Assessment. Eighty-four P1 students, 74 P2 students, and 59 mentors participated in the portfolio program during 2010-2011. Both student groups performed well overall, with only a small number of resubmissions required. P1 students showed significant improvements across semesters for 2 of the self-assessment questions; P2 students' scores did not differ significantly. The P1 scores were significantly higher than P2 scores for 3 questions during spring 2011. Mentors and students had similar levels of agreement with the extent to which students put forth their best effort on the self-assessments. Conclusion. An electronic portfolio using mentors based inside and outside the school provided students with many opportunities to practice their self-assessment skills. This system represents a useful method of incorporating self-assessments into the curriculum that allows for feedback to be provided to the students.