Combining usability evaluations to highlight the chain that leads from usability flaws to usage problems and then negative outcomes.

Science.gov (United States)

Watbled, Ludivine; Marcilly, Romaric; Guerlinger, Sandra; Bastien, J-M Christian; Beuscart-Zéphir, Marie-Catherine; Beuscart, Régis

2018-02-01

Poor usability of health technology is thought to diminish work system performance, increase error rates and, potentially, harm patients. The present study (i) used a combination of usability evaluation methods to highlight the chain that leads from usability flaws to usage problems experienced by users and, ultimately, to negative patient outcomes, and (ii) validated this approach by studying two different discharge summary production systems. To comply with quality guidelines, the process of drafting and sending discharge summaries is increasingly being automated. However, the usability of these systems may modify their impact (or the absence thereof) in terms of production times and quality, and must therefore be evaluated. Here, we applied three successive techniques for usability evaluation (heuristic evaluation, user testing and field observation) to two discharge summary production systems (underpinned by different technologies). The systems' main usability flaws led respectively to an increase in the time need to produce a discharge summary and the risk of patient misidentification. Our results are discussed with regard to the possibility of linking the usability flaws, usage problems and the negative outcomes by successively applying three methods for evaluating usability (heuristic evaluation, user testing and in situ observations) throughout the system development life cycle. Copyright © 2018 Elsevier Inc. All rights reserved.

Integrating Usability Evaluations into the Software Development Process

DEFF Research Database (Denmark)

Lizano, Fulvio

as relevant and strategic human–computer interaction (HCI) activities in the software development process, there are obstacles that limit the complete, effective and efficient integration of this kind of testing into the software development process. Two main obstacles are the cost of usability evaluations...... and the software developers' resistance to accepting users’ opinions regarding the lack of usability in their software systems. The ‘cost obstacle’ refers to the constraint of conducting usability evaluations in the software process due to the significant amount of resources required by this type of testing. Some......This thesis addresses the integration of usability evaluations into the software development process. The integration here is contextualized in terms of how to include usability evaluation as an activity in the software development lifecycle. Even though usability evaluations are considered...

Software Development and Feedback from Usability Evaluations

DEFF Research Database (Denmark)

Høegh, Rune Thaarup

2008-01-01

This paper presents a study of the strengths and weaknesses of written, multimedia and oral feedback from usability evaluations to developers. The strengths and weaknesses are related to how well the feedback supports the developers in addressing usability problems in a software system. The study...... concludes that using the traditional written usability report, as the only form of feedback from usability evaluations is associated with problems related to the report not supporting the process of addressing the usability problems. The report is criticized for representing an overwhelming amount...... of information, while still not offering the required information to address usability problems. Other forms of feedback, such as oral or multimedia feedback helps the developer in understanding the usability problems better, but are on the other hand less cost-effective than a written description....

Feedback from Usability Evaluation to User Interface Design

DEFF Research Database (Denmark)

Nielsen, C. M.; Overgaard, M.; Pedersen, M. B.

2005-01-01

This paper reports from an exploratory study of means for providing feedback from a usability evaluation to the user interface designers. In this study, we conducted a usability evaluation of a mobile system that is used by craftsmen to register use of time and materials. The results...... and weaknesses of the system. The findings indicate that detailed descriptions of problems and log descriptions of the user's interaction with the system and of system interaction are useful for the designers when trying to understand the usability problems that the users have encountered....

Evaluating Usability of E-Learning Systems in Universities

OpenAIRE

Nicholas Kipkurui Kiget; Professor G. Wanyembi; Anselemo Ikoha Peters

2014-01-01

The use of e-learning systems has increased significantly in the recent times. E-learning systems are supplementing teaching and learning in universities globally. Kenyan universities have adopted e-learning technologies as means for delivering course content. However despite adoption of these systems, there are considerable challenges facing the usability of the systems. Lecturers and students have different perceptions in regard to the usability of e-learning systems. The aim of this study ...

Evaluating Web Usability

Science.gov (United States)

Snider, Jean; Martin, Florence

2012-01-01

Web usability focuses on design elements and processes that make web pages easy to use. A website for college students was evaluated for underutilization. One-on-one testing, focus groups, web analytics, peer university review and marketing focus group and demographic data were utilized to conduct usability evaluation. The results indicated that…

Comparative study of heuristic evaluation and usability testing methods.

Science.gov (United States)

Thyvalikakath, Thankam Paul; Monaco, Valerie; Thambuganipalle, Himabindu; Schleyer, Titus

2009-01-01

Usability methods, such as heuristic evaluation, cognitive walk-throughs and user testing, are increasingly used to evaluate and improve the design of clinical software applications. There is still some uncertainty, however, as to how those methods can be used to support the development process and evaluation in the most meaningful manner. In this study, we compared the results of a heuristic evaluation with those of formal user tests in order to determine which usability problems were detected by both methods. We conducted heuristic evaluation and usability testing on four major commercial dental computer-based patient records (CPRs), which together cover 80% of the market for chairside computer systems among general dentists. Both methods yielded strong evidence that the dental CPRs have significant usability problems. An average of 50% of empirically-determined usability problems were identified by the preceding heuristic evaluation. Some statements of heuristic violations were specific enough to precisely identify the actual usability problem that study participants encountered. Other violations were less specific, but still manifested themselves in usability problems and poor task outcomes. In this study, heuristic evaluation identified a significant portion of problems found during usability testing. While we make no assumptions about the generalizability of the results to other domains and software systems, heuristic evaluation may, under certain circumstances, be a useful tool to determine design problems early in the development cycle.

Dogmas in the assessment of usability evaluation methods

DEFF Research Database (Denmark)

Hornbæk, Kasper

2010-01-01

Usability evaluation methods (UEMs) are widely recognised as an essential part of systems development. Assessments of the performance of UEMs, however, have been criticised for low validity and limited reliability. The present study extends this critique by describing seven dogmas in recent work ...... research approaches that may help move beyond the dogmas. In particular, we emphasise detailed studies of evaluation processes, assessments of the impact of UEMs on design carried out in real-world systems development and analyses of how UEMs may be combined......Usability evaluation methods (UEMs) are widely recognised as an essential part of systems development. Assessments of the performance of UEMs, however, have been criticised for low validity and limited reliability. The present study extends this critique by describing seven dogmas in recent work...... on UEMs. The dogmas include using inadequate procedures and measures for assessment, focusing on win-lose outcomes, holding simplistic models of how usability evaluators work, concentrating on evaluation rather than on design and working from the assumption that usability problems are real. We discuss...

Evaluation of Usability Utilizing Markov Models

Science.gov (United States)

Penedo, Janaina Rodrigues; Diniz, Morganna; Ferreira, Simone Bacellar Leal; Silveira, Denis S.; Capra, Eliane

2012-01-01

Purpose: The purpose of this paper is to analyze the usability of a remote learning system in its initial development phase, using a quantitative usability evaluation method through Markov models. Design/methodology/approach: The paper opted for an exploratory study. The data of interest of the research correspond to the possible accesses of users…

Design and validation of a questionnaire to evaluate the usability of computerized critical care information systems.

Science.gov (United States)

von Dincklage, Falk; Lichtner, Gregor; Suchodolski, Klaudiusz; Ragaller, Maximilian; Friesdorf, Wolfgang; Podtschaske, Beatrice

2017-08-01

The implementation of computerized critical care information systems (CCIS) can improve the quality of clinical care and staff satisfaction, but also holds risks of disrupting the workflow with consecutive negative impacts. The usability of CCIS is one of the key factors determining their benefits and weaknesses. However, no tailored instrument exists to measure the usability of such systems. Therefore, the aim of this study was to design and validate a questionnaire that measures the usability of CCIS. Following a mixed-method design approach, we developed a questionnaire comprising two evaluation models to assess the usability of CCIS: (1) the task-specific model rates the usability individually for several tasks which CCIS could support and which we derived by analyzing work processes in the ICU; (2) the characteristic-specific model rates the different aspects of the usability, as defined by the international standard "ergonomics of human-system interaction". We tested validity and reliability of the digital version of the questionnaire in a sample population. In the sample population of 535 participants both usability evaluation models showed a strong correlation with the overall rating of the system (multiple correlation coefficients ≥0.80) as well as a very high internal consistency (Cronbach's alpha ≥0.93). The novel questionnaire is a valid and reliable instrument to measure the usability of CCIS and can be used to study the influence of the usability on their implementation benefits and weaknesses.

Problem Prioritization in Usability Evaluation:

DEFF Research Database (Denmark)

Hertzum, Morten

2006-01-01

Severity assessments enable prioritization of problems encountered during usability evaluations and thereby provide a device for guiding the utilization of design resources. However, designers' response to usability evaluations is also influenced by other factors, which may overshadow severity....... With the purpose of enhancing the impact of severity assessments, this study combines a field study of factors that influence the impact of evaluations with an experimental study of severity assessments made during usability inspections. The results show that even in a project receptive to input from evaluations...... their impact was highly dependent on conducting evaluations early. This accorded with an informal method that blended elements of usability evaluation and participatory design and could be extended with user-made severity assessments. The major cost associated with the evaluations was not finding but fixing...

Analysis in usability evaluations

DEFF Research Database (Denmark)

Følstad, Asbjørn; Lai-Chong Law, Effie; Hornbæk, Kasper

2010-01-01

While the planning and implementation of usability evaluations are well described in the literature, the analysis of the evaluation data is not. We present interviews with 11 usability professionals on how they conduct analysis, describing the resources, collaboration, creation of recommendations......, and prioritization involved. The interviews indicate a lack of structure in the analysis process and suggest activities, such as generating recommendations, that are unsupported by existing methods. We discuss how to better support analysis, and propose four themes for future research on analysis in usability...

Comparing two anesthesia information management system user interfaces: a usability evaluation.

Science.gov (United States)

Wanderer, Jonathan P; Rao, Anoop V; Rothwell, Sarah H; Ehrenfeld, Jesse M

2012-11-01

Anesthesia information management systems (AIMS) have been developed by multiple vendors and are deployed in thousands of operating rooms around the world, yet not much is known about measuring and improving AIMS usability. We developed a methodology for evaluating AIMS usability in a low-fidelity simulated clinical environment and used it to compare an existing user interface with a revised version. We hypothesized that the revised user interface would be more useable. In a low-fidelity simulated clinical environment, twenty anesthesia providers documented essential anesthetic information for the start of the case using both an existing and a revised user interface. Participants had not used the revised user interface previously and completed a brief training exercise prior to the study task. All participants completed a workload assessment and a satisfaction survey. All sessions were recorded. Multiple usability metrics were measured. The primary outcome was documentation accuracy. Secondary outcomes were perceived workload, number of documentation steps, number of user interactions, and documentation time. The interfaces were compared and design problems were identified by analyzing recorded sessions and survey results. Use of the revised user interface was shown to improve documentation accuracy from 85.1% to 92.4%, a difference of 7.3% (95% confidence interval [CI] for the difference 1.8 to 12.7). The revised user interface decreased the number of user interactions by 6.5 for intravenous documentation (95% CI 2.9 to 10.1) and by 16.1 for airway documentation (95% CI 11.1 to 21.1). The revised user interface required 3.8 fewer documentation steps (95% CI 2.3 to 5.4). Airway documentation time was reduced by 30.5 seconds with the revised workflow (95% CI 8.5 to 52.4). There were no significant time differences noted in intravenous documentation or in total task time. No difference in perceived workload was found between the user interfaces. Two user interface

Usability evaluation of user interface of thesis title review system

Science.gov (United States)

Tri, Y.; Erna, A.; Gellysa, U.

2018-03-01

Presentation of programs with user interface that can be accessed online through the website of course greatly provide user benefits. User can easily access the program they need. There are usability values that serve as a benchmark for the success of a user accessible program, ie efficiency, effectiveness, and convenience. These usability values also determine the development of the program for the better use. Therefore, on the review title thesis program that will be implemented in STT Dumai was measured usability evaluation. It aims to see which sides are not yet perfect and need to be improved to improve the performance and utilization of the program. Usability evaluation was measured by using smartPLS software. Database used was the result of respondent questionnaires that include questions about the experience when they used program. The result of a review of thesis title program implemented in STT Dumai has an efficiency value of 22.615, the effectiveness of 20.612, and satisfaction of 33.177.

A Fuzzy method for the usability evaluation of nuclear medical equipment

Energy Technology Data Exchange (ETDEWEB)

Grecco, Cláudio H.S.; Santos, Isaac J.A.L.; Farias, Marcos S.; Farias, Larissa P.; Filho, Alfredo M.V., E-mail: grecco@ien.gov.br, E-mail: luquetti@ien.gov.br, E-mail: msantana@ien.gov.br, E-mail: larissapfarias@ymail.com, E-mail: marques@ien.gov.br [Instituto de Engenharia Nuclear (DENN/IEN/CNEN/RJ), Rio de Janeiro, RJ (Brazil). Departamento de Engenharia Nuclear

2017-11-01

To avoid errors when handling nuclear medical equipment, it is important to develop products with a high degree of usability. This can be achieved by performing usability evaluations in the product development process to detect and mitigate potential usability problems. Usability evaluation focuses on how well users can learn and use a product to achieve their goals. Thus, the usability evaluation has become recognized as a valuable complement to the established approaches to design good user interfaces, to reduce incidents and accidents as well the time required to learn how to use the equipment. To gather information about usability, practitioners use a variety of methods that gather feedback from users about an existing interface or plans related to a new interface. A wide range of usability evaluation methods have been proposed, but few methods focus on developing an objective and practical evaluation method for usability. Moreover, the usability evaluations are based on human judgments and most methods cannot fully solve the subjectivity of these evaluations. In order to remedy this deficiency, the purpose of this work is to adopt a Fuzzy Set Theory (FST) approach to establish a method for the usability evaluation of nuclear medical equipment based on usability heuristics for user interface design and international standards for ergonomics of human-system interaction. To exemplify the method we performed a usability evaluating of the Digital Spectrometer ESP 13004 by testing it with representative users. The results showed that the method is a proactive tool to provide a basis for checking usability of medical device interfaces. (author)

A Fuzzy method for the usability evaluation of nuclear medical equipment

International Nuclear Information System (INIS)

Grecco, Cláudio H.S.; Santos, Isaac J.A.L.; Farias, Marcos S.; Farias, Larissa P.; Filho, Alfredo M.V.

2017-01-01

To avoid errors when handling nuclear medical equipment, it is important to develop products with a high degree of usability. This can be achieved by performing usability evaluations in the product development process to detect and mitigate potential usability problems. Usability evaluation focuses on how well users can learn and use a product to achieve their goals. Thus, the usability evaluation has become recognized as a valuable complement to the established approaches to design good user interfaces, to reduce incidents and accidents as well the time required to learn how to use the equipment. To gather information about usability, practitioners use a variety of methods that gather feedback from users about an existing interface or plans related to a new interface. A wide range of usability evaluation methods have been proposed, but few methods focus on developing an objective and practical evaluation method for usability. Moreover, the usability evaluations are based on human judgments and most methods cannot fully solve the subjectivity of these evaluations. In order to remedy this deficiency, the purpose of this work is to adopt a Fuzzy Set Theory (FST) approach to establish a method for the usability evaluation of nuclear medical equipment based on usability heuristics for user interface design and international standards for ergonomics of human-system interaction. To exemplify the method we performed a usability evaluating of the Digital Spectrometer ESP 13004 by testing it with representative users. The results showed that the method is a proactive tool to provide a basis for checking usability of medical device interfaces. (author)

MATHEMATICAL MODEL FOR SOFTWARE USABILITY AUTOMATED EVALUATION AND ASSURANCE

Directory of Open Access Journals (Sweden)

І. Гученко

2011-04-01

Full Text Available The subject of the research is software usability and the aim is construction of mathematicalmodel of estimation and providing of the set level of usability. Methodology of structural analysis,methods of multicriterion optimization and theory of making decision, method of convolution,scientific methods of analysis and analogies is used in the research. The result of executed work isthe model for software usability automated evaluation and assurance that allows not only toestimate the current level of usability during every iteration of agile development but also tomanage the usability of created software products. Results can be used for the construction ofautomated support systems of management the software usability.

Physician satisfaction with a critical care clinical information system using a multimethod evaluation of usability.

Science.gov (United States)

Hudson, Darren; Kushniruk, Andre; Borycki, Elizabeth; Zuege, Danny J

2018-04-01

Physician satisfaction with electronic medical records has often been poor. Usability has frequently been identified as a source for decreased satisfaction. While surveys can identify many issues, and are logistically easier to administer, they may miss issues identified using other methods This study sought to understand the level of physician satisfaction and usability issues associated with a critical care clinical information system (eCritical Alberta) implemented throughout the province of Alberta, Canada. All critical care attending physicians using the system were invited to participate in an online survey. Questions included components of the User Acceptance of Information Technology and Usability Questionnaire as well as free text feedback on system components. Physicians were also invited to participate in a think aloud test using simulated scenarios. The transcribed think aloud text and questionnaire were subjected to textual analysis. 82% of all eligible physicians completed the on-line survey (n = 61). Eight physicians were invited and seven completed the think aloud test. Overall satisfaction with the system was moderate. Usability was identified as a significant factor contributing to satisfaction. The major usability factors identified were system response time and layout. The think aloud component identified additional factors beyond those identified in the on-line survey. This study found a modestly high level of physician satisfaction with a province-wide clinical critical care information system. Usability continues to be a significant factor in physician satisfaction. Using multiple methods of evaluation can capture the benefits of a large sample size and deeper understanding of the issues. Copyright © 2018 Elsevier B.V. All rights reserved.

Usability Evaluation in a Digitally Emerging Country

DEFF Research Database (Denmark)

Lizano, Fulvio; Sandoval, Maria Marta; Bruun, Anders

2013-01-01

Several emerging countries experience increasing software development activities. With the purpose of provide useful feedback on possible courses of action for increasing application of usability evaluation in such countries, this paper explores the status of usability evaluation in a digitally...... emerging country. Our aim is to identifying common characteristics or behavioral patterns that could be compared with digitally advanced countries. We used an online survey answered by 26 software development organizations, which gave a snapshot of the application of usability evaluation...... in these organizations. We found many similarities with advanced countries, several completely new obstacles more connected with software development matters and a relatively positive improvement in the lack of “usability culture”. These findings suggest good conditions to improve conduction of usability evaluations...

Advancing Usability Evaluation through Human Reliability Analysis

International Nuclear Information System (INIS)

Ronald L. Boring; David I. Gertman

2005-01-01

This paper introduces a novel augmentation to the current heuristic usability evaluation methodology. The SPAR-H human reliability analysis method was developed for categorizing human performance in nuclear power plants. Despite the specialized use of SPAR-H for safety critical scenarios, the method also holds promise for use in commercial off-the-shelf software usability evaluations. The SPAR-H method shares task analysis underpinnings with human-computer interaction, and it can be easily adapted to incorporate usability heuristics as performance shaping factors. By assigning probabilistic modifiers to heuristics, it is possible to arrive at the usability error probability (UEP). This UEP is not a literal probability of error but nonetheless provides a quantitative basis to heuristic evaluation. When combined with a consequence matrix for usability errors, this method affords ready prioritization of usability issues

An evaluation of patients' experienced usability of a diabetes mHealth system using a multi-method approach.

Science.gov (United States)

Georgsson, Mattias; Staggers, Nancy

2016-02-01

mHealth systems are becoming more common to aid patients in their diabetes self-management, but recent studies indicate a need for thorough evaluation of patients' experienced usability. Current evaluations lack a multi-method design for data collection and structured methods for data analyses. The purpose of this study was to provide a feasibility test of a multi-method approach for both data collection and data analyses for patients' experienced usability of a mHealth system for diabetes type 2 self-management. A random sample of 10 users was selected from a larger clinical trial. Data collection methods included user testing with eight representative tasks and Think Aloud protocol, a semi-structured interview and a questionnaire on patients' experiences using the system. The Framework Analysis (FA) method and Usability Problem Taxonomy (UPT) were used to structure, code and analyze the results. A usability severity rating was assigned after classification. The combined methods resulted in a total of 117 problems condensed into 19 usability issues with an average severity rating of 2.47 or serious. The usability test detected 50% of the initial usability problems, followed by the post-interview at 29%. The usability test found 18 of 19 consolidated usability problems while the questionnaire uncovered one unique issue. Patients experienced most usability problems (8) in the Glucose Readings View when performing complex tasks such as adding, deleting, and exporting glucose measurements. The severity ratings were the highest for the Glucose Diary View, Glucose Readings View, and Blood Pressure View with an average severity rating of 3 (serious). Most of the issues were classified under the artifact component of the UPT and primary categories of Visualness (7) and Manipulation (6). In the UPT task component, most issues were in the primary category Task-mapping (12). Multiple data collection methods yielded a more comprehensive set of usability issues. Usability

Usability Evaluation of www.budget.com

DEFF Research Database (Denmark)

Bruun, Anders; Jensen, Janne Jul; Skov, Mikael B.

This report presents the results of a usability evaluation of the budget.com website. The evaluation was carried out by a team of 4 HCI experts, and it involved 10 potential users. The evaluation is based on measurement of (1) task completion time, (2) task completion, (3) user satisfaction and (4......) perceived workload. We have also analyzed the user interaction with the website for (5) usability problems. Through the evaluation, we have found the following weaknesses of the website:   Reservation takes considerably more time than promised. The website indicates that a reservation can be made in 60...... is not useful for specific purposes. The user satisfaction is generally low. Except one high and two low scores, the user satisfaction was in the middle range. There are too many severe usability problems. The evaluation identified a considerable number of usability problems. 5 of them were critical which means...

A literature review about usability evaluation methods for e-learning platforms.

Science.gov (United States)

Freire, Luciana Lopes; Arezes, Pedro Miguel; Campos, José Creissac

2012-01-01

The usability analysis of information systems has been the target of several research studies over the past thirty years. These studies have highlighted a great diversity of points of view, including researchers from different scientific areas such as Ergonomics, Computer Science, Design and Education. Within the domain of information ergonomics, the study of tools and methods used for usability evaluation dedicated to E-learning presents evidence that there is a continuous and dynamic evolution of E-learning systems, in many different contexts -academics and corporative. These systems, also known as LMS (Learning Management Systems), can be classified according to their educational goals and their technological features. However, in these systems the usability issues are related with the relationship/interactions between user and system in the user's context. This review is a synthesis of research project about Information Ergonomics and embraces three dimensions, namely the methods, models and frameworks that have been applied to evaluate LMS. The study also includes the main usability criteria and heuristics used. The obtained results show a notorious change in the paradigms of usability, with which it will be possible to discuss about the studies carried out by different researchers that were focused on usability ergonomic principles aimed at E-learning.

Performance Evaluation of Two Different Usability Evaluation Methods in the Context of Collaborative Writing Systems

OpenAIRE

Bakhtyar, Shoaib; Afridi, Qaisar Zaman

2010-01-01

In today’s world of rapid technological development one cannot deny the importance of collaborative writing systems. Besides many advantages of a collaborative writing system the major one is to allow its end users to work in collaboration with each other without having to physically meet. In the past various researches has been carried out for the usability evaluation of collaborative writing systems using the think aloud protocol method however there is no study conducted on the comparison ...

Developing a Self-Report-Based Sequential Analysis Method for Educational Technology Systems: A Process-Based Usability Evaluation

Science.gov (United States)

Lin, Yi-Chun; Hsieh, Ya-Hui; Hou, Huei-Tse

2015-01-01

The development of a usability evaluation method for educational systems or applications, called the self-report-based sequential analysis, is described herein. The method aims to extend the current practice by proposing self-report-based sequential analysis as a new usability method, which integrates the advantages of self-report in survey…

77 FR 8217 - Evaluating the Usability of Electronic Health Record (EHR) Systems

Science.gov (United States)

2012-02-14

... usability of health information technology (HIT) systems, EHRs in particular, and performance-oriented user... the usability of health information technology (HIT) systems. NIST research is designed to: (1... DEPARTMENT OF COMMERCE National Institute of Standards and Technology [Docket No. 120123059-2058...

Health system guidance appraisal--concept evaluation and usability testing.

Science.gov (United States)

Ako-Arrey, Denis E; Brouwers, Melissa C; Lavis, John N; Giacomini, Mita K

2016-01-05

Health system guidance (HSG) provides recommendations aimed to address health system challenges. However, there is a paucity of methods to direct, appraise, and report HSG. Earlier research identified 30 candidate criteria (concepts) that can be used to evaluate the quality of HSG and guide development and reporting requirements. The objective of this paper was to describe two studies aimed at evaluating the importance of these 30 criteria, design a draft HSG appraisal tool, and test its usability. This study involved a two-step survey process. In step 1, respondents rated the 30 concepts for appropriateness to, relevance to, and priority for health system decisions and HSG. This led to a draft tool. In step 2, respondents reviewed HSG documents, appraised them using the tool, and answered a series of questions. Descriptive analyses were computed. Fifty participants were invited in step 1, and we had a response rate of 82 %. The mean response rates for each concept within each survey question were universally favorable. There was also an overall agreement about the need for a high-quality tool to systematically direct the development, appraisal, and reporting of HSG. Qualitative feedback and a consensus process by the team led to refinements to some of the concepts and the creation of a beta (draft) version of the HSG tool. In step 2, 35 participants were invited and we had a response rate of 74 %. Exploratory analyses showed that the quality of the HSGs reviewed varied as a function of the HSG item and the specific document assessed. A favorable consensus was reached with participants agreeing that the HSG items were easy to understand and easy to apply. Moreover, the overall agreement was high for the usability of the tool to systematically direct the development (85 %), appraisal (92 %), and reporting (81 %) of HSG. From this process, version 1.0 of the HSG appraisal tool was generated complete with 32 items (and their descriptions) and 4 domains. The final

Qualitative review of usability problems in health information systems for radiology.

Science.gov (United States)

Dias, Camila Rodrigues; Pereira, Marluce Rodrigues; Freire, André Pimenta

2017-12-01

Radiology processes are commonly supported by Radiology Information System (RIS), Picture Archiving and Communication System (PACS) and other software for radiology. However, these information technologies can present usability problems that affect the performance of radiologists and physicians, especially considering the complexity of the tasks involved. The purpose of this study was to extract, classify and analyze qualitatively the usability problems in PACS, RIS and other software for radiology. A systematic review was performed to extract usability problems reported in empirical usability studies in the literature. The usability problems were categorized as violations of Nielsen and Molich's usability heuristics. The qualitative analysis indicated the causes and the effects of the identified usability problems. From the 431 papers initially identified, 10 met the study criteria. The analysis of the papers identified 90 instances of usability problems, classified into categories corresponding to established usability heuristics. The five heuristics with the highest number of instances of usability problems were "Flexibility and efficiency of use", "Consistency and standards", "Match between system and the real world", "Recognition rather than recall" and "Help and documentation", respectively. These problems can make the interaction time consuming, causing delays in tasks, dissatisfaction, frustration, preventing users from enjoying all the benefits and functionalities of the system, as well as leading to more errors and difficulties in carrying out clinical analyses. Furthermore, the present paper showed a lack of studies performed on systems for radiology, especially usability evaluations using formal methods of evaluation involving the final users. Copyright © 2017 Elsevier Inc. All rights reserved.

Safety and usability evaluation of a web-based insulin self-titration system for patients with type 2 diabetes mellitus.

Science.gov (United States)

Simon, Airin C R; Holleman, Frits; Gude, Wouter T; Hoekstra, Joost B L; Peute, Linda W; Jaspers, Monique W M; Peek, Niels

2013-09-01

The rising incidence of type 2 diabetes mellitus (T2DM) induces severe challenges for the health care system. Our research group developed a web-based system named PANDIT that provides T2DM patients with insulin dosing advice using state of the art clinical decision support technology. The PANDIT interface resembles a glucose diary and provides advice through pop-up messages. Diabetes nurses (DNs) also have access to the system, allowing them to intervene when needed. The objective of this study was to establish whether T2DM patients can safely use PANDIT at home. To this end, we assessed whether patients experience usability problems with a high risk of compromising patient safety when interacting with the system, and whether PANDIT's insulin dosing advice are clinically safe. The study population consisted of patients with T2DM (aged 18-80) who used a once daily basal insulin as well as DNs from a university hospital. The usability evaluation consisted of think-aloud sessions with four patients and three DNs. Video data, audio data and verbal utterances were analyzed for usability problems encountered during PANDIT interactions. Usability problems were rated by a physician and a usability expert according to their potential impact on patient safety. The usability evaluation was followed by an implementation with a duration of four weeks. This implementation took place at the patients' homes with ten patients to evaluate clinical safety of PANDIT advice. PANDIT advice were systematically compared with DN advice. Deviating advice were evaluated with respect to patient safety by a panel of experienced physicians, which specialized in diabetes care. We detected seventeen unique usability problems, none of which was judged to have a high risk of compromising patient safety. Most usability problems concerned the lay-out of the diary, which did not clearly indicate which data entry fields had to be entered in order to obtain an advice. 27 out of 74 (36.5%) PANDIT advice

Usability Evaluation of Public Web Mapping Sites

Science.gov (United States)

Wang, C.

2014-04-01

Web mapping sites are interactive maps that are accessed via Webpages. With the rapid development of Internet and Geographic Information System (GIS) field, public web mapping sites are not foreign to people. Nowadays, people use these web mapping sites for various reasons, in that increasing maps and related map services of web mapping sites are freely available for end users. Thus, increased users of web mapping sites led to more usability studies. Usability Engineering (UE), for instance, is an approach for analyzing and improving the usability of websites through examining and evaluating an interface. In this research, UE method was employed to explore usability problems of four public web mapping sites, analyze the problems quantitatively and provide guidelines for future design based on the test results. Firstly, the development progress for usability studies were described, and simultaneously several usability evaluation methods such as Usability Engineering (UE), User-Centered Design (UCD) and Human-Computer Interaction (HCI) were generally introduced. Then the method and procedure of experiments for the usability test were presented in detail. In this usability evaluation experiment, four public web mapping sites (Google Maps, Bing maps, Mapquest, Yahoo Maps) were chosen as the testing websites. And 42 people, who having different GIS skills (test users or experts), gender (male or female), age and nationality, participated in this test to complete the several test tasks in different teams. The test comprised three parts: a pretest background information questionnaire, several test tasks for quantitative statistics and progress analysis, and a posttest questionnaire. The pretest and posttest questionnaires focused on gaining the verbal explanation of their actions qualitatively. And the design for test tasks targeted at gathering quantitative data for the errors and problems of the websites. Then, the results mainly from the test part were analyzed. The

Usability engineering: domain analysis activities for augmented-reality systems

Science.gov (United States)

Gabbard, Joseph; Swan, J. E., II; Hix, Deborah; Lanzagorta, Marco O.; Livingston, Mark; Brown, Dennis B.; Julier, Simon J.

2002-05-01

This paper discusses our usability engineering process for the Battlefield Augmented Reality System (BARS). Usability engineering is a structured, iterative, stepwise development process. Like the related disciplines of software and systems engineering, usability engineering is a combination of management principals and techniques, formal and semi- formal evaluation techniques, and computerized tools. BARS is an outdoor augmented reality system that displays heads- up battlefield intelligence information to a dismounted warrior. The paper discusses our general usability engineering process. We originally developed the process in the context of virtual reality applications, but in this work we are adapting the procedures to an augmented reality system. The focus of this paper is our work on domain analysis, the first activity of the usability engineering process. We describe our plans for and our progress to date on our domain analysis for BARS. We give results in terms of a specific urban battlefield use case we have designed.

Ionizing radiation detection meter systems usability evaluation. Technical memorandum

International Nuclear Information System (INIS)

Severs, Y.; Hughes, R.; Tack, D.; Bossi, L.L.M.

2002-12-01

In support of the Canadian Forces Nuclear Detection, Identification and Dosimetry project a usability trial of three Ionizing Radiation Detection Meter Systems (IRDMS) was conducted 18-22 June 2001 at Connaught Ranges. The systems evaluated: System 1: ANPDR 77 Operational System with PDR 77 Alpha/Beta probes and GPS (Global Positioning System) Training System; System 2: ADM 300C Operational System with ABP 100 Alpha/Beta probes and GPS Training System, and System 3: RAM R200 Operational System with PA 100M and RG12 Alpha/Beta probes and GPS Training System. Twelve military personnel (Private to Captain) determined compliance with the bid evaluation criteria. The counter-balanced study, based on a Latin square design, was divided into two assessment phases, an operational phase and a training system phase, which rotated three groups (of four participants each) concurrently through each IRDMS. Groups were trained in set-up, maintenance and operation of each bid IRDMS prior to evaluation. The operational evaluation consisted of a test/maintenance stand, an indoor/outdoor operational stand, and an indoor,/outdoor training system stand. Participants rated the utility and usability of each system, by measuring case of use/maintenance under a range of relevant operational and clothing conditions, compatibility with soldier tasks, compatibility with clothing, compatibility with equipment and operational environments, and case of learning (including an assessment of initial and refresher training implications). A 7-point Likert-like acceptability rating scale, which covered a range from completely unacceptable (1) to completely acceptable (7), was used to evaluate the measures. The combined weighted mean scores were 584.3 for System 1 (70 o /b), 682.0 for System 2 (81 %) and 756.9 for System 3 (90%) out of a total possible score of 840. Participants reported mean scores for System 1 between barely unacceptable and borderline for both indoor and outdoor operations. System 2

Usability Study of a Computer-Based Self-Management System for Older Adults with Chronic Diseases

Science.gov (United States)

Tao, Da

2012-01-01

Background Usability can influence patients’ acceptance and adoption of a health information technology. However, little research has been conducted to study the usability of a self-management health care system, especially one geared toward elderly patients. Objective This usability study evaluated a new computer-based self-management system interface for older adults with chronic diseases, using a paper prototype approach. Methods Fifty older adults with different chronic diseases participated. Two usability evaluation methods were involved: (1) a heuristics evaluation and (2) end-user testing with a think-aloud testing method, audio recording, videotaping, and interviewing. A set of usability metrics was employed to determine the overall system usability, including task incompletion rate, task completion time, frequency of error, frequency of help, satisfaction, perceived usefulness, and perceived ease of use. Interviews were used to elicit participants’ comments on the system design. The quantitative data were analyzed using descriptive statistics and the qualitative data were analyzed for content. Results The participants were able to perform the predesigned self-management tasks with the current system design and they expressed mostly positive responses about the perceived usability measures regarding the system interface. However, the heuristics evaluation, performance measures, and interviews revealed a number of usability problems related to system navigation, information search and interpretation, information presentation, and readability. Design recommendations for further system interface modifications were discussed. Conclusions This study verified the usability of the self-management system developed for older adults with chronic diseases. Also, we demonstrated that our usability evaluation approach could be used to quickly and effectively identify usability problems in a health care information system at an early stage of the system development

Cultural Cognition in Usability Evaluation

DEFF Research Database (Denmark)

Clemmensen, Torkil; Hertzum, Morten; Hornbæk, Kasper Anders Søren

2009-01-01

We discuss the impact of cultural differences on usability evaluations that are based on the thinking-aloud method (TA). The term ‘cultural differences' helps distinguish differences in the perception and thinking of Westerners (people from Western Europe and US citizens with European origins...... and evaluator. In conclusion, we point to the importance of matching the task presentation to users' cultural background, the different effects of thinking aloud on task performance between Easterners and Westerners, the differences in nonverbal behaviour that affect usability problem detection, and, finally...

Cultural Cognition in Usability Evaluation

DEFF Research Database (Denmark)

Clemmensen, Torkil; Hertzum, Morten; Hornbæk, Kasper Anders Søren

2009-01-01

We discuss the impact of cultural differences on usability evaluations that are based on the thinking-aloud method (TA). The term ‘cultural differences’ helps distinguish differences in the perception and thinking of Westerners (people from Western Europe and US citizens with European origins...... and evaluator. In conclusion, we point to the importance of matching the task presentation to users’ cultural background, the different effects of thinking aloud on task performance between Easterners and Westerners, the differences in nonverbal behaviour that affect usability problem detection, and, finally...

A usability evaluation of the prototype Afrikaanse idiome-woordeboek

African Journals Online (AJOL)

When digital tools are developed it is important to consider the usability of the tool. Usability evaluation was done on the Afrikaanse idiome-woordeboek to determine with what success it can be used. Discount usability methods, viz. heuristic evaluation and usability testing were used. This article reports on the findings from ...

Design and Usability Evaluation of Social Mobile Diabetes Management System in the Gulf Region.

Science.gov (United States)

Alanzi, Turki; Istepanian, Robert; Philip, Nada

2016-09-26

The prevalence of diabetes in the Gulf States is one of the highest globally. It is estimated that 20% of the population in the region has been diagnosed with diabetes and according to the International Diabetes Federation (IDF), five of the IDF's "top 10" countries for diabetes prevalence in 2011 and projected for 2030 are in this region. In recent years, there have been an increasing number of clinical studies advocating the use of mobile phone technology for diabetes self-management with improved clinical outcomes. However, there are few studies to date addressing the application of mobile diabetes management in the Gulf region, particularly in the Kingdom of Saudi Arabia (KSA), where there is exponential increase in mobile phone usage and access to social networking. The objective of this paper is to present the design and development of a new mobile health system for social behavioral change and management tailored for Saudi patients with diabetes called Saudi Arabia Networking for Aiding Diabetes (SANAD). A usability study for the SANAD system is presented to validate the acceptability of using mobile technologies among patients with diabetes in the KSA and the Gulf region. The SANAD system was developed using mobile phone technology with diabetes management and social networking modules. For the usability study the Questionnaire for User Interaction Satisfaction was used to evaluate the usability aspect of the SANAD system. A total of 33 users with type 2 diabetes participated in the study. The key modules of the SANAD system consist of (1) a mobile diabetes management module; (2) a social networking module; and (3) a cognitive behavioral therapy module for behavioral change issues. The preliminary results of the usability study indicated general acceptance of the patients in using the system with higher usability rating in patients with type 2 diabetes. We found that the acceptability of the system was high among Saudi patients with diabetes, and ongoing

Usability Evaluation for Business Intelligence Applications: A User Support Perspective

Directory of Open Access Journals (Sweden)

Chrisna Jooste

2014-08-01

Full Text Available Business Intelligence (BI applications provide business information to drive decision support. Usability is one of the factors determining the optimal use and eventual benefit derived from BI applications. The documented need for more BI usability research together with the practical necessity for BI evaluation guidelines in the mining industry provides the rationale for this study. The purpose of the study was to investigate the usability evaluation of BI applications in the context of a coal mining organization. The research is guided by the question: How can existing usability criteria be customized to evaluate the usability of BI applications. The research design included user observation, heuristic evaluation and a survey. Based on observations made during user support on a BI application used at a coal mining organization a log of usability issues was compiled. The usability issues extracted from this log was compared and contrasted with general usability criteria from literature to synthesize an initial set of BI usability evaluation criteria. These criteria were used as the basis for a heuristic evaluation of the BI application used at the coal mining organization. The same BI application was also evaluated using the Software Usability Measurement Inventory (SUMI standardized questionnaire. The results from the two evaluations were triangulated and then compared with the BI user issues again to contextualize the findings and synthesize a validated and refined set of criteria. The main contribution of the study is the usability evaluation criteria for BI applications presented as guidelines. These guidelines deviate from existing evaluation guidelines in the emphasis on information architecture, learnability and operability.

Usability evaluations in user-centred design

NARCIS (Netherlands)

Ruyter, de B.E.R.; Vet, de J.H.M.

1997-01-01

Improving usability of consumer applications has been the focus of much research. Estimating the advantage of these efforts, however, still is very cumbersome since evaluating product improvements in terms of their usability requires good measuring methods and tools. In this paper activities in the

A health literacy and usability heuristic evaluation of a mobile consumer health application.

Science.gov (United States)

Monkman, Helen; Kushniruk, Andre

2013-01-01

Usability and health literacy are two critical factors in the design and evaluation of consumer health information systems. However, methods for evaluating these two factors in conjunction remain limited. This study adapted a set of existing guidelines for the design of consumer health Web sites into evidence-based evaluation heuristics tailored specifically for mobile consumer health applications. In order to test the approach, a mobile consumer health application (app) was then evaluated using these heuristics. In addition to revealing ways to improve the usability of the system, this analysis identified opportunities to augment the content to make it more understandable by users with limited health literacy. This study successfully demonstrated the utility of converting existing design guidelines into heuristics for the evaluation of usability and health literacy. The heuristics generated could be applied for assessing and revising other existing consumer health information systems.

A continuous usability evaluation of an electronic medication administration record application.

Science.gov (United States)

Vicente Oliveros, Noelia; Gramage Caro, Teresa; Pérez Menéndez-Conde, Covadonga; Álvarez-Diaz, Ana María; Martín-Aragón Álvarez, Sagrario; Bermejo Vicedo, Teresa; Delgado Silveira, Eva

2017-12-01

The complexity of an electronic medication administration record (eMAR) has been underestimated by most designers in the past. Usability issues, such as poorly designed user application flow in eMAR, are therefore of vital importance, since they can have a negative impact on nursing activities and result in poor outcomes. The purpose of this study was to evaluate the usability of an eMAR application during its development. A usability evaluation was conducted during the development of the eMAR application. Two usability methods were used: a heuristic evaluation complemented by usability testing. Each eMAR application version provided by the vendor was evaluated by 2 hospital pharmacists, who applied the heuristic method. They reviewed the eMAR tasks, detected usability problems and their heuristic violations, and rated the severity of the usability problems. Usability testing was used to assess the final application version by observing how 3 nurses interacted with the application. Thirty-four versions were assessed before the eMAR application was considered usable. During the heuristic evaluation, the usability problems decreased from 46 unique usability problems in version 1 (V1) to 9 in version 34 (V34). In V1, usability problems were categorized into 154 heuristic violations, which decreased to 27 in V34. The average severity rating also decreased from major usability problem (2.96) to no problem (0.23). During usability testing, the 3 nurses did not encounter new usability problems. A thorough heuristic evaluation is a good method for obtaining a usable eMAR application. This evaluation points key areas for improvement and decreases usability problems and their severity. © 2017 John Wiley & Sons, Ltd.

Evaluation Methods on Usability of M-Learning Environments

Directory of Open Access Journals (Sweden)

Teresa Magal-Royo

2007-10-01

Full Text Available Nowadays there are different evaluation methods focused in the assessment of the usability of telematic methods. The assessment of 3rd generation web environments evaluates the effectiveness and usability of application with regard to the user needs. Wireless usability and, specifically in mobile phones, is concentrated in the validation of the features and tools management using conventional interactive environments. There is not a specific and suitable criterion to evaluate created environments and m-learning platforms, where the restricted and sequential representation is a fundamental aspect to be considered.The present paper exposes the importance of the conventional usability methods to verify both: the employed contents in wireless formats, and the possible interfaces from the conception phases, to the validations of the platform with such characteristics.The development of usability adapted inspection could be complemented with the Remote’s techniques of usability testing, which are being carried out these days in the mobile devices field and which pointed out the need to apply common criteria in the validation of non-located learning scenarios.

Usability and accessibility evaluation of the digital doorway

CSIR Research Space (South Africa)

Adebesin, TF

2011-03-01

Full Text Available The Digital Doorway (DD) is a non-standard computer system deployed to promote computer literacy amongst underprivileged communities in South Africa. Since its inception there has been no usability evaluation of the software installed on the DD...

Usability evaluation techniques in mobile commerce applications: A systematic review

Science.gov (United States)

Hussain, Azham; Mkpojiogu, Emmanuel O. C.

2016-08-01

Obviously, there are a number of literatures concerning the usability of mobile commerce (m-commerce) applications and related areas, but they do not adequately provide knowledge about usability techniques used in most of the empirical usability evaluation for m-commerce application. Therefore, this paper is aimed at producing the usability techniques frequently used in the aspect of usability evaluation for m-commerce applications. To achieve the stated objective, systematic literature review was employed. Sixty seven papers were downloaded in usability evaluation for m-commerce and related areas; twenty one most relevant studies were selected for review in order to extract the appropriate information. The results from the review shows that heuristic evaluation, formal test and think aloud methods are the most commonly used methods in m-commerce application in comparison to cognitive walkthrough and the informal test methods. Moreover, most of the studies applied control experiment (33.3% of the total studies); other studies that applied case study for usability evaluation are 14.28%. The results from this paper provide additional knowledge to the usability practitioners and research community for the current state and use of usability techniques in m-commerce application.

A usability evaluation of a SNOMED CT based compositional interface terminology for intensive care.

Science.gov (United States)

Bakhshi-Raiez, F; de Keizer, N F; Cornet, R; Dorrepaal, M; Dongelmans, D; Jaspers, M W M

2012-05-01

To evaluate the usability of a large compositional interface terminology based on SNOMED CT and the terminology application for registration of the reasons for intensive care admission in a Patient Data Management System. Observational study with user-based usability evaluations before and 3 months after the system was implemented and routinely used. Usability was defined by five aspects: effectiveness, efficiency, learnability, overall user satisfaction, and experienced usability problems. Qualitative (the Think-Aloud user testing method) and quantitative (the System Usability Scale questionnaire and Time-on-Task analyses) methods were used to examine these usability aspects. The results of the evaluation study revealed that the usability of the interface terminology fell short (SUS scores before and after implementation of 47.2 out of 100 and 37.5 respectively out of 100). The qualitative measurements revealed a high number (n=35) of distinct usability problems, leading to ineffective and inefficient registration of reasons for admission. The effectiveness and efficiency of the system did not change over time. About 14% (n=5) of the revealed usability problems were related to the terminology content based on SNOMED CT, while the remaining 86% (n=30) was related to the terminology application. The problems related to the terminology content were more severe than the problems related to the terminology application. This study provides a detailed insight into how clinicians interact with a controlled compositional terminology through a terminology application. The extensiveness, complexity of the hierarchy, and the language usage of an interface terminology are defining for its usability. Carefully crafted domain-specific subsets and a well-designed terminology application are needed to facilitate the use of a complex compositional interface terminology based on SNOMED CT. Copyright © 2011 Elsevier Ireland Ltd. All rights reserved.

Development of a Customizable Health IT Usability Evaluation Scale

Science.gov (United States)

Yen, Po-Yin; Wantland, Dean; Bakken, Suzanne

2010-01-01

We developed a customizable questionnaire, the Health Information Technology (IT) Usability Evaluation Scale (Health-ITUES) and conducted an exploratory factor analysis to examine the scale’s psychometric properties. Nurses (n=377) completed Health-ITUES to rate the usability of a web-based communication system for scheduling nursing staff. The analysis revealed a four-factor structure of Health-ITUES. The results provided preliminary evidence for the factorial validity and internal consistency reliability of Health-ITUES. PMID:21347112

Toward more usable electronic voting: testing the usability of a smartphone voting system.

Science.gov (United States)

Campbell, Bryan A; Tossell, Chad C; Byrne, Michael D; Kortum, Philip

2014-08-01

The goal of this research was to assess the usability of a voting system designed for smart-phones. Smartphones offer remote participation in elections through the use of pervasive technology. Voting on these devices could, among other benefits, increase voter participation while allowing voters to use familiar technology. However, the usability of these systems has not been assessed. A mobile voting system optimized for use on a smartphone was designed and tested against traditional voting platforms for usability. There were no reliable differences between the smartphone-based system and other voting methods in efficiency and perceived usability. More important, though, smartphone owners committed fewer errors on the mobile voting system than on the traditional voting systems. Even with the known limitations of small mobile platforms in both displays and controls, a carefully designed system can provide a usable voting method. Much of the concern about mobile voting is in the area of security; therefore, although these results are promising, security concerns and usability issues arising from mitigating them must be strongly considered. The results of this experiment may help to inform current and future election and public policy officials about the benefits of allowing voters to vote with familiar hardware.

A usability evaluation of a SNOMED CT based compositional interface terminology for intensive care

NARCIS (Netherlands)

Bakhshi-Raiez, F.; de Keizer, N. F.; Cornet, R.; Dorrepaal, M.; Dongelmans, D.; Jaspers, M. W. M.

2012-01-01

Objective: To evaluate the usability of a large compositional interface terminology based on SNOMED CT and the terminology application for registration of the reasons for intensive care admission in a Patient Data Management System. Design: Observational study with user-based usability evaluations

Usability evaluation of healthcare information systems: comparison of methods and classification of usability problems

NARCIS (Netherlands)

Khajouei, R.

2011-01-01

In ziekenhuizen wordt steeds meer gebruik gemaakt van informatiesystemen, maar de usability laat vaak te wensen over. Reza Khajouei testte twee methodes om de usability te beoordelen. Hij constateert dat beide methoden ongeveer evenveel, maar verschillende problemen aan het licht brengen. Uit een

How to Reach Evidence-Based Usability Evaluation Methods

NARCIS (Netherlands)

Marcilly, Romaric; Peute, Linda

2017-01-01

This paper discusses how and why to build evidence-based knowledge on usability evaluation methods. At each step of building evidence, requisites and difficulties to achieve it are highlighted. Specifically, the paper presents how usability evaluation studies should be designed to allow capitalizing

Agents and wearables : usability in the COMRIS system

NARCIS (Netherlands)

Haan, de G.; Terken, J.M.B.

2001-01-01

This paper discusses usability research concerning a wearable system for spoken advice: a field experiment to investigate key concepts, a laboratory experiment to investigate interaction design, and field trials to evaluate the system. Conclusions are drawn about designing wearable appliances.

Determination of the effectiveness of two methods for usability evaluation using a CPOE medication ordering system

NARCIS (Netherlands)

Khajouei, R.; Hasman, A.; Jaspers, M. W. M.

2011-01-01

Objectives: To assess the effectiveness of two usability evaluation methods, cognitive walkthrough (CW) and think aloud (TA), for identifying usability problems and to compare the performance of CW and TA in identifying different types of usability problems. Methods: A CW was performed by two

Post-deployment usability evaluation of a radiology workstation

NARCIS (Netherlands)

Jorritsma, Wiard; Cnossen, Fokie; Dierckx, Rudi; Oudkerk, Matthijs; van Ooijen, Peter

2015-01-01

Objective To evaluate the usability of a radiology workstation after deployment in a hospital. Significance In radiology, it is difficult to perform valid pre-deployment usability evaluations due to the heterogeneity of the user group, the complexity of the radiological workflow, and the complexity

Use Case Evaluation (UCE): A Method for Early Usability Evaluation in Software Development

DEFF Research Database (Denmark)

Stage, Jan; Høegh, Rune Thaarup; Hornbæk, K.

2007-01-01

t is often argued that usability problems should be identified as early as possible during software development, but many usability evaluation methods do not fit well in early development activities. We propose a method for usability evaluation of use cases, a widely used representation of design...... ideas produced early in software development processes. The method proceeds by systematic inspection of use cases with reference to a set of guidelines for usable design. To validate the method, four evaluators inspected a set of use cases for a health care application....

An Evaluation of the Usability of a Computerized Decision Support System for Nursing Homes

Science.gov (United States)

Fossum, M.; Ehnfors, M.; Fruhling, A.; Ehrenberg, A.

2011-01-01

Background Computerized decision support systems (CDSSs) have the potential to significantly improve the quality of nursing care of older people by enhancing the decision making of nursing personnel. Despite this potential, health care organizations have been slow to incorporate CDSSs into nursing home practices. Objective This study describes facilitators and barriers that impact the ability of nursing personnel to effectively use a clinical CDSS for planning and treating pressure ulcers (PUs) and malnutrition and for following the suggested risk assessment guidelines for the care of nursing home residents. Methods We employed a qualitative descriptive design using varied methods, including structured group interviews, cognitive walkthrough observations and a graphical user interface (GUI) usability evaluation. Group interviews were conducted with 25 nursing personnel from four nursing homes in southern Norway. Five nursing personnel participated in cognitive walkthrough observations and the GUI usability evaluation. Text transcripts were analyzed using qualitative content analysis. Results Group interview participants reported that ease of use, usefulness and a supportive work environment were key facilitators of CDSS use. The barriers identified were lack of training, resistance to using computers and limited integration of the CDSS with the facility’s electronic health record (EHR) system. Key findings from the usability evaluation also identified the difficulty of using the CDSS within the EHR and the poorly designed GUI integration as barriers. Conclusion Overall, we found disconnect between two types of nursing personnel. Those who were comfortable with computer technology reported positive feedback about the CDSS, while others expressed resistance to using the CDSS for various reasons. This study revealed that organizations must invest more resources in educating nursing personnel on the seriousness of PUs and poor nutrition in the elderly, providing

The significance of a usability evaluation of an emerging laboratory order entry system

NARCIS (Netherlands)

Peute, Linda W. P.; Jaspers, Monique W. M.

2007-01-01

OBJECTIVES: To assess the usability of an emerging POE system, OM/Lab, for the computer-supported ordering of laboratory tests. We were more specifically interested in the relation of the usability problems detected in the user testing sessions with the order behaviour in terms of efficiency and

Usability evaluation of the interactive Personal Patient Profile-Prostate decision support system with African American men.

Science.gov (United States)

Jaja, Cheedy; Pares-Avila, Jose; Wolpin, Seth; Berry, Donna

2010-04-01

The Personal Patient Profile-Prostate (P4) program is an interactive Web-based decision support system that provides men with localized prostate cancer customized education and coaching with which to make the best personal treatment decision. This study assessed functionality and usability of the P4 program and identified problems in user-computer interaction in a sample of African American men. Usability testing was conducted with 12 community-dwelling African American adult men. The health status of participants was not known or collected by the research team. Each participant worked with the P4 program and provided simultaneous feedback using the "think aloud" technique. Handwritten field notes were collated and assigned to 3 standard coded categories. Aspects of P4 program usability was made based on common issues in the assigned categories. Summary statistics were derived for types and frequency of usability issues noted in the coded data. Twelve participants reported a total of 122 usability comments, with a mean of 9 usability comments. The most common usability issue by participant was completeness of information content, which comprised 53 (43%) of the total issues. Comprehensibility of text and graphics was second, comprising 51 (42%) of the total issues. This study provided initial inventory of usability issues for community African American men that may potentially interfere with application of the P4 system in the community setting and overall system usability, confirming the need for usability testing of a culturally appropriate Internet-based decision support system before community application.

Usability Prediction & Ranking of SDLC Models Using Fuzzy Hierarchical Usability Model

Science.gov (United States)

Gupta, Deepak; Ahlawat, Anil K.; Sagar, Kalpna

2017-06-01

Evaluation of software quality is an important aspect for controlling and managing the software. By such evaluation, improvements in software process can be made. The software quality is significantly dependent on software usability. Many researchers have proposed numbers of usability models. Each model considers a set of usability factors but do not cover all the usability aspects. Practical implementation of these models is still missing, as there is a lack of precise definition of usability. Also, it is very difficult to integrate these models into current software engineering practices. In order to overcome these challenges, this paper aims to define the term `usability' using the proposed hierarchical usability model with its detailed taxonomy. The taxonomy considers generic evaluation criteria for identifying the quality components, which brings together factors, attributes and characteristics defined in various HCI and software models. For the first time, the usability model is also implemented to predict more accurate usability values. The proposed system is named as fuzzy hierarchical usability model that can be easily integrated into the current software engineering practices. In order to validate the work, a dataset of six software development life cycle models is created and employed. These models are ranked according to their predicted usability values. This research also focuses on the detailed comparison of proposed model with the existing usability models.

Commercial versus in-situ usability testing of healthcare information systems: towards "public" usability testing in healthcare organizations.

Science.gov (United States)

Kushniruk, Andre W; Borycki, Elizabeth M; Kannry, Joseph

2013-01-01

The need for improved usability in healthcare IT has been widely recognized. In addition, methods from usability engineering, including usability testing and usability inspection have received greater attention. Many vendors of healthcare software are now employing usability testing methods in the design and development of their products. However, despite this, the usability of healthcare IT is still considered to be problematic and many healthcare organizations that have purchased systems that have been tested at vendor testing sites are still reporting a range of usability and safety issues. In this paper we explore the distinction between commercial usability testing (conducted at centralized vendor usability laboratories and limited beta test sites) and usability testing that is carried out locally within healthcare organizations that have purchased vendor systems and products (i.e. public "in-situ" usability testing). In this paper it will be argued that both types of testing (i.e. commercial vendor-based testing) and in-situ testing are needed to ensure system usability and safety.

Direct Integration: Training Software Developers to Conduct Usability Evaluations

DEFF Research Database (Denmark)

Skov, Mikael B.; Stage, Jan

2008-01-01

is based on an empirical study where 36 teams with a total of 234 first-year university students on software development and design educations were trained in a simple approach for user-based website usability testing that was taught in a 40 hour course. This approach supported them in planning, conducting......Many improvements of the interplay between usability evaluation and software development rely either on better methods for conducting usability evaluations or on better formats for presenting evaluation results in ways that are useful for software designers and developers. Both approaches involve...... a complete division of work between developers and evaluators, which is an undesirable complexity for many software development projects. This paper takes a different approach by exploring to what extent software developers and designers can be trained to carry out their own usability evaluations. The paper...

Usability of a patient education and motivation tool using heuristic evaluation.

Science.gov (United States)

Joshi, Ashish; Arora, Mohit; Dai, Liwei; Price, Kathleen; Vizer, Lisa; Sears, Andrew

2009-11-06

Computer-mediated educational applications can provide a self-paced, interactive environment to deliver educational content to individuals about their health condition. These programs have been used to deliver health-related information about a variety of topics, including breast cancer screening, asthma management, and injury prevention. We have designed the Patient Education and Motivation Tool (PEMT), an interactive computer-based educational program based on behavioral, cognitive, and humanistic learning theories. The tool is designed to educate users and has three key components: screening, learning, and evaluation. The objective of this tutorial is to illustrate a heuristic evaluation using a computer-based patient education program (PEMT) as a case study. The aims were to improve the usability of PEMT through heuristic evaluation of the interface; to report the results of these usability evaluations; to make changes based on the findings of the usability experts; and to describe the benefits and limitations of applying usability evaluations to PEMT. PEMT was evaluated by three usability experts using Nielsen's usability heuristics while reviewing the interface to produce a list of heuristic violations with severity ratings. The violations were sorted by heuristic and ordered from most to least severe within each heuristic. A total of 127 violations were identified with a median severity of 3 (range 0 to 4 with 0 = no problem to 4 = catastrophic problem). Results showed 13 violations for visibility (median severity = 2), 38 violations for match between system and real world (median severity = 2), 6 violations for user control and freedom (median severity = 3), 34 violations for consistency and standards (median severity = 2), 11 violations for error severity (median severity = 3), 1 violation for recognition and control (median severity = 3), 7 violations for flexibility and efficiency (median severity = 2), 9 violations for aesthetic and minimalist design

Usability Evaluation of An Electronic Medication Administration Record (eMAR) Application

Science.gov (United States)

Guo, J.; Iribarren, S.; Kapsandoy, S.; Perri, S.; Staggers, N.

2011-01-01

Background Electronic medication administration records (eMARs) have been widely used in recent years. However, formal usability evaluations are not yet available for these vendor applications, especially from the perspective of nurses, the largest group of eMAR users. Objective To conduct a formal usability evaluation of an implemented eMAR. Methods Four evaluators examined a commercial vendor eMAR using heuristic evaluation techniques. The evaluators defined seven tasks typical of eMAR use and independently evaluated the application. Consensus techniques were used to obtain 100% agreement of identified usability problems and severity ratings. Findings were reviewed with 5 clinical staff nurses and the Director of Clinical Informatics who verified findings with a small group of clinical nurses. Results Evaluators found 60 usability problems categorized into 233 heuristic violations. Match, Error, and Visibility heuristics were the most frequently violated. Administer Medication and Order and Modify Medications tasks had the highest number of heuristic violations and usability problems rated as major or catastrophic. Conclusion The high number of usability problems could impact the effectiveness, efficiency and satisfaction of nurses’ medication administration activities and may include concerns about patient safety. Usability is a joint responsibility between sites and vendors. We offer a call to action for usability evaluations at all sites and eMAR application redesign as necessary to improve the user experience and promote patient safety. PMID:23616871

Social Context in Usability Evaluations: Concepts, Processes and Products

DEFF Research Database (Denmark)

Jensen, Janne Jul

social context is considered important, only little research has been done to identify how it influences usability evaluations. In this thesis I explore how social context affects the process and product of a usability evaluation and explain the findings in terms of the theory of behaviour settings...... leader) and non-operatives (members, spectators, neutrals and potentials) from the theory of behaviour settings to usability evaluations generates an understanding and create an awareness of the level of power possessed by each of the participants in the social context. 2. On the operative level...... and a field experiment. Findings from these activities are presented in five published paper contributions. I furthermore introduce the theory of behaviour settings as a tool to help characterise the key concepts of social context which, together with an understanding of usability evaluations, provide...

The evaluator effect in usability studies: Problem detection and severity judgments

DEFF Research Database (Denmark)

Jacobsen, Niels Ebbe; Hertzum, Morten; John, Bonnie E.

1998-01-01

Usability studies are commonly used in industry and applied in research as a yardstick for other usability evaluation methods. Though usability studies have been studied extensively, one potential threat to their reliability has been left virtually untouched: the evaluator effect. In this study......, four evaluators individually analyzed four videotaped usability test sessions. Only 20% of the 93 detected problems were detected by all evaluators, and 46% were detected by only a single evaluator. From the total set of 93 problems the evaluators individually selected the ten problems they considered...... most severe. None of the selected severe problems appeared on all four evaluators’ top-10 lists, and 4 of the 11 problems that were considered severe by more than one evaluator were only detected by one or two evaluators. Thus, both detection of usability problems and selection of the most severe...

Training Software Developers and Designers to Conduct Usability Evaluations

Science.gov (United States)

Skov, Mikael Brasholt; Stage, Jan

2012-01-01

Many efforts to improve the interplay between usability evaluation and software development rely either on better methods for conducting usability evaluations or on better formats for presenting evaluation results in ways that are useful for software designers and developers. Both of these approaches depend on a complete division of work between…

Health Information Technology Usability Evaluation Scale (Health-ITUES) for Usability Assessment of Mobile Health Technology: Validation Study.

Science.gov (United States)

Schnall, Rebecca; Cho, Hwayoung; Liu, Jianfang

2018-01-05

Mobile technology has become a ubiquitous technology and can be particularly useful in the delivery of health interventions. This technology can allow us to deliver interventions to scale, cover broad geographic areas, and deliver technologies in highly tailored ways based on the preferences or characteristics of users. The broad use of mobile technologies supports the need for usability assessments of these tools. Although there have been a number of usability assessment instruments developed, none have been validated for use with mobile technologies. The goal of this work was to validate the Health Information Technology Usability Evaluation Scale (Health-ITUES), a customizable usability assessment instrument in a sample of community-dwelling adults who were testing the use of a new mobile health (mHealth) technology. A sample of 92 community-dwelling adults living with HIV used a new mobile app for symptom self-management and completed the Health-ITUES to assess the usability of the app. They also completed the Post-Study System Usability Questionnaire (PSSUQ), a widely used and well-validated usability assessment tool. Correlations between these scales and each of the subscales were assessed. The subscales of the Health-ITUES showed high internal consistency reliability (Cronbach alpha=.85-.92). Each of the Health-ITUES subscales and the overall scale was moderately to strongly correlated with the PSSUQ scales (r=.46-.70), demonstrating the criterion validity of the Health-ITUES. The Health-ITUES has demonstrated reliability and validity for use in assessing the usability of mHealth technologies in community-dwelling adults living with a chronic illness. ©Rebecca Schnall, Hwayoung Cho, Jianfang Liu. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 05.01.2018.

A Usability Evaluation Model for Academic Library Websites: Efficiency, Effectiveness and Learnability

Directory of Open Access Journals (Sweden)

Soohyung Joo

2011-12-01

Full Text Available Purpose – This paper aimed to develop a usability evaluation model and associated survey tool in the context of academic libraries. This study not only proposed a usability evaluation model but also a practical survey tool tailored to academic library websites. Design/methodology – A usability evaluation model has been developed for academic library websites based on literature review and expert consultation. Then, the authors verified the reliability and validity of the usability evaluation model empirically using the survey data from actual users. Statistical analysis, such as descriptive statistics, internal consistency test, and a factor analysis, were applied to ensure both the reliability and validity of the usability evaluation tool. Findings – From the document analysis and expert consultation, this study identified eighteen measurement items to survey the three constructs of the usability, effectiveness, efficiency, and learnability, in academic library websites. The evaluation tool was then validated with regard to data distribution, reliability, and validity. The empirical examination based on 147 actual user responses proved the survey evaluation tool suggested herein is acceptable in assessing academic library website usability. Originality/Value – This research is one of the few studies to engender a practical survey tool in evaluating library website usability. The usability model and corresponding survey tool would be useful for librarians and library administrators in academic libraries who plan to conduct a usability evaluation involving large sample.

Usability Evaluation Methods for Gesture-Based Games: A Systematic Review.

Science.gov (United States)

Simor, Fernando Winckler; Brum, Manoela Rogofski; Schmidt, Jaison Dairon Ebertz; Rieder, Rafael; De Marchi, Ana Carolina Bertoletti

2016-10-04

Gestural interaction systems are increasingly being used, mainly in games, expanding the idea of entertainment and providing experiences with the purpose of promoting better physical and/or mental health. Therefore, it is necessary to establish mechanisms for evaluating the usability of these interfaces, which make gestures the basis of interaction, to achieve a balance between functionality and ease of use. This study aims to present the results of a systematic review focused on usability evaluation methods for gesture-based games, considering devices with motion-sensing capability. We considered the usability methods used, the common interface issues, and the strategies adopted to build good gesture-based games. The research was centered on four electronic databases: IEEE, Association for Computing Machinery (ACM), Springer, and Science Direct from September 4 to 21, 2015. Within 1427 studies evaluated, 10 matched the eligibility criteria. As a requirement, we considered studies about gesture-based games, Kinect and/or Wii as devices, and the use of a usability method to evaluate the user interface. In the 10 studies found, there was no standardization in the methods because they considered diverse analysis variables. Heterogeneously, authors used different instruments to evaluate gesture-based interfaces and no default approach was proposed. Questionnaires were the most used instruments (70%, 7/10), followed by interviews (30%, 3/10), and observation and video recording (20%, 2/10). Moreover, 60% (6/10) of the studies used gesture-based serious games to evaluate the performance of elderly participants in rehabilitation tasks. This highlights the need for creating an evaluation protocol for older adults to provide a user-friendly interface according to the user's age and limitations. Through this study, we conclude this field is in need of a usability evaluation method for serious games, especially games for older adults, and that the definition of a methodology

Evaluation of Virtual Teaching-Learning Environments based on usability standards

Directory of Open Access Journals (Sweden)

Jose I. Cocunubo-Suarez

2018-01-01

Full Text Available The main objective of this review article is to determine the necessary sub-characteristics or aspects for the evaluation of Virtual Teaching-Learning Environments (VTLEs as final or finished products based on ISO 9126, 14598 and 25000-SQuaRE standards. A systematic information search was carried out. A total of 108 documents were retrieved about subjects such as web usability, virtual learning environments, usability, educational software, educational web evaluation, usability evaluation and web usability evaluation. Out of the 108 documents, 70 were selected by inclusion and exclusion analysis. The eight subfeatures of greater statistical frequency were identified among the subset of documents and then integrated as a proposal for standard 25000-SQuaRE.

A human reliability based usability evaluation method for safety-critical software

International Nuclear Information System (INIS)

Boring, R. L.; Tran, T. Q.; Gertman, D. I.; Ragsdale, A.

2006-01-01

Boring and Gertman (2005) introduced a novel method that augments heuristic usability evaluation methods with that of the human reliability analysis method of SPAR-H. By assigning probabilistic modifiers to individual heuristics, it is possible to arrive at the usability error probability (UEP). Although this UEP is not a literal probability of error, it nonetheless provides a quantitative basis to heuristic evaluation. This method allows one to seamlessly prioritize and identify usability issues (i.e., a higher UEP requires more immediate fixes). However, the original version of this method required the usability evaluator to assign priority weights to the final UEP, thus allowing the priority of a usability issue to differ among usability evaluators. The purpose of this paper is to explore an alternative approach to standardize the priority weighting of the UEP in an effort to improve the method's reliability. (authors)

Usability evaluation of a locomotor therapy device considering different strategies

Directory of Open Access Journals (Sweden)

Langthaler Sonja

2016-09-01

Full Text Available Usability of medical devices is one of the main determining factors in preventing use errors in treatment and strongly correlates to patient safety and quality of treatment. This thesis demonstrates the usability testing and evaluation of a prototype for locomotor therapy of infants. Therefore, based on the normative requirements of the EN 62366, a concept combined of evaluation procedures and assessing methods was created to enable extensive testing and analysis of the different aspects of usability. On the basis of gathered information weak points were identified and appropriate measures were presented to increase the usability and operating safety of the locomotor prototype. The overall outcome showed an usability value of 77.4% and an evaluation score of 6.99, which can be interpreted as “satisfactory”.

Evaluating the Usability of a Controlled Language Authoring Assistant

Directory of Open Access Journals (Sweden)

Miyata Rei

2017-06-01

Full Text Available This paper presents experimental results of a usability evaluation of a controlled language (CL authoring assistant designed to help non-professional writers create machine translatable source texts. As the author drafts the text, the system detects CL rule violations and proscribed terms. It also incorporates several support functions to facilitate rephrasing of the source. In order to assess the usability of the system, we conducted a rewriting experiment, in which we compared two groups of participants, one with the aid of the system and the other without it. The results revealed that our system helped reduce the number of CL violations by about 9% and the time to correct violations by more than 30%. The CL-applied source text resulted in higher fluency and adequacy of MT outputs. Questionnaire and interview results also implied the improved satisfaction with the task completion of those participants who used the system.

Post-deployment usability evaluation of a radiology workstation

NARCIS (Netherlands)

Jorritsma, Wiard; Cnossen, Fokie; Dierckx, Rudi A.; Oudkerk, Matthijs; Van Ooijen, Peter M. A.

Objectives: To determine the number, nature and severity of usability issues radiologists encounter while using a commercially available radiology workstation in clinical practice, and to assess how well the results of a pre-deployment usability evaluation of this workstation generalize to clinical

A Cross-Cultural Study of How Usability Professionals Experience the Usability of Everyday Systems

DEFF Research Database (Denmark)

Jiang, Y.; Sun, X.; Li, H.

2009-01-01

in the study, their understanding of fun systems differs across cultural backgrounds. Also, easy-to-use and useful systems are perceived as being similar or different depending on the usability professional’s cultural background. Most other cross-cultural differences relate to categories of construct......Culture influences many aspects of the design and use of computer systems; understanding better this influence on their own thinking may benefit usability professionals who do cross-cultural usability work. Using Kelly’s notion of personal constructs, we focus on one mediator of culture: how...

Usability evaluation model for mobile e-book applications

Science.gov (United States)

Matraf, Munya Saleh Ba; Hussain, Azham

2017-10-01

Evaluation for mobile e-book applications are limited and did not address all the important usability measurements. Hence, this study aimed to identify the characteristics that affect user satisfaction on the usability of mobile e-book applications. Five characteristics that have a significant effect on the user satisfaction of mobile e-book applications have been identified namely readability, effectiveness, accessibility, efficiency, and navigation. A usability evaluation was conducted on three mobile e-book applications namely Adobe Acrobat Reader, Ebook Reader, and Amazon Kindle. 30 students from Universiti Utara Malaysia evaluated the mobile e-book applications and their satisfaction was measured using questionnaire. The outcomes discovered that the five characteristics have a significant positive relationship with user satisfaction. This provides insights into the main characteristics that increase user satisfaction.

Usability evaluation of mobile applications using ISO 9241 and ISO 25062 standards.

Science.gov (United States)

Moumane, Karima; Idri, Ali; Abran, Alain

2016-01-01

This paper presents an empirical study based on a set of measures to evaluate the usability of mobile applications running on different mobile operating systems, including Android, iOS and Symbian. The aim is to evaluate empirically a framework that we have developed on the use of the Software Quality Standard ISO 9126 in mobile environments, especially the usability characteristic. To do that, 32 users had participated in the experiment and we have used ISO 25062 and ISO 9241 standards for objective measures by working with two widely used mobile applications: Google Apps and Google Maps. The QUIS 7.0 questionnaire have been used to collect measures assessing the users' level of satisfaction when using these two mobile applications. By analyzing the results we highlighted a set of mobile usability issues that are related to the hardware as well as to the software and that need to be taken into account by designers and developers in order to improve the usability of mobile applications.

Interplay between usability evaluation and software development (I-USED 2009)

DEFF Research Database (Denmark)

Abrahão, S.; Hornbæk, Kasper Anders Søren; Law, E.

2009-01-01

This workshop is aimed at bringing together researchers and practitioners from the Human-Computer Interaction (HCI) and Software Engineering (SE) fields to determine the state-of-the-art in the interplay between usability evaluation and software development and to generate ideas for new...... and improved relations between these activities. The aim is to base the determination of the current state on empirical studies. Presentations of new ideas on how to improve the interplay between HCI & SE to the design of usable software systems should also be based on empirical studies....

Towards a Usability and Error "Safety Net": A Multi-Phased Multi-Method Approach to Ensuring System Usability and Safety.

Science.gov (United States)

Kushniruk, Andre; Senathirajah, Yalini; Borycki, Elizabeth

2017-01-01

The usability and safety of health information systems have become major issues in the design and implementation of useful healthcare IT. In this paper we describe a multi-phased multi-method approach to integrating usability engineering methods into system testing to ensure both usability and safety of healthcare IT upon widespread deployment. The approach involves usability testing followed by clinical simulation (conducted in-situ) and "near-live" recording of user interactions with systems. At key stages in this process, usability problems are identified and rectified forming a usability and technology-induced error "safety net" that catches different types of usability and safety problems prior to releasing systems widely in healthcare settings.

Towards an empirical method of usability testing of system parts : a methodological study

NARCIS (Netherlands)

Brinkman, W.P.; Haakma, R.; Bouwhuis, D.G.

2007-01-01

Current usability evaluation methods are essentially holistic in nature. However, engineers that apply a component-based software engineering approach might also be interested in understanding the usability of individual parts of an interactive system. This paper examines the efficiency dimension of

A Usability Evaluation of Academic Virtual Reference Services

Science.gov (United States)

Chow, Anthony S.; Croxton, Rebecca A.

2014-01-01

This study examined the usability of five virtual reference services--instant messenger chat, e-mail, telephone, text messaging, and Skype videoconferencing--by having 31 undergraduate and graduate students evaluate the usability of the virtual reference services of two different universities. The study's results suggest that user preference and…

A Longitudinal Study of Usability in Health Care

DEFF Research Database (Denmark)

Kjeldskov, Jesper; Skov, Mikael B.; Stage, Jan

2010-01-01

We report from a longitudinal laboratory-based usability evaluation of a health care information system. The purpose of the study was to inquire into the nature of usability problems experienced by novice and expert users, and to see to what extend usability problems of a health care information...... system may or may not disappear over time, as the nurses get more familiar with it-if time heals poor design? As our method for studying this, we conducted a longitudinal study with two key studies. A usability evaluation was conducted with novice users when an electronic patient record system was being......, we discuss implications for evaluating usability in health care....

Using a fuzzy comprehensive evaluation method to determine product usability: A test case.

Science.gov (United States)

Zhou, Ronggang; Chan, Alan H S

2017-01-01

In order to take into account the inherent uncertainties during product usability evaluation, Zhou and Chan [1] proposed a comprehensive method of usability evaluation for products by combining the analytic hierarchy process (AHP) and fuzzy evaluation methods for synthesizing performance data and subjective response data. This method was designed to provide an integrated framework combining the inevitable vague judgments from the multiple stages of the product evaluation process. In order to illustrate the effectiveness of the model, this study used a summative usability test case to assess the application and strength of the general fuzzy usability framework. To test the proposed fuzzy usability evaluation framework [1], a standard summative usability test was conducted to benchmark the overall usability of a specific network management software. Based on the test data, the fuzzy method was applied to incorporate both the usability scores and uncertainties involved in the multiple components of the evaluation. Then, with Monte Carlo simulation procedures, confidence intervals were used to compare the reliabilities among the fuzzy approach and two typical conventional methods combining metrics based on percentages. This case study showed that the fuzzy evaluation technique can be applied successfully for combining summative usability testing data to achieve an overall usability quality for the network software evaluated. Greater differences of confidence interval widths between the method of averaging equally percentage and weighted evaluation method, including the method of weighted percentage averages, verified the strength of the fuzzy method.

The impact of usability reports and user test observations on developers understanding of usability data

DEFF Research Database (Denmark)

Høegh, Rune Thaarup; Nielsen, Christian Monrad; Pedersen, Michael Bach

2006-01-01

of the system. This article presents results from an exploratory study of 2 ways of providing feedback from a usability evaluation: observation of user tests and reading usability reports. A case study and a field experiment were used to explore how observation and usability reports impact developers......' understanding of usability data. The results indicate that observation of user tests facilitated a rich understanding of usability problems and created empathy with the users and their work. The usability report had a strong impact on the developers' understanding of specific usability problems and supported...

Usability Evaluation Method for Agile Software Development

Directory of Open Access Journals (Sweden)

Saad Masood Butt

2015-02-01

Full Text Available Agile methods are the best fit for tremendously growing software industry due to its flexible and dynamic nature. But the software developed using agile methods do meet the usability standards? To answer this question we can see that majority of agile software development projects currently involve interactive user interface designs, which can only be possible by following User Centered Design (UCD in agile methods. The question here is, how to integrate UCD with agile models. Both Agile models and UCD are iterative in nature but agile models focus on coding and development of software; whereas, UCD focuses on user interface of the software. Similarly, both of them have testing features where the agile model involves automated tested code while UCD involves an expert or a user to test the user interface. In this paper, a new agile usability model is proposed and the evaluation is of the proposed model is presented by practically implementing it in three real life projects. . Key results from these projects clearly show: the proposed agile model incorporates usability evaluation methods, improves the relationship between usability experts to work with agile software experts; in addition, allows agile developers to incorporate the result from UCD into subsequent interactions.

Course management systems from a usability perspective

CSIR Research Space (South Africa)

Padayachee, I

2011-01-01

Full Text Available (with knowledge of usability engineering and system domain knowledge) who inspect the Indira Padayachee, Paula Kotz? and Alta van der Merwe 302 interface and determine its compliance with recognized usability principles (the ?heuristics... presents a summary of the results of the severity rating. Table 2: Severity Rating Scale for Usability Problems (Nielsen 2007b) 0 I don?t agree that this is a usability problem at all 1 Cosmetic problem only: need not be fixed unless there is extra...

Multi-method laboratory user evaluation of an actionable clinical performance information system: Implications for usability and patient safety.

Science.gov (United States)

Brown, Benjamin; Balatsoukas, Panos; Williams, Richard; Sperrin, Matthew; Buchan, Iain

2018-01-01

Electronic audit and feedback (e-A&F) systems are used worldwide for care quality improvement. They measure health professionals' performance against clinical guidelines, and some systems suggest improvement actions. However, little is known about optimal interface designs for e-A&F, in particular how to present suggested actions for improvement. We developed a novel theory-informed system for primary care (the Performance Improvement plaN GeneratoR; PINGR) that covers the four principal interface components: clinical performance summaries; patient lists; detailed patient-level information; and suggested actions. As far as we are aware, this is the first report of an e-A&F system with all four interface components. (1) Use a combination of quantitative and qualitative methods to evaluate the usability of PINGR with target end-users; (2) refine existing design recommendations for e-A&F systems; (3) determine the implications of these recommendations for patient safety. We recruited seven primary care physicians to perform seven tasks with PINGR, during which we measured on-screen behaviour and eye movements. Participants subsequently completed usability questionnaires, and were interviewed in-depth. Data were integrated to: gain a more complete understanding of usability issues; enhance and explain each other's findings; and triangulate results to increase validity. Participants committed a median of 10 errors (range 8-21) when using PINGR's interface, and completed a median of five out of seven tasks (range 4-7). Errors violated six usability heuristics: clear response options; perceptual grouping and data relationships; representational formats; unambiguous description; visually distinct screens for confusable items; and workflow integration. Eye movement analysis revealed the integration of components largely supported effective user workflow, although the modular design of clinical performance summaries unnecessarily increased cognitive load. Interviews and

Web usability evaluation on BloobIS website by using hallway usability testing method and ISO 9241:11

Science.gov (United States)

Dwi Susanto, Tony; Ingesti Prasetyo, Anisa; Astuti, Hanim Maria

2018-03-01

At this moment, the need for web as an information media is highly important. Not only confined in the infotainment area, government, and education, but health as well uses the web media as a medium for providing information effectively. BloobIS is a web based application which integrates blood supply and distribution information at the Blood Transfusion Unit. Knowing how easy information is on BloobIS is marked by how convenient the website is used. Up until now, the BloobIS website is nearing completion but testing has not yet been performed to users and is on the testing and development phase in the Development Life Cycle software. Thus, an evaluation namely the quality control software which focuses on the perspective of BloobIs web usability is required. Hallway Usability Testing and ISO 9241:11 are the methods chosen to measure the BloobIS application usability. The expected outputs of the quality control software focusing on the usability evaluation are being able to rectify the usability deficiencies on the BloobIs web and provide recommendations to develop the web as a basic BloobIS web quality upgraed which sets a goal to amplify the satisfaction of web users based on usability factors in ISO 9241:11.

Usability evaluation of augmented reality user interfaces in games and personal navigation assistants

OpenAIRE

Ivanec, Zorian

2017-01-01

This paper studies heuristic usability evaluation of user interfaces. The literature review distinguishes characteristics of augmented reality and smartphones, presents heuristic evaluation, PACMAD usability model, personal navigation assistants, video games, augmented reality and J. Nielsen usability heuristics. In research part, usability heuristics of related subject fields and design guidelines are combined. Created heuristics and control questionnaire are used for evaluation of augmented...

Tracing Impact in a Usability Improvement Process

DEFF Research Database (Denmark)

Uldall-Espersen, Tobias; Frøkjær, Erik; Hornbæk, Kasper Anders Søren

2008-01-01

Analyzing usability improvement processes as they take place in real-life organizations is necessary to understand the practice of usability work. This paper describes a case study where the usability of an information system is improved and a relationship between the improvements...... and the evaluation efforts is established. Results show that evaluation techniques complemented each other by suggesting different kinds of usability improvement. Among the techniques applied, a combination of questionnaires and Metaphors of Human Thinking (MOT) showed the largest mean impact and MOT produced...... the largest number of impacts. Logging of real-life use of the system over 6 months indicated six aspects of improved usability, where significant differences among evaluation techniques were found. Concerning five of the six aspects Think Aloud evaluations and the above-mentioned combination of questionnaire...

Usability of Low-Cost Android Data Collection System for Community-Based Participatory Research.

Science.gov (United States)

Salihu, Hamisu M; Salinas-Miranda, Abraham; Turner, DeAnne; King, Lindsey; Paothong, Arnut; Austin, Deborah; Berry, Estrellita Lo

2016-01-01

Android tablet computers can be valuable tools for data collection, but their usability has not been evaluated in community-based participatory research (CBPR). This article examines the usability of a low-cost bilingual touchscreen computerized survey system using Android tablets, piloted with a sample of 201 community residents in Tampa, Florida, from November 2013 to March 2014. Needs assessment questions were designed with the droidSURVEY software, and deployed using Android tablet computers. In addition, participants were asked questions about system usability. The mean system usability was 77.57 ± 17.66 (range, 0-100). The mean completion time for taking the 63 survey questions in the needs assessment was 23.11 ± 9.62 minutes. The survey completion rate was optimal (100%), with only 6.34% missingness per variable. We found no sociodemographic differences in usability scores. Our findings indicate that Android tablets could serve as useful tools in CBPR studies.

Evidence-based Heuristics for Evaluating Demands on eHealth Literacy and Usability in a Mobile Consumer Health Application.

Science.gov (United States)

Monkman, Helen; Griffith, Janessa; Kushniruk, Andre W

2015-01-01

Heuristic evaluations have proven to be valuable for identifying usability issues in systems. Commonly used sets of heuritics exist; however, they may not always be the most suitable, given the specific goal of the analysis. One such example is seeking to evaluate the demands on eHealth literacy and usability of consumer health information systems. In this study, eight essential heuristics and three optional heuristics subsumed from the evidence on eHealth/health literacy and usability were tested for their utility in assessing a mobile blood pressure tracking application (app). This evaluation revealed a variety of ways the design of the app could both benefit and impede users with limited eHealth literacy. This study demonstrated the utility of a low-cost, single evaluation approach for identifying both eHealth literacy and usability issues based on existing evidence in the literature.

Usability Evaluation and Implementation of a Health Information Technology Dashboard of Evidence-Based Quality Indicators.

Science.gov (United States)

Schall, Mark Christopher; Cullen, Laura; Pennathur, Priyadarshini; Chen, Howard; Burrell, Keith; Matthews, Grace

2017-06-01

Health information technology dashboards that integrate evidence-based quality indicators can efficiently and accurately display patient risk information to promote early intervention and improve overall quality of patient care. We describe the process of developing, evaluating, and implementing a dashboard designed to promote quality care through display of evidence-based quality indicators within an electronic health record. Clinician feedback was sought throughout the process. Usability evaluations were provided by three nurse pairs and one physician from medical-surgical areas. Task completion times, error rates, and ratings of system usability were collected to compare the use of quality indicators displayed on the dashboard to the indicators displayed in a conventional electronic health record across eight experimental scenarios. Participants rated the dashboard as "highly usable" following System Usability Scale (mean, 87.5 [SD, 9.6]) and Poststudy System Usability Questionnaire (mean, 1.7 [SD, 0.5]) criteria. Use of the dashboard led to reduced task completion times and error rates in comparison to the conventional electronic health record for quality indicator-related tasks. Clinician responses to the dashboard display capabilities were positive, and a multifaceted implementation plan has been used. Results suggest application of the dashboard in the care environment may lead to improved patient care.

The Design of Application-Specific Heuristics for the Usability Evaluation of the Digital Doorway

Directory of Open Access Journals (Sweden)

Funmi Adebesin

2012-06-01

Full Text Available The Digital Doorway (DD is a joint initiative between the South African Department of Science and Technology (DST and the Meraka Institute of the Council for Scientific and Industrial Research (CSIR. The DD is a non-standard computer system deployed amongst underprivileged communities in South Africa with the objective to promote computer literacy. Since its inception, there has been no usability or accessibility evaluation of the software installed on the DD, mainly due to lack of usability engineering or interaction design expertise within the development team. The goal of the research presented in this paper was to design a solution to this problem by developing a suitable instrument that could guide DD application developers in the design and development of more usable DD software and interfaces. Design research was used as a research methodology. We first investigated the applicability of the standard usability and accessibility evaluation methods for evaluating the software installed on the DD. During the first cycle of design research, we established that a heuristic-like evaluation method would be an appropriate method for evaluating the usability and direct accessibility support provided by the DD. During a second cycle of design research, embedded in the first, we also developed a set of multi-category heuristics as the ‘instrument’ that could guide the developers during design of applications as well as in the first-level (formative evaluation thereof. To verify the heuristics, we conducted a usability evaluation of the DD and triangulated the results with a direct field observation at a natural environment of DD use, together with user-administered questionnaires.

Investigation and evaluation into the usability of human-computer interfaces using a typical CAD system

Energy Technology Data Exchange (ETDEWEB)

Rickett, J D

1987-01-01

This research program covers three topics relating to the human-computer interface namely, voice recognition, tools and techniques for evaluation, and user and interface modeling. An investigation into the implementation of voice-recognition technologies examines how voice recognizers may be evaluated in commercial software. A prototype system was developed with the collaboration of FEMVIEW Ltd. (marketing a CAD package). A theoretical approach to evaluation leads to the hypothesis that human-computer interaction is affected by personality, influencing types of dialogue, preferred methods for providing helps, etc. A user model based on personality traits, or habitual-behavior patterns (HBP) is presented. Finally, a practical framework is provided for the evaluation of human-computer interfaces. It suggests that evaluation is an integral part of design and that the iterative use of evaluation techniques throughout the conceptualization, design, implementation and post-implementation stages will ensure systems that satisfy the needs of the users and fulfill the goal of usability.

The E-health Literacy Demands of Australia's My Health Record: A Heuristic Evaluation of Usability.

Science.gov (United States)

Walsh, Louisa; Hemsley, Bronwyn; Allan, Meredith; Adams, Natalie; Balandin, Susan; Georgiou, Andrew; Higgins, Isabel; McCarthy, Shaun; Hill, Sophie

2017-01-01

My Health Record is Australia's electronic personal health record system, which was introduced in July 2012. As of August 2017, approximately 21 percent of Australia's total population was registered to use My Health Record. Internationally, usability issues have been shown to negatively influence the uptake and use of electronic health record systems, and this scenario may particularly affect people who have low e-health literacy. It is likely that usability issues are negatively affecting the uptake and use of My Health Record in Australia. To identify potential e-health literacy-related usability issues within My Health Record through a heuristic evaluation method. Between September 14 and October 12, 2016, three of the authors conducted a heuristic evaluation of the two consumer-facing components of My Health Record-the information website and the electronic health record itself. These two components were evaluated against two sets of heuristics-the Health Literacy Online checklist and the Monkman Heuristics. The Health Literacy Online checklist and Monkman Heuristics are evidence-based checklists of web design elements with a focus on design for audiences with low health literacy. During this heuristic evaluation, the investigators individually navigated through the consumer-facing components of My Health Record, recording instances where the My Health Record did not conform to the checklist criteria. After the individual evaluations were completed, the investigators conferred and aggregated their results. From this process, a list of usability violations was constructed. When evaluated against the Health Literacy Online Checklist, the information website demonstrated violations in 12 of 35 criteria, and the electronic health record demonstrated violations in 16 of 35 criteria. When evaluated against the Monkman Heuristics, the information website demonstrated violations in 7 of 11 criteria, and the electronic health record demonstrated violations in 9 of 11

Evaluating the Usability of Pinchigator, a system for Navigating Virtual Worlds using Pinch Gloves

Science.gov (United States)

Hamilton, George S.; Brookman, Stephen; Dumas, Joseph D. II; Tilghman, Neal

2003-01-01

Appropriate design of two dimensional user interfaces (2D U/I) utilizing the well known WIMP (Window, Icon, Menu, Pointing device) environment for computer software is well studied and guidance can be found in several standards. Three-dimensional U/I design is not nearly so mature as 2D U/I, and standards bodies have not reached consensus on what makes a usable interface. This is especially true when the tools for interacting with the virtual environment may include stereo viewing, real time trackers and pinch gloves instead of just a mouse & keyboard. Over the last several years the authors have created a 3D U/I system dubbed Pinchigator for navigating virtual worlds based on the dVise dV/Mockup visualization software, Fakespace Pinch Gloves and Pohlemus trackers. The current work is to test the usability of the system on several virtual worlds, suggest improvements to increase Pinchigator s usability, and then to generalize about what was learned and how those lessons might be applied to improve other 3D U/I systems.

Usability Study of a Wireless Monitoring System among Alzheimer’s Disease Elderly Population

Directory of Open Access Journals (Sweden)

Stefano Abbate

2014-01-01

Full Text Available Healthcare technologies are slowly entering into our daily lives, replacing old devices and techniques with newer intelligent ones. Although they are meant to help people, the reaction and willingness to use such new devices by the people can be unexpected, especially among the elderly. We conducted a usability study of a fall monitoring system in a long-term nursing home. The subjects were the elderly with advanced Alzheimer’s disease. The study presented here highlights some of the challenges faced in the use of wearable devices and the lessons learned. The results gave us useful insights, leading to ergonomics and aesthetics modifications to our wearable systems that significantly improved their usability and acceptance. New evaluating metrics were designed for the performance evaluation of usability and acceptability.

Usability Evaluation of a Research Repository and Collaboration Web Site

Science.gov (United States)

Zhang, Tao; Maron, Deborah J.; Charles, Christopher C.

2013-01-01

This article reports results from an empirical usability evaluation of Human-Animal Bond Research Initiative Central as part of the effort to develop an open access research repository and collaboration platform for human-animal bond researchers. By repurposing and altering key features of the original HUBzero system, Human-Animal Bond Research…

A Comparison of Quantitative and Qualitative Data from a Formative Usability Evaluation of an Augmented Reality Learning Scenario

Directory of Open Access Journals (Sweden)

Dragos Daniel IORDACHE

2009-01-01

Full Text Available The proliferation of augmented reality (AR technologies creates opportunities for the devel-opment of new learning scenarios. More recently, the advances in the design and implementation of desktop AR systems make it possible the deployment of such scenarios in primary and secondary schools. Usability evaluation is a precondition for the pedagogical effectiveness of these new technologies and requires a systematic approach for finding and fixing usability problems. In this paper we present an approach to a formative usability evaluation based on heuristic evaluation and user testing. The basic idea is to compare and integrate quantitative and qualitative measures in order to increase confidence in results and enhance the descriptive power of the usability evaluation report.

Optimizing Usability Studies by Complementary Evaluation Methods

NARCIS (Netherlands)

Schmettow, Martin; Bach, Cedric; Scapin, Dominique

2014-01-01

This paper examines combinations of complementary evaluation methods as a strategy for efficient usability problem discovery. A data set from an earlier study is re-analyzed, involving three evaluation methods applied to two virtual environment applications. Results of a mixed-effects logistic

Designing Telemedicine Systems for Geriatric Patients: A Review of the Usability Studies.

Science.gov (United States)

Narasimha, Shraddhaa; Madathil, Kapil Chalil; Agnisarman, Sruthy; Rogers, Hunter; Welch, Brandon; Ashok, Aparna; Nair, Aswathi; McElligott, James

2017-06-01

One area where telemedicine may prove to be highly effective is in providing medical care to the geriatric population, an age group predicted to account for 20% of the population in the near future. However, even though telemedicine has certain advantages, the usability of these systems with this population merits investigation. This article reviews the literature published from 2000 to 2016 with the goal of analyzing the characteristics of usability-related studies conducted using geriatric participants and the subsequent usability challenges identified. Articles were found using Web of Knowledge and PubMed citation indexing portals using the keywords (1) Telemedicine* AND Geriatrics* (2) Telemedicine* AND Usability* (3) Telemedicine* AND Usability* AND Older Adults*. A total of 297 articles were obtained from the initial search. After further detailed screening, 16 articles were selected for review based on the inclusion criteria. Of these, 60% of the studies focused on the overall usability of telemedicine systems; 6.25% focused on the usability of a telepresence robot; 12.5% compared a face-to-face medical consultation with the use of telemedicine systems, and 25% focused on the study of other aspects of telemedicine in addition to its usability. Findings reported in the studies included high patient satisfaction with telemedicine in 31.25%, whereas another 31.25% indicated a high acceptance of this method of medical consultation. Care coordination in 6.25% of the studies; confidence in telemedicine in 6.25%; trust, privacy, and reliability in 6.25%; and increased convenience when compared to personal visits in 18.75% were also reported. This review suggests limited research providing scientifically valid and reproducible usability evaluation at various stages of telemedicine system development. Telemedicine system designers need to consider the age-related issues in cognition, perception, and behavior of geriatric patients while designing telemedicine

Wireless application guidelines for usability design and evaluation for developing countries

International Nuclear Information System (INIS)

Khoja, S.A.; Ursani, A.A.; Ali, A.

2009-01-01

The aim of this paper is to provide guidelines for designing wireless applications. These guidelines are developed with the help of Human Computer Interaction based Usability Evaluation surveys and tests results. These guidelines will help in making wireless applications more usable and attractive for users and will help designers to plan effective interactive applications. The guidelines presented in the paper have been derived from the results of surveys, usability tests and SET (Summative Evaluation Tests) carried out over 200 people of Karachi, Pakistan, from different walks of life with different academic and professional backgrounds, making imperative suggestions not only to improve the usability of exciting features given in wireless and mobile products, but also provide suggestions to improve usability of basic components such as, text, navigation, interactivity, multimedia, control and behavior. (author)

Business-Intelligence-Systeme im Spannungsfeld zwischen Usability und Sicherheit

OpenAIRE

Hinck, Thorsten

2010-01-01

Business-Intelligence-Systeme stehen im Spannungsfeld zwischen Usability und Sicherheit. Die Gefahren und Sicherheitsmaßnahmen für Informationssysteme werden ermittelt und ökonomisch in Bezug zur Gebrauchstauglichkeit gesetzt. Es entsteht ein Sicherheitsprofil für Business-Intelligence-Systeme. Business-Intelligence-Systems are in tension between Usability and IT-Security. Risks and safety measures of informationsystems are defined and set in (economic) regard to Usability. A security prof...

PERBANDINGAN METODE EVALUASI USABILITY (STUDI KASUS : PENGGUNAAN PERANGKAT SMARTPHONE)

OpenAIRE

Hendradewa, Andrie Pasca

2017-01-01

Usability is a key factor that determine success of a product or interactive system, like smartphone. The increasing use of smartphone and its rapid technology development, requires usability evaluation method that is more accurate and effective to found usability problem, so it can be used for product improvement in development process. This study suggests a usability evaluation method that is most effective to identify problem in smartphone usage. Three usability evaluation methods that com...

Identifying usability issues for personalization during formative evaluations: a comparisons of three methods

NARCIS (Netherlands)

van Velsen, Lex Stefan; van der Geest, Thea; Klaassen, R.F.

2011-01-01

A personalized system is one that generates unique output for each individual. As a result, personalization has transformed the interaction between the user and the system, and specific new usability issues have arisen. Methods used for evaluating personalized systems should be able to reveal the

Reducing and Sustaining Duplicate Medical Record Creation by Usability Testing and System Redesign.

Science.gov (United States)

Khunlertkit, Adjhaporn; Dorissaint, Leonard; Chen, Allen; Paine, Lori; Pronovost, Peter J

2017-10-25

Duplicate medical record creation is a common and consequential health care systems error often caused by poor search system usability and inappropriate user training. We conducted two phases of scenario-based usability testing with patient registrars working in areas at risk of generating duplicate medical records. Phase 1 evaluated the existing search system, which led to system redesigns. Phase 2 tested the redesigned system to mitigate potential errors before health system-wide implementation. To evaluate system effectiveness, we compared the monthly potential duplicate medical record rates for preimplementation and postimplementation months. The existing system could not effectively handle a misspelling, which led to failed search and duplicate medical record creation. Using the existing system, 96% of registrars found commonly spelled patient names whereas only 69% successfully found complicated names. Registrars lacked knowledge and usage of a phonetic matching function to assist in misspelling. The new system consistently captured the correct patient regardless of misspelling, but search returned more potential matches, resulting in, on average, 4 seconds longer to select common names. Potential monthly duplicate medical record rate reduced by 38%, from 4% to 2.3% after implementation of the new system, and has sustained at an average of 2.5% for 2 years. Usability testing was an effective method to reveal problems and aid system redesign to deliver a more user friendly system, hence reducing the potential for medical record duplication. Greater standards for usability would ensure that these improvements can be realized before rather than after exposing patients to risks.

Testing the usability of a personalized system: comparing the use of interviews, questionnaires and thinking -aloud

NARCIS (Netherlands)

van Velsen, Lex Stefan; van der Geest, Thea; Klaassen, R.F.

2007-01-01

Personalized systems present each user with tailored content or output. Testing the usability of such a system must take some specific usability issues and the suitability of the personalized output into account. In this study, we evaluated a personalized search engine to compare the use of

National questionnaire study on clinical ICT systems proofs: physicians suffer from poor usability.

Science.gov (United States)

Viitanen, Johanna; Hyppönen, Hannele; Lääveri, Tinja; Vänskä, Jukka; Reponen, Jarmo; Winblad, Ilkka

2011-10-01

In the health informatics field, usability studies typically focus on evaluating a single information system and involve a rather small group of end-users. However, little is known about the usability of clinical information and communication technology (ICT) environment in which healthcare professionals work daily. This paper aims at contributing to usability research and user-oriented development of healthcare technologies with three objectives: inform researchers and practitioners about the current state of usability of clinical ICT systems, increase the understanding of usability aspects specific for clinical context, and encourage a more holistic approach on studying usability issues in health informatics field. A national web questionnaire study was conducted in Finland in spring 2010 with 3929 physicians actively working in patient care. For the purposes of the study, we described three dimensions of clinical ICT system usability that reflect the physicians' viewpoint on system usage: (1) compatibility between clinical ICT systems and physicians' tasks, (2) ICT support for information exchange, communication and collaboration in clinical work, and (3) interoperability and reliability. The dimensions derive from the definitions of usability and clinical context of use analysis, and reflect the ability of ICT systems to have a positive impact on patient care by supporting physicians in achieving their goals with a pleasant user experience. The research data incorporated 32 statements with a five-point Likert-scale on physicians' experiences on usability of their currently used ICT systems and a summative question about school grade given to electronic health record (EHR) systems. Physicians' estimates of their EHR systems were very critical. With the rating scale from 4 or fail to 10 or excellent, the average of the grades varied from 6.1 to 8.4 dependent on the kind of facility the physician is working. Questionnaire results indicated several usability

Human Machine Interface and Usability Issues: Exploring a Preliminary Mission Management System Evaluation Methodology

National Research Council Canada - National Science Library

Kardos, Monique

2001-01-01

...) concept demonstrator test during June 2000. The questionnaire was designed to examine usability and interface issues relating to the design of the systems or tools, and to provide suggestions for future iterations of the system...

Expected usability is not a valid indicator of experienced usability

Directory of Open Access Journals (Sweden)

Meinald T. Thielsch

2015-09-01

Full Text Available Usability is a core construct of website evaluation and inherently defined as interactive. Yet, when analysing first impressions of websites, expected usability, i.e., before use, is of interest. Here we investigate to what extend ratings of expected usability are related to (a experienced usability, i.e., ratings after use, and (b objective usability measures, i.e., task performance. Furthermore, we try to elucidate how ratings of expected usability are correlated to aesthetic judgments. In an experiment, 57 participants submitted expected usability ratings after the presentation of website screenshots in three viewing-time conditions (50, 500, and 10,000 ms and after an interactive task (experienced usability. Additionally, objective usability measures (task completion and duration and subjective aesthetics evaluations were recorded for each website. The results at both the group and individual level show that expected usability ratings are not significantly related either to experienced usability or objective usability measures. Instead, they are highly correlated with aesthetics ratings. Taken together, our results highlight the need for interaction in empirical website usability testing, even when exploring very early usability impressions. In our study, user ratings of expected usability were no valid proxy neither for objective usability nor for experienced website usability.

Usability evaluation of a web-based support system for people with a schizophrenia diagnosis.

Science.gov (United States)

van der Krieke, Lian; Emerencia, Ando C; Aiello, Marco; Sytema, Sjoerd

2012-02-06

Routine Outcome Monitoring (ROM) is a systematic way of assessing service users' health conditions for the purpose of better aiding their care. ROM consists of various measures used to assess a service user's physical, psychological, and social condition. While ROM is becoming increasingly important in the mental health care sector, one of its weaknesses is that ROM is not always sufficiently service user-oriented. First, clinicians tend to concentrate on those ROM results that provide information about clinical symptoms and functioning, whereas it has been suggested that a service user-oriented approach needs to focus on personal recovery. Second, service users have limited access to ROM results and they are often not equipped to interpret them. These problems need to be addressed, as access to resources and the opportunity to share decision making has been indicated as a prerequisite for service users to become a more equal partner in communication with their clinicians. Furthermore, shared decision making has been shown to improve the therapeutic alliance and to lead to better care. Our aim is to build a web-based support system which makes ROM results more accessible to service users and to provide them with more concrete and personalized information about their functioning (ie, symptoms, housing, social contacts) that they can use to discuss treatment options with their clinician. In this study, we will report on the usability of the web-based support system for service users with schizophrenia. First, we developed a prototype of a web-based support system in a multidisciplinary project team, including end-users. We then conducted a usability study of the support system consisting of (1) a heuristic evaluation, (2) a qualitative evaluation and (3) a quantitative evaluation. Fifteen service users with a schizophrenia diagnosis and four information and communication technology (ICT) experts participated in the study. The results show that people with a

Using a fuzzy comprehensive evaluation method to determine product usability: A proposed theoretical framework.

Science.gov (United States)

Zhou, Ronggang; Chan, Alan H S

2017-01-01

In order to compare existing usability data to ideal goals or to that for other products, usability practitioners have tried to develop a framework for deriving an integrated metric. However, most current usability methods with this aim rely heavily on human judgment about the various attributes of a product, but often fail to take into account of the inherent uncertainties in these judgments in the evaluation process. This paper presents a universal method of usability evaluation by combining the analytic hierarchical process (AHP) and the fuzzy evaluation method. By integrating multiple sources of uncertain information during product usability evaluation, the method proposed here aims to derive an index that is structured hierarchically in terms of the three usability components of effectiveness, efficiency, and user satisfaction of a product. With consideration of the theoretical basis of fuzzy evaluation, a two-layer comprehensive evaluation index was first constructed. After the membership functions were determined by an expert panel, the evaluation appraisals were computed by using the fuzzy comprehensive evaluation technique model to characterize fuzzy human judgments. Then with the use of AHP, the weights of usability components were elicited from these experts. Compared to traditional usability evaluation methods, the major strength of the fuzzy method is that it captures the fuzziness and uncertainties in human judgments and provides an integrated framework that combines the vague judgments from multiple stages of a product evaluation process.

DEVELOPING AND VALIDATION A USABILITY EVALUATION TOOLS FOR DISTANCE EDUCATION WEBSITES: Persian Version

Directory of Open Access Journals (Sweden)

Soheila HAFEZI

2010-07-01

Full Text Available The web is playing a central role in distance education. The word “usability” is usually synonymous with functionality of the system for the user. Also, usability of a website is defined as something that can be used by a specific group of people to carry out specific objectives in an effective way, with efficiency and satisfaction. There are some useful usability evaluation instruments of websites in other languages. The aim of this study was to develop a Persian instrument to evaluate usability in distance education websites. This study was a methodology research design. According to review literature, the main structure of evaluation instrument for usability designed. It was consisted of forty-eight items divided in 8 domains. Each domain was composed of closed multiple choice questions in which an ordinal scale with three values is used (always, sometimes and never that measures the opinion of the evaluator about the frequency of visiting each item selected in the website. After that, we used face validity method. It refers to what instrument appears superficially to measure. At the final step, we omputed a formal content validity index (CVI across the experts’ ratings of each item’s relevance. The content validity index was measured by set of ten experts, who evaluated each item individually. According to CVI, the final version of instrument was composed of 40 questions divided into 8 domains: Navigation, Functionality, Feedback, Control, Language, Consistency, Error prevention and correction, and Visual clarity.CVI score for each phrase was more than 0.75. According to our findings, this instrument has enough validity to apply in evaluation usability of educational websites of Persian distance education websites. However, instrument reliability can be measured in further study.

Human Factors in the Large: Experiences from Denmark, Finland and Canada in Moving Towards Regional and National Evaluations of Health Information System Usability

Science.gov (United States)

Kaipio, J.; Nieminen, M.; Hyppönen, H.; Lääveri, T.; Nohr, C.; Kanstrup, A. M.; Berg Christiansen, M.; Kuo, M.-H.; Borycki, E.

2014-01-01

Summary Objectives The objective of this paper is to explore approaches to understanding the usability of health information systems at regional and national levels. Methods Several different methods are discussed in case studies from Denmark, Finland and Canada. They range from small scale qualitative studies involving usability testing of systems to larger scale national level questionnaire studies aimed at assessing the use and usability of health information systems by entire groups of health professionals. Results It was found that regional and national usability studies can complement smaller scale usability studies, and that they are needed in order to understand larger trends regarding system usability. Despite adoption of EHRs, many health professionals rate the usability of the systems as low. A range of usability issues have been noted when data is collected on a large scale through use of widely distributed questionnaires and websites designed to monitor user perceptions of usability. Conclusion As health information systems are deployed on a widespread basis, studies that examine systems used regionally or nationally are required. In addition, collection of large scale data on the usability of specific IT products is needed in order to complement smaller scale studies of specific systems. PMID:25123725

An Online Social Constructivist Course: Toward a Framework for Usability Evaluations

Science.gov (United States)

Phillips, Alana S.; Sheffield, Anneliese; Moore, Michelle; Robinson, Heather A.

2016-01-01

There is a need for a holistic usability evaluation framework that accommodates social constructivist online courses. Social knowledge construction may not be adequately evaluated using current frameworks. This qualitative research study examined the usability needs of a social constructivist online course. Data from an online course were analyzed…

An Iterative, Mixed Usability Approach Applied to the Telekit System from the Danish TeleCare North Trial

Science.gov (United States)

Schaarup, Clara; Hejlesen, Ole Kristian

2016-01-01

Objective. The aim of the present study is to evaluate the usability of the telehealth system, coined Telekit, by using an iterative, mixed usability approach. Materials and Methods. Ten double experts participated in two heuristic evaluations (HE1, HE2), and 11 COPD patients attended two think-aloud tests. The double experts identified usability violations and classified them into Jakob Nielsen's heuristics. These violations were then translated into measurable values on a scale of 0 to 4 indicating degree of severity. In the think-aloud tests, COPD participants were invited to verbalise their thoughts. Results. The double experts identified 86 usability violations in HE1 and 101 usability violations in HE2. The majority of the violations were rated in the 0–2 range. The findings from the think-aloud tests resulted in 12 themes and associated examples regarding the usability of the Telekit system. The use of the iterative, mixed usability approach produced both quantitative and qualitative results. Conclusion. The iterative, mixed usability approach yields a strong result owing to the high number of problems identified in the tests because the double experts and the COPD participants focus on different aspects of Telekit's usability. This trial is registered with Clinicaltrials.gov, NCT01984840, November 14, 2013. PMID:27974888

Pilot implementation Driven by Effects Specifications and Formative Usability Evaluation

DEFF Research Database (Denmark)

Barlach, Anders; Hertzum, Morten; Simonsen, Jesper

2013-01-01

This chapter reports on the usability-engineering work performed throughout the pilot implementation of an Electronic Healthcare Record (EHR). The case describes and analyzes the use of pilot implementations to formatively evaluate whether the usability of the EHR meets the effects specified for ...

A Longitudinal Study of Usability in Health Care - Does Time Heal?

DEFF Research Database (Denmark)

Kjeldskov, Jesper; Skov, Mikael B.; Stage, Jan

2007-01-01

for 15 months, we repeated the evaluation. Our aim was to inquire into the nature of usability problems experienced by novice and expert users, and to see to what extend usability problems of a health care information system may or may not disappear over time, as the nurses get more familiar......We report from a longitudinal laboratory-based usability evaluation of a health care information system. A usability evaluation was conducted with novice users when an electronic patient record system was being deployed in a large hospital. After the nurses had used the system in their daily work...... with it – if time heals poor design. On the basis of our study, we present findings on the usability of the electronic patient system as experienced by the nurses at these two different points in time and discuss implications for evaluating usability in health care....

Integrated System Validation Usability Questionnaire: Computerized Procedures

International Nuclear Information System (INIS)

Garcés, Ma. I.; Torralba, B.

2015-01-01

The Research and Development (R&D) project on “Theoretical and Methodological Approaches to Integrated System Validation of Control Rooms, 2014-2015”, in which the research activities described in this report are framed, has two main objectives: to develop the items for an usability methodology conceived as a part of the measurement framework for performance-based control room evaluation that the OECD Halden Reactor Project will test in the experiments planned for 2015; and the statistical analysis of the data generated in the experimental activities of the Halden Man-Machine Laboratory (HAMMLAB) facility, with previous usability questionnaires, in 2010 and 2011. In this report, the procedure designed to meet the first goal of the project is described, in particular, the process followed to identify the items related to operating procedures, both computer and paper-based, one of the elements to be included in the usability questionnaire. Three phases are performed, in the first one, the approaches developed by the United States Nuclear Regulatory Commission, NRC, are reviewed, the models used by the nuclear industry and their technical support organizations, mainly, the Electric Power Research Institute, EPRI, are analyzed, and scientist advances are also explored. In the remaining stages, general and specific guidelines for computerized and paper-based procedures are compared and criteria for the preliminary selection of the items that should be incorporated into the usability questionnaire are defined. This proposal will be reviewed and adapted by the Halden Reactor Project to the design of the specific experiments performed in HAMLAB.

Next Gen One Portal Usability Evaluation

Science.gov (United States)

Cross, E. V., III; Perera, J. S.; Hanson, A. M.; English, K.; Vu, L.; Amonette, W.

2018-01-01

Each exercise device on the International Space Station (ISS) has a unique, customized software system interface with unique layouts / hierarchy, and operational principles that require significant crew training. Furthermore, the software programs are not adaptable and provide no real-time feedback or motivation to enhance the exercise experience and/or prevent injuries. Additionally, the graphical user interfaces (GUI) of these systems present information through multiple layers resulting in difficulty navigating to the desired screens and functions. These limitations of current exercise device GUI's lead to increased crew time spent on initiating, loading, performing exercises, logging data and exiting the system. To address these limitations a Next Generation One Portal (NextGen One Portal) Crew Countermeasure System (CMS) was developed, which utilizes the latest industry guidelines in GUI designs to provide an intuitive ease of use approach (i.e., 80% of the functionality gained within 5-10 minutes of initial use without/limited formal training required). This is accomplished by providing a consistent interface using common software to reduce crew training, increase efficiency & user satisfaction while also reducing development & maintenance costs. Results from the usability evaluations showed the NextGen One Portal UI having greater efficiency, learnability, memorability, usability and overall user experience than the current Advanced Resistive Exercise Device (ARED) UI used by astronauts on ISS. Specifically, the design of the One-Portal UI as an app interface similar to those found on the Apple and Google's App Store, assisted many of the participants in grasping the concepts of the interface with minimum training. Although the NextGen One-Portal UI was shown to be an overall better interface, observations by the test facilitators noted specific exercise tasks appeared to have a significant impact on the NextGen One-Portal UI efficiency. Future updates to

A usability evaluation of Lazada mobile application

Science.gov (United States)

Hussain, Azham; Mkpojiogu, Emmanuel O. C.; Jamaludin, Nur Hafiza; Moh, Somia T. L.

2017-10-01

This paper reports on a usability evaluation of Lazada mobile application, an online shopping app for mobile devices. The evaluation was conducted using 12 users of ages 18 to 24. Seven (7) were expert users and the other 5 were novice users. The study objectives were to evaluate the perceived effectiveness, efficiency and satisfaction of the mobile application. The result provides a positive feedback and shows that the mobile shopping app is effective, efficient, and satisfying as perceived by the study participants. However, there are some observed usability issues with the main menu and the payment method that necessitates improvements to increase the application's effectiveness, efficiency and satisfaction. The suggested improvements include: 1) the main menu should be capitalized and place on the left side of mobile app and 2) payment method tutorial should be included as a hyperlink in the payment method page. This observation will be helpful to the owners of the application in future version development of the app.

The Focus of Current HCI Research in Usability Evaluation and Feedback

DEFF Research Database (Denmark)

Høegh, Rune Thaarup

2007-01-01

This paper presents the findings from a systematic literature study of the research on usability evaluation and feedback. The literature study covers the papers published from five key human-computer interaction journals for a period of four years, and depicts the most current research within...... the field with regard to usability evaluation and feedback....

Usability Evaluation of the Spatial OLAP Visualization and Analysis Tool (SOVAT).

Science.gov (United States)

Scotch, Matthew; Parmanto, Bambang; Monaco, Valerie

2007-02-01

Increasingly sophisticated technologies, such as On-Line Analytical Processing (OLAP) and Geospatial Information Systems (GIS), are being leveraged for conducting community health assessments (CHA). Little is known about the usability of OLAP and GIS interfaces with respect to CHA. We conducted an iterative usability evaluation of the Spatial OLAP Visualization and Analysis Tool (SOVAT), a software application that combines OLAP and GIS. A total of nine graduate students and six community health researchers were asked to think-aloud while completing five CHA questions using SOVAT. The sessions were analyzed after every three participants and changes to the interface were made based on the findings. Measures included elapsed time, answers provided, erroneous actions, and satisfaction. Traditional OLAP interface features were poorly understood by participants and combined OLAP-GIS features needed to be better emphasized. The results suggest that the changes made to the SOVAT interface resulted in increases in both usability and user satisfaction.

Can eye tracking boost usability evaluation of computer games?

DEFF Research Database (Denmark)

Johansen, Sune Alstrup; Noergaard, Mie; Soerensen, Janus Rau

2008-01-01

Good computer games need to be challenging while at the same time being easy to use. Accordingly, besides struggling with well known challenges for usability work, such as persuasiveness, the computer game industry also faces system-specific challenges, such as identifying methods that can provide...... data on players' attention during a game. This position paper discusses how eye tracking may address three core challenges faced by computer game producer IO Interactive in their on-going work to ensure games that are fun, usable, and challenging. These challenges are: (1) Persuading game designers...... about the relevance of usability results, (2) involving game designers in usability work, and (3) identifying methods that provide new data about user behaviour and experience....

Usability of clinical decision support system as a facilitator for learning the assistive technology adaptation process.

Science.gov (United States)

Danial-Saad, Alexandra; Kuflik, Tsvi; Weiss, Patrice L Tamar; Schreuer, Naomi

2016-01-01

The aim of this study was to evaluate the usability of Ontology Supported Computerized Assistive Technology Recommender (OSCAR), a Clinical Decision Support System (CDSS) for the assistive technology adaptation process, its impact on learning the matching process, and to determine the relationship between its usability and learnability. Two groups of expert and novice clinicians (total, n = 26) took part in this study. Each group filled out system usability scale (SUS) to evaluate OSCAR's usability. The novice group completed a learning questionnaire to assess OSCAR's effect on their ability to learn the matching process. Both groups rated OSCAR's usability as "very good", (M [SUS] = 80.7, SD = 11.6, median = 83.7) by the novices, and (M [SUS] = 81.2, SD = 6.8, median = 81.2) by the experts. The Mann-Whitney results indicated that no significant differences were found between the expert and novice groups in terms of OSCAR's usability. A significant positive correlation existed between the usability of OSCAR and the ability to learn the adaptation process (rs = 0.46, p = 0.04). Usability is an important factor in the acceptance of a system. The successful application of user-centered design principles during the development of OSCAR may serve as a case study that models the significant elements to be considered, theoretically and practically in developing other systems. Implications for Rehabilitation Creating a CDSS with a focus on its usability is an important factor for its acceptance by its users. Successful usability outcomes can impact the learning process of the subject matter in general, and the AT prescription process in particular. The successful application of User-Centered Design principles during the development of OSCAR may serve as a case study that models the significant elements to be considered, theoretically and practically. The study emphasizes the importance of close collaboration between the developers and

Cultural Usability

DEFF Research Database (Denmark)

Shi, Qingxin

2007-01-01

Culture has already played an important role in the global market. It not only affects products, but also impacts on usability evaluation methods. This project aims to examine in the established thinking aloud usability evaluation method (TA UEM), how does the evaluator build a supportive...

Usability styles of the plant safety monitoring and assesment system, PLASMA

International Nuclear Information System (INIS)

Green, M.; Hornaes, A.; Hulsund, J.E.; Vegh, J.; Major, Cs.; Lipcsei, S.; Borbely, S.

2001-01-01

Development of the PLASMA (Plant Safety Monitoring and Assessment) system was started in 1998 in the framework of an international Research and Development project supported by OECD/NEA. The objective of this project was to develop an on-line information system to support VVER reactor operators during the execution of symptom-based Emergency Operating Procedures (EOPs), with the Paks NPP in Hungary as the target plant. In connection with the installation of the PLASMA system at Paks NPP it was decided to perform a usability study of the system through interviewing NPP operators after they had completed a short training course on the PLASMA system and used it during a validation and training session in the training simulator at Paks. This report describes the basic process used in preparations and execution of the usability studies, including methods for gathering information and analysis of the findings from the validation. As a general conclusion from the usability studies it can be stated that the PLASMA system received very favourable ratings from the operators. User satisfaction was generally rated highly and comments from operators were positive. This is particularly encouraging considering the relatively short introduction and experience that the operators had with the system at the time of the-evaluation. (Author)

Evaluating the usability of an interactive, bi-lingual, touchscreen-enabled breastfeeding educational programme: application of Nielson’s heuristics

Directory of Open Access Journals (Sweden)

Ashish Joshi

2015-03-01

Full Text Available Background: Usability challenges have to be met in an interactive computer program development and should meet all users’ needs. Objective: The study purpose was to conduct heuristic evaluation of an interactive, bilingual touch screen enabled breastfeeding educational program for Hispanic women living in rural settings.Methods: Two usability experts used Nielsen’s heuristics while reviewing the user interface in May 2013 using principles of Nielson’s Heuristics. Nielson’s heuristics are a set of usability engineering principles developed to identify issues in user interface design and involves analysis of the interface. The heuristic evaluations were carried out in the interface, program sections, and interactive educational modules. A total of 271 screens were evaluated and included: interface (n=5, program sections (n=223 and educational content (n=43.Results: A total of 97 violations were identified and were mostly related to interface (8violations/5screens and program components (89violations/266screens. The most common violations reported were recognition rather than recall (62%, n=60, consistency and standards (14%, n=14, and match between the system and real world (9%, n= 9. Majority of the violations had minor usability issues (73%, n=71. The only catastrophic violation reported was due to the visibility of system status in the assessment modules.Conclusion: The results demonstrated that the system was more consistent with Nielsen’s usability heuristics. 

Interface design recommendations for computerised clinical audit and feedback: Hybrid usability evidence from a research-led system.

Science.gov (United States)

Brown, Benjamin; Balatsoukas, Panos; Williams, Richard; Sperrin, Matthew; Buchan, Iain

2016-10-01

Audit and Feedback (A&F) is a widely used quality improvement technique that measures clinicians' clinical performance and reports it back to them. Computerised A&F (e-A&F) system interfaces may consist of four key components: (1) Summaries of clinical performance; (2) Patient lists; (3) Patient-level data; (4) Recommended actions. There is a lack of evidence regarding how to best design e-A&F interfaces; establishing such evidence is key to maximising usability, and in turn improving patient safety. To evaluate the usability of a novel theoretically-informed and research-led e-A&F system for primary care (the Performance Improvement plaN GeneratoR: PINGR). (1) Describe PINGR's design, rationale and theoretical basis; (2) Identify usability issues with PINGR; (3) Understand how these issues may interfere with the cognitive goals of end-users; (4) Translate the issues into recommendations for the user-centred design of e-A&F systems. Eight experienced health system evaluators performed a usability inspection using an innovative hybrid approach consisting of five stages: (1) Development of representative user tasks, Goals, and Actions; (2) Combining Heuristic Evaluation and Cognitive Walkthrough methods into a single protocol to identify usability issues; (3) Consolidation of issues; (4) Severity rating of consolidated issues; (5) Analysis of issues according to usability heuristics, interface components, and Goal-Action structure. A final list of 47 issues were categorised into 8 heuristic themes. The most error-prone heuristics were 'Consistency and standards' (13 usability issues; 28% of the total) and 'Match between system and real world' (n=10, 21%). The recommended actions component of the PINGR interface had the most usability issues (n=21, 45%), followed by patient-level data (n=5, 11%), patient lists (n=4, 9%), and summaries of clinical performance (n=4, 9%). The most error-prone Actions across all user Goals were: (1) Patient selection from a list; (2) Data

Usability of Public Administration Electronic Forms

Directory of Open Access Journals (Sweden)

Miloslav Hub

2015-06-01

Full Text Available This paper is focused on the testing and evaluating of public administration electronic forms from the usability point of view. Its objective is to design a suitable methodology for usability testing of electronic forms and their description and distribution to public administration information systems professionals. Firstly, methods of usability engineering are summarized and a suitable method for usability testing and evaluation of electronic forms is selected. Farther, the methodology of electronic forms usability testing that uses the selected method is suggested. In the last part of the paper the case study that uses the proposed methodology is suggested and performed. The main benefit of the work is the design of testing methodology and proposition of the set of recommendations for new public administration electronic forms design.

Complementary role of two evaluation methods in the usability and accessibility evaluation of a non-standard system

CSIR Research Space (South Africa)

Adebesin, F

2010-10-01

Full Text Available and used by people with varying abilities. Although accessibility concerns are aimed at making systems usable for people with disabilities, support for direct accessibility, the built-in redundancies in an application that enable as many people as possible...

Investigation of the Usability of Computerized Critical Care Information Systems in Germany.

Science.gov (United States)

von Dincklage, Falk; Suchodolski, Klaudiusz; Lichtner, Gregor; Friesdorf, Wolfgang; Podtschaske, Beatrice; Ragaller, Maximilian

2017-01-01

The term "usability" describes how effectively, efficiently, and with what level of user satisfaction an information system can be used to accomplish specific goals. Computerized critical care information systems (CCISs) with high usability increase quality of care and staff satisfaction, while reducing medication errors. Conversely, systems lacking usability can interrupt clinical workflow, facilitate errors, and increase charting time. The aim of this study was to investigate and compare usability across CCIS currently used in Germany. In this study, German intensive care unit (ICU) nurses and physicians completed a specialized, previously validated, web-based questionnaire. The questionnaire assessed CCIS usability based on three rating models: an overall rating of the systems, a model rating technical usability, and a model rating task-specific usability. We analyzed results from 535 survey participants and compared eight different CCIS commonly used in Germany. Our results showed that usability strongly differs across the compared systems. The system ICUData had the best overall rating and technical usability, followed by the platforms ICM and MetaVision. The same three systems performed best in the rating of task-specific usability without significant differences between each other. Across all systems, overall ratings were more dependent on ease-of-use aspects than on aspects of utility/functionality, and the general scope of the functions offered was rated better than how well the functions are realized. Our results suggest that manufacturers should shift some of their effort away from the development of new features and focus more on improving the ease-of-use and quality of existing features.

A usability test is not an interview

DEFF Research Database (Denmark)

Hertzum, Morten

2016-01-01

Usability tests are conducted to gauge users’ experience with a system, preferably before it is released for real use, and thereby find any problems that prevent users from completing their tasks, slow them down, or otherwise degrade their user experience. Such tests are important to successful...... systems development, yet test procedures vary and the quality of test results is sometimes contested. While there is no single accepted procedure for usability specialists to follow when conducting usability tests, these tests normally involve users who think out loud while using a system and an evaluator...... who observes the users’ behavior and listens in on their thoughts. This common core of usability tests is illustrated in Figure 1. The possible variations include, for example, whether the users work individually or in pairs, whether the evaluator is in the room with the user or in an adjoining room...

Evaluating the usability of an interactive, bi-lingual, touchscreen-enabled breastfeeding educational programme: application of Nielson's heuristics.

Science.gov (United States)

Joshi, Ashish; Perin, Douglas M Puricelli; Amadi, Chioma; Trout, Kate

2015-03-05

The study purpose was to conduct heuristic evaluation of an interactive, bilingual touchscreen-enabled breastfeeding educational programme for Hispanic women living in rural settings in Nebraska. Three raters conducted the evaluation during May 2013 using principles of Nielson's heuristics. A total of 271 screens were evaluated and included: interface (n = 5), programme sections (n = 223) and educational content (n = 43). A total of 97 heuristic violations were identified and were mostly related to interface (8 violations/5 screens) and programme components (89 violations/266 screens). The most common heuristic violations reported were recognition rather than recall (62%, n = 60), consistency and standards (14%, n = 14) and match between the system and real world (9%, n = 9). Majority of the heuristic violations had minor usability issues (73%, n = 71). The only grade 4 heuristic violation reported was due to the visibility of system status in the assessment modules. The results demonstrated that the system was more consistent with Nielsen's usability heuristics. With Nielsen's usability heuristics, it is possible to identify problems in a timely manner, and help facilitate the identification and prioritisation of problems needing urgent attention at an earlier stage before the final deployment of the system.

Usability Assessment of International Office Website of Diponegoro University With Scenario Based Usability Evaluation Method and Wammi Method

OpenAIRE

Ratna Purwaningsih; Ikhsan Yenifi

2015-01-01

Website development needs to pay attention to usability aspect which consist of four factors: efficiency, effectiveness, satisfactions and error rate. The website of International Office of Diponegoro University provides academic information for Indonesian students and foreign students. The website still has some problem, especially on searching certain information. This research aims to evaluate the design of the existing website and measures the usability. A well-designed website will help ...

Integrated System Validation Usability Questionnaire: Information Display Element

International Nuclear Information System (INIS)

Garcés, Ma. I.; Torralba, B.

2015-01-01

The Research and Development (R&D) project on “Theoretical and Methodological Approaches to Integrated System Validation of Control Rooms, 2014-2015”, in which the research activities described in this report are framed, has two main objectives: to develop the items for an usability methodology conceived as a part of the measurement framework for performance-based control room evaluation that the OECD Halden Reactor Project will test in the experiments planned for 2015; and the statistical analysis of the data generated in the experimental activities of the Halden Man-Machine Laboratory (HAMMLAB) facility, with previous usability questionnaires, in 2010 and 2011. In this report, the procedure designed to meet the first goal of the project is described, in particular, the process followed to identify the items related to information displays, one of the elements to be included in the usability questionnaire. Three phases are performed, in the first one, the approaches developed by the United States Nuclear Regulatory Commission, NRC, are reviewed and the models proposed by the nuclear energy industry and their technical support organizations, mainly, the United States Electric Power Research Institute, EPRI, are analyzed. In the remaining stages, general and specific guidelines for information displays, in particular, display pages, formats, elements and data quality and update rate recommendations are compared and criteria for the preliminary selection of the items that should be incorporated into the usability questionnaire are defined. This proposal will be reviewed and adapted by the Halden Reactor Project to the design of the specific experiments performed in HAMMLAB.

EVALUATION OF THE USABILITY OF MOBILE COMPUTER SYSTEM FOR EDUCATIVE ADVISING IN THE UNIVERSIDAD AUTÓNOMA INDÍGENA DE MEXICO

Directory of Open Access Journals (Sweden)

Erasmo Miranda-Bojórquez

2011-01-01

Full Text Available Tests of evaluation and functionality of a system computer denominated FoldMobile in la Universidad Autonoma Indigena de Mexico (UAIM have taken place. This system provides with sufficient information to carry out the pursuit of the learning of student when it takes place the actual interaction of the adviser and the student in any place of the university campus and within an atmosphere totally disconnected of any network of devices used for storage. For it one has settled in a digital personal assistant (PDA. The purpose of the tests is to measure its usability and to carry out a redesign of the system being used for it the methodology based on users. Especially the propose heuristic evaluation by Nielsen through its ten rules. An experiment was carried out to evaluate the usability of the system and graphical interfaces to offer simpler solutions that they approach the mental models of the users and to ensure that the system is accepted and more used. The results gave the approval of the experts to the simplicity of the screens and the information presented/displayed through a consistent and familiar language but also suggested a redesign as far as the obtaining of aid of use of the system. In agreement with the results of the experiment the level of acceptance from the user was high, describing its experience like satisfactory, exciting and easy to use. The benefits are in cost and time of development of the system and more propitious conditions so that he is really used.

The usability of ventilators: a comparative evaluation of use safety and user experience.

Science.gov (United States)

Morita, Plinio P; Weinstein, Peter B; Flewwelling, Christopher J; Bañez, Carleene A; Chiu, Tabitha A; Iannuzzi, Mario; Patel, Aastha H; Shier, Ashleigh P; Cafazzo, Joseph A

2016-08-20

The design complexity of critical care ventilators (CCVs) can lead to use errors and patient harm. In this study, we present the results of a comparison of four CCVs from market leaders, using a rigorous methodology for the evaluation of use safety and user experience of medical devices. We carried out a comparative usability study of four CCVs: Hamilton G5, Puritan Bennett 980, Maquet SERVO-U, and Dräger Evita V500. Forty-eight critical care respiratory therapists participated in this fully counterbalanced, repeated measures study. Participants completed seven clinical scenarios composed of 16 tasks on each ventilator. Use safety was measured by percentage of tasks with use errors or close calls (UE/CCs). User experience was measured by system usability and workload metrics, using the Post-Study System Usability Questionnaire (PSSUQ) and the National Aeronautics and Space Administration Task Load Index (NASA-TLX). Nine of 18 post hoc contrasts between pairs of ventilators were significant after Bonferroni correction, with effect sizes between 0.4 and 1.09 (Cohen's d). There were significantly fewer UE/CCs with SERVO-U when compared to G5 (p = 0.044) and V500 (p = 0.020). Participants reported higher system usability for G5 when compared to PB980 (p = 0.035) and higher system usability for SERVO-U when compared to G5 (p ventilators from market leaders compare and highlights the importance of medical technology design. Within the boundaries of this study, we can infer that SERVO-U demonstrated the highest levels of use safety and user experience, followed by G5. Based on qualitative data, differences in outcomes could be explained by interaction design, quality of hardware components used in manufacturing, and influence of consumer product technology on users' expectations.

Images of Usability

DEFF Research Database (Denmark)

Hertzum, Morten

2010-01-01

unless confronted with alternative images. This study delineates six images of usability: universal usability, situational usability, perceived usability, hedonic usability, organizational usability, and cultural usability. The different foci of the images provide opportunities for becoming sensitized...... assume different images of usability and a need for supplementary methods addressing the collaborative and long-term aspects of usability. Moreover, the images call for extending the scope of practical usability work to include the effects achieved by users during their use of systems for real work....

On the Usability of Spoken Dialogue Systems

DEFF Research Database (Denmark)

Larsen, Lars Bo

 This work is centred on the methods and problems associated with defining and measuring the usability of Spoken Dialogue Systems (SDS). The starting point is the fact that speech based interfaces has several times during the last 20 years fallen short of the high expectations and predictions held...... by industry, researchers and analysts. Several studies in the literature of SDS indicate that this can be ascribed to a lack of attention from the speech technology community towards the usability of such systems. The experimental results presented in this work are based on a field trial with the OVID home...

Usability evaluation of mobile applications; where do we stand?

Science.gov (United States)

Zahra, Fatima; Hussain, Azham; Mohd, Haslina

2017-10-01

The range and availability of mobile applications is expanding rapidly. With the increased processing power available on portable devices, developers are increasing the range of services by embracing smartphones in their extensive and diverse practices. While usability testing and evaluations of mobile applications have not yet touched the accuracy level of other web based applications. The existing usability models do not adequately capture the complexities of interacting with applications on a mobile platform. Therefore, this study aims to presents review on existing usability models for mobile applications. These models are in their infancy but with time and more research they may eventually be adopted. Moreover, different categories of mobile apps (medical, entertainment, education) possess different functional and non-functional requirements thus customized models are required for diverse mobile applications.

Checking an integrated model of web accessibility and usability evaluation for disabled people.

Science.gov (United States)

Federici, Stefano; Micangeli, Andrea; Ruspantini, Irene; Borgianni, Stefano; Corradi, Fabrizio; Pasqualotto, Emanuele; Olivetti Belardinelli, Marta

2005-07-08

A combined objective-oriented and subjective-oriented method for evaluating accessibility and usability of web pages for students with disability was tested. The objective-oriented approach is devoted to verifying the conformity of interfaces to standard rules stated by national and international organizations responsible for web technology standardization, such as W3C. Conversely, the subjective-oriented approach allows assessing how the final users interact with the artificial system, accessing levels of user satisfaction based on personal factors and environmental barriers. Five kinds of measurements were applied as objective-oriented and subjective-oriented tests. Objective-oriented evaluations were performed on the Help Desk web page for students with disability, included in the website of a large Italian state university. Subjective-oriented tests were administered to 19 students labeled as disabled on the basis of their own declaration at the University enrolment: 13 students were tested by means of the SUMI test and six students by means of the 'Cooperative evaluation'. Objective-oriented and subjective-oriented methods highlighted different and sometimes conflicting results. Both methods have pointed out much more consistency regarding levels of accessibility than of usability. Since usability is largely affected by individual differences in user's own (dis)abilities, subjective-oriented measures underscored the fact that blind students encountered much more web surfing difficulties.

Cross cultural usability testing

DEFF Research Database (Denmark)

Clemmensen, Torkil; Goyal, Shivam

2005-01-01

In this paper, we present the results of a pilot study in Denmark of cross cultural effects on Think Aloud usability testing. We provide an overview of previous research on cross cultural usability evaluation with a special focus on the relationship between the evaluator and the test user....... This relation was studied in an experiment with usability testing of a localized clipart application in which eight participants from Denmark and India formed pairs of evaluator-test user. The test users were asked to think aloud and the evaluators' role were to facilitate the test users thinking aloud...... and hereby identify usability problems with the clipart application. Data on the evaluators' and test users' behaviour were recorded and analyzed by coding and summarizing statistics on these behavioural events. The results show that Think Aloud Usability Test of a localized application is most effectively...

Merits of usability testing for PACS selection

NARCIS (Netherlands)

Jorritsma, Wiard; Cnossen, Fokie; van Ooijen, Peter M. A.

Objectives: To compare the usability of different Picture Archiving and Communication System (PACS) workstations, determine whether a usability test has added value with respect to the traditional way of comparing PACSs based on functional requirements, and to evaluate the appropriateness of a

Distributed user interfaces usability and collaboration

CERN Document Server

Lozano, María D; Tesoriero, Ricardo; Penichet, Victor MR

2013-01-01

Written by international researchers in the field of Distributed User Interfaces (DUIs), this book brings together important contributions regarding collaboration and usability in Distributed User Interface settings. Throughout the thirteen chapters authors address key questions concerning how collaboration can be improved by using DUIs, including: in which situations a DUI is suitable to ease the collaboration among users; how usability standards can be used to evaluate the usability of systems based on DUIs; and accurately describe case studies and prototypes implementing these concerns

From Usability Testing to Clinical Simulations: Bringing Context into the Design and Evaluation of Usable and Safe Health Information Technologies

DEFF Research Database (Denmark)

Kushniruk, Andre; Nøhr, Christian; Jensen, Sanne

2013-01-01

of clinical context into stronger focus. This involves testing of systems with representative users doing representative tasks, in representative settings/environments. Results: Application of methods where realistic clinical scenarios are used to drive the study of users interacting with systems under...... such evaluation can be used to improve both the usability and safety of HIT. In addition, recent work has shown that clinical simulations, in particular those conducted in-situ, can lead to considerable benefits when compared to the costs of running such studies. Conclusion: In order to bring context of use...

A Benchmark Usability Study of the Tactical Decision Making Under Stress Decision Support System

National Research Council Canada - National Science Library

Schmorrow, Dylan

1998-01-01

This study evaluates the usability of a U.S. Navy Decision Support System (DSS). The DSS was developed to enhance the performance of tactical decision makers within a Navy Combat Information Center...

Defining Usability of PN Services

DEFF Research Database (Denmark)

Nicolajsen, Hanne Westh; Ahola, Titta; Fleury, Alexandre

In this deliverable usability and user experience are defined in relation to MAGNET Beyond technologies, and it is described how the main MAGNET Beyond concepts can be evaluated through the involvement of users. The concepts include the new "Activity based communication approach" for interacting...... with the MAGNET Beyond system, as well as the core concepts: Personal Network, Personal Network-Federation, Service Discovery, User Profile Management, Personal Network Management, Privacy and Security and Context Awareness. The overall plans for the final usability evaluation are documented based on the present...

Low-Cost Rapid Usability Testing: Its Application in Both Product Development and System Implementation.

Science.gov (United States)

Kushniruk, Andre; Borycki, Elizabeth

2017-01-01

In recent years there has been considerable discussion around the need for certification and regulation of healthcare information technology (IT). In particular, the usability of the products being developed needs to be evaluated. This has included the application of standards designed to ensure the process of system development is user-centered and takes usability into consideration while a product is being developed. In addition to this, in healthcare, organizations in the United States and Europe have also addressed the need and requirement for product certification. However, despite these efforts there are continued reports of unusable and unsafe implementations. In this paper we discuss the need to not only include (and require) usability testing in the one-time development process of health IT products (such as EHRs), but we also argue for the need to additionally develop specific usability standards and requirements for usability testing during the implementation of vendor products (i.e. post product development) in healthcare settings. It is further argued that health IT products that may have been certified regarding their development process will still require application of usability testing in the process of implementing them in real hospital settings in order to ensure usability and safety. This is needed in order to ensure that the final result of both product development and implementation processes take into account and apply the latest usability principles and methods.

A Novel Multifactor Authentication System Ensuring Usability and Security

OpenAIRE

Mathew, Gloriya; Thomas, Shiney

2013-01-01

User authentication is one of the most important part of information security. Computer security most commonly depends on passwords to authenticate human users. Password authentication systems will be either been usable but not secure, or secure but not usable. While there are different types of authentication systems available alphanumeric password is the most commonly used authentication mechanism. But this method has significant drawbacks. An alternative solution to the text based authenti...

Supporting cancer patients in illness management: usability evaluation of a mobile app.

Science.gov (United States)

Mirkovic, Jelena; Kaufman, David R; Ruland, Cornelia M

2014-08-13

Mobile phones and tablets currently represent a significant presence in people's everyday lives. They enable access to different information and services independent of current place and time. Such widespread connectivity offers significant potential in different app areas including health care. Our goal was to evaluate the usability of the Connect Mobile app. The mobile app enables mobile access to the Connect system, an online system that supports cancer patients in managing health-related issues. Along with symptom management, the system promotes better patient-provider communication, collaboration, and shared decision making. The Connect Mobile app enables access to the Connect system over both mobile phones and tablets. The study consisted of usability tests of a high fidelity prototype with 7 cancer patients where the objectives were to identify existing design and functionality issues and to provide patients with a real look-and-feel of the mobile system. In addition, we conducted semistructured interviews to obtain participants' feedback about app usefulness, identify the need for new system features and design requirements, and measure the acceptance of the mobile app and its features within everyday health management. The study revealed a total of 27 design issues (13 for mobile apps and 14 for tablet apps), which were mapped to source events (ie, errors, requests for help, participants' concurrent feedback, and moderator observation). We also applied usability heuristics to identify violations of usability principles. The majority of violations were related to enabling ease of input, screen readability, and glanceability (15 issues), as well as supporting an appropriate match between systems and the real world (7 issues) and consistent mapping of system functions and interactions (4 issues). Feedback from participants also showed the cancer patients' requirements for support systems and how these needs are influenced by different context-related factors

Developing an Android-Based Patient Decision Aid Based on Ottawa Standards for Patients After Kidney Transplant and Its Usability Evaluation.

Science.gov (United States)

Zare Moayedi, Mahboobeh; Aslani, Azam; Fakhrahmad, Mostafa; Ezzatzadegan J, Shahrokh

2018-01-01

This study was conducted to develop an android based patient decision aid (PDA) as a self-care instrument for patients after kidney transplant and its usability evaluation. In this study, the systematic development process of Android-based self-care application for patients after kidney transplant based on Ottawa standard was included: scoping, assemble steering group, analysis of requirements, designing, develop of a prototype and system evaluation. The PDA is a self-triage system that will help early identification of risk symptoms in patients, and help manage them. System recommendations for risk signs are: Refer to the nearest hospital or healthcare center without delay, refer to the doctor and tell your doctor in the next visit. To identify patient care needs, a semi-structured interview with members of steering group, including patients and clinical experts, was conducted by the researchers. A prototype of the decision aid was made according to identified needs in the previous step. Finally, in order to evaluate its usability rate by using the System Usability Scale (SUS) questionnaire, it was used by exerts and patients. This study identified information needs, risk signs and steps that patients need to make appropriate decisions about them. The main capabilities of the decision aid are features such as reminders for appointment/test, time of taking medication, registration of symptoms, weight, blood pressure, body temperature, advising to patient in case of signs of risk, weight, blood pressure, body temperature and test results which were reported in the diagram. The mean score of system's usability evaluated by medical informatics specialists, clinicians, and patients were 88.33, 95, and 91. PDAs was usable and desirable from the point of view of medical informatics specialists, clinicians and patients.

Integration of MSFC Usability Lab with Usability Testing

Science.gov (United States)

Cheng, Yiwei; Richardson, Sally

2010-01-01

As part of the Stage Analysis Branch, human factors engineering plays an important role in relating humans to the systems of hardware and structure designs of the new launch vehicle. While many branches are involved in the technical aspects of creating a launch vehicle, human factors connects humans to the scientific systems with the goal of improving operational performance and safety while reducing operational error and damage to the hardware. Human factors engineers use physical and computerized models to visualize possible areas for improvements to ensure human accessibility to components requiring maintenance and that the necessary maintenance activities can be accomplished with minimal risks to human and hardware. Many methods of testing are used to fulfill this goal, such as physical mockups, computerized visualization, and usability testing. In this analysis, a usability test is conducted to test how usable a website is to users who are and are not familiar with it. The testing is performed using participants and Morae software to record and analyze the results. This analysis will be a preliminary test of the usability lab in preparation for use in new spacecraft programs, NASA Enterprise, or other NASA websites. The usability lab project is divided into two parts: integration of the usability lab and a preliminary test of the usability lab.

Usability evaluation of pharmacogenomics clinical decision support aids and clinical knowledge resources in a computerized provider order entry system: a mixed methods approach.

Science.gov (United States)

Devine, Emily Beth; Lee, Chia-Ju; Overby, Casey L; Abernethy, Neil; McCune, Jeannine; Smith, Joe W; Tarczy-Hornoch, Peter

2014-07-01

Pharmacogenomics (PGx) is positioned to have a widespread impact on the practice of medicine, yet physician acceptance is low. The presentation of context-specific PGx information, in the form of clinical decision support (CDS) alerts embedded in a computerized provider order entry (CPOE) system, can aid uptake. Usability evaluations can inform optimal design, which, in turn, can spur adoption. The study objectives were to: (1) evaluate an early prototype, commercial CPOE system with PGx-CDS alerts in a simulated environment, (2) identify potential improvements to the system user interface, and (3) understand the contexts under which PGx knowledge embedded in an electronic health record is useful to prescribers. Using a mixed methods approach, we presented seven cardiologists and three oncologists with five hypothetical clinical case scenarios. Each scenario featured a drug for which a gene encoding drug metabolizing enzyme required consideration of dosage adjustment. We used Morae(®) to capture comments and on-screen movements as participants prescribed each drug. In addition to PGx-CDS alerts, 'Infobutton(®)' and 'Evidence' icons provided participants with clinical knowledge resources to aid decision-making. Nine themes emerged. Five suggested minor improvements to the CPOE user interface; two suggested presenting PGx information through PGx-CDS alerts using an 'Infobutton' or 'Evidence' icon. The remaining themes were strong recommendations to provide succinct, relevant guidelines and dosing recommendations of phenotypic information from credible and trustworthy sources; any more information was overwhelming. Participants' median rating of PGx-CDS system usability was 2 on a Likert scale ranging from 1 (strongly agree) to 7 (strongly disagree). Usability evaluation results suggest that participants considered PGx information important for improving prescribing decisions; and that they would incorporate PGx-CDS when information is presented in relevant and

Design research as a framework to evaluate the usability and accessibility of the digital doorway

CSIR Research Space (South Africa)

Adebesin, F

2011-09-01

Full Text Available , and then also evaluated the usability and direct accessibility support provided by the Digital Doorway, a non-standard computer system deployed amongst underprivileged communities in South Africa with the aim of promoting computer literacy. This paper discusses...

Hello World! - Experiencing Usability Methods without Usability Expertise

Science.gov (United States)

Eriksson, Elina; Cajander, Åsa; Gulliksen, Jan

How do you do usability work when no usability expertise is available? What happens in an organization when system developers, with no previous HCI knowledge, after a 3-day course, start applying usability methods, and particularly field studies? In order to answer these questions qualitative data were gathered through participatory observations, a feed back survey, field study documentation and interviews from 47 system developers from a public authority. Our results suggest that field studies enhance the developer’s understanding of the user perspective, and provide a more holistic overview of the use situation, but that some developers were unable to interpret their observations and see solutions to the users’ problems. The field study method was very much appreciated and has now become standard operating procedure within the organization. However, although field studies may be useful, it does not replace the need for usability pro fes sion als, as their knowledge is essential for more complex observations, analysis and for keeping the focus on usability.

Lessons from an Educational Game Usability Evaluation

Directory of Open Access Journals (Sweden)

Gabriela Trindade Perry

2012-04-01

Full Text Available In this paper, we present the results of a usability evaluation for Xenubi, a cell phone game about the periodic table. We describe and discuss the issues that we faced during the evaluation, which took place in a state high school. These issues are related to conflicts between the data that were gathered through video recordings and through a questionnaire. We also point to the poor performance of the game mechanics during the evaluation. This result was unexpected because the game had an excellent performance in the pilot test. Possible causes for these differences are discussed.

Usability Briefing for hospital design

DEFF Research Database (Denmark)

Fronczek-Munter, Aneta

This PhD thesis is a contribution to an ongoing debate in Denmark about improving the building design processes of complex buildings, especially in relation to the current hospital developments. It provides knowledge about capturing user needs and defines the process model for usability briefing...... and evaluations, can be fed into briefing and design processes. This PhD thesis proposes methods for usability briefing.Usability is a concept similar to functionality, but usability depends on: subjective view of users, context, culture, situation and experience. Understanding usability is achieved by involving...... users. This PhD thesis extends the research in usability of buildings to include all building design phases, therefore not only proposes usability evaluations, but also defines usability briefing. Briefing, also called architectural programming, is usually understood as one of the first phases...

Evaluation of the acceptability and usability of a decision support system to encourage safe and effective use of opioid therapy for chronic, noncancer pain by primary care providers.

Science.gov (United States)

Trafton, Jodie; Martins, Susana; Michel, Martha; Lewis, Eleanor; Wang, Dan; Combs, Ann; Scates, Naquell; Tu, Samson; Goldstein, Mary K

2010-04-01

To develop and evaluate a clinical decision support system (CDSS) named Assessment and Treatment in Healthcare: Evidenced-Based Automation (ATHENA)-Opioid Therapy, which encourages safe and effective use of opioid therapy for chronic, noncancer pain. CDSS development and iterative evaluation using the analysis, design, development, implementation, and evaluation process including simulation-based and in-clinic assessments of usability for providers followed by targeted system revisions. Volunteers provided detailed feedback to guide improvements in the graphical user interface, and content and design changes to increase clinical usefulness, understandability, clinical workflow fit, and ease of completing guideline recommended practices. Revisions based on feedback increased CDSS usability ratings over time. Practice concerns outside the scope of the CDSS were also identified. Usability testing optimized the CDSS to better address barriers such as lack of provider education, confusion in dosing calculations and titration schedules, access to relevant patient information, provider discontinuity, documentation, and access to validated assessment tools. It also highlighted barriers to good clinical practice that are difficult to address with CDSS technology in its current conceptualization. For example, clinicians indicated that constraints on time and competing priorities in primary care, discomfort in patient-provider communications, and lack of evidence to guide opioid prescribing decisions impeded their ability to provide effective, guideline-adherent pain management. Iterative testing was essential for designing a highly usable and acceptable CDSS; however, identified barriers may limit the impact of the ATHENA-Opioid Therapy system and other CDSS on clinical practices and outcomes unless CDSS are paired with parallel initiatives to address these issues.

Usability Testing of Two Ambulatory EHR Navigators.

Science.gov (United States)

Hultman, Gretchen; Marquard, Jenna; Arsoniadis, Elliot; Mink, Pamela; Rizvi, Rubina; Ramer, Tim; Khairat, Saif; Fickau, Keri; Melton, Genevieve B

2016-01-01

Despite widespread electronic health record (EHR) adoption, poor EHR system usability continues to be a significant barrier to effective system use for end users. One key to addressing usability problems is to employ user testing and user-centered design. To understand if redesigning an EHR-based navigation tool with clinician input improved user performance and satisfaction. A usability evaluation was conducted to compare two versions of a redesigned ambulatory navigator. Participants completed tasks for five patient cases using the navigators, while employing a think-aloud protocol. The tasks were based on Meaningful Use (MU) requirements. The version of navigator did not affect perceived workload, and time to complete tasks was longer in the redesigned navigator. A relatively small portion of navigator content was used to complete the MU-related tasks, though navigation patterns were highly variable across participants for both navigators. Preferences for EHR navigation structures appeared to be individualized. This study demonstrates the importance of EHR usability assessments to evaluate group and individual performance of different interfaces and preferences for each design.

Usability Constructs

DEFF Research Database (Denmark)

Hertzum, Morten; Clemmesen, Torkil; Hornbæk, Kasper Anders Søren

2007-01-01

Whereas research on usability predominantly employs universal definitions of the aspects that comprise usability, people experience their use of information systems through personal constructs. Based on 48 repertory-grid interviews, this study investigates how such personal constructs are affected...... use of constructs traditionally associated with usability (e.g., easy-to-use, intuitive, and liked). Further analysis of the data is ongoing...

Using scenarios and personas to enhance the effectiveness of heuristic usability evaluations for older adults and their care team.

Science.gov (United States)

Kneale, Laura; Mikles, Sean; Choi, Yong K; Thompson, Hilaire; Demiris, George

2017-09-01

Using heuristics to evaluate user experience is a common methodology for human-computer interaction studies. One challenge of this method is the inability to tailor results towards specific end-user needs. This manuscript reports on a method that uses validated scenarios and personas of older adults and care team members to enhance heuristics evaluations of the usability of commercially available personal health records for homebound older adults. Our work extends the Chisnell and Redish heuristic evaluation methodology by using a protocol that relies on multiple expert reviews of each system. It further standardizes the heuristic evaluation process through the incorporation of task-based scenarios. We were able to use the modified version of the Chisnell and Redish heuristic evaluation methodology to identify potential usability challenges of two commercially available personal health record systems. This allowed us to: (1) identify potential usability challenges for specific types of users, (2) describe improvements that would be valuable to all end-users of the system, and (3) better understand how the interactions of different users may vary within a single personal health record. The methodology described in this paper may help designers of consumer health information technology tools, such as personal health records, understand the needs of diverse end-user populations. Such methods may be particularly helpful when designing systems for populations that are difficult to recruit for end-user evaluations through traditional methods. Copyright © 2017 Elsevier Inc. All rights reserved.

Evaluating the parent-adolescent communication toolkit: Usability and preliminary content effectiveness of an online intervention.

Science.gov (United States)

Toombs, Elaine; Unruh, Anita; McGrath, Patrick

2018-01-01

This study aimed to assess the Parent-Adolescent Communication Toolkit, an online intervention designed to help improve parent communication with their adolescents. Participant preferences for two module delivery systems (sequential and unrestricted module access) were identified. Usability assessment of the PACT intervention was completed using pre-test and posttest comparisons. Usability data, including participant completion and satisfaction ratings were examined. Parents ( N  =   18) of adolescents were randomized to a sequential or unrestricted chapter access group. Parent participants completed pre-test measures, the PACT intervention and posttest measures. Participants provided feedback for the intervention to improve modules and provided usability ratings. Adolescent pre- and posttest ratings were evaluated. Usability ratings were high and parent feedback was positive. The sequential module access groups rated the intervention content higher and completed more content than the unrestricted chapter access group, indicating support for the sequential access design. Parent mean posttest communication scores were significantly higher ( p  Communication Toolkit has potential to improve parent-adolescent communication but further effectiveness assessment is required.

Evaluating the usability of web pages: a case study

NARCIS (Netherlands)

Lautenbach, M.A.E.; Schegget, I.E. ter; Schoute, A.E.; Witteman, C.L.M.

1999-01-01

An evaluation of the Utrecht University website was carried out with 240 students. New criteria were drawn from the literature and operationalized for the study. These criteria are surveyability and findability. Web pages can be said to satisfy a usability criterion if their efficiency and

Usability Testing for Developing Effective Interactive Multimedia Software: Concepts, Dimensions, and Procedures

Directory of Open Access Journals (Sweden)

Sung Heum Lee

1999-04-01

Full Text Available Usability testing is a dynamic process that can be used throughout the process of developing interactive multimedia software. The purpose of usability testing is to find problems and make recommendations to improve the utility of a product during its design and development. For developing effective interactive multimedia software, dimensions of usability testing were classified into the general categories of: learnability; performance effectiveness; flexibility; error tolerance and system integrity; and user satisfaction. In the process of usability testing, evaluation experts consider the nature of users and tasks, tradeoffs supported by the iterative design paradigm, and real world constraints to effectively evaluate and improve interactive multimedia software. Different methods address different purposes and involve a combination of user and usability testing, however, usability practitioners follow the seven general procedures of usability testing for effective multimedia development. As the knowledge about usability testing grows, evaluation experts will be able to choose more effective and efficient methods and techniques that are appropriate to their goals.

Evaluation and Satisfaction Survey on the Interface Usability of Online Publishing Software

Directory of Open Access Journals (Sweden)

Ying-Jye Lee

2014-01-01

Full Text Available Digital publishing is one of the national key programs. Different from traditional digital publishing models, consumers could create personal digital publications with the editing program provided by businesses and combine it with web-to-print to output solid publications. Nevertheless, the usability of online publishing software is related to consumers’ acceptance or intention of product purchase. In this case, Focus Group is utilized to screen representative online publishing software (including TinTint, Photobook, and Hypo for evaluating interface usability, investigating users’ Subjective Satisfaction, and further proposing suggestions for interface modification. Learnability and the number of user errors are set as the evaluation indicators of usability. Within the evaluation indicators in Learnability, the results show that nine typical tasks, except for Storing, show significant difference between various online publishing software. Typical tasks of basic information of works, appending pictures, adjusting pictures, changing type version, and changing pages in the number of user errors reveal significant difference on distinct online publishing software. Regarding the evaluation of overall Subjective Satisfaction with interface, TinTint and Hypo outperform Photobook, and no significant difference appears between TinTint and Hypo. It is expected that the research model could be the application reference of interface development and evaluation in digital content industries.

Are three methods better than one? A comparative assessment of usability evaluation methods in an EHR.

Science.gov (United States)

Walji, Muhammad F; Kalenderian, Elsbeth; Piotrowski, Mark; Tran, Duong; Kookal, Krishna K; Tokede, Oluwabunmi; White, Joel M; Vaderhobli, Ram; Ramoni, Rachel; Stark, Paul C; Kimmes, Nicole S; Lagerweij, Maxim; Patel, Vimla L

2014-05-01

To comparatively evaluate the effectiveness of three different methods involving end-users for detecting usability problems in an EHR: user testing, semi-structured interviews and surveys. Data were collected at two major urban dental schools from faculty, residents and dental students to assess the usability of a dental EHR for developing a treatment plan. These included user testing (N=32), semi-structured interviews (N=36), and surveys (N=35). The three methods together identified a total of 187 usability violations: 54% via user testing, 28% via the semi-structured interview and 18% from the survey method, with modest overlap. These usability problems were classified into 24 problem themes in 3 broad categories. User testing covered the broadest range of themes (83%), followed by the interview (63%) and survey (29%) methods. Multiple evaluation methods provide a comprehensive approach to identifying EHR usability challenges and specific problems. The three methods were found to be complementary, and thus each can provide unique insights for software enhancement. Interview and survey methods were found not to be sufficient by themselves, but when used in conjunction with the user testing method, they provided a comprehensive evaluation of the EHR. We recommend using a multi-method approach when testing the usability of health information technology because it provides a more comprehensive picture of usability challenges. Copyright © 2014 Elsevier Ireland Ltd. All rights reserved.

Intra- and Inter-Cultural Usability in Computer-Supported Collaboration

DEFF Research Database (Denmark)

Vatrapu, Ravi; Suthers, Dan

2010-01-01

In this paper, we argue for an increased scope of universal design to encompass usability and accessibility for not only users with physical disabilities but also for users from different cultures. Towards this end, we present an empirical evaluation of cultural usability in computer......-Chinese, and Chinese-Chinese) appropriated affordances and produced technological intersubjectivity. Cultural usability was analyzed through the use of performance and satisfaction measures. The results show a systemic variation in efficiency, effectiveness, and satisfaction between the two cultural groups....... Implications of these findings for the research and practice of usability, in general, and cultural usability, in particular, are discussed in this paper....

USABILITY TESTING OF JAPANESE CAPTIONS SEGMENTATION SYSTEM TO SCAFFOLD BEGINNERS TO COMPREHEND JAPANESE VIDEOS

Directory of Open Access Journals (Sweden)

Ya-Fei Yang

2013-06-01

Full Text Available A major learning difficulty of Japanese foreign language (JFL learners is the complex composition of two syllabaries, hiragana and katakana, and kanji characters adopted from logographic Chinese ones. As the number of Japanese language learners increases, computer-assisted Japanese language education gradually gains more attention. This study aimed to adopt a Japanese word segmentation system to help JFL learners overcome literacy problems. This study adopted MeCab, a Japanese morphological analyzer and part-of-speech (POS tagger, to segment Japanese texts into separate morphemes by adding spaces and to attach POS tags to each morpheme for beginners. The participants were asked to participate in three experimental activities involvingwatching two Japanese videos with general or segmented Japanese captions and complete the Nielsen’s Attributes of Usability (NAU survey and the After Scenario Questionnaire (ASQ to evaluate the usability of the learning activities. The results of the system evaluation showed that the videos with the segmented captions could increase the participants’ learning motivation and willingness to adopt the word segmentation system to learn Japanese.

Usability of the remote console for virtual reality telerehabilitation: formative evaluation.

Science.gov (United States)

Lewis, Jeffrey A; Deutsch, Judith E; Burdea, Grigore

2006-04-01

The Remote Console (ReCon) is a telerehabilitation application that allows therapists to remotely communicate with patients while monitoring and controlling their virtual rehabilitation exercises. It provides therapists visual feedback of patients' movements, their exercise simulations replicated in real time and with tools to conduct training without a face-to-face session. The Recon underwent a formative evaluation (a type of usability engineering methodology) used to refine its design. Five physical therapists from different practice settings acted as representative users. During the evaluation, these users made errors related to manipulation and finding and understanding controls. Technical issues with the server and audio communication were identified. These findings were used to fine-tune the ReCon system.

User Interface Problems of a Nationwide Inpatient Information System: A Heuristic Evaluation.

Science.gov (United States)

Atashi, Alireza; Khajouei, Reza; Azizi, Amirabbas; Dadashi, Ali

2016-01-01

While studies have shown that usability evaluation could uncover many design problems of health information systems, the usability of health information systems in developing countries using their native language is poorly studied. The objective of this study was to evaluate the usability of a nationwide inpatient information system used in many academic hospitals in Iran. Three trained usability evaluators independently evaluated the system using Nielsen's 10 usability heuristics. The evaluators combined identified problems in a single list and independently rated the severity of the problems. We statistically compared the number and severity of problems identified by HIS experienced and non-experienced evaluators. A total of 158 usability problems were identified. After removing duplications 99 unique problems were left. The highest mismatch with usability principles was related to "Consistency and standards" heuristic (25%) and the lowest related to "Flexibility and efficiency of use" (4%). The average severity of problems ranged from 2.4 (Major problem) to 3.3 (Catastrophe problem). The experienced evaluator with HIS identified significantly more problems and gave higher severities to problems (pusers' and patients' time, increase errors and finally threaten patient's safety. Many of them can be fixed with simple redesign solutions such as using clear labels and better layouts. This study suggests conducting further studies to confirm the findings concerning effect of evaluator experience on the results of Heuristic Evaluation.

Evaluation of the Usability of Different Virtual Lab Software Used in Physics Courses

Directory of Open Access Journals (Sweden)

O. Karagoz

2010-11-01

Full Text Available In recent years the use of virtual lab software has become ubiquitous in education settings. The purpose of the current study is to de¬velop an Evaluation Scale to allow the easy and fast assessment of virtual lab software. During the development of the Evaluation Scale, theoretical and experimental studies investigating the effects of different software on learn¬ing, particularly in terms of usability, were utilized. The Evaluation Scale was created by adding new attributes to pre-existing scales, and comprises three sections – attributes related to the interface of the software; attributes related to its use as a material in education; and attributes related to product and ser¬vice support – and 79 items. Testing of the Evaluation Scale was carried out using two different virtual lab programmes, with the help of a checklist. The evaluation was carried out by physics teachers and academics that had previ¬ously used similar software, and consistency between the results was consid¬ered to represent inter-rater reliability. At the end of the study, the usability of the Evaluation Scale was tested, and the instructor evaluations regarding the usability characteristics considered sufficient and that require improvement in virtual lab software were also investigated.

An Intelligent Framework for Website Usability

Directory of Open Access Journals (Sweden)

Alexiei Dingli

2014-01-01

Full Text Available With the major advances of the Internet throughout the past couple of years, websites have come to play a central role in the modern marketing business program. However, simply owning a website is not enough for a business to prosper on the Web. Indeed, it is the level of usability of a website that determines if a user stays or abandons it for another competing one. It is therefore crucial to understand the importance of usability on the web, and consequently the need for its evaluation. Nonetheless, there exist a number of obstacles preventing software organizations from successfully applying sound website usability evaluation strategies in practice. From this point of view automation of the latter is extremely beneficial, which not only assists designers in creating more usable websites, but also enhances the Internet users’ experience on the Web and increases their level of satisfaction. As a means of addressing this problem, an Intelligent Usability Evaluation (IUE tool is proposed that automates the usability evaluation process by employing a Heuristic Evaluation technique in an intelligent manner through the adoption of several research-based AI methods. Experimental results show there exists a high correlation between the tool and human annotators when identifying the considered usability violations.

A Model for Usability Evaluation for the Development and Implementation of Consumer eHealth Interventions.

Science.gov (United States)

Parry, David; Carter, Philip; Koziol-McLain, Jane; Feather, Jacqueline

2015-01-01

Consumer eHealth products are often used by people in their own homes or other settings without dedicated clinical supervision, and often with minimal training and limited support--much as eCommerce and eGovernment applications are currently deployed. Internet based self-care systems have been advocated for over a decade as a way to reduce costs and allow more convenient care, and--because of the expectation that they will be used to reduced health cost--, by increasing self-care and avoiding hospitalization. However, the history of consumer eHealth interventions is mixed, with many unsuccessful implementations. Many consumer eHealth products will form part of a broader complex intervention, with many possible benefits and effects on both individuals and society. This poster describes a model of consumer eHealth assessment based on multiple methods of usability evaluation at different stages in the design and fielding of eHealth systems. We argue that different methods of usability evaluation are able to give valuable insights into the likely effects of an intervention in a way that is congruent with software development processes.

Users' performance in lab and non-lab environments through online usability testing:A case of evaluating the usability of digital academic libraries' websites

OpenAIRE

Alharbi, Abeer; Mayhew, Pam

2015-01-01

The factors related to the environment in which users operate may be of a vital importance when trying to understand how they experience a particular system. It is required that we find out how we can get to know those factors to investigate if they affect the users' performance in usability testing. An online usability study has emerged that can be attempted by a large, varied pool of users' anywhere with an Internet connection. Would the usage of an online usability study help to give compr...

An investigation of the efficacy of collaborative virtual reality systems for moderated remote usability testing.

Science.gov (United States)

Chalil Madathil, Kapil; Greenstein, Joel S

2017-11-01

Collaborative virtual reality-based systems have integrated high fidelity voice-based communication, immersive audio and screen-sharing tools into virtual environments. Such three-dimensional collaborative virtual environments can mirror the collaboration among usability test participants and facilitators when they are physically collocated, potentially enabling moderated usability tests to be conducted effectively when the facilitator and participant are located in different places. We developed a virtual collaborative three-dimensional remote moderated usability testing laboratory and employed it in a controlled study to evaluate the effectiveness of moderated usability testing in a collaborative virtual reality-based environment with two other moderated usability testing methods: the traditional lab approach and Cisco WebEx, a web-based conferencing and screen sharing approach. Using a mixed methods experimental design, 36 test participants and 12 test facilitators were asked to complete representative tasks on a simulated online shopping website. The dependent variables included the time taken to complete the tasks; the usability defects identified and their severity; and the subjective ratings on the workload index, presence and satisfaction questionnaires. Remote moderated usability testing methodology using a collaborative virtual reality system performed similarly in terms of the total number of defects identified, the number of high severity defects identified and the time taken to complete the tasks with the other two methodologies. The overall workload experienced by the test participants and facilitators was the least with the traditional lab condition. No significant differences were identified for the workload experienced with the virtual reality and the WebEx conditions. However, test participants experienced greater involvement and a more immersive experience in the virtual environment than in the WebEx condition. The ratings for the virtual

The cultural evolution of human communication systems in different sized populations: usability trumps learnability.

Science.gov (United States)

Fay, Nicolas; Ellison, T Mark

2013-01-01

This study examines the intergenerational transfer of human communication systems. It tests if human communication systems evolve to be easy to learn or easy to use (or both), and how population size affects learnability and usability. Using an experimental-semiotic task, we find that human communication systems evolve to be easier to use (production efficiency and reproduction fidelity), but harder to learn (identification accuracy) for a second generation of naïve participants. Thus, usability trumps learnability. In addition, the communication systems that evolve in larger populations exhibit distinct advantages over those that evolve in smaller populations: the learnability loss (from the Initial signs) is more muted and the usability benefits are more pronounced. The usability benefits for human communication systems that evolve in a small and large population is explained through guided variation reducing sign complexity. The enhanced performance of the communication systems that evolve in larger populations is explained by the operation of a content bias acting on the larger pool of competing signs. The content bias selects for information-efficient iconic signs that aid learnability and enhance usability.

The McMaster Optimal Aging Portal: Usability Evaluation of a Unique Evidence-Based Health Information Website

Science.gov (United States)

Dobbins, Maureen; Haynes, R. Brian; Iorio, Alfonso; Lavis, John N; Raina, Parminder

2016-01-01

Background Increasingly, older adults and their informal caregivers are using the Internet to search for health-related information. There is a proliferation of health information online, but the quality of this information varies, often based on exaggerated or dramatic findings, and not easily comprehended by consumers. The McMaster Optimal Aging Portal (Portal) was developed to provide Internet users with high-quality evidence about aging and address some of these current limitations of health information posted online. The Portal includes content for health professionals coming from three best-in-class resources (MacPLUS, Health Evidence, and Health Systems Evidence) and four types of content specifically prepared for the general public (Evidence Summaries, Web Resource Ratings, Blog Posts, and Twitter messages). Objective Our objectives were to share the findings of the usability evaluation of the Portal with particular focus on the content features for the general public and to inform designers of health information websites and online resources for older adults about key usability themes. Methods Data analysis included task performance during usability testing and qualitative content analyses of both the usability sessions and interviews to identify core themes. Results A total of 37 participants took part in 33 usability testing sessions and 21 focused interviews. Qualitative analysis revealed common themes regarding the Portal’s strengths and challenges to usability. The strengths of the website were related to credibility, applicability, browsing function, design, and accessibility. The usability challenges included reluctance to register, process of registering, searching, terminology, and technical features. Conclusions The study reinforced the importance of including end users during the development of this unique, dynamic, evidence-based health information website. The feedback was applied to iteratively improve website usability. Our findings can be

Palliative Care Planner: A Pilot Study to Evaluate Acceptability and Usability of an Electronic Health Records System-integrated, Needs-targeted App Platform.

Science.gov (United States)

Cox, Christopher E; Jones, Derek M; Reagan, Wen; Key, Mary D; Chow, Vinca; McFarlin, Jessica; Casarett, David; Creutzfeldt, Claire J; Docherty, Sharron L

2018-01-01

The quality and patient-centeredness of intensive care unit (ICU)-based palliative care delivery is highly variable. To develop and pilot an app platform for clinicians and ICU patients and their family members that enhances the delivery of needs-targeted palliative care. In the development phase of the study, we developed an electronic health record (EHR) system-integrated mobile web app system prototype, PCplanner (Palliative Care Planner). PCplanner screens the EHR for ICU patients meeting any of five prompts (triggers) for palliative care consultation, allows families to report their unmet palliative care needs, and alerts clinicians to these needs. The evaluation phase included a prospective before/after study conducted at a large academic medical center. Two control populations were enrolled in the before period to serve as context for the intervention. First, 25 ICU patients who received palliative care consults served as patient-level controls. Second, 49 family members of ICU patients who received mechanical ventilation for at least 48 hours served as family-level controls. Afterward, 14 patients, 18 family members, and 10 clinicians participated in the intervention evaluation period. Family member outcomes measured at baseline and 4 days later included acceptability (Client Satisfaction Questionnaire [CSQ]), usability (Systems Usability Scale [SUS]), and palliative care needs, assessed with the adapted needs of social nature, existential concerns, symptoms, and therapeutic interaction (NEST) scale; the Patient-Centeredness of Care Scale (PCCS); and the Perceived Stress Scale (PSS). Patient outcomes included frequency of goal concordant treatment, hospital length of stay, and discharge disposition. Family members reported high PCplanner acceptability (mean CSQ, 14.1 [SD, 1.4]) and usability (mean SUS, 21.1 [SD, 1.7]). PCplanner family member recipients experienced a 12.7-unit reduction in NEST score compared with a 3.4-unit increase among controls (P�

Human factors review of CFMS displays for UCN 3,4 - Development of a usability evaluation method for the CFMS user interface of UCN 3,4

Energy Technology Data Exchange (ETDEWEB)

Hang, Sung Ho; Yun, Myung Hwan; Chung, Min Kyun; Kwahk, Ji Young; Hong, Sang Woo; Cho, Jang Hyun; Kim, Bum Soo [Pohang University of Science and Technology, Pohang (Korea, Republic of)

1996-11-01

The purpose of study was to perform a usability evaluation of the prototype Critical Function Monitoring System (CFMS) to be used in the Ulchin= 3 and Ulchin 4 nuclear power plant. Collection of existing regulation and literature, design review, design of the evaluation scheme, evaluation list, evaluation procedure, and evaluation method was conducted through literature survey, expert advice, design engineer consulting, interview of operators, and various systems analysis techniques. The main results of the study are as follows. 1) Usability evaluation of the CFMS screen 2) Recommendations for improved design 3) HED evaluations 4) Systematic evaluation procedure, method, and program implementation plan 5) Design of database for managing the evaluation format evaluation result. 47 tabs., 86 figs. (author)

The evaluation for the usability of the Varian Standard Couch modelingusing Treatment Planning System

Energy Technology Data Exchange (ETDEWEB)

Yang, Yong Mo; Song, Yong Min; Kim, Jin Man; Choi, Ji Min; Choi, Byeung Gi [Dept. of Radiation Oncology, Samsung Medical Center, Seoul (Korea, Republic of)

2016-06-15

When a radiation treatment, there is an attenuation by Carbon Fiber Couch. In this study, we tried to evaluate the usability of the Varian Standard Couch(VSC) by modeling with Treatment Planning System (TPS) VSC was scanned by CBCT(Cone Beam Computed Tomography) of the Linac(Clinac IX, VARIAN, USA), following the three conditions of VSC, Side Rail OutGrid(SROG), Side Rail InGrid(SRIG), Side Rail In OutSpine Down Bar(SRIOS). After scan, the data was transferred to TPS and modeled by contouring Side Rail, Side Bar Upper, Side Bar Lower, Spine Down Bar automatically. We scanned the Cheese Phantom(Middelton, USA) using Computed Tomography(Light Speed RT 16, GE, USA) and transfer the data to TPS, and apply VSC modeled previously with TPS to it. Dose was measured at the isocenter of Ion Chamber(A1SL, Standard imaging, USA) in Cheese Phantom using 4 and 10 MV radiation for every 5° gantry angle in a different filed size(3 X 3 cm{sup 2},10 X 10cm{sup 2}) without any change of MU(=100), and then we compared the calculated dose and measured dose. Also we included dose at the 127° in SRIG to compare the attenuation by Side Bar Upper. Results : The density of VSC by CBCT in TPS was 0.9g/cm{sup 3}, and in the case of Spine Down Bar, it was 0.7g/cm{sup 3}. The radiation was attenuated by 17.49%, 16.49%, 8.54%, and 7.59% at the Side Rail, Side Bar Upper, Side Bar Lower, and Spine Down Bar. For the accuracy of modeling, calculated dose and measured dose were compared. The average error was 1.13% and the maximum error was 1.98% at the 170° beam crossing the Spine Down Bar. To evaluate the usability for the VSC modeled by TPS, the maximum error was 1.98% as a result of compassion between calculated dose and measured dose. We found out that VSC modeling helped expect the dose, so we think that it will be helpful for the more accurate treatment.

The cultural evolution of human communication systems in different sized populations: usability trumps learnability.

Directory of Open Access Journals (Sweden)

Nicolas Fay

Full Text Available This study examines the intergenerational transfer of human communication systems. It tests if human communication systems evolve to be easy to learn or easy to use (or both, and how population size affects learnability and usability. Using an experimental-semiotic task, we find that human communication systems evolve to be easier to use (production efficiency and reproduction fidelity, but harder to learn (identification accuracy for a second generation of naïve participants. Thus, usability trumps learnability. In addition, the communication systems that evolve in larger populations exhibit distinct advantages over those that evolve in smaller populations: the learnability loss (from the Initial signs is more muted and the usability benefits are more pronounced. The usability benefits for human communication systems that evolve in a small and large population is explained through guided variation reducing sign complexity. The enhanced performance of the communication systems that evolve in larger populations is explained by the operation of a content bias acting on the larger pool of competing signs. The content bias selects for information-efficient iconic signs that aid learnability and enhance usability.

Usability Metrics for Gamified E-learning Course: A Multilevel Approach

Directory of Open Access Journals (Sweden)

Aleksandra Sobodić

2018-04-01

Full Text Available This paper discusses the effect of a gamified learning system for students of the master course on Web Design and Programming performed at the Faculty of Organization and Informatics. A new set of usability metrics was derived from web-based learning usability, user experience and instructional design literature and incorporated into the questionnaire which consists of three main categories: Usability, Educational Usability and User Experience. The main contribution of this paper is the development and validation of a questionnaire for measuring the usability of a gamified e-learning course from students’ perspective. Usability practitioners can use the developed metrics with confidence when evaluating the design of a gamified e-learning course in order to improve students’ engagement and motivation.

Usability evaluation of mobile ICT support used at the building construction site

DEFF Research Database (Denmark)

Christiansson, Per; Svidt, Kjeld

2006-01-01

The paper summarizes findings from field evaluations and controlled laboratory usability evaluations of new mobile Information and Communication Technology, ICT, support used by craftsmen at construction sites as well as a discussion of methodologies for user centred ICT tools design. The finding...

Barefooted Usability Evaluation: Addressing the Mindset, Resources and Competences

OpenAIRE

Bruun , Anders

2011-01-01

Part 1: Long and Short Papers; International audience; Lack of usability specialists, high resource requirements and developer mindsets are three considerable barriers for introducing usability engineering into software companies. This Ph.D. project explores the effect of letting software developers and end users apply usability engineering methods instead of a specialist, a solution which may reduce the barriers.

Personal Usability Constructs

DEFF Research Database (Denmark)

Hertzum, Morten; Clemmensen, Torkil; Hornbæk, Kasper

2011-01-01

Whereas the concept of usability is predominantly defined analytically, people relate to systems through personal usability constructs. Based on 48 repertory-grid interviews, this study investigates how such personal constructs are affected by two factors crucial to the international development...

A framework for evaluating electronic health record vendor user-centered design and usability testing processes.

Science.gov (United States)

Ratwani, Raj M; Zachary Hettinger, A; Kosydar, Allison; Fairbanks, Rollin J; Hodgkins, Michael L

2017-04-01

Currently, there are few resources for electronic health record (EHR) purchasers and end users to understand the usability processes employed by EHR vendors during product design and development. We developed a framework, based on human factors literature and industry standards, to systematically evaluate the user-centered design processes and usability testing methods used by EHR vendors. We reviewed current usability certification requirements and the human factors literature to develop a 15-point framework for evaluating EHR products. The framework is based on 3 dimensions: user-centered design process, summative testing methodology, and summative testing results. Two vendor usability reports were retrieved from the Office of the National Coordinator's Certified Health IT Product List and were evaluated using the framework. One vendor scored low on the framework (5 pts) while the other vendor scored high on the framework (15 pts). The 2 scored vendor reports demonstrate the framework's ability to discriminate between the variabilities in vendor processes and to determine which vendors are meeting best practices. The framework provides a method to more easily comprehend EHR vendors' usability processes and serves to highlight where EHR vendors may be falling short in terms of best practices. The framework provides a greater level of transparency for both purchasers and end users of EHRs. The framework highlights the need for clearer certification requirements and suggests that the authorized certification bodies that examine vendor usability reports may need to be provided with clearer guidance. © The Author 2016. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Development of a System to Assist Automatic Translation of Hand-Drawn Maps into Tactile Graphics and Its Usability Evaluation

Directory of Open Access Journals (Sweden)

Jianjun Chen

2014-01-01

Full Text Available Tactile graphics are images that use raised surfaces so that a visually impaired person can feel them. Tactile maps are used by blind and partially sighted people when navigating around an environment, and they are also used prior to a visit for orientation purposes. Since the ability to read tactile graphics deeply depends on individuals, providing tactile graphics individually is needed. This implies that producing tactile graphics should be as simple as possible. Based on this background, we are developing a system for automating production of tactile maps from hand-drawn figures. In this paper, we first present a pattern recognition method for hand-drawn maps. The usability of our system is then evaluated by comparing it with the two different methods to produce tactile graphics.

Preliminary study of online machine translation use of nursing literature: quality evaluation and perceived usability

Directory of Open Access Journals (Sweden)

Anazawa Ryoko

2012-11-01

Full Text Available Abstract Background Japanese nurses are increasingly required to read published international research in clinical, educational, and research settings. Language barriers are a significant obstacle, and online machine translation (MT is a tool that can be used to address this issue. We examined the quality of Google Translate® (English to Japanese and Korean to Japanese, which is a representative online MT, using a previously verified evaluation method. We also examined the perceived usability and current use of online MT among Japanese nurses. Findings Randomly selected nursing abstracts were translated and then evaluated for intelligibility and usability by 28 participants, including assistants and research associates from nursing universities throughout Japan. They answered a questionnaire about their online MT use. From simple comparison of mean scores between two language pairs, translation quality was significantly better, with respect to both intelligibility and usability, for Korean-Japanese than for English-Japanese. Most respondents perceived a language barrier. Online MT had been used by 61% of the respondents and was perceived as not useful enough. Conclusion Nursing articles translated from Korean into Japanese by an online MT system could be read at an acceptable level of comprehension, but the same could not be said for English-Japanese translations. Respondents with experience using online MT used it largely to grasp the overall meanings of the original text. Enrichment in technical terms appeared to be the key to better usability. Users will be better able to use MT outputs if they improve their foreign language proficiency as much as possible. Further research is being conducted with a larger sample size and detailed analysis.

Improving the Interplay between Usability Evaluation and User Interface Design

DEFF Research Database (Denmark)

Hornbæk, K.; Stage, Jan

2004-01-01

of the workshop are motivated and an outline of the contents of the papers that were presented in the workshop is given. In addition we summarize some challenges to the interplay between usability evaluation and user interface design agreed upon at the workshop, as well as some solutions that were debated....

Usability: Human Research Program - Space Human Factors and Habitability

Science.gov (United States)

Sandor, Aniko; Holden, Kritina L.

2009-01-01

The Usability project addresses the need for research in the area of metrics and methodologies used in hardware and software usability testing in order to define quantifiable and verifiable usability requirements. A usability test is a human-in-the-loop evaluation where a participant works through a realistic set of representative tasks using the hardware/software under investigation. The purpose of this research is to define metrics and methodologies for measuring and verifying usability in the aerospace domain in accordance with FY09 focus on errors, consistency, and mobility/maneuverability. Usability metrics must be predictive of success with the interfaces, must be easy to obtain and/or calculate, and must meet the intent of current Human Systems Integration Requirements (HSIR). Methodologies must work within the constraints of the aerospace domain, be cost and time efficient, and be able to be applied without extensive specialized training.

Methods of Usability Testing in Libraries Web Sites

Directory of Open Access Journals (Sweden)

Eman Fawzy

2006-03-01

Full Text Available A Study about libraries' web sites evaluation, that is the Usability, the study talking about methods of usability testing and define it, and its important in web sites evaluation, then details the methods of usability: questionnaire, core groups, testing experimental model, cards arrangement, and composed evaluation.

Multi-attribute evaluation and choice of alternatives for surplus weapons-usable plutonium disposition at uncertainty

International Nuclear Information System (INIS)

Kosterev, V.V.; Bolyatko, V.V.; Khajretdinov, S.I.; Averkin, A.N.

2014-01-01

The problem of surplus weapons-usable plutonium disposition is formalized as a multi-attribute problem of a choice of alternatives from a set of possible alternatives under fuzzy conditions. Evaluation and ordering of alternatives for the surplus weapons-usable plutonium disposition and sensitivity analysis are carried out at uncertainty [ru

Questionnaire for usability evaluation of orthopaedic shoes: construction and reliability in patients with degenerative disorders of the foot

NARCIS (Netherlands)

Jannink, M.J.A.; de Vries, Jaap; Stewart, Roy E.; Groothoff, Johan W.; Lankhorst, Gustaaf J.

2004-01-01

Objective: To develop a self-report questionnaire for patients with degenerative disorders of the foot to evaluate the usability of their orthopaedic shoes, and to assess the reproducibility and responsiveness of the instrument. Design: Development of the Questionnaire for Usability Evaluation of

Questionnaire for usability evaluation of orthopaedic shoes : Construction and reliability in patients with degenerative disorders of the foot

NARCIS (Netherlands)

Jannink, MJA; de Vries, J; Stewart, RE; Groothoff, JW; Lankhorst, GJ

2004-01-01

Objective: To develop a self-report questionnaire for patients with degenerative disorders of the foot to evaluate the usability of their orthopaedic shoes, and to assess the reproducibility and responsiveness of the instrument. Design: Development of the Questionnaire for Usability Evaluation of

Questionnaire for usability evaluation of orthopaedic shoes : Construction and reliability in patients with degenerative disorders of the foot

NARCIS (Netherlands)

Jannink, MJA; de Vries, J; Stewart, RE; Groothoff, JW; Lankhorst, GJ

Objective: To develop a self-report questionnaire for patients with degenerative disorders of the foot to evaluate the usability of their orthopaedic shoes, and to assess the reproducibility and responsiveness of the instrument. Design: Development of the Questionnaire for Usability Evaluation of

Usability in Scientific Databases

Directory of Open Access Journals (Sweden)

Ana-Maria Suduc

2012-07-01

Full Text Available Usability, most often defined as the ease of use and acceptability of a system, affects the users' performance and their job satisfaction when working with a machine. Therefore, usability is a very important aspect which must be considered in the process of a system development. The paper presents several numerical data related to the history of the scientific research of the usability of information systems, as it is viewed in the information provided by three important scientific databases, Science Direct, ACM Digital Library and IEEE Xplore Digital Library, at different queries related to this field.

Determining the Number of Participants Needed for the Usability Evaluation of E-Learning Resources: A Monte Carlo Simulation

Science.gov (United States)

Davids, Mogamat Razeen; Harvey, Justin; Halperin, Mitchell L.; Chikte, Usuf M. E.

2015-01-01

The usability of computer interfaces has a major influence on learning. Optimising the usability of e-learning resources is therefore essential. However, this may be neglected because of time and monetary constraints. User testing is a common approach to usability evaluation and involves studying typical end-users interacting with the application…

A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence.

Science.gov (United States)

Jaspers, Monique W M

2009-05-01

Usability evaluation is now widely recognized as critical to the success of interactive health care applications. However, the broad range of usability inspection and testing methods available may make it difficult to decide on a usability assessment plan. To guide novices in the human-computer interaction field, we provide an overview of the methodological and empirical research available on the three usability inspection and testing methods most often used. We describe two 'expert-based' and one 'user-based' usability method: (1) the heuristic evaluation, (2) the cognitive walkthrough, and (3) the think aloud. All three usability evaluation methods are applied in laboratory settings. Heuristic evaluation is a relatively efficient usability evaluation method with a high benefit-cost ratio, but requires high skills and usability experience of the evaluators to produce reliable results. The cognitive walkthrough is a more structured approach than the heuristic evaluation with a stronger focus on the learnability of a computer application. Major drawbacks of the cognitive walkthrough are the required level of detail of task and user background descriptions for an adequate application of the latest version of the technique. The think aloud is a very direct method to gain deep insight in the problems end users encounter in interaction with a system but data analyses is extensive and requires a high level of expertise both in the cognitive ergonomics and in computer system application domain. Each of the three usability evaluation methods has shown its usefulness, has its own advantages and disadvantages; no single method has revealed any significant results indicating that it is singularly effective in all circumstances. A combination of different techniques that compliment one another should preferably be used as their collective application will be more powerful than applied in isolation. Innovative mobile and automated solutions to support end-user testing have

Establishing usability heuristics for heuristics evaluation in a specific domain: Is there a consensus?

Science.gov (United States)

Hermawati, Setia; Lawson, Glyn

2016-09-01

Heuristics evaluation is frequently employed to evaluate usability. While general heuristics are suitable to evaluate most user interfaces, there is still a need to establish heuristics for specific domains to ensure that their specific usability issues are identified. This paper presents a comprehensive review of 70 studies related to usability heuristics for specific domains. The aim of this paper is to review the processes that were applied to establish heuristics in specific domains and identify gaps in order to provide recommendations for future research and area of improvements. The most urgent issue found is the deficiency of validation effort following heuristics proposition and the lack of robustness and rigour of validation method adopted. Whether domain specific heuristics perform better or worse than general ones is inconclusive due to lack of validation quality and clarity on how to assess the effectiveness of heuristics for specific domains. The lack of validation quality also affects effort in improving existing heuristics for specific domain as their weaknesses are not addressed. Copyright © 2016 Elsevier Ltd. All rights reserved.

Usability Problem Identification in Culturally Diverse Settings

DEFF Research Database (Denmark)

Clemmensen, Torkil

2012-01-01

There are indications that established methods for evaluating information system usability that have been developed for use in, e.g. Europe or the USA, fail to give reliable results in countries such as India, China or Malaysia. This paper presents the theoretical background, related work...... and a definition of culture that should be useful for studies of multiple-country usability testing. This includes a discussion of cultural fit and the consequences of cultural (in)consistencies between stakeholders in system development and use. As an illustrative example of the kind of academic research...... that needs to be done, a pilot study is described. The pilot study exemplifies themes to explore, who should be participants and where should the study be done, how to find examples of multiple-country usability testing, how to collect data and how to analyse that data and what kind of results and discussion...

To twist or poke? A method for identifying usability issues with the rotary controller and touch screen for control of in-vehicle information systems.

Science.gov (United States)

Harvey, Catherine; Stanton, Neville A; Pickering, Carl A; McDonald, Mike; Zheng, Pengjun

2011-07-01

In-vehicle information systems (IVIS) can be controlled by the user via direct or indirect input devices. In order to develop the next generation of usable IVIS, designers need to be able to evaluate and understand the usability issues associated with these two input types. The aim of this study was to investigate the effectiveness of a set of empirical usability evaluation methods for identifying important usability issues and distinguishing between the IVIS input devices. A number of usability issues were identified and their causal factors have been explored. These were related to the input type, the structure of the menu/tasks and hardware issues. In particular, the translation between inputs and on-screen actions and a lack of visual feedback for menu navigation resulted in lower levels of usability for the indirect device. This information will be useful in informing the design of new IVIS, with improved usability. STATEMENT OF RELEVANCE: This paper examines the use of empirical methods for distinguishing between direct and indirect IVIS input devices and identifying usability issues. Results have shown that the characteristics of indirect input devices produce more serious usability issues, compared with direct devices and can have a negative effect on the driver-vehicle interaction.

A Facebook-Based Obesity Prevention Program for Korean American Adolescents: Usability Evaluation.

Science.gov (United States)

Park, Bu Kyung; Nahm, Eun-Shim; Rogers, Valerie E; Choi, Mona; Friedmann, Erika; Wilson, Marisa; Koru, Gunes

Adolescent obesity is one of the most serious global public health challenges. Social networking sites are currently popular among adolescents. Therefore, the obesity prevention program for Korean American adolescents was developed on the most popular social networking site, Facebook. The purpose of this study was to evaluate the usability of a culturally tailored Facebook-based obesity prevention program for Korean American adolescents (Healthy Teens). An explorative descriptive design of usability testing was used. Usability testing employing one-on-one observation, the think-aloud method, audio taping, screen activity capture, and surveys was performed. Twenty participants were recruited from two Korean language schools (mean age, 15.40 ± 1.50 years). Recruitment and user testing was performed between February and April 2014. Content analysis, using the inductive coding approach, was performed by three coders to analyze transcriptions. Descriptive statistics were used to analyze quantitative data including demographic characteristics, perceived usability, eHealth literacy, and health behaviors. Testing revealed several usability issues in content, appearance, and navigation. Participants' comments regarding content were positive. Although the Facebook platform provided limited flexibility with respect to building the site, participants described the program's appearance as appropriate. Most participants did not experience difficulty in navigating the program. Our preliminary findings indicated that participants perceived the Healthy Teens program as usable and useful. This program could be used as a robust platform for the delivery of health education to adolescents. Further research is required to assess the effects of Facebook-based programs on adolescent obesity prevention. Copyright © 2016 National Association of Pediatric Nurse Practitioners. Published by Elsevier Inc. All rights reserved.

From Usability Testing to Clinical Simulations: Bringing Context into the Design and Evaluation of Usable and Safe Health Information Technologies. Contribution of the IMIA Human Factors Engineering for Healthcare Informatics Working Group.

Science.gov (United States)

Kushniruk, A; Nohr, C; Jensen, S; Borycki, E M

2013-01-01

The objective of this paper is to explore human factors approaches to understanding the use of health information technology (HIT) by extending usability engineering approaches to include analysis of the impact of clinical context through use of clinical simulations. Methods discussed are considered on a continuum from traditional laboratory-based usability testing to clinical simulations. Clinical simulations can be conducted in a simulation laboratory and they can also be conducted in real-world settings. The clinical simulation approach attempts to bring the dimension of clinical context into stronger focus. This involves testing of systems with representative users doing representative tasks, in representative settings/environments. Application of methods where realistic clinical scenarios are used to drive the study of users interacting with systems under realistic conditions and settings can lead to identification of problems and issues with systems that may not be detected using traditional usability engineering methods. In conducting such studies, careful consideration is needed in creating ecologically valid test scenarios. The evidence obtained from such evaluation can be used to improve both the usability and safety of HIT. In addition, recent work has shown that clinical simulations, in particular those conducted in-situ, can lead to considerable benefits when compared to the costs of running such studies. In order to bring context of use into the testing of HIT, clinical simulation, involving observing representative users carrying out tasks in representative settings, holds considerable promise.

How do Usability Professionals Construe Usability?

DEFF Research Database (Denmark)

Hertzum, Morten; Clemmensen, Torkil

2012-01-01

distinguishing it from user experience. On the basis of repertory-grid interviews with 24 Chinese, Danish, and Indian usability professionals we find that they make use of more utilitarian than experiential, i.e. user-experience related, constructs. This indicates that goal-related performance is central......’ constructs, particularly their experiential constructs, go considerably beyond ISO 9241 usability, indicating a discrepancy between this definition of usability and the thinking of the professionals concerned with delivering usability. Finally, usability is construed rather similarly across the three...

The Performance and Usability of a Factory-Calibrated Flash Glucose Monitoring System

OpenAIRE

Bailey, Timothy; Bode, Bruce W.; Christiansen, Mark P.; Klaff, Leslie J.; Alva, Shridhara

2015-01-01

Abstract Introduction: The purpose of the study was to evaluate the performance and usability of the FreeStyle? Libre? Flash glucose monitoring system (Abbott Diabetes Care, Alameda, CA) for interstitial glucose results compared with capillary blood glucose results. Materials and Methods: Seventy-two study participants with type 1 or type 2 diabetes were enrolled by four U.S. clinical sites. A sensor was inserted on the back of each upper arm for up to 14 days. Three factory-only calibrated s...

Usability in Government Systems User Experience Design for Citizens and Public Servants

CERN Document Server

Buie, Elizabeth

2012-01-01

As a usability specialist or interaction designer working with the government, or as a government or contractor professional involved in specifying, procuring, or managing system development, you need this book. Editors Elizabeth Buie and Dianne Murray have brought together over 30 experts to outline practical advice to both usability specialists and government technology professionals and managers. Working with internal and external government systems is a unique and difficult task because of of the sheer magnitude of the audience for external systems (the entire population of a country, a

A Usability Evaluation of a Blended MOOC Environment: An Experimental Case Study

Directory of Open Access Journals (Sweden)

Ahmed Mohamed Fahmy Yousef

2015-04-01

Full Text Available In the past few years, there has been an increasing interest in Massive Open Online Courses (MOOCs as a new form of Technology-Enhanced Learning (TEL, in higher education and beyond. Recognizing the limitations of standalone MOOCs, blended MOOCs (bMOOCs that aim at bringing in-class (i.e. face-to-face interactions and online learning components together have emerged as an alternative MOOC model of teaching and learning in a higher education context. In this paper, we present the design, implementation, and evaluation details of a bMOOC course on “Teaching Methodologies” at Fayoum University, Egypt in cooperation with RWTH Aachen University, Germany, provided using the bMOOC platform L2P-bMOOC. In order to gauge the usability and effectiveness of the course, we employed an evaluation approach based on Conole’s 12 dimensions rubrics, ISONORM 9241/110-S as a general usability evaluation, and a custom effectiveness questionnaire reflecting the different MOOC stakeholder perspectives.

The Usability of SEEQ in Quality Evaluation of Arabic Secondary Education in Malaysia

Science.gov (United States)

al-Muslim, M.; Arifin, Zamri

2015-01-01

Evaluation of the quality of Arabic education in Malaysia needs to be conducted on a continuous basis to achieve the objectives of Arabic education. Student Evaluation of Education Quality (SEEQ) was proposed as one of the evaluation instruments of the quality of Arabic education. This study aimed to evaluate the usability of SEEQ in the context…

Review of Spatial-Database System Usability: Recommendations for the ADDNS Project

National Research Council Canada - National Science Library

Abdalla, R. M; Niall, K. K

2007-01-01

...) and three-dimensional (3D) visualizations. This report presents an overview of the basic concepts of GIS and spatial databases, provides an analytical usability evaluation and critically analyses different spatial- database applications...

Templates for Cross-Cultural and Culturally Specific Usability Testing

DEFF Research Database (Denmark)

Clemmensen, Torkil

2011-01-01

The cultural diversity of users of technology challenges our methods for usability testing. This article suggests templates for cross-culturally and culturally specific usability testing, based on studies of usability testing in companies in Mumbai, Beijing, and Copenhagen. Study 1 was a cross...... tests. The result was the construction of templates for usability testing. The culturally specific templates were in Mumbai “user-centered evaluation,” Copenhagen “client-centered evaluation,” and Beijing “evaluator-centered evaluation.” The findings are compared with related research...

Distributed usability evaluation of the Pennsylvania Cancer Atlas.

Science.gov (United States)

Bhowmick, Tanuka; Robinson, Anthony C; Gruver, Adrienne; MacEachren, Alan M; Lengerich, Eugene J

2008-07-11

The Pennsylvania Cancer Atlas (PA-CA) is an interactive online atlas to help policy-makers, program managers, and epidemiologists with tasks related to cancer prevention and control. The PA-CA includes maps, graphs, tables, that are dynamically linked to support data exploration and decision-making with spatio-temporal cancer data. Our Atlas development process follows a user-centered design approach. To assess the usability of the initial versions of the PA-CA, we developed and applied a novel strategy for soliciting user feedback through multiple distributed focus groups and surveys. Our process of acquiring user feedback leverages an online web application (e-Delphi). In this paper we describe the PA-CA, detail how we have adapted e-Delphi web application to support usability and utility evaluation of the PA-CA, and present the results of our evaluation. We report results from four sets of users. Each group provided structured individual and group assessments of the PA-CA as well as input on the kinds of users and applications for which it is best suited. Overall reactions to the PA-CA are quite positive. Participants did, however, provide a range of useful suggestions. Key suggestions focused on improving interaction functions, enhancing methods of temporal analysis, addressing data issues, and providing additional data displays and help functions. These suggestions were incorporated in each design and implementation iteration for the PA-CA and used to inform a set of web-atlas design principles. For the Atlas, we find that a design that utilizes linked map, graph, and table views is understandable to and perceived to be useful by the target audience of cancer prevention and control professionals. However, it is clear that considerable variation in experience using maps and graphics exists and for those with less experience, integrated tutorials and help features are needed. In relation to our usability assessment strategy, we find that our distributed, web

Distributed usability evaluation of the Pennsylvania Cancer Atlas

Directory of Open Access Journals (Sweden)

MacEachren Alan M

2008-07-01

Full Text Available Abstract Background The Pennsylvania Cancer Atlas (PA-CA is an interactive online atlas to help policy-makers, program managers, and epidemiologists with tasks related to cancer prevention and control. The PA-CA includes maps, graphs, tables, that are dynamically linked to support data exploration and decision-making with spatio-temporal cancer data. Our Atlas development process follows a user-centered design approach. To assess the usability of the initial versions of the PA-CA, we developed and applied a novel strategy for soliciting user feedback through multiple distributed focus groups and surveys. Our process of acquiring user feedback leverages an online web application (e-Delphi. In this paper we describe the PA-CA, detail how we have adapted e-Delphi web application to support usability and utility evaluation of the PA-CA, and present the results of our evaluation. Results We report results from four sets of users. Each group provided structured individual and group assessments of the PA-CA as well as input on the kinds of users and applications for which it is best suited. Overall reactions to the PA-CA are quite positive. Participants did, however, provide a range of useful suggestions. Key suggestions focused on improving interaction functions, enhancing methods of temporal analysis, addressing data issues, and providing additional data displays and help functions. These suggestions were incorporated in each design and implementation iteration for the PA-CA and used to inform a set of web-atlas design principles. Conclusion For the Atlas, we find that a design that utilizes linked map, graph, and table views is understandable to and perceived to be useful by the target audience of cancer prevention and control professionals. However, it is clear that considerable variation in experience using maps and graphics exists and for those with less experience, integrated tutorials and help features are needed. In relation to our usability

Reusable Software Usability Specifications for mHealth Applications.

Science.gov (United States)

Cruz Zapata, Belén; Fernández-Alemán, José Luis; Toval, Ambrosio; Idri, Ali

2018-01-25

One of the key factors for the adoption of mobile technologies, and in particular of mobile health applications, is usability. A usable application will be easier to use and understand by users, and will improve user's interaction with it. This paper proposes a software requirements catalog for usable mobile health applications, which can be used for the development of new applications, or the evaluation of existing ones. The catalog is based on the main identified sources in literature on usability and mobile health applications. Our catalog was organized according to the ISO/IEC/IEEE 29148:2011 standard and follows the SIREN methodology to create reusable catalogs. The applicability of the catalog was verified by the creation of an audit method, which was used to perform the evaluation of a real app, S Health, application created by Samsung Electronics Co. The usability requirements catalog, along with the audit method, identified several usability flaws on the evaluated app, which scored 83%. Some flaws were detected in the app related to the navigation pattern. Some more issues related to the startup experience, empty screens or writing style were also found. The way a user navigates through an application improves or deteriorates user's experience with the application. We proposed a reusable usability catalog and an audit method. This proposal was used to evaluate a mobile health application. An audit report was created with the usability issues identified on the evaluated application.

Evaluating the Usability of a Professional Modeling Tool Repurposed for Middle School Learning

Science.gov (United States)

Peters, Vanessa L.; Songer, Nancy Butler

2013-10-01

This paper reports the results of a three-stage usability test of a modeling tool designed to support learners' deep understanding of the impacts of climate change on ecosystems. The design process involved repurposing an existing modeling technology used by professional scientists into a learning tool specifically designed for middle school students. To evaluate usability, we analyzed students' task performance and task completion time as they worked on an activity with the repurposed modeling technology. In stage 1, we conducted remote testing of an early modeling prototype with urban middle school students (n = 84). In stages 2 and 3, we used screencasting software to record students' mouse and keyboard movements during collaborative think-alouds (n = 22) and conducted a qualitative analysis of their peer discussions. Taken together, the study findings revealed two kinds of usability issues that interfered with students' productive use of the tool: issues related to the use of data and information, and issues related to the use of the modeling technology. The study findings resulted in design improvements that led to stronger usability outcomes and higher task performance among students. In this paper, we describe our methods for usability testing, our research findings, and our design solutions for supporting students' use of the modeling technology and use of data. The paper concludes with implications for the design and study of modeling technologies for science learning.

Enhancing usability using Near Field Communication for mobile application

Directory of Open Access Journals (Sweden)

Wihidayat Endar

2018-01-01

Full Text Available Near Field Communication (NFC as relatively new wireless communication technology pushes new challenges to application developers to make their applications easier to use and simpler to operate. This point of view known as usability element. Usability is one of the elements for creating good quality applications. This study aims to analyse the usability of mobile-based application embeds with NFC. We also try to evaluate usability in applications used by children. We developed an application called Receptionist which has a primary function as a communication tool between students, teachers and parents at a middle school. To know the impact of the NFC, the Receptionist input system is designed with two methods, via conventional navigation (using buttons and via NFC. To understand the usability of each method, we do user testing and questioners on students. The results show, using the NFC there is a significant increase in usability attributes: efficiency, effectiveness, and learnability. On the other hand, there is decreases of user satisfaction comparing to conventional method. In general, this study demonstrates the potential of new input device technologies that can improve the usability of mobile-based applications.

Teaching clinical reasoning and decision-making skills to nursing students: Design, development, and usability evaluation of a serious game.

Science.gov (United States)

Johnsen, Hege Mari; Fossum, Mariann; Vivekananda-Schmidt, Pirashanthie; Fruhling, Ann; Slettebø, Åshild

2016-10-01

Serious games (SGs) are a type of simulation technology that may provide nursing students with the opportunity to practice their clinical reasoning and decision-making skills in a safe and authentic environment. Despite the growing number of SGs developed for healthcare professionals, few SGs are video based or address the domain of home health care. This paper aims to describe the design, development, and usability evaluation of a video based SG for teaching clinical reasoning and decision-making skills to nursing students who care for patients with chronic obstructive pulmonary disease (COPD) in home healthcare settings. A prototype SG was developed. A unified framework of usability called TURF (Task, User, Representation, and Function) and SG theory were employed to ensure a user-centered design. The educational content was based on the clinical decision-making model, Bloom's taxonomy, and a Bachelor of Nursing curriculum. A purposeful sample of six participants evaluated the SG prototype in a usability laboratory. Cognitive walkthrough evaluations, a questionnaire, and individual interviews were used for the usability evaluation. The data were analyzed using qualitative deductive content analysis based on the TURF framework elements and related usability heuristics. The SG was perceived as being realistic, clinically relevant, and at an adequate level of complexity for the intended users. Usability issues regarding functionality and the user-computer interface design were identified. However, the SG was perceived as being easy to learn, and participants suggested that the SG could serve as a supplement to traditional training in laboratory and clinical settings. Using video based scenarios with an authentic COPD patient and a home healthcare registered nurse as actors contributed to increased realism. Using different theoretical approaches in the SG design was considered an advantage of the design process. The SG was perceived as being useful, usable, and

PROOF OF CONCEPT FOR A HUMAN RELIABILITY ANALYSIS METHOD FOR HEURISTIC USABILITY EVALUATION OF SOFTWARE

International Nuclear Information System (INIS)

Ronald L. Boring; David I. Gertman; Jeffrey C. Joe; Julie L. Marble

2005-01-01

An ongoing issue within human-computer interaction (HCI) is the need for simplified or ''discount'' methods. The current economic slowdown has necessitated innovative methods that are results driven and cost effective. The myriad methods of design and usability are currently being cost-justified, and new techniques are actively being explored that meet current budgets and needs. Recent efforts in human reliability analysis (HRA) are highlighted by the ten-year development of the Standardized Plant Analysis Risk HRA (SPAR-H) method. The SPAR-H method has been used primarily for determining human centered risk at nuclear power plants. The SPAR-H method, however, shares task analysis underpinnings with HCI. Despite this methodological overlap, there is currently no HRA approach deployed in heuristic usability evaluation. This paper presents an extension of the existing SPAR-H method to be used as part of heuristic usability evaluation in HCI

Usable security history, themes, and challenges

CERN Document Server

Garfinkel, Simson

2014-01-01

There has been roughly 15 years of research into approaches for aligning research in Human Computer Interaction with computer Security, more colloquially known as ``usable security.'' Although usability and security were once thought to be inherently antagonistic, today there is wide consensus that systems that are not usable will inevitably suffer security failures when they are deployed into the real world. Only by simultaneously addressing both usability and security concerns will we be able to build systems that are truly secure. This book presents the historical context of the work to dat

Understanding Usability Work as a Human Activity

DEFF Research Database (Denmark)

Nørgaard, Mie

Three core themes are explored in eight papers: Usability work as a human activity, usability practice and methods, and persuasiveness of evaluation results and feedback. We explore how usability work is much more than methods and work procedures, and argue that maturing our understanding...

Active Involvement of Software Developers in Usability Engineering

DEFF Research Database (Denmark)

Bornoe, Nis; Stage, Jan

2017-01-01

The essence of usability evaluations is to produce feedback that supports the downstream utility so the interaction design can be improved and problems can be fixed. In practice, software development organizations experience several obstacles for conducting usability engineering. One suggested...... approach is to train and involve developers in all phases of usability activities from evaluations, to problem reporting, and making redesign proposals. Only limited work has previously investigated the impact of actively involving developers in usability engineering. In this paper, we present two small......, and problem fixing. At the organizational level, we found that the attitude towards and understanding of the role of usability engineering improved....

Scrutinising usability evaluation: does thinking aloud affect behaviour and mental workload?

DEFF Research Database (Denmark)

Hertzum, Morten; Hansen, Kristin Due; Andersen, Hans Henrik

2009-01-01

thinking aloud participants took longer to solve tasks, spent a larger part of tasks on general distributed visual behaviour, issued more commands to navigate both within and between the pages of the websites used in the experiment, and experienced higher mental workload. Implications for usability......Thinking aloud is widely used for usability evaluation. The validity of the method is, however, debatable because it is generally used in a relaxed way that conflicts with the prescriptions of the classic model for obtaining valid verbalisations of thought processes. This study investigates whether...... participants that think aloud in the classic or relaxed way behave differently compared to performing in silence. Results indicate that whereas classic thinking aloud has little or no effect on behaviour apart from prolonging tasks, relaxed thinking aloud affects behaviour in multiple ways. During relaxed...

A Concise and Practical Framework for the Development and Usability Evaluation of Patient Information Websites.

Science.gov (United States)

Peute, L W; Knijnenburg, S L; Kremer, L C; Jaspers, M W M

2015-01-01

The Website Developmental Model for the Healthcare Consumer (WDMHC) is an extensive and successfully evaluated framework that incorporates user-centered design principles. However, due to its extensiveness its application is limited. In the current study we apply a subset of the WDMHC framework in a case study concerning the development and evaluation of a website aimed at childhood cancer survivors (CCS). To assess whether the implementation of a limited subset of the WDMHC-framework is sufficient to deliver a high-quality website with few usability problems, aimed at a specific patient population. The website was developed using a six-step approach divided into three phases derived from the WDMHC: 1) information needs analysis, mock-up creation and focus group discussion; 2) website prototype development; and 3) heuristic evaluation (HE) and think aloud analysis (TA). The HE was performed by three double experts (knowledgeable both in usability engineering and childhood cancer survivorship), who assessed the site using the Nielsen heuristics. Eight end-users were invited to complete three scenarios covering all functionality of the website by TA. The HE and TA were performed concurrently on the website prototype. The HE resulted in 29 unique usability issues; the end-users performing the TA encountered eleven unique problems. Four issues specifically revealed by HE concerned cosmetic design flaws, whereas two problems revealed by TA were related to website content. Based on the subset of the WDMHC framework we were able to deliver a website that closely matched the expectancy of the end-users and resulted in relatively few usability problems during end-user testing. With the successful application of this subset of the WDMHC, we provide developers with a clear and easily applicable framework for the development of healthcare websites with high usability aimed at specific medical populations.

A Usability Study of Users' Perceptions toward a Multimedia Computer-Assisted Learning Tool for Neuroanatomy

Science.gov (United States)

Gould, Douglas J.; Terrell, Mark A.; Fleming, Jo

2008-01-01

This usability study evaluated users' perceptions of a multimedia prototype for a new e-learning tool: Anatomy of the Central Nervous System: A Multimedia Course. Usability testing is a collection of formative evaluation methods that inform the developmental design of e-learning tools to maximize user acceptance, satisfaction, and adoption.…

Responsibilities in the Usability Requirements Elicitation Process

Directory of Open Access Journals (Sweden)

Marianella Aveledo

2008-12-01

Full Text Available Like any other software system quality attribute, usability places requirements on software components. In particular, it has been demonstrated that certain usability features have a direct impact throughout the software process. This paper details an approach that looks at how to deal with certain usability features in the early software development stages. In particular, we consider usability features as functional usability requirements using patterns that have been termed usability patterns to elicit requirements. Additionally, we clearly establish the responsibilities of all the players at the usability requirements elicitation stage.

Log files for testing usability

NARCIS (Netherlands)

Klein Teeselink, G.; Siepe, A.H.M.; Pijper, de J.R.

1999-01-01

The aim of this study is to gain insight in the usefulness of log file analysis as a method to evaluate the usability of individual interface components and their influence on the usability of the overall user interface. We selected a music player as application, with four different interfaces and

Educational technology "Anatomy and Vital Signs": Evaluation study of content, appearance and usability.

Science.gov (United States)

de Góes, Fernanda dos Santos Nogueira; Fonseca, Luciana Mara Monti; de Camargo, Rosangela Andrade Aukar; de Oliveira, Gustavo Faria; Felipe, Helena Reche

2015-11-01

The use of new technology has recently grown considerably as an increasing number of college students using Internet. In nursing education, the personal computer and the Internet facilitate teaching theoretical and practical knowledge. Evaluate an educational technology known as "Anatomy and Vital Signs" with respect to content, appearance and usability. This was a first stage evaluation-by specialists to verify content and functioning, prior to a second validation as to learning by students. A methodological study in which instructional technologists (11 participants) and nursing specialists (17 participants) used the technology in an unguided manner and completed three questionnaires. The evaluation was measured by the difference between disagreement and agreement for each statement in the questionnaires. Most of the items were positively evaluated at a level higher than 70% by most of the evaluators except for the following usability criteria: grouping by shape, minimum actions and user control, which did not attain the 70% agreement level among instructional technologists. The evaluation was useful to improve the technology and guarantee suitable product for nursing education. It may be a reliable educational tool for nursing education that applies technological resources. Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.

Integrating usability testing and think-aloud protocol analysis with "near-live" clinical simulations in evaluating clinical decision support.

Science.gov (United States)

Li, Alice C; Kannry, Joseph L; Kushniruk, Andre; Chrimes, Dillon; McGinn, Thomas G; Edonyabo, Daniel; Mann, Devin M

2012-11-01

Usability evaluations can improve the usability and workflow integration of clinical decision support (CDS). Traditional usability testing using scripted scenarios with think-aloud protocol analysis provide a useful but incomplete assessment of how new CDS tools interact with users and clinical workflow. "Near-live" clinical simulations are a newer usability evaluation tool that more closely mimics clinical workflow and that allows for a complementary evaluation of CDS usability as well as impact on workflow. This study employed two phases of testing a new CDS tool that embedded clinical prediction rules (an evidence-based medicine tool) into primary care workflow within a commercial electronic health record. Phase I applied usability testing involving "think-aloud" protocol analysis of 8 primary care providers encountering several scripted clinical scenarios. Phase II used "near-live" clinical simulations of 8 providers interacting with video clips of standardized trained patient actors enacting the clinical scenario. In both phases, all sessions were audiotaped and had screen-capture software activated for onscreen recordings. Transcripts were coded using qualitative analysis methods. In Phase I, the impact of the CDS on navigation and workflow were associated with the largest volume of negative comments (accounting for over 90% of user raised issues) while the overall usability and the content of the CDS were associated with the most positive comments. However, usability had a positive-to-negative comment ratio of only 0.93 reflecting mixed perceptions about the usability of the CDS. In Phase II, the duration of encounters with simulated patients was approximately 12 min with 71% of the clinical prediction rules being activated after half of the visit had already elapsed. Upon activation, providers accepted the CDS tool pathway 82% of times offered and completed all of its elements in 53% of all simulation cases. Only 12.2% of encounter time was spent using the

Usability in the lifecycle of medical software development

Directory of Open Access Journals (Sweden)

Trauzettel Franziska

2016-09-01

Full Text Available A close cooperation with users is necessary to ensure that interactive systems are robust, easy to use and accepted. Therefore, in medical technology, standards for usability are of fundamental importance. We investigated with the presented study how the concept of usability is currently understood and implemented in medical software companies. Interviews were conducted with 21 employees of German enterprises. Furthermore we extended an already existing quantitative online survey where 53 companies (including 24 from the health industry sector participated in. Results show that the importance of usability is recognized by most of the respondents. Moreover, a wide variety of methods and approaches is known and implemented for exploring user needs and evaluating system prototypes. However, it was observed that human-centered design activities mainly focus on functionality, risk prevention and accessibility. Hedonic user needs and subjective perceptions (“user experience” still play a minor role. Based on the results, practical requirements are derived and a “best case” for methodological approach is introduced.

Building a Remote Usability Testing Body of Knowledge (RUTBOK)

DEFF Research Database (Denmark)

Lizano, Fulvio; Stage, Jan

2012-01-01

Remote Usability Testing (RUT) is an alternative method of usability evaluations. The main aim of RUT is the reduction of costs in the usability evaluation process. Since middle of the 90s RUT, has emerged as a technique which seems as is gaining maturity across the time. In general any discipline...

Dynamic Tunnel Usability Study: Format Recommendations for Synthetic Vision System Primary Flight Displays

Science.gov (United States)

Arthur, Jarvis J., III; Prinzel, Lawrence J., III; Kramer, Lynda J.; Bailey, Randall E.

2006-01-01

A usability study evaluating dynamic tunnel concepts has been completed under the Aviation Safety and Security Program, Synthetic Vision Systems Project. The usability study was conducted in the Visual Imaging Simulator for Transport Aircraft Systems (VISTAS) III simulator in the form of questionnaires and pilot-in-the-loop simulation sessions. Twelve commercial pilots participated in the study to determine their preferences via paired comparisons and subjective rankings regarding the color, line thickness and sensitivity of the dynamic tunnel. The results of the study showed that color was not significant in pilot preference paired comparisons or in pilot rankings. Line thickness was significant for both pilot preference paired comparisons and in pilot rankings. The preferred line/halo thickness combination was a line width of 3 pixels and a halo of 4 pixels. Finally, pilots were asked their preference for the current dynamic tunnel compared to a less sensitive dynamic tunnel. The current dynamic tunnel constantly gives feedback to the pilot with regard to path error while the less sensitive tunnel only changes as the path error approaches the edges of the tunnel. The tunnel sensitivity comparison results were not statistically significant.

Mobile Usability Testing in Healthcare: Methodological Approaches.

Science.gov (United States)

Borycki, Elizabeth M; Monkman, Helen; Griffith, Janessa; Kushniruk, Andre W

2015-01-01

The use of mobile devices and healthcare applications is increasing exponentially worldwide. This has lead to the need for the healthcare industry to develop a better understanding of the impact of the usability of mobile software and hardware upon consumer and health professional adoption and use of these technologies. There are many methodological approaches that can be employed in conducting usability evaluation of mobile technologies. More obtrusive approaches to collecting study data may lead to changes in study participant behaviour, leading to study results that are less consistent with how the technologies will be used in the real-world. Alternatively, less obstrusive methods used in evaluating the usability of mobile software and hardware in-situ and laboratory settings can lead to less detailed information being collected about how an individual interacts with both the software and hardware. In this paper we review and discuss several innovative mobile usability evaluation methods on a contiuum from least to most obtrusive and their effects on the quality of the usability data collected. The strengths and limitations of methods are also discussed.

Does Usability Matter? An Analysis of the Impact of Usability on Technology Acceptance in ERP Settings

Directory of Open Access Journals (Sweden)

Brenda M Scholtz

2016-11-01

Full Text Available Though the field of management information systems, as a sector and a discipline, is the inventor of many guidelines and models, it appears to be a slow runner on practical implications of interface usability. This usability can influence end users’ attitude and behavior to use IT. The purpose of this paper was to examine the interface usability of a popular Enterprise Resource Planning (ERP software system, SAP, and to identify related issues and implications to the Technology Acceptance Model (TAM. A survey was conducted of 112 SAP ERP users from an organization in the heavy metal industry in Bangladesh. The partial least squares technique was used to analyze the survey data. The survey findings empirically confirmed that interface usability has a significant impact on users’ perceptions of usefulness and ease of use which ultimately affects attitudes and intention to use the ERP software. The research model extends the TAM by incorporating three criteria of interface usability. It is the first known study to investigate usability criteria as an extension of TAM.

Online user tracking and usability tools - a mapping study

OpenAIRE

Nyström, Niklas

2014-01-01

Tracking user activities online can be done in order to improve systems usability, understand user behaviour or to increase monetary prot, to mention a few things. The software tools for this purpose can vary, for example in terms of usability of the tool itself, features, security, price, accessibility and support options. In this study I have developed a methodology to evaluate a selection of modern tools and map them depending on various properties. I have also analysed learning theories t...

Exploring the Usability of Mobile Apps Supporting Radiologists' Training in Diagnostic Decision Making.

Science.gov (United States)

Kim, Min Soon; Aro, Michael R; Lage, Kraig J; Ingalls, Kevin L; Sindhwani, Vivek; Markey, Mia K

2016-03-01

The objective of this study was to conduct a usability evaluation of mobile apps for supporting education and training in radiologic diagnostic decision-making processes. Of 381 mobile apps available at two major stores (Google Play and iTunes), eight iOS apps were selected for laboratory-based usability tests. Six staff radiologists completed eight app-specific task sets, using a think-aloud strategy. The triangular methods approach included quantitative performance measures, System Usability Scale (SUS), and qualitative thematic analysis using heuristic usability principles of usability issues. Overall, radiologists achieved higher than 70% success, with favorable SUS scores, in completing the tasks for seven of the eight apps. However, task success rate and SUS score had a weak relation (r = 0.23), indicating that the perceived usability may not reflect the holistic usability of the app. Task analysis and self-report revealed 108 usability issues, which were condensed to 55 unique issues and categorized by nine usability themes and mapped to ten usability heuristics. Nonintuitive functionality (eg, nonintuitive or misleading labels) was the most frequent theme observed, leading to inefficient navigation. These usability findings were consistent with the 13 improvements the radiologists suggested. This study demonstrates the feasibility of usability evaluation of radiology mobile apps and suggests potential improvements in the development of radiology mobile apps. This study also suggests that proficiency with mobile devices may not be equivalent to being an expert user, proficient in using the apps. Copyright © 2016 American College of Radiology. Published by Elsevier Inc. All rights reserved.

Toward a More Usable Home-Based Video Telemedicine System: A Heuristic Evaluation of the Clinician User Interfaces of Home-Based Video Telemedicine Systems.

Science.gov (United States)

Agnisarman, Sruthy; Narasimha, Shraddhaa; Chalil Madathil, Kapil; Welch, Brandon; Brinda, Fnu; Ashok, Aparna; McElligott, James

2017-04-24

Telemedicine is the use of technology to provide and support health care when distance separates the clinical service and the patient. Home-based telemedicine systems involve the use of such technology for medical support and care connecting the patient from the comfort of their homes with the clinician. In order for such a system to be used extensively, it is necessary to understand not only the issues faced by the patients in using them but also the clinician. The aim of this study was to conduct a heuristic evaluation of 4 telemedicine software platforms-Doxy.me, Polycom, Vidyo, and VSee-to assess possible problems and limitations that could affect the usability of the system from the clinician's perspective. It was found that 5 experts individually evaluated all four systems using Nielsen's list of heuristics, classifying the issues based on a severity rating scale. A total of 46 unique problems were identified by the experts. The heuristics most frequently violated were visibility of system status and Error prevention amounting to 24% (11/46 issues) each. Esthetic and minimalist design was second contributing to 13% (6/46 issues) of the total errors. Heuristic evaluation coupled with a severity rating scale was found to be an effective method for identifying problems with the systems. Prioritization of these problems based on the rating provides a good starting point for resolving the issues affecting these platforms. There is a need for better transparency and a more streamlined approach for how physicians use telemedicine systems. Visibility of the system status and speaking the users' language are keys for achieving this. ©Sruthy Agnisarman, Shraddhaa Narasimha, Kapil Chalil Madathil, Brandon Welch, FNU Brinda, Aparna Ashok, James McElligott. Originally published in JMIR Human Factors (http://humanfactors.jmir.org), 24.04.2017.

Enhancing the Usability of an Optical Reader System to Support Point-of-Care Rapid Diagnostic Testing: An Iterative Design Approach.

Science.gov (United States)

Hohenstein, Jess; O'Dell, Dakota; Murnane, Elizabeth L; Lu, Zhengda; Erickson, David; Gay, Geri

2017-11-21

In today's health care environment, increasing costs and inadequate medical resources have created a worldwide need for more affordable diagnostic tools that are also portable, fast, and easy to use. To address this issue, numerous research and commercial efforts have focused on developing rapid diagnostic technologies; however, the efficacy of existing systems has been hindered by usability problems or high production costs, making them infeasible for deployment in at-home, point-of-care (POC), or resource-limited settings. The aim of this study was to create a low-cost optical reader system that integrates with any smart device and accepts any type of rapid diagnostic test strip to provide fast and accurate data collection, sample analysis, and diagnostic result reporting. An iterative design methodology was employed by a multidisciplinary research team to engineer three versions of a portable diagnostic testing device that were evaluated for usability and overall user receptivity. Repeated design critiques and usability studies identified a number of system requirements and considerations (eg, software compatibility, biomatter contamination, and physical footprint) that we worked to incrementally incorporate into successive system variants. Our final design phase culminated in the development of Tidbit, a reader that is compatible with any Wi-Fi-enabled device and test strip format. The Tidbit includes various features that support intuitive operation, including a straightforward test strip insertion point, external indicator lights, concealed electronic components, and an asymmetric shape, which inherently signals correct device orientation. Usability testing of the Tidbit indicates high usability for potential user communities. This study presents the design process, specification, and user reception of the Tidbit, an inexpensive, easy-to-use, portable optical reader for fast, accurate quantification of rapid diagnostic test results. Usability testing suggests

ReMashed - An Usability Study of a Recommender System for Mash-Ups for Learning

Directory of Open Access Journals (Sweden)

Rob Koper

2010-01-01

Full Text Available The following article presents an usability study of a Mash-up Personal Learning Environment called ReMashed that recommends items from the emerging information of a Learning Network. In ReMashed users can specify certain Web 2.0 services and combine them in a Mash-Up Personal Learning Environment. The users can rate information from an emerging amount of Web 2.0 information of a Learning Network and train a recommender system for their particular needs. In total 49 participants from 8 different countries registered to evaluate the ReMashed system. The participants contributed Web 2.0 contents and used the recommender system for one month. The evaluation was concluded with an online questionnaire where most of the participants were positive about the ReMashed system and offered helpful ideas for future developments.

Usability evaluation of low-cost virtual reality hand and arm rehabilitation games.

Science.gov (United States)

Seo, Na Jin; Arun Kumar, Jayashree; Hur, Pilwon; Crocher, Vincent; Motawar, Binal; Lakshminarayanan, Kishor

2016-01-01

The emergence of lower-cost motion tracking devices enables home-based virtual reality rehabilitation activities and increased accessibility to patients. Currently, little documentation on patients' expectations for virtual reality rehabilitation is available. This study surveyed 10 people with stroke for their expectations of virtual reality rehabilitation games. This study also evaluated the usability of three lower-cost virtual reality rehabilitation games using a survey and House of Quality analysis. The games (kitchen, archery, and puzzle) were developed in the laboratory to encourage coordinated finger and arm movements. Lower-cost motion tracking devices, the P5 Glove and Microsoft Kinect, were used to record the movements. People with stroke were found to desire motivating and easy-to-use games with clinical insights and encouragement from therapists. The House of Quality analysis revealed that the games should be improved by obtaining evidence for clinical effectiveness, including clinical feedback regarding improving functional abilities, adapting the games to the user's changing functional ability, and improving usability of the motion-tracking devices. This study reports the expectations of people with stroke for rehabilitation games and usability analysis that can help guide development of future games.

Useful and usable alarm systems : recommended properties

International Nuclear Information System (INIS)

Veland, Oeystein; Kaarstad, Magnhild; Seim, Lars Aage; Foerdestroemmen, Nils T.

2001-01-01

This document describes the result of a study on alarm systems conducted by IFE in Halden. The study was initiated by the Norwegian Petroleum Directorate. The objective was to identify and formulate a set of important properties for useful and usable alarm systems. The study is mainly based on review of the latest international recognised guidelines and standards on alarm systems available at the time of writing, with focus on realistic solutions from research and best practice from different industries. In addition, IFE experiences gathered through specification and design of alarm systems and experimental activities in HAMMLAB and bilateral projects, have been utilized where relevant. The document presents a total of 43 recommendations divided into a number of general recommendations and more detailed recommendations on alarm generation, structuring, prioritisation, presentation and handling. (Author)

Accessibility and usability OCW data: The UTPL OCW

Directory of Open Access Journals (Sweden)

Germania Rodríguez

2017-08-01

Full Text Available This data article provides a data description on article entitled “A framework for improving web accessibility and usability of Open Course Ware sites” [3] This Data in Brief presents the data obtained from the accessibility and usability evaluation of the UTPL OCW. The data obtained from the framework evaluation consists of the manual evaluation of the standards criteria and the automatic evaluation of the tools Google PageSpeed and Google Analytics. In addition, this article presents the synthetized tables from standards that are used by the framework to evaluate the accessibility and usability of OCW, and the questionnaires required to extract the data. As a result, the article also provides the data required to reproduce the evaluation of other OCW.

Usability Plans and Schemes for Low Fidelity Prototypes

DEFF Research Database (Denmark)

Schultz, Nette; Kirk Sørensen, Jannick; Sørensen, Lene

2006-01-01

Internal report of WP1 task 4 activities from August 2006 to December 2006. This report describes the setup of a usability evaluation framework for evaluating core MAGNET concepts in terms of usability. Initially, these core concepts are identified and modes of operation involving...

Ten factors to consider when developing usability scenarios and tasks for health information technology.

Science.gov (United States)

Russ, Alissa L; Saleem, Jason J

2018-02-01

The quality of usability testing is highly dependent upon the associated usability scenarios. To promote usability testing as part of electronic health record (EHR) certification, the Office of the National Coordinator (ONC) for Health Information Technology requires that vendors test specific capabilities of EHRs with clinical end-users and report their usability testing process - including the test scenarios used - along with the results. The ONC outlines basic expectations for usability testing, but there is little guidance in usability texts or scientific literature on how to develop usability scenarios for healthcare applications. The objective of this article is to outline key factors to consider when developing usability scenarios and tasks to evaluate computer-interface based health information technologies. To achieve this goal, we draw upon a decade of our experience conducting usability tests with a variety of healthcare applications and a wide range of end-users, to include healthcare professionals as well as patients. We discuss 10 key factors that influence scenario development: objectives of usability testing; roles of end-user(s); target performance goals; evaluation time constraints; clinical focus; fidelity; scenario-related bias and confounders; embedded probes; minimize risks to end-users; and healthcare related outcome measures. For each factor, we present an illustrative example. This article is intended to aid usability researchers and practitioners in their efforts to advance health information technologies. The article provides broad guidance on usability scenario development and can be applied to a wide range of clinical information systems and applications. Published by Elsevier Inc.

A review of privacy and usability issues in mobile health systems: Role of external factors.

Science.gov (United States)

Katusiime, Jane; Pinkwart, Niels

2017-10-01

The increased penetration of mobile devices has created opportunities in the health sector and led to emerging of mobile health systems. As much as the mobile health systems have registered tremendous progress, they have been faced with privacy and usability issues. Due to the sensitivity of health information, there is an ethical need to equip mobile health systems with adequate privacy measures. However, these systems should also be useable by the intended users. Even though many researchers are working on solutions, the issues still persist. External factors such as cultural differences have also contributed to the issues, yet they have been under researched. In this article, we conduct a systematic literature review of 22 articles, categorize and present privacy and usability issues and possible solutions. We then discuss the relevance and implications of external factors to the findings on privacy and usability. We end with recommendations to address these external factors.

Software architecture analysis of usability

NARCIS (Netherlands)

Folmer, E; van Gurp, J; Bosch, J; Bastide, R; Palanque, P; Roth, J

2005-01-01

Studies of software engineering projects show that a large number of usability related change requests are made after its deployment. Fixing usability problems during the later stages of development often proves to be costly, since many of the necessary changes require changes to the system that

OPTIMIZING USABILITY OF AN ECONOMIC DECISION SUPPORT TOOL: PROTOTYPE OF THE EQUIPT TOOL.

Science.gov (United States)

Cheung, Kei Long; Hiligsmann, Mickaël; Präger, Maximilian; Jones, Teresa; Józwiak-Hagymásy, Judit; Muñoz, Celia; Lester-George, Adam; Pokhrel, Subhash; López-Nicolás, Ángel; Trapero-Bertran, Marta; Evers, Silvia M A A; de Vries, Hein

2018-01-01

Economic decision-support tools can provide valuable information for tobacco control stakeholders, but their usability may impact the adoption of such tools. This study aims to illustrate a mixed-method usability evaluation of an economic decision-support tool for tobacco control, using the EQUIPT ROI tool prototype as a case study. A cross-sectional mixed methods design was used, including a heuristic evaluation, a thinking aloud approach, and a questionnaire testing and exploring the usability of the Return of Investment tool. A total of sixty-six users evaluated the tool (thinking aloud) and completed the questionnaire. For the heuristic evaluation, four experts evaluated the interface. In total twenty-one percent of the respondents perceived good usability. A total of 118 usability problems were identified, from which twenty-six problems were categorized as most severe, indicating high priority to fix them before implementation. Combining user-based and expert-based evaluation methods is recommended as these were shown to identify unique usability problems. The evaluation provides input to optimize usability of a decision-support tool, and may serve as a vantage point for other developers to conduct usability evaluations to refine similar tools before wide-scale implementation. Such studies could reduce implementation gaps by optimizing usability, enhancing in turn the research impact of such interventions.

Usability Assessment of Text-to-Speech Synthesis for Additional Detail in an Automated Telephone Banking System

OpenAIRE

Morton , Hazel; Gunson , Nancie; Marshall , Diarmid; McInnes , Fergus; Ayres , Andrea; Jack , Mervyn

2010-01-01

Abstract This paper describes a comprehensive usability evaluation of an automated telephone banking system which employs text-to-speech (TTS) synthesis in offering additional detail on customers? account transactions. The paper describes a series of four experiments in which TTS was employed to offer an extra level of detail to recent transactions listings within an established banking service which otherwise uses recorded speech from a professional recording artist. Results from ...

Usability Evaluation of a Web-Based Symptom Monitoring Application for Heart Failure.

Science.gov (United States)

Wakefield, Bonnie; Pham, Kassie; Scherubel, Melody

2015-07-01

Symptom recognition and reporting by patients with heart failure are critical to avoid hospitalization. This project evaluated a patient symptom tracking application. Fourteen end users (nine patients, five clinicians) from a Midwestern Veterans Affairs Medical Center evaluated the website using a think aloud protocol. A structured observation protocol was used to assess success or failure for each task. Measures included task time, success, and satisfaction. Patients had a mean age of 70 years; clinicians averaged 42 years in age. Patients took 9.3 min and clinicians took less than 3 min per scenario. Most patients needed some assistance, but few patients were completely unable to complete some tasks. Clinicians demonstrated few problems navigating the site. Patient System Usability Scale item scores ranged from 2.0 to 3.6; clinician item scores ranged from 1.8 to 4.0. Further work is needed to determine whether using the web-based tool improves symptom recognition and reporting. © The Author(s) 2015.

Usability: a critical success factor for managing change in the clinical info-structure.

Science.gov (United States)

Kay, S

2005-06-01

There can be no doubt that the clinical info-structure is being significantly enriched with the deployment of new systems throughout the health sector. From a technological perspective, the initial emphasis has been mainly on functionality and only latterly on the usability of these clinical information systems. However, the large scale and rapid pace of the changes being wrought in the health sector will have a major impact on clinicians and patients, not least in how they interact with the technology. Therefore, it is not only hardware and software but people-ware, too, that needs to be actively managed; not simply a one-off functional specification but an ongoing, complex relationship. Usability is the human factor that encompasses the ethical, educational, and evaluative aspects of design. There is also a strong case for regarding usability of clinical information systems as a key critical success factor for the management of change within the health-care domain. In particular, the relationship between usability, and education and training is examined.

How Do Clinical Information Systems Affect the Cognitive Demands of General Practitioners?: Usability Study with a Focus on Cognitive Workload

Directory of Open Access Journals (Sweden)

Ferran Ariza

2015-11-01

Full Text Available Background Clinical information systems in the National Health Service do not need to conform to any explicit usability requirements. Poor usability can increase the mental workload experienced by clinicians and cause fatigue, increase error rates and impact the overall patient safety. Mental workload can be used as a measure of usability.Objective To assess the subjective cognitive workload experienced by general practitioners (GPs with their systems. To raise awareness of the importance of usability in system design among users, designers, developers and policymakers.Methods We used a modified version of the NASA Task Load Index, adapted for web. We developed a set of common clinical scenarios and computer tasks on an online survey. We emailed the study link to 199 clinical commissioning groups and 1,646 GP practices in England. Results Sixty-seven responders completed the survey. The respondents had spent an average of 17 years in general practice, had experience of using a mean of 1.5 GP computer systems and had used their current system for a mean time of 6.7 years. The mental workload score was not different among systems. There were significant differences among the task scores, but these differences were not specific to particular systems. The overall score and task scores were related to the length of experience with their present system. Conclusion Four tasks imposed a higher mental workload on GPs: ‘repeat prescribing’, ‘find episode’, ‘drug management’ and ‘overview records’. Further usability studies on GP systems should focus on these tasks. Users, policymakers, designers and developers should remain aware of the importance of usability in system design.What does this study add?• Current GP systems in England do not need to conform to explicit usability requirements. Poor usability can increase the mental workload of clinicians and lead to errors.• Some clinical computer tasks incur more cognitive workload

Determining Reliability and Validity of the Persian Version of Software Usability Measurements Inventory (SUMI) Questionnaire

OpenAIRE

seyed abolfazl zakerian; Roya Azizi; Mehdi Rahgozar

2013-01-01

The term usability refers to a special index for success of an operating system. This study aimed to determine the reliability and validity of the Software Usability Measurements Inventory (SUMI) questionnaire as one of the valid and common questionnaires about usability evaluation. The back translation method was used to translate the questionnaire from English to Persian back to English. Moreover, repeatability or test-retest reliability was practically used to determine the reliability of ...

An Evaluation of the Design and Usability of a Novel Robotic Bilateral Arm Rehabilitation Device for Patients with Stroke

Directory of Open Access Journals (Sweden)

Yu-Cheng Pei

2017-07-01

Full Text Available Study designCase series.Evidence levelIV (case series.IntroductionRobot-assisted therapy for upper limb rehabilitation is an emerging research topic and its design process must integrate engineering, neurological pathophysiology, and clinical needs.Purpose of the studyThis study developed/evaluated the usefulness of a novel rehabilitation device, the MirrorPath, designed for the upper limb rehabilitation of patients with hemiplegic stroke.MethodsThe process follows Tseng’s methodology for innovative product design and development, namely two stages, device development and usability assessment. During the development process, the design was guided by patients’ rehabilitation needs as defined by patients and their therapists. The design applied synchronic movement of the bilateral upper limbs, an approach that is compatible with the bilateral movement therapy and proprioceptive neuromuscular facilitation theories. MirrorPath consists of a robotic device that guides upper limb movement linked to a control module containing software controlling the robotic movement.ResultsFive healthy subjects were recruited in the pretest, and 4 patients, 4 caregivers, and 4 therapists were recruited in the formal test for usability. All recruited subjects were allocated to the test group, completed the evaluation, and their data were all analyzed. The total system usability scale score obtained from the patients, caregivers, and therapists was 71.8 ± 11.9, indicating a high level of usability and product acceptance.Discussion and conclusionFollowing a standard development process, we could yield a design that meets clinical needs. This low-cost device provides a feasible platform for carrying out robot-assisted bilateral movement therapy of patients with hemiplegic stroke.Clinical Trial Registrationidentifier NCT02698605.

Getting a technology-based diabetes intervention ready for prime time: a review of usability testing studies.

Science.gov (United States)

Lyles, Courtney R; Sarkar, Urmimala; Osborn, Chandra Y

2014-10-01

Consumer health technologies can educate patients about diabetes and support their self-management, yet usability evidence is rarely published even though it determines patient engagement, optimal benefit of any intervention, and an understanding of generalizability. Therefore, we conducted a narrative review of peer-reviewed articles published from 2009 to 2013 that tested the usability of a web- or mobile-delivered system/application designed to educate and support patients with diabetes. Overall, the 23 papers included in our review used mixed (n = 11), descriptive quantitative (n = 9), and qualitative methods (n = 3) to assess usability, such as documenting which features performed as intended and how patients rated their experiences. More sophisticated usability evaluations combined several complementary approaches to elucidate more aspects of functionality. Future work pertaining to the design and evaluation of technology-delivered diabetes education/support interventions should aim to standardize the usability testing processes and publish usability findings to inform interpretation of why an intervention succeeded or failed and for whom.

Translation, Cross-Cultural Adaptation, and Validation of the Malay Version of the System Usability Scale Questionnaire for the Assessment of Mobile Apps.

Science.gov (United States)

Mohamad Marzuki, Muhamad Fadhil; Yaacob, Nor Azwany; Yaacob, Najib Majdi

2018-05-14

A mobile app is a programmed system designed to be used by a target user on a mobile device. The usability of such a system refers not only to the extent to which product can be used to achieve the task that it was designed for, but also its effectiveness and efficiency, as well as user satisfaction. The System Usability Scale is one of the most commonly used questionnaires used to assess the usability of a system. The original 10-item version of System Usability Scale was developed in English and thus needs to be adapted into local languages to assess the usability of a mobile apps developed in other languages. The aim of this study is to translate and validate (with cross-cultural adaptation) the English System Usability Scale questionnaire into Malay, the main language spoken in Malaysia. The development of a translated version will allow the usability of mobile apps to be assessed in Malay. Forward and backward translation of the questionnaire was conducted by groups of Malay native speakers who spoke English as their second language. The final version was obtained after reconciliation and cross-cultural adaptation. The content of the Malay System Usability Scale questionnaire for mobile apps was validated by 10 experts in mobile app development. The efficacy of the questionnaire was further probed by testing the face validity on 10 mobile phone users, followed by reliability testing involving 54 mobile phone users. The content validity index was determined to be 0.91, indicating good relevancy of the 10 items used to assess the usability of a mobile app. Calculation of the face validity index resulted in a value of 0.94, therefore indicating that the questionnaire was easily understood by the users. Reliability testing showed a Cronbach alpha value of .85 (95% CI 0.79-0.91) indicating that the translated System Usability Scale questionnaire is a reliable tool for the assessment of usability of a mobile app. The Malay System Usability Scale questionnaire is a

Examining the Use of Usability Results in a Software Development Company

DEFF Research Database (Denmark)

Høegh, Rune Thaarup; Stage, Jan

2004-01-01

This paper presents the first results of a study on a usability evaluation for a Danish software development company. The use of the results from the usability evaluation is examined through interviews with two developers from the software company. It is through an interview with a project leader...... from the company found that the traditional usability report plays a very little role for the development team. Initial results suggest that textual feedback proves more valuable when accompanied with video and oral feedback.......This paper presents the first results of a study on a usability evaluation for a Danish software development company. The use of the results from the usability evaluation is examined through interviews with two developers from the software company. It is through an interview with a project leader...

Usability/Sentiment for the Enterprise and ENTERPRISE

Science.gov (United States)

Meza, David; Berndt, Sarah

2014-01-01

The purpose of the Sentiment of Search Study for NASA Johnson Space Center (JSC) is to gain insight into the intranet search environment. With an initial usability survey, the authors were able to determine a usability score based on the Systems Usability Scale (SUS). Created in 1986, the freely available, well cited, SUS is commonly used to determine user perceptions of a system (in this case the intranet search environment). As with any improvement initiative, one must first examine and document the current reality of the situation. In this scenario, a method was needed to determine the usability of a search interface in addition to the user's perception on how well the search system was providing results. The use of the SUS provided a mechanism to quickly ascertain information in both areas, by adding one additional open-ended question at the end. The first ten questions allowed us to examine the usability of the system, while the last questions informed us on how the users rated the performance of the search results. The final analysis provides us with a better understanding of the current situation and areas to focus on for improvement. The power of search applications to enhance knowledge transfer is indisputable. The performance impact for any user unable to find needed information undermines project lifecycle, resource and scheduling requirements. Ever-increasing complexity of content and the user interface make usability considerations for the intranet, especially for search, a necessity instead of a 'nice-to-have'. Despite these arguments, intranet usability is largely disregarded due to lack of attention beyond the functionality of the infrastructure (White, 2013). The data collected from users of the JSC search system revealed their overall sentiment by means of the widely-known System Usability Scale. Results of the scores suggest 75%, +/-0.04, of the population rank the search system below average. In terms of a grading scaled, this equated to D or

Empirical studies on usability of mHealth apps: a systematic literature review.

Science.gov (United States)

Zapata, Belén Cruz; Fernández-Alemán, José Luis; Idri, Ali; Toval, Ambrosio

2015-02-01

The release of smartphones and tablets, which offer more advanced communication and computing capabilities, has led to the strong emergence of mHealth on the market. mHealth systems are being used to improve patients' lives and their health, in addition to facilitating communication between doctors and patients. Researchers are now proposing mHealth applications for many health conditions such as dementia, autism, dysarthria, Parkinson's disease, and so on. Usability becomes a key factor in the adoption of these applications, which are often used by people who have problems when using mobile devices and who have a limited experience of technology. The aim of this paper is to investigate the empirical usability evaluation processes described in a total of 22 selected studies related to mHealth applications by means of a Systematic Literature Review. Our results show that the empirical evaluation methods employed as regards usability could be improved by the adoption of automated mechanisms. The evaluation processes should also be revised to combine more than one method. This paper will help researchers and developers to create more usable applications. Our study demonstrates the importance of adapting health applications to users' need.

An approach to the efficient assessment of safety and usability of computer based control systems, VeNuS 2. Global final report

International Nuclear Information System (INIS)

Nelke, T.; Dlugosch, C.; Olaverri Monreal, C.; Sachse, K.; Thuering, M.

2015-01-01

Prior to the use of computer-based instrumentation and control the evidence of sufficient safety, development methods and the suitability of man-machine interface must be provided. For this purpose, validation methods must be available, if possible supported by appropriate tools. Based on the multitude of the data which has to be taken into account it is important to generate technical documentation, to realize efficient operation and to prevent human based errors. An approach for computer based generation of user manuals for the operation of technical systems was developed in the VeNuS 2 project. A second goal was to develop an approach to evaluate the usability of safety relevant digital human-machine-interfaces (e.g. for nuclear industries). Therefore a software tool has been developed to assess aspects of usability of user interfaces by considering safety-related priorities. Additionally new or well known methods for provision of evidence of sufficient safety and usability for computer based systems shall be developed in a prototyped way.

Qualitative Marketing Research Through Usability Testing

Directory of Open Access Journals (Sweden)

Orzan Mihai

2008-04-01

Full Text Available Usability is an attribute of any good product, just as its functionality. It refers mainly to the utility of a product for its indented users, as well as to its ease of use. And whilst a correct functionality is critical for the commercial success of any product, its value comes through the human needs that it fulfills, which is determined through various marketing research techniques. In parallel, the IT&C community has developed in the last two decades its own type of research, called usability testing, used mainly to evaluate interface ease of use and all usability problems associated with.software products. This article aims at finding the right place for usability testing and usability professionals in the marketing community, as well as drawing a wider picture, from a marketing research perspective, on one of the most popular topics in IT&C community for the benefit of marketing scholars and professionals.

Exploring the value of usability feedback formats

DEFF Research Database (Denmark)

Nørgaard, Mie; Hornbæk, Kasper Anders Søren

2009-01-01

The format used to present feedback from usability evaluations to developers affects whether problems are understood, accepted, and fixed. Yet, little research has investigated which formats are the most effective. We describe an explorative study where three developers assess 40 usability findings...... presented using five feedback formats. Our usability findings comprise 35 problems and 5 positive comments. Data suggest that feedback serves multiple purposes. Initially, feedback must convince developers about the relevance of a problem and convey an understanding of this. Feedback must next be easy...... working with the feedback to address the usability problems, there were no significant differences among the developers' ratings of the value of the different formats. This suggests that all of the formats may serve equally well as reminders in later stages of working with usability problems...

Integrating discount usability in scrum development process in Ethiopia

DEFF Research Database (Denmark)

Teka, Degif; Dittrich, Y.; Kifle, Mesfin

2017-01-01

be adapted and integrated into the Scrum-agile development with especial emphasis on the Ethiopian context. The research aims at adapting software engineering and ICT development methods to the specific situation and integrating user-centered design (UCD) and lightweight usability methods into agile...... end users and developers. Culturally adapted user pair testing and heuristic evaluation supported usability testing and supported developers in getting early feedback. Integrated approach of discount usability with the Scrum process has been developed and evaluated first with the involved...

Cost Effective Development of Usable Systems: Gaps between HCI and Software Architecture Design

Science.gov (United States)

Folmer, Eelke; Bosch, Jan

A software product with poor usability is likely to fail in a highly competitive market; therefore software developing organizations are paying more and more attention to ensuring the usability of their software. Practice, however, shows that product quality (which includes usability among others) is not that high as it could be. Studies of software projects (Pressman, 2001) reveal that organizations spend a relative large amount of money and effort on fixing usability problems during late stage development. Some of these problems could have been detected and fixed much earlier. This avoidable rework leads to high costs and because during development different tradeoffs have to be made, for example between cost and quality leads to systems with less than optimal usability. This problem has been around for a couple of decades especially after software engineering (SE) and human computer interaction (HCI) became disciplines on their own. While both disciplines developed themselves, several gaps appeared which are now receiving increased attention in research literature. Major gaps of understanding, both between suggested practice and how software is actually developed in industry, but also between the best practices of each of the fields have been identified (Carrol et al, 1994, Bass et al, 2001, Folmer and Bosch, 2002). In addition, there are gaps in the fields of differing terminology, concepts, education, and methods.

Human Factors for Nursing: From In-Situ Testing to Mobile Usability Engineering.

Science.gov (United States)

Kushniruk, Andre W; Borycki, Elizabeth M; Solvoll, Terje; Hullin, Carola

2016-01-01

The tutorial goal is to familiarize participants with human aspects of health informatics and human-centered approaches to the design, evaluation and deployment of both usable and safe healthcare information systems. The focus will be on demonstrating and teaching practical and low-cost methods for evaluating mobile applications in nursing. Basic background to testing methods will be provided, followed by live demonstration of the methods. Then the audience will break into small groups to explore the application of the methods to applications of interest (there will be a number of possible applications that will be available for applications in areas such as electronic health records and decision support, however, if the groups have applications of specific interest to them that will be possible). The challenges of conducting usability testing, and in particular mobile usability testing will be discussed along with practical solutions. The target audience includes practicing nurses and nurse researchers, nursing informatics specialists, nursing students, nursing managers and health informatics professionals interested in improving the usability and safety of healthcare applications.

Usability testing of an mHealth device for swallowing therapy in head and neck cancer survivors.

Science.gov (United States)

Constantinescu, Gabriela; Kuffel, Kristina; King, Ben; Hodgetts, William; Rieger, Jana

2018-04-01

The objective of this study was to conduct the first patient usability testing of a mobile health (mHealth) system for in-home swallowing therapy. Five participants with a history of head and neck cancer evaluated the mHealth system. After completing an in-application (app) tutorial with the clinician, participants were asked to independently complete five tasks: pair the device to the smartphone, place the device correctly, exercise, interpret progress displays, and close the system. Quantitative and qualitative methods were used to evaluate the effectiveness, efficiency, and satisfaction with the system. Critical changes to the app were found in three of the tasks, resulting in recommendations for the next iteration. These issues were related to ease of Bluetooth pairing, placement of device, and interpretation of statistics. Usability testing with patients identified issues that were essential to address prior to implementing the mHealth system in subsequent clinical trials. Of the usability methods used, video observation (synced screen capture with videoed gestures) revealed the most information.

Optimizing the user interface of a data entry module for an electronic patient record for cardiac rehabilitation: A mixed method usability approach.

Science.gov (United States)

van Engen-Verheul, Mariëtte M; Peute, Linda W P; de Keizer, Nicolette F; Peek, Niels; Jaspers, Monique W M

2016-03-01

Cumbersome electronic patient record (EPR) interfaces may complicate data-entry in clinical practice. Completeness of data entered in the EPR determines, among other things, the value of computerized clinical decision support (CCDS). Quantitative usability evaluations can provide insight into mismatches between the system design model of data entry and users' data entry behavior, but not into the underlying causes for these mismatches. Mixed method usability evaluation studies may provide these insights, and thus support generating redesign recommendations for improving an EPR system's data entry interface. To improve the usability of the data entry interface of an EPR system with CCDS in the field of cardiac rehabilitation (CR), and additionally, to assess the value of a mixed method usability approach in this context. Seven CR professionals performed a think-aloud usability evaluation both before (beta-version) and after the redesign of the system. Observed usability problems from both evaluations were analyzed and categorized using Zhang et al.'s heuristic principles of good interface design. We combined the think-aloud usability evaluation of the system's beta-version with the measurement of a new usability construct: users' deviations in action sequence from the system's predefined data entry order sequence. Recommendations for redesign were implemented. We assessed whether the redesign improved CR professionals' (1) task efficacy (with respect to the completeness of data they collected), and (2) task efficiency (with respect to the average number of mouse clicks they needed to complete data entry subtasks). With the system's beta version, 40% of health care professionals' navigation actions through the system deviated from the predefined next system action. The causes for these deviations as revealed by the think-aloud method mostly concerned mismatches between the system design model for data entry action sequences and users expectations of these action

Methods uncovering usability issues in medication-related alerting functions: results from a systematic review.

Science.gov (United States)

Marcilly, Romaric; Vasseur, Francis; Ammenwerth, Elske; Beuscart-Zephir, Marie-Catherine

2014-01-01

This paper aims at listing the methods used to evaluate the usability of medication-related alerting functions and at knowing what type of usability issues those methods allow to detect. A sub-analysis of data from this systematic review has been performed. Methods applied in the included papers were collected. Then, included papers were sorted in four types of evaluation: "expert evaluation", "user- testing/simulation", "on site observation" and "impact studies". The types of usability issues (usability flaws, usage problems and negative outcomes) uncovered by those evaluations were analyzed. Results show that a large set of methods are used. The largest proportion of papers uses "on site observation" evaluation. This is the only evaluation type for which every kind of usability flaws, usage problems and outcomes are detected. It is somehow surprising that, in a usability systematic review, most of the papers included use a method that is not often presented as a usability method. Results are discussed about the opportunity to provide usability information collected after the implementation of the technology during their design process, i.e. before their implementation.

Web usability testing with a Hispanic medically underserved population.

Science.gov (United States)

Moore, Mary; Bias, Randolph G; Prentice, Katherine; Fletcher, Robin; Vaughn, Terry

2009-04-01

Skilled website developers value usability testing to assure user needs are met. When the target audience differs substantially from the developers, it becomes essential to tailor both design and evaluation methods. In this study, researchers carried out a multifaceted usability evaluation of a website (Healthy Texas) designed for Hispanic audiences with lower computer literacy and lower health literacy. METHODS INCLUDED: (1) heuristic evaluation by a usability engineer, (2) remote end-user testing using WebEx software; and (3) face-to-face testing in a community center where use of the website was likely. Researchers found standard usability testing methods needed to be modified to provide interpreters, increased flexibility for time on task, presence of a trusted intermediary such as a librarian, and accommodation for family members who accompanied participants. Participants offered recommendations for website redesign, including simplified language, engaging and relevant graphics, culturally relevant examples, and clear navigation. User-centered design is especially important when website developers are not representative of the target audience. Failure to conduct appropriate usability testing with a representative audience can substantially reduce use and value of the website. This thorough course of usability testing identified improvements that benefit all users but become crucial when trying to reach an underserved audience.

Usability in a cultural context

DEFF Research Database (Denmark)

Clemmensen, Torkil; Yammiyavar, Pradeep; Ørngreen, Rikke

2010-01-01

This paper focuses on presenting and discussing the aim, context, challenges, results, and impact of the Cultural usability project named as CultUsab. This project was a four year international research effort from 2006 to 2009, which was supported by a grant for the Danish Research Councils...... for Independent Research in Culture and Communication. The project aimed at innovating processes in Information and Communication Technology development through an understanding of culturally sensitive aspects of usability evaluation methods....

Usability Testing of a National Substance Use Screening Tool Embedded in Electronic Health Records.

Science.gov (United States)

Press, Anne; DeStio, Catherine; McCullagh, Lauren; Kapoor, Sandeep; Morley, Jeanne; Conigliaro, Joseph

2016-07-08

Screening, brief intervention, and referral to treatment (SBIRT) is currently being implemented into health systems nationally via paper and electronic methods. The purpose of this study was to evaluate the integration of an electronic SBIRT tool into an existing paper-based SBIRT clinical workflow in a patient-centered medical home. Usability testing was conducted in an academic ambulatory clinic. Two rounds of usability testing were done with medical office assistants (MOAs) using a paper and electronic version of the SBIRT tool, with two and four participants, respectively. Qualitative and quantitative data was analyzed to determine the impact of both tools on clinical workflow. A second round of usability testing was done with the revised electronic version and compared with the first version. Personal workflow barriers cited in the first round of testing were that the electronic health record (EHR) tool was disruptive to patient's visits. In Round 2 of testing, MOAs reported favoring the electronic version due to improved layout and the inclusion of an alert system embedded in the EHR. For example, using the system usability scale (SUS), MOAs reported a grade "1" for the statement, "I would like to use this system frequently" during the first round of testing but a "5" during the second round of analysis. The importance of testing usability of various mediums of tools used in health care screening is highlighted by the findings of this study. In the first round of testing, the electronic tool was reported as less user friendly, being difficult to navigate, and time consuming. Many issues faced in the first generation of the tool were improved in the second generation after usability was evaluated. This study demonstrates how usability testing of an electronic SBRIT tool can help to identify challenges that can impact clinical workflow. However, a limitation of this study was the small sample size of MOAs that participated. The results may have been biased to

Usability Testing: Too Early? Too Much Talking? Too Many Problems?

DEFF Research Database (Denmark)

Hertzum, Morten

2016-01-01

Usability testing has evolved in response to a search for tests that are cheap, early, easy, and fast. In addition, it accords with a situational definition of usability, such as the one propounded by ISO. By approaching usability from an organizational perspective, this author argues that usabil......Usability testing has evolved in response to a search for tests that are cheap, early, easy, and fast. In addition, it accords with a situational definition of usability, such as the one propounded by ISO. By approaching usability from an organizational perspective, this author argues...... that usability should (also) be evaluated late, that usability professionals should be wary of thinking aloud, and that they should focus more on effects achievement than problem detection....

Usability and Accessibility of Air Force Intranet Web Sites

National Research Council Canada - National Science Library

Bentley, Richard S

2006-01-01

.... The Technology Acceptance Model suggests that information systems must not only be useful but also be usable and a large body of usability engineering knowledge exists to support usable design. In addition, the U.S...

On the Concepts of Usability and Reusability of Learning Objects

Directory of Open Access Journals (Sweden)

Miguel-Angel Sicilia

2003-10-01

Full Text Available “Reusable learning objects” oriented towards increasing their potential reusability are required to satisfy concerns about their granularity and their independence of concrete contexts of use. Such requirements also entail that the definition of learning object “usability,” and the techniques required to carry out their “usability evaluation” must be substantially different from those commonly used to characterize and evaluate the usability of conventional educational applications. In this article, a specific characterization of the concept of learning object usability is discussed, which places emphasis on “reusability,” the key property of learning objects residing in repositories. The concept of learning object reusability is described as the possibility and adequacy for the object to be usable in prospective educational settings, so that usability and reusability are considered two interrelated – and in many cases conflicting – properties of learning objects. Following the proposed characterization of two characteristics or properties of learning objects, a method to evaluate usability of specific learning objects will be presented.

Global usability

CERN Document Server

Douglas, Ian

2011-01-01

The concept of usability has become an increasingly important consideration in the design of all kinds of technology. As more products are aimed at global markets and developed through internationally distributed teams, usability design needs to be addressed in global terms. Interest in usability as a design issue and specialist area of research and education has developed steadily in North America and Europe since the 1980's. However, it is only over the last ten years that it has emerged as a global concern. Global Usability provides an introduction to the important issues in globalizing des

Evaluating the Acceptability and Usability of EASEL: A Mobile Application that Supports Guided Reflection for Experiential Learning Activities

Directory of Open Access Journals (Sweden)

Jerry C Schnepp

2017-07-01

Full Text Available Aim/Purpose: To examine the early perceptions (acceptability and usability of EASEL (Education through Application-Supported Experiential Learning, a mobile platform that delivers reflection prompts and content before, during, and after an experiential learning activity. Background: Experiential learning is an active learning approach in which students learn by doing and by reflecting on the experience. This approach to teaching is often used in disciplines such as humanities, business, and medicine. Reflection before, during, and after an experience allows the student to analyze what they learn and why it is important, which is vital in helping them to understand the relevance of the experience. A just-in-time tool (EASEL was needed to facilitate this. Methodology: To inform the development of a mobile application that facilitates real-time guided reflection and to determine the relevant feature set, we conducted a needs analysis with both students and faculty members. Data collected during this stage of the evaluation helped guide the creation of a prototype. The user experience of the prototype and interface interactions were evaluated during the usability phase of the evaluation study. Contribution: Both the needs analysis and usability assessment provided justification for continued development of EASEL as well as insight that guides current development. Findings: The interaction design of EASEL is understandable and usable. Both students and teachers value an application that facilitates real-time guided reflection. Recommendations for Practitioners: The use of a system such as EASEL can leverage time and location-based services to support students in field experiences. This technology aligns with evidence that guided reflection provides opportunities for metacognition. Recommendation for Researchers: Iterative prototyping, testing, and refinement can lead to a deliberate and effective app development process. Impact on Society: The EASEL

Evaluating Nurses' Satisfaction With Two Nursing Information Systems.

Science.gov (United States)

Khajouei, Reza; Abbasi, Reza

2017-06-01

Evaluating user satisfaction is one of the methods to ensure the usability of information systems. Considering the importance of nursing information systems in patient health, the objective of this study is to evaluate nurses' satisfaction with two widely used nursing information systems (Peyvand Dadeh and Tirazhe) in Iran. This descriptive-analytical study was done on 230 nurses in all teaching hospitals of Kerman University of Medical Sciences in 2015. Data were collected using an augmented version of a questionnaire developed by IBM. Data were analyzed by SPSS.16 using descriptive and analytical statistical methods including t test, analysis of variance, and Pearson correlation coefficient. The mean of overall satisfaction with the two systems was 61 ± 2.2 and 74 ± 2.4, respectively. The mean of satisfaction with different systems dimensions, that is, ease of use, information quality, and interface quality, was, respectively, 24 ± 1.9, 26 ± 9.7, and 12 ± 4.7 for Tirazhe and 29 ± 1.1, 39 ± 1.04 and 13 ± 5.3 for Peyvand Dadeh system. Nurses' satisfaction with both systems was at a medium level. The majority of nurses were relatively satisfied with the information quality and user interface quality of these systems. The results suggest that designing nursing information systems in accordance with their users' need improves usability. Hence, policy and decision makers of healthcare institutions should invest on usability when purchasing such systems.

Usability Methods for Ensuring Health Information Technology Safety: Evidence-Based Approaches. Contribution of the IMIA Working Group Health Informatics for Patient Safety.

Science.gov (United States)

Borycki, E; Kushniruk, A; Nohr, C; Takeda, H; Kuwata, S; Carvalho, C; Bainbridge, M; Kannry, J

2013-01-01

Issues related to lack of system usability and potential safety hazards continue to be reported in the health information technology (HIT) literature. Usability engineering methods are increasingly used to ensure improved system usability and they are also beginning to be applied more widely for ensuring the safety of HIT applications. These methods are being used in the design and implementation of many HIT systems. In this paper we describe evidence-based approaches to applying usability engineering methods. A multi-phased approach to ensuring system usability and safety in healthcare is described. Usability inspection methods are first described including the development of evidence-based safety heuristics for HIT. Laboratory-based usability testing is then conducted under artificial conditions to test if a system has any base level usability problems that need to be corrected. Usability problems that are detected are corrected and then a new phase is initiated where the system is tested under more realistic conditions using clinical simulations. This phase may involve testing the system with simulated patients. Finally, an additional phase may be conducted, involving a naturalistic study of system use under real-world clinical conditions. The methods described have been employed in the analysis of the usability and safety of a wide range of HIT applications, including electronic health record systems, decision support systems and consumer health applications. It has been found that at least usability inspection and usability testing should be applied prior to the widespread release of HIT. However, wherever possible, additional layers of testing involving clinical simulations and a naturalistic evaluation will likely detect usability and safety issues that may not otherwise be detected prior to widespread system release. The framework presented in the paper can be applied in order to develop more usable and safer HIT, based on multiple layers of evidence.

Evaluating the usability of an interactive, bi-lingual, touchscreen-enabled breastfeeding educational programme: application of Nielson’s heuristics

OpenAIRE

Ashish Joshi; Douglas M Puricelli Perin; Chioma Amadi; Kate Trout

2015-01-01

Background: Usability challenges have to be met in an interactive computer program development and should meet all users’ needs. Objective: The study purpose was to conduct heuristic evaluation of an interactive, bilingual touch screen enabled breastfeeding educational program for Hispanic women living in rural settings.Methods: Two usability experts used Nielsen’s heuristics while reviewing the user interface in May 2013 using principles of Nielson’s Heuristics. Nielson’s heuristics are a se...

Usability and Applicability of Microfluidic Cell Culture Systems

DEFF Research Database (Denmark)

Hemmingsen, Mette

possibilities for, for example, precise control of the chemical environment, 3D cultures, controlled co-culture of different cell types or automated, individual control of up to 96 cell culture chambers in one integrated system. Despite the great new opportunities to perform novel experimental designs......Microfluidic cell culture has been a research area with great attention the last decade due to its potential to mimic the in vivo cellular environment more closely compared to what is possible by conventional cell culture methods. Many exciting and complex devices have been presented providing......, these devices still lack general implementation into biological research laboratories. In this project, the usability and applicability of microfluidic cell culture systems have been investigated. The tested systems display good properties regarding optics and compatibility with standard laboratory equipment...

Usability study of a vineyard teleoperated compost spreader.

Science.gov (United States)

Ferrari, Ester; Cavallo, Eugenio

2012-01-01

Teleoperation has been widely applied in modern industry because of a variety of advantages, such as providing replaceable surrogates for humans in dangerous or difficult working environments over long distances. In this paper, a usability evaluation study of a teleoperation system for a compost spreader robotic machine is presented. The machine has been designed for the application of compost in small and stepping parcels of hilly vineyards. Driving and working tasks can be controlled remotely by a portable piloting unit, reducing the risk for the operator in the event of machine overturning. Participants of the study were asked to perform a series of tasks and sub-tasks and to vocalize their thoughts while working with the machine. The tasks were designed to simulate typical user experience. Once all the tasks were accomplished each participant was asked to fill a questionnaire. The evaluation considered aspects such as learnability, ease of use, understandability, controllability, frustration, mental effort, distraction, clarity of presentation, perceived usefulness, temporal efficiency and machine aesthetic. Results show that usability evaluation helped detecting design deficiencies in the teleoperated compost spreader machine.

Evaluation of usability of Spanish academic library web portals using a cognitive-emotional heuristic model

Directory of Open Access Journals (Sweden)

Daniel Muñoz-Egido

2017-03-01

Full Text Available In this paper a cognitive-emotional usability analysis model based on heuristics was used to analyze eighteen academic library web portals. Starting with a review of existing literature on the subject, a set of seventy-one heuristic criteria grouped into six aspects was designed, to which the metrics developed by SIRIUS were applied. The result of the evaluation reveals that the average usability of websites of university libraries analyzed stands at 72.30 over 100 with a standard deviation of 5.57. The aspects with the lowest scores were Attention and Long Term Memory and Learning, which were therefore approached less satisfactorily from the user’s point of view.

Methods for Evaluating the Content, Usability, and Efficacy of Commercial Mobile Health Apps

Science.gov (United States)

Silfee, Valerie J; Waring, Molly E; Boudreaux, Edwin D; Sadasivam, Rajani S; Mullen, Sean P; Carey, Jennifer L; Hayes, Rashelle B; Ding, Eric Y; Bennett, Gary G; Pagoto, Sherry L

2017-01-01

Commercial mobile apps for health behavior change are flourishing in the marketplace, but little evidence exists to support their use. This paper summarizes methods for evaluating the content, usability, and efficacy of commercially available health apps. Content analyses can be used to compare app features with clinical guidelines, evidence-based protocols, and behavior change techniques. Usability testing can establish how well an app functions and serves its intended purpose for a target population. Observational studies can explore the association between use and clinical and behavioral outcomes. Finally, efficacy testing can establish whether a commercial app impacts an outcome of interest via a variety of study designs, including randomized trials, multiphase optimization studies, and N-of-1 studies. Evidence in all these forms would increase adoption of commercial apps in clinical practice, inform the development of the next generation of apps, and ultimately increase the impact of commercial apps. PMID:29254914

Measurement properties and usability of non-contact scanners for measuring transtibial residual limb volume.

Science.gov (United States)

Kofman, Rianne; Beekman, Anna M; Emmelot, Cornelis H; Geertzen, Jan H B; Dijkstra, Pieter U

2018-06-01

Non-contact scanners may have potential for measurement of residual limb volume. Different non-contact scanners have been introduced during the last decades. Reliability and usability (practicality and user friendliness) should be assessed before introducing these systems in clinical practice. The aim of this study was to analyze the measurement properties and usability of four non-contact scanners (TT Design, Omega Scanner, BioSculptor Bioscanner, and Rodin4D Scanner). Quasi experimental. Nine (geometric and residual limb) models were measured on two occasions, each consisting of two sessions, thus in total 4 sessions. In each session, four observers used the four systems for volume measurement. Mean for each model, repeatability coefficients for each system, variance components, and their two-way interactions of measurement conditions were calculated. User satisfaction was evaluated with the Post-Study System Usability Questionnaire. Systematic differences between the systems were found in volume measurements. Most of the variances were explained by the model (97%), while error variance was 3%. Measurement system and the interaction between system and model explained 44% of the error variance. Repeatability coefficient of the systems ranged from 0.101 (Omega Scanner) to 0.131 L (Rodin4D). Differences in Post-Study System Usability Questionnaire scores between the systems were small and not significant. The systems were reliable in determining residual limb volume. Measurement systems and the interaction between system and residual limb model explained most of the error variances. The differences in repeatability coefficient and usability between the four CAD/CAM systems were small. Clinical relevance If accurate measurements of residual limb volume are required (in case of research), modern non-contact scanners should be taken in consideration nowadays.

A Usability Study of Users’ Perceptions Toward a Multimedia Computer-Assisted Learning Tool for Neuroanatomy

Science.gov (United States)

Gould, Douglas J.; Terrell, Mark A.; Fleming, Jo

2015-01-01

This usability study evaluated users’ perceptions of a multimedia prototype for a new e-learning tool: Anatomy of the Central Nervous System: A Multimedia Course. Usability testing is a collection of formative evaluation methods that inform the developmental design of e-learning tools to maximize user acceptance, satisfaction, and adoption. Sixty-two study participants piloted the prototype and completed a usability questionnaire designed to measure two usability properties: program need and program applicability. Statistical analyses were used to test the hypothesis that the multimedia prototype was well designed and highly usable, it was perceived as: 1) highly needed across a spectrum of educational contexts, 2) highly applicable in supporting the pedagogical processes of teaching and learning neuroanatomy, and 3) was highly usable by all types of users. Three independent variables represented user differences: level of expertise (faculty vs. student), age, and gender. Analysis of the results supports the research hypotheses that the prototype was designed well for different types of users in various educational contexts and for supporting the pedagogy of neuroanatomy. In addition, the results suggest that the multimedia program will be most useful as a neuroanatomy review tool for health-professions students preparing for licensing or board exams. This study demonstrates the importance of integrating quality properties of usability with principles of human learning during the instructional design process for multimedia products. PMID:19177405

The STEP database through the end-users eyes--USABILITY STUDY.

Science.gov (United States)

Salunke, Smita; Tuleu, Catherine

2015-08-15

The user-designed database of Safety and Toxicity of Excipients for Paediatrics ("STEP") is created to address the shared need of drug development community to access the relevant information of excipients effortlessly. Usability testing was performed to validate if the database satisfies the need of the end-users. Evaluation framework was developed to assess the usability. The participants performed scenario based tasks and provided feedback and post-session usability ratings. Failure Mode Effect Analysis (FMEA) was performed to prioritize the problems and improvements to the STEP database design and functionalities. The study revealed several design vulnerabilities. Tasks such as limiting the results, running complex queries, location of data and registering to access the database were challenging. The three critical attributes identified to have impact on the usability of the STEP database included (1) content and presentation (2) the navigation and search features (3) potential end-users. Evaluation framework proved to be an effective method for evaluating database effectiveness and user satisfaction. This study provides strong initial support for the usability of the STEP database. Recommendations would be incorporated into the refinement of the database to improve its usability and increase user participation towards the advancement of the database. Copyright © 2015 Elsevier B.V. All rights reserved.

Usability Assessment of E-Café Operational Management Simulation Game

Science.gov (United States)

Chang, Chiung-sui; Huang, Ya-Ping

2013-01-01

To ensure the quality of digital simulation game, we utilized the usability evaluation heuristic in the design and development processes of e-café operational management game-based learning material for students. The application of usability evaluations during this study is described. Additionally, participant selection, data collection and…

Mapping Disease Data: A Usability Test of an Internet-Based System of Disease Status Disclosure

Directory of Open Access Journals (Sweden)

Gareth Enticott

2018-01-01

Full Text Available Disease maps are important tools in the management of disease. By communicating risk, disease maps can help raise awareness of disease and encourage farmers and veterinarians to employ best practice to eliminate the spread of disease. However, despite the importance of disease maps in communicating risk and the existence of various online disease maps, there are few studies that explicitly examine their usability. Where disease maps are complicated to use, it seems that they are unlikely to be used effectively. The paper outlines an attempt to create an open access, online, searchable map of incidents of bovine tuberculosis in England and Wales, and analyzes its usability among veterinarians. The paper describes the process of creating the map before describing the results of a series of usability trials. Results show the map to score highly on different measures of usability. However, the trials also revealed a number of social and technical limitations and challenges facing the use of online disease maps, including reputational dangers, role confusion, data accuracy, and data representation. The paper considers the challenges facing disease maps and their potential role in designing new methodologies to evaluate the effectiveness of disease prevention initiatives.

Usability testing of a mobile robotic system for in-home telerehabilitation.

Science.gov (United States)

Boissy, Patrick; Brière, Simon; Corriveau, Hélène; Grant, Andrew; Lauria, Michel; Michaud, François

2011-01-01

Mobile robots designed to enhance telepresence in the support of telehealth services are being considered for numerous applications. TELEROBOT is a teleoperated mobile robotic platform equipped with videoconferencingcapabilities and designed to be used in a home environment to. In this study, learnability of the system's teleoperation interface and controls was evaluated with ten rehabilitation professionals during four training sessions in a laboratory environment and in an unknown home environment while performing the execution of a standardized evaluation protocol typically used in home care. Results show that the novice teleoperators' performances on two of the four metrics used (number of command and total time) improved significantly across training sessions (ANOVAS, phome environment during navigation tasks (r=0,77 and 0,60). With only 4 hours of training, rehabilitation professionals were able learn to teleoperate successfully TELEROBOT. However teleoperation performances remained significantly less efficient then those of an expert. Under the home task condition (navigating the home environment from one point to the other as fast as possible) this translated to completion time between 350 seconds (best performance) and 850 seconds (worse performance). Improvements in other usability aspects of the system will be needed to meet the requirements of in-home telerehabilitation.

A Textual Feedback Tool for Empowering Participants in Usability and UX Evaluations

DEFF Research Database (Denmark)

Sivaji, Ashok; Clemmensen, Torkil; Nielsen, Søren Feodor

2017-01-01

-educated, but low-status, users in UX evaluations in countries and contexts with high power distances. The proposed tool contributes to the HCI community’s pool of localized UX evaluation tools. We evaluate the tool with 40 users from two socio-economic groups in real-life UX usability evaluations setting...... in Malaysia. The results indicate that the Textual Feedback tool may help participants to give their thoughts in UX evaluation in high power distance contexts. In particular, the Textual Feedback tool helps high status females and low status males express more UX problems than they can do with traditional CTA...... alone. We found that classic concurrent think aloud UX evaluation works fine in high power contexts, but only with the addition of Textual feedback to mitigate the effects of socio-economic status in certain user groups. We suggest that future research on UX evaluation look more into how to empower...

Usability and Workflow Evaluation of “RhEumAtic Disease Activity” (READY)

Science.gov (United States)

Yen, Po-Yin; Lara, Barbara; Lopetegui, Marcelo; Bharat, Aseem; Ardoin, Stacy; Johnson, Bernadette; Mathur, Puneet; Embi, Peter J.

2016-01-01

Summary Background RhEumAtic Disease activitY (READY) is a mobile health (mHealth) application that aims to create a shared platform integrating data from both patients and physicians, with a particular emphasis on arthritis disease activity. Methods We made READY available on an iPad and pilot implemented it at a rheumatology out-patient clinic. We conducted 1) a usability evaluation study to explore patients’ and physicians’ interactions with READY, and 2) a time motion study (TMS) to observe the clinical workflow before and after the implementation. Results A total of 33 patients and 15 physicians participated in the usability evaluation. We found usability problems in navigation, data entry, pain assessment, documentation, and instructions along with error messages. Despite these issues, 25 (75,76%) patients reported they liked READY. Physicians provided mixed feedback because they were concerned about the impact of READY on clinical workflow. Six physicians participated in the TMS. We observed 47 patient visits (44.72 hours) in the pre-implementation phase, and 42 patient visits (37.82 hours) in the post-implementation phase. We found that patients spent more time on READY than paper (4.39mins vs. 2.26mins), but overall, READY did not delay the workflow (pre = 52.08 mins vs. post = 45.46 mins). This time difference may be compensated with READY eliminating a workflow step for the staff. Conclusion Patients preferred READY to paper documents. Many found it easier to input information because of the larger font size and the ease of ‘tapping’ rather than writing-out or circling answers. Even though patients spent more time on READY than using paper documents, the longer usage of READY was mainly due to when troubleshooting was needed. Most patients did not have problems after receiving initial support from the staff. This study not only enabled improvements to the software but also serves as good reference for other researchers or institutional decision

The value of Retrospective and Concurrent Think Aloud in formative usability testing of a physician data query tool.

Science.gov (United States)

Peute, Linda W P; de Keizer, Nicolette F; Jaspers, Monique W M

2015-06-01

To compare the performance of the Concurrent (CTA) and Retrospective (RTA) Think Aloud method and to assess their value in a formative usability evaluation of an Intensive Care Registry-physician data query tool designed to support ICU quality improvement processes. Sixteen representative intensive care physicians participated in the usability evaluation study. Subjects were allocated to either the CTA or RTA method by a matched randomized design. Each subject performed six usability-testing tasks of varying complexity in the query tool in a real-working context. Methods were compared with regard to number and type of problems detected. Verbal protocols of CTA and RTA were analyzed in depth to assess differences in verbal output. Standardized measures were applied to assess thoroughness in usability problem detection weighted per problem severity level and method overall effectiveness in detecting usability problems with regard to the time subjects spent per method. The usability evaluation of the data query tool revealed a total of 43 unique usability problems that the intensive care physicians encountered. CTA detected unique usability problems with regard to graphics/symbols, navigation issues, error messages, and the organization of information on the query tool's screens. RTA detected unique issues concerning system match with subjects' language and applied terminology. The in-depth verbal protocol analysis of CTA provided information on intensive care physicians' query design strategies. Overall, CTA performed significantly better than RTA in detecting usability problems. CTA usability problem detection effectiveness was 0.80 vs. 0.62 (pusability problems of a moderate (0.85 vs. 0.7) and severe nature (0.71 vs. 0.57). In this study, the CTA is more effective in usability-problem detection and provided clarification of intensive care physician query design strategies to inform redesign of the query tool. However, CTA does not outperform RTA. The RTA

Effect of improving the usability of an e-learning resource: a randomized trial.

Science.gov (United States)

Davids, Mogamat Razeen; Chikte, Usuf M E; Halperin, Mitchell L

2014-06-01

Optimizing the usability of e-learning materials is necessary to reduce extraneous cognitive load and maximize their potential educational impact. However, this is often neglected, especially when time and other resources are limited. We conducted a randomized trial to investigate whether a usability evaluation of our multimedia e-learning resource, followed by fixing of all problems identified, would translate into improvements in usability parameters and learning by medical residents. Two iterations of our e-learning resource [version 1 (V1) and version 2 (V2)] were compared. V1 was the first fully functional version and V2 was the revised version after all identified usability problems were addressed. Residents in internal medicine and anesthesiology were randomly assigned to one of the versions. Usability was evaluated by having participants complete a user satisfaction questionnaire and by recording and analyzing their interactions with the application. The effect on learning was assessed by questions designed to test the retention and transfer of knowledge. Participants reported high levels of satisfaction with both versions, with good ratings on the System Usability Scale and adjective rating scale. In contrast, analysis of video recordings revealed significant differences in the occurrence of serious usability problems between the two versions, in particular in the interactive HandsOn case with its treatment simulation, where there was a median of five serious problem instances (range: 0-50) recorded per participant for V1 and zero instances (range: 0-1) for V2 (P e-learning resource resulted in significant improvements in usability. This is likely to translate into improved motivation and willingness to engage with the learning material. In this population of relatively high-knowledge participants, learning scores were similar across the two versions. Copyright © 2014 The American Physiological Society.

Usability testing for the rest of us: the application of discount usability principles in the development of an online communications assessment application.

Science.gov (United States)

Brock, Douglas; Kim, Sara; Palmer, Odawni; Gallagher, Thomas; Holmboe, Eric

2013-01-01

Usability evaluation provides developers and educators with the means to understand user needs, improve overall product utility, and increase user satisfaction. The application of "discount usability" principles developed to make usability testing more practical and useful may improve user experience at minimal cost and require little existing expertise to conduct. We describe an application of discount usability to a high-fidelity online communications assessment application developed by the University of Washington for the American Board of Internal Medicine. Eight internal medicine physicians completed a discount usability test. Sessions were recorded and the videos analyzed for significant usability concerns. Concerns were identified, summarized, discussed, and prioritized by the authors in collaboration with the software developers before implementing any changes to the interface. Thirty-eight significant usability issues were detected and four technical problems were identified. Each issue was responded to through modification of the software, by providing additional instruction, or delayed for a later version to be developed. Discount usability can be easily implemented in academic developmental activities. Our study resulted in the discovery and remediation of significant user problems, in addition to giving important insight into the novel methods built into the application.

Usability Testing of an Academic Library Web Site: A Case Study.

Science.gov (United States)

Battleson, Brenda; Booth, Austin; Weintrop, Jane

2001-01-01

Discusses usability testing as a tool for evaluating the effectiveness and ease of use of academic library Web sites; considers human-computer interaction; reviews major usability principles; and explores the application of formal usability testing to an existing site at the University at Buffalo (NY) libraries. (Author/LRW)

Apathy towards the Integration of Usability Work

DEFF Research Database (Denmark)

Wale-Kolade, Adeola; Nielsen, Peter Axel

2016-01-01

In this article we report from a case study of a software development organization and we study in particular the developers’ and product managers’ attitudes towards integrating usability work into software development. We offer explanations based on system justification theory illuminating what...... would-be integrators might be up against. The analysis shows how the developers only pay lip service to usability work and how they treat users superficially. It further shows how that leads to stereotyping of usability designers and users in order to preserve status quo, and how internalization...... of inequality between the developers and usability designers rationalizes the preservation of status quo. These findings will have significant implications for managers of software development....

How can usability measurement affect the re-engineering process of clinical software procedures?

Science.gov (United States)

Terazzi, A; Giordano, A; Minuco, G

1998-01-01

As a consequence of the dramatic improvements achieved in information technology standards in terms of single hardware and software components, efforts in the evaluation processes have been focused on the assessment of critical human factors, such as work-flow organisation, man-machine interaction and, in general, quality of use, or usability. This trend is particularly valid when applied to medical informatics, since the human component is the basis of the information processing system in health care context. With the aim to establish an action-research project on the evaluation and assessment of clinical software procedures which constitute an integrated hospital information system, the authors adopted this strategy and considered the measurement of perceived usability as one of the main goals of the project itself: the paper reports the results of this experience.

Marketing through Usability.

Science.gov (United States)

Morgan, Eric Lease

1999-01-01

Suggests that one of the best ways to get word-of-mouth marketing of library technology is to provide "usable" products and services. Provides an international standards definition on usability and notes that understanding usability combines an understanding of user needs/wants with an understanding of goals, functions, and limitations…

Usability Assessment of the Missouri Cancer Registry’s Published Interactive Mapping Reports: Round One

Science.gov (United States)

Jackson-Thompson, Jeannette; Schmaltz, Chester Lee

2017-01-01

Background  Many users of spatial data have difficulty interpreting information in health-related spatial reports. The Missouri Cancer Registry and Research Center (MCR-ARC) has produced interactive reports for several years. These reports have never been tested for usability. Objective  The aims of this study were to: (1) conduct a multi-approach usability testing study to understand ease of use (user friendliness) and user satisfaction; and (2) evaluate the usability of MCR-ARC’s published InstantAtlas reports. Methods   An institutional review board (IRB) approved mixed methodology usability testing study using a convenience sample of health professionals. A recruiting email was sent to faculty in the Master of Public Health program and to faculty and staff in the Department of Health Management and Informatics at the University of Missouri-Columbia. The study included 7 participants. The test included a pretest questionnaire, a multi-task usability test, and the System Usability Scale (SUS). Also, the researchers collected participants’ comments about the tested maps immediately after every trial. Software was used to record the computer screen during the trial and the participants’ spoken comments. Several performance and usability metrics were measured to evaluate the usability of MCR-ARC’s published mapping reports. Results Of the 10 assigned tasks, 6 reached a 100% completion success rate, and this outcome was relative to the complexity of the tasks. The simple tasks were handled more efficiently than the complicated tasks. The SUS score ranged between 20-100 points, with an average of 62.7 points and a median of 50.5 points. The tested maps’ effectiveness outcomes were better than the efficiency and satisfaction outcomes. There was a statistically significant relationship between the subjects’ performance on the study test and the users’ previous experience with geographic information system (GIS) tools (P=.03). There were no

Usability Evaluation Methods for Special Interest Internet Information Services

Directory of Open Access Journals (Sweden)

Eva-Maria Schön

2014-06-01

Full Text Available The internet provides a wide range of scientific information for different areas of research, used by the related scientific communities. Often the design or architecture of these web pages does not correspond to the mental model of their users. As a result the wanted information is difficult to find. Methods established by Usability Engineering and User Experience can help to increase the appeal of scientific internet information services by analyzing the users’ requirements. This paper describes a procedure to analyze and optimize scientific internet information services that can be accomplished with relatively low effort. It consists of a combination of methods that already have been successfully applied to practice: Personas, usability inspections, Online Questionnaire, Kano model and Web Analytics.

Discovering the User: A Practical Glance at Usability Testing.

Science.gov (United States)

Campbell, Nicole; Walbridge, Sharon; Chisman, Janet; Diller, Karen R.

1999-01-01

This interview focuses on usability testing, one method for evaluating whether library Web sites are useful for users. Discussion includes: benefits and drawbacks of usability testing; why it is important; lessons learned from testing; advice for someone interested in doing a study; and recommendations. (AEF)

Sample size in usability studies

NARCIS (Netherlands)

Schmettow, Martin

2012-01-01

Usability studies are important for developing usable, enjoyable products, identifying design flaws (usability problems) likely to compromise the user experience. Usability testing is recommended for improving interactive design, but discovery of usability problems depends on the number of users

A Clinical Decision Support System for Chronic Pain Management in Primary Care: Usability testing and its relevance.

Science.gov (United States)

Nair, Kalpana Maria; Malaeekeh, Raheleh; Schabort, Inge; Taenzer, Paul; Radhakrishnan, Arun; Guenter, Dale

2015-08-13

Clinical decision support systems (CDSSs) that are integrated into electronic medical records may be useful for encouraging practice change compliant with clinical practice guidelines. To engage end users to inform early phase CDSS development through a process of usability testing. A sequential exploratory mixed method approach was used. Interprofessional clinician participants (seven in iteration 1 and six in iteration 2) were asked to 'think aloud' while performing various tasks on the CDSS and then complete the System Usability Scale (SUS). Changes were made to the CDSS after each iteration.Results Barriers and facilitators were identified: systemic; user interface (most numerous barriers); content (most numerous facilitators) and technical. The mean SUS score was 81.1 (SD = 12.02) in iteration 1 and 70.40 (SD = 6.78) in iteration 2 (p > 0.05). Qualitative data from usability testing were valuable in the CDSS development process. SUS scores were of limited value at this development stage.

Identifying Usability and Productivity Dimensions for Measuring the Success of Mobile Business Services

Directory of Open Access Journals (Sweden)

Maiju Vuolle

2008-01-01

Full Text Available This paper reviews existing measures used for evaluating the usability of information systems and those used for evaluating the level of the productivity of a company. We discuss the usefulness of the existing measures in the mobile business context, where both mobility and work-context pose specific demands for the mobile business services. The review showed that the existing measures rarely consider the great contextual variation caused by the mobility of the services and the demands this poses on usability; which, in turn, affects productivity. To build a measurement tool that better meets the requirements of mobile business services, we completed case studies on two mobile business services, one used in passenger transport and the other in construction sites. Based on the understanding gained from the case studies, we propose a list of dimensions and items addressing both usability and productivity aspects that work as the basis for a multidisciplinary measurement tool.

Mobile applications for diabetics: a systematic review and expert-based usability evaluation considering the special requirements of diabetes patients age 50 years or older.

Science.gov (United States)

Arnhold, Madlen; Quade, Mandy; Kirch, Wilhelm

2014-04-09

A multitude of mhealth (mobile health) apps have been developed in recent years to support effective self-management of patients with diabetes mellitus type 1 or 2. We carried out a systematic review of all currently available diabetes apps for the operating systems iOS and Android. We considered the number of newly released diabetes apps, range of functions, target user groups, languages, acquisition costs, user ratings, available interfaces, and the connection between acquisition costs and user ratings. Additionally, we examined whether the available applications serve the special needs of diabetes patients aged 50 or older by performing an expert-based usability evaluation. We identified relevant keywords, comparative categories, and their specifications. Subsequently, we performed the app review based on the information given in the Google Play Store, the Apple App Store, and the apps themselves. In addition, we carried out an expert-based usability evaluation based on a representative 10% sample of diabetes apps. In total, we analyzed 656 apps finding that 355 (54.1%) offered just one function and 348 (53.0%) provided a documentation function. The dominating app language was English (85.4%, 560/656), patients represented the main user group (96.0%, 630/656), and the analysis of the costs revealed a trend toward free apps (53.7%, 352/656). The median price of paid apps was €1.90. The average user rating was 3.6 stars (maximum 5). Our analyses indicated no clear differences in the user rating between free and paid apps. Only 30 (4.6%) of the 656 available diabetes apps offered an interface to a measurement device. We evaluated 66 apps within the usability evaluation. On average, apps were rated best regarding the criterion "comprehensibility" (4.0 out of 5.0), while showing a lack of "fault tolerance" (2.8 out of 5.0). Of the 66 apps, 48 (72.7%) offered the ability to read the screen content aloud. The number of functions was significantly negative correlated

Mobile Applications for Diabetics: A Systematic Review and Expert-Based Usability Evaluation Considering the Special Requirements of Diabetes Patients Age 50 Years or Older

Science.gov (United States)

Quade, Mandy; Kirch, Wilhelm

2014-01-01

Background A multitude of mhealth (mobile health) apps have been developed in recent years to support effective self-management of patients with diabetes mellitus type 1 or 2. Objective We carried out a systematic review of all currently available diabetes apps for the operating systems iOS and Android. We considered the number of newly released diabetes apps, range of functions, target user groups, languages, acquisition costs, user ratings, available interfaces, and the connection between acquisition costs and user ratings. Additionally, we examined whether the available applications serve the special needs of diabetes patients aged 50 or older by performing an expert-based usability evaluation. Methods We identified relevant keywords, comparative categories, and their specifications. Subsequently, we performed the app review based on the information given in the Google Play Store, the Apple App Store, and the apps themselves. In addition, we carried out an expert-based usability evaluation based on a representative 10% sample of diabetes apps. Results In total, we analyzed 656 apps finding that 355 (54.1%) offered just one function and 348 (53.0%) provided a documentation function. The dominating app language was English (85.4%, 560/656), patients represented the main user group (96.0%, 630/656), and the analysis of the costs revealed a trend toward free apps (53.7%, 352/656). The median price of paid apps was €1.90. The average user rating was 3.6 stars (maximum 5). Our analyses indicated no clear differences in the user rating between free and paid apps. Only 30 (4.6%) of the 656 available diabetes apps offered an interface to a measurement device. We evaluated 66 apps within the usability evaluation. On average, apps were rated best regarding the criterion “comprehensibility” (4.0 out of 5.0), while showing a lack of “fault tolerance” (2.8 out of 5.0). Of the 66 apps, 48 (72.7%) offered the ability to read the screen content aloud. The number of

Web-based education for low-literate parents in Neonatal Intensive Care Unit: development of a website and heuristic evaluation and usability testing.

Science.gov (United States)

Choi, Jeungok; Bakken, Suzanne

2010-08-01

Low health literacy has been associated with poor health-related outcomes. The purposes are to report the development of a website for low-literate parents in the Neonatal Intensive Care Unit (NICU), and the findings of heuristic evaluation and a usability testing of this website. To address low literacy of NICU parents, multimedia educational Website using visual aids (e.g., pictographs, photographs), voice-recorded text message in addition to a simplified text was developed. The text was created at the 5th grade readability level. The heuristic evaluation was conducted by three usability experts using 10 heuristics. End-users' performance was measured by counting the time spent completing tasks and number of errors, as well as recording users' perception of ease of use and usefulness (PEUU) in a sample of 10 NICU parents. Three evaluators identified 82 violations across the 10 heuristics. All violations, however, received scores visuals on the Website were well accepted by low-literate users and agreement of visuals with text improved understanding of the educational materials over that with text alone. The findings suggest that using concrete and realistic pictures and pictographs with clear captions would maximize the benefit of visuals. One emerging theme was "simplicity" in design (e.g., limited use of colors, one font type and size), content (e.g., avoid lengthy text), and technical features (e.g., limited use of pop-ups). The heuristic evaluation by usability experts and the usability test with actual users provided complementary expertise, which can give a richer assessment of a design for low literacy Website. These results facilitated design modification and implementation of solutions by categorizing and prioritizing the usability problems.

Software Usability: Concepts, Attributes and Associated Health Problems

Directory of Open Access Journals (Sweden)

Grīnberga Sabīne

2016-10-01

Full Text Available Digital technologies have opened a large set of opportunities for new electronic services (e-commerce, e-health, e-studies etc.. There are many considerations that need to be made when programmers are building new application software or system software. The software needs to be attractive enough that people want to look at it. It also needs to contain all necessary information that developers want to share with their readers (customers, users in order to help them achieve the objective for which they came to their website, use their software, or interact with their teaching packages. The oversupply of e-services products has created a need for usability research and development. “Usability means making products and systems easier to use, and matching them more closely to user needs and requirements”. Usability is a key concept of the human-computer interface and is concerned with making computer systems easy to learn and easy to use through a user-centered design process. The in-depth understanding of usability concepts and processes are critical for large-scale acceptance of new e-services and knowledge productivity. Poorly designed software can be extremely annoying to users. Smith and Mayes state that „usability is now recognised as a vital determining factor in the success of any new computer system or computer-based service”. Studies have shown that the main health problems of computer users are repetitive strain injuries, visual discomfort and stress-related disorders. Beside other risk factors, such as poor workstation design, uncomfortable work postures, long hours of computer use every day, stress, etc., also poor design and usability of the computer systems, as well as computer technical problems, add to the pressure felt by the user, which may in turn cause stress-related disorders.

An object-oriented mobile health system with usability features.

Science.gov (United States)

Escarfullet, Krystle; Moore, Cantera; Tucker, Shari; Wei, June

2012-01-01

Mobile health (m-health) comprises the concept of utilising mobile devices to carry out the task of viewing electronic medical records, reserving medical appointments with a patient's medical provider and electronically refilling prescriptions. This paper aims at developing a m-health system to improve usability from a user's perspective. Specifically, it first developed a m-health model by logically linking characteristics of the m-health system together based on information flows. Then, the system requirements were collected by using a developed questionnaire. These requirements were structured and further in-depth analysis was conducted by using an object-oriented approach based on unified modelling language, such as use-case, sequence and analysis class diagrams. This research will be beneficial to decision makers and developers in the mobile healthcare industry.

Usable Security and E-Banking: ease of use vis-a-vis security

Directory of Open Access Journals (Sweden)

Morten Hertzum

2004-05-01

Full Text Available Electronic banking must be secure and easy to use. An evaluation of six Danish web-based electronic banking systems indicates that the systems have serious weaknesses with respect to ease of use. Our analysis of the weaknesses suggests that security requirements are among their causes and that the weaknesses may in turn cause decreased security. We view the conflict between ease of use and security in the context of usable security, a concept that is intended to match security principles and demands against user knowledge and motivation. Automation, instruction, and understanding can be identified as different approaches to usable security. Instruction is the main approach of the systems evaluated; automation relieves the user from involvement in security, as far as possible; and understanding goes beyond step-by-step instructions, to enable users to act competently and safely in situations that transcend preconceived instructions. We discuss the pros and cons of automation and understanding as alternative approaches to the design of web-based e-banking systems.

Evaluation bibliothekarischer Online-Angebote: Usability und Usefulness – zwei Seiten einer Medaille

Directory of Open Access Journals (Sweden)

Thomas Weinhold

2013-11-01

Full Text Available Aus Anwendersicht sind für die Qualität bibliothekarischer Online-Angebote zwei Aspekte entscheidend. Einerseits müssen für sie relevante Inhalte zur Verfügung stehen. Andererseits sollte es mög­lichst einfach und intuitiv möglich sein, diese Inhalte zu finden und auf sie zuzugreifen. Die Themen Usefulness und Usability sind daher für Bibliotheken von grosser Bedeutung. Der Beitrag beschreibt wie diese beiden Aspekte evaluiert werden können und wie Bibliotheken anhand von BibEval und PECI selb­ständig ihre Angebote überprüfen können. From the users’ point of view two aspects are essential for the quality of library online services. On the one hand, relevant content must be available. On the other hand, it should be simple and intuitive to find and to access this content. Therefore, the topics usability and usefulness are of great importance for li­braries. This article describes how these two aspects can be evaluated and how libraries can use BibEval and PECI for assessments of their own services.

Exploring Multiple Usability Perspectives

DEFF Research Database (Denmark)

Uldall-Espersen, Tobias

2007-01-01

Industrial usability work often fails to produce the expected impact on software products even though significant resources have been used on uncovering problems and suggesting improvements. So, it seems that feedback from industrial usability work lacks persuasiveness, i.e. it fails to convince...... the key stakeholders that actions need to be taken. This study reports from interviews with 26 stakeholders in software development projects. Our data suggests that the interviewees address usability using different perspectives and based on our observations we describe five such perspectives. Further, we...... discuss how applying different usability perspectives might inform the persuasiveness of usability work....

Dimensions for hearing-impaired mobile application usability model

Science.gov (United States)

Nathan, Shelena Soosay; Hussain, Azham; Hashim, Nor Laily; Omar, Mohd Adan

2017-10-01

This paper discuss on the dimensions that has been derived for the hearing-impaired mobile applications usability model. General usability model consist of general dimension for evaluating mobile application however requirements for the hearing-impaired are overlooked and often scanted. This led towards mobile application developed for the hearing-impaired are left unused. It is also apparent that these usability models do not consider accessibility dimensions according to the requirement of the special users. This complicates the work of usability practitioners as well as academician that practices research usability when application are developed for the specific user needs. To overcome this issue, dimension chosen for the hearing-impaired are ensured to be align with the real need of the hearing-impaired mobile application. Besides literature studies, requirements for the hearing-impaired mobile application have been identified through interview conducted with hearing-impaired mobile application users that were recorded as video outputs and analyzed using Nvivo. Finally total of 6 out of 15 dimensions gathered are chosen for the proposed model and presented.

Formal usability evaluation of audio track widget graphical representation for two-dimensional stage audio mixing interface

OpenAIRE

Dewey, Christopher; Wakefield, Jonathan P.

2017-01-01

The two-dimensional stage paradigm (2DSP) has been suggested as an alternative audio mixing interface (AMI). This study seeks to refine the 2DSP by formally evaluating graphical track visualisation styles. Track visualisations considered were text only, circles containing text, individually coloured circles containing text, circles colour coded by instrument type with text, icons with text superimposed, circles with RMS related dynamic opacity and a traditional AMI. The usability evaluation f...

A methodological approach for designing a usable ontology-based GUI in healthcare.

Science.gov (United States)

Lasierra, N; Kushniruk, A; Alesanco, A; Borycki, E; García, J

2013-01-01

This paper presents a methodological approach to the design and evaluation of an interface for an ontology-based system used for designing care plans for monitoring patients at home. In order to define the care plans, physicians need a tool for creating instances of the ontology and configuring some rules. Our purpose is to develop an interface to allow clinicians to interact with the ontology. Although ontology-driven applications do not necessarily present the ontology in the user interface, it is our hypothesis that showing selected parts of the ontology in a "usable" way could enhance clinician's understanding and make easier the definition of the care plans. Based on prototyping and iterative testing, this methodology combines visualization techniques and usability methods. Preliminary results obtained after a formative evaluation indicate the effectiveness of suggested combination.

Assessment of Usability Benchmarks: Combining Standardized Scales with Specific Questions

Directory of Open Access Journals (Sweden)

Stephanie Bettina Linek

2011-12-01

Full Text Available The usability of Web sites and online services is of rising importance. When creating a completely new Web site, qualitative data are adequate for identifying the most usability problems. However, changes of an existing Web site should be evaluated by a quantitative benchmarking process. The proposed paper describes the creation of a questionnaire that allows a quantitative usability benchmarking, i.e. a direct comparison of the different versions of a Web site and an orientation on general standards of usability. The questionnaire is also open for qualitative data. The methodology will be explained by the digital library services of the ZBW.

Usability-Studie zu KonSearch : Evaluation der neuen Literatursuchmaschine der Universität Konstanz

OpenAIRE

Luca, Helena

2011-01-01

Der vorliegende Bericht beschreibt Hintergründe, Methodik und Ergebnisse einer Usability-Evaluation, die im Sommersemester 2011 im Zuge der Einführung der Literatursuchmaschine KonSearch an der Universität Konstanz durchgeführt wurde. Die neue Suchmaschine basiert auf dem Resource Discovery Service Summon der Firma SerialsSolutions. Sie soll den Benutzern der Bibliothek der Universität Konstanz eine effektive, effiziente und zufriedenstellende Suche nach wissenschaftlichen Informationen ermög...

Educational software usability: Artifact or Design?

Science.gov (United States)

Van Nuland, Sonya E; Eagleson, Roy; Rogers, Kem A

2017-03-01

Online educational technologies and e-learning tools are providing new opportunities for students to learn worldwide, and they continue to play an important role in anatomical sciences education. Yet, as we shift to teaching online, particularly within the anatomical sciences, it has become apparent that e-learning tool success is based on more than just user satisfaction and preliminary learning outcomes-rather it is a multidimensional construct that should be addressed from an integrated perspective. The efficiency, effectiveness and satisfaction with which a user can navigate an e-learning tool is known as usability, and represents a construct which we propose can be used to quantitatively evaluate e-learning tool success. To assess the usability of an e-learning tool, usability testing should be employed during the design and development phases (i.e., prior to its release to users) as well as during its delivery (i.e., following its release to users). However, both the commercial educational software industry and individual academic developers in the anatomical sciences have overlooked the added value of additional usability testing. Reducing learner frustration and anxiety during e-learning tool use is essential in ensuring e-learning tool success, and will require a commitment on the part of the developers to engage in usability testing during all stages of an e-learning tool's life cycle. Anat Sci Educ 10: 190-199. © 2016 American Association of Anatomists. © 2016 American Association of Anatomists.

Usability testing for dummies

CERN Multimedia

CERN. Geneva

2017-01-01

Usability testing seems complicated and time-consuming. Is it though? In fact, it is the best way to understand how real users experience your product. In this interactive session, we will do a live usability test and you will get advice on how to conduct your own usability tests.

Videoconferencing for site initiations in clinical studies: Mixed methods evaluation of usability, acceptability, and impact on recruitment.

Science.gov (United States)

Randell, Rebecca; Backhouse, Michael R; Nelson, E Andrea

2016-12-01

A critical issue for multicentre clinical studies is conducting site initiations, ensuring sites are trained in study procedures and comply with relevant governance requirements before they begin recruiting patients. How technology can support site initiations has not previously been explored. This study sought to evaluate use of off-the-shelf web-based videoconferencing to deliver site initiations for a large national multicentre study. Participants in the initiations, including podiatrists, diabetologists, trial coordinators, and research nurses, completed an online questionnaire based on the System Usability Scale (SUS) (N = 15). This was followed by semi-structured interviews, with a consultant diabetologist, a trial coordinator, and three research nurses, exploring perceived benefits and limitations of videoconferencing. The mean SUS score for the videoconferencing platform was 87.2 (SD = 13.7), suggesting a good level of usability. Interview participants perceived initiations delivered by videoconferencing as being more interactive and easier to follow than those delivered by teleconference. In comparison to face-to-face initiations, videoconferencing takes less time, easily fitting in with the work of staff at the local sites. Perceptions of impact on communication varied according to the hardware used. Off-the-shelf videoconferencing is a viable alternative to face-to-face site initiations and confers advantages over teleconferencing.

Development of a Wearable Cardiac Monitoring System for Behavioral Neurocardiac Training: A Usability Study.

Science.gov (United States)

Uddin, Akib A; Morita, Plinio P; Tallevi, Kevin; Armour, Kevin; Li, John; Nolan, Robert P; Cafazzo, Joseph A

2016-04-22

Elevated blood pressure is one of the main risk factors for death globally. Behavioral neurocardiac training (BNT) is a complementary approach to blood pressure and stress management that is intended to exercise the autonomic reflexes, improve stress recovery, and lower blood pressure. BNT involves cognitive-behavioral therapy with a paced breathing technique and heart rate variability biofeedback. BNT is limited to in-clinic delivery and faces an accessibility barrier because of the need for clinical oversight and the use of complex monitoring tools. The objective of this project was to design, develop, and evaluate a wearable electrocardiographic (ECG) sensor system for the delivery of BNT in a home setting. The wearable sensor system, Beat, consists of an ECG sensor and a mobile app. It was developed iteratively using the principles of test-driven Agile development and user-centered design. A usability study was conducted at Toronto General Hospital to evaluate feasibility and user experience and identify areas of improvement. The Beat sensor was designed as a modular patch to be worn on the user's chest and uses standard ECG electrodes. It streams a single-lead ECG wirelessly to a mobile phone using Bluetooth Low Energy. The use of small, low-power electronics, a low device profile, and a tapered enclosure allowed for a device that can be unobtrusively worn under clothing. The sensor was designed to operate with a mobile app that guides users through the BNT exercises to train them to a slow-paced breathing technique for stress recovery. The BNT app uses the ECG captured by the sensor to provide heart rate variability biofeedback in the form of a real-time heart rate waveform to complement and reinforce the impact of the training. Usability testing (n=6) indicated that the overall response to the design and user experience of the system was perceived positively. All participants indicated that the system had a positive effect on stress management and that they

Working together to improve usability

DEFF Research Database (Denmark)

Nørgaard, Mie; Hornbæk, Kasper

2010-01-01

In theory, usability work is an important and well-integrated activity in developing software. In practice, collaboration on improving usability is ridden with challenges relating to conflicting professional goals, tight project schedules, and unclear usability findings. The authors study those...... challenges through 16 interviews with software developers, usability experts, and project managers. Four themes that are key challenges to successful interaction between stakeholders are identified: poor timing when delivering usability results, results lacking relevance, little respect for other disciplines...

Usability study of a novel, self-lighted, disposable speculum: military applications.

Science.gov (United States)

Jones, Christina L; Gruber, Daniel D; Warner, William; Buller, Jerome L

2013-04-01

Data collected from a postutilization questionnaire were used to evaluate the usability of the OfficeSPEC disposable vaginal speculum, specifically the effectiveness, efficiency, and acceptability, in clinical, hospital, and austere environments. Usability data analysis showed the OfficeSPEC speculum had an effectiveness rating of 4.6/5, efficiency rating of 4.5/5, and acceptability rating of 4.6/5; overall usability in deployed environments was favorable. The overall rankings were 3.4 for plastic, 4.2 for metal (p disposable speculum as a reasonable alternative with yearly cost of $129,200, compared to traditional metal ($209,100) and plastic ($319,175). By evaluating the OfficeSPEC speculum within a usability framework, it proved to be practical, viable alternative in all environments, particularly in the forward deployed environment. Reprint & Copyright © 2013 Association of Military Surgeons of the U.S.

Training Software Developers in Usability Engineering: A Literature Review

DEFF Research Database (Denmark)

Bruun, Anders

2010-01-01

. This paper presents a comprehensive literature study of research conducted in this area, where 129 papers are analyzed in terms of research focus, empirical basis, types of training participants and training costs. Findings show a need for further empirical research regarding long term effects of training...... to fund usability specialists or comprehensive consultancy. Training of non-usability personnel in critical usability engineering methods has the potential of easing these challenges. It is, however, unknown how much and what kind of research that has been committed to novice training in UE methods......, training costs and training in user based evaluation methods....

Heuristic evaluation of paper-based Web pages: a simplified inspection usability methodology.

Science.gov (United States)

Allen, Mureen; Currie, Leanne M; Bakken, Suzanne; Patel, Vimla L; Cimino, James J

2006-08-01

Online medical information, when presented to clinicians, must be well-organized and intuitive to use, so that the clinicians can conduct their daily work efficiently and without error. It is essential to actively seek to produce good user interfaces that are acceptable to the user. This paper describes the methodology used to develop a simplified heuristic evaluation (HE) suitable for the evaluation of screen shots of Web pages, the development of an HE instrument used to conduct the evaluation, and the results of the evaluation of the aforementioned screen shots. In addition, this paper presents examples of the process of categorizing problems identified by the HE and the technological solutions identified to resolve these problems. Four usability experts reviewed 18 paper-based screen shots and made a total of 108 comments. Each expert completed the task in about an hour. We were able to implement solutions to approximately 70% of the violations. Our study found that a heuristic evaluation using paper-based screen shots of a user interface was expeditious, inexpensive, and straightforward to implement.

Usability Results from Human Performance Feedback and Development (HPFD) and ePerformance System Users: Technical Report #1

National Research Council Canada - National Science Library

Dean, Elizabeth; Aspinwall, Kimberly R; Schwerin, Michael J; Kendrick, Douglas E

2006-01-01

...) and ePerformance system. This second and final iteration of usability testing was designed to verify the functionality of system revisions made as a result of recommendations offered in Schwerin et al. (2004...

Knowledge management impact of information technology Web 2.0/3.0. The case study of agent software technology usability in knowledge management system

Science.gov (United States)

Sołtysik-Piorunkiewicz, Anna

2015-02-01

How we can measure the impact of internet technology Web 2.0/3.0 for knowledge management? How we can use the Web 2.0/3.0 technologies for generating, evaluating, sharing, organizing knowledge in knowledge-based organization? How we can evaluate it from user-centered perspective? Article aims to provide a method for evaluate the usability of web technologies to support knowledge management in knowledge-based organizations of the various stages of the cycle knowledge management, taking into account: generating knowledge, evaluating knowledge, sharing knowledge, etc. for the modern Internet technologies based on the example of agent technologies. The method focuses on five areas of evaluation: GUI, functional structure, the way of content publication, organizational aspect, technological aspect. The method is based on the proposed indicators relating respectively to assess specific areas of evaluation, taking into account the individual characteristics of the scoring. Each of the features identified in the evaluation is judged first point wise, then this score is subject to verification and clarification by means of appropriate indicators of a given feature. The article proposes appropriate indicators to measure the impact of Web 2.0/3.0 technologies for knowledge management and verification them in an example of agent technology usability in knowledge management system.

Engaging Community Stakeholders to Evaluate the Design, Usability, and Acceptability of a Chronic Obstructive Pulmonary Disease Social Media Resource Center

Science.gov (United States)

Chaney, Beth; Chaney, Don; Paige, Samantha; Payne-Purvis, Caroline; Tennant, Bethany; Walsh-Childers, Kim; Sriram, PS; Alber, Julia

2015-01-01

Background Patients with chronic obstructive pulmonary disease (COPD) often report inadequate access to comprehensive patient education resources. Objective The purpose of this study was to incorporate community-engagement principles within a mixed-method research design to evaluate the usability and acceptability of a self-tailored social media resource center for medically underserved patients with COPD. Methods A multiphase sequential design (qual → QUANT → quant + QUAL) was incorporated into the current study, whereby a small-scale qualitative (qual) study informed the design of a social media website prototype that was tested with patients during a computer-based usability study (QUANT). To identify usability violations and determine whether or not patients found the website prototype acceptable for use, each patient was asked to complete an 18-item website usability and acceptability questionnaire, as well as a retrospective, in-depth, semistructured interview (quant + QUAL). Results The majority of medically underserved patients with COPD (n=8, mean 56 years, SD 7) found the social media website prototype to be easy to navigate and relevant to their self-management information needs. Mean responses on the 18-item website usability and acceptability questionnaire were very high on a scale of 1 (strongly disagree) to 5 (strongly agree) (mean 4.72, SD 0.33). However, the majority of patients identified several usability violations related to the prototype’s information design, interactive capabilities, and navigational structure. Specifically, 6 out of 8 (75%) patients struggled to create a log-in account to access the prototype, and 7 out of 8 patients (88%) experienced difficulty posting and replying to comments on an interactive discussion forum. Conclusions Patient perceptions of most social media website prototype features (eg, clickable picture-based screenshots of videos, comment tools) were largely positive. Mixed-method stakeholder feedback was

From Usability Engineering to Evidence-based Usability in Health IT

NARCIS (Netherlands)

Marcilly, Romaric; Peute, Linda; Beuscart-Zephir, Marie-Catherine

2016-01-01

Usability is a critical factor in the acceptance, safe use, and success of health IT. The User-Centred Design process is widely promoted to improve usability. However, this traditional case by case approach that is rooted in the sound understanding of users' needs is not sufficient to improve

Usability testing of interaction components: taking the message exchange as a measure of usability

NARCIS (Netherlands)

Brinkman, W.P.; Haakma, R.; Bouwhuis, D.G.; Jacob, R.J.K.; Limbourg, Q; Vanderdonckt, J.

2004-01-01

Component-based Software Engineering (CBSE) is concerned with the development of systems from reusable parts (components), and the development and maintenance of these parts. This study addresses the issue of usability testing in a CBSE environment, and specifically automatically measuring the

A task-specific interactive game-based virtual reality rehabilitation system for patients with stroke: a usability test and two clinical experiments.

Science.gov (United States)

Shin, Joon-Ho; Ryu, Hokyoung; Jang, Seong Ho

2014-03-06

Virtual reality (VR) is not commonly used in clinical rehabilitation, and commercial VR gaming systems may have mixed effects in patients with stroke. Therefore, we developed RehabMaster™, a task-specific interactive game-based VR system for post-stroke rehabilitation of the upper extremities, and assessed its usability and clinical efficacy. A participatory design and usability tests were carried out for development of RehabMaster with representative user groups. Two clinical trials were then performed. The first was an observational study in which seven patients with chronic stroke received 30 minutes of RehabMaster intervention per day for two weeks. The second was a randomised controlled trial of 16 patients with acute or subacute stroke who received 10 sessions of conventional occupational therapy only (OT-only group) or conventional occupational therapy plus 20 minutes of RehabMaster intervention (RehabMaster + OT group). The Fugl-Meyer Assessment score (FMA), modified Barthel Index (MBI), adverse effects, and drop-out rate were recorded. The requirements of a VR system for stroke rehabilitation were established and incorporated into RehabMaster. The reported advantages from the usability tests were improved attention, the immersive flow experience, and individualised intervention. The first clinical trial showed that the RehabMaster intervention improved the FMA (P = .03) and MBI (P = .04) across evaluation times. The second trial revealed that the addition of RehabMaster intervention tended to enhance the improvement in the FMA (P = .07) but did not affect the improvement in the MBI. One patient with chronic stroke left the trial, and no adverse effects were reported. The RehabMaster is a feasible and safe VR system for enhancing upper extremity function in patients with stroke.

Key Design Characteristics for Developing Usable E-Commerce Websites in the Arab World

Directory of Open Access Journals (Sweden)

Layla Hasan

2016-09-01

Full Text Available This research aims to suggest key design characteristics that are necessary for developing usable e-commerce websites in the Arab world. A comprehensive usability evaluation of four leading Arab e-commerce websites was conducted using the heuristic evaluation method. The results identified major and minor usability problems and major and minor good design characteristics on the selected websites. Based on the results, 51 key design characteristics were suggested. The recommended key design characteristics comprised two levels according to their priority: level one which includes mandatory key design characteristics and level two which includes supplementary design characteristics. The key design characteristics in each level were categorized under specific pages and areas that can be found on any e-commerce website. Such categorizations could direct website evaluators and designers to important pages and areas that should be considered to improve the overall usability of e-commerce websites. The results of this research are particularly important to developing countries which are still facing challenges that may affect the design and accessibility of usable and useful websites. These relate to low speed of accessing the Internet and a lack of website designers who have experience in customers’ needs and websites’ usable design characteristics.

Virtual Reality for Architectural or Territorial Representations: Usability Perceptions

Directory of Open Access Journals (Sweden)

Atta Idrawani Zaini

2017-05-01

Full Text Available Virtual reality (VR is widely being researched within various aspects of real-world applications. As architecture and urban design are very much adhered to evaluating and designing space, physical representations are deemed as incompetent to deliver a full-scale depiction of a space. Similarly, digital models are very much also limited in that sense. VR can deliver a full-scale virtual environment (VE, tricking users to be immersed in the replicated environment. This is an advantage for the aforementioned design disciplines, as more relatable and realistic depiction of a space can be modelled. The notion of its usability has become important to be understood from the perspective of architecture and urban design. This paper measured the respondents’ perceptions of VR’s usability through measuring its quality of use based on several criteria. The criteria established were the ease of use, usefulness, and satisfaction. Different levels of architectural details were decided as a form of control. A total of N=96 randomly selected respondents from various backgrounds participated in the survey as they were divided into four different group of treatments. Each group experienced a different VE with different level of architectural details. The first section of analysis is a one-sample analysis and the second is a group difference analysis. From the first analysis, it was found that the respondents perceived VR as a usable tool for architectural or territorial representation. Using Kruskal-Wallis test, it was found that there was no statistically significant difference between groups, suggesting that the respondents perceived VR as usable regardless of the level of architectural details. As this paper used perception data based on the quality of use alone, the efficiency of VR system was not measured. Thus, this paper recommends further studies to be conducted on the system’s efficiency to reflect its usability in full extent.

Usability laboratories at Philips : supporting research, development, and design for consumer and professional products

NARCIS (Netherlands)

Vries, de G.; Gelderen, van T.; Brigham, F.

1994-01-01

This paper describes two of the usability laboratories at Philips, discusses practical issues arising from our experience using the facilities, gives an example of a typical usability evaluation, and briefly outlines our vision for the future of the laboratories. Usability tests at Philips can

Empirical usability testing in a component-based environment : improving test efficiency with component-specific usability measures

NARCIS (Netherlands)

Brinkman, W.P.; Haakma, R.; Bouwhuis, D.G.; Bastide, R.; Palanque, P.; Roth, J.

2005-01-01

This paper addresses the issue of usability testing in a component-based software engineering environment, specifically measuring the usability of different versions of a component in a more powerful manner than other, more holistic, usability methods. Three component-specific usability measures are

Proposed Framework for the Evaluation of Standalone Corpora Processing Systems: An Application to Arabic Corpora

Directory of Open Access Journals (Sweden)

Abdulmohsen Al-Thubaity

2014-01-01

Full Text Available Despite the accessibility of numerous online corpora, students and researchers engaged in the fields of Natural Language Processing (NLP, corpus linguistics, and language learning and teaching may encounter situations in which they need to develop their own corpora. Several commercial and free standalone corpora processing systems are available to process such corpora. In this study, we first propose a framework for the evaluation of standalone corpora processing systems and then use it to evaluate seven freely available systems. The proposed framework considers the usability, functionality, and performance of the evaluated systems while taking into consideration their suitability for Arabic corpora. While the results show that most of the evaluated systems exhibited comparable usability scores, the scores for functionality and performance were substantially different with respect to support for the Arabic language and N-grams profile generation. The results of our evaluation will help potential users of the evaluated systems to choose the system that best meets their needs. More importantly, the results will help the developers of the evaluated systems to enhance their systems and developers of new corpora processing systems by providing them with a reference framework.

Proposed framework for the evaluation of standalone corpora processing systems: an application to Arabic corpora.

Science.gov (United States)

Al-Thubaity, Abdulmohsen; Al-Khalifa, Hend; Alqifari, Reem; Almazrua, Manal

2014-01-01

Despite the accessibility of numerous online corpora, students and researchers engaged in the fields of Natural Language Processing (NLP), corpus linguistics, and language learning and teaching may encounter situations in which they need to develop their own corpora. Several commercial and free standalone corpora processing systems are available to process such corpora. In this study, we first propose a framework for the evaluation of standalone corpora processing systems and then use it to evaluate seven freely available systems. The proposed framework considers the usability, functionality, and performance of the evaluated systems while taking into consideration their suitability for Arabic corpora. While the results show that most of the evaluated systems exhibited comparable usability scores, the scores for functionality and performance were substantially different with respect to support for the Arabic language and N-grams profile generation. The results of our evaluation will help potential users of the evaluated systems to choose the system that best meets their needs. More importantly, the results will help the developers of the evaluated systems to enhance their systems and developers of new corpora processing systems by providing them with a reference framework.

A Web-Based Graphical Food Frequency Assessment System: Design, Development and Usability Metrics.

Science.gov (United States)

Franco, Rodrigo Zenun; Alawadhi, Balqees; Fallaize, Rosalind; Lovegrove, Julie A; Hwang, Faustina

2017-05-08

Food frequency questionnaires (FFQs) are well established in the nutrition field, but there remain important questions around how to develop online tools in a way that can facilitate wider uptake. Also, FFQ user acceptance and evaluation have not been investigated extensively. This paper presents a Web-based graphical food frequency assessment system that addresses challenges of reproducibility, scalability, mobile friendliness, security, and usability and also presents the utilization metrics and user feedback from a deployment study. The application design employs a single-page application Web architecture with back-end services (database, authentication, and authorization) provided by Google Firebase's free plan. Its design and responsiveness take advantage of the Bootstrap framework. The FFQ was deployed in Kuwait as part of the EatWellQ8 study during 2016. The EatWellQ8 FFQ contains 146 food items (including drinks). Participants were recruited in Kuwait without financial incentive. Completion time was based on browser timestamps and usability was measured using the System Usability Scale (SUS), scoring between 0 and 100. Products with a SUS higher than 70 are considered to be good. A total of 235 participants created accounts in the system, and 163 completed the FFQ. Of those 163 participants, 142 reported their gender (93 female, 49 male) and 144 reported their date of birth (mean age of 35 years, range from 18-65 years). The mean completion time for all FFQs (n=163), excluding periods of interruption, was 14.2 minutes (95% CI 13.3-15.1 minutes). Female participants (n=93) completed in 14.1 minutes (95% CI 12.9-15.3 minutes) and male participants (n=49) completed in 14.3 minutes (95% CI 12.6-15.9 minutes). Participants using laptops or desktops (n=69) completed the FFQ in an average of 13.9 minutes (95% CI 12.6-15.1 minutes) and participants using smartphones or tablets (n=91) completed in an average of 14.5 minutes (95% CI 13.2-15.8 minutes). The median SUS

Towards Usable E-Health. A Systematic Review of Usability Questionnaires.

Science.gov (United States)

Sousa, Vanessa E C; Dunn Lopez, Karen

2017-05-10

The use of e-health can lead to several positive outcomes. However, the potential for e-health to improve healthcare is partially dependent on its ease of use. In order to determine the usability for any technology, rigorously developed and appropriate measures must be chosen. To identify psychometrically tested questionnaires that measure usability of e-health tools, and to appraise their generalizability, attributes coverage, and quality. We conducted a systematic review of studies that measured usability of e-health tools using four databases (Scopus, PubMed, CINAHL, and HAPI). Non-primary research, studies that did not report measures, studies with children or people with cognitive limitations, and studies about assistive devices or medical equipment were systematically excluded. Two authors independently extracted information including: questionnaire name, number of questions, scoring method, item generation, and psychometrics using a data extraction tool with pre-established categories and a quality appraisal scoring table. Using a broad search strategy, 5,558 potentially relevant papers were identified. After removing duplicates and applying exclusion criteria, 35 articles remained that used 15 unique questionnaires. From the 15 questionnaires, only 5 were general enough to be used across studies. Usability attributes covered by the questionnaires were: learnability (15), efficiency (12), and satisfaction (11). Memorability (1) was the least covered attribute. Quality appraisal showed that face/content (14) and construct (7) validity were the most frequent types of validity assessed. All questionnaires reported reliability measurement. Some questionnaires scored low in the quality appraisal for the following reasons: limited validity testing (7), small sample size (3), no reporting of user centeredness (9) or feasibility estimates of time, effort, and expense (7). Existing questionnaires provide a foundation for research on e-health usability. However

The role of usability in e-lexicography

Directory of Open Access Journals (Sweden)

André H. du Plessis

2017-05-01

Full Text Available In the development of the metalexicography, the primary focus of metalexicographers was on the development of printed dictionaries. As technology advances, the focus now shifts towards online and electronic products. Tarp (2009:23–24 posits that this shift results in metalexicographic shortcomings. This means that the theories used to create and evaluate dictionaries are not sufficient anymore. Bergenholtz and Bothma (2011:54 show that dictionaries as data carriers are now not only limited to the field of lexicography, but also to information technological fields. For current lexicographers to be successful in the digital age, the theories used in designing, developing and critiquing dictionaries must adapt with regards to current information technological trends, norms and practices. One aspect that must be addressed is the usability of dictionaries and other electronic lexicographic tools. By using usability theories and standards, current metalexicographical principles can be adapted for the digital age in a streamlined fashion. This study aims to show how usability theories and standards can be advantageous for metalexicography and that future electronic metalexicographic endeavours should not be undertaken without considering the usability as part of the theoretical framework.

大學圖書館網站使用性評估之探討 The Usability Evaluation Study of University Library Websites

Directory of Open Access Journals (Sweden)

Jiann-Cherng Shieh

2009-12-01

Full Text Available 圖書館網站是圖書館服務的延伸,因此除了網站內容能否滿足讀者需求外,網站是否易於被使用,是否易於找到資料,亦是影響圖書館網站服務品質的重要指標。本研究利用任務導向之使用性測試、觀察與深度訪談等方法獲取受試者於個案圖書館網站使用性資料,以分析了解不同學科背景使用者於使用圖書館網站之差異,發掘使用者對圖書館網站使用性之問題,改善舊有圖書館網站,並透過圖書館網站使用性之再評估,驗證大學圖書館網站進行使用性評估之成效。另本研究進一步採用系統使用性量表問卷,於使用性測試前後量測使用者對圖書館網站使用性之滿意度,藉由滿意度統計分析以驗證使用者對使用性改善滿意度之提昇。本研究結果顯示不同學科背景使用者對大學圖書館網站使用性有差異,且造成差異的主要原因與其使用圖書館網站頻率有關;另依據使用者於圖書館網站使用性問題,修改重構部分個案圖書館網站並進行後測,結果顯示使用者不僅均能順利完成任務且其平均完成時間亦減少%,同時使用者對修改後圖書館網站之系統使用性滿意度亦明顯高於修改前。本研究證實大學圖書館網站進行使用性評估確能協助提昇使用者對圖書館網站之使用效率及使用性滿意度。Library websites are the platforms that libraries extend their services to. In addition to concerning the contents of library websites, their easy understand- ing and flexible usage are the most important issues that are liable to disregard their critical effects on library web sites design. Library websites of poor us- ability will frustrate users’ aspirations for libraries services. In this research, based on user centered design, we apply various methods of system usability scale, usability testing, think aloud and interview to evaluate the usability of

Evaluating the Usability of Authoring Environments for Serious Games.

Science.gov (United States)

Slootmaker, Aad; Hummel, Hans; Koper, Rob

2017-08-01

Background . The EMERGO method and online platform enable the development and delivery of scenario-based serious games that foster students to acquire professional competence. One of the main goals of the platform is to provide a user-friendly authoring environment for creating virtual environments where students can perform authentic tasks. Aim . We present the findings of an in-depth qualitative case study of the platform's authoring environment and compare our findings on usability with those found for comparable environments in literature. Method . We carried out semi-structured interviews, with two experienced game developers who have authored a game for higher education, and a literature review of comparable environments. Findings . The analysis shows that the usability of the authoring environment is problematic, especially regarding understandability and learnability , which is in line with findings of comparable environments. Other findings are that authoring is well integrated with the EMERGO method and that functionality and reliability of the authoring environment are valued. Practical implications . The lessons learned are presented in the form of general guidelines to improve the understandability and learnability of authoring environments for serious games .

What makes a control system usable? An operational viewpoint

International Nuclear Information System (INIS)

Clay, M.

1990-01-01

This report discusses the generally accepted successes and shortcomings of the various computer and hardware-based control systems at the Los Alamos Meson Physics Facility (LAMPF) from an operator's standpoint. LAMPF currently utilizes three separate control rooms that, although critically co-dependent, use distinct operating methods. The first, the Injector Control Room, which is responsible for the operation of the three ion sources, the 750 keV transport lines and the 201.25 MHz portion of the linac, uses a predominantly hardware-based control system. The second, the LANSCE Control Room, which is responsible for the operation of the Los Alamos Neutron Scattering Center, uses a graphical touch-panel interface with single-application screens as its control system. The third, the LAMPF Central Control Room, which is responsible for the overall operation of LAMPF, primarily uses a text-oriented keyboard interface with multiple applications per screen. Though each system provides generally reliable human interfacing to the enormously complex and diverse machine known as LAMPF, the operational requirements of speed, usability, and reliability are increasingly necessitating the use of a standard control system that incorporates the positive aspects of all three control systems. (orig.)

Usability at the CELSTEC Medialab

NARCIS (Netherlands)

Storm, Jeroen; Kostons, Danny

2009-01-01

Storm, J., & Kostons, D. (2009). Usability at the CELSTEC MediaLab. Presentation about the usability testing, usability research and User Centered Design conducted at the CELSTEC MediaLab. July, 7, 2009, Heerlen, The Netherlands: Open University of the Netherlands.

Usability testing of the human-machine interface for the Light Duty Utility Arm System

International Nuclear Information System (INIS)

Kiebel, G.R.; Ellis, J.E.; Masliah, M.R.

1994-01-01

This report describes the usability testing that has been done for the control and data acquisition system for the Light Duty Utility Arm (LDUA) System. A program of usability testing has been established as a part of a process for making the LDUA as easy to use as possible. The LDUA System is being designed to deploy a family of tools, called End Effectors, into underground storage tanks by means of a robotic arm on the end of a telescoping mast, and to collect and manage the data that they generate. The LDUA System uses a vertical positioning mast, to lower the arm into a tank through an existing 30.5 cm access riser. A Mobile Deployment Subsystem is used to position the mast and arm over a tank riser for deployment, and to transport them from tank to tank. The LDUA System has many ancillary subsystems including the Operations Control Trailer, the Tank Riser Interface and Confinement Subsystem, the Decontamination Subsystem, and the End Effector Exchange Subsystem. This work resulted in the identification of several important improvements to the LDUA control and data acquisition system before the design was frozen. The most important of these were color coding of joints in motion, simultaneous operator control of multiple joints, and changes to the field-of-views of the camera lenses for the robot and other camera systems

Usability testing a practical guide for librarians

CERN Document Server

Blakiston, Rebecca

2014-01-01

Do you want to improve the usability of your library website, but feel that it is too difficult, time-consuming, or expensive? Usability Testing: A Practical Guide for Librarians will teach you how to make the case for usability testing, define your audience and their goals, select a usability testing method appropriate for your particular context, plan for an in-house usability test, conduct an effective in-house usability test, analyze usability test results, and create and implement a plan for ongoing, systematic usability testing. Step-by-step instructions, along with a myriad of examples,

Usability Testing of a Multimedia e-Learning Resource for Electrolyte and Acid-Base Disorders

Science.gov (United States)

Davids, Mogamat Razeen; Chikte, Usuf; Grimmer-Somers, Karen; Halperin, Mitchell L.

2014-01-01

The usability of computer interfaces may have a major influence on learning. Design approaches that optimize usability are commonplace in the software development industry but are seldom used in the development of e-learning resources, especially in medical education. We conducted a usability evaluation of a multimedia resource for teaching…

Ten Demands of Improved Usability in eHealth and Some Progress - Co-Creation by Health and Social Care Professionals.

Science.gov (United States)

Scandurra, Isabella; Liljequist, David

2016-01-01

Current healthcare organizations often do not accomplish the intended effects of their eHealth systems due to inadequate usability. Commissioned by the Swedish Ministry of Health and Social Affairs, the usability of current eHealth systems in Swedish health and social care has been analysed from the perspective of their professionals. The objective of the study was to report on current problems, potential solutions as well as to relate these to research in relevant areas. Using a participatory approach, nine workshops were held where health informatics researchers guided staff from different care organizations, representatives of the national associations of health and social care professionals and the national eHealth system vendor organization. This paper presents ten demands that Swedish health and social care professionals find imperative to prioritize. The study emphasizes that development of eHealth systems must be integrated into the care practice improvement process and iteratively evaluated regarding usability.

Usability Briefing - a process model for healthcare facilities

DEFF Research Database (Denmark)

Fronczek-Munter, Aneta

2014-01-01

Background: In complex buildings with many types of users it can be difficult to satisfy the numerous, often contradictory requirements. Research in usability mostly focuses on evaluating products or facilities with users, after they were built. This paper is part of a PhD project “Usability...... with various users/stakeholders, using creative boundary objects at workshops.  Practical Implications: The research results have relevance to researchers, client organisations, facility managers and architects planning new complex facilities.  Research limitations: The proposed model is theoretical and needs...... briefing for hospitals”, where methods for capturing user needs and experiences at hospital facilities are investigated in order to feed into design processes and satisfy the users’ needs and maximise the effectiveness of facilities. Purpose: This paper introduces the concept of usability briefing...

Software architecture analysis of usability

NARCIS (Netherlands)

Folmer, Eelke

2005-01-01

One of the qualities that has received increased attention in recent decades is usability. A software product with poor usability is likely to fail in a highly competitive market; therefore software developing organizations are paying more and more attention to ensuring the usability of their

A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence

NARCIS (Netherlands)

Jaspers, Monique W. M.

2009-01-01

OBJECTIVE: Usability evaluation is now widely recognized as critical to the success of interactive health care applications. However, the broad range of usability inspection and testing methods available may make it difficult to decide on a usability assessment plan. To guide novices in the

Evaluation of the functional performance and technical quality of an Electronic Documentation System of the Nursing Process.

Science.gov (United States)

de Oliveira, Neurilene Batista; Peres, Heloisa Helena Ciqueto

2015-01-01

To evaluate the functional performance and the technical quality of the Electronic Documentation System of the Nursing Process of the Teaching Hospital of the University of São Paulo. exploratory-descriptive study. The Quality Model of regulatory standard 25010 and the Evaluation Process defined under regulatory standard 25040, both of the International Organization for Standardization/International Electrotechnical Commission. The quality characteristics evaluated were: functional suitability, reliability, usability, performance efficiency, compatibility, security, maintainability and portability. The sample was made up of 37 evaluators. in the evaluation of the specialists in information technology, only the characteristic of usability obtained a rate of positive responses of less than 70%. For the nurse lecturers, all the quality characteristics obtained a rate of positive responses of over 70%. The staff nurses of the medical and surgical clinics with experience in using the system) and staff nurses from other units of the hospital and from other health institutions (without experience in using the system) obtained rates of positive responses of more than 70% referent to the functional suitability, usability, and security. However, performance efficiency, reliability and compatibility all obtained rates below the parameter established. the software achieved rates of positive responses of over 70% for the majority of the quality characteristics evaluated.

Integrating Visual Mnemonics and Input Feedback With Passphrases to Improve the Usability and Security of Digital Authentication.

Science.gov (United States)

Juang, Kevin; Greenstein, Joel

2018-04-01

We developed a new authentication system based on passphrases instead of passwords. Our new system incorporates a user-generated mnemonic picture displayed during login, definition tooltips, error correction to reduce typographical errors, a decoy-based input masking technique, and random passphrase generation using either a specialized wordlist or a sentence template. Passphrases exhibit a greater level of security than traditional passwords, but their wider adoption has been hindered by human factors issues. Our assertion is that the added features of our system work particularly well with passphrases and help address these shortcomings. We conducted a study to evaluate our new system with a customized 1,450-word list and our new system with a 6-word sentence structure against the control conditions of a user-created passphrase of at least 24 characters and a system-generated passphrase using a 10,326-word list. Fifty participants completed two sessions so that we could measure the usability and security of the authentication schemes. With the new system conditions, memorability was improved, and security was equivalent to or better than the control conditions. Usability and overall ratings also favored the new system conditions over the control conditions. Our research presents a new authentication system using innovative techniques that improve on the usability and security of existing password and passphrase authentication systems. In computer security, drastic changes should never happen overnight, but we recommend that our contributions be incorporated into current authentication systems to help facilitate a transition from passwords to usable passphrases.

Upper-Extremity Rehabilitation Robot RehabRoby: Methodology, Design, Usability and Validation

Directory of Open Access Journals (Sweden)

Fatih Ozkul

2013-12-01

Full Text Available In this study, an exoskeleton type robot-assisted rehabilitation system, called RehabRoby, is developed for rehabilitation purposes. A control architecture, which contains a high-level controller and a low-level controller, is designed so that RehabRoby can complete the given rehabilitation task in a desired and safe manner. A hybrid system modelling technique is used for the high-level controller. An admittance control with an inner robust position control loop is used for the low-level control of the RehabRoby. Real-time experiments are performed to evaluate the control architecture of the robot-assisted rehabilitation system, RehabRoby. Furthermore, the usability of RehabRoby is evaluated.

Usability of computerized nursing process from the ICNP® in intensive care units

Directory of Open Access Journals (Sweden)

Daniela Couto Carvalho Barra

2015-04-01

Full Text Available OBJECTIVE To analyze the usability of Computerized Nursing Process (CNP from the ICNP® 1.0 in Intensive Care Units in accordance with the criteria established by the standards of the International Organization for Standardization and the Brazilian Association of Technical Standards of systems. METHOD This is a before-and-after semi-experimental quantitative study, with a sample of 34 participants (nurses, professors and systems programmers, carried out in three Intensive Care Units. RESULTS The evaluated criteria (use, content and interface showed that CNP has usability criteria, as it integrates a logical data structure, clinical assessment, diagnostics and nursing interventions. CONCLUSION The CNP is a source of information and knowledge that provide nurses with new ways of learning in intensive care, for it is a place that provides complete, comprehensive, and detailed content, supported by current and relevant data and scientific research information for Nursing practices.

Preliminary Evaluation of a Personal Healthcare System Prototype for Cognitive eRehabilitation in a Living Assistance Domain

Directory of Open Access Journals (Sweden)

Matteo Pastorino

2014-06-01

Full Text Available The integration of rehabilitation systems in an ambient assisted living environment can provide a powerful and versatile tool for long-term stroke rehabilitation goals. This paper introduces a novel concept of a personalized cognitive rehabilitation system in a naturalistic setting. The proposed platform was developed within the CogWatch project, with the intent of fostering independence in activities of daily living in patients with apraxia and action disorganization syndrome. Technical usability was evaluated in a series of pilot experiments, which illustrate how this approach may help to retrain patients in activities of daily living. The first system prototype has been tested with 36 participants divided into three groups, providing an exploratory evaluation of the usability of this solution and its acceptability. The technical solutions used within the CogWatch project are targeted to meet both the end users’ needs from the interaction and usability point of views and the clinical requirements associated with the use of such systems. The challenges behind the development of ambient assisted living systems for cognitive rehabilitation are discussed.

Usability Study Identifies Vocabulary, Facets, and Education as Primary Primo Discovery System Interface Problems

Directory of Open Access Journals (Sweden)

Ruby Muriel Lavallee Warren

2017-09-01

Full Text Available A Review of: Brett, K. R., Lierman, A., & Turner, C. (2016. Lessons learned: A Primo usability study. Information Technology and Libraries, 35(1, 7-25. https://doi.org/10.6017/ital.v35i1.8965 Abstract Objective – To discover whether users can effectively complete common research tasks in a modified Primo Discovery System interface. Design – Usability testing. Setting – University of Houston Libraries. Subjects – Users of the University of Houston Libraries Ex Libris Primo Discovery System interface. Methods – The researchers used a think aloud usability test methodology, with participants asked to verbalize their thought processes as they completed a set of tasks. Four tasks were developed and divided into two task sets (Test 1 and Test 2, with session facilitators alternating sets for each participant. Tasks were as follows: locating a known article, finding a peer reviewed article on a requested subject, locating a book, and finding a newspaper article on a topic. Tests were conducted in front of the library entrance using a laptop equipped with Morae (screen and audio recording software, and participants were recruited via an assigned “caller” at the table offering library merchandise and food as a research incentive. Users could opt out of having their session recorded, resulting in a total of fifteen sessions completed with fourteen recorded. Thirteen of the fifteen participants were undergraduate students, one was a graduate student, one was a post-baccalaureate student, and there were no faculty participants. Facilitators completed notes on a standard rubric, coding participant responses into successes or failures and noting participant feedback. Main Results – All eight participants assigned Test 1 successfully completed Test 1, Task 1: locating a known article. Participants expressed a need for an author limiter in advanced search, and had difficulty using the citation formatted information to locate materials

Evaluating interactive digital television applications through usability heuristics Evaluando aplicaciones de televisión digital interactiva a través de heurísticas de usabilidad

Directory of Open Access Journals (Sweden)

Andrés Solano

2013-04-01

Full Text Available Interactive Digital Television (iDT is considered as the convergence of television and computer technologies. Usability evaluation for applications based on emerging information technology brings new challenges. The main iDT feature is that the user may interact with the application; therefore usability should be a main concern when designing iDT applications. Current research usually focuses on iDT applications from a technical point of view, rather than a user-centered approach. There is a need for new usability evaluation methods or at least for the use of traditional evaluations in novel ways. A set of specific usability heuristics was defined and validated, in order to help the usability evaluations of iDT applications. A usability checklist to be used when applying iDT heuristics is also proposed.La Televisión Digital Interactiva (TDi es considerada como la convergencia de la televisión y las tecnologías de la computación. La evaluación de usabilidad para aplicaciones basadas en tecnologías emergentes trae nuevos retos. La principal característica de la TDi es que los usuarios pueden interactuar con las aplicaciones; de esta manera la usabilidad es el principal aspecto a considerar cuando se diseñan aplicaciones de TDi. Las investigaciones actuales normalmente están enfocadas sobre las aplicaciones de TDi desde un punto de vista técnico, y no desde un enfoque centrado en el usuario. Hay una necesidad de nuevos métodos de evaluación de usabilidad o, por lo menos, utilizar de nuevas formas los métodos de evaluación existentes. Un conjunto de heurísticas de usabilidad específicas ha sido definido y validado, con el objetivo de ayudar en la evaluación de usabilidad de aplicaciones de TDi. También se ha propuesto una lista de verificación para ser usada al momento de aplicar las heurísticas de usabilidad.

Seeing the System through the End Users' Eyes: Shadow Expert Technique for Evaluating the Consistency of a Learning Management System

Science.gov (United States)

Holzinger, Andreas; Stickel, Christian; Fassold, Markus; Ebner, Martin

Interface consistency is an important basic concept in web design and has an effect on performance and satisfaction of end users. Consistency also has significant effects on the learning performance of both expert and novice end users. Consequently, the evaluation of consistency within a e-learning system and the ensuing eradication of irritating discrepancies in the user interface redesign is a big issue. In this paper, we report of our experiences with the Shadow Expert Technique (SET) during the evaluation of the consistency of the user interface of a large university learning management system. The main objective of this new usability evaluation method is to understand the interaction processes of end users with a specific system interface. Two teams of usability experts worked independently from each other in order to maximize the objectivity of the results. The outcome of this SET method is a list of recommended changes to improve the user interaction processes, hence to facilitate high consistency.