Using qualitative studies to improve the usability of an EMR
Blackford Middleton2005, Journal of Biomedical …
Sign up for access to the world's latest research
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Abstract
The adoption of electronic medical records (EMRs) and user satisfaction are closely associated with the systemÕs usability. To improve the usability of a results management module of a widely deployed web-based EMR, we conducted two qualitative studies that included multiple focus group and field study sessions. Qualitative research can help focus attention on user tasks and goals and identify patterns of care that can be visualized through task modeling exercises. Findings from both studies raised issues with the amount and organization of information in the display, interference with workflow patterns of primary care physicians, and the availability of visual cues and feedback. We used the findings of these studies to recommend design changes to the user interface of the results management module.
Related papers
Evaluating physician satisfaction regarding user interactions with an electronic medical record system
Proceedings / AMIA ... Annual Symposium. AMIA Symposium
A limiting factor in realizing the full potential of electronic medical records (EMR) is physician reluctance to use these applications. There have been very few formal usability studies of experienced physician users of EMRs in routine clinical use. We distributed the Questionnaire for User Interaction Satisfaction (QUIS) to 75 primary care physicians who routinely use the Brigham and Women's Integrated Computing System (BICS). BICS scored highest in the area of screen design and lowest in the area of system capability. Overall user satisfaction was most highly correlated with screen design and layout, and surprisingly not with system response time. Human-computer interaction studies can help focus our design efforts as we strive to increase clinician usage of information technology.
Task-oriented evaluation of electronic medical records systems: development and validation of a questionnaire for physicians
BMC Medical Informatics and Decision Making, 2004
Background: Evaluation is a challenging but necessary part of the development cycle of clinical information systems like the electronic medical records (EMR) system. It is believed that such evaluations should include multiple perspectives, be comparative and employ both qualitative and quantitative methods. Self-administered questionnaires are frequently used as a quantitative evaluation method in medical informatics, but very few validated questionnaires address clinical use of EMR systems.
A mixed-methods evaluation framework for electronic health records usability studies
Journal of Biomedical Informatics, 2019
Poor EHR design adds further challenges, especially in the areas of order entry and information visualization, with a net effect of increased rates of incidents, accidents, and mortality in ICU settings. The purpose of the study was to propose a novel, mixed-methods framework to understand EHRrelated information overload by identifying and characterizing areas of suboptimal usability and clinician frustration within a vendor-based, provider-facing EHR interface. A mixed-methods, live observational usability study was conducted at a single, large, tertiary academic medical center in the Southeastern US utilizing a commercial, vendor based EHR. Physicians were asked to complete usability patient cases, provide responses to three surveys, and participant in a semi-structured interview. Of the 25 enrolled ICU physician participants, there were 5(20%) attending physicians, 9 (36%) fellows, and 11 (44%) residents; 52% of participants were females. On average, residents were the quickest in completing the tasks while attending physician took the longest to complete the same task. Attending physicians reported the highest levels of frustration followed by fellows and residents. Physicians remain frustrated with the EHR due to difficulty in finding patient information. EHR usability remains a critical challenge in healthcare, with implications for medical errors, patient safety, and clinician burnout. There is a need for scientific findings on current information needs and ways to improve EHR-related information overload.
End-user support for primary care electronic medical records: a qualitative case study of users’ needs, expectations, and realities
Health Systems, 2013
Support is considered an important factor for realizing the benefits of health information technology (HIT) but there is a dearth of research on the topic of support, especially in primary care. We conducted a qualitative multiple case study of 4 family health teams (FHTs) and one family health organization (FHO) in Ontario, Canada in an attempt to gain insight into users' expectations and needs, and the realities of end-user support for primary care electronic medical records (EMRs). Data were collected by semi-structured interviews, documents review, and observation of training sessions. The analysis highlights the important role of on-site information technology (IT) staff and super-users in liaising with various stakeholders to solve technical problems and providing hardware and functional ('how to') support; the local development of data
Usability Improvements to Electronic Health Records - Finding Realistic Paths to Success
2015
There is a need to develop more effective strategies for improving usability within information and communication technologies, specifically, electronic health record (EHR) systems. Usability incorporates the ease of use, learnability, efficiency, effectiveness and satisfaction of a system by the end user. When an EHR incorporates principles of usability, the system does a better job of meeting the needs of the end users intuitively, resulting in fewer errors and better quality outcomes for patients. Health systems and governments have been urging the widespread adoption of EHR systems to help lower costs and increase efficiency, putting themselves and vendors under pressure to develop and implement the best solution quickly. Too often stakeholders rely on future releases and enhancements to fix issues not originally planned for in the initial release of the product. Hospital system end users and vendors need better strategies for improving usability in EHR solutions. Poor usability can result in frustrated end-users, inefficient and more costly processes, and, at worst, compromises to patient safety. Proposed strategies and areas for future development are discussed.
Physician experiences of screen-level features in a prominent electronic health record: Design recommendations from a qualitative study
Health Informatics Journal
The goal of this qualitative study was to assess physicians’ perceptions around features of key screens within a prominent commercial EHR, and to solicit end-user recommendations for improved retrieval of high-priority clinical information. We conducted a qualitative, descriptive study of 25 physicians in a medical ICU setting. at a tertiary academic medical center. An in-depth, semi-structured interview guide was developed to elicit physician perceptions on information retrieval as well as favorable and unfavorable features of specific EHR screens. Transcripts were independently coded in a qualitative software management tool by at least two trained coders using a common code book. We successfully obtained vendor permission to map physicians perception’s on full Epic© screenshots. Among the 25 physician participants (13 female; 5 attending physicians, 9 fellows, 11 residents), the majority of participants reported experiencing challenges finding clinical information in the EHR. We ...
National questionnaire study on clinical ICT systems proofs: Physicians suffer from poor usability
International Journal of Medical Informatics, 2011
Medical informatics Computer systems evaluation Questionnaires Task performance User-computer interface Electronic health record Patient care a b s t r a c t Objective: In the health informatics field, usability studies typically focus on evaluating a single information system and involve a rather small group of end-users. However, little is known about the usability of clinical information and communication technology (ICT) environment in which healthcare professionals work daily. This paper aims at contributing to usability research and user-oriented development of healthcare technologies with three objectives: inform researchers and practitioners about the current state of usability of clinical ICT systems, increase the understanding of usability aspects specific for clinical context, and encourage a more holistic approach on studying usability issues in health informatics field. Methods: A national web questionnaire study was conducted in Finland in spring 2010 with 3929 physicians actively working in patient care. For the purposes of the study, we described three dimensions of clinical ICT system usability that reflect the physicians' viewpoint on system usage: (1) compatibility between clinical ICT systems and physicians' tasks, (2) ICT support for information exchange, communication and collaboration in clinical work, and (3) interoperability and reliability. The dimensions derive from the definitions of usability and clinical context of use analysis, and reflect the ability of ICT systems to have a positive impact on patient care by supporting physicians in achieving their goals with a pleasant user experience. The research data incorporated 32 statements with a five-point Likert-scale on physicians' experiences on usability of their currently used ICT systems and a summative question about school grade given to electronic health record (EHR) systems.
Usability Evaluation of the Electronic Medical Record of the Rapid Response Team: a Case Study
Applied Health Information Technology (AHIT) , 2021
Aim: This study was conducted to determine the usability of the rapid response team (RRT) electronic medical records (EMRs) system at an Abu-Ali-Sina organ transplant hospital, Shiraz, Iran. Method: This cross-sectional study was carried out in partnership with 25 direct members of RRT includes nurses and anesthesia technicians who were on the shift during the data collection for two months. To evaluate, the Questionnaire for User Interaction Satisfaction (QUIS) version 7 was used. Data were analyzed by SPSS version 19. Results: A total of 20 out of 25 questionnaires were obtained. Seven (25.0 %) of 27 sections were higher than seven, and all areas were higher than five. The highest rankings were for 1) reading characters on the computer screen 2) highlighting on the screen simplify task 3) overall reactions: wonderful and learning to operate the system. Conclusion: This study demonstrates the usability of the hospital RRT for the EMRs system and over the moderate. However, the flexibility and capability of the rapid response for EMRs tool require to be improved.
Examining perspectives on the adoption and use of computer-based patient-reported outcomes among clinicians and health professionals: a Q methodology study
Journal of the American Medical Informatics Association, 2021
Objective: To determine factors that influence the adoption and use of patient-reported outcomes (PROs) in the electronic health record (EHR) among users. Materials and Methods: Q methodology, supported by focus groups, semistructured interviews, and a review of the literature was used for data collection about opinions on PROs in the EHR. An iterative thematic analysis resulted in 49 statements that study participants sorted, from most unimportant to most important, under the following condition of instruction: "What issues are most important or most unimportant to you when you think about the adoption and use of patient-reported outcomes within the electronic health record in routine clinical care?" Using purposive sampling, 50 participants were recruited to rank and sort the 49 statements online, using HTMLQ software. Principal component analysis and Varimax rotation were used for data analysis using the PQMethod software. Results: Participants were mostly physicians (24%) or physician/researchers (20%). Eight factors were identified. Factors included the ability of PROs in the EHR to enable: efficient and reliable use; care process improvement and accountability; effective and better symptom assessment; patient involvement for care quality; actionable and practical clinical decisions; graphical review and interpretation of results; use for holistic care planning to reflect patients' needs; and seamless use for all users. Discussion: The success of PROs in the EHR in clinical settings is not dependent on a "one size fits all" strategy, demonstrated by the diversity of viewpoints identified in this study. A sociotechnical approach for implementing PROs in the EHR may help improve its success and sustainability. Conclusions: PROs in the EHR are most important to users when the technology is used to improve patient outcomes. Future research must focus on the impact of embedding this EHR functionality on care processes.
Development of a Nursing Electronic Medical Record Usability Protocol
CIN: Computers, Informatics, Nursing, 2018
Assessing usability of an electronic medical record is useful for organizations wishing to customize their electronic medical record and determine the impact on usability. The purpose of this article is to describe the development of a protocol to measure electronic medical record usability from a nursing perspective and to develop a scoring methodology. The Technical Evaluation, Testing, and Validation of the Usability of Electronic Health Records (NISTIR 7804), published by the National Institute of Standards and Technology, guided protocol development. Usability was determined by measuring effectiveness, efficiency, and satisfaction. Satisfaction was measured by the System Usability Scale, while effectiveness and efficiency were measured via performance testing by participants, who completed one of three scenarios. A protocol and scoring formula were developed and used to assess nursing usability, resulting in an overall "C" grade for the electronic medical record. Protocol implementation and the scoring/grading calculations can be replicated to assess electronic medical record usability. The three scenarios used in this protocol will be made available upon request from the primary author to promote the use of electronic medical record usability assessment. Using and expanding upon the government's recommended usability assessment guidelines, we were successful in measuring nursing electronic medical record usability and rating an electronic medical record.
Assessing Usability and Ambulatory Clinical Staff Satisfaction with Two Electronic Health Records
Applied Clinical Informatics, 2023
BACKGROUND: A growing body of literature has linked usability limitations within electronic health records (EHRs) to adverse outcomes which may in turn affect EHR system transitions. NewYork-Presbyterian Hospital (NYP), Columbia University College of Physicians and Surgeons (CU) and Weill Cornell Medical College (WC) are a tripartite organization with large academic medical centers that initiated a phased transition of their EHRs to one system, EpicCare©. OBJECTIVES: To characterize usability perceptions stratified by provider roles by surveying WC ambulatory clinical staff already utilizing EpicCare© and CU ambulatory clinical staff utilizing iterations of Allscripts© before the implementation of EpicCare© campus-wide. METHODS: A customized 19-question electronic survey utilizing usability constructs based on the Health Information Technology Usability Evaluation Scale was anonymously administered prior to EHR transition. Responses were recorded with self-reported demographics. RESULTS: 1,666 CU and 1,065 WC staff with ambulatory self-identified work setting were chosen. Select demographic statistics between campus staff were generally similar with small differences in patterns of clinical and EHR experience. Results demonstrated significant differences in EHR usability perceptions among ambulatory staff based on role and EHR system. WC staff utilizing EpicCare© accounted for more favorable usability metrics than CU across all constructs. Ordering providers (OPs) denoted less usability than non-OPs. The Perceived Usefulness and User Control constructs accounted for the largest differences in usability perceptions. The Cognitive Support and Situational Awareness construct was similarly low for both campuses. Prior EHR experience demonstrated limited associations. CONCLUSIONS: Usability perceptions can be affected by role and EHR system. OPs consistently denoted less usability overall and were more affected by EHR system than non-OPs. While there was greater perceived usability for EpicCare© to perform tasks related to care coordination, documentation and error prevention, there were persistent shortcomings regarding tab navigation and cognitive burden reduction, which have implications on provider efficiency and wellness.
Pre-post evaluation of physicians' satisfaction with a redesigned electronic medical record system
Studies in health technology and informatics, 2008
Physicians' acceptance of Electronic Medical Record Systems (EMRs) is closely related to their usability. Knowledge about end-users' opinions on usability of an EMR system may contribute to planning for the next phase of the usability cycle of the system. A demand for integration of new functionalities, such as computerized order entry and an electronic patient status led to redesign of our EMR system, which had been in use for over 8 years at the Academic Medical Center of Amsterdam. The aim of this study was to understand whether the redesigned EMR system was an improvement of the earlier EMR and which system aspects accounted for user satisfaction and which did not. We conducted a formative pre- and post usability evaluation of our former and redesigned EMR system. For the assessment of both system versions' usability, we distributed two standardized usability questionnaires among 150 clinicians who routinely had used the older EMR system and had been working with its...
A usability and safety analysis of electronic health records: a multi-center study
Journal of the American Medical Informatics Association : JAMIA, 2018
To characterize the variability in usability and safety of EHRs from two vendors across four healthcare systems (2 Epic and 2 Cerner). Twelve to 15 emergency medicine physicians participated from each site and completed six clinical scenarios. Keystroke, mouse click, and video data were collected. From the six scenarios, two diagnostic imaging, laboratory, and medication tasks were analyzed. There was wide variability in task completion time, clicks, and error rates. For certain tasks, there were an average of a nine-fold difference in time and eight-fold difference in clicks. Error rates varied by task (X-ray 16.7% to 25%, MRI: 0 to 10%, Lactate: 0% to 14.3%, Tylenol: 0 to 30%; Taper: 16.7% to 50%). The variability in time, clicks, and error rates highlights the need for improved implementation optimization. EHR implementation, in addition to vendor design and development, is critical to usable and safe products.
Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA
2013
Abstract In response to mounting evidence that use of electronic medical record systems may cause unintended consequences, and even patient harm, the AMIA Board of Directors convened a Task Force on Usability to examine evidence from the literature and make recommendations. This task force was composed of representatives from both academic settings and vendors of electronic health record (EHR) systems.
Usability Improvement of a Clinical Decision Support System
Lecture Notes in Computer Science, 2014
This paper focuses on improving the usability of an electronic health record (EHR) embedded clinical decision support system (CDSS) targeted to treat pain in elderly adults. CDSS have the potential to impact provider behavior. Optimizing CDSS-provider interaction and usability may enhance CDSS use. Five CDSS interventions were developed and deployed in test scenarios within a simulated EHR that mirrored typical Emergency Department (ED) workflow. Provider feedback was analyzed using a mixed methodology approach. The CDSS interventions were iteratively designed across three rounds of testing based upon this analysis. Iterative CDSS design led to improved provider usability and favorability scores.
Technical Evaluation, Testing, and Validation of the Usability of Electronic Health Records
National Institute of Standards and Technology, 2012
Using human factors methods to design a new interface for an electronic medical record
AMIA ... Annual Symposium proceedings / AMIA Symposium. AMIA Symposium, 2007
The Veterans Health Administration (VHA) is a leader in development and use of electronic patient records and clinical decision support. The VHA is currently reengineering a somewhat dated platform for its Computerized Patient Record System (CPRS). This process affords a unique opportunity to implement major changes to the current design and function of the system. We report on two human factors studies designed to provide input and guidance during this reengineering process. One study involved a card sort to better understand how providers tend to cognitively organize clinical data, and how that understanding can help guide interface design. The other involved a simulation to assess the impact of redesign modifications on computerized clinical reminders, a form of clinical decision support in the CPRS, on the learnability of the system for first-time users.
A National Survey of EMR Usability: Comparisons between medical and nursing professions in the hospital and primary care sectors in Australia and Finland
International Journal of Medical Informatics, 2021
Background Electronic Medical Record Systems (EMRs) are now part of nursing and medical professionals daily work in the acute and primary care sectors in Australia. Usability is an important factor in their successful adoption and impacts upon clinical workflow, safety and quality, communication, and collaboration. This study replicates a significant body of work conducted by Finnish researchers applying a usability focused survey to understand medical and nursing professionals' experiences in the Australian context. As we implement EMRs across health systems, their usability and design to support clinicians to effectively deliver and document care, is essential. Methods We conducted an observational study using a cross sectional survey, the National Usability-Focused HIS Scale (NuHISS) developed and validated by Finnish researchers. For this study 13 usability statements collected clinician impressions of EMRs related to technical quality, ease of use, benefits, and collaboration. We report the responses from medical and nursing professionals working in clinical practice settings in Australia, including primary care and hospital sectors in 2020. Results Nursing and medical professionals have different experiences with EMR usability. This depends on the sector they work in and the usability feature measured. In our sample, technical quality features were more positively experienced by doctors in the primary care sector than nurses as well as ease of obtaining patient information and prevention of errors. In the hospital sector nurses experiences with EMRs were more positive with respect to support for routine task completion, learnability, ease of obtaining patient information and entry of patient data. Conclusions The NuHISS is a suitable tool for measuring the usability experiences of Australian clinicians and the EMRs utilised. Differences in usability experiences were noted between professional groups and sectors. A focus on the usability perspectives of clinicians when enhancing or developing EMR solutions is advocated. Manuscript File Click here to view linked References
Usability and Safety in Electronic Medical Records Interface Design: A Review of Recent Literature and Guideline Formulation
Human factors, 2015
The objectives of this study were to (a) review electronic medical record (EMR) and related electronic health record (EHR) interface usability issues, (b) review how EMRs have been evaluated with safety analysis techniques along with any hazard recognition, and (c) formulate design guidelines and a concept for enhanced EMR interfaces with a focus on diagnosis and documentation processes. A major impact of information technology in health care has been the introduction of EMRs. Although numerous studies indicate use of EMRs to increase health care quality, there remain concerns with usability issues and safety. A literature search was conducted using Compendex, PubMed, CINAHL, and Web of Science databases to find EMR research published since 2000. Inclusion criteria included relevant English-language papers with subsets of keywords and any studies (manually) identified with a focus on EMR usability. Fifty studies met the inclusion criteria. Results revealed EMR and EHR usability prob...
Health IT Usability Focus Section: Data Use and Navigation Patterns among Medical ICU Clinicians during Electronic Chart Review
Applied Clinical Informatics, 2017
Background A detailed understanding of electronic health record (EHR) workflow patterns and information use is necessary to inform user-centered design of critical care information systems. While developing a longitudinal medical record visualization tool to facilitate electronic chart review (ECR) for medical intensive care unit (MICU) clinicians, we found inadequate research on clinician–EHR interactions. Objective We systematically studied EHR information use and workflow among MICU clinicians to determine the optimal selection and display of core data for a revised EHR interface. Methods We conducted a direct observational study of MICU clinicians performing ECR for unfamiliar patients during their routine daily practice at an academic medical center. Using a customized manual data collection instrument, we unobtrusively recorded the content and sequence of EHR data reviewed by clinicians. Results We performed 32 ECR observations among 24 clinicians. The median (interquartile ra...
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.