Papers by Laura April McEwen
Paediatrics and Child Health, Apr 29, 2023
Objectives In 2017, Queen’s University launched Competency-Based Medical Education (CBME) across ... more Objectives In 2017, Queen’s University launched Competency-Based Medical Education (CBME) across 29 programs simultaneously. Two years post-implementation, we asked key stakeholders (faculty, residents, and program leaders) within the Pediatrics program for their perspectives on and experiences with CBME so far. Methods Program leadership explicitly described the intended outcomes of implementing CBME. Focus groups and interviews were conducted with all stakeholders to describe the enacted implementation. The intended versus enacted implementations were compared to provide insight into needed adaptations for program improvement. Results Overall, stakeholders saw value in the concept of CBME. Residents felt they received more specific feedback and monthly Competence Committee (CC) meetings and Academic Advisors were helpful. Conversely, all stakeholders noted the increased expectations had led to a feeling of assessment fatigue. Faculty noted that direct observation and not knowing a resident’s previous performance information was challenging. Residents wanted to see faculty initiate assessments and improved transparency around progress and promotion decisions. Discussion The results provided insight into how well the intended outcomes had been achieved as well as areas for improvement. Proposed adaptations included a need for increased direct observation and exploration of faculty accessing residents’ previous performance information. Education was provided on the performance expectations of residents and how progress and promotion decisions are made. As well, “flex blocks” were created to help residents customize their training experience to meet their learning needs. The results of this study can be used to inform and guide implementation and adaptations in other programs and institutions.
Les enseignants jouent un role central dans le perfectionnement des competences des residents en ... more Les enseignants jouent un role central dans le perfectionnement des competences des residents en donnant de la retroaction a l’appui de l’apprentissage. Le present article propose un bref outil, accessible en anglais dans CFPlus [*][1], qui sert a revoir la retroaction formative exprime
Objective To pilot a survey of family medicine residents entering residency, describing their exp... more Objective To pilot a survey of family medicine residents entering residency, describing their exposure to family medicine and their perspectives related to their future intentions to practise family medicine, in order to inform curriculum planners; and to test the methodology, feasibility, and utility of delivering a longitudinal survey to multiple residency programs. Design Pilot study using surveys. Setting Five Canadian residency programs. Participants A total of 454 first-year family medicine residents were surveyed. Main outcome measures Residents’ previous exposure to family medicine, perspectives on family medicine, and future practice intentions. Results Overall, 70% of first-year residents surveyed responded (n = 317). Although only 5 residency programs participated, respondents included graduates from each of the medical schools in Canada, as well as international medical graduates. Among respondents, 92% felt positive or strongly positive about their choice to be family physicians. Most (73%) indicated they had strong or very strong exposure to family medicine in medical school, yet more than 40% had no or minimal exposure to key clinical domains of family medicine like palliative care, home care, and care of underserved groups. Similar responses were found about residents’ lack of intention to practise in these domains. Conclusion Exposure to clinical domains in family medicine could influence future practice intentions. Surveys at entrance to residency can help medical school and family medicine residency planners consider important learning experiences to include in training.
Academic Medicine, Jun 21, 2023
Residents and faculty have described a burden of assessment related to the implementation of comp... more Residents and faculty have described a burden of assessment related to the implementation of competency-based medical education (CBME), which may undermine its benefits. Although this concerning signal has been identified, little has been done to identify adaptations to address this problem. Grounded in an analysis of an early Canadian pan-institutional CBME adopter’s experience, this article describes postgraduate programs’ adaptations related to the challenges of assessment in CBME. From June 2019–September 2022, 8 residency programs underwent a standardized Rapid Evaluation guided by the Core Components Framework (CCF). Sixty interviews and 18 focus groups were held with invested partners. Transcripts were analyzed abductively using CCF, and ideal implementation was compared with enacted implementation. These findings were then shared back with program leaders, adaptations were subsequently developed, and technical reports were generated for each program. Researchers reviewed the technical reports to identify themes related to the burden of assessment with a subsequent focus on identifying adaptations across programs. Three themes were identified: (1) disparate mental models of assessment processes in CBME, (2) challenges in workplace-based assessment processes, and (3) challenges in performance review and decision making. Theme 1 included entrustment interpretation and lack of shared mindset for performance standards. Adaptations included revising entrustment scales, faculty development, and formalizing resident membership. Theme 2 involved direct observation, timeliness of assessment completion, and feedback quality. Adaptations included alternative assessment strategies beyond entrustable professional activity forms and proactive assessment planning. Theme 3 related to resident data monitoring and competence committee decision making. Adaptations included adding resident representatives to the competence committee and assessment platform enhancements. These adaptations represent responses to the concerning signal of significant burden of assessment within CBME being experienced broadly. The authors hope other programs may learn from their institution’s experience and navigate the CBME-related assessment burden their invested partners may be facing.
Problem Assessing the development and achievement of competence requires multiple formative and s... more Problem Assessing the development and achievement of competence requires multiple formative and summative assessment strategies and the coordinated efforts of trainees and faculty (who often serve in multiple roles, such as academic advisors, program directors, and competency committee members). Operationalizing programmatic assessment (PA) in competency-based medical education (CBME) requires comprehensive practice guidelines, written in accessible language with descriptions of stakeholder activities, to move assessment theory into practice and to help guide the trainees and faculty who enact PA. Approach Informed by the Appraisal of Guidelines for Research and Evaluation II (AGREE II) framework, the authors used a multiphase, multimethod approach to develop the CBME Programmatic Assessment Practice Guidelines (PA Guidelines). The 9 guidelines are organized by phases of assessment and include descriptions of stakeholder activities. A user guide provides a glossary of key terms and summarizes how the guidelines can be used by different stakeholder groups across postgraduate medical education (PGME) contexts. The 4 phases of guideline development, including internal stakeholder consultations and external expert review, occurred between August 2016 and March 2020. Outcomes Local stakeholders and external experts agreed that the PA Guidelines hold potential for guiding initial operationalization and ongoing refinement of PA in CBME by individual stakeholders, residency programs, and PGME institutions. Since July 2020, the PA Guidelines have been used at Queen’s University to inform faculty and resident development initiatives, including online CBME modules for faculty, workshops for academic advisors/competence committee members, and a guide that supports incoming residents’ transition to CBME. Next Steps Research exploring the use of the PA Guidelines and user guide in multiple programs and institutions will gather further evidence of their acceptability and utility for guiding operationalization of PA in different contexts.
Medical Education, 2020
Introduction: Competency based medical education (CBME) requires that educators structure assessm... more Introduction: Competency based medical education (CBME) requires that educators structure assessment of clinical competence using outcome frameworks. While these frameworks may serve some outcomes well (e.g., represent eventual practice), translating these into workplace-based assessment plans may undermine validity and therefore trustworthiness of assessment decisions due to a number of competing factors that may not always be visible or their impact known. Explored here is the translation process from outcome framework to formative and summative assessment plans in postgraduate medical education (PGME) in Canadian universities. Methods: We conducted a qualitative study involving in-depth semi-structured interviews with leaders of PGME programs involved in assessment and/or CBME implementation, with a focus on their assessment-based translational activities and evaluation strategies. Interviews were informed by Callon's theory of translation. Our analytical strategy involved directed content analysis, allowing us to be guided by Kane's validity framework but still participating in open coding and analytical memo-taking. We then engaged in axial coding to systematically explore themes across the dataset, various situations and our conceptual framework. Results: Twenty-four interviews were conducted involving 15 specialties across three universities. Our results suggest (a) using outcomes for assessment frameworks serves as necessary for good assessment but also as an incomplete construct; (b) there are a number of social and practical negotiations with competing factors that displace validity as a core influencer in assessment planning, including implementation, accreditation and technology; (c) validity exists as threatened, uncertain and assumed due to a number of unchecked assumptions and reliance on surrogates. Conclusions: Translational processes in CBME involve negotiating with numerous influencing actors and institutions that, from an assessment perspective, provide challenges for assessment scientists, institutions and educators to contend with. These processes are challenging validity as a core element of assessment designs. Educators must reconcile these influences when preparing for or structuring validity arguments.
MedEdPublish, 2017
This article was migrated. The article was not marked as recommended. Medical education in Canada... more This article was migrated. The article was not marked as recommended. Medical education in Canada is currently in a state of transition. In 2013, the Royal College of Physicians and Surgeons of Canada launched Competency by Design (CBD), an initiative which will see all specialty and subspecialty programs in Canada begin transitioning to competency-based medical education (CBME) by 2022. At Queen's University, we intend that beginning July 2017, residents entering any of our 29 postgraduate specialty programs will be integrated into CBME residency programs. This paper shares Queen's University's experience of an accelerated, institutional implementation of CBME in advance of the Royal College's competency by design (CBD) program.
Canadian Medical Education Journal
Competency-based medical education (CBME) curricula are becoming increasingly common in graduate ... more Competency-based medical education (CBME) curricula are becoming increasingly common in graduate medical education. Put simply, CBME is focused on educational outcomes, is independent of methods and time, and is composed of achievable competencies.1 In spite of widespread uptake, there remains much to learn about implementing CBME at the program level. Leveraging the collective experience of program leaders at Queen’s University, where CBME simultaneously launched across 29 specialty programs in 2017, this paper leverages change management theory to provide a short summary of how program leaders can navigate the successful preparation, launch, and initial implementation of CBME within their residency programs.
Journal of Evaluation in Clinical Practice
Canadian Medical Education Journal
The Royal College of Physicians and Surgeons of Canada (RCPSC) adopted a plan to transform, over ... more The Royal College of Physicians and Surgeons of Canada (RCPSC) adopted a plan to transform, over a seven-year horizon (2014-2021), residency education across all specialties to competency-based medical education (CBME) curriculum models. The RCPSC plan recommended implementing a more responsive and accountable training model with four discrete stages of training, explicit, specialty specific entrustable professional activities, with associated milestones, and a programmatic approach to assessment across residency education. Embracing this vision, the leadership at Queen’s University (in Kingston, Ontario, Canada) applied for and was granted special permission by the RCPSC to embark on an accelerated institutional path. Over a three-year period, Queen’s took CBME from concept to reality through the development and implementation of acomprehensive strategic plan. This perspective paper describes Queen’s University’s approach of creating a shared institutional vision, outlines the proc...
Canadian Journal of Ophthalmology
Abstract: This study examined the relationship between the quality of intra-group on-line collabo... more Abstract: This study examined the relationship between the quality of intra-group on-line collaboration among groups of undergraduate learners on the quality of products produced. The quality of on-line collaboration was assessed by the instructor and Teaching Assistant with the ...
Canadian family physician Medecin de famille canadien, 2018
To describe exiting family medicine (FM) residents' reported practice intentions after comple... more To describe exiting family medicine (FM) residents' reported practice intentions after completing a Triple C Competency-based Curriculum. The surveys were intended to capture residents' perceptions of FM, their perceptions of their competency-based training, and their intentions to practise FM. Entry (T1) and exit (T2) self-reported survey results were compared considering the influence of the curriculum change. Unmatched aggregate-level data were reviewed. The T1 survey was administered in the summer of 2012 and the T2 survey was administered in the spring of 2014. Six Canadian FM residency programs across 4 provinces in Canada (Alberta, Saskatchewan, Ontario, and Quebec). Overall, 341 entering FM residents in 2012 responded to the T1 survey and 325 exiting FM residents completing their residency programs in spring 2014 responded to the T2 survey. Self-reported data on FM residents' future practice intentions related to comprehensive care, providing care across clinical...
Canadian Family Physician, Apr 1, 2015
Objective To pilot a survey of family medicine residents entering residency, describing their exp... more Objective To pilot a survey of family medicine residents entering residency, describing their exposure to family medicine and their perspectives related to their future intentions to practise family medicine, in order to inform curriculum planners; and to test the methodology, feasibility, and utility of delivering a longitudinal survey to multiple residency programs.
Canadian family physician Médecin de famille canadien, 2016
Uploads
Papers by Laura April McEwen