Journal of Educational Evaluation for Health Professions, Nov 28, 2022
The study investigates the efficacy of new features introduced to the selection process for medic... more The study investigates the efficacy of new features introduced to the selection process for medical school at the University of New South Wales, Australia: (1) considering the relative ranks rather than scores of the Undergraduate Medicine and Health Sciences Admission Test and Australian Tertiary Admission Rank; (2) structured interview focusing on interpersonal interaction and concerns should the applicants become students; and (3) embracing interviewers' diverse perspectives. Methods: Data from 5 cohorts of students were analyzed, comparing outcomes of the second year in the medicine program of 4 cohorts of the old selection process and 1 of the new process. The main analysis comprised multiple linear regression models for predicting academic, clinical, and professional outcomes, by section tools and demographic variables. Results: Selection interview marks from the new interview (512 applicants, 2 interviewers each) were analyzed for inter-rater reliability, which identified a high level of agreement (kappa=0.639). No such analysis was possible for the old interview since it required interviewers to reach a consensus. Multivariate linear regression models utilizing outcomes for 5 cohorts (N=905) revealed that the new selection process was much more effective in predicting academic and clinical achievement in the program (R 2 =9.4%-17.8% vs. R 2 =1.5%-8.4%). Conclusion: The results suggest that the medical student selection process can be significantly enhanced by employing a non-compensatory selection algorithm; and using a structured interview focusing on interpersonal interaction and concerns should the applicants become students; as well as embracing interviewers' diverse perspectives.
An interactive computer-assisted learning (CAL) module on glomerulonephritis, previously identifi... more An interactive computer-assisted learning (CAL) module on glomerulonephritis, previously identified by fourth-year medical students as a difficult topic, was developed. The module comprised background material, case studies, graphics, animation, video and supporting quizzes with feedback. The impact of the module on student learning was evaluated by comparing the performance of two matched groups of students, only one of which accessed the CAL module, in an online assessment. The analysis examined the effect of having completed a clinical term in renal medicine. A significant improvement in the performance of those students who used the CAL module was demonstrated. Unexpectedly, completing a renal medicine term had no beneficial influence on students' performance. Students who used the CAL module perceived a significant decrease in the difficulty of the topic. It is concluded that the module is an effective learning tool, but important caveats are noted associated with using CAL modules in redesigned medical curricula.
Background: Radiology education is limited in undergraduate Medicine programs. Junior doctors mig... more Background: Radiology education is limited in undergraduate Medicine programs. Junior doctors might not have the necessary background to effectively order and interpret diagnostic imaging investigations. Furthermore, junior doctors are often time-poor, balancing clinical commitments with ongoing learning, leadership and teaching responsibilities. Previous studies have demonstrated the efficacy of radiology-themed online adaptive tutorials for senior medical students. Such adaptive tutorials might therefore be an efficient and effective form of radiology education for junior doctors. Methods: A randomised controlled crossover trial was performed to evaluate the impact of adaptive tutorials on learning the indications for, and interpretation of, basic imaging studies, compared with peer-reviewed web-based resources. Ninety-one volunteer junior doctors, comprising 53 postgraduate year 1 (PGY 1) and 38 postgraduate year 2 (PGY 2), were randomly allocated into two groups. In the first phase of the trial, focusing on head CT, one group accessed adaptive tutorials while the other received web-based resources. In the second phase of the trial, focusing on chest CT, the groups crossed over. Following each phase of the trial, participants completed exam-style online assessments. At the conclusion of the study, participants also completed an online questionnaire regarding perceived engagement and efficacy of each type of educational resource. Results: Junior doctors completed the adaptive tutorials significantly faster than the relevant web-based resources for both head CT and chest CT (p = 0.03 and < 0.01 respectively). Mean quiz scores were higher in the groups receiving adaptive tutorials on head CT and chest CT (86.4% vs 83.5 and 77.7% vs 75% respectively). However, in contrast to previous studies in senior medical students, these differences were not statistically significant. Participants reported higher engagement and perceived value of adaptive tutorials, compared with web-based resources.
Background: The COVID-19 pandemic and the consequent social distancing measures caused unpreceden... more Background: The COVID-19 pandemic and the consequent social distancing measures caused unprecedented disruption for medical and healthcare education. This study examined medical teachers' experience with emergency remote teaching during the pandemic and their acceptance of online teaching after the pandemic. Methods: In this sequential mixed methods study, online surveys were disseminated to teachers (n = 139) at two Asia-Pacific medical schools to evaluate their experience with emergency remote teaching during the pandemic. Subsequently, in-depth interviews were conducted with teachers from both institutions (n = 13). Each interviewee was classified into an adopter category based on Rogers' Diffusion of Innovations Theory. Interview transcripts were analyzed thematically, and the descriptive themes were mapped to broader themes partly based on the Technology Acceptance Model and these included: (i) perceived usefulness of online teaching, (ii) perceived ease of delivering online teaching, (iii) experience with institutional support and (iv) acceptance of online teaching after the pandemic. Results: Our participants described accounts of successes with their emergency remote teaching and difficulties they experienced. In general, most participants found it difficult to deliver clinical skills teaching remotely and manage large groups of students in synchronous online classes. With regards to institutional support, teachers with lower technological literacy required just-in-time technical support, while teachers who were innovative in their online teaching practices found that IT support alone could not fully address their needs. It was also found that teachers' acceptance of online teaching after the pandemic was influenced by their belief about the usefulness of online teaching. Conclusions: This study demonstrated that our participants managed to adapt to emergency remote teaching during this pandemic, and it also identified a myriad of drivers and blockers to online teaching adoption for medical teachers. It highlights the need for institutes to better support their teaching staff with diverse needs in their online teaching.
Rationale and Objectives: Radiology education is suited to delivery via e-learning which may be u... more Rationale and Objectives: Radiology education is suited to delivery via e-learning which may be used to fill gaps in knowledge and help prepare medical students for internship. There is limited evidence of effectiveness of adaptive tutorials, a form of e-learning in a senior medical student cohort. Materials and Methods: A randomized mixed methods crossover trial was performed to assess effectiveness of adaptive tutorials on engagement and understanding of appropriate use and interpretation of basic imaging studies. Eighty-one volunteer medical students from years 5 and 6 of a 6-year program were randomly allocated to one of two groups. In the first phase of the trial on head CT, one group received access to adaptive tutorials and the other to peer-reviewed web-based resources. A cross over was performed and the second phase of the trial addressing chest CT commenced. Examination style assessments were completed at the end of each phase. At the trial's conclusion, an online questionnaire was provided to evaluate student perceptions of engagement and efficacy of each educational resource. Results: Adaptive tutorial groups in both phases achieved higher mean scores than controls which were statistically significant in the first phase only. Students reported higher engagement and overall perceived value of the adaptive tutorials than controls. Conclusion: Adaptive tutorials are overwhelmingly supported by senior medical students. Questionnaire responses suggest the engaging nature of the tutorials efficiently aids participation and knowledge retention which is in principle supported by test results.
Background: Providing large numbers of undergraduate students in scientific disciplines with enga... more Background: Providing large numbers of undergraduate students in scientific disciplines with engaging, authentic laboratory experiences is important, but challenging. Virtual laboratories (vLABs) are a potential means to enable interactive learning experiences. A vLAB focusing on Western Blotting was developed and implemented in a 3rd year undergraduate Pathology course for science students to facilitate learning of technical molecular laboratory skills that are linked to development of diagnostic skills. Such skills are important for undergraduates in building a conceptual understanding of translation of laboratory techniques to changes in human biology due to disease. Methods: The Western Blotting vLAB was developed and deployed using the Adaptive eLearning Platform (AeLP) developed by Smart Sparrow (https://www.smartsparrow.com/). The vLAB was evaluated to assess students' perceptions of their laboratory skills relevant to the diagnosis of Muscular Dystrophy. A blended learning rotation model was applied in which wet laboratory and vLAB environments for Western Blotting were both delivered to three consecutive cohorts of 3rd year science undergraduates undertaking a Muscle Diseases practical class. Evaluation questionnaires were administered at the completion of the practical classes. Results: Students indicated in online questionnaires that the Western Blotting vLAB was at least equivalent to the real lab in their perceived development of concepts, laboratory skills and diagnosis of disease. Conclusions: vLABs have great potential for improving students' development of diagnostic skills. Further studies are required to determine the impact of vLABs on student learning.
Focus on health professional education : a multi-disciplinary journal, Jul 26, 2018
Introduction: Medical education has transitioned to require more active student learning, includi... more Introduction: Medical education has transitioned to require more active student learning, including self-directed and collaborative approaches. Although there are many advantages to collaborative small-group environments, learning in this setting is not always effective. One potential factor is that behaviours that are key to the success of small-group collaborative learning in medical education are yet to be clearly defined. In this study, we aimed to identify and prioritise behaviours that are relevant to individual students effective collaborative learning in small groups. Method: A two-round modified-Delphi approach was used to identify student behaviours that positively influenced the success of small-group collaborative-learning environments. In Round 1, the panel members were asked to rate the impact of 15 student behaviours on two subscales: (1) enhancing the quality of learning and (2) fostering a collaborative environment. The top 10 behaviours identified were utilised for Round 2, in which the panel members were asked to rank the behaviours based on the magnitude of their positive impact on students' quality of learning. Results: Collaborative group behaviours were prioritised in Round 2, and the top six behaviours were selected for the design of a user-friendly educational intervention for medical students. The key behaviours identified were preparation by the student, involvement in discussion, engagement in learning, listening skills, sharing views and voicing opinions, reflecting on feedback and responding appropriately.
Background: Modern clinical practice increasingly relies on collaborative and team-based approach... more Background: Modern clinical practice increasingly relies on collaborative and team-based approaches to care. Regulatory bodies in medical education emphasise the need to develop collaboration and teamwork competencies and highlight the need to do so from an early stage of medical training. In undergraduate medical education, the focus is usually on collaborative learning, associated with feedback and reflection on this learning This article describes a novel educational instrument, the Collaborative Learning Development Exercise (CLeD-EX), which aims to foster the development of key collaborative learning competencies in medical students. In this article we report on the effectiveness, feasibility and educational impact of the CLeD-EX. Methods: In this study, the "educational design research" framework was used to develop, implement and evaluate the CLeD-EX. This involved adopting a systematic approach towards designing a creative and innovative instrument which would help solve a real-world challenge in developing collaborative learning skills. The systematic approach involved a qualitative exploration of key collaborative learning behaviours which are influential in effective collaborative learning contexts. The identified competencies were employed in the design of the CLeD-EX. The design of the CLeD-EX included features to facilitate structured feedback by tutors to students, complemented by self-evaluation and reflection. The CLeD-EX was field-tested with volunteer junior medical students, using a controlled pre-test post-test design. Analysis of the completed CLeD-EX forms, self-perception surveys (i.e. pre-test and post-test surveys) and analyses of reflective reports were used to explore the educational impact of CLeD-EX, as well as its utility and practicality. Results: After using the CLeD-EX, students showed a significant improvement in critical thinking and group process as measured by a previously validated instrument. Both students and tutors recognised CLeD-EX as an effective instrument, especially as a structured basis for giving and receiving feedback and for completing the feedback loop. CLeD-EX was also found to be feasible, practical and focused, while promoting learning and effective interactions in small group learning.
Rationale and Objectives: Radiology education is suited to delivery via e-learning which may be u... more Rationale and Objectives: Radiology education is suited to delivery via e-learning which may be used to fill gaps in knowledge and help prepare medical students for internship. There is limited evidence of effectiveness of adaptive tutorials, a form of e-learning in a senior medical student cohort. Materials and Methods: A randomized mixed methods crossover trial was performed to assess effectiveness of adaptive tutorials on engagement and understanding of appropriate use and interpretation of basic imaging studies. Eighty-one volunteer medical students from years 5 and 6 of a 6-year program were randomly allocated to one of two groups. In the first phase of the trial on head CT, one group received access to adaptive tutorials and the other to peer-reviewed web-based resources. A cross over was performed and the second phase of the trial addressing chest CT commenced. Examination style assessments were completed at the end of each phase. At the trial's conclusion, an online questionnaire was provided to evaluate student perceptions of engagement and efficacy of each educational resource. Results: Adaptive tutorial groups in both phases achieved higher mean scores than controls which were statistically significant in the first phase only. Students reported higher engagement and overall perceived value of the adaptive tutorials than controls. Conclusion: Adaptive tutorials are overwhelmingly supported by senior medical students. Questionnaire responses suggest the engaging nature of the tutorials efficiently aids participation and knowledge retention which is in principle supported by test results.
Journal of Educational Evaluation for Health Professions
Purpose: This study evaluated the validity of student feedback derived from Medicine Student Expe... more Purpose: This study evaluated the validity of student feedback derived from Medicine Student Experience Questionnaire (MedSEQ), as well as the predictors of students’ satisfaction in the Medicine program.Methods: Data from MedSEQ applying to the University of New South Wales Medicine program in 2017, 2019, and 2021 were analyzed. Confirmatory factor analysis (CFA) and Cronbach’s α were used to assess the construct validity and reliability of MedSEQ respectively. Hierarchical multiple linear regressions were used to identify the factors that most impact students’ overall satisfaction with the program.Results: A total of 1,719 students (34.50%) responded to MedSEQ. CFA showed good fit indices (root mean square error of approximation=0.051; comparative fit index=0.939; chi-square/degrees of freedom=6.429). All factors yielded good (α>0.7) or very good (α>0.8) levels of reliability, except the “online resources” factor, which had acceptable reliability (α=0.687). A multiple linear...
Background: Most assessments in health professions education consist of knowledge-based examinati... more Background: Most assessments in health professions education consist of knowledge-based examinations as well as practical and clinical examinations. Among the most challenging aspects of clinical assessments is decision making related to borderline grades assigned by examiners. Borderline grades are commonly used by examiners when they do not have sufficient information to make clear pass/fail decisions. The interpretation of these borderline grades is rarely discussed in the literature. This study reports the application of the Objective Borderline Method (version 2, henceforth: OBM2) to a high stakes Objective Structured Clinical Examination undertaken at the end of the final year of a Medicine program in Australia. Methods: The OBM2 uses all examination data to reclassify borderline grades as either pass or fail. Factor analysis was used to estimate the suitability of data for application of OBM2. Student\u27s t-tests, utilising bootstrapping, were used to compare the OBM2 with \...
The ‘Phase 1 Prac Project’ (P1PP) and ‘Phase 2 Prac Project’ (P2PP) were initiatives developed as... more The ‘Phase 1 Prac Project’ (P1PP) and ‘Phase 2 Prac Project’ (P2PP) were initiatives developed as part of the BEST Network (www.best.edu.au) to systematically develop online support materials for biomedical science practicals held in Phase 1 (Years 1 and 2) and Phase 2 (Years 3 and 4) Medicine. Systematic production of academic, peer-evaluated online practical elements (OPEs), focussed on learning bottlenecks and threshold concepts in these practicals, were produced using the Smart Sparrow Adaptive eLearning Platform (AeLP). The project delivered 42 re-useable, editable OPEs, tested by students and refined using available analytical data and academic peer review to ensure student satisfaction and content quality. These OPEs will benefit future blended learning activities offered to biomedical university students globally
From a perspective of social practice, learning is a socially constituted practice that is imbued... more From a perspective of social practice, learning is a socially constituted practice that is imbued with socio-culturally significant meanings and shaped by the values and norms shared within a community of learners. This focus group study examines the role of e-learning technologies in mediating the social practice of learning among coursework university students in Sydney, Australia. Qualitative data from this study elucidated the social context and significant socio-cultural meanings that make learning technologies highly valued tools for students. Notably, students valued the interaction with others facilitated by learning technologies that enable the sharing and building of knowledge, because learning is most effective and engaging through community participation. Further, they articulated a desire for learning opportunities that are instantaneous, convenient and time-saving, against the backdrop of a busy and time-poor metropolitan lifestyle that requires the balancing of study ...
Background: Medical schools apply a range of selection methods to ensure that admitted students s... more Background: Medical schools apply a range of selection methods to ensure that admitted students succeed in the program. In Australia, selection tools typically include measures of academic achievement (e.g. the Australian Tertiary Admission Rank-ATAR) and aptitude tests (e.g. the Undergraduate Medicine and Health Sciences Admissions Test-UMAT). These are most commonly used to determine which applicants are invited for additional selection processes, such as interviews. However, no previous study has examined the efficacy of the first part of the selection process. In particular, are compensatory or non-compensatory approaches more effective in evaluating the outcomes of cognitive and aptitude tests, and do they affect the demographics of students selected for interview? Methods: This study utilised data from consecutive cohorts of mainstream domestic students who applied to enter the UNSW Medicine program between 2013 to 2018. A compensatory ranked selection model was compared with a non-compensatory ranked model. Initially, ATAR marks and UMAT scores for each applicant were ranked within each cohort. In the compensatory model, the mean of the ATAR and UMAT ranks were used to determine the outcome. In the non-compensatory model, the lowest rank of ATAR and UMAT determined the outcome for each applicant. The impact of each model on the gender and socioeconomic status of applicants selected to interview was evaluated across all cohorts. Results: The non-compensatory ranked selection model resulted in substantially higher ATAR and UMAT thresholds for invitation to interview, with no significant effect on the socioeconomic status of the selected applicants. Conclusions: These results are important, demonstrating that it is possible to raise the academic threshold for selection to medicine without having any negative impact on applicants from low socioeconomic backgrounds. Overall, the evidence gathered in this study suggests that a non-compensatory model is preferable for selecting applicants for medical student selection interview.
Background: Modern clinical practice increasingly relies on collaborative and team-based approach... more Background: Modern clinical practice increasingly relies on collaborative and team-based approaches to care. Regulatory bodies in medical education emphasise the need to develop collaborative and teamwork competencies and highlight the need to do so from an early stage of medical training. During undergraduate medical education the focus is usually to facilitate collaborative learning and increase feedback and reflection on this learning. This article describes a novel educational instrument, the Collaborative Learning Development Exercise (CLeD-EX), which aims to foster the development of key collaborative learning competencies in medical students. Methods: In this study, the “educational design research” framework was used to develop, implement and evaluate the CLeD-EX. This involved adopting a systematic approach towards designing a creative and innovative instrument which would help solve a real-world challenge in developing collaborative learning skills. The systematic approach...
Concept and knowledge maps have been shown to improve students' learning by emphasising meaningfu... more Concept and knowledge maps have been shown to improve students' learning by emphasising meaningful relationships between phenomena. A user-friendly online tool that enables assessment of students' maps with automated feedback might therefore have significant benefits for learning. For that purpose, we developed an online software platform known as Knowledge Maps. Two pilot studies were performed to evaluate the usability and efficacy of Knowledge Maps. Study A demonstrated significantly improved perceptions of learning after using Knowledge Maps to learn pathology. Study B showed significant improvement between pre-test and post-test scores in an anatomy course. These preliminary studies indicate that this software is readily accepted and may have potential benefits for learning.
This article was migrated. The article was not marked as recommended. With the increasing complex... more This article was migrated. The article was not marked as recommended. With the increasing complexity of healthcare delivery and patient case mix, and the aging population, health professionals are expected to function more often and more effectively as members of multidisciplinary teams. Competency based medical education (CBME) advocates that the professional development of doctors should be fostered from the start of medical school through to medical practice and with continuing professional development. Strategies for developing teamwork among health professionals and students range from minimal, implicit to explicit (Earnest et al., 2017). Considering this continuum of medical education and the spectrum of educational strategies, we propose a framework for the programmatic approach which can be used in developing competence in 'collaborative clinical practice'.
Introduction Narratives (as opposed to stories) can assess multiple facets of the same problem th... more Introduction Narratives (as opposed to stories) can assess multiple facets of the same problem through the viewpoints of different characters. Methods Narratives related to three cancer patients, from diagnosis to cure or death, were used to teach seven cancer-related themes in a Cancer Pathology course offered to third-year medical science and science (college) undergraduates. Results The majority of students preferred narrative-based learning compared with traditional learning methods because they felt that it improved their learning experience and retention of information. Conclusion Narrative-based learning may improve the learning experience of students by contextualizing complex concepts and highlighting real-world applications of knowledge.
In higher education, most assessments or examinations comprise either multiple-choice items or op... more In higher education, most assessments or examinations comprise either multiple-choice items or open-ended questions such as modified essay questions (MEQs). Online concept and knowledge maps are potential tools for assessment, which might emphasize meaningful, integrated understanding of phenomena. We developed an online knowledge-mapping assessment tool, which provides automated feedback on student-submitted maps. We conducted a pilot study to investigate the potential utility of online knowledge mapping as a tool for automated assessment by comparing the scores generated by the software with manual grading of a MEQ on the same topic for a cohort of first-year medical students. In addition, an online questionnaire was used to gather students' perceptions of the tool. Map items were highly discriminating between students of differing knowledge of the topic overall. Regression analysis showed a significant correlation between map scores and MEQ scores, and responses to the questi...
Journal of Educational Evaluation for Health Professions, Nov 28, 2022
The study investigates the efficacy of new features introduced to the selection process for medic... more The study investigates the efficacy of new features introduced to the selection process for medical school at the University of New South Wales, Australia: (1) considering the relative ranks rather than scores of the Undergraduate Medicine and Health Sciences Admission Test and Australian Tertiary Admission Rank; (2) structured interview focusing on interpersonal interaction and concerns should the applicants become students; and (3) embracing interviewers' diverse perspectives. Methods: Data from 5 cohorts of students were analyzed, comparing outcomes of the second year in the medicine program of 4 cohorts of the old selection process and 1 of the new process. The main analysis comprised multiple linear regression models for predicting academic, clinical, and professional outcomes, by section tools and demographic variables. Results: Selection interview marks from the new interview (512 applicants, 2 interviewers each) were analyzed for inter-rater reliability, which identified a high level of agreement (kappa=0.639). No such analysis was possible for the old interview since it required interviewers to reach a consensus. Multivariate linear regression models utilizing outcomes for 5 cohorts (N=905) revealed that the new selection process was much more effective in predicting academic and clinical achievement in the program (R 2 =9.4%-17.8% vs. R 2 =1.5%-8.4%). Conclusion: The results suggest that the medical student selection process can be significantly enhanced by employing a non-compensatory selection algorithm; and using a structured interview focusing on interpersonal interaction and concerns should the applicants become students; as well as embracing interviewers' diverse perspectives.
An interactive computer-assisted learning (CAL) module on glomerulonephritis, previously identifi... more An interactive computer-assisted learning (CAL) module on glomerulonephritis, previously identified by fourth-year medical students as a difficult topic, was developed. The module comprised background material, case studies, graphics, animation, video and supporting quizzes with feedback. The impact of the module on student learning was evaluated by comparing the performance of two matched groups of students, only one of which accessed the CAL module, in an online assessment. The analysis examined the effect of having completed a clinical term in renal medicine. A significant improvement in the performance of those students who used the CAL module was demonstrated. Unexpectedly, completing a renal medicine term had no beneficial influence on students&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39; performance. Students who used the CAL module perceived a significant decrease in the difficulty of the topic. It is concluded that the module is an effective learning tool, but important caveats are noted associated with using CAL modules in redesigned medical curricula.
Background: Radiology education is limited in undergraduate Medicine programs. Junior doctors mig... more Background: Radiology education is limited in undergraduate Medicine programs. Junior doctors might not have the necessary background to effectively order and interpret diagnostic imaging investigations. Furthermore, junior doctors are often time-poor, balancing clinical commitments with ongoing learning, leadership and teaching responsibilities. Previous studies have demonstrated the efficacy of radiology-themed online adaptive tutorials for senior medical students. Such adaptive tutorials might therefore be an efficient and effective form of radiology education for junior doctors. Methods: A randomised controlled crossover trial was performed to evaluate the impact of adaptive tutorials on learning the indications for, and interpretation of, basic imaging studies, compared with peer-reviewed web-based resources. Ninety-one volunteer junior doctors, comprising 53 postgraduate year 1 (PGY 1) and 38 postgraduate year 2 (PGY 2), were randomly allocated into two groups. In the first phase of the trial, focusing on head CT, one group accessed adaptive tutorials while the other received web-based resources. In the second phase of the trial, focusing on chest CT, the groups crossed over. Following each phase of the trial, participants completed exam-style online assessments. At the conclusion of the study, participants also completed an online questionnaire regarding perceived engagement and efficacy of each type of educational resource. Results: Junior doctors completed the adaptive tutorials significantly faster than the relevant web-based resources for both head CT and chest CT (p = 0.03 and < 0.01 respectively). Mean quiz scores were higher in the groups receiving adaptive tutorials on head CT and chest CT (86.4% vs 83.5 and 77.7% vs 75% respectively). However, in contrast to previous studies in senior medical students, these differences were not statistically significant. Participants reported higher engagement and perceived value of adaptive tutorials, compared with web-based resources.
Background: The COVID-19 pandemic and the consequent social distancing measures caused unpreceden... more Background: The COVID-19 pandemic and the consequent social distancing measures caused unprecedented disruption for medical and healthcare education. This study examined medical teachers' experience with emergency remote teaching during the pandemic and their acceptance of online teaching after the pandemic. Methods: In this sequential mixed methods study, online surveys were disseminated to teachers (n = 139) at two Asia-Pacific medical schools to evaluate their experience with emergency remote teaching during the pandemic. Subsequently, in-depth interviews were conducted with teachers from both institutions (n = 13). Each interviewee was classified into an adopter category based on Rogers' Diffusion of Innovations Theory. Interview transcripts were analyzed thematically, and the descriptive themes were mapped to broader themes partly based on the Technology Acceptance Model and these included: (i) perceived usefulness of online teaching, (ii) perceived ease of delivering online teaching, (iii) experience with institutional support and (iv) acceptance of online teaching after the pandemic. Results: Our participants described accounts of successes with their emergency remote teaching and difficulties they experienced. In general, most participants found it difficult to deliver clinical skills teaching remotely and manage large groups of students in synchronous online classes. With regards to institutional support, teachers with lower technological literacy required just-in-time technical support, while teachers who were innovative in their online teaching practices found that IT support alone could not fully address their needs. It was also found that teachers' acceptance of online teaching after the pandemic was influenced by their belief about the usefulness of online teaching. Conclusions: This study demonstrated that our participants managed to adapt to emergency remote teaching during this pandemic, and it also identified a myriad of drivers and blockers to online teaching adoption for medical teachers. It highlights the need for institutes to better support their teaching staff with diverse needs in their online teaching.
Rationale and Objectives: Radiology education is suited to delivery via e-learning which may be u... more Rationale and Objectives: Radiology education is suited to delivery via e-learning which may be used to fill gaps in knowledge and help prepare medical students for internship. There is limited evidence of effectiveness of adaptive tutorials, a form of e-learning in a senior medical student cohort. Materials and Methods: A randomized mixed methods crossover trial was performed to assess effectiveness of adaptive tutorials on engagement and understanding of appropriate use and interpretation of basic imaging studies. Eighty-one volunteer medical students from years 5 and 6 of a 6-year program were randomly allocated to one of two groups. In the first phase of the trial on head CT, one group received access to adaptive tutorials and the other to peer-reviewed web-based resources. A cross over was performed and the second phase of the trial addressing chest CT commenced. Examination style assessments were completed at the end of each phase. At the trial's conclusion, an online questionnaire was provided to evaluate student perceptions of engagement and efficacy of each educational resource. Results: Adaptive tutorial groups in both phases achieved higher mean scores than controls which were statistically significant in the first phase only. Students reported higher engagement and overall perceived value of the adaptive tutorials than controls. Conclusion: Adaptive tutorials are overwhelmingly supported by senior medical students. Questionnaire responses suggest the engaging nature of the tutorials efficiently aids participation and knowledge retention which is in principle supported by test results.
Background: Providing large numbers of undergraduate students in scientific disciplines with enga... more Background: Providing large numbers of undergraduate students in scientific disciplines with engaging, authentic laboratory experiences is important, but challenging. Virtual laboratories (vLABs) are a potential means to enable interactive learning experiences. A vLAB focusing on Western Blotting was developed and implemented in a 3rd year undergraduate Pathology course for science students to facilitate learning of technical molecular laboratory skills that are linked to development of diagnostic skills. Such skills are important for undergraduates in building a conceptual understanding of translation of laboratory techniques to changes in human biology due to disease. Methods: The Western Blotting vLAB was developed and deployed using the Adaptive eLearning Platform (AeLP) developed by Smart Sparrow (https://www.smartsparrow.com/). The vLAB was evaluated to assess students' perceptions of their laboratory skills relevant to the diagnosis of Muscular Dystrophy. A blended learning rotation model was applied in which wet laboratory and vLAB environments for Western Blotting were both delivered to three consecutive cohorts of 3rd year science undergraduates undertaking a Muscle Diseases practical class. Evaluation questionnaires were administered at the completion of the practical classes. Results: Students indicated in online questionnaires that the Western Blotting vLAB was at least equivalent to the real lab in their perceived development of concepts, laboratory skills and diagnosis of disease. Conclusions: vLABs have great potential for improving students' development of diagnostic skills. Further studies are required to determine the impact of vLABs on student learning.
Focus on health professional education : a multi-disciplinary journal, Jul 26, 2018
Introduction: Medical education has transitioned to require more active student learning, includi... more Introduction: Medical education has transitioned to require more active student learning, including self-directed and collaborative approaches. Although there are many advantages to collaborative small-group environments, learning in this setting is not always effective. One potential factor is that behaviours that are key to the success of small-group collaborative learning in medical education are yet to be clearly defined. In this study, we aimed to identify and prioritise behaviours that are relevant to individual students effective collaborative learning in small groups. Method: A two-round modified-Delphi approach was used to identify student behaviours that positively influenced the success of small-group collaborative-learning environments. In Round 1, the panel members were asked to rate the impact of 15 student behaviours on two subscales: (1) enhancing the quality of learning and (2) fostering a collaborative environment. The top 10 behaviours identified were utilised for Round 2, in which the panel members were asked to rank the behaviours based on the magnitude of their positive impact on students' quality of learning. Results: Collaborative group behaviours were prioritised in Round 2, and the top six behaviours were selected for the design of a user-friendly educational intervention for medical students. The key behaviours identified were preparation by the student, involvement in discussion, engagement in learning, listening skills, sharing views and voicing opinions, reflecting on feedback and responding appropriately.
Background: Modern clinical practice increasingly relies on collaborative and team-based approach... more Background: Modern clinical practice increasingly relies on collaborative and team-based approaches to care. Regulatory bodies in medical education emphasise the need to develop collaboration and teamwork competencies and highlight the need to do so from an early stage of medical training. In undergraduate medical education, the focus is usually on collaborative learning, associated with feedback and reflection on this learning This article describes a novel educational instrument, the Collaborative Learning Development Exercise (CLeD-EX), which aims to foster the development of key collaborative learning competencies in medical students. In this article we report on the effectiveness, feasibility and educational impact of the CLeD-EX. Methods: In this study, the "educational design research" framework was used to develop, implement and evaluate the CLeD-EX. This involved adopting a systematic approach towards designing a creative and innovative instrument which would help solve a real-world challenge in developing collaborative learning skills. The systematic approach involved a qualitative exploration of key collaborative learning behaviours which are influential in effective collaborative learning contexts. The identified competencies were employed in the design of the CLeD-EX. The design of the CLeD-EX included features to facilitate structured feedback by tutors to students, complemented by self-evaluation and reflection. The CLeD-EX was field-tested with volunteer junior medical students, using a controlled pre-test post-test design. Analysis of the completed CLeD-EX forms, self-perception surveys (i.e. pre-test and post-test surveys) and analyses of reflective reports were used to explore the educational impact of CLeD-EX, as well as its utility and practicality. Results: After using the CLeD-EX, students showed a significant improvement in critical thinking and group process as measured by a previously validated instrument. Both students and tutors recognised CLeD-EX as an effective instrument, especially as a structured basis for giving and receiving feedback and for completing the feedback loop. CLeD-EX was also found to be feasible, practical and focused, while promoting learning and effective interactions in small group learning.
Rationale and Objectives: Radiology education is suited to delivery via e-learning which may be u... more Rationale and Objectives: Radiology education is suited to delivery via e-learning which may be used to fill gaps in knowledge and help prepare medical students for internship. There is limited evidence of effectiveness of adaptive tutorials, a form of e-learning in a senior medical student cohort. Materials and Methods: A randomized mixed methods crossover trial was performed to assess effectiveness of adaptive tutorials on engagement and understanding of appropriate use and interpretation of basic imaging studies. Eighty-one volunteer medical students from years 5 and 6 of a 6-year program were randomly allocated to one of two groups. In the first phase of the trial on head CT, one group received access to adaptive tutorials and the other to peer-reviewed web-based resources. A cross over was performed and the second phase of the trial addressing chest CT commenced. Examination style assessments were completed at the end of each phase. At the trial's conclusion, an online questionnaire was provided to evaluate student perceptions of engagement and efficacy of each educational resource. Results: Adaptive tutorial groups in both phases achieved higher mean scores than controls which were statistically significant in the first phase only. Students reported higher engagement and overall perceived value of the adaptive tutorials than controls. Conclusion: Adaptive tutorials are overwhelmingly supported by senior medical students. Questionnaire responses suggest the engaging nature of the tutorials efficiently aids participation and knowledge retention which is in principle supported by test results.
Journal of Educational Evaluation for Health Professions
Purpose: This study evaluated the validity of student feedback derived from Medicine Student Expe... more Purpose: This study evaluated the validity of student feedback derived from Medicine Student Experience Questionnaire (MedSEQ), as well as the predictors of students’ satisfaction in the Medicine program.Methods: Data from MedSEQ applying to the University of New South Wales Medicine program in 2017, 2019, and 2021 were analyzed. Confirmatory factor analysis (CFA) and Cronbach’s α were used to assess the construct validity and reliability of MedSEQ respectively. Hierarchical multiple linear regressions were used to identify the factors that most impact students’ overall satisfaction with the program.Results: A total of 1,719 students (34.50%) responded to MedSEQ. CFA showed good fit indices (root mean square error of approximation=0.051; comparative fit index=0.939; chi-square/degrees of freedom=6.429). All factors yielded good (α>0.7) or very good (α>0.8) levels of reliability, except the “online resources” factor, which had acceptable reliability (α=0.687). A multiple linear...
Background: Most assessments in health professions education consist of knowledge-based examinati... more Background: Most assessments in health professions education consist of knowledge-based examinations as well as practical and clinical examinations. Among the most challenging aspects of clinical assessments is decision making related to borderline grades assigned by examiners. Borderline grades are commonly used by examiners when they do not have sufficient information to make clear pass/fail decisions. The interpretation of these borderline grades is rarely discussed in the literature. This study reports the application of the Objective Borderline Method (version 2, henceforth: OBM2) to a high stakes Objective Structured Clinical Examination undertaken at the end of the final year of a Medicine program in Australia. Methods: The OBM2 uses all examination data to reclassify borderline grades as either pass or fail. Factor analysis was used to estimate the suitability of data for application of OBM2. Student\u27s t-tests, utilising bootstrapping, were used to compare the OBM2 with \...
The ‘Phase 1 Prac Project’ (P1PP) and ‘Phase 2 Prac Project’ (P2PP) were initiatives developed as... more The ‘Phase 1 Prac Project’ (P1PP) and ‘Phase 2 Prac Project’ (P2PP) were initiatives developed as part of the BEST Network (www.best.edu.au) to systematically develop online support materials for biomedical science practicals held in Phase 1 (Years 1 and 2) and Phase 2 (Years 3 and 4) Medicine. Systematic production of academic, peer-evaluated online practical elements (OPEs), focussed on learning bottlenecks and threshold concepts in these practicals, were produced using the Smart Sparrow Adaptive eLearning Platform (AeLP). The project delivered 42 re-useable, editable OPEs, tested by students and refined using available analytical data and academic peer review to ensure student satisfaction and content quality. These OPEs will benefit future blended learning activities offered to biomedical university students globally
From a perspective of social practice, learning is a socially constituted practice that is imbued... more From a perspective of social practice, learning is a socially constituted practice that is imbued with socio-culturally significant meanings and shaped by the values and norms shared within a community of learners. This focus group study examines the role of e-learning technologies in mediating the social practice of learning among coursework university students in Sydney, Australia. Qualitative data from this study elucidated the social context and significant socio-cultural meanings that make learning technologies highly valued tools for students. Notably, students valued the interaction with others facilitated by learning technologies that enable the sharing and building of knowledge, because learning is most effective and engaging through community participation. Further, they articulated a desire for learning opportunities that are instantaneous, convenient and time-saving, against the backdrop of a busy and time-poor metropolitan lifestyle that requires the balancing of study ...
Background: Medical schools apply a range of selection methods to ensure that admitted students s... more Background: Medical schools apply a range of selection methods to ensure that admitted students succeed in the program. In Australia, selection tools typically include measures of academic achievement (e.g. the Australian Tertiary Admission Rank-ATAR) and aptitude tests (e.g. the Undergraduate Medicine and Health Sciences Admissions Test-UMAT). These are most commonly used to determine which applicants are invited for additional selection processes, such as interviews. However, no previous study has examined the efficacy of the first part of the selection process. In particular, are compensatory or non-compensatory approaches more effective in evaluating the outcomes of cognitive and aptitude tests, and do they affect the demographics of students selected for interview? Methods: This study utilised data from consecutive cohorts of mainstream domestic students who applied to enter the UNSW Medicine program between 2013 to 2018. A compensatory ranked selection model was compared with a non-compensatory ranked model. Initially, ATAR marks and UMAT scores for each applicant were ranked within each cohort. In the compensatory model, the mean of the ATAR and UMAT ranks were used to determine the outcome. In the non-compensatory model, the lowest rank of ATAR and UMAT determined the outcome for each applicant. The impact of each model on the gender and socioeconomic status of applicants selected to interview was evaluated across all cohorts. Results: The non-compensatory ranked selection model resulted in substantially higher ATAR and UMAT thresholds for invitation to interview, with no significant effect on the socioeconomic status of the selected applicants. Conclusions: These results are important, demonstrating that it is possible to raise the academic threshold for selection to medicine without having any negative impact on applicants from low socioeconomic backgrounds. Overall, the evidence gathered in this study suggests that a non-compensatory model is preferable for selecting applicants for medical student selection interview.
Background: Modern clinical practice increasingly relies on collaborative and team-based approach... more Background: Modern clinical practice increasingly relies on collaborative and team-based approaches to care. Regulatory bodies in medical education emphasise the need to develop collaborative and teamwork competencies and highlight the need to do so from an early stage of medical training. During undergraduate medical education the focus is usually to facilitate collaborative learning and increase feedback and reflection on this learning. This article describes a novel educational instrument, the Collaborative Learning Development Exercise (CLeD-EX), which aims to foster the development of key collaborative learning competencies in medical students. Methods: In this study, the “educational design research” framework was used to develop, implement and evaluate the CLeD-EX. This involved adopting a systematic approach towards designing a creative and innovative instrument which would help solve a real-world challenge in developing collaborative learning skills. The systematic approach...
Concept and knowledge maps have been shown to improve students' learning by emphasising meaningfu... more Concept and knowledge maps have been shown to improve students' learning by emphasising meaningful relationships between phenomena. A user-friendly online tool that enables assessment of students' maps with automated feedback might therefore have significant benefits for learning. For that purpose, we developed an online software platform known as Knowledge Maps. Two pilot studies were performed to evaluate the usability and efficacy of Knowledge Maps. Study A demonstrated significantly improved perceptions of learning after using Knowledge Maps to learn pathology. Study B showed significant improvement between pre-test and post-test scores in an anatomy course. These preliminary studies indicate that this software is readily accepted and may have potential benefits for learning.
This article was migrated. The article was not marked as recommended. With the increasing complex... more This article was migrated. The article was not marked as recommended. With the increasing complexity of healthcare delivery and patient case mix, and the aging population, health professionals are expected to function more often and more effectively as members of multidisciplinary teams. Competency based medical education (CBME) advocates that the professional development of doctors should be fostered from the start of medical school through to medical practice and with continuing professional development. Strategies for developing teamwork among health professionals and students range from minimal, implicit to explicit (Earnest et al., 2017). Considering this continuum of medical education and the spectrum of educational strategies, we propose a framework for the programmatic approach which can be used in developing competence in 'collaborative clinical practice'.
Introduction Narratives (as opposed to stories) can assess multiple facets of the same problem th... more Introduction Narratives (as opposed to stories) can assess multiple facets of the same problem through the viewpoints of different characters. Methods Narratives related to three cancer patients, from diagnosis to cure or death, were used to teach seven cancer-related themes in a Cancer Pathology course offered to third-year medical science and science (college) undergraduates. Results The majority of students preferred narrative-based learning compared with traditional learning methods because they felt that it improved their learning experience and retention of information. Conclusion Narrative-based learning may improve the learning experience of students by contextualizing complex concepts and highlighting real-world applications of knowledge.
In higher education, most assessments or examinations comprise either multiple-choice items or op... more In higher education, most assessments or examinations comprise either multiple-choice items or open-ended questions such as modified essay questions (MEQs). Online concept and knowledge maps are potential tools for assessment, which might emphasize meaningful, integrated understanding of phenomena. We developed an online knowledge-mapping assessment tool, which provides automated feedback on student-submitted maps. We conducted a pilot study to investigate the potential utility of online knowledge mapping as a tool for automated assessment by comparing the scores generated by the software with manual grading of a MEQ on the same topic for a cohort of first-year medical students. In addition, an online questionnaire was used to gather students' perceptions of the tool. Map items were highly discriminating between students of differing knowledge of the topic overall. Regression analysis showed a significant correlation between map scores and MEQ scores, and responses to the questi...
Uploads
Papers by Gary Velan