Background: Professional self-identity is a 'state of mind' -identifying one's-self as a member o... more Background: Professional self-identity is a 'state of mind' -identifying one's-self as a member of a professional group. Delayed professional self-identity is a barrier to successful transition from student to professional. Current trends in medical education limit student doctors' legitimate peripheral participation and may retard their developing professional self-identity compared with other health and social care students. Aims: Develop a tool to monitor the development of professional self-identity to operate across the different health and social care professions and evaluate the tool with student doctors before wider data collection. Method: Content analysis of relevant curricula, mapped to professional standards documents, defined initial content. Field tests across 10 professional groups refined questionnaire items. A cross-sectional study on 496 student doctors evaluated validity on the basis of internal structure and relationships with external variables. Results: The 9-item questionnaire indicates a three-factor structure reflecting 'interpersonal tasks', 'generic attributes' and 'profession-specific elements'. Students with greater previous experience of health or social care roles, and students with a more positive attitude to qualification had significantly more advanced scores than their peers. Scores advanced through the curriculum showing step changes after the start of clinical attachments. Conclusions: The data provides sufficient evidence of validity with student doctors to justify wider data collection.
Professional self-identity [PSI] can be defined as the degree to which an individual identifies w... more Professional self-identity [PSI] can be defined as the degree to which an individual identifies with his or her professional group. Several authors have called for a better understanding of the processes by which healthcare students develop their professional identities, and suggested helpful theoretical frameworks borrowed from the social science and psychology literature. However to our knowledge, there has been little empirical work examining these processes in actual healthcare students, and we are aware of no data driven description of PSI development in healthcare students. Here, we report a data driven model of PSI formation in healthcare students. We interviewed 17 student doctors and dentists who had indicated, on a tracking questionnaire, the most substantial changes in their PSI. We analysed their perceptions of the experiences that had influenced their PSI, to develop a descriptive model. Both the primary coder and the secondary coder considered the data without referenc...
There is an increasing literature on learning organisations as a way of fostering communication, ... more There is an increasing literature on learning organisations as a way of fostering communication, teamwork, collaboration and collective learning, thereby promoting quality improvement and enhancing patient safety. An increasing number of instruments are being developed in an attempt to measure learning organisation characteristics. However, the majority of these tools are created for a business setting, have not been scientifically tested and have not been applied in healthcare. To evaluate elements of the validity and reliability of an instrument (ie, learning practice inventory (LPI)) for diagnosing learning practice characteristics in primary healthcare. Content validity was evaluated using a modified nominal group technique and a content validity rating scale. Construct validity and reliability evaluation was undertaken with 10 staff members from 10 general practices in the west of Scotland. Staff completed the inventory twice, 4-6 weeks apart. Applying generalisability theory, a variance component analysis was performed. The main findings present evidence that the inventory has acceptable reliability and content validity. The results also demonstrate that the inventory can reflect the consistent and uniquely different perspectives of particular designations of staff within a practice. It is possible to compare practices' overall learning environments and to identify specific areas of practice strength as well as areas for development. This study demonstrates the psychometric properties of a learning practice diagnostic inventory. It highlights the consistently different perspectives that individual staff groups have on the function of their practice, suggesting that the success of quality improvement initiatives may be compromised without the involvement and true engagement of each staff group.
OBJECTIVES Free-text comments in multisource feedback are intended to facilitate change in the as... more OBJECTIVES Free-text comments in multisource feedback are intended to facilitate change in the assessee's practice. This study was designed to utilise a large dataset of free-text comments obtained in a national pilot study in order to investigate how helpful these free-text comments may be to assessees.
New guidelines require all undergraduate medical students to undertake at least one period of ass... more New guidelines require all undergraduate medical students to undertake at least one period of assistantship where they assume most of the responsibilities of a first-year graduate doctor (FY1 doctor in the UK) under supervision. To investigate the feasibility of these assistantships. All UK schools were sent a questionnaire addressing the supervision required and the main barriers around implementation. Competencies that students already engage in as part of existing clinical placements and a number of 'tacit' competencies (e.g. practice and promote infection control) were regarded by most as suitable. Activities that present a clear clinical risk (e.g. prescribing and writing clinical correspondence) were regarded by most as unsuitable or requiring continuous supervision. Some lower risk but hard to measure activities (e.g. responding in practice to audit) were also regarded as unsuitable by some. A competency was usually considered inappropriate for one of three reasons: (1) current clinical governance and patient safety protocols appeared to bar students undertaking the competency; (2) a competency was not considered to be part of the current FY1 doctors' role; or (3) brief assistantships were considered unlikely to create sufficient opportunity for performing the competency. The article presents a number of practical issues in relation to assigning responsibility to student doctors. Respondents indicate that successful assistantships will only be possible if the UK National Health Service trusts review their attitude to balancing short- and long-term risks: assistantships need to be long enough to create genuine responsibility opportunities, and will require investment in supervision beyond the current capacity.
This article describes the problem of disorientation in students as they become doctors. Disorien... more This article describes the problem of disorientation in students as they become doctors. Disorientation arises because students have a poor or inaccurate understanding of what they are training to become. If they do not know what they are becoming it is hard for them to prioritise and contextualise their learning, to make sense of information about where they are now (assessment and feedback) or to determine the steps they need to take to develop (formative feedback and "feedforward"). It is also a barrier to the early development of professional identity. Using the analogy of a map, the paper describes the idea of a curriculum that is articulated as a developmental journey--a "roadmap curriculum". This is not incompatible with a competency-based curriculum, and certainly requires the same integration of knowledge, skills and attitudes. However, the semantic essence of a roadmap curriculum is fundamentally different; it must describe the pathway or pathways of development toward being a doctor in ways that are both authentic to qualified doctors and meaningful to learners. Examples from within and outside medicine are cited. Potential advantages and implications of this kind of curricular reform are discussed.
Background: The UK Department of Health is considering a single, generic multi-source feedback (M... more Background: The UK Department of Health is considering a single, generic multi-source feedback (MSF) questionnaire to inform revalidation. Method: Evaluation of an implementation pilot, reporting: response rates, assessor mix, question redundancy and participants' perceptions. Reliability was estimated using Generalisability theory. Results: A total of 12,540 responses were received on 977 doctors. The mean time taken to complete an MSF exercise was 68.2 days. The mean number of responses received per doctor was 12.0 (range 1-17) with no significant difference between specialties. Individual question response rates and participants' comments about questions indicate that some questions are less appropriate for some specialities. There was a significant difference in the mean score between specialities. Despite guidance, there were significant differences in the mix of assessors across specialties. More favourable scores were given by progressively more junior doctors. Nurses gave the most reliable scores. Conclusions: It is feasible to electronically administer a generic questionnaire to a large population of doctors. Generic content is appropriate for most but not all specialties. The differences in mean scores and the reliability of the MSF between specialties may be in part due to the specialty differences in assessor mix. Therefore the number and assessor mix should be standardised at specialty level and scores should not be compared across specialties.
Background: Professional self-identity is a 'state of mind' -identifying one's-self as a member o... more Background: Professional self-identity is a 'state of mind' -identifying one's-self as a member of a professional group. Delayed professional self-identity is a barrier to successful transition from student to professional. Current trends in medical education limit student doctors' legitimate peripheral participation and may retard their developing professional self-identity compared with other health and social care students. Aims: Develop a tool to monitor the development of professional self-identity to operate across the different health and social care professions and evaluate the tool with student doctors before wider data collection. Method: Content analysis of relevant curricula, mapped to professional standards documents, defined initial content. Field tests across 10 professional groups refined questionnaire items. A cross-sectional study on 496 student doctors evaluated validity on the basis of internal structure and relationships with external variables. Results: The 9-item questionnaire indicates a three-factor structure reflecting 'interpersonal tasks', 'generic attributes' and 'profession-specific elements'. Students with greater previous experience of health or social care roles, and students with a more positive attitude to qualification had significantly more advanced scores than their peers. Scores advanced through the curriculum showing step changes after the start of clinical attachments. Conclusions: The data provides sufficient evidence of validity with student doctors to justify wider data collection.
OBJECTIVES The mini-clinical evaluation exercise (mini-CEX) is widely used in the UK to assess cl... more OBJECTIVES The mini-clinical evaluation exercise (mini-CEX) is widely used in the UK to assess clinical competence, but there is little evidence regarding its implementation in the undergraduate setting. This study aimed to estimate the validity and reliability of the undergraduate mini-CEX and discuss the challenges involved in its implementation.
CONTEXT The clinical consultation is an important aspect of the doctor's role. However, there is ... more CONTEXT The clinical consultation is an important aspect of the doctor's role. However, there is a particular shortage of methods for assessing its quality, and its complexity makes it a considerable assessment challenge.
CONTEXT Only a patient and his or her family can judge many of the most important aspects of the ... more CONTEXT Only a patient and his or her family can judge many of the most important aspects of the doctor-patient interaction. This study evaluates the feasibility and reliability of children and their families assessing the quality of paediatricians' interactions using a rating instrument developed specifically for this purpose.
CONTEXT Investigators applying generalisability theory to educational research and evaluation hav... more CONTEXT Investigators applying generalisability theory to educational research and evaluation have sometimes done so poorly. The main difficulties have related to: inadequate or non-random sampling of effects, dealing with naturalistic data, and interpreting and presenting variance components.
CONTEXT There are significant levels of variation in candidate multiple mini-interview (MMI) scor... more CONTEXT There are significant levels of variation in candidate multiple mini-interview (MMI) scores caused by interviewer-related factors. Multi-facet Rasch modelling (MFRM) has the capability to both identify these sources of error and partially adjust for them within a measurement model that may be fairer to the candidate.
Context Well-designed assessments of performance are urgently required for training doctors, and ... more Context Well-designed assessments of performance are urgently required for training doctors, and to provide indicators of the quality of practice.
INTRODUCTION We wished to determine whether assessors could make reliable and valid judgements ab... more INTRODUCTION We wished to determine whether assessors could make reliable and valid judgements about the quality of completed reflective personal development plans (PDPs) for the purpose of accrediting UK general practitioners (GPs) for a postgraduate education allowance using a marking matrix, and secondly, to plan a feasible model of PDP assessment in the context of forthcoming GP appraisal ⁄ revalidation that would overcome the main sources of error identified from this study.
Background: Professional self-identity is a 'state of mind' -identifying one's-self as a member o... more Background: Professional self-identity is a 'state of mind' -identifying one's-self as a member of a professional group. Delayed professional self-identity is a barrier to successful transition from student to professional. Current trends in medical education limit student doctors' legitimate peripheral participation and may retard their developing professional self-identity compared with other health and social care students. Aims: Develop a tool to monitor the development of professional self-identity to operate across the different health and social care professions and evaluate the tool with student doctors before wider data collection. Method: Content analysis of relevant curricula, mapped to professional standards documents, defined initial content. Field tests across 10 professional groups refined questionnaire items. A cross-sectional study on 496 student doctors evaluated validity on the basis of internal structure and relationships with external variables. Results: The 9-item questionnaire indicates a three-factor structure reflecting 'interpersonal tasks', 'generic attributes' and 'profession-specific elements'. Students with greater previous experience of health or social care roles, and students with a more positive attitude to qualification had significantly more advanced scores than their peers. Scores advanced through the curriculum showing step changes after the start of clinical attachments. Conclusions: The data provides sufficient evidence of validity with student doctors to justify wider data collection.
Professional self-identity [PSI] can be defined as the degree to which an individual identifies w... more Professional self-identity [PSI] can be defined as the degree to which an individual identifies with his or her professional group. Several authors have called for a better understanding of the processes by which healthcare students develop their professional identities, and suggested helpful theoretical frameworks borrowed from the social science and psychology literature. However to our knowledge, there has been little empirical work examining these processes in actual healthcare students, and we are aware of no data driven description of PSI development in healthcare students. Here, we report a data driven model of PSI formation in healthcare students. We interviewed 17 student doctors and dentists who had indicated, on a tracking questionnaire, the most substantial changes in their PSI. We analysed their perceptions of the experiences that had influenced their PSI, to develop a descriptive model. Both the primary coder and the secondary coder considered the data without referenc...
There is an increasing literature on learning organisations as a way of fostering communication, ... more There is an increasing literature on learning organisations as a way of fostering communication, teamwork, collaboration and collective learning, thereby promoting quality improvement and enhancing patient safety. An increasing number of instruments are being developed in an attempt to measure learning organisation characteristics. However, the majority of these tools are created for a business setting, have not been scientifically tested and have not been applied in healthcare. To evaluate elements of the validity and reliability of an instrument (ie, learning practice inventory (LPI)) for diagnosing learning practice characteristics in primary healthcare. Content validity was evaluated using a modified nominal group technique and a content validity rating scale. Construct validity and reliability evaluation was undertaken with 10 staff members from 10 general practices in the west of Scotland. Staff completed the inventory twice, 4-6 weeks apart. Applying generalisability theory, a variance component analysis was performed. The main findings present evidence that the inventory has acceptable reliability and content validity. The results also demonstrate that the inventory can reflect the consistent and uniquely different perspectives of particular designations of staff within a practice. It is possible to compare practices' overall learning environments and to identify specific areas of practice strength as well as areas for development. This study demonstrates the psychometric properties of a learning practice diagnostic inventory. It highlights the consistently different perspectives that individual staff groups have on the function of their practice, suggesting that the success of quality improvement initiatives may be compromised without the involvement and true engagement of each staff group.
OBJECTIVES Free-text comments in multisource feedback are intended to facilitate change in the as... more OBJECTIVES Free-text comments in multisource feedback are intended to facilitate change in the assessee's practice. This study was designed to utilise a large dataset of free-text comments obtained in a national pilot study in order to investigate how helpful these free-text comments may be to assessees.
New guidelines require all undergraduate medical students to undertake at least one period of ass... more New guidelines require all undergraduate medical students to undertake at least one period of assistantship where they assume most of the responsibilities of a first-year graduate doctor (FY1 doctor in the UK) under supervision. To investigate the feasibility of these assistantships. All UK schools were sent a questionnaire addressing the supervision required and the main barriers around implementation. Competencies that students already engage in as part of existing clinical placements and a number of 'tacit' competencies (e.g. practice and promote infection control) were regarded by most as suitable. Activities that present a clear clinical risk (e.g. prescribing and writing clinical correspondence) were regarded by most as unsuitable or requiring continuous supervision. Some lower risk but hard to measure activities (e.g. responding in practice to audit) were also regarded as unsuitable by some. A competency was usually considered inappropriate for one of three reasons: (1) current clinical governance and patient safety protocols appeared to bar students undertaking the competency; (2) a competency was not considered to be part of the current FY1 doctors' role; or (3) brief assistantships were considered unlikely to create sufficient opportunity for performing the competency. The article presents a number of practical issues in relation to assigning responsibility to student doctors. Respondents indicate that successful assistantships will only be possible if the UK National Health Service trusts review their attitude to balancing short- and long-term risks: assistantships need to be long enough to create genuine responsibility opportunities, and will require investment in supervision beyond the current capacity.
This article describes the problem of disorientation in students as they become doctors. Disorien... more This article describes the problem of disorientation in students as they become doctors. Disorientation arises because students have a poor or inaccurate understanding of what they are training to become. If they do not know what they are becoming it is hard for them to prioritise and contextualise their learning, to make sense of information about where they are now (assessment and feedback) or to determine the steps they need to take to develop (formative feedback and "feedforward"). It is also a barrier to the early development of professional identity. Using the analogy of a map, the paper describes the idea of a curriculum that is articulated as a developmental journey--a "roadmap curriculum". This is not incompatible with a competency-based curriculum, and certainly requires the same integration of knowledge, skills and attitudes. However, the semantic essence of a roadmap curriculum is fundamentally different; it must describe the pathway or pathways of development toward being a doctor in ways that are both authentic to qualified doctors and meaningful to learners. Examples from within and outside medicine are cited. Potential advantages and implications of this kind of curricular reform are discussed.
Background: The UK Department of Health is considering a single, generic multi-source feedback (M... more Background: The UK Department of Health is considering a single, generic multi-source feedback (MSF) questionnaire to inform revalidation. Method: Evaluation of an implementation pilot, reporting: response rates, assessor mix, question redundancy and participants' perceptions. Reliability was estimated using Generalisability theory. Results: A total of 12,540 responses were received on 977 doctors. The mean time taken to complete an MSF exercise was 68.2 days. The mean number of responses received per doctor was 12.0 (range 1-17) with no significant difference between specialties. Individual question response rates and participants' comments about questions indicate that some questions are less appropriate for some specialities. There was a significant difference in the mean score between specialities. Despite guidance, there were significant differences in the mix of assessors across specialties. More favourable scores were given by progressively more junior doctors. Nurses gave the most reliable scores. Conclusions: It is feasible to electronically administer a generic questionnaire to a large population of doctors. Generic content is appropriate for most but not all specialties. The differences in mean scores and the reliability of the MSF between specialties may be in part due to the specialty differences in assessor mix. Therefore the number and assessor mix should be standardised at specialty level and scores should not be compared across specialties.
Background: Professional self-identity is a 'state of mind' -identifying one's-self as a member o... more Background: Professional self-identity is a 'state of mind' -identifying one's-self as a member of a professional group. Delayed professional self-identity is a barrier to successful transition from student to professional. Current trends in medical education limit student doctors' legitimate peripheral participation and may retard their developing professional self-identity compared with other health and social care students. Aims: Develop a tool to monitor the development of professional self-identity to operate across the different health and social care professions and evaluate the tool with student doctors before wider data collection. Method: Content analysis of relevant curricula, mapped to professional standards documents, defined initial content. Field tests across 10 professional groups refined questionnaire items. A cross-sectional study on 496 student doctors evaluated validity on the basis of internal structure and relationships with external variables. Results: The 9-item questionnaire indicates a three-factor structure reflecting 'interpersonal tasks', 'generic attributes' and 'profession-specific elements'. Students with greater previous experience of health or social care roles, and students with a more positive attitude to qualification had significantly more advanced scores than their peers. Scores advanced through the curriculum showing step changes after the start of clinical attachments. Conclusions: The data provides sufficient evidence of validity with student doctors to justify wider data collection.
OBJECTIVES The mini-clinical evaluation exercise (mini-CEX) is widely used in the UK to assess cl... more OBJECTIVES The mini-clinical evaluation exercise (mini-CEX) is widely used in the UK to assess clinical competence, but there is little evidence regarding its implementation in the undergraduate setting. This study aimed to estimate the validity and reliability of the undergraduate mini-CEX and discuss the challenges involved in its implementation.
CONTEXT The clinical consultation is an important aspect of the doctor's role. However, there is ... more CONTEXT The clinical consultation is an important aspect of the doctor's role. However, there is a particular shortage of methods for assessing its quality, and its complexity makes it a considerable assessment challenge.
CONTEXT Only a patient and his or her family can judge many of the most important aspects of the ... more CONTEXT Only a patient and his or her family can judge many of the most important aspects of the doctor-patient interaction. This study evaluates the feasibility and reliability of children and their families assessing the quality of paediatricians' interactions using a rating instrument developed specifically for this purpose.
CONTEXT Investigators applying generalisability theory to educational research and evaluation hav... more CONTEXT Investigators applying generalisability theory to educational research and evaluation have sometimes done so poorly. The main difficulties have related to: inadequate or non-random sampling of effects, dealing with naturalistic data, and interpreting and presenting variance components.
CONTEXT There are significant levels of variation in candidate multiple mini-interview (MMI) scor... more CONTEXT There are significant levels of variation in candidate multiple mini-interview (MMI) scores caused by interviewer-related factors. Multi-facet Rasch modelling (MFRM) has the capability to both identify these sources of error and partially adjust for them within a measurement model that may be fairer to the candidate.
Context Well-designed assessments of performance are urgently required for training doctors, and ... more Context Well-designed assessments of performance are urgently required for training doctors, and to provide indicators of the quality of practice.
INTRODUCTION We wished to determine whether assessors could make reliable and valid judgements ab... more INTRODUCTION We wished to determine whether assessors could make reliable and valid judgements about the quality of completed reflective personal development plans (PDPs) for the purpose of accrediting UK general practitioners (GPs) for a postgraduate education allowance using a marking matrix, and secondly, to plan a feasible model of PDP assessment in the context of forthcoming GP appraisal ⁄ revalidation that would overcome the main sources of error identified from this study.
Uploads
Papers by Jim Crossley